2024-09-11T08:21:07.566 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T08:21:07.582 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T08:21:07.669 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901289 branch: wip-guits-quincy-2024-09-10-2008-quincy description: orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} email: gabrioux@ibm.com first_in_suite: false job_id: '7901289' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-10-2008-quincy ceph: conf: client.0: debug ms: 1 debug objecter: 20 debug rados: 20 global: ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug ms: 1 debug osd: 20 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd max backfills: 3 osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - \(POOL_APP_NOT_ENABLED\) sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-quincy-2024-09-10-2008-quincy sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 2282 sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-quincy-2024-09-10-2008-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 targets: smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMMNnkzegqXXpeXRHVHZoVHK3m/El5/N2pcaddAuG1Xvq6b/wOu5wVXJrAuWgj5cWtgNgNZjWLsU9ddKrsShT10= smithi145.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEkkLCbEyj0llbt78wKnNVGZAavS+Ma5LLF4J7qGkVBwRAntKWfhhrigc8VWiC4uNnufTs1qhFshJE16LXCdYu4= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - full_sequential: - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_06:53:43 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T08:21:07.670 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa; will attempt to use it 2024-09-11T08:21:07.670 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa/tasks 2024-09-11T08:21:07.671 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T08:21:07.672 INFO:teuthology.task.internal:Checking packages... 2024-09-11T08:21:07.694 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd0e6828a2016d48cf25ad84064e50742bb1c39b9' 2024-09-11T08:21:07.694 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T08:21:07.694 INFO:teuthology.packaging:ref: None 2024-09-11T08:21:07.694 INFO:teuthology.packaging:tag: None 2024-09-11T08:21:07.695 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T08:21:07.695 INFO:teuthology.packaging:sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:21:07.695 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T08:21:07.842 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1623.gd0e6828a 2024-09-11T08:21:07.843 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T08:21:07.850 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T08:21:07.850 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T08:21:07.889 INFO:teuthology.task.internal:Saving configuration 2024-09-11T08:21:07.902 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T08:21:07.913 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T08:21:07.938 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901289', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 08:16:12.528000', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMMNnkzegqXXpeXRHVHZoVHK3m/El5/N2pcaddAuG1Xvq6b/wOu5wVXJrAuWgj5cWtgNgNZjWLsU9ddKrsShT10='} 2024-09-11T08:21:07.959 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901289', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 08:16:12.529386', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEkkLCbEyj0llbt78wKnNVGZAavS+Ma5LLF4J7qGkVBwRAntKWfhhrigc8VWiC4uNnufTs1qhFshJE16LXCdYu4='} 2024-09-11T08:21:07.959 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T08:21:07.966 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-11T08:21:07.966 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-11T08:21:07.966 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T08:21:08.047 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f687241bac0>, signals=[15]) 2024-09-11T08:21:08.047 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T08:21:08.053 INFO:teuthology.task.internal:Opening connections... 2024-09-11T08:21:08.054 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-09-11T08:21:08.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:21:08.130 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi145.front.sepia.ceph.com 2024-09-11T08:21:08.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:21:08.204 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T08:21:08.211 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-09-11T08:21:08.232 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-09-11T08:21:08.233 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-09-11T08:21:08.290 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-09-11T08:21:08.291 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-09-11T08:21:08.291 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-09-11T08:21:08.291 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-09-11T08:21:08.291 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-09-11T08:21:08.291 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-09-11T08:21:08.291 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T08:21:08.291 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-09-11T08:21:08.291 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-09-11T08:21:08.291 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T08:21:08.291 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-09-11T08:21:08.291 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T08:21:08.291 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T08:21:08.291 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T08:21:08.292 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-09-11T08:21:08.313 DEBUG:teuthology.orchestra.run.smithi145:> uname -m 2024-09-11T08:21:08.332 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2024-09-11T08:21:08.333 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2024-09-11T08:21:08.390 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2024-09-11T08:21:08.390 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="9" 2024-09-11T08:21:08.390 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2024-09-11T08:21:08.390 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2024-09-11T08:21:08.390 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="9" 2024-09-11T08:21:08.390 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el9" 2024-09-11T08:21:08.390 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T08:21:08.390 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2024-09-11T08:21:08.390 INFO:teuthology.orchestra.run.smithi145.stdout:LOGO="fedora-logo-icon" 2024-09-11T08:21:08.391 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T08:21:08.391 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2024-09-11T08:21:08.391 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T08:21:08.391 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T08:21:08.391 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T08:21:08.391 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2024-09-11T08:21:08.413 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T08:21:08.423 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T08:21:08.429 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T08:21:08.429 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-09-11T08:21:08.432 DEBUG:teuthology.orchestra.run.smithi145:> test '!' -e /home/ubuntu/cephtest 2024-09-11T08:21:08.448 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T08:21:08.454 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T08:21:08.455 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-09-11T08:21:08.489 DEBUG:teuthology.orchestra.run.smithi145:> test -z $(ls -A /var/lib/ceph) 2024-09-11T08:21:08.590 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T08:21:08.649 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T08:21:08.663 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T08:21:08.663 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-11T08:21:08.663 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T08:21:08.664 DEBUG:teuthology.orchestra.run.smithi145:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T08:21:08.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:21:08.681 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-11T08:21:08.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:21:08.682 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2024-09-11T08:21:08.739 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-503.el9.x86_64 2024-09-11T08:21:08.739 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.14.0-503.el9.x86_64 2024-09-11T08:21:08.739 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2024-09-11T08:21:08.740 INFO:teuthology.orchestra.run.smithi145.stdout:5.14.0-503.el9.x86_64 2024-09-11T08:21:08.740 INFO:teuthology.task.kernel:Running kernel on smithi145: 5.14.0-503.el9.x86_64 2024-09-11T08:21:08.740 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum install -y kernel 2024-09-11T08:21:11.247 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-11T08:21:12.095 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.2 MB 00:01 2024-09-11T08:21:14.673 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - AppStream 8.9 MB/s | 20 MB 00:02 2024-09-11T08:21:18.923 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-11T08:21:20.341 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - Extras packages 90 kB/s | 19 kB 00:00 2024-09-11T08:21:21.129 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-11T08:21:26.592 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-09-11T08:21:28.574 INFO:teuthology.orchestra.run.smithi145.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T08:21:28.574 INFO:teuthology.orchestra.run.smithi145.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T08:21:28.667 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:21:28.673 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:21:28.674 INFO:teuthology.orchestra.run.smithi145.stdout: Package Architecture Version Repository Size 2024-09-11T08:21:28.674 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:21:28.674 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:21:28.674 INFO:teuthology.orchestra.run.smithi145.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T08:21:28.674 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-09-11T08:21:28.674 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T08:21:28.674 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T08:21:28.674 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T08:21:28.674 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:21:28.674 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:21:28.675 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:21:28.675 INFO:teuthology.orchestra.run.smithi145.stdout:Install 4 Packages 2024-09-11T08:21:28.675 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:21:28.675 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 79 M 2024-09-11T08:21:28.675 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 126 M 2024-09-11T08:21:28.675 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:21:28.768 INFO:teuthology.orchestra.run.smithi145.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 317 kB/s | 23 kB 00:00 2024-09-11T08:21:31.709 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 20 MB 00:18 2024-09-11T08:21:32.393 INFO:teuthology.orchestra.run.smithi145.stdout:(2/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 9.3 MB/s | 35 MB 00:03 2024-09-11T08:21:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 4.1 MB/s | 16 MB 00:03 2024-09-11T08:21:32.902 INFO:teuthology.orchestra.run.smithi145.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.9 MB/s | 29 MB 00:04 2024-09-11T08:21:32.902 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:21:32.902 INFO:teuthology.orchestra.run.smithi145.stdout:Total 19 MB/s | 79 MB 00:04 2024-09-11T08:21:33.589 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:21:33.981 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:21:33.981 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:21:34.911 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:21:34.912 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:21:36.585 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 5.9 MB/s | 6.5 MB 00:01 2024-09-11T08:21:37.346 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:21:38.030 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:21:38.085 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 19 kB 00:00 2024-09-11T08:21:38.128 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:21:38.973 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-11T08:21:40.903 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:21:41.031 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:21:44.420 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-09-11T08:21:45.885 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:21:45.924 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:21:46.434 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T08:21:46.434 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T08:21:46.523 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:21:46.532 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:21:46.532 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-09-11T08:21:46.533 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:21:46.533 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:21:46.533 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T08:21:46.533 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-11T08:21:46.533 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T08:21:46.533 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T08:21:46.533 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T08:21:46.533 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:21:46.533 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:21:46.533 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:21:46.533 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2024-09-11T08:21:46.533 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:21:46.534 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 79 M 2024-09-11T08:21:46.534 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 126 M 2024-09-11T08:21:46.534 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:21:47.138 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 41 kB/s | 23 kB 00:00 2024-09-11T08:21:47.971 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-11T08:21:48.654 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-09-11T08:21:49.313 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-11T08:21:49.313 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:21:49.314 INFO:teuthology.orchestra.run.smithi092.stdout:Total 28 MB/s | 79 MB 00:02 2024-09-11T08:21:49.998 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:21:50.351 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:21:50.351 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:21:50.780 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:21:51.330 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:21:51.331 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:21:53.782 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:21:54.497 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:21:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:21:57.360 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:21:57.492 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:22:02.345 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:22:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:22:07.343 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:22:29.136 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:22:29.175 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:22:29.452 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:22:29.452 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:22:29.453 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:22:29.453 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:22:30.285 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:22:30.285 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:22:30.286 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:22:30.286 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T08:22:30.286 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T08:22:30.286 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T08:22:30.286 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T08:22:30.286 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:22:30.286 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:22:30.619 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:22:30.704 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:22:30.705 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:22:30.705 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T08:22:30.705 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T08:22:30.705 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T08:22:30.705 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-11T08:22:30.705 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi145.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T08:22:30.713 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:22:30.772 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:22:30.772 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:22:30.772 DEBUG:teuthology.orchestra.run.smithi145:> sudo rpm -qi grub2-tools 2024-09-11T08:22:30.804 INFO:teuthology.orchestra.run.smithi145.stdout:Name : grub2-tools 2024-09-11T08:22:30.805 INFO:teuthology.orchestra.run.smithi145.stdout:Epoch : 1 2024-09-11T08:22:30.805 INFO:teuthology.orchestra.run.smithi145.stdout:Version : 2.06 2024-09-11T08:22:30.805 INFO:teuthology.orchestra.run.smithi145.stdout:Release : 82.el9 2024-09-11T08:22:30.805 INFO:teuthology.orchestra.run.smithi145.stdout:Architecture: x86_64 2024-09-11T08:22:30.805 INFO:teuthology.orchestra.run.smithi145.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T08:22:30.805 INFO:teuthology.orchestra.run.smithi145.stdout:Group : Unspecified 2024-09-11T08:22:30.805 INFO:teuthology.orchestra.run.smithi145.stdout:Size : 8274670 2024-09-11T08:22:30.805 INFO:teuthology.orchestra.run.smithi145.stdout:License : GPLv3+ 2024-09-11T08:22:30.805 INFO:teuthology.orchestra.run.smithi145.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T08:22:30.805 INFO:teuthology.orchestra.run.smithi145.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T08:22:30.805 INFO:teuthology.orchestra.run.smithi145.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T08:22:30.805 INFO:teuthology.orchestra.run.smithi145.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T08:22:30.806 INFO:teuthology.orchestra.run.smithi145.stdout:Packager : builder@centos.org 2024-09-11T08:22:30.806 INFO:teuthology.orchestra.run.smithi145.stdout:Vendor : CentOS 2024-09-11T08:22:30.806 INFO:teuthology.orchestra.run.smithi145.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T08:22:30.806 INFO:teuthology.orchestra.run.smithi145.stdout:Summary : Support tools for GRUB. 2024-09-11T08:22:30.806 INFO:teuthology.orchestra.run.smithi145.stdout:Description : 2024-09-11T08:22:30.806 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:22:30.806 INFO:teuthology.orchestra.run.smithi145.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T08:22:30.806 INFO:teuthology.orchestra.run.smithi145.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T08:22:30.806 INFO:teuthology.orchestra.run.smithi145.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T08:22:30.806 INFO:teuthology.orchestra.run.smithi145.stdout:hardware devices. 2024-09-11T08:22:30.806 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:22:30.806 INFO:teuthology.orchestra.run.smithi145.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T08:22:30.807 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T08:22:30.807 INFO:teuthology.task.kernel:Updating grub on smithi145 to boot 5.14.0-505.el9.x86_64 2024-09-11T08:22:30.807 DEBUG:teuthology.orchestra.run.smithi145:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T08:22:31.498 INFO:teuthology.orchestra.run.smithi145.stderr:Generating grub configuration file ... 2024-09-11T08:22:32.876 INFO:teuthology.orchestra.run.smithi145.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T08:22:32.906 INFO:teuthology.orchestra.run.smithi145.stderr:done 2024-09-11T08:22:32.909 DEBUG:teuthology.orchestra.run.smithi145:> mktemp 2024-09-11T08:22:32.925 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/tmp.5KufqNJQAi 2024-09-11T08:22:32.925 DEBUG:teuthology.orchestra.run.smithi145:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5KufqNJQAi 2024-09-11T08:22:32.991 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0666 /tmp/tmp.5KufqNJQAi 2024-09-11T08:22:33.144 DEBUG:teuthology.orchestra.remote:smithi145:/tmp/tmp.5KufqNJQAi is 6KB 2024-09-11T08:22:33.192 DEBUG:teuthology.orchestra.run.smithi145:> rm -fr /tmp/tmp.5KufqNJQAi 2024-09-11T08:22:33.208 DEBUG:teuthology.orchestra.run.smithi145:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T08:22:33.279 INFO:teuthology.orchestra.run.smithi145.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T08:22:33.279 INFO:teuthology.orchestra.run.smithi145.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T08:22:33.279 INFO:teuthology.orchestra.run.smithi145.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T08:22:33.279 INFO:teuthology.orchestra.run.smithi145.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T08:22:33.280 DEBUG:teuthology.orchestra.run.smithi145:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T08:22:33.440 DEBUG:teuthology.orchestra.run.smithi145:> sudo shutdown -r now 2024-09-11T08:22:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:22:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:22:46.392 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:22:46.392 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:22:46.393 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:22:46.393 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:22:47.221 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:22:47.221 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:22:47.221 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:22:47.221 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T08:22:47.221 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T08:22:47.221 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T08:22:47.221 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T08:22:47.222 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:22:47.222 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:22:47.579 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:22:47.657 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:22:47.657 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:22:47.658 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T08:22:47.658 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T08:22:47.658 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T08:22:47.658 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T08:22:47.658 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T08:22:47.658 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:22:47.716 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:22:47.717 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:22:47.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -qi grub2-tools 2024-09-11T08:22:47.749 INFO:teuthology.orchestra.run.smithi092.stdout:Name : grub2-tools 2024-09-11T08:22:47.749 INFO:teuthology.orchestra.run.smithi092.stdout:Epoch : 1 2024-09-11T08:22:47.749 INFO:teuthology.orchestra.run.smithi092.stdout:Version : 2.06 2024-09-11T08:22:47.749 INFO:teuthology.orchestra.run.smithi092.stdout:Release : 82.el9 2024-09-11T08:22:47.749 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: x86_64 2024-09-11T08:22:47.749 INFO:teuthology.orchestra.run.smithi092.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T08:22:47.749 INFO:teuthology.orchestra.run.smithi092.stdout:Group : Unspecified 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout:Size : 8274670 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout:License : GPLv3+ 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout:Packager : builder@centos.org 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout:Vendor : CentOS 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout:Summary : Support tools for GRUB. 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout:Description : 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T08:22:47.750 INFO:teuthology.orchestra.run.smithi092.stdout:hardware devices. 2024-09-11T08:22:47.751 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:22:47.751 INFO:teuthology.orchestra.run.smithi092.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T08:22:47.751 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T08:22:47.751 INFO:teuthology.task.kernel:Updating grub on smithi092 to boot 5.14.0-505.el9.x86_64 2024-09-11T08:22:47.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T08:22:48.446 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2024-09-11T08:22:49.817 INFO:teuthology.orchestra.run.smithi092.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T08:22:49.847 INFO:teuthology.orchestra.run.smithi092.stderr:done 2024-09-11T08:22:49.854 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-09-11T08:22:49.870 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.uXpvR7gcLq 2024-09-11T08:22:49.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.uXpvR7gcLq 2024-09-11T08:22:49.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.uXpvR7gcLq 2024-09-11T08:22:50.080 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.uXpvR7gcLq is 6KB 2024-09-11T08:22:50.128 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.uXpvR7gcLq 2024-09-11T08:22:50.143 DEBUG:teuthology.orchestra.run.smithi092:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T08:22:50.205 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T08:22:50.206 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T08:22:50.206 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T08:22:50.206 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T08:22:50.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T08:22:50.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2024-09-11T08:23:03.453 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-11T08:23:03.453 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-11T08:23:03.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:23:20.358 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T08:23:20.358 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-11T08:23:20.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:23:21.917 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-09-11T08:23:30.925 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-11T08:23:30.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:23:33.981 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-09-11T08:23:38.813 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-11T08:23:45.988 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-11T08:23:45.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:23:47.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-11T08:23:47.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:23:50.881 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-11T08:24:02.893 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-11T08:24:02.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:24:17.984 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-09-11T08:24:18.423 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-11T08:24:18.423 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-11T08:24:18.423 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2024-09-11T08:24:18.438 INFO:teuthology.orchestra.run.smithi145.stdout:5.14.0-505.el9.x86_64 2024-09-11T08:24:18.438 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T08:24:18.439 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T08:24:18.439 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T08:24:19.440 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-11T08:24:19.441 DEBUG:teuthology.orchestra.run.smithi145:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T08:24:19.557 INFO:teuthology.orchestra.run.smithi145.stdout:ttyS1 2024-09-11T08:24:19.581 DEBUG:teuthology.parallel:result is None 2024-09-11T08:24:34.712 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-11T08:24:35.064 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-11T08:24:35.064 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-11T08:24:35.064 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-11T08:24:35.079 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-505.el9.x86_64 2024-09-11T08:24:35.079 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T08:24:35.079 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T08:24:35.080 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T08:24:36.081 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T08:24:36.081 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T08:24:36.186 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2024-09-11T08:24:36.206 DEBUG:teuthology.parallel:result is None 2024-09-11T08:24:36.207 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T08:24:36.216 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T08:24:36.216 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T08:24:36.219 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T08:24:36.238 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T08:24:36.249 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T08:24:36.262 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T08:24:36.262 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T08:24:36.271 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T08:24:36.342 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T08:24:36.354 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T08:24:36.354 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T08:24:36.356 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T08:24:36.391 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:24:36.398 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:24:36.402 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:24:36.408 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:24:36.409 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T08:24:36.416 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T08:24:36.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T08:24:36.419 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T08:24:36.470 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T08:24:36.483 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T08:24:36.483 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T08:24:36.486 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T08:24:36.526 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:24:36.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:24:36.641 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:24:36.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:24:36.720 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:24:36.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T08:24:36.782 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:24:36.829 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:24:36.879 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:24:36.961 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:24:37.004 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-11T08:24:37.004 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T08:24:37.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-09-11T08:24:37.072 DEBUG:teuthology.orchestra.run.smithi145:> sudo service rsyslog restart 2024-09-11T08:24:37.127 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:24:37.173 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:24:37.450 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T08:24:37.460 INFO:teuthology.task.internal:Starting timer... 2024-09-11T08:24:37.460 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T08:24:37.518 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T08:24:37.532 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-11T08:24:37.532 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-09-11T08:24:37.576 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-09-11T08:24:37.578 DEBUG:teuthology.orchestra.run.smithi145:> sudo service auditd rotate 2024-09-11T08:24:37.613 INFO:teuthology.orchestra.run.smithi145.stdout:Rotating logs: 2024-09-11T08:24:37.615 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T08:24:37.615 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-09-11T08:24:37.743 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-09-11T08:24:37.744 DEBUG:teuthology.orchestra.run.smithi145:> /usr/sbin/getenforce 2024-09-11T08:24:37.776 INFO:teuthology.orchestra.run.smithi145.stdout:Permissive 2024-09-11T08:24:37.777 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi145.front.sepia.ceph.com': 'permissive'} 2024-09-11T08:24:37.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:24:37.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:24:37.802 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:24:37.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:24:37.843 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T08:24:37.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-09-11T08:24:37.872 DEBUG:teuthology.orchestra.run.smithi145:> sudo /usr/sbin/setenforce permissive 2024-09-11T08:24:37.919 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T08:24:37.932 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T08:24:37.936 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T08:24:38.056 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T08:24:38.067 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T08:24:38.068 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi145.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T08:28:20.924 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi145.front.sepia.ceph.com')] 2024-09-11T08:28:20.926 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-11T08:28:20.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:28:21.013 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-11T08:28:21.092 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-11T08:28:21.093 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-11T08:28:21.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:28:21.169 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-09-11T08:28:21.249 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-11T08:28:21.250 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T08:28:21.260 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T08:28:21.261 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T08:28:21.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:28:21.264 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T08:28:21.264 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:28:21.292 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T08:28:21.306 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T08:28:21.322 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T08:28:21.330 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-09-11T08:28:21.336 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T08:28:21.341 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-09-11T08:28:21.356 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T08:28:21.361 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: ntpd: command not found 2024-09-11T08:28:21.369 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T08:28:21.372 INFO:teuthology.orchestra.run.smithi145.stdout:506 Cannot talk to daemon 2024-09-11T08:28:21.386 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T08:28:21.400 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T08:28:21.431 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:28:21.435 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:28:21.435 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-11T08:28:21.435 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:28:21.435 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:28:21.435 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:28:21.435 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:28:21.458 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:28:21.462 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:28:21.462 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-09-11T08:28:21.463 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:28:21.463 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:28:21.463 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:28:21.463 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:28:21.463 INFO:teuthology.run_tasks:Running task pexec... 2024-09-11T08:28:21.479 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-11T08:28:21.479 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi092.front.sepia.ceph.com 2024-09-11T08:28:21.480 DEBUG:teuthology.orchestra.run.smithi092:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T08:28:21.480 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi145.front.sepia.ceph.com 2024-09-11T08:28:21.480 DEBUG:teuthology.orchestra.run.smithi145:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T08:28:21.831 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:28:21.833 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:28:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-09-11T08:28:21.833 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:28:21.833 INFO:teuthology.orchestra.run.smithi092.stdout:Removing: 2024-09-11T08:28:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T08:28:21.833 INFO:teuthology.orchestra.run.smithi092.stdout:Removing unused dependencies: 2024-09-11T08:28:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T08:28:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:28:21.834 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:28:21.834 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:28:21.834 INFO:teuthology.orchestra.run.smithi092.stdout:Remove 2 Packages 2024-09-11T08:28:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:28:21.834 INFO:teuthology.orchestra.run.smithi092.stdout:Freed space: 5.5 M 2024-09-11T08:28:21.834 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:28:21.838 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:28:21.838 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:28:21.851 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:28:21.863 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:28:21.864 INFO:teuthology.orchestra.run.smithi145.stdout: Package Architecture Version Repository Size 2024-09-11T08:28:21.864 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:28:21.864 INFO:teuthology.orchestra.run.smithi145.stdout:Removing: 2024-09-11T08:28:21.864 INFO:teuthology.orchestra.run.smithi145.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T08:28:21.864 INFO:teuthology.orchestra.run.smithi145.stdout:Removing unused dependencies: 2024-09-11T08:28:21.864 INFO:teuthology.orchestra.run.smithi145.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T08:28:21.864 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:28:21.864 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:28:21.864 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:28:21.864 INFO:teuthology.orchestra.run.smithi145.stdout:Remove 2 Packages 2024-09-11T08:28:21.864 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:28:21.864 INFO:teuthology.orchestra.run.smithi145.stdout:Freed space: 5.5 M 2024-09-11T08:28:21.865 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:28:21.868 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:28:21.868 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:28:21.916 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:28:21.917 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:28:21.933 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:28:21.933 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:28:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:28:22.068 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:28:22.142 INFO:teuthology.orchestra.run.smithi145.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T08:28:22.146 INFO:teuthology.orchestra.run.smithi092.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T08:28:22.193 INFO:teuthology.orchestra.run.smithi145.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:28:22.206 INFO:teuthology.orchestra.run.smithi092.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:28:22.720 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:28:22.720 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T08:28:22.723 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:28:22.723 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T08:28:22.968 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T08:28:22.968 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:28:22.968 INFO:teuthology.orchestra.run.smithi092.stdout:Removed: 2024-09-11T08:28:22.969 INFO:teuthology.orchestra.run.smithi092.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:28:22.969 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:28:22.969 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:28:22.987 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T08:28:22.987 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:28:22.987 INFO:teuthology.orchestra.run.smithi145.stdout:Removed: 2024-09-11T08:28:22.987 INFO:teuthology.orchestra.run.smithi145.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:28:22.987 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:28:22.987 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:28:23.800 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:49 ago on Wed 11 Sep 2024 08:27:34 AM UTC. 2024-09-11T08:28:23.809 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:49 ago on Wed 11 Sep 2024 08:27:34 AM UTC. 2024-09-11T08:28:23.938 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:28:23.939 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:28:23.939 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-11T08:28:23.939 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:28:23.939 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:28:23.939 INFO:teuthology.orchestra.run.smithi092.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T08:28:23.939 INFO:teuthology.orchestra.run.smithi092.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T08:28:23.939 INFO:teuthology.orchestra.run.smithi092.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-11T08:28:23.939 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-11T08:28:23.939 INFO:teuthology.orchestra.run.smithi092.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T08:28:23.939 INFO:teuthology.orchestra.run.smithi092.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T08:28:23.939 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T08:28:23.939 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T08:28:23.940 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:28:23.940 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:28:23.940 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:28:23.940 INFO:teuthology.orchestra.run.smithi092.stdout:Install 7 Packages 2024-09-11T08:28:23.940 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:28:23.944 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:28:23.945 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:28:23.945 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-09-11T08:28:23.945 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:28:23.945 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:28:23.945 INFO:teuthology.orchestra.run.smithi145.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T08:28:23.945 INFO:teuthology.orchestra.run.smithi145.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T08:28:23.945 INFO:teuthology.orchestra.run.smithi145.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-11T08:28:23.946 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-09-11T08:28:23.946 INFO:teuthology.orchestra.run.smithi145.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T08:28:23.946 INFO:teuthology.orchestra.run.smithi145.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T08:28:23.946 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T08:28:23.946 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T08:28:23.946 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:28:23.946 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:28:23.946 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:28:23.946 INFO:teuthology.orchestra.run.smithi145.stdout:Install 7 Packages 2024-09-11T08:28:23.946 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:28:23.951 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 5.2 M 2024-09-11T08:28:23.952 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 19 M 2024-09-11T08:28:23.952 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:28:23.961 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 5.2 M 2024-09-11T08:28:23.961 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 19 M 2024-09-11T08:28:23.961 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:28:24.225 INFO:teuthology.orchestra.run.smithi145.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 369 kB/s | 44 kB 00:00 2024-09-11T08:28:24.250 INFO:teuthology.orchestra.run.smithi145.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 727 kB/s | 106 kB 00:00 2024-09-11T08:28:24.300 INFO:teuthology.orchestra.run.smithi145.stdout:(3/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-11T08:28:24.359 INFO:teuthology.orchestra.run.smithi145.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 536 kB/s | 72 kB 00:00 2024-09-11T08:28:24.443 INFO:teuthology.orchestra.run.smithi145.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.6 MB/s | 906 kB 00:00 2024-09-11T08:28:24.454 INFO:teuthology.orchestra.run.smithi092.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 366 kB/s | 44 kB 00:00 2024-09-11T08:28:24.482 INFO:teuthology.orchestra.run.smithi092.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 723 kB/s | 106 kB 00:00 2024-09-11T08:28:24.493 INFO:teuthology.orchestra.run.smithi145.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.3 MB/s | 837 kB 00:00 2024-09-11T08:28:24.530 INFO:teuthology.orchestra.run.smithi092.stdout:(3/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-11T08:28:24.555 INFO:teuthology.orchestra.run.smithi092.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 713 kB/s | 72 kB 00:00 2024-09-11T08:28:24.605 INFO:teuthology.orchestra.run.smithi092.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.2 MB/s | 906 kB 00:00 2024-09-11T08:28:24.797 INFO:teuthology.orchestra.run.smithi092.stdout:(6/7): runc-1.1.13-4.el9.x86_64.rpm 13 MB/s | 3.2 MB 00:00 2024-09-11T08:28:24.873 INFO:teuthology.orchestra.run.smithi092.stdout:(7/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 2.4 MB/s | 837 kB 00:00 2024-09-11T08:28:24.873 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:28:24.873 INFO:teuthology.orchestra.run.smithi092.stdout:Total 5.6 MB/s | 5.2 MB 00:00 2024-09-11T08:28:24.983 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:28:24.997 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:28:24.997 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:28:25.287 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:28:25.287 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:28:25.747 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:28:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-11T08:28:25.851 INFO:teuthology.orchestra.run.smithi145.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 2.1 MB/s | 3.2 MB 00:01 2024-09-11T08:28:25.851 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:28:25.852 INFO:teuthology.orchestra.run.smithi145.stdout:Total 2.7 MB/s | 5.2 MB 00:01 2024-09-11T08:28:25.876 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-11T08:28:25.905 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-11T08:28:25.970 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:28:25.985 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:28:25.985 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:28:26.109 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-11T08:28:26.136 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T08:28:26.294 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:28:26.295 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:28:26.575 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T08:28:26.608 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T08:28:26.719 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T08:28:26.766 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:28:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:28:26.845 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-11T08:28:26.880 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-11T08:28:26.909 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-11T08:28:27.117 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-11T08:28:27.156 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T08:28:27.602 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T08:28:27.626 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T08:28:27.742 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T08:28:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:28:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-11T08:28:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-11T08:28:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-11T08:28:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-11T08:28:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-11T08:28:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-11T08:28:27.820 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:28:28.162 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:28:28.162 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:28:28.162 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:28:28.162 INFO:teuthology.orchestra.run.smithi092.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:28:28.163 INFO:teuthology.orchestra.run.smithi092.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T08:28:28.163 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T08:28:28.163 INFO:teuthology.orchestra.run.smithi092.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-11T08:28:28.163 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:28:28.163 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:28:28.573 DEBUG:teuthology.parallel:result is None 2024-09-11T08:28:28.846 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:28:28.847 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-11T08:28:28.847 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-11T08:28:28.847 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-11T08:28:28.847 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-11T08:28:28.847 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-11T08:28:28.847 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-11T08:28:29.179 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:28:29.180 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:28:29.180 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:28:29.180 INFO:teuthology.orchestra.run.smithi145.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:28:29.180 INFO:teuthology.orchestra.run.smithi145.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T08:28:29.180 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T08:28:29.180 INFO:teuthology.orchestra.run.smithi145.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-11T08:28:29.180 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:28:29.180 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:28:29.544 DEBUG:teuthology.parallel:result is None 2024-09-11T08:28:29.545 INFO:teuthology.run_tasks:Running task install... 2024-09-11T08:28:29.553 DEBUG:teuthology.task.install:project ceph 2024-09-11T08:28:29.553 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'}} 2024-09-11T08:28:29.553 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'} 2024-09-11T08:28:29.554 INFO:teuthology.task.install:Using flavor: default 2024-09-11T08:28:29.559 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T08:28:29.559 INFO:teuthology.task.install:extra packages: [] 2024-09-11T08:28:29.560 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T08:28:29.560 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:28:29.563 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T08:28:29.563 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:28:29.711 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T08:28:29.711 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T08:28:29.714 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T08:28:29.715 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T08:28:29.820 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T08:28:29.821 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-11T08:28:29.821 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T08:28:29.826 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T08:28:29.826 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:28:29.826 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T08:28:29.845 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T08:28:29.846 DEBUG:teuthology.orchestra.run.smithi145:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T08:28:29.857 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T08:28:29.857 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T08:28:29.910 DEBUG:teuthology.orchestra.run.smithi145:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T08:28:29.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T08:28:29.985 DEBUG:teuthology.orchestra.run.smithi145:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T08:28:30.004 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T08:28:30.049 INFO:teuthology.orchestra.run.smithi145.stdout:check_obsoletes = 1 2024-09-11T08:28:30.051 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum clean all 2024-09-11T08:28:30.073 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-09-11T08:28:30.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-09-11T08:28:30.269 INFO:teuthology.orchestra.run.smithi145.stdout:45 files removed 2024-09-11T08:28:30.293 INFO:teuthology.orchestra.run.smithi092.stdout:45 files removed 2024-09-11T08:28:30.297 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-radosgw 2024-09-11T08:28:30.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw 2024-09-11T08:28:30.860 INFO:teuthology.orchestra.run.smithi145.stdout:ceph packages for x86_64 281 kB/s | 77 kB 00:00 2024-09-11T08:28:30.881 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 283 kB/s | 77 kB 00:00 2024-09-11T08:28:31.143 INFO:teuthology.orchestra.run.smithi145.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-11T08:28:31.164 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-11T08:28:31.419 INFO:teuthology.orchestra.run.smithi145.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-11T08:28:31.456 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 7.0 kB/s | 1.8 kB 00:00 2024-09-11T08:28:31.923 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.2 MB 00:00 2024-09-11T08:28:32.153 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-11T08:28:34.354 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-11T08:28:39.563 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.5 MB 00:01 2024-09-11T08:28:40.907 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-11T08:28:41.255 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-11T08:28:42.042 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-11T08:28:47.156 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.5 MB 00:02 2024-09-11T08:28:47.415 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-09-11T08:28:48.799 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 74 kB/s | 19 kB 00:00 2024-09-11T08:28:49.670 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-11T08:28:49.767 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:28:49.769 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:28:49.769 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-09-11T08:28:49.769 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:28:49.769 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:28:49.769 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T08:28:49.769 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrading: 2024-09-11T08:28:49.769 INFO:teuthology.orchestra.run.smithi145.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T08:28:49.769 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T08:28:49.769 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-09-11T08:28:49.770 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T08:28:49.770 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T08:28:49.770 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T08:28:49.770 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T08:28:49.770 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T08:28:49.770 INFO:teuthology.orchestra.run.smithi145.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T08:28:49.770 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T08:28:49.770 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T08:28:49.770 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T08:28:49.770 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T08:28:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T08:28:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T08:28:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T08:28:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T08:28:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T08:28:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T08:28:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T08:28:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T08:28:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T08:28:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T08:28:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T08:28:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:28:49.771 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:28:49.772 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:28:49.772 INFO:teuthology.orchestra.run.smithi145.stdout:Install 22 Packages 2024-09-11T08:28:49.772 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrade 2 Packages 2024-09-11T08:28:49.772 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:28:49.772 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 58 M 2024-09-11T08:28:49.772 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:28:50.358 INFO:teuthology.orchestra.run.smithi145.stdout:(1/24): ceph-base-17.2.7-1623.gd0e6828a.el9.x86 11 MB/s | 5.0 MB 00:00 2024-09-11T08:28:50.400 INFO:teuthology.orchestra.run.smithi145.stdout:(2/24): ceph-selinux-17.2.7-1623.gd0e6828a.el9. 606 kB/s | 25 kB 00:00 2024-09-11T08:28:50.492 INFO:teuthology.orchestra.run.smithi145.stdout:(3/24): libcephfs2-17.2.7-1623.gd0e6828a.el9.x8 6.8 MB/s | 636 kB 00:00 2024-09-11T08:28:50.592 INFO:teuthology.orchestra.run.smithi145.stdout:(4/24): libradosstriper1-17.2.7-1623.gd0e6828a. 4.5 MB/s | 457 kB 00:00 2024-09-11T08:28:50.917 INFO:teuthology.orchestra.run.smithi145.stdout:(5/24): librgw2-17.2.7-1623.gd0e6828a.el9.x86_6 15 MB/s | 4.9 MB 00:00 2024-09-11T08:28:51.159 INFO:teuthology.orchestra.run.smithi145.stdout:(6/24): ceph-common-17.2.7-1623.gd0e6828a.el9.x 18 MB/s | 23 MB 00:01 2024-09-11T08:28:51.176 INFO:teuthology.orchestra.run.smithi145.stdout:(7/24): python3-ceph-argparse-17.2.7-1623.gd0e6 175 kB/s | 45 kB 00:00 2024-09-11T08:28:51.210 INFO:teuthology.orchestra.run.smithi145.stdout:(8/24): python3-ceph-common-17.2.7-1623.gd0e682 2.3 MB/s | 120 kB 00:00 2024-09-11T08:28:51.235 INFO:teuthology.orchestra.run.smithi145.stdout:(9/24): python3-cephfs-17.2.7-1623.gd0e6828a.el 2.6 MB/s | 156 kB 00:00 2024-09-11T08:28:51.260 INFO:teuthology.orchestra.run.smithi145.stdout:(10/24): python3-rados-17.2.7-1623.gd0e6828a.el 6.3 MB/s | 320 kB 00:00 2024-09-11T08:28:51.285 INFO:teuthology.orchestra.run.smithi145.stdout:(11/24): python3-rbd-17.2.7-1623.gd0e6828a.el9. 5.7 MB/s | 294 kB 00:00 2024-09-11T08:28:51.310 INFO:teuthology.orchestra.run.smithi145.stdout:(12/24): python3-rgw-17.2.7-1623.gd0e6828a.el9. 1.9 MB/s | 99 kB 00:00 2024-09-11T08:28:51.461 INFO:teuthology.orchestra.run.smithi145.stdout:(13/24): ceph-radosgw-17.2.7-1623.gd0e6828a.el9 9.4 MB/s | 14 MB 00:01 2024-09-11T08:28:51.519 INFO:teuthology.orchestra.run.smithi145.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 217 kB/s | 45 kB 00:00 2024-09-11T08:28:51.544 INFO:teuthology.orchestra.run.smithi145.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 618 kB/s | 160 kB 00:00 2024-09-11T08:28:51.578 INFO:teuthology.orchestra.run.smithi145.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 1.2 MB/s | 42 kB 00:00 2024-09-11T08:28:51.603 INFO:teuthology.orchestra.run.smithi145.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-11T08:28:51.654 INFO:teuthology.orchestra.run.smithi145.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-09-11T08:28:51.681 INFO:teuthology.orchestra.run.smithi145.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 646 kB/s | 49 kB 00:00 2024-09-11T08:28:51.713 INFO:teuthology.orchestra.run.smithi145.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.3 MB/s | 309 kB 00:00 2024-09-11T08:28:51.738 INFO:teuthology.orchestra.run.smithi145.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 821 kB/s | 67 kB 00:00 2024-09-11T08:28:51.788 INFO:teuthology.orchestra.run.smithi145.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T08:28:51.905 INFO:teuthology.orchestra.run.smithi145.stdout:(23/24): librados2-17.2.7-1623.gd0e6828a.el9.x8 16 MB/s | 3.1 MB 00:00 2024-09-11T08:28:51.972 INFO:teuthology.orchestra.run.smithi145.stdout:(24/24): librbd1-17.2.7-1623.gd0e6828a.el9.x86_ 13 MB/s | 2.9 MB 00:00 2024-09-11T08:28:51.977 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:28:51.977 INFO:teuthology.orchestra.run.smithi145.stdout:Total 26 MB/s | 58 MB 00:02 2024-09-11T08:28:52.037 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:28:52.070 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:28:52.070 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:28:52.533 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:28:52.533 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:28:53.724 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:28:54.116 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-11T08:28:54.182 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:28:54.275 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:28:54.360 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T08:28:54.384 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:28:54.456 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:28:54.508 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-11T08:28:54.539 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-11T08:28:54.985 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-11T08:28:55.020 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T08:28:55.066 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T08:28:55.094 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-11T08:28:55.125 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-11T08:28:55.138 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 10/26 2024-09-11T08:28:55.173 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T08:28:55.236 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T08:28:55.264 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T08:28:55.330 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T08:28:55.385 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-11T08:28:55.423 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-11T08:28:55.455 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-11T08:28:55.750 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-11T08:28:55.793 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T08:28:55.883 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T08:28:55.956 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 19/26 2024-09-11T08:28:55.987 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 20/26 2024-09-11T08:28:57.531 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:28:57.533 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:28:57.533 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-11T08:28:57.533 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:28:57.533 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:28:57.533 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T08:28:57.534 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-09-11T08:28:57.534 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T08:28:57.534 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T08:28:57.534 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-11T08:28:57.534 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T08:28:57.534 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T08:28:57.534 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T08:28:57.534 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T08:28:57.534 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T08:28:57.534 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T08:28:57.534 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T08:28:57.534 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T08:28:57.535 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T08:28:57.535 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T08:28:57.535 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T08:28:57.535 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T08:28:57.535 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T08:28:57.535 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T08:28:57.535 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T08:28:57.535 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T08:28:57.535 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T08:28:57.535 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T08:28:57.535 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T08:28:57.535 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T08:28:57.535 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T08:28:57.536 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:28:57.536 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:28:57.536 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:28:57.536 INFO:teuthology.orchestra.run.smithi092.stdout:Install 22 Packages 2024-09-11T08:28:57.536 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 2 Packages 2024-09-11T08:28:57.536 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:28:57.536 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 58 M 2024-09-11T08:28:57.536 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:28:58.176 INFO:teuthology.orchestra.run.smithi092.stdout:(1/24): ceph-base-17.2.7-1623.gd0e6828a.el9.x86 9.4 MB/s | 5.0 MB 00:00 2024-09-11T08:28:58.234 INFO:teuthology.orchestra.run.smithi092.stdout:(2/24): ceph-selinux-17.2.7-1623.gd0e6828a.el9. 433 kB/s | 25 kB 00:00 2024-09-11T08:28:58.343 INFO:teuthology.orchestra.run.smithi092.stdout:(3/24): libcephfs2-17.2.7-1623.gd0e6828a.el9.x8 5.7 MB/s | 636 kB 00:00 2024-09-11T08:28:58.443 INFO:teuthology.orchestra.run.smithi092.stdout:(4/24): libradosstriper1-17.2.7-1623.gd0e6828a. 4.5 MB/s | 457 kB 00:00 2024-09-11T08:28:58.467 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:28:58.517 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:28:58.852 INFO:teuthology.orchestra.run.smithi092.stdout:(5/24): ceph-common-17.2.7-1623.gd0e6828a.el9.x 19 MB/s | 23 MB 00:01 2024-09-11T08:28:58.994 INFO:teuthology.orchestra.run.smithi092.stdout:(6/24): ceph-radosgw-17.2.7-1623.gd0e6828a.el9. 11 MB/s | 14 MB 00:01 2024-09-11T08:28:59.019 INFO:teuthology.orchestra.run.smithi092.stdout:(7/24): python3-ceph-argparse-17.2.7-1623.gd0e6 271 kB/s | 45 kB 00:00 2024-09-11T08:28:59.044 INFO:teuthology.orchestra.run.smithi092.stdout:(8/24): python3-ceph-common-17.2.7-1623.gd0e682 2.3 MB/s | 120 kB 00:00 2024-09-11T08:28:59.046 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:28:59.070 INFO:teuthology.orchestra.run.smithi092.stdout:(9/24): python3-cephfs-17.2.7-1623.gd0e6828a.el 3.0 MB/s | 156 kB 00:00 2024-09-11T08:28:59.087 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T08:28:59.095 INFO:teuthology.orchestra.run.smithi092.stdout:(10/24): python3-rados-17.2.7-1623.gd0e6828a.el 6.3 MB/s | 320 kB 00:00 2024-09-11T08:28:59.120 INFO:teuthology.orchestra.run.smithi092.stdout:(11/24): python3-rbd-17.2.7-1623.gd0e6828a.el9. 5.8 MB/s | 294 kB 00:00 2024-09-11T08:28:59.145 INFO:teuthology.orchestra.run.smithi092.stdout:(12/24): python3-rgw-17.2.7-1623.gd0e6828a.el9. 1.9 MB/s | 99 kB 00:00 2024-09-11T08:28:59.191 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T08:28:59.192 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T08:28:59.192 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T08:28:59.192 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:28:59.220 INFO:teuthology.orchestra.run.smithi092.stdout:(13/24): librgw2-17.2.7-1623.gd0e6828a.el9.x86_ 6.3 MB/s | 4.9 MB 00:00 2024-09-11T08:28:59.221 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:28:59.287 INFO:teuthology.orchestra.run.smithi092.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 959 kB/s | 160 kB 00:00 2024-09-11T08:28:59.313 INFO:teuthology.orchestra.run.smithi092.stdout:(15/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 271 kB/s | 45 kB 00:00 2024-09-11T08:28:59.355 INFO:teuthology.orchestra.run.smithi092.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-09-11T08:28:59.380 INFO:teuthology.orchestra.run.smithi092.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 632 kB/s | 42 kB 00:00 2024-09-11T08:28:59.430 INFO:teuthology.orchestra.run.smithi092.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-09-11T08:28:59.464 INFO:teuthology.orchestra.run.smithi092.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 581 kB/s | 49 kB 00:00 2024-09-11T08:28:59.489 INFO:teuthology.orchestra.run.smithi092.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.3 MB/s | 309 kB 00:00 2024-09-11T08:28:59.514 INFO:teuthology.orchestra.run.smithi092.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 821 kB/s | 67 kB 00:00 2024-09-11T08:28:59.565 INFO:teuthology.orchestra.run.smithi092.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-11T08:28:59.707 INFO:teuthology.orchestra.run.smithi092.stdout:(23/24): librbd1-17.2.7-1623.gd0e6828a.el9.x86_ 15 MB/s | 2.9 MB 00:00 2024-09-11T08:28:59.757 INFO:teuthology.orchestra.run.smithi092.stdout:(24/24): librados2-17.2.7-1623.gd0e6828a.el9.x8 12 MB/s | 3.1 MB 00:00 2024-09-11T08:28:59.762 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:28:59.762 INFO:teuthology.orchestra.run.smithi092.stdout:Total 26 MB/s | 58 MB 00:02 2024-09-11T08:28:59.825 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:28:59.858 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:28:59.858 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:29:00.330 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:29:00.330 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:29:01.538 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:29:01.925 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-11T08:29:02.009 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:29:02.101 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:29:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T08:29:02.219 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:29:02.308 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:29:02.366 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-11T08:29:02.399 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-11T08:29:02.841 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-11T08:29:02.902 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T08:29:02.968 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T08:29:02.996 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-11T08:29:03.039 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 10/26 2024-09-11T08:29:03.076 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T08:29:03.139 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T08:29:03.166 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T08:29:03.231 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T08:29:03.281 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-11T08:29:03.324 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-11T08:29:03.357 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-11T08:29:03.651 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-11T08:29:03.703 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T08:29:03.818 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T08:29:03.900 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 19/26 2024-09-11T08:29:03.930 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 20/26 2024-09-11T08:29:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:29:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:29:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:29:07.038 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T08:29:07.131 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T08:29:07.132 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T08:29:07.132 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T08:29:07.132 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:07.173 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:29:08.973 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:29:08.973 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /sys 2024-09-11T08:29:08.973 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /proc 2024-09-11T08:29:08.973 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /mnt 2024-09-11T08:29:08.973 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /var/tmp 2024-09-11T08:29:08.973 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /home 2024-09-11T08:29:08.973 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /root 2024-09-11T08:29:08.973 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /tmp 2024-09-11T08:29:08.974 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:10.313 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T08:29:10.383 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T08:29:10.383 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:29:10.383 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T08:29:10.383 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:29:10.384 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:29:10.384 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:10.385 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T08:29:10.482 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T08:29:10.482 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:29:11.742 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:29:11.742 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/26 2024-09-11T08:29:11.742 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:29:11.742 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T08:29:11.742 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:29:11.742 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/26 2024-09-11T08:29:11.742 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 6/26 2024-09-11T08:29:11.742 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/26 2024-09-11T08:29:11.742 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 8/26 2024-09-11T08:29:11.742 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 9/26 2024-09-11T08:29:11.743 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/26 2024-09-11T08:29:11.743 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T08:29:11.743 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T08:29:11.743 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 13/26 2024-09-11T08:29:11.743 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-11T08:29:11.743 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-11T08:29:11.743 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-11T08:29:11.743 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-11T08:29:11.743 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-11T08:29:11.743 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-11T08:29:11.743 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-11T08:29:11.743 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-11T08:29:11.743 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-11T08:29:11.744 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:29:11.744 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-11T08:29:11.744 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/26 2024-09-11T08:29:12.620 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:29:12.620 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:12.620 INFO:teuthology.orchestra.run.smithi145.stdout:Upgraded: 2024-09-11T08:29:12.620 INFO:teuthology.orchestra.run.smithi145.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:12.620 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:12.620 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:29:12.620 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:12.620 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:12.621 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:12.621 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:12.621 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T08:29:12.621 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:12.621 INFO:teuthology.orchestra.run.smithi145.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T08:29:12.621 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T08:29:12.621 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T08:29:12.621 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:12.621 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T08:29:12.621 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:12.621 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T08:29:12.621 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T08:29:12.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:12.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:12.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:12.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T08:29:12.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:12.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:12.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:12.622 INFO:teuthology.orchestra.run.smithi145.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T08:29:12.622 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:12.622 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:29:12.983 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-test 2024-09-11T08:29:13.546 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:26 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:29:13.684 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:29:13.685 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:29:13.685 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-09-11T08:29:13.685 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:29:13.686 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:29:13.686 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T08:29:13.686 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-09-11T08:29:13.686 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T08:29:13.686 INFO:teuthology.orchestra.run.smithi145.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T08:29:13.686 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T08:29:13.687 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:13.687 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:29:13.687 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:29:13.687 INFO:teuthology.orchestra.run.smithi145.stdout:Install 4 Packages 2024-09-11T08:29:13.687 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:13.688 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 48 M 2024-09-11T08:29:13.688 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 193 M 2024-09-11T08:29:13.688 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:29:13.960 INFO:teuthology.orchestra.run.smithi145.stdout:(1/4): libcephsqlite-17.2.7-1623.gd0e6828a.el9. 731 kB/s | 163 kB 00:00 2024-09-11T08:29:14.335 INFO:teuthology.orchestra.run.smithi145.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 170 kB/s | 64 kB 00:00 2024-09-11T08:29:14.377 INFO:teuthology.orchestra.run.smithi145.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 475 kB/s | 304 kB 00:00 2024-09-11T08:29:15.252 INFO:teuthology.orchestra.run.smithi145.stdout:(4/4): ceph-test-17.2.7-1623.gd0e6828a.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-11T08:29:15.254 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:29:15.255 INFO:teuthology.orchestra.run.smithi145.stdout:Total 31 MB/s | 48 MB 00:01 2024-09-11T08:29:15.282 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:29:15.300 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:29:15.300 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:29:15.665 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:29:15.666 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:29:16.180 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:29:16.250 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T08:29:16.292 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T08:29:16.329 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T08:29:17.042 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:29:17.043 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-09-11T08:29:17.043 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-09-11T08:29:17.043 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-09-11T08:29:17.043 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-09-11T08:29:17.043 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-09-11T08:29:17.043 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-09-11T08:29:17.043 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-09-11T08:29:17.043 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:18.405 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T08:29:18.474 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T08:29:18.474 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:29:18.475 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T08:29:18.475 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:29:18.475 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:29:18.475 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:18.476 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T08:29:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T08:29:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:29:19.864 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:29:19.864 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/26 2024-09-11T08:29:19.864 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:29:19.864 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T08:29:19.865 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:29:19.865 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/26 2024-09-11T08:29:19.865 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 6/26 2024-09-11T08:29:19.865 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/26 2024-09-11T08:29:19.865 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 8/26 2024-09-11T08:29:19.865 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 9/26 2024-09-11T08:29:19.865 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/26 2024-09-11T08:29:19.865 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T08:29:19.865 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T08:29:19.865 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 13/26 2024-09-11T08:29:19.865 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-11T08:29:19.865 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-11T08:29:19.866 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-11T08:29:19.866 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-11T08:29:19.866 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-11T08:29:19.866 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-11T08:29:19.866 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-11T08:29:19.866 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-11T08:29:19.866 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-11T08:29:19.866 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:29:19.866 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-11T08:29:19.866 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/26 2024-09-11T08:29:20.598 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:29:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:20.617 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-09-11T08:29:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:20.617 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:29:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:20.617 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T08:29:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T08:29:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T08:29:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T08:29:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T08:29:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T08:29:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T08:29:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:20.618 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T08:29:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T08:29:20.619 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:20.619 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:29:20.733 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T08:29:20.810 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T08:29:20.963 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-test 2024-09-11T08:29:21.526 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:26 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:29:21.657 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:29:21.658 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:29:21.658 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-11T08:29:21.658 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:29:21.658 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:29:21.658 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T08:29:21.658 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-11T08:29:21.658 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T08:29:21.658 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T08:29:21.659 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T08:29:21.659 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:21.659 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:29:21.659 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:29:21.659 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2024-09-11T08:29:21.659 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:21.659 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 48 M 2024-09-11T08:29:21.660 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 193 M 2024-09-11T08:29:21.660 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:29:22.094 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.6 MB/s | 304 kB 00:00 2024-09-11T08:29:22.119 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): libcephsqlite-17.2.7-1623.gd0e6828a.el9. 781 kB/s | 163 kB 00:00 2024-09-11T08:29:22.144 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-11T08:29:23.040 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T08:29:23.040 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/4 2024-09-11T08:29:23.040 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/4 2024-09-11T08:29:23.040 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T08:29:23.348 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T08:29:23.348 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:23.348 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:29:23.348 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:23.348 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:23.348 INFO:teuthology.orchestra.run.smithi145.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T08:29:23.349 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T08:29:23.349 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:23.349 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:29:23.436 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): ceph-test-17.2.7-1623.gd0e6828a.el9.x86_ 31 MB/s | 48 MB 00:01 2024-09-11T08:29:23.438 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:29:23.438 INFO:teuthology.orchestra.run.smithi092.stdout:Total 27 MB/s | 48 MB 00:01 2024-09-11T08:29:23.465 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:29:23.483 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:29:23.483 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:29:23.612 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph 2024-09-11T08:29:23.849 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:29:23.849 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:29:24.177 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:37 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:29:24.318 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:29:24.320 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:29:24.320 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-09-11T08:29:24.320 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:29:24.320 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:29:24.320 INFO:teuthology.orchestra.run.smithi145.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T08:29:24.320 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-09-11T08:29:24.321 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T08:29:24.321 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T08:29:24.321 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T08:29:24.321 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T08:29:24.321 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T08:29:24.321 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T08:29:24.321 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T08:29:24.321 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T08:29:24.321 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T08:29:24.321 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T08:29:24.321 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T08:29:24.322 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T08:29:24.322 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections 2024-09-11T08:29:24.322 INFO:teuthology.orchestra.run.smithi145.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-11T08:29:24.322 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T08:29:24.322 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T08:29:24.322 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T08:29:24.322 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T08:29:24.322 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T08:29:24.322 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T08:29:24.322 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T08:29:24.322 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T08:29:24.323 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T08:29:24.323 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T08:29:24.323 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T08:29:24.323 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T08:29:24.323 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T08:29:24.323 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T08:29:24.323 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T08:29:24.323 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T08:29:24.323 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T08:29:24.323 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T08:29:24.323 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T08:29:24.323 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T08:29:24.323 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:24.324 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:29:24.324 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:29:24.324 INFO:teuthology.orchestra.run.smithi145.stdout:Install 34 Packages 2024-09-11T08:29:24.324 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:24.324 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 27 M 2024-09-11T08:29:24.324 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 96 M 2024-09-11T08:29:24.325 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:29:24.393 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:29:24.471 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T08:29:24.518 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T08:29:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T08:29:24.981 INFO:teuthology.orchestra.run.smithi145.stdout:(1/34): ceph-17.2.7-1623.gd0e6828a.el9.x86_64.r 51 kB/s | 6.5 kB 00:00 2024-09-11T08:29:25.164 INFO:teuthology.orchestra.run.smithi145.stdout:(2/34): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86_ 6.7 MB/s | 2.1 MB 00:00 2024-09-11T08:29:25.206 INFO:teuthology.orchestra.run.smithi145.stdout:(3/34): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86_ 4.0 MB/s | 1.4 MB 00:00 2024-09-11T08:29:25.306 INFO:teuthology.orchestra.run.smithi145.stdout:(4/34): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86_ 12 MB/s | 3.9 MB 00:00 2024-09-11T08:29:25.331 INFO:teuthology.orchestra.run.smithi145.stdout:(5/34): ceph-mgr-modules-core-17.2.7-1623.gd0e6 1.9 MB/s | 240 kB 00:00 2024-09-11T08:29:25.449 INFO:teuthology.orchestra.run.smithi145.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-09-11T08:29:25.490 INFO:teuthology.orchestra.run.smithi145.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-11T08:29:25.549 INFO:teuthology.orchestra.run.smithi145.stdout:(8/34): python3-cryptography-36.0.1-4.el9.x86_6 5.6 MB/s | 1.2 MB 00:00 2024-09-11T08:29:25.708 INFO:teuthology.orchestra.run.smithi145.stdout:(9/34): ceph-osd-17.2.7-1623.gd0e6828a.el9.x86_ 28 MB/s | 15 MB 00:00 2024-09-11T08:29:25.741 INFO:teuthology.orchestra.run.smithi145.stdout:(10/34): python3-pycparser-2.20-6.el9.noarch.rp 539 kB/s | 135 kB 00:00 2024-09-11T08:29:25.767 INFO:teuthology.orchestra.run.smithi145.stdout:(11/34): python3-requests-2.25.1-8.el9.noarch.r 574 kB/s | 125 kB 00:00 2024-09-11T08:29:25.834 INFO:teuthology.orchestra.run.smithi145.stdout:(12/34): python3-markupsafe-1.1.1-12.el9.x86_64 523 kB/s | 35 kB 00:00 2024-09-11T08:29:25.859 INFO:teuthology.orchestra.run.smithi145.stdout:(13/34): python3-urllib3-1.26.5-6.el9.noarch.rp 1.4 MB/s | 215 kB 00:00 2024-09-11T08:29:25.885 INFO:teuthology.orchestra.run.smithi145.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-11T08:29:25.909 INFO:teuthology.orchestra.run.smithi145.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 1.0 MB/s | 77 kB 00:00 2024-09-11T08:29:25.935 INFO:teuthology.orchestra.run.smithi145.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 826 kB/s | 42 kB 00:00 2024-09-11T08:29:25.960 INFO:teuthology.orchestra.run.smithi145.stdout:(17/34): python3-pytz-2021.1-5.el9.noarch.rpm 506 kB/s | 51 kB 00:00 2024-09-11T08:29:26.010 INFO:teuthology.orchestra.run.smithi145.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 433 kB/s | 43 kB 00:00 2024-09-11T08:29:26.035 INFO:teuthology.orchestra.run.smithi145.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 464 kB/s | 11 kB 00:00 2024-09-11T08:29:26.069 INFO:teuthology.orchestra.run.smithi145.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-11T08:29:26.094 INFO:teuthology.orchestra.run.smithi145.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 302 kB/s | 18 kB 00:00 2024-09-11T08:29:26.119 INFO:teuthology.orchestra.run.smithi145.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 936 kB/s | 23 kB 00:00 2024-09-11T08:29:26.145 INFO:teuthology.orchestra.run.smithi145.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 389 kB/s | 19 kB 00:00 2024-09-11T08:29:26.170 INFO:teuthology.orchestra.run.smithi145.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-11T08:29:26.195 INFO:teuthology.orchestra.run.smithi145.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 921 kB/s | 46 kB 00:00 2024-09-11T08:29:26.229 INFO:teuthology.orchestra.run.smithi145.stdout:(26/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-11T08:29:26.254 INFO:teuthology.orchestra.run.smithi145.stdout:(27/34): python3-more-itertools-8.12.0-2.el9.no 937 kB/s | 79 kB 00:00 2024-09-11T08:29:26.280 INFO:teuthology.orchestra.run.smithi145.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-11T08:29:26.305 INFO:teuthology.orchestra.run.smithi145.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-11T08:29:26.330 INFO:teuthology.orchestra.run.smithi145.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 716 kB/s | 36 kB 00:00 2024-09-11T08:29:26.355 INFO:teuthology.orchestra.run.smithi145.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-11T08:29:26.380 INFO:teuthology.orchestra.run.smithi145.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 801 kB/s | 20 kB 00:00 2024-09-11T08:29:26.414 INFO:teuthology.orchestra.run.smithi145.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-11T08:29:26.439 INFO:teuthology.orchestra.run.smithi145.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-11T08:29:26.443 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:29:26.443 INFO:teuthology.orchestra.run.smithi145.stdout:Total 13 MB/s | 27 MB 00:02 2024-09-11T08:29:26.571 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:29:26.608 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:29:26.609 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:29:26.995 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:29:26.996 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:29:28.231 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:29:28.309 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-11T08:29:28.349 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-11T08:29:28.383 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-11T08:29:28.474 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-11T08:29:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-11T08:29:28.593 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-11T08:29:28.637 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-11T08:29:28.675 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-11T08:29:28.710 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-11T08:29:28.743 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-11T08:29:28.784 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-11T08:29:28.835 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-11T08:29:28.870 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-11T08:29:28.918 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-11T08:29:28.950 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T08:29:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T08:29:29.024 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-11T08:29:29.069 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T08:29:29.163 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-11T08:29:29.242 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-11T08:29:29.303 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-11T08:29:29.341 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-11T08:29:29.390 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-11T08:29:29.446 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-11T08:29:29.632 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-11T08:29:29.687 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-11T08:29:29.765 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-11T08:29:29.862 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-11T08:29:29.910 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-11T08:29:30.026 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-11T08:29:30.183 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 29/34 2024-09-11T08:29:30.213 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T08:29:30.240 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T08:29:30.240 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:29:30.240 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T08:29:30.240 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:29:30.240 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:29:30.240 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T08:29:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/4 2024-09-11T08:29:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/4 2024-09-11T08:29:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T08:29:31.607 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T08:29:31.635 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T08:29:31.635 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:29:31.636 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T08:29:31.636 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:29:31.636 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:29:31.636 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:31.724 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T08:29:31.725 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:31.725 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:29:31.725 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:31.725 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:31.725 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T08:29:31.725 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T08:29:31.725 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:31.725 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:29:31.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph 2024-09-11T08:29:32.024 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T08:29:32.048 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T08:29:32.049 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:29:32.049 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T08:29:32.049 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:29:32.049 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:29:32.049 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:32.258 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T08:29:32.282 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T08:29:32.282 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:29:32.282 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T08:29:32.282 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:29:32.282 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:29:32.282 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:32.346 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T08:29:32.558 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:37 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:29:32.694 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:29:32.697 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:29:32.697 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-11T08:29:32.697 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:29:32.697 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:29:32.697 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T08:29:32.697 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-11T08:29:32.697 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T08:29:32.697 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T08:29:32.697 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T08:29:32.697 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T08:29:32.697 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T08:29:32.697 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T08:29:32.698 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T08:29:32.698 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T08:29:32.698 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T08:29:32.698 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T08:29:32.698 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T08:29:32.698 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T08:29:32.698 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections 2024-09-11T08:29:32.698 INFO:teuthology.orchestra.run.smithi092.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-11T08:29:32.698 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T08:29:32.698 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T08:29:32.698 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T08:29:32.698 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T08:29:32.698 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T08:29:32.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T08:29:32.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T08:29:32.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T08:29:32.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T08:29:32.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T08:29:32.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T08:29:32.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T08:29:32.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T08:29:32.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T08:29:32.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T08:29:32.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T08:29:32.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T08:29:32.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T08:29:32.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T08:29:32.700 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T08:29:32.700 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:32.700 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:29:32.700 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:29:32.700 INFO:teuthology.orchestra.run.smithi092.stdout:Install 34 Packages 2024-09-11T08:29:32.700 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:32.700 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 27 M 2024-09-11T08:29:32.701 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 96 M 2024-09-11T08:29:32.701 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:29:33.030 INFO:teuthology.orchestra.run.smithi092.stdout:(1/34): ceph-17.2.7-1623.gd0e6828a.el9.x86_64.r 47 kB/s | 6.5 kB 00:00 2024-09-11T08:29:33.214 INFO:teuthology.orchestra.run.smithi092.stdout:(2/34): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86_ 6.4 MB/s | 2.1 MB 00:00 2024-09-11T08:29:33.256 INFO:teuthology.orchestra.run.smithi092.stdout:(3/34): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86_ 3.9 MB/s | 1.4 MB 00:00 2024-09-11T08:29:33.331 INFO:teuthology.orchestra.run.smithi092.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1623.gd0e6 3.1 MB/s | 240 kB 00:00 2024-09-11T08:29:33.540 INFO:teuthology.orchestra.run.smithi092.stdout:(5/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-11T08:29:33.607 INFO:teuthology.orchestra.run.smithi092.stdout:(6/34): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86_ 6.8 MB/s | 3.9 MB 00:00 2024-09-11T08:29:33.807 INFO:teuthology.orchestra.run.smithi092.stdout:(7/34): ceph-osd-17.2.7-1623.gd0e6828a.el9.x86_ 26 MB/s | 15 MB 00:00 2024-09-11T08:29:33.915 INFO:teuthology.orchestra.run.smithi092.stdout:(8/34): python3-cryptography-36.0.1-4.el9.x86_6 3.3 MB/s | 1.2 MB 00:00 2024-09-11T08:29:33.966 INFO:teuthology.orchestra.run.smithi092.stdout:(9/34): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-11T08:29:33.991 INFO:teuthology.orchestra.run.smithi092.stdout:(10/34): python3-ply-3.11-14.el9.noarch.rpm 277 kB/s | 106 kB 00:00 2024-09-11T08:29:34.025 INFO:teuthology.orchestra.run.smithi092.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-11T08:29:34.040 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T08:29:34.040 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/34 2024-09-11T08:29:34.040 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/34 2024-09-11T08:29:34.040 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/34 2024-09-11T08:29:34.040 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/34 2024-09-11T08:29:34.040 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/34 2024-09-11T08:29:34.040 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 6/34 2024-09-11T08:29:34.040 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-11T08:29:34.040 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-11T08:29:34.040 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-11T08:29:34.040 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-11T08:29:34.041 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-11T08:29:34.041 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-11T08:29:34.041 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-11T08:29:34.041 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-11T08:29:34.041 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T08:29:34.041 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-11T08:29:34.041 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-11T08:29:34.041 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-11T08:29:34.041 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-11T08:29:34.041 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-11T08:29:34.041 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-11T08:29:34.041 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-11T08:29:34.041 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-11T08:29:34.041 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-11T08:29:34.042 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-11T08:29:34.042 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-11T08:29:34.042 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-11T08:29:34.042 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-11T08:29:34.042 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-11T08:29:34.042 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-11T08:29:34.042 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-11T08:29:34.042 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-11T08:29:34.042 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-11T08:29:34.058 INFO:teuthology.orchestra.run.smithi092.stdout:(12/34): python3-pycparser-2.20-6.el9.noarch.rp 539 kB/s | 135 kB 00:00 2024-09-11T08:29:34.575 INFO:teuthology.orchestra.run.smithi092.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 63 kB/s | 35 kB 00:00 2024-09-11T08:29:34.700 INFO:teuthology.orchestra.run.smithi092.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 120 kB/s | 77 kB 00:00 2024-09-11T08:29:34.726 INFO:teuthology.orchestra.run.smithi092.stdout:(15/34): python3-mako-1.1.4-6.el9.noarch.rpm 234 kB/s | 172 kB 00:00 2024-09-11T08:29:34.768 INFO:teuthology.orchestra.run.smithi092.stdout:(16/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.0 MB/s | 43 kB 00:00 2024-09-11T08:29:34.793 INFO:teuthology.orchestra.run.smithi092.stdout:(17/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 6.7 MB/s | 172 kB 00:00 2024-09-11T08:29:34.827 INFO:teuthology.orchestra.run.smithi092.stdout:(18/34): python3-cherrypy-18.6.1-2.el9.noarch.r 11 MB/s | 358 kB 00:00 2024-09-11T08:29:34.860 INFO:teuthology.orchestra.run.smithi092.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 410 kB/s | 11 kB 00:00 2024-09-11T08:29:34.885 INFO:teuthology.orchestra.run.smithi092.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 707 kB/s | 18 kB 00:00 2024-09-11T08:29:34.911 INFO:teuthology.orchestra.run.smithi092.stdout:(21/34): python3-jaraco-collections-3.0.0-8.el9 932 kB/s | 23 kB 00:00 2024-09-11T08:29:34.936 INFO:teuthology.orchestra.run.smithi092.stdout:(22/34): python3-jaraco-functools-3.5.0-2.el9.n 779 kB/s | 19 kB 00:00 2024-09-11T08:29:34.961 INFO:teuthology.orchestra.run.smithi092.stdout:(23/34): python3-jaraco-text-3.2.0-6.el9.noarch 810 kB/s | 20 kB 00:00 2024-09-11T08:29:34.987 INFO:teuthology.orchestra.run.smithi092.stdout:(24/34): python3-logutils-0.3.5-21.el9.noarch.r 1.8 MB/s | 46 kB 00:00 2024-09-11T08:29:35.020 INFO:teuthology.orchestra.run.smithi092.stdout:(25/34): python3-more-itertools-8.12.0-2.el9.no 3.0 MB/s | 79 kB 00:00 2024-09-11T08:29:35.054 INFO:teuthology.orchestra.run.smithi092.stdout:(26/34): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-11T08:29:35.067 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-11T08:29:35.068 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:35.068 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:29:35.068 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:35.068 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:35.068 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:35.068 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:29:35.068 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:35.068 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:35.068 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T08:29:35.068 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T08:29:35.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T08:29:35.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T08:29:35.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T08:29:35.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T08:29:35.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T08:29:35.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T08:29:35.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T08:29:35.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T08:29:35.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T08:29:35.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T08:29:35.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T08:29:35.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T08:29:35.069 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T08:29:35.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T08:29:35.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T08:29:35.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T08:29:35.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T08:29:35.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T08:29:35.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T08:29:35.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T08:29:35.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T08:29:35.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T08:29:35.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T08:29:35.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T08:29:35.070 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T08:29:35.071 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T08:29:35.071 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:35.071 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:29:35.079 INFO:teuthology.orchestra.run.smithi092.stdout:(27/34): python3-portend-3.1.0-2.el9.noarch.rpm 657 kB/s | 16 kB 00:00 2024-09-11T08:29:35.105 INFO:teuthology.orchestra.run.smithi092.stdout:(28/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.6 MB/s | 90 kB 00:00 2024-09-11T08:29:35.121 INFO:teuthology.orchestra.run.smithi092.stdout:(29/34): python3-tempora-5.0.0-2.el9.noarch.rpm 2.1 MB/s | 36 kB 00:00 2024-09-11T08:29:35.163 INFO:teuthology.orchestra.run.smithi092.stdout:(30/34): python3-webob-1.8.8-2.el9.noarch.rpm 5.5 MB/s | 230 kB 00:00 2024-09-11T08:29:35.196 INFO:teuthology.orchestra.run.smithi092.stdout:(31/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 13 MB/s | 427 kB 00:00 2024-09-11T08:29:35.222 INFO:teuthology.orchestra.run.smithi092.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 792 kB/s | 20 kB 00:00 2024-09-11T08:29:35.272 INFO:teuthology.orchestra.run.smithi092.stdout:(33/34): python3-pytz-2021.1-5.el9.noarch.rpm 73 kB/s | 51 kB 00:00 2024-09-11T08:29:35.397 INFO:teuthology.orchestra.run.smithi092.stdout:(34/34): python3-toml-0.10.2-6.el9.noarch.rpm 60 kB/s | 42 kB 00:00 2024-09-11T08:29:35.402 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:29:35.402 INFO:teuthology.orchestra.run.smithi092.stdout:Total 10 MB/s | 27 MB 00:02 2024-09-11T08:29:35.421 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-base 2024-09-11T08:29:35.533 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:29:35.570 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:29:35.570 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:29:35.953 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:29:35.954 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:29:35.991 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:48 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:29:36.081 INFO:teuthology.orchestra.run.smithi145.stdout:Package ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:29:36.123 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:29:36.124 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2024-09-11T08:29:36.124 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:29:36.183 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install cephadm 2024-09-11T08:29:36.745 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:49 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:29:36.878 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:29:36.879 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:29:36.879 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-09-11T08:29:36.879 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:29:36.879 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:29:36.879 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T08:29:36.880 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:36.880 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:29:36.880 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:29:36.880 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2024-09-11T08:29:36.880 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:36.880 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 83 k 2024-09-11T08:29:36.880 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 375 k 2024-09-11T08:29:36.880 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:29:37.083 INFO:teuthology.orchestra.run.smithi145.stdout:cephadm-17.2.7-1623.gd0e6828a.el9.noarch.rpm 407 kB/s | 83 kB 00:00 2024-09-11T08:29:37.084 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:29:37.084 INFO:teuthology.orchestra.run.smithi145.stdout:Total 404 kB/s | 83 kB 00:00 2024-09-11T08:29:37.084 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:29:37.089 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:29:37.090 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:29:37.091 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:29:37.092 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:29:37.168 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:29:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:29:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-11T08:29:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-11T08:29:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-11T08:29:37.408 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-11T08:29:37.504 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-11T08:29:37.532 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:29:37.543 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-11T08:29:37.578 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-11T08:29:37.608 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-11T08:29:37.638 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:29:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-11T08:29:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-11T08:29:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-11T08:29:37.776 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-11T08:29:37.810 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-11T08:29:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-11T08:29:37.899 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T08:29:37.973 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-11T08:29:38.042 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:29:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-11T08:29:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-11T08:29:38.262 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:29:38.262 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:38.262 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:29:38.262 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:29:38.262 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:38.263 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:29:38.269 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-11T08:29:38.307 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-11T08:29:38.355 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-11T08:29:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-11T08:29:38.462 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T08:29:38.599 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-11T08:29:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-11T08:29:38.723 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-11T08:29:38.811 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-11T08:29:38.857 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-11T08:29:38.975 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-11T08:29:39.031 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:52 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:29:39.130 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 29/34 2024-09-11T08:29:39.166 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:29:39.167 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:29:39.167 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repo Size 2024-09-11T08:29:39.167 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:29:39.167 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:29:39.167 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T08:29:39.167 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:39.167 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:29:39.167 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:29:39.167 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2024-09-11T08:29:39.167 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:39.167 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 137 k 2024-09-11T08:29:39.168 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 379 k 2024-09-11T08:29:39.168 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:29:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T08:29:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T08:29:39.196 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:29:39.196 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T08:29:39.196 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:29:39.196 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:29:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:39.398 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-immutable-object-cache-17.2.7-1623.gd0e682 599 kB/s | 137 kB 00:00 2024-09-11T08:29:39.398 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:29:39.398 INFO:teuthology.orchestra.run.smithi145.stdout:Total 594 kB/s | 137 kB 00:00 2024-09-11T08:29:39.399 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:29:39.406 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:29:39.406 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:29:39.463 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:29:39.463 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:29:39.620 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:29:39.724 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:29:39.749 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:29:39.749 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:29:39.750 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T08:29:39.750 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:40.563 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T08:29:40.592 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T08:29:40.592 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:29:40.593 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T08:29:40.593 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:29:40.593 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:29:40.593 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:40.689 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:29:40.689 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:40.689 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:29:40.689 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:40.689 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:40.689 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:29:40.904 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-mgr 2024-09-11T08:29:40.981 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T08:29:41.006 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T08:29:41.006 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:29:41.006 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T08:29:41.006 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:29:41.006 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:29:41.006 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:41.207 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T08:29:41.230 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T08:29:41.231 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:29:41.231 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T08:29:41.231 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:29:41.231 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:29:41.231 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:41.295 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T08:29:41.483 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:54 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:29:41.578 INFO:teuthology.orchestra.run.smithi145.stdout:Package ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:29:41.619 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:29:41.620 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2024-09-11T08:29:41.621 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:29:41.682 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T08:29:42.257 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:55 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:29:42.390 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:29:42.391 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:29:42.391 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-09-11T08:29:42.391 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:29:42.391 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:29:42.391 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T08:29:42.391 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-09-11T08:29:42.391 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T08:29:42.391 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T08:29:42.392 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T08:29:42.392 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T08:29:42.392 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T08:29:42.392 INFO:teuthology.orchestra.run.smithi145.stdout:Installing weak dependencies: 2024-09-11T08:29:42.392 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T08:29:42.393 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:42.393 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:29:42.393 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:29:42.393 INFO:teuthology.orchestra.run.smithi145.stdout:Install 7 Packages 2024-09-11T08:29:42.393 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:42.394 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 3.7 M 2024-09-11T08:29:42.394 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 80 M 2024-09-11T08:29:42.395 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:29:42.593 INFO:teuthology.orchestra.run.smithi145.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1623.gd0e 154 kB/s | 24 kB 00:00 2024-09-11T08:29:42.626 INFO:teuthology.orchestra.run.smithi145.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1623.gd0e6 76 kB/s | 15 kB 00:00 2024-09-11T08:29:42.668 INFO:teuthology.orchestra.run.smithi145.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 120 kB/s | 9.0 kB 00:00 2024-09-11T08:29:42.694 INFO:teuthology.orchestra.run.smithi145.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-11T08:29:42.719 INFO:teuthology.orchestra.run.smithi145.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 444 kB/s | 41 kB 00:00 2024-09-11T08:29:42.744 INFO:teuthology.orchestra.run.smithi145.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-11T08:29:42.861 INFO:teuthology.orchestra.run.smithi145.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1623.gd0e6828a 7.9 MB/s | 3.4 MB 00:00 2024-09-11T08:29:42.864 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:29:42.865 INFO:teuthology.orchestra.run.smithi145.stdout:Total 7.7 MB/s | 3.7 MB 00:00 2024-09-11T08:29:42.865 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:29:42.873 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:29:42.873 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:29:42.967 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:29:42.967 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:29:42.997 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T08:29:42.998 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/34 2024-09-11T08:29:42.998 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/34 2024-09-11T08:29:42.998 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/34 2024-09-11T08:29:42.998 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/34 2024-09-11T08:29:42.998 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/34 2024-09-11T08:29:42.998 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 6/34 2024-09-11T08:29:42.998 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-11T08:29:42.998 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-11T08:29:42.998 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-11T08:29:42.998 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-11T08:29:42.998 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-11T08:29:42.998 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-11T08:29:42.999 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-11T08:29:42.999 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-11T08:29:42.999 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T08:29:42.999 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-11T08:29:42.999 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-11T08:29:42.999 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-11T08:29:42.999 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-11T08:29:42.999 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-11T08:29:42.999 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-11T08:29:42.999 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-11T08:29:42.999 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-11T08:29:42.999 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-11T08:29:42.999 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-11T08:29:43.000 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-11T08:29:43.001 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-11T08:29:43.001 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-11T08:29:43.001 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-11T08:29:43.001 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-11T08:29:43.001 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-11T08:29:43.001 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-11T08:29:43.001 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-11T08:29:43.300 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:29:43.362 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-11T08:29:43.396 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-11T08:29:43.442 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-11T08:29:43.483 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-11T08:29:43.512 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 5/7 2024-09-11T08:29:43.918 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-11T08:29:43.918 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:43.919 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:29:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:43.919 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:29:43.920 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:43.920 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:43.920 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T08:29:43.920 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T08:29:43.920 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T08:29:43.920 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T08:29:43.921 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T08:29:43.921 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T08:29:43.921 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T08:29:43.921 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T08:29:43.921 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T08:29:43.921 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T08:29:43.922 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T08:29:43.922 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T08:29:43.922 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T08:29:43.922 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T08:29:43.922 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T08:29:43.923 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T08:29:43.923 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T08:29:43.923 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T08:29:43.923 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T08:29:43.923 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T08:29:43.924 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T08:29:43.924 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T08:29:43.924 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T08:29:43.924 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T08:29:43.924 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T08:29:43.924 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T08:29:43.925 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T08:29:43.925 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T08:29:43.925 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:43.925 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:29:44.304 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-base 2024-09-11T08:29:44.500 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 6/7 2024-09-11T08:29:44.549 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T08:29:44.869 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:49 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:29:44.960 INFO:teuthology.orchestra.run.smithi092.stdout:Package ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:29:45.002 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:29:45.003 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-11T08:29:45.003 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:29:45.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install cephadm 2024-09-11T08:29:45.533 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T08:29:45.533 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 1/7 2024-09-11T08:29:45.533 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 2/7 2024-09-11T08:29:45.533 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 3/7 2024-09-11T08:29:45.533 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-11T08:29:45.533 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-11T08:29:45.533 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-11T08:29:45.629 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:50 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:29:45.760 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:29:45.761 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:29:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-11T08:29:45.761 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:29:45.761 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:29:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T08:29:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:45.761 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:29:45.761 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:29:45.761 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-11T08:29:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:45.762 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 83 k 2024-09-11T08:29:45.762 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 375 k 2024-09-11T08:29:45.762 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:29:45.802 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-11T08:29:45.802 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:45.803 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:29:45.803 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:29:45.803 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:29:45.803 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:29:45.803 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T08:29:45.803 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T08:29:45.803 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T08:29:45.803 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T08:29:45.803 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:45.803 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:29:45.939 INFO:teuthology.orchestra.run.smithi092.stdout:cephadm-17.2.7-1623.gd0e6828a.el9.noarch.rpm 465 kB/s | 83 kB 00:00 2024-09-11T08:29:45.940 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:29:45.940 INFO:teuthology.orchestra.run.smithi092.stdout:Total 461 kB/s | 83 kB 00:00 2024-09-11T08:29:45.940 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:29:45.945 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:29:45.946 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:29:45.947 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:29:45.948 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:29:46.013 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T08:29:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:29:46.430 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:29:46.526 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:29:46.571 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:59 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:29:46.704 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:29:46.705 INFO:teuthology.orchestra.run.smithi145.stdout:======================================================================================= 2024-09-11T08:29:46.706 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-09-11T08:29:46.706 INFO:teuthology.orchestra.run.smithi145.stdout:======================================================================================= 2024-09-11T08:29:46.706 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:29:46.706 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T08:29:46.706 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-09-11T08:29:46.706 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T08:29:46.706 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T08:29:46.706 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T08:29:46.706 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T08:29:46.707 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T08:29:46.707 INFO:teuthology.orchestra.run.smithi145.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T08:29:46.707 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T08:29:46.707 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T08:29:46.707 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T08:29:46.707 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T08:29:46.707 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T08:29:46.707 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:46.707 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:29:46.707 INFO:teuthology.orchestra.run.smithi145.stdout:======================================================================================= 2024-09-11T08:29:46.707 INFO:teuthology.orchestra.run.smithi145.stdout:Install 12 Packages 2024-09-11T08:29:46.708 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:46.708 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 38 M 2024-09-11T08:29:46.708 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 212 M 2024-09-11T08:29:46.708 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:29:46.885 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:29:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:29:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:47.129 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:29:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:29:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:47.129 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:29:47.335 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T08:29:47.502 INFO:teuthology.orchestra.run.smithi145.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-16 17 MB/s | 7.4 MB 00:00 2024-09-11T08:29:47.911 INFO:teuthology.orchestra.run.smithi145.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 226 kB/s | 191 kB 00:00 2024-09-11T08:29:47.913 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:52 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:29:48.044 INFO:teuthology.orchestra.run.smithi145.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 817 kB/s | 801 kB 00:00 2024-09-11T08:29:48.048 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:29:48.048 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:29:48.048 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-09-11T08:29:48.048 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:29:48.048 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:29:48.048 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T08:29:48.048 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:48.049 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:29:48.049 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:29:48.049 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-11T08:29:48.049 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:48.049 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 137 k 2024-09-11T08:29:48.050 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 379 k 2024-09-11T08:29:48.050 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:29:48.069 INFO:teuthology.orchestra.run.smithi145.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 58 kB/s | 33 kB 00:00 2024-09-11T08:29:48.262 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-immutable-object-cache-17.2.7-1623.gd0e682 648 kB/s | 137 kB 00:00 2024-09-11T08:29:48.262 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:29:48.263 INFO:teuthology.orchestra.run.smithi092.stdout:Total 643 kB/s | 137 kB 00:00 2024-09-11T08:29:48.263 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:29:48.270 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:29:48.270 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:29:48.327 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:29:48.328 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:29:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:29:48.519 INFO:teuthology.orchestra.run.smithi145.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 38 kB/s | 18 kB 00:00 2024-09-11T08:29:48.588 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:29:48.612 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:29:48.613 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:29:48.613 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T08:29:48.613 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:48.645 INFO:teuthology.orchestra.run.smithi145.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 67 kB/s | 38 kB 00:00 2024-09-11T08:29:48.978 INFO:teuthology.orchestra.run.smithi145.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.8 MB/s | 3.0 MB 00:01 2024-09-11T08:29:49.445 INFO:teuthology.orchestra.run.smithi145.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 306 kB/s | 245 kB 00:00 2024-09-11T08:29:49.522 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:29:49.522 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:49.522 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:29:49.522 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:29:49.522 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:49.523 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:29:49.713 INFO:teuthology.orchestra.run.smithi145.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.1 MB/s | 4.9 MB 00:01 2024-09-11T08:29:49.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr 2024-09-11T08:29:50.113 INFO:teuthology.orchestra.run.smithi145.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-09-11T08:29:50.297 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:55 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:29:50.321 INFO:teuthology.orchestra.run.smithi145.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 286 kB/s | 250 kB 00:00 2024-09-11T08:29:50.390 INFO:teuthology.orchestra.run.smithi092.stdout:Package ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:29:50.431 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:29:50.432 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-11T08:29:50.432 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:29:50.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T08:29:51.089 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:56 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:29:51.154 INFO:teuthology.orchestra.run.smithi145.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-11T08:29:51.156 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:29:51.156 INFO:teuthology.orchestra.run.smithi145.stdout:Total 8.6 MB/s | 38 MB 00:04 2024-09-11T08:29:51.221 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:29:51.222 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:29:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-11T08:29:51.222 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:29:51.222 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:29:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T08:29:51.223 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-11T08:29:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T08:29:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T08:29:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T08:29:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T08:29:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T08:29:51.223 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-09-11T08:29:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T08:29:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:51.223 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:29:51.223 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:29:51.223 INFO:teuthology.orchestra.run.smithi092.stdout:Install 7 Packages 2024-09-11T08:29:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:51.224 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 3.7 M 2024-09-11T08:29:51.224 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 80 M 2024-09-11T08:29:51.224 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:29:51.516 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:29:51.523 INFO:teuthology.orchestra.run.smithi092.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1623.gd0e 176 kB/s | 24 kB 00:00 2024-09-11T08:29:51.542 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:29:51.542 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:29:51.548 INFO:teuthology.orchestra.run.smithi092.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1623.gd0e6 89 kB/s | 15 kB 00:00 2024-09-11T08:29:51.590 INFO:teuthology.orchestra.run.smithi092.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 136 kB/s | 9.0 kB 00:00 2024-09-11T08:29:51.615 INFO:teuthology.orchestra.run.smithi092.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-11T08:29:51.641 INFO:teuthology.orchestra.run.smithi092.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 444 kB/s | 41 kB 00:00 2024-09-11T08:29:51.666 INFO:teuthology.orchestra.run.smithi092.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-11T08:29:51.783 INFO:teuthology.orchestra.run.smithi092.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1623.gd0e6828a 8.4 MB/s | 3.4 MB 00:00 2024-09-11T08:29:51.786 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:29:51.786 INFO:teuthology.orchestra.run.smithi092.stdout:Total 6.5 MB/s | 3.7 MB 00:00 2024-09-11T08:29:51.787 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:29:51.795 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:29:51.795 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:29:51.882 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:29:51.882 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:29:52.063 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:29:52.063 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:29:52.219 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:29:52.292 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-11T08:29:52.327 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-11T08:29:52.393 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-11T08:29:52.430 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-11T08:29:52.459 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 5/7 2024-09-11T08:29:52.823 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:29:52.896 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T08:29:52.936 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:29:53.035 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:29:53.070 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T08:29:53.256 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T08:29:53.285 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T08:29:53.362 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T08:29:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 6/7 2024-09-11T08:29:53.496 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T08:29:54.020 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T08:29:54.132 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T08:29:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T08:29:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 1/7 2024-09-11T08:29:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 2/7 2024-09-11T08:29:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 3/7 2024-09-11T08:29:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-11T08:29:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-11T08:29:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-11T08:29:54.842 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-11T08:29:54.842 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:54.842 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:29:54.842 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:29:54.842 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:29:54.842 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:29:54.842 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T08:29:54.842 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T08:29:54.842 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T08:29:54.843 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T08:29:54.843 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:54.843 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:29:55.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T08:29:55.384 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:29:55.648 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:00 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:29:55.778 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:29:55.779 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-09-11T08:29:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-11T08:29:55.780 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-09-11T08:29:55.780 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:29:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T08:29:55.780 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-11T08:29:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T08:29:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T08:29:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T08:29:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T08:29:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T08:29:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T08:29:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T08:29:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T08:29:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T08:29:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T08:29:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T08:29:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:55.781 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:29:55.781 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-09-11T08:29:55.781 INFO:teuthology.orchestra.run.smithi092.stdout:Install 12 Packages 2024-09-11T08:29:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:29:55.782 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 38 M 2024-09-11T08:29:55.782 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 212 M 2024-09-11T08:29:55.782 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:29:56.415 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T08:29:56.443 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T08:29:56.607 INFO:teuthology.orchestra.run.smithi092.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-16 17 MB/s | 7.4 MB 00:00 2024-09-11T08:29:56.724 INFO:teuthology.orchestra.run.smithi092.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 282 kB/s | 33 kB 00:00 2024-09-11T08:29:56.966 INFO:teuthology.orchestra.run.smithi092.stdout:(3/12): libquadmath-11.5.0-2.el9.x86_64.rpm 237 kB/s | 191 kB 00:00 2024-09-11T08:29:57.074 INFO:teuthology.orchestra.run.smithi092.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 169 kB/s | 18 kB 00:00 2024-09-11T08:29:57.116 INFO:teuthology.orchestra.run.smithi092.stdout:(5/12): libgfortran-11.5.0-2.el9.x86_64.rpm 838 kB/s | 801 kB 00:00 2024-09-11T08:29:57.158 INFO:teuthology.orchestra.run.smithi092.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 461 kB/s | 38 kB 00:00 2024-09-11T08:29:57.267 INFO:teuthology.orchestra.run.smithi092.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-11T08:29:57.742 INFO:teuthology.orchestra.run.smithi092.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.9 MB/s | 3.0 MB 00:01 2024-09-11T08:29:57.892 INFO:teuthology.orchestra.run.smithi092.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.2 MB/s | 5.1 MB 00:00 2024-09-11T08:29:57.926 INFO:teuthology.orchestra.run.smithi092.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.3 MB/s | 250 kB 00:00 2024-09-11T08:29:58.226 INFO:teuthology.orchestra.run.smithi092.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 49 MB/s | 16 MB 00:00 2024-09-11T08:29:58.426 INFO:teuthology.orchestra.run.smithi092.stdout:(12/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 3.8 MB/s | 4.9 MB 00:01 2024-09-11T08:29:58.427 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:29:58.427 INFO:teuthology.orchestra.run.smithi092.stdout:Total 14 MB/s | 38 MB 00:02 2024-09-11T08:29:58.780 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:29:58.807 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:29:58.807 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:29:59.096 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T08:29:59.097 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 1/12 2024-09-11T08:29:59.097 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:29:59.097 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:29:59.097 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T08:29:59.097 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T08:29:59.097 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T08:29:59.097 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T08:29:59.097 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T08:29:59.097 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T08:29:59.097 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:29:59.097 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T08:29:59.325 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:29:59.325 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:29:59.493 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T08:29:59.493 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:59.493 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:29:59.493 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:29:59.493 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T08:29:59.493 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T08:29:59.493 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T08:29:59.493 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T08:29:59.493 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T08:29:59.493 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T08:29:59.494 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T08:29:59.494 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T08:29:59.494 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T08:29:59.494 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T08:29:59.494 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T08:29:59.494 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:29:59.494 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:29:59.938 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-mgr-rook 2024-09-11T08:30:00.118 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:30:00.200 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T08:30:00.241 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:30:00.341 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:30:00.375 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T08:30:00.520 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:13 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T08:30:00.599 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T08:30:00.661 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:00.662 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-09-11T08:30:00.662 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:00.662 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:30:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T08:30:00.662 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-09-11T08:30:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T08:30:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T08:30:00.662 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T08:30:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T08:30:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T08:30:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T08:30:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T08:30:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T08:30:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T08:30:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T08:30:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T08:30:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T08:30:00.663 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:00.663 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:30:00.663 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:00.663 INFO:teuthology.orchestra.run.smithi145.stdout:Install 13 Packages 2024-09-11T08:30:00.664 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:00.664 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 2.2 M 2024-09-11T08:30:00.664 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 26 M 2024-09-11T08:30:00.664 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:30:00.683 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T08:30:01.234 INFO:teuthology.orchestra.run.smithi145.stdout:(1/13): ceph-mgr-rook-17.2.7-1623.gd0e6828a.el9 284 kB/s | 47 kB 00:00 2024-09-11T08:30:01.335 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T08:30:01.471 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T08:30:01.601 INFO:teuthology.orchestra.run.smithi145.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 36 kB/s | 19 kB 00:00 2024-09-11T08:30:01.684 INFO:teuthology.orchestra.run.smithi145.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 41 kB/s | 26 kB 00:00 2024-09-11T08:30:01.993 INFO:teuthology.orchestra.run.smithi145.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 293 kB/s | 222 kB 00:00 2024-09-11T08:30:02.335 INFO:teuthology.orchestra.run.smithi145.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 216 kB/s | 159 kB 00:00 2024-09-11T08:30:02.377 INFO:teuthology.orchestra.run.smithi145.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 781 kB/s | 32 kB 00:00 2024-09-11T08:30:02.402 INFO:teuthology.orchestra.run.smithi145.stdout:(7/13): python3-certifi-2023.05.07-4.el9.noarch 564 kB/s | 14 kB 00:00 2024-09-11T08:30:02.427 INFO:teuthology.orchestra.run.smithi145.stdout:(8/13): python3-google-auth-2.34.0-1.el9.noarch 8.6 MB/s | 220 kB 00:00 2024-09-11T08:30:02.469 INFO:teuthology.orchestra.run.smithi145.stdout:(9/13): python3-kubernetes-26.1.0-2.el9.noarch. 25 MB/s | 1.0 MB 00:00 2024-09-11T08:30:02.494 INFO:teuthology.orchestra.run.smithi145.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T08:30:02.520 INFO:teuthology.orchestra.run.smithi145.stdout:(11/13): python3-pyasn1-modules-0.4.8-6.el9.noa 334 kB/s | 279 kB 00:00 2024-09-11T08:30:02.545 INFO:teuthology.orchestra.run.smithi145.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-11T08:30:02.645 INFO:teuthology.orchestra.run.smithi145.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 82 kB/s | 54 kB 00:00 2024-09-11T08:30:02.647 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:02.648 INFO:teuthology.orchestra.run.smithi145.stdout:Total 1.1 MB/s | 2.2 MB 00:01 2024-09-11T08:30:02.717 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:30:02.730 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:30:02.733 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:30:02.733 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:30:02.822 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:30:02.824 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:30:03.348 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:30:03.517 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T08:30:03.579 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T08:30:03.635 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T08:30:03.672 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T08:30:03.716 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T08:30:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T08:30:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T08:30:03.824 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T08:30:03.925 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T08:30:03.973 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T08:30:04.682 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T08:30:04.738 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T08:30:04.771 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T08:30:04.817 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T08:30:04.841 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T08:30:05.606 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T08:30:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 1/13 2024-09-11T08:30:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T08:30:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T08:30:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T08:30:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T08:30:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T08:30:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T08:30:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T08:30:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T08:30:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T08:30:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T08:30:05.607 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T08:30:05.941 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T08:30:05.941 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:05.941 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:30:05.942 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:30:05.942 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T08:30:05.942 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T08:30:05.942 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T08:30:05.942 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T08:30:05.942 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T08:30:05.942 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T08:30:05.942 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T08:30:05.942 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T08:30:05.942 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T08:30:05.942 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T08:30:05.942 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T08:30:05.942 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T08:30:05.943 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:05.943 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:06.193 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T08:30:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T08:30:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 1/12 2024-09-11T08:30:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:30:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:30:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T08:30:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T08:30:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T08:30:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T08:30:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T08:30:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T08:30:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:30:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T08:30:06.765 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:19 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:06.767 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T08:30:06.767 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:06.767 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:30:06.767 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:30:06.767 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T08:30:06.767 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T08:30:06.767 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T08:30:06.767 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T08:30:06.767 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T08:30:06.767 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T08:30:06.767 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T08:30:06.768 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T08:30:06.768 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T08:30:06.768 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T08:30:06.768 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T08:30:06.768 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:06.768 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:06.898 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:06.899 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:06.899 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-09-11T08:30:06.899 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:06.899 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:30:06.899 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T08:30:06.899 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-09-11T08:30:06.900 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T08:30:06.900 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T08:30:06.900 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T08:30:06.900 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T08:30:06.900 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T08:30:06.900 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:06.900 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:30:06.900 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:06.900 INFO:teuthology.orchestra.run.smithi145.stdout:Install 6 Packages 2024-09-11T08:30:06.900 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:06.901 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 7.0 M 2024-09-11T08:30:06.901 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 33 M 2024-09-11T08:30:06.901 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:30:07.149 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-rook 2024-09-11T08:30:07.723 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:12 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:07.859 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:07.860 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-11T08:30:07.860 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:07.860 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:30:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T08:30:07.860 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-11T08:30:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T08:30:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T08:30:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T08:30:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T08:30:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T08:30:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T08:30:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T08:30:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T08:30:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T08:30:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T08:30:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T08:30:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T08:30:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:07.861 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:30:07.861 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:07.862 INFO:teuthology.orchestra.run.smithi092.stdout:Install 13 Packages 2024-09-11T08:30:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:07.862 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 2.2 M 2024-09-11T08:30:07.862 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 26 M 2024-09-11T08:30:07.862 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:30:07.964 INFO:teuthology.orchestra.run.smithi145.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1623.gd0e6828a.e 305 kB/s | 124 kB 00:00 2024-09-11T08:30:08.047 INFO:teuthology.orchestra.run.smithi145.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-11T08:30:08.064 INFO:teuthology.orchestra.run.smithi145.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 3.4 MB/s | 58 kB 00:00 2024-09-11T08:30:08.089 INFO:teuthology.orchestra.run.smithi145.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-11T08:30:08.190 INFO:teuthology.orchestra.run.smithi145.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 393 kB/s | 248 kB 00:00 2024-09-11T08:30:08.314 INFO:teuthology.orchestra.run.smithi092.stdout:(1/13): ceph-mgr-rook-17.2.7-1623.gd0e6828a.el9 284 kB/s | 47 kB 00:00 2024-09-11T08:30:08.664 INFO:teuthology.orchestra.run.smithi092.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 37 kB/s | 19 kB 00:00 2024-09-11T08:30:08.706 INFO:teuthology.orchestra.run.smithi092.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 46 kB/s | 26 kB 00:00 2024-09-11T08:30:09.048 INFO:teuthology.orchestra.run.smithi092.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 303 kB/s | 222 kB 00:00 2024-09-11T08:30:09.211 INFO:teuthology.orchestra.run.smithi145.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.6 MB/s | 6.0 MB 00:01 2024-09-11T08:30:09.213 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:09.213 INFO:teuthology.orchestra.run.smithi145.stdout:Total 3.0 MB/s | 7.0 MB 00:02 2024-09-11T08:30:09.287 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:30:09.298 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:30:09.298 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:30:09.398 INFO:teuthology.orchestra.run.smithi092.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 216 kB/s | 159 kB 00:00 2024-09-11T08:30:09.398 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:30:09.399 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:30:09.440 INFO:teuthology.orchestra.run.smithi092.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 779 kB/s | 32 kB 00:00 2024-09-11T08:30:09.465 INFO:teuthology.orchestra.run.smithi092.stdout:(7/13): python3-certifi-2023.05.07-4.el9.noarch 567 kB/s | 14 kB 00:00 2024-09-11T08:30:09.490 INFO:teuthology.orchestra.run.smithi092.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 356 kB/s | 279 kB 00:00 2024-09-11T08:30:09.524 INFO:teuthology.orchestra.run.smithi092.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 3.7 MB/s | 220 kB 00:00 2024-09-11T08:30:09.549 INFO:teuthology.orchestra.run.smithi092.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.5 MB/s | 59 kB 00:00 2024-09-11T08:30:09.574 INFO:teuthology.orchestra.run.smithi092.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.5 MB/s | 90 kB 00:00 2024-09-11T08:30:09.608 INFO:teuthology.orchestra.run.smithi092.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 8.7 MB/s | 1.0 MB 00:00 2024-09-11T08:30:09.641 INFO:teuthology.orchestra.run.smithi092.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 90 kB/s | 54 kB 00:00 2024-09-11T08:30:09.645 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:09.645 INFO:teuthology.orchestra.run.smithi092.stdout:Total 1.2 MB/s | 2.2 MB 00:01 2024-09-11T08:30:09.700 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:30:09.713 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:30:09.729 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:30:09.729 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:30:09.817 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:30:09.818 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:30:09.872 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T08:30:09.933 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T08:30:10.318 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:30:10.458 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T08:30:10.496 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T08:30:10.542 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T08:30:10.550 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T08:30:10.607 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T08:30:10.613 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T08:30:10.636 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T08:30:10.644 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T08:30:10.684 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T08:30:10.804 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T08:30:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T08:30:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T08:30:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T08:30:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/6 2024-09-11T08:30:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T08:30:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T08:30:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T08:30:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T08:30:11.681 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T08:30:11.723 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T08:30:11.723 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:11.723 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:30:11.723 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:30:11.723 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T08:30:11.723 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T08:30:11.723 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T08:30:11.723 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T08:30:11.723 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T08:30:11.723 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:11.724 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T08:30:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T08:30:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T08:30:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T08:30:11.969 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-fuse 2024-09-11T08:30:12.533 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:25 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:12.578 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T08:30:12.578 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 1/13 2024-09-11T08:30:12.578 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T08:30:12.578 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T08:30:12.578 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T08:30:12.578 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T08:30:12.579 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T08:30:12.579 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T08:30:12.579 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T08:30:12.579 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T08:30:12.579 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T08:30:12.579 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T08:30:12.579 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T08:30:12.666 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:12.666 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:12.666 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-09-11T08:30:12.666 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:12.667 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:30:12.667 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T08:30:12.667 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-09-11T08:30:12.667 INFO:teuthology.orchestra.run.smithi145.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T08:30:12.667 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:12.667 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:30:12.667 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:12.667 INFO:teuthology.orchestra.run.smithi145.stdout:Install 2 Packages 2024-09-11T08:30:12.667 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:12.668 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 893 k 2024-09-11T08:30:12.668 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 2.6 M 2024-09-11T08:30:12.668 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:30:12.915 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T08:30:12.916 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:12.916 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:30:12.916 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:30:12.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T08:30:12.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T08:30:12.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T08:30:12.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T08:30:12.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T08:30:12.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T08:30:12.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T08:30:12.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T08:30:12.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T08:30:12.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T08:30:12.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T08:30:12.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T08:30:12.917 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:12.917 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:12.974 INFO:teuthology.orchestra.run.smithi145.stdout:(1/2): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x86_ 3.0 MB/s | 813 kB 00:00 2024-09-11T08:30:13.083 INFO:teuthology.orchestra.run.smithi145.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 216 kB/s | 80 kB 00:00 2024-09-11T08:30:13.083 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:13.084 INFO:teuthology.orchestra.run.smithi145.stdout:Total 2.1 MB/s | 893 kB 00:00 2024-09-11T08:30:13.096 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:30:13.119 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:30:13.119 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:30:13.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T08:30:13.203 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:30:13.203 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:30:13.388 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:30:13.515 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T08:30:13.582 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:30:13.754 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:18 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:13.889 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:13.890 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:13.890 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-11T08:30:13.890 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:13.890 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:30:13.890 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T08:30:13.890 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-11T08:30:13.890 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T08:30:13.890 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T08:30:13.890 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T08:30:13.890 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T08:30:13.890 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T08:30:13.890 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:13.891 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:30:13.891 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:13.891 INFO:teuthology.orchestra.run.smithi092.stdout:Install 6 Packages 2024-09-11T08:30:13.891 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:13.891 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 7.0 M 2024-09-11T08:30:13.891 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 33 M 2024-09-11T08:30:13.892 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:30:14.267 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:30:14.267 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/2 2024-09-11T08:30:14.426 INFO:teuthology.orchestra.run.smithi092.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 2.0 MB/s | 248 kB 00:00 2024-09-11T08:30:14.476 INFO:teuthology.orchestra.run.smithi092.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-11T08:30:14.502 INFO:teuthology.orchestra.run.smithi092.stdout:(3/6): ceph-mgr-cephadm-17.2.7-1623.gd0e6828a.e 628 kB/s | 124 kB 00:00 2024-09-11T08:30:14.527 INFO:teuthology.orchestra.run.smithi092.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-11T08:30:14.532 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T08:30:14.532 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:14.532 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:30:14.532 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T08:30:14.532 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:14.532 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:14.577 INFO:teuthology.orchestra.run.smithi092.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 991 kB/s | 74 kB 00:00 2024-09-11T08:30:14.669 INFO:teuthology.orchestra.run.smithi092.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-11T08:30:14.672 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:14.672 INFO:teuthology.orchestra.run.smithi092.stdout:Total 8.9 MB/s | 7.0 MB 00:00 2024-09-11T08:30:14.742 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:30:14.754 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:30:14.754 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:30:14.829 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-volume 2024-09-11T08:30:14.853 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:30:14.853 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:30:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:30:15.336 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T08:30:15.396 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T08:30:15.412 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:28 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:15.546 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:15.547 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:15.547 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-09-11T08:30:15.547 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:15.547 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:30:15.547 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T08:30:15.547 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:15.547 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:30:15.548 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:15.548 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2024-09-11T08:30:15.548 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:15.548 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 270 k 2024-09-11T08:30:15.548 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 1.3 M 2024-09-11T08:30:15.548 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:30:15.771 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-volume-17.2.7-1623.gd0e6828a.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-11T08:30:15.771 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:15.771 INFO:teuthology.orchestra.run.smithi145.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-11T08:30:15.772 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:30:15.780 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:30:15.780 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:30:15.808 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:30:15.809 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:30:15.931 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T08:30:16.018 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:30:16.024 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T08:30:16.079 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T08:30:16.097 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:30:16.108 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T08:30:16.114 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:30:16.114 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:30:16.114 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T08:30:16.114 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:16.893 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T08:30:16.893 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/6 2024-09-11T08:30:16.893 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T08:30:16.893 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T08:30:16.893 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T08:30:16.893 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T08:30:17.139 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:30:17.140 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:17.140 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:30:17.140 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:30:17.140 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:17.140 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:17.162 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T08:30:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:17.163 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:30:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:30:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T08:30:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T08:30:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T08:30:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T08:30:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T08:30:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:17.163 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:17.383 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-fuse 2024-09-11T08:30:17.436 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install librados-devel 2024-09-11T08:30:17.948 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:22 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:18.019 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:31 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:18.079 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:18.080 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-11T08:30:18.081 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:18.081 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:30:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T08:30:18.081 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-11T08:30:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T08:30:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:18.081 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:30:18.081 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:18.081 INFO:teuthology.orchestra.run.smithi092.stdout:Install 2 Packages 2024-09-11T08:30:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:18.082 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 893 k 2024-09-11T08:30:18.082 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 2.6 M 2024-09-11T08:30:18.082 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:30:18.155 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:18.155 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:18.156 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repo Size 2024-09-11T08:30:18.156 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:18.156 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:30:18.156 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T08:30:18.156 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:18.156 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:30:18.156 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:18.156 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2024-09-11T08:30:18.156 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:18.156 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 126 k 2024-09-11T08:30:18.156 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 448 k 2024-09-11T08:30:18.156 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:30:18.375 INFO:teuthology.orchestra.run.smithi145.stdout:librados-devel-17.2.7-1623.gd0e6828a.el9.x86_64 578 kB/s | 126 kB 00:00 2024-09-11T08:30:18.376 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:18.376 INFO:teuthology.orchestra.run.smithi145.stdout:Total 575 kB/s | 126 kB 00:00 2024-09-11T08:30:18.376 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:30:18.381 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:30:18.381 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:30:18.444 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:30:18.445 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:30:18.463 INFO:teuthology.orchestra.run.smithi092.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 607 kB/s | 80 kB 00:00 2024-09-11T08:30:18.591 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:30:18.621 INFO:teuthology.orchestra.run.smithi092.stdout:(2/2): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x86_ 2.7 MB/s | 813 kB 00:00 2024-09-11T08:30:18.622 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:18.622 INFO:teuthology.orchestra.run.smithi092.stdout:Total 1.6 MB/s | 893 kB 00:00 2024-09-11T08:30:18.634 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:30:18.656 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:30:18.656 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:30:18.689 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:18.739 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:30:18.739 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:30:18.932 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:30:19.062 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T08:30:19.140 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:30:19.383 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:19.383 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:19.383 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:30:19.383 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:19.383 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:19.383 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:19.657 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install libcephfs2 2024-09-11T08:30:19.802 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:30:19.802 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/2 2024-09-11T08:30:20.079 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T08:30:20.079 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:20.079 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:30:20.079 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T08:30:20.079 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:20.079 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:20.239 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:33 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:20.333 INFO:teuthology.orchestra.run.smithi145.stdout:Package libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:20.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-volume 2024-09-11T08:30:20.381 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:20.381 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2024-09-11T08:30:20.381 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:20.501 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install libcephfs-devel 2024-09-11T08:30:20.936 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:25 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:21.062 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:34 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:21.069 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:21.070 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-11T08:30:21.071 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:21.071 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:30:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T08:30:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:21.071 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:30:21.071 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:21.071 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-11T08:30:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:21.071 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 270 k 2024-09-11T08:30:21.071 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 1.3 M 2024-09-11T08:30:21.071 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:30:21.194 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:21.195 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:21.195 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repo Size 2024-09-11T08:30:21.195 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:21.195 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:30:21.195 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T08:30:21.195 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:21.195 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:30:21.195 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:21.195 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2024-09-11T08:30:21.195 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:21.195 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 25 k 2024-09-11T08:30:21.196 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 110 k 2024-09-11T08:30:21.196 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:30:21.308 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-volume-17.2.7-1623.gd0e6828a.el9.noarch.rp 1.1 MB/s | 270 kB 00:00 2024-09-11T08:30:21.309 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:21.309 INFO:teuthology.orchestra.run.smithi092.stdout:Total 1.1 MB/s | 270 kB 00:00 2024-09-11T08:30:21.309 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:30:21.317 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:30:21.317 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:30:21.344 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:30:21.345 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:30:21.345 INFO:teuthology.orchestra.run.smithi145.stdout:libcephfs-devel-17.2.7-1623.gd0e6828a.el9.x86_6 167 kB/s | 25 kB 00:00 2024-09-11T08:30:21.346 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:21.346 INFO:teuthology.orchestra.run.smithi145.stdout:Total 165 kB/s | 25 kB 00:00 2024-09-11T08:30:21.346 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:30:21.349 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:30:21.350 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:30:21.351 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:30:21.351 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:30:21.437 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:30:21.550 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:21.556 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:30:21.619 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:30:21.635 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:30:21.635 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:30:21.635 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T08:30:21.635 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:21.973 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:22.194 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:22.195 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:22.195 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:30:22.195 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:22.195 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:22.195 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:22.383 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install librados2 2024-09-11T08:30:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:30:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:22.704 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:30:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:30:22.705 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:22.705 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:22.944 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:35 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:23.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librados-devel 2024-09-11T08:30:23.035 INFO:teuthology.orchestra.run.smithi145.stdout:Package librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:23.078 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:23.079 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2024-09-11T08:30:23.079 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:23.143 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install librbd1 2024-09-11T08:30:23.597 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:28 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:23.703 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:23.734 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:23.735 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-09-11T08:30:23.735 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:23.735 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:30:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T08:30:23.736 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:23.736 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:30:23.736 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:23.736 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-11T08:30:23.736 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:23.736 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 126 k 2024-09-11T08:30:23.736 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 448 k 2024-09-11T08:30:23.736 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:30:23.793 INFO:teuthology.orchestra.run.smithi145.stdout:Package librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:23.836 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:23.837 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2024-09-11T08:30:23.837 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:23.897 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install python3-rados 2024-09-11T08:30:23.938 INFO:teuthology.orchestra.run.smithi092.stdout:librados-devel-17.2.7-1623.gd0e6828a.el9.x86_64 625 kB/s | 126 kB 00:00 2024-09-11T08:30:23.939 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:23.939 INFO:teuthology.orchestra.run.smithi092.stdout:Total 620 kB/s | 126 kB 00:00 2024-09-11T08:30:23.939 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:30:23.945 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:30:23.945 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:30:24.010 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:30:24.010 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:30:24.162 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:30:24.319 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:24.461 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:37 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:24.550 INFO:teuthology.orchestra.run.smithi145.stdout:Package python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:24.592 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:24.593 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2024-09-11T08:30:24.593 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:24.665 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install python3-rgw 2024-09-11T08:30:24.759 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:25.012 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:30:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:25.012 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:25.224 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:25.314 INFO:teuthology.orchestra.run.smithi145.stdout:Package python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:25.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install libcephfs2 2024-09-11T08:30:25.356 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:25.357 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2024-09-11T08:30:25.357 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:25.418 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install python3-cephfs 2024-09-11T08:30:25.899 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:30 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:25.983 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:25.990 INFO:teuthology.orchestra.run.smithi092.stdout:Package libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:26.035 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:26.036 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-11T08:30:26.036 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:26.074 INFO:teuthology.orchestra.run.smithi145.stdout:Package python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:26.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install libcephfs-devel 2024-09-11T08:30:26.116 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:26.117 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2024-09-11T08:30:26.117 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:26.184 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install python3-rbd 2024-09-11T08:30:26.671 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:31 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:26.747 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:39 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:26.803 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:26.804 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-09-11T08:30:26.804 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:26.804 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:30:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T08:30:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:26.805 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:30:26.805 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:26.805 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-11T08:30:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:26.805 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 25 k 2024-09-11T08:30:26.805 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 110 k 2024-09-11T08:30:26.805 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:30:26.838 INFO:teuthology.orchestra.run.smithi145.stdout:Package python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:26.881 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:26.882 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2024-09-11T08:30:26.882 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:26.947 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install rbd-fuse 2024-09-11T08:30:26.967 INFO:teuthology.orchestra.run.smithi092.stdout:libcephfs-devel-17.2.7-1623.gd0e6828a.el9.x86_6 154 kB/s | 25 kB 00:00 2024-09-11T08:30:26.967 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:26.967 INFO:teuthology.orchestra.run.smithi092.stdout:Total 153 kB/s | 25 kB 00:00 2024-09-11T08:30:26.968 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:30:26.971 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:30:26.971 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:30:26.972 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:30:26.973 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:30:27.047 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:30:27.163 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:27.512 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:40 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:27.541 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:27.645 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:27.646 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:27.646 INFO:teuthology.orchestra.run.smithi145.stdout: Package Architecture Version Repository Size 2024-09-11T08:30:27.646 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:27.647 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:30:27.647 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T08:30:27.647 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:27.647 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:30:27.647 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:27.647 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2024-09-11T08:30:27.647 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:27.647 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 82 k 2024-09-11T08:30:27.647 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 222 k 2024-09-11T08:30:27.647 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:30:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:27.775 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:30:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:27.775 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:27.865 INFO:teuthology.orchestra.run.smithi145.stdout:rbd-fuse-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 377 kB/s | 82 kB 00:00 2024-09-11T08:30:27.865 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:27.865 INFO:teuthology.orchestra.run.smithi145.stdout:Total 374 kB/s | 82 kB 00:00 2024-09-11T08:30:27.866 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:30:27.872 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:30:27.872 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:30:27.935 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:30:27.935 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:30:27.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librados2 2024-09-11T08:30:28.074 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:30:28.169 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:28.511 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:33 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:28.599 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:28.642 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:28.643 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-11T08:30:28.643 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:28.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librbd1 2024-09-11T08:30:28.805 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:28.805 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:28.805 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:30:28.805 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:28.806 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:28.806 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:29.026 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install rbd-mirror 2024-09-11T08:30:29.267 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:34 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:29.356 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:29.399 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:29.400 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-11T08:30:29.400 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:29.473 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rados 2024-09-11T08:30:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:42 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:29.742 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repo Size 2024-09-11T08:30:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:30:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T08:30:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:30:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2024-09-11T08:30:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 2.9 M 2024-09-11T08:30:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 12 M 2024-09-11T08:30:29.744 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:30:30.031 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:35 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:30.076 INFO:teuthology.orchestra.run.smithi145.stdout:rbd-mirror-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 8.8 MB/s | 2.9 MB 00:00 2024-09-11T08:30:30.076 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:30.077 INFO:teuthology.orchestra.run.smithi145.stdout:Total 8.8 MB/s | 2.9 MB 00:00 2024-09-11T08:30:30.077 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:30:30.086 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:30:30.086 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:30:30.119 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:30.162 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:30.163 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-11T08:30:30.163 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:30.165 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:30:30.165 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:30:30.232 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rgw 2024-09-11T08:30:30.630 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:30:30.694 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:30.721 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:30.721 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:30:30.721 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T08:30:30.721 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:30:30.721 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:30:30.721 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:30.794 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:35 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:30.882 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:30.925 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:30.926 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-11T08:30:30.926 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:30.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-cephfs 2024-09-11T08:30:31.558 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:31.649 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:31.692 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:31.693 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-11T08:30:31.693 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:31.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rbd 2024-09-11T08:30:31.837 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:31.837 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:31.837 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:30:31.837 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:31.837 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:31.837 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:32.070 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install rbd-nbd 2024-09-11T08:30:32.322 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:37 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:32.411 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:32.454 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:32.455 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-11T08:30:32.455 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:32.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-fuse 2024-09-11T08:30:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:45 ago on Wed 11 Sep 2024 08:28:47 AM UTC. 2024-09-11T08:30:32.789 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-09-11T08:30:32.790 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:32.790 INFO:teuthology.orchestra.run.smithi145.stdout: Package Architecture Version Repository Size 2024-09-11T08:30:32.790 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:32.790 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-09-11T08:30:32.790 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T08:30:32.790 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:32.790 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-09-11T08:30:32.791 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-09-11T08:30:32.791 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2024-09-11T08:30:32.791 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:32.791 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 165 k 2024-09-11T08:30:32.791 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 477 k 2024-09-11T08:30:32.791 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-09-11T08:30:32.997 INFO:teuthology.orchestra.run.smithi145.stdout:rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 804 kB/s | 165 kB 00:00 2024-09-11T08:30:32.997 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:32.997 INFO:teuthology.orchestra.run.smithi145.stdout:Total 798 kB/s | 165 kB 00:00 2024-09-11T08:30:32.998 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-09-11T08:30:33.005 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-09-11T08:30:33.005 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-09-11T08:30:33.069 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-09-11T08:30:33.069 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-09-11T08:30:33.088 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:33.220 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-09-11T08:30:33.221 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:33.221 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:33.221 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-09-11T08:30:33.221 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:33.221 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:30:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T08:30:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:33.222 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:30:33.222 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:33.222 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-11T08:30:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:33.222 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 82 k 2024-09-11T08:30:33.223 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 222 k 2024-09-11T08:30:33.223 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:30:33.319 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:33.411 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-fuse-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 434 kB/s | 82 kB 00:00 2024-09-11T08:30:33.412 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:33.412 INFO:teuthology.orchestra.run.smithi092.stdout:Total 430 kB/s | 82 kB 00:00 2024-09-11T08:30:33.413 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:30:33.419 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:30:33.419 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:30:33.483 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:30:33.483 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:30:33.635 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:30:33.730 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:34.003 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:34.003 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:34.003 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-09-11T08:30:34.003 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:34.003 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:30:34.003 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-09-11T08:30:34.103 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:34.235 DEBUG:teuthology.parallel:result is None 2024-09-11T08:30:34.350 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:34.350 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:34.350 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:30:34.350 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:34.350 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:34.350 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:34.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-mirror 2024-09-11T08:30:35.154 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:40 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:35.289 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:35.290 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:35.290 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-09-11T08:30:35.290 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:35.290 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:30:35.290 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T08:30:35.290 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:35.290 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:30:35.290 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:35.290 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-11T08:30:35.290 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:35.290 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 2.9 M 2024-09-11T08:30:35.291 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 12 M 2024-09-11T08:30:35.291 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:30:35.631 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-mirror-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 8.6 MB/s | 2.9 MB 00:00 2024-09-11T08:30:35.631 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:35.632 INFO:teuthology.orchestra.run.smithi092.stdout:Total 8.6 MB/s | 2.9 MB 00:00 2024-09-11T08:30:35.632 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:30:35.640 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:30:35.641 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:30:35.721 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:30:35.721 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:30:36.129 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:30:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:36.215 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:36.215 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:30:36.215 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T08:30:36.215 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:30:36.215 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:30:36.215 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:37.399 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:37.399 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:37.399 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:30:37.399 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:37.399 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:37.399 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:37.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-nbd 2024-09-11T08:30:38.188 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:43 ago on Wed 11 Sep 2024 08:28:55 AM UTC. 2024-09-11T08:30:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-11T08:30:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:38.323 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-09-11T08:30:38.323 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:38.324 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-11T08:30:38.324 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T08:30:38.324 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:38.324 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-11T08:30:38.324 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-11T08:30:38.324 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-11T08:30:38.324 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:38.324 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 165 k 2024-09-11T08:30:38.324 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 477 k 2024-09-11T08:30:38.324 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-11T08:30:38.526 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 819 kB/s | 165 kB 00:00 2024-09-11T08:30:38.527 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:38.527 INFO:teuthology.orchestra.run.smithi092.stdout:Total 813 kB/s | 165 kB 00:00 2024-09-11T08:30:38.527 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-11T08:30:38.535 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-11T08:30:38.535 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-11T08:30:38.599 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-11T08:30:38.599 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-11T08:30:38.754 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-11T08:30:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:39.282 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:39.557 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:30:39.558 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:39.558 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-11T08:30:39.558 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:39.558 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:30:39.558 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-11T08:30:39.788 DEBUG:teuthology.parallel:result is None 2024-09-11T08:30:39.788 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:30:39.936 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T08:30:39.956 INFO:teuthology.orchestra.run.smithi092.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T08:30:39.957 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T08:30:39.958 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T08:30:39.960 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:30:40.101 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T08:30:40.127 INFO:teuthology.orchestra.run.smithi145.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T08:30:40.127 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T08:30:40.127 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T08:30:40.129 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T08:30:40.129 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:30:40.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T08:30:40.155 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-11T08:30:40.155 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T08:30:40.197 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T08:30:40.197 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:30:40.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T08:30:40.222 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T08:30:40.286 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-11T08:30:40.286 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T08:30:40.315 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T08:30:40.384 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T08:30:40.384 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:30:40.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T08:30:40.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T08:30:40.475 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-11T08:30:40.475 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T08:30:40.503 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T08:30:40.571 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T08:30:40.571 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:30:40.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T08:30:40.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T08:30:40.661 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-11T08:30:40.661 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T08:30:40.687 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T08:30:40.754 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T08:30:40.839 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'cephadm_mode': 'root'} 2024-09-11T08:30:40.839 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:30:40.840 INFO:tasks.cephadm:Cluster fsid is 211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:30:40.840 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T08:30:40.840 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.92', 'mon.c': '[v2:172.21.15.92:3301,v1:172.21.15.92:6790]', 'mon.b': '172.21.15.145'} 2024-09-11T08:30:40.840 INFO:tasks.cephadm:First mon is mon.a on smithi092 2024-09-11T08:30:40.840 INFO:tasks.cephadm:First mgr is y 2024-09-11T08:30:40.840 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T08:30:40.841 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname $(hostname -s) 2024-09-11T08:30:40.866 DEBUG:teuthology.orchestra.run.smithi145:> sudo hostname $(hostname -s) 2024-09-11T08:30:40.894 INFO:tasks.cephadm:Downloading cephadm (repo https://git.ceph.com/ceph-ci.git ref d0e6828a2016d48cf25ad84064e50742bb1c39b9)... 2024-09-11T08:30:40.895 DEBUG:teuthology.orchestra.run.smithi092:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show d0e6828a2016d48cf25ad84064e50742bb1c39b9:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T08:30:40.921 INFO:teuthology.orchestra.run.smithi092.stderr:Cloning into 'testrepo'... 2024-09-11T08:31:41.681 INFO:teuthology.orchestra.run.smithi092.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 11 08:31 /home/ubuntu/cephtest/cephadm 2024-09-11T08:31:41.682 DEBUG:teuthology.orchestra.run.smithi145:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show d0e6828a2016d48cf25ad84064e50742bb1c39b9:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T08:31:41.702 INFO:teuthology.orchestra.run.smithi145.stderr:Cloning into 'testrepo'... 2024-09-11T08:32:38.928 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 11 08:32 /home/ubuntu/cephtest/cephadm 2024-09-11T08:32:38.929 DEBUG:teuthology.orchestra.run.smithi092:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T08:32:38.948 DEBUG:teuthology.orchestra.run.smithi145:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T08:32:38.998 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 on all hosts... 2024-09-11T08:32:38.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T08:32:39.000 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T08:32:40.637 INFO:teuthology.orchestra.run.smithi092.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:32:42.314 INFO:teuthology.orchestra.run.smithi145.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:33:06.265 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:33:06.266 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:33:06.266 INFO:teuthology.orchestra.run.smithi092.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T08:33:06.266 INFO:teuthology.orchestra.run.smithi092.stdout: "repo_digests": [ 2024-09-11T08:33:06.266 INFO:teuthology.orchestra.run.smithi092.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T08:33:06.266 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:33:06.266 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:33:11.262 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:33:11.313 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:33:11.313 INFO:teuthology.orchestra.run.smithi145.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T08:33:11.314 INFO:teuthology.orchestra.run.smithi145.stdout: "repo_digests": [ 2024-09-11T08:33:11.314 INFO:teuthology.orchestra.run.smithi145.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T08:33:11.314 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:33:11.314 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:33:11.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph 2024-09-11T08:33:11.348 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /etc/ceph 2024-09-11T08:33:11.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /etc/ceph 2024-09-11T08:33:11.417 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 777 /etc/ceph 2024-09-11T08:33:11.441 INFO:tasks.cephadm:Writing seed config... 2024-09-11T08:33:11.442 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T08:33:11.442 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T08:33:11.442 INFO:tasks.cephadm: override: [client.0] debug ms = 1 2024-09-11T08:33:11.442 INFO:tasks.cephadm: override: [client.0] debug objecter = 20 2024-09-11T08:33:11.443 INFO:tasks.cephadm: override: [client.0] debug rados = 20 2024-09-11T08:33:11.443 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-11T08:33:11.443 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T08:33:11.443 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T08:33:11.443 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T08:33:11.443 INFO:tasks.cephadm: override: [mon] mon min osdmap epochs = 50 2024-09-11T08:33:11.443 INFO:tasks.cephadm: override: [mon] mon osdmap full prune interval = 2 2024-09-11T08:33:11.443 INFO:tasks.cephadm: override: [mon] mon osdmap full prune min = 15 2024-09-11T08:33:11.443 INFO:tasks.cephadm: override: [mon] mon osdmap full prune txsize = 2 2024-09-11T08:33:11.443 INFO:tasks.cephadm: override: [mon] paxos service trim min = 10 2024-09-11T08:33:11.444 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T08:33:11.444 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T08:33:11.444 INFO:tasks.cephadm: override: [osd] osd debug reject backfill probability = 0.3 2024-09-11T08:33:11.444 INFO:tasks.cephadm: override: [osd] osd delete sleep = 1 2024-09-11T08:33:11.444 INFO:tasks.cephadm: override: [osd] osd max backfills = 3 2024-09-11T08:33:11.444 INFO:tasks.cephadm: override: [osd] osd scrub max interval = 120 2024-09-11T08:33:11.444 INFO:tasks.cephadm: override: [osd] osd scrub min interval = 60 2024-09-11T08:33:11.445 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T08:33:11.445 INFO:tasks.cephadm: override: [osd] osd snap trim sleep = 2 2024-09-11T08:33:11.445 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:33:11.445 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T08:33:11.474 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 211625d4-7018-11ef-bcea-c7b262605968 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 120 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd debug reject backfill probability = 0.3 osd delete sleep = 1 osd max backfills = 3 osd scrub min interval = 60 osd snap trim sleep = 2 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon min osdmap epochs = 50 mon osdmap full prune interval = 2 mon osdmap full prune min = 15 mon osdmap full prune txsize = 2 paxos service trim min = 10 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client.0] debug ms = 1 debug objecter = 20 debug rados = 20 2024-09-11T08:33:11.475 DEBUG:teuthology.orchestra.run.smithi092:mon.a> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T08:33:11.517 DEBUG:teuthology.orchestra.run.smithi092:mgr.y> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T08:33:11.561 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T08:33:11.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 -v bootstrap --fsid 211625d4-7018-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.92 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:33:11.766 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:33:11.767 INFO:teuthology.orchestra.run.smithi092.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9', '-v', 'bootstrap', '--fsid', '211625d4-7018-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.92', '--skip-admin-label'] 2024-09-11T08:33:11.791 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 5.2.2 2024-09-11T08:33:11.791 INFO:teuthology.orchestra.run.smithi092.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T08:33:11.792 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying podman|docker is present... 2024-09-11T08:33:11.817 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 5.2.2 2024-09-11T08:33:11.818 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying lvm2 is present... 2024-09-11T08:33:11.818 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying time synchronization is in place... 2024-09-11T08:33:11.823 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T08:33:11.823 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T08:33:11.830 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T08:33:11.830 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-11T08:33:11.837 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout enabled 2024-09-11T08:33:11.844 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout active 2024-09-11T08:33:11.845 INFO:teuthology.orchestra.run.smithi092.stdout:Unit chronyd.service is enabled and running 2024-09-11T08:33:11.845 INFO:teuthology.orchestra.run.smithi092.stdout:Repeating the final host check... 2024-09-11T08:33:11.868 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 5.2.2 2024-09-11T08:33:11.868 INFO:teuthology.orchestra.run.smithi092.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T08:33:11.868 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl is present 2024-09-11T08:33:11.868 INFO:teuthology.orchestra.run.smithi092.stdout:lvcreate is present 2024-09-11T08:33:11.875 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T08:33:11.875 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T08:33:11.882 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T08:33:11.882 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-11T08:33:11.890 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout enabled 2024-09-11T08:33:11.898 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout active 2024-09-11T08:33:11.898 INFO:teuthology.orchestra.run.smithi092.stdout:Unit chronyd.service is enabled and running 2024-09-11T08:33:11.898 INFO:teuthology.orchestra.run.smithi092.stdout:Host looks OK 2024-09-11T08:33:11.898 INFO:teuthology.orchestra.run.smithi092.stdout:Cluster fsid: 211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:33:11.899 INFO:teuthology.orchestra.run.smithi092.stdout:Acquiring lock 139904175707040 on /run/cephadm/211625d4-7018-11ef-bcea-c7b262605968.lock 2024-09-11T08:33:11.899 INFO:teuthology.orchestra.run.smithi092.stdout:Lock 139904175707040 acquired on /run/cephadm/211625d4-7018-11ef-bcea-c7b262605968.lock 2024-09-11T08:33:11.899 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying IP 172.21.15.92 port 3300 ... 2024-09-11T08:33:11.899 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying IP 172.21.15.92 port 6789 ... 2024-09-11T08:33:11.900 INFO:teuthology.orchestra.run.smithi092.stdout:Base mon IP(s) is [172.21.15.92:3300, 172.21.15.92:6789], mon addrv is [v2:172.21.15.92:3300,v1:172.21.15.92:6789] 2024-09-11T08:33:11.902 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.92 metric 100 2024-09-11T08:33:11.903 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.92 metric 100 2024-09-11T08:33:11.905 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T08:33:11.906 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-11T08:33:11.906 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-11T08:33:11.909 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T08:33:11.909 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T08:33:11.909 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T08:33:11.909 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-11T08:33:11.909 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link noprefixroute 2024-09-11T08:33:11.909 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T08:33:11.909 INFO:teuthology.orchestra.run.smithi092.stdout:Mon IP `172.21.15.92` is in CIDR network `172.21.0.0/20` 2024-09-11T08:33:11.909 INFO:teuthology.orchestra.run.smithi092.stdout:Mon IP `172.21.15.92` is in CIDR network `172.21.0.0/20` 2024-09-11T08:33:11.909 INFO:teuthology.orchestra.run.smithi092.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T08:33:11.910 INFO:teuthology.orchestra.run.smithi092.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T08:33:11.910 INFO:teuthology.orchestra.run.smithi092.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:33:12.439 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T08:33:12.440 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:33:12.440 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T08:33:12.440 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Copying blob sha256:0d2f264d63cec2f6fb0f85eed7b2c8c5b4db3ab9edd9d71a74c04647294a553f 2024-09-11T08:33:12.440 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T08:33:12.440 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Copying config sha256:4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T08:33:12.440 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T08:33:13.427 INFO:teuthology.orchestra.run.smithi092.stdout:ceph: stdout ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T08:33:13.428 INFO:teuthology.orchestra.run.smithi092.stdout:Ceph version: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T08:33:13.428 INFO:teuthology.orchestra.run.smithi092.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T08:33:14.492 INFO:teuthology.orchestra.run.smithi092.stdout:stat: stdout 167 167 2024-09-11T08:33:14.492 INFO:teuthology.orchestra.run.smithi092.stdout:Creating initial keys... 2024-09-11T08:33:15.522 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-authtool: stdout AQDKVeFmCGAbMhAAK0BGcFbDf9YpNxq9rARjHg== 2024-09-11T08:33:19.817 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-authtool: stdout AQDLVeFmnEHvNBAAq6akQ509BcytR+Tc8P1t4w== 2024-09-11T08:33:28.838 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-authtool: stdout AQDSVeFmzXGNLhAAhuQop2TEUtp8x0E7Hvr61Q== 2024-09-11T08:33:28.838 INFO:teuthology.orchestra.run.smithi092.stdout:Creating initial monmap... 2024-09-11T08:33:29.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T08:33:29.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-11T08:33:29.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:33:29.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T08:33:29.859 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool for a [v2:172.21.15.92:3300,v1:172.21.15.92:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T08:33:29.859 INFO:teuthology.orchestra.run.smithi092.stdout:setting min_mon_release = octopus 2024-09-11T08:33:29.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: set fsid to 211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:33:29.859 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T08:33:29.859 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:33:29.859 INFO:teuthology.orchestra.run.smithi092.stdout:Creating mon... 2024-09-11T08:33:30.815 INFO:teuthology.orchestra.run.smithi092.stdout:create mon.a on 2024-09-11T08:33:31.036 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T08:33:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T08:33:31.327 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-211625d4-7018-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-211625d4-7018-11ef-bcea-c7b262605968.target. 2024-09-11T08:33:31.327 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-211625d4-7018-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-211625d4-7018-11ef-bcea-c7b262605968.target. 2024-09-11T08:33:31.493 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-211625d4-7018-11ef-bcea-c7b262605968@mon.a 2024-09-11T08:33:31.493 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to reset failed state of unit ceph-211625d4-7018-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-211625d4-7018-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T08:33:31.649 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-211625d4-7018-11ef-bcea-c7b262605968.target.wants/ceph-211625d4-7018-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-211625d4-7018-11ef-bcea-c7b262605968@.service. 2024-09-11T08:33:32.135 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:31 smithi092 podman[29145]: 2024-09-11 08:33:31.897512071 +0000 UTC m=+0.133048026 container create 3f5910df82b91403054230ea95c8dee7230edfb3206bd581a925cdb0829a46b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T08:33:32.136 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:32 smithi092 podman[29145]: 2024-09-11 08:33:32.135728173 +0000 UTC m=+0.371264128 container init 3f5910df82b91403054230ea95c8dee7230edfb3206bd581a925cdb0829a46b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T08:33:32.257 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:33:32.257 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-09-11T08:33:32.264 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:33:32.264 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-11T08:33:32.264 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-09-11T08:33:32.264 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T08:33:32.265 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mon to start... 2024-09-11T08:33:32.265 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mon... 2024-09-11T08:33:32.387 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:32 smithi092 podman[29145]: 2024-09-11 08:33:32.140595362 +0000 UTC m=+0.376131317 container start 3f5910df82b91403054230ea95c8dee7230edfb3206bd581a925cdb0829a46b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:33:32.387 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:32 smithi092 ceph-mon[29182]: mkfs 211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:33:32.387 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:32 smithi092 ceph-mon[29182]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T08:33:32.387 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:32 smithi092 bash[29145]: 3f5910df82b91403054230ea95c8dee7230edfb3206bd581a925cdb0829a46b0 2024-09-11T08:33:32.387 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:32 smithi092 systemd[1]: Started Ceph mon.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:33:33.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:33 smithi092 ceph-mon[29182]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T08:33:33.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:33 smithi092 ceph-mon[29182]: monmap e1: 1 mons at {a=[v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:33:33.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:33 smithi092 ceph-mon[29182]: fsmap 2024-09-11T08:33:33.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:33 smithi092 ceph-mon[29182]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T08:33:33.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:33 smithi092 ceph-mon[29182]: mgrmap e1: no daemons active 2024-09-11T08:33:33.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:33 smithi092 ceph-mon[29182]: from='client.? 172.21.15.92:0/212762164' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T08:33:33.370 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T08:33:33.370 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout id: 211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:33:33.370 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T08:33:33.370 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:33.370 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout services: 2024-09-11T08:33:33.370 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.631277s) 2024-09-11T08:33:33.370 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T08:33:33.370 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T08:33:33.370 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:33.370 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout data: 2024-09-11T08:33:33.370 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T08:33:33.370 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T08:33:33.371 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T08:33:33.371 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T08:33:33.371 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:33.371 INFO:teuthology.orchestra.run.smithi092.stdout:mon is available 2024-09-11T08:33:33.371 INFO:teuthology.orchestra.run.smithi092.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T08:33:34.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:34 smithi092 ceph-mon[29182]: from='client.? 172.21.15.92:0/157181667' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T08:33:34.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:34 smithi092 ceph-mon[29182]: from='client.? 172.21.15.92:0/157181667' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T08:33:35.575 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:35.575 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T08:33:35.576 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout fsid = 211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:33:35.576 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T08:33:35.576 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.92:3300,v1:172.21.15.92:6789] 2024-09-11T08:33:35.576 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T08:33:35.576 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T08:33:35.576 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T08:33:35.577 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T08:33:35.577 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:35.577 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T08:33:35.577 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T08:33:35.577 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:35.577 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T08:33:35.577 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T08:33:35.577 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T08:33:35.577 INFO:teuthology.orchestra.run.smithi092.stdout:Generating new minimal ceph.conf... 2024-09-11T08:33:36.425 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:36 smithi092 ceph-mon[29182]: from='client.? 172.21.15.92:0/1638417267' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:33:36.637 INFO:teuthology.orchestra.run.smithi092.stdout:Restarting the monitor... 2024-09-11T08:33:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:36 smithi092 systemd[1]: Stopping Ceph mon.a for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T08:33:37.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:36 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a[29159]: 2024-09-11T08:33:36.931+0000 7f5b3e979640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:33:37.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:36 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a[29159]: 2024-09-11T08:33:36.931+0000 7f5b3e979640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T08:33:37.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:37 smithi092 podman[29498]: 2024-09-11 08:33:37.085173191 +0000 UTC m=+0.291241730 container died 3f5910df82b91403054230ea95c8dee7230edfb3206bd581a925cdb0829a46b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T08:33:37.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:37 smithi092 podman[29498]: 2024-09-11 08:33:37.453553063 +0000 UTC m=+0.659621594 container cleanup 3f5910df82b91403054230ea95c8dee7230edfb3206bd581a925cdb0829a46b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T08:33:37.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:37 smithi092 bash[29498]: ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a 2024-09-11T08:33:38.005 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:37 smithi092 podman[29515]: 2024-09-11 08:33:37.671082612 +0000 UTC m=+0.586417945 container remove 3f5910df82b91403054230ea95c8dee7230edfb3206bd581a925cdb0829a46b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-11T08:33:38.005 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:37 smithi092 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T08:33:38.005 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:37 smithi092 systemd[1]: Stopped Ceph mon.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:33:38.005 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:37 smithi092 systemd[1]: Starting Ceph mon.a for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T08:33:38.289 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 podman[29610]: 2024-09-11 08:33:38.005230354 +0000 UTC m=+0.108596274 container create b8d1f6a92bc5fb3bbb38871f1a17f3f9e58c0d061442f5473a6cf4984684f20b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T08:33:38.289 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 podman[29610]: 2024-09-11 08:33:37.912014815 +0000 UTC m=+0.015380703 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:33:38.289 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 podman[29610]: 2024-09-11 08:33:38.198999443 +0000 UTC m=+0.302365324 container init b8d1f6a92bc5fb3bbb38871f1a17f3f9e58c0d061442f5473a6cf4984684f20b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:33:38.290 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 podman[29610]: 2024-09-11 08:33:38.203386514 +0000 UTC m=+0.306752393 container start b8d1f6a92bc5fb3bbb38871f1a17f3f9e58c0d061442f5473a6cf4984684f20b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T08:33:38.290 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T08:33:38.290 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 7 2024-09-11T08:33:38.290 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: pidfile_write: ignore empty --pid-file 2024-09-11T08:33:38.290 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: load: jerasure load: lrc 2024-09-11T08:33:38.290 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: RocksDB version: 6.15.5 2024-09-11T08:33:38.290 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T08:33:38.290 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Compile date Sep 10 2024 2024-09-11T08:33:38.290 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: DB SUMMARY 2024-09-11T08:33:38.290 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: DB Session ID: I8V7KGTXMG0C4IM2FVXN 2024-09-11T08:33:38.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: CURRENT file: CURRENT 2024-09-11T08:33:38.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T08:33:38.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-11T08:33:38.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T08:33:38.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 89038 ; 2024-09-11T08:33:38.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.error_if_exists: 0 2024-09-11T08:33:38.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.create_if_missing: 0 2024-09-11T08:33:38.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.paranoid_checks: 1 2024-09-11T08:33:38.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T08:33:38.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.env: 0x56111baff440 2024-09-11T08:33:38.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.fs: Posix File System 2024-09-11T08:33:38.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.info_log: 0x56111d0c0ec0 2024-09-11T08:33:38.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T08:33:38.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.statistics: (nil) 2024-09-11T08:33:38.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.use_fsync: 0 2024-09-11T08:33:38.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_log_file_size: 0 2024-09-11T08:33:38.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T08:33:38.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T08:33:38.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T08:33:38.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T08:33:38.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.allow_fallocate: 1 2024-09-11T08:33:38.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T08:33:38.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T08:33:38.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.use_direct_reads: 0 2024-09-11T08:33:38.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T08:33:38.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T08:33:38.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.db_log_dir: 2024-09-11T08:33:38.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-11T08:33:38.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T08:33:38.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T08:33:38.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T08:33:38.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T08:33:38.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T08:33:38.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T08:33:38.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T08:33:38.293 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T08:33:38.294 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.write_buffer_manager: 0x56111d17e7e0 2024-09-11T08:33:38.294 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T08:33:38.294 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T08:33:38.294 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T08:33:38.294 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T08:33:38.294 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T08:33:38.294 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T08:33:38.294 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T08:33:38.294 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T08:33:38.294 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T08:33:38.294 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.unordered_write: 0 2024-09-11T08:33:38.294 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T08:33:38.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T08:33:38.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T08:33:38.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T08:33:38.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.row_cache: None 2024-09-11T08:33:38.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.wal_filter: None 2024-09-11T08:33:38.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T08:33:38.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T08:33:38.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.preserve_deletes: 0 2024-09-11T08:33:38.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.two_write_queues: 0 2024-09-11T08:33:38.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T08:33:38.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.atomic_flush: 0 2024-09-11T08:33:38.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T08:33:38.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T08:33:38.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T08:33:38.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.log_readahead_size: 0 2024-09-11T08:33:38.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T08:33:38.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T08:33:38.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T08:33:38.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T08:33:38.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T08:33:38.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T08:33:38.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_background_jobs: 2 2024-09-11T08:33:38.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_background_compactions: -1 2024-09-11T08:33:38.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_subcompactions: 1 2024-09-11T08:33:38.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T08:33:38.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T08:33:38.297 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T08:33:38.297 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T08:33:38.297 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T08:33:38.297 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T08:33:38.297 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T08:33:38.297 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T08:33:38.297 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_open_files: -1 2024-09-11T08:33:38.297 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T08:33:38.297 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T08:33:38.298 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T08:33:38.298 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T08:33:38.299 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_background_flushes: -1 2024-09-11T08:33:38.299 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Compression algorithms supported: 2024-09-11T08:33:38.299 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: kZSTD supported: 0 2024-09-11T08:33:38.299 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: kXpressCompression supported: 0 2024-09-11T08:33:38.299 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: kBZip2Compression supported: 0 2024-09-11T08:33:38.299 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T08:33:38.299 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: kLZ4Compression supported: 1 2024-09-11T08:33:38.299 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: kZlibCompression supported: 1 2024-09-11T08:33:38.299 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T08:33:38.299 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: kSnappyCompression supported: 1 2024-09-11T08:33:38.299 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T08:33:38.299 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-11T08:33:38.299 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T08:33:38.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T08:33:38.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.merge_operator: 2024-09-11T08:33:38.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compaction_filter: None 2024-09-11T08:33:38.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T08:33:38.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T08:33:38.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T08:33:38.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T08:33:38.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56111d067a38) 2024-09-11T08:33:38.300 INFO:journalctl@ceph.mon.a.smithi092.stdout: cache_index_and_filter_blocks: 1 2024-09-11T08:33:38.300 INFO:journalctl@ceph.mon.a.smithi092.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T08:33:38.300 INFO:journalctl@ceph.mon.a.smithi092.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T08:33:38.300 INFO:journalctl@ceph.mon.a.smithi092.stdout: pin_top_level_index_and_filter: 1 2024-09-11T08:33:38.301 INFO:journalctl@ceph.mon.a.smithi092.stdout: index_type: 0 2024-09-11T08:33:38.301 INFO:journalctl@ceph.mon.a.smithi092.stdout: data_block_index_type: 0 2024-09-11T08:33:38.301 INFO:journalctl@ceph.mon.a.smithi092.stdout: index_shortening: 1 2024-09-11T08:33:38.301 INFO:journalctl@ceph.mon.a.smithi092.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T08:33:38.301 INFO:journalctl@ceph.mon.a.smithi092.stdout: hash_index_allow_collision: 1 2024-09-11T08:33:38.301 INFO:journalctl@ceph.mon.a.smithi092.stdout: checksum: 1 2024-09-11T08:33:38.301 INFO:journalctl@ceph.mon.a.smithi092.stdout: no_block_cache: 0 2024-09-11T08:33:38.301 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache: 0x56111d09d1f0 2024-09-11T08:33:38.301 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache_name: BinnedLRUCache 2024-09-11T08:33:38.301 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache_options: 2024-09-11T08:33:38.301 INFO:journalctl@ceph.mon.a.smithi092.stdout: capacity : 536870912 2024-09-11T08:33:38.301 INFO:journalctl@ceph.mon.a.smithi092.stdout: num_shard_bits : 4 2024-09-11T08:33:38.302 INFO:journalctl@ceph.mon.a.smithi092.stdout: strict_capacity_limit : 0 2024-09-11T08:33:38.302 INFO:journalctl@ceph.mon.a.smithi092.stdout: high_pri_pool_ratio: 0.000 2024-09-11T08:33:38.302 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache_compressed: (nil) 2024-09-11T08:33:38.302 INFO:journalctl@ceph.mon.a.smithi092.stdout: persistent_cache: (nil) 2024-09-11T08:33:38.302 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_size: 4096 2024-09-11T08:33:38.302 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_size_deviation: 10 2024-09-11T08:33:38.302 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_restart_interval: 16 2024-09-11T08:33:38.302 INFO:journalctl@ceph.mon.a.smithi092.stdout: index_block_restart_interval: 1 2024-09-11T08:33:38.302 INFO:journalctl@ceph.mon.a.smithi092.stdout: metadata_block_size: 4096 2024-09-11T08:33:38.302 INFO:journalctl@ceph.mon.a.smithi092.stdout: partition_filters: 0 2024-09-11T08:33:38.302 INFO:journalctl@ceph.mon.a.smithi092.stdout: use_delta_encoding: 1 2024-09-11T08:33:38.302 INFO:journalctl@ceph.mon.a.smithi092.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T08:33:38.302 INFO:journalctl@ceph.mon.a.smithi092.stdout: whole_key_filtering: 1 2024-09-11T08:33:38.303 INFO:journalctl@ceph.mon.a.smithi092.stdout: verify_compression: 0 2024-09-11T08:33:38.303 INFO:journalctl@ceph.mon.a.smithi092.stdout: read_amp_bytes_per_bit: 0 2024-09-11T08:33:38.303 INFO:journalctl@ceph.mon.a.smithi092.stdout: format_version: 4 2024-09-11T08:33:38.303 INFO:journalctl@ceph.mon.a.smithi092.stdout: enable_index_compression: 1 2024-09-11T08:33:38.303 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_align: 0 2024-09-11T08:33:38.303 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T08:33:38.303 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T08:33:38.303 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compression: NoCompression 2024-09-11T08:33:38.303 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T08:33:38.303 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T08:33:38.303 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T08:33:38.303 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.num_levels: 7 2024-09-11T08:33:38.303 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T08:33:38.303 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T08:33:38.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T08:33:38.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T08:33:38.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T08:33:38.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T08:33:38.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T08:33:38.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:33:38.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T08:33:38.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T08:33:38.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T08:33:38.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T08:33:38.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T08:33:38.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T08:33:38.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:33:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T08:33:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T08:33:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T08:33:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T08:33:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T08:33:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T08:33:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T08:33:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T08:33:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T08:33:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T08:33:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T08:33:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T08:33:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T08:33:38.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T08:33:38.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T08:33:38.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T08:33:38.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T08:33:38.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T08:33:38.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T08:33:38.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T08:33:38.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T08:33:38.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T08:33:38.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T08:33:38.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T08:33:38.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T08:33:38.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T08:33:38.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T08:33:38.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T08:33:38.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T08:33:38.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T08:33:38.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T08:33:38.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T08:33:38.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T08:33:38.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T08:33:38.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T08:33:38.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.inplace_update_support: 0 2024-09-11T08:33:38.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T08:33:38.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T08:33:38.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T08:33:38.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T08:33:38.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.bloom_locality: 0 2024-09-11T08:33:38.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.max_successive_merges: 0 2024-09-11T08:33:38.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T08:33:38.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T08:33:38.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T08:33:38.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T08:33:38.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.ttl: 2592000 2024-09-11T08:33:38.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T08:33:38.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.enable_blob_files: false 2024-09-11T08:33:38.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.min_blob_size: 0 2024-09-11T08:33:38.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T08:33:38.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T08:33:38.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T08:33:38.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T08:33:38.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T08:33:38.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-11T08:33:38.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-11T08:33:38.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043618226883, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-11T08:33:38.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-11T08:33:38.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T08:33:38.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043618227734, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 85183, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 83274, "index_size": 243, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 11312, "raw_average_key_size": 48, "raw_value_size": 76984, "raw_average_value_size": 330, "num_data_blocks": 10, "num_entries": 233, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726043618, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "60560217-2838-41a5-b9aa-f583eeb2fa04", "db_session_id": "I8V7KGTXMG0C4IM2FVXN"}} 2024-09-11T08:33:38.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-11T08:33:38.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043618228161, "job": 1, "event": "recovery_finished"} 2024-09-11T08:33:38.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T08:33:38.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x56111d0d0700 2024-09-11T08:33:38.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: DB pointer 0x56111d0a0000 2024-09-11T08:33:38.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T08:33:38.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T08:33:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** DB Stats ** 2024-09-11T08:33:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:33:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T08:33:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T08:33:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:33:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T08:33:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T08:33:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:33:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-11T08:33:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** Compaction Stats [default] ** 2024-09-11T08:33:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:33:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:33:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout: L0 2/0 84.97 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 164.8 0.00 0.00 1 0.000 0 0 2024-09-11T08:33:38.311 INFO:journalctl@ceph.mon.a.smithi092.stdout: Sum 2/0 84.97 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 164.8 0.00 0.00 1 0.000 0 0 2024-09-11T08:33:38.311 INFO:journalctl@ceph.mon.a.smithi092.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 164.8 0.00 0.00 1 0.000 0 0 2024-09-11T08:33:38.311 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-11T08:33:38.311 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** Compaction Stats [default] ** 2024-09-11T08:33:38.311 INFO:journalctl@ceph.mon.a.smithi092.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:33:38.311 INFO:journalctl@ceph.mon.a.smithi092.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:33:38.311 INFO:journalctl@ceph.mon.a.smithi092.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 164.8 0.00 0.00 1 0.000 0 0 2024-09-11T08:33:38.311 INFO:journalctl@ceph.mon.a.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:33:38.311 INFO:journalctl@ceph.mon.a.smithi092.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:33:38.311 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:33:38.311 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:33:38.311 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:33:38.311 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:33:38.311 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative compaction: 0.00 GB write, 25.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:33:38.312 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval compaction: 0.00 GB write, 25.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:33:38.312 INFO:journalctl@ceph.mon.a.smithi092.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:33:38.312 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-11T08:33:38.312 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:33:38.312 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-11T08:33:38.312 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** Compaction Stats [default] ** 2024-09-11T08:33:38.312 INFO:journalctl@ceph.mon.a.smithi092.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:33:38.312 INFO:journalctl@ceph.mon.a.smithi092.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:33:38.312 INFO:journalctl@ceph.mon.a.smithi092.stdout: L0 2/0 84.97 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 164.8 0.00 0.00 1 0.000 0 0 2024-09-11T08:33:38.312 INFO:journalctl@ceph.mon.a.smithi092.stdout: Sum 2/0 84.97 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 164.8 0.00 0.00 1 0.000 0 0 2024-09-11T08:33:38.312 INFO:journalctl@ceph.mon.a.smithi092.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T08:33:38.312 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-11T08:33:38.312 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** Compaction Stats [default] ** 2024-09-11T08:33:38.313 INFO:journalctl@ceph.mon.a.smithi092.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:33:38.313 INFO:journalctl@ceph.mon.a.smithi092.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:33:38.313 INFO:journalctl@ceph.mon.a.smithi092.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 164.8 0.00 0.00 1 0.000 0 0 2024-09-11T08:33:38.313 INFO:journalctl@ceph.mon.a.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:33:38.313 INFO:journalctl@ceph.mon.a.smithi092.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:33:38.313 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:33:38.313 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:33:38.313 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:33:38.313 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:33:38.313 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative compaction: 0.00 GB write, 24.98 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:33:38.313 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:33:38.313 INFO:journalctl@ceph.mon.a.smithi092.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:33:38.313 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-11T08:33:38.314 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:33:38.314 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: starting mon.a rank 0 at public addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] at bind addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:33:38.314 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: mon.a@-1(???) e1 preinit fsid 211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:33:38.314 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: mon.a@-1(???).mds e1 new map 2024-09-11T08:33:38.314 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: mon.a@-1(???).mds e1 print_map 2024-09-11T08:33:38.314 INFO:journalctl@ceph.mon.a.smithi092.stdout: e1 2024-09-11T08:33:38.314 INFO:journalctl@ceph.mon.a.smithi092.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T08:33:38.314 INFO:journalctl@ceph.mon.a.smithi092.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T08:33:38.314 INFO:journalctl@ceph.mon.a.smithi092.stdout: legacy client fscid: -1 2024-09-11T08:33:38.314 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-11T08:33:38.314 INFO:journalctl@ceph.mon.a.smithi092.stdout: No filesystems configured 2024-09-11T08:33:38.314 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T08:33:38.314 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:33:38.314 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:33:38.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:33:38.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T08:33:38.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T08:33:38.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: monmap e1: 1 mons at {a=[v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:33:38.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: fsmap 2024-09-11T08:33:38.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T08:33:38.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 ceph-mon[29647]: mgrmap e1: no daemons active 2024-09-11T08:33:38.324 INFO:teuthology.orchestra.run.smithi092.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T08:33:38.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 bash[29610]: b8d1f6a92bc5fb3bbb38871f1a17f3f9e58c0d061442f5473a6cf4984684f20b 2024-09-11T08:33:38.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:38 smithi092 systemd[1]: Started Ceph mon.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:33:39.382 INFO:teuthology.orchestra.run.smithi092.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T08:33:39.383 INFO:teuthology.orchestra.run.smithi092.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:33:39.383 INFO:teuthology.orchestra.run.smithi092.stdout:Creating mgr... 2024-09-11T08:33:39.383 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T08:33:39.383 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T08:33:39.561 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-211625d4-7018-11ef-bcea-c7b262605968@mgr.y 2024-09-11T08:33:39.561 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to reset failed state of unit ceph-211625d4-7018-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-211625d4-7018-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-11T08:33:39.715 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-211625d4-7018-11ef-bcea-c7b262605968.target.wants/ceph-211625d4-7018-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-211625d4-7018-11ef-bcea-c7b262605968@.service. 2024-09-11T08:33:40.053 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:39 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1749326064' entity='client.admin' 2024-09-11T08:33:40.054 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:39 smithi092 podman[29883]: 2024-09-11 08:33:39.843264674 +0000 UTC m=+0.015699502 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:33:40.262 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:33:40.263 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-09-11T08:33:40.270 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:33:40.270 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-11T08:33:40.270 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-09-11T08:33:40.270 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T08:33:40.279 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:33:40.279 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-09-11T08:33:40.287 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:33:40.287 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-11T08:33:40.287 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-09-11T08:33:40.287 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T08:33:40.287 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr to start... 2024-09-11T08:33:40.288 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr... 2024-09-11T08:33:40.411 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:40 smithi092 podman[29883]: 2024-09-11 08:33:40.158110457 +0000 UTC m=+0.330545286 container init 417d2bf1fce87da9b4bf42d263b3aa91be1477b0ee0be6e0a26800c1a124978c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:33:40.411 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:40 smithi092 podman[29883]: 2024-09-11 08:33:40.162836368 +0000 UTC m=+0.335271197 container start 417d2bf1fce87da9b4bf42d263b3aa91be1477b0ee0be6e0a26800c1a124978c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T08:33:40.411 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:40 smithi092 bash[29883]: 417d2bf1fce87da9b4bf42d263b3aa91be1477b0ee0be6e0a26800c1a124978c 2024-09-11T08:33:40.411 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:40 smithi092 systemd[1]: Started Ceph mgr.y for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:33:40.411 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:40 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:40.277+0000 7fcd06934ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:33:40.411 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:40 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:40.340+0000 7fcd06934ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:33:40.411 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:40 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:40.404+0000 7fcd06934ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:33:40.671 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:40 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:40.529+0000 7fcd06934ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:33:40.945 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:40 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/406267280' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T08:33:40.945 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:40 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:40.670+0000 7fcd06934ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:33:40.945 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:40 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:40.944+0000 7fcd06934ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:33:41.205 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:41.005+0000 7fcd06934ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:33:41.206 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:41.169+0000 7fcd06934ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:33:41.481 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:41.482 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:33:41.482 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "211625d4-7018-11ef-bcea-c7b262605968", 2024-09-11T08:33:41.482 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T08:33:41.482 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T08:33:41.482 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T08:33:41.482 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T08:33:41.482 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:41.482 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T08:33:41.482 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T08:33:41.482 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-09-11T08:33:41.483 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:41.483 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T08:33:41.483 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T08:33:41.483 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:41.483 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-11T08:33:41.483 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T08:33:41.483 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:41.483 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T08:33:41.483 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T08:33:41.483 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:41.484 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T08:33:41.484 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:41.484 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T08:33:41.484 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T08:33:41.484 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T08:33:41.484 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T08:33:41.484 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T08:33:41.484 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T08:33:41.484 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:41.484 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T08:33:41.484 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T08:33:41.485 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T08:33:41.485 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T08:33:41.485 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T08:33:41.485 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T08:33:41.485 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T08:33:41.485 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T08:33:41.485 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T08:33:41.485 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:41.485 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T08:33:41.485 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:41.486 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T08:33:41.486 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T08:33:41.486 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:41.486 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T08:33:41.486 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T08:33:41.486 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T08:33:41.486 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T08:33:41.486 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T08:33:41.486 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T08:33:41.486 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T08:33:41.486 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:41.487 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:33:41.487 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:41.487 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T08:33:41.487 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:41.487 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T08:33:32.174974+0000", 2024-09-11T08:33:41.487 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:33:41.487 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:41.487 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T08:33:41.487 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:33:41.487 INFO:teuthology.orchestra.run.smithi092.stdout:mgr not available, waiting (1/15)... 2024-09-11T08:33:42.276 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:42 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:42.012+0000 7fcd06934ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:33:42.313 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:42 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:42.080+0000 7fcd06934ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:33:43.110 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:42 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:42.819+0000 7fcd06934ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:33:43.110 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:42 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:42.930+0000 7fcd06934ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:33:43.110 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:43 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:43.050+0000 7fcd06934ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:33:43.110 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:43 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:43.109+0000 7fcd06934ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:33:43.362 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:43 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:43.236+0000 7fcd06934ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:33:43.362 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:43 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:43.304+0000 7fcd06934ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:33:43.362 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:43 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:43.361+0000 7fcd06934ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:33:43.632 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:43 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:43.497+0000 7fcd06934ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:33:43.632 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:43 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:43.566+0000 7fcd06934ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:33:43.632 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:43 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:43.632+0000 7fcd06934ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:33:43.951 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:43 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:43.715+0000 7fcd06934ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:33:43.951 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:43 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:43.778+0000 7fcd06934ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:33:43.951 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:43 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:43.835+0000 7fcd06934ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:33:44.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:44 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3220208015' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T08:33:44.653 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:44.653 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:33:44.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "211625d4-7018-11ef-bcea-c7b262605968", 2024-09-11T08:33:44.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T08:33:44.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T08:33:44.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T08:33:44.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T08:33:44.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:44.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T08:33:44.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T08:33:44.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-09-11T08:33:44.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:44.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T08:33:44.654 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T08:33:44.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:44.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-11T08:33:44.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T08:33:44.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:44.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T08:33:44.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T08:33:44.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:44.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T08:33:44.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:44.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T08:33:44.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T08:33:44.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T08:33:44.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T08:33:44.655 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T08:33:44.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T08:33:44.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:44.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T08:33:44.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T08:33:44.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T08:33:44.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T08:33:44.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T08:33:44.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T08:33:44.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T08:33:44.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T08:33:44.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T08:33:44.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:44.656 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T08:33:44.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:44.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T08:33:44.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T08:33:44.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:44.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T08:33:44.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T08:33:44.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T08:33:44.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T08:33:44.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T08:33:44.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T08:33:44.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T08:33:44.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:44.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:33:44.657 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:44.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T08:33:44.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:44.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T08:33:32.174974+0000", 2024-09-11T08:33:44.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:33:44.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:44.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T08:33:44.658 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:33:44.658 INFO:teuthology.orchestra.run.smithi092.stdout:mgr not available, waiting (2/15)... 2024-09-11T08:33:45.060 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:44 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:44.600+0000 7fcd06934ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:33:45.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:45 smithi092 ceph-mon[29647]: Activating manager daemon y 2024-09-11T08:33:45.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:45 smithi092 ceph-mon[29647]: mgrmap e2: y(active, starting, since 0.00307518s) 2024-09-11T08:33:45.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:45 smithi092 ceph-mon[29647]: from='mgr.14100 172.21.15.92:0/1973124140' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:33:45.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:45 smithi092 ceph-mon[29647]: from='mgr.14100 172.21.15.92:0/1973124140' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:33:45.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:45 smithi092 ceph-mon[29647]: from='mgr.14100 172.21.15.92:0/1973124140' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:33:45.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:45 smithi092 ceph-mon[29647]: from='mgr.14100 172.21.15.92:0/1973124140' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:33:45.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:45 smithi092 ceph-mon[29647]: from='mgr.14100 172.21.15.92:0/1973124140' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:33:45.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:45 smithi092 ceph-mon[29647]: Manager daemon y is now available 2024-09-11T08:33:45.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:45 smithi092 ceph-mon[29647]: from='mgr.14100 172.21.15.92:0/1973124140' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:33:45.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:45 smithi092 ceph-mon[29647]: from='mgr.14100 172.21.15.92:0/1973124140' entity='mgr.y' 2024-09-11T08:33:45.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:45 smithi092 ceph-mon[29647]: from='mgr.14100 172.21.15.92:0/1973124140' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:33:45.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:45 smithi092 ceph-mon[29647]: from='mgr.14100 172.21.15.92:0/1973124140' entity='mgr.y' 2024-09-11T08:33:45.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:45 smithi092 ceph-mon[29647]: from='mgr.14100 172.21.15.92:0/1973124140' entity='mgr.y' 2024-09-11T08:33:46.882 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:46 smithi092 ceph-mon[29647]: mgrmap e3: y(active, since 1.0059s) 2024-09-11T08:33:47.935 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:47 smithi092 ceph-mon[29647]: mgrmap e4: y(active, since 2s) 2024-09-11T08:33:47.935 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:47 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/247153894' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T08:33:47.943 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:47.943 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:33:47.943 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "211625d4-7018-11ef-bcea-c7b262605968", 2024-09-11T08:33:47.943 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T08:33:47.943 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T08:33:47.943 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T08:33:47.943 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T08:33:47.943 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:47.943 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T08:33:47.943 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T08:33:47.944 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-09-11T08:33:47.944 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:47.944 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T08:33:47.944 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T08:33:47.944 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:47.944 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-11T08:33:47.944 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T08:33:47.944 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:47.944 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T08:33:47.944 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T08:33:47.944 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:47.944 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T08:33:47.944 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:47.944 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T08:33:47.944 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T08:33:47.945 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T08:33:47.945 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T08:33:47.945 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T08:33:47.945 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T08:33:47.945 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:47.945 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T08:33:47.945 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T08:33:47.945 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T08:33:47.945 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T08:33:47.945 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T08:33:47.945 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T08:33:47.945 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T08:33:47.945 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T08:33:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T08:33:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T08:33:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T08:33:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T08:33:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T08:33:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T08:33:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T08:33:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T08:33:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T08:33:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T08:33:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T08:33:47.946 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:47.947 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:33:47.947 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:47.947 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T08:33:47.947 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:47.947 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T08:33:32.174974+0000", 2024-09-11T08:33:47.947 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:33:47.947 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:47.947 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T08:33:47.947 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:33:47.947 INFO:teuthology.orchestra.run.smithi092.stdout:mgr is available 2024-09-11T08:33:48.864 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:48 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/977076545' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T08:33:49.118 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:49.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T08:33:49.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout fsid = 211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:33:49.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T08:33:49.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T08:33:49.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T08:33:49.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T08:33:49.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T08:33:49.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:49.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T08:33:49.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T08:33:49.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:49.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T08:33:49.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T08:33:49.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T08:33:49.121 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling cephadm module... 2024-09-11T08:33:50.187 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:49 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3724081867' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T08:33:50.188 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:49 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ignoring --setuser ceph since I am not root 2024-09-11T08:33:50.188 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:49 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ignoring --setgroup ceph since I am not root 2024-09-11T08:33:50.188 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:49 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:49.945+0000 7f13e4737ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:33:50.188 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:50 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:49.998+0000 7f13e4737ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:33:50.188 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:50 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:50.061+0000 7f13e4737ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:33:50.188 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:50 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:50.186+0000 7f13e4737ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:33:50.521 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:50 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:50.322+0000 7f13e4737ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:33:50.780 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:50 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:50.579+0000 7f13e4737ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:33:50.780 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:50 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:50.638+0000 7f13e4737ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:33:51.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:50 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3724081867' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T08:33:51.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:50 smithi092 ceph-mon[29647]: mgrmap e5: y(active, since 5s) 2024-09-11T08:33:51.066 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:50 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:50.811+0000 7f13e4737ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:33:51.738 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:51.672+0000 7f13e4737ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:33:51.738 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:51.737+0000 7f13e4737ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:33:51.773 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:33:51.773 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T08:33:51.773 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T08:33:51.773 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T08:33:51.773 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T08:33:51.773 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:33:51.773 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for the mgr to restart... 2024-09-11T08:33:51.773 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr epoch 5... 2024-09-11T08:33:52.029 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:51 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3980517388' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T08:33:52.749 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:52.459+0000 7f13e4737ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:33:52.749 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:52.569+0000 7f13e4737ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:33:52.749 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:52.688+0000 7f13e4737ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:33:53.004 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:52.749+0000 7f13e4737ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:33:53.004 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:52.879+0000 7f13e4737ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:33:53.004 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:52.945+0000 7f13e4737ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:33:53.004 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:53 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:53.003+0000 7f13e4737ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:33:53.265 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:53 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:53.136+0000 7f13e4737ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:33:53.265 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:53 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:53.199+0000 7f13e4737ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:33:53.266 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:53 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:53.264+0000 7f13e4737ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:33:53.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:53 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:53.350+0000 7f13e4737ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:33:53.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:53 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:53.406+0000 7f13e4737ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:33:53.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:53 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:53.462+0000 7f13e4737ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:33:54.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:54.192+0000 7f13e4737ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:33:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: Active manager daemon y restarted 2024-09-11T08:33:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: Activating manager daemon y 2024-09-11T08:33:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T08:33:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: mgrmap e6: y(active, starting, since 0.00382569s) 2024-09-11T08:33:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:33:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:33:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:33:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:33:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:33:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: Manager daemon y is now available 2024-09-11T08:33:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' 2024-09-11T08:33:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' 2024-09-11T08:33:54.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:54.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:54.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:33:54.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:54 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:33:55.499 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:55 smithi092 ceph-mon[29647]: Found migration_current of "None". Setting to last migration. 2024-09-11T08:33:55.499 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:55 smithi092 ceph-mon[29647]: mgrmap e7: y(active, since 1.00488s) 2024-09-11T08:33:55.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:33:55.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T08:33:55.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T08:33:55.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:33:55.953 INFO:teuthology.orchestra.run.smithi092.stdout:mgr epoch 5 is available 2024-09-11T08:33:55.954 INFO:teuthology.orchestra.run.smithi092.stdout:Setting orchestrator backend to cephadm... 2024-09-11T08:33:56.202 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:56.039+0000 7f1396b25640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:33:56] ENGINE Error in HTTPServer.serve 2024-09-11T08:33:56.202 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: Traceback (most recent call last): 2024-09-11T08:33:56.203 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:33:56.203 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: self._connections.run(self.expiration_interval) 2024-09-11T08:33:56.203 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:33:56.203 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: self._run(expiration_interval) 2024-09-11T08:33:56.203 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:33:56.203 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:33:56.203 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:33:56.203 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:33:56.203 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:33:56.203 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: s = self.context.wrap_socket( 2024-09-11T08:33:56.203 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:33:56.204 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: return self.sslsocket_class._create( 2024-09-11T08:33:56.204 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:33:56.204 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: self.do_handshake() 2024-09-11T08:33:56.204 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:33:56.204 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: self._sslobj.do_handshake() 2024-09-11T08:33:56.204 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:33:56.204 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:33:56.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-mon[29647]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T08:33:56.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-mon[29647]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T08:33:56.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' 2024-09-11T08:33:56.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:56 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:57.660 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:57 smithi092 ceph-mon[29647]: [11/Sep/2024:08:33:55] ENGINE Bus STARTING 2024-09-11T08:33:57.660 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:57 smithi092 ceph-mon[29647]: [11/Sep/2024:08:33:56] ENGINE Serving on https://172.21.15.92:7150 2024-09-11T08:33:57.660 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:57 smithi092 ceph-mon[29647]: [11/Sep/2024:08:33:56] ENGINE Bus STARTED 2024-09-11T08:33:57.661 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:57 smithi092 ceph-mon[29647]: [11/Sep/2024:08:33:56] ENGINE Error in HTTPServer.serve 2024-09-11T08:33:57.661 INFO:journalctl@ceph.mon.a.smithi092.stdout: Traceback (most recent call last): 2024-09-11T08:33:57.661 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:33:57.661 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:33:57.661 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:33:57.661 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._run(expiration_interval) 2024-09-11T08:33:57.661 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:33:57.661 INFO:journalctl@ceph.mon.a.smithi092.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:33:57.661 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:33:57.661 INFO:journalctl@ceph.mon.a.smithi092.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:33:57.661 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:33:57.661 INFO:journalctl@ceph.mon.a.smithi092.stdout: s = self.context.wrap_socket( 2024-09-11T08:33:57.661 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:33:57.661 INFO:journalctl@ceph.mon.a.smithi092.stdout: return self.sslsocket_class._create( 2024-09-11T08:33:57.662 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:33:57.662 INFO:journalctl@ceph.mon.a.smithi092.stdout: self.do_handshake() 2024-09-11T08:33:57.662 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:33:57.662 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._sslobj.do_handshake() 2024-09-11T08:33:57.662 INFO:journalctl@ceph.mon.a.smithi092.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:33:57.662 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:57 smithi092 ceph-mon[29647]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:33:57.662 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:57 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' 2024-09-11T08:33:57.662 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:57 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:57.662 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:57 smithi092 ceph-mon[29647]: mgrmap e8: y(active, since 2s) 2024-09-11T08:33:58.364 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T08:33:58.365 INFO:teuthology.orchestra.run.smithi092.stdout:Generating ssh key... 2024-09-11T08:33:58.618 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:33:58 smithi092 ceph-mon[29647]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:33:59.401 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: Generating public/private rsa key pair. 2024-09-11T08:33:59.401 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: Your identification has been saved in /tmp/tmpjrfr7lb_/key 2024-09-11T08:33:59.401 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: Your public key has been saved in /tmp/tmpjrfr7lb_/key.pub 2024-09-11T08:33:59.401 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: The key fingerprint is: 2024-09-11T08:33:59.401 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: SHA256:GyRjfwTI+R3V8N9doposTFojkGfE9tBkQOriqPkgHv4 ceph-211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:33:59.401 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: The key's randomart image is: 2024-09-11T08:33:59.401 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: +---[RSA 3072]----+ 2024-09-11T08:33:59.402 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: | ++*+ .oo | 2024-09-11T08:33:59.402 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: | +B.... .. | 2024-09-11T08:33:59.402 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: | +.*+.... o .| 2024-09-11T08:33:59.402 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: | . = =o.. . o+| 2024-09-11T08:33:59.402 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: | . . . S . . +| 2024-09-11T08:33:59.402 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: | o . * * o | 2024-09-11T08:33:59.402 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: |+.. . + + | 2024-09-11T08:33:59.402 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: |=o. . | 2024-09-11T08:33:59.402 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: |o+oE | 2024-09-11T08:33:59.402 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:33:59 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: +----[SHA256]-----+ 2024-09-11T08:34:00.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:00 smithi092 ceph-mon[29647]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:00.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:00 smithi092 ceph-mon[29647]: Generating ssh key... 2024-09-11T08:34:00.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:00 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' 2024-09-11T08:34:00.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:00 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' 2024-09-11T08:34:00.826 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKJfwWCp5YE5py0dakJhOxiPC9KJI+DfkOG9wk/gSiUrkJnJIWQDW10ZB59ntCZaTv6T+Ph3kwTrP3li9m4NseGP3FIdT9G/l5KQA7tm9roFyeYU+rN799mDpgr0zUg0R7zvl31Y9j9+3gFn8XAzt0r4IbTEvkLVhh1hN70U6EkA938pfl0s7Qgui/H6OHbCAiUanYWtwL6/ZGsonFMbaPmW0JSY93fEag0LtI2J0NHvuIpYFj99McBn91Os48F6wNDaIEWbJhkxkCrg4gmv4bhSkppdoa7wXJ13GMi1F+g+Gn1+z5oSFWfmiHJwt29DPCf/kH4z1OxMXnz8TEwkXXWIjdULWxKCAnmKHswISyB5DklH1povIDTh9yZsQTaAxyNTfV7Tkgc3mA78jjJBHDdp2MyliBEFJ/fv22YsUdluj6g2szOSZa0jEck6JhcU3pQ4kFaOCphLiImT0aR7TUjCaqu6P3mxo7WUfiCjgnIhh7LamM18S5haGaxYeQDb0= ceph-211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:34:00.826 INFO:teuthology.orchestra.run.smithi092.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T08:34:00.826 INFO:teuthology.orchestra.run.smithi092.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T08:34:00.834 INFO:teuthology.orchestra.run.smithi092.stdout:Adding host smithi092... 2024-09-11T08:34:01.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:01 smithi092 ceph-mon[29647]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:02.211 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:02 smithi092 ceph-mon[29647]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "addr": "172.21.15.92", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:03.171 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:03 smithi092 ceph-mon[29647]: Deploying cephadm binary to smithi092 2024-09-11T08:34:04.683 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Added host 'smithi092' with addr '172.21.15.92' 2024-09-11T08:34:04.683 INFO:teuthology.orchestra.run.smithi092.stdout:Deploying unmanaged mon service... 2024-09-11T08:34:05.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:05 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' 2024-09-11T08:34:05.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:05 smithi092 ceph-mon[29647]: Added host smithi092 2024-09-11T08:34:05.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:05 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:06.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:06 smithi092 ceph-mon[29647]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:06.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:06 smithi092 ceph-mon[29647]: Saving service mon spec with placement count:5 2024-09-11T08:34:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:06 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' 2024-09-11T08:34:07.007 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T08:34:07.007 INFO:teuthology.orchestra.run.smithi092.stdout:Deploying unmanaged mgr service... 2024-09-11T08:34:07.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:07 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' 2024-09-11T08:34:07.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:07 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' 2024-09-11T08:34:08.831 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:08 smithi092 ceph-mon[29647]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:08.831 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:08 smithi092 ceph-mon[29647]: Saving service mgr spec with placement count:2 2024-09-11T08:34:08.831 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:08 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' 2024-09-11T08:34:08.831 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:08 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' 2024-09-11T08:34:08.833 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T08:34:10.408 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:10 smithi092 ceph-mon[29647]: from='mgr.14118 172.21.15.92:0/3483755692' entity='mgr.y' 2024-09-11T08:34:10.409 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:10 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3712731077' entity='client.admin' 2024-09-11T08:34:12.165 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:11 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4017060810' entity='client.admin' 2024-09-11T08:34:12.167 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling the dashboard module... 2024-09-11T08:34:13.192 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:12 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2976778558' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T08:34:13.253 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:12 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ignoring --setuser ceph since I am not root 2024-09-11T08:34:13.253 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:12 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ignoring --setgroup ceph since I am not root 2024-09-11T08:34:13.253 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:13.068+0000 7f11d66d2ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:34:13.253 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:13.125+0000 7f11d66d2ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:34:13.253 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:13.192+0000 7f11d66d2ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:34:13.460 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:13.319+0000 7f11d66d2ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:34:13.460 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:13.459+0000 7f11d66d2ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:34:14.007 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:13 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2976778558' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T08:34:14.008 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:13 smithi092 ceph-mon[29647]: mgrmap e9: y(active, since 18s) 2024-09-11T08:34:14.008 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:13.723+0000 7f11d66d2ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:34:14.008 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:13.786+0000 7f11d66d2ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:34:14.008 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:13.951+0000 7f11d66d2ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:34:14.796 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:34:14.797 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T08:34:14.798 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T08:34:14.798 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T08:34:14.798 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T08:34:14.798 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:34:14.798 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for the mgr to restart... 2024-09-11T08:34:14.798 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr epoch 9... 2024-09-11T08:34:15.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:14 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/629540581' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T08:34:15.065 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:14 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:14.812+0000 7f11d66d2ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:34:15.066 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:14 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:14.879+0000 7f11d66d2ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:34:15.876 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:15 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:15.590+0000 7f11d66d2ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:34:15.876 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:15 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:15.700+0000 7f11d66d2ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:34:15.877 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:15 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:15.818+0000 7f11d66d2ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:34:15.877 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:15 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:15.876+0000 7f11d66d2ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:34:16.256 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:16.001+0000 7f11d66d2ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:34:16.257 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:16.067+0000 7f11d66d2ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:34:16.257 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:16.124+0000 7f11d66d2ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:34:16.257 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:16.256+0000 7f11d66d2ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:34:16.516 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:16.318+0000 7f11d66d2ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:34:16.516 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:16.381+0000 7f11d66d2ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:34:16.516 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:16.460+0000 7f11d66d2ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:34:16.516 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:16.516+0000 7f11d66d2ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:34:16.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:16.571+0000 7f11d66d2ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:34:17.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:17 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:17.304+0000 7f11d66d2ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:34:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:17 smithi092 ceph-mon[29647]: Active manager daemon y restarted 2024-09-11T08:34:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:17 smithi092 ceph-mon[29647]: Activating manager daemon y 2024-09-11T08:34:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:17 smithi092 ceph-mon[29647]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T08:34:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:17 smithi092 ceph-mon[29647]: mgrmap e10: y(active, starting, since 0.00340926s) 2024-09-11T08:34:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:34:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:34:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:34:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:34:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:34:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:17 smithi092 ceph-mon[29647]: Manager daemon y is now available 2024-09-11T08:34:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:17.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:34:17.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:34:19.140 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:34:19.140 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T08:34:19.140 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T08:34:19.140 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:34:19.140 INFO:teuthology.orchestra.run.smithi092.stdout:mgr epoch 9 is available 2024-09-11T08:34:19.140 INFO:teuthology.orchestra.run.smithi092.stdout:Generating a dashboard self-signed certificate... 2024-09-11T08:34:19.436 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-mon[29647]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T08:34:19.437 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-mon[29647]: mgrmap e11: y(active, since 1.00562s) 2024-09-11T08:34:19.437 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-mon[29647]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T08:34:19.437 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:19.437 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:19.356+0000 7f118722f640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:34:19] ENGINE Error in HTTPServer.serve 2024-09-11T08:34:19.438 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: Traceback (most recent call last): 2024-09-11T08:34:19.438 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:34:19.438 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: self._connections.run(self.expiration_interval) 2024-09-11T08:34:19.438 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:34:19.438 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: self._run(expiration_interval) 2024-09-11T08:34:19.438 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:34:19.439 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:34:19.439 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:34:19.439 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:34:19.439 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:34:19.439 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: s = self.context.wrap_socket( 2024-09-11T08:34:19.439 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:34:19.440 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: return self.sslsocket_class._create( 2024-09-11T08:34:19.440 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:34:19.440 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: self.do_handshake() 2024-09-11T08:34:19.440 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:34:19.440 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: self._sslobj.do_handshake() 2024-09-11T08:34:19.440 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:34:19.441 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:34:19 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:34:20.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:20 smithi092 ceph-mon[29647]: [11/Sep/2024:08:34:19] ENGINE Bus STARTING 2024-09-11T08:34:20.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:20 smithi092 ceph-mon[29647]: [11/Sep/2024:08:34:19] ENGINE Serving on https://172.21.15.92:7150 2024-09-11T08:34:20.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:20 smithi092 ceph-mon[29647]: [11/Sep/2024:08:34:19] ENGINE Bus STARTED 2024-09-11T08:34:20.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:20 smithi092 ceph-mon[29647]: [11/Sep/2024:08:34:19] ENGINE Error in HTTPServer.serve 2024-09-11T08:34:20.554 INFO:journalctl@ceph.mon.a.smithi092.stdout: Traceback (most recent call last): 2024-09-11T08:34:20.554 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:34:20.555 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:34:20.555 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:34:20.555 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._run(expiration_interval) 2024-09-11T08:34:20.555 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:34:20.555 INFO:journalctl@ceph.mon.a.smithi092.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:34:20.555 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:34:20.555 INFO:journalctl@ceph.mon.a.smithi092.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:34:20.555 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:34:20.555 INFO:journalctl@ceph.mon.a.smithi092.stdout: s = self.context.wrap_socket( 2024-09-11T08:34:20.555 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:34:20.555 INFO:journalctl@ceph.mon.a.smithi092.stdout: return self.sslsocket_class._create( 2024-09-11T08:34:20.555 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:34:20.555 INFO:journalctl@ceph.mon.a.smithi092.stdout: self.do_handshake() 2024-09-11T08:34:20.556 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:34:20.556 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._sslobj.do_handshake() 2024-09-11T08:34:20.556 INFO:journalctl@ceph.mon.a.smithi092.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:34:20.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:20 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:20.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:20 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:20.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:20 smithi092 ceph-mon[29647]: mgrmap e12: y(active, since 2s) 2024-09-11T08:34:20.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:20 smithi092 ceph-mon[29647]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:20.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:20 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:20.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:20 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:20.557 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T08:34:20.557 INFO:teuthology.orchestra.run.smithi092.stdout:Creating initial admin user... 2024-09-11T08:34:22.916 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$/wp9EgWmkWf2vvnN2I.yXeKcFHG46TGm8yOGsQ/.RR/RyBSw2P5KG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726043662, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T08:34:22.917 INFO:teuthology.orchestra.run.smithi092.stdout:Fetching dashboard port number... 2024-09-11T08:34:23.422 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:23 smithi092 ceph-mon[29647]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:23.423 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:23 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:24.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T08:34:24.175 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:34:24.175 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-09-11T08:34:24.183 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:34:24.183 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-11T08:34:24.183 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-09-11T08:34:24.183 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T08:34:24.185 INFO:teuthology.orchestra.run.smithi092.stdout:Ceph Dashboard is now available at: 2024-09-11T08:34:24.185 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:34:24.185 INFO:teuthology.orchestra.run.smithi092.stdout: URL: https://smithi092.front.sepia.ceph.com:8443/ 2024-09-11T08:34:24.185 INFO:teuthology.orchestra.run.smithi092.stdout: User: admin 2024-09-11T08:34:24.185 INFO:teuthology.orchestra.run.smithi092.stdout: Password: pw367uxjv9 2024-09-11T08:34:24.185 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:34:24.185 INFO:teuthology.orchestra.run.smithi092.stdout:Saving cluster configuration to /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config directory 2024-09-11T08:34:24.186 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling autotune for osd_memory_target 2024-09-11T08:34:24.417 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:24 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1817310890' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T08:34:25.393 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:25 smithi092 ceph-mon[29647]: mgrmap e13: y(active, since 6s) 2024-09-11T08:34:26.802 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T08:34:26.802 INFO:teuthology.orchestra.run.smithi092.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T08:34:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:34:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:34:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:34:26.803 INFO:teuthology.orchestra.run.smithi092.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T08:34:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:34:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-11T08:34:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:34:26.803 INFO:teuthology.orchestra.run.smithi092.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T08:34:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:34:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: ceph telemetry on 2024-09-11T08:34:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:34:26.803 INFO:teuthology.orchestra.run.smithi092.stdout:For more information see: 2024-09-11T08:34:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:34:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-11T08:34:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:34:26.804 INFO:teuthology.orchestra.run.smithi092.stdout:Bootstrap complete. 2024-09-11T08:34:26.843 INFO:tasks.cephadm:Fetching config... 2024-09-11T08:34:26.844 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:34:26.844 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T08:34:26.865 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T08:34:26.866 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:34:26.866 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T08:34:26.922 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T08:34:26.922 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:34:26.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T08:34:26.988 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T08:34:26.988 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:34:26.988 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T08:34:27.044 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T08:34:27.044 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKJfwWCp5YE5py0dakJhOxiPC9KJI+DfkOG9wk/gSiUrkJnJIWQDW10ZB59ntCZaTv6T+Ph3kwTrP3li9m4NseGP3FIdT9G/l5KQA7tm9roFyeYU+rN799mDpgr0zUg0R7zvl31Y9j9+3gFn8XAzt0r4IbTEvkLVhh1hN70U6EkA938pfl0s7Qgui/H6OHbCAiUanYWtwL6/ZGsonFMbaPmW0JSY93fEag0LtI2J0NHvuIpYFj99McBn91Os48F6wNDaIEWbJhkxkCrg4gmv4bhSkppdoa7wXJ13GMi1F+g+Gn1+z5oSFWfmiHJwt29DPCf/kH4z1OxMXnz8TEwkXXWIjdULWxKCAnmKHswISyB5DklH1povIDTh9yZsQTaAxyNTfV7Tkgc3mA78jjJBHDdp2MyliBEFJ/fv22YsUdluj6g2szOSZa0jEck6JhcU3pQ4kFaOCphLiImT0aR7TUjCaqu6P3mxo7WUfiCjgnIhh7LamM18S5haGaxYeQDb0= ceph-211625d4-7018-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T08:34:27.147 INFO:teuthology.orchestra.run.smithi092.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKJfwWCp5YE5py0dakJhOxiPC9KJI+DfkOG9wk/gSiUrkJnJIWQDW10ZB59ntCZaTv6T+Ph3kwTrP3li9m4NseGP3FIdT9G/l5KQA7tm9roFyeYU+rN799mDpgr0zUg0R7zvl31Y9j9+3gFn8XAzt0r4IbTEvkLVhh1hN70U6EkA938pfl0s7Qgui/H6OHbCAiUanYWtwL6/ZGsonFMbaPmW0JSY93fEag0LtI2J0NHvuIpYFj99McBn91Os48F6wNDaIEWbJhkxkCrg4gmv4bhSkppdoa7wXJ13GMi1F+g+Gn1+z5oSFWfmiHJwt29DPCf/kH4z1OxMXnz8TEwkXXWIjdULWxKCAnmKHswISyB5DklH1povIDTh9yZsQTaAxyNTfV7Tkgc3mA78jjJBHDdp2MyliBEFJ/fv22YsUdluj6g2szOSZa0jEck6JhcU3pQ4kFaOCphLiImT0aR7TUjCaqu6P3mxo7WUfiCjgnIhh7LamM18S5haGaxYeQDb0= ceph-211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:34:27.159 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKJfwWCp5YE5py0dakJhOxiPC9KJI+DfkOG9wk/gSiUrkJnJIWQDW10ZB59ntCZaTv6T+Ph3kwTrP3li9m4NseGP3FIdT9G/l5KQA7tm9roFyeYU+rN799mDpgr0zUg0R7zvl31Y9j9+3gFn8XAzt0r4IbTEvkLVhh1hN70U6EkA938pfl0s7Qgui/H6OHbCAiUanYWtwL6/ZGsonFMbaPmW0JSY93fEag0LtI2J0NHvuIpYFj99McBn91Os48F6wNDaIEWbJhkxkCrg4gmv4bhSkppdoa7wXJ13GMi1F+g+Gn1+z5oSFWfmiHJwt29DPCf/kH4z1OxMXnz8TEwkXXWIjdULWxKCAnmKHswISyB5DklH1povIDTh9yZsQTaAxyNTfV7Tkgc3mA78jjJBHDdp2MyliBEFJ/fv22YsUdluj6g2szOSZa0jEck6JhcU3pQ4kFaOCphLiImT0aR7TUjCaqu6P3mxo7WUfiCjgnIhh7LamM18S5haGaxYeQDb0= ceph-211625d4-7018-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T08:34:27.278 INFO:teuthology.orchestra.run.smithi145.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKJfwWCp5YE5py0dakJhOxiPC9KJI+DfkOG9wk/gSiUrkJnJIWQDW10ZB59ntCZaTv6T+Ph3kwTrP3li9m4NseGP3FIdT9G/l5KQA7tm9roFyeYU+rN799mDpgr0zUg0R7zvl31Y9j9+3gFn8XAzt0r4IbTEvkLVhh1hN70U6EkA938pfl0s7Qgui/H6OHbCAiUanYWtwL6/ZGsonFMbaPmW0JSY93fEag0LtI2J0NHvuIpYFj99McBn91Os48F6wNDaIEWbJhkxkCrg4gmv4bhSkppdoa7wXJ13GMi1F+g+Gn1+z5oSFWfmiHJwt29DPCf/kH4z1OxMXnz8TEwkXXWIjdULWxKCAnmKHswISyB5DklH1povIDTh9yZsQTaAxyNTfV7Tkgc3mA78jjJBHDdp2MyliBEFJ/fv22YsUdluj6g2szOSZa0jEck6JhcU3pQ4kFaOCphLiImT0aR7TUjCaqu6P3mxo7WUfiCjgnIhh7LamM18S5haGaxYeQDb0= ceph-211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:34:27.291 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T08:34:27.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:27 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2698484094' entity='client.admin' 2024-09-11T08:34:27.532 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:34:29.864 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T08:34:29.864 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T08:34:30.331 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:34:30.538 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:30 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2223486478' entity='client.admin' 2024-09-11T08:34:33.634 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi145 2024-09-11T08:34:33.635 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-11T08:34:33.635 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/etc/ceph/ceph.conf 2024-09-11T08:34:33.654 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-11T08:34:33.654 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:34:33.713 INFO:tasks.cephadm:Adding host smithi145 to orchestrator... 2024-09-11T08:34:33.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch host add smithi145 2024-09-11T08:34:33.956 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:34:34.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:33 smithi092 ceph-mon[29647]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:34.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:33 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:34.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:33 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:34.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:33 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:34.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:33 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:34:34.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:33 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:34.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:33 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:34.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:33 smithi092 ceph-mon[29647]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-11T08:34:34.998 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:34 smithi092 ceph-mon[29647]: Updating smithi092:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:34:34.998 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:34 smithi092 ceph-mon[29647]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:34:34.998 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:34 smithi092 ceph-mon[29647]: Updating smithi092:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:34:34.998 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:34.999 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:34.999 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:34.999 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:34.999 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:34.999 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:34.999 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:36.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:36 smithi092 ceph-mon[29647]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi145", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:37.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:37 smithi092 ceph-mon[29647]: Deploying cephadm binary to smithi145 2024-09-11T08:34:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:38 smithi092 ceph-mon[29647]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:38.566 INFO:teuthology.orchestra.run.smithi092.stdout:Added host 'smithi145' with addr '172.21.15.145' 2024-09-11T08:34:39.109 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T08:34:39.345 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:34:39.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:39 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:39.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:39 smithi092 ceph-mon[29647]: Added host smithi145 2024-09-11T08:34:39.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:39 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:39.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:39 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:39.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:39 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:41.025 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:34:41.025 INFO:teuthology.orchestra.run.smithi092.stdout:[{"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}, {"addr": "172.21.15.145", "hostname": "smithi145", "labels": [], "status": ""}] 2024-09-11T08:34:41.026 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:40 smithi092 ceph-mon[29647]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:41.026 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:40 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:41.675 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T08:34:41.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T08:34:41.922 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:34:43.014 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:42 smithi092 ceph-mon[29647]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:34:43.014 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:42 smithi092 ceph-mon[29647]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:43.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:42 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:43.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:42 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:43.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:42 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:43.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:42 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:43.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:42 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:34:43.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:42 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:43.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:42 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:43 smithi092 ceph-mon[29647]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-11T08:34:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:43 smithi092 ceph-mon[29647]: Updating smithi145:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:34:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:43 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:43 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:43 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:43 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3228282024' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T08:34:44.542 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2024-09-11T08:34:45.112 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:44 smithi092 ceph-mon[29647]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:34:45.113 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:44 smithi092 ceph-mon[29647]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:45.113 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:44 smithi092 ceph-mon[29647]: Updating smithi145:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:34:45.113 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:44 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3228282024' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T08:34:45.113 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:44 smithi092 ceph-mon[29647]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:34:45.143 INFO:tasks.cephadm:Adding mon.a on smithi092 2024-09-11T08:34:45.143 INFO:tasks.cephadm:Adding mon.c on smithi092 2024-09-11T08:34:45.143 INFO:tasks.cephadm:Adding mon.b on smithi145 2024-09-11T08:34:45.143 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi092:172.21.15.92=a;smithi092:[v2:172.21.15.92:3301,v1:172.21.15.92:6790]=c;smithi145:172.21.15.145=b' 2024-09-11T08:34:45.387 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:34:47.047 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled mon update... 2024-09-11T08:34:47.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:46 smithi092 ceph-mon[29647]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:47.559 DEBUG:teuthology.orchestra.run.smithi092:mon.c> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T08:34:47.562 DEBUG:teuthology.orchestra.run.smithi145:mon.b> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T08:34:47.564 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T08:34:47.564 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T08:34:48.291 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:34:48.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:48 smithi092 ceph-mon[29647]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi092:172.21.15.92=a;smithi092:[v2:172.21.15.92:3301,v1:172.21.15.92:6790]=c;smithi145:172.21.15.145=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:48.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:48 smithi092 ceph-mon[29647]: Saving service mon spec with placement smithi092:172.21.15.92=a;smithi092:[v2:172.21.15.92:3301,v1:172.21.15.92:6790]=c;smithi145:172.21.15.145=b;count:3 2024-09-11T08:34:48.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:48 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:48.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:48 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:48.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:48 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:48.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:48 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:48.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:48 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:48.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:48 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:48.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:48 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:48.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:48 smithi092 ceph-mon[29647]: Deploying daemon mon.b on smithi145 2024-09-11T08:34:48.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:48 smithi092 ceph-mon[29647]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:50.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:50 smithi092 ceph-mon[29647]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:50.845 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:34:50.846 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":1,"fsid":"211625d4-7018-11ef-bcea-c7b262605968","modified":"2024-09-11T08:33:29.232780Z","created":"2024-09-11T08:33:29.232780Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T08:34:50.846 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 1 2024-09-11T08:34:51.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:51 smithi092 ceph-mon[29647]: from='client.? 172.21.15.145:0/3712911306' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:34:52.402 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T08:34:52.402 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T08:34:52.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:52 smithi092 ceph-mon[29647]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:53.040 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: mon.b@-1(synchronizing).mds e1 new map 2024-09-11T08:34:53.040 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-11T08:34:53.040 INFO:journalctl@ceph.mon.b.smithi145.stdout: e1 2024-09-11T08:34:53.041 INFO:journalctl@ceph.mon.b.smithi145.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T08:34:53.041 INFO:journalctl@ceph.mon.b.smithi145.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T08:34:53.041 INFO:journalctl@ceph.mon.b.smithi145.stdout: legacy client fscid: -1 2024-09-11T08:34:53.041 INFO:journalctl@ceph.mon.b.smithi145.stdout: 2024-09-11T08:34:53.041 INFO:journalctl@ceph.mon.b.smithi145.stdout: No filesystems configured 2024-09-11T08:34:53.041 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-11T08:34:53.042 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-11T08:34:53.042 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-11T08:34:53.042 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-11T08:34:53.042 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-11T08:34:53.042 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-11T08:34:53.145 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:34:53.323 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T08:34:53.323 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:34:53.323 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:34:53.323 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:34:53.323 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:34:53.323 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:53.324 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: Updating smithi145:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:34:53.324 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3228282024' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T08:34:53.324 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:34:53.324 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:53.324 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi092:172.21.15.92=a;smithi092:[v2:172.21.15.92:3301,v1:172.21.15.92:6790]=c;smithi145:172.21.15.145=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:53.324 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: Saving service mon spec with placement smithi092:172.21.15.92=a;smithi092:[v2:172.21.15.92:3301,v1:172.21.15.92:6790]=c;smithi145:172.21.15.145=b;count:3 2024-09-11T08:34:53.325 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:53.325 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:53.325 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:53.325 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:53.325 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:53.325 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:53.326 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:53.326 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: Deploying daemon mon.b on smithi145 2024-09-11T08:34:53.326 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:53.326 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:53.326 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: from='client.? 172.21.15.145:0/3712911306' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:34:53.326 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:53.326 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 ceph-mon[33384]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T08:34:53.326 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 bash[33322]: 01d29144d2b3858ee0df800dc612ee1deddb3c99cdfc863cd8bded066e77fe12 2024-09-11T08:34:53.326 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:53 smithi145 systemd[1]: Started Ceph mon.b for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:34:58.387 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:34:58.388 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":2,"fsid":"211625d4-7018-11ef-bcea-c7b262605968","modified":"2024-09-11T08:34:53.045539Z","created":"2024-09-11T08:33:29.232780Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T08:34:58.388 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 2 2024-09-11T08:34:58.388 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:34:58.388 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: mon.a calling monitor election 2024-09-11T08:34:58.389 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:58.389 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:58.389 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:58.389 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:58.389 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: mon.b calling monitor election 2024-09-11T08:34:58.389 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:58.389 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:58.389 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:58.389 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:58.389 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:58.389 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:34:58.389 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: monmap e2: 2 mons at {a=[v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0],b=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:34:58.389 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: fsmap 2024-09-11T08:34:58.390 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:34:58.390 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: mgrmap e13: y(active, since 40s) 2024-09-11T08:34:58.390 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: overall HEALTH_OK 2024-09-11T08:34:58.390 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:58.390 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:58.390 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:58.390 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:58.390 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:58 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:58.503 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:34:58.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: mon.a calling monitor election 2024-09-11T08:34:58.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:58.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:58.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:58.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:58.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: mon.b calling monitor election 2024-09-11T08:34:58.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:58.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:58.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:58.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:58.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:58.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:34:58.505 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: monmap e2: 2 mons at {a=[v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0],b=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:34:58.505 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: fsmap 2024-09-11T08:34:58.505 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:34:58.505 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: mgrmap e13: y(active, since 40s) 2024-09-11T08:34:58.505 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: overall HEALTH_OK 2024-09-11T08:34:58.505 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:58.505 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:58.505 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:34:58.505 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:58.505 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:58 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:59.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:59 smithi145 ceph-mon[33384]: Deploying daemon mon.c on smithi092 2024-09-11T08:34:59.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:59 smithi145 ceph-mon[33384]: from='client.? 172.21.15.145:0/979137394' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:34:59.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:34:59 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:59.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:59 smithi092 ceph-mon[29647]: Deploying daemon mon.c on smithi092 2024-09-11T08:34:59.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:59 smithi092 ceph-mon[29647]: from='client.? 172.21.15.145:0/979137394' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:34:59.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:34:59 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:59.995 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T08:34:59.995 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T08:35:00.233 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:35:00.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:00 smithi092 ceph-mon[29647]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:00.354 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:00 smithi145 ceph-mon[33384]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:00.982 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:00 smithi092 podman[41076]: 2024-09-11 08:35:00.801107459 +0000 UTC m=+0.015630160 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:35:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 podman[41076]: 2024-09-11 08:35:01.085533266 +0000 UTC m=+0.300055960 container init 656e8e0df9f5ae5871efdde6209e2d63d3fa4fbde717060d12806f8da813c188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:35:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 podman[41076]: 2024-09-11 08:35:01.090164042 +0000 UTC m=+0.304686737 container start 656e8e0df9f5ae5871efdde6209e2d63d3fa4fbde717060d12806f8da813c188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-c, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-11T08:35:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T08:35:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 7 2024-09-11T08:35:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: pidfile_write: ignore empty --pid-file 2024-09-11T08:35:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: load: jerasure load: lrc 2024-09-11T08:35:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: RocksDB version: 6.15.5 2024-09-11T08:35:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T08:35:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Compile date Sep 10 2024 2024-09-11T08:35:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: DB SUMMARY 2024-09-11T08:35:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: DB Session ID: CQBYMU0O0CI6FR9AU3O2 2024-09-11T08:35:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: CURRENT file: CURRENT 2024-09-11T08:35:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T08:35:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-11T08:35:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-11T08:35:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-11T08:35:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.error_if_exists: 0 2024-09-11T08:35:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.create_if_missing: 0 2024-09-11T08:35:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.paranoid_checks: 1 2024-09-11T08:35:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T08:35:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.env: 0x55654b88e440 2024-09-11T08:35:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.fs: Posix File System 2024-09-11T08:35:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.info_log: 0x55654cce6ec0 2024-09-11T08:35:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T08:35:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.statistics: (nil) 2024-09-11T08:35:01.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.use_fsync: 0 2024-09-11T08:35:01.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_log_file_size: 0 2024-09-11T08:35:01.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T08:35:01.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T08:35:01.570 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T08:35:01.570 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T08:35:01.570 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.allow_fallocate: 1 2024-09-11T08:35:01.570 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T08:35:01.570 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T08:35:01.570 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.use_direct_reads: 0 2024-09-11T08:35:01.570 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T08:35:01.570 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T08:35:01.570 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.db_log_dir: 2024-09-11T08:35:01.570 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-09-11T08:35:01.570 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T08:35:01.570 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T08:35:01.570 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T08:35:01.570 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T08:35:01.571 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T08:35:01.571 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T08:35:01.571 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T08:35:01.571 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T08:35:01.571 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.write_buffer_manager: 0x55654cda47b0 2024-09-11T08:35:01.571 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T08:35:01.571 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T08:35:01.571 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T08:35:01.571 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T08:35:01.571 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T08:35:01.571 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T08:35:01.571 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T08:35:01.571 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T08:35:01.572 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T08:35:01.572 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.unordered_write: 0 2024-09-11T08:35:01.572 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T08:35:01.572 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T08:35:01.572 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T08:35:01.572 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T08:35:01.572 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.row_cache: None 2024-09-11T08:35:01.572 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.wal_filter: None 2024-09-11T08:35:01.572 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T08:35:01.572 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T08:35:01.572 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.preserve_deletes: 0 2024-09-11T08:35:01.572 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.two_write_queues: 0 2024-09-11T08:35:01.573 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T08:35:01.573 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.atomic_flush: 0 2024-09-11T08:35:01.573 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T08:35:01.573 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T08:35:01.573 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T08:35:01.573 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.log_readahead_size: 0 2024-09-11T08:35:01.573 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T08:35:01.573 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T08:35:01.573 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T08:35:01.573 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T08:35:01.573 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T08:35:01.573 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T08:35:01.573 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_background_jobs: 2 2024-09-11T08:35:01.573 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_background_compactions: -1 2024-09-11T08:35:01.573 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_subcompactions: 1 2024-09-11T08:35:01.574 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T08:35:01.574 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T08:35:01.574 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T08:35:01.574 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T08:35:01.574 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T08:35:01.574 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T08:35:01.574 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T08:35:01.574 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T08:35:01.574 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_open_files: -1 2024-09-11T08:35:01.574 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T08:35:01.574 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T08:35:01.575 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T08:35:01.575 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T08:35:01.575 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_background_flushes: -1 2024-09-11T08:35:01.575 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Compression algorithms supported: 2024-09-11T08:35:01.575 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: kZSTD supported: 0 2024-09-11T08:35:01.575 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: kXpressCompression supported: 0 2024-09-11T08:35:01.575 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: kBZip2Compression supported: 0 2024-09-11T08:35:01.575 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T08:35:01.575 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: kLZ4Compression supported: 1 2024-09-11T08:35:01.575 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: kZlibCompression supported: 1 2024-09-11T08:35:01.575 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T08:35:01.575 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: kSnappyCompression supported: 1 2024-09-11T08:35:01.575 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T08:35:01.576 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-09-11T08:35:01.576 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T08:35:01.576 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T08:35:01.576 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.merge_operator: 2024-09-11T08:35:01.576 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compaction_filter: None 2024-09-11T08:35:01.576 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T08:35:01.576 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T08:35:01.576 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T08:35:01.576 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T08:35:01.576 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55654cc8da38) 2024-09-11T08:35:01.576 INFO:journalctl@ceph.mon.c.smithi092.stdout: cache_index_and_filter_blocks: 1 2024-09-11T08:35:01.576 INFO:journalctl@ceph.mon.c.smithi092.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T08:35:01.576 INFO:journalctl@ceph.mon.c.smithi092.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T08:35:01.576 INFO:journalctl@ceph.mon.c.smithi092.stdout: pin_top_level_index_and_filter: 1 2024-09-11T08:35:01.577 INFO:journalctl@ceph.mon.c.smithi092.stdout: index_type: 0 2024-09-11T08:35:01.577 INFO:journalctl@ceph.mon.c.smithi092.stdout: data_block_index_type: 0 2024-09-11T08:35:01.577 INFO:journalctl@ceph.mon.c.smithi092.stdout: index_shortening: 1 2024-09-11T08:35:01.577 INFO:journalctl@ceph.mon.c.smithi092.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T08:35:01.577 INFO:journalctl@ceph.mon.c.smithi092.stdout: hash_index_allow_collision: 1 2024-09-11T08:35:01.577 INFO:journalctl@ceph.mon.c.smithi092.stdout: checksum: 1 2024-09-11T08:35:01.577 INFO:journalctl@ceph.mon.c.smithi092.stdout: no_block_cache: 0 2024-09-11T08:35:01.577 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_cache: 0x55654ccc31f0 2024-09-11T08:35:01.577 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_cache_name: BinnedLRUCache 2024-09-11T08:35:01.577 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_cache_options: 2024-09-11T08:35:01.577 INFO:journalctl@ceph.mon.c.smithi092.stdout: capacity : 536870912 2024-09-11T08:35:01.577 INFO:journalctl@ceph.mon.c.smithi092.stdout: num_shard_bits : 4 2024-09-11T08:35:01.577 INFO:journalctl@ceph.mon.c.smithi092.stdout: strict_capacity_limit : 0 2024-09-11T08:35:01.577 INFO:journalctl@ceph.mon.c.smithi092.stdout: high_pri_pool_ratio: 0.000 2024-09-11T08:35:01.578 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_cache_compressed: (nil) 2024-09-11T08:35:01.578 INFO:journalctl@ceph.mon.c.smithi092.stdout: persistent_cache: (nil) 2024-09-11T08:35:01.578 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_size: 4096 2024-09-11T08:35:01.578 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_size_deviation: 10 2024-09-11T08:35:01.578 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_restart_interval: 16 2024-09-11T08:35:01.578 INFO:journalctl@ceph.mon.c.smithi092.stdout: index_block_restart_interval: 1 2024-09-11T08:35:01.578 INFO:journalctl@ceph.mon.c.smithi092.stdout: metadata_block_size: 4096 2024-09-11T08:35:01.578 INFO:journalctl@ceph.mon.c.smithi092.stdout: partition_filters: 0 2024-09-11T08:35:01.578 INFO:journalctl@ceph.mon.c.smithi092.stdout: use_delta_encoding: 1 2024-09-11T08:35:01.578 INFO:journalctl@ceph.mon.c.smithi092.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T08:35:01.578 INFO:journalctl@ceph.mon.c.smithi092.stdout: whole_key_filtering: 1 2024-09-11T08:35:01.578 INFO:journalctl@ceph.mon.c.smithi092.stdout: verify_compression: 0 2024-09-11T08:35:01.578 INFO:journalctl@ceph.mon.c.smithi092.stdout: read_amp_bytes_per_bit: 0 2024-09-11T08:35:01.578 INFO:journalctl@ceph.mon.c.smithi092.stdout: format_version: 4 2024-09-11T08:35:01.579 INFO:journalctl@ceph.mon.c.smithi092.stdout: enable_index_compression: 1 2024-09-11T08:35:01.579 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_align: 0 2024-09-11T08:35:01.579 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T08:35:01.579 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T08:35:01.579 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compression: NoCompression 2024-09-11T08:35:01.579 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T08:35:01.579 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T08:35:01.579 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T08:35:01.579 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.num_levels: 7 2024-09-11T08:35:01.579 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T08:35:01.579 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T08:35:01.579 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T08:35:01.579 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T08:35:01.579 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T08:35:01.580 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T08:35:01.580 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T08:35:01.580 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:35:01.580 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T08:35:01.580 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T08:35:01.580 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T08:35:01.580 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T08:35:01.580 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T08:35:01.580 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T08:35:01.580 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:35:01.580 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T08:35:01.580 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T08:35:01.580 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T08:35:01.581 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T08:35:01.581 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T08:35:01.581 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T08:35:01.581 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T08:35:01.581 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T08:35:01.581 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T08:35:01.581 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T08:35:01.581 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T08:35:01.581 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T08:35:01.581 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T08:35:01.581 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T08:35:01.581 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T08:35:01.581 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T08:35:01.582 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T08:35:01.582 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T08:35:01.582 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T08:35:01.582 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T08:35:01.582 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T08:35:01.582 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T08:35:01.582 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T08:35:01.582 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T08:35:01.582 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T08:35:01.582 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T08:35:01.582 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T08:35:01.582 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T08:35:01.582 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T08:35:01.582 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T08:35:01.583 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T08:35:01.583 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T08:35:01.583 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T08:35:01.583 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T08:35:01.583 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T08:35:01.583 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.inplace_update_support: 0 2024-09-11T08:35:01.583 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T08:35:01.583 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T08:35:01.583 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T08:35:01.583 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T08:35:01.583 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.bloom_locality: 0 2024-09-11T08:35:01.583 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.max_successive_merges: 0 2024-09-11T08:35:01.583 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T08:35:01.583 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T08:35:01.584 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T08:35:01.584 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T08:35:01.584 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.ttl: 2592000 2024-09-11T08:35:01.584 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T08:35:01.584 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.enable_blob_files: false 2024-09-11T08:35:01.584 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.min_blob_size: 0 2024-09-11T08:35:01.584 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T08:35:01.584 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T08:35:01.584 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T08:35:01.584 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T08:35:01.584 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T08:35:01.584 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-11T08:35:01.584 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-11T08:35:01.585 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043701113952, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-11T08:35:01.585 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-11T08:35:01.585 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T08:35:01.585 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043701114253, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1696, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726043701, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "3ffa2838-1e41-47c5-ae7c-cbf213dee9a6", "db_session_id": "CQBYMU0O0CI6FR9AU3O2"}} 2024-09-11T08:35:01.585 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-11T08:35:01.585 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043701114567, "job": 1, "event": "recovery_finished"} 2024-09-11T08:35:01.585 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T08:35:01.585 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55654ccf6700 2024-09-11T08:35:01.585 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: DB pointer 0x55654ccc6000 2024-09-11T08:35:01.585 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T08:35:01.585 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T08:35:01.585 INFO:journalctl@ceph.mon.c.smithi092.stdout: ** DB Stats ** 2024-09-11T08:35:01.585 INFO:journalctl@ceph.mon.c.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:35:01.586 INFO:journalctl@ceph.mon.c.smithi092.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T08:35:01.586 INFO:journalctl@ceph.mon.c.smithi092.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T08:35:01.586 INFO:journalctl@ceph.mon.c.smithi092.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:35:01.586 INFO:journalctl@ceph.mon.c.smithi092.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T08:35:01.586 INFO:journalctl@ceph.mon.c.smithi092.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T08:35:01.586 INFO:journalctl@ceph.mon.c.smithi092.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:35:01.586 INFO:journalctl@ceph.mon.c.smithi092.stdout: 2024-09-11T08:35:01.586 INFO:journalctl@ceph.mon.c.smithi092.stdout: ** Compaction Stats [default] ** 2024-09-11T08:35:01.586 INFO:journalctl@ceph.mon.c.smithi092.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:35:01.586 INFO:journalctl@ceph.mon.c.smithi092.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:35:01.586 INFO:journalctl@ceph.mon.c.smithi092.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T08:35:01.586 INFO:journalctl@ceph.mon.c.smithi092.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T08:35:01.586 INFO:journalctl@ceph.mon.c.smithi092.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T08:35:01.586 INFO:journalctl@ceph.mon.c.smithi092.stdout: 2024-09-11T08:35:01.587 INFO:journalctl@ceph.mon.c.smithi092.stdout: ** Compaction Stats [default] ** 2024-09-11T08:35:01.587 INFO:journalctl@ceph.mon.c.smithi092.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:35:01.587 INFO:journalctl@ceph.mon.c.smithi092.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:35:01.587 INFO:journalctl@ceph.mon.c.smithi092.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T08:35:01.587 INFO:journalctl@ceph.mon.c.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:35:01.587 INFO:journalctl@ceph.mon.c.smithi092.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:35:01.587 INFO:journalctl@ceph.mon.c.smithi092.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:35:01.587 INFO:journalctl@ceph.mon.c.smithi092.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:35:01.587 INFO:journalctl@ceph.mon.c.smithi092.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:35:01.587 INFO:journalctl@ceph.mon.c.smithi092.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:35:01.587 INFO:journalctl@ceph.mon.c.smithi092.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:35:01.587 INFO:journalctl@ceph.mon.c.smithi092.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:35:01.587 INFO:journalctl@ceph.mon.c.smithi092.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:35:01.587 INFO:journalctl@ceph.mon.c.smithi092.stdout: 2024-09-11T08:35:01.588 INFO:journalctl@ceph.mon.c.smithi092.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:35:01.588 INFO:journalctl@ceph.mon.c.smithi092.stdout: 2024-09-11T08:35:01.588 INFO:journalctl@ceph.mon.c.smithi092.stdout: ** Compaction Stats [default] ** 2024-09-11T08:35:01.588 INFO:journalctl@ceph.mon.c.smithi092.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:35:01.588 INFO:journalctl@ceph.mon.c.smithi092.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:35:01.588 INFO:journalctl@ceph.mon.c.smithi092.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T08:35:01.588 INFO:journalctl@ceph.mon.c.smithi092.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T08:35:01.588 INFO:journalctl@ceph.mon.c.smithi092.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T08:35:01.588 INFO:journalctl@ceph.mon.c.smithi092.stdout: 2024-09-11T08:35:01.588 INFO:journalctl@ceph.mon.c.smithi092.stdout: ** Compaction Stats [default] ** 2024-09-11T08:35:01.588 INFO:journalctl@ceph.mon.c.smithi092.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:35:01.588 INFO:journalctl@ceph.mon.c.smithi092.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:35:01.588 INFO:journalctl@ceph.mon.c.smithi092.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T08:35:01.589 INFO:journalctl@ceph.mon.c.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:35:01.589 INFO:journalctl@ceph.mon.c.smithi092.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:35:01.589 INFO:journalctl@ceph.mon.c.smithi092.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:35:01.589 INFO:journalctl@ceph.mon.c.smithi092.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:35:01.589 INFO:journalctl@ceph.mon.c.smithi092.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:35:01.589 INFO:journalctl@ceph.mon.c.smithi092.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:35:01.589 INFO:journalctl@ceph.mon.c.smithi092.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:35:01.589 INFO:journalctl@ceph.mon.c.smithi092.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:35:01.589 INFO:journalctl@ceph.mon.c.smithi092.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:35:01.589 INFO:journalctl@ceph.mon.c.smithi092.stdout: 2024-09-11T08:35:01.589 INFO:journalctl@ceph.mon.c.smithi092.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:35:01.589 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-11T08:35:01.589 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: using public_addrv [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] 2024-09-11T08:35:01.590 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: starting mon.c rank -1 at public addrs [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] at bind addrs [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:35:01.590 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.c@-1(???) e0 preinit fsid 211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:35:01.590 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.c@-1(synchronizing).mds e1 new map 2024-09-11T08:35:01.590 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-11T08:35:01.590 INFO:journalctl@ceph.mon.c.smithi092.stdout: e1 2024-09-11T08:35:01.590 INFO:journalctl@ceph.mon.c.smithi092.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T08:35:01.590 INFO:journalctl@ceph.mon.c.smithi092.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T08:35:01.590 INFO:journalctl@ceph.mon.c.smithi092.stdout: legacy client fscid: -1 2024-09-11T08:35:01.590 INFO:journalctl@ceph.mon.c.smithi092.stdout: 2024-09-11T08:35:01.590 INFO:journalctl@ceph.mon.c.smithi092.stdout: No filesystems configured 2024-09-11T08:35:01.590 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-11T08:35:01.590 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-11T08:35:01.590 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-11T08:35:01.590 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-11T08:35:01.590 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-11T08:35:01.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-11T08:35:01.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T08:35:01.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:35:01.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:35:01.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:35:01.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:35:01.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:01.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: Updating smithi145:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:35:01.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3228282024' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T08:35:01.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:35:01.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:01.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi092:172.21.15.92=a;smithi092:[v2:172.21.15.92:3301,v1:172.21.15.92:6790]=c;smithi145:172.21.15.145=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:01.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: Saving service mon spec with placement smithi092:172.21.15.92=a;smithi092:[v2:172.21.15.92:3301,v1:172.21.15.92:6790]=c;smithi145:172.21.15.145=b;count:3 2024-09-11T08:35:01.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:01.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:01.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:01.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:01.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:01.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:35:01.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:01.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: Deploying daemon mon.b on smithi145 2024-09-11T08:35:01.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:01.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:01.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='client.? 172.21.15.145:0/3712911306' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:35:01.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:01.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:35:01.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.a calling monitor election 2024-09-11T08:35:01.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:01.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:01.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:01.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:01.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.b calling monitor election 2024-09-11T08:35:01.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:01.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:01.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:01.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:01.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:01.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:35:01.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: monmap e2: 2 mons at {a=[v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0],b=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:35:01.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: fsmap 2024-09-11T08:35:01.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:35:01.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mgrmap e13: y(active, since 40s) 2024-09-11T08:35:01.594 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: overall HEALTH_OK 2024-09-11T08:35:01.594 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:01.594 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:01.594 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:01.594 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:35:01.594 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:01.594 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: Deploying daemon mon.c on smithi092 2024-09-11T08:35:01.594 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='client.? 172.21.15.145:0/979137394' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:35:01.594 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:01.594 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:01.594 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 ceph-mon[41114]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T08:35:01.594 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 bash[41076]: 656e8e0df9f5ae5871efdde6209e2d63d3fa4fbde717060d12806f8da813c188 2024-09-11T08:35:01.594 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:01 smithi092 systemd[1]: Started Ceph mon.c for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:35:06.401 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-11T08:35:06.401 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":3,"fsid":"211625d4-7018-11ef-bcea-c7b262605968","modified":"2024-09-11T08:35:01.136454Z","created":"2024-09-11T08:33:29.232780Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3301","nonce":0},{"type":"v1","addr":"172.21.15.92:6790","nonce":0}]},"addr":"172.21.15.92:6790/0","public_addr":"172.21.15.92:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-11T08:35:06.401 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 3 2024-09-11T08:35:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:35:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: mon.a calling monitor election 2024-09-11T08:35:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: mon.b calling monitor election 2024-09-11T08:35:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: mon.c calling monitor election 2024-09-11T08:35:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T08:35:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: monmap e3: 3 mons at {a=[v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0],b=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0],c=[v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:35:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: fsmap 2024-09-11T08:35:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:35:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: mgrmap e13: y(active, since 48s) 2024-09-11T08:35:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: overall HEALTH_OK 2024-09-11T08:35:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:35:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: mon.a calling monitor election 2024-09-11T08:35:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: mon.b calling monitor election 2024-09-11T08:35:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: mon.c calling monitor election 2024-09-11T08:35:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:06.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T08:35:06.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: monmap e3: 3 mons at {a=[v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0],b=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0],c=[v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:35:06.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: fsmap 2024-09-11T08:35:06.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:35:06.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: mgrmap e13: y(active, since 48s) 2024-09-11T08:35:06.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: overall HEALTH_OK 2024-09-11T08:35:06.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:06.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:06.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:06.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:06 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:35:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: mon.a calling monitor election 2024-09-11T08:35:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: mon.b calling monitor election 2024-09-11T08:35:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: mon.c calling monitor election 2024-09-11T08:35:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T08:35:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: monmap e3: 3 mons at {a=[v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0],b=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0],c=[v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:35:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: fsmap 2024-09-11T08:35:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:35:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: mgrmap e13: y(active, since 48s) 2024-09-11T08:35:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: overall HEALTH_OK 2024-09-11T08:35:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:06.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:06.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:06.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:06 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:06.937 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T08:35:06.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T08:35:07.307 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:35:07.549 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:07 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:07.549 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:07 smithi092 ceph-mon[29647]: from='client.? 172.21.15.145:0/1862444036' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:35:07.549 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:07 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:07.549 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:07 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:07.550 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:07 smithi092 ceph-mon[41114]: from='client.? 172.21.15.145:0/1862444036' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:35:07.550 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:07 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:07.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:07 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:07 smithi145 ceph-mon[33384]: from='client.? 172.21.15.145:0/1862444036' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:35:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:07 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:08.471 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:08.471 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.471 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.471 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.471 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.471 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.471 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:08.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:08.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-11T08:35:08.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-11T08:35:08.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: Updating smithi092:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:35:08.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: Updating smithi145:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:35:08.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:35:08.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:35:08.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:08.473 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:08.473 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.473 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.473 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.473 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.473 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.473 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:08.473 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:08.473 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-11T08:35:08.473 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-11T08:35:08.473 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: Updating smithi092:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:35:08.473 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: Updating smithi145:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:35:08.474 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.474 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.474 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.474 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.474 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.474 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:35:08.474 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:35:08.474 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-11T08:35:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-11T08:35:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: Updating smithi092:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:35:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: Updating smithi145:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:35:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:35:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:35:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:09.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:09 smithi145 ceph-mon[33384]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T08:35:09.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:09 smithi145 ceph-mon[33384]: Reconfiguring daemon mon.a on smithi092 2024-09-11T08:35:09.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:09 smithi092 ceph-mon[29647]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T08:35:09.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:09 smithi092 ceph-mon[29647]: Reconfiguring daemon mon.a on smithi092 2024-09-11T08:35:09.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:09 smithi092 ceph-mon[41114]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T08:35:09.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:09 smithi092 ceph-mon[41114]: Reconfiguring daemon mon.a on smithi092 2024-09-11T08:35:09.750 INFO:teuthology.orchestra.run.smithi092.stdout:# minimal ceph.conf for 211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:35:09.750 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-09-11T08:35:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = 211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:35:09.750 INFO:teuthology.orchestra.run.smithi092.stdout: mon_host = [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] 2024-09-11T08:35:10.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[41114]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/173267151' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[41114]: Reconfiguring mon.c (monmap changed)... 2024-09-11T08:35:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:35:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:35:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[41114]: Reconfiguring daemon mon.c on smithi092 2024-09-11T08:35:10.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[29647]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:10.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/173267151' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:10.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:10.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:10.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[29647]: Reconfiguring mon.c (monmap changed)... 2024-09-11T08:35:10.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:35:10.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:35:10.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:10.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:10 smithi092 ceph-mon[29647]: Reconfiguring daemon mon.c on smithi092 2024-09-11T08:35:10.656 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T08:35:10.656 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:35:10.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T08:35:10.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:10 smithi145 ceph-mon[33384]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:10.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:10 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/173267151' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:10 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:10 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:10 smithi145 ceph-mon[33384]: Reconfiguring mon.c (monmap changed)... 2024-09-11T08:35:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:10 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:35:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:10 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:35:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:10 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:10 smithi145 ceph-mon[33384]: Reconfiguring daemon mon.c on smithi092 2024-09-11T08:35:10.684 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:35:10.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:35:10.749 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-11T08:35:10.749 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T08:35:10.782 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-11T08:35:10.782 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:35:10.851 INFO:tasks.cephadm:Adding mgr.y on smithi092 2024-09-11T08:35:10.851 INFO:tasks.cephadm:Adding mgr.x on smithi145 2024-09-11T08:35:10.851 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi092=y;smithi145=x' 2024-09-11T08:35:11.126 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:35:13.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[29647]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:13.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:13.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:13.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[29647]: Reconfiguring mon.b (monmap changed)... 2024-09-11T08:35:13.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:35:13.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:35:13.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:13.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[29647]: Reconfiguring daemon mon.b on smithi145 2024-09-11T08:35:13.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[41114]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:13.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:13.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:13.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[41114]: Reconfiguring mon.b (monmap changed)... 2024-09-11T08:35:13.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:35:13.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:35:13.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:13.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:12 smithi092 ceph-mon[41114]: Reconfiguring daemon mon.b on smithi145 2024-09-11T08:35:13.072 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:12 smithi145 ceph-mon[33384]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:13.072 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:13.073 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:13.073 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:12 smithi145 ceph-mon[33384]: Reconfiguring mon.b (monmap changed)... 2024-09-11T08:35:13.073 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:35:13.073 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:35:13.073 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:13.073 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:12 smithi145 ceph-mon[33384]: Reconfiguring daemon mon.b on smithi145 2024-09-11T08:35:13.342 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled mgr update... 2024-09-11T08:35:14.169 DEBUG:teuthology.orchestra.run.smithi145:mgr.x> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T08:35:14.171 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T08:35:14.171 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:35:14.171 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:35:14.188 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T08:35:14.188 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-09-11T08:35:14.243 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T08:35:14.243 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:35:14.243 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-11T08:35:14.243 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:35:14.243 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:35:14.244 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-11 08:35:00.634981795 +0000 2024-09-11T08:35:14.244 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-11 08:28:26.685310231 +0000 2024-09-11T08:35:14.244 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-11 08:28:26.685310231 +0000 2024-09-11T08:35:14.244 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-11 08:28:26.685310231 +0000 2024-09-11T08:35:14.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T08:35:14.309 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-11T08:35:14.309 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-11T08:35:14.309 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000273721 s, 1.9 MB/s 2024-09-11T08:35:14.310 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T08:35:14.368 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-09-11T08:35:14.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:14 smithi145 ceph-mon[33384]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:14.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:14 smithi145 ceph-mon[33384]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi092=y;smithi145=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:14.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:14 smithi145 ceph-mon[33384]: Saving service mgr spec with placement smithi092=y;smithi145=x;count:2 2024-09-11T08:35:14.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:14 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:14.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:14 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:14.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:14 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:14.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:14 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:14.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:14 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:14.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:14 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:14.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:14 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:14.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:14 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:35:14.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:14 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T08:35:14.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:14 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:35:14.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:14 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:14.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:14 smithi145 ceph-mon[33384]: Deploying daemon mgr.x on smithi145 2024-09-11T08:35:14.423 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T08:35:14.423 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:35:14.423 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 979 Links: 1 2024-09-11T08:35:14.423 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:35:14.423 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:35:14.423 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-11 08:35:00.634981795 +0000 2024-09-11T08:35:14.424 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-11 08:28:26.685310231 +0000 2024-09-11T08:35:14.424 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-11 08:28:26.685310231 +0000 2024-09-11T08:35:14.424 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-11 08:28:26.685310231 +0000 2024-09-11T08:35:14.424 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T08:35:14.488 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-11T08:35:14.489 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-11T08:35:14.489 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000232228 s, 2.2 MB/s 2024-09-11T08:35:14.490 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T08:35:14.546 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-09-11T08:35:14.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[29647]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[29647]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi092=y;smithi145=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[29647]: Saving service mgr spec with placement smithi092=y;smithi145=x;count:2 2024-09-11T08:35:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:35:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T08:35:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:35:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:14.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[29647]: Deploying daemon mgr.x on smithi145 2024-09-11T08:35:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[41114]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[41114]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi092=y;smithi145=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[41114]: Saving service mgr spec with placement smithi092=y;smithi145=x;count:2 2024-09-11T08:35:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:14.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:14.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:14.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:14.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:35:14.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T08:35:14.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:35:14.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:14.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:14 smithi092 ceph-mon[41114]: Deploying daemon mgr.x on smithi145 2024-09-11T08:35:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T08:35:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:35:14.583 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 986 Links: 1 2024-09-11T08:35:14.583 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:35:14.583 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:35:14.583 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-11 08:35:00.634981795 +0000 2024-09-11T08:35:14.583 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-11 08:28:26.687309449 +0000 2024-09-11T08:35:14.583 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-11 08:28:26.687309449 +0000 2024-09-11T08:35:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-11 08:28:26.687309449 +0000 2024-09-11T08:35:14.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T08:35:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-11T08:35:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-11T08:35:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000193416 s, 2.6 MB/s 2024-09-11T08:35:14.647 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T08:35:14.702 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-09-11T08:35:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T08:35:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:35:14.758 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-11T08:35:14.758 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:35:14.758 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:35:14.758 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-11 08:35:00.635981767 +0000 2024-09-11T08:35:14.759 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-11 08:28:26.685310231 +0000 2024-09-11T08:35:14.759 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-11 08:28:26.685310231 +0000 2024-09-11T08:35:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-11 08:28:26.685310231 +0000 2024-09-11T08:35:14.759 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T08:35:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-11T08:35:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-11T08:35:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000192889 s, 2.7 MB/s 2024-09-11T08:35:14.823 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T08:35:14.879 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-11T08:35:14.879 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:35:14.895 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T08:35:14.895 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_1 2024-09-11T08:35:14.951 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T08:35:14.951 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:35:14.951 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-09-11T08:35:14.951 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:35:14.951 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:35:14.951 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-11 08:34:52.482944451 +0000 2024-09-11T08:35:14.951 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-11 08:28:27.711861708 +0000 2024-09-11T08:35:14.951 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-11 08:28:27.711861708 +0000 2024-09-11T08:35:14.951 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-11 08:28:27.711861708 +0000 2024-09-11T08:35:14.951 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T08:35:15.015 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-11T08:35:15.016 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-11T08:35:15.016 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000271655 s, 1.9 MB/s 2024-09-11T08:35:15.017 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T08:35:15.073 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_2 2024-09-11T08:35:15.128 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T08:35:15.128 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:35:15.128 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-11T08:35:15.129 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:35:15.129 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:35:15.129 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-11 08:34:52.483944430 +0000 2024-09-11T08:35:15.129 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-11 08:28:27.710861729 +0000 2024-09-11T08:35:15.129 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-11 08:28:27.710861729 +0000 2024-09-11T08:35:15.129 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-11 08:28:27.710861729 +0000 2024-09-11T08:35:15.129 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T08:35:15.193 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-11T08:35:15.311 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-11T08:35:15.312 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000311667 s, 1.6 MB/s 2024-09-11T08:35:15.313 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T08:35:15.369 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_3 2024-09-11T08:35:15.425 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T08:35:15.425 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:35:15.426 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 984 Links: 1 2024-09-11T08:35:15.426 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:35:15.426 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:35:15.426 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-11 08:35:15.390467800 +0000 2024-09-11T08:35:15.426 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-11 08:28:27.711861708 +0000 2024-09-11T08:35:15.426 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-11 08:28:27.712861687 +0000 2024-09-11T08:35:15.426 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-11 08:28:27.711861708 +0000 2024-09-11T08:35:15.426 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T08:35:15.647 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-11T08:35:15.648 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-11T08:35:15.648 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000187164 s, 2.7 MB/s 2024-09-11T08:35:15.649 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T08:35:15.664 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_4 2024-09-11T08:35:15.722 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T08:35:15.723 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:35:15.723 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-11T08:35:15.723 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:35:15.723 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:35:15.723 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-11 08:35:15.605463330 +0000 2024-09-11T08:35:15.723 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-11 08:28:27.708861770 +0000 2024-09-11T08:35:15.723 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-11 08:28:27.708861770 +0000 2024-09-11T08:35:15.723 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-11 08:28:27.708861770 +0000 2024-09-11T08:35:15.723 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T08:35:15.789 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-11T08:35:15.789 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-11T08:35:15.789 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000209256 s, 2.4 MB/s 2024-09-11T08:35:15.791 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T08:35:15.847 INFO:tasks.cephadm:Deploying osd.0 on smithi092 with /dev/vg_nvme/lv_4... 2024-09-11T08:35:15.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T08:35:15.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:15 smithi145 podman[35339]: 2024-09-11 08:35:15.779937444 +0000 UTC m=+0.016393223 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:35:15.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:15 smithi145 podman[35339]: 2024-09-11 08:35:15.912306153 +0000 UTC m=+0.148761921 container create d360d7b619c6ebd588320c3d0c91085180219a2cca6e2ef83a656a655ee51894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:35:16.088 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:35:16.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:16 smithi145 podman[35339]: 2024-09-11 08:35:16.131553815 +0000 UTC m=+0.368009594 container init d360d7b619c6ebd588320c3d0c91085180219a2cca6e2ef83a656a655ee51894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-11T08:35:16.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:16 smithi145 podman[35339]: 2024-09-11 08:35:16.136537524 +0000 UTC m=+0.372993302 container start d360d7b619c6ebd588320c3d0c91085180219a2cca6e2ef83a656a655ee51894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T08:35:16.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:16 smithi145 ceph-mgr[35413]: -- 172.21.15.145:0/573694156 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557d7ab36340 con 0x557d7ab42800 2024-09-11T08:35:16.507 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:16 smithi145 ceph-mon[33384]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:16.507 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:16 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:16.507 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:16 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:16.507 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:16 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:16.507 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:16 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:16.508 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:16 smithi145 bash[35339]: d360d7b619c6ebd588320c3d0c91085180219a2cca6e2ef83a656a655ee51894 2024-09-11T08:35:16.508 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:16 smithi145 systemd[1]: Started Ceph mgr.x for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:35:16.508 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:16.249+0000 7f4fb1d9aec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:35:16.508 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:16.308+0000 7f4fb1d9aec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:35:16.508 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:16.376+0000 7f4fb1d9aec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:35:16.508 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:16.507+0000 7f4fb1d9aec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:35:16.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:16 smithi092 ceph-mon[29647]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:16.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:16 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:16.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:16 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:16.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:16 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:16.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:16 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:16 smithi092 ceph-mon[41114]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:16 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:16 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:16 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:16.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:16 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:16.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:16.666+0000 7f4fb1d9aec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:35:17.349 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:16.944+0000 7f4fb1d9aec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:35:17.350 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:17 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:17.009+0000 7f4fb1d9aec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:35:17.350 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:17 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:17.174+0000 7f4fb1d9aec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:35:17.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:17.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:17 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:17.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:17 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:18.015 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T08:35:18.096 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T08:35:18.109 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-09-11T08:35:18.110 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-09-11T08:35:18.110 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115303 s, 909 MB/s 2024-09-11T08:35:18.110 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-09-11T08:35:18.350 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:18.028+0000 7f4fb1d9aec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:35:18.351 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:18.092+0000 7f4fb1d9aec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:35:18.644 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[41114]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:18.644 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:18.644 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:18.644 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:18.645 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:18.645 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:18.645 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:18.645 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[41114]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T08:35:18.645 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:35:18.645 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:35:18.645 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:18.645 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[41114]: Reconfiguring daemon mgr.y on smithi092 2024-09-11T08:35:18.645 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[29647]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:18.645 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:18.646 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:18.646 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:18.646 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:18.646 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:18.646 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:18.646 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[29647]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T08:35:18.646 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:35:18.646 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:35:18.646 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:18.646 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:18 smithi092 ceph-mon[29647]: Reconfiguring daemon mgr.y on smithi092 2024-09-11T08:35:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-mon[33384]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:18.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-mon[33384]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T08:35:18.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:35:18.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:35:18.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:18.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-mon[33384]: Reconfiguring daemon mgr.y on smithi092 2024-09-11T08:35:19.098 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:18.807+0000 7f4fb1d9aec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:35:19.098 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:18 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:18.918+0000 7f4fb1d9aec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:35:19.098 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:19 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:19.038+0000 7f4fb1d9aec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:35:19.098 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:19 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:19.098+0000 7f4fb1d9aec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:35:19.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_4 2024-09-11T08:35:19.349 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:19 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:19.224+0000 7f4fb1d9aec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:35:19.349 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:19 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:19.291+0000 7f4fb1d9aec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:35:19.349 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:19 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:19.348+0000 7f4fb1d9aec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:35:19.611 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:19 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:19.482+0000 7f4fb1d9aec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:35:19.611 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:19 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:19.546+0000 7f4fb1d9aec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:35:19.612 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:19 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:19.611+0000 7f4fb1d9aec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:35:19.752 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:35:19.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:19 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:19.692+0000 7f4fb1d9aec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:35:19.912 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:19 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:19.750+0000 7f4fb1d9aec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:35:19.912 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:19 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:19.806+0000 7f4fb1d9aec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:35:20.912 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:35:20 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:35:20.551+0000 7f4fb1d9aec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:35:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:20 smithi145 ceph-mon[33384]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:20 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:20 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:20 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:20 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:20 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:20 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:20 smithi145 ceph-mon[33384]: Standby manager daemon x started 2024-09-11T08:35:20.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:20 smithi145 ceph-mon[33384]: from='mgr.? 172.21.15.145:0/1782802997' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:35:20.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:20 smithi145 ceph-mon[33384]: from='mgr.? 172.21.15.145:0/1782802997' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:35:20.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:20 smithi145 ceph-mon[33384]: from='mgr.? 172.21.15.145:0/1782802997' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:35:20.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:20 smithi145 ceph-mon[33384]: from='mgr.? 172.21.15.145:0/1782802997' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:35:20.938 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[41114]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:20.938 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:20.939 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:20.939 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:20.939 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:20.939 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:20.939 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:20.939 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[41114]: Standby manager daemon x started 2024-09-11T08:35:20.939 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[41114]: from='mgr.? 172.21.15.145:0/1782802997' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:35:20.939 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[41114]: from='mgr.? 172.21.15.145:0/1782802997' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:35:20.939 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[41114]: from='mgr.? 172.21.15.145:0/1782802997' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:35:20.939 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[41114]: from='mgr.? 172.21.15.145:0/1782802997' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:35:20.939 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[29647]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:20.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:20.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:20.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:20.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:20.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:20.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:20.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[29647]: Standby manager daemon x started 2024-09-11T08:35:20.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[29647]: from='mgr.? 172.21.15.145:0/1782802997' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:35:20.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[29647]: from='mgr.? 172.21.15.145:0/1782802997' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:35:20.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[29647]: from='mgr.? 172.21.15.145:0/1782802997' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:35:20.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:20 smithi092 ceph-mon[29647]: from='mgr.? 172.21.15.145:0/1782802997' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:35:22.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:22 smithi092 ceph-mon[41114]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-11T08:35:22.234 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:22 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:35:22.234 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:22 smithi092 ceph-mon[41114]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:22.234 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:22 smithi092 ceph-mon[41114]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:22.234 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:22 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:35:22.234 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:22 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:35:22.234 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:22 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:22.234 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:22 smithi092 ceph-mon[29647]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-11T08:35:22.234 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:22 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:35:22.235 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:22 smithi092 ceph-mon[29647]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:22.235 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:22 smithi092 ceph-mon[29647]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:22.235 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:22 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:35:22.235 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:22 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:35:22.235 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:22 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:22 smithi145 ceph-mon[33384]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-11T08:35:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:22 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:35:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:22 smithi145 ceph-mon[33384]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:22 smithi145 ceph-mon[33384]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:22 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:35:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:22 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:35:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:22 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:24.491 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:24 smithi092 ceph-mon[29647]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:24.491 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:24 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/331198306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d8a3811-e801-4b6a-9a63-83936283cb7d"}]: dispatch 2024-09-11T08:35:24.491 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:24 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/331198306' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d8a3811-e801-4b6a-9a63-83936283cb7d"}]': finished 2024-09-11T08:35:24.491 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:24 smithi092 ceph-mon[29647]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T08:35:24.491 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:24 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:35:24.491 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:24 smithi092 ceph-mon[41114]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:24.492 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:24 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/331198306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d8a3811-e801-4b6a-9a63-83936283cb7d"}]: dispatch 2024-09-11T08:35:24.492 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:24 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/331198306' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d8a3811-e801-4b6a-9a63-83936283cb7d"}]': finished 2024-09-11T08:35:24.492 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:24 smithi092 ceph-mon[41114]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T08:35:24.492 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:24 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:35:24.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:24 smithi145 ceph-mon[33384]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:24 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/331198306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d8a3811-e801-4b6a-9a63-83936283cb7d"}]: dispatch 2024-09-11T08:35:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:24 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/331198306' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d8a3811-e801-4b6a-9a63-83936283cb7d"}]': finished 2024-09-11T08:35:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:24 smithi145 ceph-mon[33384]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T08:35:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:24 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:35:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:25 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2990074919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:35:25.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:25 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2990074919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:35:25.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:25 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2990074919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:35:26.629 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:26 smithi092 ceph-mon[29647]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:26.629 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:26 smithi092 ceph-mon[41114]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:26.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:26 smithi145 ceph-mon[33384]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:28.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:28 smithi092 ceph-mon[29647]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:28.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:28 smithi092 ceph-mon[41114]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:28.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:28 smithi145 ceph-mon[33384]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:30.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:30 smithi145 ceph-mon[33384]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:30 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T08:35:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:30 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:30.728 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:30 smithi092 ceph-mon[29647]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:30.728 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:30 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T08:35:30.728 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:30 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:30.728 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:30 smithi092 ceph-mon[41114]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:30.729 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:30 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T08:35:30.729 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:30 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:31.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:31 smithi092 ceph-mon[41114]: Deploying daemon osd.0 on smithi092 2024-09-11T08:35:31.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:31 smithi092 ceph-mon[29647]: Deploying daemon osd.0 on smithi092 2024-09-11T08:35:31.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:31 smithi145 ceph-mon[33384]: Deploying daemon osd.0 on smithi092 2024-09-11T08:35:32.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:32 smithi145 ceph-mon[33384]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:32.729 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:32 smithi092 ceph-mon[29647]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:32.729 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:32 smithi092 ceph-mon[41114]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:34.503 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:34 smithi092 ceph-mon[29647]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:34.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:34 smithi092 ceph-mon[41114]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:34.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:34 smithi145 ceph-mon[33384]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:35.742 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:35.742 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:35.742 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:35.742 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:35.742 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:35.743 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:36 smithi092 ceph-mon[29647]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:36.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:36 smithi092 ceph-mon[41114]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:36.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:36 smithi145 ceph-mon[33384]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:38.573 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 0 on host 'smithi092' 2024-09-11T08:35:38.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[29647]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:38.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:38.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:38.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:38.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:38.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:38.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[29647]: from='osd.0 [v2:172.21.15.92:6802/2040613580,v1:172.21.15.92:6803/2040613580]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:35:38.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[29647]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:35:38.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:38.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:38.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:38.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[41114]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:38.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:38.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:38.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:38.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:38.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:38.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[41114]: from='osd.0 [v2:172.21.15.92:6802/2040613580,v1:172.21.15.92:6803/2040613580]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:35:38.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[41114]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:35:38.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:38.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:38.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:38 smithi145 ceph-mon[33384]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:38 smithi145 ceph-mon[33384]: from='osd.0 [v2:172.21.15.92:6802/2040613580,v1:172.21.15.92:6803/2040613580]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:35:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:38 smithi145 ceph-mon[33384]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:35:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:39.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:39.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:39.171 DEBUG:teuthology.orchestra.run.smithi092:osd.0> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T08:35:39.173 INFO:tasks.cephadm:Deploying osd.1 on smithi092 with /dev/vg_nvme/lv_3... 2024-09-11T08:35:39.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T08:35:39.949 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:35:40.064 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:39 smithi092 ceph-mon[29647]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T08:35:40.064 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:39 smithi092 ceph-mon[29647]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T08:35:40.064 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:39 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:35:40.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:39 smithi092 ceph-mon[29647]: from='osd.0 [v2:172.21.15.92:6802/2040613580,v1:172.21.15.92:6803/2040613580]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:35:40.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:39 smithi092 ceph-mon[29647]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:35:40.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:39 smithi092 ceph-mon[41114]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T08:35:40.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:39 smithi092 ceph-mon[41114]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T08:35:40.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:39 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:35:40.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:39 smithi092 ceph-mon[41114]: from='osd.0 [v2:172.21.15.92:6802/2040613580,v1:172.21.15.92:6803/2040613580]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:35:40.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:39 smithi092 ceph-mon[41114]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:35:40.065 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 08:35:39 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-0[45480]: 2024-09-11T08:35:39.786+0000 7ff515a6b640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T08:35:40.066 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 08:35:39 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-0[45480]: 2024-09-11T08:35:39.788+0000 7ff5108a2640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:35:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:39 smithi145 ceph-mon[33384]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T08:35:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:39 smithi145 ceph-mon[33384]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T08:35:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:39 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:35:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:39 smithi145 ceph-mon[33384]: from='osd.0 [v2:172.21.15.92:6802/2040613580,v1:172.21.15.92:6803/2040613580]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:35:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:39 smithi145 ceph-mon[33384]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:35:40.977 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:40 smithi092 ceph-mon[41114]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:40.977 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:40 smithi092 ceph-mon[41114]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T08:35:40.977 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:40 smithi092 ceph-mon[41114]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T08:35:40.977 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:40 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:35:40.977 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:40 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:35:40.978 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:40 smithi092 ceph-mon[29647]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:40.978 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:40 smithi092 ceph-mon[29647]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T08:35:40.978 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:40 smithi092 ceph-mon[29647]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T08:35:40.978 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:40 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:35:40.978 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:40 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:35:41.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:40 smithi145 ceph-mon[33384]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:41.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:40 smithi145 ceph-mon[33384]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T08:35:41.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:40 smithi145 ceph-mon[33384]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T08:35:41.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:40 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:35:41.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:40 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:35:41.957 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:41 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:35:41.957 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:41 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:35:41.957 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:41 smithi092 ceph-mon[29647]: osd.0 [v2:172.21.15.92:6802/2040613580,v1:172.21.15.92:6803/2040613580] boot 2024-09-11T08:35:41.957 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:41 smithi092 ceph-mon[29647]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T08:35:41.957 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:41 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:35:41.958 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:41 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:35:41.958 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:41 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:35:41.958 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:41 smithi092 ceph-mon[41114]: osd.0 [v2:172.21.15.92:6802/2040613580,v1:172.21.15.92:6803/2040613580] boot 2024-09-11T08:35:41.958 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:41 smithi092 ceph-mon[41114]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T08:35:41.958 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:41 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:35:42.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:41 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:35:42.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:41 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:35:42.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:41 smithi145 ceph-mon[33384]: osd.0 [v2:172.21.15.92:6802/2040613580,v1:172.21.15.92:6803/2040613580] boot 2024-09-11T08:35:42.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:41 smithi145 ceph-mon[33384]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T08:35:42.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:41 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:35:43.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:42 smithi092 ceph-mon[29647]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:43.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:42 smithi092 ceph-mon[29647]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T08:35:43.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:42 smithi092 ceph-mon[41114]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:43.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:42 smithi092 ceph-mon[41114]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T08:35:43.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:42 smithi145 ceph-mon[33384]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:35:43.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:42 smithi145 ceph-mon[33384]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T08:35:43.217 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T08:35:43.300 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T08:35:43.314 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-09-11T08:35:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-09-11T08:35:43.314 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119844 s, 875 MB/s 2024-09-11T08:35:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-09-11T08:35:43.880 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[29647]: Detected new or changed devices on smithi092 2024-09-11T08:35:43.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:43.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:43.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:43.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[29647]: Adjusting osd_memory_target on smithi092 to 16064M 2024-09-11T08:35:43.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:43.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:43.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:43.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:43.881 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[41114]: Detected new or changed devices on smithi092 2024-09-11T08:35:43.881 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:43.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:43.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:43.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[41114]: Adjusting osd_memory_target on smithi092 to 16064M 2024-09-11T08:35:43.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:43.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:43.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:43.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:43 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:43.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_3 2024-09-11T08:35:44.124 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:35:44.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:43 smithi145 ceph-mon[33384]: Detected new or changed devices on smithi092 2024-09-11T08:35:44.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:43 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:44.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:43 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:44.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:43 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:44.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:43 smithi145 ceph-mon[33384]: Adjusting osd_memory_target on smithi092 to 16064M 2024-09-11T08:35:44.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:43 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:44.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:43 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:44.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:43 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:44.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:43 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:35:45.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:44 smithi145 ceph-mon[33384]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:45.248 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:44 smithi092 ceph-mon[29647]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:45.248 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:44 smithi092 ceph-mon[41114]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:47.014 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:46 smithi092 ceph-mon[29647]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:47.014 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:46 smithi092 ceph-mon[29647]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:47.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:46 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:35:47.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:46 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:35:47.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:46 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:47.015 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:46 smithi092 ceph-mon[41114]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:47.015 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:46 smithi092 ceph-mon[41114]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:47.015 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:46 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:35:47.015 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:46 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:35:47.015 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:46 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:47.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:46 smithi145 ceph-mon[33384]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:47.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:46 smithi145 ceph-mon[33384]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:47.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:46 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:35:47.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:46 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:35:47.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:46 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:49.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:48 smithi092 ceph-mon[29647]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:49.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:48 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1374687843' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7a18e45-0137-45ef-8b85-e458cf3737ac"}]: dispatch 2024-09-11T08:35:49.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:48 smithi092 ceph-mon[29647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7a18e45-0137-45ef-8b85-e458cf3737ac"}]: dispatch 2024-09-11T08:35:49.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:48 smithi092 ceph-mon[29647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7a18e45-0137-45ef-8b85-e458cf3737ac"}]': finished 2024-09-11T08:35:49.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:48 smithi092 ceph-mon[29647]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T08:35:49.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:48 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:49.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:48 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1099162568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:35:49.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:48 smithi092 ceph-mon[41114]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:49.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:48 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1374687843' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7a18e45-0137-45ef-8b85-e458cf3737ac"}]: dispatch 2024-09-11T08:35:49.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:48 smithi092 ceph-mon[41114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7a18e45-0137-45ef-8b85-e458cf3737ac"}]: dispatch 2024-09-11T08:35:49.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:48 smithi092 ceph-mon[41114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7a18e45-0137-45ef-8b85-e458cf3737ac"}]': finished 2024-09-11T08:35:49.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:48 smithi092 ceph-mon[41114]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T08:35:49.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:48 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:49.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:48 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1099162568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:35:49.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:48 smithi145 ceph-mon[33384]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:49.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:48 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1374687843' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7a18e45-0137-45ef-8b85-e458cf3737ac"}]: dispatch 2024-09-11T08:35:49.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:48 smithi145 ceph-mon[33384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7a18e45-0137-45ef-8b85-e458cf3737ac"}]: dispatch 2024-09-11T08:35:49.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:48 smithi145 ceph-mon[33384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7a18e45-0137-45ef-8b85-e458cf3737ac"}]': finished 2024-09-11T08:35:49.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:48 smithi145 ceph-mon[33384]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T08:35:49.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:48 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:49.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:48 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1099162568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:35:51.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:50 smithi145 ceph-mon[33384]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:51.241 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:50 smithi092 ceph-mon[29647]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:51.241 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:50 smithi092 ceph-mon[41114]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:52.998 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:52 smithi092 ceph-mon[29647]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:52.998 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:52 smithi092 ceph-mon[41114]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:53.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:52 smithi145 ceph-mon[33384]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:54.957 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:54 smithi092 ceph-mon[29647]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:54.958 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:54 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T08:35:54.958 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:54 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:54.958 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:54 smithi092 ceph-mon[41114]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:54.958 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:54 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T08:35:54.958 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:54 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:55.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:54 smithi145 ceph-mon[33384]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:55.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:54 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T08:35:55.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:54 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:56.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:55 smithi092 ceph-mon[29647]: Deploying daemon osd.1 on smithi092 2024-09-11T08:35:56.015 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:55 smithi092 ceph-mon[41114]: Deploying daemon osd.1 on smithi092 2024-09-11T08:35:56.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:55 smithi145 ceph-mon[33384]: Deploying daemon osd.1 on smithi092 2024-09-11T08:35:57.027 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:56 smithi092 ceph-mon[29647]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:57.027 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:56 smithi092 ceph-mon[41114]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:57.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:56 smithi145 ceph-mon[33384]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:59.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:58 smithi145 ceph-mon[33384]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:59.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:58 smithi092 ceph-mon[29647]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:59.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:58 smithi092 ceph-mon[41114]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:36:00.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:59 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:00.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:59 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:00.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:35:59 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:00.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:59 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:00.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:59 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:00.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:35:59 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:00.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:59 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:00.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:59 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:00.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:35:59 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:01.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:00 smithi092 ceph-mon[29647]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:36:01.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:00 smithi092 ceph-mon[41114]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:36:01.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:00 smithi145 ceph-mon[33384]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:36:02.388 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 1 on host 'smithi092' 2024-09-11T08:36:03.123 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:02 smithi092 ceph-mon[29647]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:36:03.124 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:02 smithi092 ceph-mon[29647]: from='osd.1 [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:36:03.124 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:02 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:03.124 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:02 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:03.124 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:02 smithi092 ceph-mon[41114]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:36:03.124 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:02 smithi092 ceph-mon[41114]: from='osd.1 [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:36:03.124 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:02 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:03.124 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:02 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:03.153 DEBUG:teuthology.orchestra.run.smithi092:osd.1> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T08:36:03.155 INFO:tasks.cephadm:Deploying osd.2 on smithi092 with /dev/vg_nvme/lv_2... 2024-09-11T08:36:03.155 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T08:36:03.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:02 smithi145 ceph-mon[33384]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:36:03.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:02 smithi145 ceph-mon[33384]: from='osd.1 [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:36:03.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:02 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:03.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:02 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:03.457 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:36:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:03 smithi145 ceph-mon[33384]: from='osd.1 [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T08:36:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:03 smithi145 ceph-mon[33384]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T08:36:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:03 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:36:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:03 smithi145 ceph-mon[33384]: from='osd.1 [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:36:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:03 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:03 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:03 smithi145 ceph-mon[33384]: from='osd.1 [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T08:36:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:03 smithi145 ceph-mon[33384]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T08:36:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:03 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:36:04.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[41114]: from='osd.1 [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T08:36:04.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[41114]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T08:36:04.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:36:04.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[41114]: from='osd.1 [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:36:04.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:04.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:04.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[41114]: from='osd.1 [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T08:36:04.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[41114]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T08:36:04.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:36:04.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[29647]: from='osd.1 [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T08:36:04.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[29647]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T08:36:04.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:36:04.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[29647]: from='osd.1 [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:36:04.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:04.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:04.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[29647]: from='osd.1 [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T08:36:04.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[29647]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T08:36:04.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:36:04.318 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-1[51569]: 2024-09-11T08:36:03.888+0000 7f3db7b08640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T08:36:04.318 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 08:36:03 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-1[51569]: 2024-09-11T08:36:03.892+0000 7f3db4943640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:36:05.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:04 smithi145 ceph-mon[33384]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:36:05.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:04 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:36:05.169 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:04 smithi092 ceph-mon[29647]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:36:05.169 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:04 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:36:05.169 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:04 smithi092 ceph-mon[41114]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:36:05.169 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:04 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:36:05.537 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T08:36:05.615 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T08:36:05.630 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-09-11T08:36:05.630 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-09-11T08:36:05.630 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126455 s, 829 MB/s 2024-09-11T08:36:05.630 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-09-11T08:36:06.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:05 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:36:06.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:05 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:36:06.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:05 smithi145 ceph-mon[33384]: osd.1 [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328] boot 2024-09-11T08:36:06.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:05 smithi145 ceph-mon[33384]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T08:36:06.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:05 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:36:06.208 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:05 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:36:06.208 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:05 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:36:06.208 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:05 smithi092 ceph-mon[29647]: osd.1 [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328] boot 2024-09-11T08:36:06.208 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:05 smithi092 ceph-mon[29647]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T08:36:06.208 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:05 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:36:06.208 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:05 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:36:06.208 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:05 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:36:06.208 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:05 smithi092 ceph-mon[41114]: osd.1 [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328] boot 2024-09-11T08:36:06.209 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:05 smithi092 ceph-mon[41114]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T08:36:06.209 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:05 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:36:07.002 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:06 smithi092 ceph-mon[29647]: pgmap v57: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:36:07.003 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:06 smithi092 ceph-mon[29647]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T08:36:07.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:06 smithi092 ceph-mon[41114]: pgmap v57: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:36:07.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:06 smithi092 ceph-mon[41114]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T08:36:07.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:06 smithi145 ceph-mon[33384]: pgmap v57: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:36:07.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:06 smithi145 ceph-mon[33384]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T08:36:07.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_2 2024-09-11T08:36:08.164 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:36:08.999 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[29647]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:08.999 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:08.999 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:09.000 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:09.000 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:09.000 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:09.000 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:09.000 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:09.000 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:09.000 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:09.000 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:09.000 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:09.000 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:09.000 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[41114]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:09.001 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:09.001 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:09.001 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:09.001 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:09.001 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:09.001 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:09.001 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:09.001 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:09.001 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:09.001 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:09.001 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:09.001 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:08 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:08 smithi145 ceph-mon[33384]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:09.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:09.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:09.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:09.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:08 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:10.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:09 smithi145 ceph-mon[33384]: Detected new or changed devices on smithi092 2024-09-11T08:36:10.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:09 smithi145 ceph-mon[33384]: Adjusting osd_memory_target on smithi092 to 8032M 2024-09-11T08:36:10.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:09 smithi092 ceph-mon[29647]: Detected new or changed devices on smithi092 2024-09-11T08:36:10.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:09 smithi092 ceph-mon[29647]: Adjusting osd_memory_target on smithi092 to 8032M 2024-09-11T08:36:10.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:09 smithi092 ceph-mon[41114]: Detected new or changed devices on smithi092 2024-09-11T08:36:10.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:09 smithi092 ceph-mon[41114]: Adjusting osd_memory_target on smithi092 to 8032M 2024-09-11T08:36:11.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:10 smithi092 ceph-mon[41114]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:11.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:10 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:11.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:10 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:11.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:10 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:11.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:10 smithi092 ceph-mon[29647]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:11.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:10 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:11.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:10 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:11.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:10 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:11.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:10 smithi145 ceph-mon[33384]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:11.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:10 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:11.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:10 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:11.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:10 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:12.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:11 smithi092 ceph-mon[29647]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:12.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:11 smithi092 ceph-mon[41114]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:12.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:11 smithi145 ceph-mon[33384]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:13.208 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:12 smithi092 ceph-mon[41114]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:13.208 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:12 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/937821413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "017282e2-8053-4fa7-affc-a05408fa481e"}]: dispatch 2024-09-11T08:36:13.209 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:12 smithi092 ceph-mon[41114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "017282e2-8053-4fa7-affc-a05408fa481e"}]: dispatch 2024-09-11T08:36:13.209 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:12 smithi092 ceph-mon[41114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "017282e2-8053-4fa7-affc-a05408fa481e"}]': finished 2024-09-11T08:36:13.209 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:12 smithi092 ceph-mon[41114]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T08:36:13.209 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:13.209 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:12 smithi092 ceph-mon[29647]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:13.209 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:12 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/937821413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "017282e2-8053-4fa7-affc-a05408fa481e"}]: dispatch 2024-09-11T08:36:13.209 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:12 smithi092 ceph-mon[29647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "017282e2-8053-4fa7-affc-a05408fa481e"}]: dispatch 2024-09-11T08:36:13.209 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:12 smithi092 ceph-mon[29647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "017282e2-8053-4fa7-affc-a05408fa481e"}]': finished 2024-09-11T08:36:13.209 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:12 smithi092 ceph-mon[29647]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T08:36:13.210 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:13.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:12 smithi145 ceph-mon[33384]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:13.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:12 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/937821413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "017282e2-8053-4fa7-affc-a05408fa481e"}]: dispatch 2024-09-11T08:36:13.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:12 smithi145 ceph-mon[33384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "017282e2-8053-4fa7-affc-a05408fa481e"}]: dispatch 2024-09-11T08:36:13.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:12 smithi145 ceph-mon[33384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "017282e2-8053-4fa7-affc-a05408fa481e"}]': finished 2024-09-11T08:36:13.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:12 smithi145 ceph-mon[33384]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T08:36:13.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:14.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:13 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3879519175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:36:14.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:13 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3879519175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:36:14.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:13 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3879519175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:36:15.181 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:14 smithi092 ceph-mon[29647]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:15.181 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:14 smithi092 ceph-mon[41114]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:15.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:14 smithi145 ceph-mon[33384]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:17.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:16 smithi092 ceph-mon[29647]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:17.274 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:16 smithi092 ceph-mon[41114]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:17.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:16 smithi145 ceph-mon[33384]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:19.183 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:18 smithi092 ceph-mon[29647]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:19.184 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:18 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T08:36:19.184 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:18 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:19.184 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:18 smithi092 ceph-mon[41114]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:19.184 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:18 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T08:36:19.184 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:18 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:19.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:18 smithi145 ceph-mon[33384]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:19.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:18 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T08:36:19.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:18 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:20.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:19 smithi092 ceph-mon[29647]: Deploying daemon osd.2 on smithi092 2024-09-11T08:36:20.271 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:19 smithi092 ceph-mon[41114]: Deploying daemon osd.2 on smithi092 2024-09-11T08:36:20.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:19 smithi145 ceph-mon[33384]: Deploying daemon osd.2 on smithi092 2024-09-11T08:36:21.127 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:20 smithi092 ceph-mon[41114]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:21.127 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:20 smithi092 ceph-mon[29647]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:21.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:20 smithi145 ceph-mon[33384]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:23.263 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:22 smithi092 ceph-mon[29647]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:23.263 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:22 smithi092 ceph-mon[41114]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:23.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:22 smithi145 ceph-mon[33384]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:24.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:23 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:24.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:23 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:24.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:23 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:24.292 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:23 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:24.292 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:23 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:24.293 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:23 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:24.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:23 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:24.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:23 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:24.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:23 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:25.036 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:24 smithi092 ceph-mon[29647]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:25.037 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:24 smithi092 ceph-mon[41114]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:25.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:24 smithi145 ceph-mon[33384]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:26.923 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 2 on host 'smithi092' 2024-09-11T08:36:27.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:26 smithi092 ceph-mon[29647]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:27.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:26 smithi092 ceph-mon[29647]: from='osd.2 [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:36:27.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:26 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:27.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:26 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:27.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:26 smithi092 ceph-mon[41114]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:27.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:26 smithi092 ceph-mon[41114]: from='osd.2 [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:36:27.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:26 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:27.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:26 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:27.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:26 smithi145 ceph-mon[33384]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:27.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:26 smithi145 ceph-mon[33384]: from='osd.2 [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:36:27.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:26 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:27.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:26 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:27.711 DEBUG:teuthology.orchestra.run.smithi092:osd.2> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T08:36:27.714 INFO:tasks.cephadm:Deploying osd.3 on smithi092 with /dev/vg_nvme/lv_1... 2024-09-11T08:36:27.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T08:36:28.025 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:36:28.256 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[29647]: from='osd.2 [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T08:36:28.256 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[29647]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T08:36:28.257 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:28.257 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[29647]: from='osd.2 [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:36:28.257 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:28.257 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:28.257 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[29647]: from='osd.2 [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T08:36:28.257 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[29647]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T08:36:28.257 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:28.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[41114]: from='osd.2 [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T08:36:28.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[41114]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T08:36:28.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:28.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[41114]: from='osd.2 [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:36:28.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:28.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:28.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[41114]: from='osd.2 [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T08:36:28.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[41114]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T08:36:28.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:28.258 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-2[57505]: 2024-09-11T08:36:27.961+0000 7f2b826a1640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T08:36:28.258 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 08:36:27 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-2[57505]: 2024-09-11T08:36:27.971+0000 7f2b7b4d4640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:36:28.313 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:27 smithi145 ceph-mon[33384]: from='osd.2 [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T08:36:28.313 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:27 smithi145 ceph-mon[33384]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T08:36:28.313 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:27 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:28.313 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:27 smithi145 ceph-mon[33384]: from='osd.2 [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:36:28.313 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:27 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:28.313 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:27 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:28.313 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:27 smithi145 ceph-mon[33384]: from='osd.2 [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T08:36:28.313 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:27 smithi145 ceph-mon[33384]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T08:36:28.313 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:27 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:29.167 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:28 smithi092 ceph-mon[41114]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:29.167 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:28 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:29.167 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:28 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:29.167 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:28 smithi092 ceph-mon[29647]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:29.167 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:28 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:29.167 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:28 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:29.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:28 smithi145 ceph-mon[33384]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:28 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:28 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:30.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:29 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:36:30.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:29 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:36:30.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:29 smithi092 ceph-mon[41114]: osd.2 [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777] boot 2024-09-11T08:36:30.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:29 smithi092 ceph-mon[41114]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T08:36:30.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:29 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:30.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:29 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T08:36:30.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:29 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:36:30.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:29 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:36:30.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:29 smithi092 ceph-mon[29647]: osd.2 [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777] boot 2024-09-11T08:36:30.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:29 smithi092 ceph-mon[29647]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T08:36:30.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:29 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:30.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:29 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T08:36:30.367 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T08:36:30.367 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T08:36:30.367 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-09-11T08:36:30.367 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-09-11T08:36:30.367 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125853 s, 833 MB/s 2024-09-11T08:36:30.367 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-09-11T08:36:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:29 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:36:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:29 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:36:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:29 smithi145 ceph-mon[33384]: osd.2 [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777] boot 2024-09-11T08:36:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:29 smithi145 ceph-mon[33384]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T08:36:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:29 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:36:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:29 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T08:36:30.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_1 2024-09-11T08:36:31.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:30 smithi092 ceph-mon[41114]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:31.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:30 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T08:36:31.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:30 smithi092 ceph-mon[41114]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T08:36:31.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:30 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:31.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:30 smithi092 ceph-mon[29647]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:31.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:30 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T08:36:31.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:30 smithi092 ceph-mon[29647]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T08:36:31.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:30 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:31.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:30 smithi145 ceph-mon[33384]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:36:31.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:30 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T08:36:31.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:30 smithi145 ceph-mon[33384]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T08:36:31.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:30 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:36:31.850 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:36:31.971 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 08:36:31 smithi092 sudo[59254]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:36:31.972 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 08:36:31 smithi092 sudo[59254]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:36:31.972 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 08:36:31 smithi092 sudo[59254]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:36:31.972 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 08:36:31 smithi092 sudo[59254]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:36:32.280 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:31 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:32.280 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:31 smithi092 ceph-mon[29647]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T08:36:32.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:31 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:32.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:31 smithi092 ceph-mon[41114]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T08:36:32.281 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[59291]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:36:32.281 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[59291]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:36:32.281 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[59291]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:36:32.281 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[59291]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:36:32.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:31 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:36:32.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:31 smithi145 ceph-mon[33384]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T08:36:32.565 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[61428]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:36:32.565 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[61428]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:36:32.566 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[61428]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:36:32.566 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[61428]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:36:32.868 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[61443]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:36:32.868 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[61443]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:36:32.868 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[61443]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:36:32.868 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[61443]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:36:33.127 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:32 smithi092 ceph-mon[29647]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:33.127 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:32 smithi092 ceph-mon[29647]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T08:36:33.127 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:32 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:33.127 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:32 smithi092 ceph-mon[41114]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:33.128 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:32 smithi092 ceph-mon[41114]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T08:36:33.128 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:32 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:33.128 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[61452]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:36:33.128 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[61452]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:36:33.128 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[61452]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:36:33.128 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 08:36:32 smithi092 sudo[61452]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:36:33.389 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:33 smithi092 sudo[61469]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:36:33.390 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:33 smithi092 sudo[61469]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:36:33.390 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:33 smithi092 sudo[61469]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:36:33.390 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:33 smithi092 sudo[61469]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:36:33.390 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 08:36:33 smithi092 sudo[61457]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:36:33.390 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 08:36:33 smithi092 sudo[61457]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:36:33.390 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 08:36:33 smithi092 sudo[61457]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:36:33.390 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 08:36:33 smithi092 sudo[61457]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:36:33.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:32 smithi145 ceph-mon[33384]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:33.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:32 smithi145 ceph-mon[33384]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T08:36:33.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:32 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:33.754 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:33 smithi092 sudo[61472]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:36:33.755 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:33 smithi092 sudo[61472]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:36:33.755 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:33 smithi092 sudo[61472]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:36:33.755 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:33 smithi092 sudo[61472]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:36:34.133 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:33 smithi145 sudo[35997]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:36:34.133 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:33 smithi145 sudo[35997]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:36:34.133 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:33 smithi145 sudo[35997]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:36:34.133 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:33 smithi145 sudo[35997]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:36:34.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[29647]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:36:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[29647]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:36:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:36:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:36:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:36:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:36:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:36:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:36:34.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[41114]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:36:34.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[41114]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:36:34.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:36:34.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:36:34.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:36:34.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:34.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:36:34.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:36:34.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:36:34.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:34 smithi145 sudo[36000]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:36:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:34 smithi145 sudo[36000]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:36:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:34 smithi145 sudo[36000]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:36:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:34 smithi145 sudo[36000]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:36:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:34 smithi145 ceph-mon[33384]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:36:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:34 smithi145 ceph-mon[33384]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:36:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:36:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:36:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:36:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:36:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:36:34.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:36:34.688 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 sudo[61580]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:36:34.689 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 sudo[61580]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:36:34.689 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 sudo[61580]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:36:34.689 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 sudo[61580]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:36:34.962 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 sudo[61683]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:36:34.962 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 sudo[61683]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:36:34.962 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 sudo[61683]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:36:34.962 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:34 smithi092 sudo[61683]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:36:35.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: Detected new or changed devices on smithi092 2024-09-11T08:36:35.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:35.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:36:35.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:36:35.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:35.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:35.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:35.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T08:36:35.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:35.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: Adjusting osd_memory_target on smithi092 to 5354M 2024-09-11T08:36:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:36:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:36:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:36:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:36:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:35.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:35.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:35.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:36:35.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:35.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: Detected new or changed devices on smithi092 2024-09-11T08:36:35.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:35.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:36:35.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:36:35.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:35.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:35.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:35.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T08:36:35.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:35.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: Adjusting osd_memory_target on smithi092 to 5354M 2024-09-11T08:36:35.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:35.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:35.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:35.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:35.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:36:35.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:35.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:36:35.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:36:35.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:36:35.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:35.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:35.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:35.320 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:35.320 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:35.320 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:35.320 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:35.320 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:36:35.320 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: Detected new or changed devices on smithi092 2024-09-11T08:36:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:36:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:36:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T08:36:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: Adjusting osd_memory_target on smithi092 to 5354M 2024-09-11T08:36:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:36:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:36:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:36:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:36:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:35.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:35.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:35.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:35.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:35.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:35.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:36:35.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:36.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:36 smithi092 ceph-mon[29647]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:36.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:36 smithi092 ceph-mon[41114]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:36.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:36 smithi145 ceph-mon[33384]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:37.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:37 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2033175430' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58e5a8a4-9e58-40ad-9b40-90cb5ff5414b"}]: dispatch 2024-09-11T08:36:37.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:37 smithi145 ceph-mon[33384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58e5a8a4-9e58-40ad-9b40-90cb5ff5414b"}]: dispatch 2024-09-11T08:36:37.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:37 smithi145 ceph-mon[33384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58e5a8a4-9e58-40ad-9b40-90cb5ff5414b"}]': finished 2024-09-11T08:36:37.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:37 smithi145 ceph-mon[33384]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T08:36:37.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:37 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:37.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:37 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2033175430' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58e5a8a4-9e58-40ad-9b40-90cb5ff5414b"}]: dispatch 2024-09-11T08:36:37.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:37 smithi092 ceph-mon[29647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58e5a8a4-9e58-40ad-9b40-90cb5ff5414b"}]: dispatch 2024-09-11T08:36:37.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:37 smithi092 ceph-mon[29647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58e5a8a4-9e58-40ad-9b40-90cb5ff5414b"}]': finished 2024-09-11T08:36:37.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:37 smithi092 ceph-mon[29647]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T08:36:37.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:37 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:37.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:37 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2033175430' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58e5a8a4-9e58-40ad-9b40-90cb5ff5414b"}]: dispatch 2024-09-11T08:36:37.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:37 smithi092 ceph-mon[41114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58e5a8a4-9e58-40ad-9b40-90cb5ff5414b"}]: dispatch 2024-09-11T08:36:37.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:37 smithi092 ceph-mon[41114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58e5a8a4-9e58-40ad-9b40-90cb5ff5414b"}]': finished 2024-09-11T08:36:37.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:37 smithi092 ceph-mon[41114]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T08:36:37.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:37 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:38.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:38 smithi145 ceph-mon[33384]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:38.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:38 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3793550138' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:36:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:38 smithi092 ceph-mon[29647]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:38 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3793550138' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:36:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:38 smithi092 ceph-mon[41114]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:38 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3793550138' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:36:40.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:40 smithi145 ceph-mon[33384]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:40.989 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:40 smithi092 ceph-mon[29647]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:40.990 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:40 smithi092 ceph-mon[41114]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:42.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:42 smithi145 ceph-mon[33384]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:42.988 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:42 smithi092 ceph-mon[41114]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:42.988 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:42 smithi092 ceph-mon[29647]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:43.849 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:43 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T08:36:43.849 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:43 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:43.850 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:43 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T08:36:43.850 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:43 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:43.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:43 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T08:36:43.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:43 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:44.789 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:44 smithi092 ceph-mon[29647]: Deploying daemon osd.3 on smithi092 2024-09-11T08:36:44.790 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:44 smithi092 ceph-mon[29647]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:44.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:44 smithi092 ceph-mon[41114]: Deploying daemon osd.3 on smithi092 2024-09-11T08:36:44.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:44 smithi092 ceph-mon[41114]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:44.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:44 smithi145 ceph-mon[33384]: Deploying daemon osd.3 on smithi092 2024-09-11T08:36:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:44 smithi145 ceph-mon[33384]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:46.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:46 smithi092 ceph-mon[29647]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:46 smithi092 ceph-mon[41114]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:47.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:46 smithi145 ceph-mon[33384]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:48.783 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:48 smithi092 ceph-mon[29647]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:48.783 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:48 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:48.784 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:48 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:48.784 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:48 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:48.784 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:48 smithi092 ceph-mon[41114]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:48.784 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:48 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:48.784 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:48 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:48.784 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:48 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:49.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:48 smithi145 ceph-mon[33384]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:49.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:48 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:49.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:48 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:49.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:48 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:50.915 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:50 smithi092 ceph-mon[29647]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:50.915 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:50 smithi092 ceph-mon[41114]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:51.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:50 smithi145 ceph-mon[33384]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:51.224 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 3 on host 'smithi092' 2024-09-11T08:36:51.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:36:51.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:36:51.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[29647]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T08:36:51.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:51.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[29647]: from='osd.3 [v2:172.21.15.92:6826/878892459,v1:172.21.15.92:6827/878892459]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:36:51.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[29647]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:36:51.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:51.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:51.811 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:36:51.811 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:36:51.811 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[41114]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T08:36:51.811 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:51.811 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[41114]: from='osd.3 [v2:172.21.15.92:6826/878892459,v1:172.21.15.92:6827/878892459]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:36:51.811 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[41114]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:36:51.812 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:51.812 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:51 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:51.834 DEBUG:teuthology.orchestra.run.smithi092:osd.3> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T08:36:51.836 INFO:tasks.cephadm:Deploying osd.4 on smithi145 with /dev/vg_nvme/lv_4... 2024-09-11T08:36:51.836 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T08:36:52.084 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:36:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:51 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:36:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:51 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:36:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:51 smithi145 ceph-mon[33384]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T08:36:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:51 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:51 smithi145 ceph-mon[33384]: from='osd.3 [v2:172.21.15.92:6826/878892459,v1:172.21.15.92:6827/878892459]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:36:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:51 smithi145 ceph-mon[33384]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:36:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:51 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:51 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:52.789 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[29647]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:52.789 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[29647]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T08:36:52.789 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[29647]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T08:36:52.789 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[29647]: from='osd.3 [v2:172.21.15.92:6826/878892459,v1:172.21.15.92:6827/878892459]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:36:52.789 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:52.789 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[29647]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:36:52.789 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:52.789 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:52.789 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[29647]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T08:36:52.790 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[29647]: osdmap e25: 4 total, 3 up, 4 in 2024-09-11T08:36:52.790 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:52.790 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:52.790 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3[63650]: 2024-09-11T08:36:52.683+0000 7f087b4e3640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T08:36:52.790 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3[63650]: 2024-09-11T08:36:52.688+0000 7f087831e640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:36:52.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[41114]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:52.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[41114]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T08:36:52.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[41114]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T08:36:52.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[41114]: from='osd.3 [v2:172.21.15.92:6826/878892459,v1:172.21.15.92:6827/878892459]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:36:52.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:52.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[41114]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:36:52.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:52.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:52.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[41114]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T08:36:52.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[41114]: osdmap e25: 4 total, 3 up, 4 in 2024-09-11T08:36:52.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:52.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:52 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:52.796 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:52 smithi145 ceph-mon[33384]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:52.796 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:52 smithi145 ceph-mon[33384]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T08:36:52.796 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:52 smithi145 ceph-mon[33384]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T08:36:52.796 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:52 smithi145 ceph-mon[33384]: from='osd.3 [v2:172.21.15.92:6826/878892459,v1:172.21.15.92:6827/878892459]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:36:52.796 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:52 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:52.796 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:52 smithi145 ceph-mon[33384]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-11T08:36:52.796 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:52 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:52.796 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:52 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:52.796 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:52 smithi145 ceph-mon[33384]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-11T08:36:52.797 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:52 smithi145 ceph-mon[33384]: osdmap e25: 4 total, 3 up, 4 in 2024-09-11T08:36:52.797 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:52 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:52.797 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:52 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:53.670 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T08:36:53.759 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T08:36:53.773 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2024-09-11T08:36:53.773 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2024-09-11T08:36:53.773 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112199 s, 935 MB/s 2024-09-11T08:36:53.773 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2024-09-11T08:36:53.812 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:53 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:53.812 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:53 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:54.069 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:53 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:54.385 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_4 2024-09-11T08:36:54.622 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:36:54.972 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:54 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:36:54.973 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:54 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:36:54.973 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:54 smithi145 ceph-mon[33384]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:54.973 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:54 smithi145 ceph-mon[33384]: osd.3 [v2:172.21.15.92:6826/878892459,v1:172.21.15.92:6827/878892459] boot 2024-09-11T08:36:54.973 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:54 smithi145 ceph-mon[33384]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T08:36:54.973 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:54 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:55.049 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:54 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:36:55.049 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:54 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:36:55.049 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:54 smithi092 ceph-mon[29647]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:55.049 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:54 smithi092 ceph-mon[29647]: osd.3 [v2:172.21.15.92:6826/878892459,v1:172.21.15.92:6827/878892459] boot 2024-09-11T08:36:55.049 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:54 smithi092 ceph-mon[29647]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T08:36:55.049 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:54 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:55.050 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:54 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:36:55.050 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:54 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:36:55.050 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:54 smithi092 ceph-mon[41114]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:55.050 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:54 smithi092 ceph-mon[41114]: osd.3 [v2:172.21.15.92:6826/878892459,v1:172.21.15.92:6827/878892459] boot 2024-09-11T08:36:55.050 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:54 smithi092 ceph-mon[41114]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T08:36:55.050 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:54 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:56.004 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:55 smithi145 ceph-mon[33384]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T08:36:56.004 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:55 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:56.004 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:55 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:56.004 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:55 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:56.004 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:55 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:56.004 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:55 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:56.004 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:55 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:56.004 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:55 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:56.004 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:55 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:56.004 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:55 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:56.005 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:55 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:56.005 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:55 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:56.005 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:55 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:56.005 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:55 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:56.005 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:55 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:56.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[41114]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T08:36:56.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:56.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:56.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:56.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:56.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:56.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:56.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:56.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:56.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:56.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:56.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:56.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:56.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:56.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:56.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[29647]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T08:36:56.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:56.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:56.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:56.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:56.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:56.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:56.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:56.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:56.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:56.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:56.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:56.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:56.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:56.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:55 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:36:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:56 smithi145 ceph-mon[33384]: Detected new or changed devices on smithi092 2024-09-11T08:36:56.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:56 smithi145 ceph-mon[33384]: Adjusting osd_memory_target on smithi092 to 4016M 2024-09-11T08:36:56.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:56 smithi145 ceph-mon[33384]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:56.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:56 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:56.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:56 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:56.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:56 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:57.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:56 smithi092 ceph-mon[29647]: Detected new or changed devices on smithi092 2024-09-11T08:36:57.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:56 smithi092 ceph-mon[29647]: Adjusting osd_memory_target on smithi092 to 4016M 2024-09-11T08:36:57.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:56 smithi092 ceph-mon[29647]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:57.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:56 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:57.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:56 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:57.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:56 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:57.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:56 smithi092 ceph-mon[41114]: Detected new or changed devices on smithi092 2024-09-11T08:36:57.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:56 smithi092 ceph-mon[41114]: Adjusting osd_memory_target on smithi092 to 4016M 2024-09-11T08:36:57.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:56 smithi092 ceph-mon[41114]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:57.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:56 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:57.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:56 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:57.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:56 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:57.859 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:57 smithi145 ceph-mon[33384]: from='client.24184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:58.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:57 smithi092 ceph-mon[29647]: from='client.24184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:58.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:57 smithi092 ceph-mon[41114]: from='client.24184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:59.051 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:58 smithi145 ceph-mon[33384]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:59.051 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:58 smithi145 ceph-mon[33384]: from='client.? 172.21.15.145:0/1247408937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e144e102-963b-48b7-92a9-9391706164bf"}]: dispatch 2024-09-11T08:36:59.051 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:58 smithi145 ceph-mon[33384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e144e102-963b-48b7-92a9-9391706164bf"}]: dispatch 2024-09-11T08:36:59.051 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:58 smithi145 ceph-mon[33384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e144e102-963b-48b7-92a9-9391706164bf"}]': finished 2024-09-11T08:36:59.052 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:58 smithi145 ceph-mon[33384]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T08:36:59.052 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:58 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:59.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:58 smithi092 ceph-mon[29647]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:59.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:58 smithi092 ceph-mon[29647]: from='client.? 172.21.15.145:0/1247408937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e144e102-963b-48b7-92a9-9391706164bf"}]: dispatch 2024-09-11T08:36:59.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:58 smithi092 ceph-mon[29647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e144e102-963b-48b7-92a9-9391706164bf"}]: dispatch 2024-09-11T08:36:59.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:58 smithi092 ceph-mon[29647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e144e102-963b-48b7-92a9-9391706164bf"}]': finished 2024-09-11T08:36:59.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:58 smithi092 ceph-mon[29647]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T08:36:59.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:58 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:58 smithi092 ceph-mon[41114]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:58 smithi092 ceph-mon[41114]: from='client.? 172.21.15.145:0/1247408937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e144e102-963b-48b7-92a9-9391706164bf"}]: dispatch 2024-09-11T08:36:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:58 smithi092 ceph-mon[41114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e144e102-963b-48b7-92a9-9391706164bf"}]: dispatch 2024-09-11T08:36:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:58 smithi092 ceph-mon[41114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e144e102-963b-48b7-92a9-9391706164bf"}]': finished 2024-09-11T08:36:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:58 smithi092 ceph-mon[41114]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T08:36:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:58 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:37:00.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:36:59 smithi092 ceph-mon[29647]: from='client.? 172.21.15.145:0/530074434' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:37:00.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:36:59 smithi092 ceph-mon[41114]: from='client.? 172.21.15.145:0/530074434' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:37:00.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:36:59 smithi145 ceph-mon[33384]: from='client.? 172.21.15.145:0/530074434' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:37:01.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:00 smithi092 ceph-mon[29647]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:01.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:00 smithi092 ceph-mon[41114]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:01.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:00 smithi145 ceph-mon[33384]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:02.849 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:02 smithi145 ceph-mon[33384]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:03.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:02 smithi092 ceph-mon[29647]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:03.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:02 smithi092 ceph-mon[41114]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:05.026 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:04 smithi145 ceph-mon[33384]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:05.026 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:04 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T08:37:05.026 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:04 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:05.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:04 smithi092 ceph-mon[29647]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:05.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:04 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T08:37:05.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:04 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:05.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:04 smithi092 ceph-mon[41114]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:05.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:04 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T08:37:05.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:04 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:05.949 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:05 smithi145 ceph-mon[33384]: Deploying daemon osd.4 on smithi145 2024-09-11T08:37:06.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:05 smithi092 ceph-mon[29647]: Deploying daemon osd.4 on smithi145 2024-09-11T08:37:06.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:05 smithi092 ceph-mon[41114]: Deploying daemon osd.4 on smithi145 2024-09-11T08:37:06.752 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:06 smithi145 ceph-mon[33384]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:07.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:06 smithi092 ceph-mon[29647]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:06 smithi092 ceph-mon[41114]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:08 smithi145 ceph-mon[33384]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:09.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:08 smithi092 ceph-mon[29647]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:09.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:08 smithi092 ceph-mon[41114]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:09.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:09 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:09.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:09 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:09.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:09 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:10.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:09 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:10.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:09 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:10.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:09 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:10.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:09 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:10.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:09 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:10.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:09 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:11.309 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:11 smithi145 ceph-mon[33384]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:11.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:11 smithi092 ceph-mon[29647]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:11.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:11 smithi092 ceph-mon[41114]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:12.072 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 4 on host 'smithi145' 2024-09-11T08:37:12.639 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:12 smithi145 ceph-mon[33384]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:12.639 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:12.639 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:12.639 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:12.639 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:12.639 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:12.639 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:12.640 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:12.640 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:12.640 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:12 smithi145 ceph-mon[33384]: from='osd.4 [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:37:12.641 DEBUG:teuthology.orchestra.run.smithi145:osd.4> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T08:37:12.643 INFO:tasks.cephadm:Deploying osd.5 on smithi145 with /dev/vg_nvme/lv_3... 2024-09-11T08:37:12.643 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T08:37:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[29647]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[29647]: from='osd.4 [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:37:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[41114]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:12 smithi092 ceph-mon[41114]: from='osd.4 [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:37:13.632 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:37:14.144 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:14 smithi145 ceph-mon[33384]: from='osd.4 [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T08:37:14.144 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:14 smithi145 ceph-mon[33384]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T08:37:14.144 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:14 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:37:14.144 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:14 smithi145 ceph-mon[33384]: from='osd.4 [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:37:14.144 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:14 smithi145 ceph-mon[33384]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:14.145 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 08:37:14 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4[38644]: 2024-09-11T08:37:14.081+0000 7f9939b09640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T08:37:14.145 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 08:37:14 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4[38644]: 2024-09-11T08:37:14.085+0000 7f993413f640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:37:14.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:14 smithi092 ceph-mon[29647]: from='osd.4 [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T08:37:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:14 smithi092 ceph-mon[29647]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T08:37:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:14 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:37:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:14 smithi092 ceph-mon[29647]: from='osd.4 [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:37:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:14 smithi092 ceph-mon[29647]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:14 smithi092 ceph-mon[41114]: from='osd.4 [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T08:37:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:14 smithi092 ceph-mon[41114]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T08:37:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:14 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:37:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:14 smithi092 ceph-mon[41114]: from='osd.4 [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:37:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:14 smithi092 ceph-mon[41114]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:37:15.156 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:15 smithi145 ceph-mon[33384]: from='osd.4 [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-11T08:37:15.156 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:15 smithi145 ceph-mon[33384]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T08:37:15.157 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:15 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:37:15.157 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:15 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:37:15.512 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T08:37:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:15 smithi092 ceph-mon[29647]: from='osd.4 [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-11T08:37:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:15 smithi092 ceph-mon[29647]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T08:37:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:15 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:37:15.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:15 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:37:15.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:15 smithi092 ceph-mon[41114]: from='osd.4 [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-11T08:37:15.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:15 smithi092 ceph-mon[41114]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T08:37:15.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:15 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:37:15.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:15 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:37:15.600 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T08:37:15.615 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2024-09-11T08:37:15.615 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2024-09-11T08:37:15.615 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124714 s, 841 MB/s 2024-09-11T08:37:15.615 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2024-09-11T08:37:16.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:16 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:37:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:16 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:37:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:16 smithi145 ceph-mon[33384]: osd.4 [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669] boot 2024-09-11T08:37:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:16 smithi145 ceph-mon[33384]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T08:37:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:16 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:37:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:16 smithi145 ceph-mon[33384]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:16 smithi145 ceph-mon[33384]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T08:37:16.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:16 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:37:16.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:16 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:37:16.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:16 smithi092 ceph-mon[29647]: osd.4 [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669] boot 2024-09-11T08:37:16.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:16 smithi092 ceph-mon[29647]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T08:37:16.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:16 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:37:16.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:16 smithi092 ceph-mon[29647]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:16.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:16 smithi092 ceph-mon[29647]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T08:37:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:16 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:37:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:16 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:37:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:16 smithi092 ceph-mon[41114]: osd.4 [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669] boot 2024-09-11T08:37:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:16 smithi092 ceph-mon[41114]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T08:37:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:16 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:37:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:16 smithi092 ceph-mon[41114]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:16.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:16 smithi092 ceph-mon[41114]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T08:37:16.692 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_3 2024-09-11T08:37:16.928 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:37:17.362 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:17 smithi145 ceph-mon[33384]: Detected new or changed devices on smithi145 2024-09-11T08:37:17.362 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:17 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:17.362 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:17 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:17.363 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:17 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:17.363 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:17 smithi145 ceph-mon[33384]: Adjusting osd_memory_target on smithi145 to 17088M 2024-09-11T08:37:17.363 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:17 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:17.363 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:17 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:17.363 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:17 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:17.363 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:17 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:17.363 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:17 smithi145 ceph-mon[33384]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T08:37:17.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[41114]: Detected new or changed devices on smithi145 2024-09-11T08:37:17.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:17.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:17.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:17.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[41114]: Adjusting osd_memory_target on smithi145 to 17088M 2024-09-11T08:37:17.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:17.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:17.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:17.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:17.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[41114]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T08:37:17.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[29647]: Detected new or changed devices on smithi145 2024-09-11T08:37:17.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:17.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:17.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:17.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[29647]: Adjusting osd_memory_target on smithi145 to 17088M 2024-09-11T08:37:17.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:17.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:17.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:17.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:17.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:17 smithi092 ceph-mon[29647]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T08:37:18.510 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:18 smithi145 ceph-mon[33384]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:18.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:18 smithi092 ceph-mon[29647]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:18.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:18 smithi092 ceph-mon[41114]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:19.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:19 smithi145 ceph-mon[33384]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:37:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:19 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:37:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:19 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:37:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:19 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:20.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:19 smithi092 ceph-mon[29647]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:37:20.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:19 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:37:20.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:19 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:37:20.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:19 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:19 smithi092 ceph-mon[41114]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:37:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:19 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:37:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:19 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:37:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:19 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:20.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:20 smithi145 ceph-mon[33384]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:21.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:20 smithi092 ceph-mon[29647]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:21.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:20 smithi092 ceph-mon[41114]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:22.058 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:21 smithi092 ceph-mon[29647]: from='client.? 172.21.15.145:0/3484028116' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "811b0dc9-b7a7-439a-86c1-5dc9d0d54e16"}]: dispatch 2024-09-11T08:37:22.058 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:21 smithi092 ceph-mon[29647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "811b0dc9-b7a7-439a-86c1-5dc9d0d54e16"}]: dispatch 2024-09-11T08:37:22.058 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:21 smithi092 ceph-mon[29647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "811b0dc9-b7a7-439a-86c1-5dc9d0d54e16"}]': finished 2024-09-11T08:37:22.059 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:21 smithi092 ceph-mon[29647]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T08:37:22.059 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:21 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:22.059 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:21 smithi092 ceph-mon[29647]: from='client.? 172.21.15.145:0/100059737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:37:22.059 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:21 smithi092 ceph-mon[41114]: from='client.? 172.21.15.145:0/3484028116' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "811b0dc9-b7a7-439a-86c1-5dc9d0d54e16"}]: dispatch 2024-09-11T08:37:22.059 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:21 smithi092 ceph-mon[41114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "811b0dc9-b7a7-439a-86c1-5dc9d0d54e16"}]: dispatch 2024-09-11T08:37:22.059 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:21 smithi092 ceph-mon[41114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "811b0dc9-b7a7-439a-86c1-5dc9d0d54e16"}]': finished 2024-09-11T08:37:22.059 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:21 smithi092 ceph-mon[41114]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T08:37:22.059 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:21 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:22.059 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:21 smithi092 ceph-mon[41114]: from='client.? 172.21.15.145:0/100059737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:37:22.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:21 smithi145 ceph-mon[33384]: from='client.? 172.21.15.145:0/3484028116' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "811b0dc9-b7a7-439a-86c1-5dc9d0d54e16"}]: dispatch 2024-09-11T08:37:22.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:21 smithi145 ceph-mon[33384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "811b0dc9-b7a7-439a-86c1-5dc9d0d54e16"}]: dispatch 2024-09-11T08:37:22.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:21 smithi145 ceph-mon[33384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "811b0dc9-b7a7-439a-86c1-5dc9d0d54e16"}]': finished 2024-09-11T08:37:22.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:21 smithi145 ceph-mon[33384]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T08:37:22.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:21 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:22.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:21 smithi145 ceph-mon[33384]: from='client.? 172.21.15.145:0/100059737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:37:23.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:22 smithi092 ceph-mon[29647]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T08:37:23.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:22 smithi092 ceph-mon[41114]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T08:37:23.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:22 smithi145 ceph-mon[33384]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T08:37:25.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:24 smithi145 ceph-mon[33384]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-11T08:37:25.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:24 smithi092 ceph-mon[29647]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-11T08:37:25.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:24 smithi092 ceph-mon[41114]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-11T08:37:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:26 smithi145 ceph-mon[33384]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-11T08:37:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:26 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T08:37:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:26 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:27.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:26 smithi092 ceph-mon[29647]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-11T08:37:27.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:26 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T08:37:27.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:26 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:27.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:26 smithi092 ceph-mon[41114]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-11T08:37:27.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:26 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T08:37:27.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:26 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:28.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:27 smithi145 ceph-mon[33384]: Deploying daemon osd.5 on smithi145 2024-09-11T08:37:28.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:27 smithi092 ceph-mon[29647]: Deploying daemon osd.5 on smithi145 2024-09-11T08:37:28.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:27 smithi092 ceph-mon[41114]: Deploying daemon osd.5 on smithi145 2024-09-11T08:37:29.139 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:28 smithi145 ceph-mon[33384]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:29.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:28 smithi092 ceph-mon[29647]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:29.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:28 smithi092 ceph-mon[41114]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:31.121 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:30 smithi145 ceph-mon[33384]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:31.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:30 smithi092 ceph-mon[29647]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:31.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:30 smithi092 ceph-mon[41114]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:32.004 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:31 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:32.004 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:31 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:32.004 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:31 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:32.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:31 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:32.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:31 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:32.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:31 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:31 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:31 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:31 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:33.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:32 smithi145 ceph-mon[33384]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T08:37:33.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:32 smithi092 ceph-mon[29647]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T08:37:33.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:32 smithi092 ceph-mon[41114]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T08:37:34.587 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 5 on host 'smithi145' 2024-09-11T08:37:35.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:34 smithi145 ceph-mon[33384]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:35.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:34 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:37:35.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:34 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:37:35.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:35.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:35.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:35.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:35.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:35.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:34 smithi145 ceph-mon[33384]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T08:37:35.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:35.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:34 smithi145 ceph-mon[33384]: from='osd.5 [v2:172.21.15.145:6808/3569637188,v1:172.21.15.145:6809/3569637188]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:37:35.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:34 smithi145 ceph-mon[33384]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:37:35.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:35.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:35.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:34 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:35.195 DEBUG:teuthology.orchestra.run.smithi145:osd.5> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T08:37:35.201 INFO:tasks.cephadm:Deploying osd.6 on smithi145 with /dev/vg_nvme/lv_2... 2024-09-11T08:37:35.201 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T08:37:35.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[29647]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:35.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:37:35.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:37:35.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:35.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:35.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:35.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:35.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:35.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[29647]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T08:37:35.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:35.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[29647]: from='osd.5 [v2:172.21.15.145:6808/3569637188,v1:172.21.15.145:6809/3569637188]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:37:35.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[29647]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:37:35.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:35.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:35.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[41114]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:37:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:37:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:35.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[41114]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T08:37:35.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:35.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[41114]: from='osd.5 [v2:172.21.15.145:6808/3569637188,v1:172.21.15.145:6809/3569637188]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:37:35.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[41114]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:37:35.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:35.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:35.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:34 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:36.071 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:37:36.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:35 smithi145 ceph-mon[33384]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T08:37:36.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:35 smithi145 ceph-mon[33384]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T08:37:36.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:36.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:35 smithi145 ceph-mon[33384]: from='osd.5 [v2:172.21.15.145:6808/3569637188,v1:172.21.15.145:6809/3569637188]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:37:36.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:35 smithi145 ceph-mon[33384]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:37:36.162 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 08:37:35 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T08:37:35.873+0000 7f1da23a8640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T08:37:36.162 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 08:37:35 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T08:37:35.878+0000 7f1d9d1df640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:37:36.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:35 smithi092 ceph-mon[29647]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T08:37:36.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:35 smithi092 ceph-mon[29647]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T08:37:36.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:36.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:35 smithi092 ceph-mon[29647]: from='osd.5 [v2:172.21.15.145:6808/3569637188,v1:172.21.15.145:6809/3569637188]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:37:36.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:35 smithi092 ceph-mon[29647]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:37:36.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:35 smithi092 ceph-mon[41114]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T08:37:36.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:35 smithi092 ceph-mon[41114]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T08:37:36.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:36.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:35 smithi092 ceph-mon[41114]: from='osd.5 [v2:172.21.15.145:6808/3569637188,v1:172.21.15.145:6809/3569637188]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:37:36.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:35 smithi092 ceph-mon[41114]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:37:37.027 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:36 smithi145 ceph-mon[33384]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:37.027 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:36 smithi145 ceph-mon[33384]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-11T08:37:37.027 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:36 smithi145 ceph-mon[33384]: osdmap e37: 6 total, 5 up, 6 in 2024-09-11T08:37:37.027 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:36 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:37.027 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:36 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:37.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:36 smithi092 ceph-mon[29647]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:37.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:36 smithi092 ceph-mon[29647]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-11T08:37:37.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:36 smithi092 ceph-mon[29647]: osdmap e37: 6 total, 5 up, 6 in 2024-09-11T08:37:37.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:36 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:37.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:36 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:37.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:36 smithi092 ceph-mon[41114]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:37.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:36 smithi092 ceph-mon[41114]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-11T08:37:37.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:36 smithi092 ceph-mon[41114]: osdmap e37: 6 total, 5 up, 6 in 2024-09-11T08:37:37.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:36 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:37.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:36 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:37.907 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:37 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:37:37.907 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:37 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:37:37.907 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:37 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:37.907 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:37 smithi145 ceph-mon[33384]: osd.5 [v2:172.21.15.145:6808/3569637188,v1:172.21.15.145:6809/3569637188] boot 2024-09-11T08:37:37.907 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:37 smithi145 ceph-mon[33384]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T08:37:37.908 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:37 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:38.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:37 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:37:38.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:37 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:37:38.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:37 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:38.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:37 smithi092 ceph-mon[29647]: osd.5 [v2:172.21.15.145:6808/3569637188,v1:172.21.15.145:6809/3569637188] boot 2024-09-11T08:37:38.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:37 smithi092 ceph-mon[29647]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T08:37:38.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:37 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:38.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:37 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:37:38.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:37 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:37:38.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:37 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:38.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:37 smithi092 ceph-mon[41114]: osd.5 [v2:172.21.15.145:6808/3569637188,v1:172.21.15.145:6809/3569637188] boot 2024-09-11T08:37:38.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:37 smithi092 ceph-mon[41114]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T08:37:38.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:37 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:37:39.059 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T08:37:39.059 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:38 smithi145 ceph-mon[33384]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:39.059 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:38 smithi145 ceph-mon[33384]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:37:39.059 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:38 smithi145 ceph-mon[33384]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T08:37:39.060 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:39.060 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:39.060 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:39.060 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:39.060 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:39.060 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:39.060 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:39.060 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:39.141 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T08:37:39.161 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2024-09-11T08:37:39.161 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2024-09-11T08:37:39.161 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0185984 s, 564 MB/s 2024-09-11T08:37:39.162 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2024-09-11T08:37:39.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[29647]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:39.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[29647]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:37:39.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[29647]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T08:37:39.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:39.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:39.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:39.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:39.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:39.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:39.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:39.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:39.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[41114]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:37:39.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[41114]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:37:39.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[41114]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T08:37:39.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:39.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:39.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:39.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:39.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:39.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:39.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:39.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:39.691 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_2 2024-09-11T08:37:39.935 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:37:40.034 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:39 smithi145 ceph-mon[33384]: Detected new or changed devices on smithi145 2024-09-11T08:37:40.034 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:39 smithi145 ceph-mon[33384]: Adjusting osd_memory_target on smithi145 to 8544M 2024-09-11T08:37:40.035 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:39 smithi145 ceph-mon[33384]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T08:37:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:39 smithi092 ceph-mon[29647]: Detected new or changed devices on smithi145 2024-09-11T08:37:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:39 smithi092 ceph-mon[29647]: Adjusting osd_memory_target on smithi145 to 8544M 2024-09-11T08:37:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:39 smithi092 ceph-mon[29647]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T08:37:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:39 smithi092 ceph-mon[41114]: Detected new or changed devices on smithi145 2024-09-11T08:37:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:39 smithi092 ceph-mon[41114]: Adjusting osd_memory_target on smithi145 to 8544M 2024-09-11T08:37:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:39 smithi092 ceph-mon[41114]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T08:37:41.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:40 smithi145 ceph-mon[33384]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:41.217 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:40 smithi092 ceph-mon[29647]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:41.217 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:40 smithi092 ceph-mon[41114]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:42.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:41 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:37:42.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:41 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:37:42.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:41 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:42.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:41 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:37:42.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:41 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:37:42.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:41 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:42.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:41 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:37:42.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:41 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:37:42.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:41 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:43.144 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:42 smithi145 ceph-mon[33384]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:43.144 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:42 smithi145 ceph-mon[33384]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:37:43.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:42 smithi092 ceph-mon[29647]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:43.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:42 smithi092 ceph-mon[29647]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:37:43.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:42 smithi092 ceph-mon[41114]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:43.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:42 smithi092 ceph-mon[41114]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:37:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:44 smithi145 ceph-mon[33384]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-11T08:37:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:44 smithi145 ceph-mon[33384]: from='client.? 172.21.15.145:0/1638125363' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd162f52-097c-4c5d-9d37-1289779a196b"}]: dispatch 2024-09-11T08:37:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:44 smithi145 ceph-mon[33384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd162f52-097c-4c5d-9d37-1289779a196b"}]: dispatch 2024-09-11T08:37:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:44 smithi145 ceph-mon[33384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd162f52-097c-4c5d-9d37-1289779a196b"}]': finished 2024-09-11T08:37:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:44 smithi145 ceph-mon[33384]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T08:37:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:44 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:44 smithi145 ceph-mon[33384]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:37:44.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:44 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:37:44.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:44 smithi145 ceph-mon[33384]: from='client.? 172.21.15.145:0/3929087575' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:37:45.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[29647]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-11T08:37:45.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[29647]: from='client.? 172.21.15.145:0/1638125363' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd162f52-097c-4c5d-9d37-1289779a196b"}]: dispatch 2024-09-11T08:37:45.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[29647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd162f52-097c-4c5d-9d37-1289779a196b"}]: dispatch 2024-09-11T08:37:45.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[29647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd162f52-097c-4c5d-9d37-1289779a196b"}]': finished 2024-09-11T08:37:45.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[29647]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T08:37:45.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:45.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[29647]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:37:45.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:37:45.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[29647]: from='client.? 172.21.15.145:0/3929087575' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:37:45.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[41114]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-11T08:37:45.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[41114]: from='client.? 172.21.15.145:0/1638125363' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd162f52-097c-4c5d-9d37-1289779a196b"}]: dispatch 2024-09-11T08:37:45.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[41114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd162f52-097c-4c5d-9d37-1289779a196b"}]: dispatch 2024-09-11T08:37:45.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[41114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd162f52-097c-4c5d-9d37-1289779a196b"}]': finished 2024-09-11T08:37:45.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[41114]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T08:37:45.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:45.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[41114]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:37:45.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:37:45.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:44 smithi092 ceph-mon[41114]: from='client.? 172.21.15.145:0/3929087575' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:37:46.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:45 smithi092 ceph-mon[29647]: 1.0 scrub starts 2024-09-11T08:37:46.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:45 smithi092 ceph-mon[29647]: 1.0 scrub ok 2024-09-11T08:37:46.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:45 smithi092 ceph-mon[41114]: 1.0 scrub starts 2024-09-11T08:37:46.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:45 smithi092 ceph-mon[41114]: 1.0 scrub ok 2024-09-11T08:37:46.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:45 smithi145 ceph-mon[33384]: 1.0 scrub starts 2024-09-11T08:37:46.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:45 smithi145 ceph-mon[33384]: 1.0 scrub ok 2024-09-11T08:37:46.856 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:46 smithi145 ceph-mon[33384]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-11T08:37:47.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:46 smithi092 ceph-mon[41114]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-11T08:37:47.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:46 smithi092 ceph-mon[29647]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-11T08:37:48.783 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:48 smithi145 ceph-mon[33384]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-11T08:37:49.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:48 smithi092 ceph-mon[29647]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-11T08:37:49.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:48 smithi092 ceph-mon[41114]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-11T08:37:50.060 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:49 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T08:37:50.061 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:49 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:50.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:49 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T08:37:50.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:49 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:50.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:49 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T08:37:50.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:49 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:50.943 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:50 smithi145 ceph-mon[33384]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:50.943 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:50 smithi145 ceph-mon[33384]: Deploying daemon osd.6 on smithi145 2024-09-11T08:37:51.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:50 smithi092 ceph-mon[29647]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:51.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:50 smithi092 ceph-mon[29647]: Deploying daemon osd.6 on smithi145 2024-09-11T08:37:51.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:50 smithi092 ceph-mon[41114]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:51.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:50 smithi092 ceph-mon[41114]: Deploying daemon osd.6 on smithi145 2024-09-11T08:37:52.954 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:52 smithi145 ceph-mon[33384]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:52.954 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:52 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:37:52.954 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:52 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:37:52.954 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:52 smithi145 ceph-mon[33384]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T08:37:52.954 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:52 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:53.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:52 smithi092 ceph-mon[29647]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:53.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:52 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:37:53.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:52 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:37:53.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:52 smithi092 ceph-mon[29647]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T08:37:53.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:52 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:53.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:52 smithi092 ceph-mon[41114]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:53.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:52 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:37:53.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:52 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:37:53.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:52 smithi092 ceph-mon[41114]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T08:37:53.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:52 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:55.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:54 smithi092 ceph-mon[29647]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:55.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:54 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:55.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:54 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:55.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:54 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:55.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:54 smithi092 ceph-mon[41114]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:55.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:54 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:55.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:54 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:55.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:54 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:55.123 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:54 smithi145 ceph-mon[33384]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:55.123 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:54 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:55.123 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:54 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:55.123 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:54 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:57.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:56 smithi092 ceph-mon[29647]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:57.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:56 smithi092 ceph-mon[41114]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:57.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:56 smithi145 ceph-mon[33384]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:57.497 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 6 on host 'smithi145' 2024-09-11T08:37:58.064 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:57 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:58.064 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:57 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:58.064 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:57 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:58.065 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:57 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:58.065 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:57 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:58.065 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:57 smithi145 ceph-mon[33384]: from='osd.6 [v2:172.21.15.145:6816/2615047110,v1:172.21.15.145:6817/2615047110]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:37:58.065 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:57 smithi145 ceph-mon[33384]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:37:58.065 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:57 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:58.065 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:57 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:58.065 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:57 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[41114]: from='osd.6 [v2:172.21.15.145:6816/2615047110,v1:172.21.15.145:6817/2615047110]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:37:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[41114]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:37:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:58.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:58.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:58.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:58.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:58.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:58.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[29647]: from='osd.6 [v2:172.21.15.145:6816/2615047110,v1:172.21.15.145:6817/2615047110]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:37:58.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[29647]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:37:58.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:58.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:58.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:57 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:37:58.093 DEBUG:teuthology.orchestra.run.smithi145:osd.6> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T08:37:58.095 INFO:tasks.cephadm:Deploying osd.7 on smithi145 with /dev/vg_nvme/lv_1... 2024-09-11T08:37:58.096 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T08:37:58.773 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:37:59.037 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:58 smithi145 ceph-mon[33384]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:59.037 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:58 smithi145 ceph-mon[33384]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T08:37:59.037 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:58 smithi145 ceph-mon[33384]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T08:37:59.037 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:58 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:59.037 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:58 smithi145 ceph-mon[33384]: from='osd.6 [v2:172.21.15.145:6816/2615047110,v1:172.21.15.145:6817/2615047110]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:37:59.037 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:58 smithi145 ceph-mon[33384]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:37:59.038 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 08:37:58 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T08:37:58.788+0000 7fefc711c640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T08:37:59.038 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 08:37:58 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T08:37:58.795+0000 7fefc1f53640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:37:59.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:58 smithi092 ceph-mon[29647]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:59.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:58 smithi092 ceph-mon[29647]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T08:37:59.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:58 smithi092 ceph-mon[29647]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T08:37:59.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:58 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:59.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:58 smithi092 ceph-mon[29647]: from='osd.6 [v2:172.21.15.145:6816/2615047110,v1:172.21.15.145:6817/2615047110]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:37:59.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:58 smithi092 ceph-mon[29647]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:37:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:58 smithi092 ceph-mon[41114]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:58 smithi092 ceph-mon[41114]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T08:37:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:58 smithi092 ceph-mon[41114]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T08:37:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:58 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:58 smithi092 ceph-mon[41114]: from='osd.6 [v2:172.21.15.145:6816/2615047110,v1:172.21.15.145:6817/2615047110]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:37:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:58 smithi092 ceph-mon[41114]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:38:00.062 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:59 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:38:00.062 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:59 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:38:00.062 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:59 smithi145 ceph-mon[33384]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-11T08:38:00.062 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:59 smithi145 ceph-mon[33384]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T08:38:00.062 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:59 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:38:00.063 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:59 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:38:00.063 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:59 smithi145 ceph-mon[33384]: osd.6 [v2:172.21.15.145:6816/2615047110,v1:172.21.15.145:6817/2615047110] boot 2024-09-11T08:38:00.063 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:59 smithi145 ceph-mon[33384]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T08:38:00.063 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:37:59 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:38:00.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:38:00.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:38:00.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[41114]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-11T08:38:00.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[41114]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T08:38:00.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:38:00.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:38:00.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[41114]: osd.6 [v2:172.21.15.145:6816/2615047110,v1:172.21.15.145:6817/2615047110] boot 2024-09-11T08:38:00.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[41114]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T08:38:00.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:38:00.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:38:00.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:38:00.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[29647]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-11T08:38:00.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[29647]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T08:38:00.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:38:00.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:38:00.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[29647]: osd.6 [v2:172.21.15.145:6816/2615047110,v1:172.21.15.145:6817/2615047110] boot 2024-09-11T08:38:00.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[29647]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T08:38:00.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:37:59 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:38:00.976 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:00 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:38:00.976 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:00 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:38:00.976 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:00 smithi145 ceph-mon[33384]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:38:00.976 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:00 smithi145 ceph-mon[33384]: osdmap e46: 7 total, 7 up, 7 in 2024-09-11T08:38:01.000 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T08:38:01.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:00 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:38:01.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:00 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:38:01.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:00 smithi092 ceph-mon[41114]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:38:01.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:00 smithi092 ceph-mon[41114]: osdmap e46: 7 total, 7 up, 7 in 2024-09-11T08:38:01.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:00 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:38:01.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:00 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:38:01.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:00 smithi092 ceph-mon[29647]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:38:01.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:00 smithi092 ceph-mon[29647]: osdmap e46: 7 total, 7 up, 7 in 2024-09-11T08:38:01.101 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T08:38:01.117 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2024-09-11T08:38:01.117 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2024-09-11T08:38:01.117 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0140526 s, 746 MB/s 2024-09-11T08:38:01.117 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2024-09-11T08:38:01.754 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi145:vg_nvme/lv_1 2024-09-11T08:38:02.333 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:38:02.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:02 smithi145 ceph-mon[33384]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:02 smithi145 ceph-mon[33384]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T08:38:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:02 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:02 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:02 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:02 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:02 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:02 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:02 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:02 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:02 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:03.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[29647]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:03.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[29647]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T08:38:03.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:03.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:03.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:03.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:03.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:03.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:03.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:03.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:03.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:03.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[41114]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:03.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[41114]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T08:38:03.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:03.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:03.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:03.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:03.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:03.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:03.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:03.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:03.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:02 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:04.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:03 smithi092 ceph-mon[41114]: Detected new or changed devices on smithi145 2024-09-11T08:38:04.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:03 smithi092 ceph-mon[41114]: Adjusting osd_memory_target on smithi145 to 5696M 2024-09-11T08:38:04.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:03 smithi092 ceph-mon[29647]: Detected new or changed devices on smithi145 2024-09-11T08:38:04.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:03 smithi092 ceph-mon[29647]: Adjusting osd_memory_target on smithi145 to 5696M 2024-09-11T08:38:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:03 smithi145 ceph-mon[33384]: Detected new or changed devices on smithi145 2024-09-11T08:38:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:03 smithi145 ceph-mon[33384]: Adjusting osd_memory_target on smithi145 to 5696M 2024-09-11T08:38:05.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:04 smithi092 ceph-mon[29647]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:05.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:04 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:38:05.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:04 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:38:05.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:04 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:05.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:04 smithi092 ceph-mon[41114]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:05.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:04 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:38:05.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:04 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:38:05.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:04 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:05.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:04 smithi145 ceph-mon[33384]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:05.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:04 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:38:05.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:04 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:38:05.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:04 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:05.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:05 smithi145 ceph-mon[33384]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:06.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:05 smithi092 ceph-mon[29647]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:06.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:05 smithi092 ceph-mon[41114]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:06.985 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:06 smithi145 ceph-mon[33384]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:06.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:06 smithi145 ceph-mon[33384]: from='client.? 172.21.15.145:0/1399844913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e4021bf-78c7-4bc6-b222-3494f724ddec"}]: dispatch 2024-09-11T08:38:06.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:06 smithi145 ceph-mon[33384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e4021bf-78c7-4bc6-b222-3494f724ddec"}]: dispatch 2024-09-11T08:38:06.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:06 smithi145 ceph-mon[33384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e4021bf-78c7-4bc6-b222-3494f724ddec"}]': finished 2024-09-11T08:38:06.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:06 smithi145 ceph-mon[33384]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T08:38:06.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:06 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:07.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:06 smithi092 ceph-mon[29647]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:07.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:06 smithi092 ceph-mon[29647]: from='client.? 172.21.15.145:0/1399844913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e4021bf-78c7-4bc6-b222-3494f724ddec"}]: dispatch 2024-09-11T08:38:07.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:06 smithi092 ceph-mon[29647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e4021bf-78c7-4bc6-b222-3494f724ddec"}]: dispatch 2024-09-11T08:38:07.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:06 smithi092 ceph-mon[29647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e4021bf-78c7-4bc6-b222-3494f724ddec"}]': finished 2024-09-11T08:38:07.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:06 smithi092 ceph-mon[29647]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T08:38:07.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:06 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:06 smithi092 ceph-mon[41114]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:06 smithi092 ceph-mon[41114]: from='client.? 172.21.15.145:0/1399844913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e4021bf-78c7-4bc6-b222-3494f724ddec"}]: dispatch 2024-09-11T08:38:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:06 smithi092 ceph-mon[41114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e4021bf-78c7-4bc6-b222-3494f724ddec"}]: dispatch 2024-09-11T08:38:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:06 smithi092 ceph-mon[41114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e4021bf-78c7-4bc6-b222-3494f724ddec"}]': finished 2024-09-11T08:38:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:06 smithi092 ceph-mon[41114]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T08:38:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:06 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:08.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:07 smithi145 ceph-mon[33384]: from='client.? 172.21.15.145:0/3108133340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:38:08.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:07 smithi092 ceph-mon[29647]: from='client.? 172.21.15.145:0/3108133340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:38:08.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:07 smithi092 ceph-mon[41114]: from='client.? 172.21.15.145:0/3108133340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:38:09.121 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:08 smithi145 ceph-mon[33384]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:09.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:08 smithi092 ceph-mon[29647]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:09.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:08 smithi092 ceph-mon[41114]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:11.118 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:10 smithi092 ceph-mon[29647]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:11.118 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:10 smithi092 ceph-mon[41114]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:11.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:10 smithi145 ceph-mon[33384]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:13.074 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:12 smithi145 ceph-mon[33384]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:13.074 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:38:13.074 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:12 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:13.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:12 smithi092 ceph-mon[41114]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:13.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:38:13.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:12 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:13.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:12 smithi092 ceph-mon[29647]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:13.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:38:13.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:12 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:14.097 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:13 smithi145 ceph-mon[33384]: Deploying daemon osd.7 on smithi145 2024-09-11T08:38:14.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:13 smithi092 ceph-mon[41114]: Deploying daemon osd.7 on smithi145 2024-09-11T08:38:14.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:13 smithi092 ceph-mon[29647]: Deploying daemon osd.7 on smithi145 2024-09-11T08:38:15.055 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:14 smithi145 ceph-mon[33384]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:15.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:14 smithi092 ceph-mon[41114]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:15.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:14 smithi092 ceph-mon[29647]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:17.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:16 smithi145 ceph-mon[33384]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:17.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:16 smithi092 ceph-mon[41114]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:17.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:16 smithi092 ceph-mon[29647]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:17 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:38:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:17 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:38:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:17 smithi145 ceph-mon[33384]: osdmap e49: 8 total, 7 up, 8 in 2024-09-11T08:38:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:17 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:17 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:17 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:17 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:18.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:17 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:38:18.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:17 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:38:18.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:17 smithi092 ceph-mon[41114]: osdmap e49: 8 total, 7 up, 8 in 2024-09-11T08:38:18.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:17 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:18.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:17 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:18.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:17 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:18.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:17 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:17 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:38:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:17 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:38:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:17 smithi092 ceph-mon[29647]: osdmap e49: 8 total, 7 up, 8 in 2024-09-11T08:38:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:18.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:18.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:18.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:17 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:19.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:18 smithi145 ceph-mon[33384]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:19.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:18 smithi092 ceph-mon[29647]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:19.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:18 smithi092 ceph-mon[41114]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:20.097 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 7 on host 'smithi145' 2024-09-11T08:38:20.662 DEBUG:teuthology.orchestra.run.smithi145:osd.7> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T08:38:20.664 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-11T08:38:20.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T08:38:20.923 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:21.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:20 smithi092 ceph-mon[29647]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:21.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:20 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:21.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:20 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:21.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:20 smithi092 ceph-mon[29647]: from='osd.7 [v2:172.21.15.145:6824/3195898530,v1:172.21.15.145:6825/3195898530]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:38:21.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:20 smithi092 ceph-mon[29647]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:38:21.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:20 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:21.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:20 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:21.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:20 smithi092 ceph-mon[41114]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:21.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:20 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:21.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:20 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:21.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:20 smithi092 ceph-mon[41114]: from='osd.7 [v2:172.21.15.145:6824/3195898530,v1:172.21.15.145:6825/3195898530]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:38:21.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:20 smithi092 ceph-mon[41114]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:38:21.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:20 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:21.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:20 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:21.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:20 smithi145 ceph-mon[33384]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:21.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:20 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:21.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:20 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:21.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:20 smithi145 ceph-mon[33384]: from='osd.7 [v2:172.21.15.145:6824/3195898530,v1:172.21.15.145:6825/3195898530]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:38:21.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:20 smithi145 ceph-mon[33384]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:38:21.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:20 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:21.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:20 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:22 smithi145 ceph-mon[33384]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:38:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:22 smithi145 ceph-mon[33384]: osdmap e50: 8 total, 7 up, 8 in 2024-09-11T08:38:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:22 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:22 smithi145 ceph-mon[33384]: from='osd.7 [v2:172.21.15.145:6824/3195898530,v1:172.21.15.145:6825/3195898530]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:38:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:22 smithi145 ceph-mon[33384]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:38:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:22 smithi145 ceph-mon[33384]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:22.413 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 08:38:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T08:38:22.109+0000 7fbc9734c640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T08:38:22.413 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 08:38:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T08:38:22.116+0000 7fbc90980640 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:38:22.414 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:22 smithi092 ceph-mon[41114]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:38:22.414 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:22 smithi092 ceph-mon[41114]: osdmap e50: 8 total, 7 up, 8 in 2024-09-11T08:38:22.414 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:22 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:22.414 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:22 smithi092 ceph-mon[41114]: from='osd.7 [v2:172.21.15.145:6824/3195898530,v1:172.21.15.145:6825/3195898530]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:38:22.414 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:22 smithi092 ceph-mon[41114]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:38:22.414 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:22 smithi092 ceph-mon[41114]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:22.414 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:22 smithi092 ceph-mon[29647]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:38:22.415 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:22 smithi092 ceph-mon[29647]: osdmap e50: 8 total, 7 up, 8 in 2024-09-11T08:38:22.415 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:22 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:22.415 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:22 smithi092 ceph-mon[29647]: from='osd.7 [v2:172.21.15.145:6824/3195898530,v1:172.21.15.145:6825/3195898530]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:38:22.415 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:22 smithi092 ceph-mon[29647]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-09-11T08:38:22.415 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:22 smithi092 ceph-mon[29647]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:38:22.766 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:38:23.105 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:23 smithi145 ceph-mon[33384]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-11T08:38:23.105 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:23 smithi145 ceph-mon[33384]: osdmap e51: 8 total, 7 up, 8 in 2024-09-11T08:38:23.105 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:23 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:23.105 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:23 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:23.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:23 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2732460915' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:38:23.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:23 smithi145 ceph-mon[33384]: osd.7 [v2:172.21.15.145:6824/3195898530,v1:172.21.15.145:6825/3195898530] boot 2024-09-11T08:38:23.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:23 smithi145 ceph-mon[33384]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T08:38:23.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:23 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:23.461 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[29647]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-11T08:38:23.461 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[29647]: osdmap e51: 8 total, 7 up, 8 in 2024-09-11T08:38:23.461 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:23.461 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:23.461 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2732460915' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:38:23.461 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[29647]: osd.7 [v2:172.21.15.145:6824/3195898530,v1:172.21.15.145:6825/3195898530] boot 2024-09-11T08:38:23.461 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[29647]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T08:38:23.462 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:23.462 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[41114]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-09-11T08:38:23.462 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[41114]: osdmap e51: 8 total, 7 up, 8 in 2024-09-11T08:38:23.462 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:23.462 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:23.462 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2732460915' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:38:23.462 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[41114]: osd.7 [v2:172.21.15.145:6824/3195898530,v1:172.21.15.145:6825/3195898530] boot 2024-09-11T08:38:23.462 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[41114]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T08:38:23.462 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:23 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:23.486 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":51,"num_osds":8,"num_up_osds":7,"osd_up_since":1726043879,"num_in_osds":8,"osd_in_since":1726043886,"num_remapped_pgs":0} 2024-09-11T08:38:24.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:24 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:38:24.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:24 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:38:24.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:24 smithi145 ceph-mon[33384]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:24.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:24 smithi145 ceph-mon[33384]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T08:38:24.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:24 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:24.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:24 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:24.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:24 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:24.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:24 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:24.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:24 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:24.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:24 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:24.487 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T08:38:24.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:38:24.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:38:24.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[29647]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:24.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[29647]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T08:38:24.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:24.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:24.512 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:24.512 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:24.512 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:24.512 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:24.512 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:38:24.512 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:38:24.512 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[41114]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:24.512 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[41114]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T08:38:24.512 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:24.512 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:24.512 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:24.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:24.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:24.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:24 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:24.731 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:25 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:38:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:25 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:38:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:25 smithi145 ceph-mon[33384]: Detected new or changed devices on smithi145 2024-09-11T08:38:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:25 smithi145 ceph-mon[33384]: Adjusting osd_memory_target on smithi145 to 4272M 2024-09-11T08:38:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:25 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:25 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:25 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:25 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:25 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:25.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:25 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:25.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:25 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:25.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:25 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:25.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:25 smithi145 ceph-mon[33384]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[41114]: Detected new or changed devices on smithi145 2024-09-11T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[41114]: Adjusting osd_memory_target on smithi145 to 4272M 2024-09-11T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:25.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[41114]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:38:25.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:38:25.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:38:25.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[29647]: Detected new or changed devices on smithi145 2024-09-11T08:38:25.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[29647]: Adjusting osd_memory_target on smithi145 to 4272M 2024-09-11T08:38:25.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:25.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:25.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:25.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:25.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:25.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:25.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:25.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:25.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:25 smithi092 ceph-mon[29647]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:38:26.153 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:26 smithi092 ceph-mon[29647]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:26.153 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:26 smithi092 ceph-mon[41114]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:26 smithi145 ceph-mon[33384]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:26.502 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:38:27.048 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1726043903,"num_in_osds":8,"osd_in_since":1726043886,"num_remapped_pgs":0} 2024-09-11T08:38:27.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:38:27.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:27 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2824427858' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:38:27.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:27 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2824427858' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:38:27.329 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:27.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:27 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2824427858' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:38:28.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:28 smithi092 ceph-mon[29647]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:28.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:28 smithi092 ceph-mon[41114]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:28.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:28 smithi145 ceph-mon[33384]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:29.264 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:38:29.264 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":54,"fsid":"211625d4-7018-11ef-bcea-c7b262605968","created":"2024-09-11T08:33:32.174444+0000","modified":"2024-09-11T08:38:24.658419+0000","last_up_change":"2024-09-11T08:38:23.097480+0000","last_in_change":"2024-09-11T08:38:06.259586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:36:29.606460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"9d8a3811-e801-4b6a-9a63-83936283cb7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6803","nonce":2040613580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6805","nonce":2040613580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6809","nonce":2040613580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6807","nonce":2040613580}]},"public_addr":"172.21.15.92:6803/2040613580","cluster_addr":"172.21.15.92:6805/2040613580","heartbeat_back_addr":"172.21.15.92:6809/2040613580","heartbeat_front_addr":"172.21.15.92:6807/2040613580","state":["exists","up"]},{"osd":1,"uuid":"e7a18e45-0137-45ef-8b85-e458cf3737ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6811","nonce":544779328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6813","nonce":544779328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6817","nonce":544779328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6815","nonce":544779328}]},"public_addr":"172.21.15.92:6811/544779328","cluster_addr":"172.21.15.92:6813/544779328","heartbeat_back_addr":"172.21.15.92:6817/544779328","heartbeat_front_addr":"172.21.15.92:6815/544779328","state":["exists","up"]},{"osd":2,"uuid":"017282e2-8053-4fa7-affc-a05408fa481e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6819","nonce":933736777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6821","nonce":933736777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6825","nonce":933736777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6823","nonce":933736777}]},"public_addr":"172.21.15.92:6819/933736777","cluster_addr":"172.21.15.92:6821/933736777","heartbeat_back_addr":"172.21.15.92:6825/933736777","heartbeat_front_addr":"172.21.15.92:6823/933736777","state":["exists","up"]},{"osd":3,"uuid":"58e5a8a4-9e58-40ad-9b40-90cb5ff5414b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6827","nonce":878892459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6829","nonce":878892459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6833","nonce":878892459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6831","nonce":878892459}]},"public_addr":"172.21.15.92:6827/878892459","cluster_addr":"172.21.15.92:6829/878892459","heartbeat_back_addr":"172.21.15.92:6833/878892459","heartbeat_front_addr":"172.21.15.92:6831/878892459","state":["exists","up"]},{"osd":4,"uuid":"e144e102-963b-48b7-92a9-9391706164bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6801","nonce":2792751669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6803","nonce":2792751669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6807","nonce":2792751669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6805","nonce":2792751669}]},"public_addr":"172.21.15.145:6801/2792751669","cluster_addr":"172.21.15.145:6803/2792751669","heartbeat_back_addr":"172.21.15.145:6807/2792751669","heartbeat_front_addr":"172.21.15.145:6805/2792751669","state":["exists","up"]},{"osd":5,"uuid":"811b0dc9-b7a7-439a-86c1-5dc9d0d54e16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6809","nonce":3569637188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6811","nonce":3569637188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6815","nonce":3569637188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6813","nonce":3569637188}]},"public_addr":"172.21.15.145:6809/3569637188","cluster_addr":"172.21.15.145:6811/3569637188","heartbeat_back_addr":"172.21.15.145:6815/3569637188","heartbeat_front_addr":"172.21.15.145:6813/3569637188","state":["exists","up"]},{"osd":6,"uuid":"bd162f52-097c-4c5d-9d37-1289779a196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6817","nonce":2615047110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6819","nonce":2615047110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6823","nonce":2615047110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6821","nonce":2615047110}]},"public_addr":"172.21.15.145:6817/2615047110","cluster_addr":"172.21.15.145:6819/2615047110","heartbeat_back_addr":"172.21.15.145:6823/2615047110","heartbeat_front_addr":"172.21.15.145:6821/2615047110","state":["exists","up"]},{"osd":7,"uuid":"4e4021bf-78c7-4bc6-b222-3494f724ddec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6825","nonce":3195898530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6827","nonce":3195898530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6831","nonce":3195898530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6829","nonce":3195898530}]},"public_addr":"172.21.15.145:6825/3195898530","cluster_addr":"172.21.15.145:6827/3195898530","heartbeat_back_addr":"172.21.15.145:6831/3195898530","heartbeat_front_addr":"172.21.15.145:6829/3195898530","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:37:58.662303+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:37:33.436117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:37:51.602696+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:38:16.555386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:38:24.003323+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:37:35.445079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:37:58.504953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:38:21.129568+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/12697418":"2024-09-12T08:34:17.306389+0000","172.21.15.92:6801/2480574848":"2024-09-12T08:34:17.306389+0000","172.21.15.92:6801/4060453592":"2024-09-12T08:33:54.195816+0000","172.21.15.92:6800/4060453592":"2024-09-12T08:33:54.195816+0000","172.21.15.92:6800/2480574848":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/2680744738":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/736380628":"2024-09-12T08:33:54.195816+0000","172.21.15.92:0/1583695466":"2024-09-12T08:33:54.195816+0000","172.21.15.92:0/4280958069":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/4113910808":"2024-09-12T08:33:54.195816+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:38:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:29 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2585789728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:38:30.011 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:29 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2585789728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:38:30.012 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:29 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2585789728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:38:30.038 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T08:36:29.606460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-11T08:38:30.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T08:38:30.281 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:31.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:30 smithi092 ceph-mon[29647]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:31.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:30 smithi092 ceph-mon[41114]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:31.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:30 smithi145 ceph-mon[33384]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:32.172 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-09-11T08:38:32.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:32 smithi092 ceph-mon[29647]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:32.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:32 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1424928358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:38:32.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:32 smithi092 ceph-mon[41114]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:32.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:32 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1424928358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:38:32.860 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi092 2024-09-11T08:38:32.860 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi092=foo.a' 2024-09-11T08:38:33.113 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:38:33.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:32 smithi145 ceph-mon[33384]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:33.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:32 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1424928358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:38:34.786 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:34 smithi145 ceph-mon[33384]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:34.931 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled rgw.foo.a update... 2024-09-11T08:38:35.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:34 smithi092 ceph-mon[29647]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:35.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:34 smithi092 ceph-mon[41114]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:35.464 DEBUG:teuthology.orchestra.run.smithi092:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-11T08:38:35.466 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi145 2024-09-11T08:38:35.467 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-11T08:38:35.704 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:38:36.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[29647]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi092=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:36.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[29647]: Saving service rgw.foo.a spec with placement smithi092=foo.a;count:1 2024-09-11T08:38:36.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:36.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:36.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:36.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:36.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:36.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:38:36.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:38:36.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:36.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[41114]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi092=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[41114]: Saving service rgw.foo.a spec with placement smithi092=foo.a;count:1 2024-09-11T08:38:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:38:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:38:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:35 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:36.076 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:35 smithi145 ceph-mon[33384]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi092=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:36.076 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:35 smithi145 ceph-mon[33384]: Saving service rgw.foo.a spec with placement smithi092=foo.a;count:1 2024-09-11T08:38:36.076 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:36.076 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:36.076 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:36.076 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:36.076 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:36.076 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:38:36.076 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:38:36.076 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:36.076 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:35 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:37.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:36 smithi092 ceph-mon[29647]: Deploying daemon rgw.foo.a on smithi092 2024-09-11T08:38:37.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:36 smithi092 ceph-mon[29647]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:37.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:36 smithi092 ceph-mon[41114]: Deploying daemon rgw.foo.a on smithi092 2024-09-11T08:38:37.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:36 smithi092 ceph-mon[41114]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:37.186 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:36 smithi145 ceph-mon[33384]: Deploying daemon rgw.foo.a on smithi092 2024-09-11T08:38:37.186 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:36 smithi145 ceph-mon[33384]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:37.469 INFO:journalctl@ceph.rgw.foo.a.smithi092.stdout:Sep 11 08:38:37 smithi092 podman[68799]: 2024-09-11 08:38:37.367850182 +0000 UTC m=+0.133699086 container create 83137f9367192a5a2175d1c1308cd9ea8d736611f6ca985e43454072112eb74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-11T08:38:37.935 INFO:journalctl@ceph.rgw.foo.a.smithi092.stdout:Sep 11 08:38:37 smithi092 podman[68799]: 2024-09-11 08:38:37.586620499 +0000 UTC m=+0.352469413 container init 83137f9367192a5a2175d1c1308cd9ea8d736611f6ca985e43454072112eb74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=) 2024-09-11T08:38:37.935 INFO:journalctl@ceph.rgw.foo.a.smithi092.stdout:Sep 11 08:38:37 smithi092 podman[68799]: 2024-09-11 08:38:37.591464688 +0000 UTC m=+0.357313592 container start 83137f9367192a5a2175d1c1308cd9ea8d736611f6ca985e43454072112eb74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-rgw-foo-a, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T08:38:37.935 INFO:journalctl@ceph.rgw.foo.a.smithi092.stdout:Sep 11 08:38:37 smithi092 bash[68799]: 83137f9367192a5a2175d1c1308cd9ea8d736611f6ca985e43454072112eb74c 2024-09-11T08:38:37.935 INFO:journalctl@ceph.rgw.foo.a.smithi092.stdout:Sep 11 08:38:37 smithi092 systemd[1]: Started Ceph rgw.foo.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:38:37.942 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'datapool' created 2024-09-11T08:38:38.271 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:37 smithi145 ceph-mon[33384]: from='client.? 172.21.15.145:0/3759314708' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:38:38.271 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:37 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:38:38.272 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:37 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:38.272 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:37 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:38.272 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:37 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:38.272 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:37 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:38.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:37 smithi092 ceph-mon[41114]: from='client.? 172.21.15.145:0/3759314708' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:38:38.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:37 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:38:38.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:37 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:38.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:37 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:38.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:37 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:38.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:37 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:38.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:37 smithi092 ceph-mon[29647]: from='client.? 172.21.15.145:0/3759314708' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:38:38.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:37 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:38:38.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:37 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:38.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:37 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:38.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:37 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:38.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:37 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:38.538 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-11T08:38:38.781 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:38:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:38 smithi145 ceph-mon[33384]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:38 smithi145 ceph-mon[33384]: Saving service rgw.foo.a spec with placement smithi092=foo.a;count:1 2024-09-11T08:38:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:38 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:38:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:38 smithi145 ceph-mon[33384]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:38:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:38 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:38:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:38 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:39.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:38 smithi092 ceph-mon[29647]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:39.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:38 smithi092 ceph-mon[29647]: Saving service rgw.foo.a spec with placement smithi092=foo.a;count:1 2024-09-11T08:38:39.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:39.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:38 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:38:39.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:38 smithi092 ceph-mon[29647]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:38:39.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:38 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:38:39.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:38 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:39.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:38 smithi092 ceph-mon[41114]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:39.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:38 smithi092 ceph-mon[41114]: Saving service rgw.foo.a spec with placement smithi092=foo.a;count:1 2024-09-11T08:38:39.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:39.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:38 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:38:39.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:38 smithi092 ceph-mon[41114]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:38:39.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:38 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:38:39.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:38 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:40.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:39 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:38:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:39 smithi145 ceph-mon[33384]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:38:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:39 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:39 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:39 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:39 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:39 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:38:40.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:39 smithi092 ceph-mon[29647]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:38:40.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:39 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:40.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:39 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:40.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:39 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:40.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:39 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:39 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:38:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:39 smithi092 ceph-mon[41114]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:38:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:39 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:39 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:40.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:39 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:40.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:39 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:41.216 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[29647]: pgmap v177: 36 pgs: 7 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:41.216 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[29647]: Checking dashboard <-> RGW credentials 2024-09-11T08:38:41.216 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[29647]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:38:41.216 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[29647]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:38:41.217 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:38:41.217 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1230553190' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:38:41.217 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[29647]: from='client.? 172.21.15.145:0/1526472091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:38:41.217 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:38:41.217 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[41114]: pgmap v177: 36 pgs: 7 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:41.217 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[41114]: Checking dashboard <-> RGW credentials 2024-09-11T08:38:41.217 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[41114]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:38:41.217 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[41114]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:38:41.217 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:38:41.217 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1230553190' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:38:41.218 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[41114]: from='client.? 172.21.15.145:0/1526472091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:38:41.218 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:40 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:38:41.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:40 smithi145 ceph-mon[33384]: pgmap v177: 36 pgs: 7 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:41.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:40 smithi145 ceph-mon[33384]: Checking dashboard <-> RGW credentials 2024-09-11T08:38:41.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:40 smithi145 ceph-mon[33384]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:38:41.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:40 smithi145 ceph-mon[33384]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:38:41.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:40 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:38:41.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:40 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1230553190' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:38:41.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:40 smithi145 ceph-mon[33384]: from='client.? 172.21.15.145:0/1526472091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:38:41.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:40 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:38:42.232 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:41 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:38:42.232 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:41 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1230553190' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:38:42.232 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:41 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:38:42.232 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:41 smithi092 ceph-mon[29647]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:38:42.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:41 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:38:42.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:41 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1230553190' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:38:42.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:41 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:38:42.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:41 smithi092 ceph-mon[41114]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:38:42.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:41 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:38:42.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:41 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1230553190' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:38:42.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:41 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:38:42.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:41 smithi145 ceph-mon[33384]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:38:43.295 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:42 smithi145 ceph-mon[33384]: pgmap v180: 68 pgs: 22 creating+peering, 17 active+clean, 29 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:38:43.295 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:42 smithi145 ceph-mon[33384]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:38:43.295 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:42 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:38:43.295 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:42 smithi145 ceph-mon[33384]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:38:43.295 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:42 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1230553190' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:38:43.295 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:42 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:38:43.295 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:42 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:42 smithi092 ceph-mon[29647]: pgmap v180: 68 pgs: 22 creating+peering, 17 active+clean, 29 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:38:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:42 smithi092 ceph-mon[29647]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:38:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:42 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:38:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:42 smithi092 ceph-mon[29647]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:38:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:42 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1230553190' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:38:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:42 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:38:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:42 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:43.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:42 smithi092 ceph-mon[41114]: pgmap v180: 68 pgs: 22 creating+peering, 17 active+clean, 29 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:38:43.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:42 smithi092 ceph-mon[41114]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:38:43.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:42 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:38:43.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:42 smithi092 ceph-mon[41114]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:38:43.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:42 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1230553190' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:38:43.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:42 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:38:43.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:42 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:43.531 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.145 --placement '1;smithi145=iscsi.a' 2024-09-11T08:38:43.775 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:38:44.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:43 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1230553190' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:38:44.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:43 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:38:44.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:43 smithi145 ceph-mon[33384]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:38:44.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:43 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1230553190' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:38:44.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:43 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:38:44.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:43 smithi092 ceph-mon[29647]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:38:44.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:43 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1230553190' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:38:44.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:43 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3990203193' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:38:44.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:43 smithi092 ceph-mon[41114]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:38:45.107 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:44 smithi145 ceph-mon[33384]: pgmap v183: 100 pgs: 20 creating+peering, 47 active+clean, 33 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-11T08:38:45.107 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:44 smithi145 ceph-mon[33384]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:38:45.107 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:44 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4147753376' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:38:45.107 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:44 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1175084397' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:38:45.107 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:44 smithi145 ceph-mon[33384]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:38:45.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:44 smithi092 ceph-mon[29647]: pgmap v183: 100 pgs: 20 creating+peering, 47 active+clean, 33 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-11T08:38:45.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:44 smithi092 ceph-mon[29647]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:38:45.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:44 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4147753376' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:38:45.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:44 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1175084397' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:38:45.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:44 smithi092 ceph-mon[29647]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:38:45.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:44 smithi092 ceph-mon[41114]: pgmap v183: 100 pgs: 20 creating+peering, 47 active+clean, 33 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-11T08:38:45.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:44 smithi092 ceph-mon[41114]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:38:45.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:44 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4147753376' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:38:45.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:44 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1175084397' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:38:45.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:44 smithi092 ceph-mon[41114]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:38:45.512 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled iscsi.datapool update... 2024-09-11T08:38:46.127 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:46 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4147753376' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:38:46.127 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:46 smithi145 ceph-mon[33384]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:38:46.127 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:46 smithi145 ceph-mon[33384]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:38:46.127 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:46 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4147753376' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:38:46.128 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:46 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1175084397' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:38:46.128 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:46 smithi145 ceph-mon[33384]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:38:46.128 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:46 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:46.157 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-11T08:38:46.157 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:38:46.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T08:38:46.195 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-11T08:38:46.195 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T08:38:46.224 DEBUG:teuthology.orchestra.run.smithi145:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-11T08:38:46.269 INFO:tasks.cephadm:Adding prometheus.a on smithi145 2024-09-11T08:38:46.269 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi145=a' 2024-09-11T08:38:46.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:46 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4147753376' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:38:46.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:46 smithi092 ceph-mon[29647]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:38:46.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:46 smithi092 ceph-mon[29647]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:38:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:46 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4147753376' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:38:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:46 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1175084397' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:38:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:46 smithi092 ceph-mon[29647]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:38:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:46 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:46 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4147753376' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:38:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:46 smithi092 ceph-mon[41114]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:38:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:46 smithi092 ceph-mon[41114]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:38:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:46 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4147753376' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:38:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:46 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1175084397' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:38:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:46 smithi092 ceph-mon[41114]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:38:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:46 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:47.143 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:38:47.265 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:47 smithi145 ceph-mon[33384]: pgmap v186: 132 pgs: 1 creating+activating, 19 creating+peering, 87 active+clean, 25 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-11T08:38:47.265 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:47 smithi145 ceph-mon[33384]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.145", "placement": "1;smithi145=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:47.265 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:47 smithi145 ceph-mon[33384]: Saving service iscsi.datapool spec with placement smithi145=iscsi.a;count:1 2024-09-11T08:38:47.265 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:47 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4147753376' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:38:47.265 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:47 smithi145 ceph-mon[33384]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:38:47.265 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:47 smithi145 ceph-mon[33384]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:38:47.265 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:47 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:47.266 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:47 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:47.266 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:47 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:47.266 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:47 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:38:47.266 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:47 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:38:47.266 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:47 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:47.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[29647]: pgmap v186: 132 pgs: 1 creating+activating, 19 creating+peering, 87 active+clean, 25 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-11T08:38:47.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[29647]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.145", "placement": "1;smithi145=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:47.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[29647]: Saving service iscsi.datapool spec with placement smithi145=iscsi.a;count:1 2024-09-11T08:38:47.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4147753376' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:38:47.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[29647]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:38:47.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[29647]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:38:47.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:47.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:47.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:47.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:38:47.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:38:47.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:47.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[41114]: pgmap v186: 132 pgs: 1 creating+activating, 19 creating+peering, 87 active+clean, 25 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-11T08:38:47.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[41114]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.145", "placement": "1;smithi145=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:47.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[41114]: Saving service iscsi.datapool spec with placement smithi145=iscsi.a;count:1 2024-09-11T08:38:47.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4147753376' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:38:47.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[41114]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:38:47.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[41114]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:38:47.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:47.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:47.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:47.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:38:47.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:38:47.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:47 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:48.299 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:48 smithi145 ceph-mon[33384]: Deploying daemon iscsi.iscsi.a on smithi145 2024-09-11T08:38:48.299 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:48 smithi145 systemd[1]: Starting Ceph iscsi.iscsi.a for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T08:38:48.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:48 smithi092 ceph-mon[29647]: Deploying daemon iscsi.iscsi.a on smithi145 2024-09-11T08:38:48.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:48 smithi092 ceph-mon[41114]: Deploying daemon iscsi.iscsi.a on smithi145 2024-09-11T08:38:49.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:49 smithi145 ceph-mon[33384]: pgmap v188: 132 pgs: 1 creating+activating, 14 creating+peering, 107 active+clean, 10 unknown; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 2.0 KiB/s wr, 44 op/s 2024-09-11T08:38:49.178 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled prometheus update... 2024-09-11T08:38:49.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:49 smithi092 ceph-mon[29647]: pgmap v188: 132 pgs: 1 creating+activating, 14 creating+peering, 107 active+clean, 10 unknown; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 2.0 KiB/s wr, 44 op/s 2024-09-11T08:38:49.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:49 smithi092 ceph-mon[41114]: pgmap v188: 132 pgs: 1 creating+activating, 14 creating+peering, 107 active+clean, 10 unknown; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 2.0 KiB/s wr, 44 op/s 2024-09-11T08:38:49.633 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:49 smithi145 podman[62415]: 2024-09-11 08:38:49.127205945 +0000 UTC m=+0.017825358 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:38:49.633 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:49 smithi145 podman[62415]: 2024-09-11 08:38:49.229225062 +0000 UTC m=+0.119844461 container create d513187baac9a3e6e430da0be54c3acf39476921565b59802636792c8fb0132e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:38:49.909 DEBUG:teuthology.orchestra.run.smithi145:prometheus.a> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-11T08:38:49.912 INFO:tasks.cephadm:Adding node-exporter.a on smithi092 2024-09-11T08:38:49.912 INFO:tasks.cephadm:Adding node-exporter.b on smithi145 2024-09-11T08:38:49.912 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi092=a;smithi145=b' 2024-09-11T08:38:50.162 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:50 smithi145 podman[62415]: 2024-09-11 08:38:50.068720341 +0000 UTC m=+0.959339741 container init d513187baac9a3e6e430da0be54c3acf39476921565b59802636792c8fb0132e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-11T08:38:50.162 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:50 smithi145 podman[62415]: 2024-09-11 08:38:50.073817889 +0000 UTC m=+0.964437293 container start d513187baac9a3e6e430da0be54c3acf39476921565b59802636792c8fb0132e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-11T08:38:50.162 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:50 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu[62462]: log file path now is '/var/log/tcmu-runner.log' 2024-09-11T08:38:50.257 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:38:50.491 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:50 smithi145 ceph-mon[33384]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi145=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:50.491 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:50 smithi145 ceph-mon[33384]: Saving service prometheus spec with placement smithi145=a;count:1 2024-09-11T08:38:50.491 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:50 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:50.491 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:50 smithi145 ceph-mon[33384]: pgmap v189: 132 pgs: 1 creating+activating, 9 creating+peering, 122 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 18 KiB/s rd, 1.8 KiB/s wr, 43 op/s 2024-09-11T08:38:50.492 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:50 smithi145 bash[62415]: d513187baac9a3e6e430da0be54c3acf39476921565b59802636792c8fb0132e 2024-09-11T08:38:50.492 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:50 smithi145 podman[62606]: 2024-09-11 08:38:50.280203768 +0000 UTC m=+0.022027257 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:38:50.492 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:50 smithi145 podman[62606]: 2024-09-11 08:38:50.386342967 +0000 UTC m=+0.128166420 container create d07f9d6332065432b61b7f59ebe9920edb3d0ded130bb854fa8ae00dcd1789f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-11T08:38:50.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:50 smithi092 ceph-mon[41114]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi145=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:50.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:50 smithi092 ceph-mon[41114]: Saving service prometheus spec with placement smithi145=a;count:1 2024-09-11T08:38:50.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:50 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:50.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:50 smithi092 ceph-mon[41114]: pgmap v189: 132 pgs: 1 creating+activating, 9 creating+peering, 122 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 18 KiB/s rd, 1.8 KiB/s wr, 43 op/s 2024-09-11T08:38:50.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:50 smithi092 ceph-mon[29647]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi145=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:50.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:50 smithi092 ceph-mon[29647]: Saving service prometheus spec with placement smithi145=a;count:1 2024-09-11T08:38:50.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:50 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:50.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:50 smithi092 ceph-mon[29647]: pgmap v189: 132 pgs: 1 creating+activating, 9 creating+peering, 122 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 18 KiB/s rd, 1.8 KiB/s wr, 43 op/s 2024-09-11T08:38:51.124 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:50 smithi145 podman[62606]: 2024-09-11 08:38:50.841697159 +0000 UTC m=+0.583520611 container init d07f9d6332065432b61b7f59ebe9920edb3d0ded130bb854fa8ae00dcd1789f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T08:38:51.125 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:50 smithi145 podman[62606]: 2024-09-11 08:38:50.84678115 +0000 UTC m=+0.588604604 container start d07f9d6332065432b61b7f59ebe9920edb3d0ded130bb854fa8ae00dcd1789f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:38:51.390 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug Started the configuration object watcher 2024-09-11T08:38:51.390 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug Checking for config object changes every 1s 2024-09-11T08:38:51.391 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug Processing osd blocklist entries for this node 2024-09-11T08:38:51.391 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 bash[62606]: d07f9d6332065432b61b7f59ebe9920edb3d0ded130bb854fa8ae00dcd1789f3 2024-09-11T08:38:51.391 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 systemd[1]: Started Ceph iscsi.iscsi.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:38:51.391 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug Reading the configuration object to update local LIO configuration 2024-09-11T08:38:51.391 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug Configuration does not have an entry for this host(smithi145.front.sepia.ceph.com) - nothing to define to LIO 2024-09-11T08:38:51.391 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-11T08:38:51.391 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: * Environment: production 2024-09-11T08:38:51.391 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:38:51.391 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: Use a production WSGI server instead. 2024-09-11T08:38:51.391 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: * Debug mode: off 2024-09-11T08:38:51.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-mon[33384]: from='client.? 172.21.15.145:0/2254105876' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:38:51.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug * Running on all addresses. 2024-09-11T08:38:51.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:38:51.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: * Running on all addresses. 2024-09-11T08:38:51.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:38:51.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug * Running on http://[fe80::ec4:7aff:fe88:7791]:5000/ (Press CTRL+C to quit) 2024-09-11T08:38:51.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:38:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: * Running on http://[fe80::ec4:7aff:fe88:7791]:5000/ (Press CTRL+C to quit) 2024-09-11T08:38:51.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:51 smithi092 ceph-mon[29647]: from='client.? 172.21.15.145:0/2254105876' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:38:51.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:51 smithi092 ceph-mon[41114]: from='client.? 172.21.15.145:0/2254105876' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:38:52.385 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled node-exporter update... 2024-09-11T08:38:52.646 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:52 smithi145 ceph-mon[33384]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 63 KiB/s rd, 5.0 KiB/s wr, 152 op/s 2024-09-11T08:38:52.647 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:52 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.647 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:52 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.647 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:52 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.647 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:52 smithi145 ceph-mon[33384]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:38:52.647 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:52 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.647 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:52 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:52.647 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:52 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.647 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:52 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[29647]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 63 KiB/s rd, 5.0 KiB/s wr, 152 op/s 2024-09-11T08:38:52.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[29647]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:38:52.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:52.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[41114]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 63 KiB/s rd, 5.0 KiB/s wr, 152 op/s 2024-09-11T08:38:52.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[41114]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:38:52.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:52.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:52.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:52 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:53.087 DEBUG:teuthology.orchestra.run.smithi092:node-exporter.a> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-11T08:38:53.090 DEBUG:teuthology.orchestra.run.smithi145:node-exporter.b> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-11T08:38:53.093 INFO:tasks.cephadm:Adding alertmanager.a on smithi092 2024-09-11T08:38:53.093 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi092=a' 2024-09-11T08:38:53.613 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:38:53.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:53 smithi145 ceph-mon[33384]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi092=a;smithi145=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:53.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:53 smithi145 ceph-mon[33384]: Saving service node-exporter spec with placement smithi092=a;smithi145=b;count:2 2024-09-11T08:38:53.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:53 smithi092 ceph-mon[29647]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi092=a;smithi145=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:53.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:53 smithi092 ceph-mon[29647]: Saving service node-exporter spec with placement smithi092=a;smithi145=b;count:2 2024-09-11T08:38:53.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:53 smithi092 ceph-mon[41114]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi092=a;smithi145=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:53.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:53 smithi092 ceph-mon[41114]: Saving service node-exporter spec with placement smithi092=a;smithi145=b;count:2 2024-09-11T08:38:54.655 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:54 smithi145 ceph-mon[33384]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 55 KiB/s rd, 4.4 KiB/s wr, 135 op/s 2024-09-11T08:38:54.655 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:54 smithi145 ceph-mon[33384]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:38:54.656 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:54 smithi145 ceph-mon[33384]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:38:54.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:54 smithi092 ceph-mon[29647]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 55 KiB/s rd, 4.4 KiB/s wr, 135 op/s 2024-09-11T08:38:54.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:54 smithi092 ceph-mon[29647]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:38:54.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:54 smithi092 ceph-mon[29647]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:38:54.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:54 smithi092 ceph-mon[41114]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 55 KiB/s rd, 4.4 KiB/s wr, 135 op/s 2024-09-11T08:38:54.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:54 smithi092 ceph-mon[41114]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:38:54.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:54 smithi092 ceph-mon[41114]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:38:55.372 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled alertmanager update... 2024-09-11T08:38:56.023 DEBUG:teuthology.orchestra.run.smithi092:alertmanager.a> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-11T08:38:56.026 INFO:tasks.cephadm:Adding grafana.a on smithi145 2024-09-11T08:38:56.026 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi145=a' 2024-09-11T08:38:56.601 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:38:56.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:56 smithi145 ceph-mon[33384]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 3.8 KiB/s wr, 120 op/s 2024-09-11T08:38:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:56 smithi145 ceph-mon[33384]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi092=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:56 smithi145 ceph-mon[33384]: Saving service alertmanager spec with placement smithi092=a;count:1 2024-09-11T08:38:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:56 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:56.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:56 smithi092 ceph-mon[29647]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 3.8 KiB/s wr, 120 op/s 2024-09-11T08:38:56.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:56 smithi092 ceph-mon[29647]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi092=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:56.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:56 smithi092 ceph-mon[29647]: Saving service alertmanager spec with placement smithi092=a;count:1 2024-09-11T08:38:56.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:56 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:56.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:56 smithi092 ceph-mon[41114]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 3.8 KiB/s wr, 120 op/s 2024-09-11T08:38:56.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:56 smithi092 ceph-mon[41114]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi092=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:56.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:56 smithi092 ceph-mon[41114]: Saving service alertmanager spec with placement smithi092=a;count:1 2024-09-11T08:38:56.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:56 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:57.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:57 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:57 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:57 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:57 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:57 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:57 smithi145 ceph-mon[33384]: Deploying daemon prometheus.a on smithi145 2024-09-11T08:38:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:57 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:57.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:57 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:57 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:57 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:57 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:57 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:57 smithi092 ceph-mon[29647]: Deploying daemon prometheus.a on smithi145 2024-09-11T08:38:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:57 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:57 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:57 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:57 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:57 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:57 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:57 smithi092 ceph-mon[41114]: Deploying daemon prometheus.a on smithi145 2024-09-11T08:38:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:57 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:58.370 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled grafana update... 2024-09-11T08:38:58.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:58 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:38:58.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:58 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:38:58.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:58 smithi145 ceph-mon[33384]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 39 KiB/s rd, 3.1 KiB/s wr, 95 op/s 2024-09-11T08:38:58.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:58 smithi145 ceph-mon[33384]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:38:58.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:58 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:58 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:38:58.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:58 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:38:58.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:58 smithi092 ceph-mon[29647]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 39 KiB/s rd, 3.1 KiB/s wr, 95 op/s 2024-09-11T08:38:58.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:58 smithi092 ceph-mon[29647]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:38:58.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:58 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:58 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:38:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:58 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:38:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:58 smithi092 ceph-mon[41114]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 39 KiB/s rd, 3.1 KiB/s wr, 95 op/s 2024-09-11T08:38:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:58 smithi092 ceph-mon[41114]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:38:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:58 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:38:59.208 DEBUG:teuthology.orchestra.run.smithi145:grafana.a> sudo journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@grafana.a.service 2024-09-11T08:38:59.210 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T08:38:59.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T08:38:59.470 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:59.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:59 smithi092 ceph-mon[29647]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi145=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:59.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:38:59 smithi092 ceph-mon[29647]: Saving service grafana spec with placement smithi145=a;count:1 2024-09-11T08:38:59.925 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:59 smithi092 ceph-mon[41114]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi145=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:59.925 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:38:59 smithi092 ceph-mon[41114]: Saving service grafana spec with placement smithi145=a;count:1 2024-09-11T08:39:00.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:59 smithi145 ceph-mon[33384]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi145=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:39:00.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:38:59 smithi145 ceph-mon[33384]: Saving service grafana spec with placement smithi145=a;count:1 2024-09-11T08:39:01.010 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:00 smithi092 ceph-mon[29647]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 127 B/s wr, 1 op/s 2024-09-11T08:39:01.014 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:00 smithi092 ceph-mon[41114]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 127 B/s wr, 1 op/s 2024-09-11T08:39:01.143 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:00 smithi145 ceph-mon[33384]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 127 B/s wr, 1 op/s 2024-09-11T08:39:01.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:39:01 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:39:01.443 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2024-09-11T08:39:01.443 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQAlV+FmKa05GhAAKrerjEiYt3AfMgkGI4/aoQ== 2024-09-11T08:39:01.793 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:01 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3206149007' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:39:01.793 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:01 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3206149007' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:39:01.793 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:01 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3206149007' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:39:01.794 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:01 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3206149007' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:39:02.116 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-11T08:39:02.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T08:39:02.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T08:39:02.152 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T08:39:02.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:01 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3206149007' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:39:02.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:01 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3206149007' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:39:02.441 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:39:02.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:02 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:02.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:02 smithi145 ceph-mon[33384]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:39:03.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:02 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:03.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:02 smithi092 ceph-mon[29647]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:39:03.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:02 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:03.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:02 smithi092 ceph-mon[41114]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:39:05.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:04 smithi092 ceph-mon[29647]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:39:05.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:04 smithi092 ceph-mon[41114]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:39:05.069 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:04 smithi145 ceph-mon[33384]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:39:05.562 INFO:teuthology.orchestra.run.smithi145.stdout:[client.1] 2024-09-11T08:39:05.562 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQApV+FmqChSIRAAYj9+B4fFm5XJfg+d/Gxb2Q== 2024-09-11T08:39:06.002 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:05 smithi145 ceph-mon[33384]: from='client.? 172.21.15.145:0/4287285748' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:39:06.003 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:05 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:39:06.003 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:05 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:39:06.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:05 smithi092 ceph-mon[29647]: from='client.? 172.21.15.145:0/4287285748' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:39:06.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:05 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:39:06.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:05 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:39:06.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:05 smithi092 ceph-mon[41114]: from='client.? 172.21.15.145:0/4287285748' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:39:06.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:05 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:39:06.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:05 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:39:07.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:06 smithi092 ceph-mon[29647]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:39:07.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:06 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:39:07.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:06 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:39:07.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:06 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:39:07.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:06 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:39:07.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:06 smithi092 ceph-mon[29647]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:39:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:06 smithi092 ceph-mon[41114]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:39:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:06 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:39:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:06 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:39:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:06 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:39:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:06 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:39:07.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:06 smithi092 ceph-mon[41114]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:39:07.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:06 smithi145 ceph-mon[33384]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:39:07.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:06 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:39:07.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:06 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:39:07.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:06 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:39:07.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:06 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:39:07.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:06 smithi145 ceph-mon[33384]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:39:08.860 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-11T08:39:08.860 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-11T08:39:08.860 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-11T08:39:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:08 smithi145 ceph-mon[33384]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:39:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:08 smithi145 ceph-mon[33384]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T08:39:08.943 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T08:39:08.943 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T08:39:08.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T08:39:08.971 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:08 smithi092 ceph-mon[29647]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:39:08.971 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:08 smithi092 ceph-mon[29647]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T08:39:08.971 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:08 smithi092 ceph-mon[41114]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:39:08.971 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:08 smithi092 ceph-mon[41114]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T08:39:09.186 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 podman[64501]: 2024-09-11 08:39:09.080386502 +0000 UTC m=+0.130478492 container create c5d6ca7549fa7d0af93af4010d652147caa225b6004091609d23d17160020799 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:09.200 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:09.662 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 podman[64501]: 2024-09-11 08:39:09.274338751 +0000 UTC m=+0.324430699 container init c5d6ca7549fa7d0af93af4010d652147caa225b6004091609d23d17160020799 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:09.662 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 podman[64501]: 2024-09-11 08:39:09.278822338 +0000 UTC m=+0.328914297 container start c5d6ca7549fa7d0af93af4010d652147caa225b6004091609d23d17160020799 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:09.663 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.301Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T08:39:09.663 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.302Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T08:39:09.663 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.302Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi145 (none))" 2024-09-11T08:39:09.663 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.302Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T08:39:09.663 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.302Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T08:39:09.663 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.303Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T08:39:09.663 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.303Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T08:39:09.663 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.305Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T08:39:09.663 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.305Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T08:39:09.664 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.307Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T08:39:09.664 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.307Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.509µs 2024-09-11T08:39:09.664 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.307Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T08:39:09.664 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.307Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-11T08:39:09.664 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.307Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=16.52µs wal_replay_duration=196.365µs wbl_replay_duration=87ns total_replay_duration=230.394µs 2024-09-11T08:39:09.664 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.308Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T08:39:09.664 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.308Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T08:39:09.664 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.308Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T08:39:09.664 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.323Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.057566ms db_storage=670ns remote_storage=1.038µs web_handler=231ns query_engine=544ns scrape=472.787µs scrape_sd=17.299µs notify=711ns notify_sd=1.86µs rules=14.381972ms tracing=5.901µs 2024-09-11T08:39:09.664 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.323Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T08:39:09.665 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:39:09.323Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T08:39:09.665 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 bash[64501]: c5d6ca7549fa7d0af93af4010d652147caa225b6004091609d23d17160020799 2024-09-11T08:39:09.665 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:39:09 smithi145 systemd[1]: Started Ceph prometheus.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:39:10.695 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:10 smithi145 ceph-mon[33384]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:10.695 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:10 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:39:10.695 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:10 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:39:10.695 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:10 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:39:10.695 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:10 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:39:10.696 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:10 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ignoring --setuser ceph since I am not root 2024-09-11T08:39:10.696 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:10 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ignoring --setgroup ceph since I am not root 2024-09-11T08:39:10.696 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:10 smithi145 ceph-mgr[35413]: -- 172.21.15.145:0/1225906806 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f1f8f00340 con 0x55f1f8f0d000 2024-09-11T08:39:10.696 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:10 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:10.576+0000 7f82d8493ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:39:10.696 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:10 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:10.630+0000 7f82d8493ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:39:10.696 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:10 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:10.695+0000 7f82d8493ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:39:10.703 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-mon[29647]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:10.703 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:39:10.703 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:39:10.703 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:39:10.703 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:39:10.703 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-mon[41114]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:10.703 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:39:10.703 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:39:10.704 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' 2024-09-11T08:39:10.704 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:39:10.704 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ignoring --setuser ceph since I am not root 2024-09-11T08:39:10.704 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ignoring --setgroup ceph since I am not root 2024-09-11T08:39:10.704 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-mgr[29921]: -- 172.21.15.92:0/3680606620 <== mon.2 v2:172.21.15.92:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aebf82a340 con 0x55aebf836800 2024-09-11T08:39:10.704 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:10.578+0000 7fe4c5ac6ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:39:10.704 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:10.635+0000 7fe4c5ac6ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:39:10.704 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:10.701+0000 7fe4c5ac6ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:39:10.951 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:10 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:10.817+0000 7f82d8493ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:39:10.979 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:10.835+0000 7fe4c5ac6ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:39:10.979 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:10 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:10.977+0000 7fe4c5ac6ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:39:11.205 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:39:11.330 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:10 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:10.951+0000 7f82d8493ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:39:11.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:11 smithi092 ceph-mon[29647]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:39:11.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:11 smithi092 ceph-mon[29647]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:39:11.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:11 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3693788147' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:39:11.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:11 smithi092 ceph-mon[41114]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:39:11.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:11 smithi092 ceph-mon[41114]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:39:11.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:11 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3693788147' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:39:11.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:11.242+0000 7fe4c5ac6ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:39:11.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:11.304+0000 7fe4c5ac6ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:39:11.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:11.468+0000 7fe4c5ac6ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:39:11.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:11 smithi145 ceph-mon[33384]: from='mgr.14150 172.21.15.92:0/2693324172' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:39:11.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:11 smithi145 ceph-mon[33384]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:39:11.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:11 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3693788147' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:39:11.662 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:11.207+0000 7f82d8493ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:39:11.662 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:11.267+0000 7f82d8493ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:39:11.662 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:11.427+0000 7f82d8493ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:39:11.991 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1861854063},{"type":"v1","addr":"172.21.15.92:6801","nonce":1861854063}]},"active_addr":"172.21.15.92:6801/1861854063","active_change":"2024-09-11T08:34:17.306531+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24106,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.92:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":969221731}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":2234574262}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3501902574}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":1540079149}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":2686720854}]}]} 2024-09-11T08:39:11.995 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T08:39:11.995 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T08:39:11.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:39:12.289 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:12.661 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:12.265+0000 7f82d8493ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:39:12.661 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:12.329+0000 7f82d8493ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:39:12.719 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:12 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:12.345+0000 7fe4c5ac6ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:39:12.719 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:12 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:12.410+0000 7fe4c5ac6ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:39:13.318 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:13 smithi145 ceph-mon[33384]: 1.0 scrub starts 2024-09-11T08:39:13.318 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:13 smithi145 ceph-mon[33384]: 1.0 scrub ok 2024-09-11T08:39:13.318 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:13.027+0000 7f82d8493ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:39:13.318 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:13.139+0000 7f82d8493ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:39:13.318 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:13.259+0000 7f82d8493ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:39:13.439 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:13 smithi092 ceph-mon[29647]: 1.0 scrub starts 2024-09-11T08:39:13.440 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:13 smithi092 ceph-mon[29647]: 1.0 scrub ok 2024-09-11T08:39:13.440 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:13 smithi092 ceph-mon[41114]: 1.0 scrub starts 2024-09-11T08:39:13.440 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:13 smithi092 ceph-mon[41114]: 1.0 scrub ok 2024-09-11T08:39:13.440 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:13.144+0000 7fe4c5ac6ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:39:13.440 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:13.257+0000 7fe4c5ac6ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:39:13.441 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:13.379+0000 7fe4c5ac6ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:39:13.441 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:13.438+0000 7fe4c5ac6ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:39:13.661 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:13.318+0000 7f82d8493ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:39:13.662 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:13.442+0000 7f82d8493ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:39:13.662 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:13.509+0000 7f82d8493ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:39:13.662 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:13.567+0000 7f82d8493ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:39:13.702 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:13.566+0000 7fe4c5ac6ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:39:13.702 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:13.635+0000 7fe4c5ac6ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:39:13.702 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:13.701+0000 7fe4c5ac6ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:39:13.963 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:13.700+0000 7f82d8493ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:39:13.963 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:13.763+0000 7f82d8493ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:39:13.963 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:13.827+0000 7f82d8493ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:39:13.963 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:13.907+0000 7f82d8493ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:39:13.963 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:13.963+0000 7f82d8493ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:39:13.984 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:13.843+0000 7fe4c5ac6ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:39:13.984 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:13.913+0000 7fe4c5ac6ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:39:13.984 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:13.982+0000 7fe4c5ac6ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:39:14.241 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:14 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:14.069+0000 7fe4c5ac6ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:39:14.241 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:14 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:14.132+0000 7fe4c5ac6ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:39:14.241 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:14 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:14.194+0000 7fe4c5ac6ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:39:14.278 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:39:14.279 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":68,"fsid":"211625d4-7018-11ef-bcea-c7b262605968","created":"2024-09-11T08:33:32.174444+0000","modified":"2024-09-11T08:39:13.667757+0000","last_up_change":"2024-09-11T08:38:23.097480+0000","last_in_change":"2024-09-11T08:38:06.259586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:36:29.606460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-11T08:38:37.674272+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-11T08:38:37.912906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T08:38:39.234173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T08:38:41.216386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T08:38:43.285497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"9d8a3811-e801-4b6a-9a63-83936283cb7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6803","nonce":2040613580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6805","nonce":2040613580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6809","nonce":2040613580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6807","nonce":2040613580}]},"public_addr":"172.21.15.92:6803/2040613580","cluster_addr":"172.21.15.92:6805/2040613580","heartbeat_back_addr":"172.21.15.92:6809/2040613580","heartbeat_front_addr":"172.21.15.92:6807/2040613580","state":["exists","up"]},{"osd":1,"uuid":"e7a18e45-0137-45ef-8b85-e458cf3737ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6811","nonce":544779328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6813","nonce":544779328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6817","nonce":544779328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6815","nonce":544779328}]},"public_addr":"172.21.15.92:6811/544779328","cluster_addr":"172.21.15.92:6813/544779328","heartbeat_back_addr":"172.21.15.92:6817/544779328","heartbeat_front_addr":"172.21.15.92:6815/544779328","state":["exists","up"]},{"osd":2,"uuid":"017282e2-8053-4fa7-affc-a05408fa481e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6819","nonce":933736777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6821","nonce":933736777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6825","nonce":933736777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6823","nonce":933736777}]},"public_addr":"172.21.15.92:6819/933736777","cluster_addr":"172.21.15.92:6821/933736777","heartbeat_back_addr":"172.21.15.92:6825/933736777","heartbeat_front_addr":"172.21.15.92:6823/933736777","state":["exists","up"]},{"osd":3,"uuid":"58e5a8a4-9e58-40ad-9b40-90cb5ff5414b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6827","nonce":878892459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6829","nonce":878892459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6833","nonce":878892459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6831","nonce":878892459}]},"public_addr":"172.21.15.92:6827/878892459","cluster_addr":"172.21.15.92:6829/878892459","heartbeat_back_addr":"172.21.15.92:6833/878892459","heartbeat_front_addr":"172.21.15.92:6831/878892459","state":["exists","up"]},{"osd":4,"uuid":"e144e102-963b-48b7-92a9-9391706164bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6801","nonce":2792751669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6803","nonce":2792751669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6807","nonce":2792751669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6805","nonce":2792751669}]},"public_addr":"172.21.15.145:6801/2792751669","cluster_addr":"172.21.15.145:6803/2792751669","heartbeat_back_addr":"172.21.15.145:6807/2792751669","heartbeat_front_addr":"172.21.15.145:6805/2792751669","state":["exists","up"]},{"osd":5,"uuid":"811b0dc9-b7a7-439a-86c1-5dc9d0d54e16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6809","nonce":3569637188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6811","nonce":3569637188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6815","nonce":3569637188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6813","nonce":3569637188}]},"public_addr":"172.21.15.145:6809/3569637188","cluster_addr":"172.21.15.145:6811/3569637188","heartbeat_back_addr":"172.21.15.145:6815/3569637188","heartbeat_front_addr":"172.21.15.145:6813/3569637188","state":["exists","up"]},{"osd":6,"uuid":"bd162f52-097c-4c5d-9d37-1289779a196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6817","nonce":2615047110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6819","nonce":2615047110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6823","nonce":2615047110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6821","nonce":2615047110}]},"public_addr":"172.21.15.145:6817/2615047110","cluster_addr":"172.21.15.145:6819/2615047110","heartbeat_back_addr":"172.21.15.145:6823/2615047110","heartbeat_front_addr":"172.21.15.145:6821/2615047110","state":["exists","up"]},{"osd":7,"uuid":"4e4021bf-78c7-4bc6-b222-3494f724ddec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6825","nonce":3195898530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6827","nonce":3195898530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6831","nonce":3195898530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6829","nonce":3195898530}]},"public_addr":"172.21.15.145:6825/3195898530","cluster_addr":"172.21.15.145:6827/3195898530","heartbeat_back_addr":"172.21.15.145:6831/3195898530","heartbeat_front_addr":"172.21.15.145:6829/3195898530","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:39:05.564318+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:38:57.295592+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:37:51.602696+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:38:16.555386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:38:24.003323+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:39:05.626141+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:37:58.504953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:38:21.129568+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/12697418":"2024-09-12T08:34:17.306389+0000","172.21.15.92:6801/2480574848":"2024-09-12T08:34:17.306389+0000","172.21.15.92:6801/4060453592":"2024-09-12T08:33:54.195816+0000","172.21.15.92:6800/4060453592":"2024-09-12T08:33:54.195816+0000","172.21.15.92:6800/2480574848":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/2680744738":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/736380628":"2024-09-12T08:33:54.195816+0000","172.21.15.92:0/1583695466":"2024-09-12T08:33:54.195816+0000","172.21.15.92:0/4280958069":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/4113910808":"2024-09-12T08:33:54.195816+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:39:14.411 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:14 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:14.020+0000 7f82d8493ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:39:14.981 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:14 smithi092 ceph-mon[29647]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T08:39:14.981 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:14 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/295211280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:39:14.981 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:14 smithi092 ceph-mon[41114]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T08:39:14.981 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:14 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/295211280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:39:15.018 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T08:39:15.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:39:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:14 smithi145 ceph-mon[33384]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T08:39:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:14 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/295211280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:39:15.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:14 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:14.761+0000 7f82d8493ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:39:15.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:14 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: [11/Sep/2024:08:39:14] ENGINE Bus STARTING 2024-09-11T08:39:15.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:14 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: CherryPy Checker: 2024-09-11T08:39:15.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:14 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: The Application mounted at '' has an empty config. 2024-09-11T08:39:15.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:14 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: 2024-09-11T08:39:15.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:14 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: [11/Sep/2024:08:39:14] ENGINE Serving on http://:::9283 2024-09-11T08:39:15.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:14 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: [11/Sep/2024:08:39:14] ENGINE Bus STARTED 2024-09-11T08:39:15.298 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:14 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:14.981+0000 7fe4c5ac6ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:39:15.298 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:39:15] ENGINE Bus STARTING 2024-09-11T08:39:15.298 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: CherryPy Checker: 2024-09-11T08:39:15.298 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: The Application mounted at '' has an empty config. 2024-09-11T08:39:15.298 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:15.298 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:39:15] ENGINE Serving on http://:::9283 2024-09-11T08:39:15.298 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:39:15] ENGINE Bus STARTED 2024-09-11T08:39:15.349 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:15.893 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: Standby manager daemon x restarted 2024-09-11T08:39:15.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: Standby manager daemon x started 2024-09-11T08:39:15.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.? 172.21.15.145:0/3425412860' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:39:15.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.? 172.21.15.145:0/3425412860' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:39:15.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.? 172.21.15.145:0/3425412860' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:39:15.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.? 172.21.15.145:0/3425412860' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:39:15.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: Active manager daemon y restarted 2024-09-11T08:39:15.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: Activating manager daemon y 2024-09-11T08:39:15.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T08:39:15.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: mgrmap e18: y(active, starting, since 0.0116982s), standbys: x 2024-09-11T08:39:15.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:39:15.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:39:15.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:39:15.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:39:15.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:39:15.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:39:15.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:39:15.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:39:15.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:39:15.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:39:15.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:39:15.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:39:15.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:39:15.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:39:15.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:39:15.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:39:15.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: Manager daemon y is now available 2024-09-11T08:39:15.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:39:15.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:15.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: Standby manager daemon x restarted 2024-09-11T08:39:15.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: Standby manager daemon x started 2024-09-11T08:39:15.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.? 172.21.15.145:0/3425412860' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:39:15.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.? 172.21.15.145:0/3425412860' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:39:15.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.? 172.21.15.145:0/3425412860' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:39:15.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.? 172.21.15.145:0/3425412860' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:39:15.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: Active manager daemon y restarted 2024-09-11T08:39:15.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: Activating manager daemon y 2024-09-11T08:39:15.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T08:39:15.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: mgrmap e18: y(active, starting, since 0.0116982s), standbys: x 2024-09-11T08:39:15.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:39:15.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:39:15.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:39:15.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:39:15.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:39:15.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:39:15.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:39:15.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:39:15.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:39:15.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:39:15.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:39:15.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:39:15.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:39:15.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:39:15.898 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:39:15.898 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:39:15.898 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: Manager daemon y is now available 2024-09-11T08:39:15.898 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:39:15.898 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:15.898 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:15.898 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:39:15.898 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:39:15.898 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:39:15.898 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:39:15.899 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:15.899 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:39:15.899 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:39:15.899 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:39:15.899 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:15 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:39:15.985 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: Standby manager daemon x restarted 2024-09-11T08:39:15.985 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: Standby manager daemon x started 2024-09-11T08:39:15.985 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.? 172.21.15.145:0/3425412860' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:39:15.985 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.? 172.21.15.145:0/3425412860' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:39:15.985 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.? 172.21.15.145:0/3425412860' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:39:15.985 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.? 172.21.15.145:0/3425412860' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:39:15.985 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: Active manager daemon y restarted 2024-09-11T08:39:15.985 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: Activating manager daemon y 2024-09-11T08:39:15.985 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T08:39:15.985 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: mgrmap e18: y(active, starting, since 0.0116982s), standbys: x 2024-09-11T08:39:15.985 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:39:15.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:39:15.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:39:15.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:39:15.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:39:15.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:39:15.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:39:15.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:39:15.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:39:15.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:39:15.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:39:15.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:39:15.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:39:15.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:39:15.986 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:39:15.987 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:39:15.987 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: Manager daemon y is now available 2024-09-11T08:39:15.987 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:39:15.987 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:15.987 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:15.987 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:39:15.987 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:39:15.987 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:39:15.987 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:15 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:39:16.247 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:16.107+0000 7fe468848640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:39:16] ENGINE Error in HTTPServer.serve 2024-09-11T08:39:16.247 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: Traceback (most recent call last): 2024-09-11T08:39:16.247 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:39:16.247 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: self._connections.run(self.expiration_interval) 2024-09-11T08:39:16.247 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:39:16.247 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: self._run(expiration_interval) 2024-09-11T08:39:16.247 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:39:16.247 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:39:16.247 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:39:16.247 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:39:16.247 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:39:16.247 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: s = self.context.wrap_socket( 2024-09-11T08:39:16.248 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:39:16.248 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: return self.sslsocket_class._create( 2024-09-11T08:39:16.248 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:39:16.248 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: self.do_handshake() 2024-09-11T08:39:16.248 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:39:16.248 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: self._sslobj.do_handshake() 2024-09-11T08:39:16.248 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:39:16.248 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T08:39:16.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:39:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:39:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:16 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:39:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:16 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:39:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:16 smithi145 ceph-mon[33384]: [11/Sep/2024:08:39:15] ENGINE Bus STARTING 2024-09-11T08:39:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:16 smithi145 ceph-mon[33384]: mgrmap e19: y(active, since 1.0113s), standbys: x 2024-09-11T08:39:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:16 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:16 smithi145 ceph-mon[33384]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:16 smithi145 ceph-mon[33384]: [11/Sep/2024:08:39:16] ENGINE Serving on https://172.21.15.92:7150 2024-09-11T08:39:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:16 smithi145 ceph-mon[33384]: [11/Sep/2024:08:39:16] ENGINE Bus STARTED 2024-09-11T08:39:17.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:16 smithi145 ceph-mon[33384]: [11/Sep/2024:08:39:16] ENGINE Error in HTTPServer.serve 2024-09-11T08:39:17.163 INFO:journalctl@ceph.mon.b.smithi145.stdout: Traceback (most recent call last): 2024-09-11T08:39:17.163 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:39:17.163 INFO:journalctl@ceph.mon.b.smithi145.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:39:17.163 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:39:17.163 INFO:journalctl@ceph.mon.b.smithi145.stdout: self._run(expiration_interval) 2024-09-11T08:39:17.163 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:39:17.163 INFO:journalctl@ceph.mon.b.smithi145.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:39:17.163 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:39:17.163 INFO:journalctl@ceph.mon.b.smithi145.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:39:17.163 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:39:17.163 INFO:journalctl@ceph.mon.b.smithi145.stdout: s = self.context.wrap_socket( 2024-09-11T08:39:17.163 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:39:17.163 INFO:journalctl@ceph.mon.b.smithi145.stdout: return self.sslsocket_class._create( 2024-09-11T08:39:17.163 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:39:17.164 INFO:journalctl@ceph.mon.b.smithi145.stdout: self.do_handshake() 2024-09-11T08:39:17.164 INFO:journalctl@ceph.mon.b.smithi145.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:39:17.164 INFO:journalctl@ceph.mon.b.smithi145.stdout: self._sslobj.do_handshake() 2024-09-11T08:39:17.164 INFO:journalctl@ceph.mon.b.smithi145.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:39:17.164 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:16 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:17.164 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:16 smithi145 ceph-mon[33384]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T08:39:17.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:39:17.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:39:17.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[29647]: [11/Sep/2024:08:39:15] ENGINE Bus STARTING 2024-09-11T08:39:17.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[29647]: mgrmap e19: y(active, since 1.0113s), standbys: x 2024-09-11T08:39:17.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:17.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[29647]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:17.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[29647]: [11/Sep/2024:08:39:16] ENGINE Serving on https://172.21.15.92:7150 2024-09-11T08:39:17.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[29647]: [11/Sep/2024:08:39:16] ENGINE Bus STARTED 2024-09-11T08:39:17.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[29647]: [11/Sep/2024:08:39:16] ENGINE Error in HTTPServer.serve 2024-09-11T08:39:17.317 INFO:journalctl@ceph.mon.a.smithi092.stdout: Traceback (most recent call last): 2024-09-11T08:39:17.317 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:39:17.317 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:39:17.317 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:39:17.317 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._run(expiration_interval) 2024-09-11T08:39:17.317 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:39:17.317 INFO:journalctl@ceph.mon.a.smithi092.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:39:17.317 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:39:17.317 INFO:journalctl@ceph.mon.a.smithi092.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:39:17.317 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:39:17.317 INFO:journalctl@ceph.mon.a.smithi092.stdout: s = self.context.wrap_socket( 2024-09-11T08:39:17.317 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:39:17.317 INFO:journalctl@ceph.mon.a.smithi092.stdout: return self.sslsocket_class._create( 2024-09-11T08:39:17.318 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:39:17.318 INFO:journalctl@ceph.mon.a.smithi092.stdout: self.do_handshake() 2024-09-11T08:39:17.318 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:39:17.318 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._sslobj.do_handshake() 2024-09-11T08:39:17.318 INFO:journalctl@ceph.mon.a.smithi092.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:39:17.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:17.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[29647]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T08:39:17.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:39:17.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:39:17.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[41114]: [11/Sep/2024:08:39:15] ENGINE Bus STARTING 2024-09-11T08:39:17.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[41114]: mgrmap e19: y(active, since 1.0113s), standbys: x 2024-09-11T08:39:17.319 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:17.319 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[41114]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:17.319 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[41114]: [11/Sep/2024:08:39:16] ENGINE Serving on https://172.21.15.92:7150 2024-09-11T08:39:17.319 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[41114]: [11/Sep/2024:08:39:16] ENGINE Bus STARTED 2024-09-11T08:39:17.319 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[41114]: [11/Sep/2024:08:39:16] ENGINE Error in HTTPServer.serve 2024-09-11T08:39:17.319 INFO:journalctl@ceph.mon.c.smithi092.stdout: Traceback (most recent call last): 2024-09-11T08:39:17.319 INFO:journalctl@ceph.mon.c.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:39:17.319 INFO:journalctl@ceph.mon.c.smithi092.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:39:17.319 INFO:journalctl@ceph.mon.c.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:39:17.319 INFO:journalctl@ceph.mon.c.smithi092.stdout: self._run(expiration_interval) 2024-09-11T08:39:17.319 INFO:journalctl@ceph.mon.c.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:39:17.319 INFO:journalctl@ceph.mon.c.smithi092.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:39:17.319 INFO:journalctl@ceph.mon.c.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:39:17.320 INFO:journalctl@ceph.mon.c.smithi092.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:39:17.320 INFO:journalctl@ceph.mon.c.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:39:17.320 INFO:journalctl@ceph.mon.c.smithi092.stdout: s = self.context.wrap_socket( 2024-09-11T08:39:17.320 INFO:journalctl@ceph.mon.c.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:39:17.320 INFO:journalctl@ceph.mon.c.smithi092.stdout: return self.sslsocket_class._create( 2024-09-11T08:39:17.320 INFO:journalctl@ceph.mon.c.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:39:17.320 INFO:journalctl@ceph.mon.c.smithi092.stdout: self.do_handshake() 2024-09-11T08:39:17.320 INFO:journalctl@ceph.mon.c.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:39:17.320 INFO:journalctl@ceph.mon.c.smithi092.stdout: self._sslobj.do_handshake() 2024-09-11T08:39:17.320 INFO:journalctl@ceph.mon.c.smithi092.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:39:17.320 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:17.320 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:16 smithi092 ceph-mon[41114]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T08:39:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:39:17.372 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":70,"fsid":"211625d4-7018-11ef-bcea-c7b262605968","created":"2024-09-11T08:33:32.174444+0000","modified":"2024-09-11T08:39:16.671660+0000","last_up_change":"2024-09-11T08:38:23.097480+0000","last_in_change":"2024-09-11T08:38:06.259586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:36:29.606460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-11T08:38:37.674272+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-11T08:38:37.912906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T08:38:39.234173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T08:38:41.216386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T08:38:43.285497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"9d8a3811-e801-4b6a-9a63-83936283cb7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6803","nonce":2040613580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6805","nonce":2040613580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6809","nonce":2040613580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6807","nonce":2040613580}]},"public_addr":"172.21.15.92:6803/2040613580","cluster_addr":"172.21.15.92:6805/2040613580","heartbeat_back_addr":"172.21.15.92:6809/2040613580","heartbeat_front_addr":"172.21.15.92:6807/2040613580","state":["exists","up"]},{"osd":1,"uuid":"e7a18e45-0137-45ef-8b85-e458cf3737ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6811","nonce":544779328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6813","nonce":544779328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6817","nonce":544779328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6815","nonce":544779328}]},"public_addr":"172.21.15.92:6811/544779328","cluster_addr":"172.21.15.92:6813/544779328","heartbeat_back_addr":"172.21.15.92:6817/544779328","heartbeat_front_addr":"172.21.15.92:6815/544779328","state":["exists","up"]},{"osd":2,"uuid":"017282e2-8053-4fa7-affc-a05408fa481e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6819","nonce":933736777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6821","nonce":933736777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6825","nonce":933736777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6823","nonce":933736777}]},"public_addr":"172.21.15.92:6819/933736777","cluster_addr":"172.21.15.92:6821/933736777","heartbeat_back_addr":"172.21.15.92:6825/933736777","heartbeat_front_addr":"172.21.15.92:6823/933736777","state":["exists","up"]},{"osd":3,"uuid":"58e5a8a4-9e58-40ad-9b40-90cb5ff5414b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6827","nonce":878892459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6829","nonce":878892459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6833","nonce":878892459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6831","nonce":878892459}]},"public_addr":"172.21.15.92:6827/878892459","cluster_addr":"172.21.15.92:6829/878892459","heartbeat_back_addr":"172.21.15.92:6833/878892459","heartbeat_front_addr":"172.21.15.92:6831/878892459","state":["exists","up"]},{"osd":4,"uuid":"e144e102-963b-48b7-92a9-9391706164bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6801","nonce":2792751669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6803","nonce":2792751669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6807","nonce":2792751669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6805","nonce":2792751669}]},"public_addr":"172.21.15.145:6801/2792751669","cluster_addr":"172.21.15.145:6803/2792751669","heartbeat_back_addr":"172.21.15.145:6807/2792751669","heartbeat_front_addr":"172.21.15.145:6805/2792751669","state":["exists","up"]},{"osd":5,"uuid":"811b0dc9-b7a7-439a-86c1-5dc9d0d54e16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6809","nonce":3569637188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6811","nonce":3569637188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6815","nonce":3569637188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6813","nonce":3569637188}]},"public_addr":"172.21.15.145:6809/3569637188","cluster_addr":"172.21.15.145:6811/3569637188","heartbeat_back_addr":"172.21.15.145:6815/3569637188","heartbeat_front_addr":"172.21.15.145:6813/3569637188","state":["exists","up"]},{"osd":6,"uuid":"bd162f52-097c-4c5d-9d37-1289779a196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6817","nonce":2615047110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6819","nonce":2615047110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6823","nonce":2615047110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6821","nonce":2615047110}]},"public_addr":"172.21.15.145:6817/2615047110","cluster_addr":"172.21.15.145:6819/2615047110","heartbeat_back_addr":"172.21.15.145:6823/2615047110","heartbeat_front_addr":"172.21.15.145:6821/2615047110","state":["exists","up"]},{"osd":7,"uuid":"4e4021bf-78c7-4bc6-b222-3494f724ddec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6825","nonce":3195898530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6827","nonce":3195898530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6831","nonce":3195898530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6829","nonce":3195898530}]},"public_addr":"172.21.15.145:6825/3195898530","cluster_addr":"172.21.15.145:6827/3195898530","heartbeat_back_addr":"172.21.15.145:6831/3195898530","heartbeat_front_addr":"172.21.15.145:6829/3195898530","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:39:05.564318+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:38:57.295592+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:39:15.683927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:38:16.555386+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:38:24.003323+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:39:05.626141+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:37:58.504953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:38:21.129568+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6800/1861854063":"2024-09-12T08:39:14.984442+0000","172.21.15.92:0/4113910808":"2024-09-12T08:33:54.195816+0000","172.21.15.92:0/3501902574":"2024-09-12T08:39:14.984442+0000","172.21.15.92:0/1583695466":"2024-09-12T08:33:54.195816+0000","172.21.15.92:0/736380628":"2024-09-12T08:33:54.195816+0000","172.21.15.92:6800/4060453592":"2024-09-12T08:33:54.195816+0000","172.21.15.92:0/2234574262":"2024-09-12T08:39:14.984442+0000","172.21.15.92:6800/2480574848":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/1540079149":"2024-09-12T08:39:14.984442+0000","172.21.15.92:0/2680744738":"2024-09-12T08:34:17.306389+0000","172.21.15.92:6801/4060453592":"2024-09-12T08:33:54.195816+0000","172.21.15.92:6801/1861854063":"2024-09-12T08:39:14.984442+0000","172.21.15.92:6801/2480574848":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/12697418":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/4280958069":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/2686720854":"2024-09-12T08:39:14.984442+0000","172.21.15.92:0/969221731":"2024-09-12T08:39:14.984442+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:39:18.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:18 smithi092 ceph-mon[29647]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:18.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:18 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/422848886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:39:18.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:18 smithi092 ceph-mon[29647]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-11T08:39:18.291 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:18 smithi092 ceph-mon[41114]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:18.292 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:18 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/422848886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:39:18.292 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:18 smithi092 ceph-mon[41114]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-11T08:39:18.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T08:39:18.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T08:39:18.317 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T08:39:18.317 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T08:39:18.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T08:39:18.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T08:39:18.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T08:39:18.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T08:39:18.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:18 smithi145 ceph-mon[33384]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:18.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:18 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/422848886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:39:18.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:18 smithi145 ceph-mon[33384]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-11T08:39:18.867 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:18.872 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:18.873 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:18.876 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:18.886 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:18.928 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:18.940 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:18.940 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:19.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:19.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:19.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[29647]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T08:39:19.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:19.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:19.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:19.296 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:19.297 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:19.297 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[41114]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T08:39:19.297 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:19.297 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:19.297 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.297 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.297 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.297 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.297 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.297 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.297 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.297 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.297 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:19 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:19.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:19 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:19.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:19 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:19.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:19 smithi145 ceph-mon[33384]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T08:39:19.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:19 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:19.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:19 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:19.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:19 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:19 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:19 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:19 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:19 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:19 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:19 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:19 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:19.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:19 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:20.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:20 smithi092 ceph-mon[29647]: Adjusting osd_memory_target on smithi145 to 3760M 2024-09-11T08:39:20.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:20 smithi092 ceph-mon[29647]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:20.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:20 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:20.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:20 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:20.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:20 smithi092 ceph-mon[29647]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-11T08:39:20.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:20 smithi092 ceph-mon[41114]: Adjusting osd_memory_target on smithi145 to 3760M 2024-09-11T08:39:20.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:20 smithi092 ceph-mon[41114]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:20.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:20 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:20.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:20 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:20.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:20 smithi092 ceph-mon[41114]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-11T08:39:20.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:20 smithi145 ceph-mon[33384]: Adjusting osd_memory_target on smithi145 to 3760M 2024-09-11T08:39:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:20 smithi145 ceph-mon[33384]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:20 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:20 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:20 smithi145 ceph-mon[33384]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-11T08:39:22.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T08:39:22.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.724 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.724 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.724 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.724 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.724 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: Adjusting osd_memory_target on smithi092 to 3760M 2024-09-11T08:39:22.724 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.724 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:39:22.724 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:39:22.724 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-11T08:39:22.724 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-11T08:39:22.724 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.724 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.724 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.725 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.725 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.756 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T08:39:22.756 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.756 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.756 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.757 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.757 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.757 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.757 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.757 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.757 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.757 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.757 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: Adjusting osd_memory_target on smithi092 to 3760M 2024-09-11T08:39:22.757 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.757 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:39:22.757 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:39:22.757 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-11T08:39:22.757 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-11T08:39:22.758 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.758 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.758 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.758 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.758 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T08:39:22.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:39:22.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: Adjusting osd_memory_target on smithi092 to 3760M 2024-09-11T08:39:22.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:39:22.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:39:22.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-11T08:39:22.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: Updating smithi145:/etc/ceph/ceph.conf 2024-09-11T08:39:22.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.760 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.760 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:22.760 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:22 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:23.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:39:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:39:23.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:23 smithi092 ceph-mon[41114]: Updating smithi092:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:39:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:23 smithi092 ceph-mon[41114]: Updating smithi145:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:39:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:23 smithi092 ceph-mon[41114]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:39:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:23 smithi092 ceph-mon[41114]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:39:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:23 smithi092 ceph-mon[41114]: Updating smithi092:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:39:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:23 smithi092 ceph-mon[41114]: Updating smithi145:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:39:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:23 smithi092 ceph-mon[41114]: Deploying daemon alertmanager.a on smithi092 2024-09-11T08:39:23.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:23 smithi092 ceph-mon[29647]: Updating smithi092:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:39:23.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:23 smithi092 ceph-mon[29647]: Updating smithi145:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:39:23.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:23 smithi092 ceph-mon[29647]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:39:23.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:23 smithi092 ceph-mon[29647]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:39:23.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:23 smithi092 ceph-mon[29647]: Updating smithi092:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:39:23.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:23 smithi092 ceph-mon[29647]: Updating smithi145:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:39:23.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:23 smithi092 ceph-mon[29647]: Deploying daemon alertmanager.a on smithi092 2024-09-11T08:39:23.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:23 smithi145 ceph-mon[33384]: Updating smithi092:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:39:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:23 smithi145 ceph-mon[33384]: Updating smithi145:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:39:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:23 smithi145 ceph-mon[33384]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:39:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:23 smithi145 ceph-mon[33384]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:39:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:23 smithi145 ceph-mon[33384]: Updating smithi092:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:39:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:23 smithi145 ceph-mon[33384]: Updating smithi145:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:39:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:23 smithi145 ceph-mon[33384]: Deploying daemon alertmanager.a on smithi092 2024-09-11T08:39:24.749 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:24 smithi092 ceph-mon[29647]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:39:24.749 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:24 smithi092 ceph-mon[29647]: osdmap e72: 8 total, 8 up, 8 in 2024-09-11T08:39:24.749 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:24 smithi092 ceph-mon[41114]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:39:24.749 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:24 smithi092 ceph-mon[41114]: osdmap e72: 8 total, 8 up, 8 in 2024-09-11T08:39:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:24 smithi145 ceph-mon[33384]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:39:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:24 smithi145 ceph-mon[33384]: osdmap e72: 8 total, 8 up, 8 in 2024-09-11T08:39:25.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:25 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:39:25.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:25 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:39:25.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:25 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:39:25.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:25 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:39:25.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:25 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:39:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:25 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:39:26.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:39:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:39:26.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:26 smithi092 ceph-mon[29647]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-11T08:39:26.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:26 smithi092 ceph-mon[29647]: osdmap e73: 8 total, 8 up, 8 in 2024-09-11T08:39:26.669 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:26 smithi092 ceph-mon[41114]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-11T08:39:26.669 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:26 smithi092 ceph-mon[41114]: osdmap e73: 8 total, 8 up, 8 in 2024-09-11T08:39:26.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:26 smithi145 ceph-mon[33384]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-11T08:39:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:26 smithi145 ceph-mon[33384]: osdmap e73: 8 total, 8 up, 8 in 2024-09-11T08:39:27.075 INFO:teuthology.orchestra.run.smithi092.stdout:163208757271 2024-09-11T08:39:27.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T08:39:27.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:27 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:27.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:27 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:27.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:27 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:28.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:28 smithi092 ceph-mon[29647]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T08:39:28.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:28 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:39:28.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:28 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:39:28.616 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:28 smithi092 ceph-mon[29647]: osdmap e74: 8 total, 8 up, 8 in 2024-09-11T08:39:28.616 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:28 smithi092 ceph-mon[41114]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T08:39:28.616 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:28 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:39:28.616 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:28 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:39:28.616 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:28 smithi092 ceph-mon[41114]: osdmap e74: 8 total, 8 up, 8 in 2024-09-11T08:39:28.813 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:28 smithi145 ceph-mon[33384]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T08:39:28.813 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:28 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:39:28.813 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:28 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:39:28.813 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:28 smithi145 ceph-mon[33384]: osdmap e74: 8 total, 8 up, 8 in 2024-09-11T08:39:30.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:30 smithi092 ceph-mon[29647]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:30.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:30.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:30 smithi092 ceph-mon[41114]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:30.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:30.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:30 smithi145 ceph-mon[33384]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:30.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:32.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:32 smithi092 ceph-mon[29647]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:39:32.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:32 smithi092 ceph-mon[41114]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:39:32.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:32 smithi145 ceph-mon[33384]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:39:32.912 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:39:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:39:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:33 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:39:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:33 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:39:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:33 smithi092 ceph-mon[29647]: osdmap e75: 8 total, 8 up, 8 in 2024-09-11T08:39:33.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:33 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:39:33.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:33 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:39:33.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:33 smithi092 ceph-mon[41114]: osdmap e75: 8 total, 8 up, 8 in 2024-09-11T08:39:33.815 INFO:teuthology.orchestra.run.smithi092.stdout:111669149730 2024-09-11T08:39:33.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T08:39:33.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:33 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:39:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:33 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:39:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:33 smithi145 ceph-mon[33384]: osdmap e75: 8 total, 8 up, 8 in 2024-09-11T08:39:34.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:34 smithi092 ceph-mon[41114]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:39:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:34 smithi092 ceph-mon[41114]: osdmap e76: 8 total, 8 up, 8 in 2024-09-11T08:39:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:34 smithi092 ceph-mon[29647]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:39:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:34 smithi092 ceph-mon[29647]: osdmap e76: 8 total, 8 up, 8 in 2024-09-11T08:39:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:34 smithi145 ceph-mon[33384]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:39:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:34 smithi145 ceph-mon[33384]: osdmap e76: 8 total, 8 up, 8 in 2024-09-11T08:39:35.034 INFO:teuthology.orchestra.run.smithi092.stdout:55834574892 2024-09-11T08:39:35.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T08:39:35.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:35 smithi092 ceph-mon[29647]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:39:35.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:35 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:39:35.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:35 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:39:35.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:35 smithi092 ceph-mon[41114]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:39:35.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:35 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:39:35.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:35 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:39:35.682 INFO:teuthology.orchestra.run.smithi092.stdout:133143986205 2024-09-11T08:39:35.682 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T08:39:35.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:35 smithi145 ceph-mon[33384]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:39:35.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:35 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:39:35.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:35 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:39:36.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:39:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:39:36.570 INFO:teuthology.orchestra.run.smithi092.stdout:193273528341 2024-09-11T08:39:36.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T08:39:36.791 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:36 smithi092 ceph-mon[29647]: osdmap e77: 8 total, 8 up, 8 in 2024-09-11T08:39:36.791 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:36 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:36.792 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:36 smithi092 ceph-mon[41114]: osdmap e77: 8 total, 8 up, 8 in 2024-09-11T08:39:36.792 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:36 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:36.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:36 smithi145 ceph-mon[33384]: osdmap e77: 8 total, 8 up, 8 in 2024-09-11T08:39:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:36 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:37 smithi092 ceph-mon[29647]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:37 smithi092 ceph-mon[41114]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:37.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:37 smithi145 ceph-mon[33384]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:38.520 INFO:teuthology.orchestra.run.smithi092.stdout:223338299408 2024-09-11T08:39:38.520 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T08:39:39.167 INFO:teuthology.orchestra.run.smithi092.stdout:77309411368 2024-09-11T08:39:39.168 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T08:39:39.953 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:39 smithi092 ceph-mon[29647]: osdmap e78: 8 total, 8 up, 8 in 2024-09-11T08:39:39.953 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:39 smithi092 ceph-mon[29647]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:39.953 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:39 smithi092 ceph-mon[41114]: osdmap e78: 8 total, 8 up, 8 in 2024-09-11T08:39:39.953 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:39 smithi092 ceph-mon[41114]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:40.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:39 smithi145 ceph-mon[33384]: osdmap e78: 8 total, 8 up, 8 in 2024-09-11T08:39:40.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:39 smithi145 ceph-mon[33384]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:40.782 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:41.326 INFO:teuthology.orchestra.run.smithi092.stdout:34359738417 2024-09-11T08:39:41.326 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T08:39:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:42.038 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:42.039 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:42.042 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:42.169 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:42 smithi092 ceph-mon[41114]: 4.9 scrub starts 2024-09-11T08:39:42.169 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:42 smithi092 ceph-mon[41114]: 4.9 scrub ok 2024-09-11T08:39:42.169 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:42 smithi092 ceph-mon[41114]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 977 B/s rd, 0 op/s 2024-09-11T08:39:42.169 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:42 smithi092 ceph-mon[29647]: 4.9 scrub starts 2024-09-11T08:39:42.169 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:42 smithi092 ceph-mon[29647]: 4.9 scrub ok 2024-09-11T08:39:42.169 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:42 smithi092 ceph-mon[29647]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 977 B/s rd, 0 op/s 2024-09-11T08:39:42.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:42 smithi145 ceph-mon[33384]: 4.9 scrub starts 2024-09-11T08:39:42.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:42 smithi145 ceph-mon[33384]: 4.9 scrub ok 2024-09-11T08:39:42.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:42 smithi145 ceph-mon[33384]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 977 B/s rd, 0 op/s 2024-09-11T08:39:43.067 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:39:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:39:43.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:43 smithi145 ceph-mon[33384]: 3.6 scrub starts 2024-09-11T08:39:43.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:43 smithi145 ceph-mon[33384]: 3.6 scrub ok 2024-09-11T08:39:43.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:43 smithi145 ceph-mon[33384]: 4.10 scrub starts 2024-09-11T08:39:43.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:43 smithi145 ceph-mon[33384]: 4.10 scrub ok 2024-09-11T08:39:43.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:43 smithi092 ceph-mon[29647]: 3.6 scrub starts 2024-09-11T08:39:43.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:43 smithi092 ceph-mon[29647]: 3.6 scrub ok 2024-09-11T08:39:43.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:43 smithi092 ceph-mon[29647]: 4.10 scrub starts 2024-09-11T08:39:43.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:43 smithi092 ceph-mon[29647]: 4.10 scrub ok 2024-09-11T08:39:43.459 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:43 smithi092 ceph-mon[41114]: 3.6 scrub starts 2024-09-11T08:39:43.459 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:43 smithi092 ceph-mon[41114]: 3.6 scrub ok 2024-09-11T08:39:43.460 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:43 smithi092 ceph-mon[41114]: 4.10 scrub starts 2024-09-11T08:39:43.460 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:43 smithi092 ceph-mon[41114]: 4.10 scrub ok 2024-09-11T08:39:44.065 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:44 smithi092 systemd[1]: Starting Ceph alertmanager.a for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T08:39:44.377 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:44 smithi092 ceph-mon[29647]: 3.10 scrub starts 2024-09-11T08:39:44.377 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:44 smithi092 ceph-mon[29647]: 3.10 scrub ok 2024-09-11T08:39:44.377 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:44 smithi092 ceph-mon[29647]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 0 op/s 2024-09-11T08:39:44.377 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:44 smithi092 ceph-mon[29647]: 3.7 scrub starts 2024-09-11T08:39:44.377 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:44 smithi092 ceph-mon[29647]: 3.7 scrub ok 2024-09-11T08:39:44.378 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:44 smithi092 ceph-mon[41114]: 3.10 scrub starts 2024-09-11T08:39:44.378 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:44 smithi092 ceph-mon[41114]: 3.10 scrub ok 2024-09-11T08:39:44.378 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:44 smithi092 ceph-mon[41114]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 0 op/s 2024-09-11T08:39:44.378 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:44 smithi092 ceph-mon[41114]: 3.7 scrub starts 2024-09-11T08:39:44.378 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:44 smithi092 ceph-mon[41114]: 3.7 scrub ok 2024-09-11T08:39:44.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:44 smithi145 ceph-mon[33384]: 3.10 scrub starts 2024-09-11T08:39:44.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:44 smithi145 ceph-mon[33384]: 3.10 scrub ok 2024-09-11T08:39:44.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:44 smithi145 ceph-mon[33384]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 0 op/s 2024-09-11T08:39:44.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:44 smithi145 ceph-mon[33384]: 3.7 scrub starts 2024-09-11T08:39:44.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:44 smithi145 ceph-mon[33384]: 3.7 scrub ok 2024-09-11T08:39:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:45 smithi145 ceph-mon[33384]: 3.1 scrub starts 2024-09-11T08:39:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:45 smithi145 ceph-mon[33384]: 3.1 scrub ok 2024-09-11T08:39:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:45 smithi145 ceph-mon[33384]: 4.d scrub starts 2024-09-11T08:39:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:45 smithi145 ceph-mon[33384]: 4.d scrub ok 2024-09-11T08:39:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:45 smithi145 ceph-mon[33384]: 4.7 scrub starts 2024-09-11T08:39:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:45 smithi145 ceph-mon[33384]: 4.7 scrub ok 2024-09-11T08:39:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:45 smithi145 ceph-mon[33384]: 5.15 scrub starts 2024-09-11T08:39:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:45 smithi145 ceph-mon[33384]: 5.15 scrub ok 2024-09-11T08:39:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:45.481 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[29647]: 3.1 scrub starts 2024-09-11T08:39:45.481 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[29647]: 3.1 scrub ok 2024-09-11T08:39:45.481 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[29647]: 4.d scrub starts 2024-09-11T08:39:45.481 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[29647]: 4.d scrub ok 2024-09-11T08:39:45.481 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[29647]: 4.7 scrub starts 2024-09-11T08:39:45.481 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[29647]: 4.7 scrub ok 2024-09-11T08:39:45.481 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[29647]: 5.15 scrub starts 2024-09-11T08:39:45.481 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[29647]: 5.15 scrub ok 2024-09-11T08:39:45.481 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:45.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[41114]: 3.1 scrub starts 2024-09-11T08:39:45.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[41114]: 3.1 scrub ok 2024-09-11T08:39:45.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[41114]: 4.d scrub starts 2024-09-11T08:39:45.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[41114]: 4.d scrub ok 2024-09-11T08:39:45.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[41114]: 4.7 scrub starts 2024-09-11T08:39:45.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[41114]: 4.7 scrub ok 2024-09-11T08:39:45.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[41114]: 5.15 scrub starts 2024-09-11T08:39:45.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[41114]: 5.15 scrub ok 2024-09-11T08:39:45.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:45.930 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:46.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:46 smithi092 ceph-mon[29647]: 5.1d scrub starts 2024-09-11T08:39:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:46 smithi092 ceph-mon[29647]: 5.1d scrub ok 2024-09-11T08:39:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:46 smithi092 ceph-mon[29647]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 753 B/s rd, 0 op/s 2024-09-11T08:39:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:46 smithi092 ceph-mon[41114]: 5.1d scrub starts 2024-09-11T08:39:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:46 smithi092 ceph-mon[41114]: 5.1d scrub ok 2024-09-11T08:39:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:46 smithi092 ceph-mon[41114]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 753 B/s rd, 0 op/s 2024-09-11T08:39:46.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:39:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:39:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:46 smithi145 ceph-mon[33384]: 5.1d scrub starts 2024-09-11T08:39:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:46 smithi145 ceph-mon[33384]: 5.1d scrub ok 2024-09-11T08:39:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:46 smithi145 ceph-mon[33384]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 753 B/s rd, 0 op/s 2024-09-11T08:39:47.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:47 smithi092 ceph-mon[41114]: 4.a scrub starts 2024-09-11T08:39:47.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:47 smithi092 ceph-mon[41114]: 4.a scrub ok 2024-09-11T08:39:47.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:47 smithi092 ceph-mon[41114]: 4.19 deep-scrub starts 2024-09-11T08:39:47.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:47 smithi092 ceph-mon[41114]: 4.19 deep-scrub ok 2024-09-11T08:39:47.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:47 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:47.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:47 smithi092 ceph-mon[29647]: 4.a scrub starts 2024-09-11T08:39:47.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:47 smithi092 ceph-mon[29647]: 4.a scrub ok 2024-09-11T08:39:47.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:47 smithi092 ceph-mon[29647]: 4.19 deep-scrub starts 2024-09-11T08:39:47.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:47 smithi092 ceph-mon[29647]: 4.19 deep-scrub ok 2024-09-11T08:39:47.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:47 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:47.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:47 smithi145 ceph-mon[33384]: 4.a scrub starts 2024-09-11T08:39:47.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:47 smithi145 ceph-mon[33384]: 4.a scrub ok 2024-09-11T08:39:47.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:47 smithi145 ceph-mon[33384]: 4.19 deep-scrub starts 2024-09-11T08:39:47.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:47 smithi145 ceph-mon[33384]: 4.19 deep-scrub ok 2024-09-11T08:39:47.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:47 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:47.795 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:47.932 INFO:teuthology.orchestra.run.smithi092.stdout:163208757276 2024-09-11T08:39:48.345 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[41114]: 4.1a deep-scrub starts 2024-09-11T08:39:48.345 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[41114]: 4.1a deep-scrub ok 2024-09-11T08:39:48.345 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[41114]: 5.c scrub starts 2024-09-11T08:39:48.345 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[41114]: 5.c scrub ok 2024-09-11T08:39:48.345 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[41114]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:39:48.345 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[41114]: 5.10 scrub starts 2024-09-11T08:39:48.346 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[41114]: 5.10 scrub ok 2024-09-11T08:39:48.346 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[41114]: 6.7 scrub starts 2024-09-11T08:39:48.346 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[41114]: 6.7 scrub ok 2024-09-11T08:39:48.346 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3054035549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:39:48.346 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[29647]: 4.1a deep-scrub starts 2024-09-11T08:39:48.346 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[29647]: 4.1a deep-scrub ok 2024-09-11T08:39:48.346 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[29647]: 5.c scrub starts 2024-09-11T08:39:48.346 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[29647]: 5.c scrub ok 2024-09-11T08:39:48.346 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[29647]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:39:48.346 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[29647]: 5.10 scrub starts 2024-09-11T08:39:48.346 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[29647]: 5.10 scrub ok 2024-09-11T08:39:48.346 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[29647]: 6.7 scrub starts 2024-09-11T08:39:48.347 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[29647]: 6.7 scrub ok 2024-09-11T08:39:48.347 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:48 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3054035549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:39:48.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:48 smithi145 ceph-mon[33384]: 4.1a deep-scrub starts 2024-09-11T08:39:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:48 smithi145 ceph-mon[33384]: 4.1a deep-scrub ok 2024-09-11T08:39:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:48 smithi145 ceph-mon[33384]: 5.c scrub starts 2024-09-11T08:39:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:48 smithi145 ceph-mon[33384]: 5.c scrub ok 2024-09-11T08:39:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:48 smithi145 ceph-mon[33384]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:39:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:48 smithi145 ceph-mon[33384]: 5.10 scrub starts 2024-09-11T08:39:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:48 smithi145 ceph-mon[33384]: 5.10 scrub ok 2024-09-11T08:39:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:48 smithi145 ceph-mon[33384]: 6.7 scrub starts 2024-09-11T08:39:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:48 smithi145 ceph-mon[33384]: 6.7 scrub ok 2024-09-11T08:39:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:48 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3054035549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:39:49.086 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757271 got 163208757276 for osd.5 2024-09-11T08:39:49.086 DEBUG:teuthology.parallel:result is None 2024-09-11T08:39:49.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:49 smithi092 ceph-mon[41114]: 3.1f scrub starts 2024-09-11T08:39:49.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:49 smithi092 ceph-mon[41114]: 3.1f scrub ok 2024-09-11T08:39:49.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:49 smithi092 ceph-mon[41114]: 4.8 scrub starts 2024-09-11T08:39:49.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:49 smithi092 ceph-mon[41114]: 4.8 scrub ok 2024-09-11T08:39:49.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:49 smithi092 ceph-mon[29647]: 3.1f scrub starts 2024-09-11T08:39:49.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:49 smithi092 ceph-mon[29647]: 3.1f scrub ok 2024-09-11T08:39:49.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:49 smithi092 ceph-mon[29647]: 4.8 scrub starts 2024-09-11T08:39:49.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:49 smithi092 ceph-mon[29647]: 4.8 scrub ok 2024-09-11T08:39:49.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:49 smithi145 ceph-mon[33384]: 3.1f scrub starts 2024-09-11T08:39:49.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:49 smithi145 ceph-mon[33384]: 3.1f scrub ok 2024-09-11T08:39:49.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:49 smithi145 ceph-mon[33384]: 4.8 scrub starts 2024-09-11T08:39:49.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:49 smithi145 ceph-mon[33384]: 4.8 scrub ok 2024-09-11T08:39:50.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:50 smithi092 ceph-mon[29647]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 991 B/s rd, 0 op/s 2024-09-11T08:39:50.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:50 smithi092 ceph-mon[29647]: 5.4 scrub starts 2024-09-11T08:39:50.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:50 smithi092 ceph-mon[29647]: 5.4 scrub ok 2024-09-11T08:39:50.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:50 smithi092 ceph-mon[29647]: 5.16 scrub starts 2024-09-11T08:39:50.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:50 smithi092 ceph-mon[29647]: 5.16 scrub ok 2024-09-11T08:39:50.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:50 smithi092 ceph-mon[41114]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 991 B/s rd, 0 op/s 2024-09-11T08:39:50.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:50 smithi092 ceph-mon[41114]: 5.4 scrub starts 2024-09-11T08:39:50.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:50 smithi092 ceph-mon[41114]: 5.4 scrub ok 2024-09-11T08:39:50.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:50 smithi092 ceph-mon[41114]: 5.16 scrub starts 2024-09-11T08:39:50.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:50 smithi092 ceph-mon[41114]: 5.16 scrub ok 2024-09-11T08:39:50.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:50 smithi145 ceph-mon[33384]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 991 B/s rd, 0 op/s 2024-09-11T08:39:50.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:50 smithi145 ceph-mon[33384]: 5.4 scrub starts 2024-09-11T08:39:50.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:50 smithi145 ceph-mon[33384]: 5.4 scrub ok 2024-09-11T08:39:50.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:50 smithi145 ceph-mon[33384]: 5.16 scrub starts 2024-09-11T08:39:50.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:50 smithi145 ceph-mon[33384]: 5.16 scrub ok 2024-09-11T08:39:50.625 INFO:teuthology.orchestra.run.smithi092.stdout:133143986208 2024-09-11T08:39:51.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:51 smithi092 ceph-mon[29647]: 3.c deep-scrub starts 2024-09-11T08:39:51.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:51 smithi092 ceph-mon[29647]: 3.c deep-scrub ok 2024-09-11T08:39:51.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:51 smithi092 ceph-mon[29647]: 4.1c deep-scrub starts 2024-09-11T08:39:51.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:51 smithi092 ceph-mon[29647]: 4.1c deep-scrub ok 2024-09-11T08:39:51.276 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:51 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4279533097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:39:51.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:51 smithi092 ceph-mon[41114]: 3.c deep-scrub starts 2024-09-11T08:39:51.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:51 smithi092 ceph-mon[41114]: 3.c deep-scrub ok 2024-09-11T08:39:51.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:51 smithi092 ceph-mon[41114]: 4.1c deep-scrub starts 2024-09-11T08:39:51.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:51 smithi092 ceph-mon[41114]: 4.1c deep-scrub ok 2024-09-11T08:39:51.277 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:51 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4279533097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:39:51.277 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:50 smithi092 podman[75454]: 2024-09-11 08:39:50.86065527 +0000 UTC m=+3.055305881 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-11T08:39:51.277 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:50 smithi092 podman[75454]: 2024-09-11 08:39:50.964298747 +0000 UTC m=+3.158949346 volume create dcd905fe3f0579ce13ab5ab45e6fd49363fcfec4d63c36232e82d49b660472b3 2024-09-11T08:39:51.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:51 smithi145 ceph-mon[33384]: 3.c deep-scrub starts 2024-09-11T08:39:51.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:51 smithi145 ceph-mon[33384]: 3.c deep-scrub ok 2024-09-11T08:39:51.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:51 smithi145 ceph-mon[33384]: 4.1c deep-scrub starts 2024-09-11T08:39:51.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:51 smithi145 ceph-mon[33384]: 4.1c deep-scrub ok 2024-09-11T08:39:51.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:51 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4279533097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:39:51.969 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:51 smithi092 podman[75454]: 2024-09-11 08:39:51.785661589 +0000 UTC m=+3.980312185 container create afe2c4d9d2d034d4d244eb01e275f5ca9cb3faa81d28c1341da6f01c5841e193 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:52.349 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-mon[29647]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:52.350 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-mon[41114]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:52.350 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:52 smithi092 podman[75454]: 2024-09-11 08:39:52.096978596 +0000 UTC m=+4.291629205 container init afe2c4d9d2d034d4d244eb01e275f5ca9cb3faa81d28c1341da6f01c5841e193 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:52.350 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:52 smithi092 podman[75454]: 2024-09-11 08:39:52.10329949 +0000 UTC m=+4.297950087 container start afe2c4d9d2d034d4d244eb01e275f5ca9cb3faa81d28c1341da6f01c5841e193 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:52.350 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[75712]: ts=2024-09-11T08:39:52.117Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T08:39:52.350 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[75712]: ts=2024-09-11T08:39:52.117Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T08:39:52.350 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[75712]: ts=2024-09-11T08:39:52.118Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.92 port=9094 2024-09-11T08:39:52.350 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[75712]: ts=2024-09-11T08:39:52.119Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T08:39:52.350 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[75712]: ts=2024-09-11T08:39:52.142Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:39:52.350 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[75712]: ts=2024-09-11T08:39:52.143Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:39:52.350 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[75712]: ts=2024-09-11T08:39:52.145Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T08:39:52.351 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[75712]: ts=2024-09-11T08:39:52.145Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T08:39:52.351 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:52 smithi092 bash[75454]: afe2c4d9d2d034d4d244eb01e275f5ca9cb3faa81d28c1341da6f01c5841e193 2024-09-11T08:39:52.351 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:52 smithi092 systemd[1]: Started Ceph alertmanager.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:39:52.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:52 smithi145 ceph-mon[33384]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:52.604 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:39:52] ENGINE Bus STOPPING 2024-09-11T08:39:52.961 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:39:52] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:39:52.961 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:39:52] ENGINE Bus STOPPED 2024-09-11T08:39:52.961 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:39:52] ENGINE Bus STARTING 2024-09-11T08:39:52.961 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:39:52] ENGINE Serving on http://:::9283 2024-09-11T08:39:52.961 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:39:52 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:39:52] ENGINE Bus STARTED 2024-09-11T08:39:52.987 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986205 got 133143986208 for osd.4 2024-09-11T08:39:52.987 DEBUG:teuthology.parallel:result is None 2024-09-11T08:39:53.096 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:39:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:39:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:39:53.223 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: 5.1a deep-scrub starts 2024-09-11T08:39:53.223 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: 5.1a deep-scrub ok 2024-09-11T08:39:53.223 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: 6.18 scrub starts 2024-09-11T08:39:53.223 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: 6.18 scrub ok 2024-09-11T08:39:53.223 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.223 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.223 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.224 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.224 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.224 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.224 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:39:53.224 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:39:53.224 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.224 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: Deploying daemon grafana.a on smithi145 2024-09-11T08:39:53.224 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: 6.14 deep-scrub starts 2024-09-11T08:39:53.224 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[41114]: 6.14 deep-scrub ok 2024-09-11T08:39:53.224 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: 5.1a deep-scrub starts 2024-09-11T08:39:53.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: 5.1a deep-scrub ok 2024-09-11T08:39:53.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: 6.18 scrub starts 2024-09-11T08:39:53.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: 6.18 scrub ok 2024-09-11T08:39:53.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:39:53.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:39:53.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: Deploying daemon grafana.a on smithi145 2024-09-11T08:39:53.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: 6.14 deep-scrub starts 2024-09-11T08:39:53.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:53 smithi092 ceph-mon[29647]: 6.14 deep-scrub ok 2024-09-11T08:39:53.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: 5.1a deep-scrub starts 2024-09-11T08:39:53.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: 5.1a deep-scrub ok 2024-09-11T08:39:53.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: 6.18 scrub starts 2024-09-11T08:39:53.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: 6.18 scrub ok 2024-09-11T08:39:53.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:39:53.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:39:53.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:53.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: Deploying daemon grafana.a on smithi145 2024-09-11T08:39:53.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: 6.14 deep-scrub starts 2024-09-11T08:39:53.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:53 smithi145 ceph-mon[33384]: 6.14 deep-scrub ok 2024-09-11T08:39:53.975 INFO:teuthology.orchestra.run.smithi092.stdout:193273528345 2024-09-11T08:39:54.195 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[75712]: ts=2024-09-11T08:39:54.120Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000493857s 2024-09-11T08:39:54.195 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[29647]: 6.16 scrub starts 2024-09-11T08:39:54.195 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[29647]: 6.16 scrub ok 2024-09-11T08:39:54.196 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[29647]: 6.1d scrub starts 2024-09-11T08:39:54.196 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[29647]: 6.1d scrub ok 2024-09-11T08:39:54.196 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[29647]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:54.196 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[29647]: 3.18 scrub starts 2024-09-11T08:39:54.196 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[29647]: 3.18 scrub ok 2024-09-11T08:39:54.196 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1194355199' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:39:54.196 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[41114]: 6.16 scrub starts 2024-09-11T08:39:54.196 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[41114]: 6.16 scrub ok 2024-09-11T08:39:54.196 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[41114]: 6.1d scrub starts 2024-09-11T08:39:54.196 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[41114]: 6.1d scrub ok 2024-09-11T08:39:54.196 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[41114]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:54.197 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[41114]: 3.18 scrub starts 2024-09-11T08:39:54.197 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[41114]: 3.18 scrub ok 2024-09-11T08:39:54.197 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:54 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1194355199' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:39:54.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:54 smithi145 ceph-mon[33384]: 6.16 scrub starts 2024-09-11T08:39:54.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:54 smithi145 ceph-mon[33384]: 6.16 scrub ok 2024-09-11T08:39:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:54 smithi145 ceph-mon[33384]: 6.1d scrub starts 2024-09-11T08:39:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:54 smithi145 ceph-mon[33384]: 6.1d scrub ok 2024-09-11T08:39:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:54 smithi145 ceph-mon[33384]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:54 smithi145 ceph-mon[33384]: 3.18 scrub starts 2024-09-11T08:39:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:54 smithi145 ceph-mon[33384]: 3.18 scrub ok 2024-09-11T08:39:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:54 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1194355199' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:39:55.037 INFO:teuthology.orchestra.run.smithi092.stdout:223338299412 2024-09-11T08:39:55.148 INFO:teuthology.orchestra.run.smithi092.stdout:34359738420 2024-09-11T08:39:55.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[29647]: 3.11 scrub starts 2024-09-11T08:39:55.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[29647]: 3.11 scrub ok 2024-09-11T08:39:55.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[29647]: 4.4 deep-scrub starts 2024-09-11T08:39:55.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[29647]: 4.4 deep-scrub ok 2024-09-11T08:39:55.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[29647]: 5.11 scrub starts 2024-09-11T08:39:55.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[29647]: 5.11 scrub ok 2024-09-11T08:39:55.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3849943651' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:39:55.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:55.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[41114]: 3.11 scrub starts 2024-09-11T08:39:55.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[41114]: 3.11 scrub ok 2024-09-11T08:39:55.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[41114]: 4.4 deep-scrub starts 2024-09-11T08:39:55.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[41114]: 4.4 deep-scrub ok 2024-09-11T08:39:55.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[41114]: 5.11 scrub starts 2024-09-11T08:39:55.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[41114]: 5.11 scrub ok 2024-09-11T08:39:55.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3849943651' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:39:55.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:55 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:55.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:55 smithi145 ceph-mon[33384]: 3.11 scrub starts 2024-09-11T08:39:55.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:55 smithi145 ceph-mon[33384]: 3.11 scrub ok 2024-09-11T08:39:55.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:55 smithi145 ceph-mon[33384]: 4.4 deep-scrub starts 2024-09-11T08:39:55.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:55 smithi145 ceph-mon[33384]: 4.4 deep-scrub ok 2024-09-11T08:39:55.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:55 smithi145 ceph-mon[33384]: 5.11 scrub starts 2024-09-11T08:39:55.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:55 smithi145 ceph-mon[33384]: 5.11 scrub ok 2024-09-11T08:39:55.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:55 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3849943651' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:39:55.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:55 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:39:55.732 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299408 got 223338299412 for osd.7 2024-09-11T08:39:55.732 DEBUG:teuthology.parallel:result is None 2024-09-11T08:39:56.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:56 smithi092 ceph-mon[29647]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:56.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:56 smithi092 ceph-mon[29647]: 4.1d scrub starts 2024-09-11T08:39:56.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:56 smithi092 ceph-mon[29647]: 4.1d scrub ok 2024-09-11T08:39:56.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:56 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1359735233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:39:56.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:56 smithi092 ceph-mon[41114]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:56.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:56 smithi092 ceph-mon[41114]: 4.1d scrub starts 2024-09-11T08:39:56.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:56 smithi092 ceph-mon[41114]: 4.1d scrub ok 2024-09-11T08:39:56.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:56 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1359735233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:39:56.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:39:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:39:56.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:56 smithi145 ceph-mon[33384]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:56 smithi145 ceph-mon[33384]: 4.1d scrub starts 2024-09-11T08:39:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:56 smithi145 ceph-mon[33384]: 4.1d scrub ok 2024-09-11T08:39:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:56 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1359735233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:39:56.432 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528341 got 193273528345 for osd.6 2024-09-11T08:39:56.432 DEBUG:teuthology.parallel:result is None 2024-09-11T08:39:56.992 INFO:teuthology.orchestra.run.smithi092.stdout:77309411371 2024-09-11T08:39:57.184 INFO:teuthology.orchestra.run.smithi092.stdout:55834574897 2024-09-11T08:39:57.185 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[29647]: 2.0 scrub starts 2024-09-11T08:39:57.185 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[29647]: 2.0 scrub ok 2024-09-11T08:39:57.185 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[29647]: 5.1e scrub starts 2024-09-11T08:39:57.185 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[29647]: 5.1e scrub ok 2024-09-11T08:39:57.185 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[29647]: 3.a scrub starts 2024-09-11T08:39:57.185 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[29647]: 3.a scrub ok 2024-09-11T08:39:57.185 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:57.185 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[29647]: 3.d scrub starts 2024-09-11T08:39:57.185 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[29647]: 3.d scrub ok 2024-09-11T08:39:57.186 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3414136350' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:39:57.186 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[41114]: 2.0 scrub starts 2024-09-11T08:39:57.186 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[41114]: 2.0 scrub ok 2024-09-11T08:39:57.186 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[41114]: 5.1e scrub starts 2024-09-11T08:39:57.186 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[41114]: 5.1e scrub ok 2024-09-11T08:39:57.186 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[41114]: 3.a scrub starts 2024-09-11T08:39:57.186 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[41114]: 3.a scrub ok 2024-09-11T08:39:57.186 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:57.186 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[41114]: 3.d scrub starts 2024-09-11T08:39:57.186 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[41114]: 3.d scrub ok 2024-09-11T08:39:57.186 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:57 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3414136350' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:39:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:57 smithi145 ceph-mon[33384]: 2.0 scrub starts 2024-09-11T08:39:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:57 smithi145 ceph-mon[33384]: 2.0 scrub ok 2024-09-11T08:39:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:57 smithi145 ceph-mon[33384]: 5.1e scrub starts 2024-09-11T08:39:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:57 smithi145 ceph-mon[33384]: 5.1e scrub ok 2024-09-11T08:39:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:57 smithi145 ceph-mon[33384]: 3.a scrub starts 2024-09-11T08:39:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:57 smithi145 ceph-mon[33384]: 3.a scrub ok 2024-09-11T08:39:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:57 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:57 smithi145 ceph-mon[33384]: 3.d scrub starts 2024-09-11T08:39:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:57 smithi145 ceph-mon[33384]: 3.d scrub ok 2024-09-11T08:39:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:57 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3414136350' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:39:57.628 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411368 got 77309411371 for osd.2 2024-09-11T08:39:57.628 DEBUG:teuthology.parallel:result is None 2024-09-11T08:39:58.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:58 smithi092 ceph-mon[29647]: 4.18 deep-scrub starts 2024-09-11T08:39:58.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:58 smithi092 ceph-mon[29647]: 4.18 deep-scrub ok 2024-09-11T08:39:58.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:58 smithi092 ceph-mon[29647]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:58.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:58 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1233198914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:39:58.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:58 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:39:58.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:58 smithi092 ceph-mon[29647]: 5.8 scrub ok 2024-09-11T08:39:58.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:58 smithi092 ceph-mon[41114]: 4.18 deep-scrub starts 2024-09-11T08:39:58.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:58 smithi092 ceph-mon[41114]: 4.18 deep-scrub ok 2024-09-11T08:39:58.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:58 smithi092 ceph-mon[41114]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:58.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:58 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1233198914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:39:58.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:58 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:39:58.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:58 smithi092 ceph-mon[41114]: 5.8 scrub ok 2024-09-11T08:39:58.344 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574892 got 55834574897 for osd.1 2024-09-11T08:39:58.344 DEBUG:teuthology.parallel:result is None 2024-09-11T08:39:58.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:58 smithi145 ceph-mon[33384]: 4.18 deep-scrub starts 2024-09-11T08:39:58.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:58 smithi145 ceph-mon[33384]: 4.18 deep-scrub ok 2024-09-11T08:39:58.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:58 smithi145 ceph-mon[33384]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:58.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:58 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1233198914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:39:58.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:58 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:39:58.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:58 smithi145 ceph-mon[33384]: 5.8 scrub ok 2024-09-11T08:39:59.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:59 smithi145 ceph-mon[33384]: 5.2 deep-scrub starts 2024-09-11T08:39:59.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:59 smithi145 ceph-mon[33384]: 5.2 deep-scrub ok 2024-09-11T08:39:59.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:59 smithi145 ceph-mon[33384]: 3.1e scrub starts 2024-09-11T08:39:59.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:59 smithi145 ceph-mon[33384]: 3.1e scrub ok 2024-09-11T08:39:59.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:59 smithi145 ceph-mon[33384]: 2.2 scrub starts 2024-09-11T08:39:59.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:39:59 smithi145 ceph-mon[33384]: 2.2 scrub ok 2024-09-11T08:39:59.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:59 smithi092 ceph-mon[41114]: 5.2 deep-scrub starts 2024-09-11T08:39:59.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:59 smithi092 ceph-mon[41114]: 5.2 deep-scrub ok 2024-09-11T08:39:59.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:59 smithi092 ceph-mon[41114]: 3.1e scrub starts 2024-09-11T08:39:59.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:59 smithi092 ceph-mon[41114]: 3.1e scrub ok 2024-09-11T08:39:59.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:59 smithi092 ceph-mon[41114]: 2.2 scrub starts 2024-09-11T08:39:59.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:39:59 smithi092 ceph-mon[41114]: 2.2 scrub ok 2024-09-11T08:39:59.483 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:59 smithi092 ceph-mon[29647]: 5.2 deep-scrub starts 2024-09-11T08:39:59.483 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:59 smithi092 ceph-mon[29647]: 5.2 deep-scrub ok 2024-09-11T08:39:59.483 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:59 smithi092 ceph-mon[29647]: 3.1e scrub starts 2024-09-11T08:39:59.483 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:59 smithi092 ceph-mon[29647]: 3.1e scrub ok 2024-09-11T08:39:59.483 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:59 smithi092 ceph-mon[29647]: 2.2 scrub starts 2024-09-11T08:39:59.483 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:39:59 smithi092 ceph-mon[29647]: 2.2 scrub ok 2024-09-11T08:39:59.509 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738417 got 34359738420 for osd.0 2024-09-11T08:39:59.509 DEBUG:teuthology.parallel:result is None 2024-09-11T08:40:00.131 INFO:teuthology.orchestra.run.smithi092.stdout:111669149736 2024-09-11T08:40:00.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:00 smithi145 ceph-mon[33384]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:00.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:00 smithi145 ceph-mon[33384]: overall HEALTH_OK 2024-09-11T08:40:00.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:00.439 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:00 smithi092 ceph-mon[29647]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:00.439 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:00 smithi092 ceph-mon[29647]: overall HEALTH_OK 2024-09-11T08:40:00.439 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:00.439 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:00 smithi092 ceph-mon[41114]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:00.439 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:00 smithi092 ceph-mon[41114]: overall HEALTH_OK 2024-09-11T08:40:00.439 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:00.729 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149730 got 111669149736 for osd.3 2024-09-11T08:40:00.729 DEBUG:teuthology.parallel:result is None 2024-09-11T08:40:00.730 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T08:40:00.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:40:00.971 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:01.227 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:01 smithi145 ceph-mon[33384]: 6.1e scrub starts 2024-09-11T08:40:01.227 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:01 smithi145 ceph-mon[33384]: 6.1e scrub ok 2024-09-11T08:40:01.227 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:01 smithi145 ceph-mon[33384]: 3.e scrub starts 2024-09-11T08:40:01.227 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:01 smithi145 ceph-mon[33384]: 3.e scrub ok 2024-09-11T08:40:01.227 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:01 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/187740091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:40:01.227 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:01 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T08:40:01.228 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:01 smithi145 ceph-mon[33384]: 5.d scrub ok 2024-09-11T08:40:01.359 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:01 smithi092 ceph-mon[41114]: 6.1e scrub starts 2024-09-11T08:40:01.359 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:01 smithi092 ceph-mon[41114]: 6.1e scrub ok 2024-09-11T08:40:01.359 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:01 smithi092 ceph-mon[41114]: 3.e scrub starts 2024-09-11T08:40:01.359 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:01 smithi092 ceph-mon[41114]: 3.e scrub ok 2024-09-11T08:40:01.359 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:01 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/187740091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:40:01.359 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:01 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T08:40:01.359 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:01 smithi092 ceph-mon[41114]: 5.d scrub ok 2024-09-11T08:40:01.360 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:01 smithi092 ceph-mon[29647]: 6.1e scrub starts 2024-09-11T08:40:01.360 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:01 smithi092 ceph-mon[29647]: 6.1e scrub ok 2024-09-11T08:40:01.360 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:01 smithi092 ceph-mon[29647]: 3.e scrub starts 2024-09-11T08:40:01.360 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:01 smithi092 ceph-mon[29647]: 3.e scrub ok 2024-09-11T08:40:01.360 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:01 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/187740091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:40:01.360 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:01 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T08:40:01.360 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:01 smithi092 ceph-mon[29647]: 5.d scrub ok 2024-09-11T08:40:02.213 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:02 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[75712]: ts=2024-09-11T08:40:02.122Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002845854s 2024-09-11T08:40:02.213 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:02 smithi092 ceph-mon[29647]: 3.0 scrub starts 2024-09-11T08:40:02.213 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:02 smithi092 ceph-mon[29647]: 3.0 scrub ok 2024-09-11T08:40:02.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:02 smithi092 ceph-mon[29647]: 4.5 scrub starts 2024-09-11T08:40:02.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:02 smithi092 ceph-mon[29647]: 4.5 scrub ok 2024-09-11T08:40:02.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:02 smithi092 ceph-mon[29647]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:02.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:02 smithi092 ceph-mon[29647]: 2.1 scrub starts 2024-09-11T08:40:02.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:02 smithi092 ceph-mon[29647]: 2.1 scrub ok 2024-09-11T08:40:02.214 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:02 smithi092 ceph-mon[41114]: 3.0 scrub starts 2024-09-11T08:40:02.214 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:02 smithi092 ceph-mon[41114]: 3.0 scrub ok 2024-09-11T08:40:02.214 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:02 smithi092 ceph-mon[41114]: 4.5 scrub starts 2024-09-11T08:40:02.214 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:02 smithi092 ceph-mon[41114]: 4.5 scrub ok 2024-09-11T08:40:02.214 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:02 smithi092 ceph-mon[41114]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:02.215 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:02 smithi092 ceph-mon[41114]: 2.1 scrub starts 2024-09-11T08:40:02.215 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:02 smithi092 ceph-mon[41114]: 2.1 scrub ok 2024-09-11T08:40:02.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:02 smithi145 ceph-mon[33384]: 3.0 scrub starts 2024-09-11T08:40:02.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:02 smithi145 ceph-mon[33384]: 3.0 scrub ok 2024-09-11T08:40:02.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:02 smithi145 ceph-mon[33384]: 4.5 scrub starts 2024-09-11T08:40:02.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:02 smithi145 ceph-mon[33384]: 4.5 scrub ok 2024-09-11T08:40:02.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:02 smithi145 ceph-mon[33384]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:02.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:02 smithi145 ceph-mon[33384]: 2.1 scrub starts 2024-09-11T08:40:02.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:02 smithi145 ceph-mon[33384]: 2.1 scrub ok 2024-09-11T08:40:02.722 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:02 smithi145 podman[66430]: 2024-09-11 08:40:02.643212939 +0000 UTC m=+0.131586297 container create c1e40a1789f917c84c910f2f7fec27534d7edbb37fc606af3e0c8b491194bf05 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, name=ubi8, io.openshift.expose-services=, build-date=2022-04-29T11:48:11.055897, vendor=Red Hat, Inc., io.buildah.version=1.32.0, distribution-scope=public, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.component=ubi8-container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, maintainer=Paul Cuzner , description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8.5, architecture=x86_64, release=239.1651231664) 2024-09-11T08:40:02.954 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:40:02.956 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:40:03.030 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:40:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:40:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:03.030 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:02 smithi145 podman[66430]: 2024-09-11 08:40:02.870880307 +0000 UTC m=+0.359253672 container init c1e40a1789f917c84c910f2f7fec27534d7edbb37fc606af3e0c8b491194bf05 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a, io.openshift.tags=base rhel8, io.buildah.version=1.32.0, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, description=Ceph Grafana Container, com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, vcs-type=git, distribution-scope=public, name=ubi8, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664, vendor=Red Hat, Inc., version=8.5, summary=Grafana Container configured for Ceph mgr/dashboard integration) 2024-09-11T08:40:03.031 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:02 smithi145 podman[66430]: 2024-09-11 08:40:02.875510471 +0000 UTC m=+0.363883833 container start c1e40a1789f917c84c910f2f7fec27534d7edbb37fc606af3e0c8b491194bf05 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a, version=8.5, com.redhat.component=ubi8-container, vendor=Red Hat, Inc., maintainer=Paul Cuzner , description=Ceph Grafana Container, io.openshift.expose-services=, io.openshift.tags=base rhel8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, name=ubi8, io.buildah.version=1.32.0, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, build-date=2022-04-29T11:48:11.055897, io.k8s.display-name=Red Hat Universal Base Image 8, architecture=x86_64, release=239.1651231664, distribution-scope=public, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git) 2024-09-11T08:40:03.031 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:02 smithi145 bash[66430]: c1e40a1789f917c84c910f2f7fec27534d7edbb37fc606af3e0c8b491194bf05 2024-09-11T08:40:03.031 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:02 smithi145 systemd[1]: Started Ceph grafana.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:40:03.031 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=settings t=2024-09-11T08:40:03.030989378Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-11T08:40:03.031 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=settings t=2024-09-11T08:40:03.031163606Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-11T08:40:03.031 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=settings t=2024-09-11T08:40:03.031186618Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-11T08:40:03.031 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=settings t=2024-09-11T08:40:03.031198217Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-11T08:40:03.031 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=settings t=2024-09-11T08:40:03.031207625Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-11T08:40:03.031 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=settings t=2024-09-11T08:40:03.031216623Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-11T08:40:03.031 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=settings t=2024-09-11T08:40:03.03122505Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-11T08:40:03.031 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=settings t=2024-09-11T08:40:03.031233542Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-11T08:40:03.031 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=settings t=2024-09-11T08:40:03.031243293Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-11T08:40:03.031 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=settings t=2024-09-11T08:40:03.031252691Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-11T08:40:03.032 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=settings t=2024-09-11T08:40:03.031261335Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-11T08:40:03.032 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=settings t=2024-09-11T08:40:03.031269983Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-11T08:40:03.032 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=settings t=2024-09-11T08:40:03.031280096Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-11T08:40:03.032 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=settings t=2024-09-11T08:40:03.031288295Z level=info msg="App mode production" 2024-09-11T08:40:03.032 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=sqlstore t=2024-09-11T08:40:03.031346014Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-11T08:40:03.032 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=sqlstore t=2024-09-11T08:40:03.03136983Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-11T08:40:03.280 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-mon[33384]: 5.1c scrub starts 2024-09-11T08:40:03.281 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-mon[33384]: 5.1c scrub ok 2024-09-11T08:40:03.281 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-mon[33384]: 5.6 deep-scrub starts 2024-09-11T08:40:03.281 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-mon[33384]: 5.6 deep-scrub ok 2024-09-11T08:40:03.281 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:03.281 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:03.281 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:03.281 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:03.282 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.032590654Z level=info msg="Starting DB migrations" 2024-09-11T08:40:03.282 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.032665143Z level=info msg="Executing migration" id="create migration_log table" 2024-09-11T08:40:03.282 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.033168648Z level=info msg="Executing migration" id="create user table" 2024-09-11T08:40:03.282 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.033589346Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-11T08:40:03.282 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.034418774Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-11T08:40:03.282 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.034840928Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-11T08:40:03.282 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.037447167Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-11T08:40:03.282 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.040272678Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-11T08:40:03.282 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.043298155Z level=info msg="Executing migration" id="create user table v2" 2024-09-11T08:40:03.283 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.043758348Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-11T08:40:03.283 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.046343685Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-11T08:40:03.283 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.049045623Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-11T08:40:03.283 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.0516974Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-11T08:40:03.283 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.054661431Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-11T08:40:03.283 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.057254582Z level=info msg="Executing migration" id="Update user table charset" 2024-09-11T08:40:03.283 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.060002169Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-11T08:40:03.283 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.062786601Z level=info msg="Executing migration" id="Add missing user data" 2024-09-11T08:40:03.283 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.065633466Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-11T08:40:03.283 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.068367399Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-11T08:40:03.283 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.071243214Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-11T08:40:03.284 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.074071904Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-11T08:40:03.284 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.076823358Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-11T08:40:03.284 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.079722844Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-11T08:40:03.284 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.082605149Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-11T08:40:03.284 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.085151102Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-11T08:40:03.284 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.087881272Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-11T08:40:03.284 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.088450192Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-11T08:40:03.284 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.091114191Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-11T08:40:03.284 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.091532487Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-11T08:40:03.284 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.093715469Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-11T08:40:03.284 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.09429231Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-11T08:40:03.284 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.094701902Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-11T08:40:03.285 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.097327458Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-11T08:40:03.285 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.100019038Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-11T08:40:03.285 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.102486435Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-11T08:40:03.285 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.102941224Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-11T08:40:03.285 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.105699062Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-11T08:40:03.285 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.108548383Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-11T08:40:03.285 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.111198694Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-11T08:40:03.286 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.111692522Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-11T08:40:03.287 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.112042326Z level=info msg="Executing migration" id="create star table" 2024-09-11T08:40:03.287 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.112437039Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-11T08:40:03.287 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.112861378Z level=info msg="Executing migration" id="create org table v1" 2024-09-11T08:40:03.287 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.113281938Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-11T08:40:03.287 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.113719045Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-11T08:40:03.287 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.11413262Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-11T08:40:03.287 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.114548408Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-11T08:40:03.287 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.114984342Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-11T08:40:03.287 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.11540474Z level=info msg="Executing migration" id="Update org table charset" 2024-09-11T08:40:03.287 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.116159327Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-11T08:40:03.287 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.116502988Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-11T08:40:03.288 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.116949461Z level=info msg="Executing migration" id="create dashboard table" 2024-09-11T08:40:03.288 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.117438999Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-11T08:40:03.288 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.117871535Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-11T08:40:03.288 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.118848626Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-11T08:40:03.288 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.119582152Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-11T08:40:03.288 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.120524813Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-11T08:40:03.288 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.121257744Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-11T08:40:03.288 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.122114088Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-11T08:40:03.288 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.123018178Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-11T08:40:03.288 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.123796585Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-11T08:40:03.288 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.124645619Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-11T08:40:03.288 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.125589923Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-11T08:40:03.288 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.126367077Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-11T08:40:03.289 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.127209139Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-11T08:40:03.289 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.128068853Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-11T08:40:03.289 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.129044519Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-11T08:40:03.289 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.129779868Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-11T08:40:03.289 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.1325944Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-11T08:40:03.289 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.135358264Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-11T08:40:03.289 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.138044411Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-11T08:40:03.289 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.140640594Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-11T08:40:03.289 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.143526564Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-11T08:40:03.289 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.146148102Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-11T08:40:03.289 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.146709815Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-11T08:40:03.289 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.14885319Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-11T08:40:03.289 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.151638065Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-11T08:40:03.290 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.15425675Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-11T08:40:03.290 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.157088458Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-11T08:40:03.290 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.16000202Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-11T08:40:03.290 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.162473467Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-11T08:40:03.290 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.162816125Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-11T08:40:03.290 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.165755726Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-11T08:40:03.290 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.16841512Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-11T08:40:03.290 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.171182336Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-11T08:40:03.290 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.171611593Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-11T08:40:03.290 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.174357659Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-11T08:40:03.290 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.177166051Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-11T08:40:03.291 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.17963235Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-11T08:40:03.291 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.180047787Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-11T08:40:03.291 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.1828007Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-11T08:40:03.291 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.185424914Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-11T08:40:03.291 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.188003924Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-11T08:40:03.291 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.188357843Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-11T08:40:03.291 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.191227472Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-11T08:40:03.291 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.19393805Z level=info msg="Executing migration" id="create data_source table" 2024-09-11T08:40:03.291 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.196334242Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-11T08:40:03.291 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.196765445Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-11T08:40:03.291 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.197215253Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-11T08:40:03.291 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.200014069Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-11T08:40:03.292 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.202774182Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-11T08:40:03.292 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.203845862Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-11T08:40:03.292 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.206722741Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-11T08:40:03.292 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.209329266Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-11T08:40:03.292 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.212008856Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-11T08:40:03.292 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.212562477Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-11T08:40:03.292 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.213368074Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-11T08:40:03.292 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.21414473Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-11T08:40:03.292 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.215014656Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-11T08:40:03.292 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.215356724Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-11T08:40:03.292 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.216148364Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-11T08:40:03.292 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.216499388Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-11T08:40:03.292 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.216926024Z level=info msg="Executing migration" id="Add uid column" 2024-09-11T08:40:03.293 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.217515892Z level=info msg="Executing migration" id="Update uid value" 2024-09-11T08:40:03.293 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.217827221Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-11T08:40:03.293 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.218289118Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-11T08:40:03.293 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.218751558Z level=info msg="Executing migration" id="create api_key table" 2024-09-11T08:40:03.293 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.219653579Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-11T08:40:03.293 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.220360365Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-11T08:40:03.293 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.221176339Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-11T08:40:03.293 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.221637119Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-11T08:40:03.293 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.222400289Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-11T08:40:03.293 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.222828511Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-11T08:40:03.293 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.223258882Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-11T08:40:03.293 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.22430948Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-11T08:40:03.293 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.225134183Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-11T08:40:03.294 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.225572515Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-11T08:40:03.294 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.226009969Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-11T08:40:03.294 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.226441047Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-11T08:40:03.294 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.226757001Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-11T08:40:03.294 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.227162424Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-11T08:40:03.294 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.227465686Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-11T08:40:03.294 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.228019675Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-11T08:40:03.294 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.228568989Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-11T08:40:03.294 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.228879501Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-11T08:40:03.294 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.229439429Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-11T08:40:03.294 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.229976081Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-11T08:40:03.294 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.230376814Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-11T08:40:03.294 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.230726993Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-11T08:40:03.294 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.231140052Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-11T08:40:03.295 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.231563685Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-11T08:40:03.295 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.232007631Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-11T08:40:03.295 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.232452902Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-11T08:40:03.295 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.23274157Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-11T08:40:03.295 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.23302216Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-11T08:40:03.295 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.233627338Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-11T08:40:03.295 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.234234676Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-11T08:40:03.295 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.23458515Z level=info msg="Executing migration" id="create quota table v1" 2024-09-11T08:40:03.295 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.234992908Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-11T08:40:03.295 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.235424624Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-11T08:40:03.295 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.235716672Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-11T08:40:03.295 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.236135073Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-11T08:40:03.295 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.236559937Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-11T08:40:03.295 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.237160047Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-11T08:40:03.296 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.237472752Z level=info msg="Executing migration" id="create session table" 2024-09-11T08:40:03.296 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.237925218Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-11T08:40:03.296 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.238250644Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-11T08:40:03.296 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.238542608Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-11T08:40:03.296 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.238937898Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-11T08:40:03.296 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.23934676Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-11T08:40:03.296 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.239621968Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-11T08:40:03.296 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.23990124Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-11T08:40:03.296 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.240230408Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-11T08:40:03.296 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.24055173Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-11T08:40:03.296 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.240950601Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-11T08:40:03.296 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.241275613Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-11T08:40:03.297 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.24185734Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-11T08:40:03.297 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.242223638Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-11T08:40:03.297 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.242830987Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-11T08:40:03.297 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.243436751Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-11T08:40:03.297 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.243753868Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-11T08:40:03.297 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.244214426Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-11T08:40:03.297 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.244796668Z level=info msg="Executing migration" id="create alert table v1" 2024-09-11T08:40:03.297 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.247425623Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-11T08:40:03.297 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.250255585Z level=info msg="Executing migration" id="add index alert state" 2024-09-11T08:40:03.297 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.252850839Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-11T08:40:03.297 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.255536121Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-11T08:40:03.297 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.258174752Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-11T08:40:03.298 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.260898154Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-11T08:40:03.298 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.263581291Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-11T08:40:03.298 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.266112283Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-11T08:40:03.298 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.266914568Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-11T08:40:03.298 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.267933682Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-11T08:40:03.298 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.268622036Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-11T08:40:03.298 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.269464159Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-11T08:40:03.298 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.269864037Z level=info msg="Executing migration" id="Add column is_default" 2024-09-11T08:40:03.298 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.270533525Z level=info msg="Executing migration" id="Add column frequency" 2024-09-11T08:40:03.298 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.27138342Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-11T08:40:03.298 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.272212351Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-11T08:40:03.298 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.273061449Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-11T08:40:03.298 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.27387131Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-11T08:40:03.299 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.274830192Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-11T08:40:03.299 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.275575467Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-11T08:40:03.299 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.276419163Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-11T08:40:03.299 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.277348223Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-11T08:40:03.299 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.278077177Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-11T08:40:03.299 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.278946779Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-11T08:40:03.299 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.279794432Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-11T08:40:03.299 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.280660396Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-11T08:40:03.299 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.281622495Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-11T08:40:03.299 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.282382876Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-11T08:40:03.300 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.283234625Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-11T08:40:03.300 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.284099638Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-11T08:40:03.300 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.285041934Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-11T08:40:03.300 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.285877106Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-11T08:40:03.300 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.286726488Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-11T08:40:03.300 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.287568702Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-11T08:40:03.300 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.288466756Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-11T08:40:03.404 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[41114]: 5.1c scrub starts 2024-09-11T08:40:03.404 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[41114]: 5.1c scrub ok 2024-09-11T08:40:03.404 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[41114]: 5.6 deep-scrub starts 2024-09-11T08:40:03.404 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[41114]: 5.6 deep-scrub ok 2024-09-11T08:40:03.405 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:03.405 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:03.405 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:03.405 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:03.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[29647]: 5.1c scrub starts 2024-09-11T08:40:03.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[29647]: 5.1c scrub ok 2024-09-11T08:40:03.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[29647]: 5.6 deep-scrub starts 2024-09-11T08:40:03.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[29647]: 5.6 deep-scrub ok 2024-09-11T08:40:03.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:03.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:03.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:03.406 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:03 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:03.532 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.289237996Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-11T08:40:03.533 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.290152599Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-11T08:40:03.533 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.291013203Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-11T08:40:03.533 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.291837724Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-11T08:40:03.533 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.292889579Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-11T08:40:03.533 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.293565233Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-11T08:40:03.533 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.294395766Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-11T08:40:03.533 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.295069184Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-11T08:40:03.533 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.295489228Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-11T08:40:03.533 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.298107834Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-11T08:40:03.533 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.301161516Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-11T08:40:03.533 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.303981936Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-11T08:40:03.534 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.306710901Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-11T08:40:03.534 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.309371022Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-11T08:40:03.534 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.312194561Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-11T08:40:03.534 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.314822621Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-11T08:40:03.534 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.317753885Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-11T08:40:03.534 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.320283723Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-11T08:40:03.534 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.323196467Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-11T08:40:03.534 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.325878256Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-11T08:40:03.534 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.328588879Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-11T08:40:03.534 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.331482602Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-11T08:40:03.534 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.334146251Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-11T08:40:03.534 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.337161271Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-11T08:40:03.535 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.339904385Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-11T08:40:03.535 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.342843668Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-11T08:40:03.535 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.345362776Z level=info msg="Executing migration" id="Move region to single row" 2024-09-11T08:40:03.535 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.348195497Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-11T08:40:03.535 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.350863767Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-11T08:40:03.535 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.353659787Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-11T08:40:03.535 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.356151258Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-11T08:40:03.535 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.359060029Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-11T08:40:03.535 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.361625519Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-11T08:40:03.535 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.364331354Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-11T08:40:03.535 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.367198558Z level=info msg="Executing migration" id="create test_data table" 2024-09-11T08:40:03.535 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.369652518Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-11T08:40:03.535 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.372534716Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-11T08:40:03.536 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.37519869Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-11T08:40:03.537 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.377607446Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-11T08:40:03.537 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.38044703Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-11T08:40:03.537 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.382891037Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-11T08:40:03.537 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.385717372Z level=info msg="Executing migration" id="create team table" 2024-09-11T08:40:03.537 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.388251989Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-11T08:40:03.537 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.389079004Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-11T08:40:03.537 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.389894179Z level=info msg="Executing migration" id="create team member table" 2024-09-11T08:40:03.537 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.390841104Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-11T08:40:03.537 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.391591357Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-11T08:40:03.537 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.392407137Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-11T08:40:03.537 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.393163615Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-11T08:40:03.537 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.394052904Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-11T08:40:03.537 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.394892094Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-11T08:40:03.537 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.395740532Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-11T08:40:03.538 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.396561176Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-11T08:40:03.538 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.397507916Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-11T08:40:03.538 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.398312843Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-11T08:40:03.538 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.399264192Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-11T08:40:03.538 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.399973123Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-11T08:40:03.538 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.400771032Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-11T08:40:03.538 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.401189049Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-11T08:40:03.538 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.40219243Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-11T08:40:03.538 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.402886892Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-11T08:40:03.538 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.403726914Z level=info msg="Executing migration" id="create tag table" 2024-09-11T08:40:03.538 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.404116411Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-11T08:40:03.538 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.405006199Z level=info msg="Executing migration" id="create login attempt table" 2024-09-11T08:40:03.539 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.405978539Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-11T08:40:03.539 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.406651608Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-11T08:40:03.539 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.407524535Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-11T08:40:03.539 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.409623927Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-11T08:40:03.539 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.410028584Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-11T08:40:03.539 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.410862808Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-11T08:40:03.539 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.411707767Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-11T08:40:03.539 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.412061708Z level=info msg="Executing migration" id="create user auth table" 2024-09-11T08:40:03.539 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.412897269Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-11T08:40:03.539 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.413919732Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-11T08:40:03.539 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.4147022Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-11T08:40:03.539 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.415464513Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-11T08:40:03.539 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.416310899Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-11T08:40:03.540 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.417161229Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-11T08:40:03.540 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.419981777Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-11T08:40:03.540 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.420422928Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-11T08:40:03.540 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.423350729Z level=info msg="Executing migration" id="create server_lock table" 2024-09-11T08:40:03.540 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.4262439Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-11T08:40:03.540 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.428977788Z level=info msg="Executing migration" id="create user auth token table" 2024-09-11T08:40:03.540 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.4294407Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-11T08:40:03.540 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.432214141Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-11T08:40:03.540 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.435025136Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-11T08:40:03.540 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.437854149Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-11T08:40:03.540 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.438648007Z level=info msg="Executing migration" id="create cache_data table" 2024-09-11T08:40:03.540 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.441629127Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-11T08:40:03.541 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.444465005Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-11T08:40:03.541 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.447085897Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-11T08:40:03.541 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.447508642Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-11T08:40:03.541 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.450143406Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-11T08:40:03.541 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.450445547Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-11T08:40:03.541 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.453138974Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-11T08:40:03.541 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.456038677Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-11T08:40:03.541 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.456522011Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-11T08:40:03.541 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.456778032Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-11T08:40:03.541 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.457182823Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-11T08:40:03.541 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.457576894Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-11T08:40:03.541 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.457974877Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-11T08:40:03.541 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.458387338Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-11T08:40:03.541 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.45917163Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-11T08:40:03.542 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.459598871Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-11T08:40:03.542 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.459861844Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-11T08:40:03.542 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.460267461Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-11T08:40:03.542 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.460680462Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-11T08:40:03.542 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.461098457Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-11T08:40:03.542 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.461393837Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-11T08:40:03.542 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.461800507Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-11T08:40:03.542 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.462221121Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-11T08:40:03.542 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.462647012Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-11T08:40:03.542 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.463078214Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-11T08:40:03.542 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.463854828Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-11T08:40:03.542 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.464380568Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-11T08:40:03.542 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.464812891Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-11T08:40:03.543 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.468224902Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-11T08:40:03.543 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.471541308Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-11T08:40:03.543 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.471995881Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-11T08:40:03.543 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.472432128Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-11T08:40:03.543 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.473246917Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-11T08:40:03.543 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.473672042Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-11T08:40:03.543 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.474558085Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-11T08:40:03.543 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.475281371Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-11T08:40:03.543 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.475697713Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-11T08:40:03.543 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.476525999Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-11T08:40:03.543 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.477578867Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-11T08:40:03.543 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.478428983Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-11T08:40:03.543 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.479341982Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-11T08:40:03.543 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.479782951Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-11T08:40:03.544 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.480623607Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-11T08:40:03.544 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.481473674Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-11T08:40:03.544 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.484329005Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-11T08:40:03.544 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.485010368Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-11T08:40:03.544 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.487746484Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-11T08:40:03.544 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.490335124Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-11T08:40:03.544 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.493212016Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-11T08:40:03.544 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.49383111Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-11T08:40:03.544 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.496477594Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-11T08:40:03.544 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.499137028Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-11T08:40:03.544 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.499453423Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-11T08:40:03.544 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.500336408Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-11T08:40:03.544 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.503313737Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-11T08:40:03.545 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.506051952Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-11T08:40:03.545 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.508740494Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-11T08:40:03.545 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.511434417Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-11T08:40:03.545 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.514225735Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-11T08:40:03.545 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.517100288Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-11T08:40:03.545 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.519730794Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-11T08:40:03.545 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.522408499Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-11T08:40:03.545 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.52507963Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-11T08:40:03.545 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.527685001Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-11T08:40:03.545 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.530381056Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-11T08:40:03.546 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.532948015Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-11T08:40:03.546 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.535607392Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-11T08:40:03.708 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-11T08:40:01.010218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":928,"num_read_kb":785,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":580,"ondisk_log_size":580,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2387408,"kb_used_data":8592,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747573808,"statfs":{"total":767960285184,"available":765515579392,"internally_reserved":0,"allocated":8798208,"data_stored":4918791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002700"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:47.339397+0000","last_change":"2024-09-11T08:39:47.339366+0000","last_active":"2024-09-11T08:39:47.339397+0000","last_peered":"2024-09-11T08:39:47.339397+0000","last_clean":"2024-09-11T08:39:47.339397+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:39:47.339397+0000","last_undegraded":"2024-09-11T08:39:47.339397+0000","last_fullsized":"2024-09-11T08:39:47.339397+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.339312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.339312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.154202+0000","scrub_duration":0.001323095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:56.666807+0000","last_change":"2024-09-11T08:39:56.666764+0000","last_active":"2024-09-11T08:39:56.666807+0000","last_peered":"2024-09-11T08:39:56.666807+0000","last_clean":"2024-09-11T08:39:56.666807+0000","last_became_active":"2024-09-11T08:38:40.973398+0000","last_became_peered":"2024-09-11T08:38:40.973398+0000","last_unstale":"2024-09-11T08:39:56.666807+0000","last_undegraded":"2024-09-11T08:39:56.666807+0000","last_fullsized":"2024-09-11T08:39:56.666807+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.666703+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.544222+0000","scrub_duration":0.0041022000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499643+0000","last_change":"2024-09-11T08:38:42.980578+0000","last_active":"2024-09-11T08:39:35.499643+0000","last_peered":"2024-09-11T08:39:35.499643+0000","last_clean":"2024-09-11T08:39:35.499643+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:39:35.499643+0000","last_undegraded":"2024-09-11T08:39:35.499643+0000","last_fullsized":"2024-09-11T08:39:35.499643+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.101278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175049+0000","last_change":"2024-09-11T08:38:44.992111+0000","last_active":"2024-09-11T08:39:36.175049+0000","last_peered":"2024-09-11T08:39:36.175049+0000","last_clean":"2024-09-11T08:39:36.175049+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:39:36.175049+0000","last_undegraded":"2024-09-11T08:39:36.175049+0000","last_fullsized":"2024-09-11T08:39:36.175049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:12.528264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445456+0000","last_change":"2024-09-11T08:38:45.184844+0000","last_active":"2024-09-11T08:39:36.445456+0000","last_peered":"2024-09-11T08:39:36.445456+0000","last_clean":"2024-09-11T08:39:36.445456+0000","last_became_active":"2024-09-11T08:38:45.184215+0000","last_became_peered":"2024-09-11T08:38:45.184215+0000","last_unstale":"2024-09-11T08:39:36.445456+0000","last_undegraded":"2024-09-11T08:39:36.445456+0000","last_fullsized":"2024-09-11T08:39:36.445456+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.791964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:58.161111+0000","last_change":"2024-09-11T08:39:58.161055+0000","last_active":"2024-09-11T08:39:58.161111+0000","last_peered":"2024-09-11T08:39:58.161111+0000","last_clean":"2024-09-11T08:39:58.161111+0000","last_became_active":"2024-09-11T08:38:38.949136+0000","last_became_peered":"2024-09-11T08:38:38.949136+0000","last_unstale":"2024-09-11T08:39:58.161111+0000","last_undegraded":"2024-09-11T08:39:58.161111+0000","last_fullsized":"2024-09-11T08:39:58.161111+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:58.160984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.160984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:00.523696+0000","scrub_duration":0.0015109679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":70,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:45.997924+0000","last_change":"2024-09-11T08:39:45.997873+0000","last_active":"2024-09-11T08:39:45.997924+0000","last_peered":"2024-09-11T08:39:45.997924+0000","last_clean":"2024-09-11T08:39:45.997924+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:39:45.997924+0000","last_undegraded":"2024-09-11T08:39:45.997924+0000","last_fullsized":"2024-09-11T08:39:45.997924+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.997803+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.534414+0000","scrub_duration":0.0046870369999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.174897+0000","last_change":"2024-09-11T08:38:42.983653+0000","last_active":"2024-09-11T08:39:36.174897+0000","last_peered":"2024-09-11T08:39:36.174897+0000","last_clean":"2024-09-11T08:39:36.174897+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:39:36.174897+0000","last_undegraded":"2024-09-11T08:39:36.174897+0000","last_fullsized":"2024-09-11T08:39:36.174897+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:57.589237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501926+0000","last_change":"2024-09-11T08:38:38.949230+0000","last_active":"2024-09-11T08:39:35.501926+0000","last_peered":"2024-09-11T08:39:35.501926+0000","last_clean":"2024-09-11T08:39:35.501926+0000","last_became_active":"2024-09-11T08:38:38.948930+0000","last_became_peered":"2024-09-11T08:38:38.948930+0000","last_unstale":"2024-09-11T08:39:35.501926+0000","last_undegraded":"2024-09-11T08:39:35.501926+0000","last_fullsized":"2024-09-11T08:39:35.501926+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:57.678210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:46.620744+0000","last_change":"2024-09-11T08:39:46.620700+0000","last_active":"2024-09-11T08:39:46.620744+0000","last_peered":"2024-09-11T08:39:46.620744+0000","last_clean":"2024-09-11T08:39:46.620744+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:39:46.620744+0000","last_undegraded":"2024-09-11T08:39:46.620744+0000","last_fullsized":"2024-09-11T08:39:46.620744+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:39:46.620624+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:10.984564+0000","scrub_duration":0.0044132319999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501424+0000","last_change":"2024-09-11T08:38:42.981649+0000","last_active":"2024-09-11T08:39:35.501424+0000","last_peered":"2024-09-11T08:39:35.501424+0000","last_clean":"2024-09-11T08:39:35.501424+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:39:35.501424+0000","last_undegraded":"2024-09-11T08:39:35.501424+0000","last_fullsized":"2024-09-11T08:39:35.501424+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.721496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:51.280393+0000","last_change":"2024-09-11T08:39:51.280365+0000","last_active":"2024-09-11T08:39:51.280393+0000","last_peered":"2024-09-11T08:39:51.280393+0000","last_clean":"2024-09-11T08:39:51.280393+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:39:51.280393+0000","last_undegraded":"2024-09-11T08:39:51.280393+0000","last_fullsized":"2024-09-11T08:39:51.280393+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.280310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.280310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:06.712880+0000","scrub_duration":0.001634777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501649+0000","last_change":"2024-09-11T08:38:38.950588+0000","last_active":"2024-09-11T08:39:35.501649+0000","last_peered":"2024-09-11T08:39:35.501649+0000","last_clean":"2024-09-11T08:39:35.501649+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:39:35.501649+0000","last_undegraded":"2024-09-11T08:39:35.501649+0000","last_fullsized":"2024-09-11T08:39:35.501649+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:58.714558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175109+0000","last_change":"2024-09-11T08:38:40.974291+0000","last_active":"2024-09-11T08:39:36.175109+0000","last_peered":"2024-09-11T08:39:36.175109+0000","last_clean":"2024-09-11T08:39:36.175109+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:39:36.175109+0000","last_undegraded":"2024-09-11T08:39:36.175109+0000","last_fullsized":"2024-09-11T08:39:36.175109+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:57.136170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:51.166149+0000","last_change":"2024-09-11T08:39:51.166114+0000","last_active":"2024-09-11T08:39:51.166149+0000","last_peered":"2024-09-11T08:39:51.166149+0000","last_clean":"2024-09-11T08:39:51.166149+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:39:51.166149+0000","last_undegraded":"2024-09-11T08:39:51.166149+0000","last_fullsized":"2024-09-11T08:39:51.166149+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.166051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.570790+0000","scrub_duration":0.0016440420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501268+0000","last_change":"2024-09-11T08:38:44.994259+0000","last_active":"2024-09-11T08:39:35.501268+0000","last_peered":"2024-09-11T08:39:35.501268+0000","last_clean":"2024-09-11T08:39:35.501268+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:39:35.501268+0000","last_undegraded":"2024-09-11T08:39:35.501268+0000","last_fullsized":"2024-09-11T08:39:35.501268+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:03.579441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:59.606552+0000","last_change":"2024-09-11T08:39:59.606521+0000","last_active":"2024-09-11T08:39:59.606552+0000","last_peered":"2024-09-11T08:39:59.606552+0000","last_clean":"2024-09-11T08:39:59.606552+0000","last_became_active":"2024-09-11T08:38:45.183364+0000","last_became_peered":"2024-09-11T08:38:45.183364+0000","last_unstale":"2024-09-11T08:39:59.606552+0000","last_undegraded":"2024-09-11T08:39:59.606552+0000","last_fullsized":"2024-09-11T08:39:59.606552+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:59.606458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:59.606458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.547769+0000","scrub_duration":0.0012065960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500980+0000","last_change":"2024-09-11T08:38:38.954075+0000","last_active":"2024-09-11T08:39:35.500980+0000","last_peered":"2024-09-11T08:39:35.500980+0000","last_clean":"2024-09-11T08:39:35.500980+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:39:35.500980+0000","last_undegraded":"2024-09-11T08:39:35.500980+0000","last_fullsized":"2024-09-11T08:39:35.500980+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:04.776009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":71,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:50.472453+0000","last_change":"2024-09-11T08:39:50.472419+0000","last_active":"2024-09-11T08:39:50.472453+0000","last_peered":"2024-09-11T08:39:50.472453+0000","last_clean":"2024-09-11T08:39:50.472453+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:39:50.472453+0000","last_undegraded":"2024-09-11T08:39:50.472453+0000","last_fullsized":"2024-09-11T08:39:50.472453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.472368+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:10.784548+0000","scrub_duration":0.0049809520000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:44.632395+0000","last_change":"2024-09-11T08:39:44.632369+0000","last_active":"2024-09-11T08:39:44.632395+0000","last_peered":"2024-09-11T08:39:44.632395+0000","last_clean":"2024-09-11T08:39:44.632395+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:39:44.632395+0000","last_undegraded":"2024-09-11T08:39:44.632395+0000","last_fullsized":"2024-09-11T08:39:44.632395+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:44.632308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:44.632308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:56.408810+0000","scrub_duration":0.001420377,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445687+0000","last_change":"2024-09-11T08:38:45.183646+0000","last_active":"2024-09-11T08:39:36.445687+0000","last_peered":"2024-09-11T08:39:36.445687+0000","last_clean":"2024-09-11T08:39:36.445687+0000","last_became_active":"2024-09-11T08:38:45.182763+0000","last_became_peered":"2024-09-11T08:38:45.182763+0000","last_unstale":"2024-09-11T08:39:36.445687+0000","last_undegraded":"2024-09-11T08:39:36.445687+0000","last_fullsized":"2024-09-11T08:39:36.445687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:04.100474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175475+0000","last_change":"2024-09-11T08:38:38.949654+0000","last_active":"2024-09-11T08:39:36.175475+0000","last_peered":"2024-09-11T08:39:36.175475+0000","last_clean":"2024-09-11T08:39:36.175475+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:39:36.175475+0000","last_undegraded":"2024-09-11T08:39:36.175475+0000","last_fullsized":"2024-09-11T08:39:36.175475+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:54.740531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:55.125478+0000","last_change":"2024-09-11T08:39:55.125449+0000","last_active":"2024-09-11T08:39:55.125478+0000","last_peered":"2024-09-11T08:39:55.125478+0000","last_clean":"2024-09-11T08:39:55.125478+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:39:55.125478+0000","last_undegraded":"2024-09-11T08:39:55.125478+0000","last_fullsized":"2024-09-11T08:39:55.125478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:55.125394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.125394+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.857707+0000","scrub_duration":0.0016833810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175174+0000","last_change":"2024-09-11T08:38:42.987327+0000","last_active":"2024-09-11T08:39:36.175174+0000","last_peered":"2024-09-11T08:39:36.175174+0000","last_clean":"2024-09-11T08:39:36.175174+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:39:36.175174+0000","last_undegraded":"2024-09-11T08:39:36.175174+0000","last_fullsized":"2024-09-11T08:39:36.175174+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.157159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842563+0000","last_change":"2024-09-11T08:38:44.993281+0000","last_active":"2024-09-11T08:39:35.842563+0000","last_peered":"2024-09-11T08:39:35.842563+0000","last_clean":"2024-09-11T08:39:35.842563+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:39:35.842563+0000","last_undegraded":"2024-09-11T08:39:35.842563+0000","last_fullsized":"2024-09-11T08:39:35.842563+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:55.539078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499751+0000","last_change":"2024-09-11T08:38:38.954288+0000","last_active":"2024-09-11T08:39:35.499751+0000","last_peered":"2024-09-11T08:39:35.499751+0000","last_clean":"2024-09-11T08:39:35.499751+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:39:35.499751+0000","last_undegraded":"2024-09-11T08:39:35.499751+0000","last_fullsized":"2024-09-11T08:39:35.499751+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:56.900737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500506+0000","last_change":"2024-09-11T08:38:41.041065+0000","last_active":"2024-09-11T08:39:35.500506+0000","last_peered":"2024-09-11T08:39:35.500506+0000","last_clean":"2024-09-11T08:39:35.500506+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:39:35.500506+0000","last_undegraded":"2024-09-11T08:39:35.500506+0000","last_fullsized":"2024-09-11T08:39:35.500506+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:53.140994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":53,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.687946+0000","last_change":"2024-09-11T08:38:42.985883+0000","last_active":"2024-09-11T08:39:52.687946+0000","last_peered":"2024-09-11T08:39:52.687946+0000","last_clean":"2024-09-11T08:39:52.687946+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:39:52.687946+0000","last_undegraded":"2024-09-11T08:39:52.687946+0000","last_fullsized":"2024-09-11T08:39:52.687946+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:05.221206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500487+0000","last_change":"2024-09-11T08:38:40.979334+0000","last_active":"2024-09-11T08:39:35.500487+0000","last_peered":"2024-09-11T08:39:35.500487+0000","last_clean":"2024-09-11T08:39:35.500487+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:39:35.500487+0000","last_undegraded":"2024-09-11T08:39:35.500487+0000","last_fullsized":"2024-09-11T08:39:35.500487+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:05.848565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445242+0000","last_change":"2024-09-11T08:38:38.951009+0000","last_active":"2024-09-11T08:39:36.445242+0000","last_peered":"2024-09-11T08:39:36.445242+0000","last_clean":"2024-09-11T08:39:36.445242+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:39:36.445242+0000","last_undegraded":"2024-09-11T08:39:36.445242+0000","last_fullsized":"2024-09-11T08:39:36.445242+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:53.872027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.687918+0000","last_change":"2024-09-11T08:38:42.985885+0000","last_active":"2024-09-11T08:39:52.687918+0000","last_peered":"2024-09-11T08:39:52.687918+0000","last_clean":"2024-09-11T08:39:52.687918+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:39:52.687918+0000","last_undegraded":"2024-09-11T08:39:52.687918+0000","last_fullsized":"2024-09-11T08:39:52.687918+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501622+0000","last_change":"2024-09-11T08:38:44.991592+0000","last_active":"2024-09-11T08:39:35.501622+0000","last_peered":"2024-09-11T08:39:35.501622+0000","last_clean":"2024-09-11T08:39:35.501622+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:39:35.501622+0000","last_undegraded":"2024-09-11T08:39:35.501622+0000","last_fullsized":"2024-09-11T08:39:35.501622+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:06.716187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445123+0000","last_change":"2024-09-11T08:38:41.040597+0000","last_active":"2024-09-11T08:39:36.445123+0000","last_peered":"2024-09-11T08:39:36.445123+0000","last_clean":"2024-09-11T08:39:36.445123+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:39:36.445123+0000","last_undegraded":"2024-09-11T08:39:36.445123+0000","last_fullsized":"2024-09-11T08:39:36.445123+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:47.766795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.037635+0000","last_change":"2024-09-11T08:39:43.037605+0000","last_active":"2024-09-11T08:39:43.037635+0000","last_peered":"2024-09-11T08:39:43.037635+0000","last_clean":"2024-09-11T08:39:43.037635+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:39:43.037635+0000","last_undegraded":"2024-09-11T08:39:43.037635+0000","last_fullsized":"2024-09-11T08:39:43.037635+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.037551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.037551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.071661+0000","scrub_duration":0.001523976,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.174914+0000","last_change":"2024-09-11T08:38:42.983881+0000","last_active":"2024-09-11T08:39:36.174914+0000","last_peered":"2024-09-11T08:39:36.174914+0000","last_clean":"2024-09-11T08:39:36.174914+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:39:36.174914+0000","last_undegraded":"2024-09-11T08:39:36.174914+0000","last_fullsized":"2024-09-11T08:39:36.174914+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:49.733060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175286+0000","last_change":"2024-09-11T08:38:44.995555+0000","last_active":"2024-09-11T08:39:36.175286+0000","last_peered":"2024-09-11T08:39:36.175286+0000","last_clean":"2024-09-11T08:39:36.175286+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:39:36.175286+0000","last_undegraded":"2024-09-11T08:39:36.175286+0000","last_fullsized":"2024-09-11T08:39:36.175286+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.851425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175199+0000","last_change":"2024-09-11T08:38:40.968377+0000","last_active":"2024-09-11T08:39:36.175199+0000","last_peered":"2024-09-11T08:39:36.175199+0000","last_clean":"2024-09-11T08:39:36.175199+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:39:36.175199+0000","last_undegraded":"2024-09-11T08:39:36.175199+0000","last_fullsized":"2024-09-11T08:39:36.175199+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:09.166194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:41.267346+0000","last_change":"2024-09-11T08:39:41.267320+0000","last_active":"2024-09-11T08:39:41.267346+0000","last_peered":"2024-09-11T08:39:41.267346+0000","last_clean":"2024-09-11T08:39:41.267346+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:39:41.267346+0000","last_undegraded":"2024-09-11T08:39:41.267346+0000","last_fullsized":"2024-09-11T08:39:41.267346+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:41.267267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.267267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:01.547087+0000","scrub_duration":0.0013348209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":53,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.687796+0000","last_change":"2024-09-11T08:38:42.982794+0000","last_active":"2024-09-11T08:39:52.687796+0000","last_peered":"2024-09-11T08:39:52.687796+0000","last_clean":"2024-09-11T08:39:52.687796+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:39:52.687796+0000","last_undegraded":"2024-09-11T08:39:52.687796+0000","last_fullsized":"2024-09-11T08:39:52.687796+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:06.036142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842473+0000","last_change":"2024-09-11T08:38:45.183016+0000","last_active":"2024-09-11T08:39:35.842473+0000","last_peered":"2024-09-11T08:39:35.842473+0000","last_clean":"2024-09-11T08:39:35.842473+0000","last_became_active":"2024-09-11T08:38:45.182724+0000","last_became_peered":"2024-09-11T08:38:45.182724+0000","last_unstale":"2024-09-11T08:39:35.842473+0000","last_undegraded":"2024-09-11T08:39:35.842473+0000","last_fullsized":"2024-09-11T08:39:35.842473+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:51.221296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"63'10","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499890+0000","last_change":"2024-09-11T08:38:40.977074+0000","last_active":"2024-09-11T08:39:35.499890+0000","last_peered":"2024-09-11T08:39:35.499890+0000","last_clean":"2024-09-11T08:39:35.499890+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:39:35.499890+0000","last_undegraded":"2024-09-11T08:39:35.499890+0000","last_fullsized":"2024-09-11T08:39:35.499890+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.468488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.502041+0000","last_change":"2024-09-11T08:38:38.951168+0000","last_active":"2024-09-11T08:39:35.502041+0000","last_peered":"2024-09-11T08:39:35.502041+0000","last_clean":"2024-09-11T08:39:35.502041+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:39:35.502041+0000","last_undegraded":"2024-09-11T08:39:35.502041+0000","last_fullsized":"2024-09-11T08:39:35.502041+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.717085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.687787+0000","last_change":"2024-09-11T08:38:42.981732+0000","last_active":"2024-09-11T08:39:52.687787+0000","last_peered":"2024-09-11T08:39:52.687787+0000","last_clean":"2024-09-11T08:39:52.687787+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:39:52.687787+0000","last_undegraded":"2024-09-11T08:39:52.687787+0000","last_fullsized":"2024-09-11T08:39:52.687787+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:08.306042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500551+0000","last_change":"2024-09-11T08:38:44.998257+0000","last_active":"2024-09-11T08:39:35.500551+0000","last_peered":"2024-09-11T08:39:35.500551+0000","last_clean":"2024-09-11T08:39:35.500551+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:39:35.500551+0000","last_undegraded":"2024-09-11T08:39:35.500551+0000","last_fullsized":"2024-09-11T08:39:35.500551+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:12.569737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":65,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500898+0000","last_change":"2024-09-11T08:38:41.040034+0000","last_active":"2024-09-11T08:39:35.500898+0000","last_peered":"2024-09-11T08:39:35.500898+0000","last_clean":"2024-09-11T08:39:35.500898+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:39:35.500898+0000","last_undegraded":"2024-09-11T08:39:35.500898+0000","last_fullsized":"2024-09-11T08:39:35.500898+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:07.747665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500207+0000","last_change":"2024-09-11T08:38:38.953053+0000","last_active":"2024-09-11T08:39:35.500207+0000","last_peered":"2024-09-11T08:39:35.500207+0000","last_clean":"2024-09-11T08:39:35.500207+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:39:35.500207+0000","last_undegraded":"2024-09-11T08:39:35.500207+0000","last_fullsized":"2024-09-11T08:39:35.500207+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:57.480705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:57.597021+0000","last_change":"2024-09-11T08:39:57.596980+0000","last_active":"2024-09-11T08:39:57.597021+0000","last_peered":"2024-09-11T08:39:57.597021+0000","last_clean":"2024-09-11T08:39:57.597021+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:39:57.597021+0000","last_undegraded":"2024-09-11T08:39:57.597021+0000","last_fullsized":"2024-09-11T08:39:57.597021+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.596889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.648355+0000","scrub_duration":0.001574424,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499567+0000","last_change":"2024-09-11T08:38:45.183159+0000","last_active":"2024-09-11T08:39:35.499567+0000","last_peered":"2024-09-11T08:39:35.499567+0000","last_clean":"2024-09-11T08:39:35.499567+0000","last_became_active":"2024-09-11T08:38:45.182782+0000","last_became_peered":"2024-09-11T08:38:45.182782+0000","last_unstale":"2024-09-11T08:39:35.499567+0000","last_undegraded":"2024-09-11T08:39:35.499567+0000","last_fullsized":"2024-09-11T08:39:35.499567+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:59.864364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":113,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:53.493055+0000","last_change":"2024-09-11T08:39:53.493004+0000","last_active":"2024-09-11T08:39:53.493055+0000","last_peered":"2024-09-11T08:39:53.493055+0000","last_clean":"2024-09-11T08:39:53.493055+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:39:53.493055+0000","last_undegraded":"2024-09-11T08:39:53.493055+0000","last_fullsized":"2024-09-11T08:39:53.493055+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.492945+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.774847+0000","scrub_duration":0.005988045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175554+0000","last_change":"2024-09-11T08:38:38.955616+0000","last_active":"2024-09-11T08:39:36.175554+0000","last_peered":"2024-09-11T08:39:36.175554+0000","last_clean":"2024-09-11T08:39:36.175554+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:39:36.175554+0000","last_undegraded":"2024-09-11T08:39:36.175554+0000","last_fullsized":"2024-09-11T08:39:36.175554+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:50.638474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:58.472902+0000","last_change":"2024-09-11T08:39:58.472851+0000","last_active":"2024-09-11T08:39:58.472902+0000","last_peered":"2024-09-11T08:39:58.472902+0000","last_clean":"2024-09-11T08:39:58.472902+0000","last_became_active":"2024-09-11T08:38:38.947270+0000","last_became_peered":"2024-09-11T08:38:38.947270+0000","last_unstale":"2024-09-11T08:39:58.472902+0000","last_undegraded":"2024-09-11T08:39:58.472902+0000","last_fullsized":"2024-09-11T08:39:58.472902+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:39:58.472785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.472785+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.725781+0000","scrub_duration":0.0020806930000000002,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501165+0000","last_change":"2024-09-11T08:38:42.975757+0000","last_active":"2024-09-11T08:39:35.501165+0000","last_peered":"2024-09-11T08:39:35.501165+0000","last_clean":"2024-09-11T08:39:35.501165+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:39:35.501165+0000","last_undegraded":"2024-09-11T08:39:35.501165+0000","last_fullsized":"2024-09-11T08:39:35.501165+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:50.821530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445649+0000","last_change":"2024-09-11T08:38:44.997146+0000","last_active":"2024-09-11T08:39:36.445649+0000","last_peered":"2024-09-11T08:39:36.445649+0000","last_clean":"2024-09-11T08:39:36.445649+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:39:36.445649+0000","last_undegraded":"2024-09-11T08:39:36.445649+0000","last_fullsized":"2024-09-11T08:39:36.445649+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.168511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":66,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.585203+0000","last_change":"2024-09-11T08:39:43.585175+0000","last_active":"2024-09-11T08:39:43.585203+0000","last_peered":"2024-09-11T08:39:43.585203+0000","last_clean":"2024-09-11T08:39:43.585203+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:39:43.585203+0000","last_undegraded":"2024-09-11T08:39:43.585203+0000","last_fullsized":"2024-09-11T08:39:43.585203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:39:43.585116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.585116+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.724395+0000","scrub_duration":0.0021210309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:00.500472+0000","last_change":"2024-09-11T08:40:00.500444+0000","last_active":"2024-09-11T08:40:00.500472+0000","last_peered":"2024-09-11T08:40:00.500472+0000","last_clean":"2024-09-11T08:40:00.500472+0000","last_became_active":"2024-09-11T08:38:38.949019+0000","last_became_peered":"2024-09-11T08:38:38.949019+0000","last_unstale":"2024-09-11T08:40:00.500472+0000","last_undegraded":"2024-09-11T08:40:00.500472+0000","last_fullsized":"2024-09-11T08:40:00.500472+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.500392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.500392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.354187+0000","scrub_duration":0.001294566,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.444967+0000","last_change":"2024-09-11T08:38:42.971425+0000","last_active":"2024-09-11T08:39:36.444967+0000","last_peered":"2024-09-11T08:39:36.444967+0000","last_clean":"2024-09-11T08:39:36.444967+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:39:36.444967+0000","last_undegraded":"2024-09-11T08:39:36.444967+0000","last_fullsized":"2024-09-11T08:39:36.444967+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:59.348177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445494+0000","last_change":"2024-09-11T08:38:42.980780+0000","last_active":"2024-09-11T08:39:36.445494+0000","last_peered":"2024-09-11T08:39:36.445494+0000","last_clean":"2024-09-11T08:39:36.445494+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:39:36.445494+0000","last_undegraded":"2024-09-11T08:39:36.445494+0000","last_fullsized":"2024-09-11T08:39:36.445494+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.591149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842581+0000","last_change":"2024-09-11T08:38:45.183842+0000","last_active":"2024-09-11T08:39:35.842581+0000","last_peered":"2024-09-11T08:39:35.842581+0000","last_clean":"2024-09-11T08:39:35.842581+0000","last_became_active":"2024-09-11T08:38:45.183582+0000","last_became_peered":"2024-09-11T08:38:45.183582+0000","last_unstale":"2024-09-11T08:39:35.842581+0000","last_undegraded":"2024-09-11T08:39:35.842581+0000","last_fullsized":"2024-09-11T08:39:35.842581+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:03.954910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500582+0000","last_change":"2024-09-11T08:38:40.975966+0000","last_active":"2024-09-11T08:39:35.500582+0000","last_peered":"2024-09-11T08:39:35.500582+0000","last_clean":"2024-09-11T08:39:35.500582+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:39:35.500582+0000","last_undegraded":"2024-09-11T08:39:35.500582+0000","last_fullsized":"2024-09-11T08:39:35.500582+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.626733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.356781+0000","last_change":"2024-09-11T08:39:43.356751+0000","last_active":"2024-09-11T08:39:43.356781+0000","last_peered":"2024-09-11T08:39:43.356781+0000","last_clean":"2024-09-11T08:39:43.356781+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:39:43.356781+0000","last_undegraded":"2024-09-11T08:39:43.356781+0000","last_fullsized":"2024-09-11T08:39:43.356781+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.356699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.356699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.569637+0000","scrub_duration":0.001649455,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:59.236810+0000","last_change":"2024-09-11T08:39:55.153362+0000","last_active":"2024-09-11T08:39:59.236810+0000","last_peered":"2024-09-11T08:39:59.236810+0000","last_clean":"2024-09-11T08:39:59.236810+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:39:59.236810+0000","last_undegraded":"2024-09-11T08:39:59.236810+0000","last_fullsized":"2024-09-11T08:39:59.236810+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:39:55.153290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.153290+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.446909+0000","scrub_duration":0.0017161990000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.502124+0000","last_change":"2024-09-11T08:38:42.976525+0000","last_active":"2024-09-11T08:39:35.502124+0000","last_peered":"2024-09-11T08:39:35.502124+0000","last_clean":"2024-09-11T08:39:35.502124+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:39:35.502124+0000","last_undegraded":"2024-09-11T08:39:35.502124+0000","last_fullsized":"2024-09-11T08:39:35.502124+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:09.438148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500105+0000","last_change":"2024-09-11T08:38:44.994290+0000","last_active":"2024-09-11T08:39:35.500105+0000","last_peered":"2024-09-11T08:39:35.500105+0000","last_clean":"2024-09-11T08:39:35.500105+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:39:35.500105+0000","last_undegraded":"2024-09-11T08:39:35.500105+0000","last_fullsized":"2024-09-11T08:39:35.500105+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.617197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":60,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:00.540221+0000","last_change":"2024-09-11T08:40:00.540192+0000","last_active":"2024-09-11T08:40:00.540221+0000","last_peered":"2024-09-11T08:40:00.540221+0000","last_clean":"2024-09-11T08:40:00.540221+0000","last_became_active":"2024-09-11T08:38:41.040948+0000","last_became_peered":"2024-09-11T08:38:41.040948+0000","last_unstale":"2024-09-11T08:40:00.540221+0000","last_undegraded":"2024-09-11T08:40:00.540221+0000","last_fullsized":"2024-09-11T08:40:00.540221+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:40:00.540136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.540136+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:07.651325+0000","scrub_duration":0.001785095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445564+0000","last_change":"2024-09-11T08:38:38.948242+0000","last_active":"2024-09-11T08:39:36.445564+0000","last_peered":"2024-09-11T08:39:36.445564+0000","last_clean":"2024-09-11T08:39:36.445564+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:39:36.445564+0000","last_undegraded":"2024-09-11T08:39:36.445564+0000","last_fullsized":"2024-09-11T08:39:36.445564+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:52.890961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":74,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842755+0000","last_change":"2024-09-11T08:39:12.765244+0000","last_active":"2024-09-11T08:39:35.842755+0000","last_peered":"2024-09-11T08:39:35.842755+0000","last_clean":"2024-09-11T08:39:35.842755+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:39:35.842755+0000","last_undegraded":"2024-09-11T08:39:35.842755+0000","last_fullsized":"2024-09-11T08:39:35.842755+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'72","last_scrub_stamp":"2024-09-11T08:39:12.765190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:39:12.765190+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:16.661831+0000","scrub_duration":0.0014709829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:49.116242+0000","last_change":"2024-09-11T08:39:49.116184+0000","last_active":"2024-09-11T08:39:49.116242+0000","last_peered":"2024-09-11T08:39:49.116242+0000","last_clean":"2024-09-11T08:39:49.116242+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:39:49.116242+0000","last_undegraded":"2024-09-11T08:39:49.116242+0000","last_fullsized":"2024-09-11T08:39:49.116242+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.116113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.116113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.209747+0000","scrub_duration":0.00167223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:47.423780+0000","last_change":"2024-09-11T08:39:47.423745+0000","last_active":"2024-09-11T08:39:47.423780+0000","last_peered":"2024-09-11T08:39:47.423780+0000","last_clean":"2024-09-11T08:39:47.423780+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:39:47.423780+0000","last_undegraded":"2024-09-11T08:39:47.423780+0000","last_fullsized":"2024-09-11T08:39:47.423780+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.423684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.423684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.004042+0000","scrub_duration":0.001458972,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175659+0000","last_change":"2024-09-11T08:38:40.973744+0000","last_active":"2024-09-11T08:39:36.175659+0000","last_peered":"2024-09-11T08:39:36.175659+0000","last_clean":"2024-09-11T08:39:36.175659+0000","last_became_active":"2024-09-11T08:38:40.973373+0000","last_became_peered":"2024-09-11T08:38:40.973373+0000","last_unstale":"2024-09-11T08:39:36.175659+0000","last_undegraded":"2024-09-11T08:39:36.175659+0000","last_fullsized":"2024-09-11T08:39:36.175659+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:07.389753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175244+0000","last_change":"2024-09-11T08:38:38.948244+0000","last_active":"2024-09-11T08:39:36.175244+0000","last_peered":"2024-09-11T08:39:36.175244+0000","last_clean":"2024-09-11T08:39:36.175244+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:39:36.175244+0000","last_undegraded":"2024-09-11T08:39:36.175244+0000","last_fullsized":"2024-09-11T08:39:36.175244+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:03.299757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501998+0000","last_change":"2024-09-11T08:38:42.982947+0000","last_active":"2024-09-11T08:39:35.501998+0000","last_peered":"2024-09-11T08:39:35.501998+0000","last_clean":"2024-09-11T08:39:35.501998+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:39:35.501998+0000","last_undegraded":"2024-09-11T08:39:35.501998+0000","last_fullsized":"2024-09-11T08:39:35.501998+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.506223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445494+0000","last_change":"2024-09-11T08:38:45.185064+0000","last_active":"2024-09-11T08:39:36.445494+0000","last_peered":"2024-09-11T08:39:36.445494+0000","last_clean":"2024-09-11T08:39:36.445494+0000","last_became_active":"2024-09-11T08:38:45.184410+0000","last_became_peered":"2024-09-11T08:38:45.184410+0000","last_unstale":"2024-09-11T08:39:36.445494+0000","last_undegraded":"2024-09-11T08:39:36.445494+0000","last_fullsized":"2024-09-11T08:39:36.445494+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:58.964836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":67,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.526881+0000","last_change":"2024-09-11T08:39:43.526845+0000","last_active":"2024-09-11T08:39:43.526881+0000","last_peered":"2024-09-11T08:39:43.526881+0000","last_clean":"2024-09-11T08:39:43.526881+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:39:43.526881+0000","last_undegraded":"2024-09-11T08:39:43.526881+0000","last_fullsized":"2024-09-11T08:39:43.526881+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:39:43.526763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.526763+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.869811+0000","scrub_duration":0.0022037329999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:55.599092+0000","last_change":"2024-09-11T08:39:55.599049+0000","last_active":"2024-09-11T08:39:55.599092+0000","last_peered":"2024-09-11T08:39:55.599092+0000","last_clean":"2024-09-11T08:39:55.599092+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:39:55.599092+0000","last_undegraded":"2024-09-11T08:39:55.599092+0000","last_fullsized":"2024-09-11T08:39:55.599092+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.598961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.598961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.562190+0000","scrub_duration":0.0016789820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:46.627109+0000","last_change":"2024-09-11T08:39:46.627077+0000","last_active":"2024-09-11T08:39:46.627109+0000","last_peered":"2024-09-11T08:39:46.627109+0000","last_clean":"2024-09-11T08:39:46.627109+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:39:46.627109+0000","last_undegraded":"2024-09-11T08:39:46.627109+0000","last_fullsized":"2024-09-11T08:39:46.627109+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:46.627028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:46.627028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:56.334041+0000","scrub_duration":0.0013824029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445462+0000","last_change":"2024-09-11T08:38:44.994162+0000","last_active":"2024-09-11T08:39:36.445462+0000","last_peered":"2024-09-11T08:39:36.445462+0000","last_clean":"2024-09-11T08:39:36.445462+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:39:36.445462+0000","last_undegraded":"2024-09-11T08:39:36.445462+0000","last_fullsized":"2024-09-11T08:39:36.445462+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:09.343122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175377+0000","last_change":"2024-09-11T08:38:40.969616+0000","last_active":"2024-09-11T08:39:36.175377+0000","last_peered":"2024-09-11T08:39:36.175377+0000","last_clean":"2024-09-11T08:39:36.175377+0000","last_became_active":"2024-09-11T08:38:40.969348+0000","last_became_peered":"2024-09-11T08:38:40.969348+0000","last_unstale":"2024-09-11T08:39:36.175377+0000","last_undegraded":"2024-09-11T08:39:36.175377+0000","last_fullsized":"2024-09-11T08:39:36.175377+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:52.841663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445860+0000","last_change":"2024-09-11T08:38:38.957214+0000","last_active":"2024-09-11T08:39:36.445860+0000","last_peered":"2024-09-11T08:39:36.445860+0000","last_clean":"2024-09-11T08:39:36.445860+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:39:36.445860+0000","last_undegraded":"2024-09-11T08:39:36.445860+0000","last_fullsized":"2024-09-11T08:39:36.445860+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:04.778506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:00.473638+0000","last_change":"2024-09-11T08:40:00.473606+0000","last_active":"2024-09-11T08:40:00.473638+0000","last_peered":"2024-09-11T08:40:00.473638+0000","last_clean":"2024-09-11T08:40:00.473638+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:40:00.473638+0000","last_undegraded":"2024-09-11T08:40:00.473638+0000","last_fullsized":"2024-09-11T08:40:00.473638+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:00.473549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.473549+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.300137+0000","scrub_duration":0.0016703060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175492+0000","last_change":"2024-09-11T08:38:44.993736+0000","last_active":"2024-09-11T08:39:36.175492+0000","last_peered":"2024-09-11T08:39:36.175492+0000","last_clean":"2024-09-11T08:39:36.175492+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:39:36.175492+0000","last_undegraded":"2024-09-11T08:39:36.175492+0000","last_fullsized":"2024-09-11T08:39:36.175492+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:55.062690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500820+0000","last_change":"2024-09-11T08:38:40.977113+0000","last_active":"2024-09-11T08:39:35.500820+0000","last_peered":"2024-09-11T08:39:35.500820+0000","last_clean":"2024-09-11T08:39:35.500820+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:39:35.500820+0000","last_undegraded":"2024-09-11T08:39:35.500820+0000","last_fullsized":"2024-09-11T08:39:35.500820+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:05.072088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:50.452607+0000","last_change":"2024-09-11T08:39:50.452568+0000","last_active":"2024-09-11T08:39:50.452607+0000","last_peered":"2024-09-11T08:39:50.452607+0000","last_clean":"2024-09-11T08:39:50.452607+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:39:50.452607+0000","last_undegraded":"2024-09-11T08:39:50.452607+0000","last_fullsized":"2024-09-11T08:39:50.452607+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.452494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.631278+0000","scrub_duration":0.001547086,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.444929+0000","last_change":"2024-09-11T08:38:42.983093+0000","last_active":"2024-09-11T08:39:36.444929+0000","last_peered":"2024-09-11T08:39:36.444929+0000","last_clean":"2024-09-11T08:39:36.444929+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:39:36.444929+0000","last_undegraded":"2024-09-11T08:39:36.444929+0000","last_fullsized":"2024-09-11T08:39:36.444929+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.830046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500230+0000","last_change":"2024-09-11T08:38:44.991908+0000","last_active":"2024-09-11T08:39:35.500230+0000","last_peered":"2024-09-11T08:39:35.500230+0000","last_clean":"2024-09-11T08:39:35.500230+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:39:35.500230+0000","last_undegraded":"2024-09-11T08:39:35.500230+0000","last_fullsized":"2024-09-11T08:39:35.500230+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:12.056728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":69,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:45.575270+0000","last_change":"2024-09-11T08:39:45.575209+0000","last_active":"2024-09-11T08:39:45.575270+0000","last_peered":"2024-09-11T08:39:45.575270+0000","last_clean":"2024-09-11T08:39:45.575270+0000","last_became_active":"2024-09-11T08:38:41.040207+0000","last_became_peered":"2024-09-11T08:38:41.040207+0000","last_unstale":"2024-09-11T08:39:45.575270+0000","last_undegraded":"2024-09-11T08:39:45.575270+0000","last_fullsized":"2024-09-11T08:39:45.575270+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:39:45.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.575133+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.182090+0000","scrub_duration":0.0022792160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:56.112155+0000","last_change":"2024-09-11T08:39:56.112112+0000","last_active":"2024-09-11T08:39:56.112155+0000","last_peered":"2024-09-11T08:39:56.112155+0000","last_clean":"2024-09-11T08:39:56.112155+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:39:56.112155+0000","last_undegraded":"2024-09-11T08:39:56.112155+0000","last_fullsized":"2024-09-11T08:39:56.112155+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:56.112043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.112043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:25.809354+0000","scrub_duration":0.001517137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445023+0000","last_change":"2024-09-11T08:38:42.985952+0000","last_active":"2024-09-11T08:39:36.445023+0000","last_peered":"2024-09-11T08:39:36.445023+0000","last_clean":"2024-09-11T08:39:36.445023+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:39:36.445023+0000","last_undegraded":"2024-09-11T08:39:36.445023+0000","last_fullsized":"2024-09-11T08:39:36.445023+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:06.325377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842722+0000","last_change":"2024-09-11T08:38:44.993458+0000","last_active":"2024-09-11T08:39:35.842722+0000","last_peered":"2024-09-11T08:39:35.842722+0000","last_clean":"2024-09-11T08:39:35.842722+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:39:35.842722+0000","last_undegraded":"2024-09-11T08:39:35.842722+0000","last_fullsized":"2024-09-11T08:39:35.842722+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:04.342365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:40.524639+0000","last_change":"2024-09-11T08:39:40.524581+0000","last_active":"2024-09-11T08:39:40.524639+0000","last_peered":"2024-09-11T08:39:40.524639+0000","last_clean":"2024-09-11T08:39:40.524639+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:39:40.524639+0000","last_undegraded":"2024-09-11T08:39:40.524639+0000","last_fullsized":"2024-09-11T08:39:40.524639+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:40.524498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:40.524498+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.186922+0000","scrub_duration":0.0024035010000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:00.113613+0000","last_change":"2024-09-11T08:40:00.113587+0000","last_active":"2024-09-11T08:40:00.113613+0000","last_peered":"2024-09-11T08:40:00.113613+0000","last_clean":"2024-09-11T08:40:00.113613+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:40:00.113613+0000","last_undegraded":"2024-09-11T08:40:00.113613+0000","last_fullsized":"2024-09-11T08:40:00.113613+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.113534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.113534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.476975+0000","scrub_duration":0.0011790780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:57.454285+0000","last_change":"2024-09-11T08:39:57.454257+0000","last_active":"2024-09-11T08:39:57.454285+0000","last_peered":"2024-09-11T08:39:57.454285+0000","last_clean":"2024-09-11T08:39:57.454285+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:39:57.454285+0000","last_undegraded":"2024-09-11T08:39:57.454285+0000","last_fullsized":"2024-09-11T08:39:57.454285+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.454198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.454198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.731898+0000","scrub_duration":0.0011666669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445542+0000","last_change":"2024-09-11T08:38:44.998686+0000","last_active":"2024-09-11T08:39:36.445542+0000","last_peered":"2024-09-11T08:39:36.445542+0000","last_clean":"2024-09-11T08:39:36.445542+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:39:36.445542+0000","last_undegraded":"2024-09-11T08:39:36.445542+0000","last_fullsized":"2024-09-11T08:39:36.445542+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:02.320682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:48.462270+0000","last_change":"2024-09-11T08:39:48.462233+0000","last_active":"2024-09-11T08:39:48.462270+0000","last_peered":"2024-09-11T08:39:48.462270+0000","last_clean":"2024-09-11T08:39:48.462270+0000","last_became_active":"2024-09-11T08:38:41.040475+0000","last_became_peered":"2024-09-11T08:38:41.040475+0000","last_unstale":"2024-09-11T08:39:48.462270+0000","last_undegraded":"2024-09-11T08:39:48.462270+0000","last_fullsized":"2024-09-11T08:39:48.462270+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-11T08:39:48.462163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:48.462163+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.032207+0000","scrub_duration":0.002094753,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.843191+0000","last_change":"2024-09-11T08:38:38.956109+0000","last_active":"2024-09-11T08:39:35.843191+0000","last_peered":"2024-09-11T08:39:35.843191+0000","last_clean":"2024-09-11T08:39:35.843191+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:39:35.843191+0000","last_undegraded":"2024-09-11T08:39:35.843191+0000","last_fullsized":"2024-09-11T08:39:35.843191+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:50.337939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"63'8","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.688011+0000","last_change":"2024-09-11T08:38:42.987044+0000","last_active":"2024-09-11T08:39:52.688011+0000","last_peered":"2024-09-11T08:39:52.688011+0000","last_clean":"2024-09-11T08:39:52.688011+0000","last_became_active":"2024-09-11T08:38:42.986756+0000","last_became_peered":"2024-09-11T08:38:42.986756+0000","last_unstale":"2024-09-11T08:39:52.688011+0000","last_undegraded":"2024-09-11T08:39:52.688011+0000","last_fullsized":"2024-09-11T08:39:52.688011+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501304+0000","last_change":"2024-09-11T08:38:45.182987+0000","last_active":"2024-09-11T08:39:35.501304+0000","last_peered":"2024-09-11T08:39:35.501304+0000","last_clean":"2024-09-11T08:39:35.501304+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:39:35.501304+0000","last_undegraded":"2024-09-11T08:39:35.501304+0000","last_fullsized":"2024-09-11T08:39:35.501304+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:58.541554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:42.605410+0000","last_change":"2024-09-11T08:39:42.605373+0000","last_active":"2024-09-11T08:39:42.605410+0000","last_peered":"2024-09-11T08:39:42.605410+0000","last_clean":"2024-09-11T08:39:42.605410+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:39:42.605410+0000","last_undegraded":"2024-09-11T08:39:42.605410+0000","last_fullsized":"2024-09-11T08:39:42.605410+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:42.605301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:42.605301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:53.937210+0000","scrub_duration":0.0014798509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445861+0000","last_change":"2024-09-11T08:38:41.039820+0000","last_active":"2024-09-11T08:39:36.445861+0000","last_peered":"2024-09-11T08:39:36.445861+0000","last_clean":"2024-09-11T08:39:36.445861+0000","last_became_active":"2024-09-11T08:38:41.039552+0000","last_became_peered":"2024-09-11T08:38:41.039552+0000","last_unstale":"2024-09-11T08:39:36.445861+0000","last_undegraded":"2024-09-11T08:39:36.445861+0000","last_fullsized":"2024-09-11T08:39:36.445861+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:51.272516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:49.472804+0000","last_change":"2024-09-11T08:39:49.472764+0000","last_active":"2024-09-11T08:39:49.472804+0000","last_peered":"2024-09-11T08:39:49.472804+0000","last_clean":"2024-09-11T08:39:49.472804+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:39:49.472804+0000","last_undegraded":"2024-09-11T08:39:49.472804+0000","last_fullsized":"2024-09-11T08:39:49.472804+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.472696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.472696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:01.403308+0000","scrub_duration":0.001919104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842528+0000","last_change":"2024-09-11T08:38:45.185016+0000","last_active":"2024-09-11T08:39:35.842528+0000","last_peered":"2024-09-11T08:39:35.842528+0000","last_clean":"2024-09-11T08:39:35.842528+0000","last_became_active":"2024-09-11T08:38:45.184753+0000","last_became_peered":"2024-09-11T08:38:45.184753+0000","last_unstale":"2024-09-11T08:39:35.842528+0000","last_undegraded":"2024-09-11T08:39:35.842528+0000","last_fullsized":"2024-09-11T08:39:35.842528+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:13.726221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500656+0000","last_change":"2024-09-11T08:38:41.041476+0000","last_active":"2024-09-11T08:39:35.500656+0000","last_peered":"2024-09-11T08:39:35.500656+0000","last_clean":"2024-09-11T08:39:35.500656+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:39:35.500656+0000","last_undegraded":"2024-09-11T08:39:35.500656+0000","last_fullsized":"2024-09-11T08:39:35.500656+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:04.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:53.176120+0000","last_change":"2024-09-11T08:39:53.176095+0000","last_active":"2024-09-11T08:39:53.176120+0000","last_peered":"2024-09-11T08:39:53.176120+0000","last_clean":"2024-09-11T08:39:53.176120+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:39:53.176120+0000","last_undegraded":"2024-09-11T08:39:53.176120+0000","last_fullsized":"2024-09-11T08:39:53.176120+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.176035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.176035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:00.682857+0000","scrub_duration":0.001147531,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445708+0000","last_change":"2024-09-11T08:38:42.982083+0000","last_active":"2024-09-11T08:39:36.445708+0000","last_peered":"2024-09-11T08:39:36.445708+0000","last_clean":"2024-09-11T08:39:36.445708+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:39:36.445708+0000","last_undegraded":"2024-09-11T08:39:36.445708+0000","last_fullsized":"2024-09-11T08:39:36.445708+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:58.890541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.493090+0000","last_change":"2024-09-11T08:39:52.493064+0000","last_active":"2024-09-11T08:39:52.493090+0000","last_peered":"2024-09-11T08:39:52.493090+0000","last_clean":"2024-09-11T08:39:52.493090+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:39:52.493090+0000","last_undegraded":"2024-09-11T08:39:52.493090+0000","last_fullsized":"2024-09-11T08:39:52.493090+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.493013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:13.717323+0000","scrub_duration":0.0012699160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501853+0000","last_change":"2024-09-11T08:38:41.042115+0000","last_active":"2024-09-11T08:39:35.501853+0000","last_peered":"2024-09-11T08:39:35.501853+0000","last_clean":"2024-09-11T08:39:35.501853+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:39:35.501853+0000","last_undegraded":"2024-09-11T08:39:35.501853+0000","last_fullsized":"2024-09-11T08:39:35.501853+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.772561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500193+0000","last_change":"2024-09-11T08:38:38.958515+0000","last_active":"2024-09-11T08:39:35.500193+0000","last_peered":"2024-09-11T08:39:35.500193+0000","last_clean":"2024-09-11T08:39:35.500193+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:39:35.500193+0000","last_undegraded":"2024-09-11T08:39:35.500193+0000","last_fullsized":"2024-09-11T08:39:35.500193+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:53.785774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.687834+0000","last_change":"2024-09-11T08:38:42.981915+0000","last_active":"2024-09-11T08:39:52.687834+0000","last_peered":"2024-09-11T08:39:52.687834+0000","last_clean":"2024-09-11T08:39:52.687834+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:39:52.687834+0000","last_undegraded":"2024-09-11T08:39:52.687834+0000","last_fullsized":"2024-09-11T08:39:52.687834+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175675+0000","last_change":"2024-09-11T08:38:44.993997+0000","last_active":"2024-09-11T08:39:36.175675+0000","last_peered":"2024-09-11T08:39:36.175675+0000","last_clean":"2024-09-11T08:39:36.175675+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:39:36.175675+0000","last_undegraded":"2024-09-11T08:39:36.175675+0000","last_fullsized":"2024-09-11T08:39:36.175675+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:59.445794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":43,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445977+0000","last_change":"2024-09-11T08:38:41.040424+0000","last_active":"2024-09-11T08:39:36.445977+0000","last_peered":"2024-09-11T08:39:36.445977+0000","last_clean":"2024-09-11T08:39:36.445977+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:39:36.445977+0000","last_undegraded":"2024-09-11T08:39:36.445977+0000","last_fullsized":"2024-09-11T08:39:36.445977+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:53.450115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842811+0000","last_change":"2024-09-11T08:38:38.949874+0000","last_active":"2024-09-11T08:39:35.842811+0000","last_peered":"2024-09-11T08:39:35.842811+0000","last_clean":"2024-09-11T08:39:35.842811+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:39:35.842811+0000","last_undegraded":"2024-09-11T08:39:35.842811+0000","last_fullsized":"2024-09-11T08:39:35.842811+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:02.574659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.687975+0000","last_change":"2024-09-11T08:39:44.429546+0000","last_active":"2024-09-11T08:39:52.687975+0000","last_peered":"2024-09-11T08:39:52.687975+0000","last_clean":"2024-09-11T08:39:52.687975+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:39:52.687975+0000","last_undegraded":"2024-09-11T08:39:52.687975+0000","last_fullsized":"2024-09-11T08:39:52.687975+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:39:44.429477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:44.429477+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.023544+0000","scrub_duration":0.001890784,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.254325+0000","last_change":"2024-09-11T08:39:52.254299+0000","last_active":"2024-09-11T08:39:52.254325+0000","last_peered":"2024-09-11T08:39:52.254325+0000","last_clean":"2024-09-11T08:39:52.254325+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:39:52.254325+0000","last_undegraded":"2024-09-11T08:39:52.254325+0000","last_fullsized":"2024-09-11T08:39:52.254325+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.254245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.254245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:20.782062+0000","scrub_duration":0.001581877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175203+0000","last_change":"2024-09-11T08:38:40.973815+0000","last_active":"2024-09-11T08:39:36.175203+0000","last_peered":"2024-09-11T08:39:36.175203+0000","last_clean":"2024-09-11T08:39:36.175203+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:39:36.175203+0000","last_undegraded":"2024-09-11T08:39:36.175203+0000","last_fullsized":"2024-09-11T08:39:36.175203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.866315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175392+0000","last_change":"2024-09-11T08:38:38.952604+0000","last_active":"2024-09-11T08:39:36.175392+0000","last_peered":"2024-09-11T08:39:36.175392+0000","last_clean":"2024-09-11T08:39:36.175392+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:39:36.175392+0000","last_undegraded":"2024-09-11T08:39:36.175392+0000","last_fullsized":"2024-09-11T08:39:36.175392+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:51.807236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499809+0000","last_change":"2024-09-11T08:38:42.979138+0000","last_active":"2024-09-11T08:39:35.499809+0000","last_peered":"2024-09-11T08:39:35.499809+0000","last_clean":"2024-09-11T08:39:35.499809+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:39:35.499809+0000","last_undegraded":"2024-09-11T08:39:35.499809+0000","last_fullsized":"2024-09-11T08:39:35.499809+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:08.203025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.444988+0000","last_change":"2024-09-11T08:38:44.994241+0000","last_active":"2024-09-11T08:39:36.444988+0000","last_peered":"2024-09-11T08:39:36.444988+0000","last_clean":"2024-09-11T08:39:36.444988+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:39:36.444988+0000","last_undegraded":"2024-09-11T08:39:36.444988+0000","last_fullsized":"2024-09-11T08:39:36.444988+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:02.265888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499942+0000","last_change":"2024-09-11T08:38:40.976003+0000","last_active":"2024-09-11T08:39:35.499942+0000","last_peered":"2024-09-11T08:39:35.499942+0000","last_clean":"2024-09-11T08:39:35.499942+0000","last_became_active":"2024-09-11T08:38:40.975709+0000","last_became_peered":"2024-09-11T08:38:40.975709+0000","last_unstale":"2024-09-11T08:39:35.499942+0000","last_undegraded":"2024-09-11T08:39:35.499942+0000","last_fullsized":"2024-09-11T08:39:35.499942+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.631901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842846+0000","last_change":"2024-09-11T08:38:38.954570+0000","last_active":"2024-09-11T08:39:35.842846+0000","last_peered":"2024-09-11T08:39:35.842846+0000","last_clean":"2024-09-11T08:39:35.842846+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:39:35.842846+0000","last_undegraded":"2024-09-11T08:39:35.842846+0000","last_fullsized":"2024-09-11T08:39:35.842846+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:05.920652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445629+0000","last_change":"2024-09-11T08:38:42.984381+0000","last_active":"2024-09-11T08:39:36.445629+0000","last_peered":"2024-09-11T08:39:36.445629+0000","last_clean":"2024-09-11T08:39:36.445629+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:39:36.445629+0000","last_undegraded":"2024-09-11T08:39:36.445629+0000","last_fullsized":"2024-09-11T08:39:36.445629+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:54.394345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499567+0000","last_change":"2024-09-11T08:38:44.999759+0000","last_active":"2024-09-11T08:39:35.499567+0000","last_peered":"2024-09-11T08:39:35.499567+0000","last_clean":"2024-09-11T08:39:35.499567+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:39:35.499567+0000","last_undegraded":"2024-09-11T08:39:35.499567+0000","last_fullsized":"2024-09-11T08:39:35.499567+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:03.249276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445088+0000","last_change":"2024-09-11T08:38:41.039878+0000","last_active":"2024-09-11T08:39:36.445088+0000","last_peered":"2024-09-11T08:39:36.445088+0000","last_clean":"2024-09-11T08:39:36.445088+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:39:36.445088+0000","last_undegraded":"2024-09-11T08:39:36.445088+0000","last_fullsized":"2024-09-11T08:39:36.445088+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:08.136760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501978+0000","last_change":"2024-09-11T08:38:38.951115+0000","last_active":"2024-09-11T08:39:35.501978+0000","last_peered":"2024-09-11T08:39:35.501978+0000","last_clean":"2024-09-11T08:39:35.501978+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:39:35.501978+0000","last_undegraded":"2024-09-11T08:39:35.501978+0000","last_fullsized":"2024-09-11T08:39:35.501978+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:51.243171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:47.104746+0000","last_change":"2024-09-11T08:39:47.104702+0000","last_active":"2024-09-11T08:39:47.104746+0000","last_peered":"2024-09-11T08:39:47.104746+0000","last_clean":"2024-09-11T08:39:47.104746+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:39:47.104746+0000","last_undegraded":"2024-09-11T08:39:47.104746+0000","last_fullsized":"2024-09-11T08:39:47.104746+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.104637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.104637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:51.311803+0000","scrub_duration":0.0015238020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445776+0000","last_change":"2024-09-11T08:38:45.183913+0000","last_active":"2024-09-11T08:39:36.445776+0000","last_peered":"2024-09-11T08:39:36.445776+0000","last_clean":"2024-09-11T08:39:36.445776+0000","last_became_active":"2024-09-11T08:38:45.183548+0000","last_became_peered":"2024-09-11T08:38:45.183548+0000","last_unstale":"2024-09-11T08:39:36.445776+0000","last_undegraded":"2024-09-11T08:39:36.445776+0000","last_fullsized":"2024-09-11T08:39:36.445776+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:07.678681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":44,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:41.989766+0000","last_change":"2024-09-11T08:39:41.989740+0000","last_active":"2024-09-11T08:39:41.989766+0000","last_peered":"2024-09-11T08:39:41.989766+0000","last_clean":"2024-09-11T08:39:41.989766+0000","last_became_active":"2024-09-11T08:38:40.973799+0000","last_became_peered":"2024-09-11T08:38:40.973799+0000","last_unstale":"2024-09-11T08:39:41.989766+0000","last_undegraded":"2024-09-11T08:39:41.989766+0000","last_fullsized":"2024-09-11T08:39:41.989766+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-11T08:39:41.989684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.989684+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.687182+0000","scrub_duration":0.00162959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500414+0000","last_change":"2024-09-11T08:38:38.956548+0000","last_active":"2024-09-11T08:39:35.500414+0000","last_peered":"2024-09-11T08:39:35.500414+0000","last_clean":"2024-09-11T08:39:35.500414+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:39:35.500414+0000","last_undegraded":"2024-09-11T08:39:35.500414+0000","last_fullsized":"2024-09-11T08:39:35.500414+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:50.245039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:53.541675+0000","last_change":"2024-09-11T08:39:53.541628+0000","last_active":"2024-09-11T08:39:53.541675+0000","last_peered":"2024-09-11T08:39:53.541675+0000","last_clean":"2024-09-11T08:39:53.541675+0000","last_became_active":"2024-09-11T08:38:42.989556+0000","last_became_peered":"2024-09-11T08:38:42.989556+0000","last_unstale":"2024-09-11T08:39:53.541675+0000","last_undegraded":"2024-09-11T08:39:53.541675+0000","last_fullsized":"2024-09-11T08:39:53.541675+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.541557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.541557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:08.967817+0000","scrub_duration":0.001310285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842390+0000","last_change":"2024-09-11T08:38:44.993185+0000","last_active":"2024-09-11T08:39:35.842390+0000","last_peered":"2024-09-11T08:39:35.842390+0000","last_clean":"2024-09-11T08:39:35.842390+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:39:35.842390+0000","last_undegraded":"2024-09-11T08:39:35.842390+0000","last_fullsized":"2024-09-11T08:39:35.842390+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:53.658762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.480694+0000","last_change":"2024-09-11T08:39:52.480663+0000","last_active":"2024-09-11T08:39:52.480694+0000","last_peered":"2024-09-11T08:39:52.480694+0000","last_clean":"2024-09-11T08:39:52.480694+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:39:52.480694+0000","last_undegraded":"2024-09-11T08:39:52.480694+0000","last_fullsized":"2024-09-11T08:39:52.480694+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.480610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.480610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:55.608483+0000","scrub_duration":0.001335754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:53.038381+0000","last_change":"2024-09-11T08:39:53.038356+0000","last_active":"2024-09-11T08:39:53.038381+0000","last_peered":"2024-09-11T08:39:53.038381+0000","last_clean":"2024-09-11T08:39:53.038381+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:39:53.038381+0000","last_undegraded":"2024-09-11T08:39:53.038381+0000","last_fullsized":"2024-09-11T08:39:53.038381+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.038297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.038297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:58.424182+0000","scrub_duration":0.0012198549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.089846+0000","last_change":"2024-09-11T08:38:40.974827+0000","last_active":"2024-09-11T08:39:36.089846+0000","last_peered":"2024-09-11T08:39:36.089846+0000","last_clean":"2024-09-11T08:39:36.089846+0000","last_became_active":"2024-09-11T08:38:40.974451+0000","last_became_peered":"2024-09-11T08:38:40.974451+0000","last_unstale":"2024-09-11T08:39:36.089846+0000","last_undegraded":"2024-09-11T08:39:36.089846+0000","last_fullsized":"2024-09-11T08:39:36.089846+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:02.126833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:55.265182+0000","last_change":"2024-09-11T08:39:55.265155+0000","last_active":"2024-09-11T08:39:55.265182+0000","last_peered":"2024-09-11T08:39:55.265182+0000","last_clean":"2024-09-11T08:39:55.265182+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:39:55.265182+0000","last_undegraded":"2024-09-11T08:39:55.265182+0000","last_fullsized":"2024-09-11T08:39:55.265182+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.265097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.265097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.333633+0000","scrub_duration":0.0012893830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":423,"ondisk_log_size":423,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299414,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298788,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446364,"statfs":{"total":95995035648,"available":95689076736,"internally_reserved":0,"allocated":1470464,"data_stored":983979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.28899999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.377}]},{"osd":4,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.42999999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.73399999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.60499999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528347,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298768,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446384,"statfs":{"total":95995035648,"available":95689097216,"internally_reserved":0,"allocated":1449984,"data_stored":983206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.41299999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.42799999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.45000000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.437},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.505}]},{"osd":4,"last update":"Wed Sep 11 08:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.51100000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.38}]},{"osd":7,"last update":"Wed Sep 11 08:39:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.48899999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757279,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298204,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446948,"statfs":{"total":95995035648,"available":95689674752,"internally_reserved":0,"allocated":872448,"data_stored":394134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40899999999999997}]},{"osd":1,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.29899999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.32800000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.378}]},{"osd":4,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.376},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.311}]},{"osd":6,"last update":"Wed Sep 11 08:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.28499999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.41699999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298204,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446948,"statfs":{"total":95995035648,"available":95689674752,"internally_reserved":0,"allocated":872448,"data_stored":392829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.56299999999999994,"15min":0.57299999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68899999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.50700000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55800000000000005,"15min":0.56399999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":0.754,"15min":0.754},"last":0.48999999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70799999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.71099999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67100000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75800000000000001,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.47799999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.40500000000000003,"15min":0.38700000000000001},"min":{"1min":0.32100000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.30099999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.35999999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.374}]},{"osd":7,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.38500000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149737,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":901120,"data_stored":394654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.38},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.53200000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40300000000000002,"15min":0.40500000000000003},"min":{"1min":0.29399999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.52900000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.318}]},{"osd":1,"last update":"Wed Sep 11 08:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.40600000000000003,"15min":0.40899999999999997},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51100000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.38},{"interface":"front","average":{"1min":0.433,"5min":0.42199999999999999,"15min":0.41999999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.33900000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.443,"15min":0.441},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.435,"15min":0.438},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.29699999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40699999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.752,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40400000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.45100000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.441},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.55700000000000005}]},{"osd":7,"last update":"Wed Sep 11 08:39:27 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.496}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298180,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446972,"statfs":{"total":95995035648,"available":95689699328,"internally_reserved":0,"allocated":847872,"data_stored":392013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36499999999999999,"15min":0.35399999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.23899999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.25800000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.39100000000000001,"15min":0.376},"min":{"1min":0.28999999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.27000000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:39:57 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.29899999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.441,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:39:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.435},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.44500000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.40699999999999997}]},{"osd":7,"last update":"Wed Sep 11 08:39:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.501}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298240,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446912,"statfs":{"total":95995035648,"available":95689637888,"internally_reserved":0,"allocated":909312,"data_stored":393938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.32700000000000001,"15min":0.313},"min":{"1min":0.24199999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.49199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.378},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.34499999999999997,"15min":0.33000000000000002},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.47599999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:39:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.42199999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:39:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40400000000000003,"15min":0.41999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54400000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.433,"15min":0.434},"min":{"1min":0.307,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.47599999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79900000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48899999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:39:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.755,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82199999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.499}]},{"osd":6,"last update":"Wed Sep 11 08:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.66500000000000004}]},{"osd":7,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.622}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298792,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446360,"statfs":{"total":95995035648,"available":95689072640,"internally_reserved":0,"allocated":1474560,"data_stored":984038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.32300000000000001,"15min":0.31},"min":{"1min":0.20499999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.33400000000000002,"15min":0.312},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.34499999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.35799999999999998,"15min":0.34699999999999998},"min":{"1min":0.24199999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53500000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.34899999999999998},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.51500000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.30499999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:39:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39800000000000002,"15min":0.40000000000000002},"min":{"1min":0.28499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.503,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.28999999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69699999999999995,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.41099999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.45800000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.432}]},{"osd":7,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.53300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:40:03.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:40:03.798 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.538463379Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-11T08:40:03.798 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.540953498Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-11T08:40:03.798 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.543733694Z level=info msg="Executing migration" id="create alert_image table" 2024-09-11T08:40:03.798 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.546447745Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-11T08:40:03.799 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.549013129Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-11T08:40:03.799 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.551745503Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-11T08:40:03.799 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.55430291Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-11T08:40:03.799 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.554981787Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T08:40:03.799 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.555075864Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T08:40:03.799 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.55757374Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-11T08:40:03.799 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.560374615Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-11T08:40:03.799 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.563013048Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-11T08:40:03.799 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.563144238Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-11T08:40:03.799 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.563263513Z level=info msg="alerts found to migrate" alerts=0 2024-09-11T08:40:03.799 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.565660742Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-11T08:40:03.799 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.568332092Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-11T08:40:03.799 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.570935534Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-11T08:40:03.799 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.57161988Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-11T08:40:03.800 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.572406078Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-11T08:40:03.800 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.573207828Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-11T08:40:03.800 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.574109978Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-11T08:40:03.800 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.574855135Z level=info msg="Executing migration" id="create data_keys table" 2024-09-11T08:40:03.801 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.575803448Z level=info msg="Executing migration" id="create secrets table" 2024-09-11T08:40:03.801 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.576476156Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-11T08:40:03.801 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.580757637Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-11T08:40:03.801 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.581666914Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-11T08:40:03.801 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.582615745Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-11T08:40:03.801 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.586835113Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-11T08:40:03.801 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.591054468Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-11T08:40:03.801 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.591853648Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-11T08:40:03.801 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.592829322Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-11T08:40:03.801 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.593513111Z level=info msg="Executing migration" id="create permission table" 2024-09-11T08:40:03.801 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.594333164Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-11T08:40:03.801 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.595126237Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-11T08:40:03.802 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.595916417Z level=info msg="Executing migration" id="create role table" 2024-09-11T08:40:03.802 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.59676702Z level=info msg="Executing migration" id="add column display_name" 2024-09-11T08:40:03.802 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.597690795Z level=info msg="Executing migration" id="add column group_name" 2024-09-11T08:40:03.802 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.59862758Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-11T08:40:03.802 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.599512356Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-11T08:40:03.802 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.600480098Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-11T08:40:03.802 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.60116008Z level=info msg="Executing migration" id="create team role table" 2024-09-11T08:40:03.802 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.601984373Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-11T08:40:03.802 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.602894642Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-11T08:40:03.802 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.603745142Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-11T08:40:03.802 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.604529522Z level=info msg="Executing migration" id="create user role table" 2024-09-11T08:40:03.802 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.605292067Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-11T08:40:03.802 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.606148242Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-11T08:40:03.802 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.607005231Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-11T08:40:03.803 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.607891746Z level=info msg="Executing migration" id="create builtin role table" 2024-09-11T08:40:03.803 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.608765014Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-11T08:40:03.803 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.609528974Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-11T08:40:03.803 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.610387244Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-11T08:40:03.803 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.611362605Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-11T08:40:03.803 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.613163543Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-11T08:40:03.803 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.615805011Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-11T08:40:03.803 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.616378365Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-11T08:40:03.803 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.616809848Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-11T08:40:03.803 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.61722039Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-11T08:40:03.803 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.617693457Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-11T08:40:03.803 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.61865094Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-11T08:40:03.803 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.621304416Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-11T08:40:03.804 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.621754054Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-11T08:40:03.804 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.622026094Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-11T08:40:03.804 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.622448419Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-11T08:40:03.804 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.622918457Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-11T08:40:03.804 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.623317878Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-11T08:40:03.804 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.623613184Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-11T08:40:03.804 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.62395515Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-11T08:40:03.804 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.624367997Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-11T08:40:03.804 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.624811769Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-11T08:40:03.804 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.625774028Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-11T08:40:03.804 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.626088574Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-11T08:40:03.804 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.626506833Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-11T08:40:03.804 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.626945585Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-11T08:40:03.805 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.627403631Z level=info msg="Executing migration" id="add correlation config column" 2024-09-11T08:40:03.805 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.62842884Z level=info msg="Executing migration" id="create entity_events table" 2024-09-11T08:40:03.805 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.628858977Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-11T08:40:03.805 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.629602176Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:40:03.805 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.629778308Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:40:03.806 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.630196364Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:40:03.806 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.630357072Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:40:03.806 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.631679695Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-11T08:40:03.806 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.634436971Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-11T08:40:03.806 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.637162899Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:40:03.806 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.637759974Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:40:03.806 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.640649045Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-11T08:40:03.806 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.643377337Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T08:40:03.806 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.646056413Z level=info msg="Executing migration" id="Drop public config table" 2024-09-11T08:40:03.807 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.648747028Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-11T08:40:03.807 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.651665713Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-11T08:40:03.807 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.654207823Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T08:40:03.807 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.657021852Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-11T08:40:03.807 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.659648528Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-11T08:40:03.807 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.66267449Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-11T08:40:03.807 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.665399411Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-11T08:40:03.808 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.668042905Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-11T08:40:03.808 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.670633311Z level=info msg="Executing migration" id="add share column" 2024-09-11T08:40:03.808 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.673213902Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-11T08:40:03.808 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.676010789Z level=info msg="Executing migration" id="create file table" 2024-09-11T08:40:03.808 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.678724678Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-11T08:40:03.808 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.681280544Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-11T08:40:03.809 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.68404737Z level=info msg="Executing migration" id="create file_meta table" 2024-09-11T08:40:03.809 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.684457341Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-11T08:40:03.809 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.685285832Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-11T08:40:03.809 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.686142786Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-11T08:40:03.809 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.686616727Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-11T08:40:03.809 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.687013079Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-11T08:40:03.810 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.687477676Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-11T08:40:03.810 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.688548098Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-11T08:40:03.810 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.688889427Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-11T08:40:03.810 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.689722195Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-11T08:40:03.810 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.690568253Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-11T08:40:03.810 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.690928607Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-11T08:40:03.811 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.691811167Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-11T08:40:03.811 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.692915541Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-11T08:40:03.811 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.694943938Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-11T08:40:03.811 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.697594249Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-11T08:40:03.811 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.71377299Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-11T08:40:03.812 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.714443002Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-11T08:40:03.812 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.717378033Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-11T08:40:03.812 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.720891606Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-11T08:40:03.812 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.723532253Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-11T08:40:03.812 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.726521625Z level=info msg="Executing migration" id="create folder table" 2024-09-11T08:40:03.812 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.72926184Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-11T08:40:03.813 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.731991394Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-11T08:40:03.813 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.734798481Z level=info msg="Executing migration" id="Update folder title length" 2024-09-11T08:40:03.813 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.737559163Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-11T08:40:03.813 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=migrator t=2024-09-11T08:40:03.7402811Z level=info msg="migrations completed" performed=485 skipped=0 duration=707.636465ms 2024-09-11T08:40:03.813 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=sqlstore t=2024-09-11T08:40:03.740682831Z level=info msg="Created default organization" 2024-09-11T08:40:03.814 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=plugin.loader t=2024-09-11T08:40:03.798503828Z level=info msg="Plugin registered" pluginID=input 2024-09-11T08:40:03.982 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:04.141 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=plugin.loader t=2024-09-11T08:40:03.828042343Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-11T08:40:04.142 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=plugin.loader t=2024-09-11T08:40:03.828060527Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-11T08:40:04.142 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=secrets t=2024-09-11T08:40:03.828112715Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-11T08:40:04.142 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=query_data t=2024-09-11T08:40:03.82984709Z level=info msg="Query Service initialization" 2024-09-11T08:40:04.142 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=live.push_http t=2024-09-11T08:40:03.832352392Z level=info msg="Live Push Gateway initialization" 2024-09-11T08:40:04.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[29647]: 3.19 scrub starts 2024-09-11T08:40:04.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[29647]: 3.19 scrub ok 2024-09-11T08:40:04.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[29647]: 4.1f scrub starts 2024-09-11T08:40:04.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[29647]: 4.1f scrub ok 2024-09-11T08:40:04.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[29647]: from='client.24584 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:04.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[29647]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:04.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[29647]: Deploying daemon node-exporter.a on smithi092 2024-09-11T08:40:04.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[29647]: 5.f scrub starts 2024-09-11T08:40:04.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[29647]: 5.f scrub ok 2024-09-11T08:40:04.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[41114]: 3.19 scrub starts 2024-09-11T08:40:04.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[41114]: 3.19 scrub ok 2024-09-11T08:40:04.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[41114]: 4.1f scrub starts 2024-09-11T08:40:04.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[41114]: 4.1f scrub ok 2024-09-11T08:40:04.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[41114]: from='client.24584 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:04.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[41114]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:04.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[41114]: Deploying daemon node-exporter.a on smithi092 2024-09-11T08:40:04.319 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[41114]: 5.f scrub starts 2024-09-11T08:40:04.319 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:04 smithi092 ceph-mon[41114]: 5.f scrub ok 2024-09-11T08:40:04.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-mon[33384]: 3.19 scrub starts 2024-09-11T08:40:04.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-mon[33384]: 3.19 scrub ok 2024-09-11T08:40:04.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-mon[33384]: 4.1f scrub starts 2024-09-11T08:40:04.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-mon[33384]: 4.1f scrub ok 2024-09-11T08:40:04.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-mon[33384]: from='client.24584 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:04.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-mon[33384]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:04.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-mon[33384]: Deploying daemon node-exporter.a on smithi092 2024-09-11T08:40:04.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-mon[33384]: 5.f scrub starts 2024-09-11T08:40:04.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-mon[33384]: 5.f scrub ok 2024-09-11T08:40:04.912 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=infra.usagestats.collector t=2024-09-11T08:40:04.560721076Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-11T08:40:04.912 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=provisioning.datasources t=2024-09-11T08:40:04.5611034Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-11T08:40:04.912 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=provisioning.datasources t=2024-09-11T08:40:04.59246521Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-11T08:40:04.912 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=provisioning.datasources t=2024-09-11T08:40:04.624441315Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-11T08:40:04.912 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=provisioning.alerting t=2024-09-11T08:40:04.645758285Z level=info msg="starting to provision alerting" 2024-09-11T08:40:04.912 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=provisioning.alerting t=2024-09-11T08:40:04.645782444Z level=info msg="finished to provision alerting" 2024-09-11T08:40:04.912 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=ngalert.state.manager t=2024-09-11T08:40:04.645897007Z level=info msg="Warming state cache for startup" 2024-09-11T08:40:04.912 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=grafanaStorageLogger t=2024-09-11T08:40:04.645952315Z level=info msg="storage starting" 2024-09-11T08:40:04.912 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=http.server t=2024-09-11T08:40:04.647794265Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-11T08:40:04.912 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=ngalert.state.manager t=2024-09-11T08:40:04.656059225Z level=info msg="State cache has been initialized" states=0 duration=10.161238ms 2024-09-11T08:40:04.913 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=ngalert.multiorg.alertmanager t=2024-09-11T08:40:04.656094279Z level=info msg="starting MultiOrg Alertmanager" 2024-09-11T08:40:04.913 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=ticker t=2024-09-11T08:40:04.656107434Z level=info msg=starting first_tick=2024-09-11T08:40:10Z 2024-09-11T08:40:04.913 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=sqlstore.transactions t=2024-09-11T08:40:04.657002232Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:40:04.913 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:40:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=sqlstore.transactions t=2024-09-11T08:40:04.696895413Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:40:05.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:05 smithi145 ceph-mon[33384]: 3.f scrub starts 2024-09-11T08:40:05.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:05 smithi145 ceph-mon[33384]: 3.f scrub ok 2024-09-11T08:40:05.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:05 smithi145 ceph-mon[33384]: 3.4 scrub starts 2024-09-11T08:40:05.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:05 smithi145 ceph-mon[33384]: 3.4 scrub ok 2024-09-11T08:40:05.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:05 smithi145 ceph-mon[33384]: 5.1b scrub starts 2024-09-11T08:40:05.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:05 smithi145 ceph-mon[33384]: 5.1b scrub ok 2024-09-11T08:40:05.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:05 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:05.461 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:05 smithi092 ceph-mon[41114]: 3.f scrub starts 2024-09-11T08:40:05.461 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:05 smithi092 ceph-mon[41114]: 3.f scrub ok 2024-09-11T08:40:05.461 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:05 smithi092 ceph-mon[41114]: 3.4 scrub starts 2024-09-11T08:40:05.461 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:05 smithi092 ceph-mon[41114]: 3.4 scrub ok 2024-09-11T08:40:05.461 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:05 smithi092 ceph-mon[41114]: 5.1b scrub starts 2024-09-11T08:40:05.461 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:05 smithi092 ceph-mon[41114]: 5.1b scrub ok 2024-09-11T08:40:05.461 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:05 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:05.462 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:05 smithi092 ceph-mon[29647]: 3.f scrub starts 2024-09-11T08:40:05.462 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:05 smithi092 ceph-mon[29647]: 3.f scrub ok 2024-09-11T08:40:05.462 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:05 smithi092 ceph-mon[29647]: 3.4 scrub starts 2024-09-11T08:40:05.462 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:05 smithi092 ceph-mon[29647]: 3.4 scrub ok 2024-09-11T08:40:05.462 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:05 smithi092 ceph-mon[29647]: 5.1b scrub starts 2024-09-11T08:40:05.462 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:05 smithi092 ceph-mon[29647]: 5.1b scrub ok 2024-09-11T08:40:05.462 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:05 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:05.901 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:40:05.902 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:40:06.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:06 smithi145 ceph-mon[33384]: 6.3 scrub starts 2024-09-11T08:40:06.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:06 smithi145 ceph-mon[33384]: 6.3 scrub ok 2024-09-11T08:40:06.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:06 smithi145 ceph-mon[33384]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:06.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:06 smithi145 ceph-mon[33384]: 4.0 scrub starts 2024-09-11T08:40:06.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:06 smithi145 ceph-mon[33384]: 4.0 scrub ok 2024-09-11T08:40:06.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:06 smithi145 ceph-mon[33384]: 3.16 scrub starts 2024-09-11T08:40:06.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:06 smithi145 ceph-mon[33384]: 3.16 scrub ok 2024-09-11T08:40:06.413 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:40:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:40:06.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:06 smithi092 ceph-mon[29647]: 6.3 scrub starts 2024-09-11T08:40:06.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:06 smithi092 ceph-mon[29647]: 6.3 scrub ok 2024-09-11T08:40:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:06 smithi092 ceph-mon[29647]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:06 smithi092 ceph-mon[29647]: 4.0 scrub starts 2024-09-11T08:40:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:06 smithi092 ceph-mon[29647]: 4.0 scrub ok 2024-09-11T08:40:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:06 smithi092 ceph-mon[29647]: 3.16 scrub starts 2024-09-11T08:40:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:06 smithi092 ceph-mon[29647]: 3.16 scrub ok 2024-09-11T08:40:06.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:06 smithi092 ceph-mon[41114]: 6.3 scrub starts 2024-09-11T08:40:06.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:06 smithi092 ceph-mon[41114]: 6.3 scrub ok 2024-09-11T08:40:06.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:06 smithi092 ceph-mon[41114]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:06.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:06 smithi092 ceph-mon[41114]: 4.0 scrub starts 2024-09-11T08:40:06.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:06 smithi092 ceph-mon[41114]: 4.0 scrub ok 2024-09-11T08:40:06.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:06 smithi092 ceph-mon[41114]: 3.16 scrub starts 2024-09-11T08:40:06.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:06 smithi092 ceph-mon[41114]: 3.16 scrub ok 2024-09-11T08:40:07.065 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:06 smithi092 bash[76815]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T08:40:07.334 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 bash[76815]: Writing manifest to image destination 2024-09-11T08:40:07.334 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-mon[29647]: from='client.24590 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:07.334 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:07.334 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-mon[29647]: 6.1b scrub starts 2024-09-11T08:40:07.334 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-mon[29647]: 6.1b scrub ok 2024-09-11T08:40:07.334 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-mon[29647]: 6.12 scrub starts 2024-09-11T08:40:07.334 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-mon[29647]: 6.12 scrub ok 2024-09-11T08:40:07.335 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-mon[41114]: from='client.24590 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:07.335 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:07.335 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-mon[41114]: 6.1b scrub starts 2024-09-11T08:40:07.335 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-mon[41114]: 6.1b scrub ok 2024-09-11T08:40:07.335 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-mon[41114]: 6.12 scrub starts 2024-09-11T08:40:07.335 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-mon[41114]: 6.12 scrub ok 2024-09-11T08:40:07.357 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-09-11T08:40:05.011173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":928,"num_read_kb":785,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":580,"ondisk_log_size":580,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2387408,"kb_used_data":8592,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747573808,"statfs":{"total":767960285184,"available":765515579392,"internally_reserved":0,"allocated":8798208,"data_stored":4918791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002613"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:47.339397+0000","last_change":"2024-09-11T08:39:47.339366+0000","last_active":"2024-09-11T08:39:47.339397+0000","last_peered":"2024-09-11T08:39:47.339397+0000","last_clean":"2024-09-11T08:39:47.339397+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:39:47.339397+0000","last_undegraded":"2024-09-11T08:39:47.339397+0000","last_fullsized":"2024-09-11T08:39:47.339397+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.339312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.339312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.154202+0000","scrub_duration":0.001323095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:56.666807+0000","last_change":"2024-09-11T08:39:56.666764+0000","last_active":"2024-09-11T08:39:56.666807+0000","last_peered":"2024-09-11T08:39:56.666807+0000","last_clean":"2024-09-11T08:39:56.666807+0000","last_became_active":"2024-09-11T08:38:40.973398+0000","last_became_peered":"2024-09-11T08:38:40.973398+0000","last_unstale":"2024-09-11T08:39:56.666807+0000","last_undegraded":"2024-09-11T08:39:56.666807+0000","last_fullsized":"2024-09-11T08:39:56.666807+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.666703+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.544222+0000","scrub_duration":0.0041022000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499643+0000","last_change":"2024-09-11T08:38:42.980578+0000","last_active":"2024-09-11T08:39:35.499643+0000","last_peered":"2024-09-11T08:39:35.499643+0000","last_clean":"2024-09-11T08:39:35.499643+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:39:35.499643+0000","last_undegraded":"2024-09-11T08:39:35.499643+0000","last_fullsized":"2024-09-11T08:39:35.499643+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.101278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175049+0000","last_change":"2024-09-11T08:38:44.992111+0000","last_active":"2024-09-11T08:39:36.175049+0000","last_peered":"2024-09-11T08:39:36.175049+0000","last_clean":"2024-09-11T08:39:36.175049+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:39:36.175049+0000","last_undegraded":"2024-09-11T08:39:36.175049+0000","last_fullsized":"2024-09-11T08:39:36.175049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:12.528264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445456+0000","last_change":"2024-09-11T08:38:45.184844+0000","last_active":"2024-09-11T08:39:36.445456+0000","last_peered":"2024-09-11T08:39:36.445456+0000","last_clean":"2024-09-11T08:39:36.445456+0000","last_became_active":"2024-09-11T08:38:45.184215+0000","last_became_peered":"2024-09-11T08:38:45.184215+0000","last_unstale":"2024-09-11T08:39:36.445456+0000","last_undegraded":"2024-09-11T08:39:36.445456+0000","last_fullsized":"2024-09-11T08:39:36.445456+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.791964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:58.161111+0000","last_change":"2024-09-11T08:39:58.161055+0000","last_active":"2024-09-11T08:39:58.161111+0000","last_peered":"2024-09-11T08:39:58.161111+0000","last_clean":"2024-09-11T08:39:58.161111+0000","last_became_active":"2024-09-11T08:38:38.949136+0000","last_became_peered":"2024-09-11T08:38:38.949136+0000","last_unstale":"2024-09-11T08:39:58.161111+0000","last_undegraded":"2024-09-11T08:39:58.161111+0000","last_fullsized":"2024-09-11T08:39:58.161111+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:58.160984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.160984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:00.523696+0000","scrub_duration":0.0015109679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":70,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:45.997924+0000","last_change":"2024-09-11T08:39:45.997873+0000","last_active":"2024-09-11T08:39:45.997924+0000","last_peered":"2024-09-11T08:39:45.997924+0000","last_clean":"2024-09-11T08:39:45.997924+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:39:45.997924+0000","last_undegraded":"2024-09-11T08:39:45.997924+0000","last_fullsized":"2024-09-11T08:39:45.997924+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.997803+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.534414+0000","scrub_duration":0.0046870369999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.174897+0000","last_change":"2024-09-11T08:38:42.983653+0000","last_active":"2024-09-11T08:39:36.174897+0000","last_peered":"2024-09-11T08:39:36.174897+0000","last_clean":"2024-09-11T08:39:36.174897+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:39:36.174897+0000","last_undegraded":"2024-09-11T08:39:36.174897+0000","last_fullsized":"2024-09-11T08:39:36.174897+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:57.589237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501926+0000","last_change":"2024-09-11T08:38:38.949230+0000","last_active":"2024-09-11T08:39:35.501926+0000","last_peered":"2024-09-11T08:39:35.501926+0000","last_clean":"2024-09-11T08:39:35.501926+0000","last_became_active":"2024-09-11T08:38:38.948930+0000","last_became_peered":"2024-09-11T08:38:38.948930+0000","last_unstale":"2024-09-11T08:39:35.501926+0000","last_undegraded":"2024-09-11T08:39:35.501926+0000","last_fullsized":"2024-09-11T08:39:35.501926+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:57.678210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:46.620744+0000","last_change":"2024-09-11T08:39:46.620700+0000","last_active":"2024-09-11T08:39:46.620744+0000","last_peered":"2024-09-11T08:39:46.620744+0000","last_clean":"2024-09-11T08:39:46.620744+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:39:46.620744+0000","last_undegraded":"2024-09-11T08:39:46.620744+0000","last_fullsized":"2024-09-11T08:39:46.620744+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:39:46.620624+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:10.984564+0000","scrub_duration":0.0044132319999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501424+0000","last_change":"2024-09-11T08:38:42.981649+0000","last_active":"2024-09-11T08:39:35.501424+0000","last_peered":"2024-09-11T08:39:35.501424+0000","last_clean":"2024-09-11T08:39:35.501424+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:39:35.501424+0000","last_undegraded":"2024-09-11T08:39:35.501424+0000","last_fullsized":"2024-09-11T08:39:35.501424+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.721496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:51.280393+0000","last_change":"2024-09-11T08:39:51.280365+0000","last_active":"2024-09-11T08:39:51.280393+0000","last_peered":"2024-09-11T08:39:51.280393+0000","last_clean":"2024-09-11T08:39:51.280393+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:39:51.280393+0000","last_undegraded":"2024-09-11T08:39:51.280393+0000","last_fullsized":"2024-09-11T08:39:51.280393+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.280310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.280310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:06.712880+0000","scrub_duration":0.001634777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501649+0000","last_change":"2024-09-11T08:38:38.950588+0000","last_active":"2024-09-11T08:39:35.501649+0000","last_peered":"2024-09-11T08:39:35.501649+0000","last_clean":"2024-09-11T08:39:35.501649+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:39:35.501649+0000","last_undegraded":"2024-09-11T08:39:35.501649+0000","last_fullsized":"2024-09-11T08:39:35.501649+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:58.714558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175109+0000","last_change":"2024-09-11T08:38:40.974291+0000","last_active":"2024-09-11T08:39:36.175109+0000","last_peered":"2024-09-11T08:39:36.175109+0000","last_clean":"2024-09-11T08:39:36.175109+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:39:36.175109+0000","last_undegraded":"2024-09-11T08:39:36.175109+0000","last_fullsized":"2024-09-11T08:39:36.175109+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:57.136170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:51.166149+0000","last_change":"2024-09-11T08:39:51.166114+0000","last_active":"2024-09-11T08:39:51.166149+0000","last_peered":"2024-09-11T08:39:51.166149+0000","last_clean":"2024-09-11T08:39:51.166149+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:39:51.166149+0000","last_undegraded":"2024-09-11T08:39:51.166149+0000","last_fullsized":"2024-09-11T08:39:51.166149+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.166051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.570790+0000","scrub_duration":0.0016440420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501268+0000","last_change":"2024-09-11T08:38:44.994259+0000","last_active":"2024-09-11T08:39:35.501268+0000","last_peered":"2024-09-11T08:39:35.501268+0000","last_clean":"2024-09-11T08:39:35.501268+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:39:35.501268+0000","last_undegraded":"2024-09-11T08:39:35.501268+0000","last_fullsized":"2024-09-11T08:39:35.501268+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:03.579441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:59.606552+0000","last_change":"2024-09-11T08:39:59.606521+0000","last_active":"2024-09-11T08:39:59.606552+0000","last_peered":"2024-09-11T08:39:59.606552+0000","last_clean":"2024-09-11T08:39:59.606552+0000","last_became_active":"2024-09-11T08:38:45.183364+0000","last_became_peered":"2024-09-11T08:38:45.183364+0000","last_unstale":"2024-09-11T08:39:59.606552+0000","last_undegraded":"2024-09-11T08:39:59.606552+0000","last_fullsized":"2024-09-11T08:39:59.606552+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:59.606458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:59.606458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.547769+0000","scrub_duration":0.0012065960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500980+0000","last_change":"2024-09-11T08:38:38.954075+0000","last_active":"2024-09-11T08:39:35.500980+0000","last_peered":"2024-09-11T08:39:35.500980+0000","last_clean":"2024-09-11T08:39:35.500980+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:39:35.500980+0000","last_undegraded":"2024-09-11T08:39:35.500980+0000","last_fullsized":"2024-09-11T08:39:35.500980+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:04.776009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":71,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:50.472453+0000","last_change":"2024-09-11T08:39:50.472419+0000","last_active":"2024-09-11T08:39:50.472453+0000","last_peered":"2024-09-11T08:39:50.472453+0000","last_clean":"2024-09-11T08:39:50.472453+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:39:50.472453+0000","last_undegraded":"2024-09-11T08:39:50.472453+0000","last_fullsized":"2024-09-11T08:39:50.472453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.472368+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:10.784548+0000","scrub_duration":0.0049809520000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:44.632395+0000","last_change":"2024-09-11T08:39:44.632369+0000","last_active":"2024-09-11T08:39:44.632395+0000","last_peered":"2024-09-11T08:39:44.632395+0000","last_clean":"2024-09-11T08:39:44.632395+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:39:44.632395+0000","last_undegraded":"2024-09-11T08:39:44.632395+0000","last_fullsized":"2024-09-11T08:39:44.632395+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:44.632308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:44.632308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:56.408810+0000","scrub_duration":0.001420377,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445687+0000","last_change":"2024-09-11T08:38:45.183646+0000","last_active":"2024-09-11T08:39:36.445687+0000","last_peered":"2024-09-11T08:39:36.445687+0000","last_clean":"2024-09-11T08:39:36.445687+0000","last_became_active":"2024-09-11T08:38:45.182763+0000","last_became_peered":"2024-09-11T08:38:45.182763+0000","last_unstale":"2024-09-11T08:39:36.445687+0000","last_undegraded":"2024-09-11T08:39:36.445687+0000","last_fullsized":"2024-09-11T08:39:36.445687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:04.100474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175475+0000","last_change":"2024-09-11T08:38:38.949654+0000","last_active":"2024-09-11T08:39:36.175475+0000","last_peered":"2024-09-11T08:39:36.175475+0000","last_clean":"2024-09-11T08:39:36.175475+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:39:36.175475+0000","last_undegraded":"2024-09-11T08:39:36.175475+0000","last_fullsized":"2024-09-11T08:39:36.175475+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:54.740531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:55.125478+0000","last_change":"2024-09-11T08:39:55.125449+0000","last_active":"2024-09-11T08:39:55.125478+0000","last_peered":"2024-09-11T08:39:55.125478+0000","last_clean":"2024-09-11T08:39:55.125478+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:39:55.125478+0000","last_undegraded":"2024-09-11T08:39:55.125478+0000","last_fullsized":"2024-09-11T08:39:55.125478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:55.125394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.125394+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.857707+0000","scrub_duration":0.0016833810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175174+0000","last_change":"2024-09-11T08:38:42.987327+0000","last_active":"2024-09-11T08:39:36.175174+0000","last_peered":"2024-09-11T08:39:36.175174+0000","last_clean":"2024-09-11T08:39:36.175174+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:39:36.175174+0000","last_undegraded":"2024-09-11T08:39:36.175174+0000","last_fullsized":"2024-09-11T08:39:36.175174+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.157159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842563+0000","last_change":"2024-09-11T08:38:44.993281+0000","last_active":"2024-09-11T08:39:35.842563+0000","last_peered":"2024-09-11T08:39:35.842563+0000","last_clean":"2024-09-11T08:39:35.842563+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:39:35.842563+0000","last_undegraded":"2024-09-11T08:39:35.842563+0000","last_fullsized":"2024-09-11T08:39:35.842563+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:55.539078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499751+0000","last_change":"2024-09-11T08:38:38.954288+0000","last_active":"2024-09-11T08:39:35.499751+0000","last_peered":"2024-09-11T08:39:35.499751+0000","last_clean":"2024-09-11T08:39:35.499751+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:39:35.499751+0000","last_undegraded":"2024-09-11T08:39:35.499751+0000","last_fullsized":"2024-09-11T08:39:35.499751+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:56.900737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500506+0000","last_change":"2024-09-11T08:38:41.041065+0000","last_active":"2024-09-11T08:39:35.500506+0000","last_peered":"2024-09-11T08:39:35.500506+0000","last_clean":"2024-09-11T08:39:35.500506+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:39:35.500506+0000","last_undegraded":"2024-09-11T08:39:35.500506+0000","last_fullsized":"2024-09-11T08:39:35.500506+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:53.140994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":53,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.687946+0000","last_change":"2024-09-11T08:38:42.985883+0000","last_active":"2024-09-11T08:39:52.687946+0000","last_peered":"2024-09-11T08:39:52.687946+0000","last_clean":"2024-09-11T08:39:52.687946+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:39:52.687946+0000","last_undegraded":"2024-09-11T08:39:52.687946+0000","last_fullsized":"2024-09-11T08:39:52.687946+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:05.221206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500487+0000","last_change":"2024-09-11T08:38:40.979334+0000","last_active":"2024-09-11T08:39:35.500487+0000","last_peered":"2024-09-11T08:39:35.500487+0000","last_clean":"2024-09-11T08:39:35.500487+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:39:35.500487+0000","last_undegraded":"2024-09-11T08:39:35.500487+0000","last_fullsized":"2024-09-11T08:39:35.500487+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:05.848565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445242+0000","last_change":"2024-09-11T08:38:38.951009+0000","last_active":"2024-09-11T08:39:36.445242+0000","last_peered":"2024-09-11T08:39:36.445242+0000","last_clean":"2024-09-11T08:39:36.445242+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:39:36.445242+0000","last_undegraded":"2024-09-11T08:39:36.445242+0000","last_fullsized":"2024-09-11T08:39:36.445242+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:53.872027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.687918+0000","last_change":"2024-09-11T08:38:42.985885+0000","last_active":"2024-09-11T08:39:52.687918+0000","last_peered":"2024-09-11T08:39:52.687918+0000","last_clean":"2024-09-11T08:39:52.687918+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:39:52.687918+0000","last_undegraded":"2024-09-11T08:39:52.687918+0000","last_fullsized":"2024-09-11T08:39:52.687918+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501622+0000","last_change":"2024-09-11T08:38:44.991592+0000","last_active":"2024-09-11T08:39:35.501622+0000","last_peered":"2024-09-11T08:39:35.501622+0000","last_clean":"2024-09-11T08:39:35.501622+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:39:35.501622+0000","last_undegraded":"2024-09-11T08:39:35.501622+0000","last_fullsized":"2024-09-11T08:39:35.501622+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:06.716187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445123+0000","last_change":"2024-09-11T08:38:41.040597+0000","last_active":"2024-09-11T08:39:36.445123+0000","last_peered":"2024-09-11T08:39:36.445123+0000","last_clean":"2024-09-11T08:39:36.445123+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:39:36.445123+0000","last_undegraded":"2024-09-11T08:39:36.445123+0000","last_fullsized":"2024-09-11T08:39:36.445123+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:47.766795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.037635+0000","last_change":"2024-09-11T08:39:43.037605+0000","last_active":"2024-09-11T08:39:43.037635+0000","last_peered":"2024-09-11T08:39:43.037635+0000","last_clean":"2024-09-11T08:39:43.037635+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:39:43.037635+0000","last_undegraded":"2024-09-11T08:39:43.037635+0000","last_fullsized":"2024-09-11T08:39:43.037635+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.037551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.037551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.071661+0000","scrub_duration":0.001523976,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.174914+0000","last_change":"2024-09-11T08:38:42.983881+0000","last_active":"2024-09-11T08:39:36.174914+0000","last_peered":"2024-09-11T08:39:36.174914+0000","last_clean":"2024-09-11T08:39:36.174914+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:39:36.174914+0000","last_undegraded":"2024-09-11T08:39:36.174914+0000","last_fullsized":"2024-09-11T08:39:36.174914+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:49.733060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175286+0000","last_change":"2024-09-11T08:38:44.995555+0000","last_active":"2024-09-11T08:39:36.175286+0000","last_peered":"2024-09-11T08:39:36.175286+0000","last_clean":"2024-09-11T08:39:36.175286+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:39:36.175286+0000","last_undegraded":"2024-09-11T08:39:36.175286+0000","last_fullsized":"2024-09-11T08:39:36.175286+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.851425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175199+0000","last_change":"2024-09-11T08:38:40.968377+0000","last_active":"2024-09-11T08:39:36.175199+0000","last_peered":"2024-09-11T08:39:36.175199+0000","last_clean":"2024-09-11T08:39:36.175199+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:39:36.175199+0000","last_undegraded":"2024-09-11T08:39:36.175199+0000","last_fullsized":"2024-09-11T08:39:36.175199+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:09.166194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:41.267346+0000","last_change":"2024-09-11T08:39:41.267320+0000","last_active":"2024-09-11T08:39:41.267346+0000","last_peered":"2024-09-11T08:39:41.267346+0000","last_clean":"2024-09-11T08:39:41.267346+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:39:41.267346+0000","last_undegraded":"2024-09-11T08:39:41.267346+0000","last_fullsized":"2024-09-11T08:39:41.267346+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:41.267267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.267267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:01.547087+0000","scrub_duration":0.0013348209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":53,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.687796+0000","last_change":"2024-09-11T08:38:42.982794+0000","last_active":"2024-09-11T08:39:52.687796+0000","last_peered":"2024-09-11T08:39:52.687796+0000","last_clean":"2024-09-11T08:39:52.687796+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:39:52.687796+0000","last_undegraded":"2024-09-11T08:39:52.687796+0000","last_fullsized":"2024-09-11T08:39:52.687796+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:06.036142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842473+0000","last_change":"2024-09-11T08:38:45.183016+0000","last_active":"2024-09-11T08:39:35.842473+0000","last_peered":"2024-09-11T08:39:35.842473+0000","last_clean":"2024-09-11T08:39:35.842473+0000","last_became_active":"2024-09-11T08:38:45.182724+0000","last_became_peered":"2024-09-11T08:38:45.182724+0000","last_unstale":"2024-09-11T08:39:35.842473+0000","last_undegraded":"2024-09-11T08:39:35.842473+0000","last_fullsized":"2024-09-11T08:39:35.842473+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:51.221296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"63'10","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499890+0000","last_change":"2024-09-11T08:38:40.977074+0000","last_active":"2024-09-11T08:39:35.499890+0000","last_peered":"2024-09-11T08:39:35.499890+0000","last_clean":"2024-09-11T08:39:35.499890+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:39:35.499890+0000","last_undegraded":"2024-09-11T08:39:35.499890+0000","last_fullsized":"2024-09-11T08:39:35.499890+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.468488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.502041+0000","last_change":"2024-09-11T08:38:38.951168+0000","last_active":"2024-09-11T08:39:35.502041+0000","last_peered":"2024-09-11T08:39:35.502041+0000","last_clean":"2024-09-11T08:39:35.502041+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:39:35.502041+0000","last_undegraded":"2024-09-11T08:39:35.502041+0000","last_fullsized":"2024-09-11T08:39:35.502041+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.717085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.687787+0000","last_change":"2024-09-11T08:38:42.981732+0000","last_active":"2024-09-11T08:39:52.687787+0000","last_peered":"2024-09-11T08:39:52.687787+0000","last_clean":"2024-09-11T08:39:52.687787+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:39:52.687787+0000","last_undegraded":"2024-09-11T08:39:52.687787+0000","last_fullsized":"2024-09-11T08:39:52.687787+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:08.306042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500551+0000","last_change":"2024-09-11T08:38:44.998257+0000","last_active":"2024-09-11T08:39:35.500551+0000","last_peered":"2024-09-11T08:39:35.500551+0000","last_clean":"2024-09-11T08:39:35.500551+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:39:35.500551+0000","last_undegraded":"2024-09-11T08:39:35.500551+0000","last_fullsized":"2024-09-11T08:39:35.500551+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:12.569737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":65,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500898+0000","last_change":"2024-09-11T08:38:41.040034+0000","last_active":"2024-09-11T08:39:35.500898+0000","last_peered":"2024-09-11T08:39:35.500898+0000","last_clean":"2024-09-11T08:39:35.500898+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:39:35.500898+0000","last_undegraded":"2024-09-11T08:39:35.500898+0000","last_fullsized":"2024-09-11T08:39:35.500898+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:07.747665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500207+0000","last_change":"2024-09-11T08:38:38.953053+0000","last_active":"2024-09-11T08:39:35.500207+0000","last_peered":"2024-09-11T08:39:35.500207+0000","last_clean":"2024-09-11T08:39:35.500207+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:39:35.500207+0000","last_undegraded":"2024-09-11T08:39:35.500207+0000","last_fullsized":"2024-09-11T08:39:35.500207+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:57.480705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:57.597021+0000","last_change":"2024-09-11T08:39:57.596980+0000","last_active":"2024-09-11T08:39:57.597021+0000","last_peered":"2024-09-11T08:39:57.597021+0000","last_clean":"2024-09-11T08:39:57.597021+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:39:57.597021+0000","last_undegraded":"2024-09-11T08:39:57.597021+0000","last_fullsized":"2024-09-11T08:39:57.597021+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.596889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.648355+0000","scrub_duration":0.001574424,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499567+0000","last_change":"2024-09-11T08:38:45.183159+0000","last_active":"2024-09-11T08:39:35.499567+0000","last_peered":"2024-09-11T08:39:35.499567+0000","last_clean":"2024-09-11T08:39:35.499567+0000","last_became_active":"2024-09-11T08:38:45.182782+0000","last_became_peered":"2024-09-11T08:38:45.182782+0000","last_unstale":"2024-09-11T08:39:35.499567+0000","last_undegraded":"2024-09-11T08:39:35.499567+0000","last_fullsized":"2024-09-11T08:39:35.499567+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:59.864364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":113,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:53.493055+0000","last_change":"2024-09-11T08:39:53.493004+0000","last_active":"2024-09-11T08:39:53.493055+0000","last_peered":"2024-09-11T08:39:53.493055+0000","last_clean":"2024-09-11T08:39:53.493055+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:39:53.493055+0000","last_undegraded":"2024-09-11T08:39:53.493055+0000","last_fullsized":"2024-09-11T08:39:53.493055+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.492945+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.774847+0000","scrub_duration":0.005988045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175554+0000","last_change":"2024-09-11T08:38:38.955616+0000","last_active":"2024-09-11T08:39:36.175554+0000","last_peered":"2024-09-11T08:39:36.175554+0000","last_clean":"2024-09-11T08:39:36.175554+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:39:36.175554+0000","last_undegraded":"2024-09-11T08:39:36.175554+0000","last_fullsized":"2024-09-11T08:39:36.175554+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:50.638474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:58.472902+0000","last_change":"2024-09-11T08:39:58.472851+0000","last_active":"2024-09-11T08:39:58.472902+0000","last_peered":"2024-09-11T08:39:58.472902+0000","last_clean":"2024-09-11T08:39:58.472902+0000","last_became_active":"2024-09-11T08:38:38.947270+0000","last_became_peered":"2024-09-11T08:38:38.947270+0000","last_unstale":"2024-09-11T08:39:58.472902+0000","last_undegraded":"2024-09-11T08:39:58.472902+0000","last_fullsized":"2024-09-11T08:39:58.472902+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:39:58.472785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.472785+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.725781+0000","scrub_duration":0.0020806930000000002,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501165+0000","last_change":"2024-09-11T08:38:42.975757+0000","last_active":"2024-09-11T08:39:35.501165+0000","last_peered":"2024-09-11T08:39:35.501165+0000","last_clean":"2024-09-11T08:39:35.501165+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:39:35.501165+0000","last_undegraded":"2024-09-11T08:39:35.501165+0000","last_fullsized":"2024-09-11T08:39:35.501165+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:50.821530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445649+0000","last_change":"2024-09-11T08:38:44.997146+0000","last_active":"2024-09-11T08:39:36.445649+0000","last_peered":"2024-09-11T08:39:36.445649+0000","last_clean":"2024-09-11T08:39:36.445649+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:39:36.445649+0000","last_undegraded":"2024-09-11T08:39:36.445649+0000","last_fullsized":"2024-09-11T08:39:36.445649+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.168511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":66,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.585203+0000","last_change":"2024-09-11T08:39:43.585175+0000","last_active":"2024-09-11T08:39:43.585203+0000","last_peered":"2024-09-11T08:39:43.585203+0000","last_clean":"2024-09-11T08:39:43.585203+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:39:43.585203+0000","last_undegraded":"2024-09-11T08:39:43.585203+0000","last_fullsized":"2024-09-11T08:39:43.585203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:39:43.585116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.585116+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.724395+0000","scrub_duration":0.0021210309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:00.500472+0000","last_change":"2024-09-11T08:40:00.500444+0000","last_active":"2024-09-11T08:40:00.500472+0000","last_peered":"2024-09-11T08:40:00.500472+0000","last_clean":"2024-09-11T08:40:00.500472+0000","last_became_active":"2024-09-11T08:38:38.949019+0000","last_became_peered":"2024-09-11T08:38:38.949019+0000","last_unstale":"2024-09-11T08:40:00.500472+0000","last_undegraded":"2024-09-11T08:40:00.500472+0000","last_fullsized":"2024-09-11T08:40:00.500472+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.500392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.500392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.354187+0000","scrub_duration":0.001294566,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.444967+0000","last_change":"2024-09-11T08:38:42.971425+0000","last_active":"2024-09-11T08:39:36.444967+0000","last_peered":"2024-09-11T08:39:36.444967+0000","last_clean":"2024-09-11T08:39:36.444967+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:39:36.444967+0000","last_undegraded":"2024-09-11T08:39:36.444967+0000","last_fullsized":"2024-09-11T08:39:36.444967+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:59.348177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445494+0000","last_change":"2024-09-11T08:38:42.980780+0000","last_active":"2024-09-11T08:39:36.445494+0000","last_peered":"2024-09-11T08:39:36.445494+0000","last_clean":"2024-09-11T08:39:36.445494+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:39:36.445494+0000","last_undegraded":"2024-09-11T08:39:36.445494+0000","last_fullsized":"2024-09-11T08:39:36.445494+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.591149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842581+0000","last_change":"2024-09-11T08:38:45.183842+0000","last_active":"2024-09-11T08:39:35.842581+0000","last_peered":"2024-09-11T08:39:35.842581+0000","last_clean":"2024-09-11T08:39:35.842581+0000","last_became_active":"2024-09-11T08:38:45.183582+0000","last_became_peered":"2024-09-11T08:38:45.183582+0000","last_unstale":"2024-09-11T08:39:35.842581+0000","last_undegraded":"2024-09-11T08:39:35.842581+0000","last_fullsized":"2024-09-11T08:39:35.842581+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:03.954910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500582+0000","last_change":"2024-09-11T08:38:40.975966+0000","last_active":"2024-09-11T08:39:35.500582+0000","last_peered":"2024-09-11T08:39:35.500582+0000","last_clean":"2024-09-11T08:39:35.500582+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:39:35.500582+0000","last_undegraded":"2024-09-11T08:39:35.500582+0000","last_fullsized":"2024-09-11T08:39:35.500582+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.626733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.356781+0000","last_change":"2024-09-11T08:39:43.356751+0000","last_active":"2024-09-11T08:39:43.356781+0000","last_peered":"2024-09-11T08:39:43.356781+0000","last_clean":"2024-09-11T08:39:43.356781+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:39:43.356781+0000","last_undegraded":"2024-09-11T08:39:43.356781+0000","last_fullsized":"2024-09-11T08:39:43.356781+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.356699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.356699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.569637+0000","scrub_duration":0.001649455,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":123,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:59.236810+0000","last_change":"2024-09-11T08:39:55.153362+0000","last_active":"2024-09-11T08:39:59.236810+0000","last_peered":"2024-09-11T08:39:59.236810+0000","last_clean":"2024-09-11T08:39:59.236810+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:39:59.236810+0000","last_undegraded":"2024-09-11T08:39:59.236810+0000","last_fullsized":"2024-09-11T08:39:59.236810+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:39:55.153290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.153290+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.446909+0000","scrub_duration":0.0017161990000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.502124+0000","last_change":"2024-09-11T08:38:42.976525+0000","last_active":"2024-09-11T08:39:35.502124+0000","last_peered":"2024-09-11T08:39:35.502124+0000","last_clean":"2024-09-11T08:39:35.502124+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:39:35.502124+0000","last_undegraded":"2024-09-11T08:39:35.502124+0000","last_fullsized":"2024-09-11T08:39:35.502124+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:09.438148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500105+0000","last_change":"2024-09-11T08:38:44.994290+0000","last_active":"2024-09-11T08:39:35.500105+0000","last_peered":"2024-09-11T08:39:35.500105+0000","last_clean":"2024-09-11T08:39:35.500105+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:39:35.500105+0000","last_undegraded":"2024-09-11T08:39:35.500105+0000","last_fullsized":"2024-09-11T08:39:35.500105+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.617197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":60,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:00.540221+0000","last_change":"2024-09-11T08:40:00.540192+0000","last_active":"2024-09-11T08:40:00.540221+0000","last_peered":"2024-09-11T08:40:00.540221+0000","last_clean":"2024-09-11T08:40:00.540221+0000","last_became_active":"2024-09-11T08:38:41.040948+0000","last_became_peered":"2024-09-11T08:38:41.040948+0000","last_unstale":"2024-09-11T08:40:00.540221+0000","last_undegraded":"2024-09-11T08:40:00.540221+0000","last_fullsized":"2024-09-11T08:40:00.540221+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:40:00.540136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.540136+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:07.651325+0000","scrub_duration":0.001785095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445564+0000","last_change":"2024-09-11T08:38:38.948242+0000","last_active":"2024-09-11T08:39:36.445564+0000","last_peered":"2024-09-11T08:39:36.445564+0000","last_clean":"2024-09-11T08:39:36.445564+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:39:36.445564+0000","last_undegraded":"2024-09-11T08:39:36.445564+0000","last_fullsized":"2024-09-11T08:39:36.445564+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:52.890961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":74,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842755+0000","last_change":"2024-09-11T08:39:12.765244+0000","last_active":"2024-09-11T08:39:35.842755+0000","last_peered":"2024-09-11T08:39:35.842755+0000","last_clean":"2024-09-11T08:39:35.842755+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:39:35.842755+0000","last_undegraded":"2024-09-11T08:39:35.842755+0000","last_fullsized":"2024-09-11T08:39:35.842755+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'72","last_scrub_stamp":"2024-09-11T08:39:12.765190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:39:12.765190+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:16.661831+0000","scrub_duration":0.0014709829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:49.116242+0000","last_change":"2024-09-11T08:39:49.116184+0000","last_active":"2024-09-11T08:39:49.116242+0000","last_peered":"2024-09-11T08:39:49.116242+0000","last_clean":"2024-09-11T08:39:49.116242+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:39:49.116242+0000","last_undegraded":"2024-09-11T08:39:49.116242+0000","last_fullsized":"2024-09-11T08:39:49.116242+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.116113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.116113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.209747+0000","scrub_duration":0.00167223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:47.423780+0000","last_change":"2024-09-11T08:39:47.423745+0000","last_active":"2024-09-11T08:39:47.423780+0000","last_peered":"2024-09-11T08:39:47.423780+0000","last_clean":"2024-09-11T08:39:47.423780+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:39:47.423780+0000","last_undegraded":"2024-09-11T08:39:47.423780+0000","last_fullsized":"2024-09-11T08:39:47.423780+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.423684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.423684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.004042+0000","scrub_duration":0.001458972,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175659+0000","last_change":"2024-09-11T08:38:40.973744+0000","last_active":"2024-09-11T08:39:36.175659+0000","last_peered":"2024-09-11T08:39:36.175659+0000","last_clean":"2024-09-11T08:39:36.175659+0000","last_became_active":"2024-09-11T08:38:40.973373+0000","last_became_peered":"2024-09-11T08:38:40.973373+0000","last_unstale":"2024-09-11T08:39:36.175659+0000","last_undegraded":"2024-09-11T08:39:36.175659+0000","last_fullsized":"2024-09-11T08:39:36.175659+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:07.389753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175244+0000","last_change":"2024-09-11T08:38:38.948244+0000","last_active":"2024-09-11T08:39:36.175244+0000","last_peered":"2024-09-11T08:39:36.175244+0000","last_clean":"2024-09-11T08:39:36.175244+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:39:36.175244+0000","last_undegraded":"2024-09-11T08:39:36.175244+0000","last_fullsized":"2024-09-11T08:39:36.175244+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:03.299757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501998+0000","last_change":"2024-09-11T08:38:42.982947+0000","last_active":"2024-09-11T08:39:35.501998+0000","last_peered":"2024-09-11T08:39:35.501998+0000","last_clean":"2024-09-11T08:39:35.501998+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:39:35.501998+0000","last_undegraded":"2024-09-11T08:39:35.501998+0000","last_fullsized":"2024-09-11T08:39:35.501998+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.506223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445494+0000","last_change":"2024-09-11T08:38:45.185064+0000","last_active":"2024-09-11T08:39:36.445494+0000","last_peered":"2024-09-11T08:39:36.445494+0000","last_clean":"2024-09-11T08:39:36.445494+0000","last_became_active":"2024-09-11T08:38:45.184410+0000","last_became_peered":"2024-09-11T08:38:45.184410+0000","last_unstale":"2024-09-11T08:39:36.445494+0000","last_undegraded":"2024-09-11T08:39:36.445494+0000","last_fullsized":"2024-09-11T08:39:36.445494+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:58.964836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":67,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.526881+0000","last_change":"2024-09-11T08:39:43.526845+0000","last_active":"2024-09-11T08:39:43.526881+0000","last_peered":"2024-09-11T08:39:43.526881+0000","last_clean":"2024-09-11T08:39:43.526881+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:39:43.526881+0000","last_undegraded":"2024-09-11T08:39:43.526881+0000","last_fullsized":"2024-09-11T08:39:43.526881+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:39:43.526763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.526763+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.869811+0000","scrub_duration":0.0022037329999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:55.599092+0000","last_change":"2024-09-11T08:39:55.599049+0000","last_active":"2024-09-11T08:39:55.599092+0000","last_peered":"2024-09-11T08:39:55.599092+0000","last_clean":"2024-09-11T08:39:55.599092+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:39:55.599092+0000","last_undegraded":"2024-09-11T08:39:55.599092+0000","last_fullsized":"2024-09-11T08:39:55.599092+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.598961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.598961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.562190+0000","scrub_duration":0.0016789820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:46.627109+0000","last_change":"2024-09-11T08:39:46.627077+0000","last_active":"2024-09-11T08:39:46.627109+0000","last_peered":"2024-09-11T08:39:46.627109+0000","last_clean":"2024-09-11T08:39:46.627109+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:39:46.627109+0000","last_undegraded":"2024-09-11T08:39:46.627109+0000","last_fullsized":"2024-09-11T08:39:46.627109+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:46.627028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:46.627028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:56.334041+0000","scrub_duration":0.0013824029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445462+0000","last_change":"2024-09-11T08:38:44.994162+0000","last_active":"2024-09-11T08:39:36.445462+0000","last_peered":"2024-09-11T08:39:36.445462+0000","last_clean":"2024-09-11T08:39:36.445462+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:39:36.445462+0000","last_undegraded":"2024-09-11T08:39:36.445462+0000","last_fullsized":"2024-09-11T08:39:36.445462+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:09.343122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175377+0000","last_change":"2024-09-11T08:38:40.969616+0000","last_active":"2024-09-11T08:39:36.175377+0000","last_peered":"2024-09-11T08:39:36.175377+0000","last_clean":"2024-09-11T08:39:36.175377+0000","last_became_active":"2024-09-11T08:38:40.969348+0000","last_became_peered":"2024-09-11T08:38:40.969348+0000","last_unstale":"2024-09-11T08:39:36.175377+0000","last_undegraded":"2024-09-11T08:39:36.175377+0000","last_fullsized":"2024-09-11T08:39:36.175377+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:52.841663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445860+0000","last_change":"2024-09-11T08:38:38.957214+0000","last_active":"2024-09-11T08:39:36.445860+0000","last_peered":"2024-09-11T08:39:36.445860+0000","last_clean":"2024-09-11T08:39:36.445860+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:39:36.445860+0000","last_undegraded":"2024-09-11T08:39:36.445860+0000","last_fullsized":"2024-09-11T08:39:36.445860+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:04.778506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:00.473638+0000","last_change":"2024-09-11T08:40:00.473606+0000","last_active":"2024-09-11T08:40:00.473638+0000","last_peered":"2024-09-11T08:40:00.473638+0000","last_clean":"2024-09-11T08:40:00.473638+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:40:00.473638+0000","last_undegraded":"2024-09-11T08:40:00.473638+0000","last_fullsized":"2024-09-11T08:40:00.473638+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:00.473549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.473549+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.300137+0000","scrub_duration":0.0016703060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175492+0000","last_change":"2024-09-11T08:38:44.993736+0000","last_active":"2024-09-11T08:39:36.175492+0000","last_peered":"2024-09-11T08:39:36.175492+0000","last_clean":"2024-09-11T08:39:36.175492+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:39:36.175492+0000","last_undegraded":"2024-09-11T08:39:36.175492+0000","last_fullsized":"2024-09-11T08:39:36.175492+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:55.062690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500820+0000","last_change":"2024-09-11T08:38:40.977113+0000","last_active":"2024-09-11T08:39:35.500820+0000","last_peered":"2024-09-11T08:39:35.500820+0000","last_clean":"2024-09-11T08:39:35.500820+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:39:35.500820+0000","last_undegraded":"2024-09-11T08:39:35.500820+0000","last_fullsized":"2024-09-11T08:39:35.500820+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:05.072088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:50.452607+0000","last_change":"2024-09-11T08:39:50.452568+0000","last_active":"2024-09-11T08:39:50.452607+0000","last_peered":"2024-09-11T08:39:50.452607+0000","last_clean":"2024-09-11T08:39:50.452607+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:39:50.452607+0000","last_undegraded":"2024-09-11T08:39:50.452607+0000","last_fullsized":"2024-09-11T08:39:50.452607+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.452494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.631278+0000","scrub_duration":0.001547086,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.444929+0000","last_change":"2024-09-11T08:38:42.983093+0000","last_active":"2024-09-11T08:39:36.444929+0000","last_peered":"2024-09-11T08:39:36.444929+0000","last_clean":"2024-09-11T08:39:36.444929+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:39:36.444929+0000","last_undegraded":"2024-09-11T08:39:36.444929+0000","last_fullsized":"2024-09-11T08:39:36.444929+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.830046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500230+0000","last_change":"2024-09-11T08:38:44.991908+0000","last_active":"2024-09-11T08:39:35.500230+0000","last_peered":"2024-09-11T08:39:35.500230+0000","last_clean":"2024-09-11T08:39:35.500230+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:39:35.500230+0000","last_undegraded":"2024-09-11T08:39:35.500230+0000","last_fullsized":"2024-09-11T08:39:35.500230+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:12.056728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":69,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:45.575270+0000","last_change":"2024-09-11T08:39:45.575209+0000","last_active":"2024-09-11T08:39:45.575270+0000","last_peered":"2024-09-11T08:39:45.575270+0000","last_clean":"2024-09-11T08:39:45.575270+0000","last_became_active":"2024-09-11T08:38:41.040207+0000","last_became_peered":"2024-09-11T08:38:41.040207+0000","last_unstale":"2024-09-11T08:39:45.575270+0000","last_undegraded":"2024-09-11T08:39:45.575270+0000","last_fullsized":"2024-09-11T08:39:45.575270+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:39:45.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.575133+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.182090+0000","scrub_duration":0.0022792160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:56.112155+0000","last_change":"2024-09-11T08:39:56.112112+0000","last_active":"2024-09-11T08:39:56.112155+0000","last_peered":"2024-09-11T08:39:56.112155+0000","last_clean":"2024-09-11T08:39:56.112155+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:39:56.112155+0000","last_undegraded":"2024-09-11T08:39:56.112155+0000","last_fullsized":"2024-09-11T08:39:56.112155+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:56.112043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.112043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:25.809354+0000","scrub_duration":0.001517137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445023+0000","last_change":"2024-09-11T08:38:42.985952+0000","last_active":"2024-09-11T08:39:36.445023+0000","last_peered":"2024-09-11T08:39:36.445023+0000","last_clean":"2024-09-11T08:39:36.445023+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:39:36.445023+0000","last_undegraded":"2024-09-11T08:39:36.445023+0000","last_fullsized":"2024-09-11T08:39:36.445023+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:06.325377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842722+0000","last_change":"2024-09-11T08:38:44.993458+0000","last_active":"2024-09-11T08:39:35.842722+0000","last_peered":"2024-09-11T08:39:35.842722+0000","last_clean":"2024-09-11T08:39:35.842722+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:39:35.842722+0000","last_undegraded":"2024-09-11T08:39:35.842722+0000","last_fullsized":"2024-09-11T08:39:35.842722+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:04.342365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:40.524639+0000","last_change":"2024-09-11T08:39:40.524581+0000","last_active":"2024-09-11T08:39:40.524639+0000","last_peered":"2024-09-11T08:39:40.524639+0000","last_clean":"2024-09-11T08:39:40.524639+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:39:40.524639+0000","last_undegraded":"2024-09-11T08:39:40.524639+0000","last_fullsized":"2024-09-11T08:39:40.524639+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:40.524498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:40.524498+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.186922+0000","scrub_duration":0.0024035010000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:00.113613+0000","last_change":"2024-09-11T08:40:00.113587+0000","last_active":"2024-09-11T08:40:00.113613+0000","last_peered":"2024-09-11T08:40:00.113613+0000","last_clean":"2024-09-11T08:40:00.113613+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:40:00.113613+0000","last_undegraded":"2024-09-11T08:40:00.113613+0000","last_fullsized":"2024-09-11T08:40:00.113613+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.113534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.113534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.476975+0000","scrub_duration":0.0011790780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:57.454285+0000","last_change":"2024-09-11T08:39:57.454257+0000","last_active":"2024-09-11T08:39:57.454285+0000","last_peered":"2024-09-11T08:39:57.454285+0000","last_clean":"2024-09-11T08:39:57.454285+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:39:57.454285+0000","last_undegraded":"2024-09-11T08:39:57.454285+0000","last_fullsized":"2024-09-11T08:39:57.454285+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.454198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.454198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.731898+0000","scrub_duration":0.0011666669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445542+0000","last_change":"2024-09-11T08:38:44.998686+0000","last_active":"2024-09-11T08:39:36.445542+0000","last_peered":"2024-09-11T08:39:36.445542+0000","last_clean":"2024-09-11T08:39:36.445542+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:39:36.445542+0000","last_undegraded":"2024-09-11T08:39:36.445542+0000","last_fullsized":"2024-09-11T08:39:36.445542+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:02.320682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:48.462270+0000","last_change":"2024-09-11T08:39:48.462233+0000","last_active":"2024-09-11T08:39:48.462270+0000","last_peered":"2024-09-11T08:39:48.462270+0000","last_clean":"2024-09-11T08:39:48.462270+0000","last_became_active":"2024-09-11T08:38:41.040475+0000","last_became_peered":"2024-09-11T08:38:41.040475+0000","last_unstale":"2024-09-11T08:39:48.462270+0000","last_undegraded":"2024-09-11T08:39:48.462270+0000","last_fullsized":"2024-09-11T08:39:48.462270+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-11T08:39:48.462163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:48.462163+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.032207+0000","scrub_duration":0.002094753,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.843191+0000","last_change":"2024-09-11T08:38:38.956109+0000","last_active":"2024-09-11T08:39:35.843191+0000","last_peered":"2024-09-11T08:39:35.843191+0000","last_clean":"2024-09-11T08:39:35.843191+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:39:35.843191+0000","last_undegraded":"2024-09-11T08:39:35.843191+0000","last_fullsized":"2024-09-11T08:39:35.843191+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:50.337939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"63'8","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.688011+0000","last_change":"2024-09-11T08:38:42.987044+0000","last_active":"2024-09-11T08:39:52.688011+0000","last_peered":"2024-09-11T08:39:52.688011+0000","last_clean":"2024-09-11T08:39:52.688011+0000","last_became_active":"2024-09-11T08:38:42.986756+0000","last_became_peered":"2024-09-11T08:38:42.986756+0000","last_unstale":"2024-09-11T08:39:52.688011+0000","last_undegraded":"2024-09-11T08:39:52.688011+0000","last_fullsized":"2024-09-11T08:39:52.688011+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501304+0000","last_change":"2024-09-11T08:38:45.182987+0000","last_active":"2024-09-11T08:39:35.501304+0000","last_peered":"2024-09-11T08:39:35.501304+0000","last_clean":"2024-09-11T08:39:35.501304+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:39:35.501304+0000","last_undegraded":"2024-09-11T08:39:35.501304+0000","last_fullsized":"2024-09-11T08:39:35.501304+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:58.541554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:42.605410+0000","last_change":"2024-09-11T08:39:42.605373+0000","last_active":"2024-09-11T08:39:42.605410+0000","last_peered":"2024-09-11T08:39:42.605410+0000","last_clean":"2024-09-11T08:39:42.605410+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:39:42.605410+0000","last_undegraded":"2024-09-11T08:39:42.605410+0000","last_fullsized":"2024-09-11T08:39:42.605410+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:42.605301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:42.605301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:53.937210+0000","scrub_duration":0.0014798509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445861+0000","last_change":"2024-09-11T08:38:41.039820+0000","last_active":"2024-09-11T08:39:36.445861+0000","last_peered":"2024-09-11T08:39:36.445861+0000","last_clean":"2024-09-11T08:39:36.445861+0000","last_became_active":"2024-09-11T08:38:41.039552+0000","last_became_peered":"2024-09-11T08:38:41.039552+0000","last_unstale":"2024-09-11T08:39:36.445861+0000","last_undegraded":"2024-09-11T08:39:36.445861+0000","last_fullsized":"2024-09-11T08:39:36.445861+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:51.272516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:49.472804+0000","last_change":"2024-09-11T08:39:49.472764+0000","last_active":"2024-09-11T08:39:49.472804+0000","last_peered":"2024-09-11T08:39:49.472804+0000","last_clean":"2024-09-11T08:39:49.472804+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:39:49.472804+0000","last_undegraded":"2024-09-11T08:39:49.472804+0000","last_fullsized":"2024-09-11T08:39:49.472804+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.472696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.472696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:01.403308+0000","scrub_duration":0.001919104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842528+0000","last_change":"2024-09-11T08:38:45.185016+0000","last_active":"2024-09-11T08:39:35.842528+0000","last_peered":"2024-09-11T08:39:35.842528+0000","last_clean":"2024-09-11T08:39:35.842528+0000","last_became_active":"2024-09-11T08:38:45.184753+0000","last_became_peered":"2024-09-11T08:38:45.184753+0000","last_unstale":"2024-09-11T08:39:35.842528+0000","last_undegraded":"2024-09-11T08:39:35.842528+0000","last_fullsized":"2024-09-11T08:39:35.842528+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:13.726221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500656+0000","last_change":"2024-09-11T08:38:41.041476+0000","last_active":"2024-09-11T08:39:35.500656+0000","last_peered":"2024-09-11T08:39:35.500656+0000","last_clean":"2024-09-11T08:39:35.500656+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:39:35.500656+0000","last_undegraded":"2024-09-11T08:39:35.500656+0000","last_fullsized":"2024-09-11T08:39:35.500656+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:04.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:53.176120+0000","last_change":"2024-09-11T08:39:53.176095+0000","last_active":"2024-09-11T08:39:53.176120+0000","last_peered":"2024-09-11T08:39:53.176120+0000","last_clean":"2024-09-11T08:39:53.176120+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:39:53.176120+0000","last_undegraded":"2024-09-11T08:39:53.176120+0000","last_fullsized":"2024-09-11T08:39:53.176120+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.176035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.176035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:00.682857+0000","scrub_duration":0.001147531,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445708+0000","last_change":"2024-09-11T08:38:42.982083+0000","last_active":"2024-09-11T08:39:36.445708+0000","last_peered":"2024-09-11T08:39:36.445708+0000","last_clean":"2024-09-11T08:39:36.445708+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:39:36.445708+0000","last_undegraded":"2024-09-11T08:39:36.445708+0000","last_fullsized":"2024-09-11T08:39:36.445708+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:58.890541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.493090+0000","last_change":"2024-09-11T08:39:52.493064+0000","last_active":"2024-09-11T08:39:52.493090+0000","last_peered":"2024-09-11T08:39:52.493090+0000","last_clean":"2024-09-11T08:39:52.493090+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:39:52.493090+0000","last_undegraded":"2024-09-11T08:39:52.493090+0000","last_fullsized":"2024-09-11T08:39:52.493090+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.493013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:13.717323+0000","scrub_duration":0.0012699160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501853+0000","last_change":"2024-09-11T08:38:41.042115+0000","last_active":"2024-09-11T08:39:35.501853+0000","last_peered":"2024-09-11T08:39:35.501853+0000","last_clean":"2024-09-11T08:39:35.501853+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:39:35.501853+0000","last_undegraded":"2024-09-11T08:39:35.501853+0000","last_fullsized":"2024-09-11T08:39:35.501853+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.772561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500193+0000","last_change":"2024-09-11T08:38:38.958515+0000","last_active":"2024-09-11T08:39:35.500193+0000","last_peered":"2024-09-11T08:39:35.500193+0000","last_clean":"2024-09-11T08:39:35.500193+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:39:35.500193+0000","last_undegraded":"2024-09-11T08:39:35.500193+0000","last_fullsized":"2024-09-11T08:39:35.500193+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:53.785774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.687834+0000","last_change":"2024-09-11T08:38:42.981915+0000","last_active":"2024-09-11T08:39:52.687834+0000","last_peered":"2024-09-11T08:39:52.687834+0000","last_clean":"2024-09-11T08:39:52.687834+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:39:52.687834+0000","last_undegraded":"2024-09-11T08:39:52.687834+0000","last_fullsized":"2024-09-11T08:39:52.687834+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175675+0000","last_change":"2024-09-11T08:38:44.993997+0000","last_active":"2024-09-11T08:39:36.175675+0000","last_peered":"2024-09-11T08:39:36.175675+0000","last_clean":"2024-09-11T08:39:36.175675+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:39:36.175675+0000","last_undegraded":"2024-09-11T08:39:36.175675+0000","last_fullsized":"2024-09-11T08:39:36.175675+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:59.445794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":43,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445977+0000","last_change":"2024-09-11T08:38:41.040424+0000","last_active":"2024-09-11T08:39:36.445977+0000","last_peered":"2024-09-11T08:39:36.445977+0000","last_clean":"2024-09-11T08:39:36.445977+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:39:36.445977+0000","last_undegraded":"2024-09-11T08:39:36.445977+0000","last_fullsized":"2024-09-11T08:39:36.445977+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:53.450115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842811+0000","last_change":"2024-09-11T08:38:38.949874+0000","last_active":"2024-09-11T08:39:35.842811+0000","last_peered":"2024-09-11T08:39:35.842811+0000","last_clean":"2024-09-11T08:39:35.842811+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:39:35.842811+0000","last_undegraded":"2024-09-11T08:39:35.842811+0000","last_fullsized":"2024-09-11T08:39:35.842811+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:02.574659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.687975+0000","last_change":"2024-09-11T08:39:44.429546+0000","last_active":"2024-09-11T08:39:52.687975+0000","last_peered":"2024-09-11T08:39:52.687975+0000","last_clean":"2024-09-11T08:39:52.687975+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:39:52.687975+0000","last_undegraded":"2024-09-11T08:39:52.687975+0000","last_fullsized":"2024-09-11T08:39:52.687975+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:39:44.429477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:44.429477+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.023544+0000","scrub_duration":0.001890784,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.254325+0000","last_change":"2024-09-11T08:39:52.254299+0000","last_active":"2024-09-11T08:39:52.254325+0000","last_peered":"2024-09-11T08:39:52.254325+0000","last_clean":"2024-09-11T08:39:52.254325+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:39:52.254325+0000","last_undegraded":"2024-09-11T08:39:52.254325+0000","last_fullsized":"2024-09-11T08:39:52.254325+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.254245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.254245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:20.782062+0000","scrub_duration":0.001581877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175203+0000","last_change":"2024-09-11T08:38:40.973815+0000","last_active":"2024-09-11T08:39:36.175203+0000","last_peered":"2024-09-11T08:39:36.175203+0000","last_clean":"2024-09-11T08:39:36.175203+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:39:36.175203+0000","last_undegraded":"2024-09-11T08:39:36.175203+0000","last_fullsized":"2024-09-11T08:39:36.175203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.866315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175392+0000","last_change":"2024-09-11T08:38:38.952604+0000","last_active":"2024-09-11T08:39:36.175392+0000","last_peered":"2024-09-11T08:39:36.175392+0000","last_clean":"2024-09-11T08:39:36.175392+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:39:36.175392+0000","last_undegraded":"2024-09-11T08:39:36.175392+0000","last_fullsized":"2024-09-11T08:39:36.175392+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:51.807236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499809+0000","last_change":"2024-09-11T08:38:42.979138+0000","last_active":"2024-09-11T08:39:35.499809+0000","last_peered":"2024-09-11T08:39:35.499809+0000","last_clean":"2024-09-11T08:39:35.499809+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:39:35.499809+0000","last_undegraded":"2024-09-11T08:39:35.499809+0000","last_fullsized":"2024-09-11T08:39:35.499809+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:08.203025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.444988+0000","last_change":"2024-09-11T08:38:44.994241+0000","last_active":"2024-09-11T08:39:36.444988+0000","last_peered":"2024-09-11T08:39:36.444988+0000","last_clean":"2024-09-11T08:39:36.444988+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:39:36.444988+0000","last_undegraded":"2024-09-11T08:39:36.444988+0000","last_fullsized":"2024-09-11T08:39:36.444988+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:02.265888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499942+0000","last_change":"2024-09-11T08:38:40.976003+0000","last_active":"2024-09-11T08:39:35.499942+0000","last_peered":"2024-09-11T08:39:35.499942+0000","last_clean":"2024-09-11T08:39:35.499942+0000","last_became_active":"2024-09-11T08:38:40.975709+0000","last_became_peered":"2024-09-11T08:38:40.975709+0000","last_unstale":"2024-09-11T08:39:35.499942+0000","last_undegraded":"2024-09-11T08:39:35.499942+0000","last_fullsized":"2024-09-11T08:39:35.499942+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.631901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842846+0000","last_change":"2024-09-11T08:38:38.954570+0000","last_active":"2024-09-11T08:39:35.842846+0000","last_peered":"2024-09-11T08:39:35.842846+0000","last_clean":"2024-09-11T08:39:35.842846+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:39:35.842846+0000","last_undegraded":"2024-09-11T08:39:35.842846+0000","last_fullsized":"2024-09-11T08:39:35.842846+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:05.920652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445629+0000","last_change":"2024-09-11T08:38:42.984381+0000","last_active":"2024-09-11T08:39:36.445629+0000","last_peered":"2024-09-11T08:39:36.445629+0000","last_clean":"2024-09-11T08:39:36.445629+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:39:36.445629+0000","last_undegraded":"2024-09-11T08:39:36.445629+0000","last_fullsized":"2024-09-11T08:39:36.445629+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:54.394345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499567+0000","last_change":"2024-09-11T08:38:44.999759+0000","last_active":"2024-09-11T08:39:35.499567+0000","last_peered":"2024-09-11T08:39:35.499567+0000","last_clean":"2024-09-11T08:39:35.499567+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:39:35.499567+0000","last_undegraded":"2024-09-11T08:39:35.499567+0000","last_fullsized":"2024-09-11T08:39:35.499567+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:03.249276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445088+0000","last_change":"2024-09-11T08:38:41.039878+0000","last_active":"2024-09-11T08:39:36.445088+0000","last_peered":"2024-09-11T08:39:36.445088+0000","last_clean":"2024-09-11T08:39:36.445088+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:39:36.445088+0000","last_undegraded":"2024-09-11T08:39:36.445088+0000","last_fullsized":"2024-09-11T08:39:36.445088+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:08.136760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501978+0000","last_change":"2024-09-11T08:38:38.951115+0000","last_active":"2024-09-11T08:39:35.501978+0000","last_peered":"2024-09-11T08:39:35.501978+0000","last_clean":"2024-09-11T08:39:35.501978+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:39:35.501978+0000","last_undegraded":"2024-09-11T08:39:35.501978+0000","last_fullsized":"2024-09-11T08:39:35.501978+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:51.243171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:47.104746+0000","last_change":"2024-09-11T08:39:47.104702+0000","last_active":"2024-09-11T08:39:47.104746+0000","last_peered":"2024-09-11T08:39:47.104746+0000","last_clean":"2024-09-11T08:39:47.104746+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:39:47.104746+0000","last_undegraded":"2024-09-11T08:39:47.104746+0000","last_fullsized":"2024-09-11T08:39:47.104746+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.104637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.104637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:51.311803+0000","scrub_duration":0.0015238020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445776+0000","last_change":"2024-09-11T08:38:45.183913+0000","last_active":"2024-09-11T08:39:36.445776+0000","last_peered":"2024-09-11T08:39:36.445776+0000","last_clean":"2024-09-11T08:39:36.445776+0000","last_became_active":"2024-09-11T08:38:45.183548+0000","last_became_peered":"2024-09-11T08:38:45.183548+0000","last_unstale":"2024-09-11T08:39:36.445776+0000","last_undegraded":"2024-09-11T08:39:36.445776+0000","last_fullsized":"2024-09-11T08:39:36.445776+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:07.678681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":44,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:41.989766+0000","last_change":"2024-09-11T08:39:41.989740+0000","last_active":"2024-09-11T08:39:41.989766+0000","last_peered":"2024-09-11T08:39:41.989766+0000","last_clean":"2024-09-11T08:39:41.989766+0000","last_became_active":"2024-09-11T08:38:40.973799+0000","last_became_peered":"2024-09-11T08:38:40.973799+0000","last_unstale":"2024-09-11T08:39:41.989766+0000","last_undegraded":"2024-09-11T08:39:41.989766+0000","last_fullsized":"2024-09-11T08:39:41.989766+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-11T08:39:41.989684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.989684+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.687182+0000","scrub_duration":0.00162959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500414+0000","last_change":"2024-09-11T08:38:38.956548+0000","last_active":"2024-09-11T08:39:35.500414+0000","last_peered":"2024-09-11T08:39:35.500414+0000","last_clean":"2024-09-11T08:39:35.500414+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:39:35.500414+0000","last_undegraded":"2024-09-11T08:39:35.500414+0000","last_fullsized":"2024-09-11T08:39:35.500414+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:50.245039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:53.541675+0000","last_change":"2024-09-11T08:39:53.541628+0000","last_active":"2024-09-11T08:39:53.541675+0000","last_peered":"2024-09-11T08:39:53.541675+0000","last_clean":"2024-09-11T08:39:53.541675+0000","last_became_active":"2024-09-11T08:38:42.989556+0000","last_became_peered":"2024-09-11T08:38:42.989556+0000","last_unstale":"2024-09-11T08:39:53.541675+0000","last_undegraded":"2024-09-11T08:39:53.541675+0000","last_fullsized":"2024-09-11T08:39:53.541675+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.541557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.541557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:08.967817+0000","scrub_duration":0.001310285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842390+0000","last_change":"2024-09-11T08:38:44.993185+0000","last_active":"2024-09-11T08:39:35.842390+0000","last_peered":"2024-09-11T08:39:35.842390+0000","last_clean":"2024-09-11T08:39:35.842390+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:39:35.842390+0000","last_undegraded":"2024-09-11T08:39:35.842390+0000","last_fullsized":"2024-09-11T08:39:35.842390+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:53.658762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.480694+0000","last_change":"2024-09-11T08:39:52.480663+0000","last_active":"2024-09-11T08:39:52.480694+0000","last_peered":"2024-09-11T08:39:52.480694+0000","last_clean":"2024-09-11T08:39:52.480694+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:39:52.480694+0000","last_undegraded":"2024-09-11T08:39:52.480694+0000","last_fullsized":"2024-09-11T08:39:52.480694+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.480610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.480610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:55.608483+0000","scrub_duration":0.001335754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:53.038381+0000","last_change":"2024-09-11T08:39:53.038356+0000","last_active":"2024-09-11T08:39:53.038381+0000","last_peered":"2024-09-11T08:39:53.038381+0000","last_clean":"2024-09-11T08:39:53.038381+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:39:53.038381+0000","last_undegraded":"2024-09-11T08:39:53.038381+0000","last_fullsized":"2024-09-11T08:39:53.038381+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.038297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.038297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:58.424182+0000","scrub_duration":0.0012198549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.089846+0000","last_change":"2024-09-11T08:38:40.974827+0000","last_active":"2024-09-11T08:39:36.089846+0000","last_peered":"2024-09-11T08:39:36.089846+0000","last_clean":"2024-09-11T08:39:36.089846+0000","last_became_active":"2024-09-11T08:38:40.974451+0000","last_became_peered":"2024-09-11T08:38:40.974451+0000","last_unstale":"2024-09-11T08:39:36.089846+0000","last_undegraded":"2024-09-11T08:39:36.089846+0000","last_fullsized":"2024-09-11T08:39:36.089846+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:02.126833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:55.265182+0000","last_change":"2024-09-11T08:39:55.265155+0000","last_active":"2024-09-11T08:39:55.265182+0000","last_peered":"2024-09-11T08:39:55.265182+0000","last_clean":"2024-09-11T08:39:55.265182+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:39:55.265182+0000","last_undegraded":"2024-09-11T08:39:55.265182+0000","last_fullsized":"2024-09-11T08:39:55.265182+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.265097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.265097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.333633+0000","scrub_duration":0.0012893830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":423,"ondisk_log_size":423,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299414,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298788,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446364,"statfs":{"total":95995035648,"available":95689076736,"internally_reserved":0,"allocated":1470464,"data_stored":983979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.28899999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.377}]},{"osd":4,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.42999999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.73399999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.60499999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528347,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298768,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446384,"statfs":{"total":95995035648,"available":95689097216,"internally_reserved":0,"allocated":1449984,"data_stored":983206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.41299999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.42799999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.45000000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.437},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.505}]},{"osd":4,"last update":"Wed Sep 11 08:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.51100000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.38}]},{"osd":7,"last update":"Wed Sep 11 08:39:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.48899999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757279,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298204,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446948,"statfs":{"total":95995035648,"available":95689674752,"internally_reserved":0,"allocated":872448,"data_stored":394134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40899999999999997}]},{"osd":1,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.29899999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.32800000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.378}]},{"osd":4,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.376},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.311}]},{"osd":6,"last update":"Wed Sep 11 08:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.28499999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.41699999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298204,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446948,"statfs":{"total":95995035648,"available":95689674752,"internally_reserved":0,"allocated":872448,"data_stored":392829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.56299999999999994,"15min":0.57299999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68899999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.50700000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55800000000000005,"15min":0.56399999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":0.754,"15min":0.754},"last":0.48999999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70799999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.71099999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67100000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75800000000000001,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.47799999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.40500000000000003,"15min":0.38700000000000001},"min":{"1min":0.32100000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.30099999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.35999999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.374}]},{"osd":7,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.38500000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149737,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":901120,"data_stored":394654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.38},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.53200000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40300000000000002,"15min":0.40500000000000003},"min":{"1min":0.29399999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.52900000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.318}]},{"osd":1,"last update":"Wed Sep 11 08:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.40600000000000003,"15min":0.40899999999999997},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51100000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.38},{"interface":"front","average":{"1min":0.433,"5min":0.42199999999999999,"15min":0.41999999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.33900000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.443,"15min":0.441},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.435,"15min":0.438},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.29699999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40699999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.752,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40400000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.45100000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.441},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.55700000000000005}]},{"osd":7,"last update":"Wed Sep 11 08:39:27 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.496}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298180,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446972,"statfs":{"total":95995035648,"available":95689699328,"internally_reserved":0,"allocated":847872,"data_stored":392013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36499999999999999,"15min":0.35399999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.23899999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.25800000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.39100000000000001,"15min":0.376},"min":{"1min":0.28999999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.27000000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:39:57 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.29899999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.441,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:39:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.435},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.44500000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.40699999999999997}]},{"osd":7,"last update":"Wed Sep 11 08:39:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.501}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298240,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446912,"statfs":{"total":95995035648,"available":95689637888,"internally_reserved":0,"allocated":909312,"data_stored":393938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.32700000000000001,"15min":0.313},"min":{"1min":0.24199999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.49199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.378},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.34499999999999997,"15min":0.33000000000000002},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.47599999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:39:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.42199999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:39:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40400000000000003,"15min":0.41999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54400000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.433,"15min":0.434},"min":{"1min":0.307,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.47599999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79900000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48899999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:39:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.755,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82199999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.499}]},{"osd":6,"last update":"Wed Sep 11 08:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.66500000000000004}]},{"osd":7,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.622}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298792,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446360,"statfs":{"total":95995035648,"available":95689072640,"internally_reserved":0,"allocated":1474560,"data_stored":984038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.32300000000000001,"15min":0.31},"min":{"1min":0.20499999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.33400000000000002,"15min":0.312},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.34499999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.35799999999999998,"15min":0.34699999999999998},"min":{"1min":0.24199999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53500000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.34899999999999998},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.51500000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.30499999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:39:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39800000000000002,"15min":0.40000000000000002},"min":{"1min":0.28499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.503,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.28999999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69699999999999995,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.41099999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.45800000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.432}]},{"osd":7,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.53300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:40:07.364 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T08:40:07.364 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T08:40:07.364 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T08:40:07.365 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T08:40:07.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:07 smithi145 ceph-mon[33384]: from='client.24590 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:07.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:07 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:07.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:07 smithi145 ceph-mon[33384]: 6.1b scrub starts 2024-09-11T08:40:07.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:07 smithi145 ceph-mon[33384]: 6.1b scrub ok 2024-09-11T08:40:07.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:07 smithi145 ceph-mon[33384]: 6.12 scrub starts 2024-09-11T08:40:07.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:07 smithi145 ceph-mon[33384]: 6.12 scrub ok 2024-09-11T08:40:07.693 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 podman[76815]: 2024-09-11 08:40:07.433853622 +0000 UTC m=+2.719307357 container create 4f4e713d1f2589253d2b282470b0bd4d5c440efa367cc801450a8531088c8b37 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:40:07.693 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 podman[76815]: 2024-09-11 08:40:07.359984197 +0000 UTC m=+2.645437943 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T08:40:07.693 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 podman[76815]: 2024-09-11 08:40:07.693945945 +0000 UTC m=+2.979399683 container init 4f4e713d1f2589253d2b282470b0bd4d5c440efa367cc801450a8531088c8b37 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:40:07.815 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:07.961 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 podman[76815]: 2024-09-11 08:40:07.699181585 +0000 UTC m=+2.984635323 container start 4f4e713d1f2589253d2b282470b0bd4d5c440efa367cc801450a8531088c8b37 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:40:07.962 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.703Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T08:40:07.962 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.703Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T08:40:07.962 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.703Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T08:40:07.962 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T08:40:07.962 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T08:40:07.962 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T08:40:07.962 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T08:40:07.962 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T08:40:07.962 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T08:40:07.962 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T08:40:07.962 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T08:40:07.963 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T08:40:07.963 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T08:40:07.963 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T08:40:07.963 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T08:40:07.963 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T08:40:07.963 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T08:40:07.963 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T08:40:07.964 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T08:40:07.964 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T08:40:07.964 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T08:40:07.964 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T08:40:07.964 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T08:40:07.964 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T08:40:07.964 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T08:40:07.964 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T08:40:07.964 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T08:40:07.964 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T08:40:07.964 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T08:40:07.964 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T08:40:07.964 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T08:40:07.965 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T08:40:07.965 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T08:40:07.965 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T08:40:07.965 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T08:40:07.965 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T08:40:07.965 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T08:40:07.965 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T08:40:07.965 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T08:40:07.965 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T08:40:07.965 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T08:40:07.965 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T08:40:07.965 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T08:40:07.965 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T08:40:07.966 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T08:40:07.966 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T08:40:07.966 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T08:40:07.966 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T08:40:07.966 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T08:40:07.966 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T08:40:07.966 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.704Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T08:40:07.966 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.705Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T08:40:07.966 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a[77015]: ts=2024-09-11T08:40:07.705Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T08:40:07.966 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 bash[76815]: 4f4e713d1f2589253d2b282470b0bd4d5c440efa367cc801450a8531088c8b37 2024-09-11T08:40:07.966 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 08:40:07 smithi092 systemd[1]: Started Ceph node-exporter.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:40:08.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:08 smithi092 ceph-mon[29647]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:08.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:08 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:08.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:08 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:08.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:08 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:08.306 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:08 smithi092 ceph-mon[41114]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:08.306 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:08 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:08.306 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:08 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:08.306 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:08 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:08.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:08 smithi145 ceph-mon[33384]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:08.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:08 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:08.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:08 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:08.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:08 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:08.912 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:08 smithi145 bash[66713]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T08:40:09.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:09 smithi145 ceph-mon[33384]: Deploying daemon node-exporter.b on smithi145 2024-09-11T08:40:09.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:09 smithi145 ceph-mon[33384]: 3.1d scrub starts 2024-09-11T08:40:09.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:09 smithi145 ceph-mon[33384]: 3.1d scrub ok 2024-09-11T08:40:09.471 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:09 smithi092 ceph-mon[29647]: Deploying daemon node-exporter.b on smithi145 2024-09-11T08:40:09.471 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:09 smithi092 ceph-mon[29647]: 3.1d scrub starts 2024-09-11T08:40:09.471 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:09 smithi092 ceph-mon[29647]: 3.1d scrub ok 2024-09-11T08:40:09.471 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:09 smithi092 ceph-mon[41114]: Deploying daemon node-exporter.b on smithi145 2024-09-11T08:40:09.471 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:09 smithi092 ceph-mon[41114]: 3.1d scrub starts 2024-09-11T08:40:09.471 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:09 smithi092 ceph-mon[41114]: 3.1d scrub ok 2024-09-11T08:40:09.846 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:40:09.846 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T08:40:09.911 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:09 smithi145 bash[66713]: Getting image source signatures 2024-09-11T08:40:09.911 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:09 smithi145 bash[66713]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T08:40:09.912 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:09 smithi145 bash[66713]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T08:40:09.912 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:09 smithi145 bash[66713]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T08:40:10.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:10 smithi145 ceph-mon[33384]: 3.17 scrub starts 2024-09-11T08:40:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:10 smithi145 ceph-mon[33384]: 3.17 scrub ok 2024-09-11T08:40:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:10 smithi145 ceph-mon[33384]: 6.1 scrub starts 2024-09-11T08:40:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:10 smithi145 ceph-mon[33384]: 6.1 scrub ok 2024-09-11T08:40:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:10 smithi145 ceph-mon[33384]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:10 smithi145 ceph-mon[33384]: 5.a scrub starts 2024-09-11T08:40:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:10 smithi145 ceph-mon[33384]: 5.a scrub ok 2024-09-11T08:40:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:10 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3669214174' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:40:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:10 smithi145 ceph-mon[33384]: 6.1c scrub starts 2024-09-11T08:40:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:10 smithi145 ceph-mon[33384]: 6.1c scrub ok 2024-09-11T08:40:10.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[29647]: 3.17 scrub starts 2024-09-11T08:40:10.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[29647]: 3.17 scrub ok 2024-09-11T08:40:10.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[29647]: 6.1 scrub starts 2024-09-11T08:40:10.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[29647]: 6.1 scrub ok 2024-09-11T08:40:10.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[29647]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:10.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[29647]: 5.a scrub starts 2024-09-11T08:40:10.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[29647]: 5.a scrub ok 2024-09-11T08:40:10.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3669214174' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:40:10.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[29647]: 6.1c scrub starts 2024-09-11T08:40:10.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[29647]: 6.1c scrub ok 2024-09-11T08:40:10.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[41114]: 3.17 scrub starts 2024-09-11T08:40:10.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[41114]: 3.17 scrub ok 2024-09-11T08:40:10.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[41114]: 6.1 scrub starts 2024-09-11T08:40:10.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[41114]: 6.1 scrub ok 2024-09-11T08:40:10.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[41114]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:10.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[41114]: 5.a scrub starts 2024-09-11T08:40:10.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[41114]: 5.a scrub ok 2024-09-11T08:40:10.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3669214174' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:40:10.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[41114]: 6.1c scrub starts 2024-09-11T08:40:10.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:10 smithi092 ceph-mon[41114]: 6.1c scrub ok 2024-09-11T08:40:10.774 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T08:40:10.774 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T08:40:10.774 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-11T08:40:10.786 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-11T08:40:10.786 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-guits-quincy-2024-09-10-2008-quincy'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'cephadm_mode': 'root'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'wip-guits-quincy-2024-09-10-2008-quincy', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'}} 2024-09-11T08:40:10.786 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-11T08:40:10.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-11T08:40:11.037 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:11.411 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:10 smithi145 bash[66713]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T08:40:11.412 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 bash[66713]: Writing manifest to image destination 2024-09-11T08:40:11.723 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 podman[66713]: 2024-09-11 08:40:11.376907608 +0000 UTC m=+2.730609404 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T08:40:11.724 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 podman[66713]: 2024-09-11 08:40:11.4841771 +0000 UTC m=+2.837878896 container create 256664afc9fccad27937179d5fcf67aa40753a56ff735c49f2ed613770544542 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:40:11.724 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 podman[66713]: 2024-09-11 08:40:11.719740163 +0000 UTC m=+3.073441968 container init 256664afc9fccad27937179d5fcf67aa40753a56ff735c49f2ed613770544542 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:40:11.724 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 podman[66713]: 2024-09-11 08:40:11.724567338 +0000 UTC m=+3.078269135 container start 256664afc9fccad27937179d5fcf67aa40753a56ff735c49f2ed613770544542 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:40:12.162 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.728Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T08:40:12.162 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.728Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T08:40:12.162 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.728Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T08:40:12.163 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.728Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T08:40:12.163 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.728Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T08:40:12.163 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.728Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T08:40:12.163 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T08:40:12.163 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T08:40:12.163 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T08:40:12.163 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T08:40:12.163 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T08:40:12.163 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T08:40:12.163 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T08:40:12.163 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T08:40:12.163 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T08:40:12.163 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T08:40:12.163 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T08:40:12.164 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T08:40:12.164 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T08:40:12.164 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T08:40:12.164 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T08:40:12.164 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T08:40:12.164 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T08:40:12.164 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T08:40:12.165 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T08:40:12.165 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T08:40:12.165 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T08:40:12.165 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T08:40:12.165 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T08:40:12.165 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T08:40:12.165 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T08:40:12.165 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T08:40:12.165 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T08:40:12.165 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T08:40:12.165 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T08:40:12.165 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T08:40:12.165 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T08:40:12.165 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T08:40:12.166 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T08:40:12.166 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T08:40:12.166 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T08:40:12.166 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T08:40:12.166 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T08:40:12.166 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T08:40:12.166 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T08:40:12.166 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T08:40:12.166 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T08:40:12.166 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T08:40:12.166 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T08:40:12.166 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T08:40:12.167 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T08:40:12.167 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T08:40:12.167 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b[66786]: ts=2024-09-11T08:40:11.729Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T08:40:12.167 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 bash[66713]: 256664afc9fccad27937179d5fcf67aa40753a56ff735c49f2ed613770544542 2024-09-11T08:40:12.167 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 08:40:11 smithi145 systemd[1]: Started Ceph node-exporter.b for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:40:12.466 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[29647]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:12.466 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[29647]: 6.a scrub starts 2024-09-11T08:40:12.466 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[29647]: 6.a scrub ok 2024-09-11T08:40:12.466 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[29647]: 5.b scrub starts 2024-09-11T08:40:12.467 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[29647]: 5.b scrub ok 2024-09-11T08:40:12.467 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:12.467 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:12.467 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:12.467 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:12.467 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:12.467 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:40:12.467 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[41114]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:12.467 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[41114]: 6.a scrub starts 2024-09-11T08:40:12.467 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[41114]: 6.a scrub ok 2024-09-11T08:40:12.467 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[41114]: 5.b scrub starts 2024-09-11T08:40:12.467 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[41114]: 5.b scrub ok 2024-09-11T08:40:12.468 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:12.468 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:12.468 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:12.468 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:12.468 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:12.468 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:12 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:40:12.622 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:12 smithi145 ceph-mon[33384]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:12.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:12 smithi145 ceph-mon[33384]: 6.a scrub starts 2024-09-11T08:40:12.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:12 smithi145 ceph-mon[33384]: 6.a scrub ok 2024-09-11T08:40:12.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:12 smithi145 ceph-mon[33384]: 5.b scrub starts 2024-09-11T08:40:12.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:12 smithi145 ceph-mon[33384]: 5.b scrub ok 2024-09-11T08:40:12.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:12 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:12.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:12 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:12.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:12 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:12.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:12 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:12.758 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:12 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:12.758 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:12 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:40:12.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:40:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:40:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:13.315 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:13] ENGINE Bus STOPPING 2024-09-11T08:40:13.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-mon[29647]: 5.e scrub starts 2024-09-11T08:40:13.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-mon[29647]: 5.e scrub ok 2024-09-11T08:40:13.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-mon[29647]: 3.1c scrub starts 2024-09-11T08:40:13.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-mon[29647]: 3.1c scrub ok 2024-09-11T08:40:13.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' 2024-09-11T08:40:13.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-mon[41114]: 5.e scrub starts 2024-09-11T08:40:13.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-mon[41114]: 5.e scrub ok 2024-09-11T08:40:13.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-mon[41114]: 3.1c scrub starts 2024-09-11T08:40:13.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-mon[41114]: 3.1c scrub ok 2024-09-11T08:40:13.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' 2024-09-11T08:40:13.558 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:13 smithi145 ceph-mon[33384]: 5.e scrub starts 2024-09-11T08:40:13.558 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:13 smithi145 ceph-mon[33384]: 5.e scrub ok 2024-09-11T08:40:13.558 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:13 smithi145 ceph-mon[33384]: 3.1c scrub starts 2024-09-11T08:40:13.558 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:13 smithi145 ceph-mon[33384]: 3.1c scrub ok 2024-09-11T08:40:13.559 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:13 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' 2024-09-11T08:40:13.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:13] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:40:13.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:13] ENGINE Bus STOPPED 2024-09-11T08:40:13.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:13] ENGINE Bus STARTING 2024-09-11T08:40:13.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:13] ENGINE Serving on http://:::9283 2024-09-11T08:40:13.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:13 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:13] ENGINE Bus STARTED 2024-09-11T08:40:13.839 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-11T08:40:13.839 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T08:40:13.839 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:40:14.194 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:14.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[29647]: 5.19 scrub starts 2024-09-11T08:40:14.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[29647]: 5.19 scrub ok 2024-09-11T08:40:14.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[29647]: 5.1f scrub starts 2024-09-11T08:40:14.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[29647]: 5.1f scrub ok 2024-09-11T08:40:14.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[29647]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:14.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[29647]: 5.9 scrub starts 2024-09-11T08:40:14.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[29647]: 5.9 scrub ok 2024-09-11T08:40:14.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T08:40:14.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[29647]: 6.f scrub ok 2024-09-11T08:40:14.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[41114]: 5.19 scrub starts 2024-09-11T08:40:14.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[41114]: 5.19 scrub ok 2024-09-11T08:40:14.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[41114]: 5.1f scrub starts 2024-09-11T08:40:14.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[41114]: 5.1f scrub ok 2024-09-11T08:40:14.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[41114]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:14.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[41114]: 5.9 scrub starts 2024-09-11T08:40:14.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[41114]: 5.9 scrub ok 2024-09-11T08:40:14.448 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T08:40:14.448 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:14 smithi092 ceph-mon[41114]: 6.f scrub ok 2024-09-11T08:40:14.456 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:14 smithi145 ceph-mon[33384]: 5.19 scrub starts 2024-09-11T08:40:14.456 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:14 smithi145 ceph-mon[33384]: 5.19 scrub ok 2024-09-11T08:40:14.456 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:14 smithi145 ceph-mon[33384]: 5.1f scrub starts 2024-09-11T08:40:14.456 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:14 smithi145 ceph-mon[33384]: 5.1f scrub ok 2024-09-11T08:40:14.456 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:14 smithi145 ceph-mon[33384]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:14.456 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:14 smithi145 ceph-mon[33384]: 5.9 scrub starts 2024-09-11T08:40:14.456 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:14 smithi145 ceph-mon[33384]: 5.9 scrub ok 2024-09-11T08:40:14.457 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:14 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T08:40:14.457 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:14 smithi145 ceph-mon[33384]: 6.f scrub ok 2024-09-11T08:40:15.395 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[29647]: 4.2 scrub starts 2024-09-11T08:40:15.395 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[29647]: 4.2 scrub ok 2024-09-11T08:40:15.395 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[29647]: 3.13 scrub starts 2024-09-11T08:40:15.395 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[29647]: 3.13 scrub ok 2024-09-11T08:40:15.396 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-11T08:40:15.396 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-11T08:40:15.396 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-11T08:40:15.396 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-09-11T08:40:15.396 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-11T08:40:15.396 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-09-11T08:40:15.396 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:15.396 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:15.396 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[41114]: 4.2 scrub starts 2024-09-11T08:40:15.397 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[41114]: 4.2 scrub ok 2024-09-11T08:40:15.397 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[41114]: 3.13 scrub starts 2024-09-11T08:40:15.397 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[41114]: 3.13 scrub ok 2024-09-11T08:40:15.397 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-11T08:40:15.397 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-11T08:40:15.397 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-11T08:40:15.397 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-09-11T08:40:15.397 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-11T08:40:15.397 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-09-11T08:40:15.397 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:15.397 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:15 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:15.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:15 smithi145 ceph-mon[33384]: 4.2 scrub starts 2024-09-11T08:40:15.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:15 smithi145 ceph-mon[33384]: 4.2 scrub ok 2024-09-11T08:40:15.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:15 smithi145 ceph-mon[33384]: 3.13 scrub starts 2024-09-11T08:40:15.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:15 smithi145 ceph-mon[33384]: 3.13 scrub ok 2024-09-11T08:40:15.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-11T08:40:15.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-11T08:40:15.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:15 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-11T08:40:15.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-09-11T08:40:15.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:15 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-11T08:40:15.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:15 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-09-11T08:40:15.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:15.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:15 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:16.120 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:40:16.121 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:40:16.161 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:40:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:40:16.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:16 smithi092 ceph-mon[29647]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:16.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:16 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-11T08:40:16.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:16 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-09-11T08:40:16.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:16 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-11T08:40:16.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:16 smithi092 ceph-mon[29647]: osdmap e79: 8 total, 8 up, 8 in 2024-09-11T08:40:16.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:16 smithi092 ceph-mon[29647]: 6.5 scrub starts 2024-09-11T08:40:16.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:16 smithi092 ceph-mon[29647]: 6.5 scrub ok 2024-09-11T08:40:16.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:16 smithi092 ceph-mon[41114]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:16.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:16 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-11T08:40:16.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:16 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-09-11T08:40:16.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:16 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-11T08:40:16.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:16 smithi092 ceph-mon[41114]: osdmap e79: 8 total, 8 up, 8 in 2024-09-11T08:40:16.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:16 smithi092 ceph-mon[41114]: 6.5 scrub starts 2024-09-11T08:40:16.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:16 smithi092 ceph-mon[41114]: 6.5 scrub ok 2024-09-11T08:40:16.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:16 smithi145 ceph-mon[33384]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:16 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-11T08:40:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:16 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-09-11T08:40:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:16 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-11T08:40:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:16 smithi145 ceph-mon[33384]: osdmap e79: 8 total, 8 up, 8 in 2024-09-11T08:40:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:16 smithi145 ceph-mon[33384]: 6.5 scrub starts 2024-09-11T08:40:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:16 smithi145 ceph-mon[33384]: 6.5 scrub ok 2024-09-11T08:40:16.898 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2024-09-11T08:40:15.176030+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":938,"num_read_kb":795,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":580,"ondisk_log_size":580,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2387408,"kb_used_data":8592,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747573808,"statfs":{"total":767960285184,"available":765515579392,"internally_reserved":0,"allocated":8798208,"data_stored":4918791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.164856"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:47.339397+0000","last_change":"2024-09-11T08:39:47.339366+0000","last_active":"2024-09-11T08:39:47.339397+0000","last_peered":"2024-09-11T08:39:47.339397+0000","last_clean":"2024-09-11T08:39:47.339397+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:39:47.339397+0000","last_undegraded":"2024-09-11T08:39:47.339397+0000","last_fullsized":"2024-09-11T08:39:47.339397+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.339312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.339312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.154202+0000","scrub_duration":0.001323095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:56.666807+0000","last_change":"2024-09-11T08:39:56.666764+0000","last_active":"2024-09-11T08:39:56.666807+0000","last_peered":"2024-09-11T08:39:56.666807+0000","last_clean":"2024-09-11T08:39:56.666807+0000","last_became_active":"2024-09-11T08:38:40.973398+0000","last_became_peered":"2024-09-11T08:38:40.973398+0000","last_unstale":"2024-09-11T08:39:56.666807+0000","last_undegraded":"2024-09-11T08:39:56.666807+0000","last_fullsized":"2024-09-11T08:39:56.666807+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.666703+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.544222+0000","scrub_duration":0.0041022000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499643+0000","last_change":"2024-09-11T08:38:42.980578+0000","last_active":"2024-09-11T08:39:35.499643+0000","last_peered":"2024-09-11T08:39:35.499643+0000","last_clean":"2024-09-11T08:39:35.499643+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:39:35.499643+0000","last_undegraded":"2024-09-11T08:39:35.499643+0000","last_fullsized":"2024-09-11T08:39:35.499643+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.101278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175049+0000","last_change":"2024-09-11T08:38:44.992111+0000","last_active":"2024-09-11T08:39:36.175049+0000","last_peered":"2024-09-11T08:39:36.175049+0000","last_clean":"2024-09-11T08:39:36.175049+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:39:36.175049+0000","last_undegraded":"2024-09-11T08:39:36.175049+0000","last_fullsized":"2024-09-11T08:39:36.175049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:12.528264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:06.173996+0000","last_change":"2024-09-11T08:40:06.173950+0000","last_active":"2024-09-11T08:40:06.173996+0000","last_peered":"2024-09-11T08:40:06.173996+0000","last_clean":"2024-09-11T08:40:06.173996+0000","last_became_active":"2024-09-11T08:38:45.184215+0000","last_became_peered":"2024-09-11T08:38:45.184215+0000","last_unstale":"2024-09-11T08:40:06.173996+0000","last_undegraded":"2024-09-11T08:40:06.173996+0000","last_fullsized":"2024-09-11T08:40:06.173996+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:06.173882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:06.173882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:17.036842+0000","scrub_duration":0.001730875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:58.161111+0000","last_change":"2024-09-11T08:39:58.161055+0000","last_active":"2024-09-11T08:39:58.161111+0000","last_peered":"2024-09-11T08:39:58.161111+0000","last_clean":"2024-09-11T08:39:58.161111+0000","last_became_active":"2024-09-11T08:38:38.949136+0000","last_became_peered":"2024-09-11T08:38:38.949136+0000","last_unstale":"2024-09-11T08:39:58.161111+0000","last_undegraded":"2024-09-11T08:39:58.161111+0000","last_fullsized":"2024-09-11T08:39:58.161111+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:58.160984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.160984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:00.523696+0000","scrub_duration":0.0015109679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":70,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:45.997924+0000","last_change":"2024-09-11T08:39:45.997873+0000","last_active":"2024-09-11T08:39:45.997924+0000","last_peered":"2024-09-11T08:39:45.997924+0000","last_clean":"2024-09-11T08:39:45.997924+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:39:45.997924+0000","last_undegraded":"2024-09-11T08:39:45.997924+0000","last_fullsized":"2024-09-11T08:39:45.997924+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.997803+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.534414+0000","scrub_duration":0.0046870369999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.174897+0000","last_change":"2024-09-11T08:38:42.983653+0000","last_active":"2024-09-11T08:39:36.174897+0000","last_peered":"2024-09-11T08:39:36.174897+0000","last_clean":"2024-09-11T08:39:36.174897+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:39:36.174897+0000","last_undegraded":"2024-09-11T08:39:36.174897+0000","last_fullsized":"2024-09-11T08:39:36.174897+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:57.589237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:08.416602+0000","last_change":"2024-09-11T08:40:08.416577+0000","last_active":"2024-09-11T08:40:08.416602+0000","last_peered":"2024-09-11T08:40:08.416602+0000","last_clean":"2024-09-11T08:40:08.416602+0000","last_became_active":"2024-09-11T08:38:38.948930+0000","last_became_peered":"2024-09-11T08:38:38.948930+0000","last_unstale":"2024-09-11T08:40:08.416602+0000","last_undegraded":"2024-09-11T08:40:08.416602+0000","last_fullsized":"2024-09-11T08:40:08.416602+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:08.416522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.416522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:27.445091+0000","scrub_duration":0.001330005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:46.620744+0000","last_change":"2024-09-11T08:39:46.620700+0000","last_active":"2024-09-11T08:39:46.620744+0000","last_peered":"2024-09-11T08:39:46.620744+0000","last_clean":"2024-09-11T08:39:46.620744+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:39:46.620744+0000","last_undegraded":"2024-09-11T08:39:46.620744+0000","last_fullsized":"2024-09-11T08:39:46.620744+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:39:46.620624+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:10.984564+0000","scrub_duration":0.0044132319999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:04.450367+0000","last_change":"2024-09-11T08:40:04.450336+0000","last_active":"2024-09-11T08:40:04.450367+0000","last_peered":"2024-09-11T08:40:04.450367+0000","last_clean":"2024-09-11T08:40:04.450367+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:40:04.450367+0000","last_undegraded":"2024-09-11T08:40:04.450367+0000","last_fullsized":"2024-09-11T08:40:04.450367+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:04.450282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:04.450282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.499239+0000","scrub_duration":0.001689433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:51.280393+0000","last_change":"2024-09-11T08:39:51.280365+0000","last_active":"2024-09-11T08:39:51.280393+0000","last_peered":"2024-09-11T08:39:51.280393+0000","last_clean":"2024-09-11T08:39:51.280393+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:39:51.280393+0000","last_undegraded":"2024-09-11T08:39:51.280393+0000","last_fullsized":"2024-09-11T08:39:51.280393+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.280310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.280310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:06.712880+0000","scrub_duration":0.001634777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501649+0000","last_change":"2024-09-11T08:38:38.950588+0000","last_active":"2024-09-11T08:39:35.501649+0000","last_peered":"2024-09-11T08:39:35.501649+0000","last_clean":"2024-09-11T08:39:35.501649+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:39:35.501649+0000","last_undegraded":"2024-09-11T08:39:35.501649+0000","last_fullsized":"2024-09-11T08:39:35.501649+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:58.714558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175109+0000","last_change":"2024-09-11T08:38:40.974291+0000","last_active":"2024-09-11T08:39:36.175109+0000","last_peered":"2024-09-11T08:39:36.175109+0000","last_clean":"2024-09-11T08:39:36.175109+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:39:36.175109+0000","last_undegraded":"2024-09-11T08:39:36.175109+0000","last_fullsized":"2024-09-11T08:39:36.175109+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:57.136170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:51.166149+0000","last_change":"2024-09-11T08:39:51.166114+0000","last_active":"2024-09-11T08:39:51.166149+0000","last_peered":"2024-09-11T08:39:51.166149+0000","last_clean":"2024-09-11T08:39:51.166149+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:39:51.166149+0000","last_undegraded":"2024-09-11T08:39:51.166149+0000","last_fullsized":"2024-09-11T08:39:51.166149+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.166051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.570790+0000","scrub_duration":0.0016440420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501268+0000","last_change":"2024-09-11T08:38:44.994259+0000","last_active":"2024-09-11T08:39:35.501268+0000","last_peered":"2024-09-11T08:39:35.501268+0000","last_clean":"2024-09-11T08:39:35.501268+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:39:35.501268+0000","last_undegraded":"2024-09-11T08:39:35.501268+0000","last_fullsized":"2024-09-11T08:39:35.501268+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:03.579441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:59.606552+0000","last_change":"2024-09-11T08:39:59.606521+0000","last_active":"2024-09-11T08:39:59.606552+0000","last_peered":"2024-09-11T08:39:59.606552+0000","last_clean":"2024-09-11T08:39:59.606552+0000","last_became_active":"2024-09-11T08:38:45.183364+0000","last_became_peered":"2024-09-11T08:38:45.183364+0000","last_unstale":"2024-09-11T08:39:59.606552+0000","last_undegraded":"2024-09-11T08:39:59.606552+0000","last_fullsized":"2024-09-11T08:39:59.606552+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:59.606458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:59.606458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.547769+0000","scrub_duration":0.0012065960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500980+0000","last_change":"2024-09-11T08:38:38.954075+0000","last_active":"2024-09-11T08:39:35.500980+0000","last_peered":"2024-09-11T08:39:35.500980+0000","last_clean":"2024-09-11T08:39:35.500980+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:39:35.500980+0000","last_undegraded":"2024-09-11T08:39:35.500980+0000","last_fullsized":"2024-09-11T08:39:35.500980+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:04.776009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":71,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:50.472453+0000","last_change":"2024-09-11T08:39:50.472419+0000","last_active":"2024-09-11T08:39:50.472453+0000","last_peered":"2024-09-11T08:39:50.472453+0000","last_clean":"2024-09-11T08:39:50.472453+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:39:50.472453+0000","last_undegraded":"2024-09-11T08:39:50.472453+0000","last_fullsized":"2024-09-11T08:39:50.472453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.472368+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:10.784548+0000","scrub_duration":0.0049809520000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:44.632395+0000","last_change":"2024-09-11T08:39:44.632369+0000","last_active":"2024-09-11T08:39:44.632395+0000","last_peered":"2024-09-11T08:39:44.632395+0000","last_clean":"2024-09-11T08:39:44.632395+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:39:44.632395+0000","last_undegraded":"2024-09-11T08:39:44.632395+0000","last_fullsized":"2024-09-11T08:39:44.632395+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:44.632308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:44.632308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:56.408810+0000","scrub_duration":0.001420377,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445687+0000","last_change":"2024-09-11T08:38:45.183646+0000","last_active":"2024-09-11T08:39:36.445687+0000","last_peered":"2024-09-11T08:39:36.445687+0000","last_clean":"2024-09-11T08:39:36.445687+0000","last_became_active":"2024-09-11T08:38:45.182763+0000","last_became_peered":"2024-09-11T08:38:45.182763+0000","last_unstale":"2024-09-11T08:39:36.445687+0000","last_undegraded":"2024-09-11T08:39:36.445687+0000","last_fullsized":"2024-09-11T08:39:36.445687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:04.100474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175475+0000","last_change":"2024-09-11T08:38:38.949654+0000","last_active":"2024-09-11T08:39:36.175475+0000","last_peered":"2024-09-11T08:39:36.175475+0000","last_clean":"2024-09-11T08:39:36.175475+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:39:36.175475+0000","last_undegraded":"2024-09-11T08:39:36.175475+0000","last_fullsized":"2024-09-11T08:39:36.175475+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:54.740531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:55.125478+0000","last_change":"2024-09-11T08:39:55.125449+0000","last_active":"2024-09-11T08:39:55.125478+0000","last_peered":"2024-09-11T08:39:55.125478+0000","last_clean":"2024-09-11T08:39:55.125478+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:39:55.125478+0000","last_undegraded":"2024-09-11T08:39:55.125478+0000","last_fullsized":"2024-09-11T08:39:55.125478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:55.125394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.125394+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.857707+0000","scrub_duration":0.0016833810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:01.585507+0000","last_change":"2024-09-11T08:40:01.585472+0000","last_active":"2024-09-11T08:40:01.585507+0000","last_peered":"2024-09-11T08:40:01.585507+0000","last_clean":"2024-09-11T08:40:01.585507+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:40:01.585507+0000","last_undegraded":"2024-09-11T08:40:01.585507+0000","last_fullsized":"2024-09-11T08:40:01.585507+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:01.585411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:01.585411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.937039+0000","scrub_duration":0.0021220179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:10.113758+0000","last_change":"2024-09-11T08:40:10.113729+0000","last_active":"2024-09-11T08:40:10.113758+0000","last_peered":"2024-09-11T08:40:10.113758+0000","last_clean":"2024-09-11T08:40:10.113758+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:40:10.113758+0000","last_undegraded":"2024-09-11T08:40:10.113758+0000","last_fullsized":"2024-09-11T08:40:10.113758+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:40:10.113671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:10.113671+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:33.267918+0000","scrub_duration":0.001621302,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:02.458644+0000","last_change":"2024-09-11T08:40:02.458618+0000","last_active":"2024-09-11T08:40:02.458644+0000","last_peered":"2024-09-11T08:40:02.458644+0000","last_clean":"2024-09-11T08:40:02.458644+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:40:02.458644+0000","last_undegraded":"2024-09-11T08:40:02.458644+0000","last_fullsized":"2024-09-11T08:40:02.458644+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:02.458565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.458565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.247079+0000","scrub_duration":0.001315568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500506+0000","last_change":"2024-09-11T08:38:41.041065+0000","last_active":"2024-09-11T08:39:35.500506+0000","last_peered":"2024-09-11T08:39:35.500506+0000","last_clean":"2024-09-11T08:39:35.500506+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:39:35.500506+0000","last_undegraded":"2024-09-11T08:39:35.500506+0000","last_fullsized":"2024-09-11T08:39:35.500506+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:53.140994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":55,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:02.689774+0000","last_change":"2024-09-11T08:38:42.985883+0000","last_active":"2024-09-11T08:40:02.689774+0000","last_peered":"2024-09-11T08:40:02.689774+0000","last_clean":"2024-09-11T08:40:02.689774+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:40:02.689774+0000","last_undegraded":"2024-09-11T08:40:02.689774+0000","last_fullsized":"2024-09-11T08:40:02.689774+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500487+0000","last_change":"2024-09-11T08:38:40.979334+0000","last_active":"2024-09-11T08:39:35.500487+0000","last_peered":"2024-09-11T08:39:35.500487+0000","last_clean":"2024-09-11T08:39:35.500487+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:39:35.500487+0000","last_undegraded":"2024-09-11T08:39:35.500487+0000","last_fullsized":"2024-09-11T08:39:35.500487+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:05.848565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445242+0000","last_change":"2024-09-11T08:38:38.951009+0000","last_active":"2024-09-11T08:39:36.445242+0000","last_peered":"2024-09-11T08:39:36.445242+0000","last_clean":"2024-09-11T08:39:36.445242+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:39:36.445242+0000","last_undegraded":"2024-09-11T08:39:36.445242+0000","last_fullsized":"2024-09-11T08:39:36.445242+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:53.872027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":53,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:02.689770+0000","last_change":"2024-09-11T08:38:42.985885+0000","last_active":"2024-09-11T08:40:02.689770+0000","last_peered":"2024-09-11T08:40:02.689770+0000","last_clean":"2024-09-11T08:40:02.689770+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:40:02.689770+0000","last_undegraded":"2024-09-11T08:40:02.689770+0000","last_fullsized":"2024-09-11T08:40:02.689770+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501622+0000","last_change":"2024-09-11T08:38:44.991592+0000","last_active":"2024-09-11T08:39:35.501622+0000","last_peered":"2024-09-11T08:39:35.501622+0000","last_clean":"2024-09-11T08:39:35.501622+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:39:35.501622+0000","last_undegraded":"2024-09-11T08:39:35.501622+0000","last_fullsized":"2024-09-11T08:39:35.501622+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:06.716187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":70,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:05.197234+0000","last_change":"2024-09-11T08:40:05.197208+0000","last_active":"2024-09-11T08:40:05.197234+0000","last_peered":"2024-09-11T08:40:05.197234+0000","last_clean":"2024-09-11T08:40:05.197234+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:40:05.197234+0000","last_undegraded":"2024-09-11T08:40:05.197234+0000","last_fullsized":"2024-09-11T08:40:05.197234+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-11T08:40:05.197152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:05.197152+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:06.174279+0000","scrub_duration":0.0016976000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.037635+0000","last_change":"2024-09-11T08:39:43.037605+0000","last_active":"2024-09-11T08:39:43.037635+0000","last_peered":"2024-09-11T08:39:43.037635+0000","last_clean":"2024-09-11T08:39:43.037635+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:39:43.037635+0000","last_undegraded":"2024-09-11T08:39:43.037635+0000","last_fullsized":"2024-09-11T08:39:43.037635+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.037551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.037551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.071661+0000","scrub_duration":0.001523976,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.174914+0000","last_change":"2024-09-11T08:38:42.983881+0000","last_active":"2024-09-11T08:39:36.174914+0000","last_peered":"2024-09-11T08:39:36.174914+0000","last_clean":"2024-09-11T08:39:36.174914+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:39:36.174914+0000","last_undegraded":"2024-09-11T08:39:36.174914+0000","last_fullsized":"2024-09-11T08:39:36.174914+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:49.733060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175286+0000","last_change":"2024-09-11T08:38:44.995555+0000","last_active":"2024-09-11T08:39:36.175286+0000","last_peered":"2024-09-11T08:39:36.175286+0000","last_clean":"2024-09-11T08:39:36.175286+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:39:36.175286+0000","last_undegraded":"2024-09-11T08:39:36.175286+0000","last_fullsized":"2024-09-11T08:39:36.175286+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.851425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175199+0000","last_change":"2024-09-11T08:38:40.968377+0000","last_active":"2024-09-11T08:39:36.175199+0000","last_peered":"2024-09-11T08:39:36.175199+0000","last_clean":"2024-09-11T08:39:36.175199+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:39:36.175199+0000","last_undegraded":"2024-09-11T08:39:36.175199+0000","last_fullsized":"2024-09-11T08:39:36.175199+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:09.166194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:41.267346+0000","last_change":"2024-09-11T08:39:41.267320+0000","last_active":"2024-09-11T08:39:41.267346+0000","last_peered":"2024-09-11T08:39:41.267346+0000","last_clean":"2024-09-11T08:39:41.267346+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:39:41.267346+0000","last_undegraded":"2024-09-11T08:39:41.267346+0000","last_fullsized":"2024-09-11T08:39:41.267346+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:41.267267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.267267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:01.547087+0000","scrub_duration":0.0013348209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":55,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:02.689470+0000","last_change":"2024-09-11T08:38:42.982794+0000","last_active":"2024-09-11T08:40:02.689470+0000","last_peered":"2024-09-11T08:40:02.689470+0000","last_clean":"2024-09-11T08:40:02.689470+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:40:02.689470+0000","last_undegraded":"2024-09-11T08:40:02.689470+0000","last_fullsized":"2024-09-11T08:40:02.689470+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:04.174487+0000","last_change":"2024-09-11T08:40:04.174461+0000","last_active":"2024-09-11T08:40:04.174487+0000","last_peered":"2024-09-11T08:40:04.174487+0000","last_clean":"2024-09-11T08:40:04.174487+0000","last_became_active":"2024-09-11T08:38:45.182724+0000","last_became_peered":"2024-09-11T08:38:45.182724+0000","last_unstale":"2024-09-11T08:40:04.174487+0000","last_undegraded":"2024-09-11T08:40:04.174487+0000","last_fullsized":"2024-09-11T08:40:04.174487+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:04.174409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:04.174409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.916947+0000","scrub_duration":0.0012996889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"63'10","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499890+0000","last_change":"2024-09-11T08:38:40.977074+0000","last_active":"2024-09-11T08:39:35.499890+0000","last_peered":"2024-09-11T08:39:35.499890+0000","last_clean":"2024-09-11T08:39:35.499890+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:39:35.499890+0000","last_undegraded":"2024-09-11T08:39:35.499890+0000","last_fullsized":"2024-09-11T08:39:35.499890+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.468488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.502041+0000","last_change":"2024-09-11T08:38:38.951168+0000","last_active":"2024-09-11T08:39:35.502041+0000","last_peered":"2024-09-11T08:39:35.502041+0000","last_clean":"2024-09-11T08:39:35.502041+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:39:35.502041+0000","last_undegraded":"2024-09-11T08:39:35.502041+0000","last_fullsized":"2024-09-11T08:39:35.502041+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.717085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":53,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:02.689490+0000","last_change":"2024-09-11T08:38:42.981732+0000","last_active":"2024-09-11T08:40:02.689490+0000","last_peered":"2024-09-11T08:40:02.689490+0000","last_clean":"2024-09-11T08:40:02.689490+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:40:02.689490+0000","last_undegraded":"2024-09-11T08:40:02.689490+0000","last_fullsized":"2024-09-11T08:40:02.689490+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:08.306042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500551+0000","last_change":"2024-09-11T08:38:44.998257+0000","last_active":"2024-09-11T08:39:35.500551+0000","last_peered":"2024-09-11T08:39:35.500551+0000","last_clean":"2024-09-11T08:39:35.500551+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:39:35.500551+0000","last_undegraded":"2024-09-11T08:39:35.500551+0000","last_fullsized":"2024-09-11T08:39:35.500551+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:12.569737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":65,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500898+0000","last_change":"2024-09-11T08:38:41.040034+0000","last_active":"2024-09-11T08:39:35.500898+0000","last_peered":"2024-09-11T08:39:35.500898+0000","last_clean":"2024-09-11T08:39:35.500898+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:39:35.500898+0000","last_undegraded":"2024-09-11T08:39:35.500898+0000","last_fullsized":"2024-09-11T08:39:35.500898+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:07.747665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:03.496030+0000","last_change":"2024-09-11T08:40:03.496005+0000","last_active":"2024-09-11T08:40:03.496030+0000","last_peered":"2024-09-11T08:40:03.496030+0000","last_clean":"2024-09-11T08:40:03.496030+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:40:03.496030+0000","last_undegraded":"2024-09-11T08:40:03.496030+0000","last_fullsized":"2024-09-11T08:40:03.496030+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:03.495950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.495950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.342754+0000","scrub_duration":0.001379314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:57.597021+0000","last_change":"2024-09-11T08:39:57.596980+0000","last_active":"2024-09-11T08:39:57.597021+0000","last_peered":"2024-09-11T08:39:57.597021+0000","last_clean":"2024-09-11T08:39:57.597021+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:39:57.597021+0000","last_undegraded":"2024-09-11T08:39:57.597021+0000","last_fullsized":"2024-09-11T08:39:57.597021+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.596889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.648355+0000","scrub_duration":0.001574424,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:08.387107+0000","last_change":"2024-09-11T08:40:08.387078+0000","last_active":"2024-09-11T08:40:08.387107+0000","last_peered":"2024-09-11T08:40:08.387107+0000","last_clean":"2024-09-11T08:40:08.387107+0000","last_became_active":"2024-09-11T08:38:45.182782+0000","last_became_peered":"2024-09-11T08:38:45.182782+0000","last_unstale":"2024-09-11T08:40:08.387107+0000","last_undegraded":"2024-09-11T08:40:08.387107+0000","last_fullsized":"2024-09-11T08:40:08.387107+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:08.387027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.387027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.887729+0000","scrub_duration":0.001411336,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":115,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:02.689472+0000","last_change":"2024-09-11T08:39:53.493004+0000","last_active":"2024-09-11T08:40:02.689472+0000","last_peered":"2024-09-11T08:40:02.689472+0000","last_clean":"2024-09-11T08:40:02.689472+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:40:02.689472+0000","last_undegraded":"2024-09-11T08:40:02.689472+0000","last_fullsized":"2024-09-11T08:40:02.689472+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.492945+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.774847+0000","scrub_duration":0.005988045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175554+0000","last_change":"2024-09-11T08:38:38.955616+0000","last_active":"2024-09-11T08:39:36.175554+0000","last_peered":"2024-09-11T08:39:36.175554+0000","last_clean":"2024-09-11T08:39:36.175554+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:39:36.175554+0000","last_undegraded":"2024-09-11T08:39:36.175554+0000","last_fullsized":"2024-09-11T08:39:36.175554+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:50.638474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:58.472902+0000","last_change":"2024-09-11T08:39:58.472851+0000","last_active":"2024-09-11T08:39:58.472902+0000","last_peered":"2024-09-11T08:39:58.472902+0000","last_clean":"2024-09-11T08:39:58.472902+0000","last_became_active":"2024-09-11T08:38:38.947270+0000","last_became_peered":"2024-09-11T08:38:38.947270+0000","last_unstale":"2024-09-11T08:39:58.472902+0000","last_undegraded":"2024-09-11T08:39:58.472902+0000","last_fullsized":"2024-09-11T08:39:58.472902+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:39:58.472785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.472785+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.725781+0000","scrub_duration":0.0020806930000000002,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501165+0000","last_change":"2024-09-11T08:38:42.975757+0000","last_active":"2024-09-11T08:39:35.501165+0000","last_peered":"2024-09-11T08:39:35.501165+0000","last_clean":"2024-09-11T08:39:35.501165+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:39:35.501165+0000","last_undegraded":"2024-09-11T08:39:35.501165+0000","last_fullsized":"2024-09-11T08:39:35.501165+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:50.821530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445649+0000","last_change":"2024-09-11T08:38:44.997146+0000","last_active":"2024-09-11T08:39:36.445649+0000","last_peered":"2024-09-11T08:39:36.445649+0000","last_clean":"2024-09-11T08:39:36.445649+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:39:36.445649+0000","last_undegraded":"2024-09-11T08:39:36.445649+0000","last_fullsized":"2024-09-11T08:39:36.445649+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.168511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":66,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.585203+0000","last_change":"2024-09-11T08:39:43.585175+0000","last_active":"2024-09-11T08:39:43.585203+0000","last_peered":"2024-09-11T08:39:43.585203+0000","last_clean":"2024-09-11T08:39:43.585203+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:39:43.585203+0000","last_undegraded":"2024-09-11T08:39:43.585203+0000","last_fullsized":"2024-09-11T08:39:43.585203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:39:43.585116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.585116+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.724395+0000","scrub_duration":0.0021210309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:00.500472+0000","last_change":"2024-09-11T08:40:00.500444+0000","last_active":"2024-09-11T08:40:00.500472+0000","last_peered":"2024-09-11T08:40:00.500472+0000","last_clean":"2024-09-11T08:40:00.500472+0000","last_became_active":"2024-09-11T08:38:38.949019+0000","last_became_peered":"2024-09-11T08:38:38.949019+0000","last_unstale":"2024-09-11T08:40:00.500472+0000","last_undegraded":"2024-09-11T08:40:00.500472+0000","last_fullsized":"2024-09-11T08:40:00.500472+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.500392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.500392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.354187+0000","scrub_duration":0.001294566,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":47,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:01.488572+0000","last_change":"2024-09-11T08:40:01.488539+0000","last_active":"2024-09-11T08:40:01.488572+0000","last_peered":"2024-09-11T08:40:01.488572+0000","last_clean":"2024-09-11T08:40:01.488572+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:40:01.488572+0000","last_undegraded":"2024-09-11T08:40:01.488572+0000","last_fullsized":"2024-09-11T08:40:01.488572+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:40:01.488482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:01.488482+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.917024+0000","scrub_duration":0.001463422,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:02.444386+0000","last_change":"2024-09-11T08:40:02.444341+0000","last_active":"2024-09-11T08:40:02.444386+0000","last_peered":"2024-09-11T08:40:02.444386+0000","last_clean":"2024-09-11T08:40:02.444386+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:40:02.444386+0000","last_undegraded":"2024-09-11T08:40:02.444386+0000","last_fullsized":"2024-09-11T08:40:02.444386+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.444279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.886414+0000","scrub_duration":0.001414329,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842581+0000","last_change":"2024-09-11T08:38:45.183842+0000","last_active":"2024-09-11T08:39:35.842581+0000","last_peered":"2024-09-11T08:39:35.842581+0000","last_clean":"2024-09-11T08:39:35.842581+0000","last_became_active":"2024-09-11T08:38:45.183582+0000","last_became_peered":"2024-09-11T08:38:45.183582+0000","last_unstale":"2024-09-11T08:39:35.842581+0000","last_undegraded":"2024-09-11T08:39:35.842581+0000","last_fullsized":"2024-09-11T08:39:35.842581+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:03.954910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500582+0000","last_change":"2024-09-11T08:38:40.975966+0000","last_active":"2024-09-11T08:39:35.500582+0000","last_peered":"2024-09-11T08:39:35.500582+0000","last_clean":"2024-09-11T08:39:35.500582+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:39:35.500582+0000","last_undegraded":"2024-09-11T08:39:35.500582+0000","last_fullsized":"2024-09-11T08:39:35.500582+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.626733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.356781+0000","last_change":"2024-09-11T08:39:43.356751+0000","last_active":"2024-09-11T08:39:43.356781+0000","last_peered":"2024-09-11T08:39:43.356781+0000","last_clean":"2024-09-11T08:39:43.356781+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:39:43.356781+0000","last_undegraded":"2024-09-11T08:39:43.356781+0000","last_fullsized":"2024-09-11T08:39:43.356781+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.356699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.356699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.569637+0000","scrub_duration":0.001649455,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:09.248781+0000","last_change":"2024-09-11T08:39:55.153362+0000","last_active":"2024-09-11T08:40:09.248781+0000","last_peered":"2024-09-11T08:40:09.248781+0000","last_clean":"2024-09-11T08:40:09.248781+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:40:09.248781+0000","last_undegraded":"2024-09-11T08:40:09.248781+0000","last_fullsized":"2024-09-11T08:40:09.248781+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:39:55.153290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.153290+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.446909+0000","scrub_duration":0.0017161990000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":85,"num_read_kb":80,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.502124+0000","last_change":"2024-09-11T08:38:42.976525+0000","last_active":"2024-09-11T08:39:35.502124+0000","last_peered":"2024-09-11T08:39:35.502124+0000","last_clean":"2024-09-11T08:39:35.502124+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:39:35.502124+0000","last_undegraded":"2024-09-11T08:39:35.502124+0000","last_fullsized":"2024-09-11T08:39:35.502124+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:09.438148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500105+0000","last_change":"2024-09-11T08:38:44.994290+0000","last_active":"2024-09-11T08:39:35.500105+0000","last_peered":"2024-09-11T08:39:35.500105+0000","last_clean":"2024-09-11T08:39:35.500105+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:39:35.500105+0000","last_undegraded":"2024-09-11T08:39:35.500105+0000","last_fullsized":"2024-09-11T08:39:35.500105+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.617197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":60,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:00.540221+0000","last_change":"2024-09-11T08:40:00.540192+0000","last_active":"2024-09-11T08:40:00.540221+0000","last_peered":"2024-09-11T08:40:00.540221+0000","last_clean":"2024-09-11T08:40:00.540221+0000","last_became_active":"2024-09-11T08:38:41.040948+0000","last_became_peered":"2024-09-11T08:38:41.040948+0000","last_unstale":"2024-09-11T08:40:00.540221+0000","last_undegraded":"2024-09-11T08:40:00.540221+0000","last_fullsized":"2024-09-11T08:40:00.540221+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:40:00.540136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.540136+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:07.651325+0000","scrub_duration":0.001785095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445564+0000","last_change":"2024-09-11T08:38:38.948242+0000","last_active":"2024-09-11T08:39:36.445564+0000","last_peered":"2024-09-11T08:39:36.445564+0000","last_clean":"2024-09-11T08:39:36.445564+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:39:36.445564+0000","last_undegraded":"2024-09-11T08:39:36.445564+0000","last_fullsized":"2024-09-11T08:39:36.445564+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:52.890961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":74,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842755+0000","last_change":"2024-09-11T08:39:12.765244+0000","last_active":"2024-09-11T08:39:35.842755+0000","last_peered":"2024-09-11T08:39:35.842755+0000","last_clean":"2024-09-11T08:39:35.842755+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:39:35.842755+0000","last_undegraded":"2024-09-11T08:39:35.842755+0000","last_fullsized":"2024-09-11T08:39:35.842755+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'72","last_scrub_stamp":"2024-09-11T08:39:12.765190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:39:12.765190+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:16.661831+0000","scrub_duration":0.0014709829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:49.116242+0000","last_change":"2024-09-11T08:39:49.116184+0000","last_active":"2024-09-11T08:39:49.116242+0000","last_peered":"2024-09-11T08:39:49.116242+0000","last_clean":"2024-09-11T08:39:49.116242+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:39:49.116242+0000","last_undegraded":"2024-09-11T08:39:49.116242+0000","last_fullsized":"2024-09-11T08:39:49.116242+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.116113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.116113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.209747+0000","scrub_duration":0.00167223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:47.423780+0000","last_change":"2024-09-11T08:39:47.423745+0000","last_active":"2024-09-11T08:39:47.423780+0000","last_peered":"2024-09-11T08:39:47.423780+0000","last_clean":"2024-09-11T08:39:47.423780+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:39:47.423780+0000","last_undegraded":"2024-09-11T08:39:47.423780+0000","last_fullsized":"2024-09-11T08:39:47.423780+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.423684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.423684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.004042+0000","scrub_duration":0.001458972,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175659+0000","last_change":"2024-09-11T08:38:40.973744+0000","last_active":"2024-09-11T08:39:36.175659+0000","last_peered":"2024-09-11T08:39:36.175659+0000","last_clean":"2024-09-11T08:39:36.175659+0000","last_became_active":"2024-09-11T08:38:40.973373+0000","last_became_peered":"2024-09-11T08:38:40.973373+0000","last_unstale":"2024-09-11T08:39:36.175659+0000","last_undegraded":"2024-09-11T08:39:36.175659+0000","last_fullsized":"2024-09-11T08:39:36.175659+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:07.389753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175244+0000","last_change":"2024-09-11T08:38:38.948244+0000","last_active":"2024-09-11T08:39:36.175244+0000","last_peered":"2024-09-11T08:39:36.175244+0000","last_clean":"2024-09-11T08:39:36.175244+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:39:36.175244+0000","last_undegraded":"2024-09-11T08:39:36.175244+0000","last_fullsized":"2024-09-11T08:39:36.175244+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:03.299757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:03.430923+0000","last_change":"2024-09-11T08:40:03.430883+0000","last_active":"2024-09-11T08:40:03.430923+0000","last_peered":"2024-09-11T08:40:03.430923+0000","last_clean":"2024-09-11T08:40:03.430923+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:40:03.430923+0000","last_undegraded":"2024-09-11T08:40:03.430923+0000","last_fullsized":"2024-09-11T08:40:03.430923+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:03.430794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.430794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:20.081896+0000","scrub_duration":0.0014294430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445494+0000","last_change":"2024-09-11T08:38:45.185064+0000","last_active":"2024-09-11T08:39:36.445494+0000","last_peered":"2024-09-11T08:39:36.445494+0000","last_clean":"2024-09-11T08:39:36.445494+0000","last_became_active":"2024-09-11T08:38:45.184410+0000","last_became_peered":"2024-09-11T08:38:45.184410+0000","last_unstale":"2024-09-11T08:39:36.445494+0000","last_undegraded":"2024-09-11T08:39:36.445494+0000","last_fullsized":"2024-09-11T08:39:36.445494+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:58.964836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":67,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.526881+0000","last_change":"2024-09-11T08:39:43.526845+0000","last_active":"2024-09-11T08:39:43.526881+0000","last_peered":"2024-09-11T08:39:43.526881+0000","last_clean":"2024-09-11T08:39:43.526881+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:39:43.526881+0000","last_undegraded":"2024-09-11T08:39:43.526881+0000","last_fullsized":"2024-09-11T08:39:43.526881+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:39:43.526763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.526763+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.869811+0000","scrub_duration":0.0022037329999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:55.599092+0000","last_change":"2024-09-11T08:39:55.599049+0000","last_active":"2024-09-11T08:39:55.599092+0000","last_peered":"2024-09-11T08:39:55.599092+0000","last_clean":"2024-09-11T08:39:55.599092+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:39:55.599092+0000","last_undegraded":"2024-09-11T08:39:55.599092+0000","last_fullsized":"2024-09-11T08:39:55.599092+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.598961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.598961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.562190+0000","scrub_duration":0.0016789820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:46.627109+0000","last_change":"2024-09-11T08:39:46.627077+0000","last_active":"2024-09-11T08:39:46.627109+0000","last_peered":"2024-09-11T08:39:46.627109+0000","last_clean":"2024-09-11T08:39:46.627109+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:39:46.627109+0000","last_undegraded":"2024-09-11T08:39:46.627109+0000","last_fullsized":"2024-09-11T08:39:46.627109+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:46.627028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:46.627028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:56.334041+0000","scrub_duration":0.0013824029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445462+0000","last_change":"2024-09-11T08:38:44.994162+0000","last_active":"2024-09-11T08:39:36.445462+0000","last_peered":"2024-09-11T08:39:36.445462+0000","last_clean":"2024-09-11T08:39:36.445462+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:39:36.445462+0000","last_undegraded":"2024-09-11T08:39:36.445462+0000","last_fullsized":"2024-09-11T08:39:36.445462+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:09.343122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175377+0000","last_change":"2024-09-11T08:38:40.969616+0000","last_active":"2024-09-11T08:39:36.175377+0000","last_peered":"2024-09-11T08:39:36.175377+0000","last_clean":"2024-09-11T08:39:36.175377+0000","last_became_active":"2024-09-11T08:38:40.969348+0000","last_became_peered":"2024-09-11T08:38:40.969348+0000","last_unstale":"2024-09-11T08:39:36.175377+0000","last_undegraded":"2024-09-11T08:39:36.175377+0000","last_fullsized":"2024-09-11T08:39:36.175377+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:52.841663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445860+0000","last_change":"2024-09-11T08:38:38.957214+0000","last_active":"2024-09-11T08:39:36.445860+0000","last_peered":"2024-09-11T08:39:36.445860+0000","last_clean":"2024-09-11T08:39:36.445860+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:39:36.445860+0000","last_undegraded":"2024-09-11T08:39:36.445860+0000","last_fullsized":"2024-09-11T08:39:36.445860+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:04.778506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":63,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:02.689469+0000","last_change":"2024-09-11T08:40:00.473606+0000","last_active":"2024-09-11T08:40:02.689469+0000","last_peered":"2024-09-11T08:40:02.689469+0000","last_clean":"2024-09-11T08:40:02.689469+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:40:02.689469+0000","last_undegraded":"2024-09-11T08:40:02.689469+0000","last_fullsized":"2024-09-11T08:40:02.689469+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:00.473549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.473549+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.300137+0000","scrub_duration":0.0016703060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175492+0000","last_change":"2024-09-11T08:38:44.993736+0000","last_active":"2024-09-11T08:39:36.175492+0000","last_peered":"2024-09-11T08:39:36.175492+0000","last_clean":"2024-09-11T08:39:36.175492+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:39:36.175492+0000","last_undegraded":"2024-09-11T08:39:36.175492+0000","last_fullsized":"2024-09-11T08:39:36.175492+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:55.062690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500820+0000","last_change":"2024-09-11T08:38:40.977113+0000","last_active":"2024-09-11T08:39:35.500820+0000","last_peered":"2024-09-11T08:39:35.500820+0000","last_clean":"2024-09-11T08:39:35.500820+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:39:35.500820+0000","last_undegraded":"2024-09-11T08:39:35.500820+0000","last_fullsized":"2024-09-11T08:39:35.500820+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:05.072088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:50.452607+0000","last_change":"2024-09-11T08:39:50.452568+0000","last_active":"2024-09-11T08:39:50.452607+0000","last_peered":"2024-09-11T08:39:50.452607+0000","last_clean":"2024-09-11T08:39:50.452607+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:39:50.452607+0000","last_undegraded":"2024-09-11T08:39:50.452607+0000","last_fullsized":"2024-09-11T08:39:50.452607+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.452494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.631278+0000","scrub_duration":0.001547086,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:09.519097+0000","last_change":"2024-09-11T08:40:09.519072+0000","last_active":"2024-09-11T08:40:09.519097+0000","last_peered":"2024-09-11T08:40:09.519097+0000","last_clean":"2024-09-11T08:40:09.519097+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:40:09.519097+0000","last_undegraded":"2024-09-11T08:40:09.519097+0000","last_fullsized":"2024-09-11T08:40:09.519097+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:09.519019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:09.519019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.000837+0000","scrub_duration":0.001232471,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500230+0000","last_change":"2024-09-11T08:38:44.991908+0000","last_active":"2024-09-11T08:39:35.500230+0000","last_peered":"2024-09-11T08:39:35.500230+0000","last_clean":"2024-09-11T08:39:35.500230+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:39:35.500230+0000","last_undegraded":"2024-09-11T08:39:35.500230+0000","last_fullsized":"2024-09-11T08:39:35.500230+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:12.056728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":69,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:45.575270+0000","last_change":"2024-09-11T08:39:45.575209+0000","last_active":"2024-09-11T08:39:45.575270+0000","last_peered":"2024-09-11T08:39:45.575270+0000","last_clean":"2024-09-11T08:39:45.575270+0000","last_became_active":"2024-09-11T08:38:41.040207+0000","last_became_peered":"2024-09-11T08:38:41.040207+0000","last_unstale":"2024-09-11T08:39:45.575270+0000","last_undegraded":"2024-09-11T08:39:45.575270+0000","last_fullsized":"2024-09-11T08:39:45.575270+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:39:45.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.575133+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.182090+0000","scrub_duration":0.0022792160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:56.112155+0000","last_change":"2024-09-11T08:39:56.112112+0000","last_active":"2024-09-11T08:39:56.112155+0000","last_peered":"2024-09-11T08:39:56.112155+0000","last_clean":"2024-09-11T08:39:56.112155+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:39:56.112155+0000","last_undegraded":"2024-09-11T08:39:56.112155+0000","last_fullsized":"2024-09-11T08:39:56.112155+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:56.112043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.112043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:25.809354+0000","scrub_duration":0.001517137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445023+0000","last_change":"2024-09-11T08:38:42.985952+0000","last_active":"2024-09-11T08:39:36.445023+0000","last_peered":"2024-09-11T08:39:36.445023+0000","last_clean":"2024-09-11T08:39:36.445023+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:39:36.445023+0000","last_undegraded":"2024-09-11T08:39:36.445023+0000","last_fullsized":"2024-09-11T08:39:36.445023+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:06.325377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842722+0000","last_change":"2024-09-11T08:38:44.993458+0000","last_active":"2024-09-11T08:39:35.842722+0000","last_peered":"2024-09-11T08:39:35.842722+0000","last_clean":"2024-09-11T08:39:35.842722+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:39:35.842722+0000","last_undegraded":"2024-09-11T08:39:35.842722+0000","last_fullsized":"2024-09-11T08:39:35.842722+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:04.342365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:40.524639+0000","last_change":"2024-09-11T08:39:40.524581+0000","last_active":"2024-09-11T08:39:40.524639+0000","last_peered":"2024-09-11T08:39:40.524639+0000","last_clean":"2024-09-11T08:39:40.524639+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:39:40.524639+0000","last_undegraded":"2024-09-11T08:39:40.524639+0000","last_fullsized":"2024-09-11T08:39:40.524639+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:40.524498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:40.524498+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.186922+0000","scrub_duration":0.0024035010000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:00.113613+0000","last_change":"2024-09-11T08:40:00.113587+0000","last_active":"2024-09-11T08:40:00.113613+0000","last_peered":"2024-09-11T08:40:00.113613+0000","last_clean":"2024-09-11T08:40:00.113613+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:40:00.113613+0000","last_undegraded":"2024-09-11T08:40:00.113613+0000","last_fullsized":"2024-09-11T08:40:00.113613+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.113534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.113534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.476975+0000","scrub_duration":0.0011790780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:57.454285+0000","last_change":"2024-09-11T08:39:57.454257+0000","last_active":"2024-09-11T08:39:57.454285+0000","last_peered":"2024-09-11T08:39:57.454285+0000","last_clean":"2024-09-11T08:39:57.454285+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:39:57.454285+0000","last_undegraded":"2024-09-11T08:39:57.454285+0000","last_fullsized":"2024-09-11T08:39:57.454285+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.454198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.454198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.731898+0000","scrub_duration":0.0011666669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445542+0000","last_change":"2024-09-11T08:38:44.998686+0000","last_active":"2024-09-11T08:39:36.445542+0000","last_peered":"2024-09-11T08:39:36.445542+0000","last_clean":"2024-09-11T08:39:36.445542+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:39:36.445542+0000","last_undegraded":"2024-09-11T08:39:36.445542+0000","last_fullsized":"2024-09-11T08:39:36.445542+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:02.320682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:48.462270+0000","last_change":"2024-09-11T08:39:48.462233+0000","last_active":"2024-09-11T08:39:48.462270+0000","last_peered":"2024-09-11T08:39:48.462270+0000","last_clean":"2024-09-11T08:39:48.462270+0000","last_became_active":"2024-09-11T08:38:41.040475+0000","last_became_peered":"2024-09-11T08:38:41.040475+0000","last_unstale":"2024-09-11T08:39:48.462270+0000","last_undegraded":"2024-09-11T08:39:48.462270+0000","last_fullsized":"2024-09-11T08:39:48.462270+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-11T08:39:48.462163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:48.462163+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.032207+0000","scrub_duration":0.002094753,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":57,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:03.143842+0000","last_change":"2024-09-11T08:40:03.143806+0000","last_active":"2024-09-11T08:40:03.143842+0000","last_peered":"2024-09-11T08:40:03.143842+0000","last_clean":"2024-09-11T08:40:03.143842+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:40:03.143842+0000","last_undegraded":"2024-09-11T08:40:03.143842+0000","last_fullsized":"2024-09-11T08:40:03.143842+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:40:03.143742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.143742+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.570435+0000","scrub_duration":0.001564396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"63'8","reported_seq":52,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:02.689770+0000","last_change":"2024-09-11T08:38:42.987044+0000","last_active":"2024-09-11T08:40:02.689770+0000","last_peered":"2024-09-11T08:40:02.689770+0000","last_clean":"2024-09-11T08:40:02.689770+0000","last_became_active":"2024-09-11T08:38:42.986756+0000","last_became_peered":"2024-09-11T08:38:42.986756+0000","last_unstale":"2024-09-11T08:40:02.689770+0000","last_undegraded":"2024-09-11T08:40:02.689770+0000","last_fullsized":"2024-09-11T08:40:02.689770+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501304+0000","last_change":"2024-09-11T08:38:45.182987+0000","last_active":"2024-09-11T08:39:35.501304+0000","last_peered":"2024-09-11T08:39:35.501304+0000","last_clean":"2024-09-11T08:39:35.501304+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:39:35.501304+0000","last_undegraded":"2024-09-11T08:39:35.501304+0000","last_fullsized":"2024-09-11T08:39:35.501304+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:58.541554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:42.605410+0000","last_change":"2024-09-11T08:39:42.605373+0000","last_active":"2024-09-11T08:39:42.605410+0000","last_peered":"2024-09-11T08:39:42.605410+0000","last_clean":"2024-09-11T08:39:42.605410+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:39:42.605410+0000","last_undegraded":"2024-09-11T08:39:42.605410+0000","last_fullsized":"2024-09-11T08:39:42.605410+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:42.605301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:42.605301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:53.937210+0000","scrub_duration":0.0014798509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445861+0000","last_change":"2024-09-11T08:38:41.039820+0000","last_active":"2024-09-11T08:39:36.445861+0000","last_peered":"2024-09-11T08:39:36.445861+0000","last_clean":"2024-09-11T08:39:36.445861+0000","last_became_active":"2024-09-11T08:38:41.039552+0000","last_became_peered":"2024-09-11T08:38:41.039552+0000","last_unstale":"2024-09-11T08:39:36.445861+0000","last_undegraded":"2024-09-11T08:39:36.445861+0000","last_fullsized":"2024-09-11T08:39:36.445861+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:51.272516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:49.472804+0000","last_change":"2024-09-11T08:39:49.472764+0000","last_active":"2024-09-11T08:39:49.472804+0000","last_peered":"2024-09-11T08:39:49.472804+0000","last_clean":"2024-09-11T08:39:49.472804+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:39:49.472804+0000","last_undegraded":"2024-09-11T08:39:49.472804+0000","last_fullsized":"2024-09-11T08:39:49.472804+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.472696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.472696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:01.403308+0000","scrub_duration":0.001919104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842528+0000","last_change":"2024-09-11T08:38:45.185016+0000","last_active":"2024-09-11T08:39:35.842528+0000","last_peered":"2024-09-11T08:39:35.842528+0000","last_clean":"2024-09-11T08:39:35.842528+0000","last_became_active":"2024-09-11T08:38:45.184753+0000","last_became_peered":"2024-09-11T08:38:45.184753+0000","last_unstale":"2024-09-11T08:39:35.842528+0000","last_undegraded":"2024-09-11T08:39:35.842528+0000","last_fullsized":"2024-09-11T08:39:35.842528+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:13.726221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500656+0000","last_change":"2024-09-11T08:38:41.041476+0000","last_active":"2024-09-11T08:39:35.500656+0000","last_peered":"2024-09-11T08:39:35.500656+0000","last_clean":"2024-09-11T08:39:35.500656+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:39:35.500656+0000","last_undegraded":"2024-09-11T08:39:35.500656+0000","last_fullsized":"2024-09-11T08:39:35.500656+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:04.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:53.176120+0000","last_change":"2024-09-11T08:39:53.176095+0000","last_active":"2024-09-11T08:39:53.176120+0000","last_peered":"2024-09-11T08:39:53.176120+0000","last_clean":"2024-09-11T08:39:53.176120+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:39:53.176120+0000","last_undegraded":"2024-09-11T08:39:53.176120+0000","last_fullsized":"2024-09-11T08:39:53.176120+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.176035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.176035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:00.682857+0000","scrub_duration":0.001147531,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445708+0000","last_change":"2024-09-11T08:38:42.982083+0000","last_active":"2024-09-11T08:39:36.445708+0000","last_peered":"2024-09-11T08:39:36.445708+0000","last_clean":"2024-09-11T08:39:36.445708+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:39:36.445708+0000","last_undegraded":"2024-09-11T08:39:36.445708+0000","last_fullsized":"2024-09-11T08:39:36.445708+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:58.890541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.493090+0000","last_change":"2024-09-11T08:39:52.493064+0000","last_active":"2024-09-11T08:39:52.493090+0000","last_peered":"2024-09-11T08:39:52.493090+0000","last_clean":"2024-09-11T08:39:52.493090+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:39:52.493090+0000","last_undegraded":"2024-09-11T08:39:52.493090+0000","last_fullsized":"2024-09-11T08:39:52.493090+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.493013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:13.717323+0000","scrub_duration":0.0012699160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.501853+0000","last_change":"2024-09-11T08:38:41.042115+0000","last_active":"2024-09-11T08:39:35.501853+0000","last_peered":"2024-09-11T08:39:35.501853+0000","last_clean":"2024-09-11T08:39:35.501853+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:39:35.501853+0000","last_undegraded":"2024-09-11T08:39:35.501853+0000","last_fullsized":"2024-09-11T08:39:35.501853+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.772561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.500193+0000","last_change":"2024-09-11T08:38:38.958515+0000","last_active":"2024-09-11T08:39:35.500193+0000","last_peered":"2024-09-11T08:39:35.500193+0000","last_clean":"2024-09-11T08:39:35.500193+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:39:35.500193+0000","last_undegraded":"2024-09-11T08:39:35.500193+0000","last_fullsized":"2024-09-11T08:39:35.500193+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:53.785774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":52,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:02.689506+0000","last_change":"2024-09-11T08:38:42.981915+0000","last_active":"2024-09-11T08:40:02.689506+0000","last_peered":"2024-09-11T08:40:02.689506+0000","last_clean":"2024-09-11T08:40:02.689506+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:40:02.689506+0000","last_undegraded":"2024-09-11T08:40:02.689506+0000","last_fullsized":"2024-09-11T08:40:02.689506+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175675+0000","last_change":"2024-09-11T08:38:44.993997+0000","last_active":"2024-09-11T08:39:36.175675+0000","last_peered":"2024-09-11T08:39:36.175675+0000","last_clean":"2024-09-11T08:39:36.175675+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:39:36.175675+0000","last_undegraded":"2024-09-11T08:39:36.175675+0000","last_fullsized":"2024-09-11T08:39:36.175675+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:59.445794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":43,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445977+0000","last_change":"2024-09-11T08:38:41.040424+0000","last_active":"2024-09-11T08:39:36.445977+0000","last_peered":"2024-09-11T08:39:36.445977+0000","last_clean":"2024-09-11T08:39:36.445977+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:39:36.445977+0000","last_undegraded":"2024-09-11T08:39:36.445977+0000","last_fullsized":"2024-09-11T08:39:36.445977+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:53.450115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842811+0000","last_change":"2024-09-11T08:38:38.949874+0000","last_active":"2024-09-11T08:39:35.842811+0000","last_peered":"2024-09-11T08:39:35.842811+0000","last_clean":"2024-09-11T08:39:35.842811+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:39:35.842811+0000","last_undegraded":"2024-09-11T08:39:35.842811+0000","last_fullsized":"2024-09-11T08:39:35.842811+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:02.574659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":63,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:02.689775+0000","last_change":"2024-09-11T08:39:44.429546+0000","last_active":"2024-09-11T08:40:02.689775+0000","last_peered":"2024-09-11T08:40:02.689775+0000","last_clean":"2024-09-11T08:40:02.689775+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:40:02.689775+0000","last_undegraded":"2024-09-11T08:40:02.689775+0000","last_fullsized":"2024-09-11T08:40:02.689775+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:39:44.429477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:44.429477+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.023544+0000","scrub_duration":0.001890784,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.254325+0000","last_change":"2024-09-11T08:39:52.254299+0000","last_active":"2024-09-11T08:39:52.254325+0000","last_peered":"2024-09-11T08:39:52.254325+0000","last_clean":"2024-09-11T08:39:52.254325+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:39:52.254325+0000","last_undegraded":"2024-09-11T08:39:52.254325+0000","last_fullsized":"2024-09-11T08:39:52.254325+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.254245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.254245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:20.782062+0000","scrub_duration":0.001581877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175203+0000","last_change":"2024-09-11T08:38:40.973815+0000","last_active":"2024-09-11T08:39:36.175203+0000","last_peered":"2024-09-11T08:39:36.175203+0000","last_clean":"2024-09-11T08:39:36.175203+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:39:36.175203+0000","last_undegraded":"2024-09-11T08:39:36.175203+0000","last_fullsized":"2024-09-11T08:39:36.175203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:00.866315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.175392+0000","last_change":"2024-09-11T08:38:38.952604+0000","last_active":"2024-09-11T08:39:36.175392+0000","last_peered":"2024-09-11T08:39:36.175392+0000","last_clean":"2024-09-11T08:39:36.175392+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:39:36.175392+0000","last_undegraded":"2024-09-11T08:39:36.175392+0000","last_fullsized":"2024-09-11T08:39:36.175392+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:51.807236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499809+0000","last_change":"2024-09-11T08:38:42.979138+0000","last_active":"2024-09-11T08:39:35.499809+0000","last_peered":"2024-09-11T08:39:35.499809+0000","last_clean":"2024-09-11T08:39:35.499809+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:39:35.499809+0000","last_undegraded":"2024-09-11T08:39:35.499809+0000","last_fullsized":"2024-09-11T08:39:35.499809+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:08.203025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.444988+0000","last_change":"2024-09-11T08:38:44.994241+0000","last_active":"2024-09-11T08:39:36.444988+0000","last_peered":"2024-09-11T08:39:36.444988+0000","last_clean":"2024-09-11T08:39:36.444988+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:39:36.444988+0000","last_undegraded":"2024-09-11T08:39:36.444988+0000","last_fullsized":"2024-09-11T08:39:36.444988+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:02.265888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499942+0000","last_change":"2024-09-11T08:38:40.976003+0000","last_active":"2024-09-11T08:39:35.499942+0000","last_peered":"2024-09-11T08:39:35.499942+0000","last_clean":"2024-09-11T08:39:35.499942+0000","last_became_active":"2024-09-11T08:38:40.975709+0000","last_became_peered":"2024-09-11T08:38:40.975709+0000","last_unstale":"2024-09-11T08:39:35.499942+0000","last_undegraded":"2024-09-11T08:39:35.499942+0000","last_fullsized":"2024-09-11T08:39:35.499942+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:01.631901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.842846+0000","last_change":"2024-09-11T08:38:38.954570+0000","last_active":"2024-09-11T08:39:35.842846+0000","last_peered":"2024-09-11T08:39:35.842846+0000","last_clean":"2024-09-11T08:39:35.842846+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:39:35.842846+0000","last_undegraded":"2024-09-11T08:39:35.842846+0000","last_fullsized":"2024-09-11T08:39:35.842846+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:05.920652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445629+0000","last_change":"2024-09-11T08:38:42.984381+0000","last_active":"2024-09-11T08:39:36.445629+0000","last_peered":"2024-09-11T08:39:36.445629+0000","last_clean":"2024-09-11T08:39:36.445629+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:39:36.445629+0000","last_undegraded":"2024-09-11T08:39:36.445629+0000","last_fullsized":"2024-09-11T08:39:36.445629+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:54.394345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:35.499567+0000","last_change":"2024-09-11T08:38:44.999759+0000","last_active":"2024-09-11T08:39:35.499567+0000","last_peered":"2024-09-11T08:39:35.499567+0000","last_clean":"2024-09-11T08:39:35.499567+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:39:35.499567+0000","last_undegraded":"2024-09-11T08:39:35.499567+0000","last_fullsized":"2024-09-11T08:39:35.499567+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:03.249276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445088+0000","last_change":"2024-09-11T08:38:41.039878+0000","last_active":"2024-09-11T08:39:36.445088+0000","last_peered":"2024-09-11T08:39:36.445088+0000","last_clean":"2024-09-11T08:39:36.445088+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:39:36.445088+0000","last_undegraded":"2024-09-11T08:39:36.445088+0000","last_fullsized":"2024-09-11T08:39:36.445088+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:08.136760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:05.430798+0000","last_change":"2024-09-11T08:40:05.430740+0000","last_active":"2024-09-11T08:40:05.430798+0000","last_peered":"2024-09-11T08:40:05.430798+0000","last_clean":"2024-09-11T08:40:05.430798+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:40:05.430798+0000","last_undegraded":"2024-09-11T08:40:05.430798+0000","last_fullsized":"2024-09-11T08:40:05.430798+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:05.430666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:05.430666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.818673+0000","scrub_duration":0.001547629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:47.104746+0000","last_change":"2024-09-11T08:39:47.104702+0000","last_active":"2024-09-11T08:39:47.104746+0000","last_peered":"2024-09-11T08:39:47.104746+0000","last_clean":"2024-09-11T08:39:47.104746+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:39:47.104746+0000","last_undegraded":"2024-09-11T08:39:47.104746+0000","last_fullsized":"2024-09-11T08:39:47.104746+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.104637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.104637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:51.311803+0000","scrub_duration":0.0015238020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-11T08:39:36.445776+0000","last_change":"2024-09-11T08:38:45.183913+0000","last_active":"2024-09-11T08:39:36.445776+0000","last_peered":"2024-09-11T08:39:36.445776+0000","last_clean":"2024-09-11T08:39:36.445776+0000","last_became_active":"2024-09-11T08:38:45.183548+0000","last_became_peered":"2024-09-11T08:38:45.183548+0000","last_unstale":"2024-09-11T08:39:36.445776+0000","last_undegraded":"2024-09-11T08:39:36.445776+0000","last_fullsized":"2024-09-11T08:39:36.445776+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:07.678681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":44,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:41.989766+0000","last_change":"2024-09-11T08:39:41.989740+0000","last_active":"2024-09-11T08:39:41.989766+0000","last_peered":"2024-09-11T08:39:41.989766+0000","last_clean":"2024-09-11T08:39:41.989766+0000","last_became_active":"2024-09-11T08:38:40.973799+0000","last_became_peered":"2024-09-11T08:38:40.973799+0000","last_unstale":"2024-09-11T08:39:41.989766+0000","last_undegraded":"2024-09-11T08:39:41.989766+0000","last_fullsized":"2024-09-11T08:39:41.989766+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-11T08:39:41.989684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.989684+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.687182+0000","scrub_duration":0.00162959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:08.336399+0000","last_change":"2024-09-11T08:40:08.336369+0000","last_active":"2024-09-11T08:40:08.336399+0000","last_peered":"2024-09-11T08:40:08.336399+0000","last_clean":"2024-09-11T08:40:08.336399+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:40:08.336399+0000","last_undegraded":"2024-09-11T08:40:08.336399+0000","last_fullsized":"2024-09-11T08:40:08.336399+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:40:08.336314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.336314+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:16.115575+0000","scrub_duration":0.001394085,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:53.541675+0000","last_change":"2024-09-11T08:39:53.541628+0000","last_active":"2024-09-11T08:39:53.541675+0000","last_peered":"2024-09-11T08:39:53.541675+0000","last_clean":"2024-09-11T08:39:53.541675+0000","last_became_active":"2024-09-11T08:38:42.989556+0000","last_became_peered":"2024-09-11T08:38:42.989556+0000","last_unstale":"2024-09-11T08:39:53.541675+0000","last_undegraded":"2024-09-11T08:39:53.541675+0000","last_fullsized":"2024-09-11T08:39:53.541675+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.541557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.541557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:08.967817+0000","scrub_duration":0.001310285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:07.114916+0000","last_change":"2024-09-11T08:40:07.114884+0000","last_active":"2024-09-11T08:40:07.114916+0000","last_peered":"2024-09-11T08:40:07.114916+0000","last_clean":"2024-09-11T08:40:07.114916+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:40:07.114916+0000","last_undegraded":"2024-09-11T08:40:07.114916+0000","last_fullsized":"2024-09-11T08:40:07.114916+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:07.114815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:07.114815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.897957+0000","scrub_duration":0.0013781500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.480694+0000","last_change":"2024-09-11T08:39:52.480663+0000","last_active":"2024-09-11T08:39:52.480694+0000","last_peered":"2024-09-11T08:39:52.480694+0000","last_clean":"2024-09-11T08:39:52.480694+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:39:52.480694+0000","last_undegraded":"2024-09-11T08:39:52.480694+0000","last_fullsized":"2024-09-11T08:39:52.480694+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.480610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.480610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:55.608483+0000","scrub_duration":0.001335754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:53.038381+0000","last_change":"2024-09-11T08:39:53.038356+0000","last_active":"2024-09-11T08:39:53.038381+0000","last_peered":"2024-09-11T08:39:53.038381+0000","last_clean":"2024-09-11T08:39:53.038381+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:39:53.038381+0000","last_undegraded":"2024-09-11T08:39:53.038381+0000","last_fullsized":"2024-09-11T08:39:53.038381+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.038297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.038297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:58.424182+0000","scrub_duration":0.0012198549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":57,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:02.551089+0000","last_change":"2024-09-11T08:40:02.551056+0000","last_active":"2024-09-11T08:40:02.551089+0000","last_peered":"2024-09-11T08:40:02.551089+0000","last_clean":"2024-09-11T08:40:02.551089+0000","last_became_active":"2024-09-11T08:38:40.974451+0000","last_became_peered":"2024-09-11T08:38:40.974451+0000","last_unstale":"2024-09-11T08:40:02.551089+0000","last_undegraded":"2024-09-11T08:40:02.551089+0000","last_fullsized":"2024-09-11T08:40:02.551089+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:40:02.550999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.550999+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.173446+0000","scrub_duration":0.0016484399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:55.265182+0000","last_change":"2024-09-11T08:39:55.265155+0000","last_active":"2024-09-11T08:39:55.265182+0000","last_peered":"2024-09-11T08:39:55.265182+0000","last_clean":"2024-09-11T08:39:55.265182+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:39:55.265182+0000","last_undegraded":"2024-09-11T08:39:55.265182+0000","last_fullsized":"2024-09-11T08:39:55.265182+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.265097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.265097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.333633+0000","scrub_duration":0.0012893830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":423,"ondisk_log_size":423,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":80,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299416,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298788,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446364,"statfs":{"total":95995035648,"available":95689076736,"internally_reserved":0,"allocated":1470464,"data_stored":983979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.61399999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.55100000000000005}]},{"osd":2,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.623}]},{"osd":3,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60399999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.39400000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.41299999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.435},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.42099999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528349,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298768,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446384,"statfs":{"total":95995035648,"available":95689097216,"internally_reserved":0,"allocated":1449984,"data_stored":983206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.71699999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.42099999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.59799999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.44},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.70799999999999996,"5min":0.871,"15min":0.871},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70999999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.433},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.38}]},{"osd":3,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.66200000000000003,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66800000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.45000000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.27500000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40600000000000003},"min":{"1min":0.29199999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.29599999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.439,"15min":0.43099999999999999},"min":{"1min":0.34200000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.435,"15min":0.42999999999999999},"min":{"1min":0.30299999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.33900000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:39:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.313},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.36599999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757281,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298204,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446948,"statfs":{"total":95995035648,"available":95689674752,"internally_reserved":0,"allocated":872448,"data_stored":394134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.53900000000000003}]},{"osd":1,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.432}]},{"osd":2,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.35199999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.47799999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.376},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.499},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.59799999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.42599999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73099999999999998,"5min":0.749,"15min":0.749},"last":0.57899999999999996}]},{"osd":7,"last update":"Wed Sep 11 08:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.56299999999999994}]}]},{"osd":4,"up_from":31,"seq":133143986213,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298204,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446948,"statfs":{"total":95995035648,"available":95689674752,"internally_reserved":0,"allocated":872448,"data_stored":392829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.56299999999999994,"15min":0.57299999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68899999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.48099999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55800000000000005,"15min":0.56399999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":0.754,"15min":0.754},"last":0.46500000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70799999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.71099999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.45400000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67100000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75800000000000001,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.442}]},{"osd":5,"last update":"Wed Sep 11 08:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.40500000000000003,"15min":0.38700000000000001},"min":{"1min":0.32100000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.30099999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.34699999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:40:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.32400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.36199999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67100000000000004,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.318}]},{"osd":7,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.28799999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149739,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":901120,"data_stored":394654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.36299999999999999,"15min":0.374},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.443,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.38900000000000001,"15min":0.40000000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.496,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.315}]},{"osd":1,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.39200000000000002,"15min":0.40400000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.56200000000000006,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.40400000000000003,"15min":0.41399999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48199999999999998,"5min":0.621,"15min":0.621},"last":0.32900000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.42699999999999999,"15min":0.435},"min":{"1min":0.27100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.44,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.34100000000000003}]},{"osd":4,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40699999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.498},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.752,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.43099999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.441},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.54100000000000004}]},{"osd":6,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76300000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41899999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:39:27 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.51200000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298180,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446972,"statfs":{"total":95995035648,"available":95689699328,"internally_reserved":0,"allocated":847872,"data_stored":392013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36499999999999999,"15min":0.35399999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.35899999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.25800000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.39100000000000001,"15min":0.376},"min":{"1min":0.28999999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.28999999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:39:57 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.311},{"interface":"front","average":{"1min":0.441,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.33400000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.441,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.503}]},{"osd":5,"last update":"Wed Sep 11 08:39:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.54000000000000004}]},{"osd":6,"last update":"Wed Sep 11 08:40:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.442,"5min":0.432,"15min":0.432},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.56100000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.56299999999999994}]},{"osd":7,"last update":"Wed Sep 11 08:39:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.495},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.74399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298240,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446912,"statfs":{"total":95995035648,"available":95689637888,"internally_reserved":0,"allocated":909312,"data_stored":393938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.32700000000000001,"15min":0.313},"min":{"1min":0.24199999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.49199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.34499999999999997,"15min":0.33000000000000002},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.47599999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.45800000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:39:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.40400000000000003}]},{"osd":3,"last update":"Wed Sep 11 08:39:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40400000000000003,"15min":0.41999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54400000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.433,"15min":0.434},"min":{"1min":0.307,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.39000000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79900000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.501}]},{"osd":5,"last update":"Wed Sep 11 08:39:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.755,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82199999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57899999999999996}]},{"osd":6,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57399999999999995,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60099999999999998,"15min":0.60599999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.61299999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298792,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446360,"statfs":{"total":95995035648,"available":95689072640,"internally_reserved":0,"allocated":1474560,"data_stored":984038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:40:10 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.311},"min":{"1min":0.19800000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.436,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34000000000000002,"15min":0.314},"min":{"1min":0.23999999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.504,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.35699999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.35799999999999998,"15min":0.34699999999999998},"min":{"1min":0.24199999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53500000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.34899999999999998},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.51500000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.29999999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:39:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39800000000000002,"15min":0.40000000000000002},"min":{"1min":0.28499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.503,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.374}]},{"osd":4,"last update":"Wed Sep 11 08:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69699999999999995,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.53000000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56999999999999995}]},{"osd":6,"last update":"Wed Sep 11 08:40:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.37,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68300000000000005,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.42499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.746,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54600000000000004}]},{"osd":7,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.496},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.51800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:40:16.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:40:17.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[29647]: 6.4 scrub starts 2024-09-11T08:40:17.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[29647]: 6.4 scrub ok 2024-09-11T08:40:17.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:17.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[29647]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:17.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[29647]: osdmap e80: 8 total, 8 up, 8 in 2024-09-11T08:40:17.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:40:17.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:40:17.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:17.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:17.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[41114]: 6.4 scrub starts 2024-09-11T08:40:17.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[41114]: 6.4 scrub ok 2024-09-11T08:40:17.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:17.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[41114]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:17.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[41114]: osdmap e80: 8 total, 8 up, 8 in 2024-09-11T08:40:17.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:40:17.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:40:17.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:17.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:17 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:17 smithi145 ceph-mon[33384]: 6.4 scrub starts 2024-09-11T08:40:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:17 smithi145 ceph-mon[33384]: 6.4 scrub ok 2024-09-11T08:40:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:17 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:17 smithi145 ceph-mon[33384]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:17 smithi145 ceph-mon[33384]: osdmap e80: 8 total, 8 up, 8 in 2024-09-11T08:40:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:17 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:40:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:17 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:40:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:17 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:17 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:17.701 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:18.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:18 smithi092 ceph-mon[29647]: 4.f scrub starts 2024-09-11T08:40:18.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:18 smithi092 ceph-mon[29647]: 4.f scrub ok 2024-09-11T08:40:18.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:18 smithi092 ceph-mon[29647]: pgmap v45: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:18.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:18 smithi092 ceph-mon[29647]: 4.17 scrub starts 2024-09-11T08:40:18.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:18 smithi092 ceph-mon[29647]: 4.17 scrub ok 2024-09-11T08:40:18.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:18 smithi092 ceph-mon[29647]: osdmap e81: 8 total, 8 up, 8 in 2024-09-11T08:40:18.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:18 smithi092 ceph-mon[41114]: 4.f scrub starts 2024-09-11T08:40:18.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:18 smithi092 ceph-mon[41114]: 4.f scrub ok 2024-09-11T08:40:18.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:18 smithi092 ceph-mon[41114]: pgmap v45: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:18.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:18 smithi092 ceph-mon[41114]: 4.17 scrub starts 2024-09-11T08:40:18.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:18 smithi092 ceph-mon[41114]: 4.17 scrub ok 2024-09-11T08:40:18.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:18 smithi092 ceph-mon[41114]: osdmap e81: 8 total, 8 up, 8 in 2024-09-11T08:40:18.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:18 smithi145 ceph-mon[33384]: 4.f scrub starts 2024-09-11T08:40:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:18 smithi145 ceph-mon[33384]: 4.f scrub ok 2024-09-11T08:40:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:18 smithi145 ceph-mon[33384]: pgmap v45: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:18 smithi145 ceph-mon[33384]: 4.17 scrub starts 2024-09-11T08:40:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:18 smithi145 ceph-mon[33384]: 4.17 scrub ok 2024-09-11T08:40:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:18 smithi145 ceph-mon[33384]: osdmap e81: 8 total, 8 up, 8 in 2024-09-11T08:40:19.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:19 smithi092 ceph-mon[29647]: 5.5 scrub starts 2024-09-11T08:40:19.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:19 smithi092 ceph-mon[29647]: 5.5 scrub ok 2024-09-11T08:40:19.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:19 smithi092 ceph-mon[41114]: 5.5 scrub starts 2024-09-11T08:40:19.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:19 smithi092 ceph-mon[41114]: 5.5 scrub ok 2024-09-11T08:40:19.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:19 smithi145 ceph-mon[33384]: 5.5 scrub starts 2024-09-11T08:40:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:19 smithi145 ceph-mon[33384]: 5.5 scrub ok 2024-09-11T08:40:20.220 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:40:20.220 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[29647]: 4.1e scrub starts 2024-09-11T08:40:20.221 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[29647]: 4.1e scrub ok 2024-09-11T08:40:20.221 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[29647]: pgmap v47: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:20.221 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:20.221 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:20.221 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:40:20.221 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:40:20.221 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:20.221 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[29647]: 5.1 scrub starts 2024-09-11T08:40:20.221 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[29647]: 5.1 scrub ok 2024-09-11T08:40:20.222 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[41114]: 4.1e scrub starts 2024-09-11T08:40:20.222 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[41114]: 4.1e scrub ok 2024-09-11T08:40:20.222 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[41114]: pgmap v47: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:20.222 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:20.222 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:20.222 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:40:20.222 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:40:20.222 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:20.222 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[41114]: 5.1 scrub starts 2024-09-11T08:40:20.222 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:20 smithi092 ceph-mon[41114]: 5.1 scrub ok 2024-09-11T08:40:20.226 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:40:20.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:20 smithi145 ceph-mon[33384]: 4.1e scrub starts 2024-09-11T08:40:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:20 smithi145 ceph-mon[33384]: 4.1e scrub ok 2024-09-11T08:40:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:20 smithi145 ceph-mon[33384]: pgmap v47: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:40:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:20 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:20 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:20 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:40:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:20 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:40:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:20 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:20 smithi145 ceph-mon[33384]: 5.1 scrub starts 2024-09-11T08:40:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:20 smithi145 ceph-mon[33384]: 5.1 scrub ok 2024-09-11T08:40:21.539 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:21 smithi092 ceph-mon[29647]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:40:21.540 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:21 smithi092 ceph-mon[29647]: Reconfiguring daemon alertmanager.a on smithi092 2024-09-11T08:40:21.540 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:21 smithi092 ceph-mon[29647]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:21.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:21 smithi092 ceph-mon[41114]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:40:21.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:21 smithi092 ceph-mon[41114]: Reconfiguring daemon alertmanager.a on smithi092 2024-09-11T08:40:21.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:21 smithi092 ceph-mon[41114]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:21.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:21 smithi145 ceph-mon[33384]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:40:21.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:21 smithi145 ceph-mon[33384]: Reconfiguring daemon alertmanager.a on smithi092 2024-09-11T08:40:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:21 smithi145 ceph-mon[33384]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:22.364 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-09-11T08:40:19.014221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596078,"num_objects":192,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":926,"num_read_kb":789,"num_write":533,"num_write_kb":1670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":571,"ondisk_log_size":571,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":397,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2387504,"kb_used_data":7372,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747573712,"statfs":{"total":767960285184,"available":765515481088,"internally_reserved":0,"allocated":7548928,"data_stored":4520626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-110,"num_objects":-4,"num_object_clones":0,"num_object_copies":-12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":-12,"num_read_kb":-6,"num_write":-10,"num_write_kb":-1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000830"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:47.339416+0000","last_change":"2024-09-11T08:39:47.339366+0000","last_active":"2024-09-11T08:39:47.339416+0000","last_peered":"2024-09-11T08:39:47.339416+0000","last_clean":"2024-09-11T08:39:47.339416+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:39:47.339416+0000","last_undegraded":"2024-09-11T08:39:47.339416+0000","last_fullsized":"2024-09-11T08:39:47.339416+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.339312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.339312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.154202+0000","scrub_duration":0.001323095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":60,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:56.666836+0000","last_change":"2024-09-11T08:39:56.666764+0000","last_active":"2024-09-11T08:39:56.666836+0000","last_peered":"2024-09-11T08:39:56.666836+0000","last_clean":"2024-09-11T08:39:56.666836+0000","last_became_active":"2024-09-11T08:38:40.973398+0000","last_became_peered":"2024-09-11T08:38:40.973398+0000","last_unstale":"2024-09-11T08:39:56.666836+0000","last_undegraded":"2024-09-11T08:39:56.666836+0000","last_fullsized":"2024-09-11T08:39:56.666836+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.666703+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.544222+0000","scrub_duration":0.0041022000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:12.325164+0000","last_change":"2024-09-11T08:40:12.325099+0000","last_active":"2024-09-11T08:40:12.325164+0000","last_peered":"2024-09-11T08:40:12.325164+0000","last_clean":"2024-09-11T08:40:12.325164+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:40:12.325164+0000","last_undegraded":"2024-09-11T08:40:12.325164+0000","last_fullsized":"2024-09-11T08:40:12.325164+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:12.325037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:12.325037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:13.497524+0000","scrub_duration":0.001599736,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.678449+0000","last_change":"2024-09-11T08:38:44.992111+0000","last_active":"2024-09-11T08:39:38.678449+0000","last_peered":"2024-09-11T08:39:38.678449+0000","last_clean":"2024-09-11T08:39:38.678449+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:39:38.678449+0000","last_undegraded":"2024-09-11T08:39:38.678449+0000","last_fullsized":"2024-09-11T08:39:38.678449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:06.174014+0000","last_change":"2024-09-11T08:40:06.173950+0000","last_active":"2024-09-11T08:40:06.174014+0000","last_peered":"2024-09-11T08:40:06.174014+0000","last_clean":"2024-09-11T08:40:06.174014+0000","last_became_active":"2024-09-11T08:38:45.184215+0000","last_became_peered":"2024-09-11T08:38:45.184215+0000","last_unstale":"2024-09-11T08:40:06.174014+0000","last_undegraded":"2024-09-11T08:40:06.174014+0000","last_fullsized":"2024-09-11T08:40:06.174014+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:06.173882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:06.173882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:17.036842+0000","scrub_duration":0.001730875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:58.161134+0000","last_change":"2024-09-11T08:39:58.161055+0000","last_active":"2024-09-11T08:39:58.161134+0000","last_peered":"2024-09-11T08:39:58.161134+0000","last_clean":"2024-09-11T08:39:58.161134+0000","last_became_active":"2024-09-11T08:38:38.949136+0000","last_became_peered":"2024-09-11T08:38:38.949136+0000","last_unstale":"2024-09-11T08:39:58.161134+0000","last_undegraded":"2024-09-11T08:39:58.161134+0000","last_fullsized":"2024-09-11T08:39:58.161134+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:58.160984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.160984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:00.523696+0000","scrub_duration":0.0015109679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":71,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:45.997957+0000","last_change":"2024-09-11T08:39:45.997873+0000","last_active":"2024-09-11T08:39:45.997957+0000","last_peered":"2024-09-11T08:39:45.997957+0000","last_clean":"2024-09-11T08:39:45.997957+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:39:45.997957+0000","last_undegraded":"2024-09-11T08:39:45.997957+0000","last_fullsized":"2024-09-11T08:39:45.997957+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.997803+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.534414+0000","scrub_duration":0.0046870369999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.678125+0000","last_change":"2024-09-11T08:38:42.983653+0000","last_active":"2024-09-11T08:39:38.678125+0000","last_peered":"2024-09-11T08:39:38.678125+0000","last_clean":"2024-09-11T08:39:38.678125+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:39:38.678125+0000","last_undegraded":"2024-09-11T08:39:38.678125+0000","last_fullsized":"2024-09-11T08:39:38.678125+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:08.416611+0000","last_change":"2024-09-11T08:40:08.416577+0000","last_active":"2024-09-11T08:40:08.416611+0000","last_peered":"2024-09-11T08:40:08.416611+0000","last_clean":"2024-09-11T08:40:08.416611+0000","last_became_active":"2024-09-11T08:38:38.948930+0000","last_became_peered":"2024-09-11T08:38:38.948930+0000","last_unstale":"2024-09-11T08:40:08.416611+0000","last_undegraded":"2024-09-11T08:40:08.416611+0000","last_fullsized":"2024-09-11T08:40:08.416611+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:08.416522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.416522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:27.445091+0000","scrub_duration":0.001330005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":60,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:46.620776+0000","last_change":"2024-09-11T08:39:46.620700+0000","last_active":"2024-09-11T08:39:46.620776+0000","last_peered":"2024-09-11T08:39:46.620776+0000","last_clean":"2024-09-11T08:39:46.620776+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:39:46.620776+0000","last_undegraded":"2024-09-11T08:39:46.620776+0000","last_fullsized":"2024-09-11T08:39:46.620776+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:39:46.620624+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:10.984564+0000","scrub_duration":0.0044132319999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:04.450385+0000","last_change":"2024-09-11T08:40:04.450336+0000","last_active":"2024-09-11T08:40:04.450385+0000","last_peered":"2024-09-11T08:40:04.450385+0000","last_clean":"2024-09-11T08:40:04.450385+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:40:04.450385+0000","last_undegraded":"2024-09-11T08:40:04.450385+0000","last_fullsized":"2024-09-11T08:40:04.450385+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:04.450282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:04.450282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.499239+0000","scrub_duration":0.001689433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:51.280404+0000","last_change":"2024-09-11T08:39:51.280365+0000","last_active":"2024-09-11T08:39:51.280404+0000","last_peered":"2024-09-11T08:39:51.280404+0000","last_clean":"2024-09-11T08:39:51.280404+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:39:51.280404+0000","last_undegraded":"2024-09-11T08:39:51.280404+0000","last_fullsized":"2024-09-11T08:39:51.280404+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.280310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.280310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:06.712880+0000","scrub_duration":0.001634777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:12.372191+0000","last_change":"2024-09-11T08:40:12.372128+0000","last_active":"2024-09-11T08:40:12.372191+0000","last_peered":"2024-09-11T08:40:12.372191+0000","last_clean":"2024-09-11T08:40:12.372191+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:40:12.372191+0000","last_undegraded":"2024-09-11T08:40:12.372191+0000","last_fullsized":"2024-09-11T08:40:12.372191+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:12.372043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:12.372043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.497179+0000","scrub_duration":0.001440623,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.678887+0000","last_change":"2024-09-11T08:38:40.974291+0000","last_active":"2024-09-11T08:39:38.678887+0000","last_peered":"2024-09-11T08:39:38.678887+0000","last_clean":"2024-09-11T08:39:38.678887+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:39:38.678887+0000","last_undegraded":"2024-09-11T08:39:38.678887+0000","last_fullsized":"2024-09-11T08:39:38.678887+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:51.166163+0000","last_change":"2024-09-11T08:39:51.166114+0000","last_active":"2024-09-11T08:39:51.166163+0000","last_peered":"2024-09-11T08:39:51.166163+0000","last_clean":"2024-09-11T08:39:51.166163+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:39:51.166163+0000","last_undegraded":"2024-09-11T08:39:51.166163+0000","last_fullsized":"2024-09-11T08:39:51.166163+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.166051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.570790+0000","scrub_duration":0.0016440420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.812386+0000","last_change":"2024-09-11T08:38:44.994259+0000","last_active":"2024-09-11T08:39:38.812386+0000","last_peered":"2024-09-11T08:39:38.812386+0000","last_clean":"2024-09-11T08:39:38.812386+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:39:38.812386+0000","last_undegraded":"2024-09-11T08:39:38.812386+0000","last_fullsized":"2024-09-11T08:39:38.812386+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:59.606567+0000","last_change":"2024-09-11T08:39:59.606521+0000","last_active":"2024-09-11T08:39:59.606567+0000","last_peered":"2024-09-11T08:39:59.606567+0000","last_clean":"2024-09-11T08:39:59.606567+0000","last_became_active":"2024-09-11T08:38:45.183364+0000","last_became_peered":"2024-09-11T08:38:45.183364+0000","last_unstale":"2024-09-11T08:39:59.606567+0000","last_undegraded":"2024-09-11T08:39:59.606567+0000","last_fullsized":"2024-09-11T08:39:59.606567+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:59.606458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:59.606458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.547769+0000","scrub_duration":0.0012065960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.813042+0000","last_change":"2024-09-11T08:38:38.954075+0000","last_active":"2024-09-11T08:39:38.813042+0000","last_peered":"2024-09-11T08:39:38.813042+0000","last_clean":"2024-09-11T08:39:38.813042+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:39:38.813042+0000","last_undegraded":"2024-09-11T08:39:38.813042+0000","last_fullsized":"2024-09-11T08:39:38.813042+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:50.472482+0000","last_change":"2024-09-11T08:39:50.472419+0000","last_active":"2024-09-11T08:39:50.472482+0000","last_peered":"2024-09-11T08:39:50.472482+0000","last_clean":"2024-09-11T08:39:50.472482+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:39:50.472482+0000","last_undegraded":"2024-09-11T08:39:50.472482+0000","last_fullsized":"2024-09-11T08:39:50.472482+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.472368+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:10.784548+0000","scrub_duration":0.0049809520000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:44.632405+0000","last_change":"2024-09-11T08:39:44.632369+0000","last_active":"2024-09-11T08:39:44.632405+0000","last_peered":"2024-09-11T08:39:44.632405+0000","last_clean":"2024-09-11T08:39:44.632405+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:39:44.632405+0000","last_undegraded":"2024-09-11T08:39:44.632405+0000","last_fullsized":"2024-09-11T08:39:44.632405+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:44.632308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:44.632308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:56.408810+0000","scrub_duration":0.001420377,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679427+0000","last_change":"2024-09-11T08:38:45.183646+0000","last_active":"2024-09-11T08:39:38.679427+0000","last_peered":"2024-09-11T08:39:38.679427+0000","last_clean":"2024-09-11T08:39:38.679427+0000","last_became_active":"2024-09-11T08:38:45.182763+0000","last_became_peered":"2024-09-11T08:38:45.182763+0000","last_unstale":"2024-09-11T08:39:38.679427+0000","last_undegraded":"2024-09-11T08:39:38.679427+0000","last_fullsized":"2024-09-11T08:39:38.679427+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.678871+0000","last_change":"2024-09-11T08:38:38.949654+0000","last_active":"2024-09-11T08:39:38.678871+0000","last_peered":"2024-09-11T08:39:38.678871+0000","last_clean":"2024-09-11T08:39:38.678871+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:39:38.678871+0000","last_undegraded":"2024-09-11T08:39:38.678871+0000","last_fullsized":"2024-09-11T08:39:38.678871+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:55.125494+0000","last_change":"2024-09-11T08:39:55.125449+0000","last_active":"2024-09-11T08:39:55.125494+0000","last_peered":"2024-09-11T08:39:55.125494+0000","last_clean":"2024-09-11T08:39:55.125494+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:39:55.125494+0000","last_undegraded":"2024-09-11T08:39:55.125494+0000","last_fullsized":"2024-09-11T08:39:55.125494+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:55.125394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.125394+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.857707+0000","scrub_duration":0.0016833810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:01.585519+0000","last_change":"2024-09-11T08:40:01.585472+0000","last_active":"2024-09-11T08:40:01.585519+0000","last_peered":"2024-09-11T08:40:01.585519+0000","last_clean":"2024-09-11T08:40:01.585519+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:40:01.585519+0000","last_undegraded":"2024-09-11T08:40:01.585519+0000","last_fullsized":"2024-09-11T08:40:01.585519+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:01.585411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:01.585411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.937039+0000","scrub_duration":0.0021220179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:10.113771+0000","last_change":"2024-09-11T08:40:10.113729+0000","last_active":"2024-09-11T08:40:10.113771+0000","last_peered":"2024-09-11T08:40:10.113771+0000","last_clean":"2024-09-11T08:40:10.113771+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:40:10.113771+0000","last_undegraded":"2024-09-11T08:40:10.113771+0000","last_fullsized":"2024-09-11T08:40:10.113771+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:40:10.113671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:10.113671+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:33.267918+0000","scrub_duration":0.001621302,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:02.458653+0000","last_change":"2024-09-11T08:40:02.458618+0000","last_active":"2024-09-11T08:40:02.458653+0000","last_peered":"2024-09-11T08:40:02.458653+0000","last_clean":"2024-09-11T08:40:02.458653+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:40:02.458653+0000","last_undegraded":"2024-09-11T08:40:02.458653+0000","last_fullsized":"2024-09-11T08:40:02.458653+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:02.458565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.458565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.247079+0000","scrub_duration":0.001315568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":45,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.813030+0000","last_change":"2024-09-11T08:38:41.041065+0000","last_active":"2024-09-11T08:39:38.813030+0000","last_peered":"2024-09-11T08:39:38.813030+0000","last_clean":"2024-09-11T08:39:38.813030+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:39:38.813030+0000","last_undegraded":"2024-09-11T08:39:38.813030+0000","last_fullsized":"2024-09-11T08:39:38.813030+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":67,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:12.691883+0000","last_change":"2024-09-11T08:40:12.584703+0000","last_active":"2024-09-11T08:40:12.691883+0000","last_peered":"2024-09-11T08:40:12.691883+0000","last_clean":"2024-09-11T08:40:12.691883+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:40:12.691883+0000","last_undegraded":"2024-09-11T08:40:12.691883+0000","last_fullsized":"2024-09-11T08:40:12.691883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:12.584631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:12.584631+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:27.946233+0000","scrub_duration":0.0015754790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":55,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.812179+0000","last_change":"2024-09-11T08:38:40.979334+0000","last_active":"2024-09-11T08:39:38.812179+0000","last_peered":"2024-09-11T08:39:38.812179+0000","last_clean":"2024-09-11T08:39:38.812179+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:39:38.812179+0000","last_undegraded":"2024-09-11T08:39:38.812179+0000","last_fullsized":"2024-09-11T08:39:38.812179+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679102+0000","last_change":"2024-09-11T08:38:38.951009+0000","last_active":"2024-09-11T08:39:38.679102+0000","last_peered":"2024-09-11T08:39:38.679102+0000","last_clean":"2024-09-11T08:39:38.679102+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:39:38.679102+0000","last_undegraded":"2024-09-11T08:39:38.679102+0000","last_fullsized":"2024-09-11T08:39:38.679102+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":65,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:12.691885+0000","last_change":"2024-09-11T08:40:11.625218+0000","last_active":"2024-09-11T08:40:12.691885+0000","last_peered":"2024-09-11T08:40:12.691885+0000","last_clean":"2024-09-11T08:40:12.691885+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:40:12.691885+0000","last_undegraded":"2024-09-11T08:40:12.691885+0000","last_fullsized":"2024-09-11T08:40:12.691885+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:11.625164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:11.625164+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.094157+0000","scrub_duration":0.0014996320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.813253+0000","last_change":"2024-09-11T08:38:44.991592+0000","last_active":"2024-09-11T08:39:38.813253+0000","last_peered":"2024-09-11T08:39:38.813253+0000","last_clean":"2024-09-11T08:39:38.813253+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:39:38.813253+0000","last_undegraded":"2024-09-11T08:39:38.813253+0000","last_fullsized":"2024-09-11T08:39:38.813253+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":71,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:05.197253+0000","last_change":"2024-09-11T08:40:05.197208+0000","last_active":"2024-09-11T08:40:05.197253+0000","last_peered":"2024-09-11T08:40:05.197253+0000","last_clean":"2024-09-11T08:40:05.197253+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:40:05.197253+0000","last_undegraded":"2024-09-11T08:40:05.197253+0000","last_fullsized":"2024-09-11T08:40:05.197253+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-11T08:40:05.197152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:05.197152+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:06.174279+0000","scrub_duration":0.0016976000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.037650+0000","last_change":"2024-09-11T08:39:43.037605+0000","last_active":"2024-09-11T08:39:43.037650+0000","last_peered":"2024-09-11T08:39:43.037650+0000","last_clean":"2024-09-11T08:39:43.037650+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:39:43.037650+0000","last_undegraded":"2024-09-11T08:39:43.037650+0000","last_fullsized":"2024-09-11T08:39:43.037650+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.037551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.037551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.071661+0000","scrub_duration":0.001523976,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.678252+0000","last_change":"2024-09-11T08:38:42.983881+0000","last_active":"2024-09-11T08:39:38.678252+0000","last_peered":"2024-09-11T08:39:38.678252+0000","last_clean":"2024-09-11T08:39:38.678252+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:39:38.678252+0000","last_undegraded":"2024-09-11T08:39:38.678252+0000","last_fullsized":"2024-09-11T08:39:38.678252+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.678676+0000","last_change":"2024-09-11T08:38:44.995555+0000","last_active":"2024-09-11T08:39:38.678676+0000","last_peered":"2024-09-11T08:39:38.678676+0000","last_clean":"2024-09-11T08:39:38.678676+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:39:38.678676+0000","last_undegraded":"2024-09-11T08:39:38.678676+0000","last_fullsized":"2024-09-11T08:39:38.678676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.678530+0000","last_change":"2024-09-11T08:38:40.968377+0000","last_active":"2024-09-11T08:39:38.678530+0000","last_peered":"2024-09-11T08:39:38.678530+0000","last_clean":"2024-09-11T08:39:38.678530+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:39:38.678530+0000","last_undegraded":"2024-09-11T08:39:38.678530+0000","last_fullsized":"2024-09-11T08:39:38.678530+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:41.267356+0000","last_change":"2024-09-11T08:39:41.267320+0000","last_active":"2024-09-11T08:39:41.267356+0000","last_peered":"2024-09-11T08:39:41.267356+0000","last_clean":"2024-09-11T08:39:41.267356+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:39:41.267356+0000","last_undegraded":"2024-09-11T08:39:41.267356+0000","last_fullsized":"2024-09-11T08:39:41.267356+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:41.267267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.267267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:01.547087+0000","scrub_duration":0.0013348209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":57,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:12.691552+0000","last_change":"2024-09-11T08:38:42.982794+0000","last_active":"2024-09-11T08:40:12.691552+0000","last_peered":"2024-09-11T08:40:12.691552+0000","last_clean":"2024-09-11T08:40:12.691552+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:40:12.691552+0000","last_undegraded":"2024-09-11T08:40:12.691552+0000","last_fullsized":"2024-09-11T08:40:12.691552+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:04.174501+0000","last_change":"2024-09-11T08:40:04.174461+0000","last_active":"2024-09-11T08:40:04.174501+0000","last_peered":"2024-09-11T08:40:04.174501+0000","last_clean":"2024-09-11T08:40:04.174501+0000","last_became_active":"2024-09-11T08:38:45.182724+0000","last_became_peered":"2024-09-11T08:38:45.182724+0000","last_unstale":"2024-09-11T08:40:04.174501+0000","last_undegraded":"2024-09-11T08:40:04.174501+0000","last_fullsized":"2024-09-11T08:40:04.174501+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:04.174409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:04.174409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.916947+0000","scrub_duration":0.0012996889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"63'10","reported_seq":55,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:13.327362+0000","last_change":"2024-09-11T08:40:13.327302+0000","last_active":"2024-09-11T08:40:13.327362+0000","last_peered":"2024-09-11T08:40:13.327362+0000","last_clean":"2024-09-11T08:40:13.327362+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:40:13.327362+0000","last_undegraded":"2024-09-11T08:40:13.327362+0000","last_fullsized":"2024-09-11T08:40:13.327362+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:40:13.327245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:13.327245+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.332692+0000","scrub_duration":0.0017714300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.813282+0000","last_change":"2024-09-11T08:38:38.951168+0000","last_active":"2024-09-11T08:39:38.813282+0000","last_peered":"2024-09-11T08:39:38.813282+0000","last_clean":"2024-09-11T08:39:38.813282+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:39:38.813282+0000","last_undegraded":"2024-09-11T08:39:38.813282+0000","last_fullsized":"2024-09-11T08:39:38.813282+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":55,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:12.691551+0000","last_change":"2024-09-11T08:38:42.981732+0000","last_active":"2024-09-11T08:40:12.691551+0000","last_peered":"2024-09-11T08:40:12.691551+0000","last_clean":"2024-09-11T08:40:12.691551+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:40:12.691551+0000","last_undegraded":"2024-09-11T08:40:12.691551+0000","last_fullsized":"2024-09-11T08:40:12.691551+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.812007+0000","last_change":"2024-09-11T08:38:44.998257+0000","last_active":"2024-09-11T08:39:38.812007+0000","last_peered":"2024-09-11T08:39:38.812007+0000","last_clean":"2024-09-11T08:39:38.812007+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:39:38.812007+0000","last_undegraded":"2024-09-11T08:39:38.812007+0000","last_fullsized":"2024-09-11T08:39:38.812007+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":66,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.812872+0000","last_change":"2024-09-11T08:38:41.040034+0000","last_active":"2024-09-11T08:39:38.812872+0000","last_peered":"2024-09-11T08:39:38.812872+0000","last_clean":"2024-09-11T08:39:38.812872+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:39:38.812872+0000","last_undegraded":"2024-09-11T08:39:38.812872+0000","last_fullsized":"2024-09-11T08:39:38.812872+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:03.496042+0000","last_change":"2024-09-11T08:40:03.496005+0000","last_active":"2024-09-11T08:40:03.496042+0000","last_peered":"2024-09-11T08:40:03.496042+0000","last_clean":"2024-09-11T08:40:03.496042+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:40:03.496042+0000","last_undegraded":"2024-09-11T08:40:03.496042+0000","last_fullsized":"2024-09-11T08:40:03.496042+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:03.495950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.495950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.342754+0000","scrub_duration":0.001379314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:57.597038+0000","last_change":"2024-09-11T08:39:57.596980+0000","last_active":"2024-09-11T08:39:57.597038+0000","last_peered":"2024-09-11T08:39:57.597038+0000","last_clean":"2024-09-11T08:39:57.597038+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:39:57.597038+0000","last_undegraded":"2024-09-11T08:39:57.597038+0000","last_fullsized":"2024-09-11T08:39:57.597038+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.596889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.648355+0000","scrub_duration":0.001574424,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:08.387118+0000","last_change":"2024-09-11T08:40:08.387078+0000","last_active":"2024-09-11T08:40:08.387118+0000","last_peered":"2024-09-11T08:40:08.387118+0000","last_clean":"2024-09-11T08:40:08.387118+0000","last_became_active":"2024-09-11T08:38:45.182782+0000","last_became_peered":"2024-09-11T08:38:45.182782+0000","last_unstale":"2024-09-11T08:40:08.387118+0000","last_undegraded":"2024-09-11T08:40:08.387118+0000","last_fullsized":"2024-09-11T08:40:08.387118+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:08.387027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.387027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.887729+0000","scrub_duration":0.001411336,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":117,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:12.691572+0000","last_change":"2024-09-11T08:39:53.493004+0000","last_active":"2024-09-11T08:40:12.691572+0000","last_peered":"2024-09-11T08:40:12.691572+0000","last_clean":"2024-09-11T08:40:12.691572+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:40:12.691572+0000","last_undegraded":"2024-09-11T08:40:12.691572+0000","last_fullsized":"2024-09-11T08:40:12.691572+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.492945+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.774847+0000","scrub_duration":0.005988045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679213+0000","last_change":"2024-09-11T08:38:38.955616+0000","last_active":"2024-09-11T08:39:38.679213+0000","last_peered":"2024-09-11T08:39:38.679213+0000","last_clean":"2024-09-11T08:39:38.679213+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:39:38.679213+0000","last_undegraded":"2024-09-11T08:39:38.679213+0000","last_fullsized":"2024-09-11T08:39:38.679213+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:58.472925+0000","last_change":"2024-09-11T08:39:58.472851+0000","last_active":"2024-09-11T08:39:58.472925+0000","last_peered":"2024-09-11T08:39:58.472925+0000","last_clean":"2024-09-11T08:39:58.472925+0000","last_became_active":"2024-09-11T08:38:38.947270+0000","last_became_peered":"2024-09-11T08:38:38.947270+0000","last_unstale":"2024-09-11T08:39:58.472925+0000","last_undegraded":"2024-09-11T08:39:58.472925+0000","last_fullsized":"2024-09-11T08:39:58.472925+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:39:58.472785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.472785+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.725781+0000","scrub_duration":0.0020806930000000002,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.812544+0000","last_change":"2024-09-11T08:38:42.975757+0000","last_active":"2024-09-11T08:39:38.812544+0000","last_peered":"2024-09-11T08:39:38.812544+0000","last_clean":"2024-09-11T08:39:38.812544+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:39:38.812544+0000","last_undegraded":"2024-09-11T08:39:38.812544+0000","last_fullsized":"2024-09-11T08:39:38.812544+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679230+0000","last_change":"2024-09-11T08:38:44.997146+0000","last_active":"2024-09-11T08:39:38.679230+0000","last_peered":"2024-09-11T08:39:38.679230+0000","last_clean":"2024-09-11T08:39:38.679230+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:39:38.679230+0000","last_undegraded":"2024-09-11T08:39:38.679230+0000","last_fullsized":"2024-09-11T08:39:38.679230+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":67,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.585220+0000","last_change":"2024-09-11T08:39:43.585175+0000","last_active":"2024-09-11T08:39:43.585220+0000","last_peered":"2024-09-11T08:39:43.585220+0000","last_clean":"2024-09-11T08:39:43.585220+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:39:43.585220+0000","last_undegraded":"2024-09-11T08:39:43.585220+0000","last_fullsized":"2024-09-11T08:39:43.585220+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:39:43.585116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.585116+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.724395+0000","scrub_duration":0.0021210309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:00.500492+0000","last_change":"2024-09-11T08:40:00.500444+0000","last_active":"2024-09-11T08:40:00.500492+0000","last_peered":"2024-09-11T08:40:00.500492+0000","last_clean":"2024-09-11T08:40:00.500492+0000","last_became_active":"2024-09-11T08:38:38.949019+0000","last_became_peered":"2024-09-11T08:38:38.949019+0000","last_unstale":"2024-09-11T08:40:00.500492+0000","last_undegraded":"2024-09-11T08:40:00.500492+0000","last_fullsized":"2024-09-11T08:40:00.500492+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.500392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.500392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.354187+0000","scrub_duration":0.001294566,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:01.488594+0000","last_change":"2024-09-11T08:40:01.488539+0000","last_active":"2024-09-11T08:40:01.488594+0000","last_peered":"2024-09-11T08:40:01.488594+0000","last_clean":"2024-09-11T08:40:01.488594+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:40:01.488594+0000","last_undegraded":"2024-09-11T08:40:01.488594+0000","last_fullsized":"2024-09-11T08:40:01.488594+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:40:01.488482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:01.488482+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.917024+0000","scrub_duration":0.001463422,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:02.444408+0000","last_change":"2024-09-11T08:40:02.444341+0000","last_active":"2024-09-11T08:40:02.444408+0000","last_peered":"2024-09-11T08:40:02.444408+0000","last_clean":"2024-09-11T08:40:02.444408+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:40:02.444408+0000","last_undegraded":"2024-09-11T08:40:02.444408+0000","last_fullsized":"2024-09-11T08:40:02.444408+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.444279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.886414+0000","scrub_duration":0.001414329,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-11T08:40:15.184465+0000","last_change":"2024-09-11T08:40:15.184420+0000","last_active":"2024-09-11T08:40:15.184465+0000","last_peered":"2024-09-11T08:40:15.184465+0000","last_clean":"2024-09-11T08:40:15.184465+0000","last_became_active":"2024-09-11T08:38:45.183582+0000","last_became_peered":"2024-09-11T08:38:45.183582+0000","last_unstale":"2024-09-11T08:40:15.184465+0000","last_undegraded":"2024-09-11T08:40:15.184465+0000","last_fullsized":"2024-09-11T08:40:15.184465+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:15.184337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:15.184337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.002701+0000","scrub_duration":0.0039899209999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.813038+0000","last_change":"2024-09-11T08:38:40.975966+0000","last_active":"2024-09-11T08:39:38.813038+0000","last_peered":"2024-09-11T08:39:38.813038+0000","last_clean":"2024-09-11T08:39:38.813038+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:39:38.813038+0000","last_undegraded":"2024-09-11T08:39:38.813038+0000","last_fullsized":"2024-09-11T08:39:38.813038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.356791+0000","last_change":"2024-09-11T08:39:43.356751+0000","last_active":"2024-09-11T08:39:43.356791+0000","last_peered":"2024-09-11T08:39:43.356791+0000","last_clean":"2024-09-11T08:39:43.356791+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:39:43.356791+0000","last_undegraded":"2024-09-11T08:39:43.356791+0000","last_fullsized":"2024-09-11T08:39:43.356791+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.356699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.356699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.569637+0000","scrub_duration":0.001649455,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":139,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-11T08:40:15.182723+0000","last_change":"2024-09-11T08:39:55.153362+0000","last_active":"2024-09-11T08:40:15.182723+0000","last_peered":"2024-09-11T08:40:15.182723+0000","last_clean":"2024-09-11T08:40:15.182723+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:40:15.182723+0000","last_undegraded":"2024-09-11T08:40:15.182723+0000","last_fullsized":"2024-09-11T08:40:15.182723+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:39:55.153290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.153290+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.446909+0000","scrub_duration":0.0017161990000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":90,"num_read_kb":85,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.813348+0000","last_change":"2024-09-11T08:38:42.976525+0000","last_active":"2024-09-11T08:39:38.813348+0000","last_peered":"2024-09-11T08:39:38.813348+0000","last_clean":"2024-09-11T08:39:38.813348+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:39:38.813348+0000","last_undegraded":"2024-09-11T08:39:38.813348+0000","last_fullsized":"2024-09-11T08:39:38.813348+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-11T08:40:15.280851+0000","last_change":"2024-09-11T08:40:15.280809+0000","last_active":"2024-09-11T08:40:15.280851+0000","last_peered":"2024-09-11T08:40:15.280851+0000","last_clean":"2024-09-11T08:40:15.280851+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:40:15.280851+0000","last_undegraded":"2024-09-11T08:40:15.280851+0000","last_fullsized":"2024-09-11T08:40:15.280851+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:15.280760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:15.280760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:36.262012+0000","scrub_duration":0.0012405719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:00.540241+0000","last_change":"2024-09-11T08:40:00.540192+0000","last_active":"2024-09-11T08:40:00.540241+0000","last_peered":"2024-09-11T08:40:00.540241+0000","last_clean":"2024-09-11T08:40:00.540241+0000","last_became_active":"2024-09-11T08:38:41.040948+0000","last_became_peered":"2024-09-11T08:38:41.040948+0000","last_unstale":"2024-09-11T08:40:00.540241+0000","last_undegraded":"2024-09-11T08:40:00.540241+0000","last_fullsized":"2024-09-11T08:40:00.540241+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:40:00.540136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.540136+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:07.651325+0000","scrub_duration":0.001785095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679240+0000","last_change":"2024-09-11T08:38:38.948242+0000","last_active":"2024-09-11T08:39:38.679240+0000","last_peered":"2024-09-11T08:39:38.679240+0000","last_clean":"2024-09-11T08:39:38.679240+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:39:38.679240+0000","last_undegraded":"2024-09-11T08:39:38.679240+0000","last_fullsized":"2024-09-11T08:39:38.679240+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":75,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.678312+0000","last_change":"2024-09-11T08:39:12.765244+0000","last_active":"2024-09-11T08:39:38.678312+0000","last_peered":"2024-09-11T08:39:38.678312+0000","last_clean":"2024-09-11T08:39:38.678312+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:39:38.678312+0000","last_undegraded":"2024-09-11T08:39:38.678312+0000","last_fullsized":"2024-09-11T08:39:38.678312+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'72","last_scrub_stamp":"2024-09-11T08:39:12.765190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:39:12.765190+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:16.661831+0000","scrub_duration":0.0014709829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:49.116263+0000","last_change":"2024-09-11T08:39:49.116184+0000","last_active":"2024-09-11T08:39:49.116263+0000","last_peered":"2024-09-11T08:39:49.116263+0000","last_clean":"2024-09-11T08:39:49.116263+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:39:49.116263+0000","last_undegraded":"2024-09-11T08:39:49.116263+0000","last_fullsized":"2024-09-11T08:39:49.116263+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.116113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.116113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.209747+0000","scrub_duration":0.00167223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:47.423793+0000","last_change":"2024-09-11T08:39:47.423745+0000","last_active":"2024-09-11T08:39:47.423793+0000","last_peered":"2024-09-11T08:39:47.423793+0000","last_clean":"2024-09-11T08:39:47.423793+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:39:47.423793+0000","last_undegraded":"2024-09-11T08:39:47.423793+0000","last_fullsized":"2024-09-11T08:39:47.423793+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.423684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.423684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.004042+0000","scrub_duration":0.001458972,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":51,"reported_epoch":79,"state":"peering","last_fresh":"2024-09-11T08:40:15.179599+0000","last_change":"2024-09-11T08:40:15.179855+0000","last_active":"2024-09-11T08:40:15.179568+0000","last_peered":"2024-09-11T08:39:38.680107+0000","last_clean":"2024-09-11T08:39:38.680107+0000","last_became_active":"2024-09-11T08:38:40.973373+0000","last_became_peered":"2024-09-11T08:38:40.973373+0000","last_unstale":"2024-09-11T08:40:15.179599+0000","last_undegraded":"2024-09-11T08:40:15.179599+0000","last_fullsized":"2024-09-11T08:40:15.179599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.678676+0000","last_change":"2024-09-11T08:38:38.948244+0000","last_active":"2024-09-11T08:39:38.678676+0000","last_peered":"2024-09-11T08:39:38.678676+0000","last_clean":"2024-09-11T08:39:38.678676+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:39:38.678676+0000","last_undegraded":"2024-09-11T08:39:38.678676+0000","last_fullsized":"2024-09-11T08:39:38.678676+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:03.430941+0000","last_change":"2024-09-11T08:40:03.430883+0000","last_active":"2024-09-11T08:40:03.430941+0000","last_peered":"2024-09-11T08:40:03.430941+0000","last_clean":"2024-09-11T08:40:03.430941+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:40:03.430941+0000","last_undegraded":"2024-09-11T08:40:03.430941+0000","last_fullsized":"2024-09-11T08:40:03.430941+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:03.430794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.430794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:20.081896+0000","scrub_duration":0.0014294430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679334+0000","last_change":"2024-09-11T08:38:45.185064+0000","last_active":"2024-09-11T08:39:38.679334+0000","last_peered":"2024-09-11T08:39:38.679334+0000","last_clean":"2024-09-11T08:39:38.679334+0000","last_became_active":"2024-09-11T08:38:45.184410+0000","last_became_peered":"2024-09-11T08:38:45.184410+0000","last_unstale":"2024-09-11T08:39:38.679334+0000","last_undegraded":"2024-09-11T08:39:38.679334+0000","last_fullsized":"2024-09-11T08:39:38.679334+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":68,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:43.526903+0000","last_change":"2024-09-11T08:39:43.526845+0000","last_active":"2024-09-11T08:39:43.526903+0000","last_peered":"2024-09-11T08:39:43.526903+0000","last_clean":"2024-09-11T08:39:43.526903+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:39:43.526903+0000","last_undegraded":"2024-09-11T08:39:43.526903+0000","last_fullsized":"2024-09-11T08:39:43.526903+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:39:43.526763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.526763+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.869811+0000","scrub_duration":0.0022037329999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:55.599111+0000","last_change":"2024-09-11T08:39:55.599049+0000","last_active":"2024-09-11T08:39:55.599111+0000","last_peered":"2024-09-11T08:39:55.599111+0000","last_clean":"2024-09-11T08:39:55.599111+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:39:55.599111+0000","last_undegraded":"2024-09-11T08:39:55.599111+0000","last_fullsized":"2024-09-11T08:39:55.599111+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.598961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.598961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.562190+0000","scrub_duration":0.0016789820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:46.627126+0000","last_change":"2024-09-11T08:39:46.627077+0000","last_active":"2024-09-11T08:39:46.627126+0000","last_peered":"2024-09-11T08:39:46.627126+0000","last_clean":"2024-09-11T08:39:46.627126+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:39:46.627126+0000","last_undegraded":"2024-09-11T08:39:46.627126+0000","last_fullsized":"2024-09-11T08:39:46.627126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:46.627028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:46.627028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:56.334041+0000","scrub_duration":0.0013824029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:13.462217+0000","last_change":"2024-09-11T08:40:13.462183+0000","last_active":"2024-09-11T08:40:13.462217+0000","last_peered":"2024-09-11T08:40:13.462217+0000","last_clean":"2024-09-11T08:40:13.462217+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:40:13.462217+0000","last_undegraded":"2024-09-11T08:40:13.462217+0000","last_fullsized":"2024-09-11T08:40:13.462217+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:13.462132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:13.462132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:33.998327+0000","scrub_duration":0.001365727,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":45,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679897+0000","last_change":"2024-09-11T08:38:40.969616+0000","last_active":"2024-09-11T08:39:38.679897+0000","last_peered":"2024-09-11T08:39:38.679897+0000","last_clean":"2024-09-11T08:39:38.679897+0000","last_became_active":"2024-09-11T08:38:40.969348+0000","last_became_peered":"2024-09-11T08:38:40.969348+0000","last_unstale":"2024-09-11T08:39:38.679897+0000","last_undegraded":"2024-09-11T08:39:38.679897+0000","last_fullsized":"2024-09-11T08:39:38.679897+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679338+0000","last_change":"2024-09-11T08:38:38.957214+0000","last_active":"2024-09-11T08:39:38.679338+0000","last_peered":"2024-09-11T08:39:38.679338+0000","last_clean":"2024-09-11T08:39:38.679338+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:39:38.679338+0000","last_undegraded":"2024-09-11T08:39:38.679338+0000","last_fullsized":"2024-09-11T08:39:38.679338+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":65,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:12.691571+0000","last_change":"2024-09-11T08:40:00.473606+0000","last_active":"2024-09-11T08:40:12.691571+0000","last_peered":"2024-09-11T08:40:12.691571+0000","last_clean":"2024-09-11T08:40:12.691571+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:40:12.691571+0000","last_undegraded":"2024-09-11T08:40:12.691571+0000","last_fullsized":"2024-09-11T08:40:12.691571+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:00.473549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.473549+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.300137+0000","scrub_duration":0.0016703060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679551+0000","last_change":"2024-09-11T08:38:44.993736+0000","last_active":"2024-09-11T08:39:38.679551+0000","last_peered":"2024-09-11T08:39:38.679551+0000","last_clean":"2024-09-11T08:39:38.679551+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:39:38.679551+0000","last_undegraded":"2024-09-11T08:39:38.679551+0000","last_fullsized":"2024-09-11T08:39:38.679551+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":46,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.812811+0000","last_change":"2024-09-11T08:38:40.977113+0000","last_active":"2024-09-11T08:39:38.812811+0000","last_peered":"2024-09-11T08:39:38.812811+0000","last_clean":"2024-09-11T08:39:38.812811+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:39:38.812811+0000","last_undegraded":"2024-09-11T08:39:38.812811+0000","last_fullsized":"2024-09-11T08:39:38.812811+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:50.452621+0000","last_change":"2024-09-11T08:39:50.452568+0000","last_active":"2024-09-11T08:39:50.452621+0000","last_peered":"2024-09-11T08:39:50.452621+0000","last_clean":"2024-09-11T08:39:50.452621+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:39:50.452621+0000","last_undegraded":"2024-09-11T08:39:50.452621+0000","last_fullsized":"2024-09-11T08:39:50.452621+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.452494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.631278+0000","scrub_duration":0.001547086,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:09.519107+0000","last_change":"2024-09-11T08:40:09.519072+0000","last_active":"2024-09-11T08:40:09.519107+0000","last_peered":"2024-09-11T08:40:09.519107+0000","last_clean":"2024-09-11T08:40:09.519107+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:40:09.519107+0000","last_undegraded":"2024-09-11T08:40:09.519107+0000","last_fullsized":"2024-09-11T08:40:09.519107+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:09.519019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:09.519019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.000837+0000","scrub_duration":0.001232471,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.812055+0000","last_change":"2024-09-11T08:38:44.991908+0000","last_active":"2024-09-11T08:39:38.812055+0000","last_peered":"2024-09-11T08:39:38.812055+0000","last_clean":"2024-09-11T08:39:38.812055+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:39:38.812055+0000","last_undegraded":"2024-09-11T08:39:38.812055+0000","last_fullsized":"2024-09-11T08:39:38.812055+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":70,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:45.575299+0000","last_change":"2024-09-11T08:39:45.575209+0000","last_active":"2024-09-11T08:39:45.575299+0000","last_peered":"2024-09-11T08:39:45.575299+0000","last_clean":"2024-09-11T08:39:45.575299+0000","last_became_active":"2024-09-11T08:38:41.040207+0000","last_became_peered":"2024-09-11T08:38:41.040207+0000","last_unstale":"2024-09-11T08:39:45.575299+0000","last_undegraded":"2024-09-11T08:39:45.575299+0000","last_fullsized":"2024-09-11T08:39:45.575299+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:39:45.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.575133+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.182090+0000","scrub_duration":0.0022792160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:56.112171+0000","last_change":"2024-09-11T08:39:56.112112+0000","last_active":"2024-09-11T08:39:56.112171+0000","last_peered":"2024-09-11T08:39:56.112171+0000","last_clean":"2024-09-11T08:39:56.112171+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:39:56.112171+0000","last_undegraded":"2024-09-11T08:39:56.112171+0000","last_fullsized":"2024-09-11T08:39:56.112171+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:56.112043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.112043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:25.809354+0000","scrub_duration":0.001517137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:11.540870+0000","last_change":"2024-09-11T08:40:11.540833+0000","last_active":"2024-09-11T08:40:11.540870+0000","last_peered":"2024-09-11T08:40:11.540870+0000","last_clean":"2024-09-11T08:40:11.540870+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:40:11.540870+0000","last_undegraded":"2024-09-11T08:40:11.540870+0000","last_fullsized":"2024-09-11T08:40:11.540870+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:11.540760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:11.540760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.937538+0000","scrub_duration":0.0014212529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.678211+0000","last_change":"2024-09-11T08:38:44.993458+0000","last_active":"2024-09-11T08:39:38.678211+0000","last_peered":"2024-09-11T08:39:38.678211+0000","last_clean":"2024-09-11T08:39:38.678211+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:39:38.678211+0000","last_undegraded":"2024-09-11T08:39:38.678211+0000","last_fullsized":"2024-09-11T08:39:38.678211+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":63,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:40.524667+0000","last_change":"2024-09-11T08:39:40.524581+0000","last_active":"2024-09-11T08:39:40.524667+0000","last_peered":"2024-09-11T08:39:40.524667+0000","last_clean":"2024-09-11T08:39:40.524667+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:39:40.524667+0000","last_undegraded":"2024-09-11T08:39:40.524667+0000","last_fullsized":"2024-09-11T08:39:40.524667+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:40.524498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:40.524498+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.186922+0000","scrub_duration":0.0024035010000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:00.113623+0000","last_change":"2024-09-11T08:40:00.113587+0000","last_active":"2024-09-11T08:40:00.113623+0000","last_peered":"2024-09-11T08:40:00.113623+0000","last_clean":"2024-09-11T08:40:00.113623+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:40:00.113623+0000","last_undegraded":"2024-09-11T08:40:00.113623+0000","last_fullsized":"2024-09-11T08:40:00.113623+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.113534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.113534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.476975+0000","scrub_duration":0.0011790780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:57.454296+0000","last_change":"2024-09-11T08:39:57.454257+0000","last_active":"2024-09-11T08:39:57.454296+0000","last_peered":"2024-09-11T08:39:57.454296+0000","last_clean":"2024-09-11T08:39:57.454296+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:39:57.454296+0000","last_undegraded":"2024-09-11T08:39:57.454296+0000","last_fullsized":"2024-09-11T08:39:57.454296+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.454198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.454198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.731898+0000","scrub_duration":0.0011666669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679015+0000","last_change":"2024-09-11T08:38:44.998686+0000","last_active":"2024-09-11T08:39:38.679015+0000","last_peered":"2024-09-11T08:39:38.679015+0000","last_clean":"2024-09-11T08:39:38.679015+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:39:38.679015+0000","last_undegraded":"2024-09-11T08:39:38.679015+0000","last_fullsized":"2024-09-11T08:39:38.679015+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":65,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:48.462305+0000","last_change":"2024-09-11T08:39:48.462233+0000","last_active":"2024-09-11T08:39:48.462305+0000","last_peered":"2024-09-11T08:39:48.462305+0000","last_clean":"2024-09-11T08:39:48.462305+0000","last_became_active":"2024-09-11T08:38:41.040475+0000","last_became_peered":"2024-09-11T08:38:41.040475+0000","last_unstale":"2024-09-11T08:39:48.462305+0000","last_undegraded":"2024-09-11T08:39:48.462305+0000","last_fullsized":"2024-09-11T08:39:48.462305+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-11T08:39:48.462163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:48.462163+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.032207+0000","scrub_duration":0.002094753,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":58,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:03.143863+0000","last_change":"2024-09-11T08:40:03.143806+0000","last_active":"2024-09-11T08:40:03.143863+0000","last_peered":"2024-09-11T08:40:03.143863+0000","last_clean":"2024-09-11T08:40:03.143863+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:40:03.143863+0000","last_undegraded":"2024-09-11T08:40:03.143863+0000","last_fullsized":"2024-09-11T08:40:03.143863+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:40:03.143742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.143742+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.570435+0000","scrub_duration":0.001564396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"63'8","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:13.221754+0000","last_change":"2024-09-11T08:40:13.221708+0000","last_active":"2024-09-11T08:40:13.221754+0000","last_peered":"2024-09-11T08:40:13.221754+0000","last_clean":"2024-09-11T08:40:13.221754+0000","last_became_active":"2024-09-11T08:38:42.986756+0000","last_became_peered":"2024-09-11T08:38:42.986756+0000","last_unstale":"2024-09-11T08:40:13.221754+0000","last_undegraded":"2024-09-11T08:40:13.221754+0000","last_fullsized":"2024-09-11T08:40:13.221754+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:13.221650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:13.221650+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:28.234147+0000","scrub_duration":0.0017036169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:11.370358+0000","last_change":"2024-09-11T08:40:11.370313+0000","last_active":"2024-09-11T08:40:11.370358+0000","last_peered":"2024-09-11T08:40:11.370358+0000","last_clean":"2024-09-11T08:40:11.370358+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:40:11.370358+0000","last_undegraded":"2024-09-11T08:40:11.370358+0000","last_fullsized":"2024-09-11T08:40:11.370358+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:11.370245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:11.370245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:35.188374+0000","scrub_duration":0.0011869179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:42.605423+0000","last_change":"2024-09-11T08:39:42.605373+0000","last_active":"2024-09-11T08:39:42.605423+0000","last_peered":"2024-09-11T08:39:42.605423+0000","last_clean":"2024-09-11T08:39:42.605423+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:39:42.605423+0000","last_undegraded":"2024-09-11T08:39:42.605423+0000","last_fullsized":"2024-09-11T08:39:42.605423+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:42.605301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:42.605301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:53.937210+0000","scrub_duration":0.0014798509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":40,"reported_epoch":79,"state":"peering","last_fresh":"2024-09-11T08:40:15.179136+0000","last_change":"2024-09-11T08:40:15.179370+0000","last_active":"2024-09-11T08:40:15.179106+0000","last_peered":"2024-09-11T08:39:38.679612+0000","last_clean":"2024-09-11T08:39:38.679612+0000","last_became_active":"2024-09-11T08:38:41.039552+0000","last_became_peered":"2024-09-11T08:38:41.039552+0000","last_unstale":"2024-09-11T08:40:15.179136+0000","last_undegraded":"2024-09-11T08:40:15.179136+0000","last_fullsized":"2024-09-11T08:40:15.179136+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:49.472829+0000","last_change":"2024-09-11T08:39:49.472764+0000","last_active":"2024-09-11T08:39:49.472829+0000","last_peered":"2024-09-11T08:39:49.472829+0000","last_clean":"2024-09-11T08:39:49.472829+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:39:49.472829+0000","last_undegraded":"2024-09-11T08:39:49.472829+0000","last_fullsized":"2024-09-11T08:39:49.472829+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.472696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.472696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:01.403308+0000","scrub_duration":0.001919104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.678949+0000","last_change":"2024-09-11T08:38:45.185016+0000","last_active":"2024-09-11T08:39:38.678949+0000","last_peered":"2024-09-11T08:39:38.678949+0000","last_clean":"2024-09-11T08:39:38.678949+0000","last_became_active":"2024-09-11T08:38:45.184753+0000","last_became_peered":"2024-09-11T08:38:45.184753+0000","last_unstale":"2024-09-11T08:39:38.678949+0000","last_undegraded":"2024-09-11T08:39:38.678949+0000","last_fullsized":"2024-09-11T08:39:38.678949+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":46,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.813128+0000","last_change":"2024-09-11T08:38:41.041476+0000","last_active":"2024-09-11T08:39:38.813128+0000","last_peered":"2024-09-11T08:39:38.813128+0000","last_clean":"2024-09-11T08:39:38.813128+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:39:38.813128+0000","last_undegraded":"2024-09-11T08:39:38.813128+0000","last_fullsized":"2024-09-11T08:39:38.813128+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:53.176130+0000","last_change":"2024-09-11T08:39:53.176095+0000","last_active":"2024-09-11T08:39:53.176130+0000","last_peered":"2024-09-11T08:39:53.176130+0000","last_clean":"2024-09-11T08:39:53.176130+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:39:53.176130+0000","last_undegraded":"2024-09-11T08:39:53.176130+0000","last_fullsized":"2024-09-11T08:39:53.176130+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.176035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.176035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:00.682857+0000","scrub_duration":0.001147531,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679441+0000","last_change":"2024-09-11T08:38:42.982083+0000","last_active":"2024-09-11T08:39:38.679441+0000","last_peered":"2024-09-11T08:39:38.679441+0000","last_clean":"2024-09-11T08:39:38.679441+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:39:38.679441+0000","last_undegraded":"2024-09-11T08:39:38.679441+0000","last_fullsized":"2024-09-11T08:39:38.679441+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.493100+0000","last_change":"2024-09-11T08:39:52.493064+0000","last_active":"2024-09-11T08:39:52.493100+0000","last_peered":"2024-09-11T08:39:52.493100+0000","last_clean":"2024-09-11T08:39:52.493100+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:39:52.493100+0000","last_undegraded":"2024-09-11T08:39:52.493100+0000","last_fullsized":"2024-09-11T08:39:52.493100+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.493013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:13.717323+0000","scrub_duration":0.0012699160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":46,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.813114+0000","last_change":"2024-09-11T08:38:41.042115+0000","last_active":"2024-09-11T08:39:38.813114+0000","last_peered":"2024-09-11T08:39:38.813114+0000","last_clean":"2024-09-11T08:39:38.813114+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:39:38.813114+0000","last_undegraded":"2024-09-11T08:39:38.813114+0000","last_fullsized":"2024-09-11T08:39:38.813114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.813115+0000","last_change":"2024-09-11T08:38:38.958515+0000","last_active":"2024-09-11T08:39:38.813115+0000","last_peered":"2024-09-11T08:39:38.813115+0000","last_clean":"2024-09-11T08:39:38.813115+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:39:38.813115+0000","last_undegraded":"2024-09-11T08:39:38.813115+0000","last_fullsized":"2024-09-11T08:39:38.813115+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":54,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:12.691523+0000","last_change":"2024-09-11T08:38:42.981915+0000","last_active":"2024-09-11T08:40:12.691523+0000","last_peered":"2024-09-11T08:40:12.691523+0000","last_clean":"2024-09-11T08:40:12.691523+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:40:12.691523+0000","last_undegraded":"2024-09-11T08:40:12.691523+0000","last_fullsized":"2024-09-11T08:40:12.691523+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679548+0000","last_change":"2024-09-11T08:38:44.993997+0000","last_active":"2024-09-11T08:39:38.679548+0000","last_peered":"2024-09-11T08:39:38.679548+0000","last_clean":"2024-09-11T08:39:38.679548+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:39:38.679548+0000","last_undegraded":"2024-09-11T08:39:38.679548+0000","last_fullsized":"2024-09-11T08:39:38.679548+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":44,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679590+0000","last_change":"2024-09-11T08:38:41.040424+0000","last_active":"2024-09-11T08:39:38.679590+0000","last_peered":"2024-09-11T08:39:38.679590+0000","last_clean":"2024-09-11T08:39:38.679590+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:39:38.679590+0000","last_undegraded":"2024-09-11T08:39:38.679590+0000","last_fullsized":"2024-09-11T08:39:38.679590+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:14.202599+0000","last_change":"2024-09-11T08:40:14.202552+0000","last_active":"2024-09-11T08:40:14.202599+0000","last_peered":"2024-09-11T08:40:14.202599+0000","last_clean":"2024-09-11T08:40:14.202599+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:40:14.202599+0000","last_undegraded":"2024-09-11T08:40:14.202599+0000","last_fullsized":"2024-09-11T08:40:14.202599+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:14.202498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:14.202498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:42.406451+0000","scrub_duration":0.001204299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":65,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:12.691891+0000","last_change":"2024-09-11T08:39:44.429546+0000","last_active":"2024-09-11T08:40:12.691891+0000","last_peered":"2024-09-11T08:40:12.691891+0000","last_clean":"2024-09-11T08:40:12.691891+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:40:12.691891+0000","last_undegraded":"2024-09-11T08:40:12.691891+0000","last_fullsized":"2024-09-11T08:40:12.691891+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:39:44.429477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:44.429477+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.023544+0000","scrub_duration":0.001890784,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.254334+0000","last_change":"2024-09-11T08:39:52.254299+0000","last_active":"2024-09-11T08:39:52.254334+0000","last_peered":"2024-09-11T08:39:52.254334+0000","last_clean":"2024-09-11T08:39:52.254334+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:39:52.254334+0000","last_undegraded":"2024-09-11T08:39:52.254334+0000","last_fullsized":"2024-09-11T08:39:52.254334+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.254245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.254245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:20.782062+0000","scrub_duration":0.001581877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.678986+0000","last_change":"2024-09-11T08:38:40.973815+0000","last_active":"2024-09-11T08:39:38.678986+0000","last_peered":"2024-09-11T08:39:38.678986+0000","last_clean":"2024-09-11T08:39:38.678986+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:39:38.678986+0000","last_undegraded":"2024-09-11T08:39:38.678986+0000","last_fullsized":"2024-09-11T08:39:38.678986+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679430+0000","last_change":"2024-09-11T08:38:38.952604+0000","last_active":"2024-09-11T08:39:38.679430+0000","last_peered":"2024-09-11T08:39:38.679430+0000","last_clean":"2024-09-11T08:39:38.679430+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:39:38.679430+0000","last_undegraded":"2024-09-11T08:39:38.679430+0000","last_fullsized":"2024-09-11T08:39:38.679430+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.812402+0000","last_change":"2024-09-11T08:38:42.979138+0000","last_active":"2024-09-11T08:39:38.812402+0000","last_peered":"2024-09-11T08:39:38.812402+0000","last_clean":"2024-09-11T08:39:38.812402+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:39:38.812402+0000","last_undegraded":"2024-09-11T08:39:38.812402+0000","last_fullsized":"2024-09-11T08:39:38.812402+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.678968+0000","last_change":"2024-09-11T08:38:44.994241+0000","last_active":"2024-09-11T08:39:38.678968+0000","last_peered":"2024-09-11T08:39:38.678968+0000","last_clean":"2024-09-11T08:39:38.678968+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:39:38.678968+0000","last_undegraded":"2024-09-11T08:39:38.678968+0000","last_fullsized":"2024-09-11T08:39:38.678968+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":79,"state":"peering","last_fresh":"2024-09-11T08:40:15.182649+0000","last_change":"2024-09-11T08:40:15.182649+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T08:40:15.182649+0000","last_undegraded":"2024-09-11T08:40:15.182649+0000","last_fullsized":"2024-09-11T08:40:15.182649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679856+0000","last_change":"2024-09-11T08:38:38.954570+0000","last_active":"2024-09-11T08:39:38.679856+0000","last_peered":"2024-09-11T08:39:38.679856+0000","last_clean":"2024-09-11T08:39:38.679856+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:39:38.679856+0000","last_undegraded":"2024-09-11T08:39:38.679856+0000","last_fullsized":"2024-09-11T08:39:38.679856+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679107+0000","last_change":"2024-09-11T08:38:42.984381+0000","last_active":"2024-09-11T08:39:38.679107+0000","last_peered":"2024-09-11T08:39:38.679107+0000","last_clean":"2024-09-11T08:39:38.679107+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:39:38.679107+0000","last_undegraded":"2024-09-11T08:39:38.679107+0000","last_fullsized":"2024-09-11T08:39:38.679107+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.811690+0000","last_change":"2024-09-11T08:38:44.999759+0000","last_active":"2024-09-11T08:39:38.811690+0000","last_peered":"2024-09-11T08:39:38.811690+0000","last_clean":"2024-09-11T08:39:38.811690+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:39:38.811690+0000","last_undegraded":"2024-09-11T08:39:38.811690+0000","last_fullsized":"2024-09-11T08:39:38.811690+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679096+0000","last_change":"2024-09-11T08:38:41.039878+0000","last_active":"2024-09-11T08:39:38.679096+0000","last_peered":"2024-09-11T08:39:38.679096+0000","last_clean":"2024-09-11T08:39:38.679096+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:39:38.679096+0000","last_undegraded":"2024-09-11T08:39:38.679096+0000","last_fullsized":"2024-09-11T08:39:38.679096+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:05.430820+0000","last_change":"2024-09-11T08:40:05.430740+0000","last_active":"2024-09-11T08:40:05.430820+0000","last_peered":"2024-09-11T08:40:05.430820+0000","last_clean":"2024-09-11T08:40:05.430820+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:40:05.430820+0000","last_undegraded":"2024-09-11T08:40:05.430820+0000","last_fullsized":"2024-09-11T08:40:05.430820+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:05.430666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:05.430666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.818673+0000","scrub_duration":0.001547629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:47.104769+0000","last_change":"2024-09-11T08:39:47.104702+0000","last_active":"2024-09-11T08:39:47.104769+0000","last_peered":"2024-09-11T08:39:47.104769+0000","last_clean":"2024-09-11T08:39:47.104769+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:39:47.104769+0000","last_undegraded":"2024-09-11T08:39:47.104769+0000","last_fullsized":"2024-09-11T08:39:47.104769+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.104637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.104637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:51.311803+0000","scrub_duration":0.0015238020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:38.679518+0000","last_change":"2024-09-11T08:38:45.183913+0000","last_active":"2024-09-11T08:39:38.679518+0000","last_peered":"2024-09-11T08:39:38.679518+0000","last_clean":"2024-09-11T08:39:38.679518+0000","last_became_active":"2024-09-11T08:38:45.183548+0000","last_became_peered":"2024-09-11T08:38:45.183548+0000","last_unstale":"2024-09-11T08:39:38.679518+0000","last_undegraded":"2024-09-11T08:39:38.679518+0000","last_fullsized":"2024-09-11T08:39:38.679518+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":45,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:41.989782+0000","last_change":"2024-09-11T08:39:41.989740+0000","last_active":"2024-09-11T08:39:41.989782+0000","last_peered":"2024-09-11T08:39:41.989782+0000","last_clean":"2024-09-11T08:39:41.989782+0000","last_became_active":"2024-09-11T08:38:40.973799+0000","last_became_peered":"2024-09-11T08:38:40.973799+0000","last_unstale":"2024-09-11T08:39:41.989782+0000","last_undegraded":"2024-09-11T08:39:41.989782+0000","last_fullsized":"2024-09-11T08:39:41.989782+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-11T08:39:41.989684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.989684+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.687182+0000","scrub_duration":0.00162959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:08.336421+0000","last_change":"2024-09-11T08:40:08.336369+0000","last_active":"2024-09-11T08:40:08.336421+0000","last_peered":"2024-09-11T08:40:08.336421+0000","last_clean":"2024-09-11T08:40:08.336421+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:40:08.336421+0000","last_undegraded":"2024-09-11T08:40:08.336421+0000","last_fullsized":"2024-09-11T08:40:08.336421+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:40:08.336314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.336314+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:16.115575+0000","scrub_duration":0.001394085,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:53.541691+0000","last_change":"2024-09-11T08:39:53.541628+0000","last_active":"2024-09-11T08:39:53.541691+0000","last_peered":"2024-09-11T08:39:53.541691+0000","last_clean":"2024-09-11T08:39:53.541691+0000","last_became_active":"2024-09-11T08:38:42.989556+0000","last_became_peered":"2024-09-11T08:38:42.989556+0000","last_unstale":"2024-09-11T08:39:53.541691+0000","last_undegraded":"2024-09-11T08:39:53.541691+0000","last_fullsized":"2024-09-11T08:39:53.541691+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.541557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.541557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:08.967817+0000","scrub_duration":0.001310285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:07.114929+0000","last_change":"2024-09-11T08:40:07.114884+0000","last_active":"2024-09-11T08:40:07.114929+0000","last_peered":"2024-09-11T08:40:07.114929+0000","last_clean":"2024-09-11T08:40:07.114929+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:40:07.114929+0000","last_undegraded":"2024-09-11T08:40:07.114929+0000","last_fullsized":"2024-09-11T08:40:07.114929+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:07.114815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:07.114815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.897957+0000","scrub_duration":0.0013781500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:52.480711+0000","last_change":"2024-09-11T08:39:52.480663+0000","last_active":"2024-09-11T08:39:52.480711+0000","last_peered":"2024-09-11T08:39:52.480711+0000","last_clean":"2024-09-11T08:39:52.480711+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:39:52.480711+0000","last_undegraded":"2024-09-11T08:39:52.480711+0000","last_fullsized":"2024-09-11T08:39:52.480711+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.480610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.480610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:55.608483+0000","scrub_duration":0.001335754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:53.038393+0000","last_change":"2024-09-11T08:39:53.038356+0000","last_active":"2024-09-11T08:39:53.038393+0000","last_peered":"2024-09-11T08:39:53.038393+0000","last_clean":"2024-09-11T08:39:53.038393+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:39:53.038393+0000","last_undegraded":"2024-09-11T08:39:53.038393+0000","last_fullsized":"2024-09-11T08:39:53.038393+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.038297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.038297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:58.424182+0000","scrub_duration":0.0012198549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":58,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:40:02.551113+0000","last_change":"2024-09-11T08:40:02.551056+0000","last_active":"2024-09-11T08:40:02.551113+0000","last_peered":"2024-09-11T08:40:02.551113+0000","last_clean":"2024-09-11T08:40:02.551113+0000","last_became_active":"2024-09-11T08:38:40.974451+0000","last_became_peered":"2024-09-11T08:38:40.974451+0000","last_unstale":"2024-09-11T08:40:02.551113+0000","last_undegraded":"2024-09-11T08:40:02.551113+0000","last_fullsized":"2024-09-11T08:40:02.551113+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:40:02.550999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.550999+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.173446+0000","scrub_duration":0.0016484399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T08:39:55.265199+0000","last_change":"2024-09-11T08:39:55.265155+0000","last_active":"2024-09-11T08:39:55.265199+0000","last_peered":"2024-09-11T08:39:55.265199+0000","last_clean":"2024-09-11T08:39:55.265199+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:39:55.265199+0000","last_undegraded":"2024-09-11T08:39:55.265199+0000","last_fullsized":"2024-09-11T08:39:55.265199+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.265097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.265097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.333633+0000","scrub_duration":0.0012893830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3592,"num_objects":173,"num_object_clones":0,"num_object_copies":519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":681,"num_read_kb":444,"num_write":407,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":414,"ondisk_log_size":414,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":85,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299417,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298788,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446364,"statfs":{"total":95995035648,"available":95689076736,"internally_reserved":0,"allocated":1470464,"data_stored":983979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.496},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41199999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.47799999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.432},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.46300000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.32500000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.35099999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.442},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39400000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528350,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298784,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446368,"statfs":{"total":95995035648,"available":95689080832,"internally_reserved":0,"allocated":1241088,"data_stored":916735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.71699999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.379},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.38800000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.59799999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.70799999999999996,"5min":0.871,"15min":0.871},"last":0.34699999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70999999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.35699999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.66200000000000003,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66800000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.439}]},{"osd":4,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.27500000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40600000000000003},"min":{"1min":0.29199999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.26400000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.439,"15min":0.43099999999999999},"min":{"1min":0.34200000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.435,"15min":0.42999999999999999},"min":{"1min":0.30299999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.40799999999999997}]},{"osd":7,"last update":"Wed Sep 11 08:39:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.247},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.32100000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757282,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298220,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446932,"statfs":{"total":95995035648,"available":95689658368,"internally_reserved":0,"allocated":663552,"data_stored":327663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.38300000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.36699999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.41899999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.439}]},{"osd":4,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.376},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.47599999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.42599999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73099999999999998,"5min":0.749,"15min":0.749},"last":0.40500000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.501}]}]},{"osd":4,"up_from":31,"seq":133143986214,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298220,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446932,"statfs":{"total":95995035648,"available":95689658368,"internally_reserved":0,"allocated":663552,"data_stored":326358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.56299999999999994,"15min":0.57299999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68899999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55800000000000005,"15min":0.56399999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.496},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":0.754,"15min":0.754},"last":0.39000000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70799999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.71099999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.53200000000000003}]},{"osd":3,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67100000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75800000000000001,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.52200000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.40500000000000003,"15min":0.38700000000000001},"min":{"1min":0.32100000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.30099999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.436}]},{"osd":6,"last update":"Wed Sep 11 08:40:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.32400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.36199999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67100000000000004,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.42699999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.505}]}]},{"osd":3,"up_from":26,"seq":111669149740,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":901120,"data_stored":394654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.36299999999999999,"15min":0.374},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.443,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.38900000000000001,"15min":0.40000000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.496,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.308}]},{"osd":1,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.39200000000000002,"15min":0.40400000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.56200000000000006,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.37},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.40400000000000003,"15min":0.41399999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48199999999999998,"5min":0.621,"15min":0.621},"last":0.25600000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.42699999999999999,"15min":0.435},"min":{"1min":0.27100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.44,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.34799999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40699999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.752,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52700000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.51000000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76300000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.48799999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:39:27 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.5},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.55400000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411376,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298196,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446956,"statfs":{"total":95995035648,"available":95689682944,"internally_reserved":0,"allocated":643072,"data_stored":326203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36499999999999999,"15min":0.35399999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.38400000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.25800000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.39100000000000001,"15min":0.376},"min":{"1min":0.28999999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.33400000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:39:57 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.36699999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.441,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:39:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:40:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.442,"5min":0.432,"15min":0.432},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.56100000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55300000000000005}]},{"osd":7,"last update":"Wed Sep 11 08:39:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.57499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298256,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446896,"statfs":{"total":95995035648,"available":95689621504,"internally_reserved":0,"allocated":700416,"data_stored":327467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.32700000000000001,"15min":0.313},"min":{"1min":0.24199999999999999,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.49199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.34499999999999997,"15min":0.33000000000000002},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.47599999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.29899999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:39:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.34000000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:39:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40400000000000003,"15min":0.41999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54400000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.433,"15min":0.434},"min":{"1min":0.307,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.35399999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79900000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.51300000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:39:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.755,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.496},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82199999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56100000000000005}]},{"osd":6,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57399999999999995,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60099999999999998,"15min":0.60599999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.52500000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.505},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.53700000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298808,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446344,"statfs":{"total":95995035648,"available":95689056256,"internally_reserved":0,"allocated":1265664,"data_stored":917567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:40:10 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.311},"min":{"1min":0.19800000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.436,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34000000000000002,"15min":0.314},"min":{"1min":0.23999999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.504,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.34799999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.35799999999999998,"15min":0.34699999999999998},"min":{"1min":0.24199999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53500000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.34899999999999998},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.51500000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.312}]},{"osd":3,"last update":"Wed Sep 11 08:39:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39800000000000002,"15min":0.40000000000000002},"min":{"1min":0.28499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.503,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.371}]},{"osd":4,"last update":"Wed Sep 11 08:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69699999999999995,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.48199999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.48899999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:40:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.37,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68300000000000005,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.42499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.746,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.46000000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.56699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:40:22.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:40:22.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:22 smithi092 ceph-mon[29647]: 3.12 scrub starts 2024-09-11T08:40:22.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:22 smithi092 ceph-mon[29647]: 3.12 scrub ok 2024-09-11T08:40:22.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:22 smithi092 ceph-mon[29647]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 36 B/s, 1 objects/s recovering 2024-09-11T08:40:22.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:22 smithi092 ceph-mon[29647]: 3.2 scrub starts 2024-09-11T08:40:22.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:22 smithi092 ceph-mon[29647]: 3.2 scrub ok 2024-09-11T08:40:22.483 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:22 smithi092 ceph-mon[41114]: 3.12 scrub starts 2024-09-11T08:40:22.483 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:22 smithi092 ceph-mon[41114]: 3.12 scrub ok 2024-09-11T08:40:22.483 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:22 smithi092 ceph-mon[41114]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 36 B/s, 1 objects/s recovering 2024-09-11T08:40:22.483 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:22 smithi092 ceph-mon[41114]: 3.2 scrub starts 2024-09-11T08:40:22.483 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:22 smithi092 ceph-mon[41114]: 3.2 scrub ok 2024-09-11T08:40:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:22 smithi145 ceph-mon[33384]: 3.12 scrub starts 2024-09-11T08:40:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:22 smithi145 ceph-mon[33384]: 3.12 scrub ok 2024-09-11T08:40:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:22 smithi145 ceph-mon[33384]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 36 B/s, 1 objects/s recovering 2024-09-11T08:40:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:22 smithi145 ceph-mon[33384]: 3.2 scrub starts 2024-09-11T08:40:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:22 smithi145 ceph-mon[33384]: 3.2 scrub ok 2024-09-11T08:40:23.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:40:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:40:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:23.314 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[41114]: 4.6 scrub starts 2024-09-11T08:40:23.314 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[41114]: 4.6 scrub ok 2024-09-11T08:40:23.314 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[41114]: 4.14 scrub starts 2024-09-11T08:40:23.314 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[41114]: 4.14 scrub ok 2024-09-11T08:40:23.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[41114]: 4.12 scrub starts 2024-09-11T08:40:23.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[41114]: 4.12 scrub ok 2024-09-11T08:40:23.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[41114]: 3.3 scrub starts 2024-09-11T08:40:23.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[41114]: 3.3 scrub ok 2024-09-11T08:40:23.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[29647]: 4.6 scrub starts 2024-09-11T08:40:23.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[29647]: 4.6 scrub ok 2024-09-11T08:40:23.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[29647]: 4.14 scrub starts 2024-09-11T08:40:23.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[29647]: 4.14 scrub ok 2024-09-11T08:40:23.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[29647]: 4.12 scrub starts 2024-09-11T08:40:23.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[29647]: 4.12 scrub ok 2024-09-11T08:40:23.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[29647]: 3.3 scrub starts 2024-09-11T08:40:23.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:23 smithi092 ceph-mon[29647]: 3.3 scrub ok 2024-09-11T08:40:23.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:23 smithi145 ceph-mon[33384]: 4.6 scrub starts 2024-09-11T08:40:23.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:23 smithi145 ceph-mon[33384]: 4.6 scrub ok 2024-09-11T08:40:23.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:23 smithi145 ceph-mon[33384]: 4.14 scrub starts 2024-09-11T08:40:23.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:23 smithi145 ceph-mon[33384]: 4.14 scrub ok 2024-09-11T08:40:23.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:23 smithi145 ceph-mon[33384]: 4.12 scrub starts 2024-09-11T08:40:23.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:23 smithi145 ceph-mon[33384]: 4.12 scrub ok 2024-09-11T08:40:23.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:23 smithi145 ceph-mon[33384]: 3.3 scrub starts 2024-09-11T08:40:23.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:23 smithi145 ceph-mon[33384]: 3.3 scrub ok 2024-09-11T08:40:23.752 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:23.875 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:23 smithi092 systemd[1]: Stopping Ceph alertmanager.a for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T08:40:24.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:24 smithi092 ceph-mon[29647]: 6.10 scrub starts 2024-09-11T08:40:24.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:24 smithi092 ceph-mon[29647]: 6.10 scrub ok 2024-09-11T08:40:24.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:24 smithi092 ceph-mon[29647]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 28 B/s, 1 objects/s recovering 2024-09-11T08:40:24.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:24 smithi092 ceph-mon[29647]: 3.8 scrub starts 2024-09-11T08:40:24.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:24 smithi092 ceph-mon[29647]: 3.8 scrub ok 2024-09-11T08:40:24.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:24 smithi092 ceph-mon[41114]: 6.10 scrub starts 2024-09-11T08:40:24.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:24 smithi092 ceph-mon[41114]: 6.10 scrub ok 2024-09-11T08:40:24.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:24 smithi092 ceph-mon[41114]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 28 B/s, 1 objects/s recovering 2024-09-11T08:40:24.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:24 smithi092 ceph-mon[41114]: 3.8 scrub starts 2024-09-11T08:40:24.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:24 smithi092 ceph-mon[41114]: 3.8 scrub ok 2024-09-11T08:40:24.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:24 smithi145 ceph-mon[33384]: 6.10 scrub starts 2024-09-11T08:40:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:24 smithi145 ceph-mon[33384]: 6.10 scrub ok 2024-09-11T08:40:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:24 smithi145 ceph-mon[33384]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 28 B/s, 1 objects/s recovering 2024-09-11T08:40:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:24 smithi145 ceph-mon[33384]: 3.8 scrub starts 2024-09-11T08:40:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:24 smithi145 ceph-mon[33384]: 3.8 scrub ok 2024-09-11T08:40:25.220 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:24 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[75712]: ts=2024-09-11T08:40:24.953Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:40:25.220 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:24 smithi092 podman[78746]: 2024-09-11 08:40:24.973895996 +0000 UTC m=+0.476116320 container died afe2c4d9d2d034d4d244eb01e275f5ca9cb3faa81d28c1341da6f01c5841e193 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:40:25.519 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:25 smithi092 podman[78746]: 2024-09-11 08:40:25.360313872 +0000 UTC m=+0.862534205 container cleanup afe2c4d9d2d034d4d244eb01e275f5ca9cb3faa81d28c1341da6f01c5841e193 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:40:25.519 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:25 smithi092 bash[78746]: ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T08:40:25.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:25 smithi092 ceph-mon[29647]: 5.13 scrub starts 2024-09-11T08:40:25.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:25 smithi092 ceph-mon[29647]: 5.13 scrub ok 2024-09-11T08:40:25.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:25 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:40:25.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:25 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:40:25.520 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:25 smithi092 ceph-mon[41114]: 5.13 scrub starts 2024-09-11T08:40:25.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:25 smithi092 ceph-mon[41114]: 5.13 scrub ok 2024-09-11T08:40:25.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:25 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:40:25.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:25 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:40:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:25 smithi145 ceph-mon[33384]: 5.13 scrub starts 2024-09-11T08:40:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:25 smithi145 ceph-mon[33384]: 5.13 scrub ok 2024-09-11T08:40:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:25 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:40:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:25 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:40:26.066 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:25 smithi092 podman[78791]: 2024-09-11 08:40:25.805940666 +0000 UTC m=+0.834105622 container remove afe2c4d9d2d034d4d244eb01e275f5ca9cb3faa81d28c1341da6f01c5841e193 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:40:26.066 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:25 smithi092 podman[78791]: 2024-09-11 08:40:25.898342503 +0000 UTC m=+0.926507462 volume remove dcd905fe3f0579ce13ab5ab45e6fd49363fcfec4d63c36232e82d49b660472b3 2024-09-11T08:40:26.366 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:40:26.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:26 smithi092 ceph-mon[29647]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 115 B/s wr, 3 op/s; 24 B/s, 1 objects/s recovering 2024-09-11T08:40:26.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:26 smithi092 ceph-mon[29647]: 5.14 scrub starts 2024-09-11T08:40:26.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:26 smithi092 ceph-mon[29647]: 5.14 scrub ok 2024-09-11T08:40:26.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:26 smithi092 ceph-mon[29647]: osdmap e82: 8 total, 8 up, 8 in 2024-09-11T08:40:26.367 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:26 smithi092 ceph-mon[41114]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 115 B/s wr, 3 op/s; 24 B/s, 1 objects/s recovering 2024-09-11T08:40:26.367 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:26 smithi092 ceph-mon[41114]: 5.14 scrub starts 2024-09-11T08:40:26.367 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:26 smithi092 ceph-mon[41114]: 5.14 scrub ok 2024-09-11T08:40:26.367 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:26 smithi092 ceph-mon[41114]: osdmap e82: 8 total, 8 up, 8 in 2024-09-11T08:40:26.368 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:26 smithi092 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T08:40:26.368 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:26 smithi092 systemd[1]: Stopped Ceph alertmanager.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:40:26.368 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:26 smithi092 systemd[1]: Starting Ceph alertmanager.a for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T08:40:26.372 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:40:26.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:26 smithi145 ceph-mon[33384]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 115 B/s wr, 3 op/s; 24 B/s, 1 objects/s recovering 2024-09-11T08:40:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:26 smithi145 ceph-mon[33384]: 5.14 scrub starts 2024-09-11T08:40:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:26 smithi145 ceph-mon[33384]: 5.14 scrub ok 2024-09-11T08:40:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:26 smithi145 ceph-mon[33384]: osdmap e82: 8 total, 8 up, 8 in 2024-09-11T08:40:26.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:40:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:40:27.097 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-09-11T08:40:26.224001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":953,"num_read_kb":810,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":580,"ondisk_log_size":580,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2386068,"kb_used_data":7228,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575148,"statfs":{"total":767960285184,"available":765516951552,"internally_reserved":0,"allocated":7401472,"data_stored":4557888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.039876"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.435800+0000","last_change":"2024-09-11T08:39:47.339366+0000","last_active":"2024-09-11T08:40:17.435800+0000","last_peered":"2024-09-11T08:40:17.435800+0000","last_clean":"2024-09-11T08:40:17.435800+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:40:17.435800+0000","last_undegraded":"2024-09-11T08:40:17.435800+0000","last_fullsized":"2024-09-11T08:40:17.435800+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.339312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.339312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.154202+0000","scrub_duration":0.001323095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":63,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.184912+0000","last_change":"2024-09-11T08:39:56.666764+0000","last_active":"2024-09-11T08:40:17.184912+0000","last_peered":"2024-09-11T08:40:17.184912+0000","last_clean":"2024-09-11T08:40:17.184912+0000","last_became_active":"2024-09-11T08:38:40.973398+0000","last_became_peered":"2024-09-11T08:38:40.973398+0000","last_unstale":"2024-09-11T08:40:17.184912+0000","last_undegraded":"2024-09-11T08:40:17.184912+0000","last_fullsized":"2024-09-11T08:40:17.184912+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.666703+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.544222+0000","scrub_duration":0.0041022000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.437124+0000","last_change":"2024-09-11T08:40:12.325099+0000","last_active":"2024-09-11T08:40:17.437124+0000","last_peered":"2024-09-11T08:40:17.437124+0000","last_clean":"2024-09-11T08:40:17.437124+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:40:17.437124+0000","last_undegraded":"2024-09-11T08:40:17.437124+0000","last_fullsized":"2024-09-11T08:40:17.437124+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:12.325037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:12.325037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:13.497524+0000","scrub_duration":0.001599736,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185087+0000","last_change":"2024-09-11T08:38:44.992111+0000","last_active":"2024-09-11T08:40:17.185087+0000","last_peered":"2024-09-11T08:40:17.185087+0000","last_clean":"2024-09-11T08:40:17.185087+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:40:17.185087+0000","last_undegraded":"2024-09-11T08:40:17.185087+0000","last_fullsized":"2024-09-11T08:40:17.185087+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185156+0000","last_change":"2024-09-11T08:40:06.173950+0000","last_active":"2024-09-11T08:40:17.185156+0000","last_peered":"2024-09-11T08:40:17.185156+0000","last_clean":"2024-09-11T08:40:17.185156+0000","last_became_active":"2024-09-11T08:38:45.184215+0000","last_became_peered":"2024-09-11T08:38:45.184215+0000","last_unstale":"2024-09-11T08:40:17.185156+0000","last_undegraded":"2024-09-11T08:40:17.185156+0000","last_fullsized":"2024-09-11T08:40:17.185156+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:06.173882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:06.173882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:17.036842+0000","scrub_duration":0.001730875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185751+0000","last_change":"2024-09-11T08:39:58.161055+0000","last_active":"2024-09-11T08:40:17.185751+0000","last_peered":"2024-09-11T08:40:17.185751+0000","last_clean":"2024-09-11T08:40:17.185751+0000","last_became_active":"2024-09-11T08:38:38.949136+0000","last_became_peered":"2024-09-11T08:38:38.949136+0000","last_unstale":"2024-09-11T08:40:17.185751+0000","last_undegraded":"2024-09-11T08:40:17.185751+0000","last_fullsized":"2024-09-11T08:40:17.185751+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:58.160984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.160984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:00.523696+0000","scrub_duration":0.0015109679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":74,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185361+0000","last_change":"2024-09-11T08:39:45.997873+0000","last_active":"2024-09-11T08:40:17.185361+0000","last_peered":"2024-09-11T08:40:17.185361+0000","last_clean":"2024-09-11T08:40:17.185361+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:40:17.185361+0000","last_undegraded":"2024-09-11T08:40:17.185361+0000","last_fullsized":"2024-09-11T08:40:17.185361+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.997803+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.534414+0000","scrub_duration":0.0046870369999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.184886+0000","last_change":"2024-09-11T08:38:42.983653+0000","last_active":"2024-09-11T08:40:17.184886+0000","last_peered":"2024-09-11T08:40:17.184886+0000","last_clean":"2024-09-11T08:40:17.184886+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:40:17.184886+0000","last_undegraded":"2024-09-11T08:40:17.184886+0000","last_fullsized":"2024-09-11T08:40:17.184886+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585680+0000","last_change":"2024-09-11T08:40:08.416577+0000","last_active":"2024-09-11T08:40:17.585680+0000","last_peered":"2024-09-11T08:40:17.585680+0000","last_clean":"2024-09-11T08:40:17.585680+0000","last_became_active":"2024-09-11T08:38:38.948930+0000","last_became_peered":"2024-09-11T08:38:38.948930+0000","last_unstale":"2024-09-11T08:40:17.585680+0000","last_undegraded":"2024-09-11T08:40:17.585680+0000","last_fullsized":"2024-09-11T08:40:17.585680+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:08.416522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.416522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:27.445091+0000","scrub_duration":0.001330005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":63,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185438+0000","last_change":"2024-09-11T08:39:46.620700+0000","last_active":"2024-09-11T08:40:17.185438+0000","last_peered":"2024-09-11T08:40:17.185438+0000","last_clean":"2024-09-11T08:40:17.185438+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:40:17.185438+0000","last_undegraded":"2024-09-11T08:40:17.185438+0000","last_fullsized":"2024-09-11T08:40:17.185438+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:39:46.620624+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:10.984564+0000","scrub_duration":0.0044132319999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585251+0000","last_change":"2024-09-11T08:40:04.450336+0000","last_active":"2024-09-11T08:40:17.585251+0000","last_peered":"2024-09-11T08:40:17.585251+0000","last_clean":"2024-09-11T08:40:17.585251+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:40:17.585251+0000","last_undegraded":"2024-09-11T08:40:17.585251+0000","last_fullsized":"2024-09-11T08:40:17.585251+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:04.450282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:04.450282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.499239+0000","scrub_duration":0.001689433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.435309+0000","last_change":"2024-09-11T08:39:51.280365+0000","last_active":"2024-09-11T08:40:17.435309+0000","last_peered":"2024-09-11T08:40:17.435309+0000","last_clean":"2024-09-11T08:40:17.435309+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:40:17.435309+0000","last_undegraded":"2024-09-11T08:40:17.435309+0000","last_fullsized":"2024-09-11T08:40:17.435309+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.280310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.280310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:06.712880+0000","scrub_duration":0.001634777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585449+0000","last_change":"2024-09-11T08:40:12.372128+0000","last_active":"2024-09-11T08:40:17.585449+0000","last_peered":"2024-09-11T08:40:17.585449+0000","last_clean":"2024-09-11T08:40:17.585449+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:40:17.585449+0000","last_undegraded":"2024-09-11T08:40:17.585449+0000","last_fullsized":"2024-09-11T08:40:17.585449+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:12.372043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:12.372043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.497179+0000","scrub_duration":0.001440623,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185090+0000","last_change":"2024-09-11T08:38:40.974291+0000","last_active":"2024-09-11T08:40:17.185090+0000","last_peered":"2024-09-11T08:40:17.185090+0000","last_clean":"2024-09-11T08:40:17.185090+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:40:17.185090+0000","last_undegraded":"2024-09-11T08:40:17.185090+0000","last_fullsized":"2024-09-11T08:40:17.185090+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.465247+0000","last_change":"2024-09-11T08:39:51.166114+0000","last_active":"2024-09-11T08:40:18.465247+0000","last_peered":"2024-09-11T08:40:18.465247+0000","last_clean":"2024-09-11T08:40:18.465247+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:40:18.465247+0000","last_undegraded":"2024-09-11T08:40:18.465247+0000","last_fullsized":"2024-09-11T08:40:18.465247+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.166051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.570790+0000","scrub_duration":0.0016440420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585165+0000","last_change":"2024-09-11T08:38:44.994259+0000","last_active":"2024-09-11T08:40:17.585165+0000","last_peered":"2024-09-11T08:40:17.585165+0000","last_clean":"2024-09-11T08:40:17.585165+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:40:17.585165+0000","last_undegraded":"2024-09-11T08:40:17.585165+0000","last_fullsized":"2024-09-11T08:40:17.585165+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185494+0000","last_change":"2024-09-11T08:39:59.606521+0000","last_active":"2024-09-11T08:40:17.185494+0000","last_peered":"2024-09-11T08:40:17.185494+0000","last_clean":"2024-09-11T08:40:17.185494+0000","last_became_active":"2024-09-11T08:38:45.183364+0000","last_became_peered":"2024-09-11T08:38:45.183364+0000","last_unstale":"2024-09-11T08:40:17.185494+0000","last_undegraded":"2024-09-11T08:40:17.185494+0000","last_fullsized":"2024-09-11T08:40:17.185494+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:59.606458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:59.606458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.547769+0000","scrub_duration":0.0012065960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.435779+0000","last_change":"2024-09-11T08:38:38.954075+0000","last_active":"2024-09-11T08:40:17.435779+0000","last_peered":"2024-09-11T08:40:17.435779+0000","last_clean":"2024-09-11T08:40:17.435779+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:40:17.435779+0000","last_undegraded":"2024-09-11T08:40:17.435779+0000","last_fullsized":"2024-09-11T08:40:17.435779+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":75,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.813052+0000","last_change":"2024-09-11T08:39:50.472419+0000","last_active":"2024-09-11T08:40:17.813052+0000","last_peered":"2024-09-11T08:40:17.813052+0000","last_clean":"2024-09-11T08:40:17.813052+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:40:17.813052+0000","last_undegraded":"2024-09-11T08:40:17.813052+0000","last_fullsized":"2024-09-11T08:40:17.813052+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.472368+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:10.784548+0000","scrub_duration":0.0049809520000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.437548+0000","last_change":"2024-09-11T08:39:44.632369+0000","last_active":"2024-09-11T08:40:17.437548+0000","last_peered":"2024-09-11T08:40:17.437548+0000","last_clean":"2024-09-11T08:40:17.437548+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:40:17.437548+0000","last_undegraded":"2024-09-11T08:40:17.437548+0000","last_fullsized":"2024-09-11T08:40:17.437548+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:44.632308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:44.632308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:56.408810+0000","scrub_duration":0.001420377,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185892+0000","last_change":"2024-09-11T08:38:45.183646+0000","last_active":"2024-09-11T08:40:17.185892+0000","last_peered":"2024-09-11T08:40:17.185892+0000","last_clean":"2024-09-11T08:40:17.185892+0000","last_became_active":"2024-09-11T08:38:45.182763+0000","last_became_peered":"2024-09-11T08:38:45.182763+0000","last_unstale":"2024-09-11T08:40:17.185892+0000","last_undegraded":"2024-09-11T08:40:17.185892+0000","last_fullsized":"2024-09-11T08:40:17.185892+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185540+0000","last_change":"2024-09-11T08:38:38.949654+0000","last_active":"2024-09-11T08:40:17.185540+0000","last_peered":"2024-09-11T08:40:17.185540+0000","last_clean":"2024-09-11T08:40:17.185540+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:40:17.185540+0000","last_undegraded":"2024-09-11T08:40:17.185540+0000","last_fullsized":"2024-09-11T08:40:17.185540+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":65,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185552+0000","last_change":"2024-09-11T08:39:55.125449+0000","last_active":"2024-09-11T08:40:17.185552+0000","last_peered":"2024-09-11T08:40:17.185552+0000","last_clean":"2024-09-11T08:40:17.185552+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:40:17.185552+0000","last_undegraded":"2024-09-11T08:40:17.185552+0000","last_fullsized":"2024-09-11T08:40:17.185552+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:55.125394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.125394+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.857707+0000","scrub_duration":0.0016833810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185184+0000","last_change":"2024-09-11T08:40:01.585472+0000","last_active":"2024-09-11T08:40:17.185184+0000","last_peered":"2024-09-11T08:40:17.185184+0000","last_clean":"2024-09-11T08:40:17.185184+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:40:17.185184+0000","last_undegraded":"2024-09-11T08:40:17.185184+0000","last_fullsized":"2024-09-11T08:40:17.185184+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:01.585411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:01.585411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.937039+0000","scrub_duration":0.0021220179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.465620+0000","last_change":"2024-09-11T08:40:10.113729+0000","last_active":"2024-09-11T08:40:18.465620+0000","last_peered":"2024-09-11T08:40:18.465620+0000","last_clean":"2024-09-11T08:40:18.465620+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:40:18.465620+0000","last_undegraded":"2024-09-11T08:40:18.465620+0000","last_fullsized":"2024-09-11T08:40:18.465620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:40:10.113671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:10.113671+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:33.267918+0000","scrub_duration":0.001621302,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.437202+0000","last_change":"2024-09-11T08:40:02.458618+0000","last_active":"2024-09-11T08:40:17.437202+0000","last_peered":"2024-09-11T08:40:17.437202+0000","last_clean":"2024-09-11T08:40:17.437202+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:40:17.437202+0000","last_undegraded":"2024-09-11T08:40:17.437202+0000","last_fullsized":"2024-09-11T08:40:17.437202+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:02.458565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.458565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.247079+0000","scrub_duration":0.001315568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":58,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.467716+0000","last_change":"2024-09-11T08:40:18.467638+0000","last_active":"2024-09-11T08:40:18.467716+0000","last_peered":"2024-09-11T08:40:18.467716+0000","last_clean":"2024-09-11T08:40:18.467716+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:40:18.467716+0000","last_undegraded":"2024-09-11T08:40:18.467716+0000","last_fullsized":"2024-09-11T08:40:18.467716+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:40:18.467574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:18.467574+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:37.429174+0000","scrub_duration":0.0040976820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":72,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:22.693541+0000","last_change":"2024-09-11T08:40:12.584703+0000","last_active":"2024-09-11T08:40:22.693541+0000","last_peered":"2024-09-11T08:40:22.693541+0000","last_clean":"2024-09-11T08:40:22.693541+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:40:22.693541+0000","last_undegraded":"2024-09-11T08:40:22.693541+0000","last_fullsized":"2024-09-11T08:40:22.693541+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:12.584631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:12.584631+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:27.946233+0000","scrub_duration":0.0015754790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.437701+0000","last_change":"2024-09-11T08:40:16.290927+0000","last_active":"2024-09-11T08:40:17.437701+0000","last_peered":"2024-09-11T08:40:17.437701+0000","last_clean":"2024-09-11T08:40:17.437701+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:40:17.437701+0000","last_undegraded":"2024-09-11T08:40:17.437701+0000","last_fullsized":"2024-09-11T08:40:17.437701+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-11T08:40:16.290866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:16.290866+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:46.083697+0000","scrub_duration":0.0021215069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:23.048078+0000","last_change":"2024-09-11T08:40:23.048021+0000","last_active":"2024-09-11T08:40:23.048078+0000","last_peered":"2024-09-11T08:40:23.048078+0000","last_clean":"2024-09-11T08:40:23.048078+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:40:23.048078+0000","last_undegraded":"2024-09-11T08:40:23.048078+0000","last_fullsized":"2024-09-11T08:40:23.048078+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:23.047963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:23.047963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.577363+0000","scrub_duration":0.00148653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":70,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:22.693583+0000","last_change":"2024-09-11T08:40:11.625218+0000","last_active":"2024-09-11T08:40:22.693583+0000","last_peered":"2024-09-11T08:40:22.693583+0000","last_clean":"2024-09-11T08:40:22.693583+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:40:22.693583+0000","last_undegraded":"2024-09-11T08:40:22.693583+0000","last_fullsized":"2024-09-11T08:40:22.693583+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:11.625164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:11.625164+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.094157+0000","scrub_duration":0.0014996320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585455+0000","last_change":"2024-09-11T08:38:44.991592+0000","last_active":"2024-09-11T08:40:17.585455+0000","last_peered":"2024-09-11T08:40:17.585455+0000","last_clean":"2024-09-11T08:40:17.585455+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:40:17.585455+0000","last_undegraded":"2024-09-11T08:40:17.585455+0000","last_fullsized":"2024-09-11T08:40:17.585455+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":74,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185097+0000","last_change":"2024-09-11T08:40:05.197208+0000","last_active":"2024-09-11T08:40:17.185097+0000","last_peered":"2024-09-11T08:40:17.185097+0000","last_clean":"2024-09-11T08:40:17.185097+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:40:17.185097+0000","last_undegraded":"2024-09-11T08:40:17.185097+0000","last_fullsized":"2024-09-11T08:40:17.185097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-11T08:40:05.197152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:05.197152+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:06.174279+0000","scrub_duration":0.0016976000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.186054+0000","last_change":"2024-09-11T08:39:43.037605+0000","last_active":"2024-09-11T08:40:17.186054+0000","last_peered":"2024-09-11T08:40:17.186054+0000","last_clean":"2024-09-11T08:40:17.186054+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:40:17.186054+0000","last_undegraded":"2024-09-11T08:40:17.186054+0000","last_fullsized":"2024-09-11T08:40:17.186054+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.037551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.037551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.071661+0000","scrub_duration":0.001523976,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:19.689020+0000","last_change":"2024-09-11T08:40:19.688943+0000","last_active":"2024-09-11T08:40:19.689020+0000","last_peered":"2024-09-11T08:40:19.689020+0000","last_clean":"2024-09-11T08:40:19.689020+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:40:19.689020+0000","last_undegraded":"2024-09-11T08:40:19.689020+0000","last_fullsized":"2024-09-11T08:40:19.689020+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:19.688876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:19.688876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.993714+0000","scrub_duration":0.0014001599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185262+0000","last_change":"2024-09-11T08:38:44.995555+0000","last_active":"2024-09-11T08:40:17.185262+0000","last_peered":"2024-09-11T08:40:17.185262+0000","last_clean":"2024-09-11T08:40:17.185262+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:40:17.185262+0000","last_undegraded":"2024-09-11T08:40:17.185262+0000","last_fullsized":"2024-09-11T08:40:17.185262+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":54,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185147+0000","last_change":"2024-09-11T08:38:40.968377+0000","last_active":"2024-09-11T08:40:17.185147+0000","last_peered":"2024-09-11T08:40:17.185147+0000","last_clean":"2024-09-11T08:40:17.185147+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:40:17.185147+0000","last_undegraded":"2024-09-11T08:40:17.185147+0000","last_fullsized":"2024-09-11T08:40:17.185147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.435817+0000","last_change":"2024-09-11T08:39:41.267320+0000","last_active":"2024-09-11T08:40:17.435817+0000","last_peered":"2024-09-11T08:40:17.435817+0000","last_clean":"2024-09-11T08:40:17.435817+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:40:17.435817+0000","last_undegraded":"2024-09-11T08:40:17.435817+0000","last_fullsized":"2024-09-11T08:40:17.435817+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:41.267267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.267267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:01.547087+0000","scrub_duration":0.0013348209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:22.693306+0000","last_change":"2024-09-11T08:38:42.982794+0000","last_active":"2024-09-11T08:40:22.693306+0000","last_peered":"2024-09-11T08:40:22.693306+0000","last_clean":"2024-09-11T08:40:22.693306+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:40:22.693306+0000","last_undegraded":"2024-09-11T08:40:22.693306+0000","last_fullsized":"2024-09-11T08:40:22.693306+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.465336+0000","last_change":"2024-09-11T08:40:04.174461+0000","last_active":"2024-09-11T08:40:18.465336+0000","last_peered":"2024-09-11T08:40:18.465336+0000","last_clean":"2024-09-11T08:40:18.465336+0000","last_became_active":"2024-09-11T08:38:45.182724+0000","last_became_peered":"2024-09-11T08:38:45.182724+0000","last_unstale":"2024-09-11T08:40:18.465336+0000","last_undegraded":"2024-09-11T08:40:18.465336+0000","last_fullsized":"2024-09-11T08:40:18.465336+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:04.174409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:04.174409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.916947+0000","scrub_duration":0.0012996889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"63'10","reported_seq":58,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.437239+0000","last_change":"2024-09-11T08:40:13.327302+0000","last_active":"2024-09-11T08:40:17.437239+0000","last_peered":"2024-09-11T08:40:17.437239+0000","last_clean":"2024-09-11T08:40:17.437239+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:40:17.437239+0000","last_undegraded":"2024-09-11T08:40:17.437239+0000","last_fullsized":"2024-09-11T08:40:17.437239+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:40:13.327245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:13.327245+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.332692+0000","scrub_duration":0.0017714300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585737+0000","last_change":"2024-09-11T08:38:38.951168+0000","last_active":"2024-09-11T08:40:17.585737+0000","last_peered":"2024-09-11T08:40:17.585737+0000","last_clean":"2024-09-11T08:40:17.585737+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:40:17.585737+0000","last_undegraded":"2024-09-11T08:40:17.585737+0000","last_fullsized":"2024-09-11T08:40:17.585737+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":60,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:22.693320+0000","last_change":"2024-09-11T08:38:42.981732+0000","last_active":"2024-09-11T08:40:22.693320+0000","last_peered":"2024-09-11T08:40:22.693320+0000","last_clean":"2024-09-11T08:40:22.693320+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:40:22.693320+0000","last_undegraded":"2024-09-11T08:40:22.693320+0000","last_fullsized":"2024-09-11T08:40:22.693320+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.435483+0000","last_change":"2024-09-11T08:38:44.998257+0000","last_active":"2024-09-11T08:40:17.435483+0000","last_peered":"2024-09-11T08:40:17.435483+0000","last_clean":"2024-09-11T08:40:17.435483+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:40:17.435483+0000","last_undegraded":"2024-09-11T08:40:17.435483+0000","last_fullsized":"2024-09-11T08:40:17.435483+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":69,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.435674+0000","last_change":"2024-09-11T08:38:41.040034+0000","last_active":"2024-09-11T08:40:17.435674+0000","last_peered":"2024-09-11T08:40:17.435674+0000","last_clean":"2024-09-11T08:40:17.435674+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:40:17.435674+0000","last_undegraded":"2024-09-11T08:40:17.435674+0000","last_fullsized":"2024-09-11T08:40:17.435674+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.437571+0000","last_change":"2024-09-11T08:40:03.496005+0000","last_active":"2024-09-11T08:40:17.437571+0000","last_peered":"2024-09-11T08:40:17.437571+0000","last_clean":"2024-09-11T08:40:17.437571+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:40:17.437571+0000","last_undegraded":"2024-09-11T08:40:17.437571+0000","last_fullsized":"2024-09-11T08:40:17.437571+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:03.495950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.495950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.342754+0000","scrub_duration":0.001379314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.187701+0000","last_change":"2024-09-11T08:39:57.596980+0000","last_active":"2024-09-11T08:40:17.187701+0000","last_peered":"2024-09-11T08:40:17.187701+0000","last_clean":"2024-09-11T08:40:17.187701+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:40:17.187701+0000","last_undegraded":"2024-09-11T08:40:17.187701+0000","last_fullsized":"2024-09-11T08:40:17.187701+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.596889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.648355+0000","scrub_duration":0.001574424,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.437053+0000","last_change":"2024-09-11T08:40:08.387078+0000","last_active":"2024-09-11T08:40:17.437053+0000","last_peered":"2024-09-11T08:40:17.437053+0000","last_clean":"2024-09-11T08:40:17.437053+0000","last_became_active":"2024-09-11T08:38:45.182782+0000","last_became_peered":"2024-09-11T08:38:45.182782+0000","last_unstale":"2024-09-11T08:40:17.437053+0000","last_undegraded":"2024-09-11T08:40:17.437053+0000","last_fullsized":"2024-09-11T08:40:17.437053+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:08.387027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.387027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.887729+0000","scrub_duration":0.001411336,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":122,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:22.693309+0000","last_change":"2024-09-11T08:39:53.493004+0000","last_active":"2024-09-11T08:40:22.693309+0000","last_peered":"2024-09-11T08:40:22.693309+0000","last_clean":"2024-09-11T08:40:22.693309+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:40:22.693309+0000","last_undegraded":"2024-09-11T08:40:22.693309+0000","last_fullsized":"2024-09-11T08:40:22.693309+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.492945+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.774847+0000","scrub_duration":0.005988045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:22.750862+0000","last_change":"2024-09-11T08:40:22.750815+0000","last_active":"2024-09-11T08:40:22.750862+0000","last_peered":"2024-09-11T08:40:22.750862+0000","last_clean":"2024-09-11T08:40:22.750862+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:40:22.750862+0000","last_undegraded":"2024-09-11T08:40:22.750862+0000","last_fullsized":"2024-09-11T08:40:22.750862+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:22.750744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.750744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.795298+0000","scrub_duration":0.001679119,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585422+0000","last_change":"2024-09-11T08:39:58.472851+0000","last_active":"2024-09-11T08:40:17.585422+0000","last_peered":"2024-09-11T08:40:17.585422+0000","last_clean":"2024-09-11T08:40:17.585422+0000","last_became_active":"2024-09-11T08:38:38.947270+0000","last_became_peered":"2024-09-11T08:38:38.947270+0000","last_unstale":"2024-09-11T08:40:17.585422+0000","last_undegraded":"2024-09-11T08:40:17.585422+0000","last_fullsized":"2024-09-11T08:40:17.585422+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:39:58.472785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.472785+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.725781+0000","scrub_duration":0.0020806930000000002,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.436078+0000","last_change":"2024-09-11T08:40:17.435980+0000","last_active":"2024-09-11T08:40:17.436078+0000","last_peered":"2024-09-11T08:40:17.436078+0000","last_clean":"2024-09-11T08:40:17.436078+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:40:17.436078+0000","last_undegraded":"2024-09-11T08:40:17.436078+0000","last_fullsized":"2024-09-11T08:40:17.436078+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:17.435900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:17.435900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.732279+0000","scrub_duration":0.0018276379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185615+0000","last_change":"2024-09-11T08:38:44.997146+0000","last_active":"2024-09-11T08:40:17.185615+0000","last_peered":"2024-09-11T08:40:17.185615+0000","last_clean":"2024-09-11T08:40:17.185615+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:40:17.185615+0000","last_undegraded":"2024-09-11T08:40:17.185615+0000","last_fullsized":"2024-09-11T08:40:17.185615+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":70,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.437735+0000","last_change":"2024-09-11T08:39:43.585175+0000","last_active":"2024-09-11T08:40:17.437735+0000","last_peered":"2024-09-11T08:40:17.437735+0000","last_clean":"2024-09-11T08:40:17.437735+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:40:17.437735+0000","last_undegraded":"2024-09-11T08:40:17.437735+0000","last_fullsized":"2024-09-11T08:40:17.437735+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:39:43.585116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.585116+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.724395+0000","scrub_duration":0.0021210309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.437305+0000","last_change":"2024-09-11T08:40:00.500444+0000","last_active":"2024-09-11T08:40:17.437305+0000","last_peered":"2024-09-11T08:40:17.437305+0000","last_clean":"2024-09-11T08:40:17.437305+0000","last_became_active":"2024-09-11T08:38:38.949019+0000","last_became_peered":"2024-09-11T08:38:38.949019+0000","last_unstale":"2024-09-11T08:40:17.437305+0000","last_undegraded":"2024-09-11T08:40:17.437305+0000","last_fullsized":"2024-09-11T08:40:17.437305+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.500392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.500392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.354187+0000","scrub_duration":0.001294566,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.812736+0000","last_change":"2024-09-11T08:40:01.488539+0000","last_active":"2024-09-11T08:40:17.812736+0000","last_peered":"2024-09-11T08:40:17.812736+0000","last_clean":"2024-09-11T08:40:17.812736+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:40:17.812736+0000","last_undegraded":"2024-09-11T08:40:17.812736+0000","last_fullsized":"2024-09-11T08:40:17.812736+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:40:01.488482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:01.488482+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.917024+0000","scrub_duration":0.001463422,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.813076+0000","last_change":"2024-09-11T08:40:02.444341+0000","last_active":"2024-09-11T08:40:17.813076+0000","last_peered":"2024-09-11T08:40:17.813076+0000","last_clean":"2024-09-11T08:40:17.813076+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:40:17.813076+0000","last_undegraded":"2024-09-11T08:40:17.813076+0000","last_fullsized":"2024-09-11T08:40:17.813076+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.444279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.886414+0000","scrub_duration":0.001414329,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.465617+0000","last_change":"2024-09-11T08:40:15.184420+0000","last_active":"2024-09-11T08:40:18.465617+0000","last_peered":"2024-09-11T08:40:18.465617+0000","last_clean":"2024-09-11T08:40:18.465617+0000","last_became_active":"2024-09-11T08:38:45.183582+0000","last_became_peered":"2024-09-11T08:38:45.183582+0000","last_unstale":"2024-09-11T08:40:18.465617+0000","last_undegraded":"2024-09-11T08:40:18.465617+0000","last_fullsized":"2024-09-11T08:40:18.465617+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:15.184337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:15.184337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.002701+0000","scrub_duration":0.0039899209999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:21.450093+0000","last_change":"2024-09-11T08:40:21.450026+0000","last_active":"2024-09-11T08:40:21.450093+0000","last_peered":"2024-09-11T08:40:21.450093+0000","last_clean":"2024-09-11T08:40:21.450093+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:40:21.450093+0000","last_undegraded":"2024-09-11T08:40:21.450093+0000","last_fullsized":"2024-09-11T08:40:21.450093+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:40:21.449958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:21.449958+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:46.796354+0000","scrub_duration":0.0017585280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.435453+0000","last_change":"2024-09-11T08:39:43.356751+0000","last_active":"2024-09-11T08:40:17.435453+0000","last_peered":"2024-09-11T08:40:17.435453+0000","last_clean":"2024-09-11T08:40:17.435453+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:40:17.435453+0000","last_undegraded":"2024-09-11T08:40:17.435453+0000","last_fullsized":"2024-09-11T08:40:17.435453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.356699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.356699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.569637+0000","scrub_duration":0.001649455,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":152,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.272169+0000","last_change":"2024-09-11T08:39:55.153362+0000","last_active":"2024-09-11T08:40:25.272169+0000","last_peered":"2024-09-11T08:40:25.272169+0000","last_clean":"2024-09-11T08:40:25.272169+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:40:25.272169+0000","last_undegraded":"2024-09-11T08:40:25.272169+0000","last_fullsized":"2024-09-11T08:40:25.272169+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:39:55.153290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.153290+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.446909+0000","scrub_duration":0.0017161990000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":100,"num_read_kb":95,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585798+0000","last_change":"2024-09-11T08:38:42.976525+0000","last_active":"2024-09-11T08:40:17.585798+0000","last_peered":"2024-09-11T08:40:17.585798+0000","last_clean":"2024-09-11T08:40:17.585798+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:40:17.585798+0000","last_undegraded":"2024-09-11T08:40:17.585798+0000","last_fullsized":"2024-09-11T08:40:17.585798+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.437390+0000","last_change":"2024-09-11T08:40:15.280809+0000","last_active":"2024-09-11T08:40:17.437390+0000","last_peered":"2024-09-11T08:40:17.437390+0000","last_clean":"2024-09-11T08:40:17.437390+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:40:17.437390+0000","last_undegraded":"2024-09-11T08:40:17.437390+0000","last_fullsized":"2024-09-11T08:40:17.437390+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:15.280760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:15.280760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:36.262012+0000","scrub_duration":0.0012405719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":64,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.188017+0000","last_change":"2024-09-11T08:40:00.540192+0000","last_active":"2024-09-11T08:40:17.188017+0000","last_peered":"2024-09-11T08:40:17.188017+0000","last_clean":"2024-09-11T08:40:17.188017+0000","last_became_active":"2024-09-11T08:38:41.040948+0000","last_became_peered":"2024-09-11T08:38:41.040948+0000","last_unstale":"2024-09-11T08:40:17.188017+0000","last_undegraded":"2024-09-11T08:40:17.188017+0000","last_fullsized":"2024-09-11T08:40:17.188017+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:40:00.540136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.540136+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:07.651325+0000","scrub_duration":0.001785095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:21.086298+0000","last_change":"2024-09-11T08:40:21.086239+0000","last_active":"2024-09-11T08:40:21.086298+0000","last_peered":"2024-09-11T08:40:21.086298+0000","last_clean":"2024-09-11T08:40:21.086298+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:40:21.086298+0000","last_undegraded":"2024-09-11T08:40:21.086298+0000","last_fullsized":"2024-09-11T08:40:21.086298+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:40:21.086183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:21.086183+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:25.836107+0000","scrub_duration":0.001907838,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":78,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.465509+0000","last_change":"2024-09-11T08:39:12.765244+0000","last_active":"2024-09-11T08:40:18.465509+0000","last_peered":"2024-09-11T08:40:18.465509+0000","last_clean":"2024-09-11T08:40:18.465509+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:40:18.465509+0000","last_undegraded":"2024-09-11T08:40:18.465509+0000","last_fullsized":"2024-09-11T08:40:18.465509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'72","last_scrub_stamp":"2024-09-11T08:39:12.765190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:39:12.765190+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014709829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.465740+0000","last_change":"2024-09-11T08:39:49.116184+0000","last_active":"2024-09-11T08:40:18.465740+0000","last_peered":"2024-09-11T08:40:18.465740+0000","last_clean":"2024-09-11T08:40:18.465740+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:40:18.465740+0000","last_undegraded":"2024-09-11T08:40:18.465740+0000","last_fullsized":"2024-09-11T08:40:18.465740+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.116113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.116113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.209747+0000","scrub_duration":0.00167223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585755+0000","last_change":"2024-09-11T08:39:47.423745+0000","last_active":"2024-09-11T08:40:17.585755+0000","last_peered":"2024-09-11T08:40:17.585755+0000","last_clean":"2024-09-11T08:40:17.585755+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:40:17.585755+0000","last_undegraded":"2024-09-11T08:40:17.585755+0000","last_fullsized":"2024-09-11T08:40:17.585755+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.423684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.423684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.004042+0000","scrub_duration":0.001458972,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":74,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185601+0000","last_change":"2024-09-11T08:40:16.190092+0000","last_active":"2024-09-11T08:40:17.185601+0000","last_peered":"2024-09-11T08:40:17.185601+0000","last_clean":"2024-09-11T08:40:17.185601+0000","last_became_active":"2024-09-11T08:40:16.183867+0000","last_became_peered":"2024-09-11T08:40:16.183867+0000","last_unstale":"2024-09-11T08:40:17.185601+0000","last_undegraded":"2024-09-11T08:40:17.185601+0000","last_fullsized":"2024-09-11T08:40:17.185601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185339+0000","last_change":"2024-09-11T08:38:38.948244+0000","last_active":"2024-09-11T08:40:17.185339+0000","last_peered":"2024-09-11T08:40:17.185339+0000","last_clean":"2024-09-11T08:40:17.185339+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:40:17.185339+0000","last_undegraded":"2024-09-11T08:40:17.185339+0000","last_fullsized":"2024-09-11T08:40:17.185339+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585673+0000","last_change":"2024-09-11T08:40:03.430883+0000","last_active":"2024-09-11T08:40:17.585673+0000","last_peered":"2024-09-11T08:40:17.585673+0000","last_clean":"2024-09-11T08:40:17.585673+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:40:17.585673+0000","last_undegraded":"2024-09-11T08:40:17.585673+0000","last_fullsized":"2024-09-11T08:40:17.585673+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:03.430794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.430794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:20.081896+0000","scrub_duration":0.0014294430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185384+0000","last_change":"2024-09-11T08:38:45.185064+0000","last_active":"2024-09-11T08:40:17.185384+0000","last_peered":"2024-09-11T08:40:17.185384+0000","last_clean":"2024-09-11T08:40:17.185384+0000","last_became_active":"2024-09-11T08:38:45.184410+0000","last_became_peered":"2024-09-11T08:38:45.184410+0000","last_unstale":"2024-09-11T08:40:17.185384+0000","last_undegraded":"2024-09-11T08:40:17.185384+0000","last_fullsized":"2024-09-11T08:40:17.185384+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":71,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185648+0000","last_change":"2024-09-11T08:39:43.526845+0000","last_active":"2024-09-11T08:40:17.185648+0000","last_peered":"2024-09-11T08:40:17.185648+0000","last_clean":"2024-09-11T08:40:17.185648+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:40:17.185648+0000","last_undegraded":"2024-09-11T08:40:17.185648+0000","last_fullsized":"2024-09-11T08:40:17.185648+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:39:43.526763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.526763+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.869811+0000","scrub_duration":0.0022037329999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.187882+0000","last_change":"2024-09-11T08:39:55.599049+0000","last_active":"2024-09-11T08:40:17.187882+0000","last_peered":"2024-09-11T08:40:17.187882+0000","last_clean":"2024-09-11T08:40:17.187882+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:40:17.187882+0000","last_undegraded":"2024-09-11T08:40:17.187882+0000","last_fullsized":"2024-09-11T08:40:17.187882+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.598961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.598961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.562190+0000","scrub_duration":0.0016789820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.437384+0000","last_change":"2024-09-11T08:39:46.627077+0000","last_active":"2024-09-11T08:40:17.437384+0000","last_peered":"2024-09-11T08:40:17.437384+0000","last_clean":"2024-09-11T08:40:17.437384+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:40:17.437384+0000","last_undegraded":"2024-09-11T08:40:17.437384+0000","last_fullsized":"2024-09-11T08:40:17.437384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:46.627028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:46.627028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:56.334041+0000","scrub_duration":0.0013824029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.813089+0000","last_change":"2024-09-11T08:40:13.462183+0000","last_active":"2024-09-11T08:40:17.813089+0000","last_peered":"2024-09-11T08:40:17.813089+0000","last_clean":"2024-09-11T08:40:17.813089+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:40:17.813089+0000","last_undegraded":"2024-09-11T08:40:17.813089+0000","last_fullsized":"2024-09-11T08:40:17.813089+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:13.462132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:13.462132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:33.998327+0000","scrub_duration":0.001365727,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185259+0000","last_change":"2024-09-11T08:38:40.969616+0000","last_active":"2024-09-11T08:40:17.185259+0000","last_peered":"2024-09-11T08:40:17.185259+0000","last_clean":"2024-09-11T08:40:17.185259+0000","last_became_active":"2024-09-11T08:38:40.969348+0000","last_became_peered":"2024-09-11T08:38:40.969348+0000","last_unstale":"2024-09-11T08:40:17.185259+0000","last_undegraded":"2024-09-11T08:40:17.185259+0000","last_fullsized":"2024-09-11T08:40:17.185259+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185501+0000","last_change":"2024-09-11T08:38:38.957214+0000","last_active":"2024-09-11T08:40:17.185501+0000","last_peered":"2024-09-11T08:40:17.185501+0000","last_clean":"2024-09-11T08:40:17.185501+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:40:17.185501+0000","last_undegraded":"2024-09-11T08:40:17.185501+0000","last_fullsized":"2024-09-11T08:40:17.185501+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":70,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:22.693313+0000","last_change":"2024-09-11T08:40:00.473606+0000","last_active":"2024-09-11T08:40:22.693313+0000","last_peered":"2024-09-11T08:40:22.693313+0000","last_clean":"2024-09-11T08:40:22.693313+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:40:22.693313+0000","last_undegraded":"2024-09-11T08:40:22.693313+0000","last_fullsized":"2024-09-11T08:40:22.693313+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:00.473549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.473549+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.300137+0000","scrub_duration":0.0016703060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185373+0000","last_change":"2024-09-11T08:38:44.993736+0000","last_active":"2024-09-11T08:40:17.185373+0000","last_peered":"2024-09-11T08:40:17.185373+0000","last_clean":"2024-09-11T08:40:17.185373+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:40:17.185373+0000","last_undegraded":"2024-09-11T08:40:17.185373+0000","last_fullsized":"2024-09-11T08:40:17.185373+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.435557+0000","last_change":"2024-09-11T08:38:40.977113+0000","last_active":"2024-09-11T08:40:17.435557+0000","last_peered":"2024-09-11T08:40:17.435557+0000","last_clean":"2024-09-11T08:40:17.435557+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:40:17.435557+0000","last_undegraded":"2024-09-11T08:40:17.435557+0000","last_fullsized":"2024-09-11T08:40:17.435557+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585568+0000","last_change":"2024-09-11T08:39:50.452568+0000","last_active":"2024-09-11T08:40:17.585568+0000","last_peered":"2024-09-11T08:40:17.585568+0000","last_clean":"2024-09-11T08:40:17.585568+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:40:17.585568+0000","last_undegraded":"2024-09-11T08:40:17.585568+0000","last_fullsized":"2024-09-11T08:40:17.585568+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.452494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.631278+0000","scrub_duration":0.001547086,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.812768+0000","last_change":"2024-09-11T08:40:09.519072+0000","last_active":"2024-09-11T08:40:17.812768+0000","last_peered":"2024-09-11T08:40:17.812768+0000","last_clean":"2024-09-11T08:40:17.812768+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:40:17.812768+0000","last_undegraded":"2024-09-11T08:40:17.812768+0000","last_fullsized":"2024-09-11T08:40:17.812768+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:09.519019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:09.519019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.000837+0000","scrub_duration":0.001232471,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.435375+0000","last_change":"2024-09-11T08:38:44.991908+0000","last_active":"2024-09-11T08:40:17.435375+0000","last_peered":"2024-09-11T08:40:17.435375+0000","last_clean":"2024-09-11T08:40:17.435375+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:40:17.435375+0000","last_undegraded":"2024-09-11T08:40:17.435375+0000","last_fullsized":"2024-09-11T08:40:17.435375+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":73,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.187733+0000","last_change":"2024-09-11T08:39:45.575209+0000","last_active":"2024-09-11T08:40:17.187733+0000","last_peered":"2024-09-11T08:40:17.187733+0000","last_clean":"2024-09-11T08:40:17.187733+0000","last_became_active":"2024-09-11T08:38:41.040207+0000","last_became_peered":"2024-09-11T08:38:41.040207+0000","last_unstale":"2024-09-11T08:40:17.187733+0000","last_undegraded":"2024-09-11T08:40:17.187733+0000","last_fullsized":"2024-09-11T08:40:17.187733+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:39:45.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.575133+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.182090+0000","scrub_duration":0.0022792160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.465825+0000","last_change":"2024-09-11T08:39:56.112112+0000","last_active":"2024-09-11T08:40:18.465825+0000","last_peered":"2024-09-11T08:40:18.465825+0000","last_clean":"2024-09-11T08:40:18.465825+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:40:18.465825+0000","last_undegraded":"2024-09-11T08:40:18.465825+0000","last_fullsized":"2024-09-11T08:40:18.465825+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:56.112043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.112043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:25.809354+0000","scrub_duration":0.001517137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.812816+0000","last_change":"2024-09-11T08:40:11.540833+0000","last_active":"2024-09-11T08:40:17.812816+0000","last_peered":"2024-09-11T08:40:17.812816+0000","last_clean":"2024-09-11T08:40:17.812816+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:40:17.812816+0000","last_undegraded":"2024-09-11T08:40:17.812816+0000","last_fullsized":"2024-09-11T08:40:17.812816+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:11.540760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:11.540760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.937538+0000","scrub_duration":0.0014212529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.465545+0000","last_change":"2024-09-11T08:38:44.993458+0000","last_active":"2024-09-11T08:40:18.465545+0000","last_peered":"2024-09-11T08:40:18.465545+0000","last_clean":"2024-09-11T08:40:18.465545+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:40:18.465545+0000","last_undegraded":"2024-09-11T08:40:18.465545+0000","last_fullsized":"2024-09-11T08:40:18.465545+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":66,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185069+0000","last_change":"2024-09-11T08:39:40.524581+0000","last_active":"2024-09-11T08:40:17.185069+0000","last_peered":"2024-09-11T08:40:17.185069+0000","last_clean":"2024-09-11T08:40:17.185069+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:40:17.185069+0000","last_undegraded":"2024-09-11T08:40:17.185069+0000","last_fullsized":"2024-09-11T08:40:17.185069+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:40.524498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:40.524498+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.186922+0000","scrub_duration":0.0024035010000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.465944+0000","last_change":"2024-09-11T08:40:00.113587+0000","last_active":"2024-09-11T08:40:18.465944+0000","last_peered":"2024-09-11T08:40:18.465944+0000","last_clean":"2024-09-11T08:40:18.465944+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:40:18.465944+0000","last_undegraded":"2024-09-11T08:40:18.465944+0000","last_fullsized":"2024-09-11T08:40:18.465944+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.113534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.113534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.476975+0000","scrub_duration":0.0011790780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.812697+0000","last_change":"2024-09-11T08:39:57.454257+0000","last_active":"2024-09-11T08:40:17.812697+0000","last_peered":"2024-09-11T08:40:17.812697+0000","last_clean":"2024-09-11T08:40:17.812697+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:40:17.812697+0000","last_undegraded":"2024-09-11T08:40:17.812697+0000","last_fullsized":"2024-09-11T08:40:17.812697+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.454198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.454198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.731898+0000","scrub_duration":0.0011666669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185235+0000","last_change":"2024-09-11T08:38:44.998686+0000","last_active":"2024-09-11T08:40:17.185235+0000","last_peered":"2024-09-11T08:40:17.185235+0000","last_clean":"2024-09-11T08:40:17.185235+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:40:17.185235+0000","last_undegraded":"2024-09-11T08:40:17.185235+0000","last_fullsized":"2024-09-11T08:40:17.185235+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585075+0000","last_change":"2024-09-11T08:39:48.462233+0000","last_active":"2024-09-11T08:40:17.585075+0000","last_peered":"2024-09-11T08:40:17.585075+0000","last_clean":"2024-09-11T08:40:17.585075+0000","last_became_active":"2024-09-11T08:38:41.040475+0000","last_became_peered":"2024-09-11T08:38:41.040475+0000","last_unstale":"2024-09-11T08:40:17.585075+0000","last_undegraded":"2024-09-11T08:40:17.585075+0000","last_fullsized":"2024-09-11T08:40:17.585075+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-11T08:39:48.462163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:48.462163+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.032207+0000","scrub_duration":0.002094753,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.466020+0000","last_change":"2024-09-11T08:40:03.143806+0000","last_active":"2024-09-11T08:40:18.466020+0000","last_peered":"2024-09-11T08:40:18.466020+0000","last_clean":"2024-09-11T08:40:18.466020+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:40:18.466020+0000","last_undegraded":"2024-09-11T08:40:18.466020+0000","last_fullsized":"2024-09-11T08:40:18.466020+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:40:03.143742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.143742+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.570435+0000","scrub_duration":0.001564396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"63'8","reported_seq":69,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:22.693550+0000","last_change":"2024-09-11T08:40:13.221708+0000","last_active":"2024-09-11T08:40:22.693550+0000","last_peered":"2024-09-11T08:40:22.693550+0000","last_clean":"2024-09-11T08:40:22.693550+0000","last_became_active":"2024-09-11T08:38:42.986756+0000","last_became_peered":"2024-09-11T08:38:42.986756+0000","last_unstale":"2024-09-11T08:40:22.693550+0000","last_undegraded":"2024-09-11T08:40:22.693550+0000","last_fullsized":"2024-09-11T08:40:22.693550+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:13.221650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:13.221650+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:28.234147+0000","scrub_duration":0.0017036169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585245+0000","last_change":"2024-09-11T08:40:11.370313+0000","last_active":"2024-09-11T08:40:17.585245+0000","last_peered":"2024-09-11T08:40:17.585245+0000","last_clean":"2024-09-11T08:40:17.585245+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:40:17.585245+0000","last_undegraded":"2024-09-11T08:40:17.585245+0000","last_fullsized":"2024-09-11T08:40:17.585245+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:11.370245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:11.370245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:35.188374+0000","scrub_duration":0.0011869179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.187642+0000","last_change":"2024-09-11T08:39:42.605373+0000","last_active":"2024-09-11T08:40:17.187642+0000","last_peered":"2024-09-11T08:40:17.187642+0000","last_clean":"2024-09-11T08:40:17.187642+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:40:17.187642+0000","last_undegraded":"2024-09-11T08:40:17.187642+0000","last_fullsized":"2024-09-11T08:40:17.187642+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:42.605301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:42.605301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:53.937210+0000","scrub_duration":0.0014798509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":69,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185979+0000","last_change":"2024-09-11T08:40:17.096092+0000","last_active":"2024-09-11T08:40:17.185979+0000","last_peered":"2024-09-11T08:40:17.185979+0000","last_clean":"2024-09-11T08:40:17.185979+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:40:17.185979+0000","last_undegraded":"2024-09-11T08:40:17.185979+0000","last_fullsized":"2024-09-11T08:40:17.185979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-11T08:40:17.096034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:17.096034+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:43.002314+0000","scrub_duration":0.0019684910000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585634+0000","last_change":"2024-09-11T08:39:49.472764+0000","last_active":"2024-09-11T08:40:17.585634+0000","last_peered":"2024-09-11T08:40:17.585634+0000","last_clean":"2024-09-11T08:40:17.585634+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:40:17.585634+0000","last_undegraded":"2024-09-11T08:40:17.585634+0000","last_fullsized":"2024-09-11T08:40:17.585634+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.472696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.472696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:01.403308+0000","scrub_duration":0.001919104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.465513+0000","last_change":"2024-09-11T08:38:45.185016+0000","last_active":"2024-09-11T08:40:18.465513+0000","last_peered":"2024-09-11T08:40:18.465513+0000","last_clean":"2024-09-11T08:40:18.465513+0000","last_became_active":"2024-09-11T08:38:45.184753+0000","last_became_peered":"2024-09-11T08:38:45.184753+0000","last_unstale":"2024-09-11T08:40:18.465513+0000","last_undegraded":"2024-09-11T08:40:18.465513+0000","last_fullsized":"2024-09-11T08:40:18.465513+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.435760+0000","last_change":"2024-09-11T08:38:41.041476+0000","last_active":"2024-09-11T08:40:17.435760+0000","last_peered":"2024-09-11T08:40:17.435760+0000","last_clean":"2024-09-11T08:40:17.435760+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:40:17.435760+0000","last_undegraded":"2024-09-11T08:40:17.435760+0000","last_fullsized":"2024-09-11T08:40:17.435760+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.465406+0000","last_change":"2024-09-11T08:39:53.176095+0000","last_active":"2024-09-11T08:40:18.465406+0000","last_peered":"2024-09-11T08:40:18.465406+0000","last_clean":"2024-09-11T08:40:18.465406+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:40:18.465406+0000","last_undegraded":"2024-09-11T08:40:18.465406+0000","last_fullsized":"2024-09-11T08:40:18.465406+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.176035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.176035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:00.682857+0000","scrub_duration":0.001147531,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185888+0000","last_change":"2024-09-11T08:38:42.982083+0000","last_active":"2024-09-11T08:40:17.185888+0000","last_peered":"2024-09-11T08:40:17.185888+0000","last_clean":"2024-09-11T08:40:17.185888+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:40:17.185888+0000","last_undegraded":"2024-09-11T08:40:17.185888+0000","last_fullsized":"2024-09-11T08:40:17.185888+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.812817+0000","last_change":"2024-09-11T08:39:52.493064+0000","last_active":"2024-09-11T08:40:17.812817+0000","last_peered":"2024-09-11T08:40:17.812817+0000","last_clean":"2024-09-11T08:40:17.812817+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:40:17.812817+0000","last_undegraded":"2024-09-11T08:40:17.812817+0000","last_fullsized":"2024-09-11T08:40:17.812817+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.493013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:13.717323+0000","scrub_duration":0.0012699160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585627+0000","last_change":"2024-09-11T08:38:41.042115+0000","last_active":"2024-09-11T08:40:17.585627+0000","last_peered":"2024-09-11T08:40:17.585627+0000","last_clean":"2024-09-11T08:40:17.585627+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:40:17.585627+0000","last_undegraded":"2024-09-11T08:40:17.585627+0000","last_fullsized":"2024-09-11T08:40:17.585627+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:20.467562+0000","last_change":"2024-09-11T08:40:20.467510+0000","last_active":"2024-09-11T08:40:20.467562+0000","last_peered":"2024-09-11T08:40:20.467562+0000","last_clean":"2024-09-11T08:40:20.467562+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:40:20.467562+0000","last_undegraded":"2024-09-11T08:40:20.467562+0000","last_fullsized":"2024-09-11T08:40:20.467562+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:20.467456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:20.467456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:39.736643+0000","scrub_duration":0.001886544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":69,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.104337+0000","last_change":"2024-09-11T08:40:25.104280+0000","last_active":"2024-09-11T08:40:25.104337+0000","last_peered":"2024-09-11T08:40:25.104337+0000","last_clean":"2024-09-11T08:40:25.104337+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:40:25.104337+0000","last_undegraded":"2024-09-11T08:40:25.104337+0000","last_fullsized":"2024-09-11T08:40:25.104337+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:25.104224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:25.104224+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:46.893915+0000","scrub_duration":0.001645695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185592+0000","last_change":"2024-09-11T08:38:44.993997+0000","last_active":"2024-09-11T08:40:17.185592+0000","last_peered":"2024-09-11T08:40:17.185592+0000","last_clean":"2024-09-11T08:40:17.185592+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:40:17.185592+0000","last_undegraded":"2024-09-11T08:40:17.185592+0000","last_fullsized":"2024-09-11T08:40:17.185592+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:22.076369+0000","last_change":"2024-09-11T08:40:22.076288+0000","last_active":"2024-09-11T08:40:22.076369+0000","last_peered":"2024-09-11T08:40:22.076369+0000","last_clean":"2024-09-11T08:40:22.076369+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:40:22.076369+0000","last_undegraded":"2024-09-11T08:40:22.076369+0000","last_fullsized":"2024-09-11T08:40:22.076369+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:40:22.076198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.076198+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:48.206403+0000","scrub_duration":0.0024994370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.465681+0000","last_change":"2024-09-11T08:40:14.202552+0000","last_active":"2024-09-11T08:40:18.465681+0000","last_peered":"2024-09-11T08:40:18.465681+0000","last_clean":"2024-09-11T08:40:18.465681+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:40:18.465681+0000","last_undegraded":"2024-09-11T08:40:18.465681+0000","last_fullsized":"2024-09-11T08:40:18.465681+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:14.202498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:14.202498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:42.406451+0000","scrub_duration":0.001204299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":70,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:22.693541+0000","last_change":"2024-09-11T08:39:44.429546+0000","last_active":"2024-09-11T08:40:22.693541+0000","last_peered":"2024-09-11T08:40:22.693541+0000","last_clean":"2024-09-11T08:40:22.693541+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:40:22.693541+0000","last_undegraded":"2024-09-11T08:40:22.693541+0000","last_fullsized":"2024-09-11T08:40:22.693541+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:39:44.429477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:44.429477+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.023544+0000","scrub_duration":0.001890784,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.435494+0000","last_change":"2024-09-11T08:39:52.254299+0000","last_active":"2024-09-11T08:40:17.435494+0000","last_peered":"2024-09-11T08:40:17.435494+0000","last_clean":"2024-09-11T08:40:17.435494+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:40:17.435494+0000","last_undegraded":"2024-09-11T08:40:17.435494+0000","last_fullsized":"2024-09-11T08:40:17.435494+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.254245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.254245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:20.782062+0000","scrub_duration":0.001581877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185163+0000","last_change":"2024-09-11T08:38:40.973815+0000","last_active":"2024-09-11T08:40:17.185163+0000","last_peered":"2024-09-11T08:40:17.185163+0000","last_clean":"2024-09-11T08:40:17.185163+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:40:17.185163+0000","last_undegraded":"2024-09-11T08:40:17.185163+0000","last_fullsized":"2024-09-11T08:40:17.185163+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185289+0000","last_change":"2024-09-11T08:38:38.952604+0000","last_active":"2024-09-11T08:40:17.185289+0000","last_peered":"2024-09-11T08:40:17.185289+0000","last_clean":"2024-09-11T08:40:17.185289+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:40:17.185289+0000","last_undegraded":"2024-09-11T08:40:17.185289+0000","last_fullsized":"2024-09-11T08:40:17.185289+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.437168+0000","last_change":"2024-09-11T08:38:42.979138+0000","last_active":"2024-09-11T08:40:17.437168+0000","last_peered":"2024-09-11T08:40:17.437168+0000","last_clean":"2024-09-11T08:40:17.437168+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:40:17.437168+0000","last_undegraded":"2024-09-11T08:40:17.437168+0000","last_fullsized":"2024-09-11T08:40:17.437168+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185064+0000","last_change":"2024-09-11T08:38:44.994241+0000","last_active":"2024-09-11T08:40:17.185064+0000","last_peered":"2024-09-11T08:40:17.185064+0000","last_clean":"2024-09-11T08:40:17.185064+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:40:17.185064+0000","last_undegraded":"2024-09-11T08:40:17.185064+0000","last_fullsized":"2024-09-11T08:40:17.185064+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:22.382210+0000","last_change":"2024-09-11T08:40:22.382123+0000","last_active":"2024-09-11T08:40:22.382210+0000","last_peered":"2024-09-11T08:40:22.382210+0000","last_clean":"2024-09-11T08:40:22.382210+0000","last_became_active":"2024-09-11T08:40:16.181181+0000","last_became_peered":"2024-09-11T08:40:16.181181+0000","last_unstale":"2024-09-11T08:40:22.382210+0000","last_undegraded":"2024-09-11T08:40:22.382210+0000","last_fullsized":"2024-09-11T08:40:22.382210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:40:22.382048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.382048+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.268329+0000","scrub_duration":0.001806862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.465739+0000","last_change":"2024-09-11T08:38:38.954570+0000","last_active":"2024-09-11T08:40:18.465739+0000","last_peered":"2024-09-11T08:40:18.465739+0000","last_clean":"2024-09-11T08:40:18.465739+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:40:18.465739+0000","last_undegraded":"2024-09-11T08:40:18.465739+0000","last_fullsized":"2024-09-11T08:40:18.465739+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:24.089203+0000","last_change":"2024-09-11T08:40:24.089168+0000","last_active":"2024-09-11T08:40:24.089203+0000","last_peered":"2024-09-11T08:40:24.089203+0000","last_clean":"2024-09-11T08:40:24.089203+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:40:24.089203+0000","last_undegraded":"2024-09-11T08:40:24.089203+0000","last_fullsized":"2024-09-11T08:40:24.089203+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:24.089115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:24.089115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:27.976340+0000","scrub_duration":0.001138305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:22.499184+0000","last_change":"2024-09-11T08:40:22.499148+0000","last_active":"2024-09-11T08:40:22.499184+0000","last_peered":"2024-09-11T08:40:22.499184+0000","last_clean":"2024-09-11T08:40:22.499184+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:40:22.499184+0000","last_undegraded":"2024-09-11T08:40:22.499184+0000","last_fullsized":"2024-09-11T08:40:22.499184+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:22.499088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.499088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.706106+0000","scrub_duration":0.0015631830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185153+0000","last_change":"2024-09-11T08:38:41.039878+0000","last_active":"2024-09-11T08:40:17.185153+0000","last_peered":"2024-09-11T08:40:17.185153+0000","last_clean":"2024-09-11T08:40:17.185153+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:40:17.185153+0000","last_undegraded":"2024-09-11T08:40:17.185153+0000","last_fullsized":"2024-09-11T08:40:17.185153+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.585703+0000","last_change":"2024-09-11T08:40:05.430740+0000","last_active":"2024-09-11T08:40:17.585703+0000","last_peered":"2024-09-11T08:40:17.585703+0000","last_clean":"2024-09-11T08:40:17.585703+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:40:17.585703+0000","last_undegraded":"2024-09-11T08:40:17.585703+0000","last_fullsized":"2024-09-11T08:40:17.585703+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:05.430666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:05.430666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.818673+0000","scrub_duration":0.001547629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.465030+0000","last_change":"2024-09-11T08:39:47.104702+0000","last_active":"2024-09-11T08:40:18.465030+0000","last_peered":"2024-09-11T08:40:18.465030+0000","last_clean":"2024-09-11T08:40:18.465030+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:40:18.465030+0000","last_undegraded":"2024-09-11T08:40:18.465030+0000","last_fullsized":"2024-09-11T08:40:18.465030+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.104637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.104637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:51.311803+0000","scrub_duration":0.0015238020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185455+0000","last_change":"2024-09-11T08:38:45.183913+0000","last_active":"2024-09-11T08:40:17.185455+0000","last_peered":"2024-09-11T08:40:17.185455+0000","last_clean":"2024-09-11T08:40:17.185455+0000","last_became_active":"2024-09-11T08:38:45.183548+0000","last_became_peered":"2024-09-11T08:38:45.183548+0000","last_unstale":"2024-09-11T08:40:17.185455+0000","last_undegraded":"2024-09-11T08:40:17.185455+0000","last_fullsized":"2024-09-11T08:40:17.185455+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185101+0000","last_change":"2024-09-11T08:39:41.989740+0000","last_active":"2024-09-11T08:40:17.185101+0000","last_peered":"2024-09-11T08:40:17.185101+0000","last_clean":"2024-09-11T08:40:17.185101+0000","last_became_active":"2024-09-11T08:38:40.973799+0000","last_became_peered":"2024-09-11T08:38:40.973799+0000","last_unstale":"2024-09-11T08:40:17.185101+0000","last_undegraded":"2024-09-11T08:40:17.185101+0000","last_fullsized":"2024-09-11T08:40:17.185101+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-11T08:39:41.989684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.989684+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.687182+0000","scrub_duration":0.00162959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.435846+0000","last_change":"2024-09-11T08:40:08.336369+0000","last_active":"2024-09-11T08:40:17.435846+0000","last_peered":"2024-09-11T08:40:17.435846+0000","last_clean":"2024-09-11T08:40:17.435846+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:40:17.435846+0000","last_undegraded":"2024-09-11T08:40:17.435846+0000","last_fullsized":"2024-09-11T08:40:17.435846+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:40:08.336314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.336314+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:16.115575+0000","scrub_duration":0.001394085,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.187582+0000","last_change":"2024-09-11T08:39:53.541628+0000","last_active":"2024-09-11T08:40:17.187582+0000","last_peered":"2024-09-11T08:40:17.187582+0000","last_clean":"2024-09-11T08:40:17.187582+0000","last_became_active":"2024-09-11T08:38:42.989556+0000","last_became_peered":"2024-09-11T08:38:42.989556+0000","last_unstale":"2024-09-11T08:40:17.187582+0000","last_undegraded":"2024-09-11T08:40:17.187582+0000","last_fullsized":"2024-09-11T08:40:17.187582+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.541557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.541557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:08.967817+0000","scrub_duration":0.001310285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:18.465244+0000","last_change":"2024-09-11T08:40:07.114884+0000","last_active":"2024-09-11T08:40:18.465244+0000","last_peered":"2024-09-11T08:40:18.465244+0000","last_clean":"2024-09-11T08:40:18.465244+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:40:18.465244+0000","last_undegraded":"2024-09-11T08:40:18.465244+0000","last_fullsized":"2024-09-11T08:40:18.465244+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:07.114815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:07.114815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.897957+0000","scrub_duration":0.0013781500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.437417+0000","last_change":"2024-09-11T08:39:52.480663+0000","last_active":"2024-09-11T08:40:17.437417+0000","last_peered":"2024-09-11T08:40:17.437417+0000","last_clean":"2024-09-11T08:40:17.437417+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:40:17.437417+0000","last_undegraded":"2024-09-11T08:40:17.437417+0000","last_fullsized":"2024-09-11T08:40:17.437417+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.480610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.480610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:55.608483+0000","scrub_duration":0.001335754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.185198+0000","last_change":"2024-09-11T08:39:53.038356+0000","last_active":"2024-09-11T08:40:17.185198+0000","last_peered":"2024-09-11T08:40:17.185198+0000","last_clean":"2024-09-11T08:40:17.185198+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:40:17.185198+0000","last_undegraded":"2024-09-11T08:40:17.185198+0000","last_fullsized":"2024-09-11T08:40:17.185198+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.038297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.038297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:58.424182+0000","scrub_duration":0.0012198549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.188156+0000","last_change":"2024-09-11T08:40:02.551056+0000","last_active":"2024-09-11T08:40:17.188156+0000","last_peered":"2024-09-11T08:40:17.188156+0000","last_clean":"2024-09-11T08:40:17.188156+0000","last_became_active":"2024-09-11T08:38:40.974451+0000","last_became_peered":"2024-09-11T08:38:40.974451+0000","last_unstale":"2024-09-11T08:40:17.188156+0000","last_undegraded":"2024-09-11T08:40:17.188156+0000","last_fullsized":"2024-09-11T08:40:17.188156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:40:02.550999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.550999+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.173446+0000","scrub_duration":0.0016484399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T08:40:17.435655+0000","last_change":"2024-09-11T08:39:55.265155+0000","last_active":"2024-09-11T08:40:17.435655+0000","last_peered":"2024-09-11T08:40:17.435655+0000","last_clean":"2024-09-11T08:40:17.435655+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:40:17.435655+0000","last_undegraded":"2024-09-11T08:40:17.435655+0000","last_fullsized":"2024-09-11T08:40:17.435655+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.265097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.265097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.333633+0000","scrub_duration":0.0012893830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":423,"ondisk_log_size":423,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":95,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299419,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298608,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446544,"statfs":{"total":95995035648,"available":95689261056,"internally_reserved":0,"allocated":1286144,"data_stored":932974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.499,"15min":0.501},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.66300000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71599999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.53400000000000003}]},{"osd":1,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.50700000000000001},"min":{"1min":0.32700000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.74099999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.56999999999999995}]},{"osd":2,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.442}]},{"osd":3,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71799999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.53000000000000003},"min":{"1min":0.377,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70299999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.50900000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.42499999999999999,"15min":0.42199999999999999},"min":{"1min":0.318,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.59399999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.372}]},{"osd":5,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.30099999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.495,"5min":0.44700000000000001,"15min":0.439},"min":{"1min":0.313,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.46200000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47399999999999998,"15min":0.46899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.52300000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528352,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298604,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446548,"statfs":{"total":95995035648,"available":95689265152,"internally_reserved":0,"allocated":1277952,"data_stored":939947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.71699999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.39400000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.59799999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.70799999999999996,"5min":0.871,"15min":0.871},"last":0.53000000000000003}]},{"osd":2,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70999999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.51100000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.66200000000000003,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66800000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.627}]},{"osd":4,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.27500000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40600000000000003},"min":{"1min":0.29199999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.56000000000000005}]},{"osd":5,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.439,"15min":0.43099999999999999},"min":{"1min":0.34200000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.754},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.435,"15min":0.42999999999999999},"min":{"1min":0.30299999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.64800000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:39:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.69299999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757284,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298044,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447108,"statfs":{"total":95995035648,"available":95689838592,"internally_reserved":0,"allocated":704512,"data_stored":350985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61599999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59699999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58599999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.64800000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.376},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.51700000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.42599999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73099999999999998,"5min":0.749,"15min":0.749},"last":0.49399999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66100000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986216,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298040,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447112,"statfs":{"total":95995035648,"available":95689842688,"internally_reserved":0,"allocated":700416,"data_stored":349570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55500000000000005,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.67600000000000005}]},{"osd":1,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.38,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.66100000000000003}]},{"osd":2,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.39600000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55800000000000005,"15min":0.57799999999999996},"min":{"1min":0.38800000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72199999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.64400000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.40500000000000003,"15min":0.38700000000000001},"min":{"1min":0.32100000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.30099999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.46700000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:40:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.32400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.44},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.36199999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67100000000000004,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.56200000000000006}]},{"osd":7,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.504},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53900000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149742,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298052,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447100,"statfs":{"total":95995035648,"available":95689830400,"internally_reserved":0,"allocated":716800,"data_stored":343649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.36299999999999999,"15min":0.374},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.443,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.38900000000000001,"15min":0.40000000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.496,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.39200000000000002,"15min":0.40400000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.56200000000000006,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.40400000000000003,"15min":0.41399999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48199999999999998,"5min":0.621,"15min":0.621},"last":0.55000000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.42699999999999999,"15min":0.435},"min":{"1min":0.27100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.44,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.253}]},{"osd":4,"last update":"Wed Sep 11 08:40:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79900000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54900000000000004,"15min":0.55900000000000005},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66600000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.49299999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76300000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.67600000000000005}]},{"osd":7,"last update":"Wed Sep 11 08:39:27 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.60999999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298024,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447128,"statfs":{"total":95995035648,"available":95689859072,"internally_reserved":0,"allocated":684032,"data_stored":349525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36499999999999999,"15min":0.35399999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.29699999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.25800000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.39100000000000001,"15min":0.376},"min":{"1min":0.28999999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.33600000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:39:57 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.441,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.38500000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:40:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.751,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.57399999999999995}]},{"osd":5,"last update":"Wed Sep 11 08:39:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.48999999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:40:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.442,"5min":0.432,"15min":0.432},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.56100000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47899999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.41699999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76200000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81799999999999995,"5min":1.024,"15min":1.024},"last":0.53600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298068,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447084,"statfs":{"total":95995035648,"available":95689814016,"internally_reserved":0,"allocated":729088,"data_stored":350459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:19 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.314},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.47399999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.376},{"interface":"front","average":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33300000000000002},"min":{"1min":0.27400000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.42299999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:39:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:39:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40400000000000003,"15min":0.41999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54400000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.433,"15min":0.434},"min":{"1min":0.307,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.44700000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:40:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.76500000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55100000000000005,"15min":0.56499999999999995},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.63300000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.60899999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:39:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.755,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82199999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.59599999999999997}]},{"osd":6,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57399999999999995,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.63},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60099999999999998,"15min":0.60599999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.67700000000000005}]},{"osd":7,"last update":"Wed Sep 11 08:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.45600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79700000000000004,"5min":0.879,"15min":0.879},"last":0.68600000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298628,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446524,"statfs":{"total":95995035648,"available":95689240576,"internally_reserved":0,"allocated":1302528,"data_stored":940779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:40:10 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.311},"min":{"1min":0.19800000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.436,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.186},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34000000000000002,"15min":0.314},"min":{"1min":0.23999999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.504,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.30599999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.35799999999999998,"15min":0.34699999999999998},"min":{"1min":0.24199999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53500000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.34899999999999998},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.51500000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.22900000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:39:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39800000000000002,"15min":0.40000000000000002},"min":{"1min":0.28499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.503,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.28799999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70899999999999996,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.77200000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.504}]},{"osd":6,"last update":"Wed Sep 11 08:40:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.37,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68300000000000005,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.42499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.746,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.52000000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.76100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:40:27.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph status --format=json 2024-09-11T08:40:27.373 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:27 smithi092 podman[78974]: 2024-09-11 08:40:27.164443767 +0000 UTC m=+0.016101856 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-11T08:40:27.373 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:27 smithi092 podman[78974]: 2024-09-11 08:40:27.27305295 +0000 UTC m=+0.124711031 volume create b4d34c6f250d71a87dc252cf9c275a0865491d75f9ecdbba25271dede582b418 2024-09-11T08:40:27.373 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:27 smithi092 podman[78974]: 2024-09-11 08:40:27.373721051 +0000 UTC m=+0.225379171 container create 1fa23353947c5624b6186edf67da740fecf1ed25cc5f090fe45ee547b0f37af2 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:40:27.373 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:40:27.374 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:40:27.374 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:27.374 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[29647]: 5.12 scrub starts 2024-09-11T08:40:27.374 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[29647]: 5.12 scrub ok 2024-09-11T08:40:27.374 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[29647]: 6.8 scrub starts 2024-09-11T08:40:27.374 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[29647]: 6.8 scrub ok 2024-09-11T08:40:27.374 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[29647]: osdmap e83: 8 total, 8 up, 8 in 2024-09-11T08:40:27.374 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[29647]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:27.374 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:40:27.374 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:40:27.374 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:27.374 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[41114]: 5.12 scrub starts 2024-09-11T08:40:27.375 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[41114]: 5.12 scrub ok 2024-09-11T08:40:27.375 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[41114]: 6.8 scrub starts 2024-09-11T08:40:27.375 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[41114]: 6.8 scrub ok 2024-09-11T08:40:27.375 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[41114]: osdmap e83: 8 total, 8 up, 8 in 2024-09-11T08:40:27.375 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-mon[41114]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:27.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:27 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:40:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:27 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:40:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:27 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:27 smithi145 ceph-mon[33384]: 5.12 scrub starts 2024-09-11T08:40:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:27 smithi145 ceph-mon[33384]: 5.12 scrub ok 2024-09-11T08:40:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:27 smithi145 ceph-mon[33384]: 6.8 scrub starts 2024-09-11T08:40:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:27 smithi145 ceph-mon[33384]: 6.8 scrub ok 2024-09-11T08:40:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:27 smithi145 ceph-mon[33384]: osdmap e83: 8 total, 8 up, 8 in 2024-09-11T08:40:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:27 smithi145 ceph-mon[33384]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:27.992 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:28.065 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:27 smithi092 podman[78974]: 2024-09-11 08:40:27.711748016 +0000 UTC m=+0.563406094 container init 1fa23353947c5624b6186edf67da740fecf1ed25cc5f090fe45ee547b0f37af2 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:40:28.066 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:27 smithi092 podman[78974]: 2024-09-11 08:40:27.716568992 +0000 UTC m=+0.568227071 container start 1fa23353947c5624b6186edf67da740fecf1ed25cc5f090fe45ee547b0f37af2 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:40:28.066 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[79001]: ts=2024-09-11T08:40:27.729Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T08:40:28.066 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[79001]: ts=2024-09-11T08:40:27.729Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T08:40:28.066 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[79001]: ts=2024-09-11T08:40:27.730Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.92 port=9094 2024-09-11T08:40:28.066 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[79001]: ts=2024-09-11T08:40:27.731Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T08:40:28.066 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[79001]: ts=2024-09-11T08:40:27.754Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:40:28.066 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[79001]: ts=2024-09-11T08:40:27.754Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:40:28.066 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[79001]: ts=2024-09-11T08:40:27.757Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T08:40:28.066 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:27 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[79001]: ts=2024-09-11T08:40:27.757Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T08:40:28.066 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:27 smithi092 bash[78974]: 1fa23353947c5624b6186edf67da740fecf1ed25cc5f090fe45ee547b0f37af2 2024-09-11T08:40:28.066 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:27 smithi092 systemd[1]: Started Ceph alertmanager.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:40:28.389 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:28 smithi092 ceph-mon[29647]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 127 B/s wr, 4 op/s; 27 B/s, 1 objects/s recovering 2024-09-11T08:40:28.389 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:28 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:28.389 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:28 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:28.389 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:28 smithi092 ceph-mon[29647]: 5.17 scrub starts 2024-09-11T08:40:28.389 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:28 smithi092 ceph-mon[29647]: 5.17 scrub ok 2024-09-11T08:40:28.390 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:28 smithi092 ceph-mon[41114]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 127 B/s wr, 4 op/s; 27 B/s, 1 objects/s recovering 2024-09-11T08:40:28.390 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:28 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:28.390 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:28 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:28.390 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:28 smithi092 ceph-mon[41114]: 5.17 scrub starts 2024-09-11T08:40:28.390 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:28 smithi092 ceph-mon[41114]: 5.17 scrub ok 2024-09-11T08:40:28.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:28 smithi145 ceph-mon[33384]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 127 B/s wr, 4 op/s; 27 B/s, 1 objects/s recovering 2024-09-11T08:40:28.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:28 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:28.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:28 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:28.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:28 smithi145 ceph-mon[33384]: 5.17 scrub starts 2024-09-11T08:40:28.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:28 smithi145 ceph-mon[33384]: 5.17 scrub ok 2024-09-11T08:40:29.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:29 smithi092 ceph-mon[29647]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:40:29.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:29 smithi092 ceph-mon[29647]: Reconfiguring daemon prometheus.a on smithi145 2024-09-11T08:40:29.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:29 smithi092 ceph-mon[29647]: 3.15 scrub starts 2024-09-11T08:40:29.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:29 smithi092 ceph-mon[29647]: 3.15 scrub ok 2024-09-11T08:40:29.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:29 smithi092 ceph-mon[29647]: 3.5 scrub starts 2024-09-11T08:40:29.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:29 smithi092 ceph-mon[29647]: 3.5 scrub ok 2024-09-11T08:40:29.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:29 smithi092 ceph-mon[41114]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:40:29.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:29 smithi092 ceph-mon[41114]: Reconfiguring daemon prometheus.a on smithi145 2024-09-11T08:40:29.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:29 smithi092 ceph-mon[41114]: 3.15 scrub starts 2024-09-11T08:40:29.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:29 smithi092 ceph-mon[41114]: 3.15 scrub ok 2024-09-11T08:40:29.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:29 smithi092 ceph-mon[41114]: 3.5 scrub starts 2024-09-11T08:40:29.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:29 smithi092 ceph-mon[41114]: 3.5 scrub ok 2024-09-11T08:40:29.605 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:29 smithi145 ceph-mon[33384]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:40:29.605 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:29 smithi145 ceph-mon[33384]: Reconfiguring daemon prometheus.a on smithi145 2024-09-11T08:40:29.605 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:29 smithi145 ceph-mon[33384]: 3.15 scrub starts 2024-09-11T08:40:29.605 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:29 smithi145 ceph-mon[33384]: 3.15 scrub ok 2024-09-11T08:40:29.605 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:29 smithi145 ceph-mon[33384]: 3.5 scrub starts 2024-09-11T08:40:29.606 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:29 smithi145 ceph-mon[33384]: 3.5 scrub ok 2024-09-11T08:40:29.891 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:40:29.891 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"211625d4-7018-11ef-bcea-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":323,"monmap":{"epoch":3,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":83,"num_osds":8,"num_up_osds":8,"osd_up_since":1726043903,"num_in_osds":8,"osd_in_since":1726043886,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":132}],"num_pgs":132,"num_pools":6,"num_objects":196,"data_bytes":596188,"bytes_used":2443333632,"bytes_avail":765516951552,"bytes_total":767960285184,"read_bytes_sec":639,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.92:8443/","prometheus":"http://172.21.15.92:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-11T08:40:17.013941+0000","services":{"rgw":{"daemons":{"summary":"","24352":{"start_epoch":12,"start_stamp":"2024-09-11T08:38:46.107187+0000","gid":24352,"addr":"172.21.15.92:0/1175084397","metadata":{"arch":"x86_64","ceph_release":"quincy","ceph_version":"ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)","ceph_version_short":"17.2.7-1623-gd0e6828a","container_hostname":"smithi092","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40","cpu":"Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi092","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024","kernel_version":"5.14.0-505.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32487284","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"02abb3ce-34bc-4920-b6fd-8ad4b7819694","zone_name":"default","zonegroup_id":"8abc369a-51df-4e8f-b783-2e835134b342","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T08:40:30.065 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:29 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[79001]: ts=2024-09-11T08:40:29.731Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000612752s 2024-09-11T08:40:30.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:30 smithi092 ceph-mon[29647]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:40:30.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:30 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/394209486' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-11T08:40:30.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:30.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:30 smithi092 ceph-mon[29647]: 6.c scrub starts 2024-09-11T08:40:30.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:30 smithi092 ceph-mon[29647]: 6.c scrub ok 2024-09-11T08:40:30.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:30 smithi092 ceph-mon[41114]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:40:30.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:30 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/394209486' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-11T08:40:30.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:30.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:30 smithi092 ceph-mon[41114]: 6.c scrub starts 2024-09-11T08:40:30.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:30 smithi092 ceph-mon[41114]: 6.c scrub ok 2024-09-11T08:40:30.612 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-11T08:40:30.612 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:40:30.655 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:30 smithi145 ceph-mon[33384]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:40:30.655 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:30 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/394209486' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-11T08:40:30.655 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:30.655 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:30 smithi145 ceph-mon[33384]: 6.c scrub starts 2024-09-11T08:40:30.655 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:30 smithi145 ceph-mon[33384]: 6.c scrub ok 2024-09-11T08:40:30.857 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:31.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:31 smithi092 ceph-mon[29647]: 4.15 scrub starts 2024-09-11T08:40:31.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:31 smithi092 ceph-mon[29647]: 4.15 scrub ok 2024-09-11T08:40:31.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:31 smithi092 ceph-mon[29647]: 3.1b scrub starts 2024-09-11T08:40:31.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:31 smithi092 ceph-mon[29647]: 3.1b scrub ok 2024-09-11T08:40:31.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:31 smithi092 ceph-mon[29647]: 6.6 scrub starts 2024-09-11T08:40:31.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:31 smithi092 ceph-mon[29647]: 6.6 scrub ok 2024-09-11T08:40:31.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:31 smithi092 ceph-mon[41114]: 4.15 scrub starts 2024-09-11T08:40:31.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:31 smithi092 ceph-mon[41114]: 4.15 scrub ok 2024-09-11T08:40:31.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:31 smithi092 ceph-mon[41114]: 3.1b scrub starts 2024-09-11T08:40:31.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:31 smithi092 ceph-mon[41114]: 3.1b scrub ok 2024-09-11T08:40:31.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:31 smithi092 ceph-mon[41114]: 6.6 scrub starts 2024-09-11T08:40:31.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:31 smithi092 ceph-mon[41114]: 6.6 scrub ok 2024-09-11T08:40:31.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:31 smithi145 ceph-mon[33384]: 4.15 scrub starts 2024-09-11T08:40:31.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:31 smithi145 ceph-mon[33384]: 4.15 scrub ok 2024-09-11T08:40:31.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:31 smithi145 ceph-mon[33384]: 3.1b scrub starts 2024-09-11T08:40:31.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:31 smithi145 ceph-mon[33384]: 3.1b scrub ok 2024-09-11T08:40:31.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:31 smithi145 ceph-mon[33384]: 6.6 scrub starts 2024-09-11T08:40:31.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:31 smithi145 ceph-mon[33384]: 6.6 scrub ok 2024-09-11T08:40:32.545 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:32 smithi092 ceph-mon[41114]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:32.545 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:32 smithi092 ceph-mon[41114]: 6.15 scrub starts 2024-09-11T08:40:32.545 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:32 smithi092 ceph-mon[41114]: 6.15 scrub ok 2024-09-11T08:40:32.545 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:32 smithi092 ceph-mon[41114]: 3.14 scrub starts 2024-09-11T08:40:32.545 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:32 smithi092 ceph-mon[41114]: 3.14 scrub ok 2024-09-11T08:40:32.545 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:32 smithi092 ceph-mon[29647]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:32.545 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:32 smithi092 ceph-mon[29647]: 6.15 scrub starts 2024-09-11T08:40:32.546 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:32 smithi092 ceph-mon[29647]: 6.15 scrub ok 2024-09-11T08:40:32.546 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:32 smithi092 ceph-mon[29647]: 3.14 scrub starts 2024-09-11T08:40:32.546 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:32 smithi092 ceph-mon[29647]: 3.14 scrub ok 2024-09-11T08:40:32.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:32 smithi145 ceph-mon[33384]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:32 smithi145 ceph-mon[33384]: 6.15 scrub starts 2024-09-11T08:40:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:32 smithi145 ceph-mon[33384]: 6.15 scrub ok 2024-09-11T08:40:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:32 smithi145 ceph-mon[33384]: 3.14 scrub starts 2024-09-11T08:40:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:32 smithi145 ceph-mon[33384]: 3.14 scrub ok 2024-09-11T08:40:32.707 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:40:32.709 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:40:33.112 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:40:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:40:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:33.411 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 systemd[1]: Stopping Ceph prometheus.a for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T08:40:33.567 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-11T08:40:31.016751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":958,"num_read_kb":815,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":580,"ondisk_log_size":580,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2386192,"kb_used_data":7348,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575024,"statfs":{"total":767960285184,"available":765516824576,"internally_reserved":0,"allocated":7524352,"data_stored":4635348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001444"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418157+0000","last_change":"2024-09-11T08:39:47.339366+0000","last_active":"2024-09-11T08:40:25.418157+0000","last_peered":"2024-09-11T08:40:25.418157+0000","last_clean":"2024-09-11T08:40:25.418157+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:40:25.418157+0000","last_undegraded":"2024-09-11T08:40:25.418157+0000","last_fullsized":"2024-09-11T08:40:25.418157+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.339312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.339312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.154202+0000","scrub_duration":0.001323095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418551+0000","last_change":"2024-09-11T08:39:56.666764+0000","last_active":"2024-09-11T08:40:25.418551+0000","last_peered":"2024-09-11T08:40:25.418551+0000","last_clean":"2024-09-11T08:40:25.418551+0000","last_became_active":"2024-09-11T08:38:40.973398+0000","last_became_peered":"2024-09-11T08:38:40.973398+0000","last_unstale":"2024-09-11T08:40:25.418551+0000","last_undegraded":"2024-09-11T08:40:25.418551+0000","last_fullsized":"2024-09-11T08:40:25.418551+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.666703+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.544222+0000","scrub_duration":0.0041022000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222903+0000","last_change":"2024-09-11T08:40:12.325099+0000","last_active":"2024-09-11T08:40:25.222903+0000","last_peered":"2024-09-11T08:40:25.222903+0000","last_clean":"2024-09-11T08:40:25.222903+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:40:25.222903+0000","last_undegraded":"2024-09-11T08:40:25.222903+0000","last_fullsized":"2024-09-11T08:40:25.222903+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:12.325037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:12.325037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:13.497524+0000","scrub_duration":0.001599736,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418919+0000","last_change":"2024-09-11T08:38:44.992111+0000","last_active":"2024-09-11T08:40:25.418919+0000","last_peered":"2024-09-11T08:40:25.418919+0000","last_clean":"2024-09-11T08:40:25.418919+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:40:25.418919+0000","last_undegraded":"2024-09-11T08:40:25.418919+0000","last_fullsized":"2024-09-11T08:40:25.418919+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223066+0000","last_change":"2024-09-11T08:40:06.173950+0000","last_active":"2024-09-11T08:40:25.223066+0000","last_peered":"2024-09-11T08:40:25.223066+0000","last_clean":"2024-09-11T08:40:25.223066+0000","last_became_active":"2024-09-11T08:38:45.184215+0000","last_became_peered":"2024-09-11T08:38:45.184215+0000","last_unstale":"2024-09-11T08:40:25.223066+0000","last_undegraded":"2024-09-11T08:40:25.223066+0000","last_fullsized":"2024-09-11T08:40:25.223066+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:06.173882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:06.173882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:17.036842+0000","scrub_duration":0.001730875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223651+0000","last_change":"2024-09-11T08:39:58.161055+0000","last_active":"2024-09-11T08:40:25.223651+0000","last_peered":"2024-09-11T08:40:25.223651+0000","last_clean":"2024-09-11T08:40:25.223651+0000","last_became_active":"2024-09-11T08:38:38.949136+0000","last_became_peered":"2024-09-11T08:38:38.949136+0000","last_unstale":"2024-09-11T08:40:25.223651+0000","last_undegraded":"2024-09-11T08:40:25.223651+0000","last_fullsized":"2024-09-11T08:40:25.223651+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:58.160984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.160984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:00.523696+0000","scrub_duration":0.0015109679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":75,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222725+0000","last_change":"2024-09-11T08:39:45.997873+0000","last_active":"2024-09-11T08:40:25.222725+0000","last_peered":"2024-09-11T08:40:25.222725+0000","last_clean":"2024-09-11T08:40:25.222725+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:40:25.222725+0000","last_undegraded":"2024-09-11T08:40:25.222725+0000","last_fullsized":"2024-09-11T08:40:25.222725+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.997803+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.534414+0000","scrub_duration":0.0046870369999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418457+0000","last_change":"2024-09-11T08:38:42.983653+0000","last_active":"2024-09-11T08:40:25.418457+0000","last_peered":"2024-09-11T08:40:25.418457+0000","last_clean":"2024-09-11T08:40:25.418457+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:40:25.418457+0000","last_undegraded":"2024-09-11T08:40:25.418457+0000","last_fullsized":"2024-09-11T08:40:25.418457+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223363+0000","last_change":"2024-09-11T08:40:08.416577+0000","last_active":"2024-09-11T08:40:25.223363+0000","last_peered":"2024-09-11T08:40:25.223363+0000","last_clean":"2024-09-11T08:40:25.223363+0000","last_became_active":"2024-09-11T08:38:38.948930+0000","last_became_peered":"2024-09-11T08:38:38.948930+0000","last_unstale":"2024-09-11T08:40:25.223363+0000","last_undegraded":"2024-09-11T08:40:25.223363+0000","last_fullsized":"2024-09-11T08:40:25.223363+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:08.416522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.416522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:27.445091+0000","scrub_duration":0.001330005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420008+0000","last_change":"2024-09-11T08:39:46.620700+0000","last_active":"2024-09-11T08:40:25.420008+0000","last_peered":"2024-09-11T08:40:25.420008+0000","last_clean":"2024-09-11T08:40:25.420008+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:40:25.420008+0000","last_undegraded":"2024-09-11T08:40:25.420008+0000","last_fullsized":"2024-09-11T08:40:25.420008+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:39:46.620624+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:10.984564+0000","scrub_duration":0.0044132319999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222859+0000","last_change":"2024-09-11T08:40:04.450336+0000","last_active":"2024-09-11T08:40:25.222859+0000","last_peered":"2024-09-11T08:40:25.222859+0000","last_clean":"2024-09-11T08:40:25.222859+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:40:25.222859+0000","last_undegraded":"2024-09-11T08:40:25.222859+0000","last_fullsized":"2024-09-11T08:40:25.222859+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:04.450282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:04.450282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.499239+0000","scrub_duration":0.001689433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417575+0000","last_change":"2024-09-11T08:39:51.280365+0000","last_active":"2024-09-11T08:40:25.417575+0000","last_peered":"2024-09-11T08:40:25.417575+0000","last_clean":"2024-09-11T08:40:25.417575+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:40:25.417575+0000","last_undegraded":"2024-09-11T08:40:25.417575+0000","last_fullsized":"2024-09-11T08:40:25.417575+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.280310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.280310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:06.712880+0000","scrub_duration":0.001634777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223124+0000","last_change":"2024-09-11T08:40:12.372128+0000","last_active":"2024-09-11T08:40:25.223124+0000","last_peered":"2024-09-11T08:40:25.223124+0000","last_clean":"2024-09-11T08:40:25.223124+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:40:25.223124+0000","last_undegraded":"2024-09-11T08:40:25.223124+0000","last_fullsized":"2024-09-11T08:40:25.223124+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:12.372043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:12.372043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.497179+0000","scrub_duration":0.001440623,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419933+0000","last_change":"2024-09-11T08:38:40.974291+0000","last_active":"2024-09-11T08:40:25.419933+0000","last_peered":"2024-09-11T08:40:25.419933+0000","last_clean":"2024-09-11T08:40:25.419933+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:40:25.419933+0000","last_undegraded":"2024-09-11T08:40:25.419933+0000","last_fullsized":"2024-09-11T08:40:25.419933+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418977+0000","last_change":"2024-09-11T08:39:51.166114+0000","last_active":"2024-09-11T08:40:25.418977+0000","last_peered":"2024-09-11T08:40:25.418977+0000","last_clean":"2024-09-11T08:40:25.418977+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:40:25.418977+0000","last_undegraded":"2024-09-11T08:40:25.418977+0000","last_fullsized":"2024-09-11T08:40:25.418977+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.166051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.570790+0000","scrub_duration":0.0016440420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222737+0000","last_change":"2024-09-11T08:38:44.994259+0000","last_active":"2024-09-11T08:40:25.222737+0000","last_peered":"2024-09-11T08:40:25.222737+0000","last_clean":"2024-09-11T08:40:25.222737+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:40:25.222737+0000","last_undegraded":"2024-09-11T08:40:25.222737+0000","last_fullsized":"2024-09-11T08:40:25.222737+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420256+0000","last_change":"2024-09-11T08:39:59.606521+0000","last_active":"2024-09-11T08:40:25.420256+0000","last_peered":"2024-09-11T08:40:25.420256+0000","last_clean":"2024-09-11T08:40:25.420256+0000","last_became_active":"2024-09-11T08:38:45.183364+0000","last_became_peered":"2024-09-11T08:38:45.183364+0000","last_unstale":"2024-09-11T08:40:25.420256+0000","last_undegraded":"2024-09-11T08:40:25.420256+0000","last_fullsized":"2024-09-11T08:40:25.420256+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:59.606458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:59.606458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.547769+0000","scrub_duration":0.0012065960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:30.632514+0000","last_change":"2024-09-11T08:40:30.632483+0000","last_active":"2024-09-11T08:40:30.632514+0000","last_peered":"2024-09-11T08:40:30.632514+0000","last_clean":"2024-09-11T08:40:30.632514+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:40:30.632514+0000","last_undegraded":"2024-09-11T08:40:30.632514+0000","last_fullsized":"2024-09-11T08:40:30.632514+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:30.632430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:30.632430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:45.188817+0000","scrub_duration":0.001480947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.414027+0000","last_change":"2024-09-11T08:39:50.472419+0000","last_active":"2024-09-11T08:40:25.414027+0000","last_peered":"2024-09-11T08:40:25.414027+0000","last_clean":"2024-09-11T08:40:25.414027+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:40:25.414027+0000","last_undegraded":"2024-09-11T08:40:25.414027+0000","last_fullsized":"2024-09-11T08:40:25.414027+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.472368+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:10.784548+0000","scrub_duration":0.0049809520000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223102+0000","last_change":"2024-09-11T08:39:44.632369+0000","last_active":"2024-09-11T08:40:25.223102+0000","last_peered":"2024-09-11T08:40:25.223102+0000","last_clean":"2024-09-11T08:40:25.223102+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:40:25.223102+0000","last_undegraded":"2024-09-11T08:40:25.223102+0000","last_fullsized":"2024-09-11T08:40:25.223102+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:44.632308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:44.632308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:56.408810+0000","scrub_duration":0.001420377,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223786+0000","last_change":"2024-09-11T08:38:45.183646+0000","last_active":"2024-09-11T08:40:25.223786+0000","last_peered":"2024-09-11T08:40:25.223786+0000","last_clean":"2024-09-11T08:40:25.223786+0000","last_became_active":"2024-09-11T08:38:45.182763+0000","last_became_peered":"2024-09-11T08:38:45.182763+0000","last_unstale":"2024-09-11T08:40:25.223786+0000","last_undegraded":"2024-09-11T08:40:25.223786+0000","last_fullsized":"2024-09-11T08:40:25.223786+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419889+0000","last_change":"2024-09-11T08:38:38.949654+0000","last_active":"2024-09-11T08:40:25.419889+0000","last_peered":"2024-09-11T08:40:25.419889+0000","last_clean":"2024-09-11T08:40:25.419889+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:40:25.419889+0000","last_undegraded":"2024-09-11T08:40:25.419889+0000","last_fullsized":"2024-09-11T08:40:25.419889+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223370+0000","last_change":"2024-09-11T08:39:55.125449+0000","last_active":"2024-09-11T08:40:25.223370+0000","last_peered":"2024-09-11T08:40:25.223370+0000","last_clean":"2024-09-11T08:40:25.223370+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:40:25.223370+0000","last_undegraded":"2024-09-11T08:40:25.223370+0000","last_fullsized":"2024-09-11T08:40:25.223370+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:55.125394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.125394+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.857707+0000","scrub_duration":0.0016833810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418855+0000","last_change":"2024-09-11T08:40:01.585472+0000","last_active":"2024-09-11T08:40:25.418855+0000","last_peered":"2024-09-11T08:40:25.418855+0000","last_clean":"2024-09-11T08:40:25.418855+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:40:25.418855+0000","last_undegraded":"2024-09-11T08:40:25.418855+0000","last_fullsized":"2024-09-11T08:40:25.418855+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:01.585411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:01.585411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.937039+0000","scrub_duration":0.0021220179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419857+0000","last_change":"2024-09-11T08:40:10.113729+0000","last_active":"2024-09-11T08:40:25.419857+0000","last_peered":"2024-09-11T08:40:25.419857+0000","last_clean":"2024-09-11T08:40:25.419857+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:40:25.419857+0000","last_undegraded":"2024-09-11T08:40:25.419857+0000","last_fullsized":"2024-09-11T08:40:25.419857+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:40:10.113671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:10.113671+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:33.267918+0000","scrub_duration":0.001621302,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222971+0000","last_change":"2024-09-11T08:40:02.458618+0000","last_active":"2024-09-11T08:40:25.222971+0000","last_peered":"2024-09-11T08:40:25.222971+0000","last_clean":"2024-09-11T08:40:25.222971+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:40:25.222971+0000","last_undegraded":"2024-09-11T08:40:25.222971+0000","last_fullsized":"2024-09-11T08:40:25.222971+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:02.458565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.458565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.247079+0000","scrub_duration":0.001315568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418113+0000","last_change":"2024-09-11T08:40:18.467638+0000","last_active":"2024-09-11T08:40:25.418113+0000","last_peered":"2024-09-11T08:40:25.418113+0000","last_clean":"2024-09-11T08:40:25.418113+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:40:25.418113+0000","last_undegraded":"2024-09-11T08:40:25.418113+0000","last_fullsized":"2024-09-11T08:40:25.418113+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:40:18.467574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:18.467574+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:37.429174+0000","scrub_duration":0.0040976820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":74,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:27.487746+0000","last_change":"2024-09-11T08:40:12.584703+0000","last_active":"2024-09-11T08:40:27.487746+0000","last_peered":"2024-09-11T08:40:27.487746+0000","last_clean":"2024-09-11T08:40:27.487746+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:40:27.487746+0000","last_undegraded":"2024-09-11T08:40:27.487746+0000","last_fullsized":"2024-09-11T08:40:27.487746+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:12.584631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:12.584631+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:27.946233+0000","scrub_duration":0.0015754790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223715+0000","last_change":"2024-09-11T08:40:16.290927+0000","last_active":"2024-09-11T08:40:25.223715+0000","last_peered":"2024-09-11T08:40:25.223715+0000","last_clean":"2024-09-11T08:40:25.223715+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:40:25.223715+0000","last_undegraded":"2024-09-11T08:40:25.223715+0000","last_fullsized":"2024-09-11T08:40:25.223715+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-11T08:40:16.290866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:16.290866+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:46.083697+0000","scrub_duration":0.0021215069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222683+0000","last_change":"2024-09-11T08:40:23.048021+0000","last_active":"2024-09-11T08:40:25.222683+0000","last_peered":"2024-09-11T08:40:25.222683+0000","last_clean":"2024-09-11T08:40:25.222683+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:40:25.222683+0000","last_undegraded":"2024-09-11T08:40:25.222683+0000","last_fullsized":"2024-09-11T08:40:25.222683+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:23.047963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:23.047963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.577363+0000","scrub_duration":0.00148653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:27.282426+0000","last_change":"2024-09-11T08:40:11.625218+0000","last_active":"2024-09-11T08:40:27.282426+0000","last_peered":"2024-09-11T08:40:27.282426+0000","last_clean":"2024-09-11T08:40:27.282426+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:40:27.282426+0000","last_undegraded":"2024-09-11T08:40:27.282426+0000","last_fullsized":"2024-09-11T08:40:27.282426+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:11.625164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:11.625164+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.094157+0000","scrub_duration":0.0014996320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223059+0000","last_change":"2024-09-11T08:38:44.991592+0000","last_active":"2024-09-11T08:40:25.223059+0000","last_peered":"2024-09-11T08:40:25.223059+0000","last_clean":"2024-09-11T08:40:25.223059+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:40:25.223059+0000","last_undegraded":"2024-09-11T08:40:25.223059+0000","last_fullsized":"2024-09-11T08:40:25.223059+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":75,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222500+0000","last_change":"2024-09-11T08:40:05.197208+0000","last_active":"2024-09-11T08:40:25.222500+0000","last_peered":"2024-09-11T08:40:25.222500+0000","last_clean":"2024-09-11T08:40:25.222500+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:40:25.222500+0000","last_undegraded":"2024-09-11T08:40:25.222500+0000","last_fullsized":"2024-09-11T08:40:25.222500+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-11T08:40:05.197152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:05.197152+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:06.174279+0000","scrub_duration":0.0016976000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223932+0000","last_change":"2024-09-11T08:39:43.037605+0000","last_active":"2024-09-11T08:40:25.223932+0000","last_peered":"2024-09-11T08:40:25.223932+0000","last_clean":"2024-09-11T08:40:25.223932+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:40:25.223932+0000","last_undegraded":"2024-09-11T08:40:25.223932+0000","last_fullsized":"2024-09-11T08:40:25.223932+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.037551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.037551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.071661+0000","scrub_duration":0.001523976,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418369+0000","last_change":"2024-09-11T08:40:19.688943+0000","last_active":"2024-09-11T08:40:25.418369+0000","last_peered":"2024-09-11T08:40:25.418369+0000","last_clean":"2024-09-11T08:40:25.418369+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:40:25.418369+0000","last_undegraded":"2024-09-11T08:40:25.418369+0000","last_fullsized":"2024-09-11T08:40:25.418369+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:19.688876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:19.688876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.993714+0000","scrub_duration":0.0014001599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419130+0000","last_change":"2024-09-11T08:38:44.995555+0000","last_active":"2024-09-11T08:40:25.419130+0000","last_peered":"2024-09-11T08:40:25.419130+0000","last_clean":"2024-09-11T08:40:25.419130+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:40:25.419130+0000","last_undegraded":"2024-09-11T08:40:25.419130+0000","last_fullsized":"2024-09-11T08:40:25.419130+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":55,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418981+0000","last_change":"2024-09-11T08:38:40.968377+0000","last_active":"2024-09-11T08:40:25.418981+0000","last_peered":"2024-09-11T08:40:25.418981+0000","last_clean":"2024-09-11T08:40:25.418981+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:40:25.418981+0000","last_undegraded":"2024-09-11T08:40:25.418981+0000","last_fullsized":"2024-09-11T08:40:25.418981+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418178+0000","last_change":"2024-09-11T08:39:41.267320+0000","last_active":"2024-09-11T08:40:25.418178+0000","last_peered":"2024-09-11T08:40:25.418178+0000","last_clean":"2024-09-11T08:40:25.418178+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:40:25.418178+0000","last_undegraded":"2024-09-11T08:40:25.418178+0000","last_fullsized":"2024-09-11T08:40:25.418178+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:41.267267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.267267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:01.547087+0000","scrub_duration":0.0013348209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":64,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:26.225641+0000","last_change":"2024-09-11T08:38:42.982794+0000","last_active":"2024-09-11T08:40:26.225641+0000","last_peered":"2024-09-11T08:40:26.225641+0000","last_clean":"2024-09-11T08:40:26.225641+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:40:26.225641+0000","last_undegraded":"2024-09-11T08:40:26.225641+0000","last_fullsized":"2024-09-11T08:40:26.225641+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418431+0000","last_change":"2024-09-11T08:40:04.174461+0000","last_active":"2024-09-11T08:40:25.418431+0000","last_peered":"2024-09-11T08:40:25.418431+0000","last_clean":"2024-09-11T08:40:25.418431+0000","last_became_active":"2024-09-11T08:38:45.182724+0000","last_became_peered":"2024-09-11T08:38:45.182724+0000","last_unstale":"2024-09-11T08:40:25.418431+0000","last_undegraded":"2024-09-11T08:40:25.418431+0000","last_fullsized":"2024-09-11T08:40:25.418431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:04.174409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:04.174409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.916947+0000","scrub_duration":0.0012996889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"63'10","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223578+0000","last_change":"2024-09-11T08:40:13.327302+0000","last_active":"2024-09-11T08:40:25.223578+0000","last_peered":"2024-09-11T08:40:25.223578+0000","last_clean":"2024-09-11T08:40:25.223578+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:40:25.223578+0000","last_undegraded":"2024-09-11T08:40:25.223578+0000","last_fullsized":"2024-09-11T08:40:25.223578+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:40:13.327245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:13.327245+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.332692+0000","scrub_duration":0.0017714300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:28.351843+0000","last_change":"2024-09-11T08:40:28.351815+0000","last_active":"2024-09-11T08:40:28.351843+0000","last_peered":"2024-09-11T08:40:28.351843+0000","last_clean":"2024-09-11T08:40:28.351843+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:40:28.351843+0000","last_undegraded":"2024-09-11T08:40:28.351843+0000","last_fullsized":"2024-09-11T08:40:28.351843+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:28.351762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:28.351762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:39.082689+0000","scrub_duration":0.001710488,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":62,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:27.486388+0000","last_change":"2024-09-11T08:38:42.981732+0000","last_active":"2024-09-11T08:40:27.486388+0000","last_peered":"2024-09-11T08:40:27.486388+0000","last_clean":"2024-09-11T08:40:27.486388+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:40:27.486388+0000","last_undegraded":"2024-09-11T08:40:27.486388+0000","last_fullsized":"2024-09-11T08:40:27.486388+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417732+0000","last_change":"2024-09-11T08:38:44.998257+0000","last_active":"2024-09-11T08:40:25.417732+0000","last_peered":"2024-09-11T08:40:25.417732+0000","last_clean":"2024-09-11T08:40:25.417732+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:40:25.417732+0000","last_undegraded":"2024-09-11T08:40:25.417732+0000","last_fullsized":"2024-09-11T08:40:25.417732+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417915+0000","last_change":"2024-09-11T08:38:41.040034+0000","last_active":"2024-09-11T08:40:25.417915+0000","last_peered":"2024-09-11T08:40:25.417915+0000","last_clean":"2024-09-11T08:40:25.417915+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:40:25.417915+0000","last_undegraded":"2024-09-11T08:40:25.417915+0000","last_fullsized":"2024-09-11T08:40:25.417915+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223367+0000","last_change":"2024-09-11T08:40:03.496005+0000","last_active":"2024-09-11T08:40:25.223367+0000","last_peered":"2024-09-11T08:40:25.223367+0000","last_clean":"2024-09-11T08:40:25.223367+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:40:25.223367+0000","last_undegraded":"2024-09-11T08:40:25.223367+0000","last_fullsized":"2024-09-11T08:40:25.223367+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:03.495950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.495950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.342754+0000","scrub_duration":0.001379314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418347+0000","last_change":"2024-09-11T08:39:57.596980+0000","last_active":"2024-09-11T08:40:25.418347+0000","last_peered":"2024-09-11T08:40:25.418347+0000","last_clean":"2024-09-11T08:40:25.418347+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:40:25.418347+0000","last_undegraded":"2024-09-11T08:40:25.418347+0000","last_fullsized":"2024-09-11T08:40:25.418347+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.596889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.648355+0000","scrub_duration":0.001574424,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222875+0000","last_change":"2024-09-11T08:40:08.387078+0000","last_active":"2024-09-11T08:40:25.222875+0000","last_peered":"2024-09-11T08:40:25.222875+0000","last_clean":"2024-09-11T08:40:25.222875+0000","last_became_active":"2024-09-11T08:38:45.182782+0000","last_became_peered":"2024-09-11T08:38:45.182782+0000","last_unstale":"2024-09-11T08:40:25.222875+0000","last_undegraded":"2024-09-11T08:40:25.222875+0000","last_fullsized":"2024-09-11T08:40:25.222875+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:08.387027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.387027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.887729+0000","scrub_duration":0.001411336,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":124,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:27.192495+0000","last_change":"2024-09-11T08:39:53.493004+0000","last_active":"2024-09-11T08:40:27.192495+0000","last_peered":"2024-09-11T08:40:27.192495+0000","last_clean":"2024-09-11T08:40:27.192495+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:40:27.192495+0000","last_undegraded":"2024-09-11T08:40:27.192495+0000","last_fullsized":"2024-09-11T08:40:27.192495+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.492945+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.774847+0000","scrub_duration":0.005988045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420227+0000","last_change":"2024-09-11T08:40:22.750815+0000","last_active":"2024-09-11T08:40:25.420227+0000","last_peered":"2024-09-11T08:40:25.420227+0000","last_clean":"2024-09-11T08:40:25.420227+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:40:25.420227+0000","last_undegraded":"2024-09-11T08:40:25.420227+0000","last_fullsized":"2024-09-11T08:40:25.420227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:22.750744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.750744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.795298+0000","scrub_duration":0.001679119,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223085+0000","last_change":"2024-09-11T08:39:58.472851+0000","last_active":"2024-09-11T08:40:25.223085+0000","last_peered":"2024-09-11T08:40:25.223085+0000","last_clean":"2024-09-11T08:40:25.223085+0000","last_became_active":"2024-09-11T08:38:38.947270+0000","last_became_peered":"2024-09-11T08:38:38.947270+0000","last_unstale":"2024-09-11T08:40:25.223085+0000","last_undegraded":"2024-09-11T08:40:25.223085+0000","last_fullsized":"2024-09-11T08:40:25.223085+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:39:58.472785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.472785+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.725781+0000","scrub_duration":0.0020806930000000002,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418012+0000","last_change":"2024-09-11T08:40:17.435980+0000","last_active":"2024-09-11T08:40:25.418012+0000","last_peered":"2024-09-11T08:40:25.418012+0000","last_clean":"2024-09-11T08:40:25.418012+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:40:25.418012+0000","last_undegraded":"2024-09-11T08:40:25.418012+0000","last_fullsized":"2024-09-11T08:40:25.418012+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:17.435900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:17.435900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.732279+0000","scrub_duration":0.0018276379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223509+0000","last_change":"2024-09-11T08:38:44.997146+0000","last_active":"2024-09-11T08:40:25.223509+0000","last_peered":"2024-09-11T08:40:25.223509+0000","last_clean":"2024-09-11T08:40:25.223509+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:40:25.223509+0000","last_undegraded":"2024-09-11T08:40:25.223509+0000","last_fullsized":"2024-09-11T08:40:25.223509+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":71,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.224030+0000","last_change":"2024-09-11T08:39:43.585175+0000","last_active":"2024-09-11T08:40:25.224030+0000","last_peered":"2024-09-11T08:40:25.224030+0000","last_clean":"2024-09-11T08:40:25.224030+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:40:25.224030+0000","last_undegraded":"2024-09-11T08:40:25.224030+0000","last_fullsized":"2024-09-11T08:40:25.224030+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:39:43.585116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.585116+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.724395+0000","scrub_duration":0.0021210309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222736+0000","last_change":"2024-09-11T08:40:00.500444+0000","last_active":"2024-09-11T08:40:25.222736+0000","last_peered":"2024-09-11T08:40:25.222736+0000","last_clean":"2024-09-11T08:40:25.222736+0000","last_became_active":"2024-09-11T08:38:38.949019+0000","last_became_peered":"2024-09-11T08:38:38.949019+0000","last_unstale":"2024-09-11T08:40:25.222736+0000","last_undegraded":"2024-09-11T08:40:25.222736+0000","last_fullsized":"2024-09-11T08:40:25.222736+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.500392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.500392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.354187+0000","scrub_duration":0.001294566,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":52,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.413758+0000","last_change":"2024-09-11T08:40:01.488539+0000","last_active":"2024-09-11T08:40:25.413758+0000","last_peered":"2024-09-11T08:40:25.413758+0000","last_clean":"2024-09-11T08:40:25.413758+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:40:25.413758+0000","last_undegraded":"2024-09-11T08:40:25.413758+0000","last_fullsized":"2024-09-11T08:40:25.413758+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:40:01.488482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:01.488482+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.917024+0000","scrub_duration":0.001463422,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.414198+0000","last_change":"2024-09-11T08:40:02.444341+0000","last_active":"2024-09-11T08:40:25.414198+0000","last_peered":"2024-09-11T08:40:25.414198+0000","last_clean":"2024-09-11T08:40:25.414198+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:40:25.414198+0000","last_undegraded":"2024-09-11T08:40:25.414198+0000","last_fullsized":"2024-09-11T08:40:25.414198+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.444279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.886414+0000","scrub_duration":0.001414329,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419559+0000","last_change":"2024-09-11T08:40:15.184420+0000","last_active":"2024-09-11T08:40:25.419559+0000","last_peered":"2024-09-11T08:40:25.419559+0000","last_clean":"2024-09-11T08:40:25.419559+0000","last_became_active":"2024-09-11T08:38:45.183582+0000","last_became_peered":"2024-09-11T08:38:45.183582+0000","last_unstale":"2024-09-11T08:40:25.419559+0000","last_undegraded":"2024-09-11T08:40:25.419559+0000","last_fullsized":"2024-09-11T08:40:25.419559+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:15.184337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:15.184337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.002701+0000","scrub_duration":0.0039899209999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418097+0000","last_change":"2024-09-11T08:40:21.450026+0000","last_active":"2024-09-11T08:40:25.418097+0000","last_peered":"2024-09-11T08:40:25.418097+0000","last_clean":"2024-09-11T08:40:25.418097+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:40:25.418097+0000","last_undegraded":"2024-09-11T08:40:25.418097+0000","last_fullsized":"2024-09-11T08:40:25.418097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:40:21.449958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:21.449958+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:46.796354+0000","scrub_duration":0.0017585280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417684+0000","last_change":"2024-09-11T08:39:43.356751+0000","last_active":"2024-09-11T08:40:25.417684+0000","last_peered":"2024-09-11T08:40:25.417684+0000","last_clean":"2024-09-11T08:40:25.417684+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:40:25.417684+0000","last_undegraded":"2024-09-11T08:40:25.417684+0000","last_fullsized":"2024-09-11T08:40:25.417684+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.356699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.356699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.569637+0000","scrub_duration":0.001649455,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:29.277663+0000","last_change":"2024-09-11T08:39:55.153362+0000","last_active":"2024-09-11T08:40:29.277663+0000","last_peered":"2024-09-11T08:40:29.277663+0000","last_clean":"2024-09-11T08:40:29.277663+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:40:29.277663+0000","last_undegraded":"2024-09-11T08:40:29.277663+0000","last_fullsized":"2024-09-11T08:40:29.277663+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:39:55.153290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.153290+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.446909+0000","scrub_duration":0.0017161990000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":105,"num_read_kb":100,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223583+0000","last_change":"2024-09-11T08:38:42.976525+0000","last_active":"2024-09-11T08:40:25.223583+0000","last_peered":"2024-09-11T08:40:25.223583+0000","last_clean":"2024-09-11T08:40:25.223583+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:40:25.223583+0000","last_undegraded":"2024-09-11T08:40:25.223583+0000","last_fullsized":"2024-09-11T08:40:25.223583+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223080+0000","last_change":"2024-09-11T08:40:15.280809+0000","last_active":"2024-09-11T08:40:25.223080+0000","last_peered":"2024-09-11T08:40:25.223080+0000","last_clean":"2024-09-11T08:40:25.223080+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:40:25.223080+0000","last_undegraded":"2024-09-11T08:40:25.223080+0000","last_fullsized":"2024-09-11T08:40:25.223080+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:15.280760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:15.280760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:36.262012+0000","scrub_duration":0.0012405719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419421+0000","last_change":"2024-09-11T08:40:00.540192+0000","last_active":"2024-09-11T08:40:25.419421+0000","last_peered":"2024-09-11T08:40:25.419421+0000","last_clean":"2024-09-11T08:40:25.419421+0000","last_became_active":"2024-09-11T08:38:41.040948+0000","last_became_peered":"2024-09-11T08:38:41.040948+0000","last_unstale":"2024-09-11T08:40:25.419421+0000","last_undegraded":"2024-09-11T08:40:25.419421+0000","last_fullsized":"2024-09-11T08:40:25.419421+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:40:00.540136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.540136+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:07.651325+0000","scrub_duration":0.001785095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223089+0000","last_change":"2024-09-11T08:40:21.086239+0000","last_active":"2024-09-11T08:40:25.223089+0000","last_peered":"2024-09-11T08:40:25.223089+0000","last_clean":"2024-09-11T08:40:25.223089+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:40:25.223089+0000","last_undegraded":"2024-09-11T08:40:25.223089+0000","last_fullsized":"2024-09-11T08:40:25.223089+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:40:21.086183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:21.086183+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:25.836107+0000","scrub_duration":0.001907838,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":79,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419314+0000","last_change":"2024-09-11T08:39:12.765244+0000","last_active":"2024-09-11T08:40:25.419314+0000","last_peered":"2024-09-11T08:40:25.419314+0000","last_clean":"2024-09-11T08:40:25.419314+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:40:25.419314+0000","last_undegraded":"2024-09-11T08:40:25.419314+0000","last_fullsized":"2024-09-11T08:40:25.419314+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'72","last_scrub_stamp":"2024-09-11T08:39:12.765190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:39:12.765190+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014709829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419459+0000","last_change":"2024-09-11T08:39:49.116184+0000","last_active":"2024-09-11T08:40:25.419459+0000","last_peered":"2024-09-11T08:40:25.419459+0000","last_clean":"2024-09-11T08:40:25.419459+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:40:25.419459+0000","last_undegraded":"2024-09-11T08:40:25.419459+0000","last_fullsized":"2024-09-11T08:40:25.419459+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.116113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.116113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.209747+0000","scrub_duration":0.00167223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223528+0000","last_change":"2024-09-11T08:39:47.423745+0000","last_active":"2024-09-11T08:40:25.223528+0000","last_peered":"2024-09-11T08:40:25.223528+0000","last_clean":"2024-09-11T08:40:25.223528+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:40:25.223528+0000","last_undegraded":"2024-09-11T08:40:25.223528+0000","last_fullsized":"2024-09-11T08:40:25.223528+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.423684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.423684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.004042+0000","scrub_duration":0.001458972,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":75,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420377+0000","last_change":"2024-09-11T08:40:16.190092+0000","last_active":"2024-09-11T08:40:25.420377+0000","last_peered":"2024-09-11T08:40:25.420377+0000","last_clean":"2024-09-11T08:40:25.420377+0000","last_became_active":"2024-09-11T08:40:16.183867+0000","last_became_peered":"2024-09-11T08:40:16.183867+0000","last_unstale":"2024-09-11T08:40:25.420377+0000","last_undegraded":"2024-09-11T08:40:25.420377+0000","last_fullsized":"2024-09-11T08:40:25.420377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419129+0000","last_change":"2024-09-11T08:38:38.948244+0000","last_active":"2024-09-11T08:40:25.419129+0000","last_peered":"2024-09-11T08:40:25.419129+0000","last_clean":"2024-09-11T08:40:25.419129+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:40:25.419129+0000","last_undegraded":"2024-09-11T08:40:25.419129+0000","last_fullsized":"2024-09-11T08:40:25.419129+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223434+0000","last_change":"2024-09-11T08:40:03.430883+0000","last_active":"2024-09-11T08:40:25.223434+0000","last_peered":"2024-09-11T08:40:25.223434+0000","last_clean":"2024-09-11T08:40:25.223434+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:40:25.223434+0000","last_undegraded":"2024-09-11T08:40:25.223434+0000","last_fullsized":"2024-09-11T08:40:25.223434+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:03.430794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.430794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:20.081896+0000","scrub_duration":0.0014294430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:30.215968+0000","last_change":"2024-09-11T08:40:30.215943+0000","last_active":"2024-09-11T08:40:30.215968+0000","last_peered":"2024-09-11T08:40:30.215968+0000","last_clean":"2024-09-11T08:40:30.215968+0000","last_became_active":"2024-09-11T08:38:45.184410+0000","last_became_peered":"2024-09-11T08:38:45.184410+0000","last_unstale":"2024-09-11T08:40:30.215968+0000","last_undegraded":"2024-09-11T08:40:30.215968+0000","last_fullsized":"2024-09-11T08:40:30.215968+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:30.215890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:30.215890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:45.729251+0000","scrub_duration":0.0012073710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":72,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420528+0000","last_change":"2024-09-11T08:39:43.526845+0000","last_active":"2024-09-11T08:40:25.420528+0000","last_peered":"2024-09-11T08:40:25.420528+0000","last_clean":"2024-09-11T08:40:25.420528+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:40:25.420528+0000","last_undegraded":"2024-09-11T08:40:25.420528+0000","last_fullsized":"2024-09-11T08:40:25.420528+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:39:43.526763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.526763+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.869811+0000","scrub_duration":0.0022037329999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418766+0000","last_change":"2024-09-11T08:39:55.599049+0000","last_active":"2024-09-11T08:40:25.418766+0000","last_peered":"2024-09-11T08:40:25.418766+0000","last_clean":"2024-09-11T08:40:25.418766+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:40:25.418766+0000","last_undegraded":"2024-09-11T08:40:25.418766+0000","last_fullsized":"2024-09-11T08:40:25.418766+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.598961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.598961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.562190+0000","scrub_duration":0.0016789820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223145+0000","last_change":"2024-09-11T08:39:46.627077+0000","last_active":"2024-09-11T08:40:25.223145+0000","last_peered":"2024-09-11T08:40:25.223145+0000","last_clean":"2024-09-11T08:40:25.223145+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:40:25.223145+0000","last_undegraded":"2024-09-11T08:40:25.223145+0000","last_fullsized":"2024-09-11T08:40:25.223145+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:46.627028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:46.627028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:56.334041+0000","scrub_duration":0.0013824029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.414066+0000","last_change":"2024-09-11T08:40:13.462183+0000","last_active":"2024-09-11T08:40:25.414066+0000","last_peered":"2024-09-11T08:40:25.414066+0000","last_clean":"2024-09-11T08:40:25.414066+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:40:25.414066+0000","last_undegraded":"2024-09-11T08:40:25.414066+0000","last_fullsized":"2024-09-11T08:40:25.414066+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:13.462132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:13.462132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:33.998327+0000","scrub_duration":0.001365727,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419981+0000","last_change":"2024-09-11T08:38:40.969616+0000","last_active":"2024-09-11T08:40:25.419981+0000","last_peered":"2024-09-11T08:40:25.419981+0000","last_clean":"2024-09-11T08:40:25.419981+0000","last_became_active":"2024-09-11T08:38:40.969348+0000","last_became_peered":"2024-09-11T08:38:40.969348+0000","last_unstale":"2024-09-11T08:40:25.419981+0000","last_undegraded":"2024-09-11T08:40:25.419981+0000","last_fullsized":"2024-09-11T08:40:25.419981+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223230+0000","last_change":"2024-09-11T08:38:38.957214+0000","last_active":"2024-09-11T08:40:25.223230+0000","last_peered":"2024-09-11T08:40:25.223230+0000","last_clean":"2024-09-11T08:40:25.223230+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:40:25.223230+0000","last_undegraded":"2024-09-11T08:40:25.223230+0000","last_fullsized":"2024-09-11T08:40:25.223230+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:26.223752+0000","last_change":"2024-09-11T08:40:00.473606+0000","last_active":"2024-09-11T08:40:26.223752+0000","last_peered":"2024-09-11T08:40:26.223752+0000","last_clean":"2024-09-11T08:40:26.223752+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:40:26.223752+0000","last_undegraded":"2024-09-11T08:40:26.223752+0000","last_fullsized":"2024-09-11T08:40:26.223752+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:00.473549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.473549+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.300137+0000","scrub_duration":0.0016703060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420281+0000","last_change":"2024-09-11T08:38:44.993736+0000","last_active":"2024-09-11T08:40:25.420281+0000","last_peered":"2024-09-11T08:40:25.420281+0000","last_clean":"2024-09-11T08:40:25.420281+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:40:25.420281+0000","last_undegraded":"2024-09-11T08:40:25.420281+0000","last_fullsized":"2024-09-11T08:40:25.420281+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417888+0000","last_change":"2024-09-11T08:38:40.977113+0000","last_active":"2024-09-11T08:40:25.417888+0000","last_peered":"2024-09-11T08:40:25.417888+0000","last_clean":"2024-09-11T08:40:25.417888+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:40:25.417888+0000","last_undegraded":"2024-09-11T08:40:25.417888+0000","last_fullsized":"2024-09-11T08:40:25.417888+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223196+0000","last_change":"2024-09-11T08:39:50.452568+0000","last_active":"2024-09-11T08:40:25.223196+0000","last_peered":"2024-09-11T08:40:25.223196+0000","last_clean":"2024-09-11T08:40:25.223196+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:40:25.223196+0000","last_undegraded":"2024-09-11T08:40:25.223196+0000","last_fullsized":"2024-09-11T08:40:25.223196+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.452494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.631278+0000","scrub_duration":0.001547086,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.413842+0000","last_change":"2024-09-11T08:40:09.519072+0000","last_active":"2024-09-11T08:40:25.413842+0000","last_peered":"2024-09-11T08:40:25.413842+0000","last_clean":"2024-09-11T08:40:25.413842+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:40:25.413842+0000","last_undegraded":"2024-09-11T08:40:25.413842+0000","last_fullsized":"2024-09-11T08:40:25.413842+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:09.519019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:09.519019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.000837+0000","scrub_duration":0.001232471,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417656+0000","last_change":"2024-09-11T08:38:44.991908+0000","last_active":"2024-09-11T08:40:25.417656+0000","last_peered":"2024-09-11T08:40:25.417656+0000","last_clean":"2024-09-11T08:40:25.417656+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:40:25.417656+0000","last_undegraded":"2024-09-11T08:40:25.417656+0000","last_fullsized":"2024-09-11T08:40:25.417656+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":74,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418424+0000","last_change":"2024-09-11T08:39:45.575209+0000","last_active":"2024-09-11T08:40:25.418424+0000","last_peered":"2024-09-11T08:40:25.418424+0000","last_clean":"2024-09-11T08:40:25.418424+0000","last_became_active":"2024-09-11T08:38:41.040207+0000","last_became_peered":"2024-09-11T08:38:41.040207+0000","last_unstale":"2024-09-11T08:40:25.418424+0000","last_undegraded":"2024-09-11T08:40:25.418424+0000","last_fullsized":"2024-09-11T08:40:25.418424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:39:45.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.575133+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.182090+0000","scrub_duration":0.0022792160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419947+0000","last_change":"2024-09-11T08:39:56.112112+0000","last_active":"2024-09-11T08:40:25.419947+0000","last_peered":"2024-09-11T08:40:25.419947+0000","last_clean":"2024-09-11T08:40:25.419947+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:40:25.419947+0000","last_undegraded":"2024-09-11T08:40:25.419947+0000","last_fullsized":"2024-09-11T08:40:25.419947+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:56.112043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.112043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:25.809354+0000","scrub_duration":0.001517137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.413787+0000","last_change":"2024-09-11T08:40:11.540833+0000","last_active":"2024-09-11T08:40:25.413787+0000","last_peered":"2024-09-11T08:40:25.413787+0000","last_clean":"2024-09-11T08:40:25.413787+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:40:25.413787+0000","last_undegraded":"2024-09-11T08:40:25.413787+0000","last_fullsized":"2024-09-11T08:40:25.413787+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:11.540760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:11.540760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.937538+0000","scrub_duration":0.0014212529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:26.222721+0000","last_change":"2024-09-11T08:40:26.222640+0000","last_active":"2024-09-11T08:40:26.222721+0000","last_peered":"2024-09-11T08:40:26.222721+0000","last_clean":"2024-09-11T08:40:26.222721+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:40:26.222721+0000","last_undegraded":"2024-09-11T08:40:26.222721+0000","last_fullsized":"2024-09-11T08:40:26.222721+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:26.222567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:26.222567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:54.940352+0000","scrub_duration":0.001672593,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418760+0000","last_change":"2024-09-11T08:39:40.524581+0000","last_active":"2024-09-11T08:40:25.418760+0000","last_peered":"2024-09-11T08:40:25.418760+0000","last_clean":"2024-09-11T08:40:25.418760+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:40:25.418760+0000","last_undegraded":"2024-09-11T08:40:25.418760+0000","last_fullsized":"2024-09-11T08:40:25.418760+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:40.524498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:40.524498+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.186922+0000","scrub_duration":0.0024035010000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420067+0000","last_change":"2024-09-11T08:40:00.113587+0000","last_active":"2024-09-11T08:40:25.420067+0000","last_peered":"2024-09-11T08:40:25.420067+0000","last_clean":"2024-09-11T08:40:25.420067+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:40:25.420067+0000","last_undegraded":"2024-09-11T08:40:25.420067+0000","last_fullsized":"2024-09-11T08:40:25.420067+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.113534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.113534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.476975+0000","scrub_duration":0.0011790780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.413695+0000","last_change":"2024-09-11T08:39:57.454257+0000","last_active":"2024-09-11T08:40:25.413695+0000","last_peered":"2024-09-11T08:40:25.413695+0000","last_clean":"2024-09-11T08:40:25.413695+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:40:25.413695+0000","last_undegraded":"2024-09-11T08:40:25.413695+0000","last_fullsized":"2024-09-11T08:40:25.413695+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.454198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.454198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.731898+0000","scrub_duration":0.0011666669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223108+0000","last_change":"2024-09-11T08:38:44.998686+0000","last_active":"2024-09-11T08:40:25.223108+0000","last_peered":"2024-09-11T08:40:25.223108+0000","last_clean":"2024-09-11T08:40:25.223108+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:40:25.223108+0000","last_undegraded":"2024-09-11T08:40:25.223108+0000","last_fullsized":"2024-09-11T08:40:25.223108+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222617+0000","last_change":"2024-09-11T08:39:48.462233+0000","last_active":"2024-09-11T08:40:25.222617+0000","last_peered":"2024-09-11T08:40:25.222617+0000","last_clean":"2024-09-11T08:40:25.222617+0000","last_became_active":"2024-09-11T08:38:41.040475+0000","last_became_peered":"2024-09-11T08:38:41.040475+0000","last_unstale":"2024-09-11T08:40:25.222617+0000","last_undegraded":"2024-09-11T08:40:25.222617+0000","last_fullsized":"2024-09-11T08:40:25.222617+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-11T08:39:48.462163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:48.462163+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.032207+0000","scrub_duration":0.002094753,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419804+0000","last_change":"2024-09-11T08:40:03.143806+0000","last_active":"2024-09-11T08:40:25.419804+0000","last_peered":"2024-09-11T08:40:25.419804+0000","last_clean":"2024-09-11T08:40:25.419804+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:40:25.419804+0000","last_undegraded":"2024-09-11T08:40:25.419804+0000","last_fullsized":"2024-09-11T08:40:25.419804+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:40:03.143742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.143742+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.570435+0000","scrub_duration":0.001564396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"63'8","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:27.148969+0000","last_change":"2024-09-11T08:40:13.221708+0000","last_active":"2024-09-11T08:40:27.148969+0000","last_peered":"2024-09-11T08:40:27.148969+0000","last_clean":"2024-09-11T08:40:27.148969+0000","last_became_active":"2024-09-11T08:38:42.986756+0000","last_became_peered":"2024-09-11T08:38:42.986756+0000","last_unstale":"2024-09-11T08:40:27.148969+0000","last_undegraded":"2024-09-11T08:40:27.148969+0000","last_fullsized":"2024-09-11T08:40:27.148969+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:13.221650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:13.221650+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:28.234147+0000","scrub_duration":0.0017036169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222829+0000","last_change":"2024-09-11T08:40:11.370313+0000","last_active":"2024-09-11T08:40:25.222829+0000","last_peered":"2024-09-11T08:40:25.222829+0000","last_clean":"2024-09-11T08:40:25.222829+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:40:25.222829+0000","last_undegraded":"2024-09-11T08:40:25.222829+0000","last_fullsized":"2024-09-11T08:40:25.222829+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:11.370245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:11.370245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:35.188374+0000","scrub_duration":0.0011869179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418548+0000","last_change":"2024-09-11T08:39:42.605373+0000","last_active":"2024-09-11T08:40:25.418548+0000","last_peered":"2024-09-11T08:40:25.418548+0000","last_clean":"2024-09-11T08:40:25.418548+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:40:25.418548+0000","last_undegraded":"2024-09-11T08:40:25.418548+0000","last_fullsized":"2024-09-11T08:40:25.418548+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:42.605301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:42.605301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:53.937210+0000","scrub_duration":0.0014798509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223866+0000","last_change":"2024-09-11T08:40:17.096092+0000","last_active":"2024-09-11T08:40:25.223866+0000","last_peered":"2024-09-11T08:40:25.223866+0000","last_clean":"2024-09-11T08:40:25.223866+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:40:25.223866+0000","last_undegraded":"2024-09-11T08:40:25.223866+0000","last_fullsized":"2024-09-11T08:40:25.223866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-11T08:40:17.096034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:17.096034+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:43.002314+0000","scrub_duration":0.0019684910000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223471+0000","last_change":"2024-09-11T08:39:49.472764+0000","last_active":"2024-09-11T08:40:25.223471+0000","last_peered":"2024-09-11T08:40:25.223471+0000","last_clean":"2024-09-11T08:40:25.223471+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:40:25.223471+0000","last_undegraded":"2024-09-11T08:40:25.223471+0000","last_fullsized":"2024-09-11T08:40:25.223471+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.472696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.472696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:01.403308+0000","scrub_duration":0.001919104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419788+0000","last_change":"2024-09-11T08:38:45.185016+0000","last_active":"2024-09-11T08:40:25.419788+0000","last_peered":"2024-09-11T08:40:25.419788+0000","last_clean":"2024-09-11T08:40:25.419788+0000","last_became_active":"2024-09-11T08:38:45.184753+0000","last_became_peered":"2024-09-11T08:38:45.184753+0000","last_unstale":"2024-09-11T08:40:25.419788+0000","last_undegraded":"2024-09-11T08:40:25.419788+0000","last_fullsized":"2024-09-11T08:40:25.419788+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418143+0000","last_change":"2024-09-11T08:38:41.041476+0000","last_active":"2024-09-11T08:40:25.418143+0000","last_peered":"2024-09-11T08:40:25.418143+0000","last_clean":"2024-09-11T08:40:25.418143+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:40:25.418143+0000","last_undegraded":"2024-09-11T08:40:25.418143+0000","last_fullsized":"2024-09-11T08:40:25.418143+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418789+0000","last_change":"2024-09-11T08:39:53.176095+0000","last_active":"2024-09-11T08:40:25.418789+0000","last_peered":"2024-09-11T08:40:25.418789+0000","last_clean":"2024-09-11T08:40:25.418789+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:40:25.418789+0000","last_undegraded":"2024-09-11T08:40:25.418789+0000","last_fullsized":"2024-09-11T08:40:25.418789+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.176035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.176035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:00.682857+0000","scrub_duration":0.001147531,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:28.199712+0000","last_change":"2024-09-11T08:40:28.199687+0000","last_active":"2024-09-11T08:40:28.199712+0000","last_peered":"2024-09-11T08:40:28.199712+0000","last_clean":"2024-09-11T08:40:28.199712+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:40:28.199712+0000","last_undegraded":"2024-09-11T08:40:28.199712+0000","last_fullsized":"2024-09-11T08:40:28.199712+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:28.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:28.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:57.722333+0000","scrub_duration":0.001581287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.413880+0000","last_change":"2024-09-11T08:39:52.493064+0000","last_active":"2024-09-11T08:40:25.413880+0000","last_peered":"2024-09-11T08:40:25.413880+0000","last_clean":"2024-09-11T08:40:25.413880+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:40:25.413880+0000","last_undegraded":"2024-09-11T08:40:25.413880+0000","last_fullsized":"2024-09-11T08:40:25.413880+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.493013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:13.717323+0000","scrub_duration":0.0012699160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":60,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:30.270690+0000","last_change":"2024-09-11T08:40:30.270654+0000","last_active":"2024-09-11T08:40:30.270690+0000","last_peered":"2024-09-11T08:40:30.270690+0000","last_clean":"2024-09-11T08:40:30.270690+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:40:30.270690+0000","last_undegraded":"2024-09-11T08:40:30.270690+0000","last_fullsized":"2024-09-11T08:40:30.270690+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:40:30.270582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:30.270582+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:35.812455+0000","scrub_duration":0.0025147979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417760+0000","last_change":"2024-09-11T08:40:20.467510+0000","last_active":"2024-09-11T08:40:25.417760+0000","last_peered":"2024-09-11T08:40:25.417760+0000","last_clean":"2024-09-11T08:40:25.417760+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:40:25.417760+0000","last_undegraded":"2024-09-11T08:40:25.417760+0000","last_fullsized":"2024-09-11T08:40:25.417760+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:20.467456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:20.467456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:39.736643+0000","scrub_duration":0.001886544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:26.226480+0000","last_change":"2024-09-11T08:40:25.104280+0000","last_active":"2024-09-11T08:40:26.226480+0000","last_peered":"2024-09-11T08:40:26.226480+0000","last_clean":"2024-09-11T08:40:26.226480+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:40:26.226480+0000","last_undegraded":"2024-09-11T08:40:26.226480+0000","last_fullsized":"2024-09-11T08:40:26.226480+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:25.104224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:25.104224+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:46.893915+0000","scrub_duration":0.001645695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420367+0000","last_change":"2024-09-11T08:38:44.993997+0000","last_active":"2024-09-11T08:40:25.420367+0000","last_peered":"2024-09-11T08:40:25.420367+0000","last_clean":"2024-09-11T08:40:25.420367+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:40:25.420367+0000","last_undegraded":"2024-09-11T08:40:25.420367+0000","last_fullsized":"2024-09-11T08:40:25.420367+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223541+0000","last_change":"2024-09-11T08:40:22.076288+0000","last_active":"2024-09-11T08:40:25.223541+0000","last_peered":"2024-09-11T08:40:25.223541+0000","last_clean":"2024-09-11T08:40:25.223541+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:40:25.223541+0000","last_undegraded":"2024-09-11T08:40:25.223541+0000","last_fullsized":"2024-09-11T08:40:25.223541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:40:22.076198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.076198+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:48.206403+0000","scrub_duration":0.0024994370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419508+0000","last_change":"2024-09-11T08:40:14.202552+0000","last_active":"2024-09-11T08:40:25.419508+0000","last_peered":"2024-09-11T08:40:25.419508+0000","last_clean":"2024-09-11T08:40:25.419508+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:40:25.419508+0000","last_undegraded":"2024-09-11T08:40:25.419508+0000","last_fullsized":"2024-09-11T08:40:25.419508+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:14.202498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:14.202498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:42.406451+0000","scrub_duration":0.001204299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:26.224562+0000","last_change":"2024-09-11T08:39:44.429546+0000","last_active":"2024-09-11T08:40:26.224562+0000","last_peered":"2024-09-11T08:40:26.224562+0000","last_clean":"2024-09-11T08:40:26.224562+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:40:26.224562+0000","last_undegraded":"2024-09-11T08:40:26.224562+0000","last_fullsized":"2024-09-11T08:40:26.224562+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:39:44.429477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:44.429477+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.023544+0000","scrub_duration":0.001890784,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417905+0000","last_change":"2024-09-11T08:39:52.254299+0000","last_active":"2024-09-11T08:40:25.417905+0000","last_peered":"2024-09-11T08:40:25.417905+0000","last_clean":"2024-09-11T08:40:25.417905+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:40:25.417905+0000","last_undegraded":"2024-09-11T08:40:25.417905+0000","last_fullsized":"2024-09-11T08:40:25.417905+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.254245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.254245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:20.782062+0000","scrub_duration":0.001581877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419998+0000","last_change":"2024-09-11T08:38:40.973815+0000","last_active":"2024-09-11T08:40:25.419998+0000","last_peered":"2024-09-11T08:40:25.419998+0000","last_clean":"2024-09-11T08:40:25.419998+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:40:25.419998+0000","last_undegraded":"2024-09-11T08:40:25.419998+0000","last_fullsized":"2024-09-11T08:40:25.419998+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420081+0000","last_change":"2024-09-11T08:38:38.952604+0000","last_active":"2024-09-11T08:40:25.420081+0000","last_peered":"2024-09-11T08:40:25.420081+0000","last_clean":"2024-09-11T08:40:25.420081+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:40:25.420081+0000","last_undegraded":"2024-09-11T08:40:25.420081+0000","last_fullsized":"2024-09-11T08:40:25.420081+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:26.218290+0000","last_change":"2024-09-11T08:40:26.218247+0000","last_active":"2024-09-11T08:40:26.218290+0000","last_peered":"2024-09-11T08:40:26.218290+0000","last_clean":"2024-09-11T08:40:26.218290+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:40:26.218290+0000","last_undegraded":"2024-09-11T08:40:26.218290+0000","last_fullsized":"2024-09-11T08:40:26.218290+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:26.218191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:26.218191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:48.264069+0000","scrub_duration":0.0016980210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222401+0000","last_change":"2024-09-11T08:38:44.994241+0000","last_active":"2024-09-11T08:40:25.222401+0000","last_peered":"2024-09-11T08:40:25.222401+0000","last_clean":"2024-09-11T08:40:25.222401+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:40:25.222401+0000","last_undegraded":"2024-09-11T08:40:25.222401+0000","last_fullsized":"2024-09-11T08:40:25.222401+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222744+0000","last_change":"2024-09-11T08:40:22.382123+0000","last_active":"2024-09-11T08:40:25.222744+0000","last_peered":"2024-09-11T08:40:25.222744+0000","last_clean":"2024-09-11T08:40:25.222744+0000","last_became_active":"2024-09-11T08:40:16.181181+0000","last_became_peered":"2024-09-11T08:40:16.181181+0000","last_unstale":"2024-09-11T08:40:25.222744+0000","last_undegraded":"2024-09-11T08:40:25.222744+0000","last_fullsized":"2024-09-11T08:40:25.222744+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:40:22.382048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.382048+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.268329+0000","scrub_duration":0.001806862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:28.273976+0000","last_change":"2024-09-11T08:40:28.273944+0000","last_active":"2024-09-11T08:40:28.273976+0000","last_peered":"2024-09-11T08:40:28.273976+0000","last_clean":"2024-09-11T08:40:28.273976+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:40:28.273976+0000","last_undegraded":"2024-09-11T08:40:28.273976+0000","last_fullsized":"2024-09-11T08:40:28.273976+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:28.273889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:28.273889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:42.340922+0000","scrub_duration":0.0011932570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223136+0000","last_change":"2024-09-11T08:40:24.089168+0000","last_active":"2024-09-11T08:40:25.223136+0000","last_peered":"2024-09-11T08:40:25.223136+0000","last_clean":"2024-09-11T08:40:25.223136+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:40:25.223136+0000","last_undegraded":"2024-09-11T08:40:25.223136+0000","last_fullsized":"2024-09-11T08:40:25.223136+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:24.089115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:24.089115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:27.976340+0000","scrub_duration":0.001138305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417516+0000","last_change":"2024-09-11T08:40:22.499148+0000","last_active":"2024-09-11T08:40:25.417516+0000","last_peered":"2024-09-11T08:40:25.417516+0000","last_clean":"2024-09-11T08:40:25.417516+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:40:25.417516+0000","last_undegraded":"2024-09-11T08:40:25.417516+0000","last_fullsized":"2024-09-11T08:40:25.417516+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:22.499088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.499088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.706106+0000","scrub_duration":0.0015631830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":52,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222515+0000","last_change":"2024-09-11T08:38:41.039878+0000","last_active":"2024-09-11T08:40:25.222515+0000","last_peered":"2024-09-11T08:40:25.222515+0000","last_clean":"2024-09-11T08:40:25.222515+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:40:25.222515+0000","last_undegraded":"2024-09-11T08:40:25.222515+0000","last_fullsized":"2024-09-11T08:40:25.222515+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223462+0000","last_change":"2024-09-11T08:40:05.430740+0000","last_active":"2024-09-11T08:40:25.223462+0000","last_peered":"2024-09-11T08:40:25.223462+0000","last_clean":"2024-09-11T08:40:25.223462+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:40:25.223462+0000","last_undegraded":"2024-09-11T08:40:25.223462+0000","last_fullsized":"2024-09-11T08:40:25.223462+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:05.430666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:05.430666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.818673+0000","scrub_duration":0.001547629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418844+0000","last_change":"2024-09-11T08:39:47.104702+0000","last_active":"2024-09-11T08:40:25.418844+0000","last_peered":"2024-09-11T08:40:25.418844+0000","last_clean":"2024-09-11T08:40:25.418844+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:40:25.418844+0000","last_undegraded":"2024-09-11T08:40:25.418844+0000","last_fullsized":"2024-09-11T08:40:25.418844+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.104637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.104637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:51.311803+0000","scrub_duration":0.0015238020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223207+0000","last_change":"2024-09-11T08:38:45.183913+0000","last_active":"2024-09-11T08:40:25.223207+0000","last_peered":"2024-09-11T08:40:25.223207+0000","last_clean":"2024-09-11T08:40:25.223207+0000","last_became_active":"2024-09-11T08:38:45.183548+0000","last_became_peered":"2024-09-11T08:38:45.183548+0000","last_unstale":"2024-09-11T08:40:25.223207+0000","last_undegraded":"2024-09-11T08:40:25.223207+0000","last_fullsized":"2024-09-11T08:40:25.223207+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222427+0000","last_change":"2024-09-11T08:39:41.989740+0000","last_active":"2024-09-11T08:40:25.222427+0000","last_peered":"2024-09-11T08:40:25.222427+0000","last_clean":"2024-09-11T08:40:25.222427+0000","last_became_active":"2024-09-11T08:38:40.973799+0000","last_became_peered":"2024-09-11T08:38:40.973799+0000","last_unstale":"2024-09-11T08:40:25.222427+0000","last_undegraded":"2024-09-11T08:40:25.222427+0000","last_fullsized":"2024-09-11T08:40:25.222427+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-11T08:39:41.989684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.989684+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.687182+0000","scrub_duration":0.00162959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418167+0000","last_change":"2024-09-11T08:40:08.336369+0000","last_active":"2024-09-11T08:40:25.418167+0000","last_peered":"2024-09-11T08:40:25.418167+0000","last_clean":"2024-09-11T08:40:25.418167+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:40:25.418167+0000","last_undegraded":"2024-09-11T08:40:25.418167+0000","last_fullsized":"2024-09-11T08:40:25.418167+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:40:08.336314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.336314+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:16.115575+0000","scrub_duration":0.001394085,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418539+0000","last_change":"2024-09-11T08:39:53.541628+0000","last_active":"2024-09-11T08:40:25.418539+0000","last_peered":"2024-09-11T08:40:25.418539+0000","last_clean":"2024-09-11T08:40:25.418539+0000","last_became_active":"2024-09-11T08:38:42.989556+0000","last_became_peered":"2024-09-11T08:38:42.989556+0000","last_unstale":"2024-09-11T08:40:25.418539+0000","last_undegraded":"2024-09-11T08:40:25.418539+0000","last_fullsized":"2024-09-11T08:40:25.418539+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.541557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.541557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:08.967817+0000","scrub_duration":0.001310285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418891+0000","last_change":"2024-09-11T08:40:07.114884+0000","last_active":"2024-09-11T08:40:25.418891+0000","last_peered":"2024-09-11T08:40:25.418891+0000","last_clean":"2024-09-11T08:40:25.418891+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:40:25.418891+0000","last_undegraded":"2024-09-11T08:40:25.418891+0000","last_fullsized":"2024-09-11T08:40:25.418891+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:07.114815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:07.114815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.897957+0000","scrub_duration":0.0013781500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222869+0000","last_change":"2024-09-11T08:39:52.480663+0000","last_active":"2024-09-11T08:40:25.222869+0000","last_peered":"2024-09-11T08:40:25.222869+0000","last_clean":"2024-09-11T08:40:25.222869+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:40:25.222869+0000","last_undegraded":"2024-09-11T08:40:25.222869+0000","last_fullsized":"2024-09-11T08:40:25.222869+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.480610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.480610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:55.608483+0000","scrub_duration":0.001335754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222617+0000","last_change":"2024-09-11T08:39:53.038356+0000","last_active":"2024-09-11T08:40:25.222617+0000","last_peered":"2024-09-11T08:40:25.222617+0000","last_clean":"2024-09-11T08:40:25.222617+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:40:25.222617+0000","last_undegraded":"2024-09-11T08:40:25.222617+0000","last_fullsized":"2024-09-11T08:40:25.222617+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.038297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.038297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:58.424182+0000","scrub_duration":0.0012198549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419494+0000","last_change":"2024-09-11T08:40:02.551056+0000","last_active":"2024-09-11T08:40:25.419494+0000","last_peered":"2024-09-11T08:40:25.419494+0000","last_clean":"2024-09-11T08:40:25.419494+0000","last_became_active":"2024-09-11T08:38:40.974451+0000","last_became_peered":"2024-09-11T08:38:40.974451+0000","last_unstale":"2024-09-11T08:40:25.419494+0000","last_undegraded":"2024-09-11T08:40:25.419494+0000","last_fullsized":"2024-09-11T08:40:25.419494+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:40:02.550999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.550999+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.173446+0000","scrub_duration":0.0016484399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418027+0000","last_change":"2024-09-11T08:39:55.265155+0000","last_active":"2024-09-11T08:40:25.418027+0000","last_peered":"2024-09-11T08:40:25.418027+0000","last_clean":"2024-09-11T08:40:25.418027+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:40:25.418027+0000","last_undegraded":"2024-09-11T08:40:25.418027+0000","last_fullsized":"2024-09-11T08:40:25.418027+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.265097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.265097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.333633+0000","scrub_duration":0.0012893830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":423,"ondisk_log_size":423,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":100,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299420,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298636,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446516,"statfs":{"total":95995035648,"available":95689232384,"internally_reserved":0,"allocated":1310720,"data_stored":948466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.499,"15min":0.501},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.66300000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71599999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.36899999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.50700000000000001},"min":{"1min":0.32700000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.74099999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.39800000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.46000000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71799999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.53000000000000003},"min":{"1min":0.377,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70299999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.33700000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.42499999999999999,"15min":0.42199999999999999},"min":{"1min":0.318,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.59399999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.35599999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.30099999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.379},{"interface":"front","average":{"1min":0.495,"5min":0.44700000000000001,"15min":0.439},"min":{"1min":0.313,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.34699999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47399999999999998,"15min":0.46899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.40899999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528353,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298620,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446532,"statfs":{"total":95995035648,"available":95689248768,"internally_reserved":0,"allocated":1290240,"data_stored":947693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.71699999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.499}]},{"osd":1,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.59799999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.70799999999999996,"5min":0.871,"15min":0.871},"last":0.65000000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70999999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.66200000000000003,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66800000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.86199999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.27500000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40600000000000003},"min":{"1min":0.29199999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.45700000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.439,"15min":0.43099999999999999},"min":{"1min":0.34200000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.435,"15min":0.42999999999999999},"min":{"1min":0.30299999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.48699999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:40:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.44400000000000001,"15min":0.432},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.88800000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757285,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298056,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447096,"statfs":{"total":95995035648,"available":95689826304,"internally_reserved":0,"allocated":716800,"data_stored":358731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.51300000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.47099999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56000000000000005}]},{"osd":3,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.626}]},{"osd":4,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.376},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.58799999999999997}]},{"osd":6,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.42599999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73099999999999998,"5min":0.749,"15min":0.749},"last":0.65100000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:40:27 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.60799999999999998,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.40899999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.63700000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986217,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298048,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447104,"statfs":{"total":95995035648,"available":95689834496,"internally_reserved":0,"allocated":712704,"data_stored":357316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55500000000000005,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.41699999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.38,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.39900000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.39600000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.443}]},{"osd":3,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55800000000000005,"15min":0.57799999999999996},"min":{"1min":0.38800000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72199999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.39000000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.40500000000000003,"15min":0.38700000000000001},"min":{"1min":0.32100000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.30099999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.29499999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:40:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.32400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.31},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.36199999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67100000000000004,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.32900000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:40:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48899999999999999,"15min":0.48099999999999998},"min":{"1min":0.30199999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.47599999999999998,"15min":0.46800000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.36899999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149743,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298076,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447076,"statfs":{"total":95995035648,"available":95689805824,"internally_reserved":0,"allocated":741376,"data_stored":359141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.36299999999999999,"15min":0.374},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.443,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.38900000000000001,"15min":0.40000000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.496,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.34100000000000003}]},{"osd":1,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.39200000000000002,"15min":0.40400000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.56200000000000006,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.40400000000000003,"15min":0.41399999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48199999999999998,"5min":0.621,"15min":0.621},"last":0.314}]},{"osd":2,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.42699999999999999,"15min":0.435},"min":{"1min":0.27100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.44,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.375},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.35799999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:40:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79900000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54900000000000004,"15min":0.55900000000000005},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66600000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46800000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.442},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.41799999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76300000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.434}]},{"osd":7,"last update":"Wed Sep 11 08:40:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60999999999999999,"15min":0.61799999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86799999999999999,"5min":1.22,"15min":1.22},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.86099999999999999,"5min":1.125,"15min":1.125},"last":0.45200000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298036,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447116,"statfs":{"total":95995035648,"available":95689846784,"internally_reserved":0,"allocated":696320,"data_stored":357271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36499999999999999,"15min":0.35399999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.38900000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.25800000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.372},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.39100000000000001,"15min":0.376},"min":{"1min":0.28999999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.29199999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:39:57 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.441,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.34499999999999997}]},{"osd":4,"last update":"Wed Sep 11 08:40:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.751,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.48699999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:39:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed Sep 11 08:40:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.442,"5min":0.432,"15min":0.432},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.56100000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57499999999999996}]},{"osd":7,"last update":"Wed Sep 11 08:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.41699999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76200000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.496},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81799999999999995,"5min":1.024,"15min":1.024},"last":0.56599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298080,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447072,"statfs":{"total":95995035648,"available":95689801728,"internally_reserved":0,"allocated":741376,"data_stored":358205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:19 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.314},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.47399999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.376},{"interface":"front","average":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33300000000000002},"min":{"1min":0.27400000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.42299999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:39:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:39:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40400000000000003,"15min":0.41999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54400000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.433,"15min":0.434},"min":{"1min":0.307,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.44700000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:40:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.76500000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55100000000000005,"15min":0.56499999999999995},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.63300000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.60899999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:39:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.755,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82199999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.59599999999999997}]},{"osd":6,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57399999999999995,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.63},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60099999999999998,"15min":0.60599999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.67700000000000005}]},{"osd":7,"last update":"Wed Sep 11 08:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.45600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79700000000000004,"5min":0.879,"15min":0.879},"last":0.68600000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298640,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446512,"statfs":{"total":95995035648,"available":95689228288,"internally_reserved":0,"allocated":1314816,"data_stored":948525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:40:10 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.311},"min":{"1min":0.19800000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.436,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34000000000000002,"15min":0.314},"min":{"1min":0.23999999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.504,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.27400000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.35799999999999998,"15min":0.34699999999999998},"min":{"1min":0.24199999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53500000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.34899999999999998},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.51500000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.317}]},{"osd":3,"last update":"Wed Sep 11 08:39:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39800000000000002,"15min":0.40000000000000002},"min":{"1min":0.28499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.503,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.39200000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70899999999999996,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.41699999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.43099999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:40:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.37,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68300000000000005,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.42499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.746,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48799999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:40:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.47699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:40:33.733 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:40:33.476Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:40:33.733 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:40:33.476Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T08:40:33.733 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:40:33.476Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T08:40:33.733 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:40:33.476Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T08:40:33.733 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:40:33.476Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T08:40:33.733 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:40:33.476Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T08:40:33.734 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:40:33.476Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T08:40:33.734 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:40:33.476Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T08:40:33.734 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:40:33.477Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T08:40:33.734 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:40:33.477Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T08:40:33.734 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:40:33.477Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T08:40:33.734 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[64515]: ts=2024-09-11T08:40:33.477Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T08:40:33.734 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 podman[67726]: 2024-09-11 08:40:33.487045323 +0000 UTC m=+0.157388707 container died c5d6ca7549fa7d0af93af4010d652147caa225b6004091609d23d17160020799 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:40:34.091 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 podman[67726]: 2024-09-11 08:40:33.832270377 +0000 UTC m=+0.502613750 container cleanup c5d6ca7549fa7d0af93af4010d652147caa225b6004091609d23d17160020799 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:40:34.091 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:33 smithi145 bash[67726]: ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T08:40:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-mon[33384]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-mon[33384]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-mon[33384]: 4.16 scrub starts 2024-09-11T08:40:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-mon[33384]: 4.16 scrub ok 2024-09-11T08:40:34.412 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 podman[67742]: 2024-09-11 08:40:34.092192812 +0000 UTC m=+0.607271288 container remove c5d6ca7549fa7d0af93af4010d652147caa225b6004091609d23d17160020799 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:40:34.412 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-11T08:40:34.412 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 systemd[1]: Stopped Ceph prometheus.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:40:34.412 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.083s CPU time. 2024-09-11T08:40:34.412 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 systemd[1]: Starting Ceph prometheus.a for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T08:40:34.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:34 smithi092 ceph-mon[29647]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:34.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:34 smithi092 ceph-mon[29647]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:34.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:34 smithi092 ceph-mon[29647]: 4.16 scrub starts 2024-09-11T08:40:34.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:34 smithi092 ceph-mon[29647]: 4.16 scrub ok 2024-09-11T08:40:34.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:34 smithi092 ceph-mon[41114]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:34.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:34 smithi092 ceph-mon[41114]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:34.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:34 smithi092 ceph-mon[41114]: 4.16 scrub starts 2024-09-11T08:40:34.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:34 smithi092 ceph-mon[41114]: 4.16 scrub ok 2024-09-11T08:40:34.711 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 podman[67843]: 2024-09-11 08:40:34.353449835 +0000 UTC m=+0.017027960 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T08:40:34.711 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 podman[67843]: 2024-09-11 08:40:34.476579445 +0000 UTC m=+0.140157574 container create 5220b1a6b4555900dfa461815783f66aa11382e4016ec968d4d8fffd4ba5126d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:40:34.711 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 podman[67843]: 2024-09-11 08:40:34.7119309 +0000 UTC m=+0.375509022 container init 5220b1a6b4555900dfa461815783f66aa11382e4016ec968d4d8fffd4ba5126d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:40:35.111 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:34 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:34] ENGINE Bus STOPPING 2024-09-11T08:40:35.112 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:35] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:40:35.112 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:35] ENGINE Bus STOPPED 2024-09-11T08:40:35.112 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:35] ENGINE Bus STARTING 2024-09-11T08:40:35.112 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:35] ENGINE Serving on http://:::9283 2024-09-11T08:40:35.162 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 podman[67843]: 2024-09-11 08:40:34.716572916 +0000 UTC m=+0.380151037 container start 5220b1a6b4555900dfa461815783f66aa11382e4016ec968d4d8fffd4ba5126d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:40:35.162 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.740Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T08:40:35.162 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.740Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T08:40:35.162 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.740Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi145 (none))" 2024-09-11T08:40:35.162 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.740Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T08:40:35.163 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.740Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T08:40:35.163 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.742Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T08:40:35.163 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.742Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T08:40:35.163 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.743Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T08:40:35.163 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.743Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T08:40:35.163 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.744Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T08:40:35.163 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.744Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.505µs 2024-09-11T08:40:35.163 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.744Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T08:40:35.163 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.746Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-11T08:40:35.163 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.746Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-11T08:40:35.163 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.746Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=17.381µs wal_replay_duration=1.873716ms wbl_replay_duration=112ns total_replay_duration=1.907083ms 2024-09-11T08:40:35.163 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.747Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T08:40:35.163 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.747Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T08:40:35.164 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.747Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T08:40:35.164 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.761Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.100526ms db_storage=715ns remote_storage=1.12µs web_handler=361ns query_engine=418ns scrape=388.745µs scrape_sd=27.048µs notify=24.148µs notify_sd=6.497µs rules=13.421357ms tracing=5.426µs 2024-09-11T08:40:35.164 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.761Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T08:40:35.164 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T08:40:34.761Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T08:40:35.164 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 bash[67843]: 5220b1a6b4555900dfa461815783f66aa11382e4016ec968d4d8fffd4ba5126d 2024-09-11T08:40:35.164 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 08:40:34 smithi145 systemd[1]: Started Ceph prometheus.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T08:40:35.518 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:35] ENGINE Bus STARTED 2024-09-11T08:40:35.518 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:35] ENGINE Bus STOPPING 2024-09-11T08:40:35.807 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:35] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:40:35.807 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:35] ENGINE Bus STOPPED 2024-09-11T08:40:35.807 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:35] ENGINE Bus STARTING 2024-09-11T08:40:35.807 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:35] ENGINE Serving on http://:::9283 2024-09-11T08:40:35.807 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:35] ENGINE Bus STARTED 2024-09-11T08:40:35.807 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:35] ENGINE Bus STOPPING 2024-09-11T08:40:36.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: 1.0 scrub starts 2024-09-11T08:40:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: 1.0 scrub ok 2024-09-11T08:40:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: 4.c scrub starts 2024-09-11T08:40:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: 4.c scrub ok 2024-09-11T08:40:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:40:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:40:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:40:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:40:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:40:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:40:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi145.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:40:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi145.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:40:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:40:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:40:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi145.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:40:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi145.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:40:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:40:36.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[41114]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:40:36.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: 1.0 scrub starts 2024-09-11T08:40:36.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: 1.0 scrub ok 2024-09-11T08:40:36.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: 4.c scrub starts 2024-09-11T08:40:36.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: 4.c scrub ok 2024-09-11T08:40:36.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:36.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:36.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:40:36.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:40:36.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:40:36.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:40:36.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:36.069 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:40:36.069 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:40:36.069 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi145.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:40:36.069 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi145.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:40:36.069 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:36.069 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:40:36.069 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:40:36.069 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi145.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:40:36.069 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi145.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:40:36.069 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:36.069 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:40:36.069 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:35 smithi092 ceph-mon[29647]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:40:36.078 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: 1.0 scrub starts 2024-09-11T08:40:36.078 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: 1.0 scrub ok 2024-09-11T08:40:36.078 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: 4.c scrub starts 2024-09-11T08:40:36.078 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: 4.c scrub ok 2024-09-11T08:40:36.078 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:36.078 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:36.078 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:40:36.078 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:40:36.078 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:40:36.078 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi092.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:40:36.078 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:36.078 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:40:36.078 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:40:36.079 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi145.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:40:36.079 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi145.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:40:36.079 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:36.079 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:40:36.079 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:40:36.079 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi145.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:40:36.079 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi145.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:40:36.079 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:36.079 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:40:36.079 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:35 smithi145 ceph-mon[33384]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:40:36.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:40:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:40:36.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:36 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:36] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:40:36.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:36 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:36] ENGINE Bus STOPPED 2024-09-11T08:40:36.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:36 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:36] ENGINE Bus STARTING 2024-09-11T08:40:36.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:36 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:36] ENGINE Serving on http://:::9283 2024-09-11T08:40:36.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:36 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:08:40:36] ENGINE Bus STARTED 2024-09-11T08:40:36.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:40:37.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:36 smithi145 ceph-mon[33384]: 3.1a scrub starts 2024-09-11T08:40:37.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:36 smithi145 ceph-mon[33384]: 3.1a scrub ok 2024-09-11T08:40:37.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:36 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:37.163 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:37.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:36 smithi092 ceph-mon[29647]: 3.1a scrub starts 2024-09-11T08:40:37.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:36 smithi092 ceph-mon[29647]: 3.1a scrub ok 2024-09-11T08:40:37.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:36 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:37.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:36 smithi092 ceph-mon[41114]: 3.1a scrub starts 2024-09-11T08:40:37.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:36 smithi092 ceph-mon[41114]: 3.1a scrub ok 2024-09-11T08:40:37.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:36 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:37.804 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 08:40:37 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[79001]: ts=2024-09-11T08:40:37.733Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002082201s 2024-09-11T08:40:38.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:37 smithi092 ceph-mon[29647]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 948 B/s rd, 0 op/s 2024-09-11T08:40:38.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:37 smithi092 ceph-mon[41114]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 948 B/s rd, 0 op/s 2024-09-11T08:40:38.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:37 smithi145 ceph-mon[33384]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 948 B/s rd, 0 op/s 2024-09-11T08:40:39.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:38 smithi092 ceph-mon[29647]: 6.11 scrub starts 2024-09-11T08:40:39.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:38 smithi092 ceph-mon[29647]: 6.11 scrub ok 2024-09-11T08:40:39.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:38 smithi092 ceph-mon[41114]: 6.11 scrub starts 2024-09-11T08:40:39.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:38 smithi092 ceph-mon[41114]: 6.11 scrub ok 2024-09-11T08:40:39.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:38 smithi145 ceph-mon[33384]: 6.11 scrub starts 2024-09-11T08:40:39.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:38 smithi145 ceph-mon[33384]: 6.11 scrub ok 2024-09-11T08:40:39.709 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:40:39.711 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:40:40.247 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:39 smithi145 ceph-mon[33384]: 6.e deep-scrub starts 2024-09-11T08:40:40.247 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:39 smithi145 ceph-mon[33384]: 6.e deep-scrub ok 2024-09-11T08:40:40.247 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:39 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:40.247 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:39 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:40.247 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:39 smithi145 ceph-mon[33384]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:40.247 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:39 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:40.248 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:39 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:40.248 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:39 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:40:40.248 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:39 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:40:40.248 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:39 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:40.248 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:39 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:40:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[29647]: 6.e deep-scrub starts 2024-09-11T08:40:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[29647]: 6.e deep-scrub ok 2024-09-11T08:40:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:40.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:40.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[29647]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:40.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:40.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:40.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:40:40.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:40:40.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:40.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:40:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[41114]: 6.e deep-scrub starts 2024-09-11T08:40:40.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[41114]: 6.e deep-scrub ok 2024-09-11T08:40:40.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:40.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:40.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[41114]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:40.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:40.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:40.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:40:40.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:40:40.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:40.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:39 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:40:40.426 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-09-11T08:40:39.018658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":963,"num_read_kb":820,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":580,"ondisk_log_size":580,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2386172,"kb_used_data":7348,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575044,"statfs":{"total":767960285184,"available":765516845056,"internally_reserved":0,"allocated":7524352,"data_stored":4635348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002685"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418157+0000","last_change":"2024-09-11T08:39:47.339366+0000","last_active":"2024-09-11T08:40:25.418157+0000","last_peered":"2024-09-11T08:40:25.418157+0000","last_clean":"2024-09-11T08:40:25.418157+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:40:25.418157+0000","last_undegraded":"2024-09-11T08:40:25.418157+0000","last_fullsized":"2024-09-11T08:40:25.418157+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.339312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.339312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.154202+0000","scrub_duration":0.001323095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418551+0000","last_change":"2024-09-11T08:39:56.666764+0000","last_active":"2024-09-11T08:40:25.418551+0000","last_peered":"2024-09-11T08:40:25.418551+0000","last_clean":"2024-09-11T08:40:25.418551+0000","last_became_active":"2024-09-11T08:38:40.973398+0000","last_became_peered":"2024-09-11T08:38:40.973398+0000","last_unstale":"2024-09-11T08:40:25.418551+0000","last_undegraded":"2024-09-11T08:40:25.418551+0000","last_fullsized":"2024-09-11T08:40:25.418551+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.666703+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.544222+0000","scrub_duration":0.0041022000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222903+0000","last_change":"2024-09-11T08:40:12.325099+0000","last_active":"2024-09-11T08:40:25.222903+0000","last_peered":"2024-09-11T08:40:25.222903+0000","last_clean":"2024-09-11T08:40:25.222903+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:40:25.222903+0000","last_undegraded":"2024-09-11T08:40:25.222903+0000","last_fullsized":"2024-09-11T08:40:25.222903+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:12.325037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:12.325037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:13.497524+0000","scrub_duration":0.001599736,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418919+0000","last_change":"2024-09-11T08:38:44.992111+0000","last_active":"2024-09-11T08:40:25.418919+0000","last_peered":"2024-09-11T08:40:25.418919+0000","last_clean":"2024-09-11T08:40:25.418919+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:40:25.418919+0000","last_undegraded":"2024-09-11T08:40:25.418919+0000","last_fullsized":"2024-09-11T08:40:25.418919+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223066+0000","last_change":"2024-09-11T08:40:06.173950+0000","last_active":"2024-09-11T08:40:25.223066+0000","last_peered":"2024-09-11T08:40:25.223066+0000","last_clean":"2024-09-11T08:40:25.223066+0000","last_became_active":"2024-09-11T08:38:45.184215+0000","last_became_peered":"2024-09-11T08:38:45.184215+0000","last_unstale":"2024-09-11T08:40:25.223066+0000","last_undegraded":"2024-09-11T08:40:25.223066+0000","last_fullsized":"2024-09-11T08:40:25.223066+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:06.173882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:06.173882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:17.036842+0000","scrub_duration":0.001730875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223651+0000","last_change":"2024-09-11T08:39:58.161055+0000","last_active":"2024-09-11T08:40:25.223651+0000","last_peered":"2024-09-11T08:40:25.223651+0000","last_clean":"2024-09-11T08:40:25.223651+0000","last_became_active":"2024-09-11T08:38:38.949136+0000","last_became_peered":"2024-09-11T08:38:38.949136+0000","last_unstale":"2024-09-11T08:40:25.223651+0000","last_undegraded":"2024-09-11T08:40:25.223651+0000","last_fullsized":"2024-09-11T08:40:25.223651+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:58.160984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.160984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:00.523696+0000","scrub_duration":0.0015109679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":75,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222725+0000","last_change":"2024-09-11T08:39:45.997873+0000","last_active":"2024-09-11T08:40:25.222725+0000","last_peered":"2024-09-11T08:40:25.222725+0000","last_clean":"2024-09-11T08:40:25.222725+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:40:25.222725+0000","last_undegraded":"2024-09-11T08:40:25.222725+0000","last_fullsized":"2024-09-11T08:40:25.222725+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.997803+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.534414+0000","scrub_duration":0.0046870369999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418457+0000","last_change":"2024-09-11T08:38:42.983653+0000","last_active":"2024-09-11T08:40:25.418457+0000","last_peered":"2024-09-11T08:40:25.418457+0000","last_clean":"2024-09-11T08:40:25.418457+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:40:25.418457+0000","last_undegraded":"2024-09-11T08:40:25.418457+0000","last_fullsized":"2024-09-11T08:40:25.418457+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223363+0000","last_change":"2024-09-11T08:40:08.416577+0000","last_active":"2024-09-11T08:40:25.223363+0000","last_peered":"2024-09-11T08:40:25.223363+0000","last_clean":"2024-09-11T08:40:25.223363+0000","last_became_active":"2024-09-11T08:38:38.948930+0000","last_became_peered":"2024-09-11T08:38:38.948930+0000","last_unstale":"2024-09-11T08:40:25.223363+0000","last_undegraded":"2024-09-11T08:40:25.223363+0000","last_fullsized":"2024-09-11T08:40:25.223363+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:08.416522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.416522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:27.445091+0000","scrub_duration":0.001330005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420008+0000","last_change":"2024-09-11T08:39:46.620700+0000","last_active":"2024-09-11T08:40:25.420008+0000","last_peered":"2024-09-11T08:40:25.420008+0000","last_clean":"2024-09-11T08:40:25.420008+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:40:25.420008+0000","last_undegraded":"2024-09-11T08:40:25.420008+0000","last_fullsized":"2024-09-11T08:40:25.420008+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:39:46.620624+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:10.984564+0000","scrub_duration":0.0044132319999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222859+0000","last_change":"2024-09-11T08:40:04.450336+0000","last_active":"2024-09-11T08:40:25.222859+0000","last_peered":"2024-09-11T08:40:25.222859+0000","last_clean":"2024-09-11T08:40:25.222859+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:40:25.222859+0000","last_undegraded":"2024-09-11T08:40:25.222859+0000","last_fullsized":"2024-09-11T08:40:25.222859+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:04.450282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:04.450282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.499239+0000","scrub_duration":0.001689433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417575+0000","last_change":"2024-09-11T08:39:51.280365+0000","last_active":"2024-09-11T08:40:25.417575+0000","last_peered":"2024-09-11T08:40:25.417575+0000","last_clean":"2024-09-11T08:40:25.417575+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:40:25.417575+0000","last_undegraded":"2024-09-11T08:40:25.417575+0000","last_fullsized":"2024-09-11T08:40:25.417575+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.280310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.280310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:06.712880+0000","scrub_duration":0.001634777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223124+0000","last_change":"2024-09-11T08:40:12.372128+0000","last_active":"2024-09-11T08:40:25.223124+0000","last_peered":"2024-09-11T08:40:25.223124+0000","last_clean":"2024-09-11T08:40:25.223124+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:40:25.223124+0000","last_undegraded":"2024-09-11T08:40:25.223124+0000","last_fullsized":"2024-09-11T08:40:25.223124+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:12.372043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:12.372043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.497179+0000","scrub_duration":0.001440623,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419933+0000","last_change":"2024-09-11T08:38:40.974291+0000","last_active":"2024-09-11T08:40:25.419933+0000","last_peered":"2024-09-11T08:40:25.419933+0000","last_clean":"2024-09-11T08:40:25.419933+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:40:25.419933+0000","last_undegraded":"2024-09-11T08:40:25.419933+0000","last_fullsized":"2024-09-11T08:40:25.419933+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418977+0000","last_change":"2024-09-11T08:39:51.166114+0000","last_active":"2024-09-11T08:40:25.418977+0000","last_peered":"2024-09-11T08:40:25.418977+0000","last_clean":"2024-09-11T08:40:25.418977+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:40:25.418977+0000","last_undegraded":"2024-09-11T08:40:25.418977+0000","last_fullsized":"2024-09-11T08:40:25.418977+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.166051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.570790+0000","scrub_duration":0.0016440420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222737+0000","last_change":"2024-09-11T08:38:44.994259+0000","last_active":"2024-09-11T08:40:25.222737+0000","last_peered":"2024-09-11T08:40:25.222737+0000","last_clean":"2024-09-11T08:40:25.222737+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:40:25.222737+0000","last_undegraded":"2024-09-11T08:40:25.222737+0000","last_fullsized":"2024-09-11T08:40:25.222737+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420256+0000","last_change":"2024-09-11T08:39:59.606521+0000","last_active":"2024-09-11T08:40:25.420256+0000","last_peered":"2024-09-11T08:40:25.420256+0000","last_clean":"2024-09-11T08:40:25.420256+0000","last_became_active":"2024-09-11T08:38:45.183364+0000","last_became_peered":"2024-09-11T08:38:45.183364+0000","last_unstale":"2024-09-11T08:40:25.420256+0000","last_undegraded":"2024-09-11T08:40:25.420256+0000","last_fullsized":"2024-09-11T08:40:25.420256+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:59.606458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:59.606458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.547769+0000","scrub_duration":0.0012065960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:30.632514+0000","last_change":"2024-09-11T08:40:30.632483+0000","last_active":"2024-09-11T08:40:30.632514+0000","last_peered":"2024-09-11T08:40:30.632514+0000","last_clean":"2024-09-11T08:40:30.632514+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:40:30.632514+0000","last_undegraded":"2024-09-11T08:40:30.632514+0000","last_fullsized":"2024-09-11T08:40:30.632514+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:30.632430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:30.632430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:45.188817+0000","scrub_duration":0.001480947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.414027+0000","last_change":"2024-09-11T08:39:50.472419+0000","last_active":"2024-09-11T08:40:25.414027+0000","last_peered":"2024-09-11T08:40:25.414027+0000","last_clean":"2024-09-11T08:40:25.414027+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:40:25.414027+0000","last_undegraded":"2024-09-11T08:40:25.414027+0000","last_fullsized":"2024-09-11T08:40:25.414027+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.472368+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:10.784548+0000","scrub_duration":0.0049809520000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223102+0000","last_change":"2024-09-11T08:39:44.632369+0000","last_active":"2024-09-11T08:40:25.223102+0000","last_peered":"2024-09-11T08:40:25.223102+0000","last_clean":"2024-09-11T08:40:25.223102+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:40:25.223102+0000","last_undegraded":"2024-09-11T08:40:25.223102+0000","last_fullsized":"2024-09-11T08:40:25.223102+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:44.632308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:44.632308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:56.408810+0000","scrub_duration":0.001420377,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223786+0000","last_change":"2024-09-11T08:38:45.183646+0000","last_active":"2024-09-11T08:40:25.223786+0000","last_peered":"2024-09-11T08:40:25.223786+0000","last_clean":"2024-09-11T08:40:25.223786+0000","last_became_active":"2024-09-11T08:38:45.182763+0000","last_became_peered":"2024-09-11T08:38:45.182763+0000","last_unstale":"2024-09-11T08:40:25.223786+0000","last_undegraded":"2024-09-11T08:40:25.223786+0000","last_fullsized":"2024-09-11T08:40:25.223786+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:35.714476+0000","last_change":"2024-09-11T08:40:35.714447+0000","last_active":"2024-09-11T08:40:35.714476+0000","last_peered":"2024-09-11T08:40:35.714476+0000","last_clean":"2024-09-11T08:40:35.714476+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:40:35.714476+0000","last_undegraded":"2024-09-11T08:40:35.714476+0000","last_fullsized":"2024-09-11T08:40:35.714476+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:35.714393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:35.714393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:40.832830+0000","scrub_duration":0.0013591110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223370+0000","last_change":"2024-09-11T08:39:55.125449+0000","last_active":"2024-09-11T08:40:25.223370+0000","last_peered":"2024-09-11T08:40:25.223370+0000","last_clean":"2024-09-11T08:40:25.223370+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:40:25.223370+0000","last_undegraded":"2024-09-11T08:40:25.223370+0000","last_fullsized":"2024-09-11T08:40:25.223370+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:55.125394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.125394+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.857707+0000","scrub_duration":0.0016833810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418855+0000","last_change":"2024-09-11T08:40:01.585472+0000","last_active":"2024-09-11T08:40:25.418855+0000","last_peered":"2024-09-11T08:40:25.418855+0000","last_clean":"2024-09-11T08:40:25.418855+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:40:25.418855+0000","last_undegraded":"2024-09-11T08:40:25.418855+0000","last_fullsized":"2024-09-11T08:40:25.418855+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:01.585411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:01.585411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.937039+0000","scrub_duration":0.0021220179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419857+0000","last_change":"2024-09-11T08:40:10.113729+0000","last_active":"2024-09-11T08:40:25.419857+0000","last_peered":"2024-09-11T08:40:25.419857+0000","last_clean":"2024-09-11T08:40:25.419857+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:40:25.419857+0000","last_undegraded":"2024-09-11T08:40:25.419857+0000","last_fullsized":"2024-09-11T08:40:25.419857+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:40:10.113671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:10.113671+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:33.267918+0000","scrub_duration":0.001621302,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222971+0000","last_change":"2024-09-11T08:40:02.458618+0000","last_active":"2024-09-11T08:40:25.222971+0000","last_peered":"2024-09-11T08:40:25.222971+0000","last_clean":"2024-09-11T08:40:25.222971+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:40:25.222971+0000","last_undegraded":"2024-09-11T08:40:25.222971+0000","last_fullsized":"2024-09-11T08:40:25.222971+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:02.458565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.458565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.247079+0000","scrub_duration":0.001315568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418113+0000","last_change":"2024-09-11T08:40:18.467638+0000","last_active":"2024-09-11T08:40:25.418113+0000","last_peered":"2024-09-11T08:40:25.418113+0000","last_clean":"2024-09-11T08:40:25.418113+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:40:25.418113+0000","last_undegraded":"2024-09-11T08:40:25.418113+0000","last_fullsized":"2024-09-11T08:40:25.418113+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:40:18.467574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:18.467574+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:37.429174+0000","scrub_duration":0.0040976820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":75,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:27.694081+0000","last_change":"2024-09-11T08:40:12.584703+0000","last_active":"2024-09-11T08:40:27.694081+0000","last_peered":"2024-09-11T08:40:27.694081+0000","last_clean":"2024-09-11T08:40:27.694081+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:40:27.694081+0000","last_undegraded":"2024-09-11T08:40:27.694081+0000","last_fullsized":"2024-09-11T08:40:27.694081+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:12.584631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:12.584631+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:27.946233+0000","scrub_duration":0.0015754790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223715+0000","last_change":"2024-09-11T08:40:16.290927+0000","last_active":"2024-09-11T08:40:25.223715+0000","last_peered":"2024-09-11T08:40:25.223715+0000","last_clean":"2024-09-11T08:40:25.223715+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:40:25.223715+0000","last_undegraded":"2024-09-11T08:40:25.223715+0000","last_fullsized":"2024-09-11T08:40:25.223715+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-11T08:40:16.290866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:16.290866+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:46.083697+0000","scrub_duration":0.0021215069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222683+0000","last_change":"2024-09-11T08:40:23.048021+0000","last_active":"2024-09-11T08:40:25.222683+0000","last_peered":"2024-09-11T08:40:25.222683+0000","last_clean":"2024-09-11T08:40:25.222683+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:40:25.222683+0000","last_undegraded":"2024-09-11T08:40:25.222683+0000","last_fullsized":"2024-09-11T08:40:25.222683+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:23.047963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:23.047963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.577363+0000","scrub_duration":0.00148653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:27.694068+0000","last_change":"2024-09-11T08:40:11.625218+0000","last_active":"2024-09-11T08:40:27.694068+0000","last_peered":"2024-09-11T08:40:27.694068+0000","last_clean":"2024-09-11T08:40:27.694068+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:40:27.694068+0000","last_undegraded":"2024-09-11T08:40:27.694068+0000","last_fullsized":"2024-09-11T08:40:27.694068+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:11.625164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:11.625164+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.094157+0000","scrub_duration":0.0014996320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223059+0000","last_change":"2024-09-11T08:38:44.991592+0000","last_active":"2024-09-11T08:40:25.223059+0000","last_peered":"2024-09-11T08:40:25.223059+0000","last_clean":"2024-09-11T08:40:25.223059+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:40:25.223059+0000","last_undegraded":"2024-09-11T08:40:25.223059+0000","last_fullsized":"2024-09-11T08:40:25.223059+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":75,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222500+0000","last_change":"2024-09-11T08:40:05.197208+0000","last_active":"2024-09-11T08:40:25.222500+0000","last_peered":"2024-09-11T08:40:25.222500+0000","last_clean":"2024-09-11T08:40:25.222500+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:40:25.222500+0000","last_undegraded":"2024-09-11T08:40:25.222500+0000","last_fullsized":"2024-09-11T08:40:25.222500+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-11T08:40:05.197152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:05.197152+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:06.174279+0000","scrub_duration":0.0016976000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223932+0000","last_change":"2024-09-11T08:39:43.037605+0000","last_active":"2024-09-11T08:40:25.223932+0000","last_peered":"2024-09-11T08:40:25.223932+0000","last_clean":"2024-09-11T08:40:25.223932+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:40:25.223932+0000","last_undegraded":"2024-09-11T08:40:25.223932+0000","last_fullsized":"2024-09-11T08:40:25.223932+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.037551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.037551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.071661+0000","scrub_duration":0.001523976,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418369+0000","last_change":"2024-09-11T08:40:19.688943+0000","last_active":"2024-09-11T08:40:25.418369+0000","last_peered":"2024-09-11T08:40:25.418369+0000","last_clean":"2024-09-11T08:40:25.418369+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:40:25.418369+0000","last_undegraded":"2024-09-11T08:40:25.418369+0000","last_fullsized":"2024-09-11T08:40:25.418369+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:19.688876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:19.688876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.993714+0000","scrub_duration":0.0014001599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419130+0000","last_change":"2024-09-11T08:38:44.995555+0000","last_active":"2024-09-11T08:40:25.419130+0000","last_peered":"2024-09-11T08:40:25.419130+0000","last_clean":"2024-09-11T08:40:25.419130+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:40:25.419130+0000","last_undegraded":"2024-09-11T08:40:25.419130+0000","last_fullsized":"2024-09-11T08:40:25.419130+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":55,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418981+0000","last_change":"2024-09-11T08:38:40.968377+0000","last_active":"2024-09-11T08:40:25.418981+0000","last_peered":"2024-09-11T08:40:25.418981+0000","last_clean":"2024-09-11T08:40:25.418981+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:40:25.418981+0000","last_undegraded":"2024-09-11T08:40:25.418981+0000","last_fullsized":"2024-09-11T08:40:25.418981+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418178+0000","last_change":"2024-09-11T08:39:41.267320+0000","last_active":"2024-09-11T08:40:25.418178+0000","last_peered":"2024-09-11T08:40:25.418178+0000","last_clean":"2024-09-11T08:40:25.418178+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:40:25.418178+0000","last_undegraded":"2024-09-11T08:40:25.418178+0000","last_fullsized":"2024-09-11T08:40:25.418178+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:41.267267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.267267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:01.547087+0000","scrub_duration":0.0013348209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":65,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:27.693661+0000","last_change":"2024-09-11T08:38:42.982794+0000","last_active":"2024-09-11T08:40:27.693661+0000","last_peered":"2024-09-11T08:40:27.693661+0000","last_clean":"2024-09-11T08:40:27.693661+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:40:27.693661+0000","last_undegraded":"2024-09-11T08:40:27.693661+0000","last_fullsized":"2024-09-11T08:40:27.693661+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418431+0000","last_change":"2024-09-11T08:40:04.174461+0000","last_active":"2024-09-11T08:40:25.418431+0000","last_peered":"2024-09-11T08:40:25.418431+0000","last_clean":"2024-09-11T08:40:25.418431+0000","last_became_active":"2024-09-11T08:38:45.182724+0000","last_became_peered":"2024-09-11T08:38:45.182724+0000","last_unstale":"2024-09-11T08:40:25.418431+0000","last_undegraded":"2024-09-11T08:40:25.418431+0000","last_fullsized":"2024-09-11T08:40:25.418431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:04.174409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:04.174409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.916947+0000","scrub_duration":0.0012996889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"63'10","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223578+0000","last_change":"2024-09-11T08:40:13.327302+0000","last_active":"2024-09-11T08:40:25.223578+0000","last_peered":"2024-09-11T08:40:25.223578+0000","last_clean":"2024-09-11T08:40:25.223578+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:40:25.223578+0000","last_undegraded":"2024-09-11T08:40:25.223578+0000","last_fullsized":"2024-09-11T08:40:25.223578+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:40:13.327245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:13.327245+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.332692+0000","scrub_duration":0.0017714300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:28.351843+0000","last_change":"2024-09-11T08:40:28.351815+0000","last_active":"2024-09-11T08:40:28.351843+0000","last_peered":"2024-09-11T08:40:28.351843+0000","last_clean":"2024-09-11T08:40:28.351843+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:40:28.351843+0000","last_undegraded":"2024-09-11T08:40:28.351843+0000","last_fullsized":"2024-09-11T08:40:28.351843+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:28.351762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:28.351762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:39.082689+0000","scrub_duration":0.001710488,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":63,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:27.693840+0000","last_change":"2024-09-11T08:38:42.981732+0000","last_active":"2024-09-11T08:40:27.693840+0000","last_peered":"2024-09-11T08:40:27.693840+0000","last_clean":"2024-09-11T08:40:27.693840+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:40:27.693840+0000","last_undegraded":"2024-09-11T08:40:27.693840+0000","last_fullsized":"2024-09-11T08:40:27.693840+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417732+0000","last_change":"2024-09-11T08:38:44.998257+0000","last_active":"2024-09-11T08:40:25.417732+0000","last_peered":"2024-09-11T08:40:25.417732+0000","last_clean":"2024-09-11T08:40:25.417732+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:40:25.417732+0000","last_undegraded":"2024-09-11T08:40:25.417732+0000","last_fullsized":"2024-09-11T08:40:25.417732+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417915+0000","last_change":"2024-09-11T08:38:41.040034+0000","last_active":"2024-09-11T08:40:25.417915+0000","last_peered":"2024-09-11T08:40:25.417915+0000","last_clean":"2024-09-11T08:40:25.417915+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:40:25.417915+0000","last_undegraded":"2024-09-11T08:40:25.417915+0000","last_fullsized":"2024-09-11T08:40:25.417915+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223367+0000","last_change":"2024-09-11T08:40:03.496005+0000","last_active":"2024-09-11T08:40:25.223367+0000","last_peered":"2024-09-11T08:40:25.223367+0000","last_clean":"2024-09-11T08:40:25.223367+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:40:25.223367+0000","last_undegraded":"2024-09-11T08:40:25.223367+0000","last_fullsized":"2024-09-11T08:40:25.223367+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:03.495950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.495950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.342754+0000","scrub_duration":0.001379314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418347+0000","last_change":"2024-09-11T08:39:57.596980+0000","last_active":"2024-09-11T08:40:25.418347+0000","last_peered":"2024-09-11T08:40:25.418347+0000","last_clean":"2024-09-11T08:40:25.418347+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:40:25.418347+0000","last_undegraded":"2024-09-11T08:40:25.418347+0000","last_fullsized":"2024-09-11T08:40:25.418347+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.596889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.648355+0000","scrub_duration":0.001574424,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222875+0000","last_change":"2024-09-11T08:40:08.387078+0000","last_active":"2024-09-11T08:40:25.222875+0000","last_peered":"2024-09-11T08:40:25.222875+0000","last_clean":"2024-09-11T08:40:25.222875+0000","last_became_active":"2024-09-11T08:38:45.182782+0000","last_became_peered":"2024-09-11T08:38:45.182782+0000","last_unstale":"2024-09-11T08:40:25.222875+0000","last_undegraded":"2024-09-11T08:40:25.222875+0000","last_fullsized":"2024-09-11T08:40:25.222875+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:08.387027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.387027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.887729+0000","scrub_duration":0.001411336,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":125,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:27.693600+0000","last_change":"2024-09-11T08:39:53.493004+0000","last_active":"2024-09-11T08:40:27.693600+0000","last_peered":"2024-09-11T08:40:27.693600+0000","last_clean":"2024-09-11T08:40:27.693600+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:40:27.693600+0000","last_undegraded":"2024-09-11T08:40:27.693600+0000","last_fullsized":"2024-09-11T08:40:27.693600+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.492945+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.774847+0000","scrub_duration":0.005988045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420227+0000","last_change":"2024-09-11T08:40:22.750815+0000","last_active":"2024-09-11T08:40:25.420227+0000","last_peered":"2024-09-11T08:40:25.420227+0000","last_clean":"2024-09-11T08:40:25.420227+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:40:25.420227+0000","last_undegraded":"2024-09-11T08:40:25.420227+0000","last_fullsized":"2024-09-11T08:40:25.420227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:22.750744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.750744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.795298+0000","scrub_duration":0.001679119,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223085+0000","last_change":"2024-09-11T08:39:58.472851+0000","last_active":"2024-09-11T08:40:25.223085+0000","last_peered":"2024-09-11T08:40:25.223085+0000","last_clean":"2024-09-11T08:40:25.223085+0000","last_became_active":"2024-09-11T08:38:38.947270+0000","last_became_peered":"2024-09-11T08:38:38.947270+0000","last_unstale":"2024-09-11T08:40:25.223085+0000","last_undegraded":"2024-09-11T08:40:25.223085+0000","last_fullsized":"2024-09-11T08:40:25.223085+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:39:58.472785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.472785+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.725781+0000","scrub_duration":0.0020806930000000002,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418012+0000","last_change":"2024-09-11T08:40:17.435980+0000","last_active":"2024-09-11T08:40:25.418012+0000","last_peered":"2024-09-11T08:40:25.418012+0000","last_clean":"2024-09-11T08:40:25.418012+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:40:25.418012+0000","last_undegraded":"2024-09-11T08:40:25.418012+0000","last_fullsized":"2024-09-11T08:40:25.418012+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:17.435900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:17.435900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.732279+0000","scrub_duration":0.0018276379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:31.206613+0000","last_change":"2024-09-11T08:40:31.206582+0000","last_active":"2024-09-11T08:40:31.206613+0000","last_peered":"2024-09-11T08:40:31.206613+0000","last_clean":"2024-09-11T08:40:31.206613+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:40:31.206613+0000","last_undegraded":"2024-09-11T08:40:31.206613+0000","last_fullsized":"2024-09-11T08:40:31.206613+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:31.206531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:31.206531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:49.078201+0000","scrub_duration":0.001945928,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":71,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.224030+0000","last_change":"2024-09-11T08:39:43.585175+0000","last_active":"2024-09-11T08:40:25.224030+0000","last_peered":"2024-09-11T08:40:25.224030+0000","last_clean":"2024-09-11T08:40:25.224030+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:40:25.224030+0000","last_undegraded":"2024-09-11T08:40:25.224030+0000","last_fullsized":"2024-09-11T08:40:25.224030+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:39:43.585116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.585116+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.724395+0000","scrub_duration":0.0021210309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222736+0000","last_change":"2024-09-11T08:40:00.500444+0000","last_active":"2024-09-11T08:40:25.222736+0000","last_peered":"2024-09-11T08:40:25.222736+0000","last_clean":"2024-09-11T08:40:25.222736+0000","last_became_active":"2024-09-11T08:38:38.949019+0000","last_became_peered":"2024-09-11T08:38:38.949019+0000","last_unstale":"2024-09-11T08:40:25.222736+0000","last_undegraded":"2024-09-11T08:40:25.222736+0000","last_fullsized":"2024-09-11T08:40:25.222736+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.500392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.500392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.354187+0000","scrub_duration":0.001294566,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":52,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.413758+0000","last_change":"2024-09-11T08:40:01.488539+0000","last_active":"2024-09-11T08:40:25.413758+0000","last_peered":"2024-09-11T08:40:25.413758+0000","last_clean":"2024-09-11T08:40:25.413758+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:40:25.413758+0000","last_undegraded":"2024-09-11T08:40:25.413758+0000","last_fullsized":"2024-09-11T08:40:25.413758+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:40:01.488482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:01.488482+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.917024+0000","scrub_duration":0.001463422,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.414198+0000","last_change":"2024-09-11T08:40:02.444341+0000","last_active":"2024-09-11T08:40:25.414198+0000","last_peered":"2024-09-11T08:40:25.414198+0000","last_clean":"2024-09-11T08:40:25.414198+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:40:25.414198+0000","last_undegraded":"2024-09-11T08:40:25.414198+0000","last_fullsized":"2024-09-11T08:40:25.414198+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.444279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.886414+0000","scrub_duration":0.001414329,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419559+0000","last_change":"2024-09-11T08:40:15.184420+0000","last_active":"2024-09-11T08:40:25.419559+0000","last_peered":"2024-09-11T08:40:25.419559+0000","last_clean":"2024-09-11T08:40:25.419559+0000","last_became_active":"2024-09-11T08:38:45.183582+0000","last_became_peered":"2024-09-11T08:38:45.183582+0000","last_unstale":"2024-09-11T08:40:25.419559+0000","last_undegraded":"2024-09-11T08:40:25.419559+0000","last_fullsized":"2024-09-11T08:40:25.419559+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:15.184337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:15.184337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.002701+0000","scrub_duration":0.0039899209999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418097+0000","last_change":"2024-09-11T08:40:21.450026+0000","last_active":"2024-09-11T08:40:25.418097+0000","last_peered":"2024-09-11T08:40:25.418097+0000","last_clean":"2024-09-11T08:40:25.418097+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:40:25.418097+0000","last_undegraded":"2024-09-11T08:40:25.418097+0000","last_fullsized":"2024-09-11T08:40:25.418097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:40:21.449958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:21.449958+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:46.796354+0000","scrub_duration":0.0017585280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417684+0000","last_change":"2024-09-11T08:39:43.356751+0000","last_active":"2024-09-11T08:40:25.417684+0000","last_peered":"2024-09-11T08:40:25.417684+0000","last_clean":"2024-09-11T08:40:25.417684+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:40:25.417684+0000","last_undegraded":"2024-09-11T08:40:25.417684+0000","last_fullsized":"2024-09-11T08:40:25.417684+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.356699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.356699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.569637+0000","scrub_duration":0.001649455,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":163,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:34.285291+0000","last_change":"2024-09-11T08:39:55.153362+0000","last_active":"2024-09-11T08:40:34.285291+0000","last_peered":"2024-09-11T08:40:34.285291+0000","last_clean":"2024-09-11T08:40:34.285291+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:40:34.285291+0000","last_undegraded":"2024-09-11T08:40:34.285291+0000","last_fullsized":"2024-09-11T08:40:34.285291+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:39:55.153290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.153290+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.446909+0000","scrub_duration":0.0017161990000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":110,"num_read_kb":105,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223583+0000","last_change":"2024-09-11T08:38:42.976525+0000","last_active":"2024-09-11T08:40:25.223583+0000","last_peered":"2024-09-11T08:40:25.223583+0000","last_clean":"2024-09-11T08:40:25.223583+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:40:25.223583+0000","last_undegraded":"2024-09-11T08:40:25.223583+0000","last_fullsized":"2024-09-11T08:40:25.223583+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:38:41.959472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223080+0000","last_change":"2024-09-11T08:40:15.280809+0000","last_active":"2024-09-11T08:40:25.223080+0000","last_peered":"2024-09-11T08:40:25.223080+0000","last_clean":"2024-09-11T08:40:25.223080+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:40:25.223080+0000","last_undegraded":"2024-09-11T08:40:25.223080+0000","last_fullsized":"2024-09-11T08:40:25.223080+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:15.280760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:15.280760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:36.262012+0000","scrub_duration":0.0012405719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419421+0000","last_change":"2024-09-11T08:40:00.540192+0000","last_active":"2024-09-11T08:40:25.419421+0000","last_peered":"2024-09-11T08:40:25.419421+0000","last_clean":"2024-09-11T08:40:25.419421+0000","last_became_active":"2024-09-11T08:38:41.040948+0000","last_became_peered":"2024-09-11T08:38:41.040948+0000","last_unstale":"2024-09-11T08:40:25.419421+0000","last_undegraded":"2024-09-11T08:40:25.419421+0000","last_fullsized":"2024-09-11T08:40:25.419421+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:40:00.540136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.540136+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:07.651325+0000","scrub_duration":0.001785095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223089+0000","last_change":"2024-09-11T08:40:21.086239+0000","last_active":"2024-09-11T08:40:25.223089+0000","last_peered":"2024-09-11T08:40:25.223089+0000","last_clean":"2024-09-11T08:40:25.223089+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:40:25.223089+0000","last_undegraded":"2024-09-11T08:40:25.223089+0000","last_fullsized":"2024-09-11T08:40:25.223089+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:40:21.086183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:21.086183+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:25.836107+0000","scrub_duration":0.001907838,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":89,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:34.271877+0000","last_change":"2024-09-11T08:40:34.271853+0000","last_active":"2024-09-11T08:40:34.271877+0000","last_peered":"2024-09-11T08:40:34.271877+0000","last_clean":"2024-09-11T08:40:34.271877+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:40:34.271877+0000","last_undegraded":"2024-09-11T08:40:34.271877+0000","last_fullsized":"2024-09-11T08:40:34.271877+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'79","last_scrub_stamp":"2024-09-11T08:40:34.271800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:40:34.271800+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:35.589139+0000","scrub_duration":0.001626796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419459+0000","last_change":"2024-09-11T08:39:49.116184+0000","last_active":"2024-09-11T08:40:25.419459+0000","last_peered":"2024-09-11T08:40:25.419459+0000","last_clean":"2024-09-11T08:40:25.419459+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:40:25.419459+0000","last_undegraded":"2024-09-11T08:40:25.419459+0000","last_fullsized":"2024-09-11T08:40:25.419459+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.116113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.116113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:04.209747+0000","scrub_duration":0.00167223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223528+0000","last_change":"2024-09-11T08:39:47.423745+0000","last_active":"2024-09-11T08:40:25.223528+0000","last_peered":"2024-09-11T08:40:25.223528+0000","last_clean":"2024-09-11T08:40:25.223528+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:40:25.223528+0000","last_undegraded":"2024-09-11T08:40:25.223528+0000","last_fullsized":"2024-09-11T08:40:25.223528+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.423684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.423684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:57.004042+0000","scrub_duration":0.001458972,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":75,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420377+0000","last_change":"2024-09-11T08:40:16.190092+0000","last_active":"2024-09-11T08:40:25.420377+0000","last_peered":"2024-09-11T08:40:25.420377+0000","last_clean":"2024-09-11T08:40:25.420377+0000","last_became_active":"2024-09-11T08:40:16.183867+0000","last_became_peered":"2024-09-11T08:40:16.183867+0000","last_unstale":"2024-09-11T08:40:25.420377+0000","last_undegraded":"2024-09-11T08:40:25.420377+0000","last_fullsized":"2024-09-11T08:40:25.420377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419129+0000","last_change":"2024-09-11T08:38:38.948244+0000","last_active":"2024-09-11T08:40:25.419129+0000","last_peered":"2024-09-11T08:40:25.419129+0000","last_clean":"2024-09-11T08:40:25.419129+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:40:25.419129+0000","last_undegraded":"2024-09-11T08:40:25.419129+0000","last_fullsized":"2024-09-11T08:40:25.419129+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223434+0000","last_change":"2024-09-11T08:40:03.430883+0000","last_active":"2024-09-11T08:40:25.223434+0000","last_peered":"2024-09-11T08:40:25.223434+0000","last_clean":"2024-09-11T08:40:25.223434+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:40:25.223434+0000","last_undegraded":"2024-09-11T08:40:25.223434+0000","last_fullsized":"2024-09-11T08:40:25.223434+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:03.430794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.430794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:20.081896+0000","scrub_duration":0.0014294430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:30.215968+0000","last_change":"2024-09-11T08:40:30.215943+0000","last_active":"2024-09-11T08:40:30.215968+0000","last_peered":"2024-09-11T08:40:30.215968+0000","last_clean":"2024-09-11T08:40:30.215968+0000","last_became_active":"2024-09-11T08:38:45.184410+0000","last_became_peered":"2024-09-11T08:38:45.184410+0000","last_unstale":"2024-09-11T08:40:30.215968+0000","last_undegraded":"2024-09-11T08:40:30.215968+0000","last_fullsized":"2024-09-11T08:40:30.215968+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:30.215890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:30.215890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:45.729251+0000","scrub_duration":0.0012073710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":72,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420528+0000","last_change":"2024-09-11T08:39:43.526845+0000","last_active":"2024-09-11T08:40:25.420528+0000","last_peered":"2024-09-11T08:40:25.420528+0000","last_clean":"2024-09-11T08:40:25.420528+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:40:25.420528+0000","last_undegraded":"2024-09-11T08:40:25.420528+0000","last_fullsized":"2024-09-11T08:40:25.420528+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:39:43.526763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.526763+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.869811+0000","scrub_duration":0.0022037329999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418766+0000","last_change":"2024-09-11T08:39:55.599049+0000","last_active":"2024-09-11T08:40:25.418766+0000","last_peered":"2024-09-11T08:40:25.418766+0000","last_clean":"2024-09-11T08:40:25.418766+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:40:25.418766+0000","last_undegraded":"2024-09-11T08:40:25.418766+0000","last_fullsized":"2024-09-11T08:40:25.418766+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.598961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.598961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.562190+0000","scrub_duration":0.0016789820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223145+0000","last_change":"2024-09-11T08:39:46.627077+0000","last_active":"2024-09-11T08:40:25.223145+0000","last_peered":"2024-09-11T08:40:25.223145+0000","last_clean":"2024-09-11T08:40:25.223145+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:40:25.223145+0000","last_undegraded":"2024-09-11T08:40:25.223145+0000","last_fullsized":"2024-09-11T08:40:25.223145+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:46.627028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:46.627028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:56.334041+0000","scrub_duration":0.0013824029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.414066+0000","last_change":"2024-09-11T08:40:13.462183+0000","last_active":"2024-09-11T08:40:25.414066+0000","last_peered":"2024-09-11T08:40:25.414066+0000","last_clean":"2024-09-11T08:40:25.414066+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:40:25.414066+0000","last_undegraded":"2024-09-11T08:40:25.414066+0000","last_fullsized":"2024-09-11T08:40:25.414066+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:13.462132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:13.462132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:33.998327+0000","scrub_duration":0.001365727,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":59,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:34.764069+0000","last_change":"2024-09-11T08:40:34.764019+0000","last_active":"2024-09-11T08:40:34.764069+0000","last_peered":"2024-09-11T08:40:34.764069+0000","last_clean":"2024-09-11T08:40:34.764069+0000","last_became_active":"2024-09-11T08:38:40.969348+0000","last_became_peered":"2024-09-11T08:38:40.969348+0000","last_unstale":"2024-09-11T08:40:34.764069+0000","last_undegraded":"2024-09-11T08:40:34.764069+0000","last_fullsized":"2024-09-11T08:40:34.764069+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:40:34.763952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:34.763952+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:00.532192+0000","scrub_duration":0.002030423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223230+0000","last_change":"2024-09-11T08:38:38.957214+0000","last_active":"2024-09-11T08:40:25.223230+0000","last_peered":"2024-09-11T08:40:25.223230+0000","last_clean":"2024-09-11T08:40:25.223230+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:40:25.223230+0000","last_undegraded":"2024-09-11T08:40:25.223230+0000","last_fullsized":"2024-09-11T08:40:25.223230+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:38:37.932007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:27.693599+0000","last_change":"2024-09-11T08:40:00.473606+0000","last_active":"2024-09-11T08:40:27.693599+0000","last_peered":"2024-09-11T08:40:27.693599+0000","last_clean":"2024-09-11T08:40:27.693599+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:40:27.693599+0000","last_undegraded":"2024-09-11T08:40:27.693599+0000","last_fullsized":"2024-09-11T08:40:27.693599+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:00.473549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.473549+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.300137+0000","scrub_duration":0.0016703060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420281+0000","last_change":"2024-09-11T08:38:44.993736+0000","last_active":"2024-09-11T08:40:25.420281+0000","last_peered":"2024-09-11T08:40:25.420281+0000","last_clean":"2024-09-11T08:40:25.420281+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:40:25.420281+0000","last_undegraded":"2024-09-11T08:40:25.420281+0000","last_fullsized":"2024-09-11T08:40:25.420281+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417888+0000","last_change":"2024-09-11T08:38:40.977113+0000","last_active":"2024-09-11T08:40:25.417888+0000","last_peered":"2024-09-11T08:40:25.417888+0000","last_clean":"2024-09-11T08:40:25.417888+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:40:25.417888+0000","last_undegraded":"2024-09-11T08:40:25.417888+0000","last_fullsized":"2024-09-11T08:40:25.417888+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223196+0000","last_change":"2024-09-11T08:39:50.452568+0000","last_active":"2024-09-11T08:40:25.223196+0000","last_peered":"2024-09-11T08:40:25.223196+0000","last_clean":"2024-09-11T08:40:25.223196+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:40:25.223196+0000","last_undegraded":"2024-09-11T08:40:25.223196+0000","last_fullsized":"2024-09-11T08:40:25.223196+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.452494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.631278+0000","scrub_duration":0.001547086,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.413842+0000","last_change":"2024-09-11T08:40:09.519072+0000","last_active":"2024-09-11T08:40:25.413842+0000","last_peered":"2024-09-11T08:40:25.413842+0000","last_clean":"2024-09-11T08:40:25.413842+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:40:25.413842+0000","last_undegraded":"2024-09-11T08:40:25.413842+0000","last_fullsized":"2024-09-11T08:40:25.413842+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:09.519019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:09.519019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.000837+0000","scrub_duration":0.001232471,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417656+0000","last_change":"2024-09-11T08:38:44.991908+0000","last_active":"2024-09-11T08:40:25.417656+0000","last_peered":"2024-09-11T08:40:25.417656+0000","last_clean":"2024-09-11T08:40:25.417656+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:40:25.417656+0000","last_undegraded":"2024-09-11T08:40:25.417656+0000","last_fullsized":"2024-09-11T08:40:25.417656+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":74,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418424+0000","last_change":"2024-09-11T08:39:45.575209+0000","last_active":"2024-09-11T08:40:25.418424+0000","last_peered":"2024-09-11T08:40:25.418424+0000","last_clean":"2024-09-11T08:40:25.418424+0000","last_became_active":"2024-09-11T08:38:41.040207+0000","last_became_peered":"2024-09-11T08:38:41.040207+0000","last_unstale":"2024-09-11T08:40:25.418424+0000","last_undegraded":"2024-09-11T08:40:25.418424+0000","last_fullsized":"2024-09-11T08:40:25.418424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:39:45.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.575133+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.182090+0000","scrub_duration":0.0022792160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419947+0000","last_change":"2024-09-11T08:39:56.112112+0000","last_active":"2024-09-11T08:40:25.419947+0000","last_peered":"2024-09-11T08:40:25.419947+0000","last_clean":"2024-09-11T08:40:25.419947+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:40:25.419947+0000","last_undegraded":"2024-09-11T08:40:25.419947+0000","last_fullsized":"2024-09-11T08:40:25.419947+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:56.112043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.112043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:25.809354+0000","scrub_duration":0.001517137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.413787+0000","last_change":"2024-09-11T08:40:11.540833+0000","last_active":"2024-09-11T08:40:25.413787+0000","last_peered":"2024-09-11T08:40:25.413787+0000","last_clean":"2024-09-11T08:40:25.413787+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:40:25.413787+0000","last_undegraded":"2024-09-11T08:40:25.413787+0000","last_fullsized":"2024-09-11T08:40:25.413787+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:11.540760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:11.540760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.937538+0000","scrub_duration":0.0014212529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:26.222721+0000","last_change":"2024-09-11T08:40:26.222640+0000","last_active":"2024-09-11T08:40:26.222721+0000","last_peered":"2024-09-11T08:40:26.222721+0000","last_clean":"2024-09-11T08:40:26.222721+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:40:26.222721+0000","last_undegraded":"2024-09-11T08:40:26.222721+0000","last_fullsized":"2024-09-11T08:40:26.222721+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:26.222567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:26.222567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:54.940352+0000","scrub_duration":0.001672593,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418760+0000","last_change":"2024-09-11T08:39:40.524581+0000","last_active":"2024-09-11T08:40:25.418760+0000","last_peered":"2024-09-11T08:40:25.418760+0000","last_clean":"2024-09-11T08:40:25.418760+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:40:25.418760+0000","last_undegraded":"2024-09-11T08:40:25.418760+0000","last_fullsized":"2024-09-11T08:40:25.418760+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:40.524498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:40.524498+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:03.186922+0000","scrub_duration":0.0024035010000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420067+0000","last_change":"2024-09-11T08:40:00.113587+0000","last_active":"2024-09-11T08:40:25.420067+0000","last_peered":"2024-09-11T08:40:25.420067+0000","last_clean":"2024-09-11T08:40:25.420067+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:40:25.420067+0000","last_undegraded":"2024-09-11T08:40:25.420067+0000","last_fullsized":"2024-09-11T08:40:25.420067+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.113534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.113534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.476975+0000","scrub_duration":0.0011790780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.413695+0000","last_change":"2024-09-11T08:39:57.454257+0000","last_active":"2024-09-11T08:40:25.413695+0000","last_peered":"2024-09-11T08:40:25.413695+0000","last_clean":"2024-09-11T08:40:25.413695+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:40:25.413695+0000","last_undegraded":"2024-09-11T08:40:25.413695+0000","last_fullsized":"2024-09-11T08:40:25.413695+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.454198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.454198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.731898+0000","scrub_duration":0.0011666669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223108+0000","last_change":"2024-09-11T08:38:44.998686+0000","last_active":"2024-09-11T08:40:25.223108+0000","last_peered":"2024-09-11T08:40:25.223108+0000","last_clean":"2024-09-11T08:40:25.223108+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:40:25.223108+0000","last_undegraded":"2024-09-11T08:40:25.223108+0000","last_fullsized":"2024-09-11T08:40:25.223108+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222617+0000","last_change":"2024-09-11T08:39:48.462233+0000","last_active":"2024-09-11T08:40:25.222617+0000","last_peered":"2024-09-11T08:40:25.222617+0000","last_clean":"2024-09-11T08:40:25.222617+0000","last_became_active":"2024-09-11T08:38:41.040475+0000","last_became_peered":"2024-09-11T08:38:41.040475+0000","last_unstale":"2024-09-11T08:40:25.222617+0000","last_undegraded":"2024-09-11T08:40:25.222617+0000","last_fullsized":"2024-09-11T08:40:25.222617+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-11T08:39:48.462163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:48.462163+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.032207+0000","scrub_duration":0.002094753,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419804+0000","last_change":"2024-09-11T08:40:03.143806+0000","last_active":"2024-09-11T08:40:25.419804+0000","last_peered":"2024-09-11T08:40:25.419804+0000","last_clean":"2024-09-11T08:40:25.419804+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:40:25.419804+0000","last_undegraded":"2024-09-11T08:40:25.419804+0000","last_fullsized":"2024-09-11T08:40:25.419804+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:40:03.143742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.143742+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.570435+0000","scrub_duration":0.001564396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"63'8","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:27.694063+0000","last_change":"2024-09-11T08:40:13.221708+0000","last_active":"2024-09-11T08:40:27.694063+0000","last_peered":"2024-09-11T08:40:27.694063+0000","last_clean":"2024-09-11T08:40:27.694063+0000","last_became_active":"2024-09-11T08:38:42.986756+0000","last_became_peered":"2024-09-11T08:38:42.986756+0000","last_unstale":"2024-09-11T08:40:27.694063+0000","last_undegraded":"2024-09-11T08:40:27.694063+0000","last_fullsized":"2024-09-11T08:40:27.694063+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:13.221650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:13.221650+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:28.234147+0000","scrub_duration":0.0017036169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222829+0000","last_change":"2024-09-11T08:40:11.370313+0000","last_active":"2024-09-11T08:40:25.222829+0000","last_peered":"2024-09-11T08:40:25.222829+0000","last_clean":"2024-09-11T08:40:25.222829+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:40:25.222829+0000","last_undegraded":"2024-09-11T08:40:25.222829+0000","last_fullsized":"2024-09-11T08:40:25.222829+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:11.370245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:11.370245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:35.188374+0000","scrub_duration":0.0011869179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418548+0000","last_change":"2024-09-11T08:39:42.605373+0000","last_active":"2024-09-11T08:40:25.418548+0000","last_peered":"2024-09-11T08:40:25.418548+0000","last_clean":"2024-09-11T08:40:25.418548+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:40:25.418548+0000","last_undegraded":"2024-09-11T08:40:25.418548+0000","last_fullsized":"2024-09-11T08:40:25.418548+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:42.605301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:42.605301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:53.937210+0000","scrub_duration":0.0014798509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223866+0000","last_change":"2024-09-11T08:40:17.096092+0000","last_active":"2024-09-11T08:40:25.223866+0000","last_peered":"2024-09-11T08:40:25.223866+0000","last_clean":"2024-09-11T08:40:25.223866+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:40:25.223866+0000","last_undegraded":"2024-09-11T08:40:25.223866+0000","last_fullsized":"2024-09-11T08:40:25.223866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-11T08:40:17.096034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:17.096034+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:43.002314+0000","scrub_duration":0.0019684910000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223471+0000","last_change":"2024-09-11T08:39:49.472764+0000","last_active":"2024-09-11T08:40:25.223471+0000","last_peered":"2024-09-11T08:40:25.223471+0000","last_clean":"2024-09-11T08:40:25.223471+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:40:25.223471+0000","last_undegraded":"2024-09-11T08:40:25.223471+0000","last_fullsized":"2024-09-11T08:40:25.223471+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.472696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.472696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:01.403308+0000","scrub_duration":0.001919104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:31.223728+0000","last_change":"2024-09-11T08:40:31.223694+0000","last_active":"2024-09-11T08:40:31.223728+0000","last_peered":"2024-09-11T08:40:31.223728+0000","last_clean":"2024-09-11T08:40:31.223728+0000","last_became_active":"2024-09-11T08:38:45.184753+0000","last_became_peered":"2024-09-11T08:38:45.184753+0000","last_unstale":"2024-09-11T08:40:31.223728+0000","last_undegraded":"2024-09-11T08:40:31.223728+0000","last_fullsized":"2024-09-11T08:40:31.223728+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:31.223635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:31.223635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:37.656818+0000","scrub_duration":0.001429636,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":60,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:33.692713+0000","last_change":"2024-09-11T08:40:33.692680+0000","last_active":"2024-09-11T08:40:33.692713+0000","last_peered":"2024-09-11T08:40:33.692713+0000","last_clean":"2024-09-11T08:40:33.692713+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:40:33.692713+0000","last_undegraded":"2024-09-11T08:40:33.692713+0000","last_fullsized":"2024-09-11T08:40:33.692713+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:40:33.692610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:33.692610+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:52.519660+0000","scrub_duration":0.0023198749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418789+0000","last_change":"2024-09-11T08:39:53.176095+0000","last_active":"2024-09-11T08:40:25.418789+0000","last_peered":"2024-09-11T08:40:25.418789+0000","last_clean":"2024-09-11T08:40:25.418789+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:40:25.418789+0000","last_undegraded":"2024-09-11T08:40:25.418789+0000","last_fullsized":"2024-09-11T08:40:25.418789+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.176035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.176035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:00.682857+0000","scrub_duration":0.001147531,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:28.199712+0000","last_change":"2024-09-11T08:40:28.199687+0000","last_active":"2024-09-11T08:40:28.199712+0000","last_peered":"2024-09-11T08:40:28.199712+0000","last_clean":"2024-09-11T08:40:28.199712+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:40:28.199712+0000","last_undegraded":"2024-09-11T08:40:28.199712+0000","last_fullsized":"2024-09-11T08:40:28.199712+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:28.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:28.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:57.722333+0000","scrub_duration":0.001581287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.413880+0000","last_change":"2024-09-11T08:39:52.493064+0000","last_active":"2024-09-11T08:40:25.413880+0000","last_peered":"2024-09-11T08:40:25.413880+0000","last_clean":"2024-09-11T08:40:25.413880+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:40:25.413880+0000","last_undegraded":"2024-09-11T08:40:25.413880+0000","last_fullsized":"2024-09-11T08:40:25.413880+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.493013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:13.717323+0000","scrub_duration":0.0012699160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":60,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:30.270690+0000","last_change":"2024-09-11T08:40:30.270654+0000","last_active":"2024-09-11T08:40:30.270690+0000","last_peered":"2024-09-11T08:40:30.270690+0000","last_clean":"2024-09-11T08:40:30.270690+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:40:30.270690+0000","last_undegraded":"2024-09-11T08:40:30.270690+0000","last_fullsized":"2024-09-11T08:40:30.270690+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:40:30.270582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:30.270582+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:35.812455+0000","scrub_duration":0.0025147979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417760+0000","last_change":"2024-09-11T08:40:20.467510+0000","last_active":"2024-09-11T08:40:25.417760+0000","last_peered":"2024-09-11T08:40:25.417760+0000","last_clean":"2024-09-11T08:40:25.417760+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:40:25.417760+0000","last_undegraded":"2024-09-11T08:40:25.417760+0000","last_fullsized":"2024-09-11T08:40:25.417760+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:20.467456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:20.467456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:39.736643+0000","scrub_duration":0.001886544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:27.693608+0000","last_change":"2024-09-11T08:40:25.104280+0000","last_active":"2024-09-11T08:40:27.693608+0000","last_peered":"2024-09-11T08:40:27.693608+0000","last_clean":"2024-09-11T08:40:27.693608+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:40:27.693608+0000","last_undegraded":"2024-09-11T08:40:27.693608+0000","last_fullsized":"2024-09-11T08:40:27.693608+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:25.104224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:25.104224+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:46.893915+0000","scrub_duration":0.001645695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.420367+0000","last_change":"2024-09-11T08:38:44.993997+0000","last_active":"2024-09-11T08:40:25.420367+0000","last_peered":"2024-09-11T08:40:25.420367+0000","last_clean":"2024-09-11T08:40:25.420367+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:40:25.420367+0000","last_undegraded":"2024-09-11T08:40:25.420367+0000","last_fullsized":"2024-09-11T08:40:25.420367+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223541+0000","last_change":"2024-09-11T08:40:22.076288+0000","last_active":"2024-09-11T08:40:25.223541+0000","last_peered":"2024-09-11T08:40:25.223541+0000","last_clean":"2024-09-11T08:40:25.223541+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:40:25.223541+0000","last_undegraded":"2024-09-11T08:40:25.223541+0000","last_fullsized":"2024-09-11T08:40:25.223541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:40:22.076198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.076198+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:48.206403+0000","scrub_duration":0.0024994370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419508+0000","last_change":"2024-09-11T08:40:14.202552+0000","last_active":"2024-09-11T08:40:25.419508+0000","last_peered":"2024-09-11T08:40:25.419508+0000","last_clean":"2024-09-11T08:40:25.419508+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:40:25.419508+0000","last_undegraded":"2024-09-11T08:40:25.419508+0000","last_fullsized":"2024-09-11T08:40:25.419508+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:14.202498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:14.202498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:42.406451+0000","scrub_duration":0.001204299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:27.694039+0000","last_change":"2024-09-11T08:39:44.429546+0000","last_active":"2024-09-11T08:40:27.694039+0000","last_peered":"2024-09-11T08:40:27.694039+0000","last_clean":"2024-09-11T08:40:27.694039+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:40:27.694039+0000","last_undegraded":"2024-09-11T08:40:27.694039+0000","last_fullsized":"2024-09-11T08:40:27.694039+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:39:44.429477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:44.429477+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.023544+0000","scrub_duration":0.001890784,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417905+0000","last_change":"2024-09-11T08:39:52.254299+0000","last_active":"2024-09-11T08:40:25.417905+0000","last_peered":"2024-09-11T08:40:25.417905+0000","last_clean":"2024-09-11T08:40:25.417905+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:40:25.417905+0000","last_undegraded":"2024-09-11T08:40:25.417905+0000","last_fullsized":"2024-09-11T08:40:25.417905+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.254245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.254245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:20.782062+0000","scrub_duration":0.001581877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419998+0000","last_change":"2024-09-11T08:38:40.973815+0000","last_active":"2024-09-11T08:40:25.419998+0000","last_peered":"2024-09-11T08:40:25.419998+0000","last_clean":"2024-09-11T08:40:25.419998+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:40:25.419998+0000","last_undegraded":"2024-09-11T08:40:25.419998+0000","last_fullsized":"2024-09-11T08:40:25.419998+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:31.755583+0000","last_change":"2024-09-11T08:40:31.755542+0000","last_active":"2024-09-11T08:40:31.755583+0000","last_peered":"2024-09-11T08:40:31.755583+0000","last_clean":"2024-09-11T08:40:31.755583+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:40:31.755583+0000","last_undegraded":"2024-09-11T08:40:31.755583+0000","last_fullsized":"2024-09-11T08:40:31.755583+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:31.755476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:31.755476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:52.682119+0000","scrub_duration":0.001622713,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:26.218290+0000","last_change":"2024-09-11T08:40:26.218247+0000","last_active":"2024-09-11T08:40:26.218290+0000","last_peered":"2024-09-11T08:40:26.218290+0000","last_clean":"2024-09-11T08:40:26.218290+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:40:26.218290+0000","last_undegraded":"2024-09-11T08:40:26.218290+0000","last_fullsized":"2024-09-11T08:40:26.218290+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:26.218191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:26.218191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:48.264069+0000","scrub_duration":0.0016980210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222401+0000","last_change":"2024-09-11T08:38:44.994241+0000","last_active":"2024-09-11T08:40:25.222401+0000","last_peered":"2024-09-11T08:40:25.222401+0000","last_clean":"2024-09-11T08:40:25.222401+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:40:25.222401+0000","last_undegraded":"2024-09-11T08:40:25.222401+0000","last_fullsized":"2024-09-11T08:40:25.222401+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222744+0000","last_change":"2024-09-11T08:40:22.382123+0000","last_active":"2024-09-11T08:40:25.222744+0000","last_peered":"2024-09-11T08:40:25.222744+0000","last_clean":"2024-09-11T08:40:25.222744+0000","last_became_active":"2024-09-11T08:40:16.181181+0000","last_became_peered":"2024-09-11T08:40:16.181181+0000","last_unstale":"2024-09-11T08:40:25.222744+0000","last_undegraded":"2024-09-11T08:40:25.222744+0000","last_fullsized":"2024-09-11T08:40:25.222744+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:40:22.382048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.382048+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.268329+0000","scrub_duration":0.001806862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T08:40:28.273976+0000","last_change":"2024-09-11T08:40:28.273944+0000","last_active":"2024-09-11T08:40:28.273976+0000","last_peered":"2024-09-11T08:40:28.273976+0000","last_clean":"2024-09-11T08:40:28.273976+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:40:28.273976+0000","last_undegraded":"2024-09-11T08:40:28.273976+0000","last_fullsized":"2024-09-11T08:40:28.273976+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:28.273889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:28.273889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:42.340922+0000","scrub_duration":0.0011932570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223136+0000","last_change":"2024-09-11T08:40:24.089168+0000","last_active":"2024-09-11T08:40:25.223136+0000","last_peered":"2024-09-11T08:40:25.223136+0000","last_clean":"2024-09-11T08:40:25.223136+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:40:25.223136+0000","last_undegraded":"2024-09-11T08:40:25.223136+0000","last_fullsized":"2024-09-11T08:40:25.223136+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:24.089115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:24.089115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:27.976340+0000","scrub_duration":0.001138305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.417516+0000","last_change":"2024-09-11T08:40:22.499148+0000","last_active":"2024-09-11T08:40:25.417516+0000","last_peered":"2024-09-11T08:40:25.417516+0000","last_clean":"2024-09-11T08:40:25.417516+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:40:25.417516+0000","last_undegraded":"2024-09-11T08:40:25.417516+0000","last_fullsized":"2024-09-11T08:40:25.417516+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:22.499088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.499088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.706106+0000","scrub_duration":0.0015631830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":52,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222515+0000","last_change":"2024-09-11T08:38:41.039878+0000","last_active":"2024-09-11T08:40:25.222515+0000","last_peered":"2024-09-11T08:40:25.222515+0000","last_clean":"2024-09-11T08:40:25.222515+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:40:25.222515+0000","last_undegraded":"2024-09-11T08:40:25.222515+0000","last_fullsized":"2024-09-11T08:40:25.222515+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:38:39.947194+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223462+0000","last_change":"2024-09-11T08:40:05.430740+0000","last_active":"2024-09-11T08:40:25.223462+0000","last_peered":"2024-09-11T08:40:25.223462+0000","last_clean":"2024-09-11T08:40:25.223462+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:40:25.223462+0000","last_undegraded":"2024-09-11T08:40:25.223462+0000","last_fullsized":"2024-09-11T08:40:25.223462+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:05.430666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:05.430666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.818673+0000","scrub_duration":0.001547629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418844+0000","last_change":"2024-09-11T08:39:47.104702+0000","last_active":"2024-09-11T08:40:25.418844+0000","last_peered":"2024-09-11T08:40:25.418844+0000","last_clean":"2024-09-11T08:40:25.418844+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:40:25.418844+0000","last_undegraded":"2024-09-11T08:40:25.418844+0000","last_fullsized":"2024-09-11T08:40:25.418844+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.104637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.104637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:51.311803+0000","scrub_duration":0.0015238020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.223207+0000","last_change":"2024-09-11T08:38:45.183913+0000","last_active":"2024-09-11T08:40:25.223207+0000","last_peered":"2024-09-11T08:40:25.223207+0000","last_clean":"2024-09-11T08:40:25.223207+0000","last_became_active":"2024-09-11T08:38:45.183548+0000","last_became_peered":"2024-09-11T08:38:45.183548+0000","last_unstale":"2024-09-11T08:40:25.223207+0000","last_undegraded":"2024-09-11T08:40:25.223207+0000","last_fullsized":"2024-09-11T08:40:25.223207+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222427+0000","last_change":"2024-09-11T08:39:41.989740+0000","last_active":"2024-09-11T08:40:25.222427+0000","last_peered":"2024-09-11T08:40:25.222427+0000","last_clean":"2024-09-11T08:40:25.222427+0000","last_became_active":"2024-09-11T08:38:40.973799+0000","last_became_peered":"2024-09-11T08:38:40.973799+0000","last_unstale":"2024-09-11T08:40:25.222427+0000","last_undegraded":"2024-09-11T08:40:25.222427+0000","last_fullsized":"2024-09-11T08:40:25.222427+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-11T08:39:41.989684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.989684+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:02.687182+0000","scrub_duration":0.00162959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418167+0000","last_change":"2024-09-11T08:40:08.336369+0000","last_active":"2024-09-11T08:40:25.418167+0000","last_peered":"2024-09-11T08:40:25.418167+0000","last_clean":"2024-09-11T08:40:25.418167+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:40:25.418167+0000","last_undegraded":"2024-09-11T08:40:25.418167+0000","last_fullsized":"2024-09-11T08:40:25.418167+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:40:08.336314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.336314+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:16.115575+0000","scrub_duration":0.001394085,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418539+0000","last_change":"2024-09-11T08:39:53.541628+0000","last_active":"2024-09-11T08:40:25.418539+0000","last_peered":"2024-09-11T08:40:25.418539+0000","last_clean":"2024-09-11T08:40:25.418539+0000","last_became_active":"2024-09-11T08:38:42.989556+0000","last_became_peered":"2024-09-11T08:38:42.989556+0000","last_unstale":"2024-09-11T08:40:25.418539+0000","last_undegraded":"2024-09-11T08:40:25.418539+0000","last_fullsized":"2024-09-11T08:40:25.418539+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.541557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.541557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:08.967817+0000","scrub_duration":0.001310285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418891+0000","last_change":"2024-09-11T08:40:07.114884+0000","last_active":"2024-09-11T08:40:25.418891+0000","last_peered":"2024-09-11T08:40:25.418891+0000","last_clean":"2024-09-11T08:40:25.418891+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:40:25.418891+0000","last_undegraded":"2024-09-11T08:40:25.418891+0000","last_fullsized":"2024-09-11T08:40:25.418891+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:07.114815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:07.114815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.897957+0000","scrub_duration":0.0013781500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222869+0000","last_change":"2024-09-11T08:39:52.480663+0000","last_active":"2024-09-11T08:40:25.222869+0000","last_peered":"2024-09-11T08:40:25.222869+0000","last_clean":"2024-09-11T08:40:25.222869+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:40:25.222869+0000","last_undegraded":"2024-09-11T08:40:25.222869+0000","last_fullsized":"2024-09-11T08:40:25.222869+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.480610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.480610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:55.608483+0000","scrub_duration":0.001335754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.222617+0000","last_change":"2024-09-11T08:39:53.038356+0000","last_active":"2024-09-11T08:40:25.222617+0000","last_peered":"2024-09-11T08:40:25.222617+0000","last_clean":"2024-09-11T08:40:25.222617+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:40:25.222617+0000","last_undegraded":"2024-09-11T08:40:25.222617+0000","last_fullsized":"2024-09-11T08:40:25.222617+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.038297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.038297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:40:58.424182+0000","scrub_duration":0.0012198549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.419494+0000","last_change":"2024-09-11T08:40:02.551056+0000","last_active":"2024-09-11T08:40:25.419494+0000","last_peered":"2024-09-11T08:40:25.419494+0000","last_clean":"2024-09-11T08:40:25.419494+0000","last_became_active":"2024-09-11T08:38:40.974451+0000","last_became_peered":"2024-09-11T08:38:40.974451+0000","last_unstale":"2024-09-11T08:40:25.419494+0000","last_undegraded":"2024-09-11T08:40:25.419494+0000","last_fullsized":"2024-09-11T08:40:25.419494+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:40:02.550999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.550999+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.173446+0000","scrub_duration":0.0016484399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T08:40:25.418027+0000","last_change":"2024-09-11T08:39:55.265155+0000","last_active":"2024-09-11T08:40:25.418027+0000","last_peered":"2024-09-11T08:40:25.418027+0000","last_clean":"2024-09-11T08:40:25.418027+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:40:25.418027+0000","last_undegraded":"2024-09-11T08:40:25.418027+0000","last_fullsized":"2024-09-11T08:40:25.418027+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.265097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.265097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:09.333633+0000","scrub_duration":0.0012893830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":423,"ondisk_log_size":423,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299421,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298632,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446520,"statfs":{"total":95995035648,"available":95689236480,"internally_reserved":0,"allocated":1310720,"data_stored":948466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.499,"15min":0.501},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.66300000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71599999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.50900000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.50700000000000001},"min":{"1min":0.32700000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.74099999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.56499999999999995}]},{"osd":3,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71799999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.495},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.53000000000000003},"min":{"1min":0.377,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70299999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60299999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.42499999999999999,"15min":0.42199999999999999},"min":{"1min":0.318,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.59399999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.46800000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.30099999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.44700000000000001,"15min":0.439},"min":{"1min":0.313,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.44500000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47399999999999998,"15min":0.46899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.42799999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528354,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298612,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446540,"statfs":{"total":95995035648,"available":95689256960,"internally_reserved":0,"allocated":1290240,"data_stored":947693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.71699999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.65200000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.59799999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.70799999999999996,"5min":0.871,"15min":0.871},"last":0.66400000000000003}]},{"osd":2,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70999999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.755},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.71399999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.66200000000000003,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66800000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.63700000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.27500000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40600000000000003},"min":{"1min":0.29199999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.51000000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.439,"15min":0.43099999999999999},"min":{"1min":0.34200000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.435,"15min":0.42999999999999999},"min":{"1min":0.30299999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.54500000000000004}]},{"osd":7,"last update":"Wed Sep 11 08:40:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.44400000000000001,"15min":0.432},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58099999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757286,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298052,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447100,"statfs":{"total":95995035648,"available":95689830400,"internally_reserved":0,"allocated":716800,"data_stored":358731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61599999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.5},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.53400000000000003}]},{"osd":2,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.47999999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63200000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.376},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.34000000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.42599999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73099999999999998,"5min":0.749,"15min":0.749},"last":0.313}]},{"osd":7,"last update":"Wed Sep 11 08:40:27 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.60799999999999998,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.40899999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.371}]}]},{"osd":4,"up_from":31,"seq":133143986218,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298048,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447104,"statfs":{"total":95995035648,"available":95689834496,"internally_reserved":0,"allocated":712704,"data_stored":357316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55500000000000005,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.40500000000000003}]},{"osd":1,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.38,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.38800000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.39600000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.45200000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55800000000000005,"15min":0.57799999999999996},"min":{"1min":0.38800000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72199999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.371},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.433}]},{"osd":5,"last update":"Wed Sep 11 08:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.40500000000000003,"15min":0.38700000000000001},"min":{"1min":0.32100000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.499},{"interface":"front","average":{"1min":0.495,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.30099999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.318}]},{"osd":6,"last update":"Wed Sep 11 08:40:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.32400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.36199999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67100000000000004,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:40:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48899999999999999,"15min":0.48099999999999998},"min":{"1min":0.30199999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.47599999999999998,"15min":0.46800000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48199999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149744,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298076,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447076,"statfs":{"total":95995035648,"available":95689805824,"internally_reserved":0,"allocated":741376,"data_stored":359141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.36299999999999999,"15min":0.374},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.443,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.38900000000000001,"15min":0.40000000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.496,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.35699999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.39200000000000002,"15min":0.40400000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.56200000000000006,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.318},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.40400000000000003,"15min":0.41399999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48199999999999998,"5min":0.621,"15min":0.621},"last":0.26800000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.42699999999999999,"15min":0.435},"min":{"1min":0.27100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.44,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.33700000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:40:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79900000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54900000000000004,"15min":0.55900000000000005},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66600000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50800000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.55200000000000005}]},{"osd":6,"last update":"Wed Sep 11 08:40:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76300000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.60399999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:40:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60999999999999999,"15min":0.61799999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86799999999999999,"5min":1.22,"15min":1.22},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.86099999999999999,"5min":1.125,"15min":1.125},"last":0.58099999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411380,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298036,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447116,"statfs":{"total":95995035648,"available":95689846784,"internally_reserved":0,"allocated":696320,"data_stored":357271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36499999999999999,"15min":0.35399999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.38900000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.25800000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.372},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.39100000000000001,"15min":0.376},"min":{"1min":0.28999999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.29199999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:39:57 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.441,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.34499999999999997}]},{"osd":4,"last update":"Wed Sep 11 08:40:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.751,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.48699999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:39:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed Sep 11 08:40:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.442,"5min":0.432,"15min":0.432},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.56100000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57499999999999996}]},{"osd":7,"last update":"Wed Sep 11 08:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.41699999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76200000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.496},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81799999999999995,"5min":1.024,"15min":1.024},"last":0.56599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298080,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447072,"statfs":{"total":95995035648,"available":95689801728,"internally_reserved":0,"allocated":741376,"data_stored":358205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:19 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.314},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.47399999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33300000000000002},"min":{"1min":0.27400000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.33000000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:39:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.36899999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.45000000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:39:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40400000000000003,"15min":0.41999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54400000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.433,"15min":0.434},"min":{"1min":0.307,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:40:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.76500000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.625},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55100000000000005,"15min":0.56499999999999995},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.63300000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47299999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:39:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.755,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82199999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51600000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:40:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57399999999999995,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60099999999999998,"15min":0.60599999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54700000000000004}]},{"osd":7,"last update":"Wed Sep 11 08:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.45600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79700000000000004,"5min":0.879,"15min":0.879},"last":0.56899999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298636,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446516,"statfs":{"total":95995035648,"available":95689232384,"internally_reserved":0,"allocated":1314816,"data_stored":948525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:40:10 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.311},"min":{"1min":0.19800000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.436,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34000000000000002,"15min":0.314},"min":{"1min":0.23999999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.504,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.34699999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.35799999999999998,"15min":0.34699999999999998},"min":{"1min":0.24199999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53500000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.34899999999999998},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.51500000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.27700000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:39:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39800000000000002,"15min":0.40000000000000002},"min":{"1min":0.28499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.503,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.311}]},{"osd":4,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70899999999999996,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.53600000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.503}]},{"osd":6,"last update":"Wed Sep 11 08:40:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.37,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68300000000000005,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.42499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.746,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48299999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:40:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.44}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:40:40.433 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T08:40:40.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd dump 2024-09-11T08:40:41.262 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:41.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:40 smithi092 ceph-mon[29647]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:41.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:40 smithi092 ceph-mon[29647]: 4.b scrub starts 2024-09-11T08:40:41.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:40 smithi092 ceph-mon[29647]: 4.b scrub ok 2024-09-11T08:40:41.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:40 smithi092 ceph-mon[41114]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:41.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:40 smithi092 ceph-mon[41114]: 4.b scrub starts 2024-09-11T08:40:41.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:40 smithi092 ceph-mon[41114]: 4.b scrub ok 2024-09-11T08:40:41.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:40 smithi145 ceph-mon[33384]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:40:41.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:40 smithi145 ceph-mon[33384]: 4.b scrub starts 2024-09-11T08:40:41.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:40 smithi145 ceph-mon[33384]: 4.b scrub ok 2024-09-11T08:40:41.590 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:40:41] "GET /metrics HTTP/1.1" 200 220039 "" "Prometheus/2.43.0" 2024-09-11T08:40:42.289 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:42 smithi092 ceph-mon[29647]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:42.289 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:42 smithi092 ceph-mon[29647]: 6.19 scrub starts 2024-09-11T08:40:42.290 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:42 smithi092 ceph-mon[29647]: 6.19 scrub ok 2024-09-11T08:40:42.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:42 smithi092 ceph-mon[41114]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:42.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:42 smithi092 ceph-mon[41114]: 6.19 scrub starts 2024-09-11T08:40:42.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:42 smithi092 ceph-mon[41114]: 6.19 scrub ok 2024-09-11T08:40:42.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:42 smithi145 ceph-mon[33384]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:42.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:42 smithi145 ceph-mon[33384]: 6.19 scrub starts 2024-09-11T08:40:42.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:42 smithi145 ceph-mon[33384]: 6.19 scrub ok 2024-09-11T08:40:42.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:40:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:40:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:43.232 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:43 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:40:43.232 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:43 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:40:43.232 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:43 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:40:43.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:43 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:40:43.334 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:43 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:40:43.380 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:43 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:40:43.416 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 84 2024-09-11T08:40:43.416 INFO:teuthology.orchestra.run.smithi092.stdout:fsid 211625d4-7018-11ef-bcea-c7b262605968 2024-09-11T08:40:43.416 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-09-11T08:33:32.174444+0000 2024-09-11T08:40:43.416 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-09-11T08:40:43.001964+0000 2024-09-11T08:40:43.416 INFO:teuthology.orchestra.run.smithi092.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T08:40:43.416 INFO:teuthology.orchestra.run.smithi092.stdout:crush_version 18 2024-09-11T08:40:43.416 INFO:teuthology.orchestra.run.smithi092.stdout:full_ratio 0.95 2024-09-11T08:40:43.416 INFO:teuthology.orchestra.run.smithi092.stdout:backfillfull_ratio 0.9 2024-09-11T08:40:43.416 INFO:teuthology.orchestra.run.smithi092.stdout:nearfull_ratio 0.85 2024-09-11T08:40:43.417 INFO:teuthology.orchestra.run.smithi092.stdout:require_min_compat_client luminous 2024-09-11T08:40:43.417 INFO:teuthology.orchestra.run.smithi092.stdout:min_compat_client luminous 2024-09-11T08:40:43.417 INFO:teuthology.orchestra.run.smithi092.stdout:require_osd_release quincy 2024-09-11T08:40:43.417 INFO:teuthology.orchestra.run.smithi092.stdout:stretch_mode_enabled false 2024-09-11T08:40:43.417 INFO:teuthology.orchestra.run.smithi092.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2024-09-11T08:40:43.417 INFO:teuthology.orchestra.run.smithi092.stdout:pool 2 'datapool' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 3 pgp_num 3 autoscale_mode off last_change 60 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd 2024-09-11T08:40:43.417 INFO:teuthology.orchestra.run.smithi092.stdout:pool 3 '.rgw.root' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 57 flags hashpspool stripe_width 0 application rgw 2024-09-11T08:40:43.417 INFO:teuthology.orchestra.run.smithi092.stdout:pool 4 'default.rgw.log' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 59 flags hashpspool stripe_width 0 application rgw 2024-09-11T08:40:43.417 INFO:teuthology.orchestra.run.smithi092.stdout:pool 5 'default.rgw.control' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 61 flags hashpspool stripe_width 0 application rgw 2024-09-11T08:40:43.417 INFO:teuthology.orchestra.run.smithi092.stdout:pool 6 'default.rgw.meta' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 63 flags hashpspool stripe_width 0 pg_autoscale_bias 4 application rgw 2024-09-11T08:40:43.417 INFO:teuthology.orchestra.run.smithi092.stdout:max_osd 8 2024-09-11T08:40:43.417 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 up in weight 1 up_from 8 up_thru 61 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6802/2040613580,v1:172.21.15.92:6803/2040613580] [v2:172.21.15.92:6804/2040613580,v1:172.21.15.92:6805/2040613580] exists,up 9d8a3811-e801-4b6a-9a63-83936283cb7d 2024-09-11T08:40:43.417 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 up in weight 1 up_from 13 up_thru 61 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328] [v2:172.21.15.92:6812/544779328,v1:172.21.15.92:6813/544779328] exists,up e7a18e45-0137-45ef-8b85-e458cf3737ac 2024-09-11T08:40:43.418 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 up in weight 1 up_from 18 up_thru 61 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777] [v2:172.21.15.92:6820/933736777,v1:172.21.15.92:6821/933736777] exists,up 017282e2-8053-4fa7-affc-a05408fa481e 2024-09-11T08:40:43.418 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 up in weight 1 up_from 26 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6826/878892459,v1:172.21.15.92:6827/878892459] [v2:172.21.15.92:6828/878892459,v1:172.21.15.92:6829/878892459] exists,up 58e5a8a4-9e58-40ad-9b40-90cb5ff5414b 2024-09-11T08:40:43.418 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 up in weight 1 up_from 31 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669] [v2:172.21.15.145:6802/2792751669,v1:172.21.15.145:6803/2792751669] exists,up e144e102-963b-48b7-92a9-9391706164bf 2024-09-11T08:40:43.418 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 up in weight 1 up_from 38 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6808/3569637188,v1:172.21.15.145:6809/3569637188] [v2:172.21.15.145:6810/3569637188,v1:172.21.15.145:6811/3569637188] exists,up 811b0dc9-b7a7-439a-86c1-5dc9d0d54e16 2024-09-11T08:40:43.418 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6 up in weight 1 up_from 45 up_thru 59 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6816/2615047110,v1:172.21.15.145:6817/2615047110] [v2:172.21.15.145:6818/2615047110,v1:172.21.15.145:6819/2615047110] exists,up bd162f52-097c-4c5d-9d37-1289779a196b 2024-09-11T08:40:43.418 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7 up in weight 1 up_from 52 up_thru 61 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6824/3195898530,v1:172.21.15.145:6825/3195898530] [v2:172.21.15.145:6826/3195898530,v1:172.21.15.145:6827/3195898530] exists,up 4e4021bf-78c7-4bc6-b222-3494f724ddec 2024-09-11T08:40:43.418 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 4.e [1,2] 2024-09-11T08:40:43.418 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 4.12 [1,5] 2024-09-11T08:40:43.418 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 4.17 [1,2] 2024-09-11T08:40:43.418 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6800/1861854063 expires 2024-09-12T08:39:14.984442+0000 2024-09-11T08:40:43.418 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/4113910808 expires 2024-09-12T08:33:54.195816+0000 2024-09-11T08:40:43.418 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/3501902574 expires 2024-09-12T08:39:14.984442+0000 2024-09-11T08:40:43.419 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/1583695466 expires 2024-09-12T08:33:54.195816+0000 2024-09-11T08:40:43.419 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/736380628 expires 2024-09-12T08:33:54.195816+0000 2024-09-11T08:40:43.419 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6800/4060453592 expires 2024-09-12T08:33:54.195816+0000 2024-09-11T08:40:43.419 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/2234574262 expires 2024-09-12T08:39:14.984442+0000 2024-09-11T08:40:43.419 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6800/2480574848 expires 2024-09-12T08:34:17.306389+0000 2024-09-11T08:40:43.419 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/1540079149 expires 2024-09-12T08:39:14.984442+0000 2024-09-11T08:40:43.419 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/2680744738 expires 2024-09-12T08:34:17.306389+0000 2024-09-11T08:40:43.419 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6801/4060453592 expires 2024-09-12T08:33:54.195816+0000 2024-09-11T08:40:43.419 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6801/1861854063 expires 2024-09-12T08:39:14.984442+0000 2024-09-11T08:40:43.419 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6801/2480574848 expires 2024-09-12T08:34:17.306389+0000 2024-09-11T08:40:43.419 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/12697418 expires 2024-09-12T08:34:17.306389+0000 2024-09-11T08:40:43.419 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/4280958069 expires 2024-09-12T08:34:17.306389+0000 2024-09-11T08:40:43.419 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/2686720854 expires 2024-09-12T08:39:14.984442+0000 2024-09-11T08:40:43.419 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/969221731 expires 2024-09-12T08:39:14.984442+0000 2024-09-11T08:40:44.160 INFO:tasks.cephadm.ceph_manager.ceph:['osd.0 up in weight 1 up_from 8 up_thru 61 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6802/2040613580,v1:172.21.15.92:6803/2040613580] [v2:172.21.15.92:6804/2040613580,v1:172.21.15.92:6805/2040613580] exists,up 9d8a3811-e801-4b6a-9a63-83936283cb7d', 'osd.1 up in weight 1 up_from 13 up_thru 61 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6810/544779328,v1:172.21.15.92:6811/544779328] [v2:172.21.15.92:6812/544779328,v1:172.21.15.92:6813/544779328] exists,up e7a18e45-0137-45ef-8b85-e458cf3737ac', 'osd.2 up in weight 1 up_from 18 up_thru 61 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6818/933736777,v1:172.21.15.92:6819/933736777] [v2:172.21.15.92:6820/933736777,v1:172.21.15.92:6821/933736777] exists,up 017282e2-8053-4fa7-affc-a05408fa481e', 'osd.3 up in weight 1 up_from 26 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6826/878892459,v1:172.21.15.92:6827/878892459] [v2:172.21.15.92:6828/878892459,v1:172.21.15.92:6829/878892459] exists,up 58e5a8a4-9e58-40ad-9b40-90cb5ff5414b', 'osd.4 up in weight 1 up_from 31 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6800/2792751669,v1:172.21.15.145:6801/2792751669] [v2:172.21.15.145:6802/2792751669,v1:172.21.15.145:6803/2792751669] exists,up e144e102-963b-48b7-92a9-9391706164bf', 'osd.5 up in weight 1 up_from 38 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6808/3569637188,v1:172.21.15.145:6809/3569637188] [v2:172.21.15.145:6810/3569637188,v1:172.21.15.145:6811/3569637188] exists,up 811b0dc9-b7a7-439a-86c1-5dc9d0d54e16', 'osd.6 up in weight 1 up_from 45 up_thru 59 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6816/2615047110,v1:172.21.15.145:6817/2615047110] [v2:172.21.15.145:6818/2615047110,v1:172.21.15.145:6819/2615047110] exists,up bd162f52-097c-4c5d-9d37-1289779a196b', 'osd.7 up in weight 1 up_from 52 up_thru 61 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6824/3195898530,v1:172.21.15.145:6825/3195898530] [v2:172.21.15.145:6826/3195898530,v1:172.21.15.145:6827/3195898530] exists,up 4e4021bf-78c7-4bc6-b222-3494f724ddec'] 2024-09-11T08:40:44.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon mon.a config show 2024-09-11T08:40:44.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[41114]: 4.1b scrub starts 2024-09-11T08:40:44.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[41114]: 4.1b scrub ok 2024-09-11T08:40:44.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[41114]: osdmap e84: 8 total, 8 up, 8 in 2024-09-11T08:40:44.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[41114]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:40:44.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:40:44.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:40:44.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3214921642' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T08:40:44.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:44.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[29647]: 4.1b scrub starts 2024-09-11T08:40:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[29647]: 4.1b scrub ok 2024-09-11T08:40:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[29647]: osdmap e84: 8 total, 8 up, 8 in 2024-09-11T08:40:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[29647]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:40:44.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:40:44.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:40:44.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3214921642' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T08:40:44.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:44.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:44 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:44.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:44 smithi145 ceph-mon[33384]: 4.1b scrub starts 2024-09-11T08:40:44.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:44 smithi145 ceph-mon[33384]: 4.1b scrub ok 2024-09-11T08:40:44.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:44 smithi145 ceph-mon[33384]: osdmap e84: 8 total, 8 up, 8 in 2024-09-11T08:40:44.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:44 smithi145 ceph-mon[33384]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:40:44.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:44 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:40:44.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:44 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:40:44.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:44 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3214921642' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T08:40:44.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:44 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:44.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:44 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:44.912 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:45.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:45 smithi092 ceph-mon[29647]: osdmap e85: 8 total, 8 up, 8 in 2024-09-11T08:40:45.277 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:45 smithi092 ceph-mon[41114]: osdmap e85: 8 total, 8 up, 8 in 2024-09-11T08:40:45.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:45 smithi145 ceph-mon[33384]: osdmap e85: 8 total, 8 up, 8 in 2024-09-11T08:40:46.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[29647]: 6.b scrub starts 2024-09-11T08:40:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[29647]: 6.b scrub ok 2024-09-11T08:40:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:40:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:40:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[29647]: 5.18 scrub starts 2024-09-11T08:40:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[29647]: 5.18 scrub ok 2024-09-11T08:40:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[29647]: 4.3 scrub starts 2024-09-11T08:40:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[29647]: 4.3 scrub ok 2024-09-11T08:40:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[29647]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:40:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[29647]: osdmap e86: 8 total, 8 up, 8 in 2024-09-11T08:40:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[29647]: 6.d scrub starts 2024-09-11T08:40:46.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[29647]: 6.d scrub ok 2024-09-11T08:40:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[41114]: 6.b scrub starts 2024-09-11T08:40:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[41114]: 6.b scrub ok 2024-09-11T08:40:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:40:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:40:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[41114]: 5.18 scrub starts 2024-09-11T08:40:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[41114]: 5.18 scrub ok 2024-09-11T08:40:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[41114]: 4.3 scrub starts 2024-09-11T08:40:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[41114]: 4.3 scrub ok 2024-09-11T08:40:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[41114]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:40:46.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[41114]: osdmap e86: 8 total, 8 up, 8 in 2024-09-11T08:40:46.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:46.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[41114]: 6.d scrub starts 2024-09-11T08:40:46.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:46 smithi092 ceph-mon[41114]: 6.d scrub ok 2024-09-11T08:40:46.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:40:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:40:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:46 smithi145 ceph-mon[33384]: 6.b scrub starts 2024-09-11T08:40:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:46 smithi145 ceph-mon[33384]: 6.b scrub ok 2024-09-11T08:40:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:46 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:40:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:46 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:40:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:46 smithi145 ceph-mon[33384]: 5.18 scrub starts 2024-09-11T08:40:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:46 smithi145 ceph-mon[33384]: 5.18 scrub ok 2024-09-11T08:40:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:46 smithi145 ceph-mon[33384]: 4.3 scrub starts 2024-09-11T08:40:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:46 smithi145 ceph-mon[33384]: 4.3 scrub ok 2024-09-11T08:40:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:46 smithi145 ceph-mon[33384]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:40:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:46 smithi145 ceph-mon[33384]: osdmap e86: 8 total, 8 up, 8 in 2024-09-11T08:40:46.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:46 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:46.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:46 smithi145 ceph-mon[33384]: 6.d scrub starts 2024-09-11T08:40:46.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:46 smithi145 ceph-mon[33384]: 6.d scrub ok 2024-09-11T08:40:46.952 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:40:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mon.a", 2024-09-11T08:40:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster": "ceph", 2024-09-11T08:40:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-11T08:40:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "admin_socket_mode": "", 2024-09-11T08:40:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_allow_insecure_global_id_reclaim": "false", 2024-09-11T08:40:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_client_required": "cephx, none", 2024-09-11T08:40:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_cluster_required": "cephx", 2024-09-11T08:40:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_debug": "true", 2024-09-11T08:40:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-11T08:40:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-11T08:40:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_service_required": "cephx", 2024-09-11T08:40:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-11T08:40:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_supported": "", 2024-09-11T08:40:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio": "true", 2024-09-11T08:40:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-11T08:40:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio_poll_ms": "250", 2024-09-11T08:40:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio_reap_max": "16", 2024-09-11T08:40:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_async_discard": "false", 2024-09-11T08:40:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_block_size": "4096", 2024-09-11T08:40:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_aio": "false", 2024-09-11T08:40:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-11T08:40:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-11T08:40:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_inflight_ios": "false", 2024-09-11T08:40:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_enable_discard": "false", 2024-09-11T08:40:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_flock_retry": "3", 2024-09-11T08:40:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-11T08:40:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_inject_crash": "0", 2024-09-11T08:40:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-11T08:40:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_ioring": "false", 2024-09-11T08:40:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_ioring_hipri": "false", 2024-09-11T08:40:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-11T08:40:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-11T08:40:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-11T08:40:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-11T08:40:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_type": "", 2024-09-11T08:40:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_alloc_size": "1048576", 2024-09-11T08:40:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_allocator": "hybrid", 2024-09-11T08:40:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_buffered_io": "true", 2024-09-11T08:40:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_check_for_zeros": "false", 2024-09-11T08:40:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-11T08:40:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-11T08:40:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_compact_log_sync": "false", 2024-09-11T08:40:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-11T08:40:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-11T08:40:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-11T08:40:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-11T08:40:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_max_log_runway": "4194304", 2024-09-11T08:40:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_max_prefetch": "1048576", 2024-09-11T08:40:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_min_flush_size": "524288", 2024-09-11T08:40:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_min_log_runway": "1048576", 2024-09-11T08:40:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_replay_recovery": "false", 2024-09-11T08:40:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-11T08:40:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-11T08:40:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_sync_write": "false", 2024-09-11T08:40:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-11T08:40:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-11T08:40:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-11T08:40:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_allocation_from_file": "true", 2024-09-11T08:40:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_allocator": "hybrid", 2024-09-11T08:40:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_async_db_compaction": "true", 2024-09-11T08:40:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-11T08:40:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-11T08:40:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-11T08:40:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-11T08:40:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-11T08:40:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-11T08:40:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-11T08:40:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_create": "true", 2024-09-11T08:40:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_db_create": "false", 2024-09-11T08:40:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_db_path": "", 2024-09-11T08:40:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_db_size": "0", 2024-09-11T08:40:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_path": "", 2024-09-11T08:40:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_preallocate_file": "false", 2024-09-11T08:40:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_size": "107374182400", 2024-09-11T08:40:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_wal_create": "false", 2024-09-11T08:40:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_wal_path": "", 2024-09-11T08:40:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_wal_size": "100663296", 2024-09-11T08:40:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bluefs": "true", 2024-09-11T08:40:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-11T08:40:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-11T08:40:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-11T08:40:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-11T08:40:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-11T08:40:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-11T08:40:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-11T08:40:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-11T08:40:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_autotune": "true", 2024-09-11T08:40:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-11T08:40:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-11T08:40:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-11T08:40:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-11T08:40:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_size": "0", 2024-09-11T08:40:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-11T08:40:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-11T08:40:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-11T08:40:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-11T08:40:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_type": "2q", 2024-09-11T08:40:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-11T08:40:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_clone_cow": "true", 2024-09-11T08:40:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-11T08:40:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-11T08:40:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-11T08:40:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-11T08:40:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-11T08:40:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-11T08:40:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-11T08:40:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_mode": "none", 2024-09-11T08:40:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-11T08:40:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_csum_type": "crc32c", 2024-09-11T08:40:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-11T08:40:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_freelist": "false", 2024-09-11T08:40:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-11T08:40:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-11T08:40:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-11T08:40:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-11T08:40:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-11T08:40:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-11T08:40:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-11T08:40:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-11T08:40:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-11T08:40:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-11T08:40:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-11T08:40:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-11T08:40:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-11T08:40:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-11T08:40:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_small_allocations": "0", 2024-09-11T08:40:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-11T08:40:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_default_buffered_read": "true", 2024-09-11T08:40:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_default_buffered_write": "false", 2024-09-11T08:40:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-11T08:40:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-11T08:40:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-11T08:40:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-11T08:40:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-11T08:40:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-11T08:40:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-11T08:40:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-11T08:40:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fail_eio": "false", 2024-09-11T08:40:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-11T08:40:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-11T08:40:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-11T08:40:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-11T08:40:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-11T08:40:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-11T08:40:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mount": "false", 2024-09-11T08:40:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-11T08:40:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_umount": "false", 2024-09-11T08:40:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-11T08:40:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-11T08:40:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-11T08:40:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-11T08:40:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-11T08:40:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-11T08:40:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-11T08:40:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-11T08:40:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_ignore_data_csum": "false", 2024-09-11T08:40:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-11T08:40:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-11T08:40:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-11T08:40:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-11T08:40:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_log_op_age": "5.000000", 2024-09-11T08:40:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_alloc_size": "0", 2024-09-11T08:40:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_blob_size": "0", 2024-09-11T08:40:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-11T08:40:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-11T08:40:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-11T08:40:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_deferred_txc": "32", 2024-09-11T08:40:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_min_alloc_size": "0", 2024-09-11T08:40:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-11T08:40:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-11T08:40:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_nid_prealloc": "1024", 2024-09-11T08:40:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-11T08:40:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-11T08:40:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-11T08:40:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-11T08:40:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_retry_disk_reads": "3", 2024-09-11T08:40:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_rocksdb_cf": "true", 2024-09-11T08:40:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-09-11T08:40:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-09-11T08:40:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-11T08:40:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-11T08:40:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-11T08:40:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-11T08:40:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_mem": "512", 2024-09-11T08:40:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-11T08:40:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-11T08:40:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-11T08:40:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-11T08:40:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-11T08:40:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-11T08:40:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-11T08:40:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_tracing": "false", 2024-09-11T08:40:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-11T08:40:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-11T08:40:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-11T08:40:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-11T08:40:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-11T08:40:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-11T08:40:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-11T08:40:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-11T08:40:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-11T08:40:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_zero_block_detection": "false", 2024-09-11T08:40:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-11T08:40:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-11T08:40:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-11T08:40:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-11T08:40:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-11T08:40:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-11T08:40:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-11T08:40:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-11T08:40:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-11T08:40:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-11T08:40:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-11T08:40:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-11T08:40:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-11T08:40:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_cluster_require_signatures": "false", 2024-09-11T08:40:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_cluster_require_version": "2", 2024-09-11T08:40:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_require_signatures": "false", 2024-09-11T08:40:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_require_version": "2", 2024-09-11T08:40:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_service_require_signatures": "false", 2024-09-11T08:40:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_service_require_version": "2", 2024-09-11T08:40:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_sign_messages": "true", 2024-09-11T08:40:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "chdir": "", 2024-09-11T08:40:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "client_acl_type": "", 2024-09-11T08:40:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "client_asio_thread_count": "2", 2024-09-11T08:40:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_cache_mid": "0.750000", 2024-09-11T08:40:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_cache_size": "16384", 2024-09-11T08:40:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_caps_release_delay": "5", 2024-09-11T08:40:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_check_pool_perm": "true", 2024-09-11T08:40:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-11T08:40:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_debug_force_sync_read": "false", 2024-09-11T08:40:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_debug_getattr_caps": "false", 2024-09-11T08:40:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_debug_inject_features": "", 2024-09-11T08:40:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_debug_inject_tick_delay": "0", 2024-09-11T08:40:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-11T08:40:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_die_on_failed_remount": "false", 2024-09-11T08:40:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_dirsize_rbytes": "true", 2024-09-11T08:40:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_force_lazyio": "false", 2024-09-11T08:40:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_fs": "", 2024-09-11T08:40:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-11T08:40:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_inject_release_failure": "false", 2024-09-11T08:40:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_max_inline_size": "4096", 2024-09-11T08:40:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-11T08:40:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mds_namespace": "", 2024-09-11T08:40:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_metadata": "", 2024-09-11T08:40:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mount_gid": "-1", 2024-09-11T08:40:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mount_timeout": "300", 2024-09-11T08:40:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mount_uid": "-1", 2024-09-11T08:40:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mountpoint": "/", 2024-09-11T08:40:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_notify_timeout": "10", 2024-09-11T08:40:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc": "true", 2024-09-11T08:40:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_max_dirty": "104857600", 2024-09-11T08:40:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-11T08:40:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_max_objects": "1000", 2024-09-11T08:40:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_size": "209715200", 2024-09-11T08:40:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_target_dirty": "8388608", 2024-09-11T08:40:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_permissions": "true", 2024-09-11T08:40:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_quota": "true", 2024-09-11T08:40:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_quota_df": "true", 2024-09-11T08:40:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_readahead_max_bytes": "0", 2024-09-11T08:40:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_readahead_max_periods": "4", 2024-09-11T08:40:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_readahead_min": "131072", 2024-09-11T08:40:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_reconnect_stale": "false", 2024-09-11T08:40:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_shutdown_timeout": "30", 2024-09-11T08:40:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_snapdir": ".snap", 2024-09-11T08:40:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client_tick_interval": "1", 2024-09-11T08:40:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client_trace": "", 2024-09-11T08:40:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client_try_dentry_invalidate": "false", 2024-09-11T08:40:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client_use_faked_inos": "false", 2024-09-11T08:40:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client_use_random_mds": "false", 2024-09-11T08:40:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_graylog": "false", 2024-09-11T08:40:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-11T08:40:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_graylog_port": "12201", 2024-09-11T08:40:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_monitors": "default=true", 2024-09-11T08:40:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_syslog": "false", 2024-09-11T08:40:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-11T08:40:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_syslog_level": "info", 2024-09-11T08:40:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "-", 2024-09-11T08:40:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_network": "", 2024-09-11T08:40:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_network_interface": "", 2024-09-11T08:40:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zlib_isal": "false", 2024-09-11T08:40:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zlib_level": "5", 2024-09-11T08:40:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zlib_winsize": "-15", 2024-09-11T08:40:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zstd_level": "1", 2024-09-11T08:40:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40", 2024-09-11T08:40:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-11T08:40:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_alien_op_num_threads": "6", 2024-09-11T08:40:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-11T08:40:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-11T08:40:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-11T08:40:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_location": "", 2024-09-11T08:40:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_location_hook": "", 2024-09-11T08:40:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_location_hook_timeout": "10", 2024-09-11T08:40:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "daemonize": "false", 2024-09-11T08:40:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_alienstore": "0/5", 2024-09-11T08:40:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_allow_any_pool_priority": "false", 2024-09-11T08:40:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_asok": "1/5", 2024-09-11T08:40:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_asok_assert_abort": "false", 2024-09-11T08:40:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_asserts_on_shutdown": "true", 2024-09-11T08:40:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_auth": "1/5", 2024-09-11T08:40:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_bdev": "1/3", 2024-09-11T08:40:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_bluefs": "1/5", 2024-09-11T08:40:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_bluestore": "1/5", 2024-09-11T08:40:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_buffer": "0/1", 2024-09-11T08:40:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_ceph_exporter": "1/5", 2024-09-11T08:40:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_cephfs_mirror": "0/5", 2024-09-11T08:40:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_cephsqlite": "0/5", 2024-09-11T08:40:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_civetweb": "1/10", 2024-09-11T08:40:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_client": "0/5", 2024-09-11T08:40:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_compressor": "1/5", 2024-09-11T08:40:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_context": "0/1", 2024-09-11T08:40:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_crush": "1/1", 2024-09-11T08:40:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_crypto": "1/5", 2024-09-11T08:40:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_deliberately_leak_memory": "false", 2024-09-11T08:40:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_disable_randomized_ping": "false", 2024-09-11T08:40:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_dpdk": "1/5", 2024-09-11T08:40:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_eventtrace": "1/5", 2024-09-11T08:40:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_filer": "0/1", 2024-09-11T08:40:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_filestore": "1/3", 2024-09-11T08:40:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_finisher": "1/1", 2024-09-11T08:40:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_fuse": "1/5", 2024-09-11T08:40:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_heartbeat_testing_span": "0", 2024-09-11T08:40:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_heartbeatmap": "1/5", 2024-09-11T08:40:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-11T08:40:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_javaclient": "1/5", 2024-09-11T08:40:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_journal": "1/3", 2024-09-11T08:40:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_journaler": "0/5", 2024-09-11T08:40:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_kstore": "1/5", 2024-09-11T08:40:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_leveldb": "4/5", 2024-09-11T08:40:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_lockdep": "0/1", 2024-09-11T08:40:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mclock": "1/5", 2024-09-11T08:40:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds": "1/5", 2024-09-11T08:40:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_balancer": "1/5", 2024-09-11T08:40:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_locker": "1/5", 2024-09-11T08:40:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_log": "1/5", 2024-09-11T08:40:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_log_expire": "1/5", 2024-09-11T08:40:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_migrator": "1/5", 2024-09-11T08:40:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_memdb": "4/5", 2024-09-11T08:40:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mgr": "2/5", 2024-09-11T08:40:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mgrc": "1/5", 2024-09-11T08:40:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mon": "20/20", 2024-09-11T08:40:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_monc": "0/10", 2024-09-11T08:40:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_ms": "1/1", 2024-09-11T08:40:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_none": "0/5", 2024-09-11T08:40:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_objclass": "0/5", 2024-09-11T08:40:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_objectcacher": "0/5", 2024-09-11T08:40:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_objecter": "0/1", 2024-09-11T08:40:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_optracker": "0/5", 2024-09-11T08:40:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_osd": "1/5", 2024-09-11T08:40:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_paxos": "20/20", 2024-09-11T08:40:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_perfcounter": "1/5", 2024-09-11T08:40:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_prioritycache": "1/5", 2024-09-11T08:40:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rados": "0/5", 2024-09-11T08:40:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd": "0/5", 2024-09-11T08:40:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd_mirror": "0/5", 2024-09-11T08:40:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd_pwl": "0/5", 2024-09-11T08:40:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd_replay": "0/5", 2024-09-11T08:40:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_refs": "0/0", 2024-09-11T08:40:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_reserver": "1/1", 2024-09-11T08:40:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw": "1/5", 2024-09-11T08:40:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_datacache": "1/5", 2024-09-11T08:40:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_sync": "1/5", 2024-09-11T08:40:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rocksdb": "4/5", 2024-09-11T08:40:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore": "0/5", 2024-09-11T08:40:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_cache": "0/5", 2024-09-11T08:40:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_cleaner": "0/5", 2024-09-11T08:40:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_device": "0/5", 2024-09-11T08:40:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_journal": "0/5", 2024-09-11T08:40:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_lba": "0/5", 2024-09-11T08:40:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_odata": "0/5", 2024-09-11T08:40:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_omap": "0/5", 2024-09-11T08:40:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_onode": "0/5", 2024-09-11T08:40:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_tm": "0/5", 2024-09-11T08:40:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_striper": "0/1", 2024-09-11T08:40:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_test": "0/5", 2024-09-11T08:40:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_throttle": "1/1", 2024-09-11T08:40:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_timer": "0/1", 2024-09-11T08:40:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_tp": "0/5", 2024-09-11T08:40:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-11T08:40:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "device_failure_prediction_mode": "none", 2024-09-11T08:40:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-11T08:40:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-11T08:40:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "err_to_graylog": "false", 2024-09-11T08:40:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "err_to_journald": "false", 2024-09-11T08:40:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "err_to_stderr": "true", 2024-09-11T08:40:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "err_to_syslog": "false", 2024-09-11T08:40:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event_tracing": "false", 2024-09-11T08:40:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_addr": "0.0.0.0", 2024-09-11T08:40:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_http_port": "9926", 2024-09-11T08:40:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_prio_limit": "5", 2024-09-11T08:40:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-11T08:40:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_sort_metrics": "true", 2024-09-11T08:40:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_stats_period": "5", 2024-09-11T08:40:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "fake_statfs_for_testing": "0", 2024-09-11T08:40:46.983 INFO:teuthology.orchestra.run.smithi092.stdout: "fatal_signal_handlers": "true", 2024-09-11T08:40:46.983 INFO:teuthology.orchestra.run.smithi092.stdout: "filer_max_purge_ops": "10", 2024-09-11T08:40:46.983 INFO:teuthology.orchestra.run.smithi092.stdout: "filer_max_truncate_ops": "128", 2024-09-11T08:40:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_apply_finisher_threads": "1", 2024-09-11T08:40:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_blackhole": "false", 2024-09-11T08:40:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_btrfs_clone_range": "true", 2024-09-11T08:40:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_btrfs_snap": "true", 2024-09-11T08:40:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_caller_concurrency": "10", 2024-09-11T08:40:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_collect_device_partition_information": "true", 2024-09-11T08:40:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_commit_timeout": "600.000000", 2024-09-11T08:40:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_debug_inject_read_err": "false", 2024-09-11T08:40:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_debug_omap_check": "false", 2024-09-11T08:40:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-11T08:40:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_debug_verify_split": "false", 2024-09-11T08:40:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_dump_file": "", 2024-09-11T08:40:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-11T08:40:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-11T08:40:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fadvise": "true", 2024-09-11T08:40:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fail_eio": "true", 2024-09-11T08:40:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fd_cache_shards": "16", 2024-09-11T08:40:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fd_cache_size": "128", 2024-09-11T08:40:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fiemap": "false", 2024-09-11T08:40:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fiemap_threshold": "4096", 2024-09-11T08:40:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-11T08:40:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-11T08:40:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_inject_stall": "0", 2024-09-11T08:40:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_journal_parallel": "false", 2024-09-11T08:40:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_journal_trailing": "false", 2024-09-11T08:40:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_journal_writeahead": "false", 2024-09-11T08:40:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_kill_at": "0", 2024-09-11T08:40:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-11T08:40:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-11T08:40:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-11T08:40:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-11T08:40:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-11T08:40:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs": "0", 2024-09-11T08:40:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-11T08:40:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-11T08:40:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-11T08:40:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-11T08:40:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size": "0", 2024-09-11T08:40:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-11T08:40:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-11T08:40:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-11T08:40:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_merge_threshold": "-10", 2024-09-11T08:40:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-11T08:40:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_odsync_write": "false", 2024-09-11T08:40:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_omap_backend": "rocksdb", 2024-09-11T08:40:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_omap_backend_path": "", 2024-09-11T08:40:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-11T08:40:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-11T08:40:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-11T08:40:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_op_thread_timeout": "60", 2024-09-11T08:40:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_op_threads": "2", 2024-09-11T08:40:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_punch_hole": "false", 2024-09-11T08:40:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-11T08:40:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-11T08:40:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-11T08:40:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-11T08:40:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-11T08:40:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-11T08:40:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-11T08:40:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-11T08:40:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-11T08:40:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_ops": "50", 2024-09-11T08:40:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-11T08:40:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_seek_data_hole": "false", 2024-09-11T08:40:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_sloppy_crc": "false", 2024-09-11T08:40:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-11T08:40:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_splice": "false", 2024-09-11T08:40:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_split_multiple": "2", 2024-09-11T08:40:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_split_rand_factor": "20", 2024-09-11T08:40:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_update_to": "1000", 2024-09-11T08:40:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-11T08:40:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-11T08:40:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-11T08:40:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-11T08:40:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-11T08:40:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-11T08:40:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_enable": "true", 2024-09-11T08:40:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-11T08:40:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-11T08:40:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-11T08:40:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-11T08:40:46.990 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-11T08:40:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-11T08:40:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_xfs_extsize": "false", 2024-09-11T08:40:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_zfs_snap": "false", 2024-09-11T08:40:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: "fio_dir": "/tmp/fio", 2024-09-11T08:40:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: "fsid": "211625d4-7018-11ef-bcea-c7b262605968", 2024-09-11T08:40:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_allow_other": "true", 2024-09-11T08:40:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_atomic_o_trunc": "true", 2024-09-11T08:40:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_big_writes": "true", 2024-09-11T08:40:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_debug": "false", 2024-09-11T08:40:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_default_permissions": "false", 2024-09-11T08:40:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_disable_pagecache": "false", 2024-09-11T08:40:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_max_write": "0", 2024-09-11T08:40:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_multithreaded": "true", 2024-09-11T08:40:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_require_active_mds": "true", 2024-09-11T08:40:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_set_user_groups": "true", 2024-09-11T08:40:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_splice_move": "true", 2024-09-11T08:40:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_splice_read": "true", 2024-09-11T08:40:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_splice_write": "true", 2024-09-11T08:40:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-11T08:40:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_use_invalidate_cb": "true", 2024-09-11T08:40:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-11T08:40:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "gss_target_name": "ceph", 2024-09-11T08:40:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_file": "", 2024-09-11T08:40:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_inject_failure": "0", 2024-09-11T08:40:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_interval": "5", 2024-09-11T08:40:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "host": "smithi092", 2024-09-11T08:40:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-11T08:40:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-11T08:40:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-11T08:40:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-11T08:40:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-11T08:40:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-11T08:40:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-11T08:40:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-11T08:40:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-11T08:40:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-11T08:40:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-11T08:40:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-11T08:40:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-11T08:40:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "inject_early_sigterm": "false", 2024-09-11T08:40:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "jaeger_tracing_enable": "false", 2024-09-11T08:40:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_aio": "true", 2024-09-11T08:40:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_align_min_size": "65536", 2024-09-11T08:40:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_block_align": "true", 2024-09-11T08:40:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_block_size": "4096", 2024-09-11T08:40:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_dio": "true", 2024-09-11T08:40:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_discard": "false", 2024-09-11T08:40:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_force_aio": "false", 2024-09-11T08:40:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_ignore_corruption": "false", 2024-09-11T08:40:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_max_write_bytes": "10485760", 2024-09-11T08:40:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_max_write_entries": "100", 2024-09-11T08:40:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_replay_from": "0", 2024-09-11T08:40:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-11T08:40:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-11T08:40:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-11T08:40:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-11T08:40:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_write_header_frequency": "0", 2024-09-11T08:40:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_zero_on_create": "false", 2024-09-11T08:40:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "journaler_prefetch_periods": "10", 2024-09-11T08:40:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "journaler_prezero_periods": "5", 2024-09-11T08:40:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "journaler_write_head_interval": "15", 2024-09-11T08:40:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "", 2024-09-11T08:40:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "keyfile": "", 2024-09-11T08:40:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-11T08:40:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_backend": "rocksdb", 2024-09-11T08:40:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_default_stripe_size": "65536", 2024-09-11T08:40:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_fsck_on_mount": "false", 2024-09-11T08:40:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-11T08:40:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_max_bytes": "67108864", 2024-09-11T08:40:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_max_ops": "512", 2024-09-11T08:40:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_nid_prealloc": "1024", 2024-09-11T08:40:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_onode_map_size": "1024", 2024-09-11T08:40:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-11T08:40:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_sync_submit_transaction": "false", 2024-09-11T08:40:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_sync_transaction": "false", 2024-09-11T08:40:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_block_size": "65536", 2024-09-11T08:40:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_bloom_size": "0", 2024-09-11T08:40:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_cache_size": "536870912", 2024-09-11T08:40:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_compact_on_mount": "false", 2024-09-11T08:40:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_compression": "false", 2024-09-11T08:40:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_log": "", 2024-09-11T08:40:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-11T08:40:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_max_open_files": "0", 2024-09-11T08:40:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_paranoid": "false", 2024-09-11T08:40:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-11T08:40:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "librados_thread_count": "2", 2024-09-11T08:40:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "lockdep": "false", 2024-09-11T08:40:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "lockdep_force_backtrace": "false", 2024-09-11T08:40:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "log_coarse_timestamps": "true", 2024-09-11T08:40:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-11T08:40:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "log_flush_on_exit": "false", 2024-09-11T08:40:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "log_graylog_host": "127.0.0.1", 2024-09-11T08:40:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "log_graylog_port": "12201", 2024-09-11T08:40:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "log_max_new": "1000", 2024-09-11T08:40:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "log_max_recent": "10000", 2024-09-11T08:40:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "log_stderr_prefix": "", 2024-09-11T08:40:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "log_stop_at_utilization": "0.970000", 2024-09-11T08:40:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": "true", 2024-09-11T08:40:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_graylog": "false", 2024-09-11T08:40:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_journald": "false", 2024-09-11T08:40:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_stderr": "false", 2024-09-11T08:40:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_syslog": "false", 2024-09-11T08:40:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "max_rotating_auth_attempts": "10", 2024-09-11T08:40:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-11T08:40:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_action_on_write_error": "1", 2024-09-11T08:40:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_alternate_name_max": "8192", 2024-09-11T08:40:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_asio_thread_count": "2", 2024-09-11T08:40:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_export_pin": "true", 2024-09-11T08:40:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_dirs": "true", 2024-09-11T08:40:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-11T08:40:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_interval": "5", 2024-09-11T08:40:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-11T08:40:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-11T08:40:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_interval": "10", 2024-09-11T08:40:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_max": "-1", 2024-09-11T08:40:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_max_until": "-1", 2024-09-11T08:40:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_merge_size": "50", 2024-09-11T08:40:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_midchunk": "0.300000", 2024-09-11T08:40:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-11T08:40:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_min_start": "0.200000", 2024-09-11T08:40:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_minchunk": "0.001000", 2024-09-11T08:40:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_mode": "0", 2024-09-11T08:40:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_need_max": "1.200000", 2024-09-11T08:40:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_need_min": "0.800000", 2024-09-11T08:40:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-11T08:40:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-11T08:40:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_bits": "3", 2024-09-11T08:40:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-11T08:40:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_size": "10000", 2024-09-11T08:40:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-11T08:40:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_target_decay": "10.000000", 2024-09-11T08:40:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-11T08:40:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_beacon_grace": "15.000000", 2024-09-11T08:40:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_beacon_interval": "4.000000", 2024-09-11T08:40:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-11T08:40:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-11T08:40:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_mid": "0.700000", 2024-09-11T08:40:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_release_free_interval": "10", 2024-09-11T08:40:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_reservation": "0.050000", 2024-09-11T08:40:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-11T08:40:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_trim_interval": "1", 2024-09-11T08:40:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_trim_threshold": "262144", 2024-09-11T08:40:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-11T08:40:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-11T08:40:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-11T08:40:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_client_prealloc_inos": "1000", 2024-09-11T08:40:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-11T08:40:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_connect_bootstrapping": "false", 2024-09-11T08:40:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_damage_table_max_entries": "10000", 2024-09-11T08:40:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-11T08:40:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_auth_pins": "false", 2024-09-11T08:40:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_frag": "false", 2024-09-11T08:40:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_scatterstat": "false", 2024-09-11T08:40:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_subtrees": "false", 2024-09-11T08:40:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_decay_halflife": "5.000000", 2024-09-11T08:40:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_default_dir_hash": "2", 2024-09-11T08:40:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_defer_session_stale": "true", 2024-09-11T08:40:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_deny_all_reconnect": "false", 2024-09-11T08:40:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dir_keys_per_op": "16384", 2024-09-11T08:40:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dir_max_commit_size": "10", 2024-09-11T08:40:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dir_max_entries": "0", 2024-09-11T08:40:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-11T08:40:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-11T08:40:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_on_map": "false", 2024-09-11T08:40:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-11T08:40:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-11T08:40:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_early_reply": "true", 2024-09-11T08:40:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_enable_op_tracker": "true", 2024-09-11T08:40:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_enforce_unique_name": "true", 2024-09-11T08:40:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-11T08:40:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-11T08:40:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_random": "true", 2024-09-11T08:40:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-11T08:40:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-11T08:40:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-11T08:40:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-11T08:40:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-11T08:40:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-11T08:40:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_health_summarize_threshold": "10", 2024-09-11T08:40:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-11T08:40:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-11T08:40:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_health_dummy": "false", 2024-09-11T08:40:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-11T08:40:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_migrator_session_race": "false", 2024-09-11T08:40:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-11T08:40:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-11T08:40:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-11T08:40:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_join_fs": "", 2024-09-11T08:40:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_journal_format": "1", 2024-09-11T08:40:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_create_at": "0", 2024-09-11T08:40:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_export_at": "0", 2024-09-11T08:40:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_import_at": "0", 2024-09-11T08:40:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_journal_at": "0", 2024-09-11T08:40:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_journal_expire_at": "0", 2024-09-11T08:40:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_journal_replay_at": "0", 2024-09-11T08:40:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_link_at": "0", 2024-09-11T08:40:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_mdstable_at": "0", 2024-09-11T08:40:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_openc_at": "0", 2024-09-11T08:40:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_rename_at": "0", 2024-09-11T08:40:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-09-11T08:40:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_events_per_segment": "1024", 2024-09-11T08:40:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_max_events": "-1", 2024-09-11T08:40:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_max_segments": "128", 2024-09-11T08:40:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_pause": "false", 2024-09-11T08:40:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_segment_size": "0", 2024-09-11T08:40:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-11T08:40:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_warn_factor": "2.000000", 2024-09-11T08:40:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_caps_per_client": "1048576", 2024-09-11T08:40:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_completed_flushes": "100000", 2024-09-11T08:40:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_completed_requests": "100000", 2024-09-11T08:40:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_export_size": "20971520", 2024-09-11T08:40:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_file_recover": "32", 2024-09-11T08:40:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_purge_files": "64", 2024-09-11T08:40:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_purge_ops": "8192", 2024-09-11T08:40:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-11T08:40:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-11T08:40:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_snaps_per_dir": "100", 2024-09-11T08:40:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_xattr_pairs_size": "65536", 2024-09-11T08:40:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_metrics_update_interval": "2", 2024-09-11T08:40:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_min_caps_per_client": "100", 2024-09-11T08:40:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_min_caps_working_set": "10000", 2024-09-11T08:40:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-11T08:40:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_numa_node": "-1", 2024-09-11T08:40:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-11T08:40:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_complaint_time": "30.000000", 2024-09-11T08:40:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_history_duration": "600", 2024-09-11T08:40:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_history_size": "20", 2024-09-11T08:40:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_history_slow_op_size": "20", 2024-09-11T08:40:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-11T08:40:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_log_threshold": "5", 2024-09-11T08:40:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_ping_grace": "15", 2024-09-11T08:40:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_ping_interval": "5", 2024-09-11T08:40:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-11T08:40:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-11T08:40:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_max_caps": "30000", 2024-09-11T08:40:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-11T08:40:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-11T08:40:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-11T08:40:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_warning_threshold": "262144", 2024-09-11T08:40:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-11T08:40:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_replay_interval": "1.000000", 2024-09-11T08:40:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-11T08:40:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-11T08:40:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_root_ino_gid": "0", 2024-09-11T08:40:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_root_ino_uid": "0", 2024-09-11T08:40:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-11T08:40:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-11T08:40:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-11T08:40:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-11T08:40:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-11T08:40:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-11T08:40:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-11T08:40:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-11T08:40:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-11T08:40:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-11T08:40:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_shutdown_check": "0", 2024-09-11T08:40:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_skip_ino": "0", 2024-09-11T08:40:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-11T08:40:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_snap_max_uid": "4294967294", 2024-09-11T08:40:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_snap_min_uid": "0", 2024-09-11T08:40:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_snap_rstat": "false", 2024-09-11T08:40:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_standby_replay_damaged": "false", 2024-09-11T08:40:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_symlink_recovery": "true", 2024-09-11T08:40:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-11T08:40:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_thrash_exports": "0", 2024-09-11T08:40:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_thrash_fragments": "0", 2024-09-11T08:40:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_tick_interval": "5.000000", 2024-09-11T08:40:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_valgrind_exit": "false", 2024-09-11T08:40:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_verify_backtrace": "1", 2024-09-11T08:40:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_verify_scatter": "false", 2024-09-11T08:40:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-11T08:40:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_wipe_sessions": "false", 2024-09-11T08:40:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool_debug": "false", 2024-09-11T08:40:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-11T08:40:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_device_bytes": "1073741824", 2024-09-11T08:40:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_page_set": "false", 2024-09-11T08:40:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_page_size": "65536", 2024-09-11T08:40:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_client_bytes": "134217728", 2024-09-11T08:40:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_client_messages": "512", 2024-09-11T08:40:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-11T08:40:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-11T08:40:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-11T08:40:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-11T08:40:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_disabled_modules": "diskprediction_local", 2024-09-11T08:40:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-11T08:40:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_max_pg_num_change": "128", 2024-09-11T08:40:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mds_bytes": "134217728", 2024-09-11T08:40:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mds_messages": "128", 2024-09-11T08:40:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-11T08:40:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mon_bytes": "134217728", 2024-09-11T08:40:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mon_messages": "128", 2024-09-11T08:40:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_osd_bytes": "536870912", 2024-09-11T08:40:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_osd_messages": "8192", 2024-09-11T08:40:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_pool": "true", 2024-09-11T08:40:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-11T08:40:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_standby_modules": "true", 2024-09-11T08:40:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_stats_period": "5", 2024-09-11T08:40:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_stats_threshold": "5", 2024-09-11T08:40:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_tick_period": "2", 2024-09-11T08:40:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-11T08:40:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-11T08:40:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_allow_pool_delete": "true", 2024-09-11T08:40:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_allow_pool_size_one": "false", 2024-09-11T08:40:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_auth_validate_all_caps": "true", 2024-09-11T08:40:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-11T08:40:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-11T08:40:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_bytes": "104857600", 2024-09-11T08:40:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_directed_command_retry": "2", 2024-09-11T08:40:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-11T08:40:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-11T08:40:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-11T08:40:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-11T08:40:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_parallel": "3", 2024-09-11T08:40:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_log_interval": "1.000000", 2024-09-11T08:40:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-11T08:40:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_ping_interval": "10.000000", 2024-09-11T08:40:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-11T08:40:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-11T08:40:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-11T08:40:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-11T08:40:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_level": "debug", 2024-09-11T08:40:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_file": "false", 2024-09-11T08:40:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-11T08:40:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-11T08:40:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-11T08:40:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_journald": "true", 2024-09-11T08:40:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-11T08:40:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-11T08:40:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-11T08:40:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_compact_on_bootstrap": "false", 2024-09-11T08:40:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_compact_on_start": "false", 2024-09-11T08:40:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_compact_on_trim": "true", 2024-09-11T08:40:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-11T08:40:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-11T08:40:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-11T08:40:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cpu_threads": "4", 2024-09-11T08:40:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_crush_min_required_version": "hammer", 2024-09-11T08:40:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_daemon_bytes": "419430400", 2024-09-11T08:40:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-11T08:40:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data_avail_crit": "5", 2024-09-11T08:40:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data_avail_warn": "5", 2024-09-11T08:40:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data_size_warn": "16106127360", 2024-09-11T08:40:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-11T08:40:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-11T08:40:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_dump_json": "false", 2024-09-11T08:40:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-11T08:40:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_dump_transactions": "false", 2024-09-11T08:40:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_extra_checks": "false", 2024-09-11T08:40:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-11T08:40:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-11T08:40:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_require_pacific": "false", 2024-09-11T08:40:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_require_quincy": "false", 2024-09-11T08:40:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-11T08:40:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-11T08:40:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-11T08:40:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_down_mkfs_grace": "60", 2024-09-11T08:40:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_election_default_strategy": "1", 2024-09-11T08:40:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_election_timeout": "5.000000", 2024-09-11T08:40:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-11T08:40:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_elector_ping_divisor": "2", 2024-09-11T08:40:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-11T08:40:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_enable_op_tracker": "true", 2024-09-11T08:40:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_fake_pool_delete": "false", 2024-09-11T08:40:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_force_quorum_join": "false", 2024-09-11T08:40:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_globalid_prealloc": "10000", 2024-09-11T08:40:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_detail_to_clog": "true", 2024-09-11T08:40:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_log_update_period": "5", 2024-09-11T08:40:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_max_detail": "50", 2024-09-11T08:40:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_to_clog": "true", 2024-09-11T08:40:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_to_clog_interval": "600", 2024-09-11T08:40:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-11T08:40:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_host": "[v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0]", 2024-09-11T08:40:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_host_override": "", 2024-09-11T08:40:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_initial_members": "", 2024-09-11T08:40:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-11T08:40:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-11T08:40:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-11T08:40:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-11T08:40:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-11T08:40:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_lease": "5.000000", 2024-09-11T08:40:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-11T08:40:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-11T08:40:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_log_full_interval": "50", 2024-09-11T08:40:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_log_max": "10000", 2024-09-11T08:40:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_log_max_summary": "50", 2024-09-11T08:40:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-11T08:40:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_log_epochs": "500", 2024-09-11T08:40:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-11T08:40:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-11T08:40:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_osd": "10000", 2024-09-11T08:40:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_pg_per_osd": "10000", 2024-09-11T08:40:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_pool_pg_num": "65536", 2024-09-11T08:40:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-11T08:40:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-11T08:40:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mds_force_trim_to": "0", 2024-09-11T08:40:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mds_skip_sanity": "false", 2024-09-11T08:40:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_memory_autotune": "true", 2024-09-11T08:40:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_memory_target": "2147483648", 2024-09-11T08:40:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_beacon_grace": "30", 2024-09-11T08:40:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-11T08:40:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_digest_period": "5", 2024-09-11T08:40:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_inactive_grace": "60", 2024-09-11T08:40:47.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-11T08:40:47.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-11T08:40:47.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_min_osdmap_epochs": "50", 2024-09-11T08:40:47.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_complaint_time": "30", 2024-09-11T08:40:47.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_duration": "600", 2024-09-11T08:40:47.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_size": "20", 2024-09-11T08:40:47.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_slow_op_size": "20", 2024-09-11T08:40:47.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-11T08:40:47.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_log_threshold": "5", 2024-09-11T08:40:47.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-11T08:40:47.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-11T08:40:47.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-11T08:40:47.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_auto_mark_in": "false", 2024-09-11T08:40:47.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-11T08:40:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-11T08:40:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-11T08:40:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_cache_size": "500", 2024-09-11T08:40:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-11T08:40:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-11T08:40:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-11T08:40:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_down_out_interval": "600", 2024-09-11T08:40:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-11T08:40:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-11T08:40:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_force_trim_to": "0", 2024-09-11T08:40:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-11T08:40:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-11T08:40:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-11T08:40:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-11T08:40:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-11T08:40:47.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-11T08:40:47.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-11T08:40:47.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-11T08:40:47.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_min_down_reporters": "2", 2024-09-11T08:40:47.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-11T08:40:47.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-11T08:40:47.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-11T08:40:47.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-11T08:40:47.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-11T08:40:47.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-11T08:40:47.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_report_timeout": "900", 2024-09-11T08:40:47.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-11T08:40:47.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-11T08:40:47.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-11T08:40:47.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-11T08:40:47.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-11T08:40:47.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_interval": "2", 2024-09-11T08:40:47.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_min": "15", 2024-09-11T08:40:47.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-09-11T08:40:47.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-11T08:40:47.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_stuck_threshold": "60", 2024-09-11T08:40:47.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-11T08:40:47.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-11T08:40:47.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-11T08:40:47.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-11T08:40:47.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-11T08:40:47.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-11T08:40:47.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_probe_timeout": "2.000000", 2024-09-11T08:40:47.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_max_change": "0.050000", 2024-09-11T08:40:47.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_max_osds": "4", 2024-09-11T08:40:47.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-11T08:40:47.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-11T08:40:47.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-11T08:40:47.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-11T08:40:47.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-11T08:40:47.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_interval": "86400", 2024-09-11T08:40:47.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_max_keys": "100", 2024-09-11T08:40:47.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_timeout": "300", 2024-09-11T08:40:47.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_session_timeout": "300", 2024-09-11T08:40:47.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_smart_report_timeout": "5", 2024-09-11T08:40:47.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stat_smooth_intervals": "6", 2024-09-11T08:40:47.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-11T08:40:47.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_pool_min_size": "2", 2024-09-11T08:40:47.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_pool_size": "4", 2024-09-11T08:40:47.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-11T08:40:47.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-11T08:40:47.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_debug": "false", 2024-09-11T08:40:47.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-11T08:40:47.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-11T08:40:47.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_provider_kill_at": "0", 2024-09-11T08:40:47.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_requester_kill_at": "0", 2024-09-11T08:40:47.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_timeout": "60.000000", 2024-09-11T08:40:47.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_target_pg_per_osd": "100", 2024-09-11T08:40:47.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_tick_interval": "5", 2024-09-11T08:40:47.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_timecheck_interval": "300.000000", 2024-09-11T08:40:47.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-11T08:40:47.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_older_version_delay": "604800", 2024-09-11T08:40:47.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-11T08:40:47.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-11T08:40:47.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-11T08:40:47.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_filestore_osds": "true", 2024-09-11T08:40:47.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-11T08:40:47.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-11T08:40:47.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-11T08:40:47.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_misplaced": "false", 2024-09-11T08:40:47.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-11T08:40:47.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_older_version": "true", 2024-09-11T08:40:47.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-11T08:40:47.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-11T08:40:47.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_pool_no_redundancy": "true", 2024-09-11T08:40:47.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-11T08:40:47.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-11T08:40:47.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-11T08:40:47.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-11T08:40:47.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-11T08:40:47.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-11T08:40:47.029 INFO:teuthology.orchestra.run.smithi092.stdout: "monmap": "", 2024-09-11T08:40:47.029 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_op_threads": "3", 2024-09-11T08:40:47.029 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-11T08:40:47.029 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_cm": "false", 2024-09-11T08:40:47.029 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_device_name": "", 2024-09-11T08:40:47.029 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_dscp": "96", 2024-09-11T08:40:47.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-11T08:40:47.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-11T08:40:47.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_local_gid": "", 2024-09-11T08:40:47.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-11T08:40:47.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_port_num": "1", 2024-09-11T08:40:47.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-11T08:40:47.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-11T08:40:47.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-11T08:40:47.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-11T08:40:47.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_sl": "3", 2024-09-11T08:40:47.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_support_srq": "true", 2024-09-11T08:40:47.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_type": "ib", 2024-09-11T08:40:47.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_reap_threshold": "5", 2024-09-11T08:40:47.031 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_before_connect": "false", 2024-09-11T08:40:47.031 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_ipv4": "true", 2024-09-11T08:40:47.031 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_ipv6": "false", 2024-09-11T08:40:47.031 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_msgr1": "true", 2024-09-11T08:40:47.031 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_msgr2": "true", 2024-09-11T08:40:47.031 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_port_max": "7568", 2024-09-11T08:40:47.031 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_port_min": "6800", 2024-09-11T08:40:47.031 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-11T08:40:47.031 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_retry_count": "3", 2024-09-11T08:40:47.031 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_retry_delay": "5", 2024-09-11T08:40:47.031 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_client": "false", 2024-09-11T08:40:47.031 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_mds": "false", 2024-09-11T08:40:47.031 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_mgr": "false", 2024-09-11T08:40:47.031 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_mon": "false", 2024-09-11T08:40:47.032 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_osd": "false", 2024-09-11T08:40:47.032 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_client_mode": "crc secure", 2024-09-11T08:40:47.032 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_cluster_mode": "crc secure", 2024-09-11T08:40:47.032 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_cluster_type": "", 2024-09-11T08:40:47.032 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_compress_secure": "false", 2024-09-11T08:40:47.032 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_connection_idle_timeout": "900", 2024-09-11T08:40:47.032 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_connection_ready_timeout": "10", 2024-09-11T08:40:47.032 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_crc_data": "true", 2024-09-11T08:40:47.032 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_crc_header": "true", 2024-09-11T08:40:47.032 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_bad_msg": "false", 2024-09-11T08:40:47.032 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_bug": "true", 2024-09-11T08:40:47.032 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_old_message": "true", 2024-09-11T08:40:47.032 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_skipped_message": "false", 2024-09-11T08:40:47.032 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-11T08:40:47.033 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-11T08:40:47.033 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_coremask": "0xF", 2024-09-11T08:40:47.033 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-11T08:40:47.033 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-11T08:40:47.033 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_enable_tso": "true", 2024-09-11T08:40:47.033 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-11T08:40:47.033 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-11T08:40:47.033 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_hugepages": "", 2024-09-11T08:40:47.033 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-11T08:40:47.033 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-11T08:40:47.033 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_lro": "true", 2024-09-11T08:40:47.033 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_memory_channel": "4", 2024-09-11T08:40:47.033 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-11T08:40:47.033 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_pmd": "", 2024-09-11T08:40:47.034 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_port_id": "0", 2024-09-11T08:40:47.034 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-11T08:40:47.034 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-11T08:40:47.034 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dump_on_send": "false", 2024-09-11T08:40:47.034 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_initial_backoff": "0.200000", 2024-09-11T08:40:47.034 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_delay_max": "1.000000", 2024-09-11T08:40:47.034 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-11T08:40:47.034 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_delay_type": "", 2024-09-11T08:40:47.034 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-11T08:40:47.034 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_socket_failures": "0", 2024-09-11T08:40:47.034 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_learn_addr_from_peer": "true", 2024-09-11T08:40:47.034 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_max_accept_failures": "4", 2024-09-11T08:40:47.034 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_max_backoff": "15.000000", 2024-09-11T08:40:47.034 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_mon_client_mode": "secure crc", 2024-09-11T08:40:47.035 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-11T08:40:47.035 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_mon_service_mode": "secure crc", 2024-09-11T08:40:47.035 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_osd_compress_min_size": "1024", 2024-09-11T08:40:47.035 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_osd_compress_mode": "none", 2024-09-11T08:40:47.035 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2024-09-11T08:40:47.035 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-11T08:40:47.035 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_pq_min_cost": "65536", 2024-09-11T08:40:47.035 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_public_type": "", 2024-09-11T08:40:47.035 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_service_mode": "crc secure", 2024-09-11T08:40:47.035 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_listen_backlog": "512", 2024-09-11T08:40:47.035 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_nodelay": "true", 2024-09-11T08:40:47.035 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-11T08:40:47.035 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_rcvbuf": "0", 2024-09-11T08:40:47.035 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_type": "async+posix", 2024-09-11T08:40:47.036 INFO:teuthology.orchestra.run.smithi092.stdout: "no_config_file": "false", 2024-09-11T08:40:47.036 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_completion_locks_per_session": "32", 2024-09-11T08:40:47.036 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-11T08:40:47.036 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-11T08:40:47.036 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_inflight_ops": "1024", 2024-09-11T08:40:47.036 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-11T08:40:47.036 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-11T08:40:47.036 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_tick_interval": "5.000000", 2024-09-11T08:40:47.036 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_timeout": "10.000000", 2024-09-11T08:40:47.036 INFO:teuthology.orchestra.run.smithi092.stdout: "objectstore_blackhole": "false", 2024-09-11T08:40:47.036 INFO:teuthology.orchestra.run.smithi092.stdout: "openssl_engine_opts": "", 2024-09-11T08:40:47.036 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_delay_time": "5.000000", 2024-09-11T08:40:47.036 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_hist_halflife": "1000", 2024-09-11T08:40:47.037 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_max_low_ops": "2", 2024-09-11T08:40:47.037 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_max_ops": "4", 2024-09-11T08:40:47.037 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-11T08:40:47.037 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-11T08:40:47.037 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_slop": "0.020000", 2024-09-11T08:40:47.037 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-11T08:40:47.037 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-11T08:40:47.037 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_asio_thread_count": "2", 2024-09-11T08:40:47.037 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_async_recovery_min_cost": "100", 2024-09-11T08:40:47.037 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-11T08:40:47.037 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-11T08:40:47.037 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backfill_scan_max": "512", 2024-09-11T08:40:47.037 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backfill_scan_min": "64", 2024-09-11T08:40:47.037 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backoff_on_degraded": "false", 2024-09-11T08:40:47.038 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backoff_on_peering": "false", 2024-09-11T08:40:47.038 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backoff_on_unfound": "true", 2024-09-11T08:40:47.038 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_beacon_report_interval": "300", 2024-09-11T08:40:47.038 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_duration": "30", 2024-09-11T08:40:47.038 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-11T08:40:47.038 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_max_block_size": "67108864", 2024-09-11T08:40:47.038 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-11T08:40:47.038 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_blkin_trace_all": "false", 2024-09-11T08:40:47.038 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-11T08:40:47.038 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-11T08:40:47.038 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_check_for_log_corruption": "false", 2024-09-11T08:40:47.038 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-11T08:40:47.038 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-11T08:40:47.038 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-11T08:40:47.039 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-11T08:40:47.039 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_update_on_start": "true", 2024-09-11T08:40:47.039 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_message_cap": "256", 2024-09-11T08:40:47.039 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_message_size_cap": "524288000", 2024-09-11T08:40:47.039 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_op_priority": "63", 2024-09-11T08:40:47.039 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_watch_timeout": "30", 2024-09-11T08:40:47.039 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_command_max_records": "256", 2024-09-11T08:40:47.039 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_compact_on_start": "false", 2024-09-11T08:40:47.039 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-11T08:40:47.039 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_chooseleaf_type": "1", 2024-09-11T08:40:47.039 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-11T08:40:47.039 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_update_on_start": "true", 2024-09-11T08:40:47.039 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_update_weight_set": "true", 2024-09-11T08:40:47.040 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-11T08:40:47.040 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-11T08:40:47.040 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-11T08:40:47.040 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-11T08:40:47.040 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-11T08:40:47.040 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_feed_pullee": "-1", 2024-09-11T08:40:47.040 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-11T08:40:47.040 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-11T08:40:47.040 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-11T08:40:47.040 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_misdirected_ops": "false", 2024-09-11T08:40:47.040 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_no_acting_change": "false", 2024-09-11T08:40:47.040 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_no_purge_strays": "false", 2024-09-11T08:40:47.040 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_op_order": "false", 2024-09-11T08:40:47.040 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-11T08:40:47.041 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-11T08:40:47.041 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-11T08:40:47.041 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-11T08:40:47.041 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_shutdown": "false", 2024-09-11T08:40:47.041 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-11T08:40:47.041 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-11T08:40:47.041 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-11T08:40:47.041 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-11T08:40:47.041 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_snaps": "false", 2024-09-11T08:40:47.041 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-11T08:40:47.041 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-11T08:40:47.041 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_keys": "1024", 2024-09-11T08:40:47.041 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-11T08:40:47.041 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-11T08:40:47.042 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-11T08:40:47.042 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_stride": "524288", 2024-09-11T08:40:47.042 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-11T08:40:47.042 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_default_data_pool_replay_window": "45", 2024-09-11T08:40:47.042 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_default_notify_timeout": "30", 2024-09-11T08:40:47.042 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep": "0.000000", 2024-09-11T08:40:47.042 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-11T08:40:47.042 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-11T08:40:47.042 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-11T08:40:47.042 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_discard_disconnected_ops": "true", 2024-09-11T08:40:47.042 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_enable_op_tracker": "true", 2024-09-11T08:40:47.042 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-11T08:40:47.043 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-11T08:40:47.043 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-11T08:40:47.043 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_info": "true", 2024-09-11T08:40:47.043 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_shutdown": "true", 2024-09-11T08:40:47.043 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-11T08:40:47.043 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-11T08:40:47.043 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-11T08:40:47.043 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-11T08:40:47.043 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-11T08:40:47.043 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_function_tracing": "false", 2024-09-11T08:40:47.043 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_grace": "20", 2024-09-11T08:40:47.043 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_interval": "6", 2024-09-11T08:40:47.043 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-11T08:40:47.044 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_min_peers": "10", 2024-09-11T08:40:47.044 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_min_size": "2000", 2024-09-11T08:40:47.044 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_stale": "600", 2024-09-11T08:40:47.044 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-11T08:40:47.044 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_hit_set_max_size": "100000", 2024-09-11T08:40:47.044 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_hit_set_min_size": "1000", 2024-09-11T08:40:47.044 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-11T08:40:47.044 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-11T08:40:47.044 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-11T08:40:47.044 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-11T08:40:47.044 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-11T08:40:47.044 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-11T08:40:47.044 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_journal_size": "5120", 2024-09-11T08:40:47.044 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_kill_backfill_at": "0", 2024-09-11T08:40:47.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-11T08:40:47.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_cache_size": "50", 2024-09-11T08:40:47.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_dedup": "true", 2024-09-11T08:40:47.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_message_max": "40", 2024-09-11T08:40:47.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-11T08:40:47.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_share_max_epochs": "40", 2024-09-11T08:40:47.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_attr_name_len": "100", 2024-09-11T08:40:47.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_attr_size": "0", 2024-09-11T08:40:47.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_backfills": "1", 2024-09-11T08:40:47.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_markdown_count": "5", 2024-09-11T08:40:47.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_markdown_period": "600", 2024-09-11T08:40:47.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_object_name_len": "2048", 2024-09-11T08:40:47.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_object_namespace_len": "256", 2024-09-11T08:40:47.045 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_object_size": "134217728", 2024-09-11T08:40:47.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-11T08:40:47.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-11T08:40:47.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pg_blocked_by": "16", 2024-09-11T08:40:47.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pg_log_entries": "10000", 2024-09-11T08:40:47.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-11T08:40:47.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pgls": "1024", 2024-09-11T08:40:47.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_push_cost": "8388608", 2024-09-11T08:40:47.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_push_objects": "10", 2024-09-11T08:40:47.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_scrubs": "3", 2024-09-11T08:40:47.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-11T08:40:47.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_trimming_pgs": "2", 2024-09-11T08:40:47.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_write_op_reply_len": "64", 2024-09-11T08:40:47.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_write_size": "90", 2024-09-11T08:40:47.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-11T08:40:47.047 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-11T08:40:47.047 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-11T08:40:47.047 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-11T08:40:47.047 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-11T08:40:47.047 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-11T08:40:47.047 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-11T08:40:47.047 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-11T08:40:47.047 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_profile": "balanced", 2024-09-11T08:40:47.047 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-11T08:40:47.047 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-11T08:40:47.047 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-11T08:40:47.047 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-11T08:40:47.047 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-11T08:40:47.048 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-11T08:40:47.048 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-11T08:40:47.048 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-11T08:40:47.048 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-11T08:40:47.048 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-11T08:40:47.048 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-11T08:40:47.048 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_base": "805306368", 2024-09-11T08:40:47.048 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_cache_min": "134217728", 2024-09-11T08:40:47.048 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-11T08:40:47.048 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-11T08:40:47.048 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_target": "4294967296", 2024-09-11T08:40:47.048 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_target_autotune": "false", 2024-09-11T08:40:47.048 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-11T08:40:47.049 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_min_pg_log_entries": "250", 2024-09-11T08:40:47.049 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_min_recovery_priority": "0", 2024-09-11T08:40:47.049 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-11T08:40:47.049 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-11T08:40:47.049 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_report_interval": "5", 2024-09-11T08:40:47.049 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-11T08:40:47.049 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-11T08:40:47.049 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_num_cache_shards": "32", 2024-09-11T08:40:47.049 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_num_op_tracker_shard": "32", 2024-09-11T08:40:47.049 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_numa_auto_affinity": "true", 2024-09-11T08:40:47.049 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_numa_node": "-1", 2024-09-11T08:40:47.049 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_numa_prefer_iface": "true", 2024-09-11T08:40:47.049 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-11T08:40:47.049 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objecter_finishers": "1", 2024-09-11T08:40:47.050 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objectstore": "bluestore", 2024-09-11T08:40:47.050 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objectstore_fuse": "false", 2024-09-11T08:40:47.050 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objectstore_tracing": "false", 2024-09-11T08:40:47.050 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_complaint_time": "30.000000", 2024-09-11T08:40:47.050 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_duration": "600", 2024-09-11T08:40:47.050 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_size": "20", 2024-09-11T08:40:47.050 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_slow_op_size": "20", 2024-09-11T08:40:47.050 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-11T08:40:47.050 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_log_threshold": "5", 2024-09-11T08:40:47.050 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_shards": "0", 2024-09-11T08:40:47.050 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_shards_hdd": "5", 2024-09-11T08:40:47.050 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_shards_ssd": "8", 2024-09-11T08:40:47.050 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-11T08:40:47.050 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-11T08:40:47.051 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-11T08:40:47.051 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-11T08:40:47.051 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_pq_min_cost": "65536", 2024-09-11T08:40:47.051 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-11T08:40:47.051 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_queue_cut_off": "high", 2024-09-11T08:40:47.051 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-11T08:40:47.051 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_thread_timeout": "15", 2024-09-11T08:40:47.051 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_open_classes_on_start": "true", 2024-09-11T08:40:47.051 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_os_flags": "0", 2024-09-11T08:40:47.051 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_peering_op_priority": "255", 2024-09-11T08:40:47.051 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_delete_cost": "1048576", 2024-09-11T08:40:47.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_delete_priority": "5", 2024-09-11T08:40:47.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-11T08:40:47.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-11T08:40:47.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-11T08:40:47.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_log_trim_max": "10000", 2024-09-11T08:40:47.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_log_trim_min": "100", 2024-09-11T08:40:47.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-11T08:40:47.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-11T08:40:47.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-11T08:40:47.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-11T08:40:47.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-11T08:40:47.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-11T08:40:47.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-11T08:40:47.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-11T08:40:47.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-11T08:40:47.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-11T08:40:47.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-11T08:40:47.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-11T08:40:47.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-11T08:40:47.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-11T08:40:47.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-11T08:40:47.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-11T08:40:47.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-11T08:40:47.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flags": "0", 2024-09-11T08:40:47.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-11T08:40:47.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_min_size": "0", 2024-09-11T08:40:47.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-11T08:40:47.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_pg_num": "32", 2024-09-11T08:40:47.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_pgp_num": "0", 2024-09-11T08:40:47.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-11T08:40:47.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_size": "3", 2024-09-11T08:40:47.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_type": "replicated", 2024-09-11T08:40:47.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-11T08:40:47.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-11T08:40:47.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_push_per_object_cost": "1000", 2024-09-11T08:40:47.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-11T08:40:47.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recover_clone_overlap": "true", 2024-09-11T08:40:47.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-11T08:40:47.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_cost": "20971520", 2024-09-11T08:40:47.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-11T08:40:47.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_active": "0", 2024-09-11T08:40:47.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-11T08:40:47.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-11T08:40:47.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-11T08:40:47.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-11T08:40:47.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_single_start": "1", 2024-09-11T08:40:47.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_op_priority": "3", 2024-09-11T08:40:47.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-11T08:40:47.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_priority": "5", 2024-09-11T08:40:47.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-11T08:40:47.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep": "0.000000", 2024-09-11T08:40:47.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-11T08:40:47.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-11T08:40:47.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-11T08:40:47.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_repair_during_recovery": "false", 2024-09-11T08:40:47.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_requested_scrub_priority": "120", 2024-09-11T08:40:47.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-11T08:40:47.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-11T08:40:47.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_auto_repair": "false", 2024-09-11T08:40:47.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-11T08:40:47.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-11T08:40:47.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_begin_hour": "0", 2024-09-11T08:40:47.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_begin_week_day": "0", 2024-09-11T08:40:47.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_chunk_max": "25", 2024-09-11T08:40:47.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_chunk_min": "5", 2024-09-11T08:40:47.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_cost": "52428800", 2024-09-11T08:40:47.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_during_recovery": "false", 2024-09-11T08:40:47.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_end_hour": "0", 2024-09-11T08:40:47.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_end_week_day": "0", 2024-09-11T08:40:47.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_event_cost": "4096", 2024-09-11T08:40:47.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-11T08:40:47.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-11T08:40:47.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_invalid_stats": "true", 2024-09-11T08:40:47.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-11T08:40:47.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-11T08:40:47.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_max_preemptions": "5", 2024-09-11T08:40:47.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-11T08:40:47.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_priority": "5", 2024-09-11T08:40:47.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_sleep": "0.000000", 2024-09-11T08:40:47.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-11T08:40:47.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_skip_data_digest": "false", 2024-09-11T08:40:47.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_smart_report_timeout": "5", 2024-09-11T08:40:47.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_cost": "1048576", 2024-09-11T08:40:47.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_priority": "5", 2024-09-11T08:40:47.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-11T08:40:47.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-11T08:40:47.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-11T08:40:47.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-11T08:40:47.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-11T08:40:47.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_target_transaction_size": "30", 2024-09-11T08:40:47.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-11T08:40:47.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-11T08:40:47.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-11T08:40:47.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-11T08:40:47.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-11T08:40:47.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-11T08:40:47.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-11T08:40:47.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-11T08:40:47.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-11T08:40:47.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-11T08:40:47.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tracing": "false", 2024-09-11T08:40:47.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_use_stale_snap": "false", 2024-09-11T08:40:47.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-11T08:40:47.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osdc_blkin_trace_all": "false", 2024-09-11T08:40:47.060 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_kill_at": "0", 2024-09-11T08:40:47.060 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_max_join_drift": "10", 2024-09-11T08:40:47.060 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_min": "500", 2024-09-11T08:40:47.060 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_min_wait": "0.050000", 2024-09-11T08:40:47.061 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_propose_interval": "1.000000", 2024-09-11T08:40:47.061 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_service_trim_max": "500", 2024-09-11T08:40:47.061 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-11T08:40:47.061 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_service_trim_min": "10", 2024-09-11T08:40:47.061 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_stash_full_interval": "25", 2024-09-11T08:40:47.061 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_trim_max": "500", 2024-09-11T08:40:47.061 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_trim_min": "250", 2024-09-11T08:40:47.061 INFO:teuthology.orchestra.run.smithi092.stdout: "perf": "true", 2024-09-11T08:40:47.061 INFO:teuthology.orchestra.run.smithi092.stdout: "pid_file": "", 2024-09-11T08:40:47.061 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-11T08:40:47.061 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-11T08:40:47.061 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "-", 2024-09-11T08:40:47.061 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrv": "", 2024-09-11T08:40:47.061 INFO:teuthology.orchestra.run.smithi092.stdout: "public_bind_addr": "-", 2024-09-11T08:40:47.062 INFO:teuthology.orchestra.run.smithi092.stdout: "public_network": "", 2024-09-11T08:40:47.062 INFO:teuthology.orchestra.run.smithi092.stdout: "public_network_interface": "", 2024-09-11T08:40:47.062 INFO:teuthology.orchestra.run.smithi092.stdout: "qat_compressor_enabled": "false", 2024-09-11T08:40:47.062 INFO:teuthology.orchestra.run.smithi092.stdout: "rados_mon_op_timeout": "0", 2024-09-11T08:40:47.062 INFO:teuthology.orchestra.run.smithi092.stdout: "rados_osd_op_timeout": "0", 2024-09-11T08:40:47.062 INFO:teuthology.orchestra.run.smithi092.stdout: "rados_tracing": "false", 2024-09-11T08:40:47.062 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_atime_update_interval": "60", 2024-09-11T08:40:47.062 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-11T08:40:47.062 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_balance_parent_reads": "false", 2024-09-11T08:40:47.062 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_balance_snap_reads": "false", 2024-09-11T08:40:47.062 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_blkin_trace_all": "false", 2024-09-11T08:40:47.062 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-11T08:40:47.062 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-11T08:40:47.062 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache": "true", 2024-09-11T08:40:47.063 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-11T08:40:47.063 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-11T08:40:47.063 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-11T08:40:47.063 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-11T08:40:47.063 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_policy": "writearound", 2024-09-11T08:40:47.063 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_size": "33554432", 2024-09-11T08:40:47.063 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-11T08:40:47.063 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-11T08:40:47.063 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_clone_copy_on_read": "false", 2024-09-11T08:40:47.063 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_compression_hint": "none", 2024-09-11T08:40:47.063 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_concurrent_management_ops": "10", 2024-09-11T08:40:47.063 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-11T08:40:47.063 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_clone_format": "auto", 2024-09-11T08:40:47.063 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_data_pool": "", 2024-09-11T08:40:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_features": "61", 2024-09-11T08:40:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_format": "2", 2024-09-11T08:40:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_map_options": "", 2024-09-11T08:40:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_order": "22", 2024-09-11T08:40:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_pool": "rbd", 2024-09-11T08:40:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-11T08:40:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_stripe_count": "0", 2024-09-11T08:40:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_stripe_unit": "0", 2024-09-11T08:40:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-11T08:40:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-11T08:40:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-11T08:40:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_enable_alloc_hint": "true", 2024-09-11T08:40:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-11T08:40:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_io_scheduler": "simple", 2024-09-11T08:40:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-11T08:40:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-11T08:40:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-11T08:40:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-11T08:40:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-11T08:40:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-11T08:40:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-11T08:40:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-11T08:40:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-11T08:40:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_order": "24", 2024-09-11T08:40:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_pool": "", 2024-09-11T08:40:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_splay_width": "4", 2024-09-11T08:40:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_localize_parent_reads": "false", 2024-09-11T08:40:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_localize_snap_reads": "false", 2024-09-11T08:40:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-11T08:40:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-11T08:40:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-11T08:40:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-11T08:40:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-11T08:40:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-11T08:40:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-11T08:40:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-11T08:40:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-11T08:40:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-11T08:40:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-11T08:40:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-11T08:40:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-11T08:40:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-11T08:40:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-11T08:40:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-11T08:40:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-11T08:40:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-11T08:40:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-11T08:40:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-11T08:40:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-11T08:40:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-11T08:40:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-11T08:40:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-11T08:40:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-11T08:40:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-11T08:40:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-11T08:40:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-11T08:40:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-11T08:40:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-11T08:40:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-11T08:40:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mtime_update_interval": "60", 2024-09-11T08:40:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_non_blocking_aio": "true", 2024-09-11T08:40:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_op_thread_timeout": "60", 2024-09-11T08:40:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_op_threads": "1", 2024-09-11T08:40:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_parent_cache_enabled": "false", 2024-09-11T08:40:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-11T08:40:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-11T08:40:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-11T08:40:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_plugins": "", 2024-09-11T08:40:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_bps_burst": "0", 2024-09-11T08:40:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-11T08:40:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_bps_limit": "0", 2024-09-11T08:40:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_exclude_ops": "0", 2024-09-11T08:40:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_iops_burst": "0", 2024-09-11T08:40:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-11T08:40:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_iops_limit": "0", 2024-09-11T08:40:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-11T08:40:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-11T08:40:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-11T08:40:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-11T08:40:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-11T08:40:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-11T08:40:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-11T08:40:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-11T08:40:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-11T08:40:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-11T08:40:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-11T08:40:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-11T08:40:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-11T08:40:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-11T08:40:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_read_from_replica_policy": "default", 2024-09-11T08:40:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-11T08:40:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-11T08:40:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-11T08:40:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-11T08:40:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_skip_partial_discard": "true", 2024-09-11T08:40:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-11T08:40:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_tracing": "false", 2024-09-11T08:40:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_validate_names": "true", 2024-09-11T08:40:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_validate_pool": "true", 2024-09-11T08:40:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "restapi_base_url": "", 2024-09-11T08:40:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "restapi_log_level": "", 2024-09-11T08:40:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_acl_grants_max_num": "100", 2024-09-11T08:40:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_admin_entry": "admin", 2024-09-11T08:40:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_backend_store": "rados", 2024-09-11T08:40:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_barbican_url": "", 2024-09-11T08:40:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_beast_enable_async": "true", 2024-09-11T08:40:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-11T08:40:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-11T08:40:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-11T08:40:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-11T08:40:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-11T08:40:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cache_enabled": "true", 2024-09-11T08:40:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cache_expiry_interval": "900", 2024-09-11T08:40:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cache_lru_size": "10000", 2024-09-11T08:40:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_content_length_compat": "false", 2024-09-11T08:40:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_copy_obj_progress": "true", 2024-09-11T08:40:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-11T08:40:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cors_rules_max_num": "100", 2024-09-11T08:40:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cross_domain_policy": "", 2024-09-11T08:40:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-11T08:40:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_addr": "", 2024-09-11T08:40:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-11T08:40:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-11T08:40:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-11T08:40:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-11T08:40:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_password": "", 2024-09-11T08:40:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-11T08:40:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_username": "", 2024-09-11T08:40:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_require_ssl": "true", 2024-09-11T08:40:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-11T08:40:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-11T08:40:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-11T08:40:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-11T08:40:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-11T08:40:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-11T08:40:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-11T08:40:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-11T08:40:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-11T08:40:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-11T08:40:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-11T08:40:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-11T08:40:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-11T08:40:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-11T08:40:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-11T08:40:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_addr": "", 2024-09-11T08:40:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_auth": "token", 2024-09-11T08:40:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_namespace": "", 2024-09-11T08:40:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_prefix": "", 2024-09-11T08:40:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-11T08:40:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-11T08:40:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-11T08:40:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-11T08:40:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_token_file": "", 2024-09-11T08:40:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-11T08:40:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_buffersize": "524288", 2024-09-11T08:40:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-11T08:40:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_low_speed_time": "300", 2024-09-11T08:40:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-11T08:40:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-11T08:40:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-11T08:40:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-11T08:40:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-11T08:40:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-11T08:40:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-11T08:40:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-11T08:40:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-11T08:40:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-11T08:40:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_changes_size": "1000", 2024-09-11T08:40:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_num_shards": "128", 2024-09-11T08:40:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-11T08:40:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_window": "30", 2024-09-11T08:40:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_notify_interval_msec": "200", 2024-09-11T08:40:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-11T08:40:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-11T08:40:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-11T08:40:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-11T08:40:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-11T08:40:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-11T08:40:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-11T08:40:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-11T08:40:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-11T08:40:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-11T08:40:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-11T08:40:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-11T08:40:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-11T08:40:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-11T08:40:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-11T08:40:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-11T08:40:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-11T08:40:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-11T08:40:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-11T08:40:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-11T08:40:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-11T08:40:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-11T08:40:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dns_name": "", 2024-09-11T08:40:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dns_s3website_name": "", 2024-09-11T08:40:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dynamic_resharding": "true", 2024-09-11T08:40:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-11T08:40:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_gc_threads": "true", 2024-09-11T08:40:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_lc_threads": "true", 2024-09-11T08:40:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_ops_log": "false", 2024-09-11T08:40:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_quota_threads": "true", 2024-09-11T08:40:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_static_website": "false", 2024-09-11T08:40:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_usage_log": "false", 2024-09-11T08:40:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enforce_swift_acls": "true", 2024-09-11T08:40:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_exit_timeout_secs": "120", 2024-09-11T08:40:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_expose_bucket": "false", 2024-09-11T08:40:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_extended_http_attrs": "", 2024-09-11T08:40:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-11T08:40:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_frontends": "beast port=7480", 2024-09-11T08:40:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-11T08:40:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_deferred": "50", 2024-09-11T08:40:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-11T08:40:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_objs": "32", 2024-09-11T08:40:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-11T08:40:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-11T08:40:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-11T08:40:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-11T08:40:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_processor_period": "3600", 2024-09-11T08:40:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-11T08:40:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-11T08:40:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-11T08:40:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-11T08:40:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_init_timeout": "300", 2024-09-11T08:40:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-11T08:40:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_kafka_connection_idle": "300", 2024-09-11T08:40:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-11T08:40:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-11T08:40:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-11T08:40:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_domain": "", 2024-09-11T08:40:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_password": "", 2024-09-11T08:40:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_password_path": "", 2024-09-11T08:40:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_project": "", 2024-09-11T08:40:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_tenant": "", 2024-09-11T08:40:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_token": "", 2024-09-11T08:40:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_token_path": "", 2024-09-11T08:40:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_user": "", 2024-09-11T08:40:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_api_version": "2", 2024-09-11T08:40:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_domain": "", 2024-09-11T08:40:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_password": "", 2024-09-11T08:40:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_project": "", 2024-09-11T08:40:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-11T08:40:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_user": "", 2024-09-11T08:40:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-11T08:40:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-11T08:40:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-11T08:40:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-11T08:40:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-11T08:40:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_url": "", 2024-09-11T08:40:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-11T08:40:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_debug_interval": "-1", 2024-09-11T08:40:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_lock_max_time": "90", 2024-09-11T08:40:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_objs": "32", 2024-09-11T08:40:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_rules": "1000", 2024-09-11T08:40:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_worker": "3", 2024-09-11T08:40:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-11T08:40:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_thread_delay": "0", 2024-09-11T08:40:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-11T08:40:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_dnattr": "uid", 2024-09-11T08:40:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-11T08:40:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_searchfilter": "", 2024-09-11T08:40:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-11T08:40:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_uri": "", 2024-09-11T08:40:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-11T08:40:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-11T08:40:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-11T08:40:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_http_headers": "", 2024-09-11T08:40:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-11T08:40:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-11T08:40:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_object_name_utc": "false", 2024-09-11T08:40:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-11T08:40:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_attr_name_len": "0", 2024-09-11T08:40:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_attr_size": "0", 2024-09-11T08:40:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-11T08:40:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_chunk_size": "4194304", 2024-09-11T08:40:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-11T08:40:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-11T08:40:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_listing_results": "1000", 2024-09-11T08:40:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_notify_retries": "10", 2024-09-11T08:40:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-11T08:40:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_put_param_size": "1048576", 2024-09-11T08:40:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_put_size": "5368709120", 2024-09-11T08:40:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_slo_entries": "1000", 2024-09-11T08:40:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_md_log_max_shards": "64", 2024-09-11T08:40:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-11T08:40:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-11T08:40:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_mp_lock_max_time": "600", 2024-09-11T08:40:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-11T08:40:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-11T08:40:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-11T08:40:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-11T08:40:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-11T08:40:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-11T08:40:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-11T08:40:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_max_gc": "300", 2024-09-11T08:40:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-11T08:40:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-11T08:40:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-11T08:40:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-11T08:40:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-11T08:40:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-11T08:40:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-11T08:40:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_num_async_rados_threads": "32", 2024-09-11T08:40:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_num_control_oids": "8", 2024-09-11T08:40:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_numa_node": "-1", 2024-09-11T08:40:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-11T08:40:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-11T08:40:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_objexp_chunk_size": "100", 2024-09-11T08:40:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_objexp_gc_interval": "600", 2024-09-11T08:40:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-11T08:40:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-11T08:40:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-11T08:40:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_op_thread_timeout": "600", 2024-09-11T08:40:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_op_tracing": "false", 2024-09-11T08:40:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_opa_token": "", 2024-09-11T08:40:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_opa_url": "", 2024-09-11T08:40:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_opa_verify_ssl": "true", 2024-09-11T08:40:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-11T08:40:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ops_log_file_path": "", 2024-09-11T08:40:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ops_log_rados": "true", 2024-09-11T08:40:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ops_log_socket_path": "", 2024-09-11T08:40:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-11T08:40:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-11T08:40:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_push_interval": "2.000000", 2024-09-11T08:40:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-11T08:40:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-11T08:40:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_print_continue": "true", 2024-09-11T08:40:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-11T08:40:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-11T08:40:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-11T08:40:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-11T08:40:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-11T08:40:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rados_tracing": "false", 2024-09-11T08:40:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_realm": "", 2024-09-11T08:40:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_realm_id": "", 2024-09-11T08:40:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-11T08:40:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_region": "", 2024-09-11T08:40:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-11T08:40:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-11T08:40:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-11T08:40:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-11T08:40:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_request_uri": "", 2024-09-11T08:40:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_batch_size": "64", 2024-09-11T08:40:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-11T08:40:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_max_aio": "128", 2024-09-11T08:40:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_num_logs": "16", 2024-09-11T08:40:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_thread_interval": "600", 2024-09-11T08:40:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_resolve_cname": "false", 2024-09-11T08:40:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-11T08:40:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_run_sync_thread": "true", 2024-09-11T08:40:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-11T08:40:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-11T08:40:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-11T08:40:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-11T08:40:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-11T08:40:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-11T08:40:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-11T08:40:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-11T08:40:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-11T08:40:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_scheduler_type": "throttler", 2024-09-11T08:40:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_script_uri": "", 2024-09-11T08:40:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_service_provider_name": "", 2024-09-11T08:40:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-11T08:40:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_client_id": "", 2024-09-11T08:40:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_client_secret": "", 2024-09-11T08:40:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_entry": "sts", 2024-09-11T08:40:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_key": "sts", 2024-09-11T08:40:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-11T08:40:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_min_session_duration": "900", 2024-09-11T08:40:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_token_introspection_url": "", 2024-09-11T08:40:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_account_in_url": "false", 2024-09-11T08:40:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_auth_entry": "auth", 2024-09-11T08:40:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_auth_url": "", 2024-09-11T08:40:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_custom_header": "", 2024-09-11T08:40:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-11T08:40:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_need_stats": "true", 2024-09-11T08:40:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_tenant_name": "", 2024-09-11T08:40:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_token_expiration": "86400", 2024-09-11T08:40:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_url": "", 2024-09-11T08:40:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_url_prefix": "swift", 2024-09-11T08:40:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-11T08:40:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-11T08:40:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_lease_period": "120", 2024-09-11T08:40:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-11T08:40:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-11T08:40:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-11T08:40:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-11T08:40:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-11T08:40:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-11T08:40:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-11T08:40:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-11T08:40:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-11T08:40:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_thread_pool_size": "512", 2024-09-11T08:40:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_comment": "", 2024-09-11T08:40:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_createby": "", 2024-09-11T08:40:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_encoding": "", 2024-09-11T08:40:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_flag": "false", 2024-09-11T08:40:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_origin": "", 2024-09-11T08:40:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-11T08:40:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_tracker": "", 2024-09-11T08:40:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_trust_forwarded_https": "false", 2024-09-11T08:40:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-11T08:40:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-11T08:40:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_max_shards": "32", 2024-09-11T08:40:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_max_user_shards": "1", 2024-09-11T08:40:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_use_opa_authz": "false", 2024-09-11T08:40:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-11T08:40:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-11T08:40:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_max_buckets": "1000", 2024-09-11T08:40:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-11T08:40:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-11T08:40:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-11T08:40:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-11T08:40:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_unique_email": "true", 2024-09-11T08:40:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_verify_ssl": "true", 2024-09-11T08:40:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-11T08:40:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zone": "", 2024-09-11T08:40:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zone_id": "", 2024-09-11T08:40:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-11T08:40:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zonegroup": "", 2024-09-11T08:40:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zonegroup_id": "", 2024-09-11T08:40:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-11T08:40:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_block_size": "4096", 2024-09-11T08:40:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-11T08:40:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-11T08:40:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-11T08:40:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-11T08:40:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-11T08:40:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_size": "536870912", 2024-09-11T08:40:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-11T08:40:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-11T08:40:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-11T08:40:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-11T08:40:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-11T08:40:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-11T08:40:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-11T08:40:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-11T08:40:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_index_type": "binary_search", 2024-09-11T08:40:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-11T08:40:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-11T08:40:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_partition_filters": "false", 2024-09-11T08:40:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_perf": "false", 2024-09-11T08:40:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-11T08:40:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-11T08:40:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-11T08:40:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "run_dir": "/var/run/ceph", 2024-09-11T08:40:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_block_create": "true", 2024-09-11T08:40:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_cache_lru_size": "67108864", 2024-09-11T08:40:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_default_max_object_size": "16777216", 2024-09-11T08:40:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-11T08:40:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_device_size": "107374182400", 2024-09-11T08:40:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_init_rewrite_segments_num_per_device": "6", 2024-09-11T08:40:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_journal_batch_capacity": "16", 2024-09-11T08:40:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-11T08:40:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-11T08:40:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-11T08:40:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_segment_size": "67108864", 2024-09-11T08:40:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "setgroup": "ceph", 2024-09-11T08:40:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "setuser": "ceph", 2024-09-11T08:40:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "setuser_match_path": "", 2024-09-11T08:40:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-11T08:40:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "thp": "false", 2024-09-11T08:40:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "threadpool_default_timeout": "60", 2024-09-11T08:40:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-11T08:40:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "throttler_perf_counter": "true" 2024-09-11T08:40:47.098 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:40:47.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:47 smithi092 ceph-mon[29647]: 5.3 scrub starts 2024-09-11T08:40:47.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:47 smithi092 ceph-mon[29647]: 5.3 scrub ok 2024-09-11T08:40:47.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:47 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:47.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:47 smithi092 ceph-mon[41114]: 5.3 scrub starts 2024-09-11T08:40:47.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:47 smithi092 ceph-mon[41114]: 5.3 scrub ok 2024-09-11T08:40:47.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:47 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:47.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:47 smithi145 ceph-mon[33384]: 5.3 scrub starts 2024-09-11T08:40:47.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:47 smithi145 ceph-mon[33384]: 5.3 scrub ok 2024-09-11T08:40:47.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:47 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:48.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[29647]: 6.17 scrub starts 2024-09-11T08:40:48.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[29647]: 6.17 scrub ok 2024-09-11T08:40:48.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[29647]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:48.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:48.261 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:48.261 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:40:48.261 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:40:48.261 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:48.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[41114]: 6.17 scrub starts 2024-09-11T08:40:48.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[41114]: 6.17 scrub ok 2024-09-11T08:40:48.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[41114]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:48.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:48.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:48.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:40:48.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:40:48.262 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:48 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:48.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-11T08:40:48.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:48 smithi145 ceph-mon[33384]: 6.17 scrub starts 2024-09-11T08:40:48.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:48 smithi145 ceph-mon[33384]: 6.17 scrub ok 2024-09-11T08:40:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:48 smithi145 ceph-mon[33384]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:48 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:48 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:48 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:40:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:48 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:40:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:48 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:40:48.527 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:49.928 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:49 smithi092 ceph-mon[29647]: 6.1f scrub starts 2024-09-11T08:40:49.928 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:49 smithi092 ceph-mon[29647]: 6.1f scrub ok 2024-09-11T08:40:49.928 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:49 smithi092 ceph-mon[29647]: 6.2 scrub starts 2024-09-11T08:40:49.928 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:49 smithi092 ceph-mon[29647]: 6.2 scrub ok 2024-09-11T08:40:49.929 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:49 smithi092 ceph-mon[29647]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 851 B/s rd, 0 op/s 2024-09-11T08:40:49.929 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:49 smithi092 ceph-mon[41114]: 6.1f scrub starts 2024-09-11T08:40:49.929 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:49 smithi092 ceph-mon[41114]: 6.1f scrub ok 2024-09-11T08:40:49.929 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:49 smithi092 ceph-mon[41114]: 6.2 scrub starts 2024-09-11T08:40:49.929 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:49 smithi092 ceph-mon[41114]: 6.2 scrub ok 2024-09-11T08:40:49.929 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:49 smithi092 ceph-mon[41114]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 851 B/s rd, 0 op/s 2024-09-11T08:40:50.145 INFO:teuthology.orchestra.run.smithi092.stdout:mon.a: {} 2024-09-11T08:40:50.145 INFO:teuthology.orchestra.run.smithi092.stdout:mon.b: {} 2024-09-11T08:40:50.145 INFO:teuthology.orchestra.run.smithi092.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T08:40:50.145 INFO:teuthology.orchestra.run.smithi092.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T08:40:50.152 INFO:teuthology.orchestra.run.smithi092.stdout:mon.c: {} 2024-09-11T08:40:50.153 INFO:teuthology.orchestra.run.smithi092.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T08:40:50.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:49 smithi145 ceph-mon[33384]: 6.1f scrub starts 2024-09-11T08:40:50.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:49 smithi145 ceph-mon[33384]: 6.1f scrub ok 2024-09-11T08:40:50.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:49 smithi145 ceph-mon[33384]: 6.2 scrub starts 2024-09-11T08:40:50.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:49 smithi145 ceph-mon[33384]: 6.2 scrub ok 2024-09-11T08:40:50.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:49 smithi145 ceph-mon[33384]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 851 B/s rd, 0 op/s 2024-09-11T08:40:50.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:50 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1009186577' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:40:50.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:50 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1009186577' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:40:50.854 INFO:teuthology.run_tasks:Running task full_sequential... 2024-09-11T08:40:50.904 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-11T08:40:50.908 INFO:tasks.radosbench:Beginning radosbench... 2024-09-11T08:40:50.908 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-11T08:40:50.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool create unique_pool_0 16 2024-09-11T08:40:50.909 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-11T08:40:50.909 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:40:50.909 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:40:50.909 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-11T08:40:50.910 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:40:50.910 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-11T08:40:50.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:40:50.911 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-11T08:40:50.911 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-11T08:40:50.911 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-11T08:40:50.911 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:40:50.939 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:50.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:50.941 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:40:51.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:40:51.073 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:51.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:51.075 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:40:51.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:50 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1009186577' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:40:51.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:40:51.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:51.207 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:51.208 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:40:51.218 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:51.219 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:51.219 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:51.221 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:40:51.309 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:40:51.330 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:51.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:51.331 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:40:51.432 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:40:51.458 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:51.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:51.459 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:40:51.560 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:40:51.586 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:51.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:51.587 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:40:51.611 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:40:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:40:51] "GET /metrics HTTP/1.1" 200 220023 "" "Prometheus/2.43.0" 2024-09-11T08:40:51.688 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:40:51.713 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:51.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:51.715 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:40:51.815 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:40:51.840 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:51.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:51.841 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:40:51.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:51 smithi092 ceph-mon[29647]: 3.b scrub starts 2024-09-11T08:40:51.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:51 smithi092 ceph-mon[29647]: 3.b scrub ok 2024-09-11T08:40:51.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:51 smithi092 ceph-mon[29647]: 4.13 scrub starts 2024-09-11T08:40:51.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:51 smithi092 ceph-mon[29647]: 4.13 scrub ok 2024-09-11T08:40:51.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:51 smithi092 ceph-mon[29647]: 6.9 scrub starts 2024-09-11T08:40:51.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:51 smithi092 ceph-mon[29647]: 6.9 scrub ok 2024-09-11T08:40:51.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:51 smithi092 ceph-mon[29647]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:51.917 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:51 smithi092 ceph-mon[41114]: 3.b scrub starts 2024-09-11T08:40:51.917 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:51 smithi092 ceph-mon[41114]: 3.b scrub ok 2024-09-11T08:40:51.917 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:51 smithi092 ceph-mon[41114]: 4.13 scrub starts 2024-09-11T08:40:51.917 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:51 smithi092 ceph-mon[41114]: 4.13 scrub ok 2024-09-11T08:40:51.917 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:51 smithi092 ceph-mon[41114]: 6.9 scrub starts 2024-09-11T08:40:51.917 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:51 smithi092 ceph-mon[41114]: 6.9 scrub ok 2024-09-11T08:40:51.917 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:51 smithi092 ceph-mon[41114]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:51.942 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:40:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:51.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:51.964 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:40:52.065 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:40:52.086 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:52.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:52.087 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:40:52.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:51 smithi145 ceph-mon[33384]: 3.b scrub starts 2024-09-11T08:40:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:51 smithi145 ceph-mon[33384]: 3.b scrub ok 2024-09-11T08:40:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:51 smithi145 ceph-mon[33384]: 4.13 scrub starts 2024-09-11T08:40:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:51 smithi145 ceph-mon[33384]: 4.13 scrub ok 2024-09-11T08:40:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:51 smithi145 ceph-mon[33384]: 6.9 scrub starts 2024-09-11T08:40:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:51 smithi145 ceph-mon[33384]: 6.9 scrub ok 2024-09-11T08:40:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:51 smithi145 ceph-mon[33384]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:52.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:40:52.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:52.209 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:52.210 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:40:52.310 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:40:52.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:52.337 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:52.337 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:40:52.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:40:52.460 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:52.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:52.461 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:40:52.562 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:40:52.588 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:52.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:52.589 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:40:52.690 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:40:52.716 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:52.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:52.720 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:40:52.821 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:40:52.846 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:52.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:52.847 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:40:52.948 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:40:52.969 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:52.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:52.971 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:40:53.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:52 smithi092 ceph-mon[29647]: 5.0 scrub starts 2024-09-11T08:40:53.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:52 smithi092 ceph-mon[29647]: 5.0 scrub ok 2024-09-11T08:40:53.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:52 smithi092 ceph-mon[29647]: 6.0 deep-scrub starts 2024-09-11T08:40:53.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:52 smithi092 ceph-mon[29647]: 6.0 deep-scrub ok 2024-09-11T08:40:53.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:52 smithi092 ceph-mon[41114]: 5.0 scrub starts 2024-09-11T08:40:53.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:52 smithi092 ceph-mon[41114]: 5.0 scrub ok 2024-09-11T08:40:53.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:52 smithi092 ceph-mon[41114]: 6.0 deep-scrub starts 2024-09-11T08:40:53.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:52 smithi092 ceph-mon[41114]: 6.0 deep-scrub ok 2024-09-11T08:40:53.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:40:53.092 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:53.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:53.093 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:40:53.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:52 smithi145 ceph-mon[33384]: 5.0 scrub starts 2024-09-11T08:40:53.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:52 smithi145 ceph-mon[33384]: 5.0 scrub ok 2024-09-11T08:40:53.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:52 smithi145 ceph-mon[33384]: 6.0 deep-scrub starts 2024-09-11T08:40:53.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:52 smithi145 ceph-mon[33384]: 6.0 deep-scrub ok 2024-09-11T08:40:53.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:40:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:40:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:53.194 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:40:53.220 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:53.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:53.221 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:40:53.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:40:53.343 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:53.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:53.344 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:40:53.445 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:40:53.466 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:53.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:53.467 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:40:53.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:40:53.589 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:53.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:53.590 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:40:53.690 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:40:53.717 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:53.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:53.718 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:40:53.819 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:40:53.845 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:53.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:53.846 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:40:53.946 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:40:53.972 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:53.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:53.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:40:53.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:53 smithi092 ceph-mon[29647]: 6.13 scrub starts 2024-09-11T08:40:53.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:53 smithi092 ceph-mon[29647]: 6.13 scrub ok 2024-09-11T08:40:53.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:53 smithi092 ceph-mon[29647]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:40:53.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:53 smithi092 ceph-mon[29647]: 5.10 scrub starts 2024-09-11T08:40:53.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:53 smithi092 ceph-mon[29647]: 5.10 scrub ok 2024-09-11T08:40:53.986 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:53 smithi092 ceph-mon[41114]: 6.13 scrub starts 2024-09-11T08:40:53.986 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:53 smithi092 ceph-mon[41114]: 6.13 scrub ok 2024-09-11T08:40:53.986 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:53 smithi092 ceph-mon[41114]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:40:53.986 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:53 smithi092 ceph-mon[41114]: 5.10 scrub starts 2024-09-11T08:40:53.986 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:53 smithi092 ceph-mon[41114]: 5.10 scrub ok 2024-09-11T08:40:54.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:40:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:54.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:54.097 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:40:54.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:53 smithi145 ceph-mon[33384]: 6.13 scrub starts 2024-09-11T08:40:54.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:53 smithi145 ceph-mon[33384]: 6.13 scrub ok 2024-09-11T08:40:54.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:53 smithi145 ceph-mon[33384]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:40:54.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:53 smithi145 ceph-mon[33384]: 5.10 scrub starts 2024-09-11T08:40:54.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:53 smithi145 ceph-mon[33384]: 5.10 scrub ok 2024-09-11T08:40:54.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:40:54.220 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:54.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:54.221 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:40:54.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:40:54.343 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:54.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:54.344 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:40:54.445 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:40:54.465 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:54.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:54.466 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:40:54.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:40:54.588 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:54.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:54.589 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:40:54.690 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:40:54.717 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:54.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:54.718 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:40:54.818 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:40:54.844 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:54.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:54.845 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:40:54.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:40:54.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:54.970 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:54.970 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:40:55.007 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:54 smithi092 ceph-mon[29647]: 4.e scrub starts 2024-09-11T08:40:55.008 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:54 smithi092 ceph-mon[29647]: 4.e scrub ok 2024-09-11T08:40:55.008 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:54 smithi092 ceph-mon[41114]: 4.e scrub starts 2024-09-11T08:40:55.008 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:54 smithi092 ceph-mon[41114]: 4.e scrub ok 2024-09-11T08:40:55.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:40:55.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:55.093 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:55.093 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:40:55.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:54 smithi145 ceph-mon[33384]: 4.e scrub starts 2024-09-11T08:40:55.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:54 smithi145 ceph-mon[33384]: 4.e scrub ok 2024-09-11T08:40:55.194 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:40:55.216 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:55.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:55.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:40:55.317 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:40:55.343 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:55.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:55.344 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:40:55.445 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:40:55.470 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:55.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:55.471 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:40:55.572 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:40:55.598 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:55.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:55.599 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:40:55.700 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:40:55.725 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:55.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:55.726 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:40:55.827 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:40:55.850 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:55.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:55.851 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:40:55.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:40:55.975 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:55 smithi092 ceph-mon[29647]: 4.11 scrub starts 2024-09-11T08:40:55.975 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:55 smithi092 ceph-mon[29647]: 4.11 scrub ok 2024-09-11T08:40:55.975 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:55 smithi092 ceph-mon[29647]: 3.9 deep-scrub starts 2024-09-11T08:40:55.975 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:55 smithi092 ceph-mon[29647]: 3.9 deep-scrub ok 2024-09-11T08:40:55.975 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:55 smithi092 ceph-mon[29647]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:40:55.975 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:55 smithi092 ceph-mon[41114]: 4.11 scrub starts 2024-09-11T08:40:55.975 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:55 smithi092 ceph-mon[41114]: 4.11 scrub ok 2024-09-11T08:40:55.975 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:55 smithi092 ceph-mon[41114]: 3.9 deep-scrub starts 2024-09-11T08:40:55.976 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:55 smithi092 ceph-mon[41114]: 3.9 deep-scrub ok 2024-09-11T08:40:55.976 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:55 smithi092 ceph-mon[41114]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:40:55.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:55.977 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:55.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:40:56.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:40:56.098 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:55 smithi145 ceph-mon[33384]: 4.11 scrub starts 2024-09-11T08:40:56.098 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:55 smithi145 ceph-mon[33384]: 4.11 scrub ok 2024-09-11T08:40:56.098 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:55 smithi145 ceph-mon[33384]: 3.9 deep-scrub starts 2024-09-11T08:40:56.098 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:55 smithi145 ceph-mon[33384]: 3.9 deep-scrub ok 2024-09-11T08:40:56.098 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:55 smithi145 ceph-mon[33384]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:40:56.100 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:56.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:56.100 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:40:56.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:40:56.222 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:56.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:56.223 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:40:56.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:40:56.345 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:56.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:56.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:40:56.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:40:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:40:56.447 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:40:56.473 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:56.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:56.474 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:40:56.575 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:40:56.586 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:40:56.586 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:40:56.592 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:40:56.592 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:40:56.601 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:56.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:56.605 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:40:56.607 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:40:56.607 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:40:56.622 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:40:56.622 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:40:56.637 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:40:56.638 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:40:56.653 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:40:56.654 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:40:56.669 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:40:56.669 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:40:56.685 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:40:56.685 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:40:56.706 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:40:56.732 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:56.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:56.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:40:56.834 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:40:56.859 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:56.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:56.860 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:40:56.961 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:40:56.986 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:56.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:56.987 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:40:56.989 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[29647]: 3.10 deep-scrub starts 2024-09-11T08:40:56.989 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[29647]: 3.10 deep-scrub ok 2024-09-11T08:40:56.989 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[29647]: 4.1 deep-scrub starts 2024-09-11T08:40:56.989 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[29647]: 4.1 deep-scrub ok 2024-09-11T08:40:56.989 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:56.990 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[29647]: 6.1d scrub starts 2024-09-11T08:40:56.990 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[29647]: 6.1d scrub ok 2024-09-11T08:40:56.990 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[29647]: 5.7 scrub starts 2024-09-11T08:40:56.990 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[29647]: 5.7 scrub ok 2024-09-11T08:40:56.990 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/557854865' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:40:56.990 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[41114]: 3.10 deep-scrub starts 2024-09-11T08:40:56.990 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[41114]: 3.10 deep-scrub ok 2024-09-11T08:40:56.990 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[41114]: 4.1 deep-scrub starts 2024-09-11T08:40:56.990 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[41114]: 4.1 deep-scrub ok 2024-09-11T08:40:56.990 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:56.991 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[41114]: 6.1d scrub starts 2024-09-11T08:40:56.991 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[41114]: 6.1d scrub ok 2024-09-11T08:40:56.991 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[41114]: 5.7 scrub starts 2024-09-11T08:40:56.991 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[41114]: 5.7 scrub ok 2024-09-11T08:40:56.991 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:56 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/557854865' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:40:57.088 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:40:57.106 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:40:57.106 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:40:57.114 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:57.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:57.115 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:40:57.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:56 smithi145 ceph-mon[33384]: 3.10 deep-scrub starts 2024-09-11T08:40:57.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:56 smithi145 ceph-mon[33384]: 3.10 deep-scrub ok 2024-09-11T08:40:57.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:56 smithi145 ceph-mon[33384]: 4.1 deep-scrub starts 2024-09-11T08:40:57.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:56 smithi145 ceph-mon[33384]: 4.1 deep-scrub ok 2024-09-11T08:40:57.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:56 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:57.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:56 smithi145 ceph-mon[33384]: 6.1d scrub starts 2024-09-11T08:40:57.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:56 smithi145 ceph-mon[33384]: 6.1d scrub ok 2024-09-11T08:40:57.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:56 smithi145 ceph-mon[33384]: 5.7 scrub starts 2024-09-11T08:40:57.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:56 smithi145 ceph-mon[33384]: 5.7 scrub ok 2024-09-11T08:40:57.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:56 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/557854865' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:40:57.217 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:40:57.242 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:57.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:57.243 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:40:57.344 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:40:57.369 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:57.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:57.370 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:40:57.471 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:40:57.496 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:57.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:57.497 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:40:57.598 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:40:57.623 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:57.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:57.624 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:40:57.725 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:40:57.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:57.752 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:57.752 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:40:57.852 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:40:57.875 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:57.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:57.876 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:40:57.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:40:57.980 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:57 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:40:57.980 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:57 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:40:57.980 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:57 smithi092 ceph-mon[29647]: osdmap e87: 8 total, 8 up, 8 in 2024-09-11T08:40:57.981 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:57 smithi092 ceph-mon[29647]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:40:57.981 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:57 smithi092 ceph-mon[29647]: 5.c scrub starts 2024-09-11T08:40:57.981 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:57 smithi092 ceph-mon[29647]: 5.c scrub ok 2024-09-11T08:40:57.981 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:57 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:40:57.981 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:57 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:40:57.981 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:57 smithi092 ceph-mon[41114]: osdmap e87: 8 total, 8 up, 8 in 2024-09-11T08:40:57.981 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:57 smithi092 ceph-mon[41114]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:40:57.981 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:57 smithi092 ceph-mon[41114]: 5.c scrub starts 2024-09-11T08:40:57.981 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:57 smithi092 ceph-mon[41114]: 5.c scrub ok 2024-09-11T08:40:57.998 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:57.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:57.999 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:40:58.064 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:57 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:40:58.064 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:57 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:40:58.064 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:57 smithi145 ceph-mon[33384]: osdmap e87: 8 total, 8 up, 8 in 2024-09-11T08:40:58.064 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:57 smithi145 ceph-mon[33384]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:40:58.064 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:57 smithi145 ceph-mon[33384]: 5.c scrub starts 2024-09-11T08:40:58.064 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:57 smithi145 ceph-mon[33384]: 5.c scrub ok 2024-09-11T08:40:58.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:40:58.121 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:58.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:58.122 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:40:58.222 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:40:58.248 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:58.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:58.249 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:40:58.350 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:40:58.372 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:58.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:58.373 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:40:58.412 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-11T08:40:58.474 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:40:58.495 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:58.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:58.496 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:40:58.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:40:58.618 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:58.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:58.619 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:40:58.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:40:58.738 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'unique_pool_0' created 2024-09-11T08:40:58.744 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:58.745 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:40:58.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:58.746 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:40:58.846 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:40:58.867 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:58.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:58.868 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:40:58.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:58 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1269592915' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-11T08:40:58.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:58 smithi145 ceph-mon[33384]: 2.0 scrub starts 2024-09-11T08:40:58.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:58 smithi145 ceph-mon[33384]: 2.0 scrub ok 2024-09-11T08:40:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:58 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3591128086' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:40:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:58 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:40:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:58 smithi145 ceph-mon[33384]: 5.1d scrub starts 2024-09-11T08:40:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:58 smithi145 ceph-mon[33384]: 5.1d scrub ok 2024-09-11T08:40:58.969 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:40:58.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:58.993 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:58.993 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:40:59.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:58 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1269592915' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-11T08:40:59.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:58 smithi092 ceph-mon[29647]: 2.0 scrub starts 2024-09-11T08:40:59.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:58 smithi092 ceph-mon[29647]: 2.0 scrub ok 2024-09-11T08:40:59.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:58 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3591128086' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:40:59.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:58 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:40:59.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:58 smithi092 ceph-mon[29647]: 5.1d scrub starts 2024-09-11T08:40:59.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:58 smithi092 ceph-mon[29647]: 5.1d scrub ok 2024-09-11T08:40:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:58 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1269592915' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-11T08:40:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:58 smithi092 ceph-mon[41114]: 2.0 scrub starts 2024-09-11T08:40:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:58 smithi092 ceph-mon[41114]: 2.0 scrub ok 2024-09-11T08:40:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:58 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3591128086' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:40:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:58 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:40:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:58 smithi092 ceph-mon[41114]: 5.1d scrub starts 2024-09-11T08:40:59.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:58 smithi092 ceph-mon[41114]: 5.1d scrub ok 2024-09-11T08:40:59.094 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:40:59.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:59.116 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:59.117 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:40:59.218 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:40:59.238 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:59.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:59.239 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:40:59.340 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:40:59.361 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:59.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:59.362 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:40:59.462 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:40:59.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:59.484 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:59.485 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:40:59.585 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:40:59.607 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:59.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:59.608 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:40:59.708 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:40:59.730 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:59.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:59.731 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:40:59.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:59 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:40:59.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:59 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1269592915' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-11T08:40:59.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:59 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:40:59.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:59 smithi092 ceph-mon[41114]: osdmap e88: 8 total, 8 up, 8 in 2024-09-11T08:40:59.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:59 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:40:59.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:59 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:40:59.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:40:59 smithi092 ceph-mon[41114]: pgmap v74: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:40:59.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:59 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:40:59.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:59 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1269592915' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-11T08:40:59.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:59 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:40:59.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:59 smithi092 ceph-mon[29647]: osdmap e88: 8 total, 8 up, 8 in 2024-09-11T08:40:59.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:59 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:40:59.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:59 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:40:59.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:40:59 smithi092 ceph-mon[29647]: pgmap v74: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:40:59.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:40:59.853 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:40:59.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:59.854 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:40:59.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-11T08:40:59.955 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:40:59.977 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:40:59.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:40:59.978 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:00.080 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:00.102 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:00.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:00.103 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:00.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:59 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:41:00.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:59 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1269592915' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-11T08:41:00.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:59 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:41:00.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:59 smithi145 ceph-mon[33384]: osdmap e88: 8 total, 8 up, 8 in 2024-09-11T08:41:00.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:59 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:41:00.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:59 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:41:00.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:40:59 smithi145 ceph-mon[33384]: pgmap v74: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:41:00.204 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:00.225 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:00.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:00.226 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:00.327 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:00.348 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:00.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:00.349 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:00.384 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:00.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:00.471 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:00.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:00.472 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:00.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:00.594 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:00.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:00.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:00.696 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:00.717 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:00.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:00.718 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:00.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:00.841 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:00.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:00.842 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:00.921 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:00 smithi092 ceph-mon[29647]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:41:00.921 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:00 smithi092 ceph-mon[29647]: osdmap e89: 8 total, 8 up, 8 in 2024-09-11T08:41:00.921 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:00 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:00.921 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:00.921 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:00 smithi092 ceph-mon[41114]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:41:00.922 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:00 smithi092 ceph-mon[41114]: osdmap e89: 8 total, 8 up, 8 in 2024-09-11T08:41:00.922 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:00 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:00.922 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:00.943 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:00.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:41:00.965 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:00.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:00.966 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:01.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:01.088 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:01.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:01.089 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:01.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:00 smithi145 ceph-mon[33384]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:41:01.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:00 smithi145 ceph-mon[33384]: osdmap e89: 8 total, 8 up, 8 in 2024-09-11T08:41:01.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:00 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:01.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:01.190 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:01.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:01.213 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:01.214 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:01.256 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:01.286 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:01.314 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:01.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:01.338 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:01.338 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:01.439 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:01.460 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:01.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:01.461 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:01.562 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:01.585 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:01.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:01.586 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:01.687 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:01.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:01.712 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:01.748 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:41:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:41:01] "GET /metrics HTTP/1.1" 200 222522 "" "Prometheus/2.43.0" 2024-09-11T08:41:01.812 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:01.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:01.836 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:01.836 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:01.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:01.959 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:01.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:01.961 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:02.040 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:01 smithi092 ceph-mon[29647]: osdmap e90: 8 total, 8 up, 8 in 2024-09-11T08:41:02.040 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:01 smithi092 ceph-mon[29647]: pgmap v77: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:02.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:01 smithi092 ceph-mon[41114]: osdmap e90: 8 total, 8 up, 8 in 2024-09-11T08:41:02.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:01 smithi092 ceph-mon[41114]: pgmap v77: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:02.062 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:02.087 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:02.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:02.088 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:02.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:01 smithi145 ceph-mon[33384]: osdmap e90: 8 total, 8 up, 8 in 2024-09-11T08:41:02.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:01 smithi145 ceph-mon[33384]: pgmap v77: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:02.189 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:02.214 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:02.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:02.215 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:02.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:02.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:02.342 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:02.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:41:02.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:02.465 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:02.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:02.466 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:02.566 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:02.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:02.588 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:02.588 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:02.689 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:02.714 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:02.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:02.716 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:02.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:02.838 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:02.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:02.839 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:02.940 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:02.967 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:02.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:02.968 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:03.069 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:03.091 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:03.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:03.092 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:03.165 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:41:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:41:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:03.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:03.193 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:41:03.193 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:41:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:03.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:03.214 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:03.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:03.336 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:03.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:03.337 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:03.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:03.459 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:03.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:03.459 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:03.560 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:03.586 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:03.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:03.587 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:03.688 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:03.710 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:03.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:03.711 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:03.812 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:03.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:03.834 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:03.834 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:03.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:03.956 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:03.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:03.957 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:04.058 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:04.080 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:04.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:04.081 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:04.182 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:04.208 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:04.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:04.209 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:04.310 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:04.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:04 smithi092 ceph-mon[29647]: pgmap v78: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:04.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:04 smithi092 ceph-mon[41114]: pgmap v78: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:04.336 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:04.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:04.337 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:04.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:04 smithi145 ceph-mon[33384]: pgmap v78: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:04.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:04.461 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:04.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:04.462 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:04.525 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-11T08:41:04.563 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:04.588 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:04.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:04.593 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:04.694 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:04.719 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:04.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:04.721 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:04.821 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:04.847 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:04.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:04.848 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:04.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:04.972 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:04.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:04.973 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:05.074 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:05.100 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:05.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:05.101 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:05.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:05.224 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:05.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:05.225 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:05.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:05.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:05.347 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:05.385 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:05.448 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:05.474 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:05.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:05.475 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:05.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:05.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:05.599 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:05.599 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:05.700 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:05.725 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:05.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:05.726 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:05.827 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:05.852 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:05.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:05.853 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:05.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:05.977 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:05.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:05.978 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:06.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:06.100 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:06.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:06.101 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:06.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:06.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:06.224 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:06.224 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:06.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:06 smithi092 ceph-mon[29647]: pgmap v79: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 814 B/s rd, 0 op/s 2024-09-11T08:41:06.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:06 smithi092 ceph-mon[41114]: pgmap v79: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 814 B/s rd, 0 op/s 2024-09-11T08:41:06.325 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:06.351 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:06.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:06.352 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:06.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:06 smithi145 ceph-mon[33384]: pgmap v79: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 814 B/s rd, 0 op/s 2024-09-11T08:41:06.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:41:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:41:06.453 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:06.479 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:06.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:06.480 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:06.546 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:41:06.546 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:41:06.551 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:41:06.551 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:41:06.566 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:41:06.566 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:41:06.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:06.582 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:41:06.583 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:41:06.598 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:41:06.598 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:41:06.603 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:06.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:06.604 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:06.614 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:41:06.614 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:41:06.630 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:41:06.630 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:41:06.646 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:41:06.646 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:41:06.705 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:06.731 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:06.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:06.732 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:06.833 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:06.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:06.861 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:06.862 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:06.962 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:06.990 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:06.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:06.991 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:07.092 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:07.116 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:07.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:07.117 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:07.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:07.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:07.240 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:07.241 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:07.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:07.363 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:07.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:07.364 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:07.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:07 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:07.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:07 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3605679216' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:41:07.464 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:07.470 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:07 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:07.470 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:07 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3605679216' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:41:07.470 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:07 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:07.470 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:07 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3605679216' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:41:07.490 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:07.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:07.491 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:07.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:07.613 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:07.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:07.614 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:07.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:07.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:07.737 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:07.737 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:07.838 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:07.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:07.866 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:07.866 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:07.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:07.989 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:07.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:07.990 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:08.090 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:08.116 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:08.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:08.117 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:08.199 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:08 smithi092 ceph-mon[29647]: pgmap v80: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:08.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:08 smithi092 ceph-mon[41114]: pgmap v80: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:08.204 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:08.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:08.244 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:08.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:08.245 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:08.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:08.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:08.368 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:08.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:08 smithi145 ceph-mon[33384]: pgmap v80: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:08.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:08.491 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:08.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:08.492 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:08.593 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:08.619 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:08.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:08.620 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:08.721 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:08.745 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:08.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:08.746 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:08.847 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:08.871 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:08.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:08.872 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:08.973 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:09.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:09.008 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:09.008 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:09.097 INFO:teuthology.orchestra.run.smithi092.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-11T08:41:09.109 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:09.131 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:09.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:09.132 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:09.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:09.255 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:09.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:09.256 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:09.357 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:09.379 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:09.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:09.380 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:09.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:09 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3146388395' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:09.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:09 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:09.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:09 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3146388395' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:09.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:09 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:09.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:09 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3146388395' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:09.448 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:09 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:41:09.481 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:09.504 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:09.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:09.504 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:09.605 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:09.627 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:09.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:09.628 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:09.728 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:09.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:09.750 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:09.751 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:09.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:09.873 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:09.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:09.874 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:09.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:41:09.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:09.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:09.999 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:09.999 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:10.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:10.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:10.123 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:10.124 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:10.224 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:10.247 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:10.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:10.248 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:10.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:10 smithi092 ceph-mon[29647]: pgmap v81: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:41:10.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:10 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:10.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:10 smithi092 ceph-mon[29647]: osdmap e91: 8 total, 8 up, 8 in 2024-09-11T08:41:10.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:10 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3482937895' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:41:10.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:10 smithi092 ceph-mon[41114]: pgmap v81: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:41:10.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:10 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:10.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:10 smithi092 ceph-mon[41114]: osdmap e91: 8 total, 8 up, 8 in 2024-09-11T08:41:10.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:10 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3482937895' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:41:10.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:10.371 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:10.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:10.372 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:10.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:10 smithi145 ceph-mon[33384]: pgmap v81: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:41:10.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:10 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:41:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:10 smithi145 ceph-mon[33384]: osdmap e91: 8 total, 8 up, 8 in 2024-09-11T08:41:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:10 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3482937895' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:41:10.473 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:10.496 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:10.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:10.497 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:10.598 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:10.619 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:10.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:10.620 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:10.699 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:41:10.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:10.744 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:10.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:10.745 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:10.845 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:10.870 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:10.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:10.871 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:10.972 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:10.996 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:10.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:10.998 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:11.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:11.120 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:11.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:11.121 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:11.222 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:11.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:11.244 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:11.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:11.367 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:11.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:11.368 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:11.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:11 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:41:11.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:11 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3482937895' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:41:11.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:11 smithi145 ceph-mon[33384]: osdmap e92: 8 total, 8 up, 8 in 2024-09-11T08:41:11.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:11 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:41:11.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:11 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3482937895' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:41:11.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:11 smithi092 ceph-mon[29647]: osdmap e92: 8 total, 8 up, 8 in 2024-09-11T08:41:11.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:11 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:41:11.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:11 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3482937895' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:41:11.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:11 smithi092 ceph-mon[41114]: osdmap e92: 8 total, 8 up, 8 in 2024-09-11T08:41:11.468 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:11.494 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:11.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:11.495 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:11.581 DEBUG:teuthology.orchestra.run.smithi092:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_0 bench 90 write' 2024-09-11T08:41:11.582 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-11T08:41:11.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:41:11.584 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-11T08:41:11.584 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-11T08:41:11.595 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:11.620 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:11.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:11.621 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:11.681 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:hints = 1 2024-09-11T08:41:11.681 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-11T08:41:11.681 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Object prefix: benchmark_data_smithi092_84565 2024-09-11T08:41:11.681 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:41:11.681 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 0 0 0 0 0 0 - 0 2024-09-11T08:41:11.723 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:11.755 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:11.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:11.756 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:11.783 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:41:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:41:11] "GET /metrics HTTP/1.1" 200 222522 "" "Prometheus/2.43.0" 2024-09-11T08:41:11.809 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:41:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-11T08:41:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-11T08:41:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:41:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14156.0:69 1.0 e21/19 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 21'47, mlcod=21'43)", 2024-09-11T08:41:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:36:34.147246+0000", 2024-09-11T08:41:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 277.65894519599999, 2024-09-11T08:41:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004378743, 2024-09-11T08:41:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.147246+0000", 2024-09-11T08:41:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.147246+0000", 2024-09-11T08:41:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.147246+0000", 2024-09-11T08:41:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:41:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.147268+0000", 2024-09-11T08:41:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.19e-05 2024-09-11T08:41:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.147282+0000", 2024-09-11T08:41:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4249e-05 2024-09-11T08:41:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.147287+0000", 2024-09-11T08:41:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4e-06 2024-09-11T08:41:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.147322+0000", 2024-09-11T08:41:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4439e-05 2024-09-11T08:41:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.147340+0000", 2024-09-11T08:41:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8536e-05 2024-09-11T08:41:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151586+0000", 2024-09-11T08:41:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042452000000000002 2024-09-11T08:41:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151625+0000", 2024-09-11T08:41:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9598999999999998e-05 2024-09-11T08:41:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14156.0:70 1.0 e21/19 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 21'48, mlcod=21'45)", 2024-09-11T08:41:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:36:34.148262+0000", 2024-09-11T08:41:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 277.65792985600001, 2024-09-11T08:41:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006523437, 2024-09-11T08:41:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.148262+0000", 2024-09-11T08:41:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.148262+0000", 2024-09-11T08:41:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.148261+0000", 2024-09-11T08:41:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:41:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.148268+0000", 2024-09-11T08:41:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8820000000000003e-06 2024-09-11T08:41:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.148282+0000", 2024-09-11T08:41:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3603e-05 2024-09-11T08:41:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.148287+0000", 2024-09-11T08:41:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7150000000000003e-06 2024-09-11T08:41:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151399+0000", 2024-09-11T08:41:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031121040000000001 2024-09-11T08:41:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151423+0000", 2024-09-11T08:41:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4114e-05 2024-09-11T08:41:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.154753+0000", 2024-09-11T08:41:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003329912 2024-09-11T08:41:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.154785+0000", 2024-09-11T08:41:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1806e-05 2024-09-11T08:41:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14156.0:71 1.0 e21/19 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 21'49, mlcod=21'45)", 2024-09-11T08:41:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:36:34.151387+0000", 2024-09-11T08:41:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 277.65480454599998, 2024-09-11T08:41:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034223750000000001, 2024-09-11T08:41:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151387+0000", 2024-09-11T08:41:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151387+0000", 2024-09-11T08:41:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151386+0000", 2024-09-11T08:41:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:41:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151398+0000", 2024-09-11T08:41:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1790000000000001e-05 2024-09-11T08:41:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151413+0000", 2024-09-11T08:41:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5347999999999999e-05 2024-09-11T08:41:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151421+0000", 2024-09-11T08:41:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8050000000000007e-06 2024-09-11T08:41:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151660+0000", 2024-09-11T08:41:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000238557 2024-09-11T08:41:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151675+0000", 2024-09-11T08:41:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5526999999999999e-05 2024-09-11T08:41:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.154791+0000", 2024-09-11T08:41:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031155879999999999 2024-09-11T08:41:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.154809+0000", 2024-09-11T08:41:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8779000000000001e-05 2024-09-11T08:41:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14156.0:72 1.0 e21/19 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 21'50, mlcod=21'45)", 2024-09-11T08:41:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:36:34.151428+0000", 2024-09-11T08:41:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 277.65476395600001, 2024-09-11T08:41:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038935660000000002, 2024-09-11T08:41:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151428+0000", 2024-09-11T08:41:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151428+0000", 2024-09-11T08:41:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151427+0000", 2024-09-11T08:41:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:41:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151488+0000", 2024-09-11T08:41:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0278000000000001e-05 2024-09-11T08:41:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151495+0000", 2024-09-11T08:41:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9160000000000001e-06 2024-09-11T08:41:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151500+0000", 2024-09-11T08:41:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6240000000000001e-06 2024-09-11T08:41:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151815+0000", 2024-09-11T08:41:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031481800000000002 2024-09-11T08:41:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151843+0000", 2024-09-11T08:41:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8005999999999999e-05 2024-09-11T08:41:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.155301+0000", 2024-09-11T08:41:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034579979999999999 2024-09-11T08:41:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.155321+0000", 2024-09-11T08:41:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0285999999999999e-05 2024-09-11T08:41:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14156.0:73 1.0 e21/19 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 21'51, mlcod=21'45)", 2024-09-11T08:41:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:36:34.151505+0000", 2024-09-11T08:41:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 277.65468634500002, 2024-09-11T08:41:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003834718, 2024-09-11T08:41:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151505+0000", 2024-09-11T08:41:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151505+0000", 2024-09-11T08:41:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151505+0000", 2024-09-11T08:41:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:41:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151509+0000", 2024-09-11T08:41:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9589999999999997e-06 2024-09-11T08:41:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151517+0000", 2024-09-11T08:41:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7780000000000007e-06 2024-09-11T08:41:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151520+0000", 2024-09-11T08:41:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2024-09-11T08:41:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.154438+0000", 2024-09-11T08:41:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029174409999999998 2024-09-11T08:41:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.154460+0000", 2024-09-11T08:41:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1838999999999999e-05 2024-09-11T08:41:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.155323+0000", 2024-09-11T08:41:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00086321499999999997 2024-09-11T08:41:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.155340+0000", 2024-09-11T08:41:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.717e-05 2024-09-11T08:41:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14156.0:74 1.0 e21/19 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 21'52, mlcod=21'45)", 2024-09-11T08:41:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:36:34.151524+0000", 2024-09-11T08:41:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 277.65466744100002, 2024-09-11T08:41:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003838887, 2024-09-11T08:41:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151524+0000", 2024-09-11T08:41:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151524+0000", 2024-09-11T08:41:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151524+0000", 2024-09-11T08:41:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967296 2024-09-11T08:41:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151527+0000", 2024-09-11T08:41:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3890000000000001e-06 2024-09-11T08:41:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151535+0000", 2024-09-11T08:41:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2250000000000003e-06 2024-09-11T08:41:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151538+0000", 2024-09-11T08:41:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.236e-06 2024-09-11T08:41:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.154628+0000", 2024-09-11T08:41:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030904930000000001 2024-09-11T08:41:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.154646+0000", 2024-09-11T08:41:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7742000000000001e-05 2024-09-11T08:41:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.155342+0000", 2024-09-11T08:41:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069605800000000005 2024-09-11T08:41:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.155363+0000", 2024-09-11T08:41:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0951999999999999e-05 2024-09-11T08:41:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14156.0:75 1.0 e21/19 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 21'53, mlcod=21'45)", 2024-09-11T08:41:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:36:34.151541+0000", 2024-09-11T08:41:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 277.65465045299999, 2024-09-11T08:41:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038424570000000001, 2024-09-11T08:41:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151541+0000", 2024-09-11T08:41:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151541+0000", 2024-09-11T08:41:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151541+0000", 2024-09-11T08:41:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967296 2024-09-11T08:41:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151595+0000", 2024-09-11T08:41:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4268000000000003e-05 2024-09-11T08:41:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151602+0000", 2024-09-11T08:41:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8959999999999997e-06 2024-09-11T08:41:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151611+0000", 2024-09-11T08:41:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0680000000000003e-06 2024-09-11T08:41:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.154879+0000", 2024-09-11T08:41:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003267505 2024-09-11T08:41:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.154896+0000", 2024-09-11T08:41:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7583000000000001e-05 2024-09-11T08:41:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.155365+0000", 2024-09-11T08:41:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000468548 2024-09-11T08:41:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.155384+0000", 2024-09-11T08:41:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8805999999999999e-05 2024-09-11T08:41:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14156.0:76 1.0 e21/19 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 21'54, mlcod=21'45)", 2024-09-11T08:41:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:36:34.151668+0000", 2024-09-11T08:41:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 277.65452370700001, 2024-09-11T08:41:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038762839999999998, 2024-09-11T08:41:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151668+0000", 2024-09-11T08:41:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151668+0000", 2024-09-11T08:41:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151667+0000", 2024-09-11T08:41:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:41:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151673+0000", 2024-09-11T08:41:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4999999999999999e-06 2024-09-11T08:41:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151684+0000", 2024-09-11T08:41:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.145e-05 2024-09-11T08:41:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.151690+0000", 2024-09-11T08:41:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2859999999999999e-06 2024-09-11T08:41:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.155146+0000", 2024-09-11T08:41:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034560860000000001 2024-09-11T08:41:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.155163+0000", 2024-09-11T08:41:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7045e-05 2024-09-11T08:41:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.155506+0000", 2024-09-11T08:41:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034336799999999998 2024-09-11T08:41:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.155544+0000", 2024-09-11T08:41:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8044999999999999e-05 2024-09-11T08:41:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.849 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.849 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14156.0:90 1.0 e21/19 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 21'64, mlcod=21'60)", 2024-09-11T08:41:11.849 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:36:34.699985+0000", 2024-09-11T08:41:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 277.10620649200001, 2024-09-11T08:41:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001450448, 2024-09-11T08:41:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.699985+0000", 2024-09-11T08:41:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.699985+0000", 2024-09-11T08:41:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.699985+0000", 2024-09-11T08:41:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:41:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.700010+0000", 2024-09-11T08:41:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5890000000000001e-05 2024-09-11T08:41:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.700020+0000", 2024-09-11T08:41:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0739999999999999e-06 2024-09-11T08:41:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.700024+0000", 2024-09-11T08:41:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8339999999999998e-06 2024-09-11T08:41:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.700074+0000", 2024-09-11T08:41:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9876999999999997e-05 2024-09-11T08:41:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.700096+0000", 2024-09-11T08:41:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1392000000000001e-05 2024-09-11T08:41:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.701398+0000", 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013020099999999999 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:34.701436+0000", 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7904999999999997e-05 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14448.0:2 3.12 3:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:38:38.945014+0000", 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 152.861178013, 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013036414, 2024-09-11T08:41:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:41:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14448", 2024-09-11T08:41:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3990203193", 2024-09-11T08:41:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 2 2024-09-11T08:41:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:38.945014+0000", 2024-09-11T08:41:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:38.945014+0000", 2024-09-11T08:41:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:38.945007+0000", 2024-09-11T08:41:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999928 2024-09-11T08:41:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:38.945020+0000", 2024-09-11T08:41:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3237999999999999e-05 2024-09-11T08:41:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:38.945036+0000", 2024-09-11T08:41:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6339000000000001e-05 2024-09-11T08:41:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:38.945045+0000", 2024-09-11T08:41:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4880000000000007e-06 2024-09-11T08:41:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:38.945127+0000", 2024-09-11T08:41:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2123e-05 2024-09-11T08:41:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-11T08:41:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:38.945133+0000", 2024-09-11T08:41:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3330000000000002e-06 2024-09-11T08:41:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:38.949805+0000", 2024-09-11T08:41:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046715200000000002 2024-09-11T08:41:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:41:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:38.949812+0000", 2024-09-11T08:41:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2019999999999998e-06 2024-09-11T08:41:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:38.957968+0000", 2024-09-11T08:41:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0081566 2024-09-11T08:41:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:38.958050+0000", 2024-09-11T08:41:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1496999999999996e-05 2024-09-11T08:41:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14472.0:97 5.3 5:c609908c:::notify.5:head [watch watch cookie 94817299759312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-11T08:41:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:38:42.997794+0000", 2024-09-11T08:41:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 148.80839725199999, 2024-09-11T08:41:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019299160000000001, 2024-09-11T08:41:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14472", 2024-09-11T08:41:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1230553190", 2024-09-11T08:41:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 97 2024-09-11T08:41:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.997794+0000", 2024-09-11T08:41:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.997794+0000", 2024-09-11T08:41:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.997793+0000", 2024-09-11T08:41:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:41:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.997798+0000", 2024-09-11T08:41:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5789999999999997e-06 2024-09-11T08:41:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.997807+0000", 2024-09-11T08:41:11.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5390000000000007e-06 2024-09-11T08:41:11.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.997811+0000", 2024-09-11T08:41:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.561e-06 2024-09-11T08:41:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.998049+0000", 2024-09-11T08:41:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000237982 2024-09-11T08:41:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-11T08:41:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.998119+0000", 2024-09-11T08:41:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9628999999999994e-05 2024-09-11T08:41:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.998777+0000", 2024-09-11T08:41:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00065783400000000004 2024-09-11T08:41:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.998838+0000", 2024-09-11T08:41:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1360999999999997e-05 2024-09-11T08:41:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:41:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.998946+0000", 2024-09-11T08:41:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108374 2024-09-11T08:41:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.999228+0000", 2024-09-11T08:41:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028172100000000002 2024-09-11T08:41:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.999555+0000", 2024-09-11T08:41:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000327146 2024-09-11T08:41:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.999622+0000", 2024-09-11T08:41:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6453999999999998e-05 2024-09-11T08:41:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.999669+0000", 2024-09-11T08:41:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7110000000000001e-05 2024-09-11T08:41:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:42.999724+0000", 2024-09-11T08:41:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5559000000000001e-05 2024-09-11T08:41:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.24352.0:4 4.b 4:d242335b:gc::gc.2:head [create,call version.check_conds in=18b,call rgw_gc.rgw_gc_queue_init in=23b,call version.set in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-11T08:41:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:38:46.004394+0000", 2024-09-11T08:41:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 145.801797932, 2024-09-11T08:41:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001550385, 2024-09-11T08:41:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.24352", 2024-09-11T08:41:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1175084397", 2024-09-11T08:41:11.876 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 4 2024-09-11T08:41:11.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.876 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.004394+0000", 2024-09-11T08:41:11.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.004394+0000", 2024-09-11T08:41:11.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.004391+0000", 2024-09-11T08:41:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999971 2024-09-11T08:41:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.004401+0000", 2024-09-11T08:41:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9170000000000001e-06 2024-09-11T08:41:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.004413+0000", 2024-09-11T08:41:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1778e-05 2024-09-11T08:41:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.004420+0000", 2024-09-11T08:41:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3100000000000003e-06 2024-09-11T08:41:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.004487+0000", 2024-09-11T08:41:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7254000000000004e-05 2024-09-11T08:41:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.004651+0000", 2024-09-11T08:41:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016424000000000001 2024-09-11T08:41:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,4", 2024-09-11T08:41:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.004986+0000", 2024-09-11T08:41:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000334693 2024-09-11T08:41:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.005494+0000", 2024-09-11T08:41:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050761700000000003 2024-09-11T08:41:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.005766+0000", 2024-09-11T08:41:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027211999999999999 2024-09-11T08:41:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.005812+0000", 2024-09-11T08:41:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6468999999999998e-05 2024-09-11T08:41:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.005916+0000", 2024-09-11T08:41:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104067 2024-09-11T08:41:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.005944+0000", 2024-09-11T08:41:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7579000000000001e-05 2024-09-11T08:41:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.24352.0:18 4.3 4:c623b173:gc::gc.16:head [create,call version.check_conds in=18b,call rgw_gc.rgw_gc_queue_init in=23b,call version.set in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-11T08:41:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:38:46.024620+0000", 2024-09-11T08:41:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 145.78157119400001, 2024-09-11T08:41:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001445265, 2024-09-11T08:41:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.24352", 2024-09-11T08:41:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1175084397", 2024-09-11T08:41:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18 2024-09-11T08:41:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.024620+0000", 2024-09-11T08:41:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.024620+0000", 2024-09-11T08:41:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.024618+0000", 2024-09-11T08:41:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999976 2024-09-11T08:41:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.024627+0000", 2024-09-11T08:41:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7700000000000007e-06 2024-09-11T08:41:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.024646+0000", 2024-09-11T08:41:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9332000000000001e-05 2024-09-11T08:41:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.024656+0000", 2024-09-11T08:41:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0037000000000001e-05 2024-09-11T08:41:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.024705+0000", 2024-09-11T08:41:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9008000000000002e-05 2024-09-11T08:41:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.024809+0000", 2024-09-11T08:41:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010367800000000001 2024-09-11T08:41:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,7", 2024-09-11T08:41:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.025030+0000", 2024-09-11T08:41:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022060499999999999 2024-09-11T08:41:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.025425+0000", 2024-09-11T08:41:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00039532100000000002 2024-09-11T08:41:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.025935+0000", 2024-09-11T08:41:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050997599999999998 2024-09-11T08:41:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.025977+0000", 2024-09-11T08:41:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2240000000000002e-05 2024-09-11T08:41:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.026028+0000", 2024-09-11T08:41:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0337000000000003e-05 2024-09-11T08:41:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.026066+0000", 2024-09-11T08:41:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8139999999999997e-05 2024-09-11T08:41:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.24352.0:82 4.3 4:c3a682d4:reshard::reshard.0000000004:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-11T08:41:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:38:46.061611+0000", 2024-09-11T08:41:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 145.74458081099999, 2024-09-11T08:41:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015676520000000001, 2024-09-11T08:41:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.24352", 2024-09-11T08:41:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1175084397", 2024-09-11T08:41:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 82 2024-09-11T08:41:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.061611+0000", 2024-09-11T08:41:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.061611+0000", 2024-09-11T08:41:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.061609+0000", 2024-09-11T08:41:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:41:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.061617+0000", 2024-09-11T08:41:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7000000000000008e-06 2024-09-11T08:41:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.061628+0000", 2024-09-11T08:41:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1724e-05 2024-09-11T08:41:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.061635+0000", 2024-09-11T08:41:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1920000000000002e-06 2024-09-11T08:41:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.061686+0000", 2024-09-11T08:41:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1814000000000002e-05 2024-09-11T08:41:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.061795+0000", 2024-09-11T08:41:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108146 2024-09-11T08:41:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,7", 2024-09-11T08:41:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.061994+0000", 2024-09-11T08:41:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019977999999999999 2024-09-11T08:41:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.062489+0000", 2024-09-11T08:41:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049495099999999998 2024-09-11T08:41:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.063019+0000", 2024-09-11T08:41:11.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052937500000000001 2024-09-11T08:41:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.063093+0000", 2024-09-11T08:41:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3759999999999999e-05 2024-09-11T08:41:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.063139+0000", 2024-09-11T08:41:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6411e-05 2024-09-11T08:41:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.063178+0000", 2024-09-11T08:41:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9499000000000002e-05 2024-09-11T08:41:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.24352.0:92 4.6 4:66924383:reshard::reshard.0000000005:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-11T08:41:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:38:46.065330+0000", 2024-09-11T08:41:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 145.74086179, 2024-09-11T08:41:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001690179, 2024-09-11T08:41:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.24352", 2024-09-11T08:41:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1175084397", 2024-09-11T08:41:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 92 2024-09-11T08:41:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.065330+0000", 2024-09-11T08:41:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.065330+0000", 2024-09-11T08:41:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.065329+0000", 2024-09-11T08:41:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:41:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.065334+0000", 2024-09-11T08:41:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2909999999999998e-06 2024-09-11T08:41:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.065344+0000", 2024-09-11T08:41:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9070000000000002e-06 2024-09-11T08:41:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.065349+0000", 2024-09-11T08:41:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7809999999999996e-06 2024-09-11T08:41:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.065400+0000", 2024-09-11T08:41:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1122000000000003e-05 2024-09-11T08:41:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.065470+0000", 2024-09-11T08:41:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9703999999999995e-05 2024-09-11T08:41:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:41:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.065580+0000", 2024-09-11T08:41:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110371 2024-09-11T08:41:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.066031+0000", 2024-09-11T08:41:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045074699999999999 2024-09-11T08:41:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.066273+0000", 2024-09-11T08:41:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024166300000000001 2024-09-11T08:41:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.066942+0000", 2024-09-11T08:41:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066972099999999997 2024-09-11T08:41:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.066997+0000", 2024-09-11T08:41:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4836999999999997e-05 2024-09-11T08:41:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.067020+0000", 2024-09-11T08:41:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2773e-05 2024-09-11T08:41:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.24352.0:105 4.16 4:6849fd06:gc::gc.17:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-11T08:41:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:38:46.070306+0000", 2024-09-11T08:41:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 145.73588590200001, 2024-09-11T08:41:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001490503, 2024-09-11T08:41:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.24352", 2024-09-11T08:41:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1175084397", 2024-09-11T08:41:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105 2024-09-11T08:41:11.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.908 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.070306+0000", 2024-09-11T08:41:11.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.070306+0000", 2024-09-11T08:41:11.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.070305+0000", 2024-09-11T08:41:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:41:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.070311+0000", 2024-09-11T08:41:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8780000000000003e-06 2024-09-11T08:41:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.070325+0000", 2024-09-11T08:41:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3991e-05 2024-09-11T08:41:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.070332+0000", 2024-09-11T08:41:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7130000000000005e-06 2024-09-11T08:41:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.070362+0000", 2024-09-11T08:41:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9475000000000002e-05 2024-09-11T08:41:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.070430+0000", 2024-09-11T08:41:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8738999999999994e-05 2024-09-11T08:41:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3,7", 2024-09-11T08:41:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.070587+0000", 2024-09-11T08:41:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015694299999999999 2024-09-11T08:41:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.070981+0000", 2024-09-11T08:41:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00039361000000000002 2024-09-11T08:41:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.071170+0000", 2024-09-11T08:41:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018907 2024-09-11T08:41:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.071714+0000", 2024-09-11T08:41:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054438000000000002 2024-09-11T08:41:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.071757+0000", 2024-09-11T08:41:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2670000000000003e-05 2024-09-11T08:41:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.071796+0000", 2024-09-11T08:41:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9020999999999998e-05 2024-09-11T08:41:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.913 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.24352.0:108 4.16 4:6dc3b799:reshard::reshard.0000000006:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-11T08:41:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:38:46.071114+0000", 2024-09-11T08:41:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 145.73507799399999, 2024-09-11T08:41:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001374576, 2024-09-11T08:41:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.24352", 2024-09-11T08:41:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1175084397", 2024-09-11T08:41:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 108 2024-09-11T08:41:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.071114+0000", 2024-09-11T08:41:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.071114+0000", 2024-09-11T08:41:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.071113+0000", 2024-09-11T08:41:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:41:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.071120+0000", 2024-09-11T08:41:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7329999999999997e-06 2024-09-11T08:41:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.071129+0000", 2024-09-11T08:41:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7100000000000002e-06 2024-09-11T08:41:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.071188+0000", 2024-09-11T08:41:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9015000000000003e-05 2024-09-11T08:41:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.071222+0000", 2024-09-11T08:41:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3507999999999998e-05 2024-09-11T08:41:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.071296+0000", 2024-09-11T08:41:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4301999999999994e-05 2024-09-11T08:41:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3,7", 2024-09-11T08:41:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.071452+0000", 2024-09-11T08:41:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156136 2024-09-11T08:41:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.071815+0000", 2024-09-11T08:41:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000363011 2024-09-11T08:41:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.072065+0000", 2024-09-11T08:41:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025003500000000001 2024-09-11T08:41:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.072415+0000", 2024-09-11T08:41:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035013299999999998 2024-09-11T08:41:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.072458+0000", 2024-09-11T08:41:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2978000000000001e-05 2024-09-11T08:41:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.072488+0000", 2024-09-11T08:41:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9828999999999999e-05 2024-09-11T08:41:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.24352.0:123 4.6 4:64760185:::obj_delete_at_hint.0000000010:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-11T08:41:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:38:46.076338+0000", 2024-09-11T08:41:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 145.729853623, 2024-09-11T08:41:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001349278, 2024-09-11T08:41:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.24352", 2024-09-11T08:41:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1175084397", 2024-09-11T08:41:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 123 2024-09-11T08:41:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.076338+0000", 2024-09-11T08:41:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.076338+0000", 2024-09-11T08:41:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.076337+0000", 2024-09-11T08:41:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:41:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.076343+0000", 2024-09-11T08:41:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6489999999999996e-06 2024-09-11T08:41:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.076357+0000", 2024-09-11T08:41:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4681999999999999e-05 2024-09-11T08:41:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.076363+0000", 2024-09-11T08:41:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1589999999999998e-06 2024-09-11T08:41:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.076434+0000", 2024-09-11T08:41:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0607999999999999e-05 2024-09-11T08:41:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.076595+0000", 2024-09-11T08:41:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160946 2024-09-11T08:41:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:41:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.076766+0000", 2024-09-11T08:41:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017143599999999999 2024-09-11T08:41:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.077186+0000", 2024-09-11T08:41:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00041912900000000002 2024-09-11T08:41:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.077434+0000", 2024-09-11T08:41:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024858200000000001 2024-09-11T08:41:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.077610+0000", 2024-09-11T08:41:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017596500000000001 2024-09-11T08:41:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.077655+0000", 2024-09-11T08:41:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5346999999999998e-05 2024-09-11T08:41:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.077687+0000", 2024-09-11T08:41:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1794000000000001e-05 2024-09-11T08:41:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.24352.0:364 4.1e 4:7af5a05b:::obj_delete_at_hint.0000000044:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-11T08:41:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:38:46.168933+0000", 2024-09-11T08:41:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 145.637259038, 2024-09-11T08:41:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002213156, 2024-09-11T08:41:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.24352", 2024-09-11T08:41:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1175084397", 2024-09-11T08:41:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 364 2024-09-11T08:41:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.168933+0000", 2024-09-11T08:41:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.168933+0000", 2024-09-11T08:41:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.168932+0000", 2024-09-11T08:41:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:41:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.168937+0000", 2024-09-11T08:41:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8779999999999997e-06 2024-09-11T08:41:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.168948+0000", 2024-09-11T08:41:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1545e-05 2024-09-11T08:41:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.168954+0000", 2024-09-11T08:41:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6790000000000002e-06 2024-09-11T08:41:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.168996+0000", 2024-09-11T08:41:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2265999999999998e-05 2024-09-11T08:41:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.169047+0000", 2024-09-11T08:41:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0683000000000002e-05 2024-09-11T08:41:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3,7", 2024-09-11T08:41:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.169143+0000", 2024-09-11T08:41:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6192000000000003e-05 2024-09-11T08:41:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.169729+0000", 2024-09-11T08:41:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00058565099999999997 2024-09-11T08:41:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.171018+0000", 2024-09-11T08:41:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001289503 2024-09-11T08:41:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.171061+0000", 2024-09-11T08:41:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2729999999999999e-05 2024-09-11T08:41:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.171107+0000", 2024-09-11T08:41:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6069999999999998e-05 2024-09-11T08:41:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.171146+0000", 2024-09-11T08:41:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8692999999999997e-05 2024-09-11T08:41:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.24352.0:365 4.16 4:6f6aba0e:::obj_delete_at_hint.0000000045:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-11T08:41:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:38:46.171243+0000", 2024-09-11T08:41:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 145.634949019, 2024-09-11T08:41:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026400199999999999, 2024-09-11T08:41:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.24352", 2024-09-11T08:41:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1175084397", 2024-09-11T08:41:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 365 2024-09-11T08:41:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.171243+0000", 2024-09-11T08:41:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.171243+0000", 2024-09-11T08:41:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.171242+0000", 2024-09-11T08:41:11.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:41:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.171248+0000", 2024-09-11T08:41:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3110000000000002e-06 2024-09-11T08:41:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.171264+0000", 2024-09-11T08:41:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5834000000000001e-05 2024-09-11T08:41:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.171270+0000", 2024-09-11T08:41:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9560000000000002e-06 2024-09-11T08:41:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.171308+0000", 2024-09-11T08:41:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8497000000000001e-05 2024-09-11T08:41:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.171436+0000", 2024-09-11T08:41:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000128336 2024-09-11T08:41:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3,7", 2024-09-11T08:41:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.171612+0000", 2024-09-11T08:41:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017508000000000001 2024-09-11T08:41:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.172145+0000", 2024-09-11T08:41:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00053349300000000001 2024-09-11T08:41:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.173679+0000", 2024-09-11T08:41:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001533988 2024-09-11T08:41:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.173796+0000", 2024-09-11T08:41:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116852 2024-09-11T08:41:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.173845+0000", 2024-09-11T08:41:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9227000000000001e-05 2024-09-11T08:41:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:38:46.173883+0000", 2024-09-11T08:41:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7471000000000001e-05 2024-09-11T08:41:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:11.957 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:41:11.990 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:11.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:11.991 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:12.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:12.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:12.137 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:12.137 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:12.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:12.278 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:12.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:12.279 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:12.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:12.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:12 smithi145 ceph-mon[33384]: pgmap v84: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:12.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:12 smithi145 ceph-mon[33384]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:41:12.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:12.418 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:12.418 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:12.519 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:12.537 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:12 smithi092 ceph-mon[29647]: pgmap v84: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:12.537 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:12 smithi092 ceph-mon[29647]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:41:12.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:12 smithi092 ceph-mon[41114]: pgmap v84: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:12.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:12 smithi092 ceph-mon[41114]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:41:12.548 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:12.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:12.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:12.650 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:12.679 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:12.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:12.680 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:12.681 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 1 16 3135 3119 194.927 194.938 0.00275544 0.00510886 2024-09-11T08:41:12.780 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:12.807 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:12.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:12.808 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:12.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-11T08:41:12.860 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:12.908 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:12.936 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:12.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:12.937 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:13.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:13.079 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:13.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:13.081 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:13.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:41:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:41:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:13.181 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:13.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:13.209 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:13.209 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:13.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:13.347 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:13.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:13.348 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:13.449 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:13.487 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:13.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:13.489 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:13.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:41:13.590 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:13.615 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:13.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:13.616 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:13.681 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 2 16 5849 5833 182.229 169.625 0.0135784 0.00547192 2024-09-11T08:41:13.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:13.761 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:13.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:13.762 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:13.863 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:13.891 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:13.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:13.892 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:13.992 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:14.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:14.032 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:14.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:14.172 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:14.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:14.173 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:14.187 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:14 smithi092 ceph-mon[41114]: pgmap v85: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:14.187 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:14 smithi092 ceph-mon[29647]: pgmap v85: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:14.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:14.311 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:14.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:14.312 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:14.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:14 smithi145 ceph-mon[33384]: pgmap v85: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:14.413 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:14.451 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:14.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:14.452 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:14.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:14.592 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:14.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:14.593 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:14.681 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 3 15 8880 8865 184.646 189.5 0.00466563 0.00540546 2024-09-11T08:41:14.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:14.735 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:14.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:14.736 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:14.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:14.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:14.877 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:14.877 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:14.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:15.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:15.017 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:15.017 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:15.118 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:15.145 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:15.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:15.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:15.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:15.285 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:15.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:15.286 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:15.387 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:15.417 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:15.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:15.418 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:15.519 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:15.549 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:15.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:15.550 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:15.650 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:15.678 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:15.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:15.679 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:15.681 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 4 16 11999 11983 187.199 194.875 0.00677318 0.0053316 2024-09-11T08:41:15.744 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:15.780 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:15.811 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:15.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:15.812 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:15.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:15.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:15.958 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:15.958 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:16.059 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:16.087 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:16.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:16.089 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:16.190 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:16.229 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:16.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:16.230 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:16.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:16 smithi092 ceph-mon[41114]: pgmap v86: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:41:16.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:16 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:16.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:16 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:16.281 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:16 smithi092 ceph-mon[29647]: pgmap v86: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:41:16.281 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:16 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:16.281 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:16 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:16.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:16.370 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:16.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:16.371 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:16.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:41:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:41:16.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:16 smithi145 ceph-mon[33384]: pgmap v86: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:41:16.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:16 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:16.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:16 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:16.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:16.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:16.511 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:16.511 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:16.588 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:41:16.589 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:41:16.612 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:16.624 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:41:16.624 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:41:16.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:16.651 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:16.651 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:16.662 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:41:16.663 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:41:16.682 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 5 16 14755 14739 184.184 172.25 0.010676 0.00541983 2024-09-11T08:41:16.703 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:41:16.703 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:41:16.734 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:41:16.734 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:41:16.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:16.767 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:41:16.767 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:41:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:16.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:16.790 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:16.798 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:41:16.798 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:41:16.828 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:41:16.829 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:41:16.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:16.927 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:16.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:16.928 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:17.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:17.069 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:17.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:17.070 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:17.170 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:17.199 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:17.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:17.200 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:17.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:17 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:17.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:17 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/654567140' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:41:17.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:17 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:17.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:17 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/654567140' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:41:17.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:17.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:17.341 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:17.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:17 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:17.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:17 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/654567140' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:41:17.442 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:17.470 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:17.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:17.471 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:17.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:17.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:17.611 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:17.611 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:17.682 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 6 15 17494 17479 182.026 171.25 0.00396363 0.00548568 2024-09-11T08:41:17.712 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:17.740 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:17.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:17.741 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:17.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:17.882 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:17.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:17.883 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:17.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:18.019 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:18.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:18.020 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:18.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:18.157 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:18.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:18.158 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:18.259 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:18.275 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:41:18.275 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:18 smithi092 ceph-mon[29647]: pgmap v87: 148 pgs: 148 active+clean; 808 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-11T08:41:18.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:18 smithi092 ceph-mon[41114]: pgmap v87: 148 pgs: 148 active+clean; 808 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-11T08:41:18.284 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:41:18.287 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:18.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:18.288 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:18.389 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:18.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:18 smithi145 ceph-mon[33384]: pgmap v87: 148 pgs: 148 active+clean; 808 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-11T08:41:18.424 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:18.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:18.425 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:18.526 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:18.554 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:18.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:18.555 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:18.656 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:18.682 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 7 15 20027 20012 178.637 158.312 0.00235163 0.00559183 2024-09-11T08:41:18.684 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:18.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:18.685 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:18.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:18.825 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:18.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:18.826 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:18.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:18.961 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:18.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:18.962 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:19.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:19.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:19.091 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:19.091 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:19.192 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:19.220 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:19.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:19.221 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:19.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:19.365 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:19.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:19.366 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:19.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:19 smithi145 ceph-mon[33384]: from='client.24640 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:41:19.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:19.498 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:19.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:19.499 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:19.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:19 smithi092 ceph-mon[29647]: from='client.24640 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:41:19.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:19 smithi092 ceph-mon[41114]: from='client.24640 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:41:19.601 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:19.621 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:19.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:19.622 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:19.682 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 8 16 22378 22362 174.665 146.875 0.00513604 0.00571974 2024-09-11T08:41:19.723 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:19.751 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:19.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:19.752 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:19.853 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:19.880 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:19.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:19.882 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:19.982 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:20.003 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:20.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:20.004 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:20.104 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:20.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:20.136 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:20.136 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:20.237 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:20.266 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:20.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:20.267 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:20.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:20.405 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:20.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:20.406 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:20.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:20 smithi145 ceph-mon[33384]: pgmap v88: 148 pgs: 148 active+clean; 808 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-11T08:41:20.507 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:20.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:20.528 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:20.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:20 smithi092 ceph-mon[29647]: pgmap v88: 148 pgs: 148 active+clean; 808 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-11T08:41:20.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:20 smithi092 ceph-mon[41114]: pgmap v88: 148 pgs: 148 active+clean; 808 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-11T08:41:20.629 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:20.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:20.667 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:20.667 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:20.682 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 9 16 24412 24396 169.382 127.125 0.00947101 0.00589686 2024-09-11T08:41:20.768 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:20.803 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:20.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:20.804 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:20.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:20.949 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:20.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:20.950 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:21.051 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:21.080 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:21.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:21.081 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:21.182 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:21.212 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:21.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:21.213 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:21.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:21.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:21.348 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:21.348 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:21.449 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:21.477 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:21.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:21.479 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:21.580 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:21.609 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:21.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:21.610 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:21.683 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 10 16 26431 26415 165.062 126.188 0.00160436 0.00602909 2024-09-11T08:41:21.711 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:21.732 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:21.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:21.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:21.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:41:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:41:21] "GET /metrics HTTP/1.1" 200 222486 "" "Prometheus/2.43.0" 2024-09-11T08:41:21.834 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:21.862 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:21.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:21.864 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:21.965 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:21.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:21.994 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:21.994 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:22.095 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:22.132 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:22.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:22.133 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:22.234 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:22.265 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:22.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:22.266 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:22.367 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:22.401 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:22.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:22.402 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:22.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:22 smithi092 ceph-mon[29647]: pgmap v89: 148 pgs: 148 active+clean; 1.2 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 1.5 KiB/s rd, 116 MiB/s wr, 1.85k op/s 2024-09-11T08:41:22.405 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:22 smithi092 ceph-mon[41114]: pgmap v89: 148 pgs: 148 active+clean; 1.2 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 1.5 KiB/s rd, 116 MiB/s wr, 1.85k op/s 2024-09-11T08:41:22.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:22 smithi145 ceph-mon[33384]: pgmap v89: 148 pgs: 148 active+clean; 1.2 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 1.5 KiB/s rd, 116 MiB/s wr, 1.85k op/s 2024-09-11T08:41:22.503 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:22.525 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:22.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:22.526 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:22.626 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:22.657 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:22.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:22.658 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:22.683 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 11 15 28149 28134 159.823 107.438 0.00538792 0.00625114 2024-09-11T08:41:22.758 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:22.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:22.788 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:22.789 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:22.889 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:22.910 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:22.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:22.911 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:23.012 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:23.044 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:23.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:23.045 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:23.146 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:23.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:41:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:41:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:23.182 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:23.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:23.183 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:23.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:23.323 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:23.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:23.324 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:23.425 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:23.456 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:23.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:23.457 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:23.558 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:23.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:23.580 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:23.581 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:23.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:23.683 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 12 16 29761 29745 154.895 100.688 0.0034742 0.00645046 2024-09-11T08:41:23.716 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:23.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:23.717 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:23.818 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:23.854 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:23.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:23.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:23.957 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:23.977 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:23.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:23.978 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:24.079 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:24.107 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:24.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:24.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:24.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:24.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:24.247 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:24.247 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:24.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:24 smithi092 ceph-mon[29647]: pgmap v90: 148 pgs: 148 active+clean; 1.5 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 131 MiB/s wr, 2.09k op/s 2024-09-11T08:41:24.323 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:24 smithi092 ceph-mon[41114]: pgmap v90: 148 pgs: 148 active+clean; 1.5 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 131 MiB/s wr, 2.09k op/s 2024-09-11T08:41:24.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:24.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:24.385 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:24.385 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:24.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:24 smithi145 ceph-mon[33384]: pgmap v90: 148 pgs: 148 active+clean; 1.5 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 131 MiB/s wr, 2.09k op/s 2024-09-11T08:41:24.486 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:24.525 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:24.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:24.526 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:24.627 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:24.654 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:24.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:24.655 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:24.683 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 13 15 31755 31740 152.571 124.688 0.0110673 0.00654908 2024-09-11T08:41:24.756 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:24.784 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:24.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:24.785 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:24.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:24.926 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:24.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:24.927 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:25.028 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:25.059 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:25.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:25.060 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:25.161 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:25.189 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:25.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:25.190 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:25.291 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:25.321 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:25.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:25.322 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:25.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:25.458 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:25.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:25.460 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:25.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:25.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:25.600 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:25.600 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:25.683 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 14 16 33948 33932 151.458 137 0.00722425 0.00659782 2024-09-11T08:41:25.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:25.739 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:25.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:25.740 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:25.841 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:25.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:25.871 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:25.872 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:25.972 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:26.000 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:26.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:26.001 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:26.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:26.142 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:26.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:26.143 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:26.244 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:26.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:26.275 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:26.275 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:26.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:26.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:26 smithi145 ceph-mon[33384]: pgmap v91: 148 pgs: 148 active+clean; 1.5 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 131 MiB/s wr, 2.09k op/s 2024-09-11T08:41:26.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:41:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:41:26.414 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:26.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:26.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:26.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:26 smithi092 ceph-mon[29647]: pgmap v91: 148 pgs: 148 active+clean; 1.5 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 131 MiB/s wr, 2.09k op/s 2024-09-11T08:41:26.458 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:26 smithi092 ceph-mon[41114]: pgmap v91: 148 pgs: 148 active+clean; 1.5 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 131 MiB/s wr, 2.09k op/s 2024-09-11T08:41:26.497 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-09-11T08:41:17.028001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":847321308,"num_objects":13116,"num_object_clones":0,"num_object_copies":39348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13116,"num_whiteouts":0,"num_read":1003,"num_read_kb":860,"num_write":13463,"num_write_kb":828551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13507,"ondisk_log_size":13507,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":4649044,"kb_used_data":2245140,"kb_used_omap":13,"kb_used_meta":2403698,"kb_avail":745312172,"statfs":{"total":767960285184,"available":763199664128,"internally_reserved":0,"allocated":2299023360,"data_stored":2295828844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2461387456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,7,8,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":846725120,"num_objects":12920,"num_object_clones":0,"num_object_copies":38760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12920,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":12920,"num_write_kb":826880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002684"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.097364+0000","last_change":"2024-09-11T08:39:47.339366+0000","last_active":"2024-09-11T08:41:09.097364+0000","last_peered":"2024-09-11T08:41:09.097364+0000","last_clean":"2024-09-11T08:41:09.097364+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:41:09.097364+0000","last_undegraded":"2024-09-11T08:41:09.097364+0000","last_fullsized":"2024-09-11T08:41:09.097364+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.339312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.339312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001323095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.522758+0000","last_change":"2024-09-11T08:39:56.666764+0000","last_active":"2024-09-11T08:41:09.522758+0000","last_peered":"2024-09-11T08:41:09.522758+0000","last_clean":"2024-09-11T08:41:09.522758+0000","last_became_active":"2024-09-11T08:38:40.973398+0000","last_became_peered":"2024-09-11T08:38:40.973398+0000","last_unstale":"2024-09-11T08:41:09.522758+0000","last_undegraded":"2024-09-11T08:41:09.522758+0000","last_fullsized":"2024-09-11T08:41:09.522758+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.666703+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.544222+0000","scrub_duration":0.0041022000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.228570+0000","last_change":"2024-09-11T08:40:12.325099+0000","last_active":"2024-09-11T08:41:09.228570+0000","last_peered":"2024-09-11T08:41:09.228570+0000","last_clean":"2024-09-11T08:41:09.228570+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:41:09.228570+0000","last_undegraded":"2024-09-11T08:41:09.228570+0000","last_fullsized":"2024-09-11T08:41:09.228570+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:12.325037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:12.325037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:13.497524+0000","scrub_duration":0.001599736,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.522840+0000","last_change":"2024-09-11T08:38:44.992111+0000","last_active":"2024-09-11T08:41:09.522840+0000","last_peered":"2024-09-11T08:41:09.522840+0000","last_clean":"2024-09-11T08:41:09.522840+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:41:09.522840+0000","last_undegraded":"2024-09-11T08:41:09.522840+0000","last_fullsized":"2024-09-11T08:41:09.522840+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:38:43.973425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328147+0000","last_change":"2024-09-11T08:40:06.173950+0000","last_active":"2024-09-11T08:41:09.328147+0000","last_peered":"2024-09-11T08:41:09.328147+0000","last_clean":"2024-09-11T08:41:09.328147+0000","last_became_active":"2024-09-11T08:38:45.184215+0000","last_became_peered":"2024-09-11T08:38:45.184215+0000","last_unstale":"2024-09-11T08:41:09.328147+0000","last_undegraded":"2024-09-11T08:41:09.328147+0000","last_fullsized":"2024-09-11T08:41:09.328147+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:06.173882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:06.173882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:17.036842+0000","scrub_duration":0.001730875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328942+0000","last_change":"2024-09-11T08:39:58.161055+0000","last_active":"2024-09-11T08:41:09.328942+0000","last_peered":"2024-09-11T08:41:09.328942+0000","last_clean":"2024-09-11T08:41:09.328942+0000","last_became_active":"2024-09-11T08:38:38.949136+0000","last_became_peered":"2024-09-11T08:38:38.949136+0000","last_unstale":"2024-09-11T08:41:09.328942+0000","last_undegraded":"2024-09-11T08:41:09.328942+0000","last_fullsized":"2024-09-11T08:41:09.328942+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:58.160984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.160984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015109679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":84,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328487+0000","last_change":"2024-09-11T08:39:45.997873+0000","last_active":"2024-09-11T08:41:09.328487+0000","last_peered":"2024-09-11T08:41:09.328487+0000","last_clean":"2024-09-11T08:41:09.328487+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:41:09.328487+0000","last_undegraded":"2024-09-11T08:41:09.328487+0000","last_fullsized":"2024-09-11T08:41:09.328487+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.997803+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.534414+0000","scrub_duration":0.0046870369999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.522677+0000","last_change":"2024-09-11T08:40:44.721741+0000","last_active":"2024-09-11T08:41:09.522677+0000","last_peered":"2024-09-11T08:41:09.522677+0000","last_clean":"2024-09-11T08:41:09.522677+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:41:09.522677+0000","last_undegraded":"2024-09-11T08:41:09.522677+0000","last_fullsized":"2024-09-11T08:41:09.522677+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:44.721659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:44.721659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:06.347417+0000","scrub_duration":0.001413777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.487047+0000","last_change":"2024-09-11T08:40:08.416577+0000","last_active":"2024-09-11T08:41:09.487047+0000","last_peered":"2024-09-11T08:41:09.487047+0000","last_clean":"2024-09-11T08:41:09.487047+0000","last_became_active":"2024-09-11T08:38:38.948930+0000","last_became_peered":"2024-09-11T08:38:38.948930+0000","last_unstale":"2024-09-11T08:41:09.487047+0000","last_undegraded":"2024-09-11T08:41:09.487047+0000","last_fullsized":"2024-09-11T08:41:09.487047+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:08.416522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.416522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:27.445091+0000","scrub_duration":0.001330005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.523001+0000","last_change":"2024-09-11T08:39:46.620700+0000","last_active":"2024-09-11T08:41:09.523001+0000","last_peered":"2024-09-11T08:41:09.523001+0000","last_clean":"2024-09-11T08:41:09.523001+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:41:09.523001+0000","last_undegraded":"2024-09-11T08:41:09.523001+0000","last_fullsized":"2024-09-11T08:41:09.523001+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:39:46.620624+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:10.984564+0000","scrub_duration":0.0044132319999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.486534+0000","last_change":"2024-09-11T08:40:04.450336+0000","last_active":"2024-09-11T08:41:09.486534+0000","last_peered":"2024-09-11T08:41:09.486534+0000","last_clean":"2024-09-11T08:41:09.486534+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:41:09.486534+0000","last_undegraded":"2024-09-11T08:41:09.486534+0000","last_fullsized":"2024-09-11T08:41:09.486534+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:04.450282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:04.450282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.499239+0000","scrub_duration":0.001689433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.096637+0000","last_change":"2024-09-11T08:39:51.280365+0000","last_active":"2024-09-11T08:41:09.096637+0000","last_peered":"2024-09-11T08:41:09.096637+0000","last_clean":"2024-09-11T08:41:09.096637+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:41:09.096637+0000","last_undegraded":"2024-09-11T08:41:09.096637+0000","last_fullsized":"2024-09-11T08:41:09.096637+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.280310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.280310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001634777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.486709+0000","last_change":"2024-09-11T08:40:12.372128+0000","last_active":"2024-09-11T08:41:09.486709+0000","last_peered":"2024-09-11T08:41:09.486709+0000","last_clean":"2024-09-11T08:41:09.486709+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:41:09.486709+0000","last_undegraded":"2024-09-11T08:41:09.486709+0000","last_fullsized":"2024-09-11T08:41:09.486709+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:12.372043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:12.372043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.497179+0000","scrub_duration":0.001440623,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.522948+0000","last_change":"2024-09-11T08:40:42.708917+0000","last_active":"2024-09-11T08:41:09.522948+0000","last_peered":"2024-09-11T08:41:09.522948+0000","last_clean":"2024-09-11T08:41:09.522948+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:41:09.522948+0000","last_undegraded":"2024-09-11T08:41:09.522948+0000","last_fullsized":"2024-09-11T08:41:09.522948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:40:42.708860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:42.708860+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:04.395693+0000","scrub_duration":0.001717023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.823037+0000","last_change":"2024-09-11T08:39:51.166114+0000","last_active":"2024-09-11T08:41:09.823037+0000","last_peered":"2024-09-11T08:41:09.823037+0000","last_clean":"2024-09-11T08:41:09.823037+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:41:09.823037+0000","last_undegraded":"2024-09-11T08:41:09.823037+0000","last_fullsized":"2024-09-11T08:41:09.823037+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:39:51.166051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.570790+0000","scrub_duration":0.0016440420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.486345+0000","last_change":"2024-09-11T08:40:41.248743+0000","last_active":"2024-09-11T08:41:09.486345+0000","last_peered":"2024-09-11T08:41:09.486345+0000","last_clean":"2024-09-11T08:41:09.486345+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:41:09.486345+0000","last_undegraded":"2024-09-11T08:41:09.486345+0000","last_fullsized":"2024-09-11T08:41:09.486345+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:41.248685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:41.248685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:10.549708+0000","scrub_duration":0.0014710680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.522617+0000","last_change":"2024-09-11T08:39:59.606521+0000","last_active":"2024-09-11T08:41:09.522617+0000","last_peered":"2024-09-11T08:41:09.522617+0000","last_clean":"2024-09-11T08:41:09.522617+0000","last_became_active":"2024-09-11T08:38:45.183364+0000","last_became_peered":"2024-09-11T08:38:45.183364+0000","last_unstale":"2024-09-11T08:41:09.522617+0000","last_undegraded":"2024-09-11T08:41:09.522617+0000","last_fullsized":"2024-09-11T08:41:09.522617+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:59.606458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:59.606458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.547769+0000","scrub_duration":0.0012065960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.096969+0000","last_change":"2024-09-11T08:40:30.632483+0000","last_active":"2024-09-11T08:41:09.096969+0000","last_peered":"2024-09-11T08:41:09.096969+0000","last_clean":"2024-09-11T08:41:09.096969+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:41:09.096969+0000","last_undegraded":"2024-09-11T08:41:09.096969+0000","last_fullsized":"2024-09-11T08:41:09.096969+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:30.632430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:30.632430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:45.188817+0000","scrub_duration":0.001480947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":85,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.096465+0000","last_change":"2024-09-11T08:39:50.472419+0000","last_active":"2024-09-11T08:41:09.096465+0000","last_peered":"2024-09-11T08:41:09.096465+0000","last_clean":"2024-09-11T08:41:09.096465+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:41:09.096465+0000","last_undegraded":"2024-09-11T08:41:09.096465+0000","last_fullsized":"2024-09-11T08:41:09.096465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.472368+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049809520000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.229039+0000","last_change":"2024-09-11T08:40:58.209767+0000","last_active":"2024-09-11T08:41:09.229039+0000","last_peered":"2024-09-11T08:41:09.229039+0000","last_clean":"2024-09-11T08:41:09.229039+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:41:09.229039+0000","last_undegraded":"2024-09-11T08:41:09.229039+0000","last_fullsized":"2024-09-11T08:41:09.229039+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.209713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.209713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:03.759075+0000","scrub_duration":0.001536363,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328080+0000","last_change":"2024-09-11T08:40:48.329989+0000","last_active":"2024-09-11T08:41:09.328080+0000","last_peered":"2024-09-11T08:41:09.328080+0000","last_clean":"2024-09-11T08:41:09.328080+0000","last_became_active":"2024-09-11T08:38:45.182763+0000","last_became_peered":"2024-09-11T08:38:45.182763+0000","last_unstale":"2024-09-11T08:41:09.328080+0000","last_undegraded":"2024-09-11T08:41:09.328080+0000","last_fullsized":"2024-09-11T08:41:09.328080+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:40:48.329879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:48.329879+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:06.299928+0000","scrub_duration":0.002306252,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.523078+0000","last_change":"2024-09-11T08:40:35.714447+0000","last_active":"2024-09-11T08:41:09.523078+0000","last_peered":"2024-09-11T08:41:09.523078+0000","last_clean":"2024-09-11T08:41:09.523078+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:41:09.523078+0000","last_undegraded":"2024-09-11T08:41:09.523078+0000","last_fullsized":"2024-09-11T08:41:09.523078+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:35.714393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:35.714393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:40.832830+0000","scrub_duration":0.0013591110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":75,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328749+0000","last_change":"2024-09-11T08:39:55.125449+0000","last_active":"2024-09-11T08:41:09.328749+0000","last_peered":"2024-09-11T08:41:09.328749+0000","last_clean":"2024-09-11T08:41:09.328749+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:41:09.328749+0000","last_undegraded":"2024-09-11T08:41:09.328749+0000","last_fullsized":"2024-09-11T08:41:09.328749+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:55.125394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.125394+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.857707+0000","scrub_duration":0.0016833810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.523203+0000","last_change":"2024-09-11T08:40:01.585472+0000","last_active":"2024-09-11T08:41:09.523203+0000","last_peered":"2024-09-11T08:41:09.523203+0000","last_clean":"2024-09-11T08:41:09.523203+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:41:09.523203+0000","last_undegraded":"2024-09-11T08:41:09.523203+0000","last_fullsized":"2024-09-11T08:41:09.523203+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:01.585411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:01.585411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.937039+0000","scrub_duration":0.0021220179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.823251+0000","last_change":"2024-09-11T08:40:10.113729+0000","last_active":"2024-09-11T08:41:09.823251+0000","last_peered":"2024-09-11T08:41:09.823251+0000","last_clean":"2024-09-11T08:41:09.823251+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:41:09.823251+0000","last_undegraded":"2024-09-11T08:41:09.823251+0000","last_fullsized":"2024-09-11T08:41:09.823251+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:40:10.113671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:10.113671+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:33.267918+0000","scrub_duration":0.001621302,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.228632+0000","last_change":"2024-09-11T08:40:02.458618+0000","last_active":"2024-09-11T08:41:09.228632+0000","last_peered":"2024-09-11T08:41:09.228632+0000","last_clean":"2024-09-11T08:41:09.228632+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:41:09.228632+0000","last_undegraded":"2024-09-11T08:41:09.228632+0000","last_fullsized":"2024-09-11T08:41:09.228632+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:02.458565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.458565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.247079+0000","scrub_duration":0.001315568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":68,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.096948+0000","last_change":"2024-09-11T08:40:18.467638+0000","last_active":"2024-09-11T08:41:09.096948+0000","last_peered":"2024-09-11T08:41:09.096948+0000","last_clean":"2024-09-11T08:41:09.096948+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:41:09.096948+0000","last_undegraded":"2024-09-11T08:41:09.096948+0000","last_fullsized":"2024-09-11T08:41:09.096948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:40:18.467574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:18.467574+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:37.429174+0000","scrub_duration":0.0040976820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":92,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:11.661321+0000","last_change":"2024-09-11T08:40:12.584703+0000","last_active":"2024-09-11T08:41:11.661321+0000","last_peered":"2024-09-11T08:41:11.661321+0000","last_clean":"2024-09-11T08:41:11.661321+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:41:11.661321+0000","last_undegraded":"2024-09-11T08:41:11.661321+0000","last_fullsized":"2024-09-11T08:41:11.661321+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:12.584631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:12.584631+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:27.946233+0000","scrub_duration":0.0015754790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"92'763","reported_seq":775,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:15.981408+0000","last_change":"2024-09-11T08:40:59.754739+0000","last_active":"2024-09-11T08:41:15.981408+0000","last_peered":"2024-09-11T08:41:15.981408+0000","last_clean":"2024-09-11T08:41:15.981408+0000","last_became_active":"2024-09-11T08:40:59.754455+0000","last_became_peered":"2024-09-11T08:40:59.754455+0000","last_unstale":"2024-09-11T08:41:15.981408+0000","last_undegraded":"2024-09-11T08:41:15.981408+0000","last_fullsized":"2024-09-11T08:41:15.981408+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:00.369832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50003968,"num_objects":763,"num_object_clones":0,"num_object_copies":2289,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":763,"num_write_kb":48832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":78,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.229155+0000","last_change":"2024-09-11T08:40:16.290927+0000","last_active":"2024-09-11T08:41:09.229155+0000","last_peered":"2024-09-11T08:41:09.229155+0000","last_clean":"2024-09-11T08:41:09.229155+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:41:09.229155+0000","last_undegraded":"2024-09-11T08:41:09.229155+0000","last_fullsized":"2024-09-11T08:41:09.229155+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-11T08:40:16.290866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:16.290866+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:46.083697+0000","scrub_duration":0.0021215069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328385+0000","last_change":"2024-09-11T08:40:23.048021+0000","last_active":"2024-09-11T08:41:09.328385+0000","last_peered":"2024-09-11T08:41:09.328385+0000","last_clean":"2024-09-11T08:41:09.328385+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:41:09.328385+0000","last_undegraded":"2024-09-11T08:41:09.328385+0000","last_fullsized":"2024-09-11T08:41:09.328385+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:23.047963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:23.047963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.577363+0000","scrub_duration":0.00148653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":90,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:10.700113+0000","last_change":"2024-09-11T08:40:11.625218+0000","last_active":"2024-09-11T08:41:10.700113+0000","last_peered":"2024-09-11T08:41:10.700113+0000","last_clean":"2024-09-11T08:41:10.700113+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:41:10.700113+0000","last_undegraded":"2024-09-11T08:41:10.700113+0000","last_fullsized":"2024-09-11T08:41:10.700113+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:11.625164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:11.625164+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.094157+0000","scrub_duration":0.0014996320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.486818+0000","last_change":"2024-09-11T08:40:45.210478+0000","last_active":"2024-09-11T08:41:09.486818+0000","last_peered":"2024-09-11T08:41:09.486818+0000","last_clean":"2024-09-11T08:41:09.486818+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:41:09.486818+0000","last_undegraded":"2024-09-11T08:41:09.486818+0000","last_fullsized":"2024-09-11T08:41:09.486818+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:45.210423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:45.210423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:48.427373+0000","scrub_duration":0.0020645780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"92'842","reported_seq":854,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:16.000919+0000","last_change":"2024-09-11T08:40:59.753044+0000","last_active":"2024-09-11T08:41:16.000919+0000","last_peered":"2024-09-11T08:41:16.000919+0000","last_clean":"2024-09-11T08:41:16.000919+0000","last_became_active":"2024-09-11T08:40:59.752731+0000","last_became_peered":"2024-09-11T08:40:59.752731+0000","last_unstale":"2024-09-11T08:41:16.000919+0000","last_undegraded":"2024-09-11T08:41:16.000919+0000","last_fullsized":"2024-09-11T08:41:16.000919+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:22.456843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55181312,"num_objects":842,"num_object_clones":0,"num_object_copies":2526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":842,"num_write_kb":53888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":84,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328274+0000","last_change":"2024-09-11T08:40:05.197208+0000","last_active":"2024-09-11T08:41:09.328274+0000","last_peered":"2024-09-11T08:41:09.328274+0000","last_clean":"2024-09-11T08:41:09.328274+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:41:09.328274+0000","last_undegraded":"2024-09-11T08:41:09.328274+0000","last_fullsized":"2024-09-11T08:41:09.328274+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-11T08:40:05.197152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:05.197152+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016976000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.329047+0000","last_change":"2024-09-11T08:39:43.037605+0000","last_active":"2024-09-11T08:41:09.329047+0000","last_peered":"2024-09-11T08:41:09.329047+0000","last_clean":"2024-09-11T08:41:09.329047+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:41:09.329047+0000","last_undegraded":"2024-09-11T08:41:09.329047+0000","last_fullsized":"2024-09-11T08:41:09.329047+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.037551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.037551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001523976,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.522795+0000","last_change":"2024-09-11T08:40:19.688943+0000","last_active":"2024-09-11T08:41:09.522795+0000","last_peered":"2024-09-11T08:41:09.522795+0000","last_clean":"2024-09-11T08:41:09.522795+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:41:09.522795+0000","last_undegraded":"2024-09-11T08:41:09.522795+0000","last_fullsized":"2024-09-11T08:41:09.522795+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:19.688876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:19.688876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.993714+0000","scrub_duration":0.0014001599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.522936+0000","last_change":"2024-09-11T08:40:48.768901+0000","last_active":"2024-09-11T08:41:09.522936+0000","last_peered":"2024-09-11T08:41:09.522936+0000","last_clean":"2024-09-11T08:41:09.522936+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:41:09.522936+0000","last_undegraded":"2024-09-11T08:41:09.522936+0000","last_fullsized":"2024-09-11T08:41:09.522936+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:48.768833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:48.768833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:00.167235+0000","scrub_duration":0.001532563,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"92'785","reported_seq":797,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:15.983071+0000","last_change":"2024-09-11T08:40:59.753523+0000","last_active":"2024-09-11T08:41:15.983071+0000","last_peered":"2024-09-11T08:41:15.983071+0000","last_clean":"2024-09-11T08:41:15.983071+0000","last_became_active":"2024-09-11T08:40:59.752898+0000","last_became_peered":"2024-09-11T08:40:59.752898+0000","last_unstale":"2024-09-11T08:41:15.983071+0000","last_undegraded":"2024-09-11T08:41:15.983071+0000","last_fullsized":"2024-09-11T08:41:15.983071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:19.828210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51445760,"num_objects":785,"num_object_clones":0,"num_object_copies":2355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":785,"num_write_kb":50240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"86'21","reported_seq":81,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.523008+0000","last_change":"2024-09-11T08:40:55.628101+0000","last_active":"2024-09-11T08:41:09.523008+0000","last_peered":"2024-09-11T08:41:09.523008+0000","last_clean":"2024-09-11T08:41:09.523008+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:41:09.523008+0000","last_undegraded":"2024-09-11T08:41:09.523008+0000","last_fullsized":"2024-09-11T08:41:09.523008+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'21","last_scrub_stamp":"2024-09-11T08:40:55.628034+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:40:55.628034+0000","last_clean_scrub_stamp":"2024-09-11T08:40:55.628034+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:06.433453+0000","scrub_duration":0.0051231169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.097110+0000","last_change":"2024-09-11T08:39:41.267320+0000","last_active":"2024-09-11T08:41:09.097110+0000","last_peered":"2024-09-11T08:41:09.097110+0000","last_clean":"2024-09-11T08:41:09.097110+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:41:09.097110+0000","last_undegraded":"2024-09-11T08:41:09.097110+0000","last_fullsized":"2024-09-11T08:41:09.097110+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:41.267267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.267267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013348209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":92,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:11.277135+0000","last_change":"2024-09-11T08:40:51.309316+0000","last_active":"2024-09-11T08:41:11.277135+0000","last_peered":"2024-09-11T08:41:11.277135+0000","last_clean":"2024-09-11T08:41:11.277135+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:41:11.277135+0000","last_undegraded":"2024-09-11T08:41:11.277135+0000","last_fullsized":"2024-09-11T08:41:11.277135+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:51.309259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:51.309259+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:11.655227+0000","scrub_duration":0.0017959670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.822802+0000","last_change":"2024-09-11T08:40:04.174461+0000","last_active":"2024-09-11T08:41:09.822802+0000","last_peered":"2024-09-11T08:41:09.822802+0000","last_clean":"2024-09-11T08:41:09.822802+0000","last_became_active":"2024-09-11T08:38:45.182724+0000","last_became_peered":"2024-09-11T08:38:45.182724+0000","last_unstale":"2024-09-11T08:41:09.822802+0000","last_undegraded":"2024-09-11T08:41:09.822802+0000","last_fullsized":"2024-09-11T08:41:09.822802+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:04.174409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:04.174409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.916947+0000","scrub_duration":0.0012996889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"92'838","reported_seq":850,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:16.004446+0000","last_change":"2024-09-11T08:40:59.753707+0000","last_active":"2024-09-11T08:41:16.004446+0000","last_peered":"2024-09-11T08:41:16.004446+0000","last_clean":"2024-09-11T08:41:16.004446+0000","last_became_active":"2024-09-11T08:40:59.753449+0000","last_became_peered":"2024-09-11T08:40:59.753449+0000","last_unstale":"2024-09-11T08:41:16.004446+0000","last_undegraded":"2024-09-11T08:41:16.004446+0000","last_fullsized":"2024-09-11T08:41:16.004446+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":838,"log_dups_size":0,"ondisk_log_size":838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:22.342627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54919168,"num_objects":838,"num_object_clones":0,"num_object_copies":2514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":838,"num_write_kb":53632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"63'10","reported_seq":68,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.228785+0000","last_change":"2024-09-11T08:40:13.327302+0000","last_active":"2024-09-11T08:41:09.228785+0000","last_peered":"2024-09-11T08:41:09.228785+0000","last_clean":"2024-09-11T08:41:09.228785+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:41:09.228785+0000","last_undegraded":"2024-09-11T08:41:09.228785+0000","last_fullsized":"2024-09-11T08:41:09.228785+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:40:13.327245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:13.327245+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.332692+0000","scrub_duration":0.0017714300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.487085+0000","last_change":"2024-09-11T08:40:28.351815+0000","last_active":"2024-09-11T08:41:09.487085+0000","last_peered":"2024-09-11T08:41:09.487085+0000","last_clean":"2024-09-11T08:41:09.487085+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:41:09.487085+0000","last_undegraded":"2024-09-11T08:41:09.487085+0000","last_fullsized":"2024-09-11T08:41:09.487085+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:28.351762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:28.351762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:39.082689+0000","scrub_duration":0.001710488,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":90,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:10.702191+0000","last_change":"2024-09-11T08:40:45.772463+0000","last_active":"2024-09-11T08:41:10.702191+0000","last_peered":"2024-09-11T08:41:10.702191+0000","last_clean":"2024-09-11T08:41:10.702191+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:41:10.702191+0000","last_undegraded":"2024-09-11T08:41:10.702191+0000","last_fullsized":"2024-09-11T08:41:10.702191+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:45.772375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:45.772375+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:57.912394+0000","scrub_duration":0.0021085829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.095682+0000","last_change":"2024-09-11T08:40:51.896069+0000","last_active":"2024-09-11T08:41:09.095682+0000","last_peered":"2024-09-11T08:41:09.095682+0000","last_clean":"2024-09-11T08:41:09.095682+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:41:09.095682+0000","last_undegraded":"2024-09-11T08:41:09.095682+0000","last_fullsized":"2024-09-11T08:41:09.095682+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:40:51.896002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:02.304994+0000","scrub_duration":0.001232085,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"92'830","reported_seq":842,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:15.997461+0000","last_change":"2024-09-11T08:40:59.753618+0000","last_active":"2024-09-11T08:41:15.997461+0000","last_peered":"2024-09-11T08:41:15.997461+0000","last_clean":"2024-09-11T08:41:15.997461+0000","last_became_active":"2024-09-11T08:40:59.753346+0000","last_became_peered":"2024-09-11T08:40:59.753346+0000","last_unstale":"2024-09-11T08:41:15.997461+0000","last_undegraded":"2024-09-11T08:41:15.997461+0000","last_fullsized":"2024-09-11T08:41:15.997461+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:24.059860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54394880,"num_objects":830,"num_object_clones":0,"num_object_copies":2490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":830,"num_write_kb":53120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":89,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.096731+0000","last_change":"2024-09-11T08:40:44.749339+0000","last_active":"2024-09-11T08:41:09.096731+0000","last_peered":"2024-09-11T08:41:09.096731+0000","last_clean":"2024-09-11T08:41:09.096731+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:41:09.096731+0000","last_undegraded":"2024-09-11T08:41:09.096731+0000","last_fullsized":"2024-09-11T08:41:09.096731+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:40:44.749290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:44.749290+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:14.581605+0000","scrub_duration":0.0022036899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.228980+0000","last_change":"2024-09-11T08:40:03.496005+0000","last_active":"2024-09-11T08:41:09.228980+0000","last_peered":"2024-09-11T08:41:09.228980+0000","last_clean":"2024-09-11T08:41:09.228980+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:41:09.228980+0000","last_undegraded":"2024-09-11T08:41:09.228980+0000","last_fullsized":"2024-09-11T08:41:09.228980+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:03.495950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.495950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.342754+0000","scrub_duration":0.001379314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.097226+0000","last_change":"2024-09-11T08:39:57.596980+0000","last_active":"2024-09-11T08:41:09.097226+0000","last_peered":"2024-09-11T08:41:09.097226+0000","last_clean":"2024-09-11T08:41:09.097226+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:41:09.097226+0000","last_undegraded":"2024-09-11T08:41:09.097226+0000","last_fullsized":"2024-09-11T08:41:09.097226+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.596889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:12.648355+0000","scrub_duration":0.001574424,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.228331+0000","last_change":"2024-09-11T08:40:08.387078+0000","last_active":"2024-09-11T08:41:09.228331+0000","last_peered":"2024-09-11T08:41:09.228331+0000","last_clean":"2024-09-11T08:41:09.228331+0000","last_became_active":"2024-09-11T08:38:45.182782+0000","last_became_peered":"2024-09-11T08:38:45.182782+0000","last_unstale":"2024-09-11T08:41:09.228331+0000","last_undegraded":"2024-09-11T08:41:09.228331+0000","last_fullsized":"2024-09-11T08:41:09.228331+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:08.387027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.387027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.887729+0000","scrub_duration":0.001411336,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"92'802","reported_seq":814,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:16.004515+0000","last_change":"2024-09-11T08:40:59.758072+0000","last_active":"2024-09-11T08:41:16.004515+0000","last_peered":"2024-09-11T08:41:16.004515+0000","last_clean":"2024-09-11T08:41:16.004515+0000","last_became_active":"2024-09-11T08:40:59.757849+0000","last_became_peered":"2024-09-11T08:40:59.757849+0000","last_unstale":"2024-09-11T08:41:16.004515+0000","last_undegraded":"2024-09-11T08:41:16.004515+0000","last_fullsized":"2024-09-11T08:41:16.004515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:10.638357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52559872,"num_objects":802,"num_object_clones":0,"num_object_copies":2406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":802,"num_write_kb":51328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":142,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:11.268719+0000","last_change":"2024-09-11T08:39:53.493004+0000","last_active":"2024-09-11T08:41:11.268719+0000","last_peered":"2024-09-11T08:41:11.268719+0000","last_clean":"2024-09-11T08:41:11.268719+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:41:11.268719+0000","last_undegraded":"2024-09-11T08:41:11.268719+0000","last_fullsized":"2024-09-11T08:41:11.268719+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.492945+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005988045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.523184+0000","last_change":"2024-09-11T08:40:22.750815+0000","last_active":"2024-09-11T08:41:09.523184+0000","last_peered":"2024-09-11T08:41:09.523184+0000","last_clean":"2024-09-11T08:41:09.523184+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:41:09.523184+0000","last_undegraded":"2024-09-11T08:41:09.523184+0000","last_fullsized":"2024-09-11T08:41:09.523184+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:22.750744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.750744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.795298+0000","scrub_duration":0.001679119,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.486573+0000","last_change":"2024-09-11T08:39:58.472851+0000","last_active":"2024-09-11T08:41:09.486573+0000","last_peered":"2024-09-11T08:41:09.486573+0000","last_clean":"2024-09-11T08:41:09.486573+0000","last_became_active":"2024-09-11T08:38:38.947270+0000","last_became_peered":"2024-09-11T08:38:38.947270+0000","last_unstale":"2024-09-11T08:41:09.486573+0000","last_undegraded":"2024-09-11T08:41:09.486573+0000","last_fullsized":"2024-09-11T08:41:09.486573+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:39:58.472785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:58.472785+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.725781+0000","scrub_duration":0.0020806930000000002,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.096334+0000","last_change":"2024-09-11T08:40:17.435980+0000","last_active":"2024-09-11T08:41:09.096334+0000","last_peered":"2024-09-11T08:41:09.096334+0000","last_clean":"2024-09-11T08:41:09.096334+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:41:09.096334+0000","last_undegraded":"2024-09-11T08:41:09.096334+0000","last_fullsized":"2024-09-11T08:41:09.096334+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:17.435900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:17.435900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.732279+0000","scrub_duration":0.0018276379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328806+0000","last_change":"2024-09-11T08:40:31.206582+0000","last_active":"2024-09-11T08:41:09.328806+0000","last_peered":"2024-09-11T08:41:09.328806+0000","last_clean":"2024-09-11T08:41:09.328806+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:41:09.328806+0000","last_undegraded":"2024-09-11T08:41:09.328806+0000","last_fullsized":"2024-09-11T08:41:09.328806+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:31.206531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:31.206531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:49.078201+0000","scrub_duration":0.001945928,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"92'764","reported_seq":776,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:15.992317+0000","last_change":"2024-09-11T08:40:59.756855+0000","last_active":"2024-09-11T08:41:15.992317+0000","last_peered":"2024-09-11T08:41:15.992317+0000","last_clean":"2024-09-11T08:41:15.992317+0000","last_became_active":"2024-09-11T08:40:59.756563+0000","last_became_peered":"2024-09-11T08:40:59.756563+0000","last_unstale":"2024-09-11T08:41:15.992317+0000","last_undegraded":"2024-09-11T08:41:15.992317+0000","last_fullsized":"2024-09-11T08:41:15.992317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:23.426877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50069504,"num_objects":764,"num_object_clones":0,"num_object_copies":2292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":764,"num_write_kb":48896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":80,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.229186+0000","last_change":"2024-09-11T08:39:43.585175+0000","last_active":"2024-09-11T08:41:09.229186+0000","last_peered":"2024-09-11T08:41:09.229186+0000","last_clean":"2024-09-11T08:41:09.229186+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:41:09.229186+0000","last_undegraded":"2024-09-11T08:41:09.229186+0000","last_fullsized":"2024-09-11T08:41:09.229186+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:39:43.585116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.585116+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021210309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.228719+0000","last_change":"2024-09-11T08:40:00.500444+0000","last_active":"2024-09-11T08:41:09.228719+0000","last_peered":"2024-09-11T08:41:09.228719+0000","last_clean":"2024-09-11T08:41:09.228719+0000","last_became_active":"2024-09-11T08:38:38.949019+0000","last_became_peered":"2024-09-11T08:38:38.949019+0000","last_unstale":"2024-09-11T08:41:09.228719+0000","last_undegraded":"2024-09-11T08:41:09.228719+0000","last_fullsized":"2024-09-11T08:41:09.228719+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.500392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.500392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:22.354187+0000","scrub_duration":0.001294566,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.096360+0000","last_change":"2024-09-11T08:40:01.488539+0000","last_active":"2024-09-11T08:41:09.096360+0000","last_peered":"2024-09-11T08:41:09.096360+0000","last_clean":"2024-09-11T08:41:09.096360+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:41:09.096360+0000","last_undegraded":"2024-09-11T08:41:09.096360+0000","last_fullsized":"2024-09-11T08:41:09.096360+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:40:01.488482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:01.488482+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001463422,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.096515+0000","last_change":"2024-09-11T08:40:02.444341+0000","last_active":"2024-09-11T08:41:09.096515+0000","last_peered":"2024-09-11T08:41:09.096515+0000","last_clean":"2024-09-11T08:41:09.096515+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:41:09.096515+0000","last_undegraded":"2024-09-11T08:41:09.096515+0000","last_fullsized":"2024-09-11T08:41:09.096515+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.444279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001414329,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.822813+0000","last_change":"2024-09-11T08:40:15.184420+0000","last_active":"2024-09-11T08:41:09.822813+0000","last_peered":"2024-09-11T08:41:09.822813+0000","last_clean":"2024-09-11T08:41:09.822813+0000","last_became_active":"2024-09-11T08:38:45.183582+0000","last_became_peered":"2024-09-11T08:38:45.183582+0000","last_unstale":"2024-09-11T08:41:09.822813+0000","last_undegraded":"2024-09-11T08:41:09.822813+0000","last_fullsized":"2024-09-11T08:41:09.822813+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:15.184337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:15.184337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.002701+0000","scrub_duration":0.0039899209999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"92'785","reported_seq":797,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:15.985994+0000","last_change":"2024-09-11T08:40:59.755961+0000","last_active":"2024-09-11T08:41:15.985994+0000","last_peered":"2024-09-11T08:41:15.985994+0000","last_clean":"2024-09-11T08:41:15.985994+0000","last_became_active":"2024-09-11T08:40:59.755652+0000","last_became_peered":"2024-09-11T08:40:59.755652+0000","last_unstale":"2024-09-11T08:41:15.985994+0000","last_undegraded":"2024-09-11T08:41:15.985994+0000","last_fullsized":"2024-09-11T08:41:15.985994+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:20.387084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51445760,"num_objects":785,"num_object_clones":0,"num_object_copies":2355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":785,"num_write_kb":50240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.097029+0000","last_change":"2024-09-11T08:40:21.450026+0000","last_active":"2024-09-11T08:41:09.097029+0000","last_peered":"2024-09-11T08:41:09.097029+0000","last_clean":"2024-09-11T08:41:09.097029+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:41:09.097029+0000","last_undegraded":"2024-09-11T08:41:09.097029+0000","last_fullsized":"2024-09-11T08:41:09.097029+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:40:21.449958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:21.449958+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:46.796354+0000","scrub_duration":0.0017585280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.097124+0000","last_change":"2024-09-11T08:39:43.356751+0000","last_active":"2024-09-11T08:41:09.097124+0000","last_peered":"2024-09-11T08:41:09.097124+0000","last_clean":"2024-09-11T08:41:09.097124+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:41:09.097124+0000","last_undegraded":"2024-09-11T08:41:09.097124+0000","last_fullsized":"2024-09-11T08:41:09.097124+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:43.356699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.356699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001649455,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":222,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:14.344289+0000","last_change":"2024-09-11T08:40:58.105377+0000","last_active":"2024-09-11T08:41:14.344289+0000","last_peered":"2024-09-11T08:41:14.344289+0000","last_clean":"2024-09-11T08:41:14.344289+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:41:14.344289+0000","last_undegraded":"2024-09-11T08:41:14.344289+0000","last_fullsized":"2024-09-11T08:41:14.344289+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:40:58.105306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.105306+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:21.452749+0000","scrub_duration":0.0019341060000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":150,"num_read_kb":145,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.487247+0000","last_change":"2024-09-11T08:40:56.275231+0000","last_active":"2024-09-11T08:41:09.487247+0000","last_peered":"2024-09-11T08:41:09.487247+0000","last_clean":"2024-09-11T08:41:09.487247+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:41:09.487247+0000","last_undegraded":"2024-09-11T08:41:09.487247+0000","last_fullsized":"2024-09-11T08:41:09.487247+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:56.275174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:56.275174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:58.117600+0000","scrub_duration":0.0019278870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.228701+0000","last_change":"2024-09-11T08:40:15.280809+0000","last_active":"2024-09-11T08:41:09.228701+0000","last_peered":"2024-09-11T08:41:09.228701+0000","last_clean":"2024-09-11T08:41:09.228701+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:41:09.228701+0000","last_undegraded":"2024-09-11T08:41:09.228701+0000","last_fullsized":"2024-09-11T08:41:09.228701+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:15.280760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:15.280760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:36.262012+0000","scrub_duration":0.0012405719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"92'809","reported_seq":821,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:16.004659+0000","last_change":"2024-09-11T08:40:59.756179+0000","last_active":"2024-09-11T08:41:16.004659+0000","last_peered":"2024-09-11T08:41:16.004659+0000","last_clean":"2024-09-11T08:41:16.004659+0000","last_became_active":"2024-09-11T08:40:59.755912+0000","last_became_peered":"2024-09-11T08:40:59.755912+0000","last_unstale":"2024-09-11T08:41:16.004659+0000","last_undegraded":"2024-09-11T08:41:16.004659+0000","last_fullsized":"2024-09-11T08:41:16.004659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:01.655246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53018624,"num_objects":809,"num_object_clones":0,"num_object_copies":2427,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":809,"num_write_kb":51776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.097727+0000","last_change":"2024-09-11T08:40:00.540192+0000","last_active":"2024-09-11T08:41:09.097727+0000","last_peered":"2024-09-11T08:41:09.097727+0000","last_clean":"2024-09-11T08:41:09.097727+0000","last_became_active":"2024-09-11T08:38:41.040948+0000","last_became_peered":"2024-09-11T08:38:41.040948+0000","last_unstale":"2024-09-11T08:41:09.097727+0000","last_undegraded":"2024-09-11T08:41:09.097727+0000","last_fullsized":"2024-09-11T08:41:09.097727+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:40:00.540136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.540136+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001785095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328569+0000","last_change":"2024-09-11T08:40:21.086239+0000","last_active":"2024-09-11T08:41:09.328569+0000","last_peered":"2024-09-11T08:41:09.328569+0000","last_clean":"2024-09-11T08:41:09.328569+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:41:09.328569+0000","last_undegraded":"2024-09-11T08:41:09.328569+0000","last_fullsized":"2024-09-11T08:41:09.328569+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:40:21.086183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:21.086183+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:25.836107+0000","scrub_duration":0.001907838,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":98,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.823443+0000","last_change":"2024-09-11T08:40:34.271853+0000","last_active":"2024-09-11T08:41:09.823443+0000","last_peered":"2024-09-11T08:41:09.823443+0000","last_clean":"2024-09-11T08:41:09.823443+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:41:09.823443+0000","last_undegraded":"2024-09-11T08:41:09.823443+0000","last_fullsized":"2024-09-11T08:41:09.823443+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'79","last_scrub_stamp":"2024-09-11T08:40:34.271800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:40:34.271800+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:35.589139+0000","scrub_duration":0.001626796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.823314+0000","last_change":"2024-09-11T08:39:49.116184+0000","last_active":"2024-09-11T08:41:09.823314+0000","last_peered":"2024-09-11T08:41:09.823314+0000","last_clean":"2024-09-11T08:41:09.823314+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:41:09.823314+0000","last_undegraded":"2024-09-11T08:41:09.823314+0000","last_fullsized":"2024-09-11T08:41:09.823314+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.116113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.116113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00167223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.487208+0000","last_change":"2024-09-11T08:39:47.423745+0000","last_active":"2024-09-11T08:41:09.487208+0000","last_peered":"2024-09-11T08:41:09.487208+0000","last_clean":"2024-09-11T08:41:09.487208+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:41:09.487208+0000","last_undegraded":"2024-09-11T08:41:09.487208+0000","last_fullsized":"2024-09-11T08:41:09.487208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:47.423684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:47.423684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001458972,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"92'804","reported_seq":816,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:15.992663+0000","last_change":"2024-09-11T08:40:59.756162+0000","last_active":"2024-09-11T08:41:15.992663+0000","last_peered":"2024-09-11T08:41:15.992663+0000","last_clean":"2024-09-11T08:41:15.992663+0000","last_became_active":"2024-09-11T08:40:59.755894+0000","last_became_peered":"2024-09-11T08:40:59.755894+0000","last_unstale":"2024-09-11T08:41:15.992663+0000","last_undegraded":"2024-09-11T08:41:15.992663+0000","last_fullsized":"2024-09-11T08:41:15.992663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:25.885752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52690944,"num_objects":804,"num_object_clones":0,"num_object_copies":2412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":804,"num_write_kb":51456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.523391+0000","last_change":"2024-09-11T08:40:53.717259+0000","last_active":"2024-09-11T08:41:09.523391+0000","last_peered":"2024-09-11T08:41:09.523391+0000","last_clean":"2024-09-11T08:41:09.523391+0000","last_became_active":"2024-09-11T08:40:16.183867+0000","last_became_peered":"2024-09-11T08:40:16.183867+0000","last_unstale":"2024-09-11T08:41:09.523391+0000","last_undegraded":"2024-09-11T08:41:09.523391+0000","last_fullsized":"2024-09-11T08:41:09.523391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:40:53.717193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:53.717193+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:59.406353+0000","scrub_duration":0.0019056640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.523102+0000","last_change":"2024-09-11T08:40:54.671894+0000","last_active":"2024-09-11T08:41:09.523102+0000","last_peered":"2024-09-11T08:41:09.523102+0000","last_clean":"2024-09-11T08:41:09.523102+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:41:09.523102+0000","last_undegraded":"2024-09-11T08:41:09.523102+0000","last_fullsized":"2024-09-11T08:41:09.523102+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:40:54.671832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:14.247137+0000","scrub_duration":0.001438541,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.487156+0000","last_change":"2024-09-11T08:40:03.430883+0000","last_active":"2024-09-11T08:41:09.487156+0000","last_peered":"2024-09-11T08:41:09.487156+0000","last_clean":"2024-09-11T08:41:09.487156+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:41:09.487156+0000","last_undegraded":"2024-09-11T08:41:09.487156+0000","last_fullsized":"2024-09-11T08:41:09.487156+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:03.430794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.430794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:20.081896+0000","scrub_duration":0.0014294430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328059+0000","last_change":"2024-09-11T08:40:30.215943+0000","last_active":"2024-09-11T08:41:09.328059+0000","last_peered":"2024-09-11T08:41:09.328059+0000","last_clean":"2024-09-11T08:41:09.328059+0000","last_became_active":"2024-09-11T08:38:45.184410+0000","last_became_peered":"2024-09-11T08:38:45.184410+0000","last_unstale":"2024-09-11T08:41:09.328059+0000","last_undegraded":"2024-09-11T08:41:09.328059+0000","last_fullsized":"2024-09-11T08:41:09.328059+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:30.215890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:30.215890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:45.729251+0000","scrub_duration":0.0012073710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"92'802","reported_seq":814,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:15.996699+0000","last_change":"2024-09-11T08:40:59.757516+0000","last_active":"2024-09-11T08:41:15.996699+0000","last_peered":"2024-09-11T08:41:15.996699+0000","last_clean":"2024-09-11T08:41:15.996699+0000","last_became_active":"2024-09-11T08:40:59.755838+0000","last_became_peered":"2024-09-11T08:40:59.755838+0000","last_unstale":"2024-09-11T08:41:15.996699+0000","last_undegraded":"2024-09-11T08:41:15.996699+0000","last_fullsized":"2024-09-11T08:41:15.996699+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:04.535548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52559872,"num_objects":802,"num_object_clones":0,"num_object_copies":2406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":802,"num_write_kb":51328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":81,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.523371+0000","last_change":"2024-09-11T08:39:43.526845+0000","last_active":"2024-09-11T08:41:09.523371+0000","last_peered":"2024-09-11T08:41:09.523371+0000","last_clean":"2024-09-11T08:41:09.523371+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:41:09.523371+0000","last_undegraded":"2024-09-11T08:41:09.523371+0000","last_fullsized":"2024-09-11T08:41:09.523371+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:39:43.526763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:43.526763+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022037329999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.097451+0000","last_change":"2024-09-11T08:39:55.599049+0000","last_active":"2024-09-11T08:41:09.097451+0000","last_peered":"2024-09-11T08:41:09.097451+0000","last_clean":"2024-09-11T08:41:09.097451+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:41:09.097451+0000","last_undegraded":"2024-09-11T08:41:09.097451+0000","last_fullsized":"2024-09-11T08:41:09.097451+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.598961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.598961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:23.562190+0000","scrub_duration":0.0016789820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.228809+0000","last_change":"2024-09-11T08:40:57.196146+0000","last_active":"2024-09-11T08:41:09.228809+0000","last_peered":"2024-09-11T08:41:09.228809+0000","last_clean":"2024-09-11T08:41:09.228809+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:41:09.228809+0000","last_undegraded":"2024-09-11T08:41:09.228809+0000","last_fullsized":"2024-09-11T08:41:09.228809+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:57.196087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:57.196087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:12.479198+0000","scrub_duration":0.0015986399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.095994+0000","last_change":"2024-09-11T08:40:13.462183+0000","last_active":"2024-09-11T08:41:09.095994+0000","last_peered":"2024-09-11T08:41:09.095994+0000","last_clean":"2024-09-11T08:41:09.095994+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:41:09.095994+0000","last_undegraded":"2024-09-11T08:41:09.095994+0000","last_fullsized":"2024-09-11T08:41:09.095994+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:13.462132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:13.462132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:33.998327+0000","scrub_duration":0.001365727,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"92'831","reported_seq":843,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:15.996186+0000","last_change":"2024-09-11T08:40:59.755613+0000","last_active":"2024-09-11T08:41:15.996186+0000","last_peered":"2024-09-11T08:41:15.996186+0000","last_clean":"2024-09-11T08:41:15.996186+0000","last_became_active":"2024-09-11T08:40:59.755307+0000","last_became_peered":"2024-09-11T08:40:59.755307+0000","last_unstale":"2024-09-11T08:41:15.996186+0000","last_undegraded":"2024-09-11T08:41:15.996186+0000","last_fullsized":"2024-09-11T08:41:15.996186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:14.469907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54460416,"num_objects":831,"num_object_clones":0,"num_object_copies":2493,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":831,"num_write_kb":53184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":68,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.523263+0000","last_change":"2024-09-11T08:40:34.764019+0000","last_active":"2024-09-11T08:41:09.523263+0000","last_peered":"2024-09-11T08:41:09.523263+0000","last_clean":"2024-09-11T08:41:09.523263+0000","last_became_active":"2024-09-11T08:38:40.969348+0000","last_became_peered":"2024-09-11T08:38:40.969348+0000","last_unstale":"2024-09-11T08:41:09.523263+0000","last_undegraded":"2024-09-11T08:41:09.523263+0000","last_fullsized":"2024-09-11T08:41:09.523263+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:40:34.763952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:34.763952+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:00.532192+0000","scrub_duration":0.002030423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328643+0000","last_change":"2024-09-11T08:40:50.328492+0000","last_active":"2024-09-11T08:41:09.328643+0000","last_peered":"2024-09-11T08:41:09.328643+0000","last_clean":"2024-09-11T08:41:09.328643+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:41:09.328643+0000","last_undegraded":"2024-09-11T08:41:09.328643+0000","last_fullsized":"2024-09-11T08:41:09.328643+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:50.328435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:50.328435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:03.482341+0000","scrub_duration":0.0016054349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":90,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:11.649124+0000","last_change":"2024-09-11T08:40:00.473606+0000","last_active":"2024-09-11T08:41:11.649124+0000","last_peered":"2024-09-11T08:41:11.649124+0000","last_clean":"2024-09-11T08:41:11.649124+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:41:11.649124+0000","last_undegraded":"2024-09-11T08:41:11.649124+0000","last_fullsized":"2024-09-11T08:41:11.649124+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:00.473549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.473549+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.300137+0000","scrub_duration":0.0016703060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.523279+0000","last_change":"2024-09-11T08:40:38.664034+0000","last_active":"2024-09-11T08:41:09.523279+0000","last_peered":"2024-09-11T08:41:09.523279+0000","last_clean":"2024-09-11T08:41:09.523279+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:41:09.523279+0000","last_undegraded":"2024-09-11T08:41:09.523279+0000","last_fullsized":"2024-09-11T08:41:09.523279+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:40:38.663952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:45.400249+0000","scrub_duration":0.0014392509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"92'839","reported_seq":851,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:16.005545+0000","last_change":"2024-09-11T08:40:59.754632+0000","last_active":"2024-09-11T08:41:16.005545+0000","last_peered":"2024-09-11T08:41:16.005545+0000","last_clean":"2024-09-11T08:41:16.005545+0000","last_became_active":"2024-09-11T08:40:59.754331+0000","last_became_peered":"2024-09-11T08:40:59.754331+0000","last_unstale":"2024-09-11T08:41:16.005545+0000","last_undegraded":"2024-09-11T08:41:16.005545+0000","last_fullsized":"2024-09-11T08:41:16.005545+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":839,"log_dups_size":0,"ondisk_log_size":839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:59.236292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54984704,"num_objects":839,"num_object_clones":0,"num_object_copies":2517,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":839,"num_write_kb":53696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.096650+0000","last_change":"2024-09-11T08:40:39.719818+0000","last_active":"2024-09-11T08:41:09.096650+0000","last_peered":"2024-09-11T08:41:09.096650+0000","last_clean":"2024-09-11T08:41:09.096650+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:41:09.096650+0000","last_undegraded":"2024-09-11T08:41:09.096650+0000","last_fullsized":"2024-09-11T08:41:09.096650+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:40:39.719754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:39.719754+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:07.308170+0000","scrub_duration":0.0016593619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.486779+0000","last_change":"2024-09-11T08:39:50.452568+0000","last_active":"2024-09-11T08:41:09.486779+0000","last_peered":"2024-09-11T08:41:09.486779+0000","last_clean":"2024-09-11T08:41:09.486779+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:41:09.486779+0000","last_undegraded":"2024-09-11T08:41:09.486779+0000","last_fullsized":"2024-09-11T08:41:09.486779+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:39:50.452494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:18.631278+0000","scrub_duration":0.001547086,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.096323+0000","last_change":"2024-09-11T08:40:09.519072+0000","last_active":"2024-09-11T08:41:09.096323+0000","last_peered":"2024-09-11T08:41:09.096323+0000","last_clean":"2024-09-11T08:41:09.096323+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:41:09.096323+0000","last_undegraded":"2024-09-11T08:41:09.096323+0000","last_fullsized":"2024-09-11T08:41:09.096323+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:09.519019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:09.519019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:31.000837+0000","scrub_duration":0.001232471,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.097058+0000","last_change":"2024-09-11T08:40:50.866588+0000","last_active":"2024-09-11T08:41:09.097058+0000","last_peered":"2024-09-11T08:41:09.097058+0000","last_clean":"2024-09-11T08:41:09.097058+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:41:09.097058+0000","last_undegraded":"2024-09-11T08:41:09.097058+0000","last_fullsized":"2024-09-11T08:41:09.097058+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:50.866523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:50.866523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:53.269682+0000","scrub_duration":0.0016545760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"92'770","reported_seq":782,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:15.992273+0000","last_change":"2024-09-11T08:40:59.756192+0000","last_active":"2024-09-11T08:41:15.992273+0000","last_peered":"2024-09-11T08:41:15.992273+0000","last_clean":"2024-09-11T08:41:15.992273+0000","last_became_active":"2024-09-11T08:40:59.755928+0000","last_became_peered":"2024-09-11T08:40:59.755928+0000","last_unstale":"2024-09-11T08:41:15.992273+0000","last_undegraded":"2024-09-11T08:41:15.992273+0000","last_fullsized":"2024-09-11T08:41:15.992273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:01.874033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50462720,"num_objects":770,"num_object_clones":0,"num_object_copies":2310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":770,"num_write_kb":49280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":83,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.097286+0000","last_change":"2024-09-11T08:39:45.575209+0000","last_active":"2024-09-11T08:41:09.097286+0000","last_peered":"2024-09-11T08:41:09.097286+0000","last_clean":"2024-09-11T08:41:09.097286+0000","last_became_active":"2024-09-11T08:38:41.040207+0000","last_became_peered":"2024-09-11T08:38:41.040207+0000","last_unstale":"2024-09-11T08:41:09.097286+0000","last_undegraded":"2024-09-11T08:41:09.097286+0000","last_fullsized":"2024-09-11T08:41:09.097286+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:39:45.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:45.575133+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.182090+0000","scrub_duration":0.0022792160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.823493+0000","last_change":"2024-09-11T08:39:56.112112+0000","last_active":"2024-09-11T08:41:09.823493+0000","last_peered":"2024-09-11T08:41:09.823493+0000","last_clean":"2024-09-11T08:41:09.823493+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:41:09.823493+0000","last_undegraded":"2024-09-11T08:41:09.823493+0000","last_fullsized":"2024-09-11T08:41:09.823493+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:56.112043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:56.112043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:25.809354+0000","scrub_duration":0.001517137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.095780+0000","last_change":"2024-09-11T08:40:11.540833+0000","last_active":"2024-09-11T08:41:09.095780+0000","last_peered":"2024-09-11T08:41:09.095780+0000","last_clean":"2024-09-11T08:41:09.095780+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:41:09.095780+0000","last_undegraded":"2024-09-11T08:41:09.095780+0000","last_fullsized":"2024-09-11T08:41:09.095780+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:11.540760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:11.540760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.937538+0000","scrub_duration":0.0014212529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.823268+0000","last_change":"2024-09-11T08:40:26.222640+0000","last_active":"2024-09-11T08:41:09.823268+0000","last_peered":"2024-09-11T08:41:09.823268+0000","last_clean":"2024-09-11T08:41:09.823268+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:41:09.823268+0000","last_undegraded":"2024-09-11T08:41:09.823268+0000","last_fullsized":"2024-09-11T08:41:09.823268+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:26.222567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:26.222567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:54.940352+0000","scrub_duration":0.001672593,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"92'849","reported_seq":859,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:16.006629+0000","last_change":"2024-09-11T08:40:59.753630+0000","last_active":"2024-09-11T08:41:16.006629+0000","last_peered":"2024-09-11T08:41:16.006629+0000","last_clean":"2024-09-11T08:41:16.006629+0000","last_became_active":"2024-09-11T08:40:59.753273+0000","last_became_peered":"2024-09-11T08:40:59.753273+0000","last_unstale":"2024-09-11T08:41:16.006629+0000","last_undegraded":"2024-09-11T08:41:16.006629+0000","last_fullsized":"2024-09-11T08:41:16.006629+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":849,"log_dups_size":0,"ondisk_log_size":849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:01.717985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55640064,"num_objects":849,"num_object_clones":0,"num_object_copies":2547,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":849,"num_write_kb":54336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":76,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.522910+0000","last_change":"2024-09-11T08:39:40.524581+0000","last_active":"2024-09-11T08:41:09.522910+0000","last_peered":"2024-09-11T08:41:09.522910+0000","last_clean":"2024-09-11T08:41:09.522910+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:41:09.522910+0000","last_undegraded":"2024-09-11T08:41:09.522910+0000","last_fullsized":"2024-09-11T08:41:09.522910+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:39:40.524498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:40.524498+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024035010000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.823593+0000","last_change":"2024-09-11T08:40:00.113587+0000","last_active":"2024-09-11T08:41:09.823593+0000","last_peered":"2024-09-11T08:41:09.823593+0000","last_clean":"2024-09-11T08:41:09.823593+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:41:09.823593+0000","last_undegraded":"2024-09-11T08:41:09.823593+0000","last_fullsized":"2024-09-11T08:41:09.823593+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:00.113534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:00.113534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:21.476975+0000","scrub_duration":0.0011790780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.095762+0000","last_change":"2024-09-11T08:39:57.454257+0000","last_active":"2024-09-11T08:41:09.095762+0000","last_peered":"2024-09-11T08:41:09.095762+0000","last_clean":"2024-09-11T08:41:09.095762+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:41:09.095762+0000","last_undegraded":"2024-09-11T08:41:09.095762+0000","last_fullsized":"2024-09-11T08:41:09.095762+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:57.454198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:57.454198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.731898+0000","scrub_duration":0.0011666669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328531+0000","last_change":"2024-09-11T08:40:44.366556+0000","last_active":"2024-09-11T08:41:09.328531+0000","last_peered":"2024-09-11T08:41:09.328531+0000","last_clean":"2024-09-11T08:41:09.328531+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:41:09.328531+0000","last_undegraded":"2024-09-11T08:41:09.328531+0000","last_fullsized":"2024-09-11T08:41:09.328531+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:44.366503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:44.366503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:06.210160+0000","scrub_duration":0.0015578059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"92'807","reported_seq":819,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:16.000418+0000","last_change":"2024-09-11T08:40:59.753680+0000","last_active":"2024-09-11T08:41:16.000418+0000","last_peered":"2024-09-11T08:41:16.000418+0000","last_clean":"2024-09-11T08:41:16.000418+0000","last_became_active":"2024-09-11T08:40:59.753438+0000","last_became_peered":"2024-09-11T08:40:59.753438+0000","last_unstale":"2024-09-11T08:41:16.000418+0000","last_undegraded":"2024-09-11T08:41:16.000418+0000","last_fullsized":"2024-09-11T08:41:16.000418+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:27.525527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52887552,"num_objects":807,"num_object_clones":0,"num_object_copies":2421,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":807,"num_write_kb":51648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":78,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.486246+0000","last_change":"2024-09-11T08:39:48.462233+0000","last_active":"2024-09-11T08:41:09.486246+0000","last_peered":"2024-09-11T08:41:09.486246+0000","last_clean":"2024-09-11T08:41:09.486246+0000","last_became_active":"2024-09-11T08:38:41.040475+0000","last_became_peered":"2024-09-11T08:38:41.040475+0000","last_unstale":"2024-09-11T08:41:09.486246+0000","last_undegraded":"2024-09-11T08:41:09.486246+0000","last_fullsized":"2024-09-11T08:41:09.486246+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-11T08:39:48.462163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:48.462163+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:11.032207+0000","scrub_duration":0.002094753,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.823641+0000","last_change":"2024-09-11T08:40:03.143806+0000","last_active":"2024-09-11T08:41:09.823641+0000","last_peered":"2024-09-11T08:41:09.823641+0000","last_clean":"2024-09-11T08:41:09.823641+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:41:09.823641+0000","last_undegraded":"2024-09-11T08:41:09.823641+0000","last_fullsized":"2024-09-11T08:41:09.823641+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:40:03.143742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:03.143742+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:30.570435+0000","scrub_duration":0.001564396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"63'8","reported_seq":89,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:11.005824+0000","last_change":"2024-09-11T08:40:13.221708+0000","last_active":"2024-09-11T08:41:11.005824+0000","last_peered":"2024-09-11T08:41:11.005824+0000","last_clean":"2024-09-11T08:41:11.005824+0000","last_became_active":"2024-09-11T08:38:42.986756+0000","last_became_peered":"2024-09-11T08:38:42.986756+0000","last_unstale":"2024-09-11T08:41:11.005824+0000","last_undegraded":"2024-09-11T08:41:11.005824+0000","last_fullsized":"2024-09-11T08:41:11.005824+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:13.221650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:13.221650+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:28.234147+0000","scrub_duration":0.0017036169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.486119+0000","last_change":"2024-09-11T08:40:11.370313+0000","last_active":"2024-09-11T08:41:09.486119+0000","last_peered":"2024-09-11T08:41:09.486119+0000","last_clean":"2024-09-11T08:41:09.486119+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:41:09.486119+0000","last_undegraded":"2024-09-11T08:41:09.486119+0000","last_fullsized":"2024-09-11T08:41:09.486119+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:11.370245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:11.370245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:35.188374+0000","scrub_duration":0.0011869179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":64,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.097193+0000","last_change":"2024-09-11T08:40:55.587948+0000","last_active":"2024-09-11T08:41:09.097193+0000","last_peered":"2024-09-11T08:41:09.097193+0000","last_clean":"2024-09-11T08:41:09.097193+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:41:09.097193+0000","last_undegraded":"2024-09-11T08:41:09.097193+0000","last_fullsized":"2024-09-11T08:41:09.097193+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:55.587871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:55.587871+0000","last_clean_scrub_stamp":"2024-09-11T08:40:55.587871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:06.797380+0000","scrub_duration":0.001953878,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":79,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.329015+0000","last_change":"2024-09-11T08:40:17.096092+0000","last_active":"2024-09-11T08:41:09.329015+0000","last_peered":"2024-09-11T08:41:09.329015+0000","last_clean":"2024-09-11T08:41:09.329015+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:41:09.329015+0000","last_undegraded":"2024-09-11T08:41:09.329015+0000","last_fullsized":"2024-09-11T08:41:09.329015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-11T08:40:17.096034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:17.096034+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:43.002314+0000","scrub_duration":0.0019684910000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.487027+0000","last_change":"2024-09-11T08:39:49.472764+0000","last_active":"2024-09-11T08:41:09.487027+0000","last_peered":"2024-09-11T08:41:09.487027+0000","last_clean":"2024-09-11T08:41:09.487027+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:41:09.487027+0000","last_undegraded":"2024-09-11T08:41:09.487027+0000","last_fullsized":"2024-09-11T08:41:09.487027+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:49.472696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:49.472696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001919104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.822813+0000","last_change":"2024-09-11T08:40:31.223694+0000","last_active":"2024-09-11T08:41:09.822813+0000","last_peered":"2024-09-11T08:41:09.822813+0000","last_clean":"2024-09-11T08:41:09.822813+0000","last_became_active":"2024-09-11T08:38:45.184753+0000","last_became_peered":"2024-09-11T08:38:45.184753+0000","last_unstale":"2024-09-11T08:41:09.822813+0000","last_undegraded":"2024-09-11T08:41:09.822813+0000","last_fullsized":"2024-09-11T08:41:09.822813+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:31.223635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:31.223635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:37.656818+0000","scrub_duration":0.001429636,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.097059+0000","last_change":"2024-09-11T08:40:33.692680+0000","last_active":"2024-09-11T08:41:09.097059+0000","last_peered":"2024-09-11T08:41:09.097059+0000","last_clean":"2024-09-11T08:41:09.097059+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:41:09.097059+0000","last_undegraded":"2024-09-11T08:41:09.097059+0000","last_fullsized":"2024-09-11T08:41:09.097059+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:40:33.692610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:33.692610+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:52.519660+0000","scrub_duration":0.0023198749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.823194+0000","last_change":"2024-09-11T08:39:53.176095+0000","last_active":"2024-09-11T08:41:09.823194+0000","last_peered":"2024-09-11T08:41:09.823194+0000","last_clean":"2024-09-11T08:41:09.823194+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:41:09.823194+0000","last_undegraded":"2024-09-11T08:41:09.823194+0000","last_fullsized":"2024-09-11T08:41:09.823194+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.176035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.176035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001147531,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328975+0000","last_change":"2024-09-11T08:40:28.199687+0000","last_active":"2024-09-11T08:41:09.328975+0000","last_peered":"2024-09-11T08:41:09.328975+0000","last_clean":"2024-09-11T08:41:09.328975+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:41:09.328975+0000","last_undegraded":"2024-09-11T08:41:09.328975+0000","last_fullsized":"2024-09-11T08:41:09.328975+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:28.199630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:28.199630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:57.722333+0000","scrub_duration":0.001581287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.095873+0000","last_change":"2024-09-11T08:39:52.493064+0000","last_active":"2024-09-11T08:41:09.095873+0000","last_peered":"2024-09-11T08:41:09.095873+0000","last_clean":"2024-09-11T08:41:09.095873+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:41:09.095873+0000","last_undegraded":"2024-09-11T08:41:09.095873+0000","last_fullsized":"2024-09-11T08:41:09.095873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.493013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:13.717323+0000","scrub_duration":0.0012699160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.487014+0000","last_change":"2024-09-11T08:40:30.270654+0000","last_active":"2024-09-11T08:41:09.487014+0000","last_peered":"2024-09-11T08:41:09.487014+0000","last_clean":"2024-09-11T08:41:09.487014+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:41:09.487014+0000","last_undegraded":"2024-09-11T08:41:09.487014+0000","last_fullsized":"2024-09-11T08:41:09.487014+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:40:30.270582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:30.270582+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:35.812455+0000","scrub_duration":0.0025147979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.096737+0000","last_change":"2024-09-11T08:40:20.467510+0000","last_active":"2024-09-11T08:41:09.096737+0000","last_peered":"2024-09-11T08:41:09.096737+0000","last_clean":"2024-09-11T08:41:09.096737+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:41:09.096737+0000","last_undegraded":"2024-09-11T08:41:09.096737+0000","last_fullsized":"2024-09-11T08:41:09.096737+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:20.467456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:20.467456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:39.736643+0000","scrub_duration":0.001886544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":89,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:11.277647+0000","last_change":"2024-09-11T08:40:25.104280+0000","last_active":"2024-09-11T08:41:11.277647+0000","last_peered":"2024-09-11T08:41:11.277647+0000","last_clean":"2024-09-11T08:41:11.277647+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:41:11.277647+0000","last_undegraded":"2024-09-11T08:41:11.277647+0000","last_fullsized":"2024-09-11T08:41:11.277647+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:25.104224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:25.104224+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:46.893915+0000","scrub_duration":0.001645695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.523437+0000","last_change":"2024-09-11T08:40:46.709390+0000","last_active":"2024-09-11T08:41:09.523437+0000","last_peered":"2024-09-11T08:41:09.523437+0000","last_clean":"2024-09-11T08:41:09.523437+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:41:09.523437+0000","last_undegraded":"2024-09-11T08:41:09.523437+0000","last_fullsized":"2024-09-11T08:41:09.523437+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:46.709322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:46.709322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:12.733287+0000","scrub_duration":0.0017239779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":67,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328785+0000","last_change":"2024-09-11T08:40:22.076288+0000","last_active":"2024-09-11T08:41:09.328785+0000","last_peered":"2024-09-11T08:41:09.328785+0000","last_clean":"2024-09-11T08:41:09.328785+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:41:09.328785+0000","last_undegraded":"2024-09-11T08:41:09.328785+0000","last_fullsized":"2024-09-11T08:41:09.328785+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:40:22.076198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.076198+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:48.206403+0000","scrub_duration":0.0024994370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.823447+0000","last_change":"2024-09-11T08:40:14.202552+0000","last_active":"2024-09-11T08:41:09.823447+0000","last_peered":"2024-09-11T08:41:09.823447+0000","last_clean":"2024-09-11T08:41:09.823447+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:41:09.823447+0000","last_undegraded":"2024-09-11T08:41:09.823447+0000","last_fullsized":"2024-09-11T08:41:09.823447+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:14.202498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:14.202498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:42.406451+0000","scrub_duration":0.001204299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":90,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-11T08:41:10.700433+0000","last_change":"2024-09-11T08:39:44.429546+0000","last_active":"2024-09-11T08:41:10.700433+0000","last_peered":"2024-09-11T08:41:10.700433+0000","last_clean":"2024-09-11T08:41:10.700433+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:41:10.700433+0000","last_undegraded":"2024-09-11T08:41:10.700433+0000","last_fullsized":"2024-09-11T08:41:10.700433+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:39:44.429477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:44.429477+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001890784,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.095855+0000","last_change":"2024-09-11T08:39:52.254299+0000","last_active":"2024-09-11T08:41:09.095855+0000","last_peered":"2024-09-11T08:41:09.095855+0000","last_clean":"2024-09-11T08:41:09.095855+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:41:09.095855+0000","last_undegraded":"2024-09-11T08:41:09.095855+0000","last_fullsized":"2024-09-11T08:41:09.095855+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:52.254245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:39:52.254245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:20.782062+0000","scrub_duration":0.001581877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.523029+0000","last_change":"2024-09-11T08:40:50.753192+0000","last_active":"2024-09-11T08:41:09.523029+0000","last_peered":"2024-09-11T08:41:09.523029+0000","last_clean":"2024-09-11T08:41:09.523029+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:41:09.523029+0000","last_undegraded":"2024-09-11T08:41:09.523029+0000","last_fullsized":"2024-09-11T08:41:09.523029+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:40:50.753126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:50.753126+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:10.210419+0000","scrub_duration":0.0019957540000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.523276+0000","last_change":"2024-09-11T08:40:31.755542+0000","last_active":"2024-09-11T08:41:09.523276+0000","last_peered":"2024-09-11T08:41:09.523276+0000","last_clean":"2024-09-11T08:41:09.523276+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:41:09.523276+0000","last_undegraded":"2024-09-11T08:41:09.523276+0000","last_fullsized":"2024-09-11T08:41:09.523276+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:31.755476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:31.755476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:52.682119+0000","scrub_duration":0.001622713,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.228668+0000","last_change":"2024-09-11T08:40:26.218247+0000","last_active":"2024-09-11T08:41:09.228668+0000","last_peered":"2024-09-11T08:41:09.228668+0000","last_clean":"2024-09-11T08:41:09.228668+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:41:09.228668+0000","last_undegraded":"2024-09-11T08:41:09.228668+0000","last_fullsized":"2024-09-11T08:41:09.228668+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:26.218191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:26.218191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:48.264069+0000","scrub_duration":0.0016980210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328261+0000","last_change":"2024-09-11T08:40:37.233576+0000","last_active":"2024-09-11T08:41:09.328261+0000","last_peered":"2024-09-11T08:41:09.328261+0000","last_clean":"2024-09-11T08:41:09.328261+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:41:09.328261+0000","last_undegraded":"2024-09-11T08:41:09.328261+0000","last_fullsized":"2024-09-11T08:41:09.328261+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:37.233522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:37.233522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:58.908369+0000","scrub_duration":0.001522357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.486275+0000","last_change":"2024-09-11T08:40:22.382123+0000","last_active":"2024-09-11T08:41:09.486275+0000","last_peered":"2024-09-11T08:41:09.486275+0000","last_clean":"2024-09-11T08:41:09.486275+0000","last_became_active":"2024-09-11T08:40:16.181181+0000","last_became_peered":"2024-09-11T08:40:16.181181+0000","last_unstale":"2024-09-11T08:41:09.486275+0000","last_undegraded":"2024-09-11T08:41:09.486275+0000","last_fullsized":"2024-09-11T08:41:09.486275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:40:22.382048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.382048+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:26.268329+0000","scrub_duration":0.001806862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.823369+0000","last_change":"2024-09-11T08:40:28.273944+0000","last_active":"2024-09-11T08:41:09.823369+0000","last_peered":"2024-09-11T08:41:09.823369+0000","last_clean":"2024-09-11T08:41:09.823369+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:41:09.823369+0000","last_undegraded":"2024-09-11T08:41:09.823369+0000","last_fullsized":"2024-09-11T08:41:09.823369+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:28.273889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:28.273889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:42.340922+0000","scrub_duration":0.0011932570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328538+0000","last_change":"2024-09-11T08:40:24.089168+0000","last_active":"2024-09-11T08:41:09.328538+0000","last_peered":"2024-09-11T08:41:09.328538+0000","last_clean":"2024-09-11T08:41:09.328538+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:41:09.328538+0000","last_undegraded":"2024-09-11T08:41:09.328538+0000","last_fullsized":"2024-09-11T08:41:09.328538+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:24.089115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:24.089115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:27.976340+0000","scrub_duration":0.001138305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.095545+0000","last_change":"2024-09-11T08:40:22.499148+0000","last_active":"2024-09-11T08:41:09.095545+0000","last_peered":"2024-09-11T08:41:09.095545+0000","last_clean":"2024-09-11T08:41:09.095545+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:41:09.095545+0000","last_undegraded":"2024-09-11T08:41:09.095545+0000","last_fullsized":"2024-09-11T08:41:09.095545+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:22.499088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:22.499088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:41.706106+0000","scrub_duration":0.0015631830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328352+0000","last_change":"2024-09-11T08:40:54.351650+0000","last_active":"2024-09-11T08:41:09.328352+0000","last_peered":"2024-09-11T08:41:09.328352+0000","last_clean":"2024-09-11T08:41:09.328352+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:41:09.328352+0000","last_undegraded":"2024-09-11T08:41:09.328352+0000","last_fullsized":"2024-09-11T08:41:09.328352+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:40:54.351585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:54.351585+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:14.160580+0000","scrub_duration":0.0021705309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.487018+0000","last_change":"2024-09-11T08:40:05.430740+0000","last_active":"2024-09-11T08:41:09.487018+0000","last_peered":"2024-09-11T08:41:09.487018+0000","last_clean":"2024-09-11T08:41:09.487018+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:41:09.487018+0000","last_undegraded":"2024-09-11T08:41:09.487018+0000","last_fullsized":"2024-09-11T08:41:09.487018+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:05.430666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:05.430666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:15.818673+0000","scrub_duration":0.001547629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.822876+0000","last_change":"2024-09-11T08:40:53.111577+0000","last_active":"2024-09-11T08:41:09.822876+0000","last_peered":"2024-09-11T08:41:09.822876+0000","last_clean":"2024-09-11T08:41:09.822876+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:41:09.822876+0000","last_undegraded":"2024-09-11T08:41:09.822876+0000","last_fullsized":"2024-09-11T08:41:09.822876+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:53.111510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:53.111510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:57.906928+0000","scrub_duration":0.0014220470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328047+0000","last_change":"2024-09-11T08:40:52.333417+0000","last_active":"2024-09-11T08:41:09.328047+0000","last_peered":"2024-09-11T08:41:09.328047+0000","last_clean":"2024-09-11T08:41:09.328047+0000","last_became_active":"2024-09-11T08:38:45.183548+0000","last_became_peered":"2024-09-11T08:38:45.183548+0000","last_unstale":"2024-09-11T08:41:09.328047+0000","last_undegraded":"2024-09-11T08:41:09.328047+0000","last_fullsized":"2024-09-11T08:41:09.328047+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:52.333365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:52.333365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:09.978029+0000","scrub_duration":0.0014027620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328204+0000","last_change":"2024-09-11T08:39:41.989740+0000","last_active":"2024-09-11T08:41:09.328204+0000","last_peered":"2024-09-11T08:41:09.328204+0000","last_clean":"2024-09-11T08:41:09.328204+0000","last_became_active":"2024-09-11T08:38:40.973799+0000","last_became_peered":"2024-09-11T08:38:40.973799+0000","last_unstale":"2024-09-11T08:41:09.328204+0000","last_undegraded":"2024-09-11T08:41:09.328204+0000","last_fullsized":"2024-09-11T08:41:09.328204+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-11T08:39:41.989684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:39:41.989684+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00162959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.097391+0000","last_change":"2024-09-11T08:40:08.336369+0000","last_active":"2024-09-11T08:41:09.097391+0000","last_peered":"2024-09-11T08:41:09.097391+0000","last_clean":"2024-09-11T08:41:09.097391+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:41:09.097391+0000","last_undegraded":"2024-09-11T08:41:09.097391+0000","last_fullsized":"2024-09-11T08:41:09.097391+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:40:08.336314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:40:08.336314+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:16.115575+0000","scrub_duration":0.001394085,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.097209+0000","last_change":"2024-09-11T08:39:53.541628+0000","last_active":"2024-09-11T08:41:09.097209+0000","last_peered":"2024-09-11T08:41:09.097209+0000","last_clean":"2024-09-11T08:41:09.097209+0000","last_became_active":"2024-09-11T08:38:42.989556+0000","last_became_peered":"2024-09-11T08:38:42.989556+0000","last_unstale":"2024-09-11T08:41:09.097209+0000","last_undegraded":"2024-09-11T08:41:09.097209+0000","last_fullsized":"2024-09-11T08:41:09.097209+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.541557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.541557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001310285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.823002+0000","last_change":"2024-09-11T08:40:07.114884+0000","last_active":"2024-09-11T08:41:09.823002+0000","last_peered":"2024-09-11T08:41:09.823002+0000","last_clean":"2024-09-11T08:41:09.823002+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:41:09.823002+0000","last_undegraded":"2024-09-11T08:41:09.823002+0000","last_fullsized":"2024-09-11T08:41:09.823002+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:07.114815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:07.114815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.897957+0000","scrub_duration":0.0013781500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.228895+0000","last_change":"2024-09-11T08:40:56.229818+0000","last_active":"2024-09-11T08:41:09.228895+0000","last_peered":"2024-09-11T08:41:09.228895+0000","last_clean":"2024-09-11T08:41:09.228895+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:41:09.228895+0000","last_undegraded":"2024-09-11T08:41:09.228895+0000","last_fullsized":"2024-09-11T08:41:09.228895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:56.229723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:56.229723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:42:17.824574+0000","scrub_duration":0.0014891800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.328324+0000","last_change":"2024-09-11T08:39:53.038356+0000","last_active":"2024-09-11T08:41:09.328324+0000","last_peered":"2024-09-11T08:41:09.328324+0000","last_clean":"2024-09-11T08:41:09.328324+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:41:09.328324+0000","last_undegraded":"2024-09-11T08:41:09.328324+0000","last_fullsized":"2024-09-11T08:41:09.328324+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:53.038297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:39:53.038297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012198549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.097804+0000","last_change":"2024-09-11T08:40:02.551056+0000","last_active":"2024-09-11T08:41:09.097804+0000","last_peered":"2024-09-11T08:41:09.097804+0000","last_clean":"2024-09-11T08:41:09.097804+0000","last_became_active":"2024-09-11T08:38:40.974451+0000","last_became_peered":"2024-09-11T08:38:40.974451+0000","last_unstale":"2024-09-11T08:41:09.097804+0000","last_undegraded":"2024-09-11T08:41:09.097804+0000","last_fullsized":"2024-09-11T08:41:09.097804+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:40:02.550999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:02.550999+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:41:24.173446+0000","scrub_duration":0.0016484399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-11T08:41:09.096870+0000","last_change":"2024-09-11T08:39:55.265155+0000","last_active":"2024-09-11T08:41:09.096870+0000","last_peered":"2024-09-11T08:41:09.096870+0000","last_clean":"2024-09-11T08:41:09.096870+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:41:09.096870+0000","last_undegraded":"2024-09-11T08:41:09.096870+0000","last_fullsized":"2024-09-11T08:41:09.096870+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:39:55.265097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:39:55.265097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012893830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":846725120,"num_objects":12920,"num_object_clones":0,"num_object_copies":38760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12920,"num_write_kb":826880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2541813760,"data_stored":2541813760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12920,"ondisk_log_size":12920,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":430,"ondisk_log_size":430,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":145,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":550260,"kb_used_data":252908,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93194892,"statfs":{"total":95995035648,"available":95431569408,"internally_reserved":0,"allocated":258977792,"data_stored":258577349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.499,"15min":0.501},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.66300000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71599999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.27100000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.50700000000000001},"min":{"1min":0.32700000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.74099999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.71499999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.255},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.373}]},{"osd":3,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71799999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.53000000000000003},"min":{"1min":0.377,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70299999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":1.7270000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.42499999999999999,"15min":0.42199999999999999},"min":{"1min":0.318,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.59399999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.30099999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.44700000000000001,"15min":0.439},"min":{"1min":0.313,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.54200000000000004}]},{"osd":6,"last update":"Wed Sep 11 08:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47399999999999998,"15min":0.46899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.433},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.39100000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528362,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":545120,"kb_used_data":247704,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93200032,"statfs":{"total":95995035648,"available":95436832768,"internally_reserved":0,"allocated":253648896,"data_stored":253268160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52000000000000002,"15min":0.52900000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76400000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.28999999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51500000000000001,"15min":0.52700000000000002},"min":{"1min":0.34200000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71799999999999997,"5min":0.871,"15min":0.871},"last":0.27600000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.34200000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.73899999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51400000000000001,"15min":0.51800000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69999999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":1.2390000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.41599999999999998,"15min":0.40899999999999997},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.69399999999999995,"5min":0.871,"15min":0.871},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.41899999999999998,"15min":0.40899999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.33800000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45100000000000001,"15min":0.435},"min":{"1min":0.29099999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.754,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.434},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.38900000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:40:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.44400000000000001,"15min":0.432},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.41399999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":491184,"kb_used_data":181352,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93253968,"statfs":{"total":95995035648,"available":95492063232,"internally_reserved":0,"allocated":185704448,"data_stored":185308190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70999999999999996,"5min":0.879,"15min":0.879},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.52700000000000002,"15min":0.52800000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.68300000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.314}]},{"osd":1,"last update":"Wed Sep 11 08:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.55600000000000005,"15min":0.56799999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72199999999999998,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.51500000000000001,"15min":0.51300000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68000000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":1.9450000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.71899999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.55200000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65300000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.68000000000000005}]},{"osd":3,"last update":"Wed Sep 11 08:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.53200000000000003,"15min":0.53700000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.38},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.60599999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38800000000000001,"15min":0.378},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53100000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.40999999999999998,"15min":0.40000000000000002},"min":{"1min":0.309,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.627,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.503}]},{"osd":6,"last update":"Wed Sep 11 08:41:08 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.441,"15min":0.42899999999999999},"min":{"1min":0.29799999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66400000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.439,"15min":0.42399999999999999},"min":{"1min":0.30399999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.72799999999999998,"5min":0.749,"15min":0.749},"last":0.745}]},{"osd":7,"last update":"Wed Sep 11 08:40:27 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.60799999999999998,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.40899999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.56000000000000005}]}]},{"osd":4,"up_from":31,"seq":133143986226,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":528300,"kb_used_data":230948,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93216852,"statfs":{"total":95995035648,"available":95454056448,"internally_reserved":0,"allocated":236490752,"data_stored":236097175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55500000000000005,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.57799999999999996}]},{"osd":1,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.38,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.23599999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.39600000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.372},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.30399999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55800000000000005,"15min":0.57799999999999996},"min":{"1min":0.38800000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72199999999999998,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":1.018},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.42899999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:40:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.42699999999999999,"15min":0.39500000000000002},"min":{"1min":0.27400000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.432,"15min":0.41199999999999998},"min":{"1min":0.29499999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.26600000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.44900000000000001,"15min":0.42999999999999999},"min":{"1min":0.31,"5min":0.248,"15min":0.248},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.495},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.48199999999999998,"15min":0.46500000000000002},"min":{"1min":0.318,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.39100000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:40:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48899999999999999,"15min":0.48099999999999998},"min":{"1min":0.30199999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.316},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.47599999999999998,"15min":0.46800000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.66900000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149752,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":698504,"kb_used_data":401088,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93046648,"statfs":{"total":95995035648,"available":95279767552,"internally_reserved":0,"allocated":410714112,"data_stored":410293688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.28799999999999998,"5min":0.34499999999999997,"15min":0.36799999999999999},"min":{"1min":0.216,"5min":0.184,"15min":0.184},"max":{"1min":0.46400000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.376,"15min":0.39600000000000002},"min":{"1min":0.23400000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.50800000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":1.347}]},{"osd":1,"last update":"Wed Sep 11 08:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.38100000000000001,"15min":0.40100000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58399999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":1.292},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.38700000000000001,"15min":0.40899999999999997},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57399999999999995,"5min":0.621,"15min":0.621},"last":0.53600000000000003}]},{"osd":2,"last update":"Wed Sep 11 08:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.41199999999999998,"15min":0.42999999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63800000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":2.3980000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.39800000000000002,"15min":0.42599999999999999},"min":{"1min":0.253,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51000000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":2.4489999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:40:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79900000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54900000000000004,"15min":0.55900000000000005},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66600000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.70299999999999996}]},{"osd":5,"last update":"Wed Sep 11 08:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77900000000000003,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.41799999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82499999999999996,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Wed Sep 11 08:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58399999999999996,"15min":0.59299999999999997},"min":{"1min":0.39800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":1.25,"15min":1.25},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":1.4259999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:40:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60999999999999999,"15min":0.61799999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86799999999999999,"5min":1.22,"15min":1.22},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.86099999999999999,"5min":1.125,"15min":1.125},"last":0.64600000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":587292,"kb_used_data":289940,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93157860,"statfs":{"total":95995035648,"available":95393648640,"internally_reserved":0,"allocated":296898560,"data_stored":296521322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.36299999999999999,"15min":0.35299999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.55500000000000005,"5min":1.085,"15min":1.085},"last":0.62},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.376,"15min":0.375},"min":{"1min":0.23899999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.437,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.97499999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.255,"5min":0.215,"15min":0.215},"max":{"1min":0.47399999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38800000000000001,"15min":0.375},"min":{"1min":0.27000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.57599999999999996,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":1.202}]},{"osd":3,"last update":"Wed Sep 11 08:40:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.41799999999999998,"15min":0.42599999999999999},"min":{"1min":0.27900000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.44600000000000001,"5min":1.167,"15min":1.167},"last":1.347},{"interface":"front","average":{"1min":0.376,"5min":0.435,"15min":0.438},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.58099999999999996,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.747}]},{"osd":4,"last update":"Wed Sep 11 08:40:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.751,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.33600000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.435,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.71399999999999997,"5min":1.008,"15min":1.008},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.377,"5min":0.373,"15min":0.373},"max":{"1min":0.71999999999999997,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.28999999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77400000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.69799999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.39200000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.41699999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76200000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.436},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81799999999999995,"5min":1.024,"15min":1.024},"last":0.54900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":565960,"kb_used_data":256128,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93179192,"statfs":{"total":95995035648,"available":95415492608,"internally_reserved":0,"allocated":262275072,"data_stored":261853712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:40:19 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.314},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.47399999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33300000000000002},"min":{"1min":0.27400000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":2.774}]},{"osd":2,"last update":"Wed Sep 11 08:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.379,"15min":0.38},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.36499999999999999,"15min":0.36699999999999999},"min":{"1min":0.255,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.55700000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.84899999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38900000000000001,"15min":0.41499999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.47199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.42499999999999999,"15min":0.43099999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.51500000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":1.194}]},{"osd":4,"last update":"Wed Sep 11 08:40:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.76500000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55100000000000005,"15min":0.56499999999999995},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.63300000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.40200000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56200000000000006,"15min":0.57299999999999995},"min":{"1min":0.40600000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71199999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55400000000000005,"15min":0.56399999999999995},"min":{"1min":0.378,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81799999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.50800000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.39100000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.78700000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.92600000000000005}]},{"osd":7,"last update":"Wed Sep 11 08:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.45600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79700000000000004,"5min":0.879,"15min":0.879},"last":0.435}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":682424,"kb_used_data":385072,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93062728,"statfs":{"total":95995035648,"available":95296233472,"internally_reserved":0,"allocated":394313728,"data_stored":393909248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.33200000000000002,"15min":0.313},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.3440000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.38,"15min":0.32700000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":4.0270000000000001,"5min":4.0270000000000001,"15min":4.0270000000000001},"last":4.0270000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35699999999999998,"15min":0.34699999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.47699999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":4.1189999999999998},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.35199999999999998,"15min":0.34799999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.377}]},{"osd":3,"last update":"Wed Sep 11 08:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.38500000000000001,"15min":0.39600000000000002},"min":{"1min":0.246,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.39200000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":3.3860000000000001},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.378,"15min":0.38900000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.39200000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":1.276}]},{"osd":4,"last update":"Wed Sep 11 08:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":1.028},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70899999999999996,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":4.0620000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:40:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53000000000000003,"15min":0.52700000000000002},"min":{"1min":0.40600000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.58099999999999996,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":1.5860000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:41:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68899999999999995,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":1.6200000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:40:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":4.0919999999999996},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.66400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":417464320,"data_stored":417464320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312016896,"data_stored":312016896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321191936,"data_stored":321191936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":482279424,"data_stored":482279424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":265682944,"data_stored":265682944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":215613440,"data_stored":215613440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":263389184,"data_stored":263389184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":264175616,"data_stored":264175616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:41:26.505 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool set .mgr pgp_num 1 2024-09-11T08:41:26.516 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:26.544 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:26.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:26.545 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:26.646 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:26.672 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:26.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:26.673 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:26.684 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 15 15 35971 35956 149.784 126.5 0.00678694 0.00667119 2024-09-11T08:41:26.774 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:26.801 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:26.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:26.802 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:26.902 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:26.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:26.944 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:26.944 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:27.045 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:27.072 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:27.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:27.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:27.145 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:41:27.174 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:27.200 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:27.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:27.201 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:27.302 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:27.326 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:27.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:27.328 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:27.389 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:27 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:27.389 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:27 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:27.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:27 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:27.428 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:27.453 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:27.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:27.454 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:27.505 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:41:27.505 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:41:27.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:27.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:27.592 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:27.592 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:27.684 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 16 16 37678 37662 147.086 106.625 0.0057014 0.00679286 2024-09-11T08:41:27.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:27.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:27.741 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:27.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:27.879 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:27.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:27.880 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:27.981 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:28.009 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:28.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:28.011 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:28.112 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:28.138 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:28.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:28.140 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:28.142 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:41:28.215 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-11T08:41:28.240 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:28.267 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:28.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:28.268 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:28.369 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:28.399 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:28.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:28.400 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:28.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:28 smithi145 ceph-mon[33384]: pgmap v92: 148 pgs: 148 active+clean; 2.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 180 MiB/s wr, 2.89k op/s 2024-09-11T08:41:28.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:28 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2247844140' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:41:28.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:28.559 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:28.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:28.560 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:28.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:28 smithi092 ceph-mon[29647]: pgmap v92: 148 pgs: 148 active+clean; 2.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 180 MiB/s wr, 2.89k op/s 2024-09-11T08:41:28.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:28 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2247844140' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:41:28.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:28 smithi092 ceph-mon[41114]: pgmap v92: 148 pgs: 148 active+clean; 2.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 180 MiB/s wr, 2.89k op/s 2024-09-11T08:41:28.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:28 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2247844140' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:41:28.661 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:28.684 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 17 16 39528 39512 145.236 115.625 0.00673484 0.006879 2024-09-11T08:41:28.691 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:28.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:28.692 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:28.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:28.833 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:28.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:28.834 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:28.935 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:28.965 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:28.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:28.966 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:29.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:29.109 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:29.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:29.110 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:29.212 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:29.239 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:29.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:29.240 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:29.260 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:29.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:29.346 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:29 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:41:29.346 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:29 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2247844140' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:41:29.346 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:29 smithi092 ceph-mon[29647]: osdmap e93: 8 total, 8 up, 8 in 2024-09-11T08:41:29.346 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:29 smithi092 ceph-mon[29647]: osdmap e94: 8 total, 8 up, 8 in 2024-09-11T08:41:29.346 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:29 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:41:29.346 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:29 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2247844140' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:41:29.347 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:29 smithi092 ceph-mon[41114]: osdmap e93: 8 total, 8 up, 8 in 2024-09-11T08:41:29.347 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:29 smithi092 ceph-mon[41114]: osdmap e94: 8 total, 8 up, 8 in 2024-09-11T08:41:29.379 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:29.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:29.381 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:29.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:29 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:41:29.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:29 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2247844140' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:41:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:29 smithi145 ceph-mon[33384]: osdmap e93: 8 total, 8 up, 8 in 2024-09-11T08:41:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:29 smithi145 ceph-mon[33384]: osdmap e94: 8 total, 8 up, 8 in 2024-09-11T08:41:29.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:29.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:29.527 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:29.527 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:29.628 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:29.660 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:29.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:29.661 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:29.684 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 18 16 41337 41321 143.448 113.062 0.00787219 0.00696608 2024-09-11T08:41:29.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:29.800 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:29.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:29.801 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:29.902 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:29.932 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:29.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:29.933 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:30.034 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:30.069 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:30.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:30.070 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:30.170 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:30.198 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:30.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:30.199 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:30.300 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:30.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:30 smithi092 ceph-mon[29647]: pgmap v95: 148 pgs: 148 active+clean; 2.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 121 MiB/s wr, 1.94k op/s 2024-09-11T08:41:30.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:30 smithi092 ceph-mon[29647]: 6.7 scrub starts 2024-09-11T08:41:30.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:30 smithi092 ceph-mon[29647]: 6.7 scrub ok 2024-09-11T08:41:30.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:30.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:30 smithi092 ceph-mon[29647]: 3.11 scrub starts 2024-09-11T08:41:30.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:30 smithi092 ceph-mon[29647]: 3.11 scrub ok 2024-09-11T08:41:30.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:30 smithi092 ceph-mon[41114]: pgmap v95: 148 pgs: 148 active+clean; 2.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 121 MiB/s wr, 1.94k op/s 2024-09-11T08:41:30.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:30 smithi092 ceph-mon[41114]: 6.7 scrub starts 2024-09-11T08:41:30.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:30 smithi092 ceph-mon[41114]: 6.7 scrub ok 2024-09-11T08:41:30.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:30.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:30 smithi092 ceph-mon[41114]: 3.11 scrub starts 2024-09-11T08:41:30.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:30 smithi092 ceph-mon[41114]: 3.11 scrub ok 2024-09-11T08:41:30.328 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:30.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:30.329 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:30.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:30 smithi145 ceph-mon[33384]: pgmap v95: 148 pgs: 148 active+clean; 2.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 121 MiB/s wr, 1.94k op/s 2024-09-11T08:41:30.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:30 smithi145 ceph-mon[33384]: 6.7 scrub starts 2024-09-11T08:41:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:30 smithi145 ceph-mon[33384]: 6.7 scrub ok 2024-09-11T08:41:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:30 smithi145 ceph-mon[33384]: 3.11 scrub starts 2024-09-11T08:41:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:30 smithi145 ceph-mon[33384]: 3.11 scrub ok 2024-09-11T08:41:30.430 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:30.471 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:30.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:30.472 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:30.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:30.611 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:30.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:30.612 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:30.684 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 19 16 43219 43203 142.088 117.625 0.00306889 0.00703185 2024-09-11T08:41:30.713 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:30.742 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:30.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:30.743 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:30.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:30.882 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:30.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:30.883 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:30.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:31.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:31.029 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:31.029 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:31.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:31.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:41:31.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:31.177 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:31.177 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:31.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:31 smithi092 ceph-mon[29647]: 4.4 scrub starts 2024-09-11T08:41:31.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:31 smithi092 ceph-mon[29647]: 4.4 scrub ok 2024-09-11T08:41:31.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:31 smithi092 ceph-mon[29647]: 3.1f scrub starts 2024-09-11T08:41:31.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:31 smithi092 ceph-mon[29647]: 3.1f scrub ok 2024-09-11T08:41:31.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:31 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:41:31.274 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:31 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:41:31.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:31 smithi092 ceph-mon[41114]: 4.4 scrub starts 2024-09-11T08:41:31.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:31 smithi092 ceph-mon[41114]: 4.4 scrub ok 2024-09-11T08:41:31.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:31 smithi092 ceph-mon[41114]: 3.1f scrub starts 2024-09-11T08:41:31.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:31 smithi092 ceph-mon[41114]: 3.1f scrub ok 2024-09-11T08:41:31.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:31 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:41:31.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:31 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:41:31.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:31.330 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:31.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:31.332 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:31.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:31.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:31.476 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:31.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:41:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:41:31] "GET /metrics HTTP/1.1" 200 223394 "" "Prometheus/2.43.0" 2024-09-11T08:41:31.577 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:31.605 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:31.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:31.606 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:31.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:31 smithi145 ceph-mon[33384]: 4.4 scrub starts 2024-09-11T08:41:31.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:31 smithi145 ceph-mon[33384]: 4.4 scrub ok 2024-09-11T08:41:31.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:31 smithi145 ceph-mon[33384]: 3.1f scrub starts 2024-09-11T08:41:31.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:31 smithi145 ceph-mon[33384]: 3.1f scrub ok 2024-09-11T08:41:31.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:31 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:41:31.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:31 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:41:31.684 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:41:31.685928+0000 min lat: 0.00133167 max lat: 0.172138 avg lat: 0.00711844 2024-09-11T08:41:31.685 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:41:31.685 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 20 16 44944 44928 140.374 107.812 0.0163644 0.00711844 2024-09-11T08:41:31.707 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:31.735 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:31.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:31.736 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:31.837 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:31.866 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:31.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:31.867 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:31.968 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:31.996 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:31.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:31.998 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:32.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:32.142 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:32.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:32.143 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:32.243 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:32.279 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:32.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:32.280 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:32.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:32 smithi092 ceph-mon[29647]: pgmap v96: 148 pgs: 148 active+clean; 2.4 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 110 MiB/s wr, 1.77k op/s 2024-09-11T08:41:32.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:32 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:41:32.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:32 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:41:32.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:32 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:41:32.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:32 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:41:32.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:32 smithi092 ceph-mon[29647]: osdmap e95: 8 total, 8 up, 8 in 2024-09-11T08:41:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:32 smithi092 ceph-mon[41114]: pgmap v96: 148 pgs: 148 active+clean; 2.4 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 110 MiB/s wr, 1.77k op/s 2024-09-11T08:41:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:32 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:41:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:32 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:41:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:32 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:41:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:32 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:41:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:32 smithi092 ceph-mon[41114]: osdmap e95: 8 total, 8 up, 8 in 2024-09-11T08:41:32.381 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:32.409 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:32.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:32.409 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:32.510 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:32.537 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:32.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:32.539 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:32.639 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:32.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:32 smithi145 ceph-mon[33384]: pgmap v96: 148 pgs: 148 active+clean; 2.4 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 110 MiB/s wr, 1.77k op/s 2024-09-11T08:41:32.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:32 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:41:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:32 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:41:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:32 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:41:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:32 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:41:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:32 smithi145 ceph-mon[33384]: osdmap e95: 8 total, 8 up, 8 in 2024-09-11T08:41:32.672 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:32.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:32.673 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:32.684 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 21 16 46755 46739 139.079 113.188 0.00707225 0.00718518 2024-09-11T08:41:32.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:32.819 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:32.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:32.820 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:32.920 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:32.950 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:32.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:32.951 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:33.052 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:33.087 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:33.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:33.089 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:33.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:41:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:41:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:33.190 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:33.228 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:33.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:33.229 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:33.330 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:33.357 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:33.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:33.358 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:33.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:33 smithi092 ceph-mon[29647]: 4.7 scrub starts 2024-09-11T08:41:33.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:33 smithi092 ceph-mon[29647]: 4.7 scrub ok 2024-09-11T08:41:33.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:33 smithi092 ceph-mon[29647]: 3.6 scrub starts 2024-09-11T08:41:33.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:33 smithi092 ceph-mon[29647]: 3.6 scrub ok 2024-09-11T08:41:33.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:33 smithi092 ceph-mon[29647]: 4.5 scrub starts 2024-09-11T08:41:33.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:33 smithi092 ceph-mon[29647]: 4.5 scrub ok 2024-09-11T08:41:33.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:33 smithi092 ceph-mon[41114]: 4.7 scrub starts 2024-09-11T08:41:33.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:33 smithi092 ceph-mon[41114]: 4.7 scrub ok 2024-09-11T08:41:33.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:33 smithi092 ceph-mon[41114]: 3.6 scrub starts 2024-09-11T08:41:33.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:33 smithi092 ceph-mon[41114]: 3.6 scrub ok 2024-09-11T08:41:33.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:33 smithi092 ceph-mon[41114]: 4.5 scrub starts 2024-09-11T08:41:33.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:33 smithi092 ceph-mon[41114]: 4.5 scrub ok 2024-09-11T08:41:33.459 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:33.483 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:33.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:33.484 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:33.585 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:33.612 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:33.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:33.613 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:33.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:33 smithi145 ceph-mon[33384]: 4.7 scrub starts 2024-09-11T08:41:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:33 smithi145 ceph-mon[33384]: 4.7 scrub ok 2024-09-11T08:41:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:33 smithi145 ceph-mon[33384]: 3.6 scrub starts 2024-09-11T08:41:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:33 smithi145 ceph-mon[33384]: 3.6 scrub ok 2024-09-11T08:41:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:33 smithi145 ceph-mon[33384]: 4.5 scrub starts 2024-09-11T08:41:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:33 smithi145 ceph-mon[33384]: 4.5 scrub ok 2024-09-11T08:41:33.685 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 22 16 48461 48445 137.604 106.625 0.0184835 0.00726156 2024-09-11T08:41:33.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:33.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:33.763 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:33.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:33.865 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:33.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:33.904 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:33.904 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:34.005 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:34.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:34.033 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:34.033 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:34.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:34.172 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:34.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:34.174 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:34.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:34.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:34.314 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:34.314 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:34.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:34.452 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:34.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:34.453 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:34.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:34.614 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: 5.19 scrub starts 2024-09-11T08:41:34.614 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: 5.19 scrub ok 2024-09-11T08:41:34.614 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: 3.18 scrub starts 2024-09-11T08:41:34.614 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: 3.18 scrub ok 2024-09-11T08:41:34.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: 5.6 scrub starts 2024-09-11T08:41:34.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: 5.6 scrub ok 2024-09-11T08:41:34.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: pgmap v98: 148 pgs: 148 active+clean; 2.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-11T08:41:34.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: 5.16 scrub starts 2024-09-11T08:41:34.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: 5.16 scrub ok 2024-09-11T08:41:34.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: 2.1 scrub starts 2024-09-11T08:41:34.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: 2.1 scrub ok 2024-09-11T08:41:34.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: osdmap e96: 8 total, 8 up, 8 in 2024-09-11T08:41:34.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: 5.4 scrub starts 2024-09-11T08:41:34.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: 5.4 scrub ok 2024-09-11T08:41:34.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2494512483' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]: dispatch 2024-09-11T08:41:34.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]: dispatch 2024-09-11T08:41:34.616 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: 5.19 scrub starts 2024-09-11T08:41:34.616 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: 5.19 scrub ok 2024-09-11T08:41:34.616 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: 3.18 scrub starts 2024-09-11T08:41:34.616 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: 3.18 scrub ok 2024-09-11T08:41:34.616 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: 5.6 scrub starts 2024-09-11T08:41:34.616 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: 5.6 scrub ok 2024-09-11T08:41:34.616 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: pgmap v98: 148 pgs: 148 active+clean; 2.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-11T08:41:34.616 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: 5.16 scrub starts 2024-09-11T08:41:34.616 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: 5.16 scrub ok 2024-09-11T08:41:34.616 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: 2.1 scrub starts 2024-09-11T08:41:34.617 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: 2.1 scrub ok 2024-09-11T08:41:34.617 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: osdmap e96: 8 total, 8 up, 8 in 2024-09-11T08:41:34.617 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: 5.4 scrub starts 2024-09-11T08:41:34.617 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: 5.4 scrub ok 2024-09-11T08:41:34.617 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2494512483' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]: dispatch 2024-09-11T08:41:34.617 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:34 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]: dispatch 2024-09-11T08:41:34.657 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:34.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:34.658 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:34.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: 5.19 scrub starts 2024-09-11T08:41:34.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: 5.19 scrub ok 2024-09-11T08:41:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: 3.18 scrub starts 2024-09-11T08:41:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: 3.18 scrub ok 2024-09-11T08:41:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: 5.6 scrub starts 2024-09-11T08:41:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: 5.6 scrub ok 2024-09-11T08:41:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: pgmap v98: 148 pgs: 148 active+clean; 2.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-11T08:41:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: 5.16 scrub starts 2024-09-11T08:41:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: 5.16 scrub ok 2024-09-11T08:41:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: 2.1 scrub starts 2024-09-11T08:41:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: 2.1 scrub ok 2024-09-11T08:41:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: osdmap e96: 8 total, 8 up, 8 in 2024-09-11T08:41:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: 5.4 scrub starts 2024-09-11T08:41:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: 5.4 scrub ok 2024-09-11T08:41:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2494512483' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]: dispatch 2024-09-11T08:41:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:34 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]: dispatch 2024-09-11T08:41:34.685 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 23 16 50063 50047 135.974 100.125 0.00542603 0.00734929 2024-09-11T08:41:34.724 INFO:teuthology.orchestra.run.smithi092.stderr:set pool 1 pgp_num to 1 2024-09-11T08:41:34.759 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:34.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:34.800 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:34.800 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:34.901 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:34.928 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:34.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:34.929 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:35.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:35.060 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:35.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:35.062 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:35.162 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:35.206 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:35.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:35.207 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:35.308 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:35.336 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:35.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:35.337 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:35.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:35.477 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:35.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:35.478 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:35.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[29647]: 3.4 scrub starts 2024-09-11T08:41:35.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[29647]: 3.4 scrub ok 2024-09-11T08:41:35.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[29647]: 3.1e scrub starts 2024-09-11T08:41:35.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[29647]: 3.1e scrub ok 2024-09-11T08:41:35.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[29647]: 5.15 scrub starts 2024-09-11T08:41:35.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[29647]: 5.15 scrub ok 2024-09-11T08:41:35.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-11T08:41:35.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[29647]: osdmap e97: 8 total, 8 up, 8 in 2024-09-11T08:41:35.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[29647]: 5.1a scrub starts 2024-09-11T08:41:35.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[29647]: 5.1a scrub ok 2024-09-11T08:41:35.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[41114]: 3.4 scrub starts 2024-09-11T08:41:35.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[41114]: 3.4 scrub ok 2024-09-11T08:41:35.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[41114]: 3.1e scrub starts 2024-09-11T08:41:35.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[41114]: 3.1e scrub ok 2024-09-11T08:41:35.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[41114]: 5.15 scrub starts 2024-09-11T08:41:35.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[41114]: 5.15 scrub ok 2024-09-11T08:41:35.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-11T08:41:35.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[41114]: osdmap e97: 8 total, 8 up, 8 in 2024-09-11T08:41:35.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[41114]: 5.1a scrub starts 2024-09-11T08:41:35.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:35 smithi092 ceph-mon[41114]: 5.1a scrub ok 2024-09-11T08:41:35.579 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:35.606 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:35.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:35.607 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:35.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:35 smithi145 ceph-mon[33384]: 3.4 scrub starts 2024-09-11T08:41:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:35 smithi145 ceph-mon[33384]: 3.4 scrub ok 2024-09-11T08:41:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:35 smithi145 ceph-mon[33384]: 3.1e scrub starts 2024-09-11T08:41:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:35 smithi145 ceph-mon[33384]: 3.1e scrub ok 2024-09-11T08:41:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:35 smithi145 ceph-mon[33384]: 5.15 scrub starts 2024-09-11T08:41:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:35 smithi145 ceph-mon[33384]: 5.15 scrub ok 2024-09-11T08:41:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:35 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-11T08:41:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:35 smithi145 ceph-mon[33384]: osdmap e97: 8 total, 8 up, 8 in 2024-09-11T08:41:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:35 smithi145 ceph-mon[33384]: 5.1a scrub starts 2024-09-11T08:41:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:35 smithi145 ceph-mon[33384]: 5.1a scrub ok 2024-09-11T08:41:35.685 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 24 15 51628 51613 134.386 97.875 0.00718108 0.00743633 2024-09-11T08:41:35.708 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:35.734 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:35.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:35.735 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:35.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:35.892 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:35.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:35.893 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:35.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:36.035 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:36.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:36.036 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:36.137 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:36.165 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:36.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:36.166 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:36.266 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:36.294 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:36.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:36.295 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:36.378 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:36 smithi092 ceph-mon[29647]: 3.1 scrub starts 2024-09-11T08:41:36.379 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:36 smithi092 ceph-mon[29647]: 3.1 scrub ok 2024-09-11T08:41:36.379 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:36 smithi092 ceph-mon[29647]: 5.11 scrub starts 2024-09-11T08:41:36.379 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:36 smithi092 ceph-mon[29647]: 5.11 scrub ok 2024-09-11T08:41:36.379 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:36 smithi092 ceph-mon[29647]: pgmap v101: 148 pgs: 148 active+clean; 2.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-11T08:41:36.379 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:36 smithi092 ceph-mon[41114]: 3.1 scrub starts 2024-09-11T08:41:36.379 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:36 smithi092 ceph-mon[41114]: 3.1 scrub ok 2024-09-11T08:41:36.380 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:36 smithi092 ceph-mon[41114]: 5.11 scrub starts 2024-09-11T08:41:36.380 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:36 smithi092 ceph-mon[41114]: 5.11 scrub ok 2024-09-11T08:41:36.380 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:36 smithi092 ceph-mon[41114]: pgmap v101: 148 pgs: 148 active+clean; 2.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-11T08:41:36.396 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:36.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:41:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:41:36.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:36 smithi145 ceph-mon[33384]: 3.1 scrub starts 2024-09-11T08:41:36.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:36 smithi145 ceph-mon[33384]: 3.1 scrub ok 2024-09-11T08:41:36.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:36 smithi145 ceph-mon[33384]: 5.11 scrub starts 2024-09-11T08:41:36.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:36 smithi145 ceph-mon[33384]: 5.11 scrub ok 2024-09-11T08:41:36.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:36 smithi145 ceph-mon[33384]: pgmap v101: 148 pgs: 148 active+clean; 2.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-11T08:41:36.423 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:36.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:36.424 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:36.450 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:41:36.450 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:41:36.481 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:41:36.481 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:41:36.512 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:41:36.512 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:41:36.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:36.546 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:41:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:41:36.563 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:36.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:36.564 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:36.575 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:41:36.575 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:41:36.607 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:41:36.607 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:41:36.640 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:41:36.641 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:41:36.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:36.681 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:41:36.682 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:41:36.685 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 25 16 53200 53184 132.938 98.1875 0.00816003 0.00751602 2024-09-11T08:41:36.706 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:36.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:36.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:36.745 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:41:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:41:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:41:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:41:36.745 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:41:36.808 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:36.837 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:36.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:36.838 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:36.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:36.978 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:36.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:36.979 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:37.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:37.119 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:37.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:37.120 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:37.221 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:37.249 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:37.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:37.250 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:37.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:37 smithi092 ceph-mon[29647]: 6.18 scrub starts 2024-09-11T08:41:37.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:37 smithi092 ceph-mon[29647]: 6.18 scrub ok 2024-09-11T08:41:37.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:37 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:37.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:37 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4088518594' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:41:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:37 smithi092 ceph-mon[41114]: 6.18 scrub starts 2024-09-11T08:41:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:37 smithi092 ceph-mon[41114]: 6.18 scrub ok 2024-09-11T08:41:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:37 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:37 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4088518594' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:41:37.351 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:37.379 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:37.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:37.380 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:37.481 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:37.510 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:37.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:37.511 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:37.612 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:37.640 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:37.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:37.641 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:37 smithi145 ceph-mon[33384]: 6.18 scrub starts 2024-09-11T08:41:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:37 smithi145 ceph-mon[33384]: 6.18 scrub ok 2024-09-11T08:41:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:37 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:37 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4088518594' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:41:37.685 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 26 16 54808 54792 131.69 100.5 0.00402025 0.0075889 2024-09-11T08:41:37.743 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:37.771 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:37.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:37.772 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:37.873 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:37.914 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:37.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:37.915 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:38.016 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:38.044 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:38.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:38.045 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:38.146 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:38.174 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:38.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:38.175 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:38.176 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-11T08:41:38.276 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:38.304 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:38.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:38.305 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:38.406 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:38.431 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:38.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:38.432 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:38.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[29647]: 4.10 scrub starts 2024-09-11T08:41:38.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[29647]: 4.10 scrub ok 2024-09-11T08:41:38.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[29647]: pgmap v102: 148 pgs: 148 active+clean; 3.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 135 MiB/s wr, 2.16k op/s 2024-09-11T08:41:38.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[29647]: 3.0 scrub starts 2024-09-11T08:41:38.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[29647]: 3.0 scrub ok 2024-09-11T08:41:38.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[29647]: 4.8 scrub starts 2024-09-11T08:41:38.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[29647]: 4.8 scrub ok 2024-09-11T08:41:38.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T08:41:38.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[29647]: 4.1c scrub ok 2024-09-11T08:41:38.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[29647]: 3.e scrub starts 2024-09-11T08:41:38.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[29647]: 3.e scrub ok 2024-09-11T08:41:38.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[41114]: 4.10 scrub starts 2024-09-11T08:41:38.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[41114]: 4.10 scrub ok 2024-09-11T08:41:38.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[41114]: pgmap v102: 148 pgs: 148 active+clean; 3.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 135 MiB/s wr, 2.16k op/s 2024-09-11T08:41:38.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[41114]: 3.0 scrub starts 2024-09-11T08:41:38.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[41114]: 3.0 scrub ok 2024-09-11T08:41:38.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[41114]: 4.8 scrub starts 2024-09-11T08:41:38.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[41114]: 4.8 scrub ok 2024-09-11T08:41:38.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T08:41:38.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[41114]: 4.1c scrub ok 2024-09-11T08:41:38.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[41114]: 3.e scrub starts 2024-09-11T08:41:38.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:38 smithi092 ceph-mon[41114]: 3.e scrub ok 2024-09-11T08:41:38.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:38.575 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:38.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:38.577 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:38.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:38 smithi145 ceph-mon[33384]: 4.10 scrub starts 2024-09-11T08:41:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:38 smithi145 ceph-mon[33384]: 4.10 scrub ok 2024-09-11T08:41:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:38 smithi145 ceph-mon[33384]: pgmap v102: 148 pgs: 148 active+clean; 3.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 135 MiB/s wr, 2.16k op/s 2024-09-11T08:41:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:38 smithi145 ceph-mon[33384]: 3.0 scrub starts 2024-09-11T08:41:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:38 smithi145 ceph-mon[33384]: 3.0 scrub ok 2024-09-11T08:41:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:38 smithi145 ceph-mon[33384]: 4.8 scrub starts 2024-09-11T08:41:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:38 smithi145 ceph-mon[33384]: 4.8 scrub ok 2024-09-11T08:41:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:38 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T08:41:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:38 smithi145 ceph-mon[33384]: 4.1c scrub ok 2024-09-11T08:41:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:38 smithi145 ceph-mon[33384]: 3.e scrub starts 2024-09-11T08:41:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:38 smithi145 ceph-mon[33384]: 3.e scrub ok 2024-09-11T08:41:38.678 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:38.685 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 27 16 56307 56291 130.283 93.6875 0.01432 0.0076698 2024-09-11T08:41:38.704 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:38.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:38.705 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:38.805 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:38.841 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:38.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:38.842 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:38.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:38.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:38.979 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:38.979 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:39.080 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:39.109 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:39.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:39.110 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:39.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:39.250 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:39.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:39.251 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:39.352 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:39.387 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:39.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:39.388 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:39.489 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:39.521 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:39.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:39.522 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:39 smithi092 ceph-mon[29647]: 5.1e scrub starts 2024-09-11T08:41:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:39 smithi092 ceph-mon[29647]: 5.1e scrub ok 2024-09-11T08:41:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:39 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:41:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:39 smithi092 ceph-mon[29647]: osdmap e98: 8 total, 8 up, 8 in 2024-09-11T08:41:39.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:39 smithi092 ceph-mon[41114]: 5.1e scrub starts 2024-09-11T08:41:39.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:39 smithi092 ceph-mon[41114]: 5.1e scrub ok 2024-09-11T08:41:39.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:39 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:41:39.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:39 smithi092 ceph-mon[41114]: osdmap e98: 8 total, 8 up, 8 in 2024-09-11T08:41:39.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:39.656 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:39.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:39.657 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:39.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:39 smithi145 ceph-mon[33384]: 5.1e scrub starts 2024-09-11T08:41:39.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:39 smithi145 ceph-mon[33384]: 5.1e scrub ok 2024-09-11T08:41:39.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:39 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:41:39.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:39 smithi145 ceph-mon[33384]: osdmap e98: 8 total, 8 up, 8 in 2024-09-11T08:41:39.687 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 28 16 57909 57893 129.205 100.125 0.00499824 0.0077343 2024-09-11T08:41:39.758 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:39.785 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:39.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:39.786 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:39.821 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:39.886 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:39.914 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:39.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:39.915 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:39.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:41:40.016 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:40.043 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:40.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:40.044 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:40.145 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:40.172 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:40.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:40.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:40.274 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:40.302 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:40.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:40.303 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:40.403 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:40.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:40 smithi145 ceph-mon[33384]: 5.8 scrub ok 2024-09-11T08:41:40.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:40 smithi145 ceph-mon[33384]: 5.2 scrub starts 2024-09-11T08:41:40.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:40 smithi145 ceph-mon[33384]: 5.2 scrub ok 2024-09-11T08:41:40.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:40 smithi145 ceph-mon[33384]: pgmap v104: 148 pgs: 148 active+clean; 3.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-11T08:41:40.431 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:40.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:40.432 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:40.486 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:40 smithi092 ceph-mon[29647]: 5.8 scrub ok 2024-09-11T08:41:40.487 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:40 smithi092 ceph-mon[29647]: 5.2 scrub starts 2024-09-11T08:41:40.487 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:40 smithi092 ceph-mon[29647]: 5.2 scrub ok 2024-09-11T08:41:40.487 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:40 smithi092 ceph-mon[29647]: pgmap v104: 148 pgs: 148 active+clean; 3.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-11T08:41:40.487 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:40 smithi092 ceph-mon[41114]: 5.8 scrub ok 2024-09-11T08:41:40.487 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:40 smithi092 ceph-mon[41114]: 5.2 scrub starts 2024-09-11T08:41:40.487 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:40 smithi092 ceph-mon[41114]: 5.2 scrub ok 2024-09-11T08:41:40.487 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:40 smithi092 ceph-mon[41114]: pgmap v104: 148 pgs: 148 active+clean; 3.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-11T08:41:40.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:40.586 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:40.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:40.587 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:40.686 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 29 16 59256 59240 127.652 84.1875 0.00986828 0.00782905 2024-09-11T08:41:40.688 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:40.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:40.732 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:40.732 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:40.833 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:40.859 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:40.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:40.860 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:40.885 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:41:40.885 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:41:40.886 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-11T08:41:40.886 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T08:41:40.886 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T08:41:40.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 config set filestore_inject_stall 3 2024-09-11T08:41:40.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:41.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:41.010 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:41.111 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:41.139 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:41.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:41.140 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:41.240 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:41.245 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:41:41.266 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:41.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:41.267 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:41.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[29647]: 4.a scrub starts 2024-09-11T08:41:41.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[29647]: 4.a scrub ok 2024-09-11T08:41:41.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[29647]: 3.19 scrub starts 2024-09-11T08:41:41.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[29647]: 3.19 scrub ok 2024-09-11T08:41:41.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[29647]: 6.1a scrub starts 2024-09-11T08:41:41.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[29647]: 6.1a scrub ok 2024-09-11T08:41:41.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[29647]: 3.16 scrub starts 2024-09-11T08:41:41.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[29647]: 3.16 scrub ok 2024-09-11T08:41:41.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[29647]: 6.14 scrub starts 2024-09-11T08:41:41.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[29647]: 6.14 scrub ok 2024-09-11T08:41:41.320 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2524950487' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:41:41.320 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:41:41.320 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[41114]: 4.a scrub starts 2024-09-11T08:41:41.320 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[41114]: 4.a scrub ok 2024-09-11T08:41:41.320 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[41114]: 3.19 scrub starts 2024-09-11T08:41:41.320 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[41114]: 3.19 scrub ok 2024-09-11T08:41:41.320 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[41114]: 6.1a scrub starts 2024-09-11T08:41:41.320 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[41114]: 6.1a scrub ok 2024-09-11T08:41:41.320 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[41114]: 3.16 scrub starts 2024-09-11T08:41:41.321 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[41114]: 3.16 scrub ok 2024-09-11T08:41:41.321 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[41114]: 6.14 scrub starts 2024-09-11T08:41:41.321 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[41114]: 6.14 scrub ok 2024-09-11T08:41:41.321 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2524950487' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:41:41.321 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:41:41.368 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:41.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:41.395 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:41.395 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:41.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:41.533 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:41.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:41.534 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:41.575 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:41.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:41 smithi145 ceph-mon[33384]: 4.a scrub starts 2024-09-11T08:41:41.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:41 smithi145 ceph-mon[33384]: 4.a scrub ok 2024-09-11T08:41:41.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:41 smithi145 ceph-mon[33384]: 3.19 scrub starts 2024-09-11T08:41:41.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:41 smithi145 ceph-mon[33384]: 3.19 scrub ok 2024-09-11T08:41:41.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:41 smithi145 ceph-mon[33384]: 6.1a scrub starts 2024-09-11T08:41:41.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:41 smithi145 ceph-mon[33384]: 6.1a scrub ok 2024-09-11T08:41:41.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:41 smithi145 ceph-mon[33384]: 3.16 scrub starts 2024-09-11T08:41:41.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:41 smithi145 ceph-mon[33384]: 3.16 scrub ok 2024-09-11T08:41:41.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:41 smithi145 ceph-mon[33384]: 6.14 scrub starts 2024-09-11T08:41:41.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:41 smithi145 ceph-mon[33384]: 6.14 scrub ok 2024-09-11T08:41:41.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:41 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2524950487' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:41:41.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:41 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:41:41.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:41.683 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:41.683 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:41.685 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 30 16 60604 60588 126.205 84.25 0.00723618 0.00791795 2024-09-11T08:41:41.719 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:41:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:41:41] "GET /metrics HTTP/1.1" 200 223394 "" "Prometheus/2.43.0" 2024-09-11T08:41:41.785 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:41.813 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:41.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:41.814 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:41.915 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:41.940 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:41.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:41.941 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:42.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:42.079 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:42.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:42.080 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:42.181 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:42.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:42.209 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:42.210 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:42.310 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:42.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[29647]: 3.17 scrub starts 2024-09-11T08:41:42.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[29647]: 3.17 scrub ok 2024-09-11T08:41:42.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[29647]: pgmap v105: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.4 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-11T08:41:42.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:41:42.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:41:42.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:41:42.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[29647]: osdmap e99: 8 total, 8 up, 8 in 2024-09-11T08:41:42.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T08:41:42.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[29647]: 5.d scrub ok 2024-09-11T08:41:42.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[29647]: 6.12 scrub starts 2024-09-11T08:41:42.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[29647]: 6.12 scrub ok 2024-09-11T08:41:42.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[41114]: 3.17 scrub starts 2024-09-11T08:41:42.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[41114]: 3.17 scrub ok 2024-09-11T08:41:42.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[41114]: pgmap v105: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.4 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-11T08:41:42.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:41:42.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:41:42.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:41:42.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[41114]: osdmap e99: 8 total, 8 up, 8 in 2024-09-11T08:41:42.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T08:41:42.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[41114]: 5.d scrub ok 2024-09-11T08:41:42.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[41114]: 6.12 scrub starts 2024-09-11T08:41:42.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:42 smithi092 ceph-mon[41114]: 6.12 scrub ok 2024-09-11T08:41:42.338 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:42.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:42.339 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:42.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:42.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:42.490 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:42.490 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:42.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:42.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:42.641 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:42.641 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:42 smithi145 ceph-mon[33384]: 3.17 scrub starts 2024-09-11T08:41:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:42 smithi145 ceph-mon[33384]: 3.17 scrub ok 2024-09-11T08:41:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:42 smithi145 ceph-mon[33384]: pgmap v105: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.4 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-11T08:41:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:42 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:41:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:42 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:41:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:42 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:41:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:42 smithi145 ceph-mon[33384]: osdmap e99: 8 total, 8 up, 8 in 2024-09-11T08:41:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:42 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T08:41:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:42 smithi145 ceph-mon[33384]: 5.d scrub ok 2024-09-11T08:41:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:42 smithi145 ceph-mon[33384]: 6.12 scrub starts 2024-09-11T08:41:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:42 smithi145 ceph-mon[33384]: 6.12 scrub ok 2024-09-11T08:41:42.686 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 31 16 61964 61948 124.873 85 0.00758754 0.00800323 2024-09-11T08:41:42.742 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:42.767 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:42.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:42.769 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:42.870 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:42.897 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:42.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:42.898 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:42.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:43.039 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:43.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:43.042 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:43.043 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:43.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:43.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:41:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:41:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:43.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:43.193 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:43.194 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:43.294 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:43.322 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:43.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:43.323 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:43.424 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:43.457 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:43.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:43.459 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:43.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:43.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:43 smithi092 ceph-mon[29647]: 6.16 scrub starts 2024-09-11T08:41:43.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:43 smithi092 ceph-mon[29647]: 6.16 scrub ok 2024-09-11T08:41:43.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:43 smithi092 ceph-mon[29647]: 3.d scrub starts 2024-09-11T08:41:43.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:43 smithi092 ceph-mon[29647]: 3.d scrub ok 2024-09-11T08:41:43.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:43 smithi092 ceph-mon[41114]: 6.16 scrub starts 2024-09-11T08:41:43.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:43 smithi092 ceph-mon[41114]: 6.16 scrub ok 2024-09-11T08:41:43.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:43 smithi092 ceph-mon[41114]: 3.d scrub starts 2024-09-11T08:41:43.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:43 smithi092 ceph-mon[41114]: 3.d scrub ok 2024-09-11T08:41:43.594 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:43.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:43.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:43.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:43 smithi145 ceph-mon[33384]: 6.16 scrub starts 2024-09-11T08:41:43.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:43 smithi145 ceph-mon[33384]: 6.16 scrub ok 2024-09-11T08:41:43.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:43 smithi145 ceph-mon[33384]: 3.d scrub starts 2024-09-11T08:41:43.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:43 smithi145 ceph-mon[33384]: 3.d scrub ok 2024-09-11T08:41:43.686 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 32 16 63208 63192 123.401 77.75 0.0156216 0.00809807 2024-09-11T08:41:43.696 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:43.720 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:43.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:43.721 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:43.822 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:43.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:43.865 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:43.966 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:43.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:43.988 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:43.988 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:44.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:44.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:44.124 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:44.124 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:44.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:44.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:44.259 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:44.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[29647]: 4.1e scrub starts 2024-09-11T08:41:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[29647]: 3.a scrub starts 2024-09-11T08:41:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[29647]: 3.a scrub ok 2024-09-11T08:41:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[29647]: 4.1e scrub ok 2024-09-11T08:41:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[29647]: pgmap v107: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 124 MiB/s wr, 1.98k op/s 2024-09-11T08:41:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[29647]: 3.7 scrub starts 2024-09-11T08:41:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[29647]: 3.7 scrub ok 2024-09-11T08:41:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[29647]: 6.3 scrub starts 2024-09-11T08:41:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[29647]: 6.3 scrub ok 2024-09-11T08:41:44.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[41114]: 4.1e scrub starts 2024-09-11T08:41:44.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[41114]: 3.a scrub starts 2024-09-11T08:41:44.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[41114]: 3.a scrub ok 2024-09-11T08:41:44.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[41114]: 4.1e scrub ok 2024-09-11T08:41:44.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[41114]: pgmap v107: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 124 MiB/s wr, 1.98k op/s 2024-09-11T08:41:44.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[41114]: 3.7 scrub starts 2024-09-11T08:41:44.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[41114]: 3.7 scrub ok 2024-09-11T08:41:44.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[41114]: 6.3 scrub starts 2024-09-11T08:41:44.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:44 smithi092 ceph-mon[41114]: 6.3 scrub ok 2024-09-11T08:41:44.360 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:44.388 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:44.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:44.389 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:44.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:44.526 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:44.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:44.527 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:44.628 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:44 smithi145 ceph-mon[33384]: 4.1e scrub starts 2024-09-11T08:41:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:44 smithi145 ceph-mon[33384]: 3.a scrub starts 2024-09-11T08:41:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:44 smithi145 ceph-mon[33384]: 3.a scrub ok 2024-09-11T08:41:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:44 smithi145 ceph-mon[33384]: 4.1e scrub ok 2024-09-11T08:41:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:44 smithi145 ceph-mon[33384]: pgmap v107: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 124 MiB/s wr, 1.98k op/s 2024-09-11T08:41:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:44 smithi145 ceph-mon[33384]: 3.7 scrub starts 2024-09-11T08:41:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:44 smithi145 ceph-mon[33384]: 3.7 scrub ok 2024-09-11T08:41:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:44 smithi145 ceph-mon[33384]: 6.3 scrub starts 2024-09-11T08:41:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:44 smithi145 ceph-mon[33384]: 6.3 scrub ok 2024-09-11T08:41:44.664 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:44.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:44.665 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:44.686 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 33 16 64476 64460 122.063 79.25 0.00632085 0.0081876 2024-09-11T08:41:44.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:44.800 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:44.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:44.813 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:44.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:44.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:44.953 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:44.953 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:45.054 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:45.080 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:45.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:45.081 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:45.182 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:45.210 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:45.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:45.211 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:45.312 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:45.342 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:45.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:45.344 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:45.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:45.481 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:45.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:45.482 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:45.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[29647]: 3.c scrub starts 2024-09-11T08:41:45.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[29647]: 3.c scrub ok 2024-09-11T08:41:45.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[29647]: 3.12 scrub starts 2024-09-11T08:41:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[29647]: 3.12 scrub ok 2024-09-11T08:41:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[29647]: 5.b scrub starts 2024-09-11T08:41:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[29647]: 5.b scrub ok 2024-09-11T08:41:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[41114]: 3.c scrub starts 2024-09-11T08:41:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[41114]: 3.c scrub ok 2024-09-11T08:41:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[41114]: 3.12 scrub starts 2024-09-11T08:41:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[41114]: 3.12 scrub ok 2024-09-11T08:41:45.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[41114]: 5.b scrub starts 2024-09-11T08:41:45.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[41114]: 5.b scrub ok 2024-09-11T08:41:45.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:45.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:45.583 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:45.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:45.612 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:45.612 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:45 smithi145 ceph-mon[33384]: 3.c scrub starts 2024-09-11T08:41:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:45 smithi145 ceph-mon[33384]: 3.c scrub ok 2024-09-11T08:41:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:45 smithi145 ceph-mon[33384]: 3.12 scrub starts 2024-09-11T08:41:45.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:45 smithi145 ceph-mon[33384]: 3.12 scrub ok 2024-09-11T08:41:45.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:45 smithi145 ceph-mon[33384]: 5.b scrub starts 2024-09-11T08:41:45.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:45 smithi145 ceph-mon[33384]: 5.b scrub ok 2024-09-11T08:41:45.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:45 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:45.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:45.686 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 34 16 65713 65697 120.746 77.3125 0.00770939 0.00827657 2024-09-11T08:41:45.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:41:45.713 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:45.738 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:45.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:45.739 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:45.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:45.861 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:45.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:45.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:45.963 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:46.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:46.005 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:46.005 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:46.106 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:46.128 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:46.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:46.129 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:46.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:46.251 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:46.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:46.252 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:46.352 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:46.383 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:46.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:46.384 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:46.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:41:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:41:46.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:46 smithi145 ceph-mon[33384]: 6.10 scrub starts 2024-09-11T08:41:46.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:46 smithi145 ceph-mon[33384]: 6.10 scrub ok 2024-09-11T08:41:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:46 smithi145 ceph-mon[33384]: pgmap v108: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 58 MiB/s wr, 935 op/s 2024-09-11T08:41:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:46 smithi145 ceph-mon[33384]: 4.d scrub starts 2024-09-11T08:41:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:46 smithi145 ceph-mon[33384]: 4.d scrub ok 2024-09-11T08:41:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:46 smithi145 ceph-mon[33384]: 4.0 scrub starts 2024-09-11T08:41:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:46 smithi145 ceph-mon[33384]: 4.0 scrub ok 2024-09-11T08:41:46.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:46.527 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:46.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:46.528 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:46.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:46 smithi092 ceph-mon[29647]: 6.10 scrub starts 2024-09-11T08:41:46.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:46 smithi092 ceph-mon[29647]: 6.10 scrub ok 2024-09-11T08:41:46.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:46 smithi092 ceph-mon[29647]: pgmap v108: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 58 MiB/s wr, 935 op/s 2024-09-11T08:41:46.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:46 smithi092 ceph-mon[29647]: 4.d scrub starts 2024-09-11T08:41:46.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:46 smithi092 ceph-mon[29647]: 4.d scrub ok 2024-09-11T08:41:46.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:46 smithi092 ceph-mon[29647]: 4.0 scrub starts 2024-09-11T08:41:46.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:46 smithi092 ceph-mon[29647]: 4.0 scrub ok 2024-09-11T08:41:46.562 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:46 smithi092 ceph-mon[41114]: 6.10 scrub starts 2024-09-11T08:41:46.562 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:46 smithi092 ceph-mon[41114]: 6.10 scrub ok 2024-09-11T08:41:46.562 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:46 smithi092 ceph-mon[41114]: pgmap v108: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 58 MiB/s wr, 935 op/s 2024-09-11T08:41:46.562 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:46 smithi092 ceph-mon[41114]: 4.d scrub starts 2024-09-11T08:41:46.562 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:46 smithi092 ceph-mon[41114]: 4.d scrub ok 2024-09-11T08:41:46.562 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:46 smithi092 ceph-mon[41114]: 4.0 scrub starts 2024-09-11T08:41:46.562 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:46 smithi092 ceph-mon[41114]: 4.0 scrub ok 2024-09-11T08:41:46.629 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:46.654 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:46.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:46.655 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:46.686 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 35 16 66777 66761 119.197 66.5 0.0232698 0.00838206 2024-09-11T08:41:46.755 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:46.796 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:46.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:46.797 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:46.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:46.932 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:46.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:46.934 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:47.034 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:47.055 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:47.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:47.056 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:47.157 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:47.185 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:47.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:47.186 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:47.287 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:47.314 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:47.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:47.316 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:47.417 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:47.442 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:47.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:47.443 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:47.544 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:47.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:47 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:47.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:47 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:47.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:47.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:47.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:47 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:47.680 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:47.688 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 36 16 67976 67960 117.965 74.9375 0.0120099 0.00847259 2024-09-11T08:41:47.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:47.717 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:47.717 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:47.818 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:47.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:47.863 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:47.863 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:47.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:47.985 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:47.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:47.986 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:48.087 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:48.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:48.116 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:48.117 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:48.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:48.255 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:48.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:48.256 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:48.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:48.406 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:48.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:48.407 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:48.508 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:48.535 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:48.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:48.536 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:48.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:48 smithi092 ceph-mon[41114]: 4.1f scrub starts 2024-09-11T08:41:48.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:48 smithi092 ceph-mon[41114]: 4.1f scrub ok 2024-09-11T08:41:48.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:48 smithi092 ceph-mon[41114]: pgmap v109: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-11T08:41:48.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:48 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:41:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:48 smithi092 ceph-mon[29647]: 4.1f scrub starts 2024-09-11T08:41:48.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:48 smithi092 ceph-mon[29647]: 4.1f scrub ok 2024-09-11T08:41:48.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:48 smithi092 ceph-mon[29647]: pgmap v109: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-11T08:41:48.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:48 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:41:48.637 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:48.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:48 smithi145 ceph-mon[33384]: 4.1f scrub starts 2024-09-11T08:41:48.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:48 smithi145 ceph-mon[33384]: 4.1f scrub ok 2024-09-11T08:41:48.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:48 smithi145 ceph-mon[33384]: pgmap v109: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-11T08:41:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:48 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:41:48.677 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:48.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:48.678 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:48.687 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 37 16 69150 69134 116.759 73.375 0.00825727 0.00855943 2024-09-11T08:41:48.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:48.819 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:48.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:48.820 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:48.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:48.975 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:48.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:48.976 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:49.077 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:49.104 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:49.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:49.105 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:49.130 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:41:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-11T08:41:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-11T08:41:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:41:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14156.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-11T08:41:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:36:30.992002+0000", 2024-09-11T08:41:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 318.136316265, 2024-09-11T08:41:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.51953536199999995, 2024-09-11T08:41:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.131 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14156", 2024-09-11T08:41:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3501902574", 2024-09-11T08:41:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1 2024-09-11T08:41:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992002+0000", 2024-09-11T08:41:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992002+0000", 2024-09-11T08:41:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992000+0000", 2024-09-11T08:41:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:41:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992008+0000", 2024-09-11T08:41:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9279999999999993e-06 2024-09-11T08:41:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992018+0000", 2024-09-11T08:41:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0045e-05 2024-09-11T08:41:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992026+0000", 2024-09-11T08:41:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8520000000000004e-06 2024-09-11T08:41:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992080+0000", 2024-09-11T08:41:49.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3891999999999998e-05 2024-09-11T08:41:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:41:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992090+0000", 2024-09-11T08:41:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9750000000000002e-06 2024-09-11T08:41:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.510497+0000", 2024-09-11T08:41:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.51840699700000004 2024-09-11T08:41:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.510602+0000", 2024-09-11T08:41:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105664 2024-09-11T08:41:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:41:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.510729+0000", 2024-09-11T08:41:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012635399999999999 2024-09-11T08:41:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.511254+0000", 2024-09-11T08:41:49.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052565800000000003 2024-09-11T08:41:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.511424+0000", 2024-09-11T08:41:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169438 2024-09-11T08:41:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.511483+0000", 2024-09-11T08:41:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9095999999999998e-05 2024-09-11T08:41:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.511516+0000", 2024-09-11T08:41:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2801e-05 2024-09-11T08:41:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.511537+0000", 2024-09-11T08:41:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1376999999999999e-05 2024-09-11T08:41:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:59029 7.d e98/88 7:b0032386:::benchmark_data_smithi092_84565_object59028:head v 98'3693, mlcod=98'3690)", 2024-09-11T08:41:49.138 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:40.500616+0000", 2024-09-11T08:41:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6277022250000002, 2024-09-11T08:41:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024260430999999999, 2024-09-11T08:41:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.500616+0000", 2024-09-11T08:41:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.500616+0000", 2024-09-11T08:41:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.500615+0000", 2024-09-11T08:41:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:41:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.512917+0000", 2024-09-11T08:41:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012301459000000001 2024-09-11T08:41:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.512967+0000", 2024-09-11T08:41:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9988000000000003e-05 2024-09-11T08:41:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.512977+0000", 2024-09-11T08:41:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0879e-05 2024-09-11T08:41:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.514106+0000", 2024-09-11T08:41:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011284660000000001 2024-09-11T08:41:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.514127+0000", 2024-09-11T08:41:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0839000000000002e-05 2024-09-11T08:41:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.524783+0000", 2024-09-11T08:41:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010656083 2024-09-11T08:41:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.524876+0000", 2024-09-11T08:41:49.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3298e-05 2024-09-11T08:41:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:60554 7.b 7:d3eaa59a:::benchmark_data_smithi092_84565_object60553:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:41:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:41.636491+0000", 2024-09-11T08:41:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.4918271670000003, 2024-09-11T08:41:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028113464000000001, 2024-09-11T08:41:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:41:49.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:41:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 60554 2024-09-11T08:41:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:41.636491+0000", 2024-09-11T08:41:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:41.636491+0000", 2024-09-11T08:41:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:41.636490+0000", 2024-09-11T08:41:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:41:49.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:41.636517+0000", 2024-09-11T08:41:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6472999999999999e-05 2024-09-11T08:41:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:41.636525+0000", 2024-09-11T08:41:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0879999999999995e-06 2024-09-11T08:41:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:41.636529+0000", 2024-09-11T08:41:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5120000000000002e-06 2024-09-11T08:41:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:41.640285+0000", 2024-09-11T08:41:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003755956 2024-09-11T08:41:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:41.640607+0000", 2024-09-11T08:41:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032220300000000002 2024-09-11T08:41:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:41:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:41.641338+0000", 2024-09-11T08:41:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000730402 2024-09-11T08:41:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:41.647680+0000", 2024-09-11T08:41:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063419360000000003 2024-09-11T08:41:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:41.647857+0000", 2024-09-11T08:41:49.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017768499999999999 2024-09-11T08:41:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:41.664455+0000", 2024-09-11T08:41:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016597324 2024-09-11T08:41:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:41.664558+0000", 2024-09-11T08:41:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010364799999999999 2024-09-11T08:41:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:41.664604+0000", 2024-09-11T08:41:49.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5825999999999998e-05 2024-09-11T08:41:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:61794 7.b 7:d1fa1dde:::benchmark_data_smithi092_84565_object61793:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:41:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:42.540187+0000", 2024-09-11T08:41:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.5881312359999997, 2024-09-11T08:41:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023935980999999999, 2024-09-11T08:41:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:41:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:41:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61794 2024-09-11T08:41:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.540187+0000", 2024-09-11T08:41:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.540187+0000", 2024-09-11T08:41:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.540186+0000", 2024-09-11T08:41:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:41:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.540208+0000", 2024-09-11T08:41:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2291000000000001e-05 2024-09-11T08:41:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.540216+0000", 2024-09-11T08:41:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9340000000000006e-06 2024-09-11T08:41:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.540220+0000", 2024-09-11T08:41:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0160000000000002e-06 2024-09-11T08:41:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.551194+0000", 2024-09-11T08:41:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010973289000000001 2024-09-11T08:41:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.551301+0000", 2024-09-11T08:41:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107228 2024-09-11T08:41:49.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:41:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.551422+0000", 2024-09-11T08:41:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000120715 2024-09-11T08:41:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.562166+0000", 2024-09-11T08:41:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010744046 2024-09-11T08:41:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.562331+0000", 2024-09-11T08:41:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000164983 2024-09-11T08:41:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.564043+0000", 2024-09-11T08:41:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017125759999999999 2024-09-11T08:41:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.564089+0000", 2024-09-11T08:41:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6026999999999998e-05 2024-09-11T08:41:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.564123+0000", 2024-09-11T08:41:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3430999999999999e-05 2024-09-11T08:41:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:61812 7.9 e99/88 7:96a1cc08:::benchmark_data_smithi092_84565_object61811:head v 99'3861, mlcod=99'3858)", 2024-09-11T08:41:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:42.563485+0000", 2024-09-11T08:41:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.5648330750000001, 2024-09-11T08:41:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025598387, 2024-09-11T08:41:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.563485+0000", 2024-09-11T08:41:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.563485+0000", 2024-09-11T08:41:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.563484+0000", 2024-09-11T08:41:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:41:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.563509+0000", 2024-09-11T08:41:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4834999999999999e-05 2024-09-11T08:41:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.563522+0000", 2024-09-11T08:41:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3192e-05 2024-09-11T08:41:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.563528+0000", 2024-09-11T08:41:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4020000000000001e-06 2024-09-11T08:41:49.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.566843+0000", 2024-09-11T08:41:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033158710000000002 2024-09-11T08:41:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.566863+0000", 2024-09-11T08:41:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9157999999999999e-05 2024-09-11T08:41:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.584137+0000", 2024-09-11T08:41:49.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017274701999999999 2024-09-11T08:41:49.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:42.589083+0000", 2024-09-11T08:41:49.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049459370000000001 2024-09-11T08:41:49.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.160 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.160 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:62578 7.b 7:d6c70b1b:::benchmark_data_smithi092_84565_object62577:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:41:49.160 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:43.204470+0000", 2024-09-11T08:41:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.9238479670000004, 2024-09-11T08:41:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02882154, 2024-09-11T08:41:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:41:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:41:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62578 2024-09-11T08:41:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.204470+0000", 2024-09-11T08:41:49.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.204470+0000", 2024-09-11T08:41:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.204468+0000", 2024-09-11T08:41:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:41:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.204496+0000", 2024-09-11T08:41:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7628e-05 2024-09-11T08:41:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.204520+0000", 2024-09-11T08:41:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4158000000000001e-05 2024-09-11T08:41:49.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.204525+0000", 2024-09-11T08:41:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3499999999999996e-06 2024-09-11T08:41:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.204951+0000", 2024-09-11T08:41:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000426007 2024-09-11T08:41:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.205064+0000", 2024-09-11T08:41:49.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113354 2024-09-11T08:41:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:41:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.205360+0000", 2024-09-11T08:41:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029578500000000001 2024-09-11T08:41:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.215758+0000", 2024-09-11T08:41:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010397827 2024-09-11T08:41:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.215902+0000", 2024-09-11T08:41:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143537 2024-09-11T08:41:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.233212+0000", 2024-09-11T08:41:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017309992 2024-09-11T08:41:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.233257+0000", 2024-09-11T08:41:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5330000000000001e-05 2024-09-11T08:41:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.233291+0000", 2024-09-11T08:41:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4524999999999999e-05 2024-09-11T08:41:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:62581 7.b 7:db5fc509:::benchmark_data_smithi092_84565_object62580:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:41:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:43.205083+0000", 2024-09-11T08:41:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.9232353279999996, 2024-09-11T08:41:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028273936, 2024-09-11T08:41:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.168 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:41:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:41:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62581 2024-09-11T08:41:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.205083+0000", 2024-09-11T08:41:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.205083+0000", 2024-09-11T08:41:49.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.205082+0000", 2024-09-11T08:41:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:41:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.205103+0000", 2024-09-11T08:41:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1189999999999999e-05 2024-09-11T08:41:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.205112+0000", 2024-09-11T08:41:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6440000000000006e-06 2024-09-11T08:41:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.205119+0000", 2024-09-11T08:41:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9160000000000001e-06 2024-09-11T08:41:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.216680+0000", 2024-09-11T08:41:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011560587000000001 2024-09-11T08:41:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.217126+0000", 2024-09-11T08:41:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044581299999999998 2024-09-11T08:41:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:41:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.217377+0000", 2024-09-11T08:41:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025149800000000001 2024-09-11T08:41:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.220161+0000", 2024-09-11T08:41:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027839129999999998 2024-09-11T08:41:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.223718+0000", 2024-09-11T08:41:49.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035565760000000001 2024-09-11T08:41:49.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:49.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.233299+0000", 2024-09-11T08:41:49.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095809810000000006 2024-09-11T08:41:49.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.233330+0000", 2024-09-11T08:41:49.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1414000000000001e-05 2024-09-11T08:41:49.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.233357+0000", 2024-09-11T08:41:49.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6248e-05 2024-09-11T08:41:49.174 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.174 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.174 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.174 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:63433 7.b 7:d5f17b84:::benchmark_data_smithi092_84565_object63432:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:41:49.174 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:43.866782+0000", 2024-09-11T08:41:49.174 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2615363799999999, 2024-09-11T08:41:49.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026069106000000002, 2024-09-11T08:41:49.175 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.175 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.175 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:49.175 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:41:49.175 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:41:49.175 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 63433 2024-09-11T08:41:49.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.175 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.866782+0000", 2024-09-11T08:41:49.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.866782+0000", 2024-09-11T08:41:49.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.866781+0000", 2024-09-11T08:41:49.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:41:49.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.866804+0000", 2024-09-11T08:41:49.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3683e-05 2024-09-11T08:41:49.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.866814+0000", 2024-09-11T08:41:49.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8470000000000008e-06 2024-09-11T08:41:49.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.866839+0000", 2024-09-11T08:41:49.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4831e-05 2024-09-11T08:41:49.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.885155+0000", 2024-09-11T08:41:49.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018315674000000001 2024-09-11T08:41:49.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.885262+0000", 2024-09-11T08:41:49.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010728799999999999 2024-09-11T08:41:49.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:41:49.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.885383+0000", 2024-09-11T08:41:49.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121335 2024-09-11T08:41:49.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:49.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.887496+0000", 2024-09-11T08:41:49.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021130599999999999 2024-09-11T08:41:49.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.887531+0000", 2024-09-11T08:41:49.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4329999999999998e-05 2024-09-11T08:41:49.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.892482+0000", 2024-09-11T08:41:49.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049513960000000003 2024-09-11T08:41:49.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.892808+0000", 2024-09-11T08:41:49.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000326354 2024-09-11T08:41:49.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:43.892851+0000", 2024-09-11T08:41:49.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2256000000000003e-05 2024-09-11T08:41:49.180 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.180 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.180 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.181 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:65535 7.0 e99/88 7:0d2b4b81:::benchmark_data_smithi092_84565_object65534:head v 99'4164, mlcod=99'4162)", 2024-09-11T08:41:49.181 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:45.509325+0000", 2024-09-11T08:41:49.181 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.6189929689999998, 2024-09-11T08:41:49.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039677954000000001, 2024-09-11T08:41:49.181 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.181 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.181 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.509325+0000", 2024-09-11T08:41:49.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.509325+0000", 2024-09-11T08:41:49.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.509324+0000", 2024-09-11T08:41:49.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:41:49.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.509425+0000", 2024-09-11T08:41:49.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100966 2024-09-11T08:41:49.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.509433+0000", 2024-09-11T08:41:49.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3769999999999998e-06 2024-09-11T08:41:49.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.509436+0000", 2024-09-11T08:41:49.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4400000000000001e-06 2024-09-11T08:41:49.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.509471+0000", 2024-09-11T08:41:49.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4272e-05 2024-09-11T08:41:49.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.509483+0000", 2024-09-11T08:41:49.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2029e-05 2024-09-11T08:41:49.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.511014+0000", 2024-09-11T08:41:49.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015314110000000001 2024-09-11T08:41:49.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.549003+0000", 2024-09-11T08:41:49.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037988919000000003 2024-09-11T08:41:49.185 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.185 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.185 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.185 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:65541 7.d e99/88 7:bf83f958:::benchmark_data_smithi092_84565_object65540:head v 99'4089, mlcod=99'4085)", 2024-09-11T08:41:49.186 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:45.514074+0000", 2024-09-11T08:41:49.186 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.614243638, 2024-09-11T08:41:49.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037781418999999997, 2024-09-11T08:41:49.186 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.186 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.186 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.514074+0000", 2024-09-11T08:41:49.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.514074+0000", 2024-09-11T08:41:49.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.514073+0000", 2024-09-11T08:41:49.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:41:49.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.514173+0000", 2024-09-11T08:41:49.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100022 2024-09-11T08:41:49.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.548600+0000", 2024-09-11T08:41:49.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034426382999999998 2024-09-11T08:41:49.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.548607+0000", 2024-09-11T08:41:49.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4909999999999999e-06 2024-09-11T08:41:49.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.548662+0000", 2024-09-11T08:41:49.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4469000000000003e-05 2024-09-11T08:41:49.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.548679+0000", 2024-09-11T08:41:49.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6960999999999999e-05 2024-09-11T08:41:49.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.551807+0000", 2024-09-11T08:41:49.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031287760000000002 2024-09-11T08:41:49.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.551856+0000", 2024-09-11T08:41:49.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8352000000000001e-05 2024-09-11T08:41:49.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.191 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:66215 7.4 e99/88 7:223f9d8a:::benchmark_data_smithi092_84565_object66214:head v 99'4092, mlcod=99'4089)", 2024-09-11T08:41:49.191 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.170867+0000", 2024-09-11T08:41:49.191 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.957450825, 2024-09-11T08:41:49.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103303007, 2024-09-11T08:41:49.191 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.191 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.191 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.170867+0000", 2024-09-11T08:41:49.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.170867+0000", 2024-09-11T08:41:49.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.170866+0000", 2024-09-11T08:41:49.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:41:49.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.170893+0000", 2024-09-11T08:41:49.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7039999999999999e-05 2024-09-11T08:41:49.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.170906+0000", 2024-09-11T08:41:49.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2996000000000001e-05 2024-09-11T08:41:49.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.170913+0000", 2024-09-11T08:41:49.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6139999999999997e-06 2024-09-11T08:41:49.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.170984+0000", 2024-09-11T08:41:49.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0864000000000005e-05 2024-09-11T08:41:49.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.171002+0000", 2024-09-11T08:41:49.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8230999999999999e-05 2024-09-11T08:41:49.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.175422+0000", 2024-09-11T08:41:49.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044200940000000003 2024-09-11T08:41:49.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274170+0000", 2024-09-11T08:41:49.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.098748194999999997 2024-09-11T08:41:49.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.195 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.195 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:66220 7.0 e99/88 7:0e1c1cb9:::benchmark_data_smithi092_84565_object66219:head v 99'4219, mlcod=99'4217)", 2024-09-11T08:41:49.195 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.172989+0000", 2024-09-11T08:41:49.195 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.955329216, 2024-09-11T08:41:49.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10152572, 2024-09-11T08:41:49.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.196 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.172989+0000", 2024-09-11T08:41:49.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.172989+0000", 2024-09-11T08:41:49.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.172979+0000", 2024-09-11T08:41:49.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999905 2024-09-11T08:41:49.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173083+0000", 2024-09-11T08:41:49.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104143 2024-09-11T08:41:49.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173095+0000", 2024-09-11T08:41:49.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1891e-05 2024-09-11T08:41:49.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173101+0000", 2024-09-11T08:41:49.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3580000000000002e-06 2024-09-11T08:41:49.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173156+0000", 2024-09-11T08:41:49.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4836000000000001e-05 2024-09-11T08:41:49.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173175+0000", 2024-09-11T08:41:49.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9369999999999999e-05 2024-09-11T08:41:49.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.175513+0000", 2024-09-11T08:41:49.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023384289999999999 2024-09-11T08:41:49.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274514+0000", 2024-09-11T08:41:49.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.099001071999999996 2024-09-11T08:41:49.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.200 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.200 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:66218 7.4 e99/88 7:2ebb945f:::benchmark_data_smithi092_84565_object66217:head v 99'4093, mlcod=99'4090)", 2024-09-11T08:41:49.200 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.173196+0000", 2024-09-11T08:41:49.200 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9551215960000001, 2024-09-11T08:41:49.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101057367, 2024-09-11T08:41:49.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.200 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.200 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173196+0000", 2024-09-11T08:41:49.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173196+0000", 2024-09-11T08:41:49.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173196+0000", 2024-09-11T08:41:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:41:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173219+0000", 2024-09-11T08:41:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3541999999999999e-05 2024-09-11T08:41:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173230+0000", 2024-09-11T08:41:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0504e-05 2024-09-11T08:41:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173235+0000", 2024-09-11T08:41:49.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3789999999999997e-06 2024-09-11T08:41:49.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173403+0000", 2024-09-11T08:41:49.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016734700000000001 2024-09-11T08:41:49.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173419+0000", 2024-09-11T08:41:49.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6478e-05 2024-09-11T08:41:49.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274219+0000", 2024-09-11T08:41:49.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10079982799999999 2024-09-11T08:41:49.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274254+0000", 2024-09-11T08:41:49.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4863999999999997e-05 2024-09-11T08:41:49.204 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.204 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.204 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.205 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:66214 7.2 e99/88 7:43d51050:::benchmark_data_smithi092_84565_object66213:head v 99'4137, mlcod=99'4135)", 2024-09-11T08:41:49.205 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.174111+0000", 2024-09-11T08:41:49.205 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.954207002, 2024-09-11T08:41:49.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.106495039, 2024-09-11T08:41:49.205 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.205 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.205 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174111+0000", 2024-09-11T08:41:49.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174111+0000", 2024-09-11T08:41:49.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174110+0000", 2024-09-11T08:41:49.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:41:49.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174138+0000", 2024-09-11T08:41:49.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7681000000000001e-05 2024-09-11T08:41:49.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174149+0000", 2024-09-11T08:41:49.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1185e-05 2024-09-11T08:41:49.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174155+0000", 2024-09-11T08:41:49.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6230000000000002e-06 2024-09-11T08:41:49.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174224+0000", 2024-09-11T08:41:49.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9218e-05 2024-09-11T08:41:49.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174241+0000", 2024-09-11T08:41:49.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7356000000000001e-05 2024-09-11T08:41:49.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280502+0000", 2024-09-11T08:41:49.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10626126499999999 2024-09-11T08:41:49.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280606+0000", 2024-09-11T08:41:49.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103496 2024-09-11T08:41:49.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.209 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.209 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:66225 7.0 e99/88 7:01cb4ea1:::benchmark_data_smithi092_84565_object66224:head v 99'4220, mlcod=99'4217)", 2024-09-11T08:41:49.209 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.176937+0000", 2024-09-11T08:41:49.209 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9513804920000002, 2024-09-11T08:41:49.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103855564, 2024-09-11T08:41:49.209 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.209 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.210 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.176937+0000", 2024-09-11T08:41:49.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.176937+0000", 2024-09-11T08:41:49.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.176936+0000", 2024-09-11T08:41:49.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:41:49.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.176985+0000", 2024-09-11T08:41:49.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.939e-05 2024-09-11T08:41:49.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274110+0000", 2024-09-11T08:41:49.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.097124391000000004 2024-09-11T08:41:49.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274120+0000", 2024-09-11T08:41:49.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0302999999999999e-05 2024-09-11T08:41:49.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274544+0000", 2024-09-11T08:41:49.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000424151 2024-09-11T08:41:49.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274563+0000", 2024-09-11T08:41:49.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.876e-05 2024-09-11T08:41:49.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280720+0000", 2024-09-11T08:41:49.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061573000000000001 2024-09-11T08:41:49.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280793+0000", 2024-09-11T08:41:49.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2708999999999998e-05 2024-09-11T08:41:49.213 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.213 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.213 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.214 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:66217 7.2 e99/88 7:49541d7a:::benchmark_data_smithi092_84565_object66216:head v 99'4138, mlcod=99'4135)", 2024-09-11T08:41:49.214 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.178474+0000", 2024-09-11T08:41:49.214 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9498434219999998, 2024-09-11T08:41:49.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102289936, 2024-09-11T08:41:49.214 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.214 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.214 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.178474+0000", 2024-09-11T08:41:49.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.178474+0000", 2024-09-11T08:41:49.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.178473+0000", 2024-09-11T08:41:49.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:41:49.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.178504+0000", 2024-09-11T08:41:49.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0511999999999999e-05 2024-09-11T08:41:49.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274100+0000", 2024-09-11T08:41:49.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.095596001999999999 2024-09-11T08:41:49.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274117+0000", 2024-09-11T08:41:49.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6860000000000001e-05 2024-09-11T08:41:49.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274161+0000", 2024-09-11T08:41:49.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4122000000000002e-05 2024-09-11T08:41:49.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274188+0000", 2024-09-11T08:41:49.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6631999999999998e-05 2024-09-11T08:41:49.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280611+0000", 2024-09-11T08:41:49.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064237349999999999 2024-09-11T08:41:49.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280764+0000", 2024-09-11T08:41:49.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015312600000000001 2024-09-11T08:41:49.218 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.218 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.218 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.218 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:66238 7.9 e99/88 7:939fc3d5:::benchmark_data_smithi092_84565_object66237:head v 99'4158, mlcod=99'4156)", 2024-09-11T08:41:49.218 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.188360+0000", 2024-09-11T08:41:49.218 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9399581079999999, 2024-09-11T08:41:49.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.088677141000000001, 2024-09-11T08:41:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.188360+0000", 2024-09-11T08:41:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.188360+0000", 2024-09-11T08:41:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.188358+0000", 2024-09-11T08:41:49.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:41:49.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.188387+0000", 2024-09-11T08:41:49.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8728999999999999e-05 2024-09-11T08:41:49.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274139+0000", 2024-09-11T08:41:49.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.085752140000000004 2024-09-11T08:41:49.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274148+0000", 2024-09-11T08:41:49.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2480000000000002e-06 2024-09-11T08:41:49.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274183+0000", 2024-09-11T08:41:49.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4448e-05 2024-09-11T08:41:49.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274203+0000", 2024-09-11T08:41:49.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0472e-05 2024-09-11T08:41:49.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.276918+0000", 2024-09-11T08:41:49.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027144780000000002 2024-09-11T08:41:49.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.277037+0000", 2024-09-11T08:41:49.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011909300000000001 2024-09-11T08:41:49.222 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.222 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.222 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.223 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:68825 7.b 7:da2aaa27:::benchmark_data_smithi092_84565_object68824:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:41:49.223 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:48.428213+0000", 2024-09-11T08:41:49.223 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.70010496200000005, 2024-09-11T08:41:49.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035070601, 2024-09-11T08:41:49.223 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.223 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:49.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:41:49.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:41:49.223 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 68825 2024-09-11T08:41:49.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.223 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.428213+0000", 2024-09-11T08:41:49.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.428213+0000", 2024-09-11T08:41:49.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.428208+0000", 2024-09-11T08:41:49.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999957 2024-09-11T08:41:49.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.428246+0000", 2024-09-11T08:41:49.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8025999999999998e-05 2024-09-11T08:41:49.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.428276+0000", 2024-09-11T08:41:49.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0012e-05 2024-09-11T08:41:49.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.428286+0000", 2024-09-11T08:41:49.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5880000000000004e-06 2024-09-11T08:41:49.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.428330+0000", 2024-09-11T08:41:49.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3794000000000001e-05 2024-09-11T08:41:49.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.428464+0000", 2024-09-11T08:41:49.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133912 2024-09-11T08:41:49.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:41:49.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.431031+0000", 2024-09-11T08:41:49.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025677209999999998 2024-09-11T08:41:49.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.434425+0000", 2024-09-11T08:41:49.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033934439999999998 2024-09-11T08:41:49.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.434555+0000", 2024-09-11T08:41:49.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000130094 2024-09-11T08:41:49.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:49.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.462494+0000", 2024-09-11T08:41:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027938484999999999 2024-09-11T08:41:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.463239+0000", 2024-09-11T08:41:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074591299999999996 2024-09-11T08:41:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.463284+0000", 2024-09-11T08:41:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4134000000000001e-05 2024-09-11T08:41:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.229 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:68828 7.b 7:d82769e9:::benchmark_data_smithi092_84565_object68827:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:41:49.229 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:48.432957+0000", 2024-09-11T08:41:49.229 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.69536099799999995, 2024-09-11T08:41:49.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030629256000000001, 2024-09-11T08:41:49.229 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.229 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:49.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:41:49.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:41:49.229 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 68828 2024-09-11T08:41:49.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.229 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.432957+0000", 2024-09-11T08:41:49.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.432957+0000", 2024-09-11T08:41:49.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.432955+0000", 2024-09-11T08:41:49.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:41:49.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.432978+0000", 2024-09-11T08:41:49.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3159e-05 2024-09-11T08:41:49.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.432991+0000", 2024-09-11T08:41:49.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2089000000000001e-05 2024-09-11T08:41:49.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.432997+0000", 2024-09-11T08:41:49.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3389999999999998e-06 2024-09-11T08:41:49.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.434633+0000", 2024-09-11T08:41:49.232 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:49.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001636258 2024-09-11T08:41:49.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.437606+0000", 2024-09-11T08:41:49.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029726050000000001 2024-09-11T08:41:49.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:41:49.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.437735+0000", 2024-09-11T08:41:49.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012962 2024-09-11T08:41:49.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.452848+0000", 2024-09-11T08:41:49.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015113 2024-09-11T08:41:49.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.452880+0000", 2024-09-11T08:41:49.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1837e-05 2024-09-11T08:41:49.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:49.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.463296+0000", 2024-09-11T08:41:49.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010415555 2024-09-11T08:41:49.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.463540+0000", 2024-09-11T08:41:49.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000243897 2024-09-11T08:41:49.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.463586+0000", 2024-09-11T08:41:49.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6560000000000001e-05 2024-09-11T08:41:49.236 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.236 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.236 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.236 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:68832 7.b 7:d5f03e3b:::benchmark_data_smithi092_84565_object68831:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:41:49.236 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:48.435025+0000", 2024-09-11T08:41:49.236 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.69329307799999995, 2024-09-11T08:41:49.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028894473, 2024-09-11T08:41:49.236 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:41:49.236 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:41:49.236 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:41:49.236 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:41:49.237 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:41:49.237 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 68832 2024-09-11T08:41:49.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.237 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:41:49.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:41:49.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.435025+0000", 2024-09-11T08:41:49.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:41:49.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.435025+0000", 2024-09-11T08:41:49.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:41:49.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:41:49.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.435024+0000", 2024-09-11T08:41:49.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:41:49.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:41:49.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.435048+0000", 2024-09-11T08:41:49.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3881999999999999e-05 2024-09-11T08:41:49.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:41:49.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.435057+0000", 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7380000000000006e-06 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.435062+0000", 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9790000000000001e-06 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.441570+0000", 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065079600000000001 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.441722+0000", 2024-09-11T08:41:49.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000152139 2024-09-11T08:41:49.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:41:49.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.441919+0000", 2024-09-11T08:41:49.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019684699999999999 2024-09-11T08:41:49.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.452912+0000", 2024-09-11T08:41:49.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010992849000000001 2024-09-11T08:41:49.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:41:49.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.452942+0000", 2024-09-11T08:41:49.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9850000000000001e-05 2024-09-11T08:41:49.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:41:49.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.463630+0000", 2024-09-11T08:41:49.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010688321000000001 2024-09-11T08:41:49.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:41:49.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.463871+0000", 2024-09-11T08:41:49.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024049 2024-09-11T08:41:49.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:41:49.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:41:49.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:41:49.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.463919+0000", 2024-09-11T08:41:49.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8507999999999997e-05 2024-09-11T08:41:49.242 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.242 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.242 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.242 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:41:49.242 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:41:49.242 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:41:49.261 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:49.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:49.262 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:49.363 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:49.392 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:49.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:49.393 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:49.494 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:49.521 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:49.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:49.522 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:49.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[29647]: 5.1f scrub starts 2024-09-11T08:41:49.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[29647]: 5.1f scrub ok 2024-09-11T08:41:49.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[29647]: 4.19 scrub starts 2024-09-11T08:41:49.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[29647]: 4.19 scrub ok 2024-09-11T08:41:49.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[29647]: 5.f scrub starts 2024-09-11T08:41:49.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[29647]: 5.f scrub ok 2024-09-11T08:41:49.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:41:49.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:41:49.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:49.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[41114]: 5.1f scrub starts 2024-09-11T08:41:49.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[41114]: 5.1f scrub ok 2024-09-11T08:41:49.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[41114]: 4.19 scrub starts 2024-09-11T08:41:49.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[41114]: 4.19 scrub ok 2024-09-11T08:41:49.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[41114]: 5.f scrub starts 2024-09-11T08:41:49.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[41114]: 5.f scrub ok 2024-09-11T08:41:49.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:41:49.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:41:49.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:49 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:49.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:49 smithi145 ceph-mon[33384]: 5.1f scrub starts 2024-09-11T08:41:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:49 smithi145 ceph-mon[33384]: 5.1f scrub ok 2024-09-11T08:41:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:49 smithi145 ceph-mon[33384]: 4.19 scrub starts 2024-09-11T08:41:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:49 smithi145 ceph-mon[33384]: 4.19 scrub ok 2024-09-11T08:41:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:49 smithi145 ceph-mon[33384]: 5.f scrub starts 2024-09-11T08:41:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:49 smithi145 ceph-mon[33384]: 5.f scrub ok 2024-09-11T08:41:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:49 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:41:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:49 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:41:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:49 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:41:49.665 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:49.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:49.666 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:49.687 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 38 16 70447 70431 115.82 81.0625 0.0100171 0.00862889 2024-09-11T08:41:49.767 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:49.794 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:49.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:49.794 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:49.895 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:49.923 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:49.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:49.925 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:50.025 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:50.054 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:50.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:50.055 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:50.156 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:50.182 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:50.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:50.184 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:50.285 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:50.330 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:50.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:50.331 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:50.427 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[29647]: pgmap v110: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-11T08:41:50.427 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[29647]: 4.9 scrub starts 2024-09-11T08:41:50.427 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[29647]: 4.9 scrub ok 2024-09-11T08:41:50.427 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[29647]: 2.2 scrub starts 2024-09-11T08:41:50.427 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[29647]: 2.2 scrub ok 2024-09-11T08:41:50.428 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[29647]: 5.a scrub starts 2024-09-11T08:41:50.428 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[29647]: 5.a scrub ok 2024-09-11T08:41:50.428 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[29647]: 5.9 scrub starts 2024-09-11T08:41:50.428 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[29647]: 5.9 scrub ok 2024-09-11T08:41:50.428 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[41114]: pgmap v110: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-11T08:41:50.428 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[41114]: 4.9 scrub starts 2024-09-11T08:41:50.428 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[41114]: 4.9 scrub ok 2024-09-11T08:41:50.428 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[41114]: 2.2 scrub starts 2024-09-11T08:41:50.429 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[41114]: 2.2 scrub ok 2024-09-11T08:41:50.429 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[41114]: 5.a scrub starts 2024-09-11T08:41:50.429 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[41114]: 5.a scrub ok 2024-09-11T08:41:50.429 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[41114]: 5.9 scrub starts 2024-09-11T08:41:50.429 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:50 smithi092 ceph-mon[41114]: 5.9 scrub ok 2024-09-11T08:41:50.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:50.474 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:50.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:50.475 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:50.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:50.621 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:50.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:50.622 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:50.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:50 smithi145 ceph-mon[33384]: pgmap v110: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-11T08:41:50.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:50 smithi145 ceph-mon[33384]: 4.9 scrub starts 2024-09-11T08:41:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:50 smithi145 ceph-mon[33384]: 4.9 scrub ok 2024-09-11T08:41:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:50 smithi145 ceph-mon[33384]: 2.2 scrub starts 2024-09-11T08:41:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:50 smithi145 ceph-mon[33384]: 2.2 scrub ok 2024-09-11T08:41:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:50 smithi145 ceph-mon[33384]: 5.a scrub starts 2024-09-11T08:41:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:50 smithi145 ceph-mon[33384]: 5.a scrub ok 2024-09-11T08:41:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:50 smithi145 ceph-mon[33384]: 5.9 scrub starts 2024-09-11T08:41:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:50 smithi145 ceph-mon[33384]: 5.9 scrub ok 2024-09-11T08:41:50.687 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 39 16 71771 71755 114.972 82.75 0.0204218 0.00869317 2024-09-11T08:41:50.723 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:50.751 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:50.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:50.752 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:50.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:50.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:50.874 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:50.874 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:50.975 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:51.018 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:51.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:51.019 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:51.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:51.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:51.183 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:51.183 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:51.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:51.340 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:51.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:51.341 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:51.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:51.479 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:51.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:51.481 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:51.581 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:51.612 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:51.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:51.613 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:51.670 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:41:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:41:51] "GET /metrics HTTP/1.1" 200 223548 "" "Prometheus/2.43.0" 2024-09-11T08:41:51.688 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:41:51.689184+0000 min lat: 0.00133167 max lat: 0.172138 avg lat: 0.00876896 2024-09-11T08:41:51.688 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:41:51.688 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 40 16 72967 72951 113.966 74.75 0.0104466 0.00876896 2024-09-11T08:41:51.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:51.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:51.757 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:51.757 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:51.858 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:51.868 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T08:41:51.884 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:51.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:51.885 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:51.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:52.026 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:52.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:52.027 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:52.052 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:41:52.052 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:41:52.052 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:41:52.052 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:41:52.053 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:41:52.053 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:41:52.053 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:41:52.053 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:41:52.053 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:41:52.053 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:41:52.053 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:41:52.053 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:41:52.053 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:41:52.054 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:41:52.054 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:41:52.054 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:41:52.128 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:52.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:52.172 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:52.173 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:52.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:52.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:52.336 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:52.336 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:52.437 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:52.464 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:52.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:52.465 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:52.566 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:52.595 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:52.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:52.596 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:52 smithi145 ceph-mon[33384]: pgmap v111: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-11T08:41:52.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:52 smithi145 ceph-mon[33384]: 6.1b scrub starts 2024-09-11T08:41:52.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:52 smithi145 ceph-mon[33384]: 6.1b scrub ok 2024-09-11T08:41:52.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:52 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3667032525' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:41:52.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:52 smithi145 ceph-mon[33384]: 6.1 scrub starts 2024-09-11T08:41:52.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:52 smithi145 ceph-mon[33384]: 6.1 scrub ok 2024-09-11T08:41:52.688 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 41 16 74211 74195 113.083 77.75 0.0118523 0.00883808 2024-09-11T08:41:52.697 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:52.738 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:52.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:52.739 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:52.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:52 smithi092 ceph-mon[29647]: pgmap v111: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-11T08:41:52.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:52 smithi092 ceph-mon[29647]: 6.1b scrub starts 2024-09-11T08:41:52.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:52 smithi092 ceph-mon[29647]: 6.1b scrub ok 2024-09-11T08:41:52.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:52 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3667032525' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:41:52.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:52 smithi092 ceph-mon[29647]: 6.1 scrub starts 2024-09-11T08:41:52.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:52 smithi092 ceph-mon[29647]: 6.1 scrub ok 2024-09-11T08:41:52.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:52 smithi092 ceph-mon[41114]: pgmap v111: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-11T08:41:52.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:52 smithi092 ceph-mon[41114]: 6.1b scrub starts 2024-09-11T08:41:52.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:52 smithi092 ceph-mon[41114]: 6.1b scrub ok 2024-09-11T08:41:52.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:52 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3667032525' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:41:52.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:52 smithi092 ceph-mon[41114]: 6.1 scrub starts 2024-09-11T08:41:52.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:52 smithi092 ceph-mon[41114]: 6.1 scrub ok 2024-09-11T08:41:52.840 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:52.840 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:52.864 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:52.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:52.865 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:52.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:53.005 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:53.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:53.006 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:53.107 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:53.133 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:53.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:53.135 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:53.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:41:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:41:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:53.236 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:53.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:53.264 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:53.264 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:53.366 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:53.394 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:53.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:53.395 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:53.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:53.502 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:53 smithi092 ceph-mon[29647]: 4.12 scrub starts 2024-09-11T08:41:53.502 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:53 smithi092 ceph-mon[29647]: 4.12 scrub ok 2024-09-11T08:41:53.502 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:53 smithi092 ceph-mon[29647]: 5.5 scrub starts 2024-09-11T08:41:53.502 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:53 smithi092 ceph-mon[29647]: 5.5 scrub ok 2024-09-11T08:41:53.502 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:53 smithi092 ceph-mon[29647]: 6.4 deep-scrub starts 2024-09-11T08:41:53.502 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:53 smithi092 ceph-mon[29647]: 6.4 deep-scrub ok 2024-09-11T08:41:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:53 smithi092 ceph-mon[41114]: 4.12 scrub starts 2024-09-11T08:41:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:53 smithi092 ceph-mon[41114]: 4.12 scrub ok 2024-09-11T08:41:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:53 smithi092 ceph-mon[41114]: 5.5 scrub starts 2024-09-11T08:41:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:53 smithi092 ceph-mon[41114]: 5.5 scrub ok 2024-09-11T08:41:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:53 smithi092 ceph-mon[41114]: 6.4 deep-scrub starts 2024-09-11T08:41:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:53 smithi092 ceph-mon[41114]: 6.4 deep-scrub ok 2024-09-11T08:41:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:53.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:53.538 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:53.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:53.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:53 smithi145 ceph-mon[33384]: 4.12 scrub starts 2024-09-11T08:41:53.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:53 smithi145 ceph-mon[33384]: 4.12 scrub ok 2024-09-11T08:41:53.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:53 smithi145 ceph-mon[33384]: 5.5 scrub starts 2024-09-11T08:41:53.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:53 smithi145 ceph-mon[33384]: 5.5 scrub ok 2024-09-11T08:41:53.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:53 smithi145 ceph-mon[33384]: 6.4 deep-scrub starts 2024-09-11T08:41:53.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:53 smithi145 ceph-mon[33384]: 6.4 deep-scrub ok 2024-09-11T08:41:53.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:53.673 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:53.689 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 42 16 75448 75432 112.231 77.3125 0.0145231 0.00890547 2024-09-11T08:41:53.774 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:53.801 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:53.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:53.802 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:53.839 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:41:53.839 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T08:41:53.839 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:41:53.903 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:53.936 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:53.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:53.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:54.038 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:54.062 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:54.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:54.063 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:54.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:54.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:54.205 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:54.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:54.306 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:54.334 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:54.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:54.336 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:54.437 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:54.479 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:54.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:54.480 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:54.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:54 smithi092 ceph-mon[29647]: pgmap v112: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 867 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-11T08:41:54.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:54 smithi092 ceph-mon[29647]: 3.1c scrub starts 2024-09-11T08:41:54.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:54 smithi092 ceph-mon[29647]: 3.1c scrub ok 2024-09-11T08:41:54.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:54 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T08:41:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:54 smithi092 ceph-mon[29647]: 6.f scrub ok 2024-09-11T08:41:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:54 smithi092 ceph-mon[29647]: 3.1b scrub starts 2024-09-11T08:41:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:54 smithi092 ceph-mon[29647]: 3.1b scrub ok 2024-09-11T08:41:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:54 smithi092 ceph-mon[41114]: pgmap v112: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 867 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-11T08:41:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:54 smithi092 ceph-mon[41114]: 3.1c scrub starts 2024-09-11T08:41:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:54 smithi092 ceph-mon[41114]: 3.1c scrub ok 2024-09-11T08:41:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:54 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T08:41:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:54 smithi092 ceph-mon[41114]: 6.f scrub ok 2024-09-11T08:41:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:54 smithi092 ceph-mon[41114]: 3.1b scrub starts 2024-09-11T08:41:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:54 smithi092 ceph-mon[41114]: 3.1b scrub ok 2024-09-11T08:41:54.581 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:54.607 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:54.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:54.609 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:54.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:54 smithi145 ceph-mon[33384]: pgmap v112: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 867 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-11T08:41:54.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:54 smithi145 ceph-mon[33384]: 3.1c scrub starts 2024-09-11T08:41:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:54 smithi145 ceph-mon[33384]: 3.1c scrub ok 2024-09-11T08:41:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:54 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T08:41:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:54 smithi145 ceph-mon[33384]: 6.f scrub ok 2024-09-11T08:41:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:54 smithi145 ceph-mon[33384]: 3.1b scrub starts 2024-09-11T08:41:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:54 smithi145 ceph-mon[33384]: 3.1b scrub ok 2024-09-11T08:41:54.688 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 43 16 76691 76675 111.427 77.6875 0.00845885 0.0089696 2024-09-11T08:41:54.709 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:41:54.736 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:54.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:54.737 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:41:54.837 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:54.864 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:54.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:54.865 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:54.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:41:54.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:55.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:55.086 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:55.086 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:55.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:55.213 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:55.240 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:55.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:55.241 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:55.342 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:55.371 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:55.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:55.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:55.473 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:55.501 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:55.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:55.503 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:55.604 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:55.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:55.630 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:55.630 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:55.688 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 44 16 77881 77865 110.585 74.375 0.00906861 0.00903791 2024-09-11T08:41:55.731 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:55.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:55.768 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:55.768 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:55.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:55.905 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:55.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:55.906 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:56.007 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:56.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:56.064 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:56.065 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:56.165 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:56.193 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:56.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:56.193 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:56.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:56.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:56.332 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:56.332 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:56.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:41:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:41:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:56 smithi145 ceph-mon[33384]: pgmap v113: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-11T08:41:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:56 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:41:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:56 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:41:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:56 smithi145 ceph-mon[33384]: 4.1a scrub starts 2024-09-11T08:41:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:56 smithi145 ceph-mon[33384]: 4.1a scrub ok 2024-09-11T08:41:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:56 smithi145 ceph-mon[33384]: osdmap e100: 8 total, 8 up, 8 in 2024-09-11T08:41:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:56 smithi145 ceph-mon[33384]: 4.2 deep-scrub starts 2024-09-11T08:41:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:56 smithi145 ceph-mon[33384]: 4.2 deep-scrub ok 2024-09-11T08:41:56.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:56.470 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:56.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:56.471 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:56.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:56.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:56.611 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:56.612 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:56.688 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 45 15 79138 79123 109.875 78.625 0.00594058 0.00909639 2024-09-11T08:41:56.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:56.749 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:56.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:56.750 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:56.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[29647]: pgmap v113: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-11T08:41:56.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:41:56.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:41:56.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[29647]: 4.1a scrub starts 2024-09-11T08:41:56.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[29647]: 4.1a scrub ok 2024-09-11T08:41:56.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[29647]: osdmap e100: 8 total, 8 up, 8 in 2024-09-11T08:41:56.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[29647]: 4.2 deep-scrub starts 2024-09-11T08:41:56.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[29647]: 4.2 deep-scrub ok 2024-09-11T08:41:56.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[41114]: pgmap v113: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-11T08:41:56.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:41:56.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:41:56.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[41114]: 4.1a scrub starts 2024-09-11T08:41:56.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[41114]: 4.1a scrub ok 2024-09-11T08:41:56.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[41114]: osdmap e100: 8 total, 8 up, 8 in 2024-09-11T08:41:56.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[41114]: 4.2 deep-scrub starts 2024-09-11T08:41:56.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:56 smithi092 ceph-mon[41114]: 4.2 deep-scrub ok 2024-09-11T08:41:56.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:41:56.853 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:41:56.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:56.898 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:56.898 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:41:56.999 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:57.025 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:57.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:57.027 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:57.127 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:57.154 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:57.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:57.155 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:57.256 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:57.293 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:57.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:57.294 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:57.395 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:57.423 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:57.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:57.424 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:57.525 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:57.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:57.566 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:57.566 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:57.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:57 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:57.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:57 smithi145 ceph-mon[33384]: 4.6 scrub starts 2024-09-11T08:41:57.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:57 smithi145 ceph-mon[33384]: 4.6 scrub ok 2024-09-11T08:41:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:57 smithi145 ceph-mon[33384]: 6.5 scrub starts 2024-09-11T08:41:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:57 smithi145 ceph-mon[33384]: 6.5 scrub ok 2024-09-11T08:41:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:57 smithi145 ceph-mon[33384]: 4.f scrub starts 2024-09-11T08:41:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:57 smithi145 ceph-mon[33384]: 4.f scrub ok 2024-09-11T08:41:57.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:57.689 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 46 16 80293 80277 109.053 72.125 0.00722483 0.00916496 2024-09-11T08:41:57.700 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:57.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:57.701 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:57.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:57.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:57 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:57.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:57 smithi092 ceph-mon[29647]: 4.6 scrub starts 2024-09-11T08:41:57.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:57 smithi092 ceph-mon[29647]: 4.6 scrub ok 2024-09-11T08:41:57.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:57 smithi092 ceph-mon[29647]: 6.5 scrub starts 2024-09-11T08:41:57.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:57 smithi092 ceph-mon[29647]: 6.5 scrub ok 2024-09-11T08:41:57.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:57 smithi092 ceph-mon[29647]: 4.f scrub starts 2024-09-11T08:41:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:57 smithi092 ceph-mon[29647]: 4.f scrub ok 2024-09-11T08:41:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:57 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:57 smithi092 ceph-mon[41114]: 4.6 scrub starts 2024-09-11T08:41:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:57 smithi092 ceph-mon[41114]: 4.6 scrub ok 2024-09-11T08:41:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:57 smithi092 ceph-mon[41114]: 6.5 scrub starts 2024-09-11T08:41:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:57 smithi092 ceph-mon[41114]: 6.5 scrub ok 2024-09-11T08:41:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:57 smithi092 ceph-mon[41114]: 4.f scrub starts 2024-09-11T08:41:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:57 smithi092 ceph-mon[41114]: 4.f scrub ok 2024-09-11T08:41:57.823 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:57.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:57.824 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:57.925 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:57.952 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:57.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:57.953 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:58.054 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:41:58.082 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:58.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:58.083 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:41:58.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:58.221 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:58.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:58.222 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:58.323 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:58.349 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:58.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:58.350 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:58.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:58.472 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:58.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:58.473 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:58.574 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:58.600 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:58.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:58.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:58.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:58 smithi145 ceph-mon[33384]: pgmap v115: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-11T08:41:58.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:58 smithi145 ceph-mon[33384]: 3.1d deep-scrub starts 2024-09-11T08:41:58.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:58 smithi145 ceph-mon[33384]: 3.1d deep-scrub ok 2024-09-11T08:41:58.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:58 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:41:58.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:58 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:41:58.689 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 47 15 81399 81384 108.204 69.1875 0.010669 0.00923697 2024-09-11T08:41:58.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:58.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:58.737 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:58 smithi092 ceph-mon[29647]: pgmap v115: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-11T08:41:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:58 smithi092 ceph-mon[29647]: 3.1d deep-scrub starts 2024-09-11T08:41:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:58 smithi092 ceph-mon[29647]: 3.1d deep-scrub ok 2024-09-11T08:41:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:58 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:41:58.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:58 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:41:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:58 smithi092 ceph-mon[41114]: pgmap v115: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-11T08:41:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:58 smithi092 ceph-mon[41114]: 3.1d deep-scrub starts 2024-09-11T08:41:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:58 smithi092 ceph-mon[41114]: 3.1d deep-scrub ok 2024-09-11T08:41:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:58 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:41:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:58 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:41:58.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:58.882 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:58.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:58.883 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:58.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:59.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:59.021 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:59.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:41:59.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:59.155 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:59.155 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:41:59.256 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:59.295 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:59.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:59.296 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:59.397 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:41:59.427 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:59.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:59.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:41:59.528 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:41:59.559 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:41:59.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:59.560 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:41:59.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:59 smithi092 ceph-mon[29647]: 5.1b scrub starts 2024-09-11T08:41:59.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:59 smithi092 ceph-mon[29647]: 5.1b scrub ok 2024-09-11T08:41:59.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:59 smithi092 ceph-mon[29647]: osdmap e101: 8 total, 8 up, 8 in 2024-09-11T08:41:59.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:59 smithi092 ceph-mon[29647]: 4.16 scrub starts 2024-09-11T08:41:59.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:59 smithi092 ceph-mon[29647]: 4.16 scrub ok 2024-09-11T08:41:59.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:59 smithi092 ceph-mon[29647]: 3.f scrub starts 2024-09-11T08:41:59.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:41:59 smithi092 ceph-mon[29647]: 3.f scrub ok 2024-09-11T08:41:59.637 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:59 smithi092 ceph-mon[41114]: 5.1b scrub starts 2024-09-11T08:41:59.637 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:59 smithi092 ceph-mon[41114]: 5.1b scrub ok 2024-09-11T08:41:59.638 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:59 smithi092 ceph-mon[41114]: osdmap e101: 8 total, 8 up, 8 in 2024-09-11T08:41:59.638 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:59 smithi092 ceph-mon[41114]: 4.16 scrub starts 2024-09-11T08:41:59.638 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:59 smithi092 ceph-mon[41114]: 4.16 scrub ok 2024-09-11T08:41:59.638 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:59 smithi092 ceph-mon[41114]: 3.f scrub starts 2024-09-11T08:41:59.638 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:41:59 smithi092 ceph-mon[41114]: 3.f scrub ok 2024-09-11T08:41:59.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:41:59.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:59 smithi145 ceph-mon[33384]: 5.1b scrub starts 2024-09-11T08:41:59.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:59 smithi145 ceph-mon[33384]: 5.1b scrub ok 2024-09-11T08:41:59.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:59 smithi145 ceph-mon[33384]: osdmap e101: 8 total, 8 up, 8 in 2024-09-11T08:41:59.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:59 smithi145 ceph-mon[33384]: 4.16 scrub starts 2024-09-11T08:41:59.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:59 smithi145 ceph-mon[33384]: 4.16 scrub ok 2024-09-11T08:41:59.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:59 smithi145 ceph-mon[33384]: 3.f scrub starts 2024-09-11T08:41:59.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:41:59 smithi145 ceph-mon[33384]: 3.f scrub ok 2024-09-11T08:41:59.689 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 48 16 82579 82563 107.485 73.6875 0.0153142 0.00929844 2024-09-11T08:41:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:59.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:59.718 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:41:59.740 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:41:59.741 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:41:59.741 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-11T08:41:59.741 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 1.000000 2024-09-11T08:41:59.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 7 1 2024-09-11T08:41:59.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:41:59.859 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:41:59.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:41:59.861 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:41:59.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:00.032 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:00.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:00.033 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:00.134 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:00.163 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:00.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:00.164 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:00.266 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:00.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:00.292 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:00.292 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:00.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:00.430 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:00.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:00.432 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:00.529 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:00 smithi092 ceph-mon[29647]: pgmap v117: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-11T08:42:00.529 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:00.530 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:00 smithi092 ceph-mon[41114]: pgmap v117: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-11T08:42:00.530 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:00.532 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:00.556 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:00.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:00.557 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:00.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:00.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:00 smithi145 ceph-mon[33384]: pgmap v117: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-11T08:42:00.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:00.689 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 49 16 83630 83614 106.632 65.6875 0.004522 0.00937191 2024-09-11T08:42:00.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:00.692 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:00.692 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:00.693 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:42:00.693 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:42:00.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:00.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:00.833 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:00.834 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:00.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:00.970 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:00.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:00.971 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:01.072 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:01.100 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:01.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:01.101 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:01.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:01.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:01.249 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:01.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:01.350 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:01.387 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:01.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:01.388 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:42:01.490 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:01.518 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:01.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:01.518 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:01.614 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:01 smithi092 ceph-mon[29647]: 7.8 scrub starts 2024-09-11T08:42:01.614 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:01 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/351034605' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:42:01.614 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:01 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:42:01.614 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:01 smithi092 ceph-mon[29647]: 6.1c scrub starts 2024-09-11T08:42:01.615 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:01 smithi092 ceph-mon[29647]: 6.1c scrub ok 2024-09-11T08:42:01.615 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:01 smithi092 ceph-mon[41114]: 7.8 scrub starts 2024-09-11T08:42:01.615 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:01 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/351034605' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:42:01.615 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:01 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:42:01.615 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:01 smithi092 ceph-mon[41114]: 6.1c scrub starts 2024-09-11T08:42:01.615 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:01 smithi092 ceph-mon[41114]: 6.1c scrub ok 2024-09-11T08:42:01.615 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:42:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:42:01] "GET /metrics HTTP/1.1" 200 223569 "" "Prometheus/2.43.0" 2024-09-11T08:42:01.620 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:01.645 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:01.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:01.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:01.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-11T08:42:01.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:01 smithi145 ceph-mon[33384]: 7.8 scrub starts 2024-09-11T08:42:01.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:01 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/351034605' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:42:01.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:01 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:42:01.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:01 smithi145 ceph-mon[33384]: 6.1c scrub starts 2024-09-11T08:42:01.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:01 smithi145 ceph-mon[33384]: 6.1c scrub ok 2024-09-11T08:42:01.689 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 50 16 84711 84695 105.851 67.5625 0.0163846 0.00944268 2024-09-11T08:42:01.747 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:01.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:01.795 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:01.795 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:01.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:01.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:01.943 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:01.944 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:02.044 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:02.071 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:02.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:02.072 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:02.173 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:02.198 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:02.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:02.199 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:02.300 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:02.329 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:02.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:02.330 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:02.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:02.473 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:02.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:02.474 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:02.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:02 smithi092 ceph-mon[29647]: pgmap v118: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 70 MiB/s wr, 1.12k op/s 2024-09-11T08:42:02.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:02 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:42:02.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:02 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:42:02.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:02 smithi092 ceph-mon[29647]: osdmap e102: 8 total, 8 up, 8 in 2024-09-11T08:42:02.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:02 smithi092 ceph-mon[41114]: pgmap v118: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 70 MiB/s wr, 1.12k op/s 2024-09-11T08:42:02.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:02 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:42:02.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:02 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:42:02.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:02 smithi092 ceph-mon[41114]: osdmap e102: 8 total, 8 up, 8 in 2024-09-11T08:42:02.575 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:02.603 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:02.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:02.604 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:02 smithi145 ceph-mon[33384]: pgmap v118: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 70 MiB/s wr, 1.12k op/s 2024-09-11T08:42:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:02 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:42:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:02 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:42:02.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:02 smithi145 ceph-mon[33384]: osdmap e102: 8 total, 8 up, 8 in 2024-09-11T08:42:02.689 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 51 16 85877 85861 105.204 72.875 0.0206155 0.00950032 2024-09-11T08:42:02.705 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:02.731 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:02.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:02.732 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:02.833 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:02.860 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:02.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:02.861 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:02.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:03.039 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:03.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:03.041 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:03.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:03.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:42:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:42:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:03.174 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:03.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:03.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:03.277 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:03.305 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:03.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:03.306 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:03.407 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:03.435 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:03.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:03.437 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:03.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:03.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:03 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:42:03.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:03 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:42:03.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:03 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:42:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:03 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:42:03.571 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:03.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:03.573 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:03.655 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:03.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:03.695 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 52 15 86989 86974 104.508 69.5625 0.017495 0.00956453 2024-09-11T08:42:03.730 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:03.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:03.732 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:03.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:03.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:03.876 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:03.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:03 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:42:03.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:03 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:42:03.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:04.012 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:04.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:04.013 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:04.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:04.156 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:04.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:04.157 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:04.257 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:04.295 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:04.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:04.296 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:04.396 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:04.426 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:04.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:04.427 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:04.527 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:04.537 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:04 smithi092 ceph-mon[29647]: pgmap v120: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.3 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-11T08:42:04.537 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:04 smithi092 ceph-mon[29647]: osdmap e103: 8 total, 8 up, 8 in 2024-09-11T08:42:04.537 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:04 smithi092 ceph-mon[41114]: pgmap v120: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.3 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-11T08:42:04.537 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:04 smithi092 ceph-mon[41114]: osdmap e103: 8 total, 8 up, 8 in 2024-09-11T08:42:04.554 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:04.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:04.555 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:04.655 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:04.680 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:04.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:04.681 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:04.695 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 53 16 88101 88085 103.846 69.4375 0.010846 0.00962485 2024-09-11T08:42:04.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:04.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:04.820 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:04.820 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:04 smithi145 ceph-mon[33384]: pgmap v120: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.3 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-11T08:42:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:04 smithi145 ceph-mon[33384]: osdmap e103: 8 total, 8 up, 8 in 2024-09-11T08:42:04.921 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:04.950 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:04.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:04.951 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:04.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:42:05.051 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:05.111 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:05.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:05.113 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:05.213 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:05.240 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:05.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:05.242 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:05.342 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:05.369 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:05.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:05.370 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:05.471 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:05.498 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:05.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:05.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:05.600 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:05.627 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:05.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:05.628 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:05.695 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 54 16 89230 89214 103.23 70.5625 0.0105762 0.00968207 2024-09-11T08:42:05.729 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:05.755 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:05.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:05.756 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:05.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:05.898 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:05.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:05.899 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:06.000 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:06.028 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:06.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:06.029 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:06.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:06.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:06.165 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:06.166 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:06.267 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:06.293 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:06.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:06.295 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:06.396 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:06.421 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:06.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:06.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:06.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:06.560 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:06.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:06.561 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:06.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:06 smithi145 ceph-mon[33384]: pgmap v122: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 766 B/s rd, 53 MiB/s wr, 845 op/s 2024-09-11T08:42:06.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:42:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:42:06.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:06.695 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 55 16 90385 90369 102.665 72.1875 0.0234322 0.00973551 2024-09-11T08:42:06.695 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:06.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:06.696 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:06.797 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:06.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:06 smithi092 ceph-mon[29647]: pgmap v122: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 766 B/s rd, 53 MiB/s wr, 845 op/s 2024-09-11T08:42:06.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:06 smithi092 ceph-mon[41114]: pgmap v122: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 766 B/s rd, 53 MiB/s wr, 845 op/s 2024-09-11T08:42:06.824 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:06.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:06.825 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:06.926 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:06.951 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:06.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:06.952 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:07.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:07.096 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:07.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:07.097 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:07.198 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:07.225 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:07.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:07.226 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:07.292 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:42:07.293 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:42:07.303 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:42:07.304 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:42:07.326 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:07.343 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:42:07.343 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:42:07.352 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:07.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:07.354 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:07.374 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:42:07.375 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:42:07.403 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:42:07.403 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:42:07.436 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:42:07.437 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:42:07.454 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:07.469 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:42:07.469 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:42:07.481 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:07.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:07.482 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:07.506 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:42:07.506 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:42:07.583 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:07.609 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:07.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:07.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:07.653 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:07 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:07.654 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:07 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2440938977' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:42:07.654 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:07 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:07.654 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:07 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2440938977' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:42:07.695 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 56 16 91515 91499 102.093 70.625 0.0145978 0.00978994 2024-09-11T08:42:07.711 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:07.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:07.737 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:07.738 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:07.838 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:07.865 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:07.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:07.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:07.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:07 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:07.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:07 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2440938977' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:42:07.968 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:07.995 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:07.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:07.996 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:08.097 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:08.123 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:08.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:08.124 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:08.225 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:08.251 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:08.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:08.252 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:08.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:08.427 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:08.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:08.429 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:08.502 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:42:08.502 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:42:08.502 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:42:08.503 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:42:08.503 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:42:08.503 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:08 smithi092 ceph-mon[29647]: pgmap v123: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-11T08:42:08.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:08 smithi092 ceph-mon[41114]: pgmap v123: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-11T08:42:08.530 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:08.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:08.576 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:08.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:08.696 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 57 16 92616 92600 101.508 68.8125 0.0145714 0.00984687 2024-09-11T08:42:08.719 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:08.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:08.721 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:08.821 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:08.835 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:08.845 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:08.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:08.846 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:08 smithi145 ceph-mon[33384]: pgmap v123: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-11T08:42:08.947 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:08.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:08.975 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:08.976 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:09.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:09.114 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:09.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:09.115 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:09.216 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:09.241 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:09.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:09.242 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:09.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:09.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:09.381 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:09.481 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:09.503 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:09.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:09.504 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:09.605 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:09.633 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:09.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:09.634 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:09.696 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 58 16 93750 93734 100.98 70.875 0.011147 0.00989838 2024-09-11T08:42:09.735 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:09.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:09.763 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:09.763 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:09.864 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:09.894 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:09.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:09.895 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:09.996 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:10.013 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:09 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:42:10.013 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:09 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:42:10.013 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:09 smithi092 ceph-mon[29647]: pgmap v124: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-11T08:42:10.013 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:09 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:42:10.014 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:09 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:42:10.014 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:09 smithi092 ceph-mon[41114]: pgmap v124: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-11T08:42:10.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:10.025 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:10.025 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:10.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:10.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:09 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:42:10.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:09 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:42:10.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:09 smithi145 ceph-mon[33384]: pgmap v124: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-11T08:42:10.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:10.164 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:10.165 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:10.265 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:10.303 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:10.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:10.305 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:10.405 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:10.430 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:10.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:10.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:10.532 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:10.557 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:10.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:10.558 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:10.658 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:10.685 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:10.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:10.686 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:10.696 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 59 16 94824 94808 100.406 67.125 0.010399 0.00995488 2024-09-11T08:42:10.756 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-11T08:42:10.787 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:10.812 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:10.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:10.813 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:10.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:10.949 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:10.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:10.950 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:11.005 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:10 smithi092 ceph-mon[29647]: osdmap e104: 8 total, 8 up, 8 in 2024-09-11T08:42:11.005 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:10 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1194414924' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T08:42:11.005 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:10 smithi092 ceph-mon[41114]: osdmap e104: 8 total, 8 up, 8 in 2024-09-11T08:42:11.005 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:10 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1194414924' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T08:42:11.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-11T08:42:11.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:11.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:11.131 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:11.131 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:11.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:10 smithi145 ceph-mon[33384]: osdmap e104: 8 total, 8 up, 8 in 2024-09-11T08:42:11.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:10 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1194414924' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T08:42:11.232 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:11.286 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:11.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:11.288 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:11.388 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:11.440 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:11.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:11.442 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:11.542 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:11.568 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:11.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:11.569 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:11.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:11.696 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:42:11.697697+0000 min lat: 0.00133167 max lat: 0.172138 avg lat: 0.0100112 2024-09-11T08:42:11.696 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:42:11.696 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 60 16 95890 95874 99.8431 66.625 0.00767539 0.0100112 2024-09-11T08:42:11.704 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:11.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:11.705 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:11.739 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:42:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:42:11] "GET /metrics HTTP/1.1" 200 223569 "" "Prometheus/2.43.0" 2024-09-11T08:42:11.806 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:11.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:11.835 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:11.835 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:11.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:11.976 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:11.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:11.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:12.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:42:12.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:11 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1194414924' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-11T08:42:12.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:11 smithi092 ceph-mon[29647]: osdmap e105: 8 total, 8 up, 8 in 2024-09-11T08:42:12.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:11 smithi092 ceph-mon[29647]: pgmap v127: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.3 KiB/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T08:42:12.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:11 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:42:12.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:11 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:42:12.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:11 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1194414924' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-11T08:42:12.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:11 smithi092 ceph-mon[41114]: osdmap e105: 8 total, 8 up, 8 in 2024-09-11T08:42:12.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:11 smithi092 ceph-mon[41114]: pgmap v127: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.3 KiB/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T08:42:12.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:11 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:42:12.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:11 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:42:12.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:12.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:12.147 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:12.147 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:12.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:11 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1194414924' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-11T08:42:12.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:11 smithi145 ceph-mon[33384]: osdmap e105: 8 total, 8 up, 8 in 2024-09-11T08:42:12.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:11 smithi145 ceph-mon[33384]: pgmap v127: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.3 KiB/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T08:42:12.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:11 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:42:12.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:11 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:42:12.248 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:12.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:12.274 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:12.274 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:12.374 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:12.421 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:12.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:12.422 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:12.523 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:12.546 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:12.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:12.547 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:12.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:12.672 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:12.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:12.673 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:12.696 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 61 16 96829 96813 99.1684 58.6875 0.0140795 0.0100778 2024-09-11T08:42:12.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:12.825 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:12.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:12.826 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:12.927 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:12.951 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:12.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:12.952 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:13.036 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:12 smithi092 ceph-mon[29647]: osdmap e106: 8 total, 8 up, 8 in 2024-09-11T08:42:13.037 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:12 smithi092 ceph-mon[41114]: osdmap e106: 8 total, 8 up, 8 in 2024-09-11T08:42:13.052 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:13.076 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:13.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:13.077 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:13.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:12 smithi145 ceph-mon[33384]: osdmap e106: 8 total, 8 up, 8 in 2024-09-11T08:42:13.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:42:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:42:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:13.178 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:13.204 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:13.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:13.205 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:13.298 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:13.300 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:13.305 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:13.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:13.368 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:13.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:13.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:13.518 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:13.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:13.519 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:13.620 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:13.666 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:13.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:13.668 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:13.696 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 62 16 97710 97694 98.457 55.0625 0.0156794 0.0101521 2024-09-11T08:42:13.736 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:42:13.768 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:13.793 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:13.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:13.794 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:13.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:13.932 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:13.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:13.933 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:14.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:13 smithi092 ceph-mon[29647]: pgmap v129: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 58 MiB/s wr, 928 op/s 2024-09-11T08:42:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:13 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4142299431' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:42:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:13 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:42:14.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:13 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4142299431' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:42:14.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:13 smithi092 ceph-mon[29647]: osdmap e107: 8 total, 8 up, 8 in 2024-09-11T08:42:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:13 smithi092 ceph-mon[41114]: pgmap v129: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 58 MiB/s wr, 928 op/s 2024-09-11T08:42:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:13 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4142299431' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:42:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:13 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:42:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:13 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4142299431' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:42:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:13 smithi092 ceph-mon[41114]: osdmap e107: 8 total, 8 up, 8 in 2024-09-11T08:42:14.069 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:14.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:14.070 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:14.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:13 smithi145 ceph-mon[33384]: pgmap v129: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 58 MiB/s wr, 928 op/s 2024-09-11T08:42:14.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:13 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4142299431' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:42:14.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:13 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:42:14.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:13 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4142299431' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:42:14.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:13 smithi145 ceph-mon[33384]: osdmap e107: 8 total, 8 up, 8 in 2024-09-11T08:42:14.171 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:14.195 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:14.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:14.195 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:14.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:14.332 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:14.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:14.333 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:14.434 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:14.463 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:14.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:14.464 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:14.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:14.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:14.605 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:14.696 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 63 16 98744 98728 97.92 64.625 0.0178486 0.0102076 2024-09-11T08:42:14.707 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:14.734 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:14.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:14.735 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:14.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:14.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:14.875 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:14.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:14.976 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:15.001 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:15.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:15.002 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:15.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:15.148 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:15.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:15.149 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:15.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:15.289 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:15.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:15.290 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:15.390 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:15.436 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:15.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:15.437 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:15.538 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:15.565 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:15.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:15.566 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:15.667 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:15.693 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:15.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:15.694 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:15.696 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 64 16 99878 99862 97.4974 70.875 0.019694 0.0102522 2024-09-11T08:42:15.795 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:15.820 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:15.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:15.821 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:15.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:15.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:15.959 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:15.959 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:16.060 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:16.085 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:16.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:16.086 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:16.187 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:16.216 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:16.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:16.217 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:16.279 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:16 smithi092 ceph-mon[29647]: pgmap v131: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 963 B/s rd, 65 MiB/s wr, 1.05k op/s 2024-09-11T08:42:16.279 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:16 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:16.279 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:16 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:16.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:16 smithi092 ceph-mon[41114]: pgmap v131: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 963 B/s rd, 65 MiB/s wr, 1.05k op/s 2024-09-11T08:42:16.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:16 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:16.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:16 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:16.318 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:16.343 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:16.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:16.344 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:16.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:42:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:42:16.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:16 smithi145 ceph-mon[33384]: pgmap v131: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 963 B/s rd, 65 MiB/s wr, 1.05k op/s 2024-09-11T08:42:16.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:16 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:16.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:16 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:16.445 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:16.471 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:16.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:16.473 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:16.573 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:16.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:16.601 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:16.601 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:16.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:42:16.697 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 65 16 100989 100973 97.0656 69.4375 0.0159499 0.0102967 2024-09-11T08:42:16.702 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:16.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:16.752 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:16.779 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:42:16.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 0 1 2024-09-11T08:42:16.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:16.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:16.901 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:16.901 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:17.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:17.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:17.074 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:17.074 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:17.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:17.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:17.242 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:17.245 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:42:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-11T08:42:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-11T08:42:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:42:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:58474 7.e 7:7d7736e0:::benchmark_data_smithi092_84565_object58473:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-11T08:42:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:40.077767+0000", 2024-09-11T08:42:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.156528305000002, 2024-09-11T08:42:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052254118000000002, 2024-09-11T08:42:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 58474 2024-09-11T08:42:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.077767+0000", 2024-09-11T08:42:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.077767+0000", 2024-09-11T08:42:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.077766+0000", 2024-09-11T08:42:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:42:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.077790+0000", 2024-09-11T08:42:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3989999999999999e-05 2024-09-11T08:42:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.077801+0000", 2024-09-11T08:42:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1083e-05 2024-09-11T08:42:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.077806+0000", 2024-09-11T08:42:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2290000000000002e-06 2024-09-11T08:42:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.079204+0000", 2024-09-11T08:42:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001397674 2024-09-11T08:42:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.079303+0000", 2024-09-11T08:42:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9698999999999995e-05 2024-09-11T08:42:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:42:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.112271+0000", 2024-09-11T08:42:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032968048999999999 2024-09-11T08:42:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.118986+0000", 2024-09-11T08:42:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006715024 2024-09-11T08:42:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.118992+0000", 2024-09-11T08:42:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9120000000000003e-06 2024-09-11T08:42:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.129942+0000", 2024-09-11T08:42:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010949530000000001 2024-09-11T08:42:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.129988+0000", 2024-09-11T08:42:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5772000000000002e-05 2024-09-11T08:42:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.130021+0000", 2024-09-11T08:42:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3271e-05 2024-09-11T08:42:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:58478 7.e 7:75c7d0a4:::benchmark_data_smithi092_84565_object58477:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-11T08:42:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:40.078230+0000", 2024-09-11T08:42:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.156065597999998, 2024-09-11T08:42:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051881568000000003, 2024-09-11T08:42:17.252 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.252 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.252 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 58478 2024-09-11T08:42:17.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.252 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.078230+0000", 2024-09-11T08:42:17.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.078230+0000", 2024-09-11T08:42:17.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.078229+0000", 2024-09-11T08:42:17.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:42:17.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.078249+0000", 2024-09-11T08:42:17.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9902999999999999e-05 2024-09-11T08:42:17.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.078257+0000", 2024-09-11T08:42:17.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4039999999999995e-06 2024-09-11T08:42:17.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.078262+0000", 2024-09-11T08:42:17.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5170000000000001e-06 2024-09-11T08:42:17.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.114316+0000", 2024-09-11T08:42:17.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036054332000000001 2024-09-11T08:42:17.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.114432+0000", 2024-09-11T08:42:17.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115641 2024-09-11T08:42:17.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:42:17.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.114555+0000", 2024-09-11T08:42:17.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012347000000000001 2024-09-11T08:42:17.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.119000+0000", 2024-09-11T08:42:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004444712 2024-09-11T08:42:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.119034+0000", 2024-09-11T08:42:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4090000000000001e-05 2024-09-11T08:42:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.130051+0000", 2024-09-11T08:42:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011017422000000001 2024-09-11T08:42:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.130084+0000", 2024-09-11T08:42:17.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2783000000000001e-05 2024-09-11T08:42:17.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:40.130111+0000", 2024-09-11T08:42:17.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7010999999999999e-05 2024-09-11T08:42:17.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.257 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.257 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:65018 7.e 7:7bf36b07:::benchmark_data_smithi092_84565_object65017:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:42:17.257 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:45.059616+0000", 2024-09-11T08:42:17.258 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.174678624999999, 2024-09-11T08:42:17.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.112304603, 2024-09-11T08:42:17.258 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.258 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.258 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 65018 2024-09-11T08:42:17.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.258 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059616+0000", 2024-09-11T08:42:17.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059616+0000", 2024-09-11T08:42:17.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059615+0000", 2024-09-11T08:42:17.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:42:17.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059639+0000", 2024-09-11T08:42:17.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4264999999999999e-05 2024-09-11T08:42:17.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059653+0000", 2024-09-11T08:42:17.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3691e-05 2024-09-11T08:42:17.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059659+0000", 2024-09-11T08:42:17.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.263e-06 2024-09-11T08:42:17.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062216+0000", 2024-09-11T08:42:17.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025571280000000001 2024-09-11T08:42:17.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062351+0000", 2024-09-11T08:42:17.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013464400000000001 2024-09-11T08:42:17.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:42:17.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062504+0000", 2024-09-11T08:42:17.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015295099999999999 2024-09-11T08:42:17.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.064861+0000", 2024-09-11T08:42:17.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023569789999999999 2024-09-11T08:42:17.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.069304+0000", 2024-09-11T08:42:17.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044433959999999996 2024-09-11T08:42:17.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.171603+0000", 2024-09-11T08:42:17.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10229874 2024-09-11T08:42:17.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.171882+0000", 2024-09-11T08:42:17.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027927099999999999 2024-09-11T08:42:17.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.171921+0000", 2024-09-11T08:42:17.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8711000000000003e-05 2024-09-11T08:42:17.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.264 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.264 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.264 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:65021 7.e 7:7eb86f3b:::benchmark_data_smithi092_84565_object65020:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:42:17.264 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:45.062298+0000", 2024-09-11T08:42:17.264 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.171996997000001, 2024-09-11T08:42:17.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11573973999999999, 2024-09-11T08:42:17.264 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.264 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.264 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 65021 2024-09-11T08:42:17.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.265 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062298+0000", 2024-09-11T08:42:17.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062298+0000", 2024-09-11T08:42:17.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062296+0000", 2024-09-11T08:42:17.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:42:17.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062323+0000", 2024-09-11T08:42:17.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6262999999999999e-05 2024-09-11T08:42:17.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062337+0000", 2024-09-11T08:42:17.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3883000000000001e-05 2024-09-11T08:42:17.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062343+0000", 2024-09-11T08:42:17.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.438e-06 2024-09-11T08:42:17.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.064881+0000", 2024-09-11T08:42:17.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025380279999999999 2024-09-11T08:42:17.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.065038+0000", 2024-09-11T08:42:17.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015697000000000001 2024-09-11T08:42:17.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:42:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.065171+0000", 2024-09-11T08:42:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013319499999999999 2024-09-11T08:42:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.069338+0000", 2024-09-11T08:42:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041663999999999998 2024-09-11T08:42:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.069603+0000", 2024-09-11T08:42:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026567600000000001 2024-09-11T08:42:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.177874+0000", 2024-09-11T08:42:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.108271144 2024-09-11T08:42:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.178003+0000", 2024-09-11T08:42:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012872900000000001 2024-09-11T08:42:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.178038+0000", 2024-09-11T08:42:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4688999999999999e-05 2024-09-11T08:42:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:66141 7.e 7:722bb3c0:::benchmark_data_smithi092_84565_object66140:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:42:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.071334+0000", 2024-09-11T08:42:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.162961202000002, 2024-09-11T08:42:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.063473135, 2024-09-11T08:42:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 66141 2024-09-11T08:42:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.071334+0000", 2024-09-11T08:42:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.071334+0000", 2024-09-11T08:42:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.071332+0000", 2024-09-11T08:42:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:42:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.071356+0000", 2024-09-11T08:42:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3530999999999999e-05 2024-09-11T08:42:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.071369+0000", 2024-09-11T08:42:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3358e-05 2024-09-11T08:42:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.071375+0000", 2024-09-11T08:42:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9190000000000004e-06 2024-09-11T08:42:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.071439+0000", 2024-09-11T08:42:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3909000000000001e-05 2024-09-11T08:42:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.071546+0000", 2024-09-11T08:42:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010659599999999999 2024-09-11T08:42:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:42:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.071670+0000", 2024-09-11T08:42:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000124762 2024-09-11T08:42:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.074698+0000", 2024-09-11T08:42:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030272860000000001 2024-09-11T08:42:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.074952+0000", 2024-09-11T08:42:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025483800000000002 2024-09-11T08:42:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.134614+0000", 2024-09-11T08:42:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.059661244000000002 2024-09-11T08:42:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.134711+0000", 2024-09-11T08:42:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7072999999999997e-05 2024-09-11T08:42:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.134807+0000", 2024-09-11T08:42:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6318999999999997e-05 2024-09-11T08:42:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:66159 7.e 7:7e8e6b38:::benchmark_data_smithi092_84565_object66158:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:42:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.080605+0000", 2024-09-11T08:42:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.153690405999999, 2024-09-11T08:42:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075863979999999998, 2024-09-11T08:42:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 66159 2024-09-11T08:42:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.080605+0000", 2024-09-11T08:42:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.080605+0000", 2024-09-11T08:42:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.080603+0000", 2024-09-11T08:42:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:42:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.080625+0000", 2024-09-11T08:42:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1738000000000001e-05 2024-09-11T08:42:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.080636+0000", 2024-09-11T08:42:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1518999999999999e-05 2024-09-11T08:42:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.080641+0000", 2024-09-11T08:42:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1630000000000004e-06 2024-09-11T08:42:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.080688+0000", 2024-09-11T08:42:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6635000000000003e-05 2024-09-11T08:42:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.080851+0000", 2024-09-11T08:42:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016264900000000001 2024-09-11T08:42:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:42:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.080948+0000", 2024-09-11T08:42:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7114000000000006e-05 2024-09-11T08:42:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.084506+0000", 2024-09-11T08:42:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035585289999999999 2024-09-11T08:42:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.084543+0000", 2024-09-11T08:42:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6483999999999998e-05 2024-09-11T08:42:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.156386+0000", 2024-09-11T08:42:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.071843370000000004 2024-09-11T08:42:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.156432+0000", 2024-09-11T08:42:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5413e-05 2024-09-11T08:42:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.156469+0000", 2024-09-11T08:42:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7004e-05 2024-09-11T08:42:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.283 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:66160 7.e 7:74adec4f:::benchmark_data_smithi092_84565_object66159:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:42:17.283 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.080730+0000", 2024-09-11T08:42:17.283 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.153564769999999, 2024-09-11T08:42:17.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.075857965999999999, 2024-09-11T08:42:17.283 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.283 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.283 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 66160 2024-09-11T08:42:17.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.283 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.080730+0000", 2024-09-11T08:42:17.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.080730+0000", 2024-09-11T08:42:17.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.080730+0000", 2024-09-11T08:42:17.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:42:17.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.080750+0000", 2024-09-11T08:42:17.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0463999999999999e-05 2024-09-11T08:42:17.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.080757+0000", 2024-09-11T08:42:17.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8709999999999999e-06 2024-09-11T08:42:17.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.080760+0000", 2024-09-11T08:42:17.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1640000000000001e-06 2024-09-11T08:42:17.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.082666+0000", 2024-09-11T08:42:17.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019055680000000001 2024-09-11T08:42:17.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.082774+0000", 2024-09-11T08:42:17.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108275 2024-09-11T08:42:17.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:42:17.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.082897+0000", 2024-09-11T08:42:17.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012319100000000001 2024-09-11T08:42:17.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.084537+0000", 2024-09-11T08:42:17.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016395940000000001 2024-09-11T08:42:17.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.084562+0000", 2024-09-11T08:42:17.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4995e-05 2024-09-11T08:42:17.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.156503+0000", 2024-09-11T08:42:17.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.071940910999999996 2024-09-11T08:42:17.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.156546+0000", 2024-09-11T08:42:17.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3541000000000002e-05 2024-09-11T08:42:17.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.156588+0000", 2024-09-11T08:42:17.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2169000000000002e-05 2024-09-11T08:42:17.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.289 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.290 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:67209 7.e 7:714b6da6:::benchmark_data_smithi092_84565_object67208:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:42:17.290 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:47.018639+0000", 2024-09-11T08:42:17.290 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.215656561999999, 2024-09-11T08:42:17.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053758876999999997, 2024-09-11T08:42:17.290 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.290 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.290 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.290 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.290 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.290 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 67209 2024-09-11T08:42:17.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.290 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.018639+0000", 2024-09-11T08:42:17.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.018639+0000", 2024-09-11T08:42:17.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.018637+0000", 2024-09-11T08:42:17.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:42:17.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.018662+0000", 2024-09-11T08:42:17.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5763999999999999e-05 2024-09-11T08:42:17.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.018678+0000", 2024-09-11T08:42:17.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5480000000000001e-05 2024-09-11T08:42:17.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.018685+0000", 2024-09-11T08:42:17.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1289999999999998e-06 2024-09-11T08:42:17.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.018731+0000", 2024-09-11T08:42:17.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5911000000000001e-05 2024-09-11T08:42:17.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.018867+0000", 2024-09-11T08:42:17.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013625700000000001 2024-09-11T08:42:17.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:42:17.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.018990+0000", 2024-09-11T08:42:17.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122731 2024-09-11T08:42:17.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.021608+0000", 2024-09-11T08:42:17.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002618408 2024-09-11T08:42:17.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.021935+0000", 2024-09-11T08:42:17.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032678900000000002 2024-09-11T08:42:17.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.072310+0000", 2024-09-11T08:42:17.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.050375030000000001 2024-09-11T08:42:17.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.072360+0000", 2024-09-11T08:42:17.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9379999999999998e-05 2024-09-11T08:42:17.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.072397+0000", 2024-09-11T08:42:17.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7916000000000001e-05 2024-09-11T08:42:17.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.296 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.296 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:68297 7.e 7:77e7e9ab:::benchmark_data_smithi092_84565_object68296:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:42:17.296 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:47.948562+0000", 2024-09-11T08:42:17.296 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.285733059000002, 2024-09-11T08:42:17.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074319260999999998, 2024-09-11T08:42:17.296 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.296 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.297 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 68297 2024-09-11T08:42:17.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.297 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.948562+0000", 2024-09-11T08:42:17.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.948562+0000", 2024-09-11T08:42:17.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.948560+0000", 2024-09-11T08:42:17.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:42:17.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.948585+0000", 2024-09-11T08:42:17.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4561000000000001e-05 2024-09-11T08:42:17.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.948599+0000", 2024-09-11T08:42:17.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3485e-05 2024-09-11T08:42:17.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.948604+0000", 2024-09-11T08:42:17.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9329999999999998e-06 2024-09-11T08:42:17.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.948653+0000", 2024-09-11T08:42:17.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8179000000000001e-05 2024-09-11T08:42:17.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.948779+0000", 2024-09-11T08:42:17.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012656899999999999 2024-09-11T08:42:17.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:42:17.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.948945+0000", 2024-09-11T08:42:17.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016627499999999999 2024-09-11T08:42:17.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.953356+0000", 2024-09-11T08:42:17.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044110130000000001 2024-09-11T08:42:17.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.954279+0000", 2024-09-11T08:42:17.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00092233300000000001 2024-09-11T08:42:17.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.022737+0000", 2024-09-11T08:42:17.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068457849000000001 2024-09-11T08:42:17.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.022835+0000", 2024-09-11T08:42:17.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8004000000000006e-05 2024-09-11T08:42:17.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.022881+0000", 2024-09-11T08:42:17.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6632999999999999e-05 2024-09-11T08:42:17.302 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.302 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.302 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.302 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:68298 7.e 7:7004fbd6:::benchmark_data_smithi092_84565_object68297:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:42:17.302 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:47.949133+0000", 2024-09-11T08:42:17.302 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.285162471, 2024-09-11T08:42:17.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.073992506, 2024-09-11T08:42:17.303 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.303 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.303 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 68298 2024-09-11T08:42:17.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.303 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.949133+0000", 2024-09-11T08:42:17.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.949133+0000", 2024-09-11T08:42:17.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.949132+0000", 2024-09-11T08:42:17.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:42:17.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.949160+0000", 2024-09-11T08:42:17.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8331999999999999e-05 2024-09-11T08:42:17.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.949169+0000", 2024-09-11T08:42:17.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0450000000000006e-06 2024-09-11T08:42:17.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.949175+0000", 2024-09-11T08:42:17.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.463e-06 2024-09-11T08:42:17.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.951403+0000", 2024-09-11T08:42:17.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022284470000000002 2024-09-11T08:42:17.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.951509+0000", 2024-09-11T08:42:17.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0001054 2024-09-11T08:42:17.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:42:17.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.951630+0000", 2024-09-11T08:42:17.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121117 2024-09-11T08:42:17.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.953544+0000", 2024-09-11T08:42:17.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019147540000000001 2024-09-11T08:42:17.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:47.954311+0000", 2024-09-11T08:42:17.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00076657600000000004 2024-09-11T08:42:17.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.023028+0000", 2024-09-11T08:42:17.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068716658 2024-09-11T08:42:17.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.023093+0000", 2024-09-11T08:42:17.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5860999999999998e-05 2024-09-11T08:42:17.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:48.023125+0000", 2024-09-11T08:42:17.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1668999999999998e-05 2024-09-11T08:42:17.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.308 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.309 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:71972 7.e 7:7d20a00c:::benchmark_data_smithi092_84565_object71971:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:42:17.309 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:50.835878+0000", 2024-09-11T08:42:17.309 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.398417381000002, 2024-09-11T08:42:17.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051404442000000002, 2024-09-11T08:42:17.309 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.309 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.309 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 71972 2024-09-11T08:42:17.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.309 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:50.835878+0000", 2024-09-11T08:42:17.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:50.835878+0000", 2024-09-11T08:42:17.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:50.835876+0000", 2024-09-11T08:42:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:42:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:50.835900+0000", 2024-09-11T08:42:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3618e-05 2024-09-11T08:42:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:50.835912+0000", 2024-09-11T08:42:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2201000000000001e-05 2024-09-11T08:42:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:50.835918+0000", 2024-09-11T08:42:17.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6300000000000003e-06 2024-09-11T08:42:17.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:50.835965+0000", 2024-09-11T08:42:17.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7666000000000001e-05 2024-09-11T08:42:17.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:50.836091+0000", 2024-09-11T08:42:17.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012572199999999999 2024-09-11T08:42:17.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:42:17.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:50.836190+0000", 2024-09-11T08:42:17.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8827999999999996e-05 2024-09-11T08:42:17.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:50.838090+0000", 2024-09-11T08:42:17.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018999869999999999 2024-09-11T08:42:17.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:50.838818+0000", 2024-09-11T08:42:17.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072856099999999997 2024-09-11T08:42:17.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:50.887145+0000", 2024-09-11T08:42:17.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048326655000000003 2024-09-11T08:42:17.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:50.887207+0000", 2024-09-11T08:42:17.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2244999999999998e-05 2024-09-11T08:42:17.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:50.887282+0000", 2024-09-11T08:42:17.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4967000000000002e-05 2024-09-11T08:42:17.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.315 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.315 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.315 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:93532 7.e 7:7dfa3160:::benchmark_data_smithi092_84565_object93531:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T08:42:17.315 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:09.486963+0000", 2024-09-11T08:42:17.315 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.7473324879999996, 2024-09-11T08:42:17.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053991863000000001, 2024-09-11T08:42:17.315 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.315 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.316 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93532 2024-09-11T08:42:17.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.316 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.486963+0000", 2024-09-11T08:42:17.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.486963+0000", 2024-09-11T08:42:17.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.486961+0000", 2024-09-11T08:42:17.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:42:17.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.486996+0000", 2024-09-11T08:42:17.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4601999999999998e-05 2024-09-11T08:42:17.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.487005+0000", 2024-09-11T08:42:17.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5980000000000003e-06 2024-09-11T08:42:17.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.487010+0000", 2024-09-11T08:42:17.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9030000000000003e-06 2024-09-11T08:42:17.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.487058+0000", 2024-09-11T08:42:17.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7963000000000001e-05 2024-09-11T08:42:17.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.487151+0000", 2024-09-11T08:42:17.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3249000000000001e-05 2024-09-11T08:42:17.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:42:17.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.487237+0000", 2024-09-11T08:42:17.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5410999999999999e-05 2024-09-11T08:42:17.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.523520+0000", 2024-09-11T08:42:17.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036283351999999998 2024-09-11T08:42:17.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.527031+0000", 2024-09-11T08:42:17.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00351089 2024-09-11T08:42:17.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.540865+0000", 2024-09-11T08:42:17.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01383412 2024-09-11T08:42:17.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.540917+0000", 2024-09-11T08:42:17.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2228999999999997e-05 2024-09-11T08:42:17.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.540954+0000", 2024-09-11T08:42:17.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7097000000000001e-05 2024-09-11T08:42:17.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.321 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.321 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:93534 7.e 7:7f32305f:::benchmark_data_smithi092_84565_object93533:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T08:42:17.321 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:09.487024+0000", 2024-09-11T08:42:17.321 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.7472707209999996, 2024-09-11T08:42:17.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054052653999999999, 2024-09-11T08:42:17.321 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.321 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.322 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.322 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.322 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.322 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93534 2024-09-11T08:42:17.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.322 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.487024+0000", 2024-09-11T08:42:17.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.487024+0000", 2024-09-11T08:42:17.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.487024+0000", 2024-09-11T08:42:17.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:42:17.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.487043+0000", 2024-09-11T08:42:17.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9394000000000001e-05 2024-09-11T08:42:17.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.487050+0000", 2024-09-11T08:42:17.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8369999999999999e-06 2024-09-11T08:42:17.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.487054+0000", 2024-09-11T08:42:17.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7759999999999999e-06 2024-09-11T08:42:17.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.523544+0000", 2024-09-11T08:42:17.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036490250000000002 2024-09-11T08:42:17.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.523670+0000", 2024-09-11T08:42:17.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012623699999999999 2024-09-11T08:42:17.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:42:17.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.523813+0000", 2024-09-11T08:42:17.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014306900000000001 2024-09-11T08:42:17.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.533225+0000", 2024-09-11T08:42:17.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0094112880000000003 2024-09-11T08:42:17.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.533260+0000", 2024-09-11T08:42:17.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4950000000000002e-05 2024-09-11T08:42:17.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.541004+0000", 2024-09-11T08:42:17.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077439129999999998 2024-09-11T08:42:17.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.541039+0000", 2024-09-11T08:42:17.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5439999999999999e-05 2024-09-11T08:42:17.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.541077+0000", 2024-09-11T08:42:17.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7965e-05 2024-09-11T08:42:17.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.327 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.329 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:93535 7.8 7:16b04067:::benchmark_data_smithi092_84565_object93534:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T08:42:17.329 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:09.490279+0000", 2024-09-11T08:42:17.329 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.7440159829999997, 2024-09-11T08:42:17.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052183001999999999, 2024-09-11T08:42:17.329 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.329 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.329 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.329 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.329 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.329 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93535 2024-09-11T08:42:17.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.330 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.490279+0000", 2024-09-11T08:42:17.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.490279+0000", 2024-09-11T08:42:17.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.490278+0000", 2024-09-11T08:42:17.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:42:17.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.490297+0000", 2024-09-11T08:42:17.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9412e-05 2024-09-11T08:42:17.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.525214+0000", 2024-09-11T08:42:17.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034916256999999999 2024-09-11T08:42:17.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.525226+0000", 2024-09-11T08:42:17.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2153e-05 2024-09-11T08:42:17.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.525531+0000", 2024-09-11T08:42:17.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030531299999999998 2024-09-11T08:42:17.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.525951+0000", 2024-09-11T08:42:17.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00041990799999999998 2024-09-11T08:42:17.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,7", 2024-09-11T08:42:17.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.526343+0000", 2024-09-11T08:42:17.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00039228499999999998 2024-09-11T08:42:17.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.533611+0000", 2024-09-11T08:42:17.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072674339999999997 2024-09-11T08:42:17.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.533641+0000", 2024-09-11T08:42:17.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0162e-05 2024-09-11T08:42:17.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.542360+0000", 2024-09-11T08:42:17.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00871917 2024-09-11T08:42:17.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.542420+0000", 2024-09-11T08:42:17.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9809000000000003e-05 2024-09-11T08:42:17.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:09.542462+0000", 2024-09-11T08:42:17.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2141999999999997e-05 2024-09-11T08:42:17.335 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.335 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.335 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.335 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:96577 7.8 7:1c38cb72:::benchmark_data_smithi092_84565_object96576:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-11T08:42:17.335 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:12.380400+0000", 2024-09-11T08:42:17.335 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8538946919999999, 2024-09-11T08:42:17.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.089158344, 2024-09-11T08:42:17.335 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.335 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.336 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.336 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.336 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.336 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96577 2024-09-11T08:42:17.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.336 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.380400+0000", 2024-09-11T08:42:17.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.380400+0000", 2024-09-11T08:42:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.380399+0000", 2024-09-11T08:42:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:42:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.380423+0000", 2024-09-11T08:42:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4709e-05 2024-09-11T08:42:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.380441+0000", 2024-09-11T08:42:17.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8063999999999998e-05 2024-09-11T08:42:17.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.380448+0000", 2024-09-11T08:42:17.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6760000000000001e-06 2024-09-11T08:42:17.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.380507+0000", 2024-09-11T08:42:17.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8705000000000001e-05 2024-09-11T08:42:17.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.380667+0000", 2024-09-11T08:42:17.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016031800000000001 2024-09-11T08:42:17.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,7", 2024-09-11T08:42:17.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.380850+0000", 2024-09-11T08:42:17.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000182824 2024-09-11T08:42:17.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.385598+0000", 2024-09-11T08:42:17.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004748342 2024-09-11T08:42:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.388535+0000", 2024-09-11T08:42:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029364690000000001 2024-09-11T08:42:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.469265+0000", 2024-09-11T08:42:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080730730000000001 2024-09-11T08:42:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.469413+0000", 2024-09-11T08:42:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014701499999999999 2024-09-11T08:42:17.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.469559+0000", 2024-09-11T08:42:17.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014625200000000001 2024-09-11T08:42:17.341 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.341 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.341 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.341 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:96587 7.8 7:1bec6a88:::benchmark_data_smithi092_84565_object96586:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-11T08:42:17.341 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:12.389976+0000", 2024-09-11T08:42:17.341 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8443193569999998, 2024-09-11T08:42:17.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.086759923000000003, 2024-09-11T08:42:17.342 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.342 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.342 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.342 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.342 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.342 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96587 2024-09-11T08:42:17.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.342 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.389976+0000", 2024-09-11T08:42:17.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.389976+0000", 2024-09-11T08:42:17.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.389974+0000", 2024-09-11T08:42:17.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:42:17.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.389997+0000", 2024-09-11T08:42:17.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2240999999999999e-05 2024-09-11T08:42:17.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.390012+0000", 2024-09-11T08:42:17.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5469e-05 2024-09-11T08:42:17.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.390018+0000", 2024-09-11T08:42:17.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2024-09-11T08:42:17.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.390066+0000", 2024-09-11T08:42:17.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8467999999999997e-05 2024-09-11T08:42:17.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.390194+0000", 2024-09-11T08:42:17.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012714000000000001 2024-09-11T08:42:17.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,7", 2024-09-11T08:42:17.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.390339+0000", 2024-09-11T08:42:17.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014524300000000001 2024-09-11T08:42:17.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.394586+0000", 2024-09-11T08:42:17.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042470379999999999 2024-09-11T08:42:17.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.394605+0000", 2024-09-11T08:42:17.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8841000000000001e-05 2024-09-11T08:42:17.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.476642+0000", 2024-09-11T08:42:17.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082037192999999994 2024-09-11T08:42:17.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.476697+0000", 2024-09-11T08:42:17.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4669000000000001e-05 2024-09-11T08:42:17.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.476736+0000", 2024-09-11T08:42:17.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9153999999999998e-05 2024-09-11T08:42:17.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.347 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.348 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:96596 7.8 7:18ae13b6:::benchmark_data_smithi092_84565_object96595:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-11T08:42:17.348 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:12.401972+0000", 2024-09-11T08:42:17.348 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8323231719999997, 2024-09-11T08:42:17.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.074874011000000004, 2024-09-11T08:42:17.348 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.348 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.348 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96596 2024-09-11T08:42:17.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.348 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.401972+0000", 2024-09-11T08:42:17.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.401972+0000", 2024-09-11T08:42:17.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.401970+0000", 2024-09-11T08:42:17.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:42:17.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.401994+0000", 2024-09-11T08:42:17.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3433000000000001e-05 2024-09-11T08:42:17.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.402013+0000", 2024-09-11T08:42:17.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8940000000000002e-05 2024-09-11T08:42:17.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.402019+0000", 2024-09-11T08:42:17.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2110000000000003e-06 2024-09-11T08:42:17.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.402818+0000", 2024-09-11T08:42:17.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00079930500000000005 2024-09-11T08:42:17.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.403199+0000", 2024-09-11T08:42:17.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000381283 2024-09-11T08:42:17.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,7", 2024-09-11T08:42:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.403564+0000", 2024-09-11T08:42:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036506100000000002 2024-09-11T08:42:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.409528+0000", 2024-09-11T08:42:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059631179999999999 2024-09-11T08:42:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.413859+0000", 2024-09-11T08:42:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004331428 2024-09-11T08:42:17.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.476770+0000", 2024-09-11T08:42:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062911280999999999 2024-09-11T08:42:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.476805+0000", 2024-09-11T08:42:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5024999999999998e-05 2024-09-11T08:42:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.476846+0000", 2024-09-11T08:42:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0655000000000003e-05 2024-09-11T08:42:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:96603 7.8 7:14aa1708:::benchmark_data_smithi092_84565_object96602:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-11T08:42:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:12.409571+0000", 2024-09-11T08:42:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8247243769999999, 2024-09-11T08:42:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067496139999999996, 2024-09-11T08:42:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.354 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.355 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96603 2024-09-11T08:42:17.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.355 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.409571+0000", 2024-09-11T08:42:17.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.409571+0000", 2024-09-11T08:42:17.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.409569+0000", 2024-09-11T08:42:17.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:42:17.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.409595+0000", 2024-09-11T08:42:17.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6401999999999999e-05 2024-09-11T08:42:17.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.409611+0000", 2024-09-11T08:42:17.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5591e-05 2024-09-11T08:42:17.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.409617+0000", 2024-09-11T08:42:17.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2199999999999997e-06 2024-09-11T08:42:17.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.409725+0000", 2024-09-11T08:42:17.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010751699999999999 2024-09-11T08:42:17.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.410018+0000", 2024-09-11T08:42:17.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029350399999999999 2024-09-11T08:42:17.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,7", 2024-09-11T08:42:17.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.411212+0000", 2024-09-11T08:42:17.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011938579999999999 2024-09-11T08:42:17.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.413898+0000", 2024-09-11T08:42:17.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026860959999999998 2024-09-11T08:42:17.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.413987+0000", 2024-09-11T08:42:17.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8629999999999997e-05 2024-09-11T08:42:17.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.476929+0000", 2024-09-11T08:42:17.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062942373999999995 2024-09-11T08:42:17.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.476971+0000", 2024-09-11T08:42:17.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1891000000000003e-05 2024-09-11T08:42:17.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.477067+0000", 2024-09-11T08:42:17.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5727e-05 2024-09-11T08:42:17.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.360 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.360 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:96609 7.8 7:1fc69e9f:::benchmark_data_smithi092_84565_object96608:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-11T08:42:17.360 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:12.418560+0000", 2024-09-11T08:42:17.360 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8157353629999999, 2024-09-11T08:42:17.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.06102349, 2024-09-11T08:42:17.360 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.360 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.361 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.361 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.361 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.361 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96609 2024-09-11T08:42:17.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.361 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.418560+0000", 2024-09-11T08:42:17.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.418560+0000", 2024-09-11T08:42:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.418558+0000", 2024-09-11T08:42:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:42:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.418583+0000", 2024-09-11T08:42:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4604e-05 2024-09-11T08:42:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.418603+0000", 2024-09-11T08:42:17.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0203e-05 2024-09-11T08:42:17.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.418610+0000", 2024-09-11T08:42:17.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5360000000000004e-06 2024-09-11T08:42:17.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.418661+0000", 2024-09-11T08:42:17.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0445999999999998e-05 2024-09-11T08:42:17.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.418967+0000", 2024-09-11T08:42:17.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030649399999999998 2024-09-11T08:42:17.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,7", 2024-09-11T08:42:17.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.419333+0000", 2024-09-11T08:42:17.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000366175 2024-09-11T08:42:17.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.424718+0000", 2024-09-11T08:42:17.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053851000000000003 2024-09-11T08:42:17.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.424744+0000", 2024-09-11T08:42:17.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.552e-05 2024-09-11T08:42:17.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.479497+0000", 2024-09-11T08:42:17.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054752699000000002 2024-09-11T08:42:17.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.479546+0000", 2024-09-11T08:42:17.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9069000000000001e-05 2024-09-11T08:42:17.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.479583+0000", 2024-09-11T08:42:17.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7452e-05 2024-09-11T08:42:17.366 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.366 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.366 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.367 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:96611 7.8 7:1714cc5c:::benchmark_data_smithi092_84565_object96610:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-11T08:42:17.367 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:12.420200+0000", 2024-09-11T08:42:17.367 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8140949549999998, 2024-09-11T08:42:17.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.059476468999999997, 2024-09-11T08:42:17.367 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:42:17.367 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:42:17.367 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:42:17.367 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:42:17.367 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:42:17.367 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96611 2024-09-11T08:42:17.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.367 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:42:17.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:42:17.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.420200+0000", 2024-09-11T08:42:17.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:42:17.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.420200+0000", 2024-09-11T08:42:17.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:42:17.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:42:17.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.420199+0000", 2024-09-11T08:42:17.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:42:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:42:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.420230+0000", 2024-09-11T08:42:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0715e-05 2024-09-11T08:42:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:42:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.420240+0000", 2024-09-11T08:42:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0550000000000001e-05 2024-09-11T08:42:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:42:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.420245+0000", 2024-09-11T08:42:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1159999999999998e-06 2024-09-11T08:42:17.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:42:17.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.421917+0000", 2024-09-11T08:42:17.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001671992 2024-09-11T08:42:17.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:42:17.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.422055+0000", 2024-09-11T08:42:17.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000137069 2024-09-11T08:42:17.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,7", 2024-09-11T08:42:17.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.422212+0000", 2024-09-11T08:42:17.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015728300000000001 2024-09-11T08:42:17.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.424739+0000", 2024-09-11T08:42:17.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025274609999999999 2024-09-11T08:42:17.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:42:17.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.424839+0000", 2024-09-11T08:42:17.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9448000000000001e-05 2024-09-11T08:42:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:42:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.479616+0000", 2024-09-11T08:42:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054777304999999998 2024-09-11T08:42:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:42:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.479650+0000", 2024-09-11T08:42:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3509000000000001e-05 2024-09-11T08:42:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:42:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:42:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:42:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:12.479677+0000", 2024-09-11T08:42:17.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7089000000000001e-05 2024-09-11T08:42:17.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.373 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:42:17.373 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:42:17.373 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:42:17.373 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:17 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:17.374 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:17 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:17.374 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:42:17.374 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:42:17.374 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:42:17.374 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:42:17.374 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:42:17.374 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:42:17.375 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:42:17.375 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:42:17.375 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:42:17.375 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:42:17.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:17.383 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:42:17.383 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:42:17.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:17 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:17.412 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:42:17.412 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:42:17.417 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:17.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:17.418 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:17.441 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:42:17.442 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:42:17.518 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:17.552 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:17.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:17.553 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:17.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:17.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:17.687 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:17.697 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 66 16 101865 101849 96.4245 54.75 0.0135056 0.0103659 2024-09-11T08:42:17.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:17.826 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:17.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:17.827 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:17.928 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:17.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:17.966 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:17.966 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:18.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:18.117 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:18.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:18.118 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:18.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:18.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:18.254 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:18.255 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:18.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:18.393 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:18.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:18.394 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:18.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:18 smithi145 ceph-mon[33384]: pgmap v132: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.6 KiB/s rd, 106 MiB/s wr, 1.69k op/s 2024-09-11T08:42:18.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:18 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1349391864' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:42:18.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:18.532 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:18.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:18.533 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:18.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:18 smithi092 ceph-mon[29647]: pgmap v132: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.6 KiB/s rd, 106 MiB/s wr, 1.69k op/s 2024-09-11T08:42:18.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:18 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1349391864' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:42:18.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:18 smithi092 ceph-mon[41114]: pgmap v132: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.6 KiB/s rd, 106 MiB/s wr, 1.69k op/s 2024-09-11T08:42:18.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:18 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1349391864' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:42:18.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:18.697 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 67 16 102957 102941 96.003 68.25 0.0239859 0.0104122 2024-09-11T08:42:18.716 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:18.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:18.717 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:18.818 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:18.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:18.845 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:18.845 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:18.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:18.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:18.985 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:19.086 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:19.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:19.127 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:19.228 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:19.254 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:19.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:19.255 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:19.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-11T08:42:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:19.300 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:19.356 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:19.382 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:19.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:19.383 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:19.484 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:19.507 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:19.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:19.508 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:19.609 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:19.664 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:19.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:19.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:19.698 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 68 16 103996 103980 95.546 64.9375 0.00707506 0.0104617 2024-09-11T08:42:19.766 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:19.793 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:19.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:19.794 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:19.861 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:19 smithi092 ceph-mon[41114]: pgmap v133: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-11T08:42:19.861 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:19 smithi092 ceph-mon[29647]: pgmap v133: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-11T08:42:19.895 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:19.922 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:19.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:19.923 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:20.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:20.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:20.063 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:20.063 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:20.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:19 smithi145 ceph-mon[33384]: pgmap v133: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-11T08:42:20.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:20.204 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:20.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:20.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:20.305 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:20.350 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:20.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:20.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:20.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:20.493 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:20.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:20.494 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:20.595 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:20.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:42:20.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:20.708 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:20.709 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:20.709 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 69 16 105100 105084 95.1613 69 0.00877977 0.0105038 2024-09-11T08:42:20.810 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:20.836 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:20.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:20.837 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:20.938 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:20.962 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:20.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:20.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:21.063 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:21.091 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:21.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:21.092 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:21.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:21.231 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:21.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:21.232 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:21.333 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:21.362 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:21.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:21.363 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:21.463 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:21.489 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:21.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:21.490 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:21.591 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:21.617 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:21.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:21.618 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:21.698 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 70 16 106206 106190 94.7893 69.125 0.00626118 0.0105448 2024-09-11T08:42:21.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:21.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:21.759 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:21.759 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:21.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:42:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:42:21] "GET /metrics HTTP/1.1" 200 223615 "" "Prometheus/2.43.0" 2024-09-11T08:42:21.860 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:21.883 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:21.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:21.884 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:21.985 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:22.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:22.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:22.137 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:22.162 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:22.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:22.163 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:22.264 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:22.292 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:22.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:22.293 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:22.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:22 smithi092 ceph-mon[29647]: pgmap v134: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 47 MiB/s wr, 753 op/s 2024-09-11T08:42:22.366 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:22 smithi092 ceph-mon[41114]: pgmap v134: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 47 MiB/s wr, 753 op/s 2024-09-11T08:42:22.394 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:22.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:22 smithi145 ceph-mon[33384]: pgmap v134: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 47 MiB/s wr, 753 op/s 2024-09-11T08:42:22.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:22.442 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:22.442 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:22.543 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:22.590 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:22.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:22.591 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:22.692 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:22.698 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 71 16 107236 107220 94.3609 64.375 0.0120668 0.0105926 2024-09-11T08:42:22.717 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:22.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:22.719 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:22.819 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:22.844 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:22.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:22.845 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:22.850 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:22.946 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:22.973 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:22.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:22.974 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:23.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:23.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:23.114 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:23.114 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:23.115 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:42:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:42:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:23.130 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:42:23.132 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T08:42:23.215 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:23.257 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:23.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:23.258 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:23.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:23 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4002472440' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:42:23.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:23 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:42:23.259 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:23 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1390897547' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T08:42:23.259 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:23 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4002472440' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:42:23.259 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:23 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:42:23.259 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:23 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1390897547' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T08:42:23.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:23.402 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:23.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:23.403 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:23.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:23 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4002472440' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:42:23.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:23 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:42:23.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:23 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1390897547' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T08:42:23.504 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:23.546 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:23.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:23.547 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:23.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:23.689 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:23.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:23.690 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:23.698 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 72 16 108199 108183 93.8863 60.1875 0.00802516 0.0106467 2024-09-11T08:42:23.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:23.834 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:23.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:23.835 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:23.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:23.973 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:23.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:23.974 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:24.075 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:24.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:24.102 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:24.103 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:24.194 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:24 smithi092 ceph-mon[29647]: pgmap v135: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-11T08:42:24.195 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:24 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:42:24.195 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:24 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:42:24.195 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:24 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1390897547' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T08:42:24.195 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:24 smithi092 ceph-mon[29647]: osdmap e108: 8 total, 8 up, 8 in 2024-09-11T08:42:24.195 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:24 smithi092 ceph-mon[41114]: pgmap v135: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-11T08:42:24.195 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:24 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:42:24.195 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:24 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:42:24.195 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:24 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1390897547' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T08:42:24.195 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:24 smithi092 ceph-mon[41114]: osdmap e108: 8 total, 8 up, 8 in 2024-09-11T08:42:24.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:24.242 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:24.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:24.243 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:24.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:24.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:24.384 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:24.384 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:24.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:24 smithi145 ceph-mon[33384]: pgmap v135: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-11T08:42:24.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:24 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:42:24.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:24 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:42:24.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:24 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1390897547' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T08:42:24.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:24 smithi145 ceph-mon[33384]: osdmap e108: 8 total, 8 up, 8 in 2024-09-11T08:42:24.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:24.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:24.535 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:24.636 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:24.660 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:24.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:24.661 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:24.698 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 73 16 109220 109204 93.4743 63.8125 0.0163942 0.0106934 2024-09-11T08:42:24.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:24.801 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:24.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:24.802 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:24.903 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:24.946 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:24.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:24.947 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:25.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:25.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:25.091 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:25.091 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:25.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 1 1 2024-09-11T08:42:25.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:25.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:25.243 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:25.243 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:25.344 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:25.370 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:25.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:25.371 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:25.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:25 smithi145 ceph-mon[33384]: 4.18 scrub starts 2024-09-11T08:42:25.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:25 smithi145 ceph-mon[33384]: 4.18 scrub ok 2024-09-11T08:42:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:25 smithi145 ceph-mon[33384]: 7.c scrub starts 2024-09-11T08:42:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:25 smithi145 ceph-mon[33384]: 1.0 scrub starts 2024-09-11T08:42:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:25 smithi145 ceph-mon[33384]: 1.0 scrub ok 2024-09-11T08:42:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:25 smithi145 ceph-mon[33384]: 5.12 scrub starts 2024-09-11T08:42:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:25 smithi145 ceph-mon[33384]: 5.12 scrub ok 2024-09-11T08:42:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:25 smithi145 ceph-mon[33384]: 6.a scrub starts 2024-09-11T08:42:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:25 smithi145 ceph-mon[33384]: 6.a scrub ok 2024-09-11T08:42:25.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:25 smithi145 ceph-mon[33384]: 7.e scrub starts 2024-09-11T08:42:25.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:25.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:25.521 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:25.521 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:25.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[29647]: 4.18 scrub starts 2024-09-11T08:42:25.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[29647]: 4.18 scrub ok 2024-09-11T08:42:25.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[29647]: 7.c scrub starts 2024-09-11T08:42:25.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[29647]: 1.0 scrub starts 2024-09-11T08:42:25.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[29647]: 1.0 scrub ok 2024-09-11T08:42:25.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[29647]: 5.12 scrub starts 2024-09-11T08:42:25.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[29647]: 5.12 scrub ok 2024-09-11T08:42:25.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[29647]: 6.a scrub starts 2024-09-11T08:42:25.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[29647]: 6.a scrub ok 2024-09-11T08:42:25.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[29647]: 7.e scrub starts 2024-09-11T08:42:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[41114]: 4.18 scrub starts 2024-09-11T08:42:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[41114]: 4.18 scrub ok 2024-09-11T08:42:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[41114]: 7.c scrub starts 2024-09-11T08:42:25.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[41114]: 1.0 scrub starts 2024-09-11T08:42:25.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[41114]: 1.0 scrub ok 2024-09-11T08:42:25.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[41114]: 5.12 scrub starts 2024-09-11T08:42:25.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[41114]: 5.12 scrub ok 2024-09-11T08:42:25.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[41114]: 6.a scrub starts 2024-09-11T08:42:25.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[41114]: 6.a scrub ok 2024-09-11T08:42:25.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:25 smithi092 ceph-mon[41114]: 7.e scrub starts 2024-09-11T08:42:25.623 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:25.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:25.660 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:25.660 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:25.698 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 74 16 110187 110171 93.0279 60.4375 0.00343538 0.0107403 2024-09-11T08:42:25.761 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:25.786 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:25.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:25.787 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:25.874 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:25.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:25.940 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:25.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:25.941 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:26.042 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:26.069 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:26.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:26.070 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:26.171 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:26.195 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:26.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:26.196 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:26.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:26.324 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:26.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:26.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:26.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:42:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:42:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:26 smithi145 ceph-mon[33384]: 4.1d scrub starts 2024-09-11T08:42:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:26 smithi145 ceph-mon[33384]: 4.1d scrub ok 2024-09-11T08:42:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:26 smithi145 ceph-mon[33384]: 6.15 scrub starts 2024-09-11T08:42:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:26 smithi145 ceph-mon[33384]: 6.15 scrub ok 2024-09-11T08:42:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:26 smithi145 ceph-mon[33384]: pgmap v137: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-11T08:42:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:26 smithi145 ceph-mon[33384]: 5.1d scrub starts 2024-09-11T08:42:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:26 smithi145 ceph-mon[33384]: 5.1d scrub ok 2024-09-11T08:42:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:26 smithi145 ceph-mon[33384]: 4.15 scrub starts 2024-09-11T08:42:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:26 smithi145 ceph-mon[33384]: 4.15 scrub ok 2024-09-11T08:42:26.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:26.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:26.459 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:26.459 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:26.560 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[29647]: 4.1d scrub starts 2024-09-11T08:42:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[29647]: 4.1d scrub ok 2024-09-11T08:42:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[29647]: 6.15 scrub starts 2024-09-11T08:42:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[29647]: 6.15 scrub ok 2024-09-11T08:42:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[29647]: pgmap v137: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-11T08:42:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[29647]: 5.1d scrub starts 2024-09-11T08:42:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[29647]: 5.1d scrub ok 2024-09-11T08:42:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[29647]: 4.15 scrub starts 2024-09-11T08:42:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[29647]: 4.15 scrub ok 2024-09-11T08:42:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[41114]: 4.1d scrub starts 2024-09-11T08:42:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[41114]: 4.1d scrub ok 2024-09-11T08:42:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[41114]: 6.15 scrub starts 2024-09-11T08:42:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[41114]: 6.15 scrub ok 2024-09-11T08:42:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[41114]: pgmap v137: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-11T08:42:26.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[41114]: 5.1d scrub starts 2024-09-11T08:42:26.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[41114]: 5.1d scrub ok 2024-09-11T08:42:26.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[41114]: 4.15 scrub starts 2024-09-11T08:42:26.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:26 smithi092 ceph-mon[41114]: 4.15 scrub ok 2024-09-11T08:42:26.587 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:26.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:26.588 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:26.689 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:26.698 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 75 16 111064 111048 92.5183 54.8125 0.0031799 0.0108008 2024-09-11T08:42:26.709 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:26.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:26.710 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:26.811 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:26.838 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:26.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:26.839 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:26.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:26.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:26.963 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:27.064 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:27.091 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:27.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:27.092 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:27.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:27.224 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:27.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:27.225 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:27.325 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:27.350 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:27.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:27.351 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:27.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:27 smithi145 ceph-mon[33384]: 7.c scrub ok 2024-09-11T08:42:27.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:27 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:27.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:27.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:27.474 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:27.474 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:27.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:27 smithi092 ceph-mon[29647]: 7.c scrub ok 2024-09-11T08:42:27.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:27 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:27.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:27 smithi092 ceph-mon[41114]: 7.c scrub ok 2024-09-11T08:42:27.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:27 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:27.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:27.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:27.619 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:27.619 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:27.698 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 76 16 112006 111990 92.0757 58.875 0.0190154 0.0108556 2024-09-11T08:42:27.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:27.763 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:27.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:27.765 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:27.865 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:27.906 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:27.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:27.907 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:27.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:42:28.008 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:28.033 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:28.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:28.035 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:28.135 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:28.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:28.162 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:28.162 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:28.237 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:28 smithi092 ceph-mon[29647]: 6.9 scrub starts 2024-09-11T08:42:28.237 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:28 smithi092 ceph-mon[29647]: 6.9 scrub ok 2024-09-11T08:42:28.237 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:28 smithi092 ceph-mon[29647]: pgmap v138: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.7 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T08:42:28.237 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:28 smithi092 ceph-mon[29647]: 3.5 scrub starts 2024-09-11T08:42:28.237 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:28 smithi092 ceph-mon[29647]: 3.5 scrub ok 2024-09-11T08:42:28.238 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:28 smithi092 ceph-mon[41114]: 6.9 scrub starts 2024-09-11T08:42:28.238 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:28 smithi092 ceph-mon[41114]: 6.9 scrub ok 2024-09-11T08:42:28.238 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:28 smithi092 ceph-mon[41114]: pgmap v138: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.7 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T08:42:28.238 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:28 smithi092 ceph-mon[41114]: 3.5 scrub starts 2024-09-11T08:42:28.238 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:28 smithi092 ceph-mon[41114]: 3.5 scrub ok 2024-09-11T08:42:28.264 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:28.288 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:28.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:28.289 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:28.390 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:28.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:28 smithi145 ceph-mon[33384]: 6.9 scrub starts 2024-09-11T08:42:28.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:28 smithi145 ceph-mon[33384]: 6.9 scrub ok 2024-09-11T08:42:28.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:28 smithi145 ceph-mon[33384]: pgmap v138: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.7 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T08:42:28.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:28 smithi145 ceph-mon[33384]: 3.5 scrub starts 2024-09-11T08:42:28.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:28 smithi145 ceph-mon[33384]: 3.5 scrub ok 2024-09-11T08:42:28.417 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:28.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:28.418 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:28.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:28.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:28.562 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:28.563 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:28.663 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:28.691 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:28.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:28.693 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:28.699 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 77 16 112974 112958 91.6646 60.5 0.0180814 0.010905 2024-09-11T08:42:28.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:28.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:28.834 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:28.834 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:28.936 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:28.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:28.965 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:28.965 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:29.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:29.108 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:29.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:29.109 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:29.210 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:29.240 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:29.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:29.241 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:29.341 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:29.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:29.370 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:29.371 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:29.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:29 smithi145 ceph-mon[33384]: 5.3 scrub starts 2024-09-11T08:42:29.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:29 smithi145 ceph-mon[33384]: 5.3 scrub ok 2024-09-11T08:42:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:29 smithi145 ceph-mon[33384]: 7.8 scrub starts 2024-09-11T08:42:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:29 smithi145 ceph-mon[33384]: 7.e scrub ok 2024-09-11T08:42:29.412 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:29 smithi092 ceph-mon[41114]: 5.3 scrub starts 2024-09-11T08:42:29.412 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:29 smithi092 ceph-mon[41114]: 5.3 scrub ok 2024-09-11T08:42:29.413 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:29 smithi092 ceph-mon[41114]: 7.8 scrub starts 2024-09-11T08:42:29.413 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:29 smithi092 ceph-mon[41114]: 7.e scrub ok 2024-09-11T08:42:29.414 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:29 smithi092 ceph-mon[29647]: 5.3 scrub starts 2024-09-11T08:42:29.414 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:29 smithi092 ceph-mon[29647]: 5.3 scrub ok 2024-09-11T08:42:29.414 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:29 smithi092 ceph-mon[29647]: 7.8 scrub starts 2024-09-11T08:42:29.414 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:29 smithi092 ceph-mon[29647]: 7.e scrub ok 2024-09-11T08:42:29.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:29.536 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:29.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:29.537 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:29.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:29.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:29.679 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:29.679 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:29.701 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 78 16 114018 114002 91.3259 65.25 0.019117 0.0109449 2024-09-11T08:42:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:29.733 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:42:29.733 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:42:29.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:29.817 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:29.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:29.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:29.919 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:29.964 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:29.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:29.965 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:30.065 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:30.106 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:30.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:30.108 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:30.209 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:30.237 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:30 smithi092 ceph-mon[29647]: 3.2 scrub starts 2024-09-11T08:42:30.237 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:30 smithi092 ceph-mon[29647]: 3.2 scrub ok 2024-09-11T08:42:30.238 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:30 smithi092 ceph-mon[29647]: 3.15 scrub starts 2024-09-11T08:42:30.238 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:30 smithi092 ceph-mon[29647]: 3.15 scrub ok 2024-09-11T08:42:30.238 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:30 smithi092 ceph-mon[29647]: pgmap v139: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.7 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T08:42:30.238 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:30.238 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:30 smithi092 ceph-mon[41114]: 3.2 scrub starts 2024-09-11T08:42:30.238 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:30 smithi092 ceph-mon[41114]: 3.2 scrub ok 2024-09-11T08:42:30.238 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:30 smithi092 ceph-mon[41114]: 3.15 scrub starts 2024-09-11T08:42:30.239 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:30 smithi092 ceph-mon[41114]: 3.15 scrub ok 2024-09-11T08:42:30.239 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:30 smithi092 ceph-mon[41114]: pgmap v139: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.7 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T08:42:30.239 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:30.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:30.240 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:30.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:30.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:30.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:30.385 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:30.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:30 smithi145 ceph-mon[33384]: 3.2 scrub starts 2024-09-11T08:42:30.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:30 smithi145 ceph-mon[33384]: 3.2 scrub ok 2024-09-11T08:42:30.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:30 smithi145 ceph-mon[33384]: 3.15 scrub starts 2024-09-11T08:42:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:30 smithi145 ceph-mon[33384]: 3.15 scrub ok 2024-09-11T08:42:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:30 smithi145 ceph-mon[33384]: pgmap v139: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.7 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T08:42:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:30.486 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:30.512 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:30.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:30.514 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:30.537 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:42:30.537 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:42:30.573 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:42:30.573 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:42:30.614 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:42:30.614 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:42:30.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:30.649 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:42:30.649 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:42:30.651 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:30.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:30.653 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:30.679 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:42:30.679 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:42:30.699 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 79 16 114964 114948 90.9183 59.125 0.0157298 0.0109936 2024-09-11T08:42:30.718 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:42:30.719 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:42:30.749 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:42:30.749 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:42:30.755 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:30.781 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:42:30.781 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:42:30.797 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:30.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:30.798 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:30.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:30.937 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:30.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:30.938 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:31.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:31.084 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:31.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:31.085 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:31.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-11T08:42:31.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:31.217 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:31.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:31.218 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:31.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:31.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:31.380 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:31.380 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:31.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:31 smithi145 ceph-mon[33384]: 5.13 scrub starts 2024-09-11T08:42:31.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:31 smithi145 ceph-mon[33384]: 5.13 scrub ok 2024-09-11T08:42:31.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:31 smithi145 ceph-mon[33384]: 7.6 scrub starts 2024-09-11T08:42:31.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:31 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1021649064' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:42:31.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:31 smithi092 ceph-mon[29647]: 5.13 scrub starts 2024-09-11T08:42:31.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:31 smithi092 ceph-mon[29647]: 5.13 scrub ok 2024-09-11T08:42:31.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:31 smithi092 ceph-mon[29647]: 7.6 scrub starts 2024-09-11T08:42:31.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:31 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1021649064' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:42:31.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:31 smithi092 ceph-mon[41114]: 5.13 scrub starts 2024-09-11T08:42:31.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:31 smithi092 ceph-mon[41114]: 5.13 scrub ok 2024-09-11T08:42:31.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:31 smithi092 ceph-mon[41114]: 7.6 scrub starts 2024-09-11T08:42:31.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:31 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1021649064' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:42:31.450 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:42:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:42:31] "GET /metrics HTTP/1.1" 200 223632 "" "Prometheus/2.43.0" 2024-09-11T08:42:31.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:31.517 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:31.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:31.518 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:31.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:31.653 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:31.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:31.656 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:31.700 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:42:31.701102+0000 min lat: 0.00133167 max lat: 0.172138 avg lat: 0.0110469 2024-09-11T08:42:31.700 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:42:31.700 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 80 16 115854 115838 90.4771 55.625 0.00961504 0.0110469 2024-09-11T08:42:31.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:31.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:31.790 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:31.891 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:31.918 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:31.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:31.919 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:32.020 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:32.047 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:32.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:32.048 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:32.149 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:32.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:32.175 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:32.175 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:32.276 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:32.303 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:32.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:32.307 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:32.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:32.447 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:32.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:32.448 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:32.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:32 smithi092 ceph-mon[29647]: 5.1 scrub starts 2024-09-11T08:42:32.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:32 smithi092 ceph-mon[29647]: 5.1 scrub ok 2024-09-11T08:42:32.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:32 smithi092 ceph-mon[29647]: 7.8 scrub ok 2024-09-11T08:42:32.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:32 smithi092 ceph-mon[29647]: pgmap v140: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-11T08:42:32.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:32 smithi092 ceph-mon[29647]: 7.9 scrub starts 2024-09-11T08:42:32.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:32 smithi092 ceph-mon[29647]: 5.c scrub starts 2024-09-11T08:42:32.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:32 smithi092 ceph-mon[29647]: 5.c scrub ok 2024-09-11T08:42:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:32 smithi092 ceph-mon[41114]: 5.1 scrub starts 2024-09-11T08:42:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:32 smithi092 ceph-mon[41114]: 5.1 scrub ok 2024-09-11T08:42:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:32 smithi092 ceph-mon[41114]: 7.8 scrub ok 2024-09-11T08:42:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:32 smithi092 ceph-mon[41114]: pgmap v140: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-11T08:42:32.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:32 smithi092 ceph-mon[41114]: 7.9 scrub starts 2024-09-11T08:42:32.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:32 smithi092 ceph-mon[41114]: 5.c scrub starts 2024-09-11T08:42:32.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:32 smithi092 ceph-mon[41114]: 5.c scrub ok 2024-09-11T08:42:32.596 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:32.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:32.597 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:32.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:32 smithi145 ceph-mon[33384]: 5.1 scrub starts 2024-09-11T08:42:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:32 smithi145 ceph-mon[33384]: 5.1 scrub ok 2024-09-11T08:42:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:32 smithi145 ceph-mon[33384]: 7.8 scrub ok 2024-09-11T08:42:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:32 smithi145 ceph-mon[33384]: pgmap v140: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-11T08:42:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:32 smithi145 ceph-mon[33384]: 7.9 scrub starts 2024-09-11T08:42:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:32 smithi145 ceph-mon[33384]: 5.c scrub starts 2024-09-11T08:42:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:32 smithi145 ceph-mon[33384]: 5.c scrub ok 2024-09-11T08:42:32.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:32.704 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 81 16 116852 116836 90.1283 62.375 0.0123198 0.0110906 2024-09-11T08:42:32.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:32.746 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:32.746 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:32.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:32.891 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:32.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:32.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:32.993 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:32.996 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:33.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:33.047 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:33.047 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:33.148 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:33.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:42:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:42:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:33.198 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:33.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:33.199 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:33.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:33.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:33.348 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:33.348 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:33.449 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:33.476 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:33.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:33.477 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:33.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:33 smithi092 ceph-mon[41114]: 6.1e scrub starts 2024-09-11T08:42:33.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:33 smithi092 ceph-mon[41114]: 6.1e scrub ok 2024-09-11T08:42:33.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:33 smithi092 ceph-mon[41114]: 6.0 scrub starts 2024-09-11T08:42:33.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:33 smithi092 ceph-mon[41114]: 6.0 scrub ok 2024-09-11T08:42:33.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:33 smithi092 ceph-mon[41114]: 6.1d scrub starts 2024-09-11T08:42:33.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:33 smithi092 ceph-mon[41114]: 6.1d scrub ok 2024-09-11T08:42:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:33 smithi092 ceph-mon[29647]: 6.1e scrub starts 2024-09-11T08:42:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:33 smithi092 ceph-mon[29647]: 6.1e scrub ok 2024-09-11T08:42:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:33 smithi092 ceph-mon[29647]: 6.0 scrub starts 2024-09-11T08:42:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:33 smithi092 ceph-mon[29647]: 6.0 scrub ok 2024-09-11T08:42:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:33 smithi092 ceph-mon[29647]: 6.1d scrub starts 2024-09-11T08:42:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:33 smithi092 ceph-mon[29647]: 6.1d scrub ok 2024-09-11T08:42:33.578 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:33.605 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:33.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:33.606 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:33.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:33 smithi145 ceph-mon[33384]: 6.1e scrub starts 2024-09-11T08:42:33.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:33 smithi145 ceph-mon[33384]: 6.1e scrub ok 2024-09-11T08:42:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:33 smithi145 ceph-mon[33384]: 6.0 scrub starts 2024-09-11T08:42:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:33 smithi145 ceph-mon[33384]: 6.0 scrub ok 2024-09-11T08:42:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:33 smithi145 ceph-mon[33384]: 6.1d scrub starts 2024-09-11T08:42:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:33 smithi145 ceph-mon[33384]: 6.1d scrub ok 2024-09-11T08:42:33.701 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 82 16 117820 117804 89.767 60.5 0.00282936 0.0111334 2024-09-11T08:42:33.708 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:33.733 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:33.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:33.734 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:33.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:33.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:33.874 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:33.874 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:33.975 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:34.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:34.023 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:34.023 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:34.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:34.158 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:34.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:34.159 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:34.260 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:34.303 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:34.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:34.304 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:34.405 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:34.430 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:34.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:34.431 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:34.532 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:34.560 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:34.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:34.561 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:34.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:34 smithi092 ceph-mon[29647]: pgmap v141: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 61 MiB/s wr, 980 op/s 2024-09-11T08:42:34.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:34 smithi092 ceph-mon[41114]: pgmap v141: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 61 MiB/s wr, 980 op/s 2024-09-11T08:42:34.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:34 smithi145 ceph-mon[33384]: pgmap v141: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 61 MiB/s wr, 980 op/s 2024-09-11T08:42:34.663 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:34.692 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:34.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:34.693 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:34.701 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 83 16 118780 118764 89.4084 60 0.00707395 0.01118 2024-09-11T08:42:34.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:34.836 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:34.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:34.837 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:34.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:35.016 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:35.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:35.017 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:35.118 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:35.145 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:35.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:35.146 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:35.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:35.283 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:35.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:35.284 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:35.385 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:35.411 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:35.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:35.412 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:35.488 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:35 smithi092 ceph-mon[41114]: 7.6 scrub ok 2024-09-11T08:42:35.488 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:35 smithi092 ceph-mon[41114]: 3.13 scrub starts 2024-09-11T08:42:35.488 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:35 smithi092 ceph-mon[41114]: 3.13 scrub ok 2024-09-11T08:42:35.490 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:35 smithi092 ceph-mon[29647]: 7.6 scrub ok 2024-09-11T08:42:35.490 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:35 smithi092 ceph-mon[29647]: 3.13 scrub starts 2024-09-11T08:42:35.490 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:35 smithi092 ceph-mon[29647]: 3.13 scrub ok 2024-09-11T08:42:35.513 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:35.537 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:35.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:35.538 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:35.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:35.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:35 smithi145 ceph-mon[33384]: 7.6 scrub ok 2024-09-11T08:42:35.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:35 smithi145 ceph-mon[33384]: 3.13 scrub starts 2024-09-11T08:42:35.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:35 smithi145 ceph-mon[33384]: 3.13 scrub ok 2024-09-11T08:42:35.681 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:35.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:35.682 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:35.702 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 84 16 119905 119889 89.181 70.3125 0.00670723 0.0112087 2024-09-11T08:42:35.783 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:35.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:35.812 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:35.812 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:35.913 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:35.938 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:35.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:35.939 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:36.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:36.076 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:36.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:36.077 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:36.178 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:36.206 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:36.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:36.206 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:36.256 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:36 smithi092 ceph-mon[29647]: 6.d scrub starts 2024-09-11T08:42:36.256 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:36 smithi092 ceph-mon[29647]: 6.d scrub ok 2024-09-11T08:42:36.256 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:36 smithi092 ceph-mon[29647]: 7.9 scrub ok 2024-09-11T08:42:36.256 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:36 smithi092 ceph-mon[29647]: 4.b scrub starts 2024-09-11T08:42:36.256 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:36 smithi092 ceph-mon[29647]: pgmap v142: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 858 B/s rd, 51 MiB/s wr, 822 op/s 2024-09-11T08:42:36.256 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:36 smithi092 ceph-mon[29647]: 4.b scrub ok 2024-09-11T08:42:36.256 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:36 smithi092 ceph-mon[41114]: 6.d scrub starts 2024-09-11T08:42:36.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:36 smithi092 ceph-mon[41114]: 6.d scrub ok 2024-09-11T08:42:36.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:36 smithi092 ceph-mon[41114]: 7.9 scrub ok 2024-09-11T08:42:36.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:36 smithi092 ceph-mon[41114]: 4.b scrub starts 2024-09-11T08:42:36.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:36 smithi092 ceph-mon[41114]: pgmap v142: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 858 B/s rd, 51 MiB/s wr, 822 op/s 2024-09-11T08:42:36.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:36 smithi092 ceph-mon[41114]: 4.b scrub ok 2024-09-11T08:42:36.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:36.350 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:36.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:36.351 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:36.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:36.491 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:36.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:36.492 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:36.593 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:36.618 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:36.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:36.619 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:36.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:42:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:42:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:36 smithi145 ceph-mon[33384]: 6.d scrub starts 2024-09-11T08:42:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:36 smithi145 ceph-mon[33384]: 6.d scrub ok 2024-09-11T08:42:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:36 smithi145 ceph-mon[33384]: 7.9 scrub ok 2024-09-11T08:42:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:36 smithi145 ceph-mon[33384]: 4.b scrub starts 2024-09-11T08:42:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:36 smithi145 ceph-mon[33384]: pgmap v142: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 858 B/s rd, 51 MiB/s wr, 822 op/s 2024-09-11T08:42:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:36 smithi145 ceph-mon[33384]: 4.b scrub ok 2024-09-11T08:42:36.703 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 85 16 120957 120941 88.9038 65.75 0.0260213 0.0112434 2024-09-11T08:42:36.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:36.778 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:36.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:36.779 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:36.827 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:36.881 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:36.907 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:36.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:36.908 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:36.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:42:37.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:37.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:37.070 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:37.070 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:37.171 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:37.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:37.253 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:37.253 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:37.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:37.402 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:37.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:37.403 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:37.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:37 smithi092 ceph-mon[29647]: 3.8 scrub starts 2024-09-11T08:42:37.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:37 smithi092 ceph-mon[29647]: 3.8 scrub ok 2024-09-11T08:42:37.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:37 smithi092 ceph-mon[29647]: 7.f scrub starts 2024-09-11T08:42:37.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:37 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:37.464 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:37 smithi092 ceph-mon[29647]: 7.b scrub starts 2024-09-11T08:42:37.464 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:37 smithi092 ceph-mon[29647]: 3.3 scrub starts 2024-09-11T08:42:37.464 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:37 smithi092 ceph-mon[29647]: 3.3 scrub ok 2024-09-11T08:42:37.464 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:37 smithi092 ceph-mon[41114]: 3.8 scrub starts 2024-09-11T08:42:37.464 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:37 smithi092 ceph-mon[41114]: 3.8 scrub ok 2024-09-11T08:42:37.464 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:37 smithi092 ceph-mon[41114]: 7.f scrub starts 2024-09-11T08:42:37.464 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:37 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:37.464 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:37 smithi092 ceph-mon[41114]: 7.b scrub starts 2024-09-11T08:42:37.464 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:37 smithi092 ceph-mon[41114]: 3.3 scrub starts 2024-09-11T08:42:37.464 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:37 smithi092 ceph-mon[41114]: 3.3 scrub ok 2024-09-11T08:42:37.505 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:37.531 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:37.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:37.532 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:37.633 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:37.656 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:37.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:37.657 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:37 smithi145 ceph-mon[33384]: 3.8 scrub starts 2024-09-11T08:42:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:37 smithi145 ceph-mon[33384]: 3.8 scrub ok 2024-09-11T08:42:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:37 smithi145 ceph-mon[33384]: 7.f scrub starts 2024-09-11T08:42:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:37 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:37 smithi145 ceph-mon[33384]: 7.b scrub starts 2024-09-11T08:42:37.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:37 smithi145 ceph-mon[33384]: 3.3 scrub starts 2024-09-11T08:42:37.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:37 smithi145 ceph-mon[33384]: 3.3 scrub ok 2024-09-11T08:42:37.703 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 86 16 121777 121761 88.466 51.25 0.00895071 0.0112986 2024-09-11T08:42:37.759 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:37.795 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:37.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:37.796 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:37.897 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:37.925 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:37.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:37.926 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:38.027 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:38.052 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:38.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:38.053 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:38.154 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:38.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:38.183 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:38.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:38.217 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.1 to 1 (10000) 2024-09-11T08:42:38.217 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:42:38.284 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:38.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:38.311 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:38.311 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:38.412 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:38.437 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:38.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:38.438 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:38.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[29647]: 5.7 scrub starts 2024-09-11T08:42:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[29647]: 5.7 scrub ok 2024-09-11T08:42:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[29647]: 4.17 scrub starts 2024-09-11T08:42:38.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[29647]: 4.17 scrub ok 2024-09-11T08:42:38.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[29647]: pgmap v143: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-11T08:42:38.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1411959527' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T08:42:38.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T08:42:38.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/76035351' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:42:38.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:42:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[41114]: 5.7 scrub starts 2024-09-11T08:42:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[41114]: 5.7 scrub ok 2024-09-11T08:42:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[41114]: 4.17 scrub starts 2024-09-11T08:42:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[41114]: 4.17 scrub ok 2024-09-11T08:42:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[41114]: pgmap v143: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-11T08:42:38.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1411959527' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T08:42:38.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T08:42:38.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/76035351' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:42:38.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:38 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:42:38.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:38.583 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:38.583 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:38.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:38 smithi145 ceph-mon[33384]: 5.7 scrub starts 2024-09-11T08:42:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:38 smithi145 ceph-mon[33384]: 5.7 scrub ok 2024-09-11T08:42:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:38 smithi145 ceph-mon[33384]: 4.17 scrub starts 2024-09-11T08:42:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:38 smithi145 ceph-mon[33384]: 4.17 scrub ok 2024-09-11T08:42:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:38 smithi145 ceph-mon[33384]: pgmap v143: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-11T08:42:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:38 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1411959527' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T08:42:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:38 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T08:42:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:38 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/76035351' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:42:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:38 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:42:38.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:38.704 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 87 16 122667 122651 88.0885 55.625 0.00938724 0.0113479 2024-09-11T08:42:38.715 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:38.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:38.716 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:38.817 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:38.842 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:38.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:38.843 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:38.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:38.965 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:38.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:38.966 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:39.066 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:39.092 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:39.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:39.093 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:39.194 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:39.221 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:39.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:39.222 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:39.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:39.366 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:39.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:39.367 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:39.468 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:39.494 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:39.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:39.495 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:39.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[29647]: 3.10 scrub starts 2024-09-11T08:42:39.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[29647]: 3.10 scrub ok 2024-09-11T08:42:39.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:42:39.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:42:39.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T08:42:39.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:42:39.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[29647]: osdmap e109: 8 total, 8 up, 8 in 2024-09-11T08:42:39.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[29647]: 4.7 scrub starts 2024-09-11T08:42:39.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[29647]: 4.7 scrub ok 2024-09-11T08:42:39.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[41114]: 3.10 scrub starts 2024-09-11T08:42:39.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[41114]: 3.10 scrub ok 2024-09-11T08:42:39.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:42:39.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:42:39.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T08:42:39.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:42:39.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[41114]: osdmap e109: 8 total, 8 up, 8 in 2024-09-11T08:42:39.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[41114]: 4.7 scrub starts 2024-09-11T08:42:39.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:39 smithi092 ceph-mon[41114]: 4.7 scrub ok 2024-09-11T08:42:39.596 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:39.622 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:39.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:39.623 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:39.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:39 smithi145 ceph-mon[33384]: 3.10 scrub starts 2024-09-11T08:42:39.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:39 smithi145 ceph-mon[33384]: 3.10 scrub ok 2024-09-11T08:42:39.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:39 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:42:39.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:39 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:42:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:39 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T08:42:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:39 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:42:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:39 smithi145 ceph-mon[33384]: osdmap e109: 8 total, 8 up, 8 in 2024-09-11T08:42:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:39 smithi145 ceph-mon[33384]: 4.7 scrub starts 2024-09-11T08:42:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:39 smithi145 ceph-mon[33384]: 4.7 scrub ok 2024-09-11T08:42:39.703 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 88 16 123561 123545 87.7224 55.875 0.00483648 0.0113951 2024-09-11T08:42:39.724 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:39.750 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:39.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:39.751 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:39.852 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:39.879 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:39.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:39.880 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:39.981 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:40.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:40.011 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:40.011 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:40.112 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:40.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:40.142 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:40.142 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:40.243 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:40.279 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:40.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:40.280 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:40.381 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:40.411 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:40.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:40.412 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:40.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:40.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:40.553 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:40.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:40.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:40 smithi092 ceph-mon[29647]: 7.7 scrub starts 2024-09-11T08:42:40.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:40 smithi092 ceph-mon[29647]: pgmap v145: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 61 MiB/s wr, 979 op/s 2024-09-11T08:42:40.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:40 smithi092 ceph-mon[29647]: 7.b scrub ok 2024-09-11T08:42:40.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:40 smithi092 ceph-mon[29647]: 4.4 scrub starts 2024-09-11T08:42:40.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:40 smithi092 ceph-mon[29647]: 4.4 scrub ok 2024-09-11T08:42:40.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:40 smithi092 ceph-mon[29647]: 6.8 scrub starts 2024-09-11T08:42:40.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:40 smithi092 ceph-mon[29647]: 6.8 scrub ok 2024-09-11T08:42:40.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:40 smithi092 ceph-mon[41114]: 7.7 scrub starts 2024-09-11T08:42:40.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:40 smithi092 ceph-mon[41114]: pgmap v145: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 61 MiB/s wr, 979 op/s 2024-09-11T08:42:40.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:40 smithi092 ceph-mon[41114]: 7.b scrub ok 2024-09-11T08:42:40.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:40 smithi092 ceph-mon[41114]: 4.4 scrub starts 2024-09-11T08:42:40.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:40 smithi092 ceph-mon[41114]: 4.4 scrub ok 2024-09-11T08:42:40.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:40 smithi092 ceph-mon[41114]: 6.8 scrub starts 2024-09-11T08:42:40.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:40 smithi092 ceph-mon[41114]: 6.8 scrub ok 2024-09-11T08:42:40.654 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:40.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:40 smithi145 ceph-mon[33384]: 7.7 scrub starts 2024-09-11T08:42:40.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:40 smithi145 ceph-mon[33384]: pgmap v145: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 61 MiB/s wr, 979 op/s 2024-09-11T08:42:40.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:40 smithi145 ceph-mon[33384]: 7.b scrub ok 2024-09-11T08:42:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:40 smithi145 ceph-mon[33384]: 4.4 scrub starts 2024-09-11T08:42:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:40 smithi145 ceph-mon[33384]: 4.4 scrub ok 2024-09-11T08:42:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:40 smithi145 ceph-mon[33384]: 6.8 scrub starts 2024-09-11T08:42:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:40 smithi145 ceph-mon[33384]: 6.8 scrub ok 2024-09-11T08:42:40.684 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:40.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:40.685 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:40.703 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 89 15 124656 124641 87.5064 68.5 0.0147851 0.0114233 2024-09-11T08:42:40.786 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:40.812 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:40.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:40.814 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:40.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 2 1 2024-09-11T08:42:40.915 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:40.940 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:40.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:40.942 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:41.042 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:41.068 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:41.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:41.069 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:41.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:41.239 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:41.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:41.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:41.341 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:41.364 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:41.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:41.365 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:41.393 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:41.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:41 smithi092 ceph-mon[29647]: 5.19 scrub starts 2024-09-11T08:42:41.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:41 smithi092 ceph-mon[29647]: 5.19 scrub ok 2024-09-11T08:42:41.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:41 smithi092 ceph-mon[29647]: 5.1c scrub starts 2024-09-11T08:42:41.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:41 smithi092 ceph-mon[29647]: 5.1c scrub ok 2024-09-11T08:42:41.449 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:42:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:42:41] "GET /metrics HTTP/1.1" 200 223632 "" "Prometheus/2.43.0" 2024-09-11T08:42:41.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:41 smithi092 ceph-mon[41114]: 5.19 scrub starts 2024-09-11T08:42:41.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:41 smithi092 ceph-mon[41114]: 5.19 scrub ok 2024-09-11T08:42:41.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:41 smithi092 ceph-mon[41114]: 5.1c scrub starts 2024-09-11T08:42:41.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:41 smithi092 ceph-mon[41114]: 5.1c scrub ok 2024-09-11T08:42:41.466 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:41.494 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:41.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:41.495 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:41.596 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:41.620 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:41.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:41.621 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:41 smithi145 ceph-mon[33384]: 5.19 scrub starts 2024-09-11T08:42:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:41 smithi145 ceph-mon[33384]: 5.19 scrub ok 2024-09-11T08:42:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:41 smithi145 ceph-mon[33384]: 5.1c scrub starts 2024-09-11T08:42:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:41 smithi145 ceph-mon[33384]: 5.1c scrub ok 2024-09-11T08:42:41.704 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Total time run: 90.0146 2024-09-11T08:42:41.704 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Total writes made: 125525 2024-09-11T08:42:41.704 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Write size: 65536 2024-09-11T08:42:41.704 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Object size: 65536 2024-09-11T08:42:41.704 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Bandwidth (MB/sec): 87.156 2024-09-11T08:42:41.704 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Stddev Bandwidth: 34.249 2024-09-11T08:42:41.704 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Max bandwidth (MB/sec): 194.938 2024-09-11T08:42:41.704 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Min bandwidth (MB/sec): 51.25 2024-09-11T08:42:41.704 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Average IOPS: 1394 2024-09-11T08:42:41.704 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Stddev IOPS: 547.984 2024-09-11T08:42:41.704 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Max IOPS: 3119 2024-09-11T08:42:41.704 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Min IOPS: 820 2024-09-11T08:42:41.705 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Average Latency(s): 0.0114691 2024-09-11T08:42:41.705 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Stddev Latency(s): 0.00877017 2024-09-11T08:42:41.705 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Max latency(s): 0.172138 2024-09-11T08:42:41.705 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Min latency(s): 0.00133167 2024-09-11T08:42:41.710 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Cleaning up (deleting benchmark objects) 2024-09-11T08:42:41.723 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:41.773 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:41.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:41.774 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:41.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:41.915 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:41.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:41.916 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:42.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:42.057 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:42.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:42.058 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:42.159 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:42.187 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:42.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:42.188 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:42.289 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:42.314 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:42.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:42.315 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:42.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:42 smithi092 ceph-mon[29647]: 6.19 deep-scrub starts 2024-09-11T08:42:42.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:42 smithi092 ceph-mon[29647]: 6.19 deep-scrub ok 2024-09-11T08:42:42.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:42 smithi092 ceph-mon[29647]: 6.c scrub starts 2024-09-11T08:42:42.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:42 smithi092 ceph-mon[29647]: 6.c scrub ok 2024-09-11T08:42:42.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:42 smithi092 ceph-mon[29647]: pgmap v146: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.5 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 62 MiB/s wr, 985 op/s 2024-09-11T08:42:42.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:42 smithi092 ceph-mon[41114]: 6.19 deep-scrub starts 2024-09-11T08:42:42.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:42 smithi092 ceph-mon[41114]: 6.19 deep-scrub ok 2024-09-11T08:42:42.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:42 smithi092 ceph-mon[41114]: 6.c scrub starts 2024-09-11T08:42:42.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:42 smithi092 ceph-mon[41114]: 6.c scrub ok 2024-09-11T08:42:42.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:42 smithi092 ceph-mon[41114]: pgmap v146: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.5 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 62 MiB/s wr, 985 op/s 2024-09-11T08:42:42.337 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:42:42.338 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:42:42.338 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:42:42.338 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:42:42.338 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:42:42.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:42.462 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:42.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:42.463 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:42.564 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:42.591 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:42.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:42.593 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:42 smithi145 ceph-mon[33384]: 6.19 deep-scrub starts 2024-09-11T08:42:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:42 smithi145 ceph-mon[33384]: 6.19 deep-scrub ok 2024-09-11T08:42:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:42 smithi145 ceph-mon[33384]: 6.c scrub starts 2024-09-11T08:42:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:42 smithi145 ceph-mon[33384]: 6.c scrub ok 2024-09-11T08:42:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:42 smithi145 ceph-mon[33384]: pgmap v146: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.5 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 62 MiB/s wr, 985 op/s 2024-09-11T08:42:42.693 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:42.720 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:42.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:42.721 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:42.822 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:42.858 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:42.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:42.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:42.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:42:42.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:43.027 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:43.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:43.028 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:43.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-11T08:42:43.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:43.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:42:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:42:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:43.192 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:43.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:43.193 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:43.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:43 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:42:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:43 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:42:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:43 smithi092 ceph-mon[29647]: 7.f scrub ok 2024-09-11T08:42:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:43 smithi092 ceph-mon[29647]: 7.d scrub starts 2024-09-11T08:42:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:43 smithi092 ceph-mon[29647]: osdmap e110: 8 total, 8 up, 8 in 2024-09-11T08:42:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:43 smithi092 ceph-mon[29647]: 5.e scrub starts 2024-09-11T08:42:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:43 smithi092 ceph-mon[29647]: 5.e scrub ok 2024-09-11T08:42:43.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:43 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:42:43.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:43 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:42:43.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:43 smithi092 ceph-mon[41114]: 7.f scrub ok 2024-09-11T08:42:43.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:43 smithi092 ceph-mon[41114]: 7.d scrub starts 2024-09-11T08:42:43.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:43 smithi092 ceph-mon[41114]: osdmap e110: 8 total, 8 up, 8 in 2024-09-11T08:42:43.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:43 smithi092 ceph-mon[41114]: 5.e scrub starts 2024-09-11T08:42:43.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:43 smithi092 ceph-mon[41114]: 5.e scrub ok 2024-09-11T08:42:43.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:43.342 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:43.342 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:43.443 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:43.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:43.474 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:43.475 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:43.575 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:43.609 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:43.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:43.610 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:43.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:43 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:42:43.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:43 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:42:43.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:43 smithi145 ceph-mon[33384]: 7.f scrub ok 2024-09-11T08:42:43.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:43 smithi145 ceph-mon[33384]: 7.d scrub starts 2024-09-11T08:42:43.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:43 smithi145 ceph-mon[33384]: osdmap e110: 8 total, 8 up, 8 in 2024-09-11T08:42:43.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:43 smithi145 ceph-mon[33384]: 5.e scrub starts 2024-09-11T08:42:43.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:43 smithi145 ceph-mon[33384]: 5.e scrub ok 2024-09-11T08:42:43.711 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:43.736 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:43.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:43.737 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:43.838 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:43.862 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:43.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:43.863 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:43.932 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:43.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:44.022 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:44.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:44.023 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:44.124 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:44.151 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:44.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:44.152 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:44.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:44.299 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:44.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:44.300 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:44.333 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:44 smithi092 ceph-mon[29647]: pgmap v148: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-11T08:42:44.333 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:44 smithi092 ceph-mon[29647]: 7.7 scrub ok 2024-09-11T08:42:44.333 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:44 smithi092 ceph-mon[41114]: pgmap v148: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-11T08:42:44.334 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:44 smithi092 ceph-mon[41114]: 7.7 scrub ok 2024-09-11T08:42:44.401 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:44.440 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:44.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:44.442 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:44.542 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:44.571 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:44.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:44.572 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:44 smithi145 ceph-mon[33384]: pgmap v148: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-11T08:42:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:44 smithi145 ceph-mon[33384]: 7.7 scrub ok 2024-09-11T08:42:44.673 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:44.700 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:44.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:44.701 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:44.802 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:44.827 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:44.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:44.828 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:44.929 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:44.955 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:44.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:44.956 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:45.057 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:45.094 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:45.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:45.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:45.180 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:42:45.180 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:42:45.180 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:42:45.180 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:42:45.180 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:42:45.180 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:42:45.180 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:42:45.180 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:42:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:42:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:42:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:42:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:42:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:42:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:42:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:42:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:42:45.197 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:45.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:45.226 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:45.226 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:45.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:45.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:45.364 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:45.364 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:45.465 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:45.491 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:45.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:45.492 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:45.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:45 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1234869970' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:42:45.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:45 smithi092 ceph-mon[41114]: 5.10 scrub starts 2024-09-11T08:42:45.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:45 smithi092 ceph-mon[41114]: 5.10 scrub ok 2024-09-11T08:42:45.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:45 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:45.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:45.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:45 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1234869970' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:42:45.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:45 smithi092 ceph-mon[29647]: 5.10 scrub starts 2024-09-11T08:42:45.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:45 smithi092 ceph-mon[29647]: 5.10 scrub ok 2024-09-11T08:42:45.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:45 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:45.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:45.559 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:45.593 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:45.635 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:45.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:45.636 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:45 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1234869970' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:42:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:45 smithi145 ceph-mon[33384]: 5.10 scrub starts 2024-09-11T08:42:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:45 smithi145 ceph-mon[33384]: 5.10 scrub ok 2024-09-11T08:42:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:45 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:45.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:45.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:45.774 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:45.775 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:45.875 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:45.903 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:45.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:45.904 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:46.004 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:46.031 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:46.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:46.032 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:46.133 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:46.162 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:46.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:46.163 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:46.263 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:46.293 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:46.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:46.294 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:46.395 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:46.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:46.421 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:46.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:46.523 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:46.549 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:46.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:46.551 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:46 smithi092 ceph-mon[29647]: 7.0 deep-scrub starts 2024-09-11T08:42:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:46 smithi092 ceph-mon[29647]: 7.d scrub ok 2024-09-11T08:42:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:46 smithi092 ceph-mon[29647]: pgmap v149: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 37 MiB/s wr, 588 op/s 2024-09-11T08:42:46.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:46 smithi092 ceph-mon[29647]: 4.3 scrub starts 2024-09-11T08:42:46.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:46 smithi092 ceph-mon[29647]: 4.3 scrub ok 2024-09-11T08:42:46.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:46 smithi092 ceph-mon[29647]: 7.5 scrub starts 2024-09-11T08:42:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:46 smithi092 ceph-mon[41114]: 7.0 deep-scrub starts 2024-09-11T08:42:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:46 smithi092 ceph-mon[41114]: 7.d scrub ok 2024-09-11T08:42:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:46 smithi092 ceph-mon[41114]: pgmap v149: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 37 MiB/s wr, 588 op/s 2024-09-11T08:42:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:46 smithi092 ceph-mon[41114]: 4.3 scrub starts 2024-09-11T08:42:46.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:46 smithi092 ceph-mon[41114]: 4.3 scrub ok 2024-09-11T08:42:46.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:46 smithi092 ceph-mon[41114]: 7.5 scrub starts 2024-09-11T08:42:46.651 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:46.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:42:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:42:46.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:46 smithi145 ceph-mon[33384]: 7.0 deep-scrub starts 2024-09-11T08:42:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:46 smithi145 ceph-mon[33384]: 7.d scrub ok 2024-09-11T08:42:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:46 smithi145 ceph-mon[33384]: pgmap v149: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 37 MiB/s wr, 588 op/s 2024-09-11T08:42:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:46 smithi145 ceph-mon[33384]: 4.3 scrub starts 2024-09-11T08:42:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:46 smithi145 ceph-mon[33384]: 4.3 scrub ok 2024-09-11T08:42:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:46 smithi145 ceph-mon[33384]: 7.5 scrub starts 2024-09-11T08:42:46.680 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:46.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:46.681 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:46.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:46.820 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:46.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:46.822 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:46.922 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:46.950 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:46.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:46.951 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:47.052 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:47.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:47.091 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:47.092 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:47.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:47.194 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:47.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:47.258 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:47.258 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:47.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:47.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:47.399 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:47.500 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:47.530 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:47.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:47.531 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:47.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:47 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:42:47.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:47 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:42:47.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:47 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:47.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:47 smithi092 ceph-mon[41114]: 2.0 scrub starts 2024-09-11T08:42:47.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:47 smithi092 ceph-mon[41114]: 2.0 scrub ok 2024-09-11T08:42:47.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:47 smithi092 ceph-mon[41114]: osdmap e111: 8 total, 8 up, 8 in 2024-09-11T08:42:47.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:47 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:42:47.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:47 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:42:47.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:47 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:47.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:47 smithi092 ceph-mon[29647]: 2.0 scrub starts 2024-09-11T08:42:47.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:47 smithi092 ceph-mon[29647]: 2.0 scrub ok 2024-09-11T08:42:47.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:47 smithi092 ceph-mon[29647]: osdmap e111: 8 total, 8 up, 8 in 2024-09-11T08:42:47.631 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:47.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:47 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:42:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:47 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:42:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:47 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:47 smithi145 ceph-mon[33384]: 2.0 scrub starts 2024-09-11T08:42:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:47 smithi145 ceph-mon[33384]: 2.0 scrub ok 2024-09-11T08:42:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:47 smithi145 ceph-mon[33384]: osdmap e111: 8 total, 8 up, 8 in 2024-09-11T08:42:47.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:47.663 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:47.664 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:47.765 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:47.791 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:47.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:47.792 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:47.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:47.934 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:47.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:47.935 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:48.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:48.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:48.094 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:48.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:48.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:48.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:48.236 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:48.236 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:48.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:48 smithi092 ceph-mon[29647]: 5.14 scrub starts 2024-09-11T08:42:48.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:48 smithi092 ceph-mon[29647]: 5.14 scrub ok 2024-09-11T08:42:48.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:48 smithi092 ceph-mon[29647]: pgmap v150: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 7.4 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 37 MiB/s wr, 1.07k op/s 2024-09-11T08:42:48.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:48 smithi092 ceph-mon[41114]: 5.14 scrub starts 2024-09-11T08:42:48.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:48 smithi092 ceph-mon[41114]: 5.14 scrub ok 2024-09-11T08:42:48.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:48 smithi092 ceph-mon[41114]: pgmap v150: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 7.4 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 37 MiB/s wr, 1.07k op/s 2024-09-11T08:42:48.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:48.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:48.375 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:48.376 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:48.477 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:48.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:48.507 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:48.507 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:48.608 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:48.635 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:48.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:48.636 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:48.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:48 smithi145 ceph-mon[33384]: 5.14 scrub starts 2024-09-11T08:42:48.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:48 smithi145 ceph-mon[33384]: 5.14 scrub ok 2024-09-11T08:42:48.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:48 smithi145 ceph-mon[33384]: pgmap v150: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 7.4 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 37 MiB/s wr, 1.07k op/s 2024-09-11T08:42:48.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:48.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:42:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:48.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:48.784 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:48.885 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:48.913 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:48.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:48.914 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:49.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:49.060 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:49.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:49.061 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:49.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:49.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:49.201 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:49.201 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:49.302 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:49.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:49.336 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:49.336 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:49.437 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:49.462 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:49.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:49.463 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:49.559 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[29647]: 4.14 scrub starts 2024-09-11T08:42:49.559 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[29647]: 4.14 scrub ok 2024-09-11T08:42:49.559 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:42:49.559 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:42:49.559 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[29647]: 7.5 scrub ok 2024-09-11T08:42:49.559 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[29647]: osdmap e112: 8 total, 8 up, 8 in 2024-09-11T08:42:49.559 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:42:49.559 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[29647]: 3.1a scrub starts 2024-09-11T08:42:49.559 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[29647]: 3.1a scrub ok 2024-09-11T08:42:49.560 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[41114]: 4.14 scrub starts 2024-09-11T08:42:49.560 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[41114]: 4.14 scrub ok 2024-09-11T08:42:49.560 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:42:49.560 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:42:49.560 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[41114]: 7.5 scrub ok 2024-09-11T08:42:49.560 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[41114]: osdmap e112: 8 total, 8 up, 8 in 2024-09-11T08:42:49.560 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:42:49.560 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[41114]: 3.1a scrub starts 2024-09-11T08:42:49.560 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:49 smithi092 ceph-mon[41114]: 3.1a scrub ok 2024-09-11T08:42:49.563 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:49.589 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:49.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:49.590 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:49 smithi145 ceph-mon[33384]: 4.14 scrub starts 2024-09-11T08:42:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:49 smithi145 ceph-mon[33384]: 4.14 scrub ok 2024-09-11T08:42:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:49 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:42:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:49 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:42:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:49 smithi145 ceph-mon[33384]: 7.5 scrub ok 2024-09-11T08:42:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:49 smithi145 ceph-mon[33384]: osdmap e112: 8 total, 8 up, 8 in 2024-09-11T08:42:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:49 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:42:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:49 smithi145 ceph-mon[33384]: 3.1a scrub starts 2024-09-11T08:42:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:49 smithi145 ceph-mon[33384]: 3.1a scrub ok 2024-09-11T08:42:49.691 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:49.732 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:49.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:49.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:49.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:49.870 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:49.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:49.871 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:49.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:50.009 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:50.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:50.010 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:50.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:50.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:50.161 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:50.161 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:50.262 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:50.308 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:50.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:50.309 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:50.389 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[29647]: pgmap v153: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 7.4 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 901 B/s rd, 4.4 MiB/s wr, 706 op/s 2024-09-11T08:42:50.389 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:42:50.389 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:42:50.390 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[29647]: 2.1 scrub starts 2024-09-11T08:42:50.390 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[29647]: 2.1 scrub ok 2024-09-11T08:42:50.390 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:50.390 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[29647]: 3.11 deep-scrub starts 2024-09-11T08:42:50.390 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[29647]: 3.11 deep-scrub ok 2024-09-11T08:42:50.390 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1471719464' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T08:42:50.390 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T08:42:50.390 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[29647]: 6.e scrub starts 2024-09-11T08:42:50.390 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[29647]: 6.e scrub ok 2024-09-11T08:42:50.390 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[41114]: pgmap v153: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 7.4 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 901 B/s rd, 4.4 MiB/s wr, 706 op/s 2024-09-11T08:42:50.390 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:42:50.391 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:42:50.391 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[41114]: 2.1 scrub starts 2024-09-11T08:42:50.391 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[41114]: 2.1 scrub ok 2024-09-11T08:42:50.391 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:50.391 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[41114]: 3.11 deep-scrub starts 2024-09-11T08:42:50.391 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[41114]: 3.11 deep-scrub ok 2024-09-11T08:42:50.391 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1471719464' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T08:42:50.391 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T08:42:50.391 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[41114]: 6.e scrub starts 2024-09-11T08:42:50.391 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:50 smithi092 ceph-mon[41114]: 6.e scrub ok 2024-09-11T08:42:50.410 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:50.433 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:50.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:50.434 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:50.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:50.574 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:50.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:50.575 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:50.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:50 smithi145 ceph-mon[33384]: pgmap v153: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 7.4 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 901 B/s rd, 4.4 MiB/s wr, 706 op/s 2024-09-11T08:42:50.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:50 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:42:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:50 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:42:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:50 smithi145 ceph-mon[33384]: 2.1 scrub starts 2024-09-11T08:42:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:50 smithi145 ceph-mon[33384]: 2.1 scrub ok 2024-09-11T08:42:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:50 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:42:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:50 smithi145 ceph-mon[33384]: 3.11 deep-scrub starts 2024-09-11T08:42:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:50 smithi145 ceph-mon[33384]: 3.11 deep-scrub ok 2024-09-11T08:42:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:50 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1471719464' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T08:42:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:50 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T08:42:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:50 smithi145 ceph-mon[33384]: 6.e scrub starts 2024-09-11T08:42:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:50 smithi145 ceph-mon[33384]: 6.e scrub ok 2024-09-11T08:42:50.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:50.710 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:50.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:50.711 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:50.752 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.2 to 1 (10000) 2024-09-11T08:42:50.811 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:50.839 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:50.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:50.840 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:50.941 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:50.966 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:50.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:50.967 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:51.068 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:51.094 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:51.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:51.095 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:51.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:51.234 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:51.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:51.235 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:51.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[29647]: 5.11 scrub starts 2024-09-11T08:42:51.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[29647]: 5.11 scrub ok 2024-09-11T08:42:51.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[29647]: 6.6 scrub starts 2024-09-11T08:42:51.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[29647]: 6.6 scrub ok 2024-09-11T08:42:51.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T08:42:51.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[29647]: 4.1c scrub ok 2024-09-11T08:42:51.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T08:42:51.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[29647]: osdmap e113: 8 total, 8 up, 8 in 2024-09-11T08:42:51.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[29647]: 6.12 scrub starts 2024-09-11T08:42:51.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[29647]: 6.12 scrub ok 2024-09-11T08:42:51.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[41114]: 5.11 scrub starts 2024-09-11T08:42:51.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[41114]: 5.11 scrub ok 2024-09-11T08:42:51.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[41114]: 6.6 scrub starts 2024-09-11T08:42:51.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[41114]: 6.6 scrub ok 2024-09-11T08:42:51.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T08:42:51.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[41114]: 4.1c scrub ok 2024-09-11T08:42:51.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T08:42:51.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[41114]: osdmap e113: 8 total, 8 up, 8 in 2024-09-11T08:42:51.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[41114]: 6.12 scrub starts 2024-09-11T08:42:51.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-mon[41114]: 6.12 scrub ok 2024-09-11T08:42:51.336 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:51.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:51.385 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:51.385 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:51.486 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:51.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:51.531 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:51.531 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:51.632 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:51.658 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:51.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:51.659 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:51.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:51 smithi145 ceph-mon[33384]: 5.11 scrub starts 2024-09-11T08:42:51.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:51 smithi145 ceph-mon[33384]: 5.11 scrub ok 2024-09-11T08:42:51.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:51 smithi145 ceph-mon[33384]: 6.6 scrub starts 2024-09-11T08:42:51.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:51 smithi145 ceph-mon[33384]: 6.6 scrub ok 2024-09-11T08:42:51.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:51 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T08:42:51.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:51 smithi145 ceph-mon[33384]: 4.1c scrub ok 2024-09-11T08:42:51.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:51 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T08:42:51.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:51 smithi145 ceph-mon[33384]: osdmap e113: 8 total, 8 up, 8 in 2024-09-11T08:42:51.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:51 smithi145 ceph-mon[33384]: 6.12 scrub starts 2024-09-11T08:42:51.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:51 smithi145 ceph-mon[33384]: 6.12 scrub ok 2024-09-11T08:42:51.761 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:51.788 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:51.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:51.789 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:51.797 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:42:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:42:51] "GET /metrics HTTP/1.1" 200 223668 "" "Prometheus/2.43.0" 2024-09-11T08:42:51.821 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:42:51.821 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:42:51.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:51.934 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:51.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:51.935 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:52.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:52.081 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:52.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:52.082 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:52.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:52.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:52.219 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:52.220 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:52.321 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:52.345 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:52.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:52.346 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:52.446 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:52.474 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:52.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:52.475 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:52.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:52 smithi092 ceph-mon[29647]: 5.f scrub starts 2024-09-11T08:42:52.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:52 smithi092 ceph-mon[29647]: 5.f scrub ok 2024-09-11T08:42:52.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:52 smithi092 ceph-mon[29647]: 7.4 scrub starts 2024-09-11T08:42:52.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:52 smithi092 ceph-mon[29647]: pgmap v155: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.8 KiB/s rd, 5.0 MiB/s wr, 1.09k op/s 2024-09-11T08:42:52.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:52 smithi092 ceph-mon[29647]: 3.d scrub starts 2024-09-11T08:42:52.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:52 smithi092 ceph-mon[29647]: 3.d scrub ok 2024-09-11T08:42:52.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:52 smithi092 ceph-mon[41114]: 5.f scrub starts 2024-09-11T08:42:52.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:52 smithi092 ceph-mon[41114]: 5.f scrub ok 2024-09-11T08:42:52.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:52 smithi092 ceph-mon[41114]: 7.4 scrub starts 2024-09-11T08:42:52.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:52 smithi092 ceph-mon[41114]: pgmap v155: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.8 KiB/s rd, 5.0 MiB/s wr, 1.09k op/s 2024-09-11T08:42:52.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:52 smithi092 ceph-mon[41114]: 3.d scrub starts 2024-09-11T08:42:52.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:52 smithi092 ceph-mon[41114]: 3.d scrub ok 2024-09-11T08:42:52.575 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:52.599 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:52.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:52.600 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:52 smithi145 ceph-mon[33384]: 5.f scrub starts 2024-09-11T08:42:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:52 smithi145 ceph-mon[33384]: 5.f scrub ok 2024-09-11T08:42:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:52 smithi145 ceph-mon[33384]: 7.4 scrub starts 2024-09-11T08:42:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:52 smithi145 ceph-mon[33384]: pgmap v155: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.8 KiB/s rd, 5.0 MiB/s wr, 1.09k op/s 2024-09-11T08:42:52.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:52 smithi145 ceph-mon[33384]: 3.d scrub starts 2024-09-11T08:42:52.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:52 smithi145 ceph-mon[33384]: 3.d scrub ok 2024-09-11T08:42:52.701 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:52.724 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-11T08:42:52.729 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:52.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:52.730 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:52.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:52.867 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:52.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:52.868 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:52.969 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:53.001 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:53.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:53.002 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:53.059 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:42:53.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:53.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:53.137 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:53.138 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:53.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:42:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:42:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:53.239 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:53.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 3 1 2024-09-11T08:42:53.263 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:53.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:53.264 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:53.365 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:53.411 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:53.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:53.412 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:53.438 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:53 smithi145 ceph-mon[33384]: 3.a scrub starts 2024-09-11T08:42:53.438 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:53 smithi145 ceph-mon[33384]: 3.a scrub ok 2024-09-11T08:42:53.501 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:53 smithi092 ceph-mon[29647]: 3.a scrub starts 2024-09-11T08:42:53.501 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:53 smithi092 ceph-mon[29647]: 3.a scrub ok 2024-09-11T08:42:53.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:53 smithi092 ceph-mon[41114]: 3.a scrub starts 2024-09-11T08:42:53.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:53 smithi092 ceph-mon[41114]: 3.a scrub ok 2024-09-11T08:42:53.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:53.565 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:53.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:53.570 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:53.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:53.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:53.732 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:53.732 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:53.833 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:53.859 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:53.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:53.860 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:53.961 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:53.982 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:53.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:53.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:54.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:54.118 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:54.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:54.119 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:54.220 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:54.246 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:54.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:54.247 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:54.269 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:54.347 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:54.350 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:42:54.372 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:54.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:54.373 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:54.473 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:54.495 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:54.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:54.496 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:54.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[29647]: 3.1f scrub starts 2024-09-11T08:42:54.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[29647]: 3.1f scrub ok 2024-09-11T08:42:54.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[29647]: pgmap v156: 148 pgs: 1 active+clean+scrubbing, 1 active+clean+scrubbing+deep, 146 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 790 op/s 2024-09-11T08:42:54.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[29647]: 5.9 scrub starts 2024-09-11T08:42:54.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[29647]: 5.9 scrub ok 2024-09-11T08:42:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[29647]: 3.14 scrub starts 2024-09-11T08:42:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[29647]: 3.14 scrub ok 2024-09-11T08:42:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3527493846' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:42:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[41114]: 3.1f scrub starts 2024-09-11T08:42:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[41114]: 3.1f scrub ok 2024-09-11T08:42:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[41114]: pgmap v156: 148 pgs: 1 active+clean+scrubbing, 1 active+clean+scrubbing+deep, 146 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 790 op/s 2024-09-11T08:42:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[41114]: 5.9 scrub starts 2024-09-11T08:42:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[41114]: 5.9 scrub ok 2024-09-11T08:42:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[41114]: 3.14 scrub starts 2024-09-11T08:42:54.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[41114]: 3.14 scrub ok 2024-09-11T08:42:54.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:54 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3527493846' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:42:54.596 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:54.620 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:54.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:54.620 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:54.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:54 smithi145 ceph-mon[33384]: 3.1f scrub starts 2024-09-11T08:42:54.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:54 smithi145 ceph-mon[33384]: 3.1f scrub ok 2024-09-11T08:42:54.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:54 smithi145 ceph-mon[33384]: pgmap v156: 148 pgs: 1 active+clean+scrubbing, 1 active+clean+scrubbing+deep, 146 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 790 op/s 2024-09-11T08:42:54.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:54 smithi145 ceph-mon[33384]: 5.9 scrub starts 2024-09-11T08:42:54.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:54 smithi145 ceph-mon[33384]: 5.9 scrub ok 2024-09-11T08:42:54.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:54 smithi145 ceph-mon[33384]: 3.14 scrub starts 2024-09-11T08:42:54.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:54 smithi145 ceph-mon[33384]: 3.14 scrub ok 2024-09-11T08:42:54.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:54 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3527493846' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:42:54.722 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:54.743 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:54.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:54.744 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:54.844 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:54.871 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:54.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:54.872 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:54.973 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:55.006 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:55.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:55.007 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:55.108 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:55.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:55.137 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:55.137 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:55.238 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:55.266 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:55.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:55.267 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:55.368 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:55.394 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:55.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:55.395 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:55.486 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:42:55.486 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:42:55.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:55.538 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:55.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:55.539 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:55.640 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:55.658 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:55 smithi092 ceph-mon[29647]: 7.4 scrub ok 2024-09-11T08:42:55.658 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:55 smithi092 ceph-mon[29647]: 5.17 scrub starts 2024-09-11T08:42:55.658 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:55 smithi092 ceph-mon[29647]: 5.17 scrub ok 2024-09-11T08:42:55.658 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:55 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:42:55.658 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:55 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3527493846' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:42:55.658 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:55 smithi092 ceph-mon[29647]: osdmap e114: 8 total, 8 up, 8 in 2024-09-11T08:42:55.658 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:55 smithi092 ceph-mon[41114]: 7.4 scrub ok 2024-09-11T08:42:55.659 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:55 smithi092 ceph-mon[41114]: 5.17 scrub starts 2024-09-11T08:42:55.659 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:55 smithi092 ceph-mon[41114]: 5.17 scrub ok 2024-09-11T08:42:55.659 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:55 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:42:55.659 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:55 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3527493846' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:42:55.659 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:55 smithi092 ceph-mon[41114]: osdmap e114: 8 total, 8 up, 8 in 2024-09-11T08:42:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:55 smithi145 ceph-mon[33384]: 7.4 scrub ok 2024-09-11T08:42:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:55 smithi145 ceph-mon[33384]: 5.17 scrub starts 2024-09-11T08:42:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:55 smithi145 ceph-mon[33384]: 5.17 scrub ok 2024-09-11T08:42:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:55 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:42:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:55 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3527493846' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:42:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:55 smithi145 ceph-mon[33384]: osdmap e114: 8 total, 8 up, 8 in 2024-09-11T08:42:55.677 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:55.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:55.678 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:55.779 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:55.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:55.803 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:55.803 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:55.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:55.940 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:55.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:55.941 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:56.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:56.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:56.073 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:56.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:56.214 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:56.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:56.215 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:56.315 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:56.343 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:56.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:56.344 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:56.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:56 smithi145 ceph-mon[33384]: pgmap v158: 148 pgs: 1 active+clean+scrubbing, 1 active+clean+scrubbing+deep, 146 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 756 B/s rd, 0 B/s wr, 701 op/s 2024-09-11T08:42:56.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:42:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:42:56.445 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:56.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:56.473 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:56.474 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:56.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:56 smithi092 ceph-mon[29647]: pgmap v158: 148 pgs: 1 active+clean+scrubbing, 1 active+clean+scrubbing+deep, 146 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 756 B/s rd, 0 B/s wr, 701 op/s 2024-09-11T08:42:56.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:56 smithi092 ceph-mon[41114]: pgmap v158: 148 pgs: 1 active+clean+scrubbing, 1 active+clean+scrubbing+deep, 146 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 756 B/s rd, 0 B/s wr, 701 op/s 2024-09-11T08:42:56.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:56.591 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-11T08:42:56.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:56.617 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:56.618 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:56.718 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:56.748 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:56.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:56.749 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:56.850 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:56.886 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:56.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:56.887 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:56.919 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:42:56.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:57.025 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:57.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:57.026 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:57.127 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:57.132 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:57.152 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:57.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:57.153 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:57.254 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:57.291 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:57.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:57.292 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:57.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:57.431 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:57.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:57.432 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:57.532 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:57.557 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:57.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:57.558 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:57.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:57 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:57.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:57 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:57.659 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:57.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:57 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:57.691 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:57.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:57.692 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:57.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:42:57.792 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:57.817 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:57.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:57.818 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:57.919 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:57.942 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:57.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:57.943 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:58.044 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:58.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:58.103 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:58.103 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:58.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:58.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:58.243 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:58.243 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:58.344 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:58.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:58.370 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:58.370 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:58.470 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:42:58.495 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:58.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:58.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:42:58.503 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:42:58 smithi145 ceph-mon[33384]: pgmap v159: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-11T08:42:58.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:58.639 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:58.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:58.640 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:58.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:42:58.777 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:58.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:58.778 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:42:58.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:42:58 smithi092 ceph-mon[41114]: pgmap v159: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-11T08:42:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:42:58 smithi092 ceph-mon[29647]: pgmap v159: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-11T08:42:58.879 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:58.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:58.907 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:58.907 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:58.926 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:42:58.926 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:42:59.008 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:59.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:59.041 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:59.041 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:59.142 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:42:59.175 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:59.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:59.176 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:42:59.277 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:42:59.305 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:59.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:59.306 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:42:59.406 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:42:59.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:59.450 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:59.450 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:42:59.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:42:59.558 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:42:59.558 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:42:59.591 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:59.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:59.592 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:42:59.594 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:42:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:42:59.626 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:42:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:42:59.660 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:42:59.661 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:42:59.691 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:42:59.691 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:42:59.693 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:42:59.725 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:42:59.725 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:42:59.727 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:42:59.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:59.729 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:42:59.757 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:42:59.757 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:42:59.764 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-11T08:42:59.786 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:42:59.790 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:42:59.791 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:42:59.829 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:42:59.872 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:42:59.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:42:59.873 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:42:59.974 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:00.026 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:00.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:00.027 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:00.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:00.162 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:43:00.165 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:00.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:00.166 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:00.267 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:00.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:00.291 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:00.292 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:00.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:00.426 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:00.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:00.427 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:00.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:00.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:00 smithi092 ceph-mon[29647]: pgmap v160: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.11k op/s 2024-09-11T08:43:00.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:00 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2417332497' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:00.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:00 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:00.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:00 smithi092 ceph-mon[41114]: pgmap v160: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.11k op/s 2024-09-11T08:43:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:00 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2417332497' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:00 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:00.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:00.567 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:00.567 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:00.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:00 smithi145 ceph-mon[33384]: pgmap v160: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.11k op/s 2024-09-11T08:43:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:00 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2417332497' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:00 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:00.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:00.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:00.701 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:00.802 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:00.827 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:00.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:00.829 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:00.929 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:00.969 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:00.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:00.970 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:01.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:01.112 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:01.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:01.113 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:01.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:01.250 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:01.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:01.251 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:01.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:01.381 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T08:43:01.397 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:01.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:01.398 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:01.499 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:01.529 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:01.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:01.530 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:01.631 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:01.658 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:01.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:01.659 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:01.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:01 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1658638458' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T08:43:01.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:01 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T08:43:01.760 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:01.792 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:01.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:01.793 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:01.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:01 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1658638458' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T08:43:01.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:01 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T08:43:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:01 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1658638458' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T08:43:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:01 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T08:43:01.816 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:43:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:43:01] "GET /metrics HTTP/1.1" 200 223755 "" "Prometheus/2.43.0" 2024-09-11T08:43:01.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:01.932 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:01.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:01.933 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:02.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:02.056 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:02.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:02.057 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:02.158 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:02.182 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:02.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:02.183 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:02.284 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:02.309 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:02.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:02.310 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:02.411 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:02.439 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:02.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:02.440 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:02.541 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:02.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:02 smithi092 ceph-mon[29647]: pgmap v161: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.7 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 926 op/s 2024-09-11T08:43:02.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:02 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T08:43:02.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:02 smithi092 ceph-mon[29647]: osdmap e115: 8 total, 8 up, 8 in 2024-09-11T08:43:02.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:02 smithi092 ceph-mon[41114]: pgmap v161: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.7 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 926 op/s 2024-09-11T08:43:02.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:02 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T08:43:02.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:02 smithi092 ceph-mon[41114]: osdmap e115: 8 total, 8 up, 8 in 2024-09-11T08:43:02.577 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:02.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:02.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:02 smithi145 ceph-mon[33384]: pgmap v161: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.7 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 926 op/s 2024-09-11T08:43:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:02 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T08:43:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:02 smithi145 ceph-mon[33384]: osdmap e115: 8 total, 8 up, 8 in 2024-09-11T08:43:02.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:02.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:02.721 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:02.721 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:02.822 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:02.856 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:02.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:02.857 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:02.958 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:02.985 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:43:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:43:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:02.985 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:02.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:02.986 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:03.087 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:03.121 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:03.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:03.122 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:03.222 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:03.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:03.259 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:03.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 4 1 2024-09-11T08:43:03.361 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:03.378 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:43:03.378 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T08:43:03.378 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T08:43:03.378 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:43:03.378 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.378 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:58487 7.1 7:8ef209d6:::benchmark_data_smithi092_84565_object58486:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-11T08:43:03.378 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:40.083328+0000", 2024-09-11T08:43:03.378 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 83.291776420999994, 2024-09-11T08:43:03.378 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.034294940000000003, 2024-09-11T08:43:03.378 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.378 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.379 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.379 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.379 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.379 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 58487 2024-09-11T08:43:03.379 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.379 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.379 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.379 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.379 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:40.083328+0000", 2024-09-11T08:43:03.379 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.379 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.379 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.379 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.379 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:40.083328+0000", 2024-09-11T08:43:03.380 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.380 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.380 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.380 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.380 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:40.083327+0000", 2024-09-11T08:43:03.380 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:43:03.380 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.380 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.380 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.380 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:40.083396+0000", 2024-09-11T08:43:03.380 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9227000000000007e-05 2024-09-11T08:43:03.380 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.380 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.380 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.381 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:40.083414+0000", 2024-09-11T08:43:03.381 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7819999999999999e-05 2024-09-11T08:43:03.381 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.381 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.381 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.381 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:40.083420+0000", 2024-09-11T08:43:03.381 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2589999999999997e-06 2024-09-11T08:43:03.381 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.381 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.381 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.381 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:40.083462+0000", 2024-09-11T08:43:03.381 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2398999999999998e-05 2024-09-11T08:43:03.381 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.381 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.382 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.382 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:40.083581+0000", 2024-09-11T08:43:03.382 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000118423 2024-09-11T08:43:03.382 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.382 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.382 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.382 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:40.083711+0000", 2024-09-11T08:43:03.382 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000130316 2024-09-11T08:43:03.382 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.382 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.382 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.382 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:40.085604+0000", 2024-09-11T08:43:03.383 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0018928090000000001 2024-09-11T08:43:03.383 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.383 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.383 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.383 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:40.085641+0000", 2024-09-11T08:43:03.383 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6678999999999999e-05 2024-09-11T08:43:03.383 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.383 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.383 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.383 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:40.117522+0000", 2024-09-11T08:43:03.383 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.031880959 2024-09-11T08:43:03.383 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.384 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.384 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.384 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:40.117580+0000", 2024-09-11T08:43:03.384 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8829999999999997e-05 2024-09-11T08:43:03.384 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.384 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.384 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.384 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:40.117623+0000", 2024-09-11T08:43:03.384 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2913000000000001e-05 2024-09-11T08:43:03.384 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.385 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.385 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.385 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.385 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.385 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:63550 7.1 7:839dabea:::benchmark_data_smithi092_84565_object63549:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:43:03.385 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:43.960658+0000", 2024-09-11T08:43:03.385 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 79.414446416000004, 2024-09-11T08:43:03.385 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.090364671999999993, 2024-09-11T08:43:03.385 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.385 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.385 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.385 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.385 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.385 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 63550 2024-09-11T08:43:03.385 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.386 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.386 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.386 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.386 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960658+0000", 2024-09-11T08:43:03.386 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.386 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.386 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.386 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.386 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960658+0000", 2024-09-11T08:43:03.386 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.386 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.386 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.386 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.386 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960657+0000", 2024-09-11T08:43:03.387 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:43:03.387 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.387 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.387 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.387 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960724+0000", 2024-09-11T08:43:03.387 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6901000000000002e-05 2024-09-11T08:43:03.387 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.387 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.387 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.387 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960734+0000", 2024-09-11T08:43:03.387 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0206e-05 2024-09-11T08:43:03.387 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.387 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.387 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.387 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960738+0000", 2024-09-11T08:43:03.388 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8500000000000004e-06 2024-09-11T08:43:03.388 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.388 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.388 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.388 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960767+0000", 2024-09-11T08:43:03.388 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.8841999999999999e-05 2024-09-11T08:43:03.388 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.388 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.388 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.388 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960844+0000", 2024-09-11T08:43:03.388 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7350999999999999e-05 2024-09-11T08:43:03.388 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.389 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.389 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.389 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960925+0000", 2024-09-11T08:43:03.389 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.0576999999999998e-05 2024-09-11T08:43:03.389 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.389 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.389 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.389 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963102+0000", 2024-09-11T08:43:03.389 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0021772459999999999 2024-09-11T08:43:03.389 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.389 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.389 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.389 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963265+0000", 2024-09-11T08:43:03.390 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00016236399999999999 2024-09-11T08:43:03.390 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.390 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.390 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.390 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:44.050915+0000", 2024-09-11T08:43:03.390 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.087650790000000006 2024-09-11T08:43:03.390 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.390 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.390 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.390 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:44.050982+0000", 2024-09-11T08:43:03.390 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6919000000000001e-05 2024-09-11T08:43:03.390 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.390 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.390 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.391 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:44.051023+0000", 2024-09-11T08:43:03.391 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0818000000000001e-05 2024-09-11T08:43:03.391 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.391 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.391 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.391 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.391 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.391 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:63554 7.1 7:824a3d8d:::benchmark_data_smithi092_84565_object63553:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:43:03.391 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:43.963333+0000", 2024-09-11T08:43:03.391 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 79.411772108999997, 2024-09-11T08:43:03.391 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.087833008000000004, 2024-09-11T08:43:03.391 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.391 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.391 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.392 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.392 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.392 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 63554 2024-09-11T08:43:03.392 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.392 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.392 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.392 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.392 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963333+0000", 2024-09-11T08:43:03.392 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.392 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.392 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.392 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.392 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963333+0000", 2024-09-11T08:43:03.393 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.393 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.393 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.393 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.393 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963332+0000", 2024-09-11T08:43:03.393 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:43:03.393 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.393 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.393 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.393 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963380+0000", 2024-09-11T08:43:03.393 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8263999999999997e-05 2024-09-11T08:43:03.393 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.393 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.394 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.394 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963395+0000", 2024-09-11T08:43:03.394 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4732999999999999e-05 2024-09-11T08:43:03.394 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.394 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.394 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.394 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963410+0000", 2024-09-11T08:43:03.394 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5902000000000001e-05 2024-09-11T08:43:03.394 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.394 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.394 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.394 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963458+0000", 2024-09-11T08:43:03.394 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7958999999999999e-05 2024-09-11T08:43:03.394 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.394 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963595+0000", 2024-09-11T08:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013652799999999999 2024-09-11T08:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963727+0000", 2024-09-11T08:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000132127 2024-09-11T08:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.965814+0000", 2024-09-11T08:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0020865129999999999 2024-09-11T08:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.396 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.396 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.966175+0000", 2024-09-11T08:43:03.396 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00036101699999999999 2024-09-11T08:43:03.396 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.396 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.396 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.396 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:44.051074+0000", 2024-09-11T08:43:03.396 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.084899530000000001 2024-09-11T08:43:03.396 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.396 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.396 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.396 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:44.051132+0000", 2024-09-11T08:43:03.397 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7462e-05 2024-09-11T08:43:03.397 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.397 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.397 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.397 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:44.051166+0000", 2024-09-11T08:43:03.397 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4175999999999999e-05 2024-09-11T08:43:03.397 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.397 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.397 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.397 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.397 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.397 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:65855 7.1 7:8319a3f3:::benchmark_data_smithi092_84565_object65854:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:43:03.397 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:45.795970+0000", 2024-09-11T08:43:03.398 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 77.579135238000006, 2024-09-11T08:43:03.398 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.075141500999999999, 2024-09-11T08:43:03.398 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.398 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.398 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.398 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.398 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.398 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 65855 2024-09-11T08:43:03.398 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.398 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.398 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.398 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.399 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.795970+0000", 2024-09-11T08:43:03.399 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.399 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.399 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.399 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.399 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.795970+0000", 2024-09-11T08:43:03.399 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.399 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.399 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.399 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.399 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.795968+0000", 2024-09-11T08:43:03.399 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:43:03.399 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.399 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.400 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.400 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796007+0000", 2024-09-11T08:43:03.400 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8816999999999998e-05 2024-09-11T08:43:03.400 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.400 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.400 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.400 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796019+0000", 2024-09-11T08:43:03.400 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1541e-05 2024-09-11T08:43:03.400 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.400 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.400 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.400 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796025+0000", 2024-09-11T08:43:03.401 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8749999999999997e-06 2024-09-11T08:43:03.401 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.401 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.401 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.401 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796071+0000", 2024-09-11T08:43:03.401 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6728999999999999e-05 2024-09-11T08:43:03.401 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.401 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.401 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.401 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796165+0000", 2024-09-11T08:43:03.401 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.4105999999999997e-05 2024-09-11T08:43:03.401 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.401 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.401 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.402 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796293+0000", 2024-09-11T08:43:03.402 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012787000000000001 2024-09-11T08:43:03.402 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.402 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.402 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.402 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.803126+0000", 2024-09-11T08:43:03.402 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0068326630000000001 2024-09-11T08:43:03.402 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.402 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.402 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.402 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.803182+0000", 2024-09-11T08:43:03.402 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6140000000000001e-05 2024-09-11T08:43:03.402 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.402 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.402 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.403 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.870909+0000", 2024-09-11T08:43:03.403 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.06772649 2024-09-11T08:43:03.403 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.403 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.403 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.403 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.871071+0000", 2024-09-11T08:43:03.403 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00016238400000000001 2024-09-11T08:43:03.403 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.403 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.403 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.403 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.871111+0000", 2024-09-11T08:43:03.403 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0237000000000001e-05 2024-09-11T08:43:03.403 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.403 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.403 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.404 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.404 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.404 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:65857 7.1 7:8929e230:::benchmark_data_smithi092_84565_object65856:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:43:03.404 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:45.796146+0000", 2024-09-11T08:43:03.404 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 77.578959071, 2024-09-11T08:43:03.404 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.075128144999999993, 2024-09-11T08:43:03.404 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.404 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.404 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.404 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.404 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.404 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 65857 2024-09-11T08:43:03.404 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.404 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.405 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.405 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.405 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796146+0000", 2024-09-11T08:43:03.405 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.405 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.405 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.405 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.405 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796146+0000", 2024-09-11T08:43:03.405 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.405 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.405 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.405 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.406 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796145+0000", 2024-09-11T08:43:03.406 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:43:03.406 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.406 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.406 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.406 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796222+0000", 2024-09-11T08:43:03.406 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7466999999999995e-05 2024-09-11T08:43:03.406 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.406 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.406 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.406 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796231+0000", 2024-09-11T08:43:03.407 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2269999999999993e-06 2024-09-11T08:43:03.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.407 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.407 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.407 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796235+0000", 2024-09-11T08:43:03.407 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2640000000000002e-06 2024-09-11T08:43:03.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.407 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.407 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.407 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.798402+0000", 2024-09-11T08:43:03.407 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.002166992 2024-09-11T08:43:03.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.407 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.407 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.407 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.798510+0000", 2024-09-11T08:43:03.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000107765 2024-09-11T08:43:03.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.798635+0000", 2024-09-11T08:43:03.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012517600000000001 2024-09-11T08:43:03.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.807303+0000", 2024-09-11T08:43:03.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0086681359999999999 2024-09-11T08:43:03.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.409 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.807353+0000", 2024-09-11T08:43:03.409 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0176999999999997e-05 2024-09-11T08:43:03.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.409 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.871167+0000", 2024-09-11T08:43:03.409 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.063814256 2024-09-11T08:43:03.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.409 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.871246+0000", 2024-09-11T08:43:03.409 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.8881000000000006e-05 2024-09-11T08:43:03.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.410 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.871274+0000", 2024-09-11T08:43:03.410 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7671999999999999e-05 2024-09-11T08:43:03.410 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.410 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.410 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.410 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.410 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.410 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:65858 7.1 7:8b3fee21:::benchmark_data_smithi092_84565_object65857:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:43:03.410 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:45.796253+0000", 2024-09-11T08:43:03.410 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 77.57885229, 2024-09-11T08:43:03.410 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.075149819000000007, 2024-09-11T08:43:03.410 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.410 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.411 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.411 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.411 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.411 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 65858 2024-09-11T08:43:03.411 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.411 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.411 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.411 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.411 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796253+0000", 2024-09-11T08:43:03.411 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.411 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.411 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.412 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.412 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796253+0000", 2024-09-11T08:43:03.412 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.412 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.412 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.412 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.412 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796252+0000", 2024-09-11T08:43:03.412 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:43:03.412 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.412 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.412 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.412 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796279+0000", 2024-09-11T08:43:03.412 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.6812999999999999e-05 2024-09-11T08:43:03.412 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.413 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.413 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.413 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796285+0000", 2024-09-11T08:43:03.413 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0399999999999998e-06 2024-09-11T08:43:03.413 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.413 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.413 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.413 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.796288+0000", 2024-09-11T08:43:03.413 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4180000000000001e-06 2024-09-11T08:43:03.413 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.413 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.413 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.413 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.800843+0000", 2024-09-11T08:43:03.413 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0045542220000000001 2024-09-11T08:43:03.413 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.414 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.414 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.414 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.800948+0000", 2024-09-11T08:43:03.414 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010540900000000001 2024-09-11T08:43:03.414 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.414 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.414 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.414 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.801092+0000", 2024-09-11T08:43:03.414 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000143843 2024-09-11T08:43:03.414 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.414 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.414 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.414 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.807406+0000", 2024-09-11T08:43:03.414 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0063143299999999999 2024-09-11T08:43:03.414 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.415 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.415 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.415 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.807411+0000", 2024-09-11T08:43:03.415 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5070000000000002e-06 2024-09-11T08:43:03.415 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.415 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.415 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.415 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.871320+0000", 2024-09-11T08:43:03.415 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.063909571999999998 2024-09-11T08:43:03.415 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.415 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.415 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.415 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.871366+0000", 2024-09-11T08:43:03.415 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5741999999999997e-05 2024-09-11T08:43:03.415 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.416 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.416 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.416 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.871402+0000", 2024-09-11T08:43:03.416 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6418999999999998e-05 2024-09-11T08:43:03.416 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.416 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.416 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.416 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.416 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.416 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:65864 7.1 7:89e5d122:::benchmark_data_smithi092_84565_object65863:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:43:03.416 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:45.799575+0000", 2024-09-11T08:43:03.416 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 77.575529560000007, 2024-09-11T08:43:03.416 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.071938929999999998, 2024-09-11T08:43:03.416 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.416 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.417 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.417 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.417 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.417 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 65864 2024-09-11T08:43:03.417 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.417 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.417 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.417 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.417 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.799575+0000", 2024-09-11T08:43:03.417 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.417 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.417 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.417 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.417 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.799575+0000", 2024-09-11T08:43:03.417 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.418 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.418 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.418 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.418 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.799574+0000", 2024-09-11T08:43:03.418 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:43:03.418 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.418 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.418 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.418 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.801559+0000", 2024-09-11T08:43:03.418 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0019852590000000001 2024-09-11T08:43:03.418 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.418 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.418 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.418 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.801574+0000", 2024-09-11T08:43:03.418 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5443000000000001e-05 2024-09-11T08:43:03.419 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.419 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.419 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.419 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.801580+0000", 2024-09-11T08:43:03.419 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6319999999999996e-06 2024-09-11T08:43:03.419 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.419 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.419 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.419 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.803208+0000", 2024-09-11T08:43:03.419 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0016278740000000001 2024-09-11T08:43:03.419 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.419 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.419 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.419 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.803304+0000", 2024-09-11T08:43:03.419 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.6167000000000003e-05 2024-09-11T08:43:03.420 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.420 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.420 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.420 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.803426+0000", 2024-09-11T08:43:03.420 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000121569 2024-09-11T08:43:03.420 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.420 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.420 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.420 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.807415+0000", 2024-09-11T08:43:03.420 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0039897090000000001 2024-09-11T08:43:03.420 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.420 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.420 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.420 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.807439+0000", 2024-09-11T08:43:03.420 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3385000000000001e-05 2024-09-11T08:43:03.421 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.421 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.421 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.421 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.871436+0000", 2024-09-11T08:43:03.421 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.063997338000000001 2024-09-11T08:43:03.421 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.421 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.421 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.421 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.871480+0000", 2024-09-11T08:43:03.421 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3809e-05 2024-09-11T08:43:03.421 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.421 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.421 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.421 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.871514+0000", 2024-09-11T08:43:03.422 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4276000000000002e-05 2024-09-11T08:43:03.422 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.422 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.422 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.422 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.422 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.422 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:65866 7.1 7:83762ad7:::benchmark_data_smithi092_84565_object65865:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:43:03.422 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:45.802481+0000", 2024-09-11T08:43:03.422 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 77.572624083999997, 2024-09-11T08:43:03.422 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.074204000000000006, 2024-09-11T08:43:03.422 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.422 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.422 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.423 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.423 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.423 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 65866 2024-09-11T08:43:03.423 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.423 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.423 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.423 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.423 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.802481+0000", 2024-09-11T08:43:03.423 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.423 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.423 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.423 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.423 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.802481+0000", 2024-09-11T08:43:03.423 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.423 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.424 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.424 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.424 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.802479+0000", 2024-09-11T08:43:03.424 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:43:03.424 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.424 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.424 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.424 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.802539+0000", 2024-09-11T08:43:03.424 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9972000000000001e-05 2024-09-11T08:43:03.424 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.424 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.424 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.424 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.802550+0000", 2024-09-11T08:43:03.424 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1034e-05 2024-09-11T08:43:03.424 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.425 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.425 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.425 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.802555+0000", 2024-09-11T08:43:03.425 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9729999999999997e-06 2024-09-11T08:43:03.425 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.425 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.425 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.425 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.805358+0000", 2024-09-11T08:43:03.425 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.002802625 2024-09-11T08:43:03.425 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.425 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.425 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.425 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.805469+0000", 2024-09-11T08:43:03.425 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000111644 2024-09-11T08:43:03.425 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.805604+0000", 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013406099999999999 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.807443+0000", 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00183943 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.807554+0000", 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000111181 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.426 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.427 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.427 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.876559+0000", 2024-09-11T08:43:03.427 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.069004411000000002 2024-09-11T08:43:03.427 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.427 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.427 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.427 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.876641+0000", 2024-09-11T08:43:03.427 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2798000000000002e-05 2024-09-11T08:43:03.427 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.427 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.427 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.427 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.876685+0000", 2024-09-11T08:43:03.427 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3473000000000002e-05 2024-09-11T08:43:03.427 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.428 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.428 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.428 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:67206 7.1 7:8c8de506:::benchmark_data_smithi092_84565_object67205:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:43:03.428 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:47.018247+0000", 2024-09-11T08:43:03.428 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 76.356857413, 2024-09-11T08:43:03.428 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.05006476, 2024-09-11T08:43:03.428 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.428 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.428 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.428 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.428 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.428 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 67206 2024-09-11T08:43:03.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.018247+0000", 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.018247+0000", 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.018246+0000", 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:43:03.429 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.430 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.430 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.430 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.018292+0000", 2024-09-11T08:43:03.430 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6159999999999999e-05 2024-09-11T08:43:03.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.430 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.430 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.430 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.018307+0000", 2024-09-11T08:43:03.430 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4491000000000001e-05 2024-09-11T08:43:03.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.430 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.430 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.430 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.018313+0000", 2024-09-11T08:43:03.430 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0839999999999997e-06 2024-09-11T08:43:03.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.431 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.431 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.018340+0000", 2024-09-11T08:43:03.431 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7569999999999999e-05 2024-09-11T08:43:03.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.431 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.431 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.018425+0000", 2024-09-11T08:43:03.431 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.4866999999999999e-05 2024-09-11T08:43:03.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.431 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.431 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.018517+0000", 2024-09-11T08:43:03.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:03.432 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:03.433 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:03.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.2032000000000002e-05 2024-09-11T08:43:03.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.434 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.434 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.434 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.020467+0000", 2024-09-11T08:43:03.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001950177 2024-09-11T08:43:03.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.434 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.434 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.434 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.020909+0000", 2024-09-11T08:43:03.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00044175900000000001 2024-09-11T08:43:03.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.434 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.434 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.068219+0000", 2024-09-11T08:43:03.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.047309991000000003 2024-09-11T08:43:03.435 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.435 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.435 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.068274+0000", 2024-09-11T08:43:03.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4447000000000002e-05 2024-09-11T08:43:03.435 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.435 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.435 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.068312+0000", 2024-09-11T08:43:03.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8720999999999997e-05 2024-09-11T08:43:03.435 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.435 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.436 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.436 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.436 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:67212 7.1 7:82e7f96c:::benchmark_data_smithi092_84565_object67211:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:43:03.436 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:47.022217+0000", 2024-09-11T08:43:03.436 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 76.352888034000003, 2024-09-11T08:43:03.436 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.049122262, 2024-09-11T08:43:03.436 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.436 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.436 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.436 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.436 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.436 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 67212 2024-09-11T08:43:03.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.437 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.437 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.437 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.437 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.022217+0000", 2024-09-11T08:43:03.437 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.437 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.437 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.437 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.437 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.022217+0000", 2024-09-11T08:43:03.437 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.437 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.437 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.437 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.438 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.022212+0000", 2024-09-11T08:43:03.438 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999957 2024-09-11T08:43:03.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.438 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.438 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.438 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.022262+0000", 2024-09-11T08:43:03.438 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9638999999999997e-05 2024-09-11T08:43:03.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.438 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.438 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.438 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.022277+0000", 2024-09-11T08:43:03.438 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5268e-05 2024-09-11T08:43:03.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.438 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.439 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.022283+0000", 2024-09-11T08:43:03.439 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0349999999999998e-06 2024-09-11T08:43:03.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.439 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.439 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.022315+0000", 2024-09-11T08:43:03.439 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1955000000000002e-05 2024-09-11T08:43:03.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.439 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.439 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.022432+0000", 2024-09-11T08:43:03.439 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011705900000000001 2024-09-11T08:43:03.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.440 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.440 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.022519+0000", 2024-09-11T08:43:03.440 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.6407000000000001e-05 2024-09-11T08:43:03.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.440 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.440 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.024152+0000", 2024-09-11T08:43:03.440 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0016329409999999999 2024-09-11T08:43:03.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.440 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.440 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.024400+0000", 2024-09-11T08:43:03.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000247903 2024-09-11T08:43:03.441 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.441 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.441 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.441 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.071220+0000", 2024-09-11T08:43:03.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.046820762000000002 2024-09-11T08:43:03.441 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.441 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.441 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.441 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.071293+0000", 2024-09-11T08:43:03.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2962999999999999e-05 2024-09-11T08:43:03.441 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.441 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.442 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.442 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.071339+0000", 2024-09-11T08:43:03.442 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5788999999999998e-05 2024-09-11T08:43:03.442 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.442 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.442 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.442 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.442 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.442 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:67868 7.1 7:859b3c13:::benchmark_data_smithi092_84565_object67867:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:43:03.442 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:47.558098+0000", 2024-09-11T08:43:03.442 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 75.817007042, 2024-09-11T08:43:03.442 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.068992345999999996, 2024-09-11T08:43:03.442 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.443 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.443 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.443 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.443 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.443 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 67868 2024-09-11T08:43:03.443 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.443 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.443 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.443 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.443 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.558098+0000", 2024-09-11T08:43:03.443 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.443 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.444 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.558098+0000", 2024-09-11T08:43:03.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.444 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.558096+0000", 2024-09-11T08:43:03.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:43:03.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.444 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.558195+0000", 2024-09-11T08:43:03.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.8434e-05 2024-09-11T08:43:03.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.445 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.445 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.558209+0000", 2024-09-11T08:43:03.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4195999999999999e-05 2024-09-11T08:43:03.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.445 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.445 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.558215+0000", 2024-09-11T08:43:03.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9780000000000002e-06 2024-09-11T08:43:03.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.445 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.445 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.558264+0000", 2024-09-11T08:43:03.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9054000000000002e-05 2024-09-11T08:43:03.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.558389+0000", 2024-09-11T08:43:03.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000125501 2024-09-11T08:43:03.446 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.558528+0000", 2024-09-11T08:43:03.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013844199999999999 2024-09-11T08:43:03.446 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.561517+0000", 2024-09-11T08:43:03.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0029894629999999999 2024-09-11T08:43:03.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.562674+0000", 2024-09-11T08:43:03.447 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001156783 2024-09-11T08:43:03.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.627020+0000", 2024-09-11T08:43:03.447 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.064345924999999998 2024-09-11T08:43:03.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.627063+0000", 2024-09-11T08:43:03.447 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3538000000000002e-05 2024-09-11T08:43:03.448 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.448 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.448 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.448 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.627090+0000", 2024-09-11T08:43:03.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.6741999999999999e-05 2024-09-11T08:43:03.448 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.448 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.448 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.448 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.448 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.448 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:68305 7.1 7:81256ad5:::benchmark_data_smithi092_84565_object68304:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:43:03.448 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:47.952728+0000", 2024-09-11T08:43:03.448 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 75.422376752000005, 2024-09-11T08:43:03.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.070934228000000002, 2024-09-11T08:43:03.449 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.449 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.449 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.449 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.449 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.449 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 68305 2024-09-11T08:43:03.449 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.449 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.449 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.449 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.449 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.952728+0000", 2024-09-11T08:43:03.449 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.449 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.952728+0000", 2024-09-11T08:43:03.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.952726+0000", 2024-09-11T08:43:03.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:43:03.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.952796+0000", 2024-09-11T08:43:03.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9204999999999999e-05 2024-09-11T08:43:03.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.952808+0000", 2024-09-11T08:43:03.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2388999999999999e-05 2024-09-11T08:43:03.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.952813+0000", 2024-09-11T08:43:03.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8969999999999999e-06 2024-09-11T08:43:03.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.952853+0000", 2024-09-11T08:43:03.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0157999999999997e-05 2024-09-11T08:43:03.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.952939+0000", 2024-09-11T08:43:03.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.6100000000000006e-05 2024-09-11T08:43:03.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.953037+0000", 2024-09-11T08:43:03.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.7985000000000005e-05 2024-09-11T08:43:03.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.957998+0000", 2024-09-11T08:43:03.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0049605379999999996 2024-09-11T08:43:03.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.958311+0000", 2024-09-11T08:43:03.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00031344700000000001 2024-09-11T08:43:03.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:48.023592+0000", 2024-09-11T08:43:03.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.065281236000000006 2024-09-11T08:43:03.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:48.023626+0000", 2024-09-11T08:43:03.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3314e-05 2024-09-11T08:43:03.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:48.023662+0000", 2024-09-11T08:43:03.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6677999999999997e-05 2024-09-11T08:43:03.454 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.454 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.454 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.455 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:68309 7.1 7:88131e66:::benchmark_data_smithi092_84565_object68308:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:43:03.455 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:47.954508+0000", 2024-09-11T08:43:03.455 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 75.420597326000006, 2024-09-11T08:43:03.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.069261477000000002, 2024-09-11T08:43:03.455 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.455 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.455 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.455 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.455 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.455 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 68309 2024-09-11T08:43:03.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.455 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.455 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.455 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.954508+0000", 2024-09-11T08:43:03.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.456 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.456 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.456 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.954508+0000", 2024-09-11T08:43:03.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.456 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.456 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.456 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.954506+0000", 2024-09-11T08:43:03.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:43:03.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.456 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.954701+0000", 2024-09-11T08:43:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000195012 2024-09-11T08:43:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.954719+0000", 2024-09-11T08:43:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.8090000000000001e-05 2024-09-11T08:43:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.954726+0000", 2024-09-11T08:43:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.7309999999999996e-06 2024-09-11T08:43:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.955347+0000", 2024-09-11T08:43:03.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000621742 2024-09-11T08:43:03.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.955464+0000", 2024-09-11T08:43:03.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000116873 2024-09-11T08:43:03.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.955591+0000", 2024-09-11T08:43:03.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012672200000000001 2024-09-11T08:43:03.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.958129+0000", 2024-09-11T08:43:03.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0025383440000000001 2024-09-11T08:43:03.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:47.958434+0000", 2024-09-11T08:43:03.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00030509899999999997 2024-09-11T08:43:03.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:48.023696+0000", 2024-09-11T08:43:03.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.065261822999999997 2024-09-11T08:43:03.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:48.023731+0000", 2024-09-11T08:43:03.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4671000000000001e-05 2024-09-11T08:43:03.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:48.023769+0000", 2024-09-11T08:43:03.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8198999999999998e-05 2024-09-11T08:43:03.461 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.461 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.461 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.461 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.14814.0:96577 7.8 e106/88 7:1c38cb72:::benchmark_data_smithi092_84565_object96576:head v 106'6110, mlcod=106'6107)", 2024-09-11T08:43:03.461 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:12.381078+0000", 2024-09-11T08:43:03.461 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 50.994027068000001, 2024-09-11T08:43:03.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.087516295999999993, 2024-09-11T08:43:03.462 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.462 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.462 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.381078+0000", 2024-09-11T08:43:03.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.381078+0000", 2024-09-11T08:43:03.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.381077+0000", 2024-09-11T08:43:03.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:43:03.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.381135+0000", 2024-09-11T08:43:03.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7889000000000001e-05 2024-09-11T08:43:03.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.381144+0000", 2024-09-11T08:43:03.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.6049999999999998e-06 2024-09-11T08:43:03.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.381148+0000", 2024-09-11T08:43:03.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4349999999999999e-06 2024-09-11T08:43:03.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.459852+0000", 2024-09-11T08:43:03.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.078704273000000005 2024-09-11T08:43:03.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.459882+0000", 2024-09-11T08:43:03.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0083e-05 2024-09-11T08:43:03.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.468519+0000", 2024-09-11T08:43:03.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0086362590000000003 2024-09-11T08:43:03.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.468594+0000", 2024-09-11T08:43:03.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.5454e-05 2024-09-11T08:43:03.466 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.466 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.466 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.466 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.14814.0:96575 7.9 e106/88 7:9264d9f2:::benchmark_data_smithi092_84565_object96574:head v 106'6063, mlcod=106'6061)", 2024-09-11T08:43:03.466 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:12.381626+0000", 2024-09-11T08:43:03.466 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 50.993478680999999, 2024-09-11T08:43:03.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.086658006999999995, 2024-09-11T08:43:03.466 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.466 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.466 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.381626+0000", 2024-09-11T08:43:03.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.467 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.381626+0000", 2024-09-11T08:43:03.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.467 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.381626+0000", 2024-09-11T08:43:03.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:43:03.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.381741+0000", 2024-09-11T08:43:03.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000115 2024-09-11T08:43:03.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.459847+0000", 2024-09-11T08:43:03.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.078106696000000003 2024-09-11T08:43:03.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.459862+0000", 2024-09-11T08:43:03.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5098e-05 2024-09-11T08:43:03.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.469 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.469 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.459994+0000", 2024-09-11T08:43:03.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013129999999999999 2024-09-11T08:43:03.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.469 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.469 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.460014+0000", 2024-09-11T08:43:03.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0443000000000001e-05 2024-09-11T08:43:03.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.469 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.468249+0000", 2024-09-11T08:43:03.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0082347789999999994 2024-09-11T08:43:03.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.468284+0000", 2024-09-11T08:43:03.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5349999999999999e-05 2024-09-11T08:43:03.470 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.470 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.470 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.470 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.14814.0:96587 7.8 e106/88 7:1bec6a88:::benchmark_data_smithi092_84565_object96586:head v 106'6111, mlcod=106'6108)", 2024-09-11T08:43:03.471 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:12.390554+0000", 2024-09-11T08:43:03.471 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 50.984550526, 2024-09-11T08:43:03.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.080715526999999995, 2024-09-11T08:43:03.471 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.471 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.471 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.390554+0000", 2024-09-11T08:43:03.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.390554+0000", 2024-09-11T08:43:03.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.390553+0000", 2024-09-11T08:43:03.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:43:03.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.390633+0000", 2024-09-11T08:43:03.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.9363000000000006e-05 2024-09-11T08:43:03.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.459847+0000", 2024-09-11T08:43:03.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.069214247000000007 2024-09-11T08:43:03.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.459862+0000", 2024-09-11T08:43:03.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4842e-05 2024-09-11T08:43:03.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.461977+0000", 2024-09-11T08:43:03.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0021147449999999999 2024-09-11T08:43:03.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.461990+0000", 2024-09-11T08:43:03.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3825000000000001e-05 2024-09-11T08:43:03.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.471233+0000", 2024-09-11T08:43:03.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.009242732 2024-09-11T08:43:03.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.471270+0000", 2024-09-11T08:43:03.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6825000000000001e-05 2024-09-11T08:43:03.474 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.475 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.475 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.475 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.14814.0:96589 7.3 e106/88 7:caf5745c:::benchmark_data_smithi092_84565_object96588:head v 106'6039, mlcod=106'6037)", 2024-09-11T08:43:03.475 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:12.391322+0000", 2024-09-11T08:43:03.475 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 50.983783039000002, 2024-09-11T08:43:03.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.074135303999999999, 2024-09-11T08:43:03.475 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.475 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.475 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.391322+0000", 2024-09-11T08:43:03.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.391322+0000", 2024-09-11T08:43:03.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.391321+0000", 2024-09-11T08:43:03.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:43:03.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.391366+0000", 2024-09-11T08:43:03.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4991999999999999e-05 2024-09-11T08:43:03.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.459859+0000", 2024-09-11T08:43:03.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.068492549 2024-09-11T08:43:03.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.459864+0000", 2024-09-11T08:43:03.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8459999999999996e-06 2024-09-11T08:43:03.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.459914+0000", 2024-09-11T08:43:03.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9418e-05 2024-09-11T08:43:03.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.459933+0000", 2024-09-11T08:43:03.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.9029e-05 2024-09-11T08:43:03.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.465399+0000", 2024-09-11T08:43:03.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0054664889999999997 2024-09-11T08:43:03.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.479 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.479 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:12.465457+0000", 2024-09-11T08:43:03.479 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7852000000000001e-05 2024-09-11T08:43:03.479 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.479 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.479 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.479 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.479 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.479 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:115874 7.1 7:8a8f80c2:::benchmark_data_smithi092_84565_object115873:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:43:03.479 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:31.717534+0000", 2024-09-11T08:43:03.479 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 31.657571198999999, 2024-09-11T08:43:03.479 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.030650013, 2024-09-11T08:43:03.479 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.480 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.480 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.480 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.480 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.480 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 115874 2024-09-11T08:43:03.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.480 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.480 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.480 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.480 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.717534+0000", 2024-09-11T08:43:03.480 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.480 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.480 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.717534+0000", 2024-09-11T08:43:03.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.481 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.717532+0000", 2024-09-11T08:43:03.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:43:03.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.481 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.717592+0000", 2024-09-11T08:43:03.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0062000000000002e-05 2024-09-11T08:43:03.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.717606+0000", 2024-09-11T08:43:03.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3267e-05 2024-09-11T08:43:03.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.717611+0000", 2024-09-11T08:43:03.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.57e-06 2024-09-11T08:43:03.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.717657+0000", 2024-09-11T08:43:03.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5772000000000002e-05 2024-09-11T08:43:03.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.717803+0000", 2024-09-11T08:43:03.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014576199999999999 2024-09-11T08:43:03.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.717899+0000", 2024-09-11T08:43:03.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.6459999999999995e-05 2024-09-11T08:43:03.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.484 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.484 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.720615+0000", 2024-09-11T08:43:03.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0027163090000000001 2024-09-11T08:43:03.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.484 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.484 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.721056+0000", 2024-09-11T08:43:03.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00044056100000000001 2024-09-11T08:43:03.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.484 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.484 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.748039+0000", 2024-09-11T08:43:03.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.026983330999999999 2024-09-11T08:43:03.485 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.485 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.485 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.485 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.748126+0000", 2024-09-11T08:43:03.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.6750999999999997e-05 2024-09-11T08:43:03.485 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.485 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.485 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.485 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.748184+0000", 2024-09-11T08:43:03.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7568000000000002e-05 2024-09-11T08:43:03.485 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.485 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.485 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:125264 7.1 7:8012218c:::benchmark_data_smithi092_84565_object125263:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:41.375793+0000", 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 21.999311809000002, 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.033993763000000003, 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 125264 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.487 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:41.375793+0000", 2024-09-11T08:43:03.487 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.487 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.487 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.487 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.487 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:41.375793+0000", 2024-09-11T08:43:03.487 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.487 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.487 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.487 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.487 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:41.375792+0000", 2024-09-11T08:43:03.487 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:43:03.487 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.487 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:41.375843+0000", 2024-09-11T08:43:03.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1672000000000003e-05 2024-09-11T08:43:03.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:41.375853+0000", 2024-09-11T08:43:03.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.9080000000000007e-06 2024-09-11T08:43:03.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:41.375857+0000", 2024-09-11T08:43:03.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.749e-06 2024-09-11T08:43:03.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:41.375892+0000", 2024-09-11T08:43:03.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5163000000000002e-05 2024-09-11T08:43:03.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:41.376018+0000", 2024-09-11T08:43:03.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012540300000000001 2024-09-11T08:43:03.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:41.376118+0000", 2024-09-11T08:43:03.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010000700000000001 2024-09-11T08:43:03.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:41.379357+0000", 2024-09-11T08:43:03.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0032397139999999999 2024-09-11T08:43:03.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:41.379540+0000", 2024-09-11T08:43:03.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00018284 2024-09-11T08:43:03.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:41.409694+0000", 2024-09-11T08:43:03.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.030153480999999999 2024-09-11T08:43:03.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.491 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.491 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.491 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:41.409752+0000", 2024-09-11T08:43:03.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8063000000000003e-05 2024-09-11T08:43:03.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.491 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.491 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.491 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:41.409787+0000", 2024-09-11T08:43:03.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5123999999999998e-05 2024-09-11T08:43:03.491 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.491 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.491 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.492 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:144709 7.1 7:82e4b26f:::benchmark_data_smithi092_84565_object19181:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-11T08:43:03.492 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:02.027180+0000", 2024-09-11T08:43:03.492 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 1.347924696, 2024-09-11T08:43:03.492 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.049816552, 2024-09-11T08:43:03.492 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:03.492 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:03.492 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:03.492 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:03.492 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:03.492 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 144709 2024-09-11T08:43:03.492 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.492 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:03.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.493 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:03.493 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.027180+0000", 2024-09-11T08:43:03.493 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.493 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.493 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:03.493 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.027180+0000", 2024-09-11T08:43:03.493 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:03.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.493 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.493 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:03.493 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.027179+0000", 2024-09-11T08:43:03.493 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:43:03.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.494 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.494 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:03.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.027185+0000", 2024-09-11T08:43:03.494 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8300000000000001e-06 2024-09-11T08:43:03.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.494 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.494 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:03.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.027196+0000", 2024-09-11T08:43:03.494 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1345000000000001e-05 2024-09-11T08:43:03.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.494 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.494 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:03.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.027201+0000", 2024-09-11T08:43:03.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6059999999999996e-06 2024-09-11T08:43:03.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:03.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.027238+0000", 2024-09-11T08:43:03.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7067000000000003e-05 2024-09-11T08:43:03.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:03.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.027346+0000", 2024-09-11T08:43:03.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000107865 2024-09-11T08:43:03.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:43:03.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.027452+0000", 2024-09-11T08:43:03.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000106733 2024-09-11T08:43:03.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.029803+0000", 2024-09-11T08:43:03.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0023508489999999999 2024-09-11T08:43:03.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:03.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.029897+0000", 2024-09-11T08:43:03.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.4236999999999999e-05 2024-09-11T08:43:03.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.497 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.497 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:03.497 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.076920+0000", 2024-09-11T08:43:03.497 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.047022384 2024-09-11T08:43:03.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.497 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.497 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:03.497 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.076951+0000", 2024-09-11T08:43:03.497 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1222e-05 2024-09-11T08:43:03.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:03.497 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:03.497 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:03.498 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.076997+0000", 2024-09-11T08:43:03.498 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5649000000000003e-05 2024-09-11T08:43:03.498 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.498 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.498 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.498 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:03.498 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:03.498 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:43:03.498 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:03 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:43:03.498 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:03 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:43:03.499 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:03 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:43:03.499 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:03 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:43:03.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:03.583 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:03.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:03.584 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:03.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:03 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:43:03.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:03 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:43:03.684 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:03.708 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:03.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:03.709 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:03.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:03.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:03.867 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:03.867 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:03.963 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:03.968 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:03.990 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:03.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:03.991 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:04.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:04.150 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:04.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:04.151 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:04.215 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:43:04.252 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:04.277 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:04.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:04.278 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:04.301 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-11T08:43:04.379 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:04.427 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:43:04.436 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:04.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:04.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:04.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:04.587 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:04.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:04.592 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:04.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:04 smithi145 ceph-mon[33384]: pgmap v163: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T08:43:04.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:04 smithi145 ceph-mon[33384]: osdmap e116: 8 total, 8 up, 8 in 2024-09-11T08:43:04.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:04 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2740716718' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:04.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:04 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:04.676 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:43:04.693 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:04.742 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:04.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:04.743 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:04.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:04 smithi092 ceph-mon[29647]: pgmap v163: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T08:43:04.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:04 smithi092 ceph-mon[29647]: osdmap e116: 8 total, 8 up, 8 in 2024-09-11T08:43:04.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:04 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2740716718' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:04.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:04 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:04.808 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:04 smithi092 ceph-mon[41114]: pgmap v163: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T08:43:04.808 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:04 smithi092 ceph-mon[41114]: osdmap e116: 8 total, 8 up, 8 in 2024-09-11T08:43:04.808 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:04 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2740716718' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:04.808 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:04 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:04.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:04.882 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:04.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:04.883 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:04.984 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:05.018 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:05.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:05.020 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:05.120 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:05.146 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:05.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:05.147 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:05.248 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:05.274 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:05.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:05.275 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:05.376 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:05.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:05.407 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:05.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:05.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:05 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:43:05.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:05 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:43:05.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:05 smithi145 ceph-mon[33384]: osdmap e117: 8 total, 8 up, 8 in 2024-09-11T08:43:05.508 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:05.534 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:05.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:05.536 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:05.636 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:05.672 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:05.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:05.674 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:05.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:05 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:43:05.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:05 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:43:05.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:05 smithi092 ceph-mon[29647]: osdmap e117: 8 total, 8 up, 8 in 2024-09-11T08:43:05.684 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:05 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:43:05.684 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:05 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:43:05.684 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:05 smithi092 ceph-mon[41114]: osdmap e117: 8 total, 8 up, 8 in 2024-09-11T08:43:05.775 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:05.803 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:05.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:05.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:05.905 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:05.933 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:05.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:05.935 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:06.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:06.071 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:06.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:06.072 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:06.155 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:06.174 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:06.198 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:06.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:06.199 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:06.300 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:06.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:06.325 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:06.325 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:06.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:06.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:06.461 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:06.461 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:06.562 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:06.594 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:06.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:06.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:06.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:06 smithi145 ceph-mon[33384]: pgmap v166: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 788 op/s 2024-09-11T08:43:06.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:43:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:43:06.696 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:06.787 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:06.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:06.788 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:06.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:06 smithi092 ceph-mon[29647]: pgmap v166: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 788 op/s 2024-09-11T08:43:06.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:06 smithi092 ceph-mon[41114]: pgmap v166: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 788 op/s 2024-09-11T08:43:06.889 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:06.910 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:06.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:06.911 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:07.012 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:07.040 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:07.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:07.041 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:07.142 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:07.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:43:07.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:07.178 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:07.179 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:07.280 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:07.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:07.357 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:07.358 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:07.397 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:43:07.397 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:43:07.397 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:07.397 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:150185 7.0 7:0c2fe576:::benchmark_data_smithi092_84565_object24657:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:43:07.397 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:07.393665+0000", 2024-09-11T08:43:07.397 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.0030206999999999999, 2024-09-11T08:43:07.397 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003042646, 2024-09-11T08:43:07.397 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:07.397 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:43:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 150185 2024-09-11T08:43:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.393665+0000", 2024-09-11T08:43:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.393665+0000", 2024-09-11T08:43:07.399 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:07.399 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:07.399 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:07.399 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:07.399 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.393664+0000", 2024-09-11T08:43:07.399 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:43:07.399 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:07.399 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:07.399 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:07.399 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.393670+0000", 2024-09-11T08:43:07.399 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2190000000000001e-06 2024-09-11T08:43:07.399 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:07.399 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:07.399 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:07.400 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.393684+0000", 2024-09-11T08:43:07.400 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3653e-05 2024-09-11T08:43:07.400 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:07.400 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:07.400 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:07.400 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.393690+0000", 2024-09-11T08:43:07.400 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8139999999999997e-06 2024-09-11T08:43:07.400 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:07.400 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:07.400 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:07.400 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.393721+0000", 2024-09-11T08:43:07.400 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.154e-05 2024-09-11T08:43:07.400 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:07.401 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:07.401 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:07.401 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.393806+0000", 2024-09-11T08:43:07.401 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.5326000000000003e-05 2024-09-11T08:43:07.401 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:07.401 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:07.401 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:43:07.401 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.393879+0000", 2024-09-11T08:43:07.401 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2484999999999995e-05 2024-09-11T08:43:07.401 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:07.401 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:07.401 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:07.401 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.396168+0000", 2024-09-11T08:43:07.401 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0022888700000000001 2024-09-11T08:43:07.402 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:07.402 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:07.402 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:07.402 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:07.402 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2024-09-11T08:43:07.402 INFO:teuthology.orchestra.run.smithi145.stdout: "num_ops": 1 2024-09-11T08:43:07.402 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:43:07.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:07.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:07.488 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:07.488 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:07.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:07 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:07.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:07 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:07.589 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:07.618 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:07.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:07.619 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:07.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:07 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:07.720 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:07.741 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:07.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:07.742 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:07.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:07.887 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:07.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:07.888 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:07.989 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:08.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:08.010 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:08.011 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:08.111 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:08.140 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:08.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:08.141 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:08.242 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:08.281 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:08.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:08.282 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:08.382 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:08.417 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:08.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:08.418 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:08.486 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-11T08:43:08.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:08.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:08.569 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:08.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:08.706 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:08.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:08.707 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:08.808 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:08.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:08 smithi092 ceph-mon[29647]: pgmap v167: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-11T08:43:08.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:08 smithi092 ceph-mon[41114]: pgmap v167: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-11T08:43:08.816 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:43:08.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:08.852 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:08.852 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:08 smithi145 ceph-mon[33384]: pgmap v167: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-11T08:43:08.953 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:08.981 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:08.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:08.982 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:09.083 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:09.111 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:09.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:09.112 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:09.212 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:09.258 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:09.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:09.259 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:09.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:09.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:09.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:09.500 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:09.525 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:09.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:09.527 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:09.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:09.668 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:09.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:09.669 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:09.763 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T08:43:09.769 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:09.770 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:09 smithi145 ceph-mon[33384]: pgmap v168: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 664 B/s rd, 0 B/s wr, 657 op/s 2024-09-11T08:43:09.796 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:09.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:09.797 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:09.898 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:09.924 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:09.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:09.925 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:10.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:10.061 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:10.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:10.062 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:10.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:09 smithi092 ceph-mon[29647]: pgmap v168: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 664 B/s rd, 0 B/s wr, 657 op/s 2024-09-11T08:43:10.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:09 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/606829270' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T08:43:10.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:09 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T08:43:10.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:09 smithi092 ceph-mon[41114]: pgmap v168: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 664 B/s rd, 0 B/s wr, 657 op/s 2024-09-11T08:43:10.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:09 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/606829270' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T08:43:10.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:09 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T08:43:10.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:09 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/606829270' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T08:43:10.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:09 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T08:43:10.163 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:10.188 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:10.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:10.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:10.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:10.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:10.337 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:10.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:10.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:10.539 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:10.539 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:10.640 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:10.673 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:10.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:10.674 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:10.775 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:10.793 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:10 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T08:43:10.793 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:10 smithi145 ceph-mon[33384]: osdmap e118: 8 total, 8 up, 8 in 2024-09-11T08:43:10.816 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:10.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:10.817 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:10.836 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:43:10.836 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T08:43:10.836 INFO:teuthology.orchestra.run.smithi145.stdout: "complaint_time": 30, 2024-09-11T08:43:10.836 INFO:teuthology.orchestra.run.smithi145.stdout: "num_blocked_ops": 0 2024-09-11T08:43:10.837 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:43:10.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:10.956 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:10.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:10.957 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:11.058 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:11.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:10 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T08:43:11.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:10 smithi092 ceph-mon[41114]: osdmap e118: 8 total, 8 up, 8 in 2024-09-11T08:43:11.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:10 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T08:43:11.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:10 smithi092 ceph-mon[29647]: osdmap e118: 8 total, 8 up, 8 in 2024-09-11T08:43:11.086 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:11.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:11.087 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:11.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:11.225 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:11.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:11.226 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:11.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 5 1 2024-09-11T08:43:11.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:11.374 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:11.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:11.375 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:11.476 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:11.501 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:11.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:11.502 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:11.594 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:11.603 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:11.627 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:11.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:11.629 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:11.650 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:43:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:43:11] "GET /metrics HTTP/1.1" 200 223755 "" "Prometheus/2.43.0" 2024-09-11T08:43:11.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:11.777 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:11.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:11.778 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:11.805 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-11T08:43:11.879 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:11.903 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:11.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:11.904 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:11.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:11 smithi092 ceph-mon[29647]: pgmap v170: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 913 op/s 2024-09-11T08:43:11.926 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:43:11.926 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:11.926 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:11 smithi092 ceph-mon[41114]: pgmap v170: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 913 op/s 2024-09-11T08:43:11.960 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:43:11.960 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:11.999 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:43:11.999 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:12.005 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:12.031 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:12.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:12.032 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:12.038 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:43:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:12.065 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:43:12.065 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:12.096 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:43:12.096 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:12.118 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:43:12.130 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:43:12.130 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:12.132 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:12.158 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:12.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:12.159 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:12.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:11 smithi145 ceph-mon[33384]: pgmap v170: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 913 op/s 2024-09-11T08:43:12.167 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:43:12.167 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:12.260 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:12.285 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:12.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:12.286 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:12.387 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:12.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:12.425 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:12.526 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:12.552 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:12.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:12.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:12.654 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:12.679 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:12.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:12.681 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:12.781 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:12.805 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:12.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:12.806 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:12 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3683238845' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:12.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:12 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3683238845' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:12.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:12.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:12 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3683238845' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:12.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:43:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:43:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:12.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:12.939 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:12.940 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:13.040 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:13.066 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:13.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:13.067 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:13.168 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:13.191 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:13.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:13.192 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:13.293 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:13.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:13.317 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:13.318 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:13.410 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:13.418 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:13.442 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:13.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:13.443 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:13.544 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:13.581 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:13.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:13.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:13.683 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:13.708 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:13.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:13.709 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:13.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:13 smithi092 ceph-mon[29647]: pgmap v171: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-11T08:43:13.810 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:13.851 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:13.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:13.852 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:13.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:13.991 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:13.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:13.992 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:14.058 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:13 smithi145 ceph-mon[33384]: pgmap v171: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-11T08:43:14.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:13 smithi092 ceph-mon[41114]: pgmap v171: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-11T08:43:14.072 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:14.072 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:14.094 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:14.122 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:14.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:14.123 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:14.224 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:14.253 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:14.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:14.254 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:14.355 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:14.380 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:14.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:14.381 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:14.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:14.514 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:14.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:14.515 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:14.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:14.653 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:14.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:14.654 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:14.755 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:14.780 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:14.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:14.781 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:14.864 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-11T08:43:14.881 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:14.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:43:14.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:14.938 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:14.938 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:15.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:15.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:14 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:43:15.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:14 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:43:15.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:14 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:43:15.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:14 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:43:15.081 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:15.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:15.082 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:15.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:14 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:43:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:14 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:43:15.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:15.189 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:43:15.247 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:15.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:15.248 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:15.349 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:15.375 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:15.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:15.376 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:15.477 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:15.527 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:15.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:15.529 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:15.629 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:15.674 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:15.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:15.675 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:15.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:15.817 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:15.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:15.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:15.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:15 smithi145 ceph-mon[33384]: osdmap e119: 8 total, 8 up, 8 in 2024-09-11T08:43:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:15 smithi145 ceph-mon[33384]: pgmap v173: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 651 op/s 2024-09-11T08:43:15.919 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:15.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:15.949 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:15.949 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:16.040 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:15 smithi092 ceph-mon[29647]: osdmap e119: 8 total, 8 up, 8 in 2024-09-11T08:43:16.040 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:16.040 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:15 smithi092 ceph-mon[29647]: pgmap v173: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 651 op/s 2024-09-11T08:43:16.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:15 smithi092 ceph-mon[41114]: osdmap e119: 8 total, 8 up, 8 in 2024-09-11T08:43:16.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:16.041 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:15 smithi092 ceph-mon[41114]: pgmap v173: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 651 op/s 2024-09-11T08:43:16.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:16.091 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:16.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:16.092 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:16.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:16.242 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:16.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:16.243 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:16.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:16.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:16.381 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:16.381 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:16.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:43:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:43:16.482 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:16.507 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:16.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:16.509 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:16.610 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:16.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:16.654 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:16.654 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:16.755 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:16.780 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:16.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:16.781 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.5 to 1 (10000) 2024-09-11T08:43:16.853 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:43:16.882 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:16.918 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:16.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:16.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:17.020 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:17.044 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:17.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:17.046 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:17.147 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:17.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:16 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:17.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:16 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4031724831' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:43:17.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:16 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:43:17.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:16 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3091736278' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T08:43:17.182 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:17.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:17.183 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:17.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:17.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:16 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:17.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:16 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4031724831' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:43:17.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:16 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:43:17.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:16 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3091736278' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T08:43:17.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:16 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:17.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:16 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4031724831' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:43:17.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:16 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:43:17.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:16 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3091736278' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T08:43:17.317 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:17.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:17.319 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:17.419 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:17.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:17.455 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:17.455 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:17.499 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:17.499 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:17.556 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:17.597 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:17.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:17.598 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:17.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:17.738 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:17.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:17.739 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:17.840 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:17.866 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:17.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:17.867 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:17.968 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:18.007 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:18.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:18.008 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:18.109 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:18.144 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:18.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:18.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:18.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:17 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:43:18.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:17 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:43:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:17 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3091736278' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T08:43:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:17 smithi145 ceph-mon[33384]: osdmap e120: 8 total, 8 up, 8 in 2024-09-11T08:43:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:17 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:43:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:17 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:43:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:17 smithi145 ceph-mon[33384]: 3.e scrub starts 2024-09-11T08:43:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:17 smithi145 ceph-mon[33384]: 3.e scrub ok 2024-09-11T08:43:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:17 smithi145 ceph-mon[33384]: pgmap v175: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-11T08:43:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:17 smithi145 ceph-mon[33384]: 3.17 scrub starts 2024-09-11T08:43:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:17 smithi145 ceph-mon[33384]: 3.17 scrub ok 2024-09-11T08:43:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:17 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:43:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:17 smithi145 ceph-mon[33384]: 5.8 scrub ok 2024-09-11T08:43:18.248 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:18.256 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-11T08:43:18.273 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:18.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:18.274 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:43:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:43:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3091736278' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T08:43:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[29647]: osdmap e120: 8 total, 8 up, 8 in 2024-09-11T08:43:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:43:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:43:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[29647]: 3.e scrub starts 2024-09-11T08:43:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[29647]: 3.e scrub ok 2024-09-11T08:43:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[29647]: pgmap v175: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-11T08:43:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[29647]: 3.17 scrub starts 2024-09-11T08:43:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[29647]: 3.17 scrub ok 2024-09-11T08:43:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:43:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[29647]: 5.8 scrub ok 2024-09-11T08:43:18.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:43:18.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:43:18.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3091736278' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T08:43:18.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[41114]: osdmap e120: 8 total, 8 up, 8 in 2024-09-11T08:43:18.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:43:18.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:43:18.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[41114]: 3.e scrub starts 2024-09-11T08:43:18.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[41114]: 3.e scrub ok 2024-09-11T08:43:18.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[41114]: pgmap v175: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-11T08:43:18.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[41114]: 3.17 scrub starts 2024-09-11T08:43:18.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[41114]: 3.17 scrub ok 2024-09-11T08:43:18.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:43:18.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:17 smithi092 ceph-mon[41114]: 5.8 scrub ok 2024-09-11T08:43:18.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:18.412 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:18.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:18.413 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:18.514 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:18.546 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:18.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:18.547 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:18.616 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:43:18.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:18.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:18.673 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:18.673 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:18.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:18.817 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:18.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:18.818 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:18.919 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:18.941 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:18.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:18.943 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:19.044 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:19.052 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:18 smithi145 ceph-mon[33384]: 7.0 scrub starts 2024-09-11T08:43:19.052 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:18 smithi145 ceph-mon[33384]: osdmap e121: 8 total, 8 up, 8 in 2024-09-11T08:43:19.052 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:18 smithi145 ceph-mon[33384]: 6.11 scrub starts 2024-09-11T08:43:19.052 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:18 smithi145 ceph-mon[33384]: 6.11 scrub ok 2024-09-11T08:43:19.052 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:18 smithi145 ceph-mon[33384]: 5.1a scrub starts 2024-09-11T08:43:19.053 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:18 smithi145 ceph-mon[33384]: 5.1a scrub ok 2024-09-11T08:43:19.053 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:18 smithi145 ceph-mon[33384]: 6.18 scrub starts 2024-09-11T08:43:19.053 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:18 smithi145 ceph-mon[33384]: 6.18 scrub ok 2024-09-11T08:43:19.082 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:19.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:19.084 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:19.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:19.219 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:19.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:19.220 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:19.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[29647]: 7.0 scrub starts 2024-09-11T08:43:19.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[29647]: osdmap e121: 8 total, 8 up, 8 in 2024-09-11T08:43:19.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[29647]: 6.11 scrub starts 2024-09-11T08:43:19.279 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[29647]: 6.11 scrub ok 2024-09-11T08:43:19.279 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[29647]: 5.1a scrub starts 2024-09-11T08:43:19.279 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[29647]: 5.1a scrub ok 2024-09-11T08:43:19.279 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[29647]: 6.18 scrub starts 2024-09-11T08:43:19.279 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[29647]: 6.18 scrub ok 2024-09-11T08:43:19.279 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[41114]: 7.0 scrub starts 2024-09-11T08:43:19.279 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[41114]: osdmap e121: 8 total, 8 up, 8 in 2024-09-11T08:43:19.279 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[41114]: 6.11 scrub starts 2024-09-11T08:43:19.279 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[41114]: 6.11 scrub ok 2024-09-11T08:43:19.279 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[41114]: 5.1a scrub starts 2024-09-11T08:43:19.279 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[41114]: 5.1a scrub ok 2024-09-11T08:43:19.279 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[41114]: 6.18 scrub starts 2024-09-11T08:43:19.279 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:18 smithi092 ceph-mon[41114]: 6.18 scrub ok 2024-09-11T08:43:19.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 6 1 2024-09-11T08:43:19.321 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:19.346 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:19.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:19.348 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:19.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:19.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:19.491 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:19.492 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:19.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:19.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:19.638 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:19.639 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:19.740 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:19.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:19.765 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:19.765 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:19.867 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:19.895 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:19.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:19.896 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:19.969 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:19 smithi092 ceph-mon[29647]: 5.16 scrub starts 2024-09-11T08:43:19.969 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:19 smithi092 ceph-mon[29647]: 5.16 scrub ok 2024-09-11T08:43:19.969 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:19 smithi092 ceph-mon[29647]: pgmap v177: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 867 op/s 2024-09-11T08:43:19.969 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:19 smithi092 ceph-mon[29647]: 4.a scrub starts 2024-09-11T08:43:19.969 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:19 smithi092 ceph-mon[29647]: 4.a scrub ok 2024-09-11T08:43:19.969 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:19 smithi092 ceph-mon[29647]: 6.14 scrub starts 2024-09-11T08:43:19.969 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:19 smithi092 ceph-mon[29647]: 6.14 scrub ok 2024-09-11T08:43:19.970 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:19 smithi092 ceph-mon[41114]: 5.16 scrub starts 2024-09-11T08:43:19.970 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:19 smithi092 ceph-mon[41114]: 5.16 scrub ok 2024-09-11T08:43:19.970 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:19 smithi092 ceph-mon[41114]: pgmap v177: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 867 op/s 2024-09-11T08:43:19.970 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:19 smithi092 ceph-mon[41114]: 4.a scrub starts 2024-09-11T08:43:19.970 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:19 smithi092 ceph-mon[41114]: 4.a scrub ok 2024-09-11T08:43:19.970 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:19 smithi092 ceph-mon[41114]: 6.14 scrub starts 2024-09-11T08:43:19.970 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:19 smithi092 ceph-mon[41114]: 6.14 scrub ok 2024-09-11T08:43:19.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:20.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:20.045 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:20.045 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:20.133 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:19 smithi145 ceph-mon[33384]: 5.16 scrub starts 2024-09-11T08:43:20.134 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:19 smithi145 ceph-mon[33384]: 5.16 scrub ok 2024-09-11T08:43:20.134 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:19 smithi145 ceph-mon[33384]: pgmap v177: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 867 op/s 2024-09-11T08:43:20.134 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:19 smithi145 ceph-mon[33384]: 4.a scrub starts 2024-09-11T08:43:20.134 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:19 smithi145 ceph-mon[33384]: 4.a scrub ok 2024-09-11T08:43:20.134 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:19 smithi145 ceph-mon[33384]: 6.14 scrub starts 2024-09-11T08:43:20.134 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:19 smithi145 ceph-mon[33384]: 6.14 scrub ok 2024-09-11T08:43:20.146 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:20.175 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:20.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:20.176 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:20.277 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:20.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:20.306 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:20.306 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:20.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:20.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:20.442 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:20.442 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:20.543 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:20.557 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:20.557 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:20.578 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:20.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:20.579 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:20.680 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:20.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:20.724 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:20.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:20.825 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:20.863 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:20.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:20.864 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:20.965 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:20.990 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:20.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:20.991 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:21.044 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:20 smithi092 ceph-mon[29647]: 3.4 scrub starts 2024-09-11T08:43:21.044 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:20 smithi092 ceph-mon[29647]: 3.4 scrub ok 2024-09-11T08:43:21.044 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:20 smithi092 ceph-mon[29647]: 7.0 scrub ok 2024-09-11T08:43:21.044 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:20 smithi092 ceph-mon[29647]: 5.4 scrub starts 2024-09-11T08:43:21.044 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:20 smithi092 ceph-mon[29647]: 5.4 scrub ok 2024-09-11T08:43:21.044 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:20 smithi092 ceph-mon[29647]: 5.b scrub starts 2024-09-11T08:43:21.044 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:20 smithi092 ceph-mon[29647]: 5.b scrub ok 2024-09-11T08:43:21.045 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:20 smithi092 ceph-mon[41114]: 3.4 scrub starts 2024-09-11T08:43:21.045 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:20 smithi092 ceph-mon[41114]: 3.4 scrub ok 2024-09-11T08:43:21.045 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:20 smithi092 ceph-mon[41114]: 7.0 scrub ok 2024-09-11T08:43:21.045 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:20 smithi092 ceph-mon[41114]: 5.4 scrub starts 2024-09-11T08:43:21.045 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:20 smithi092 ceph-mon[41114]: 5.4 scrub ok 2024-09-11T08:43:21.045 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:20 smithi092 ceph-mon[41114]: 5.b scrub starts 2024-09-11T08:43:21.045 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:20 smithi092 ceph-mon[41114]: 5.b scrub ok 2024-09-11T08:43:21.092 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:21.108 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:21.115 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:43:21.115 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:21.118 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:21.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:21.119 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:21.149 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:43:21.149 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:21.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:20 smithi145 ceph-mon[33384]: 3.4 scrub starts 2024-09-11T08:43:21.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:20 smithi145 ceph-mon[33384]: 3.4 scrub ok 2024-09-11T08:43:21.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:20 smithi145 ceph-mon[33384]: 7.0 scrub ok 2024-09-11T08:43:21.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:20 smithi145 ceph-mon[33384]: 5.4 scrub starts 2024-09-11T08:43:21.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:20 smithi145 ceph-mon[33384]: 5.4 scrub ok 2024-09-11T08:43:21.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:20 smithi145 ceph-mon[33384]: 5.b scrub starts 2024-09-11T08:43:21.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:20 smithi145 ceph-mon[33384]: 5.b scrub ok 2024-09-11T08:43:21.186 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:43:21.186 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:21.220 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:21.225 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:43:21.225 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:21.251 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:43:21.252 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:21.271 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:21.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:21.272 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:21.283 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:43:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:21.312 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:43:21.312 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:21.344 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:43:21.345 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:21.365 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-11T08:43:21.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:21.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:21.417 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:21.417 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:21.518 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:21.552 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:21.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:21.553 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:21.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:43:21] "GET /metrics HTTP/1.1" 200 223767 "" "Prometheus/2.43.0" 2024-09-11T08:43:21.655 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:21.691 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:43:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:21.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:21.695 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:21.796 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:21.823 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:21.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:21.824 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:21.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:21.963 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:21.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:21.964 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:22.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:22.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[41114]: 3.0 scrub starts 2024-09-11T08:43:22.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[41114]: 3.0 scrub ok 2024-09-11T08:43:22.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[41114]: 4.12 scrub starts 2024-09-11T08:43:22.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[41114]: 4.12 scrub ok 2024-09-11T08:43:22.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[41114]: 4.e scrub starts 2024-09-11T08:43:22.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[41114]: 4.e scrub ok 2024-09-11T08:43:22.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2715498929' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:22.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[41114]: pgmap v178: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T08:43:22.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[41114]: 5.6 scrub starts 2024-09-11T08:43:22.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[41114]: 5.6 scrub ok 2024-09-11T08:43:22.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[29647]: 3.0 scrub starts 2024-09-11T08:43:22.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[29647]: 3.0 scrub ok 2024-09-11T08:43:22.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[29647]: 4.12 scrub starts 2024-09-11T08:43:22.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[29647]: 4.12 scrub ok 2024-09-11T08:43:22.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[29647]: 4.e scrub starts 2024-09-11T08:43:22.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[29647]: 4.e scrub ok 2024-09-11T08:43:22.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2715498929' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:22.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[29647]: pgmap v178: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T08:43:22.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[29647]: 5.6 scrub starts 2024-09-11T08:43:22.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:21 smithi092 ceph-mon[29647]: 5.6 scrub ok 2024-09-11T08:43:22.087 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:22.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:22.088 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:22.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:21 smithi145 ceph-mon[33384]: 3.0 scrub starts 2024-09-11T08:43:22.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:21 smithi145 ceph-mon[33384]: 3.0 scrub ok 2024-09-11T08:43:22.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:21 smithi145 ceph-mon[33384]: 4.12 scrub starts 2024-09-11T08:43:22.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:21 smithi145 ceph-mon[33384]: 4.12 scrub ok 2024-09-11T08:43:22.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:21 smithi145 ceph-mon[33384]: 4.e scrub starts 2024-09-11T08:43:22.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:21 smithi145 ceph-mon[33384]: 4.e scrub ok 2024-09-11T08:43:22.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:21 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2715498929' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:22.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:21 smithi145 ceph-mon[33384]: pgmap v178: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T08:43:22.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:21 smithi145 ceph-mon[33384]: 5.6 scrub starts 2024-09-11T08:43:22.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:21 smithi145 ceph-mon[33384]: 5.6 scrub ok 2024-09-11T08:43:22.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:22.228 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:22.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:22.229 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:22.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:22.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:22.367 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:22.468 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:22.498 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:22.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:22.499 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:22.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:43:22.599 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:22.625 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:22.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:22.626 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:22.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:22.791 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:22.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:22.793 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:22.892 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:43:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:43:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:22.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:22.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:22.938 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:22.938 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:23.039 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:23.065 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:23.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:23.067 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:23.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:22 smithi145 ceph-mon[33384]: 6.2 scrub starts 2024-09-11T08:43:23.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:22 smithi145 ceph-mon[33384]: 6.2 scrub ok 2024-09-11T08:43:23.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:22 smithi145 ceph-mon[33384]: 6.3 scrub starts 2024-09-11T08:43:23.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:22 smithi145 ceph-mon[33384]: 6.3 scrub ok 2024-09-11T08:43:23.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:22 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T08:43:23.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:22 smithi145 ceph-mon[33384]: 5.d scrub ok 2024-09-11T08:43:23.168 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:23.206 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:23.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:23.207 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:23.302 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:22 smithi092 ceph-mon[29647]: 6.2 scrub starts 2024-09-11T08:43:23.302 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:22 smithi092 ceph-mon[29647]: 6.2 scrub ok 2024-09-11T08:43:23.302 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:22 smithi092 ceph-mon[29647]: 6.3 scrub starts 2024-09-11T08:43:23.302 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:22 smithi092 ceph-mon[29647]: 6.3 scrub ok 2024-09-11T08:43:23.302 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:22 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T08:43:23.302 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:22 smithi092 ceph-mon[29647]: 5.d scrub ok 2024-09-11T08:43:23.302 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:22 smithi092 ceph-mon[41114]: 6.2 scrub starts 2024-09-11T08:43:23.302 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:22 smithi092 ceph-mon[41114]: 6.2 scrub ok 2024-09-11T08:43:23.303 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:22 smithi092 ceph-mon[41114]: 6.3 scrub starts 2024-09-11T08:43:23.303 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:22 smithi092 ceph-mon[41114]: 6.3 scrub ok 2024-09-11T08:43:23.303 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:22 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T08:43:23.303 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:22 smithi092 ceph-mon[41114]: 5.d scrub ok 2024-09-11T08:43:23.309 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:23.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:23.336 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:23.337 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:23.415 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:23.438 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:23.465 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:23.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:23.466 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:23.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:23.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:23.617 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:23.718 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:23.744 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:23.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:23.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:23.846 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:23.870 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:23.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:23.872 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:23.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:24.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:24.011 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:24.085 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:23 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:43:24.085 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:23 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:43:24.086 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:23 smithi145 ceph-mon[33384]: osdmap e122: 8 total, 8 up, 8 in 2024-09-11T08:43:24.086 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:23 smithi145 ceph-mon[33384]: 7.a scrub starts 2024-09-11T08:43:24.086 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:23 smithi145 ceph-mon[33384]: pgmap v180: 148 pgs: 148 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 820 B/s rd, 0 B/s wr, 870 op/s 2024-09-11T08:43:24.086 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:23 smithi145 ceph-mon[33384]: 5.2 scrub starts 2024-09-11T08:43:24.086 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:23 smithi145 ceph-mon[33384]: 5.2 scrub ok 2024-09-11T08:43:24.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:24.153 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:24.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:24.155 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:24.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:43:24.255 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:24.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:23 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:43:24.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:23 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:43:24.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:23 smithi092 ceph-mon[41114]: osdmap e122: 8 total, 8 up, 8 in 2024-09-11T08:43:24.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:23 smithi092 ceph-mon[41114]: 7.a scrub starts 2024-09-11T08:43:24.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:23 smithi092 ceph-mon[41114]: pgmap v180: 148 pgs: 148 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 820 B/s rd, 0 B/s wr, 870 op/s 2024-09-11T08:43:24.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:23 smithi092 ceph-mon[41114]: 5.2 scrub starts 2024-09-11T08:43:24.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:23 smithi092 ceph-mon[41114]: 5.2 scrub ok 2024-09-11T08:43:24.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:23 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:43:24.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:23 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:43:24.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:23 smithi092 ceph-mon[29647]: osdmap e122: 8 total, 8 up, 8 in 2024-09-11T08:43:24.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:23 smithi092 ceph-mon[29647]: 7.a scrub starts 2024-09-11T08:43:24.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:23 smithi092 ceph-mon[29647]: pgmap v180: 148 pgs: 148 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 820 B/s rd, 0 B/s wr, 870 op/s 2024-09-11T08:43:24.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:23 smithi092 ceph-mon[29647]: 5.2 scrub starts 2024-09-11T08:43:24.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:23 smithi092 ceph-mon[29647]: 5.2 scrub ok 2024-09-11T08:43:24.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:24.285 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:24.285 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:24.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:24.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:24.426 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:24.426 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:24.459 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:43:24.459 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T08:43:24.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T08:43:24.459 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:43:24.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.460 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:115418 7.7 7:eb0976f3:::benchmark_data_smithi092_84565_object115417:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:43:24.460 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:31.169655+0000", 2024-09-11T08:43:24.460 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 53.286011039999998, 2024-09-11T08:43:24.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.116402226, 2024-09-11T08:43:24.460 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.460 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.460 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.460 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.460 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.460 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 115418 2024-09-11T08:43:24.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.460 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.169655+0000", 2024-09-11T08:43:24.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.169655+0000", 2024-09-11T08:43:24.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.169653+0000", 2024-09-11T08:43:24.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:43:24.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.169724+0000", 2024-09-11T08:43:24.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.0636999999999995e-05 2024-09-11T08:43:24.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.169740+0000", 2024-09-11T08:43:24.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6290000000000002e-05 2024-09-11T08:43:24.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.169747+0000", 2024-09-11T08:43:24.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6239999999999996e-06 2024-09-11T08:43:24.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.170207+0000", 2024-09-11T08:43:24.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00046066800000000001 2024-09-11T08:43:24.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.170323+0000", 2024-09-11T08:43:24.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000115939 2024-09-11T08:43:24.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,3", 2024-09-11T08:43:24.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.170468+0000", 2024-09-11T08:43:24.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000144876 2024-09-11T08:43:24.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.173268+0000", 2024-09-11T08:43:24.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0028000450000000001 2024-09-11T08:43:24.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.182445+0000", 2024-09-11T08:43:24.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0091763790000000001 2024-09-11T08:43:24.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.285955+0000", 2024-09-11T08:43:24.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.103510519 2024-09-11T08:43:24.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.286011+0000", 2024-09-11T08:43:24.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5918000000000002e-05 2024-09-11T08:43:24.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.286057+0000", 2024-09-11T08:43:24.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6059000000000001e-05 2024-09-11T08:43:24.465 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.465 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.465 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.466 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:115424 7.7 7:e3afe89e:::benchmark_data_smithi092_84565_object115423:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:43:24.466 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:31.177358+0000", 2024-09-11T08:43:24.466 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 53.278308318000001, 2024-09-11T08:43:24.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10883133, 2024-09-11T08:43:24.466 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.466 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.466 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.466 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.466 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.466 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 115424 2024-09-11T08:43:24.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.466 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177358+0000", 2024-09-11T08:43:24.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.467 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177358+0000", 2024-09-11T08:43:24.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.467 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177356+0000", 2024-09-11T08:43:24.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:43:24.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177384+0000", 2024-09-11T08:43:24.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.815e-05 2024-09-11T08:43:24.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177398+0000", 2024-09-11T08:43:24.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4226000000000001e-05 2024-09-11T08:43:24.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177403+0000", 2024-09-11T08:43:24.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0830000000000003e-06 2024-09-11T08:43:24.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.469 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.469 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177440+0000", 2024-09-11T08:43:24.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6229999999999997e-05 2024-09-11T08:43:24.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.469 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.469 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177523+0000", 2024-09-11T08:43:24.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.3542999999999996e-05 2024-09-11T08:43:24.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,3", 2024-09-11T08:43:24.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177612+0000", 2024-09-11T08:43:24.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8450999999999997e-05 2024-09-11T08:43:24.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.181788+0000", 2024-09-11T08:43:24.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0041767280000000002 2024-09-11T08:43:24.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.185739+0000", 2024-09-11T08:43:24.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003950963 2024-09-11T08:43:24.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.286106+0000", 2024-09-11T08:43:24.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.100366205 2024-09-11T08:43:24.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.286151+0000", 2024-09-11T08:43:24.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5905000000000002e-05 2024-09-11T08:43:24.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.286189+0000", 2024-09-11T08:43:24.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7607000000000001e-05 2024-09-11T08:43:24.472 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.472 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.472 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.472 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:115425 7.7 7:e4dc9892:::benchmark_data_smithi092_84565_object115424:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:43:24.472 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:31.177874+0000", 2024-09-11T08:43:24.472 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 53.277792253000001, 2024-09-11T08:43:24.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.11603140200000001, 2024-09-11T08:43:24.472 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.473 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.473 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.473 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.473 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.473 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 115425 2024-09-11T08:43:24.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.473 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177874+0000", 2024-09-11T08:43:24.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177874+0000", 2024-09-11T08:43:24.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177873+0000", 2024-09-11T08:43:24.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:43:24.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177919+0000", 2024-09-11T08:43:24.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6035000000000002e-05 2024-09-11T08:43:24.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177930+0000", 2024-09-11T08:43:24.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1321999999999999e-05 2024-09-11T08:43:24.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177935+0000", 2024-09-11T08:43:24.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5029999999999999e-06 2024-09-11T08:43:24.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.179993+0000", 2024-09-11T08:43:24.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0020583189999999999 2024-09-11T08:43:24.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.180067+0000", 2024-09-11T08:43:24.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4243000000000001e-05 2024-09-11T08:43:24.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,3", 2024-09-11T08:43:24.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.180152+0000", 2024-09-11T08:43:24.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.4531999999999996e-05 2024-09-11T08:43:24.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.181892+0000", 2024-09-11T08:43:24.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0017399970000000001 2024-09-11T08:43:24.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.185772+0000", 2024-09-11T08:43:24.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0038804880000000001 2024-09-11T08:43:24.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.293815+0000", 2024-09-11T08:43:24.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.108043282 2024-09-11T08:43:24.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.293870+0000", 2024-09-11T08:43:24.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4389999999999999e-05 2024-09-11T08:43:24.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.293905+0000", 2024-09-11T08:43:24.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5357e-05 2024-09-11T08:43:24.478 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.478 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.479 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.479 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.479 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.479 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122193 7.d 7:b3845457:::benchmark_data_smithi092_84565_object122192:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:43:24.479 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.173551+0000", 2024-09-11T08:43:24.479 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 46.282114845000002, 2024-09-11T08:43:24.479 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10099198600000001, 2024-09-11T08:43:24.479 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.479 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.479 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.479 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.479 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.479 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122193 2024-09-11T08:43:24.479 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.480 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.480 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.480 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.480 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.173551+0000", 2024-09-11T08:43:24.480 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.480 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.480 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.480 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.173551+0000", 2024-09-11T08:43:24.480 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.480 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.481 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.173550+0000", 2024-09-11T08:43:24.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:43:24.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.481 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.173710+0000", 2024-09-11T08:43:24.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015988299999999999 2024-09-11T08:43:24.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.481 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.173726+0000", 2024-09-11T08:43:24.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6319000000000001e-05 2024-09-11T08:43:24.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.173731+0000", 2024-09-11T08:43:24.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6539999999999998e-06 2024-09-11T08:43:24.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.174092+0000", 2024-09-11T08:43:24.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00036093099999999999 2024-09-11T08:43:24.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.174216+0000", 2024-09-11T08:43:24.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012413699999999999 2024-09-11T08:43:24.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.174311+0000", 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.4907000000000005e-05 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.177466+0000", 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0031548879999999998 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.181038+0000", 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00357267 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.274472+0000", 2024-09-11T08:43:24.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.093433547000000006 2024-09-11T08:43:24.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.484 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.484 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.274518+0000", 2024-09-11T08:43:24.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5716000000000001e-05 2024-09-11T08:43:24.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.484 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.484 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.274543+0000", 2024-09-11T08:43:24.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5432999999999999e-05 2024-09-11T08:43:24.484 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.484 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.485 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.485 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.485 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.485 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122205 7.d 7:b6835570:::benchmark_data_smithi092_84565_object122204:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:43:24.485 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.181777+0000", 2024-09-11T08:43:24.485 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 46.273888794000001, 2024-09-11T08:43:24.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10647788399999999, 2024-09-11T08:43:24.485 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.485 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.485 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.485 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.485 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.485 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122205 2024-09-11T08:43:24.485 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.486 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.181777+0000", 2024-09-11T08:43:24.486 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.181777+0000", 2024-09-11T08:43:24.486 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.181776+0000", 2024-09-11T08:43:24.487 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:43:24.487 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.181808+0000", 2024-09-11T08:43:24.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1843e-05 2024-09-11T08:43:24.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.181820+0000", 2024-09-11T08:43:24.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2157999999999999e-05 2024-09-11T08:43:24.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.181825+0000", 2024-09-11T08:43:24.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4909999999999996e-06 2024-09-11T08:43:24.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.181867+0000", 2024-09-11T08:43:24.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1916999999999999e-05 2024-09-11T08:43:24.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.182000+0000", 2024-09-11T08:43:24.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013226199999999999 2024-09-11T08:43:24.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:43:24.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.182142+0000", 2024-09-11T08:43:24.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014258000000000001 2024-09-11T08:43:24.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.185449+0000", 2024-09-11T08:43:24.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0033071989999999998 2024-09-11T08:43:24.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.243415+0000", 2024-09-11T08:43:24.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.057965847000000001 2024-09-11T08:43:24.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.491 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.491 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.491 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.288188+0000", 2024-09-11T08:43:24.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.044772431000000001 2024-09-11T08:43:24.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.491 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.491 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.491 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.288227+0000", 2024-09-11T08:43:24.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9731999999999998e-05 2024-09-11T08:43:24.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.491 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.491 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.491 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.288255+0000", 2024-09-11T08:43:24.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7753e-05 2024-09-11T08:43:24.492 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.492 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.492 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.492 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.492 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122657 7.d 7:be93a7fa:::benchmark_data_smithi092_84565_object122656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:43:24.492 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.699124+0000", 2024-09-11T08:43:24.492 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 45.756541886000001, 2024-09-11T08:43:24.492 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.099699880000000005, 2024-09-11T08:43:24.492 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.492 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.492 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.492 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.492 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.493 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122657 2024-09-11T08:43:24.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.493 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.493 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.493 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.493 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.699124+0000", 2024-09-11T08:43:24.493 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.493 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.493 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.493 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.699124+0000", 2024-09-11T08:43:24.493 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.494 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.494 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.699123+0000", 2024-09-11T08:43:24.494 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:43:24.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.494 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.494 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.699203+0000", 2024-09-11T08:43:24.494 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.0150999999999999e-05 2024-09-11T08:43:24.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.494 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.494 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.699215+0000", 2024-09-11T08:43:24.494 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1459e-05 2024-09-11T08:43:24.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.699221+0000", 2024-09-11T08:43:24.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2779999999999999e-06 2024-09-11T08:43:24.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.699269+0000", 2024-09-11T08:43:24.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8214000000000003e-05 2024-09-11T08:43:24.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.699392+0000", 2024-09-11T08:43:24.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000122454 2024-09-11T08:43:24.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:43:24.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.699544+0000", 2024-09-11T08:43:24.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000152669 2024-09-11T08:43:24.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.705547+0000", 2024-09-11T08:43:24.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0060028549999999996 2024-09-11T08:43:24.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.497 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.774617+0000", 2024-09-11T08:43:24.497 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.069069869000000006 2024-09-11T08:43:24.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.497 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.497 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.497 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.798733+0000", 2024-09-11T08:43:24.497 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.024115806 2024-09-11T08:43:24.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.497 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.497 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.497 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.798781+0000", 2024-09-11T08:43:24.498 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8492000000000003e-05 2024-09-11T08:43:24.498 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.498 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.498 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.498 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.798824+0000", 2024-09-11T08:43:24.498 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2728000000000002e-05 2024-09-11T08:43:24.498 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.498 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.498 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.498 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.498 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.498 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122658 7.d 7:b25e6c52:::benchmark_data_smithi092_84565_object122657:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:43:24.498 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.699235+0000", 2024-09-11T08:43:24.498 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 45.756430874000003, 2024-09-11T08:43:24.498 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.106644026, 2024-09-11T08:43:24.499 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.499 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.499 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.499 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.499 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.499 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122658 2024-09-11T08:43:24.499 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.499 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.499 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.499 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.499 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.699235+0000", 2024-09-11T08:43:24.499 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.499 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.499 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.499 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.500 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.699235+0000", 2024-09-11T08:43:24.500 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.500 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.500 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.500 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.699234+0000", 2024-09-11T08:43:24.500 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:43:24.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.500 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.500 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.500 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.699278+0000", 2024-09-11T08:43:24.500 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3723999999999997e-05 2024-09-11T08:43:24.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.500 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.500 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.501 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.699285+0000", 2024-09-11T08:43:24.501 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5629999999999997e-06 2024-09-11T08:43:24.501 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.501 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.501 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.501 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.699287+0000", 2024-09-11T08:43:24.501 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.751e-06 2024-09-11T08:43:24.501 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.501 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.501 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.501 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702342+0000", 2024-09-11T08:43:24.501 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003054991 2024-09-11T08:43:24.501 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.501 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.502 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.502 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702434+0000", 2024-09-11T08:43:24.502 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.1712000000000006e-05 2024-09-11T08:43:24.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.502 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.502 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:43:24.502 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702532+0000", 2024-09-11T08:43:24.502 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.7770999999999996e-05 2024-09-11T08:43:24.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.502 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.502 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.502 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.708682+0000", 2024-09-11T08:43:24.502 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0061501749999999999 2024-09-11T08:43:24.503 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.503 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.503 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.503 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.774685+0000", 2024-09-11T08:43:24.503 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.066002569999999997 2024-09-11T08:43:24.503 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.503 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.503 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.503 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.805771+0000", 2024-09-11T08:43:24.503 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.031086560999999999 2024-09-11T08:43:24.503 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.504 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.504 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.504 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.805845+0000", 2024-09-11T08:43:24.504 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4128999999999994e-05 2024-09-11T08:43:24.504 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.504 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.504 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.504 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.805879+0000", 2024-09-11T08:43:24.504 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3815000000000001e-05 2024-09-11T08:43:24.504 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.504 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.504 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.504 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.505 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.505 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122659 7.c 7:37de432f:::benchmark_data_smithi092_84565_object122658:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:43:24.505 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.702815+0000", 2024-09-11T08:43:24.505 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 45.752850975999998, 2024-09-11T08:43:24.505 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.11031606400000001, 2024-09-11T08:43:24.505 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.505 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.505 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.505 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.505 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.505 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122659 2024-09-11T08:43:24.505 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.505 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.506 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.506 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.506 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702815+0000", 2024-09-11T08:43:24.506 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.506 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.506 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.506 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702815+0000", 2024-09-11T08:43:24.506 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.506 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.506 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.506 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702814+0000", 2024-09-11T08:43:24.506 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:43:24.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.507 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.507 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.507 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702886+0000", 2024-09-11T08:43:24.507 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2298999999999993e-05 2024-09-11T08:43:24.507 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.507 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.507 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.507 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702899+0000", 2024-09-11T08:43:24.507 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2330000000000001e-05 2024-09-11T08:43:24.507 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.507 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.507 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.507 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702905+0000", 2024-09-11T08:43:24.508 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1399999999999997e-06 2024-09-11T08:43:24.508 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.508 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.508 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.508 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702948+0000", 2024-09-11T08:43:24.508 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2651999999999998e-05 2024-09-11T08:43:24.508 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.508 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.508 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.508 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.703071+0000", 2024-09-11T08:43:24.508 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000123812 2024-09-11T08:43:24.508 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.508 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.509 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,7", 2024-09-11T08:43:24.509 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.703227+0000", 2024-09-11T08:43:24.509 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000155699 2024-09-11T08:43:24.509 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.509 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.509 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.509 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.706692+0000", 2024-09-11T08:43:24.509 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0034647789999999999 2024-09-11T08:43:24.509 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.509 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.509 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.509 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.711638+0000", 2024-09-11T08:43:24.509 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.004945952 2024-09-11T08:43:24.509 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.813048+0000", 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.101410371 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.813097+0000", 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8430999999999998e-05 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.813131+0000", 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4517999999999997e-05 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.510 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122664 7.d 7:b030b7ee:::benchmark_data_smithi092_84565_object122663:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.702916+0000", 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 45.752749532000003, 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.106758768, 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122664 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.511 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.512 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.512 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702916+0000", 2024-09-11T08:43:24.512 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.512 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.512 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.512 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.512 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702916+0000", 2024-09-11T08:43:24.512 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.512 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.512 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.512 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.512 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702916+0000", 2024-09-11T08:43:24.512 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:43:24.512 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.512 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.513 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.513 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702948+0000", 2024-09-11T08:43:24.513 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2489e-05 2024-09-11T08:43:24.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.513 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.513 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.513 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702962+0000", 2024-09-11T08:43:24.513 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3122000000000001e-05 2024-09-11T08:43:24.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.513 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.513 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.513 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702967+0000", 2024-09-11T08:43:24.513 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0359999999999997e-06 2024-09-11T08:43:24.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.513 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.514 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.514 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.705587+0000", 2024-09-11T08:43:24.514 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0026201620000000001 2024-09-11T08:43:24.514 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.514 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.514 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.514 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.705728+0000", 2024-09-11T08:43:24.514 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000141407 2024-09-11T08:43:24.514 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.514 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.514 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:43:24.514 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.705891+0000", 2024-09-11T08:43:24.514 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00016304000000000001 2024-09-11T08:43:24.514 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.514 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.515 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.515 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.708839+0000", 2024-09-11T08:43:24.515 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.002947613 2024-09-11T08:43:24.515 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.515 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.515 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.515 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.774749+0000", 2024-09-11T08:43:24.515 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.065909764999999995 2024-09-11T08:43:24.515 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.515 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.515 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.515 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.809590+0000", 2024-09-11T08:43:24.515 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.034841641999999999 2024-09-11T08:43:24.515 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.515 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.516 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.516 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.809645+0000", 2024-09-11T08:43:24.516 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4506000000000003e-05 2024-09-11T08:43:24.516 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.516 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.516 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.516 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.809675+0000", 2024-09-11T08:43:24.516 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0537999999999998e-05 2024-09-11T08:43:24.516 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.516 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.516 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.516 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.516 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.517 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122670 7.7 7:e7a085d3:::benchmark_data_smithi092_84565_object122669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:43:24.517 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.707082+0000", 2024-09-11T08:43:24.517 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 45.748583760000002, 2024-09-11T08:43:24.517 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.114477678, 2024-09-11T08:43:24.517 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.517 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.517 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.517 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.517 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.517 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122670 2024-09-11T08:43:24.517 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.517 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.517 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.517 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.518 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707082+0000", 2024-09-11T08:43:24.518 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.518 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.518 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.518 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.518 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707082+0000", 2024-09-11T08:43:24.518 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.518 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.518 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.518 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.518 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707072+0000", 2024-09-11T08:43:24.518 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999895 2024-09-11T08:43:24.518 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.518 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.519 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.519 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707134+0000", 2024-09-11T08:43:24.519 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2637000000000004e-05 2024-09-11T08:43:24.519 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.519 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.519 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.519 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707148+0000", 2024-09-11T08:43:24.519 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3801999999999999e-05 2024-09-11T08:43:24.519 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.519 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.519 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.519 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707153+0000", 2024-09-11T08:43:24.519 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1939999999999997e-06 2024-09-11T08:43:24.519 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.519 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.520 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.520 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707202+0000", 2024-09-11T08:43:24.520 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8974000000000002e-05 2024-09-11T08:43:24.520 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.520 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.520 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.520 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707333+0000", 2024-09-11T08:43:24.520 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013053300000000001 2024-09-11T08:43:24.520 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.520 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.520 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,3", 2024-09-11T08:43:24.520 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707436+0000", 2024-09-11T08:43:24.520 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010262700000000001 2024-09-11T08:43:24.520 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.521 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.521 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.521 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.710596+0000", 2024-09-11T08:43:24.521 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0031604189999999998 2024-09-11T08:43:24.521 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.521 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.521 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.521 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.809030+0000", 2024-09-11T08:43:24.521 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.098434113000000004 2024-09-11T08:43:24.521 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.521 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.521 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.521 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.821512+0000", 2024-09-11T08:43:24.521 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.012481628999999999 2024-09-11T08:43:24.521 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.522 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.522 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.522 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.821539+0000", 2024-09-11T08:43:24.522 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7396e-05 2024-09-11T08:43:24.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.522 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.522 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.522 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.821560+0000", 2024-09-11T08:43:24.522 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0849e-05 2024-09-11T08:43:24.522 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.522 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.522 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.522 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.522 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122672 7.c 7:30125efa:::benchmark_data_smithi092_84565_object122671:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:43:24.523 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.707596+0000", 2024-09-11T08:43:24.523 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 45.748069848999997, 2024-09-11T08:43:24.523 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.113226995, 2024-09-11T08:43:24.523 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.523 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.523 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.523 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.523 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.523 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122672 2024-09-11T08:43:24.523 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.523 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.523 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.523 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.523 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707596+0000", 2024-09-11T08:43:24.523 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.524 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.524 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.524 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.524 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707596+0000", 2024-09-11T08:43:24.524 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.524 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.524 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.524 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.524 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707595+0000", 2024-09-11T08:43:24.524 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:43:24.524 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.524 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.524 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.524 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707624+0000", 2024-09-11T08:43:24.524 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.8662000000000001e-05 2024-09-11T08:43:24.525 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.525 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.525 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.525 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707638+0000", 2024-09-11T08:43:24.525 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3611000000000001e-05 2024-09-11T08:43:24.525 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.525 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.525 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.525 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707642+0000", 2024-09-11T08:43:24.525 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6689999999999997e-06 2024-09-11T08:43:24.525 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.525 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.525 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.525 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707681+0000", 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8810000000000003e-05 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707776+0000", 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.4856000000000001e-05 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,7", 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707900+0000", 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000124156 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.710823+0000", 2024-09-11T08:43:24.526 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0029230459999999999 2024-09-11T08:43:24.527 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.527 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.527 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.527 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.711721+0000", 2024-09-11T08:43:24.527 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000897385 2024-09-11T08:43:24.527 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.527 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.527 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.527 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.820742+0000", 2024-09-11T08:43:24.527 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10902121300000001 2024-09-11T08:43:24.527 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.527 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.527 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.527 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.820787+0000", 2024-09-11T08:43:24.527 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5170000000000003e-05 2024-09-11T08:43:24.528 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.528 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.528 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.528 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.820823+0000", 2024-09-11T08:43:24.528 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6226999999999997e-05 2024-09-11T08:43:24.528 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.528 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.528 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.528 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.528 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.528 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:149681 7.d 7:b6ab2c94:::benchmark_data_smithi092_84565_object24153:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:43:24.528 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:06.851807+0000", 2024-09-11T08:43:24.528 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 17.603858595999998, 2024-09-11T08:43:24.528 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.115632797, 2024-09-11T08:43:24.528 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.529 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.529 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.529 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.529 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.529 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 149681 2024-09-11T08:43:24.533 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.533 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.533 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.533 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.533 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.851807+0000", 2024-09-11T08:43:24.533 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.533 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.533 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.533 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.533 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.851807+0000", 2024-09-11T08:43:24.533 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.533 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.534 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.534 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.534 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.851807+0000", 2024-09-11T08:43:24.534 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:43:24.534 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.534 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.534 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.534 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.851811+0000", 2024-09-11T08:43:24.534 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6879999999999998e-06 2024-09-11T08:43:24.534 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.534 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.534 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.534 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.851822+0000", 2024-09-11T08:43:24.534 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0293e-05 2024-09-11T08:43:24.535 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.535 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.535 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.535 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.851827+0000", 2024-09-11T08:43:24.535 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4029999999999998e-06 2024-09-11T08:43:24.535 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.535 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.535 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.535 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.851972+0000", 2024-09-11T08:43:24.535 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014493700000000001 2024-09-11T08:43:24.535 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.535 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.535 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.535 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852343+0000", 2024-09-11T08:43:24.536 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00037107900000000002 2024-09-11T08:43:24.536 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.536 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.536 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:43:24.536 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852440+0000", 2024-09-11T08:43:24.536 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.7342000000000004e-05 2024-09-11T08:43:24.536 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.536 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.536 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.536 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.860532+0000", 2024-09-11T08:43:24.536 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0080917650000000008 2024-09-11T08:43:24.536 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.536 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.537 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.537 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.860574+0000", 2024-09-11T08:43:24.537 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1653999999999998e-05 2024-09-11T08:43:24.537 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.537 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.537 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.537 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.967384+0000", 2024-09-11T08:43:24.537 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.106809742 2024-09-11T08:43:24.537 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.537 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.537 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.537 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.967416+0000", 2024-09-11T08:43:24.537 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2274000000000003e-05 2024-09-11T08:43:24.538 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.538 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.538 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.538 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.967440+0000", 2024-09-11T08:43:24.538 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4416000000000001e-05 2024-09-11T08:43:24.538 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.538 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.538 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.538 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.538 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.538 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:149682 7.d 7:b1cf1844:::benchmark_data_smithi092_84565_object24154:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:43:24.538 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:06.852894+0000", 2024-09-11T08:43:24.538 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 17.602772069, 2024-09-11T08:43:24.538 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.114631739, 2024-09-11T08:43:24.539 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.539 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.539 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.539 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.539 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.539 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 149682 2024-09-11T08:43:24.539 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.539 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.539 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.539 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.539 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852894+0000", 2024-09-11T08:43:24.539 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.539 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.539 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.540 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.540 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852894+0000", 2024-09-11T08:43:24.540 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.540 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.540 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.540 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.540 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852893+0000", 2024-09-11T08:43:24.540 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:43:24.540 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.540 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.540 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.540 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852898+0000", 2024-09-11T08:43:24.540 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.2000000000000002e-06 2024-09-11T08:43:24.540 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.541 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.541 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.541 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852912+0000", 2024-09-11T08:43:24.541 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3471e-05 2024-09-11T08:43:24.541 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.541 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.541 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.541 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852918+0000", 2024-09-11T08:43:24.541 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8760000000000002e-06 2024-09-11T08:43:24.541 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.541 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.541 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.541 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.857394+0000", 2024-09-11T08:43:24.541 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0044762539999999998 2024-09-11T08:43:24.542 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.542 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.542 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.542 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.857536+0000", 2024-09-11T08:43:24.542 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000142022 2024-09-11T08:43:24.542 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.542 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.542 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:43:24.542 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.857661+0000", 2024-09-11T08:43:24.542 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000125069 2024-09-11T08:43:24.542 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.542 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.542 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.542 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.860677+0000", 2024-09-11T08:43:24.543 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003015804 2024-09-11T08:43:24.543 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.543 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.543 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.543 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.861044+0000", 2024-09-11T08:43:24.543 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00036762900000000001 2024-09-11T08:43:24.543 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.543 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.543 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.543 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.967467+0000", 2024-09-11T08:43:24.543 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.106422302 2024-09-11T08:43:24.543 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.543 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.543 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.544 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.967496+0000", 2024-09-11T08:43:24.544 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.8878999999999999e-05 2024-09-11T08:43:24.544 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.544 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.544 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.544 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.967526+0000", 2024-09-11T08:43:24.544 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0128999999999999e-05 2024-09-11T08:43:24.544 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.544 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.544 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.544 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.544 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.544 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:150188 7.7 7:e5c33454:::benchmark_data_smithi092_84565_object24660:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:43:24.545 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:07.397005+0000", 2024-09-11T08:43:24.545 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 17.058661251, 2024-09-11T08:43:24.545 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:24.546 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.11073683600000001, 2024-09-11T08:43:24.546 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.547 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.547 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.547 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.547 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.547 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 150188 2024-09-11T08:43:24.547 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.547 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.547 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.547 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.547 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.397005+0000", 2024-09-11T08:43:24.547 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.547 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.547 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.547 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.548 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.397005+0000", 2024-09-11T08:43:24.548 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.548 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.548 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.548 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.548 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.397003+0000", 2024-09-11T08:43:24.548 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:43:24.548 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.548 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.548 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.548 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.397012+0000", 2024-09-11T08:43:24.548 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.4440000000000008e-06 2024-09-11T08:43:24.548 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.549 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.549 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.549 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.397032+0000", 2024-09-11T08:43:24.549 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0397000000000001e-05 2024-09-11T08:43:24.549 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.549 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.549 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.549 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.397040+0000", 2024-09-11T08:43:24.549 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6090000000000003e-06 2024-09-11T08:43:24.549 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.549 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.550 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.550 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.397092+0000", 2024-09-11T08:43:24.550 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1969999999999999e-05 2024-09-11T08:43:24.550 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.550 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.550 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.550 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.397206+0000", 2024-09-11T08:43:24.550 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011472900000000001 2024-09-11T08:43:24.550 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.550 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.550 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,3", 2024-09-11T08:43:24.550 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.397319+0000", 2024-09-11T08:43:24.550 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000112339 2024-09-11T08:43:24.551 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.551 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.551 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.551 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.400903+0000", 2024-09-11T08:43:24.551 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003584739 2024-09-11T08:43:24.551 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.551 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.551 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.551 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.409207+0000", 2024-09-11T08:43:24.551 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0083031930000000004 2024-09-11T08:43:24.551 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.551 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.551 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.551 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.507686+0000", 2024-09-11T08:43:24.552 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.098479312999999999 2024-09-11T08:43:24.552 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.552 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.552 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.552 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.507715+0000", 2024-09-11T08:43:24.552 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9581999999999999e-05 2024-09-11T08:43:24.552 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.552 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.552 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.552 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.507742+0000", 2024-09-11T08:43:24.552 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.6149999999999999e-05 2024-09-11T08:43:24.552 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.552 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.553 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.553 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.553 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.553 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:150198 7.7 7:e4f1bebe:::benchmark_data_smithi092_84565_object24670:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:43:24.553 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:07.404760+0000", 2024-09-11T08:43:24.553 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 17.050905615000001, 2024-09-11T08:43:24.553 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.117250244, 2024-09-11T08:43:24.553 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.553 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.553 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.553 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.553 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.553 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 150198 2024-09-11T08:43:24.553 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.554 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.554 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.554 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.554 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404760+0000", 2024-09-11T08:43:24.554 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.554 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.554 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.554 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.554 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404760+0000", 2024-09-11T08:43:24.554 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.554 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.554 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.554 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.554 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404759+0000", 2024-09-11T08:43:24.555 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:43:24.555 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.555 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.555 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.555 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404767+0000", 2024-09-11T08:43:24.555 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7179999999999996e-06 2024-09-11T08:43:24.555 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.555 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.555 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.555 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404781+0000", 2024-09-11T08:43:24.555 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4462e-05 2024-09-11T08:43:24.555 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.555 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.555 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.556 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404787+0000", 2024-09-11T08:43:24.556 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4720000000000003e-06 2024-09-11T08:43:24.556 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.556 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.556 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.556 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404832+0000", 2024-09-11T08:43:24.556 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5782999999999999e-05 2024-09-11T08:43:24.556 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.556 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.556 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.556 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404977+0000", 2024-09-11T08:43:24.556 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014502100000000001 2024-09-11T08:43:24.556 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.557 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.557 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,3", 2024-09-11T08:43:24.557 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.405110+0000", 2024-09-11T08:43:24.557 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013258700000000001 2024-09-11T08:43:24.557 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.557 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.557 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.557 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.408834+0000", 2024-09-11T08:43:24.557 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003723933 2024-09-11T08:43:24.557 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.557 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.557 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.557 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.409512+0000", 2024-09-11T08:43:24.557 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00067832799999999996 2024-09-11T08:43:24.558 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.558 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.558 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.558 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.521917+0000", 2024-09-11T08:43:24.558 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.11240472999999999 2024-09-11T08:43:24.558 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.558 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.558 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.558 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.521975+0000", 2024-09-11T08:43:24.558 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7726999999999998e-05 2024-09-11T08:43:24.558 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.558 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.558 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.558 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.522011+0000", 2024-09-11T08:43:24.558 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5930000000000003e-05 2024-09-11T08:43:24.559 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.559 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.559 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.559 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.559 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.559 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:150331 7.d 7:b45ac87d:::benchmark_data_smithi092_84565_object24803:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:43:24.559 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:07.601826+0000", 2024-09-11T08:43:24.559 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 16.853839582999999, 2024-09-11T08:43:24.559 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.199883954, 2024-09-11T08:43:24.559 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.559 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.559 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.559 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.559 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.560 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 150331 2024-09-11T08:43:24.560 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.560 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.560 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.560 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.560 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.601826+0000", 2024-09-11T08:43:24.560 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.560 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.560 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.560 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.560 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.601826+0000", 2024-09-11T08:43:24.560 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.560 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.561 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.561 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.561 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.601826+0000", 2024-09-11T08:43:24.561 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:43:24.561 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.561 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.561 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.561 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.601829+0000", 2024-09-11T08:43:24.561 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2940000000000001e-06 2024-09-11T08:43:24.561 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.561 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.561 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.561 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.601838+0000", 2024-09-11T08:43:24.562 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.6319999999999997e-06 2024-09-11T08:43:24.562 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.562 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.562 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.562 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.601842+0000", 2024-09-11T08:43:24.562 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0339999999999998e-06 2024-09-11T08:43:24.562 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.562 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.562 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.562 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.601890+0000", 2024-09-11T08:43:24.562 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8466e-05 2024-09-11T08:43:24.562 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.562 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.562 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.562 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.602002+0000", 2024-09-11T08:43:24.563 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000112199 2024-09-11T08:43:24.563 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.563 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.563 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:43:24.563 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.602104+0000", 2024-09-11T08:43:24.563 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000101413 2024-09-11T08:43:24.563 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.563 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.563 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.563 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.605993+0000", 2024-09-11T08:43:24.563 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0038893019999999999 2024-09-11T08:43:24.563 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.563 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.563 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.564 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.606054+0000", 2024-09-11T08:43:24.564 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0368999999999997e-05 2024-09-11T08:43:24.564 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.564 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.564 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.564 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801629+0000", 2024-09-11T08:43:24.564 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.195575569 2024-09-11T08:43:24.564 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.564 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.564 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.564 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801670+0000", 2024-09-11T08:43:24.564 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1189999999999997e-05 2024-09-11T08:43:24.564 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.565 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.565 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.565 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801710+0000", 2024-09-11T08:43:24.565 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0058999999999997e-05 2024-09-11T08:43:24.565 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.565 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.565 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.565 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.565 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.565 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:150339 7.d 7:b48099e1:::benchmark_data_smithi092_84565_object24811:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:43:24.565 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:07.610231+0000", 2024-09-11T08:43:24.565 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 16.845435194, 2024-09-11T08:43:24.565 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.19162894, 2024-09-11T08:43:24.566 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.566 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.566 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.566 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.566 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.566 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 150339 2024-09-11T08:43:24.566 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.566 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.566 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.566 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.566 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610231+0000", 2024-09-11T08:43:24.566 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.566 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.566 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.567 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.567 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610231+0000", 2024-09-11T08:43:24.567 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.567 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.567 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.567 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.567 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610230+0000", 2024-09-11T08:43:24.567 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:43:24.567 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.567 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.567 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.567 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610235+0000", 2024-09-11T08:43:24.567 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3020000000000002e-06 2024-09-11T08:43:24.567 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.568 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.568 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.568 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610246+0000", 2024-09-11T08:43:24.568 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0465999999999999e-05 2024-09-11T08:43:24.568 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.568 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.568 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.568 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610250+0000", 2024-09-11T08:43:24.568 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2379999999999999e-06 2024-09-11T08:43:24.568 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.568 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.568 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.568 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610325+0000", 2024-09-11T08:43:24.569 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4641000000000006e-05 2024-09-11T08:43:24.569 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.569 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.569 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.569 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610514+0000", 2024-09-11T08:43:24.569 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00018959599999999999 2024-09-11T08:43:24.569 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.569 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.569 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:43:24.569 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610635+0000", 2024-09-11T08:43:24.569 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012111499999999999 2024-09-11T08:43:24.569 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.569 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.569 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.570 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.616121+0000", 2024-09-11T08:43:24.570 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0054855390000000002 2024-09-11T08:43:24.570 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.570 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.570 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.570 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.616739+0000", 2024-09-11T08:43:24.570 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00061781100000000001 2024-09-11T08:43:24.570 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.570 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.571 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.571 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801782+0000", 2024-09-11T08:43:24.571 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.185043292 2024-09-11T08:43:24.571 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.571 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.571 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.571 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801827+0000", 2024-09-11T08:43:24.571 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5318000000000002e-05 2024-09-11T08:43:24.571 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.571 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.571 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.571 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801860+0000", 2024-09-11T08:43:24.571 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2393e-05 2024-09-11T08:43:24.571 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.572 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.572 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.572 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.572 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.572 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:150344 7.d 7:bc2fa16d:::benchmark_data_smithi092_84565_object24816:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:43:24.572 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:07.613435+0000", 2024-09-11T08:43:24.572 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 16.842230581999999, 2024-09-11T08:43:24.572 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.18853381399999999, 2024-09-11T08:43:24.572 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.572 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.572 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.572 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.572 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.573 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 150344 2024-09-11T08:43:24.573 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.573 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.573 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.573 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.573 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613435+0000", 2024-09-11T08:43:24.573 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.573 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.573 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.573 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.573 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613435+0000", 2024-09-11T08:43:24.573 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.573 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.574 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.574 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.574 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613435+0000", 2024-09-11T08:43:24.574 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:43:24.574 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.574 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.574 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.574 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613439+0000", 2024-09-11T08:43:24.574 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2139999999999998e-06 2024-09-11T08:43:24.574 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.574 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.574 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.574 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613446+0000", 2024-09-11T08:43:24.575 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.3259999999999998e-06 2024-09-11T08:43:24.575 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.575 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.575 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.575 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613449+0000", 2024-09-11T08:43:24.575 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2480000000000001e-06 2024-09-11T08:43:24.575 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.575 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.575 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.575 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613623+0000", 2024-09-11T08:43:24.575 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00017342599999999999 2024-09-11T08:43:24.575 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.575 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.575 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.576 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613752+0000", 2024-09-11T08:43:24.576 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000129467 2024-09-11T08:43:24.576 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.576 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.576 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:43:24.576 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613849+0000", 2024-09-11T08:43:24.576 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.6953999999999994e-05 2024-09-11T08:43:24.576 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.576 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.576 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.576 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.616228+0000", 2024-09-11T08:43:24.576 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0023789060000000001 2024-09-11T08:43:24.577 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.577 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.577 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.577 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.616867+0000", 2024-09-11T08:43:24.577 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00063850099999999998 2024-09-11T08:43:24.577 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.577 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.577 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.577 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801899+0000", 2024-09-11T08:43:24.577 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.185032274 2024-09-11T08:43:24.577 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.577 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.577 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.577 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801935+0000", 2024-09-11T08:43:24.578 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5729000000000003e-05 2024-09-11T08:43:24.578 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.578 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.578 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.578 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801969+0000", 2024-09-11T08:43:24.578 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4638000000000003e-05 2024-09-11T08:43:24.578 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.578 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.578 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.578 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.578 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.578 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:165674 7.d 7:b3f385a8:::benchmark_data_smithi092_84565_object40146:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T08:43:24.578 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:22.015476+0000", 2024-09-11T08:43:24.578 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 2.4401902799999999, 2024-09-11T08:43:24.579 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.108198423, 2024-09-11T08:43:24.579 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.579 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.579 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.579 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.579 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.579 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 165674 2024-09-11T08:43:24.579 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.579 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.579 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.579 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.579 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.015476+0000", 2024-09-11T08:43:24.579 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.579 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.580 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.580 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.580 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.015476+0000", 2024-09-11T08:43:24.580 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.580 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.580 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.580 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.580 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.015474+0000", 2024-09-11T08:43:24.580 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:43:24.580 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.580 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.580 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.580 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.015483+0000", 2024-09-11T08:43:24.580 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.6459999999999999e-06 2024-09-11T08:43:24.581 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.581 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.581 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.581 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.015502+0000", 2024-09-11T08:43:24.581 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.8992999999999999e-05 2024-09-11T08:43:24.581 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.581 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.581 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.581 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.015511+0000", 2024-09-11T08:43:24.581 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.1970000000000002e-06 2024-09-11T08:43:24.581 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.581 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.581 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.582 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.110912+0000", 2024-09-11T08:43:24.582 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.095401154000000002 2024-09-11T08:43:24.582 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.582 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.582 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.582 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.111029+0000", 2024-09-11T08:43:24.582 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000116989 2024-09-11T08:43:24.582 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.582 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.582 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:43:24.582 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.111159+0000", 2024-09-11T08:43:24.582 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012966400000000001 2024-09-11T08:43:24.582 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.582 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.583 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.583 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.121685+0000", 2024-09-11T08:43:24.583 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.010526021999999999 2024-09-11T08:43:24.583 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.583 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.583 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.583 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.121762+0000", 2024-09-11T08:43:24.583 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7467999999999998e-05 2024-09-11T08:43:24.583 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.583 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.583 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.583 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.123585+0000", 2024-09-11T08:43:24.583 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0018222589999999999 2024-09-11T08:43:24.584 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.584 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.584 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.584 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.123629+0000", 2024-09-11T08:43:24.584 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4183e-05 2024-09-11T08:43:24.584 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.584 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.584 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.584 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.123674+0000", 2024-09-11T08:43:24.584 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5414999999999998e-05 2024-09-11T08:43:24.584 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.584 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.584 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.585 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.585 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.585 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:165676 7.7 7:e42dc50b:::benchmark_data_smithi092_84565_object40148:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T08:43:24.585 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:22.015680+0000", 2024-09-11T08:43:24.585 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 2.4399864679999999, 2024-09-11T08:43:24.585 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10988466299999999, 2024-09-11T08:43:24.585 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:24.585 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:43:24.585 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:24.585 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:24.585 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:24.585 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 165676 2024-09-11T08:43:24.585 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.015680+0000", 2024-09-11T08:43:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.015680+0000", 2024-09-11T08:43:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.015678+0000", 2024-09-11T08:43:24.586 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:43:24.587 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.587 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.587 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:24.587 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.015685+0000", 2024-09-11T08:43:24.587 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.7530000000000004e-06 2024-09-11T08:43:24.587 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.587 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.587 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:24.587 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.110701+0000", 2024-09-11T08:43:24.587 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.095016046000000007 2024-09-11T08:43:24.587 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.587 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.587 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:24.588 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.110727+0000", 2024-09-11T08:43:24.588 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.6106e-05 2024-09-11T08:43:24.588 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.588 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.588 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:24.588 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.110878+0000", 2024-09-11T08:43:24.588 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000150927 2024-09-11T08:43:24.588 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.588 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.588 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:24.588 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.111029+0000", 2024-09-11T08:43:24.588 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000151139 2024-09-11T08:43:24.588 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.589 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.589 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,3", 2024-09-11T08:43:24.589 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.111159+0000", 2024-09-11T08:43:24.589 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012943699999999999 2024-09-11T08:43:24.589 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.589 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.589 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:24.589 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.119164+0000", 2024-09-11T08:43:24.589 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0080047420000000005 2024-09-11T08:43:24.589 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.589 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.589 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.589 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.122893+0000", 2024-09-11T08:43:24.589 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003729076 2024-09-11T08:43:24.590 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.590 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.590 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:24.590 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.125484+0000", 2024-09-11T08:43:24.590 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.002591398 2024-09-11T08:43:24.590 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.590 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.590 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:43:24.590 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.125528+0000", 2024-09-11T08:43:24.590 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3924999999999997e-05 2024-09-11T08:43:24.590 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:24.590 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:24.590 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:43:24.590 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:22.125564+0000", 2024-09-11T08:43:24.591 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6257000000000002e-05 2024-09-11T08:43:24.591 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.591 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.591 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.591 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:24.591 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:24.591 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:43:24.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:24.613 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:24.613 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:24.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:24.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:24.757 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:24.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:24.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:24.899 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:24.900 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:25.000 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:25.028 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:25.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:25.029 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:25.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:25.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:25.167 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:25.227 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:24 smithi092 ceph-mon[29647]: 6.1c scrub starts 2024-09-11T08:43:25.227 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:24 smithi092 ceph-mon[29647]: 6.1c scrub ok 2024-09-11T08:43:25.227 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:24 smithi092 ceph-mon[41114]: 6.1c scrub starts 2024-09-11T08:43:25.227 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:24 smithi092 ceph-mon[41114]: 6.1c scrub ok 2024-09-11T08:43:25.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:25.303 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:25.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:25.304 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:25.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:25.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:24 smithi145 ceph-mon[33384]: 6.1c scrub starts 2024-09-11T08:43:25.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:24 smithi145 ceph-mon[33384]: 6.1c scrub ok 2024-09-11T08:43:25.439 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:25.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:25.440 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:25.460 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T08:43:25.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:25.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:25.578 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:25.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:25.677 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:25.679 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:25.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:25.706 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:25.706 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:25.791 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:43:25.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:25.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:25.849 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:25.943 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.6 to 1 (10000) 2024-09-11T08:43:25.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:25.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:25.997 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:25.998 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:26.099 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:26.129 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:26.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:26.130 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:26.228 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:25 smithi145 ceph-mon[33384]: 3.c scrub starts 2024-09-11T08:43:26.229 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:25 smithi145 ceph-mon[33384]: 3.c scrub ok 2024-09-11T08:43:26.229 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:25 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3187244960' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T08:43:26.229 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:25 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T08:43:26.229 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:25 smithi145 ceph-mon[33384]: 3.b scrub starts 2024-09-11T08:43:26.229 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:25 smithi145 ceph-mon[33384]: 3.b scrub ok 2024-09-11T08:43:26.229 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:25 smithi145 ceph-mon[33384]: 4.c scrub starts 2024-09-11T08:43:26.229 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:25 smithi145 ceph-mon[33384]: 4.c scrub ok 2024-09-11T08:43:26.229 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:25 smithi145 ceph-mon[33384]: 6.5 scrub starts 2024-09-11T08:43:26.229 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:25 smithi145 ceph-mon[33384]: 6.5 scrub ok 2024-09-11T08:43:26.229 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:25 smithi145 ceph-mon[33384]: pgmap v181: 148 pgs: 148 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 678 op/s 2024-09-11T08:43:26.229 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:25 smithi145 ceph-mon[33384]: 5.a scrub starts 2024-09-11T08:43:26.229 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:25 smithi145 ceph-mon[33384]: 5.a scrub ok 2024-09-11T08:43:26.231 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:26.261 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:26.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:26.262 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:26.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[29647]: 3.c scrub starts 2024-09-11T08:43:26.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[29647]: 3.c scrub ok 2024-09-11T08:43:26.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3187244960' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T08:43:26.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T08:43:26.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[29647]: 3.b scrub starts 2024-09-11T08:43:26.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[29647]: 3.b scrub ok 2024-09-11T08:43:26.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[29647]: 4.c scrub starts 2024-09-11T08:43:26.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[29647]: 4.c scrub ok 2024-09-11T08:43:26.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[29647]: 6.5 scrub starts 2024-09-11T08:43:26.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[29647]: 6.5 scrub ok 2024-09-11T08:43:26.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[29647]: pgmap v181: 148 pgs: 148 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 678 op/s 2024-09-11T08:43:26.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[29647]: 5.a scrub starts 2024-09-11T08:43:26.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[29647]: 5.a scrub ok 2024-09-11T08:43:26.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[41114]: 3.c scrub starts 2024-09-11T08:43:26.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[41114]: 3.c scrub ok 2024-09-11T08:43:26.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3187244960' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T08:43:26.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T08:43:26.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[41114]: 3.b scrub starts 2024-09-11T08:43:26.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[41114]: 3.b scrub ok 2024-09-11T08:43:26.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[41114]: 4.c scrub starts 2024-09-11T08:43:26.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[41114]: 4.c scrub ok 2024-09-11T08:43:26.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[41114]: 6.5 scrub starts 2024-09-11T08:43:26.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[41114]: 6.5 scrub ok 2024-09-11T08:43:26.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[41114]: pgmap v181: 148 pgs: 148 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 678 op/s 2024-09-11T08:43:26.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[41114]: 5.a scrub starts 2024-09-11T08:43:26.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:25 smithi092 ceph-mon[41114]: 5.a scrub ok 2024-09-11T08:43:26.363 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:26.387 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:26.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:26.388 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:26.489 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:26.515 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:26.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:26.515 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:26.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:26.655 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:26.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:26.656 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:26.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:43:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:43:26.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:26.791 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:26.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:26.792 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:26.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:26.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:26.936 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:27.002 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:26 smithi092 ceph-mon[29647]: 7.a scrub ok 2024-09-11T08:43:27.002 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:26 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T08:43:27.003 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:26 smithi092 ceph-mon[29647]: osdmap e123: 8 total, 8 up, 8 in 2024-09-11T08:43:27.003 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:26 smithi092 ceph-mon[29647]: 4.5 scrub starts 2024-09-11T08:43:27.003 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:26 smithi092 ceph-mon[29647]: 4.5 scrub ok 2024-09-11T08:43:27.003 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:26 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:27.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:26 smithi092 ceph-mon[41114]: 7.a scrub ok 2024-09-11T08:43:27.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:26 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T08:43:27.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:26 smithi092 ceph-mon[41114]: osdmap e123: 8 total, 8 up, 8 in 2024-09-11T08:43:27.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:26 smithi092 ceph-mon[41114]: 4.5 scrub starts 2024-09-11T08:43:27.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:26 smithi092 ceph-mon[41114]: 4.5 scrub ok 2024-09-11T08:43:27.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:26 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:27.037 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:27.062 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:27.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:27.063 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:27.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:26 smithi145 ceph-mon[33384]: 7.a scrub ok 2024-09-11T08:43:27.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:26 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T08:43:27.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:26 smithi145 ceph-mon[33384]: osdmap e123: 8 total, 8 up, 8 in 2024-09-11T08:43:27.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:26 smithi145 ceph-mon[33384]: 4.5 scrub starts 2024-09-11T08:43:27.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:26 smithi145 ceph-mon[33384]: 4.5 scrub ok 2024-09-11T08:43:27.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:26 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:27.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:27.206 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:27.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:27.207 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:27.308 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:27.339 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:27.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:27.340 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:27.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:27.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 7 1 2024-09-11T08:43:27.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:27.487 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:27.487 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:27.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:27.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:27.627 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:27.628 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:27.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:27.771 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:27.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:27.772 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:27.873 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:27.898 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:27.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:27.898 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:27.999 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:28.028 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:28.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:28.029 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:28.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[29647]: 4.1b scrub starts 2024-09-11T08:43:28.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[29647]: 4.1b scrub ok 2024-09-11T08:43:28.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[29647]: pgmap v183: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-11T08:43:28.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[29647]: 3.f scrub starts 2024-09-11T08:43:28.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[29647]: 3.f scrub ok 2024-09-11T08:43:28.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[29647]: 3.1 scrub starts 2024-09-11T08:43:28.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[29647]: 3.1 scrub ok 2024-09-11T08:43:28.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T08:43:28.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[29647]: 6.f scrub ok 2024-09-11T08:43:28.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[41114]: 4.1b scrub starts 2024-09-11T08:43:28.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[41114]: 4.1b scrub ok 2024-09-11T08:43:28.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[41114]: pgmap v183: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-11T08:43:28.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[41114]: 3.f scrub starts 2024-09-11T08:43:28.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[41114]: 3.f scrub ok 2024-09-11T08:43:28.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[41114]: 3.1 scrub starts 2024-09-11T08:43:28.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[41114]: 3.1 scrub ok 2024-09-11T08:43:28.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T08:43:28.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:27 smithi092 ceph-mon[41114]: 6.f scrub ok 2024-09-11T08:43:28.130 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:28.158 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:28.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:28.159 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:28.260 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:28.287 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:28.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:28.288 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:28.389 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:28.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:27 smithi145 ceph-mon[33384]: 4.1b scrub starts 2024-09-11T08:43:28.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:27 smithi145 ceph-mon[33384]: 4.1b scrub ok 2024-09-11T08:43:28.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:27 smithi145 ceph-mon[33384]: pgmap v183: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-11T08:43:28.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:27 smithi145 ceph-mon[33384]: 3.f scrub starts 2024-09-11T08:43:28.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:27 smithi145 ceph-mon[33384]: 3.f scrub ok 2024-09-11T08:43:28.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:27 smithi145 ceph-mon[33384]: 3.1 scrub starts 2024-09-11T08:43:28.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:27 smithi145 ceph-mon[33384]: 3.1 scrub ok 2024-09-11T08:43:28.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:27 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T08:43:28.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:27 smithi145 ceph-mon[33384]: 6.f scrub ok 2024-09-11T08:43:28.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:28.415 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:28.415 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:28.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:28.583 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:28.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:28.584 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:28.685 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:28.721 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:28.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:28.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:28.811 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:43:28.811 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:43:28.811 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.812 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:172772 7.5 7:ae25417a:::benchmark_data_smithi092_84565_object47244:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-11T08:43:28.812 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:28.794244+0000", 2024-09-11T08:43:28.812 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.016669690000000001, 2024-09-11T08:43:28.812 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.016698274999999999, 2024-09-11T08:43:28.812 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:28.813 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:43:28.813 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:28.813 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:28.813 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:28.813 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 172772 2024-09-11T08:43:28.814 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.814 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:28.814 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.814 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:28.814 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.794244+0000", 2024-09-11T08:43:28.814 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:28.815 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.815 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.815 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:28.815 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.794244+0000", 2024-09-11T08:43:28.815 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:28.816 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.816 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.816 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:28.816 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.794242+0000", 2024-09-11T08:43:28.816 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999976 2024-09-11T08:43:28.817 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.817 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.817 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:28.817 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.794249+0000", 2024-09-11T08:43:28.817 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4460000000000004e-06 2024-09-11T08:43:28.817 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.818 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.818 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:28.818 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.794264+0000", 2024-09-11T08:43:28.818 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4501999999999999e-05 2024-09-11T08:43:28.818 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.819 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.819 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.819 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.794270+0000", 2024-09-11T08:43:28.819 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3280000000000002e-06 2024-09-11T08:43:28.819 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.820 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.820 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:28.820 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.794324+0000", 2024-09-11T08:43:28.820 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3411000000000001e-05 2024-09-11T08:43:28.820 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.820 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.821 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:28.821 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.794448+0000", 2024-09-11T08:43:28.821 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012402999999999999 2024-09-11T08:43:28.821 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.821 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.822 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:43:28.822 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.794564+0000", 2024-09-11T08:43:28.822 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000116267 2024-09-11T08:43:28.822 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.822 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.822 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:28.823 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.800813+0000", 2024-09-11T08:43:28.823 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0062490949999999997 2024-09-11T08:43:28.823 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.823 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.823 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.824 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.804371+0000", 2024-09-11T08:43:28.824 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0035575469999999999 2024-09-11T08:43:28.824 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:28.824 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:28.824 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:28.824 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.825 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.825 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:172773 7.5 7:ad9886d2:::benchmark_data_smithi092_84565_object47245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-11T08:43:28.825 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:28.795390+0000", 2024-09-11T08:43:28.825 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.015524111, 2024-09-11T08:43:28.825 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.015653885999999999, 2024-09-11T08:43:28.826 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:28.826 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:43:28.826 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:43:28.826 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:43:28.826 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:28.827 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 172773 2024-09-11T08:43:28.827 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.827 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:28.827 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.827 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:28.827 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.795390+0000", 2024-09-11T08:43:28.828 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:28.828 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.828 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.828 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:28.828 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.795390+0000", 2024-09-11T08:43:28.829 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:28.829 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.829 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.829 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:28.829 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.795388+0000", 2024-09-11T08:43:28.830 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:43:28.830 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.830 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.830 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:28.830 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.795394+0000", 2024-09-11T08:43:28.831 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1109999999999999e-06 2024-09-11T08:43:28.831 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.831 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.831 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:28.831 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.795409+0000", 2024-09-11T08:43:28.831 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5761999999999999e-05 2024-09-11T08:43:28.832 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.832 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.832 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.832 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.795416+0000", 2024-09-11T08:43:28.832 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6270000000000002e-06 2024-09-11T08:43:28.833 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.833 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.833 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:28.833 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.797374+0000", 2024-09-11T08:43:28.833 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0019581730000000001 2024-09-11T08:43:28.834 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.834 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.834 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:28.834 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.797475+0000", 2024-09-11T08:43:28.834 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010074800000000001 2024-09-11T08:43:28.834 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.835 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.835 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:43:28.835 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.797863+0000", 2024-09-11T08:43:28.835 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00038838000000000002 2024-09-11T08:43:28.835 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.836 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.836 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:43:28.836 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.801300+0000", 2024-09-11T08:43:28.836 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0034371179999999999 2024-09-11T08:43:28.836 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.837 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.837 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:28.837 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.807302+0000", 2024-09-11T08:43:28.837 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0060011639999999998 2024-09-11T08:43:28.837 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:28.838 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:28.838 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:28.838 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.838 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.838 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.14814.0:172787 7.c e123/88 7:3ff9fc49:::benchmark_data_smithi092_84565_object47259:head v 123'10688, mlcod=123'10686)", 2024-09-11T08:43:28.839 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:28.808323+0000", 2024-09-11T08:43:28.839 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.002590831, 2024-09-11T08:43:28.839 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0027740859999999998, 2024-09-11T08:43:28.839 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:43:28.839 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T08:43:28.839 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:43:28.840 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.840 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:43:28.840 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.808323+0000", 2024-09-11T08:43:28.840 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:28.840 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.841 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.841 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:43:28.841 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.808323+0000", 2024-09-11T08:43:28.841 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:43:28.841 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.842 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.842 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:43:28.842 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.808322+0000", 2024-09-11T08:43:28.842 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:43:28.842 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.843 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.843 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:43:28.843 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.808328+0000", 2024-09-11T08:43:28.843 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0299999999999999e-06 2024-09-11T08:43:28.843 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.843 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.844 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:43:28.844 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.808342+0000", 2024-09-11T08:43:28.844 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3908e-05 2024-09-11T08:43:28.844 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.844 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.845 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:43:28.845 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.808347+0000", 2024-09-11T08:43:28.845 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0660000000000003e-06 2024-09-11T08:43:28.845 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.845 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.845 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:43:28.846 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.808384+0000", 2024-09-11T08:43:28.846 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6248999999999998e-05 2024-09-11T08:43:28.846 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:43:28.846 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:43:28.846 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:43:28.846 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:28.808401+0000", 2024-09-11T08:43:28.847 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7549000000000001e-05 2024-09-11T08:43:28.847 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:28.847 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:43:28.847 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:28.847 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:43:28.847 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2024-09-11T08:43:28.848 INFO:teuthology.orchestra.run.smithi145.stdout: "num_ops": 3 2024-09-11T08:43:28.848 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:43:28.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:28.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:28.894 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:28.895 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:28.977 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:43:28.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:29.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:29.039 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:29.039 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:29.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:29.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:29.179 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:29.179 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:29.194 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[29647]: 3.19 scrub starts 2024-09-11T08:43:29.194 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[29647]: 3.19 scrub ok 2024-09-11T08:43:29.194 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[29647]: 6.b scrub starts 2024-09-11T08:43:29.194 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[29647]: 6.b scrub ok 2024-09-11T08:43:29.194 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1131430465' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:29.194 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:29.194 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[29647]: 6.16 scrub starts 2024-09-11T08:43:29.194 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[29647]: 6.16 scrub ok 2024-09-11T08:43:29.194 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[41114]: 3.19 scrub starts 2024-09-11T08:43:29.194 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[41114]: 3.19 scrub ok 2024-09-11T08:43:29.195 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[41114]: 6.b scrub starts 2024-09-11T08:43:29.195 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[41114]: 6.b scrub ok 2024-09-11T08:43:29.195 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1131430465' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:29.195 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:29.195 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[41114]: 6.16 scrub starts 2024-09-11T08:43:29.195 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:28 smithi092 ceph-mon[41114]: 6.16 scrub ok 2024-09-11T08:43:29.281 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:29.314 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:29.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:29.315 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:29.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:28 smithi145 ceph-mon[33384]: 3.19 scrub starts 2024-09-11T08:43:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:28 smithi145 ceph-mon[33384]: 3.19 scrub ok 2024-09-11T08:43:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:28 smithi145 ceph-mon[33384]: 6.b scrub starts 2024-09-11T08:43:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:28 smithi145 ceph-mon[33384]: 6.b scrub ok 2024-09-11T08:43:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:28 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1131430465' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:28 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:28 smithi145 ceph-mon[33384]: 6.16 scrub starts 2024-09-11T08:43:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:28 smithi145 ceph-mon[33384]: 6.16 scrub ok 2024-09-11T08:43:29.416 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:29.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:29.449 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:29.449 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:29.550 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:29.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:29.588 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:29.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:29.689 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:29.710 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:29.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:29.711 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:29.812 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:29.853 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:29.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:29.855 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:29.955 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:29.983 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:29.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:29.984 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:30.085 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:30.106 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:30.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:30.107 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:30.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:30.242 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:30.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:30.243 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:30.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[29647]: 5.15 scrub starts 2024-09-11T08:43:30.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[29647]: 5.15 scrub ok 2024-09-11T08:43:30.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:43:30.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:43:30.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:43:30.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[29647]: osdmap e124: 8 total, 8 up, 8 in 2024-09-11T08:43:30.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[29647]: 5.18 deep-scrub starts 2024-09-11T08:43:30.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[29647]: 5.18 deep-scrub ok 2024-09-11T08:43:30.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[29647]: pgmap v185: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 829 B/s rd, 0 B/s wr, 843 op/s 2024-09-11T08:43:30.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3736129210' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:30.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[29647]: 6.10 scrub starts 2024-09-11T08:43:30.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[29647]: 6.10 scrub ok 2024-09-11T08:43:30.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[41114]: 5.15 scrub starts 2024-09-11T08:43:30.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[41114]: 5.15 scrub ok 2024-09-11T08:43:30.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:43:30.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:43:30.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:43:30.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[41114]: osdmap e124: 8 total, 8 up, 8 in 2024-09-11T08:43:30.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[41114]: 5.18 deep-scrub starts 2024-09-11T08:43:30.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[41114]: 5.18 deep-scrub ok 2024-09-11T08:43:30.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[41114]: pgmap v185: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 829 B/s rd, 0 B/s wr, 843 op/s 2024-09-11T08:43:30.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3736129210' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:30.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[41114]: 6.10 scrub starts 2024-09-11T08:43:30.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:29 smithi092 ceph-mon[41114]: 6.10 scrub ok 2024-09-11T08:43:30.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:30.381 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:30.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:30.382 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:30.407 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:29 smithi145 ceph-mon[33384]: 5.15 scrub starts 2024-09-11T08:43:30.407 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:29 smithi145 ceph-mon[33384]: 5.15 scrub ok 2024-09-11T08:43:30.407 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:29 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:43:30.407 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:29 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:43:30.407 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:29 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:43:30.407 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:29 smithi145 ceph-mon[33384]: osdmap e124: 8 total, 8 up, 8 in 2024-09-11T08:43:30.407 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:29 smithi145 ceph-mon[33384]: 5.18 deep-scrub starts 2024-09-11T08:43:30.407 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:29 smithi145 ceph-mon[33384]: 5.18 deep-scrub ok 2024-09-11T08:43:30.407 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:29 smithi145 ceph-mon[33384]: pgmap v185: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 829 B/s rd, 0 B/s wr, 843 op/s 2024-09-11T08:43:30.407 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:29 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3736129210' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:30.407 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:29 smithi145 ceph-mon[33384]: 6.10 scrub starts 2024-09-11T08:43:30.408 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:29 smithi145 ceph-mon[33384]: 6.10 scrub ok 2024-09-11T08:43:30.445 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-11T08:43:30.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:30.511 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:30.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:30.512 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:30.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:30.655 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:30.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:30.657 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:30.740 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:43:30.741 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:43:30.741 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:43:30.741 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:43:30.741 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:43:30.741 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:43:30.741 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:43:30.741 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:43:30.741 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:30.741 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:30.741 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:30.742 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:30.742 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:30.742 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:30.742 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:30.742 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:30.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:30.790 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:43:30.793 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:30.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:30.794 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:30.895 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:30.923 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:30.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:30.924 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:30.968 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:31.025 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:31.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:31.052 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:31.052 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:31.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:31.191 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:31.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:31.192 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:31.277 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:31 smithi145 ceph-mon[33384]: 6.7 scrub starts 2024-09-11T08:43:31.277 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:31 smithi145 ceph-mon[33384]: 6.7 scrub ok 2024-09-11T08:43:31.277 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:31 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:31.277 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:31 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:31.277 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:31 smithi145 ceph-mon[33384]: 5.1e scrub starts 2024-09-11T08:43:31.278 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:31 smithi145 ceph-mon[33384]: 5.1e scrub ok 2024-09-11T08:43:31.293 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:31.323 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:31.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:31.324 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:31.425 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:31.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:31 smithi092 ceph-mon[29647]: 6.7 scrub starts 2024-09-11T08:43:31.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:31 smithi092 ceph-mon[29647]: 6.7 scrub ok 2024-09-11T08:43:31.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:31 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:31.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:31 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:31.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:31 smithi092 ceph-mon[29647]: 5.1e scrub starts 2024-09-11T08:43:31.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:31 smithi092 ceph-mon[29647]: 5.1e scrub ok 2024-09-11T08:43:31.448 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:31 smithi092 ceph-mon[41114]: 6.7 scrub starts 2024-09-11T08:43:31.448 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:31 smithi092 ceph-mon[41114]: 6.7 scrub ok 2024-09-11T08:43:31.448 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:31 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:31.448 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:31 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:31.448 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:31 smithi092 ceph-mon[41114]: 5.1e scrub starts 2024-09-11T08:43:31.448 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:31 smithi092 ceph-mon[41114]: 5.1e scrub ok 2024-09-11T08:43:31.449 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:31.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:31.450 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:31.551 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:31.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:31.576 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:31.576 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:31.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:31.717 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:31.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:31.717 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:31.765 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:43:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:43:31] "GET /metrics HTTP/1.1" 200 223813 "" "Prometheus/2.43.0" 2024-09-11T08:43:31.819 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:31.846 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:31.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:31.847 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:31.948 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:31.989 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:31.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:31.990 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:32.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:32.122 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:32.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:32.124 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:32.224 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:32.259 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:32.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:32.260 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:32.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:32 smithi092 ceph-mon[29647]: 4.1 deep-scrub starts 2024-09-11T08:43:32.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:32 smithi092 ceph-mon[29647]: 4.1 deep-scrub ok 2024-09-11T08:43:32.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:32 smithi092 ceph-mon[29647]: 6.1f scrub starts 2024-09-11T08:43:32.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:32 smithi092 ceph-mon[29647]: pgmap v186: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 913 op/s 2024-09-11T08:43:32.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:32 smithi092 ceph-mon[29647]: 6.1f scrub ok 2024-09-11T08:43:32.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:32 smithi092 ceph-mon[29647]: 4.1f scrub starts 2024-09-11T08:43:32.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:32 smithi092 ceph-mon[29647]: 4.1f scrub ok 2024-09-11T08:43:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:32 smithi092 ceph-mon[41114]: 4.1 deep-scrub starts 2024-09-11T08:43:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:32 smithi092 ceph-mon[41114]: 4.1 deep-scrub ok 2024-09-11T08:43:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:32 smithi092 ceph-mon[41114]: 6.1f scrub starts 2024-09-11T08:43:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:32 smithi092 ceph-mon[41114]: pgmap v186: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 913 op/s 2024-09-11T08:43:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:32 smithi092 ceph-mon[41114]: 6.1f scrub ok 2024-09-11T08:43:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:32 smithi092 ceph-mon[41114]: 4.1f scrub starts 2024-09-11T08:43:32.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:32 smithi092 ceph-mon[41114]: 4.1f scrub ok 2024-09-11T08:43:32.361 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:32.392 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:32.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:32.393 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:32.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:32 smithi145 ceph-mon[33384]: 4.1 deep-scrub starts 2024-09-11T08:43:32.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:32 smithi145 ceph-mon[33384]: 4.1 deep-scrub ok 2024-09-11T08:43:32.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:32 smithi145 ceph-mon[33384]: 6.1f scrub starts 2024-09-11T08:43:32.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:32 smithi145 ceph-mon[33384]: pgmap v186: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 913 op/s 2024-09-11T08:43:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:32 smithi145 ceph-mon[33384]: 6.1f scrub ok 2024-09-11T08:43:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:32 smithi145 ceph-mon[33384]: 4.1f scrub starts 2024-09-11T08:43:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:32 smithi145 ceph-mon[33384]: 4.1f scrub ok 2024-09-11T08:43:32.494 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:32.530 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:32.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:32.531 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:32.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:32.673 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:32.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:32.674 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:32.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:32.803 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:32.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:32.804 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:32.905 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:32.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:32.934 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:32.935 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:33.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:33.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:33.082 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:33.082 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:33.084 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:43:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:43:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:33.183 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:33.210 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:33.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:33.211 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:33.255 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:33.255 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:33.312 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:33.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:33 smithi092 ceph-mon[29647]: 6.1 deep-scrub starts 2024-09-11T08:43:33.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:33 smithi092 ceph-mon[29647]: 6.1 deep-scrub ok 2024-09-11T08:43:33.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:33 smithi092 ceph-mon[29647]: 6.13 scrub starts 2024-09-11T08:43:33.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:33 smithi092 ceph-mon[29647]: 6.13 scrub ok 2024-09-11T08:43:33.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:33 smithi092 ceph-mon[41114]: 6.1 deep-scrub starts 2024-09-11T08:43:33.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:33 smithi092 ceph-mon[41114]: 6.1 deep-scrub ok 2024-09-11T08:43:33.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:33 smithi092 ceph-mon[41114]: 6.13 scrub starts 2024-09-11T08:43:33.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:33 smithi092 ceph-mon[41114]: 6.13 scrub ok 2024-09-11T08:43:33.350 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:33.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:33.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:33.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:33 smithi145 ceph-mon[33384]: 6.1 deep-scrub starts 2024-09-11T08:43:33.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:33 smithi145 ceph-mon[33384]: 6.1 deep-scrub ok 2024-09-11T08:43:33.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:33 smithi145 ceph-mon[33384]: 6.13 scrub starts 2024-09-11T08:43:33.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:33 smithi145 ceph-mon[33384]: 6.13 scrub ok 2024-09-11T08:43:33.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:33.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:33.490 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:33.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:33.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:43:33.625 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:33.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:33.626 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:33.727 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:33.752 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:33.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:33.753 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:33.854 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:33.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:33.882 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:33.882 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:33.983 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:34.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:34.010 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:34.010 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:34.078 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-11T08:43:34.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:34.156 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:34.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:34.157 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:34.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:34.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:34 smithi092 ceph-mon[41114]: 4.8 deep-scrub starts 2024-09-11T08:43:34.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:34 smithi092 ceph-mon[41114]: 4.8 deep-scrub ok 2024-09-11T08:43:34.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:34 smithi092 ceph-mon[41114]: pgmap v187: 148 pgs: 148 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T08:43:34.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:34 smithi092 ceph-mon[41114]: 3.1b scrub starts 2024-09-11T08:43:34.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:34 smithi092 ceph-mon[41114]: 3.1b scrub ok 2024-09-11T08:43:34.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:34 smithi092 ceph-mon[29647]: 4.8 deep-scrub starts 2024-09-11T08:43:34.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:34 smithi092 ceph-mon[29647]: 4.8 deep-scrub ok 2024-09-11T08:43:34.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:34 smithi092 ceph-mon[29647]: pgmap v187: 148 pgs: 148 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T08:43:34.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:34 smithi092 ceph-mon[29647]: 3.1b scrub starts 2024-09-11T08:43:34.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:34 smithi092 ceph-mon[29647]: 3.1b scrub ok 2024-09-11T08:43:34.302 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:34.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:34.303 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:34.403 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:34.404 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:43:34.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:34 smithi145 ceph-mon[33384]: 4.8 deep-scrub starts 2024-09-11T08:43:34.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:34 smithi145 ceph-mon[33384]: 4.8 deep-scrub ok 2024-09-11T08:43:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:34 smithi145 ceph-mon[33384]: pgmap v187: 148 pgs: 148 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T08:43:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:34 smithi145 ceph-mon[33384]: 3.1b scrub starts 2024-09-11T08:43:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:34 smithi145 ceph-mon[33384]: 3.1b scrub ok 2024-09-11T08:43:34.438 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:34.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:34.439 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:34.487 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:34.487 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:43:34.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:34.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:34.610 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:34.610 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:34.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:34.778 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:34.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:34.779 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:34.880 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:34.909 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:34.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:34.910 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:35.011 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:35.037 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:35.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:35.038 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:35.108 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.7 to 1 (10000) 2024-09-11T08:43:35.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:35.181 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:35.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:35.182 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:35.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:35.322 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:35.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:35.323 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:35.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:35 smithi145 ceph-mon[33384]: 4.f deep-scrub starts 2024-09-11T08:43:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:35 smithi145 ceph-mon[33384]: 4.f deep-scrub ok 2024-09-11T08:43:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:35 smithi145 ceph-mon[33384]: 5.1b scrub starts 2024-09-11T08:43:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:35 smithi145 ceph-mon[33384]: 5.1b scrub ok 2024-09-11T08:43:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:35 smithi145 ceph-mon[33384]: 3.6 scrub starts 2024-09-11T08:43:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:35 smithi145 ceph-mon[33384]: 3.6 scrub ok 2024-09-11T08:43:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:35 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2632773584' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T08:43:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:35 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T08:43:35.424 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:35.458 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:35.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:35.459 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:35.514 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[41114]: 4.f deep-scrub starts 2024-09-11T08:43:35.514 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[41114]: 4.f deep-scrub ok 2024-09-11T08:43:35.514 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[41114]: 5.1b scrub starts 2024-09-11T08:43:35.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[41114]: 5.1b scrub ok 2024-09-11T08:43:35.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[41114]: 3.6 scrub starts 2024-09-11T08:43:35.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[41114]: 3.6 scrub ok 2024-09-11T08:43:35.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2632773584' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T08:43:35.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T08:43:35.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[29647]: 4.f deep-scrub starts 2024-09-11T08:43:35.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[29647]: 4.f deep-scrub ok 2024-09-11T08:43:35.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[29647]: 5.1b scrub starts 2024-09-11T08:43:35.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[29647]: 5.1b scrub ok 2024-09-11T08:43:35.516 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[29647]: 3.6 scrub starts 2024-09-11T08:43:35.516 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[29647]: 3.6 scrub ok 2024-09-11T08:43:35.516 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2632773584' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T08:43:35.516 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:35 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T08:43:35.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:35.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:35.599 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:35.599 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:35.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:35.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:35.733 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:35.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:35.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:35.872 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:35.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:35.895 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T08:43:35.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:43:35.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:36.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:36.019 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:36.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:36.120 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:36.147 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:36.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:36.148 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:36.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:36.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:36.286 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:36.286 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:36.345 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:43:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:43:36.346 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:36 smithi145 ceph-mon[33384]: 5.1f scrub starts 2024-09-11T08:43:36.346 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:36 smithi145 ceph-mon[33384]: 5.1f scrub ok 2024-09-11T08:43:36.346 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:36 smithi145 ceph-mon[33384]: pgmap v188: 148 pgs: 148 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-11T08:43:36.346 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:36 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T08:43:36.346 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:36 smithi145 ceph-mon[33384]: osdmap e125: 8 total, 8 up, 8 in 2024-09-11T08:43:36.365 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:36.365 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:36.387 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:36.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:36.415 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:36.415 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:36.516 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:36.542 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:36.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:36.543 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:36 smithi092 ceph-mon[29647]: 5.1f scrub starts 2024-09-11T08:43:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:36 smithi092 ceph-mon[29647]: 5.1f scrub ok 2024-09-11T08:43:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:36 smithi092 ceph-mon[29647]: pgmap v188: 148 pgs: 148 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-11T08:43:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:36 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T08:43:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:36 smithi092 ceph-mon[29647]: osdmap e125: 8 total, 8 up, 8 in 2024-09-11T08:43:36.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:36 smithi092 ceph-mon[41114]: 5.1f scrub starts 2024-09-11T08:43:36.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:36 smithi092 ceph-mon[41114]: 5.1f scrub ok 2024-09-11T08:43:36.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:36 smithi092 ceph-mon[41114]: pgmap v188: 148 pgs: 148 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-11T08:43:36.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:36 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T08:43:36.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:36 smithi092 ceph-mon[41114]: osdmap e125: 8 total, 8 up, 8 in 2024-09-11T08:43:36.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:36.684 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:36.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:36.685 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:36.786 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:36.812 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:36.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:36.813 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:36.914 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:36.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:36.943 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:36.943 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:36.991 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:37.044 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:37.068 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:37.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:37.069 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:37.170 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:37.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-11T08:43:37.197 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:37.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:37.198 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:37.236 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[29647]: 6.4 scrub starts 2024-09-11T08:43:37.236 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[29647]: 6.4 scrub ok 2024-09-11T08:43:37.236 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:43:37.236 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:43:37.236 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[29647]: 3.16 scrub starts 2024-09-11T08:43:37.236 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[29647]: 3.16 scrub ok 2024-09-11T08:43:37.236 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[29647]: osdmap e126: 8 total, 8 up, 8 in 2024-09-11T08:43:37.236 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:37.237 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[41114]: 6.4 scrub starts 2024-09-11T08:43:37.237 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[41114]: 6.4 scrub ok 2024-09-11T08:43:37.237 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:43:37.237 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:43:37.237 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[41114]: 3.16 scrub starts 2024-09-11T08:43:37.237 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[41114]: 3.16 scrub ok 2024-09-11T08:43:37.237 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[41114]: osdmap e126: 8 total, 8 up, 8 in 2024-09-11T08:43:37.237 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:37 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:37.299 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:37.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:37.326 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:37.326 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:37.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:37 smithi145 ceph-mon[33384]: 6.4 scrub starts 2024-09-11T08:43:37.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:37 smithi145 ceph-mon[33384]: 6.4 scrub ok 2024-09-11T08:43:37.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:37 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:43:37.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:37 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:43:37.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:37 smithi145 ceph-mon[33384]: 3.16 scrub starts 2024-09-11T08:43:37.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:37 smithi145 ceph-mon[33384]: 3.16 scrub ok 2024-09-11T08:43:37.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:37 smithi145 ceph-mon[33384]: osdmap e126: 8 total, 8 up, 8 in 2024-09-11T08:43:37.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:37 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:37.427 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:37.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:37.453 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:37.454 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:37.554 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:37.580 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:37.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:37.582 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:37.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:37.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:37.730 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:37.730 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:37.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:37.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:37.881 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:37.882 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:37.982 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:38.011 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:38.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:38.012 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:38.113 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:38.130 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:38.137 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:38.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:38.138 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:38.239 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:38.265 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:38.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:38.266 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:38.366 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:38.394 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:38.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:38.395 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:38.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:38 smithi145 ceph-mon[33384]: 4.2 scrub starts 2024-09-11T08:43:38.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:38 smithi145 ceph-mon[33384]: 4.2 scrub ok 2024-09-11T08:43:38.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:38 smithi145 ceph-mon[33384]: pgmap v191: 148 pgs: 148 active+clean; 4.3 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-09-11T08:43:38.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:38 smithi145 ceph-mon[33384]: 4.13 scrub starts 2024-09-11T08:43:38.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:38 smithi145 ceph-mon[33384]: 4.13 scrub ok 2024-09-11T08:43:38.496 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:38.523 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:38.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:38.524 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:38 smithi092 ceph-mon[29647]: 4.2 scrub starts 2024-09-11T08:43:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:38 smithi092 ceph-mon[29647]: 4.2 scrub ok 2024-09-11T08:43:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:38 smithi092 ceph-mon[29647]: pgmap v191: 148 pgs: 148 active+clean; 4.3 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-09-11T08:43:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:38 smithi092 ceph-mon[29647]: 4.13 scrub starts 2024-09-11T08:43:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:38 smithi092 ceph-mon[29647]: 4.13 scrub ok 2024-09-11T08:43:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:38 smithi092 ceph-mon[41114]: 4.2 scrub starts 2024-09-11T08:43:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:38 smithi092 ceph-mon[41114]: 4.2 scrub ok 2024-09-11T08:43:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:38 smithi092 ceph-mon[41114]: pgmap v191: 148 pgs: 148 active+clean; 4.3 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-09-11T08:43:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:38 smithi092 ceph-mon[41114]: 4.13 scrub starts 2024-09-11T08:43:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:38 smithi092 ceph-mon[41114]: 4.13 scrub ok 2024-09-11T08:43:38.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:38.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:38.663 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:38.764 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:38.800 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:38.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:38.801 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:38.902 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:38.939 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:38.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:38.940 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:39.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:39.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:39.081 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:39.182 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:39.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:39.210 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:39.210 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:39.311 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:39.338 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:39.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:39.339 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:39.358 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:43:39.419 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:39 smithi145 ceph-mon[33384]: 3.15 scrub starts 2024-09-11T08:43:39.419 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:39 smithi145 ceph-mon[33384]: 3.15 scrub ok 2024-09-11T08:43:39.419 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:39 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2026620999' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:43:39.441 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:39.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:39.463 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:39.463 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:39.564 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:39 smithi092 ceph-mon[29647]: 3.15 scrub starts 2024-09-11T08:43:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:39 smithi092 ceph-mon[29647]: 3.15 scrub ok 2024-09-11T08:43:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:39 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2026620999' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:43:39.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:39 smithi092 ceph-mon[41114]: 3.15 scrub starts 2024-09-11T08:43:39.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:39 smithi092 ceph-mon[41114]: 3.15 scrub ok 2024-09-11T08:43:39.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:39 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2026620999' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:43:39.592 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:39.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:39.593 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:39.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:39.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:39.730 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:39.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:39.868 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:39.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:39.869 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:39.970 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:39.996 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:39.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:39.997 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:40.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:40.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:40.139 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:40.139 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:40.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:40.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:40.279 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:40.279 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:40 smithi092 ceph-mon[29647]: 2.2 scrub starts 2024-09-11T08:43:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:40 smithi092 ceph-mon[29647]: 2.2 scrub ok 2024-09-11T08:43:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:40 smithi092 ceph-mon[29647]: pgmap v192: 148 pgs: 148 active+clean; 4.3 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-11T08:43:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:40 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:43:40.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:40 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2026620999' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:43:40.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:40 smithi092 ceph-mon[29647]: osdmap e127: 8 total, 8 up, 8 in 2024-09-11T08:43:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:40 smithi092 ceph-mon[41114]: 2.2 scrub starts 2024-09-11T08:43:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:40 smithi092 ceph-mon[41114]: 2.2 scrub ok 2024-09-11T08:43:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:40 smithi092 ceph-mon[41114]: pgmap v192: 148 pgs: 148 active+clean; 4.3 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-11T08:43:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:40 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:43:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:40 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2026620999' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:43:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:40 smithi092 ceph-mon[41114]: osdmap e127: 8 total, 8 up, 8 in 2024-09-11T08:43:40.380 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:40.410 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:40.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:40.411 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:40.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:40 smithi145 ceph-mon[33384]: 2.2 scrub starts 2024-09-11T08:43:40.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:40 smithi145 ceph-mon[33384]: 2.2 scrub ok 2024-09-11T08:43:40.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:40 smithi145 ceph-mon[33384]: pgmap v192: 148 pgs: 148 active+clean; 4.3 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-11T08:43:40.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:40 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:43:40.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:40 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2026620999' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:43:40.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:40 smithi145 ceph-mon[33384]: osdmap e127: 8 total, 8 up, 8 in 2024-09-11T08:43:40.512 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:40.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:40.534 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:40.534 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:40.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:40.667 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:40.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:40.668 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:40.769 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:40.792 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:40.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:40.794 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:40.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:40.928 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:40.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:40.929 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:41.030 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:41.071 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:41.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:41.072 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:41.173 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:41.212 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:41.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:41.213 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:41.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:41.350 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:41.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:41.351 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:41.452 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:41.478 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:41.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:41.479 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:41.580 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:41.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:41.613 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:41.613 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:41.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:41.752 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:41.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:41.753 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:41.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:43:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:43:41] "GET /metrics HTTP/1.1" 200 223813 "" "Prometheus/2.43.0" 2024-09-11T08:43:41.855 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:41.881 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:41.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:41.882 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:41.983 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:42.010 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:42.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:42.011 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:42.112 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:42.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:42.136 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:42.137 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:42.237 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:42.262 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:42.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:42.263 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:42.364 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:42.388 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:42.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:42.389 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:42.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:42.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:42.526 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:42.526 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:42.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:42 smithi092 ceph-mon[29647]: pgmap v194: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T08:43:42.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:42 smithi092 ceph-mon[41114]: pgmap v194: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T08:43:42.627 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:42.650 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:42.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:42.652 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:42 smithi145 ceph-mon[33384]: pgmap v194: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T08:43:42.753 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:42.793 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:42.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:42.794 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:42.895 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:42.920 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:42.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:42.921 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:43.016 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:43.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:43.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:43.081 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:43.081 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:43.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:43:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:43:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:43.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:43.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:43.228 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:43.228 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:43.329 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:43.354 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:43.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:43.355 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:43.456 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:43.483 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:43.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:43.484 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:43.585 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:43.610 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:43.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:43.611 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:43.712 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:43.739 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:43.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:43.740 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:43.841 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:43.878 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:43.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:43.879 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:43.980 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:44.005 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:44.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:44.006 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:44.107 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:44.133 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:44.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:44.134 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:44.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:44.278 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:44.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:44.279 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:44.291 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:43:44.291 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:44.331 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:43:44.331 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:44.365 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:43:44.365 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:44.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:44 smithi092 ceph-mon[29647]: pgmap v195: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-09-11T08:43:44.366 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:44 smithi092 ceph-mon[41114]: pgmap v195: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-09-11T08:43:44.379 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:44.403 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:43:44.403 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:44.420 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:44.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:44.421 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:44.433 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:43:44.433 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:44.463 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:43:44.463 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:44.494 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:43:44.494 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:44.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:44.532 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:43:44.533 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:44.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:44.570 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:44.571 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:44 smithi145 ceph-mon[33384]: pgmap v195: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-09-11T08:43:44.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:44.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:44.710 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:44.710 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:44.811 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:44.835 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:44.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:44.836 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:44.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:43:44.936 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:44.964 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:44.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:44.965 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:45.066 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:45.092 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:45.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:45.094 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:45.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:45.240 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:45.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:45.241 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:45.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:45 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1385555094' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:45.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:45 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:45.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:45.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:45 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1385555094' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:45.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:45 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:45.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:45.342 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:45.368 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:45.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:45.369 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:45.470 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:45.495 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:45.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:45.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:45.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:45.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:45.629 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:45.629 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:45 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1385555094' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:45 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:45.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:45.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:45.789 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:45.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:45.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:45.891 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:45.917 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:45.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:45.918 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:46.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:46.058 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:46.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:46.059 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:46.160 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:46.194 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:46.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:46.195 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:46.296 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:46 smithi092 ceph-mon[29647]: pgmap v196: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 570 B/s rd, 0 B/s wr, 987 op/s 2024-09-11T08:43:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:46 smithi092 ceph-mon[41114]: pgmap v196: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 570 B/s rd, 0 B/s wr, 987 op/s 2024-09-11T08:43:46.322 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:46.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:46.323 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:46.424 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:46.452 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:46.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:46.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:46.553 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:46.592 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:46.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:46.593 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:46.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:46 smithi145 ceph-mon[33384]: pgmap v196: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 570 B/s rd, 0 B/s wr, 987 op/s 2024-09-11T08:43:46.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:43:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:43:46.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:46.730 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:46.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:46.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:46.832 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:46.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:46.861 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:46.861 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:46.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:47.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:47.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:47.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:47.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:47.142 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:47.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:47.190 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:47 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:47.243 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:47.243 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:47.268 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:47.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:47.270 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:47.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:47.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:47.411 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:47.411 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:47.481 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:47 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:47.512 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:47.554 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:43:47.557 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:43:47.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:47.559 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:47.560 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:47.660 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:47.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:47 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:47.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:47.689 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:47.689 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:47.790 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:47.830 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:47.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:47.831 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:47.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:47.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:47.972 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:48.073 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:48.101 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:48.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:48.101 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:48.202 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:48.227 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:48.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:48.228 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:48.328 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:48.358 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:48.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:48.359 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:48.460 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:48.485 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:48.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:48.487 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:48 smithi092 ceph-mon[29647]: pgmap v197: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T08:43:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:48 smithi092 ceph-mon[29647]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:43:48.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:48 smithi092 ceph-mon[41114]: pgmap v197: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T08:43:48.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:48 smithi092 ceph-mon[41114]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:43:48.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:48.625 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:48.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:48.626 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:48.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:48 smithi145 ceph-mon[33384]: pgmap v197: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T08:43:48.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:48 smithi145 ceph-mon[33384]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:43:48.679 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-11T08:43:47.101266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3905951988,"num_objects":59788,"num_object_clones":0,"num_object_copies":179364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59788,"num_whiteouts":0,"num_read":1154,"num_read_kb":1011,"num_write":192003,"num_write_kb":8035272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90533,"ondisk_log_size":90533,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":13962544,"kb_used_data":11521072,"kb_used_omap":13,"kb_used_meta":2441138,"kb_avail":735998672,"statfs":{"total":767960285184,"available":753662640128,"internally_reserved":0,"allocated":11797577728,"data_stored":11794892108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2499726016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-698482688,"num_objects":-10658,"num_object_clones":0,"num_object_copies":-31974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10658,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10658,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.010882"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.136396+0000","last_change":"2024-09-11T08:42:53.042630+0000","last_active":"2024-09-11T08:43:36.136396+0000","last_peered":"2024-09-11T08:43:36.136396+0000","last_clean":"2024-09-11T08:43:36.136396+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:43:36.136396+0000","last_undegraded":"2024-09-11T08:43:36.136396+0000","last_fullsized":"2024-09-11T08:43:36.136396+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:53.042556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:53.042556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:11.005058+0000","scrub_duration":0.0084633999999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":118,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123391+0000","last_change":"2024-09-11T08:42:23.321548+0000","last_active":"2024-09-11T08:43:36.123391+0000","last_peered":"2024-09-11T08:43:36.123391+0000","last_clean":"2024-09-11T08:43:36.123391+0000","last_became_active":"2024-09-11T08:38:40.973398+0000","last_became_peered":"2024-09-11T08:38:40.973398+0000","last_unstale":"2024-09-11T08:43:36.123391+0000","last_undegraded":"2024-09-11T08:43:36.123391+0000","last_fullsized":"2024-09-11T08:43:36.123391+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:42:23.321492+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:42:23.321492+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056603720000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.119206+0000","last_change":"2024-09-11T08:42:40.282406+0000","last_active":"2024-09-11T08:43:36.119206+0000","last_peered":"2024-09-11T08:43:36.119206+0000","last_clean":"2024-09-11T08:43:36.119206+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:43:36.119206+0000","last_undegraded":"2024-09-11T08:43:36.119206+0000","last_fullsized":"2024-09-11T08:43:36.119206+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:40.282323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:40.282323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:04.412797+0000","scrub_duration":0.013570553000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":84,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123434+0000","last_change":"2024-09-11T08:41:40.371901+0000","last_active":"2024-09-11T08:43:36.123434+0000","last_peered":"2024-09-11T08:43:36.123434+0000","last_clean":"2024-09-11T08:43:36.123434+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:43:36.123434+0000","last_undegraded":"2024-09-11T08:43:36.123434+0000","last_fullsized":"2024-09-11T08:43:36.123434+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:40.371846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:41:40.371846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014056730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.132706+0000","last_change":"2024-09-11T08:41:51.406079+0000","last_active":"2024-09-11T08:43:36.132706+0000","last_peered":"2024-09-11T08:43:36.132706+0000","last_clean":"2024-09-11T08:43:36.132706+0000","last_became_active":"2024-09-11T08:38:45.184215+0000","last_became_peered":"2024-09-11T08:38:45.184215+0000","last_unstale":"2024-09-11T08:43:36.132706+0000","last_undegraded":"2024-09-11T08:43:36.132706+0000","last_fullsized":"2024-09-11T08:43:36.132706+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:51.405971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:41:51.405971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013018258,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":99,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.132917+0000","last_change":"2024-09-11T08:41:33.298790+0000","last_active":"2024-09-11T08:43:36.132917+0000","last_peered":"2024-09-11T08:43:36.132917+0000","last_clean":"2024-09-11T08:43:36.132917+0000","last_became_active":"2024-09-11T08:38:38.949136+0000","last_became_peered":"2024-09-11T08:38:38.949136+0000","last_unstale":"2024-09-11T08:43:36.132917+0000","last_undegraded":"2024-09-11T08:43:36.132917+0000","last_fullsized":"2024-09-11T08:43:36.132917+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:33.298719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:41:33.298719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027845299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":129,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.131053+0000","last_change":"2024-09-11T08:41:48.443143+0000","last_active":"2024-09-11T08:43:36.131053+0000","last_peered":"2024-09-11T08:43:36.131053+0000","last_clean":"2024-09-11T08:43:36.131053+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:43:36.131053+0000","last_undegraded":"2024-09-11T08:43:36.131053+0000","last_fullsized":"2024-09-11T08:43:36.131053+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:41:48.442900+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:41:48.442900+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010545605,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123250+0000","last_change":"2024-09-11T08:43:28.988751+0000","last_active":"2024-09-11T08:43:36.123250+0000","last_peered":"2024-09-11T08:43:36.123250+0000","last_clean":"2024-09-11T08:43:36.123250+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:43:36.123250+0000","last_undegraded":"2024-09-11T08:43:36.123250+0000","last_fullsized":"2024-09-11T08:43:36.123250+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:43:28.988697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:58.396363+0000","scrub_duration":0.00375019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":100,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.117001+0000","last_change":"2024-09-11T08:41:57.349049+0000","last_active":"2024-09-11T08:43:36.117001+0000","last_peered":"2024-09-11T08:43:36.117001+0000","last_clean":"2024-09-11T08:43:36.117001+0000","last_became_active":"2024-09-11T08:38:38.948930+0000","last_became_peered":"2024-09-11T08:38:38.948930+0000","last_unstale":"2024-09-11T08:43:36.117001+0000","last_undegraded":"2024-09-11T08:43:36.117001+0000","last_fullsized":"2024-09-11T08:43:36.117001+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:41:57.348997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001221261,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":118,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123595+0000","last_change":"2024-09-11T08:41:55.318568+0000","last_active":"2024-09-11T08:43:36.123595+0000","last_peered":"2024-09-11T08:43:36.123595+0000","last_clean":"2024-09-11T08:43:36.123595+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:43:36.123595+0000","last_undegraded":"2024-09-11T08:43:36.123595+0000","last_fullsized":"2024-09-11T08:43:36.123595+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:41:55.318476+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:41:55.318476+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006520601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116811+0000","last_change":"2024-09-11T08:43:33.644944+0000","last_active":"2024-09-11T08:43:36.116811+0000","last_peered":"2024-09-11T08:43:36.116811+0000","last_clean":"2024-09-11T08:43:36.116811+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:43:36.116811+0000","last_undegraded":"2024-09-11T08:43:36.116811+0000","last_fullsized":"2024-09-11T08:43:36.116811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:33.644869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.644869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:58.010377+0000","scrub_duration":0.0020832210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.135664+0000","last_change":"2024-09-11T08:43:18.293424+0000","last_active":"2024-09-11T08:43:36.135664+0000","last_peered":"2024-09-11T08:43:36.135664+0000","last_clean":"2024-09-11T08:43:36.135664+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:43:36.135664+0000","last_undegraded":"2024-09-11T08:43:36.135664+0000","last_fullsized":"2024-09-11T08:43:36.135664+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:18.293342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:18.293342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:44.371461+0000","scrub_duration":0.0078301269999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":100,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116786+0000","last_change":"2024-09-11T08:41:53.294030+0000","last_active":"2024-09-11T08:43:36.116786+0000","last_peered":"2024-09-11T08:43:36.116786+0000","last_clean":"2024-09-11T08:43:36.116786+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:43:36.116786+0000","last_undegraded":"2024-09-11T08:43:36.116786+0000","last_fullsized":"2024-09-11T08:43:36.116786+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:53.293920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:41:53.293920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029725110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":108,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123733+0000","last_change":"2024-09-11T08:43:26.983719+0000","last_active":"2024-09-11T08:43:36.123733+0000","last_peered":"2024-09-11T08:43:36.123733+0000","last_clean":"2024-09-11T08:43:36.123733+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:43:36.123733+0000","last_undegraded":"2024-09-11T08:43:36.123733+0000","last_fullsized":"2024-09-11T08:43:36.123733+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:43:26.983587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:26.983587+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:43.437727+0000","scrub_duration":0.0082679390000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":105,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124784+0000","last_change":"2024-09-11T08:43:17.981506+0000","last_active":"2024-09-11T08:43:36.124784+0000","last_peered":"2024-09-11T08:43:36.124784+0000","last_clean":"2024-09-11T08:43:36.124784+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:43:36.124784+0000","last_undegraded":"2024-09-11T08:43:36.124784+0000","last_fullsized":"2024-09-11T08:43:36.124784+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:17.981432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:43:17.981432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:33.453163+0000","scrub_duration":0.001732162,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116563+0000","last_change":"2024-09-11T08:42:40.554777+0000","last_active":"2024-09-11T08:43:36.116563+0000","last_peered":"2024-09-11T08:43:36.116563+0000","last_clean":"2024-09-11T08:43:36.116563+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:43:36.116563+0000","last_undegraded":"2024-09-11T08:43:36.116563+0000","last_fullsized":"2024-09-11T08:43:36.116563+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:42:40.554717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:47.099561+0000","scrub_duration":0.01079478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124297+0000","last_change":"2024-09-11T08:42:31.315236+0000","last_active":"2024-09-11T08:43:36.124297+0000","last_peered":"2024-09-11T08:43:36.124297+0000","last_clean":"2024-09-11T08:43:36.124297+0000","last_became_active":"2024-09-11T08:38:45.183364+0000","last_became_peered":"2024-09-11T08:38:45.183364+0000","last_unstale":"2024-09-11T08:43:36.124297+0000","last_undegraded":"2024-09-11T08:43:36.124297+0000","last_fullsized":"2024-09-11T08:43:36.124297+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:31.315163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:31.315163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001316897,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.143982+0000","last_change":"2024-09-11T08:43:33.346941+0000","last_active":"2024-09-11T08:43:36.143982+0000","last_peered":"2024-09-11T08:43:36.143982+0000","last_clean":"2024-09-11T08:43:36.143982+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:43:36.143982+0000","last_undegraded":"2024-09-11T08:43:36.143982+0000","last_fullsized":"2024-09-11T08:43:36.143982+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:33.346856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.346856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:45:01.258719+0000","scrub_duration":0.0053605370000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":140,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.126781+0000","last_change":"2024-09-11T08:42:50.585164+0000","last_active":"2024-09-11T08:43:36.126781+0000","last_peered":"2024-09-11T08:43:36.126781+0000","last_clean":"2024-09-11T08:43:36.126781+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:43:36.126781+0000","last_undegraded":"2024-09-11T08:43:36.126781+0000","last_fullsized":"2024-09-11T08:43:36.126781+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:42:50.585063+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:42:50.585063+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:02.994702+0000","scrub_duration":0.012166819000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.120460+0000","last_change":"2024-09-11T08:42:25.205701+0000","last_active":"2024-09-11T08:43:36.120460+0000","last_peered":"2024-09-11T08:43:36.120460+0000","last_clean":"2024-09-11T08:43:36.120460+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:43:36.120460+0000","last_undegraded":"2024-09-11T08:43:36.120460+0000","last_fullsized":"2024-09-11T08:43:36.120460+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:25.205161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:25.205161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015856230999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.133110+0000","last_change":"2024-09-11T08:43:31.091880+0000","last_active":"2024-09-11T08:43:36.133110+0000","last_peered":"2024-09-11T08:43:36.133110+0000","last_clean":"2024-09-11T08:43:36.133110+0000","last_became_active":"2024-09-11T08:38:45.182763+0000","last_became_peered":"2024-09-11T08:38:45.182763+0000","last_unstale":"2024-09-11T08:43:36.133110+0000","last_undegraded":"2024-09-11T08:43:36.133110+0000","last_fullsized":"2024-09-11T08:43:36.133110+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:43:31.091591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:31.091591+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:48.160729+0000","scrub_duration":0.010466677000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":100,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123670+0000","last_change":"2024-09-11T08:42:49.064565+0000","last_active":"2024-09-11T08:43:36.123670+0000","last_peered":"2024-09-11T08:43:36.123670+0000","last_clean":"2024-09-11T08:43:36.123670+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:43:36.123670+0000","last_undegraded":"2024-09-11T08:43:36.123670+0000","last_fullsized":"2024-09-11T08:43:36.123670+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:49.064487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:49.064487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:14.376741+0000","scrub_duration":0.0037677629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":120,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.130912+0000","last_change":"2024-09-11T08:42:24.655590+0000","last_active":"2024-09-11T08:43:36.130912+0000","last_peered":"2024-09-11T08:43:36.130912+0000","last_clean":"2024-09-11T08:43:36.130912+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:43:36.130912+0000","last_undegraded":"2024-09-11T08:43:36.130912+0000","last_fullsized":"2024-09-11T08:43:36.130912+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:42:24.655488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:42:24.655488+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:48.696109+0000","scrub_duration":0.014786436,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123929+0000","last_change":"2024-09-11T08:42:41.184901+0000","last_active":"2024-09-11T08:43:36.123929+0000","last_peered":"2024-09-11T08:43:36.123929+0000","last_clean":"2024-09-11T08:43:36.123929+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:43:36.123929+0000","last_undegraded":"2024-09-11T08:43:36.123929+0000","last_fullsized":"2024-09-11T08:43:36.123929+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:41.184788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:41.184788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:52.636191+0000","scrub_duration":0.005508303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.125198+0000","last_change":"2024-09-11T08:43:24.014133+0000","last_active":"2024-09-11T08:43:36.125198+0000","last_peered":"2024-09-11T08:43:36.125198+0000","last_clean":"2024-09-11T08:43:36.125198+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:43:36.125198+0000","last_undegraded":"2024-09-11T08:43:36.125198+0000","last_fullsized":"2024-09-11T08:43:36.125198+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:43:24.014057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:24.014057+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:44.940101+0000","scrub_duration":0.0037254929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.119019+0000","last_change":"2024-09-11T08:43:27.542277+0000","last_active":"2024-09-11T08:43:36.119019+0000","last_peered":"2024-09-11T08:43:36.119019+0000","last_clean":"2024-09-11T08:43:36.119019+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:43:36.119019+0000","last_undegraded":"2024-09-11T08:43:36.119019+0000","last_fullsized":"2024-09-11T08:43:36.119019+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:27.542189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:27.542189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:27.664241+0000","scrub_duration":0.0020045390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":113,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.144101+0000","last_change":"2024-09-11T08:41:42.846955+0000","last_active":"2024-09-11T08:43:36.144101+0000","last_peered":"2024-09-11T08:43:36.144101+0000","last_clean":"2024-09-11T08:43:36.144101+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:43:36.144101+0000","last_undegraded":"2024-09-11T08:43:36.144101+0000","last_fullsized":"2024-09-11T08:43:36.144101+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:41:42.846865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:41:42.846865+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013901284999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":177,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:39.166789+0000","last_change":"2024-09-11T08:43:35.027781+0000","last_active":"2024-09-11T08:43:39.166789+0000","last_peered":"2024-09-11T08:43:39.166789+0000","last_clean":"2024-09-11T08:43:39.166789+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:43:39.166789+0000","last_undegraded":"2024-09-11T08:43:39.166789+0000","last_fullsized":"2024-09-11T08:43:39.166789+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:43:35.027725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:35.027725+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:45:00.160919+0000","scrub_duration":0.0030466019999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"127'11859","reported_seq":12186,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:46.032664+0000","last_change":"2024-09-11T08:42:25.722233+0000","last_active":"2024-09-11T08:43:46.032664+0000","last_peered":"2024-09-11T08:43:46.032664+0000","last_clean":"2024-09-11T08:43:46.032664+0000","last_became_active":"2024-09-11T08:40:59.754455+0000","last_became_peered":"2024-09-11T08:40:59.754455+0000","last_unstale":"2024-09-11T08:43:46.032664+0000","last_undegraded":"2024-09-11T08:43:46.032664+0000","last_fullsized":"2024-09-11T08:43:46.032664+0000","mapping_epoch":88,"log_start":"103'5600","ondisk_log_start":"103'5600","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6791","last_scrub_stamp":"2024-09-11T08:42:25.722182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:25.722182+0000","objects_scrubbed":6745,"log_size":6259,"log_dups_size":0,"ondisk_log_size":6259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:49.298754+0000","scrub_duration":1.8191191760000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":235732992,"num_objects":3597,"num_object_clones":0,"num_object_copies":10791,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11859,"num_write_kb":494592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"124'22","reported_seq":140,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.118772+0000","last_change":"2024-09-11T08:43:33.630995+0000","last_active":"2024-09-11T08:43:36.118772+0000","last_peered":"2024-09-11T08:43:36.118772+0000","last_clean":"2024-09-11T08:43:36.118772+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:43:36.118772+0000","last_undegraded":"2024-09-11T08:43:36.118772+0000","last_fullsized":"2024-09-11T08:43:36.118772+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.630901+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:45:03.420729+0000","scrub_duration":0.01477299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":99,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.135935+0000","last_change":"2024-09-11T08:42:35.769631+0000","last_active":"2024-09-11T08:43:36.135935+0000","last_peered":"2024-09-11T08:43:36.135935+0000","last_clean":"2024-09-11T08:43:36.135935+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:43:36.135935+0000","last_undegraded":"2024-09-11T08:43:36.135935+0000","last_fullsized":"2024-09-11T08:43:36.135935+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:35.769558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:35.769558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:44.560878+0000","scrub_duration":0.0096185300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":165,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:39.164488+0000","last_change":"2024-09-11T08:42:43.134695+0000","last_active":"2024-09-11T08:43:39.164488+0000","last_peered":"2024-09-11T08:43:39.164488+0000","last_clean":"2024-09-11T08:43:39.164488+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:43:39.164488+0000","last_undegraded":"2024-09-11T08:43:39.164488+0000","last_fullsized":"2024-09-11T08:43:39.164488+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:42:43.134610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:43.134610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:56.769219+0000","scrub_duration":0.0084874449999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116744+0000","last_change":"2024-09-11T08:42:34.555469+0000","last_active":"2024-09-11T08:43:36.116744+0000","last_peered":"2024-09-11T08:43:36.116744+0000","last_clean":"2024-09-11T08:43:36.116744+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:43:36.116744+0000","last_undegraded":"2024-09-11T08:43:36.116744+0000","last_fullsized":"2024-09-11T08:43:36.116744+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:34.555379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:34.555379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:54.229410+0000","scrub_duration":0.0073682610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"127'11968","reported_seq":12015,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:46.317430+0000","last_change":"2024-09-11T08:40:59.753044+0000","last_active":"2024-09-11T08:43:46.317430+0000","last_peered":"2024-09-11T08:43:46.317430+0000","last_clean":"2024-09-11T08:43:46.317430+0000","last_became_active":"2024-09-11T08:40:59.752731+0000","last_became_peered":"2024-09-11T08:40:59.752731+0000","last_unstale":"2024-09-11T08:43:46.317430+0000","last_undegraded":"2024-09-11T08:43:46.317430+0000","last_fullsized":"2024-09-11T08:43:46.317430+0000","mapping_epoch":88,"log_start":"108'7300","ondisk_log_start":"108'7300","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":4668,"log_dups_size":0,"ondisk_log_size":4668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":252313600,"num_objects":3850,"num_object_clones":0,"num_object_copies":11550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11968,"num_write_kb":506176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":129,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.135756+0000","last_change":"2024-09-11T08:41:45.395630+0000","last_active":"2024-09-11T08:43:36.135756+0000","last_peered":"2024-09-11T08:43:36.135756+0000","last_clean":"2024-09-11T08:43:36.135756+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:43:36.135756+0000","last_undegraded":"2024-09-11T08:43:36.135756+0000","last_fullsized":"2024-09-11T08:43:36.135756+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-11T08:41:45.395547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:41:45.395547+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060283000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":99,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.133460+0000","last_change":"2024-09-11T08:41:43.421072+0000","last_active":"2024-09-11T08:43:36.133460+0000","last_peered":"2024-09-11T08:43:36.133460+0000","last_clean":"2024-09-11T08:43:36.133460+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:43:36.133460+0000","last_undegraded":"2024-09-11T08:43:36.133460+0000","last_fullsized":"2024-09-11T08:43:36.133460+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:43.421000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:41:43.421000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015196105,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123521+0000","last_change":"2024-09-11T08:42:30.305683+0000","last_active":"2024-09-11T08:43:36.123521+0000","last_peered":"2024-09-11T08:43:36.123521+0000","last_clean":"2024-09-11T08:43:36.123521+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:43:36.123521+0000","last_undegraded":"2024-09-11T08:43:36.123521+0000","last_fullsized":"2024-09-11T08:43:36.123521+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:30.305585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:30.305585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:49.696009+0000","scrub_duration":0.0023568949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123556+0000","last_change":"2024-09-11T08:43:21.980665+0000","last_active":"2024-09-11T08:43:36.123556+0000","last_peered":"2024-09-11T08:43:36.123556+0000","last_clean":"2024-09-11T08:43:36.123556+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:43:36.123556+0000","last_undegraded":"2024-09-11T08:43:36.123556+0000","last_fullsized":"2024-09-11T08:43:36.123556+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:21.980584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:21.980584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:22.213311+0000","scrub_duration":0.019382835000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"127'12049","reported_seq":12096,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:46.311503+0000","last_change":"2024-09-11T08:40:59.753523+0000","last_active":"2024-09-11T08:43:46.311503+0000","last_peered":"2024-09-11T08:43:46.311503+0000","last_clean":"2024-09-11T08:43:46.311503+0000","last_became_active":"2024-09-11T08:40:59.752898+0000","last_became_peered":"2024-09-11T08:40:59.752898+0000","last_unstale":"2024-09-11T08:43:46.311503+0000","last_undegraded":"2024-09-11T08:43:46.311503+0000","last_fullsized":"2024-09-11T08:43:46.311503+0000","mapping_epoch":88,"log_start":"108'7400","ondisk_log_start":"108'7400","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":4649,"log_dups_size":0,"ondisk_log_size":4649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":248053760,"num_objects":3785,"num_object_clones":0,"num_object_copies":11355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12049,"num_write_kb":506688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"86'21","reported_seq":126,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123717+0000","last_change":"2024-09-11T08:43:30.997804+0000","last_active":"2024-09-11T08:43:36.123717+0000","last_peered":"2024-09-11T08:43:36.123717+0000","last_clean":"2024-09-11T08:43:36.123717+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:43:36.123717+0000","last_undegraded":"2024-09-11T08:43:36.123717+0000","last_fullsized":"2024-09-11T08:43:36.123717+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'21","last_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:43:30.997710+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:45.668349+0000","scrub_duration":0.0025287299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.144474+0000","last_change":"2024-09-11T08:43:34.347814+0000","last_active":"2024-09-11T08:43:36.144474+0000","last_peered":"2024-09-11T08:43:36.144474+0000","last_clean":"2024-09-11T08:43:36.144474+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:43:36.144474+0000","last_undegraded":"2024-09-11T08:43:36.144474+0000","last_fullsized":"2024-09-11T08:43:36.144474+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:34.346299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:34.346299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:37.960188+0000","scrub_duration":0.0059519810000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":157,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:39.167676+0000","last_change":"2024-09-11T08:40:51.309316+0000","last_active":"2024-09-11T08:43:39.167676+0000","last_peered":"2024-09-11T08:43:39.167676+0000","last_clean":"2024-09-11T08:43:39.167676+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:43:39.167676+0000","last_undegraded":"2024-09-11T08:43:39.167676+0000","last_fullsized":"2024-09-11T08:43:39.167676+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:51.309259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:51.309259+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017959670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":103,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.125082+0000","last_change":"2024-09-11T08:43:22.119204+0000","last_active":"2024-09-11T08:43:36.125082+0000","last_peered":"2024-09-11T08:43:36.125082+0000","last_clean":"2024-09-11T08:43:36.125082+0000","last_became_active":"2024-09-11T08:38:45.182724+0000","last_became_peered":"2024-09-11T08:38:45.182724+0000","last_unstale":"2024-09-11T08:43:36.125082+0000","last_undegraded":"2024-09-11T08:43:36.125082+0000","last_fullsized":"2024-09-11T08:43:36.125082+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:22.119108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:22.119108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:38.805197+0000","scrub_duration":0.093065431000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"127'11934","reported_seq":11981,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:46.002924+0000","last_change":"2024-09-11T08:40:59.753707+0000","last_active":"2024-09-11T08:43:46.002924+0000","last_peered":"2024-09-11T08:43:46.002924+0000","last_clean":"2024-09-11T08:43:46.002924+0000","last_became_active":"2024-09-11T08:40:59.753449+0000","last_became_peered":"2024-09-11T08:40:59.753449+0000","last_unstale":"2024-09-11T08:43:46.002924+0000","last_undegraded":"2024-09-11T08:43:46.002924+0000","last_fullsized":"2024-09-11T08:43:46.002924+0000","mapping_epoch":88,"log_start":"108'7100","ondisk_log_start":"108'7100","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":4834,"log_dups_size":0,"ondisk_log_size":4834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":246677504,"num_objects":3764,"num_object_clones":0,"num_object_copies":11292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11934,"num_write_kb":502336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"100'15","reported_seq":128,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.603065+0000","last_change":"2024-09-11T08:43:36.602971+0000","last_active":"2024-09-11T08:43:36.603065+0000","last_peered":"2024-09-11T08:43:36.603065+0000","last_clean":"2024-09-11T08:43:36.603065+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:43:36.603065+0000","last_undegraded":"2024-09-11T08:43:36.603065+0000","last_fullsized":"2024-09-11T08:43:36.603065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'15","last_scrub_stamp":"2024-09-11T08:43:36.602876+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:41:56.250393+0000","last_clean_scrub_stamp":"2024-09-11T08:43:36.602876+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:39.522462+0000","scrub_duration":0.0051581880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":100,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.117094+0000","last_change":"2024-09-11T08:42:27.472376+0000","last_active":"2024-09-11T08:43:36.117094+0000","last_peered":"2024-09-11T08:43:36.117094+0000","last_clean":"2024-09-11T08:43:36.117094+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:43:36.117094+0000","last_undegraded":"2024-09-11T08:43:36.117094+0000","last_fullsized":"2024-09-11T08:43:36.117094+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:27.472327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:27.472327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:46.139070+0000","scrub_duration":0.001165318,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":165,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:39.161647+0000","last_change":"2024-09-11T08:42:28.036180+0000","last_active":"2024-09-11T08:43:39.161647+0000","last_peered":"2024-09-11T08:43:39.161647+0000","last_clean":"2024-09-11T08:43:39.161647+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:43:39.161647+0000","last_undegraded":"2024-09-11T08:43:39.161647+0000","last_fullsized":"2024-09-11T08:43:39.161647+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:42:28.036083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:28.036083+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:57.370485+0000","scrub_duration":0.0022379549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.135777+0000","last_change":"2024-09-11T08:42:32.112362+0000","last_active":"2024-09-11T08:43:36.135777+0000","last_peered":"2024-09-11T08:43:36.135777+0000","last_clean":"2024-09-11T08:43:36.135777+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:43:36.135777+0000","last_undegraded":"2024-09-11T08:43:36.135777+0000","last_fullsized":"2024-09-11T08:43:36.135777+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:32.112285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:42:32.112285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:39.646427+0000","scrub_duration":0.0073483730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"127'14983","reported_seq":15407,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:46.026370+0000","last_change":"2024-09-11T08:43:19.659282+0000","last_active":"2024-09-11T08:43:46.026370+0000","last_peered":"2024-09-11T08:43:46.026370+0000","last_clean":"2024-09-11T08:43:46.026370+0000","last_became_active":"2024-09-11T08:40:59.753346+0000","last_became_peered":"2024-09-11T08:40:59.753346+0000","last_unstale":"2024-09-11T08:43:46.026370+0000","last_undegraded":"2024-09-11T08:43:46.026370+0000","last_fullsized":"2024-09-11T08:43:46.026370+0000","mapping_epoch":88,"log_start":"112'9200","ondisk_log_start":"112'9200","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"121'13223","last_scrub_stamp":"2024-09-11T08:43:19.659224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:43:19.659224+0000","objects_scrubbed":5482,"log_size":5783,"log_dups_size":0,"ondisk_log_size":5783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:43.964355+0000","scrub_duration":1.9369766209999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239992832,"num_objects":3662,"num_object_clones":0,"num_object_copies":10986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12016,"num_write_kb":501696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":134,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.143727+0000","last_change":"2024-09-11T08:42:45.105667+0000","last_active":"2024-09-11T08:43:36.143727+0000","last_peered":"2024-09-11T08:43:36.143727+0000","last_clean":"2024-09-11T08:43:36.143727+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:43:36.143727+0000","last_undegraded":"2024-09-11T08:43:36.143727+0000","last_fullsized":"2024-09-11T08:43:36.143727+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:42:45.104549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:42:45.104549+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:14.762641+0000","scrub_duration":0.0073378940000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.119480+0000","last_change":"2024-09-11T08:43:19.502930+0000","last_active":"2024-09-11T08:43:36.119480+0000","last_peered":"2024-09-11T08:43:36.119480+0000","last_clean":"2024-09-11T08:43:36.119480+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:43:36.119480+0000","last_undegraded":"2024-09-11T08:43:36.119480+0000","last_fullsized":"2024-09-11T08:43:36.119480+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:19.502844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:19.502844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:26.390433+0000","scrub_duration":0.0023577619999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":105,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.121847+0000","last_change":"2024-09-11T08:43:23.161380+0000","last_active":"2024-09-11T08:43:36.121847+0000","last_peered":"2024-09-11T08:43:36.121847+0000","last_clean":"2024-09-11T08:43:36.121847+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:43:36.121847+0000","last_undegraded":"2024-09-11T08:43:36.121847+0000","last_fullsized":"2024-09-11T08:43:36.121847+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:23.161300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:43:23.161300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:40.395478+0000","scrub_duration":0.0020977869999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.118884+0000","last_change":"2024-09-11T08:43:31.631898+0000","last_active":"2024-09-11T08:43:36.118884+0000","last_peered":"2024-09-11T08:43:36.118884+0000","last_clean":"2024-09-11T08:43:36.118884+0000","last_became_active":"2024-09-11T08:38:45.182782+0000","last_became_peered":"2024-09-11T08:38:45.182782+0000","last_unstale":"2024-09-11T08:43:36.118884+0000","last_undegraded":"2024-09-11T08:43:36.118884+0000","last_fullsized":"2024-09-11T08:43:36.118884+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:43:31.631810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:52.381803+0000","scrub_duration":0.0074672260000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"127'11892","reported_seq":12260,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:46.002861+0000","last_change":"2024-09-11T08:42:43.178301+0000","last_active":"2024-09-11T08:43:46.002861+0000","last_peered":"2024-09-11T08:43:46.002861+0000","last_clean":"2024-09-11T08:43:46.002861+0000","last_became_active":"2024-09-11T08:40:59.757849+0000","last_became_peered":"2024-09-11T08:40:59.757849+0000","last_unstale":"2024-09-11T08:43:46.002861+0000","last_undegraded":"2024-09-11T08:43:46.002861+0000","last_fullsized":"2024-09-11T08:43:46.002861+0000","mapping_epoch":88,"log_start":"103'5600","ondisk_log_start":"103'5600","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7880","last_scrub_stamp":"2024-09-11T08:42:43.178246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:43.178246+0000","objects_scrubbed":7738,"log_size":6292,"log_dups_size":0,"ondisk_log_size":6292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:02.385368+0000","scrub_duration":4.2909816630000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243793920,"num_objects":3720,"num_object_clones":0,"num_object_copies":11160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11892,"num_write_kb":499584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":227,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:39.159688+0000","last_change":"2024-09-11T08:42:39.259633+0000","last_active":"2024-09-11T08:43:39.159688+0000","last_peered":"2024-09-11T08:43:39.159688+0000","last_clean":"2024-09-11T08:43:39.159688+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:43:39.159688+0000","last_undegraded":"2024-09-11T08:43:39.159688+0000","last_fullsized":"2024-09-11T08:43:39.159688+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:42:39.259542+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:42:39.259542+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:51.826159+0000","scrub_duration":0.012956343,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":100,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124043+0000","last_change":"2024-09-11T08:42:36.267727+0000","last_active":"2024-09-11T08:43:36.124043+0000","last_peered":"2024-09-11T08:43:36.124043+0000","last_clean":"2024-09-11T08:43:36.124043+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:43:36.124043+0000","last_undegraded":"2024-09-11T08:43:36.124043+0000","last_fullsized":"2024-09-11T08:43:36.124043+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:36.267644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:36.267644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:40.844732+0000","scrub_duration":0.002745029,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":117,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:38.670059+0000","last_change":"2024-09-11T08:43:38.669897+0000","last_active":"2024-09-11T08:43:38.670059+0000","last_peered":"2024-09-11T08:43:38.670059+0000","last_clean":"2024-09-11T08:43:38.670059+0000","last_became_active":"2024-09-11T08:38:38.947270+0000","last_became_peered":"2024-09-11T08:38:38.947270+0000","last_unstale":"2024-09-11T08:43:38.670059+0000","last_undegraded":"2024-09-11T08:43:38.670059+0000","last_fullsized":"2024-09-11T08:43:38.670059+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:43:38.669837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:38.669837+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:45.074560+0000","scrub_duration":0.001723715,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.150756+0000","last_change":"2024-09-11T08:41:52.983114+0000","last_active":"2024-09-11T08:43:36.150756+0000","last_peered":"2024-09-11T08:43:36.150756+0000","last_clean":"2024-09-11T08:43:36.150756+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:43:36.150756+0000","last_undegraded":"2024-09-11T08:43:36.150756+0000","last_fullsized":"2024-09-11T08:43:36.150756+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:52.983038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:41:52.983038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0098101790000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.132674+0000","last_change":"2024-09-11T08:42:49.847720+0000","last_active":"2024-09-11T08:43:36.132674+0000","last_peered":"2024-09-11T08:43:36.132674+0000","last_clean":"2024-09-11T08:43:36.132674+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:43:36.132674+0000","last_undegraded":"2024-09-11T08:43:36.132674+0000","last_fullsized":"2024-09-11T08:43:36.132674+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:49.847548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:49.847548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:07.405718+0000","scrub_duration":0.011241151,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"127'11840","reported_seq":12181,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:46.400417+0000","last_change":"2024-09-11T08:42:53.794983+0000","last_active":"2024-09-11T08:43:46.400417+0000","last_peered":"2024-09-11T08:43:46.400417+0000","last_clean":"2024-09-11T08:43:46.400417+0000","last_became_active":"2024-09-11T08:40:59.756563+0000","last_became_peered":"2024-09-11T08:40:59.756563+0000","last_unstale":"2024-09-11T08:43:46.400417+0000","last_undegraded":"2024-09-11T08:43:46.400417+0000","last_fullsized":"2024-09-11T08:43:46.400417+0000","mapping_epoch":88,"log_start":"107'6200","ondisk_log_start":"107'6200","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'8427","last_scrub_stamp":"2024-09-11T08:42:53.794889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:53.794889+0000","objects_scrubbed":7152,"log_size":5640,"log_dups_size":0,"ondisk_log_size":5640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:02.167218+0000","scrub_duration":2.7347385429999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240254976,"num_objects":3666,"num_object_clones":0,"num_object_copies":10998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11840,"num_write_kb":496192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":135,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.118849+0000","last_change":"2024-09-11T08:42:38.313016+0000","last_active":"2024-09-11T08:43:36.118849+0000","last_peered":"2024-09-11T08:43:36.118849+0000","last_clean":"2024-09-11T08:43:36.118849+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:43:36.118849+0000","last_undegraded":"2024-09-11T08:43:36.118849+0000","last_fullsized":"2024-09-11T08:43:36.118849+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:42:38.312937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:42:38.312937+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.026297491999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.118690+0000","last_change":"2024-09-11T08:43:20.533696+0000","last_active":"2024-09-11T08:43:36.118690+0000","last_peered":"2024-09-11T08:43:36.118690+0000","last_clean":"2024-09-11T08:43:36.118690+0000","last_became_active":"2024-09-11T08:38:38.949019+0000","last_became_peered":"2024-09-11T08:38:38.949019+0000","last_unstale":"2024-09-11T08:43:36.118690+0000","last_undegraded":"2024-09-11T08:43:36.118690+0000","last_fullsized":"2024-09-11T08:43:36.118690+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:20.533619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:20.533619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:27.628937+0000","scrub_duration":0.0026926540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":116,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.126335+0000","last_change":"2024-09-11T08:42:49.606256+0000","last_active":"2024-09-11T08:43:36.126335+0000","last_peered":"2024-09-11T08:43:36.126335+0000","last_clean":"2024-09-11T08:43:36.126335+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:43:36.126335+0000","last_undegraded":"2024-09-11T08:43:36.126335+0000","last_fullsized":"2024-09-11T08:43:36.126335+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:42:49.606178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:49.606178+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:52.569663+0000","scrub_duration":0.0073928789999999998,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.128728+0000","last_change":"2024-09-11T08:43:21.506389+0000","last_active":"2024-09-11T08:43:36.128728+0000","last_peered":"2024-09-11T08:43:36.128728+0000","last_clean":"2024-09-11T08:43:36.128728+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:43:36.128728+0000","last_undegraded":"2024-09-11T08:43:36.128728+0000","last_fullsized":"2024-09-11T08:43:36.128728+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:21.506299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_clean_scrub_stamp":"2024-09-11T08:43:21.506299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:45.821602+0000","scrub_duration":0.0016198600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":103,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.126124+0000","last_change":"2024-09-11T08:43:25.055354+0000","last_active":"2024-09-11T08:43:36.126124+0000","last_peered":"2024-09-11T08:43:36.126124+0000","last_clean":"2024-09-11T08:43:36.126124+0000","last_became_active":"2024-09-11T08:38:45.183582+0000","last_became_peered":"2024-09-11T08:38:45.183582+0000","last_unstale":"2024-09-11T08:43:36.126124+0000","last_undegraded":"2024-09-11T08:43:36.126124+0000","last_fullsized":"2024-09-11T08:43:36.126124+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:25.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:25.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:40.207781+0000","scrub_duration":0.0032371499999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"127'12028","reported_seq":12387,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:45.973930+0000","last_change":"2024-09-11T08:42:48.186060+0000","last_active":"2024-09-11T08:43:45.973930+0000","last_peered":"2024-09-11T08:43:45.973930+0000","last_clean":"2024-09-11T08:43:45.973930+0000","last_became_active":"2024-09-11T08:40:59.755652+0000","last_became_peered":"2024-09-11T08:40:59.755652+0000","last_unstale":"2024-09-11T08:43:45.973930+0000","last_undegraded":"2024-09-11T08:43:45.973930+0000","last_fullsized":"2024-09-11T08:43:45.973930+0000","mapping_epoch":88,"log_start":"108'6800","ondisk_log_start":"108'6800","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'8307","last_scrub_stamp":"2024-09-11T08:42:48.185997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:48.185997+0000","objects_scrubbed":7567,"log_size":5228,"log_dups_size":0,"ondisk_log_size":5228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:00.620699+0000","scrub_duration":2.2213046419999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247201792,"num_objects":3772,"num_object_clones":0,"num_object_copies":11316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12028,"num_write_kb":505600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":116,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.144318+0000","last_change":"2024-09-11T08:41:56.978529+0000","last_active":"2024-09-11T08:43:36.144318+0000","last_peered":"2024-09-11T08:43:36.144318+0000","last_clean":"2024-09-11T08:43:36.144318+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:43:36.144318+0000","last_undegraded":"2024-09-11T08:43:36.144318+0000","last_fullsized":"2024-09-11T08:43:36.144318+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:41:56.978383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:41:56.978383+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017271161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.136432+0000","last_change":"2024-09-11T08:43:27.359104+0000","last_active":"2024-09-11T08:43:36.136432+0000","last_peered":"2024-09-11T08:43:36.136432+0000","last_clean":"2024-09-11T08:43:36.136432+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:43:36.136432+0000","last_undegraded":"2024-09-11T08:43:36.136432+0000","last_fullsized":"2024-09-11T08:43:36.136432+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:27.359029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:27.359029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:36.281548+0000","scrub_duration":0.002472747,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":417,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:44.504223+0000","last_change":"2024-09-11T08:42:47.044666+0000","last_active":"2024-09-11T08:43:44.504223+0000","last_peered":"2024-09-11T08:43:44.504223+0000","last_clean":"2024-09-11T08:43:44.504223+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:43:44.504223+0000","last_undegraded":"2024-09-11T08:43:44.504223+0000","last_fullsized":"2024-09-11T08:43:44.504223+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:42:47.044598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:47.044598+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:56.754879+0000","scrub_duration":0.086046497999999999,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":300,"num_read_kb":295,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.117465+0000","last_change":"2024-09-11T08:42:36.575430+0000","last_active":"2024-09-11T08:43:36.117465+0000","last_peered":"2024-09-11T08:43:36.117465+0000","last_clean":"2024-09-11T08:43:36.117465+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:43:36.117465+0000","last_undegraded":"2024-09-11T08:43:36.117465+0000","last_fullsized":"2024-09-11T08:43:36.117465+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:36.575347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:36.575347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:42.841995+0000","scrub_duration":0.0078670330000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.119078+0000","last_change":"2024-09-11T08:43:35.596874+0000","last_active":"2024-09-11T08:43:36.119078+0000","last_peered":"2024-09-11T08:43:36.119078+0000","last_clean":"2024-09-11T08:43:36.119078+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:43:36.119078+0000","last_undegraded":"2024-09-11T08:43:36.119078+0000","last_fullsized":"2024-09-11T08:43:36.119078+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:35.596727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:43:35.596727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:45:02.202079+0000","scrub_duration":0.019062624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"127'11655","reported_seq":11996,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:46.382019+0000","last_change":"2024-09-11T08:42:34.330523+0000","last_active":"2024-09-11T08:43:46.382019+0000","last_peered":"2024-09-11T08:43:46.382019+0000","last_clean":"2024-09-11T08:43:46.382019+0000","last_became_active":"2024-09-11T08:40:59.755912+0000","last_became_peered":"2024-09-11T08:40:59.755912+0000","last_unstale":"2024-09-11T08:43:46.382019+0000","last_undegraded":"2024-09-11T08:43:46.382019+0000","last_fullsized":"2024-09-11T08:43:46.382019+0000","mapping_epoch":88,"log_start":"107'6000","ondisk_log_start":"107'6000","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'7230","last_scrub_stamp":"2024-09-11T08:42:34.330327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:34.330327+0000","objects_scrubbed":7114,"log_size":5655,"log_dups_size":0,"ondisk_log_size":5655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"queued for scrub","scrub_duration":4.2856508010000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240713728,"num_objects":3673,"num_object_clones":0,"num_object_copies":11019,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11655,"num_write_kb":490496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":129,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.122299+0000","last_change":"2024-09-11T08:43:26.127930+0000","last_active":"2024-09-11T08:43:36.122299+0000","last_peered":"2024-09-11T08:43:36.122299+0000","last_clean":"2024-09-11T08:43:36.122299+0000","last_became_active":"2024-09-11T08:38:41.040948+0000","last_became_peered":"2024-09-11T08:38:41.040948+0000","last_unstale":"2024-09-11T08:43:36.122299+0000","last_undegraded":"2024-09-11T08:43:36.122299+0000","last_fullsized":"2024-09-11T08:43:36.122299+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:43:26.127412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:26.127412+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:39.494701+0000","scrub_duration":0.0029787630000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":108,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.132035+0000","last_change":"2024-09-11T08:42:28.766183+0000","last_active":"2024-09-11T08:43:36.132035+0000","last_peered":"2024-09-11T08:43:36.132035+0000","last_clean":"2024-09-11T08:43:36.132035+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:43:36.132035+0000","last_undegraded":"2024-09-11T08:43:36.132035+0000","last_fullsized":"2024-09-11T08:43:36.132035+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:42:28.765466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:28.765466+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:55.584403+0000","scrub_duration":0.0053952599999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124997+0000","last_change":"2024-09-11T08:42:23.973455+0000","last_active":"2024-09-11T08:43:36.124997+0000","last_peered":"2024-09-11T08:43:36.124997+0000","last_clean":"2024-09-11T08:43:36.124997+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:43:36.124997+0000","last_undegraded":"2024-09-11T08:43:36.124997+0000","last_fullsized":"2024-09-11T08:43:36.124997+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'79","last_scrub_stamp":"2024-09-11T08:42:23.973378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:42:23.973378+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:49.122950+0000","scrub_duration":0.0076470410000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":105,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.125335+0000","last_change":"2024-09-11T08:43:19.974333+0000","last_active":"2024-09-11T08:43:36.125335+0000","last_peered":"2024-09-11T08:43:36.125335+0000","last_clean":"2024-09-11T08:43:36.125335+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:43:36.125335+0000","last_undegraded":"2024-09-11T08:43:36.125335+0000","last_fullsized":"2024-09-11T08:43:36.125335+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:19.974257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:19.974257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:22.526340+0000","scrub_duration":0.0018788069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.117378+0000","last_change":"2024-09-11T08:43:29.570321+0000","last_active":"2024-09-11T08:43:36.117378+0000","last_peered":"2024-09-11T08:43:36.117378+0000","last_clean":"2024-09-11T08:43:36.117378+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:43:36.117378+0000","last_undegraded":"2024-09-11T08:43:36.117378+0000","last_fullsized":"2024-09-11T08:43:36.117378+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.570246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.570246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:58.852687+0000","scrub_duration":0.0041809610000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"127'11949","reported_seq":12316,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:46.030032+0000","last_change":"2024-09-11T08:42:44.958701+0000","last_active":"2024-09-11T08:43:46.030032+0000","last_peered":"2024-09-11T08:43:46.030032+0000","last_clean":"2024-09-11T08:43:46.030032+0000","last_became_active":"2024-09-11T08:40:59.755894+0000","last_became_peered":"2024-09-11T08:40:59.755894+0000","last_unstale":"2024-09-11T08:43:46.030032+0000","last_undegraded":"2024-09-11T08:43:46.030032+0000","last_fullsized":"2024-09-11T08:43:46.030032+0000","mapping_epoch":88,"log_start":"103'5600","ondisk_log_start":"103'5600","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'8054","last_scrub_stamp":"2024-09-11T08:42:44.958606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:44.958606+0000","objects_scrubbed":7720,"log_size":6349,"log_dups_size":0,"ondisk_log_size":6349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:55.311677+0000","scrub_duration":3.0641676269999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":244842520,"num_objects":3737,"num_object_clones":0,"num_object_copies":11211,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11949,"num_write_kb":501889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":139,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124440+0000","last_change":"2024-09-11T08:43:21.002231+0000","last_active":"2024-09-11T08:43:36.124440+0000","last_peered":"2024-09-11T08:43:36.124440+0000","last_clean":"2024-09-11T08:43:36.124440+0000","last_became_active":"2024-09-11T08:40:16.183867+0000","last_became_peered":"2024-09-11T08:40:16.183867+0000","last_unstale":"2024-09-11T08:43:36.124440+0000","last_undegraded":"2024-09-11T08:43:36.124440+0000","last_fullsized":"2024-09-11T08:43:36.124440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:21.002140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:21.002140+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:25.563338+0000","scrub_duration":0.0025090400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":90,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123790+0000","last_change":"2024-09-11T08:40:54.671894+0000","last_active":"2024-09-11T08:43:36.123790+0000","last_peered":"2024-09-11T08:43:36.123790+0000","last_clean":"2024-09-11T08:43:36.123790+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:43:36.123790+0000","last_undegraded":"2024-09-11T08:43:36.123790+0000","last_fullsized":"2024-09-11T08:43:36.123790+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:40:54.671832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001438541,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.117273+0000","last_change":"2024-09-11T08:42:50.619383+0000","last_active":"2024-09-11T08:43:36.117273+0000","last_peered":"2024-09-11T08:43:36.117273+0000","last_clean":"2024-09-11T08:43:36.117273+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:43:36.117273+0000","last_undegraded":"2024-09-11T08:43:36.117273+0000","last_fullsized":"2024-09-11T08:43:36.117273+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:50.619330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:50.619330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:15.881053+0000","scrub_duration":0.001262085,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.131037+0000","last_change":"2024-09-11T08:42:40.853874+0000","last_active":"2024-09-11T08:43:36.131037+0000","last_peered":"2024-09-11T08:43:36.131037+0000","last_clean":"2024-09-11T08:43:36.131037+0000","last_became_active":"2024-09-11T08:38:45.184410+0000","last_became_peered":"2024-09-11T08:38:45.184410+0000","last_unstale":"2024-09-11T08:43:36.131037+0000","last_undegraded":"2024-09-11T08:43:36.131037+0000","last_fullsized":"2024-09-11T08:43:36.131037+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:40.853782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:40.853782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:41.342555+0000","scrub_duration":0.0045130580000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"127'12134","reported_seq":12473,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:46.112966+0000","last_change":"2024-09-11T08:42:28.551092+0000","last_active":"2024-09-11T08:43:46.112966+0000","last_peered":"2024-09-11T08:43:46.112966+0000","last_clean":"2024-09-11T08:43:46.112966+0000","last_became_active":"2024-09-11T08:40:59.755838+0000","last_became_peered":"2024-09-11T08:40:59.755838+0000","last_unstale":"2024-09-11T08:43:46.112966+0000","last_undegraded":"2024-09-11T08:43:46.112966+0000","last_fullsized":"2024-09-11T08:43:46.112966+0000","mapping_epoch":88,"log_start":"101'5300","ondisk_log_start":"101'5300","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'7153","last_scrub_stamp":"2024-09-11T08:42:28.550888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:28.550888+0000","objects_scrubbed":7055,"log_size":6834,"log_dups_size":0,"ondisk_log_size":6834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"queued for scrub","scrub_duration":4.1207098550000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241696768,"num_objects":3688,"num_object_clones":0,"num_object_copies":11064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12134,"num_write_kb":506304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":126,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124089+0000","last_change":"2024-09-11T08:41:45.328463+0000","last_active":"2024-09-11T08:43:36.124089+0000","last_peered":"2024-09-11T08:43:36.124089+0000","last_clean":"2024-09-11T08:43:36.124089+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:43:36.124089+0000","last_undegraded":"2024-09-11T08:43:36.124089+0000","last_fullsized":"2024-09-11T08:43:36.124089+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:41:45.328360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:41:45.328360+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048419429999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":109,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.122020+0000","last_change":"2024-09-11T08:42:51.805713+0000","last_active":"2024-09-11T08:43:36.122020+0000","last_peered":"2024-09-11T08:43:36.122020+0000","last_clean":"2024-09-11T08:43:36.122020+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:43:36.122020+0000","last_undegraded":"2024-09-11T08:43:36.122020+0000","last_fullsized":"2024-09-11T08:43:36.122020+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:51.805660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:51.805660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:17.456311+0000","scrub_duration":0.0032519020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.119143+0000","last_change":"2024-09-11T08:42:31.256936+0000","last_active":"2024-09-11T08:43:36.119143+0000","last_peered":"2024-09-11T08:43:36.119143+0000","last_clean":"2024-09-11T08:43:36.119143+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:43:36.119143+0000","last_undegraded":"2024-09-11T08:43:36.119143+0000","last_fullsized":"2024-09-11T08:43:36.119143+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:31.256727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:31.256727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:57.081019+0000","scrub_duration":0.055088536,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.127022+0000","last_change":"2024-09-11T08:43:27.580755+0000","last_active":"2024-09-11T08:43:36.127022+0000","last_peered":"2024-09-11T08:43:36.127022+0000","last_clean":"2024-09-11T08:43:36.127022+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:43:36.127022+0000","last_undegraded":"2024-09-11T08:43:36.127022+0000","last_fullsized":"2024-09-11T08:43:36.127022+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:27.580677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:27.580677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:31.501515+0000","scrub_duration":0.012590228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"127'11927","reported_seq":12290,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:46.391837+0000","last_change":"2024-09-11T08:42:41.749687+0000","last_active":"2024-09-11T08:43:46.391837+0000","last_peered":"2024-09-11T08:43:46.391837+0000","last_clean":"2024-09-11T08:43:46.391837+0000","last_became_active":"2024-09-11T08:40:59.755307+0000","last_became_peered":"2024-09-11T08:40:59.755307+0000","last_unstale":"2024-09-11T08:43:46.391837+0000","last_undegraded":"2024-09-11T08:43:46.391837+0000","last_fullsized":"2024-09-11T08:43:46.391837+0000","mapping_epoch":88,"log_start":"107'6300","ondisk_log_start":"107'6300","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7805","last_scrub_stamp":"2024-09-11T08:42:41.749598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:41.749598+0000","objects_scrubbed":7659,"log_size":5627,"log_dups_size":0,"ondisk_log_size":5627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:05.039544+0000","scrub_duration":5.7357807019999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240975872,"num_objects":3677,"num_object_clones":0,"num_object_copies":11031,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11927,"num_write_kb":499328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":113,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124066+0000","last_change":"2024-09-11T08:43:24.983794+0000","last_active":"2024-09-11T08:43:36.124066+0000","last_peered":"2024-09-11T08:43:36.124066+0000","last_clean":"2024-09-11T08:43:36.124066+0000","last_became_active":"2024-09-11T08:38:40.969348+0000","last_became_peered":"2024-09-11T08:38:40.969348+0000","last_unstale":"2024-09-11T08:43:36.124066+0000","last_undegraded":"2024-09-11T08:43:36.124066+0000","last_fullsized":"2024-09-11T08:43:36.124066+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:43:24.983740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:24.983740+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:38.537961+0000","scrub_duration":0.0069750209999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":99,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.133032+0000","last_change":"2024-09-11T08:43:24.975941+0000","last_active":"2024-09-11T08:43:36.133032+0000","last_peered":"2024-09-11T08:43:36.133032+0000","last_clean":"2024-09-11T08:43:36.133032+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:43:36.133032+0000","last_undegraded":"2024-09-11T08:43:36.133032+0000","last_fullsized":"2024-09-11T08:43:36.133032+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:24.975395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:24.975395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:50.930192+0000","scrub_duration":0.0047857070000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":175,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:39.154418+0000","last_change":"2024-09-11T08:43:22.553542+0000","last_active":"2024-09-11T08:43:39.154418+0000","last_peered":"2024-09-11T08:43:39.154418+0000","last_clean":"2024-09-11T08:43:39.154418+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:43:39.154418+0000","last_undegraded":"2024-09-11T08:43:39.154418+0000","last_fullsized":"2024-09-11T08:43:39.154418+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:43:22.553458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:22.553458+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:47.971884+0000","scrub_duration":0.0089134629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124618+0000","last_change":"2024-09-11T08:42:50.100332+0000","last_active":"2024-09-11T08:43:36.124618+0000","last_peered":"2024-09-11T08:43:36.124618+0000","last_clean":"2024-09-11T08:43:36.124618+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:43:36.124618+0000","last_undegraded":"2024-09-11T08:43:36.124618+0000","last_fullsized":"2024-09-11T08:43:36.124618+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:50.100253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:42:50.100253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:18.100785+0000","scrub_duration":0.013241047000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"127'12083","reported_seq":12511,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:46.133884+0000","last_change":"2024-09-11T08:42:30.989633+0000","last_active":"2024-09-11T08:43:46.133884+0000","last_peered":"2024-09-11T08:43:46.133884+0000","last_clean":"2024-09-11T08:43:46.133884+0000","last_became_active":"2024-09-11T08:40:59.754331+0000","last_became_peered":"2024-09-11T08:40:59.754331+0000","last_unstale":"2024-09-11T08:43:46.133884+0000","last_undegraded":"2024-09-11T08:43:46.133884+0000","last_fullsized":"2024-09-11T08:43:46.133884+0000","mapping_epoch":88,"log_start":"101'5200","ondisk_log_start":"101'5200","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'7272","last_scrub_stamp":"2024-09-11T08:42:30.989546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:30.989546+0000","objects_scrubbed":7197,"log_size":6883,"log_dups_size":0,"ondisk_log_size":6883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:50.349989+0000","scrub_duration":2.605776192,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243990528,"num_objects":3723,"num_object_clones":0,"num_object_copies":11169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12083,"num_write_kb":505792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":114,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.143561+0000","last_change":"2024-09-11T08:42:35.072141+0000","last_active":"2024-09-11T08:43:36.143561+0000","last_peered":"2024-09-11T08:43:36.143561+0000","last_clean":"2024-09-11T08:43:36.143561+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:43:36.143561+0000","last_undegraded":"2024-09-11T08:43:36.143561+0000","last_fullsized":"2024-09-11T08:43:36.143561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:42:35.072054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:42:35.072054+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:45.379420+0000","scrub_duration":0.022552168000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116863+0000","last_change":"2024-09-11T08:43:24.583745+0000","last_active":"2024-09-11T08:43:36.116863+0000","last_peered":"2024-09-11T08:43:36.116863+0000","last_clean":"2024-09-11T08:43:36.116863+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:43:36.116863+0000","last_undegraded":"2024-09-11T08:43:36.116863+0000","last_fullsized":"2024-09-11T08:43:36.116863+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:24.583665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:43:24.583665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:38.086636+0000","scrub_duration":0.0081591830000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.127671+0000","last_change":"2024-09-11T08:43:25.643701+0000","last_active":"2024-09-11T08:43:36.127671+0000","last_peered":"2024-09-11T08:43:36.127671+0000","last_clean":"2024-09-11T08:43:36.127671+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:43:36.127671+0000","last_undegraded":"2024-09-11T08:43:36.127671+0000","last_fullsized":"2024-09-11T08:43:36.127671+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:25.643625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:25.643625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:53.950125+0000","scrub_duration":0.010514694999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.136781+0000","last_change":"2024-09-11T08:42:27.029550+0000","last_active":"2024-09-11T08:43:36.136781+0000","last_peered":"2024-09-11T08:43:36.136781+0000","last_clean":"2024-09-11T08:43:36.136781+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:43:36.136781+0000","last_undegraded":"2024-09-11T08:43:36.136781+0000","last_fullsized":"2024-09-11T08:43:36.136781+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:27.029300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:27.029300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:41.134838+0000","scrub_duration":0.0029176419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"127'12094","reported_seq":12446,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:46.397274+0000","last_change":"2024-09-11T08:42:34.635136+0000","last_active":"2024-09-11T08:43:46.397274+0000","last_peered":"2024-09-11T08:43:46.397274+0000","last_clean":"2024-09-11T08:43:46.397274+0000","last_became_active":"2024-09-11T08:40:59.755928+0000","last_became_peered":"2024-09-11T08:40:59.755928+0000","last_unstale":"2024-09-11T08:43:46.397274+0000","last_undegraded":"2024-09-11T08:43:46.397274+0000","last_fullsized":"2024-09-11T08:43:46.397274+0000","mapping_epoch":88,"log_start":"107'6500","ondisk_log_start":"107'6500","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'7501","last_scrub_stamp":"2024-09-11T08:42:34.634925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:34.634925+0000","objects_scrubbed":7381,"log_size":5594,"log_dups_size":0,"ondisk_log_size":5594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3.5299405990000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":248119296,"num_objects":3786,"num_object_clones":0,"num_object_copies":11358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12094,"num_write_kb":508160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":138,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.121847+0000","last_change":"2024-09-11T08:43:19.146635+0000","last_active":"2024-09-11T08:43:36.121847+0000","last_peered":"2024-09-11T08:43:36.121847+0000","last_clean":"2024-09-11T08:43:36.121847+0000","last_became_active":"2024-09-11T08:38:41.040207+0000","last_became_peered":"2024-09-11T08:38:41.040207+0000","last_unstale":"2024-09-11T08:43:36.121847+0000","last_undegraded":"2024-09-11T08:43:36.121847+0000","last_fullsized":"2024-09-11T08:43:36.121847+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:43:19.146540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:19.146540+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:41.603801+0000","scrub_duration":0.0038449669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":109,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.126467+0000","last_change":"2024-09-11T08:42:51.960897+0000","last_active":"2024-09-11T08:43:36.126467+0000","last_peered":"2024-09-11T08:43:36.126467+0000","last_clean":"2024-09-11T08:43:36.126467+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:43:36.126467+0000","last_undegraded":"2024-09-11T08:43:36.126467+0000","last_fullsized":"2024-09-11T08:43:36.126467+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:51.960847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:51.960847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:11.954924+0000","scrub_duration":0.0017378840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.127344+0000","last_change":"2024-09-11T08:43:20.462355+0000","last_active":"2024-09-11T08:43:36.127344+0000","last_peered":"2024-09-11T08:43:36.127344+0000","last_clean":"2024-09-11T08:43:36.127344+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:43:36.127344+0000","last_undegraded":"2024-09-11T08:43:36.127344+0000","last_fullsized":"2024-09-11T08:43:36.127344+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:20.462184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:20.462184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:25.409093+0000","scrub_duration":0.0037598269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124902+0000","last_change":"2024-09-11T08:42:40.024673+0000","last_active":"2024-09-11T08:43:36.124902+0000","last_peered":"2024-09-11T08:43:36.124902+0000","last_clean":"2024-09-11T08:43:36.124902+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:43:36.124902+0000","last_undegraded":"2024-09-11T08:43:36.124902+0000","last_fullsized":"2024-09-11T08:43:36.124902+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:40.024586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:40.024586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:02.493996+0000","scrub_duration":0.0063827069999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"127'12224","reported_seq":12492,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:46.312477+0000","last_change":"2024-09-11T08:43:25.901583+0000","last_active":"2024-09-11T08:43:46.312477+0000","last_peered":"2024-09-11T08:43:46.312477+0000","last_clean":"2024-09-11T08:43:46.312477+0000","last_became_active":"2024-09-11T08:40:59.753273+0000","last_became_peered":"2024-09-11T08:40:59.753273+0000","last_unstale":"2024-09-11T08:43:46.312477+0000","last_undegraded":"2024-09-11T08:43:46.312477+0000","last_fullsized":"2024-09-11T08:43:46.312477+0000","mapping_epoch":88,"log_start":"108'7600","ondisk_log_start":"108'7600","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"122'10757","last_scrub_stamp":"2024-09-11T08:43:25.901499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:43:25.901499+0000","objects_scrubbed":5291,"log_size":4624,"log_dups_size":0,"ondisk_log_size":4624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:48.183540+0000","scrub_duration":2.8764546480000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":244449280,"num_objects":3730,"num_object_clones":0,"num_object_copies":11190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12224,"num_write_kb":510528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":121,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123600+0000","last_change":"2024-09-11T08:41:49.328291+0000","last_active":"2024-09-11T08:43:36.123600+0000","last_peered":"2024-09-11T08:43:36.123600+0000","last_clean":"2024-09-11T08:43:36.123600+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:43:36.123600+0000","last_undegraded":"2024-09-11T08:43:36.123600+0000","last_fullsized":"2024-09-11T08:43:36.123600+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:41:49.328238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:41:49.328238+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063967870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":109,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.125686+0000","last_change":"2024-09-11T08:43:17.025878+0000","last_active":"2024-09-11T08:43:36.125686+0000","last_peered":"2024-09-11T08:43:36.125686+0000","last_clean":"2024-09-11T08:43:36.125686+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:43:36.125686+0000","last_undegraded":"2024-09-11T08:43:36.125686+0000","last_fullsized":"2024-09-11T08:43:36.125686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:17.025803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:17.025803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:40.286553+0000","scrub_duration":0.0041414900000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.128113+0000","last_change":"2024-09-11T08:43:17.495291+0000","last_active":"2024-09-11T08:43:36.128113+0000","last_peered":"2024-09-11T08:43:36.128113+0000","last_clean":"2024-09-11T08:43:36.128113+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:43:36.128113+0000","last_undegraded":"2024-09-11T08:43:36.128113+0000","last_fullsized":"2024-09-11T08:43:36.128113+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:17.495214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:17.495214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:25.389508+0000","scrub_duration":0.0045506940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.132773+0000","last_change":"2024-09-11T08:43:28.055111+0000","last_active":"2024-09-11T08:43:36.132773+0000","last_peered":"2024-09-11T08:43:36.132773+0000","last_clean":"2024-09-11T08:43:36.132773+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:43:36.132773+0000","last_undegraded":"2024-09-11T08:43:36.132773+0000","last_fullsized":"2024-09-11T08:43:36.132773+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:28.055031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:28.055031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:56.188615+0000","scrub_duration":0.0082310189999999991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"127'11808","reported_seq":12166,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:46.147762+0000","last_change":"2024-09-11T08:42:39.077974+0000","last_active":"2024-09-11T08:43:46.147762+0000","last_peered":"2024-09-11T08:43:46.147762+0000","last_clean":"2024-09-11T08:43:46.147762+0000","last_became_active":"2024-09-11T08:40:59.753438+0000","last_became_peered":"2024-09-11T08:40:59.753438+0000","last_unstale":"2024-09-11T08:43:46.147762+0000","last_undegraded":"2024-09-11T08:43:46.147762+0000","last_fullsized":"2024-09-11T08:43:46.147762+0000","mapping_epoch":88,"log_start":"108'6800","ondisk_log_start":"108'6800","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7616","last_scrub_stamp":"2024-09-11T08:42:39.077893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:39.077893+0000","objects_scrubbed":7545,"log_size":5008,"log_dups_size":0,"ondisk_log_size":5008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2.8168520839999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":246546432,"num_objects":3762,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11808,"num_write_kb":498240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"124'22","reported_seq":140,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116421+0000","last_change":"2024-09-11T08:43:32.625436+0000","last_active":"2024-09-11T08:43:36.116421+0000","last_peered":"2024-09-11T08:43:36.116421+0000","last_clean":"2024-09-11T08:43:36.116421+0000","last_became_active":"2024-09-11T08:38:41.040475+0000","last_became_peered":"2024-09-11T08:38:41.040475+0000","last_unstale":"2024-09-11T08:43:36.116421+0000","last_undegraded":"2024-09-11T08:43:36.116421+0000","last_fullsized":"2024-09-11T08:43:36.116421+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:43:32.625376+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:41.442241+0000","scrub_duration":0.0073394039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":126,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.125561+0000","last_change":"2024-09-11T08:43:27.137539+0000","last_active":"2024-09-11T08:43:36.125561+0000","last_peered":"2024-09-11T08:43:36.125561+0000","last_clean":"2024-09-11T08:43:36.125561+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:43:36.125561+0000","last_undegraded":"2024-09-11T08:43:36.125561+0000","last_fullsized":"2024-09-11T08:43:36.125561+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:43:27.137095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:27.137095+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:49.370638+0000","scrub_duration":0.024287229,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"63'8","reported_seq":174,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:39.164515+0000","last_change":"2024-09-11T08:42:54.045854+0000","last_active":"2024-09-11T08:43:39.164515+0000","last_peered":"2024-09-11T08:43:39.164515+0000","last_clean":"2024-09-11T08:43:39.164515+0000","last_became_active":"2024-09-11T08:38:42.986756+0000","last_became_peered":"2024-09-11T08:38:42.986756+0000","last_unstale":"2024-09-11T08:43:39.164515+0000","last_undegraded":"2024-09-11T08:43:39.164515+0000","last_fullsized":"2024-09-11T08:43:39.164515+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:42:54.045763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:54.045763+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:08.949073+0000","scrub_duration":0.001794164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116501+0000","last_change":"2024-09-11T08:42:24.399593+0000","last_active":"2024-09-11T08:43:36.116501+0000","last_peered":"2024-09-11T08:43:36.116501+0000","last_clean":"2024-09-11T08:43:36.116501+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:43:36.116501+0000","last_undegraded":"2024-09-11T08:43:36.116501+0000","last_fullsized":"2024-09-11T08:43:36.116501+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:24.399510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:24.399510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:39.903679+0000","scrub_duration":0.002003959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":109,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.121866+0000","last_change":"2024-09-11T08:42:37.900470+0000","last_active":"2024-09-11T08:43:36.121866+0000","last_peered":"2024-09-11T08:43:36.121866+0000","last_clean":"2024-09-11T08:43:36.121866+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:43:36.121866+0000","last_undegraded":"2024-09-11T08:43:36.121866+0000","last_fullsized":"2024-09-11T08:43:36.121866+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:37.900396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:55.587871+0000","last_clean_scrub_stamp":"2024-09-11T08:42:37.900396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:44.921779+0000","scrub_duration":0.0037448859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":124,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.133325+0000","last_change":"2024-09-11T08:42:36.813790+0000","last_active":"2024-09-11T08:43:36.133325+0000","last_peered":"2024-09-11T08:43:36.133325+0000","last_clean":"2024-09-11T08:43:36.133325+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:43:36.133325+0000","last_undegraded":"2024-09-11T08:43:36.133325+0000","last_fullsized":"2024-09-11T08:43:36.133325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-11T08:42:36.813701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:42:36.813701+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:44.153537+0000","scrub_duration":0.030877587000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.118425+0000","last_change":"2024-09-11T08:43:18.699360+0000","last_active":"2024-09-11T08:43:36.118425+0000","last_peered":"2024-09-11T08:43:36.118425+0000","last_clean":"2024-09-11T08:43:36.118425+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:43:36.118425+0000","last_undegraded":"2024-09-11T08:43:36.118425+0000","last_fullsized":"2024-09-11T08:43:36.118425+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:18.699275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:18.699275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:36.698947+0000","scrub_duration":0.0041993969999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.126042+0000","last_change":"2024-09-11T08:42:24.944327+0000","last_active":"2024-09-11T08:43:36.126042+0000","last_peered":"2024-09-11T08:43:36.126042+0000","last_clean":"2024-09-11T08:43:36.126042+0000","last_became_active":"2024-09-11T08:38:45.184753+0000","last_became_peered":"2024-09-11T08:38:45.184753+0000","last_unstale":"2024-09-11T08:43:36.126042+0000","last_undegraded":"2024-09-11T08:43:36.126042+0000","last_fullsized":"2024-09-11T08:43:36.126042+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:24.944242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:24.944242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:44.035607+0000","scrub_duration":0.001616784,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":114,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.144399+0000","last_change":"2024-09-11T08:41:59.013658+0000","last_active":"2024-09-11T08:43:36.144399+0000","last_peered":"2024-09-11T08:43:36.144399+0000","last_clean":"2024-09-11T08:43:36.144399+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:43:36.144399+0000","last_undegraded":"2024-09-11T08:43:36.144399+0000","last_fullsized":"2024-09-11T08:43:36.144399+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:41:59.013187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:41:59.013187+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012093042,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":109,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124965+0000","last_change":"2024-09-11T08:42:49.945200+0000","last_active":"2024-09-11T08:43:36.124965+0000","last_peered":"2024-09-11T08:43:36.124965+0000","last_clean":"2024-09-11T08:43:36.124965+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:43:36.124965+0000","last_undegraded":"2024-09-11T08:43:36.124965+0000","last_fullsized":"2024-09-11T08:43:36.124965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:49.945135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:49.945135+0000","last_clean_scrub_stamp":"2024-09-11T08:42:49.945135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:18.914924+0000","scrub_duration":0.0012025040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":95,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.133180+0000","last_change":"2024-09-11T08:42:54.010343+0000","last_active":"2024-09-11T08:43:36.133180+0000","last_peered":"2024-09-11T08:43:36.133180+0000","last_clean":"2024-09-11T08:43:36.133180+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:43:36.133180+0000","last_undegraded":"2024-09-11T08:43:36.133180+0000","last_fullsized":"2024-09-11T08:43:36.133180+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:54.010203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:54.010203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:23.410125+0000","scrub_duration":0.019363425,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.126423+0000","last_change":"2024-09-11T08:43:19.468579+0000","last_active":"2024-09-11T08:43:36.126423+0000","last_peered":"2024-09-11T08:43:36.126423+0000","last_clean":"2024-09-11T08:43:36.126423+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:43:36.126423+0000","last_undegraded":"2024-09-11T08:43:36.126423+0000","last_fullsized":"2024-09-11T08:43:36.126423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:19.468499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_clean_scrub_stamp":"2024-09-11T08:43:19.468499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:43.282595+0000","scrub_duration":0.0015738169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":114,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116986+0000","last_change":"2024-09-11T08:42:25.441884+0000","last_active":"2024-09-11T08:43:36.116986+0000","last_peered":"2024-09-11T08:43:36.116986+0000","last_clean":"2024-09-11T08:43:36.116986+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:43:36.116986+0000","last_undegraded":"2024-09-11T08:43:36.116986+0000","last_fullsized":"2024-09-11T08:43:36.116986+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:42:25.441804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:42:25.441804+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:42.898593+0000","scrub_duration":0.0036498849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":100,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.137158+0000","last_change":"2024-09-11T08:41:43.851759+0000","last_active":"2024-09-11T08:43:36.137158+0000","last_peered":"2024-09-11T08:43:36.137158+0000","last_clean":"2024-09-11T08:43:36.137158+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:43:36.137158+0000","last_undegraded":"2024-09-11T08:43:36.137158+0000","last_fullsized":"2024-09-11T08:43:36.137158+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:43.851663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:41:43.851663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062910420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":164,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:39.167822+0000","last_change":"2024-09-11T08:42:46.918288+0000","last_active":"2024-09-11T08:43:39.167822+0000","last_peered":"2024-09-11T08:43:39.167822+0000","last_clean":"2024-09-11T08:43:39.167822+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:43:39.167822+0000","last_undegraded":"2024-09-11T08:43:39.167822+0000","last_fullsized":"2024-09-11T08:43:39.167822+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:42:46.918041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:46.918041+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:55.837955+0000","scrub_duration":0.0042043150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":84,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124076+0000","last_change":"2024-09-11T08:40:46.709390+0000","last_active":"2024-09-11T08:43:36.124076+0000","last_peered":"2024-09-11T08:43:36.124076+0000","last_clean":"2024-09-11T08:43:36.124076+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:43:36.124076+0000","last_undegraded":"2024-09-11T08:43:36.124076+0000","last_fullsized":"2024-09-11T08:43:36.124076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:46.709322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:46.709322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017239779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":112,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.133261+0000","last_change":"2024-09-11T08:42:47.929375+0000","last_active":"2024-09-11T08:43:36.133261+0000","last_peered":"2024-09-11T08:43:36.133261+0000","last_clean":"2024-09-11T08:43:36.133261+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:43:36.133261+0000","last_undegraded":"2024-09-11T08:43:36.133261+0000","last_fullsized":"2024-09-11T08:43:36.133261+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:42:47.929291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:42:47.929291+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:06.084342+0000","scrub_duration":0.017023771,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":99,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.125214+0000","last_change":"2024-09-11T08:42:35.039885+0000","last_active":"2024-09-11T08:43:36.125214+0000","last_peered":"2024-09-11T08:43:36.125214+0000","last_clean":"2024-09-11T08:43:36.125214+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:43:36.125214+0000","last_undegraded":"2024-09-11T08:43:36.125214+0000","last_fullsized":"2024-09-11T08:43:36.125214+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:35.039796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:35.039796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:04.620638+0000","scrub_duration":0.012556869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":175,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:39.153308+0000","last_change":"2024-09-11T08:43:28.525724+0000","last_active":"2024-09-11T08:43:39.153308+0000","last_peered":"2024-09-11T08:43:39.153308+0000","last_clean":"2024-09-11T08:43:39.153308+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:43:39.153308+0000","last_undegraded":"2024-09-11T08:43:39.153308+0000","last_fullsized":"2024-09-11T08:43:39.153308+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:43:28.525595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:28.525595+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:52.637367+0000","scrub_duration":0.0046784749999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.136601+0000","last_change":"2024-09-11T08:43:28.384507+0000","last_active":"2024-09-11T08:43:36.136601+0000","last_peered":"2024-09-11T08:43:36.136601+0000","last_clean":"2024-09-11T08:43:36.136601+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:43:36.136601+0000","last_undegraded":"2024-09-11T08:43:36.136601+0000","last_fullsized":"2024-09-11T08:43:36.136601+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:28.384312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:28.384312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:32.247444+0000","scrub_duration":0.010892657,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":117,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:37.137328+0000","last_change":"2024-09-11T08:43:37.137260+0000","last_active":"2024-09-11T08:43:37.137328+0000","last_peered":"2024-09-11T08:43:37.137328+0000","last_clean":"2024-09-11T08:43:37.137328+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:43:37.137328+0000","last_undegraded":"2024-09-11T08:43:37.137328+0000","last_fullsized":"2024-09-11T08:43:37.137328+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:37.137173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:37.137173+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:56.344201+0000","scrub_duration":0.0085039490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":100,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124172+0000","last_change":"2024-09-11T08:42:54.072438+0000","last_active":"2024-09-11T08:43:36.124172+0000","last_peered":"2024-09-11T08:43:36.124172+0000","last_clean":"2024-09-11T08:43:36.124172+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:43:36.124172+0000","last_undegraded":"2024-09-11T08:43:36.124172+0000","last_fullsized":"2024-09-11T08:43:36.124172+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:54.072359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:54.072359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:04.188518+0000","scrub_duration":0.00138809,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.119431+0000","last_change":"2024-09-11T08:42:24.194351+0000","last_active":"2024-09-11T08:43:36.119431+0000","last_peered":"2024-09-11T08:43:36.119431+0000","last_clean":"2024-09-11T08:43:36.119431+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:43:36.119431+0000","last_undegraded":"2024-09-11T08:43:36.119431+0000","last_fullsized":"2024-09-11T08:43:36.119431+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:24.194249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:24.194249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0084760800000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.130539+0000","last_change":"2024-09-11T08:43:17.962352+0000","last_active":"2024-09-11T08:43:36.130539+0000","last_peered":"2024-09-11T08:43:36.130539+0000","last_clean":"2024-09-11T08:43:36.130539+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:43:36.130539+0000","last_undegraded":"2024-09-11T08:43:36.130539+0000","last_fullsized":"2024-09-11T08:43:36.130539+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:17.962284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:17.962284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:46.961030+0000","scrub_duration":0.015011818,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":97,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116319+0000","last_change":"2024-09-11T08:43:20.621567+0000","last_active":"2024-09-11T08:43:36.116319+0000","last_peered":"2024-09-11T08:43:36.116319+0000","last_clean":"2024-09-11T08:43:36.116319+0000","last_became_active":"2024-09-11T08:40:16.181181+0000","last_became_peered":"2024-09-11T08:40:16.181181+0000","last_unstale":"2024-09-11T08:43:36.116319+0000","last_undegraded":"2024-09-11T08:43:36.116319+0000","last_fullsized":"2024-09-11T08:43:36.116319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:43:20.621471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:20.621471+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:50.034800+0000","scrub_duration":0.0039709369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":109,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:37.206228+0000","last_change":"2024-09-11T08:43:37.206161+0000","last_active":"2024-09-11T08:43:37.206228+0000","last_peered":"2024-09-11T08:43:37.206228+0000","last_clean":"2024-09-11T08:43:37.206228+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:43:37.206228+0000","last_undegraded":"2024-09-11T08:43:37.206228+0000","last_fullsized":"2024-09-11T08:43:37.206228+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:37.206096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:37.206096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:41.725411+0000","scrub_duration":0.001561808,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":95,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.132844+0000","last_change":"2024-09-11T08:42:29.790258+0000","last_active":"2024-09-11T08:43:36.132844+0000","last_peered":"2024-09-11T08:43:36.132844+0000","last_clean":"2024-09-11T08:43:36.132844+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:43:36.132844+0000","last_undegraded":"2024-09-11T08:43:36.132844+0000","last_fullsized":"2024-09-11T08:43:36.132844+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:29.790174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:29.790174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:46.828100+0000","scrub_duration":0.016936698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.135549+0000","last_change":"2024-09-11T08:43:29.349431+0000","last_active":"2024-09-11T08:43:36.135549+0000","last_peered":"2024-09-11T08:43:36.135549+0000","last_clean":"2024-09-11T08:43:36.135549+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:43:36.135549+0000","last_undegraded":"2024-09-11T08:43:36.135549+0000","last_fullsized":"2024-09-11T08:43:36.135549+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.349351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.349351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:57.738748+0000","scrub_duration":0.006276294,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.130663+0000","last_change":"2024-09-11T08:40:54.351650+0000","last_active":"2024-09-11T08:43:36.130663+0000","last_peered":"2024-09-11T08:43:36.130663+0000","last_clean":"2024-09-11T08:43:36.130663+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:43:36.130663+0000","last_undegraded":"2024-09-11T08:43:36.130663+0000","last_fullsized":"2024-09-11T08:43:36.130663+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:40:54.351585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:54.351585+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021705309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.118462+0000","last_change":"2024-09-11T08:43:35.725634+0000","last_active":"2024-09-11T08:43:36.118462+0000","last_peered":"2024-09-11T08:43:36.118462+0000","last_clean":"2024-09-11T08:43:36.118462+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:43:36.118462+0000","last_undegraded":"2024-09-11T08:43:36.118462+0000","last_fullsized":"2024-09-11T08:43:36.118462+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:35.725556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:35.725556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:42.652307+0000","scrub_duration":0.0043755219999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":105,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124561+0000","last_change":"2024-09-11T08:42:44.949659+0000","last_active":"2024-09-11T08:43:36.124561+0000","last_peered":"2024-09-11T08:43:36.124561+0000","last_clean":"2024-09-11T08:43:36.124561+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:43:36.124561+0000","last_undegraded":"2024-09-11T08:43:36.124561+0000","last_fullsized":"2024-09-11T08:43:36.124561+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:44.949525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:44.949525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:09.104543+0000","scrub_duration":0.001556291,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.132967+0000","last_change":"2024-09-11T08:43:32.115066+0000","last_active":"2024-09-11T08:43:36.132967+0000","last_peered":"2024-09-11T08:43:36.132967+0000","last_clean":"2024-09-11T08:43:36.132967+0000","last_became_active":"2024-09-11T08:38:45.183548+0000","last_became_peered":"2024-09-11T08:38:45.183548+0000","last_unstale":"2024-09-11T08:43:36.132967+0000","last_undegraded":"2024-09-11T08:43:36.132967+0000","last_fullsized":"2024-09-11T08:43:36.132967+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:32.114990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:32.114990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:59.616575+0000","scrub_duration":0.014089513999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":103,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.135685+0000","last_change":"2024-09-11T08:41:36.323880+0000","last_active":"2024-09-11T08:43:36.135685+0000","last_peered":"2024-09-11T08:43:36.135685+0000","last_clean":"2024-09-11T08:43:36.135685+0000","last_became_active":"2024-09-11T08:38:40.973799+0000","last_became_peered":"2024-09-11T08:38:40.973799+0000","last_unstale":"2024-09-11T08:43:36.135685+0000","last_undegraded":"2024-09-11T08:43:36.135685+0000","last_fullsized":"2024-09-11T08:43:36.135685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-11T08:41:36.323786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:41:36.323786+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071016270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":117,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.136462+0000","last_change":"2024-09-11T08:43:17.340312+0000","last_active":"2024-09-11T08:43:36.136462+0000","last_peered":"2024-09-11T08:43:36.136462+0000","last_clean":"2024-09-11T08:43:36.136462+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:43:36.136462+0000","last_undegraded":"2024-09-11T08:43:36.136462+0000","last_fullsized":"2024-09-11T08:43:36.136462+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:43:17.339897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:17.339897+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:36.637359+0000","scrub_duration":0.012677927,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":105,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.122140+0000","last_change":"2024-09-11T08:42:49.803075+0000","last_active":"2024-09-11T08:43:36.122140+0000","last_peered":"2024-09-11T08:43:36.122140+0000","last_clean":"2024-09-11T08:43:36.122140+0000","last_became_active":"2024-09-11T08:38:42.989556+0000","last_became_peered":"2024-09-11T08:38:42.989556+0000","last_unstale":"2024-09-11T08:43:36.122140+0000","last_undegraded":"2024-09-11T08:43:36.122140+0000","last_fullsized":"2024-09-11T08:43:36.122140+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:49.802985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:49.802985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:03.649380+0000","scrub_duration":0.0014514,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":103,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124712+0000","last_change":"2024-09-11T08:42:50.985633+0000","last_active":"2024-09-11T08:43:36.124712+0000","last_peered":"2024-09-11T08:43:36.124712+0000","last_clean":"2024-09-11T08:43:36.124712+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:43:36.124712+0000","last_undegraded":"2024-09-11T08:43:36.124712+0000","last_fullsized":"2024-09-11T08:43:36.124712+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:50.985537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:50.985537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:59.594451+0000","scrub_duration":0.0077766800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.120244+0000","last_change":"2024-09-11T08:42:32.215521+0000","last_active":"2024-09-11T08:43:36.120244+0000","last_peered":"2024-09-11T08:43:36.120244+0000","last_clean":"2024-09-11T08:43:36.120244+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:43:36.120244+0000","last_undegraded":"2024-09-11T08:43:36.120244+0000","last_fullsized":"2024-09-11T08:43:36.120244+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:32.215417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:32.215417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053169189999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":99,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.135875+0000","last_change":"2024-09-11T08:41:32.353132+0000","last_active":"2024-09-11T08:43:36.135875+0000","last_peered":"2024-09-11T08:43:36.135875+0000","last_clean":"2024-09-11T08:43:36.135875+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:43:36.135875+0000","last_undegraded":"2024-09-11T08:43:36.135875+0000","last_fullsized":"2024-09-11T08:43:36.135875+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:32.353055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:41:32.353055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011132267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":126,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.122406+0000","last_change":"2024-09-11T08:43:31.108299+0000","last_active":"2024-09-11T08:43:36.122406+0000","last_peered":"2024-09-11T08:43:36.122406+0000","last_clean":"2024-09-11T08:43:36.122406+0000","last_became_active":"2024-09-11T08:38:40.974451+0000","last_became_peered":"2024-09-11T08:38:40.974451+0000","last_unstale":"2024-09-11T08:43:36.122406+0000","last_undegraded":"2024-09-11T08:43:36.122406+0000","last_fullsized":"2024-09-11T08:43:36.122406+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:31.108210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:31.108210+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:45:00.896181+0000","scrub_duration":0.0045306019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.144235+0000","last_change":"2024-09-11T08:43:30.336272+0000","last_active":"2024-09-11T08:43:36.144235+0000","last_peered":"2024-09-11T08:43:36.144235+0000","last_clean":"2024-09-11T08:43:36.144235+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:43:36.144235+0000","last_undegraded":"2024-09-11T08:43:36.144235+0000","last_fullsized":"2024-09-11T08:43:36.144235+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:30.336188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:30.336188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:45.754555+0000","scrub_duration":0.0044263979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":3905355800,"num_objects":59592,"num_object_clones":0,"num_object_copies":178776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59592,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191460,"num_write_kb":8033601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11717062656,"data_stored":11717050440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89927,"ondisk_log_size":89927,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":449,"ondisk_log_size":449,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":300,"num_read_kb":295,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299459,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1511020,"kb_used_data":1205348,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92234132,"statfs":{"total":95995035648,"available":94447751168,"internally_reserved":0,"allocated":1234276352,"data_stored":1233942950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:43:35 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":0.78600000000000003,"15min":0.59699999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.9260000000000002,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.879,"15min":0.63100000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":3.806,"5min":5.9790000000000001,"15min":5.9790000000000001},"last":0.51500000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:43:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":0.93799999999999994,"15min":0.65000000000000002},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":8.5030000000000001,"5min":8.5030000000000001,"15min":8.5030000000000001},"last":0.437},{"interface":"front","average":{"1min":1.2330000000000001,"5min":0.78300000000000003,"15min":0.60999999999999999},"min":{"1min":0.30599999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.9830000000000001,"5min":6.593,"15min":6.593},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:43:35 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.78800000000000003,"15min":0.60099999999999998},"min":{"1min":0.28499999999999998,"5min":0.255,"15min":0.255},"max":{"1min":2.2749999999999999,"5min":4.6029999999999998,"15min":4.6029999999999998},"last":0.315},{"interface":"front","average":{"1min":0.876,"5min":0.69199999999999995,"15min":0.55700000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.6549999999999998,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":0.26600000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:43:35 2024","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":0.90500000000000003,"15min":0.65100000000000002},"min":{"1min":0.28000000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":7.8609999999999998,"5min":7.8609999999999998,"15min":7.8609999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":2.0369999999999999,"5min":1.0580000000000001,"15min":0.70599999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":6.2080000000000002,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":0.52900000000000003}]},{"osd":4,"last update":"Wed Sep 11 08:43:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.47799999999999998,"15min":0.44},"min":{"1min":0.249,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46300000000000002,"15min":0.435},"min":{"1min":0.28100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70099999999999996,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.28199999999999997}]},{"osd":5,"last update":"Wed Sep 11 08:43:35 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.496,"15min":0.45600000000000002},"min":{"1min":0.26600000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.57399999999999995,"15min":0.48199999999999998},"min":{"1min":0.29099999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.45100000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:43:35 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.59899999999999998,"15min":0.51100000000000001},"min":{"1min":0.317,"5min":0.315,"15min":0.315},"max":{"1min":3.5979999999999999,"5min":3.5979999999999999,"15min":3.5979999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.56999999999999995,"15min":0.504},"min":{"1min":0.309,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":0.69499999999999995}]}]},{"osd":6,"up_from":45,"seq":193273528392,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1493988,"kb_used_data":1192476,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":92251164,"statfs":{"total":95995035648,"available":94465191936,"internally_reserved":0,"allocated":1221095424,"data_stored":1220770074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":2.3929999999999998,"5min":1.1839999999999999,"15min":0.75},"min":{"1min":0.23599999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":28.302,"5min":28.302,"15min":28.302},"last":0.41099999999999998},{"interface":"front","average":{"1min":2.7890000000000001,"5min":1.097,"15min":0.72099999999999997},"min":{"1min":0.24399999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":28.314,"5min":28.314,"15min":28.314},"last":0.64400000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.69999999999999996,"15min":0.58799999999999997},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.8639999999999999,"5min":4.0839999999999996,"15min":4.0839999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.69599999999999995,"15min":0.57999999999999996},"min":{"1min":0.254,"5min":0.253,"15min":0.253},"max":{"1min":3.4209999999999998,"5min":3.4209999999999998,"15min":3.4209999999999998},"last":1.681}]},{"osd":2,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.65500000000000003,"15min":0.58699999999999997},"min":{"1min":0.28299999999999997,"5min":0.25,"15min":0.25},"max":{"1min":3.0710000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":1.746},{"interface":"front","average":{"1min":1.01,"5min":0.68300000000000005,"15min":0.56699999999999995},"min":{"1min":0.27200000000000002,"5min":0.251,"15min":0.251},"max":{"1min":3.282,"5min":4.25,"15min":4.25},"last":0.93600000000000005}]},{"osd":3,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":0.85899999999999999,"15min":0.63300000000000001},"min":{"1min":0.36899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":8.1869999999999994,"5min":8.1869999999999994,"15min":8.1869999999999994},"last":0.38800000000000001},{"interface":"front","average":{"1min":1.6240000000000001,"5min":0.92400000000000004,"15min":0.66600000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":9.1069999999999993,"5min":9.1069999999999993,"15min":9.1069999999999993},"last":0.436}]},{"osd":4,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42799999999999999,"15min":0.41299999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.41999999999999998,"15min":0.40999999999999998},"min":{"1min":0.28599999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.51700000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47599999999999998,"15min":0.44400000000000001},"min":{"1min":0.27000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.46999999999999997,"15min":0.442},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.44700000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:43:30 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.49199999999999999,"15min":0.44800000000000001},"min":{"1min":0.25900000000000001,"5min":0.247,"15min":0.247},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":0.495},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.495},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.46100000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757324,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1262952,"kb_used_data":957280,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92482200,"statfs":{"total":95995035648,"available":94701772800,"internally_reserved":0,"allocated":980254720,"data_stored":979925503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:42:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.64900000000000002,"15min":0.59299999999999997},"min":{"1min":0.22,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.0249999999999999,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.776,"5min":0.81399999999999995,"15min":0.624},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":4.6619999999999999,"5min":4.6619999999999999,"15min":4.6619999999999999},"last":0.34399999999999997}]},{"osd":1,"last update":"Wed Sep 11 08:42:46 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.72099999999999997,"15min":0.623},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":3.4340000000000002,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":0.995},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.63900000000000001,"15min":0.55500000000000005},"min":{"1min":0.29899999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":6.3719999999999999,"5min":6.3719999999999999,"15min":6.3719999999999999},"last":0.314}]},{"osd":2,"last update":"Wed Sep 11 08:42:46 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":0.71399999999999997,"15min":0.59999999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":5.0309999999999997,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":3.0099999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.60499999999999998,"15min":0.57499999999999996},"min":{"1min":0.28000000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.39000000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:42:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.70199999999999996,"15min":0.59399999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.052,"5min":3.7370000000000001,"15min":3.7370000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":1.3640000000000001,"5min":0.79300000000000004,"15min":0.64000000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":7.5250000000000004,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":2.3140000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:42:46 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40699999999999997,"15min":0.38400000000000001},"min":{"1min":0.23799999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.41799999999999998,"15min":0.40300000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.59299999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.57399999999999995}]},{"osd":6,"last update":"Wed Sep 11 08:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45700000000000002,"15min":0.435},"min":{"1min":0.29299999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0149999999999999,"5min":1.702,"15min":1.702},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.44800000000000001,"15min":0.42699999999999999},"min":{"1min":0.27900000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.86699999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.41399999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:43:28 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46600000000000003,"15min":0.47999999999999998},"min":{"1min":0.32500000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71899999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.497,"15min":0.53300000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.622,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46300000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986256,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1516008,"kb_used_data":1210336,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92229144,"statfs":{"total":95995035648,"available":94442643456,"internally_reserved":0,"allocated":1239384064,"data_stored":1239053676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:43:27 2024","interfaces":[{"interface":"back","average":{"1min":1.9299999999999999,"5min":2.004,"15min":1.036},"min":{"1min":0.24099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":7.4450000000000003,"5min":89.510000000000005,"15min":89.510000000000005},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.579,"5min":1.964,"15min":1.04},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":3.73,"5min":89.765000000000001,"15min":89.765000000000001},"last":1.248}]},{"osd":1,"last update":"Wed Sep 11 08:43:27 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":0.70099999999999996,"15min":0.61199999999999999},"min":{"1min":0.27200000000000002,"5min":0.254,"15min":0.254},"max":{"1min":5.4059999999999997,"5min":5.4059999999999997,"15min":5.4059999999999997},"last":7.7480000000000002},{"interface":"front","average":{"1min":1.2470000000000001,"5min":0.73399999999999999,"15min":0.62},"min":{"1min":0.255,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":3.319,"5min":3.319,"15min":3.319},"last":1}]},{"osd":2,"last update":"Wed Sep 11 08:43:27 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":0.73299999999999998,"15min":0.63200000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":5.3330000000000002,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.7070000000000001,"5min":0.81499999999999995,"15min":0.66100000000000003},"min":{"1min":0.29799999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.28399999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:43:27 2024","interfaces":[{"interface":"back","average":{"1min":2.097,"5min":2.1539999999999999,"15min":1.1100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":8.8919999999999995,"5min":89.881,"15min":89.881},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.861,"5min":2.0609999999999999,"15min":1.0629999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":5.7060000000000004,"5min":89.715000000000003,"15min":89.715000000000003},"last":4.0759999999999996}]},{"osd":5,"last update":"Wed Sep 11 08:42:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45600000000000002,"15min":0.40400000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.439,"15min":0.41499999999999998},"min":{"1min":0.29099999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.374}]},{"osd":6,"last update":"Wed Sep 11 08:43:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46100000000000002,"15min":0.434},"min":{"1min":0.28000000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.79600000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.318},{"interface":"front","average":{"1min":0.629,"5min":0.53100000000000003,"15min":0.48099999999999998},"min":{"1min":0.30099999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.42199999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:43:31 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.504,"15min":0.48599999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.1339999999999999,"5min":1.194,"15min":1.194},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.30499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.41099999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149782,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2456896,"kb_used_data":2146808,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91288256,"statfs":{"total":95995035648,"available":93479174144,"internally_reserved":0,"allocated":2198331392,"data_stored":2197978009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":2.5190000000000001,"5min":0.97899999999999998,"15min":0.57899999999999996},"min":{"1min":0.189,"5min":0.184,"15min":0.184},"max":{"1min":10.901999999999999,"5min":10.901999999999999,"15min":10.901999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":2.6240000000000001,"5min":1.099,"15min":0.63700000000000001},"min":{"1min":0.51800000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":8.5790000000000006,"5min":8.5790000000000006,"15min":8.5790000000000006},"last":2.3799999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":2.7480000000000002,"5min":1.1160000000000001,"15min":0.64600000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":6.1100000000000003,"5min":8.2050000000000001,"15min":8.2050000000000001},"last":8.0909999999999993},{"interface":"front","average":{"1min":2.2559999999999998,"5min":0.93999999999999995,"15min":0.59299999999999997},"min":{"1min":0.20300000000000001,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":8.1470000000000002,"5min":8.1470000000000002,"15min":8.1470000000000002},"last":3.0150000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":2.1269999999999998,"5min":0.95399999999999996,"15min":0.61099999999999999},"min":{"1min":0.251,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":8.2129999999999992,"5min":8.2129999999999992,"15min":8.2129999999999992},"last":2.9820000000000002},{"interface":"front","average":{"1min":2.319,"5min":1.026,"15min":0.63500000000000001},"min":{"1min":0.52400000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":10.695,"5min":10.695,"15min":10.695},"last":4.5620000000000003}]},{"osd":4,"last update":"Wed Sep 11 08:43:26 2024","interfaces":[{"interface":"back","average":{"1min":2.0150000000000001,"5min":0.97999999999999998,"15min":0.70699999999999996},"min":{"1min":0.27500000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":8.9969999999999999,"5min":8.9969999999999999,"15min":8.9969999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":1.6519999999999999,"5min":0.86699999999999999,"15min":0.66500000000000004},"min":{"1min":0.28000000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.9740000000000002,"5min":3.9740000000000002,"15min":3.9740000000000002},"last":0.55200000000000005}]},{"osd":5,"last update":"Wed Sep 11 08:43:43 2024","interfaces":[{"interface":"back","average":{"1min":2.0350000000000001,"5min":1.0760000000000001,"15min":0.73799999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":11.526999999999999,"5min":11.526999999999999,"15min":11.526999999999999},"last":0.436},{"interface":"front","average":{"1min":1.3919999999999999,"5min":0.90500000000000003,"15min":0.66900000000000004},"min":{"1min":0.40600000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.9449999999999998,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.57099999999999995}]},{"osd":6,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":1.8779999999999999,"5min":0.90700000000000003,"15min":0.70099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":9.0139999999999993,"5min":9.0139999999999993,"15min":9.0139999999999993},"last":0.497},{"interface":"front","average":{"1min":1.8,"5min":0.97899999999999998,"15min":0.70099999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":5.4349999999999996,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":2.4140000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:43:32 2024","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":1.0740000000000001,"15min":0.77200000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":11.561,"5min":11.561,"15min":11.561},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.635,"5min":0.93600000000000005,"15min":0.70999999999999996},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.008,"5min":4.008,"15min":4.008},"last":0.52800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1763412,"kb_used_data":1457740,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91981740,"statfs":{"total":95995035648,"available":94189301760,"internally_reserved":0,"allocated":1492725760,"data_stored":1492415563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:42:45 2024","interfaces":[{"interface":"back","average":{"1min":2.0579999999999998,"5min":0.79200000000000004,"15min":0.496},"min":{"1min":0.51900000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":6.2240000000000002,"5min":6.2240000000000002,"15min":6.2240000000000002},"last":6.4480000000000004},{"interface":"front","average":{"1min":2.5630000000000002,"5min":0.97499999999999998,"15min":0.57499999999999996},"min":{"1min":0.28899999999999998,"5min":0.221,"15min":0.221},"max":{"1min":14.667,"5min":14.667,"15min":14.667},"last":4.827}]},{"osd":1,"last update":"Wed Sep 11 08:42:45 2024","interfaces":[{"interface":"back","average":{"1min":2.1059999999999999,"5min":0.80900000000000005,"15min":0.52500000000000002},"min":{"1min":0.46999999999999997,"5min":0.19,"15min":0.19},"max":{"1min":6.1189999999999998,"5min":6.3460000000000001,"15min":6.3460000000000001},"last":3.3140000000000001},{"interface":"front","average":{"1min":2.089,"5min":0.84299999999999997,"15min":0.52700000000000002},"min":{"1min":0.33200000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":8.1270000000000007,"5min":8.1270000000000007,"15min":8.1270000000000007},"last":3.661}]},{"osd":3,"last update":"Wed Sep 11 08:43:02 2024","interfaces":[{"interface":"back","average":{"1min":2.0529999999999999,"5min":1.014,"15min":0.625},"min":{"1min":0.42099999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":6.2590000000000003,"5min":6.718,"15min":6.718},"last":3.4380000000000002},{"interface":"front","average":{"1min":2.5419999999999998,"5min":1.101,"15min":0.66000000000000003},"min":{"1min":0.36799999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":6.9870000000000001,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":6.4969999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:43:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":0.93300000000000005,"15min":0.68000000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":6.181,"5min":6.181,"15min":6.181},"last":3.3439999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":0.89100000000000001,"15min":0.66200000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":6.6660000000000004,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":3.464}]},{"osd":5,"last update":"Wed Sep 11 08:42:45 2024","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":0.80500000000000005,"15min":0.64400000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":6.7039999999999997,"5min":6.7039999999999997,"15min":6.7039999999999997},"last":3.4870000000000001},{"interface":"front","average":{"1min":1.498,"5min":0.76400000000000001,"15min":0.628},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":5.9530000000000003,"5min":5.9530000000000003,"15min":5.9530000000000003},"last":0.34000000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:43:13 2024","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":0.95899999999999996,"15min":0.70699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":6.2030000000000003,"5min":6.2030000000000003,"15min":6.2030000000000003},"last":3.415},{"interface":"front","average":{"1min":1.472,"5min":0.88600000000000001,"15min":0.66700000000000004},"min":{"1min":0.40100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":6.726,"5min":6.726,"15min":6.726},"last":3.5110000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:43:28 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":0.88,"15min":0.68600000000000005},"min":{"1min":0.34200000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.1820000000000004,"5min":5.9909999999999997,"15min":5.9909999999999997},"last":0.36899999999999999},{"interface":"front","average":{"1min":1.135,"5min":0.998,"15min":0.73599999999999999},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.3090000000000002,"5min":6.157,"15min":6.157},"last":3.3839999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1751048,"kb_used_data":1453696,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91994104,"statfs":{"total":95995035648,"available":94201962496,"internally_reserved":0,"allocated":1488584704,"data_stored":1488222532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:43:27 2024","interfaces":[{"interface":"back","average":{"1min":1.774,"5min":2.1160000000000001,"15min":0.91400000000000003},"min":{"1min":0.22800000000000001,"5min":0.183,"15min":0.183},"max":{"1min":6.7919999999999998,"5min":117.01600000000001,"15min":117.01600000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":3.1920000000000002,"5min":2.5009999999999999,"15min":1.0529999999999999},"min":{"1min":0.308,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":9.4830000000000005,"5min":117.62,"15min":117.62},"last":1.571}]},{"osd":2,"last update":"Wed Sep 11 08:43:43 2024","interfaces":[{"interface":"back","average":{"1min":2.4630000000000001,"5min":1.407,"15min":0.72399999999999998},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":9.5719999999999992,"5min":20.628,"15min":20.628},"last":1.544},{"interface":"front","average":{"1min":1.9570000000000001,"5min":1.3,"15min":0.68000000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":5.1689999999999996,"5min":20.148,"15min":20.148},"last":1.6890000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:43:02 2024","interfaces":[{"interface":"back","average":{"1min":2.258,"5min":1.9299999999999999,"15min":0.92800000000000005},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":7.6849999999999996,"5min":116.98,"15min":116.98},"last":0.437},{"interface":"front","average":{"1min":2.681,"5min":2.2080000000000002,"15min":1.026},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":10.379,"5min":117.56699999999999,"15min":117.56699999999999},"last":1.6359999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:43:27 2024","interfaces":[{"interface":"back","average":{"1min":1.788,"5min":0.89000000000000001,"15min":0.65200000000000002},"min":{"1min":0.30599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":11.19,"5min":11.19,"15min":11.19},"last":1.667},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.76600000000000001,"15min":0.63700000000000001},"min":{"1min":0.28499999999999998,"5min":0.251,"15min":0.251},"max":{"1min":4.8310000000000004,"5min":7.7169999999999996,"15min":7.7169999999999996},"last":0.46400000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:42:49 2024","interfaces":[{"interface":"back","average":{"1min":1.869,"5min":0.88700000000000001,"15min":0.68100000000000005},"min":{"1min":0.371,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":8.2569999999999997,"5min":8.2569999999999997,"15min":8.2569999999999997},"last":1.617},{"interface":"front","average":{"1min":1.7509999999999999,"5min":0.81999999999999995,"15min":0.65200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":11.382999999999999,"5min":11.382999999999999,"15min":11.382999999999999},"last":1.708}]},{"osd":6,"last update":"Wed Sep 11 08:43:13 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":0.79500000000000004,"15min":0.65500000000000003},"min":{"1min":0.33100000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.9149999999999991,"5min":8.9149999999999991,"15min":8.9149999999999991},"last":0.503},{"interface":"front","average":{"1min":2.3420000000000001,"5min":1.018,"15min":0.745},"min":{"1min":0.312,"5min":0.31,"15min":0.31},"max":{"1min":9.5199999999999996,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":1.5960000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:43:27 2024","interfaces":[{"interface":"back","average":{"1min":1.8300000000000001,"5min":0.96199999999999997,"15min":0.70599999999999996},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":11.315,"5min":11.315,"15min":11.315},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.085,"5min":0.84599999999999997,"15min":0.67700000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":4.9100000000000001,"5min":8.8800000000000008,"15min":8.8800000000000008},"last":0.52200000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2207220,"kb_used_data":1897388,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91537932,"statfs":{"total":95995035648,"available":93734842368,"internally_reserved":0,"allocated":1942925312,"data_stored":1942583801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:43:20 2024","interfaces":[{"interface":"back","average":{"1min":2.7599999999999998,"5min":1.131,"15min":0.58199999999999996},"min":{"1min":0.215,"5min":0.186,"15min":0.186},"max":{"1min":7.758,"5min":7.758,"15min":7.758},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.7430000000000001,"5min":0.88,"15min":0.499},"min":{"1min":0.32000000000000001,"5min":0.186,"15min":0.186},"max":{"1min":4.2320000000000002,"5min":4.5359999999999996,"15min":4.5359999999999996},"last":0.33400000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:42:43 2024","interfaces":[{"interface":"back","average":{"1min":2.0550000000000002,"5min":1.046,"15min":0.57599999999999996},"min":{"1min":0.49099999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":5.7169999999999996,"5min":19.940000000000001,"15min":19.940000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.52,"5min":0.96899999999999997,"15min":0.55300000000000005},"min":{"1min":0.36199999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":6.9770000000000003,"5min":20.757000000000001,"15min":20.757000000000001},"last":0.72499999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:43:02 2024","interfaces":[{"interface":"back","average":{"1min":2.5070000000000001,"5min":1.1319999999999999,"15min":0.64500000000000002},"min":{"1min":0.58799999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":6.9329999999999998,"5min":6.9329999999999998,"15min":6.9329999999999998},"last":0.749},{"interface":"front","average":{"1min":2.6059999999999999,"5min":1.1619999999999999,"15min":0.65000000000000002},"min":{"1min":0.27000000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":7.5099999999999998,"5min":7.5099999999999998,"15min":7.5099999999999998},"last":1.0149999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:43:26 2024","interfaces":[{"interface":"back","average":{"1min":2.532,"5min":1.125,"15min":0.72899999999999998},"min":{"1min":0.34000000000000002,"5min":0.25,"15min":0.25},"max":{"1min":7.4690000000000003,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":1.0580000000000001},{"interface":"front","average":{"1min":2.121,"5min":1.054,"15min":0.70199999999999996},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":6.9539999999999997,"5min":6.9539999999999997,"15min":6.9539999999999997},"last":0.77600000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:42:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":0.873,"15min":0.64100000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":6.7690000000000001,"5min":6.7690000000000001,"15min":6.7690000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":2.0600000000000001,"5min":0.91900000000000004,"15min":0.67500000000000004},"min":{"1min":0.30499999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":7.423,"5min":7.423,"15min":7.423},"last":1.038}]},{"osd":6,"last update":"Wed Sep 11 08:43:08 2024","interfaces":[{"interface":"back","average":{"1min":1.7010000000000001,"5min":0.97499999999999998,"15min":0.68700000000000006},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.8419999999999996,"5min":6.8419999999999996,"15min":6.8419999999999996},"last":0.38},{"interface":"front","average":{"1min":2.2090000000000001,"5min":1.089,"15min":0.751},"min":{"1min":0.32500000000000001,"5min":0.311,"15min":0.311},"max":{"1min":7.4489999999999998,"5min":7.4489999999999998,"15min":7.4489999999999998},"last":1.077}]},{"osd":7,"last update":"Wed Sep 11 08:43:33 2024","interfaces":[{"interface":"back","average":{"1min":2.6880000000000002,"5min":1.1890000000000001,"15min":0.78100000000000003},"min":{"1min":0.42599999999999999,"5min":0.372,"15min":0.372},"max":{"1min":7.0049999999999999,"5min":7.0049999999999999,"15min":7.0049999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":2.0960000000000001,"5min":1.1830000000000001,"15min":0.76700000000000002},"min":{"1min":0.36399999999999999,"5min":0.307,"15min":0.307},"max":{"1min":5.8410000000000002,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":0.39900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1941442560,"data_stored":1941438488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1468272640,"data_stored":1468268568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1479278592,"data_stored":1479278592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2187591680,"data_stored":2187591680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1236533248,"data_stored":1236533248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":974454784,"data_stored":974454784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1218056192,"data_stored":1218052120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1211432960,"data_stored":1211432960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:43:48.687 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:43:48.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:48.781 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:48.782 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:43:48.782 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:43:48.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.783 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:193260 7.7 e127/88 7:e55ef93c:::benchmark_data_smithi092_84565_object67732:head v 127'11996, mlcod=127'11993)", 2024-09-11T08:43:48.783 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:47.882056+0000", 2024-09-11T08:43:48.783 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012734753999999999, 2024-09-11T08:43:48.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012764763, 2024-09-11T08:43:48.783 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:43:48.783 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:43:48.783 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:43:48.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:43:48.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.882056+0000", 2024-09-11T08:43:48.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:43:48.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.882056+0000", 2024-09-11T08:43:48.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:43:48.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.882056+0000", 2024-09-11T08:43:48.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:43:48.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:43:48.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.882060+0000", 2024-09-11T08:43:48.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4959999999999998e-06 2024-09-11T08:43:48.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:43:48.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.882068+0000", 2024-09-11T08:43:48.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1939999999999998e-06 2024-09-11T08:43:48.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:43:48.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.882072+0000", 2024-09-11T08:43:48.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2780000000000004e-06 2024-09-11T08:43:48.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:43:48.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.882140+0000", 2024-09-11T08:43:48.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.792e-05 2024-09-11T08:43:48.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:43:48.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.882161+0000", 2024-09-11T08:43:48.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0635000000000001e-05 2024-09-11T08:43:48.786 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.786 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:43:48.786 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.786 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:193264 7.9 7:99384b69:::benchmark_data_smithi092_84565_object67736:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T08:43:48.786 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:47.885996+0000", 2024-09-11T08:43:48.787 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0087947909999999997, 2024-09-11T08:43:48.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088965369999999995, 2024-09-11T08:43:48.787 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:43:48.787 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:43:48.787 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:43:48.787 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:43:48.787 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:48.787 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 193264 2024-09-11T08:43:48.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.787 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:43:48.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:43:48.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.885996+0000", 2024-09-11T08:43:48.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:43:48.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.885996+0000", 2024-09-11T08:43:48.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:43:48.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.885994+0000", 2024-09-11T08:43:48.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:43:48.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:43:48.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.886001+0000", 2024-09-11T08:43:48.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3339999999999996e-06 2024-09-11T08:43:48.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:43:48.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.886017+0000", 2024-09-11T08:43:48.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5364999999999999e-05 2024-09-11T08:43:48.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:43:48.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.886023+0000", 2024-09-11T08:43:48.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0229999999999998e-06 2024-09-11T08:43:48.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:43:48.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.886081+0000", 2024-09-11T08:43:48.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8085999999999999e-05 2024-09-11T08:43:48.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:43:48.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.886223+0000", 2024-09-11T08:43:48.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014190599999999999 2024-09-11T08:43:48.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,4", 2024-09-11T08:43:48.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.886350+0000", 2024-09-11T08:43:48.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012714700000000001 2024-09-11T08:43:48.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:48.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.889471+0000", 2024-09-11T08:43:48.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031214680000000001 2024-09-11T08:43:48.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.791 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:43:48.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.791 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:193266 7.f 7:f8f2b67d:::benchmark_data_smithi092_84565_object67738:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T08:43:48.791 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:47.887805+0000", 2024-09-11T08:43:48.791 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0069858740000000004, 2024-09-11T08:43:48.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071620939999999999, 2024-09-11T08:43:48.791 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:43:48.792 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:43:48.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:43:48.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:43:48.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:48.792 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 193266 2024-09-11T08:43:48.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.792 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:43:48.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:43:48.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.887805+0000", 2024-09-11T08:43:48.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:43:48.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.887805+0000", 2024-09-11T08:43:48.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:43:48.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.887803+0000", 2024-09-11T08:43:48.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:43:48.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:43:48.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.887810+0000", 2024-09-11T08:43:48.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9909999999999996e-06 2024-09-11T08:43:48.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:43:48.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.887833+0000", 2024-09-11T08:43:48.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2969999999999999e-05 2024-09-11T08:43:48.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:43:48.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.887840+0000", 2024-09-11T08:43:48.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.233e-06 2024-09-11T08:43:48.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:43:48.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.887885+0000", 2024-09-11T08:43:48.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4233000000000002e-05 2024-09-11T08:43:48.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:43:48.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.888006+0000", 2024-09-11T08:43:48.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121323 2024-09-11T08:43:48.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3,7", 2024-09-11T08:43:48.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.888124+0000", 2024-09-11T08:43:48.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118353 2024-09-11T08:43:48.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:43:48.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.891883+0000", 2024-09-11T08:43:48.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037582689999999998 2024-09-11T08:43:48.796 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.796 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:43:48.796 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.796 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:193269 7.9 7:9478cb83:::benchmark_data_smithi092_84565_object67741:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T08:43:48.796 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:47.889172+0000", 2024-09-11T08:43:48.796 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0056191360000000003, 2024-09-11T08:43:48.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005865599, 2024-09-11T08:43:48.796 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:43:48.796 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:43:48.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:43:48.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:43:48.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:48.797 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 193269 2024-09-11T08:43:48.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.797 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:43:48.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:43:48.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.889172+0000", 2024-09-11T08:43:48.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:43:48.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.889172+0000", 2024-09-11T08:43:48.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:43:48.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.889171+0000", 2024-09-11T08:43:48.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:43:48.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:43:48.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.889174+0000", 2024-09-11T08:43:48.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.642e-06 2024-09-11T08:43:48.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:43:48.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.889183+0000", 2024-09-11T08:43:48.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6209999999999993e-06 2024-09-11T08:43:48.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:43:48.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.889188+0000", 2024-09-11T08:43:48.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0980000000000001e-06 2024-09-11T08:43:48.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:43:48.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.889495+0000", 2024-09-11T08:43:48.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030648900000000001 2024-09-11T08:43:48.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:43:48.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.889590+0000", 2024-09-11T08:43:48.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5459999999999997e-05 2024-09-11T08:43:48.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,4", 2024-09-11T08:43:48.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.889687+0000", 2024-09-11T08:43:48.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6478999999999996e-05 2024-09-11T08:43:48.800 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.800 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:43:48.800 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.800 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.14814.0:193256 7.f e127/88 ondisk, result = 0)", 2024-09-11T08:43:48.800 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:47.891974+0000", 2024-09-11T08:43:48.800 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.002817221, 2024-09-11T08:43:48.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003126228, 2024-09-11T08:43:48.801 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:43:48.801 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:43:48.801 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:43:48.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:43:48.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.891974+0000", 2024-09-11T08:43:48.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:43:48.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.891974+0000", 2024-09-11T08:43:48.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:43:48.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.891973+0000", 2024-09-11T08:43:48.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:43:48.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:43:48.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.891976+0000", 2024-09-11T08:43:48.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.774e-06 2024-09-11T08:43:48.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:43:48.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.891983+0000", 2024-09-11T08:43:48.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6590000000000001e-06 2024-09-11T08:43:48.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:43:48.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.891987+0000", 2024-09-11T08:43:48.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8979999999999998e-06 2024-09-11T08:43:48.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:43:48.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892343+0000", 2024-09-11T08:43:48.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035638800000000002 2024-09-11T08:43:48.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:43:48.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892349+0000", 2024-09-11T08:43:48.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4139999999999999e-06 2024-09-11T08:43:48.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.804 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:43:48.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.804 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:193265 7.7 e127/88 7:edaebf14:::benchmark_data_smithi092_84565_object67737:head v 127'11997, mlcod=127'11994)", 2024-09-11T08:43:48.804 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:47.892003+0000", 2024-09-11T08:43:48.804 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0027881849999999999, 2024-09-11T08:43:48.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031491790000000002, 2024-09-11T08:43:48.804 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:43:48.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:43:48.805 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:43:48.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:43:48.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892003+0000", 2024-09-11T08:43:48.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:43:48.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892003+0000", 2024-09-11T08:43:48.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:43:48.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892002+0000", 2024-09-11T08:43:48.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:43:48.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:43:48.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892007+0000", 2024-09-11T08:43:48.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3809999999999998e-06 2024-09-11T08:43:48.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:43:48.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892015+0000", 2024-09-11T08:43:48.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3289999999999996e-06 2024-09-11T08:43:48.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:43:48.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892020+0000", 2024-09-11T08:43:48.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.729e-06 2024-09-11T08:43:48.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:43:48.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892330+0000", 2024-09-11T08:43:48.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031080099999999998 2024-09-11T08:43:48.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:43:48.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892348+0000", 2024-09-11T08:43:48.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7662000000000001e-05 2024-09-11T08:43:48.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.808 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:43:48.808 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.808 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:193263 7.e e127/88 7:77fe14ff:::benchmark_data_smithi092_84565_object67735:head v 127'12264, mlcod=127'12262)", 2024-09-11T08:43:48.808 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:47.892043+0000", 2024-09-11T08:43:48.808 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0027482169999999998, 2024-09-11T08:43:48.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031618800000000002, 2024-09-11T08:43:48.808 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:43:48.808 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:43:48.808 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:43:48.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:43:48.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892043+0000", 2024-09-11T08:43:48.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:43:48.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892043+0000", 2024-09-11T08:43:48.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:43:48.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892042+0000", 2024-09-11T08:43:48.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:43:48.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:43:48.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892046+0000", 2024-09-11T08:43:48.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3189999999999996e-06 2024-09-11T08:43:48.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:43:48.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892054+0000", 2024-09-11T08:43:48.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6170000000000001e-06 2024-09-11T08:43:48.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:43:48.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892058+0000", 2024-09-11T08:43:48.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0960000000000003e-06 2024-09-11T08:43:48.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:48.811 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:48.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:43:48.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892471+0000", 2024-09-11T08:43:48.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000412965 2024-09-11T08:43:48.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:43:48.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.892489+0000", 2024-09-11T08:43:48.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7617000000000001e-05 2024-09-11T08:43:48.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.812 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:43:48.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.813 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:193254 7.6 7:63d0bf89:::benchmark_data_smithi092_84565_object67726:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T08:43:48.813 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:47.874725+0000", 2024-09-11T08:43:48.813 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.020066019000000001, 2024-09-11T08:43:48.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020538668999999999, 2024-09-11T08:43:48.813 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:43:48.813 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:43:48.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:43:48.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:43:48.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:43:48.813 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 193254 2024-09-11T08:43:48.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.813 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:43:48.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:43:48.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.874725+0000", 2024-09-11T08:43:48.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:43:48.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.874725+0000", 2024-09-11T08:43:48.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:43:48.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:43:48.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.874723+0000", 2024-09-11T08:43:48.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:43:48.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:43:48.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.874729+0000", 2024-09-11T08:43:48.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9179999999999999e-06 2024-09-11T08:43:48.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:43:48.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.874739+0000", 2024-09-11T08:43:48.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0205e-05 2024-09-11T08:43:48.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:43:48.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.874744+0000", 2024-09-11T08:43:48.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1220000000000003e-06 2024-09-11T08:43:48.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:43:48.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.877721+0000", 2024-09-11T08:43:48.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029765019999999998 2024-09-11T08:43:48.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:43:48.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.877845+0000", 2024-09-11T08:43:48.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012388899999999999 2024-09-11T08:43:48.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3,7", 2024-09-11T08:43:48.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.877950+0000", 2024-09-11T08:43:48.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105338 2024-09-11T08:43:48.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:43:48.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.880721+0000", 2024-09-11T08:43:48.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027711699999999999 2024-09-11T08:43:48.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:43:48.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:43:48.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:43:48.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:47.883353+0000", 2024-09-11T08:43:48.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002631862 2024-09-11T08:43:48.817 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.817 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:43:48.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:43:48.818 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:43:48.818 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 8 2024-09-11T08:43:48.818 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:43:48.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:48.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:48.963 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:48.963 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:48.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:43:49.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:49.112 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:49.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:49.114 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:49.214 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:49.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:49.240 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:49.240 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:49.341 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:49.370 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:49.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:49.371 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:49.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:49.510 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:49.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:49.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:49.612 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:49.639 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:49.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:49.640 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:49.741 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:49.772 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:49.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:49.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:49.874 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:49.901 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:49 smithi092 ceph-mon[29647]: pgmap v198: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T08:43:49.902 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:49 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:43:49.902 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:49 smithi092 ceph-mon[41114]: pgmap v198: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T08:43:49.902 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:49 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:43:49.924 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:49.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:49.925 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:49.942 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-11T08:43:50.029 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:50.060 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:50.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:50.061 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:50.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:49 smithi145 ceph-mon[33384]: pgmap v198: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T08:43:50.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:49 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:43:50.162 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:50.213 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:50.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:50.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:50.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:50.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:50.357 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:50.357 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:50.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:50.496 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:50.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:50.497 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:50.598 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:50.619 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:50.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:50.620 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:50.721 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:50.746 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:50.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:50.747 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:50.847 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:50.873 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:50.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:50.874 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:50.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:51.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:51.028 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:51.028 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:51.115 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:51.130 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:51.152 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:51.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:51.153 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:51.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:51.308 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:51.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:51.309 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:51.410 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:51.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:51.452 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:51.453 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:51.543 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:43:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:43:51] "GET /metrics HTTP/1.1" 200 223856 "" "Prometheus/2.43.0" 2024-09-11T08:43:51.554 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:51.579 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:51.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:51.580 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:51.681 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:51.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:51.707 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:51.707 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:51.808 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:51.844 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:51.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:51.845 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:51.909 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:43:51.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:51.992 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:51.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:51.993 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:52.094 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:52.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:52.143 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:52.143 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:52.244 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:52.270 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:52.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:52.271 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:52.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:52 smithi092 ceph-mon[29647]: pgmap v199: 148 pgs: 148 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T08:43:52.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:52 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:43:52.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:52 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:43:52.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:52 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:52.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:52 smithi092 ceph-mon[41114]: pgmap v199: 148 pgs: 148 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T08:43:52.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:52 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:43:52.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:52 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:43:52.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:52 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:52.372 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:52.397 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:52.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:52.398 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:52.499 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:52.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:52.540 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:52.641 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:52 smithi145 ceph-mon[33384]: pgmap v199: 148 pgs: 148 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T08:43:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:52 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:43:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:52 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:43:52.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:52 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:43:52.666 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:52.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:52.667 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:52.768 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:52.795 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:52.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:52.797 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:52.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:52.935 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:52.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:52.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:53.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:53.076 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:53.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:53.077 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:53.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:43:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:43:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:53.175 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:43:53.179 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:53.205 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:53.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:53.206 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:53.307 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:53.333 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:53.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:53.334 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:53.435 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:53.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:53.476 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:53.476 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:53.541 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:53 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3740993125' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:53.541 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:53 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:53.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:53 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3740993125' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:53.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:53 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:53.577 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:53.598 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:53.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:53.600 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:53.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:53 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3740993125' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:53.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:53 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:43:53.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:53.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:53.744 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:53.845 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:53.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:53.874 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:53.875 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:53.975 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:54.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:54.011 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:54.011 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:54.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:54.151 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:54.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:54.152 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:54.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:54.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:54.307 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:54.307 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:54.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:54.437 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:54 smithi092 ceph-mon[29647]: pgmap v200: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T08:43:54.437 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:54 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:43:54.438 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:54 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:43:54.438 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:54 smithi092 ceph-mon[29647]: osdmap e128: 8 total, 8 up, 8 in 2024-09-11T08:43:54.438 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:54 smithi092 ceph-mon[41114]: pgmap v200: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T08:43:54.438 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:54 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:43:54.438 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:54 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:43:54.438 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:54 smithi092 ceph-mon[41114]: osdmap e128: 8 total, 8 up, 8 in 2024-09-11T08:43:54.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:54.445 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:54.445 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:54.546 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:54.573 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:54.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:54.574 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:54.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:54 smithi145 ceph-mon[33384]: pgmap v200: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T08:43:54.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:54 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:43:54.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:54 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:43:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:54 smithi145 ceph-mon[33384]: osdmap e128: 8 total, 8 up, 8 in 2024-09-11T08:43:54.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:54.718 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:54.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:54.720 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:54.820 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:54.850 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:54.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:54.851 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:54.952 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:54.979 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:54.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:54.981 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:55.081 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:55.105 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:55.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:55.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:55.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:55.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:55.242 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:55.242 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:55.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:55.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:55.381 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:55.382 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:55.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:55.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:55.520 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:55.520 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:55.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:55.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:55.679 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:55.679 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:55.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:55.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:55.821 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:55.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:55.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:55.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:56.065 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:56.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:56.092 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:56.093 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:56.194 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:56.221 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:56.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:56.222 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:56.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:56 smithi092 ceph-mon[29647]: pgmap v202: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-11T08:43:56.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:56 smithi092 ceph-mon[41114]: pgmap v202: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-11T08:43:56.323 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:56.347 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:56.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:56.348 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:56.449 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:56.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:56.491 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:56.491 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:56.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:56.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:56.634 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:56.634 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:56.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:56 smithi145 ceph-mon[33384]: pgmap v202: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-11T08:43:56.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:43:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:43:56.668 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:43:56.671 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:43:56.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:56.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:56.773 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:56.773 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:56.874 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:56.898 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:56.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:56.899 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:56.977 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:43:56.977 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:43:57.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:57.015 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:43:57.016 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:43:57.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:57.039 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:57.040 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:57.049 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:43:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:43:57.077 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:43:57.077 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:43:57.104 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:43:57.104 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:43:57.132 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:43:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:43:57.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:57.165 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:43:57.165 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:43:57.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:43:57.180 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:57.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:57.181 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:57.195 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:43:57.195 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:43:57.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:57.312 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:57.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:57.313 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:57.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:57 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:57.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:57 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/768933602' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:57.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:57 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:57.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:57 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/768933602' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:57.414 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:57.440 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:57.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:57.441 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:57.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:57.579 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:57.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:57.581 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:57.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:57 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:57.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:57 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/768933602' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:43:57.682 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:57.709 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:57.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:57.710 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:57.811 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:57.837 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:57.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:57.838 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:57.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:43:57.980 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:57.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:57.981 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:43:58.082 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:58.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:58.117 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:58.117 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:58.218 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:43:58.245 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:58.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:58.246 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:43:58.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:58 smithi092 ceph-mon[29647]: from='client.24854 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:43:58.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:58 smithi092 ceph-mon[29647]: pgmap v203: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s 2024-09-11T08:43:58.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:58 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:43:58.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:58 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:43:58.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:43:58 smithi092 ceph-mon[29647]: osdmap e129: 8 total, 8 up, 8 in 2024-09-11T08:43:58.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:58 smithi092 ceph-mon[41114]: from='client.24854 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:43:58.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:58 smithi092 ceph-mon[41114]: pgmap v203: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s 2024-09-11T08:43:58.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:58 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:43:58.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:58 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:43:58.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:43:58 smithi092 ceph-mon[41114]: osdmap e129: 8 total, 8 up, 8 in 2024-09-11T08:43:58.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:58.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:58.395 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:58.395 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:58.497 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:58.523 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:58.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:58.525 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:58.562 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:58 smithi145 ceph-mon[33384]: from='client.24854 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:43:58.562 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:58 smithi145 ceph-mon[33384]: pgmap v203: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s 2024-09-11T08:43:58.562 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:58 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:43:58.562 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:58 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:43:58.562 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:43:58 smithi145 ceph-mon[33384]: osdmap e129: 8 total, 8 up, 8 in 2024-09-11T08:43:58.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:58.661 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:58.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:58.662 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:58.763 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:43:58.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:58.797 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:58.798 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:43:58.898 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:43:58.927 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:58.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:58.928 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:43:59.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:59.070 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:59.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:59.072 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:59.173 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:59.200 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:59.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:59.201 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:59.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:43:59.340 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:59.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:59.341 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:43:59.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:43:59.476 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:43:59.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:59.477 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:43:59.577 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:43:59.605 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:59.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:59.606 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:43:59.672 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:43:59.673 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:43:59.707 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:59.734 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:59.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:59.735 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:59.836 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:43:59.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:43:59.863 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:43:59.864 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:43:59.887 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-11T08:43:55.103727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3561756916,"num_objects":54536,"num_object_clones":0,"num_object_copies":163608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54536,"num_whiteouts":0,"num_read":1159,"num_read_kb":1016,"num_write":197255,"num_write_kb":8035272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90685,"ondisk_log_size":90685,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":12981232,"kb_used_data":10544176,"kb_used_omap":13,"kb_used_meta":2436978,"kb_avail":736979984,"statfs":{"total":767960285184,"available":754667503616,"internally_reserved":0,"allocated":10797236224,"data_stored":10794550604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2495466176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,6,5,9,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-685637632,"num_objects":-10462,"num_object_clones":0,"num_object_copies":-31386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10462,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10462,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003006"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.136396+0000","last_change":"2024-09-11T08:42:53.042630+0000","last_active":"2024-09-11T08:43:36.136396+0000","last_peered":"2024-09-11T08:43:36.136396+0000","last_clean":"2024-09-11T08:43:36.136396+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:43:36.136396+0000","last_undegraded":"2024-09-11T08:43:36.136396+0000","last_fullsized":"2024-09-11T08:43:36.136396+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:53.042556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:53.042556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:11.005058+0000","scrub_duration":0.0084633999999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":118,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123391+0000","last_change":"2024-09-11T08:42:23.321548+0000","last_active":"2024-09-11T08:43:36.123391+0000","last_peered":"2024-09-11T08:43:36.123391+0000","last_clean":"2024-09-11T08:43:36.123391+0000","last_became_active":"2024-09-11T08:38:40.973398+0000","last_became_peered":"2024-09-11T08:38:40.973398+0000","last_unstale":"2024-09-11T08:43:36.123391+0000","last_undegraded":"2024-09-11T08:43:36.123391+0000","last_fullsized":"2024-09-11T08:43:36.123391+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:42:23.321492+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:42:23.321492+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056603720000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.119206+0000","last_change":"2024-09-11T08:42:40.282406+0000","last_active":"2024-09-11T08:43:36.119206+0000","last_peered":"2024-09-11T08:43:36.119206+0000","last_clean":"2024-09-11T08:43:36.119206+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:43:36.119206+0000","last_undegraded":"2024-09-11T08:43:36.119206+0000","last_fullsized":"2024-09-11T08:43:36.119206+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:40.282323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:40.282323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:04.412797+0000","scrub_duration":0.013570553000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":84,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123434+0000","last_change":"2024-09-11T08:41:40.371901+0000","last_active":"2024-09-11T08:43:36.123434+0000","last_peered":"2024-09-11T08:43:36.123434+0000","last_clean":"2024-09-11T08:43:36.123434+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:43:36.123434+0000","last_undegraded":"2024-09-11T08:43:36.123434+0000","last_fullsized":"2024-09-11T08:43:36.123434+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:40.371846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:41:40.371846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014056730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.132706+0000","last_change":"2024-09-11T08:41:51.406079+0000","last_active":"2024-09-11T08:43:36.132706+0000","last_peered":"2024-09-11T08:43:36.132706+0000","last_clean":"2024-09-11T08:43:36.132706+0000","last_became_active":"2024-09-11T08:38:45.184215+0000","last_became_peered":"2024-09-11T08:38:45.184215+0000","last_unstale":"2024-09-11T08:43:36.132706+0000","last_undegraded":"2024-09-11T08:43:36.132706+0000","last_fullsized":"2024-09-11T08:43:36.132706+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:51.405971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:41:51.405971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013018258,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":99,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.132917+0000","last_change":"2024-09-11T08:41:33.298790+0000","last_active":"2024-09-11T08:43:36.132917+0000","last_peered":"2024-09-11T08:43:36.132917+0000","last_clean":"2024-09-11T08:43:36.132917+0000","last_became_active":"2024-09-11T08:38:38.949136+0000","last_became_peered":"2024-09-11T08:38:38.949136+0000","last_unstale":"2024-09-11T08:43:36.132917+0000","last_undegraded":"2024-09-11T08:43:36.132917+0000","last_fullsized":"2024-09-11T08:43:36.132917+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:33.298719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:41:33.298719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027845299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":129,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.131053+0000","last_change":"2024-09-11T08:41:48.443143+0000","last_active":"2024-09-11T08:43:36.131053+0000","last_peered":"2024-09-11T08:43:36.131053+0000","last_clean":"2024-09-11T08:43:36.131053+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:43:36.131053+0000","last_undegraded":"2024-09-11T08:43:36.131053+0000","last_fullsized":"2024-09-11T08:43:36.131053+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:41:48.442900+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:41:48.442900+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010545605,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123250+0000","last_change":"2024-09-11T08:43:28.988751+0000","last_active":"2024-09-11T08:43:36.123250+0000","last_peered":"2024-09-11T08:43:36.123250+0000","last_clean":"2024-09-11T08:43:36.123250+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:43:36.123250+0000","last_undegraded":"2024-09-11T08:43:36.123250+0000","last_fullsized":"2024-09-11T08:43:36.123250+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:43:28.988697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:58.396363+0000","scrub_duration":0.00375019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":100,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.117001+0000","last_change":"2024-09-11T08:41:57.349049+0000","last_active":"2024-09-11T08:43:36.117001+0000","last_peered":"2024-09-11T08:43:36.117001+0000","last_clean":"2024-09-11T08:43:36.117001+0000","last_became_active":"2024-09-11T08:38:38.948930+0000","last_became_peered":"2024-09-11T08:38:38.948930+0000","last_unstale":"2024-09-11T08:43:36.117001+0000","last_undegraded":"2024-09-11T08:43:36.117001+0000","last_fullsized":"2024-09-11T08:43:36.117001+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:41:57.348997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001221261,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":118,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123595+0000","last_change":"2024-09-11T08:41:55.318568+0000","last_active":"2024-09-11T08:43:36.123595+0000","last_peered":"2024-09-11T08:43:36.123595+0000","last_clean":"2024-09-11T08:43:36.123595+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:43:36.123595+0000","last_undegraded":"2024-09-11T08:43:36.123595+0000","last_fullsized":"2024-09-11T08:43:36.123595+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:41:55.318476+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:41:55.318476+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006520601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116811+0000","last_change":"2024-09-11T08:43:33.644944+0000","last_active":"2024-09-11T08:43:36.116811+0000","last_peered":"2024-09-11T08:43:36.116811+0000","last_clean":"2024-09-11T08:43:36.116811+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:43:36.116811+0000","last_undegraded":"2024-09-11T08:43:36.116811+0000","last_fullsized":"2024-09-11T08:43:36.116811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:33.644869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.644869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:58.010377+0000","scrub_duration":0.0020832210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.135664+0000","last_change":"2024-09-11T08:43:18.293424+0000","last_active":"2024-09-11T08:43:36.135664+0000","last_peered":"2024-09-11T08:43:36.135664+0000","last_clean":"2024-09-11T08:43:36.135664+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:43:36.135664+0000","last_undegraded":"2024-09-11T08:43:36.135664+0000","last_fullsized":"2024-09-11T08:43:36.135664+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:18.293342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:18.293342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:44.371461+0000","scrub_duration":0.0078301269999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":100,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116786+0000","last_change":"2024-09-11T08:41:53.294030+0000","last_active":"2024-09-11T08:43:36.116786+0000","last_peered":"2024-09-11T08:43:36.116786+0000","last_clean":"2024-09-11T08:43:36.116786+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:43:36.116786+0000","last_undegraded":"2024-09-11T08:43:36.116786+0000","last_fullsized":"2024-09-11T08:43:36.116786+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:53.293920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:41:53.293920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029725110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":108,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123733+0000","last_change":"2024-09-11T08:43:26.983719+0000","last_active":"2024-09-11T08:43:36.123733+0000","last_peered":"2024-09-11T08:43:36.123733+0000","last_clean":"2024-09-11T08:43:36.123733+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:43:36.123733+0000","last_undegraded":"2024-09-11T08:43:36.123733+0000","last_fullsized":"2024-09-11T08:43:36.123733+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:43:26.983587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:26.983587+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:43.437727+0000","scrub_duration":0.0082679390000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":105,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124784+0000","last_change":"2024-09-11T08:43:17.981506+0000","last_active":"2024-09-11T08:43:36.124784+0000","last_peered":"2024-09-11T08:43:36.124784+0000","last_clean":"2024-09-11T08:43:36.124784+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:43:36.124784+0000","last_undegraded":"2024-09-11T08:43:36.124784+0000","last_fullsized":"2024-09-11T08:43:36.124784+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:17.981432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:43:17.981432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:33.453163+0000","scrub_duration":0.001732162,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116563+0000","last_change":"2024-09-11T08:42:40.554777+0000","last_active":"2024-09-11T08:43:36.116563+0000","last_peered":"2024-09-11T08:43:36.116563+0000","last_clean":"2024-09-11T08:43:36.116563+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:43:36.116563+0000","last_undegraded":"2024-09-11T08:43:36.116563+0000","last_fullsized":"2024-09-11T08:43:36.116563+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:42:40.554717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:47.099561+0000","scrub_duration":0.01079478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124297+0000","last_change":"2024-09-11T08:42:31.315236+0000","last_active":"2024-09-11T08:43:36.124297+0000","last_peered":"2024-09-11T08:43:36.124297+0000","last_clean":"2024-09-11T08:43:36.124297+0000","last_became_active":"2024-09-11T08:38:45.183364+0000","last_became_peered":"2024-09-11T08:38:45.183364+0000","last_unstale":"2024-09-11T08:43:36.124297+0000","last_undegraded":"2024-09-11T08:43:36.124297+0000","last_fullsized":"2024-09-11T08:43:36.124297+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:31.315163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:31.315163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001316897,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.143982+0000","last_change":"2024-09-11T08:43:33.346941+0000","last_active":"2024-09-11T08:43:36.143982+0000","last_peered":"2024-09-11T08:43:36.143982+0000","last_clean":"2024-09-11T08:43:36.143982+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:43:36.143982+0000","last_undegraded":"2024-09-11T08:43:36.143982+0000","last_fullsized":"2024-09-11T08:43:36.143982+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:33.346856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.346856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:45:01.258719+0000","scrub_duration":0.0053605370000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":140,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.126781+0000","last_change":"2024-09-11T08:42:50.585164+0000","last_active":"2024-09-11T08:43:36.126781+0000","last_peered":"2024-09-11T08:43:36.126781+0000","last_clean":"2024-09-11T08:43:36.126781+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:43:36.126781+0000","last_undegraded":"2024-09-11T08:43:36.126781+0000","last_fullsized":"2024-09-11T08:43:36.126781+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:42:50.585063+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:42:50.585063+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:02.994702+0000","scrub_duration":0.012166819000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.120460+0000","last_change":"2024-09-11T08:42:25.205701+0000","last_active":"2024-09-11T08:43:36.120460+0000","last_peered":"2024-09-11T08:43:36.120460+0000","last_clean":"2024-09-11T08:43:36.120460+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:43:36.120460+0000","last_undegraded":"2024-09-11T08:43:36.120460+0000","last_fullsized":"2024-09-11T08:43:36.120460+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:25.205161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:25.205161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015856230999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.133110+0000","last_change":"2024-09-11T08:43:31.091880+0000","last_active":"2024-09-11T08:43:36.133110+0000","last_peered":"2024-09-11T08:43:36.133110+0000","last_clean":"2024-09-11T08:43:36.133110+0000","last_became_active":"2024-09-11T08:38:45.182763+0000","last_became_peered":"2024-09-11T08:38:45.182763+0000","last_unstale":"2024-09-11T08:43:36.133110+0000","last_undegraded":"2024-09-11T08:43:36.133110+0000","last_fullsized":"2024-09-11T08:43:36.133110+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:43:31.091591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:31.091591+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:48.160729+0000","scrub_duration":0.010466677000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":100,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123670+0000","last_change":"2024-09-11T08:42:49.064565+0000","last_active":"2024-09-11T08:43:36.123670+0000","last_peered":"2024-09-11T08:43:36.123670+0000","last_clean":"2024-09-11T08:43:36.123670+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:43:36.123670+0000","last_undegraded":"2024-09-11T08:43:36.123670+0000","last_fullsized":"2024-09-11T08:43:36.123670+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:49.064487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:49.064487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:14.376741+0000","scrub_duration":0.0037677629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":120,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.130912+0000","last_change":"2024-09-11T08:42:24.655590+0000","last_active":"2024-09-11T08:43:36.130912+0000","last_peered":"2024-09-11T08:43:36.130912+0000","last_clean":"2024-09-11T08:43:36.130912+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:43:36.130912+0000","last_undegraded":"2024-09-11T08:43:36.130912+0000","last_fullsized":"2024-09-11T08:43:36.130912+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:42:24.655488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:42:24.655488+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:48.696109+0000","scrub_duration":0.014786436,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123929+0000","last_change":"2024-09-11T08:42:41.184901+0000","last_active":"2024-09-11T08:43:36.123929+0000","last_peered":"2024-09-11T08:43:36.123929+0000","last_clean":"2024-09-11T08:43:36.123929+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:43:36.123929+0000","last_undegraded":"2024-09-11T08:43:36.123929+0000","last_fullsized":"2024-09-11T08:43:36.123929+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:41.184788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:41.184788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:52.636191+0000","scrub_duration":0.005508303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.125198+0000","last_change":"2024-09-11T08:43:24.014133+0000","last_active":"2024-09-11T08:43:36.125198+0000","last_peered":"2024-09-11T08:43:36.125198+0000","last_clean":"2024-09-11T08:43:36.125198+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:43:36.125198+0000","last_undegraded":"2024-09-11T08:43:36.125198+0000","last_fullsized":"2024-09-11T08:43:36.125198+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:43:24.014057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:24.014057+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:44.940101+0000","scrub_duration":0.0037254929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.119019+0000","last_change":"2024-09-11T08:43:27.542277+0000","last_active":"2024-09-11T08:43:36.119019+0000","last_peered":"2024-09-11T08:43:36.119019+0000","last_clean":"2024-09-11T08:43:36.119019+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:43:36.119019+0000","last_undegraded":"2024-09-11T08:43:36.119019+0000","last_fullsized":"2024-09-11T08:43:36.119019+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:27.542189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:27.542189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:27.664241+0000","scrub_duration":0.0020045390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":113,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.144101+0000","last_change":"2024-09-11T08:41:42.846955+0000","last_active":"2024-09-11T08:43:36.144101+0000","last_peered":"2024-09-11T08:43:36.144101+0000","last_clean":"2024-09-11T08:43:36.144101+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:43:36.144101+0000","last_undegraded":"2024-09-11T08:43:36.144101+0000","last_fullsized":"2024-09-11T08:43:36.144101+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:41:42.846865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:41:42.846865+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013901284999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":178,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:42.731613+0000","last_change":"2024-09-11T08:43:35.027781+0000","last_active":"2024-09-11T08:43:42.731613+0000","last_peered":"2024-09-11T08:43:42.731613+0000","last_clean":"2024-09-11T08:43:42.731613+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:43:42.731613+0000","last_undegraded":"2024-09-11T08:43:42.731613+0000","last_fullsized":"2024-09-11T08:43:42.731613+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:43:35.027725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:35.027725+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:45:00.160919+0000","scrub_duration":0.0030466019999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"127'12191","reported_seq":12516,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:51.032232+0000","last_change":"2024-09-11T08:42:25.722233+0000","last_active":"2024-09-11T08:43:51.032232+0000","last_peered":"2024-09-11T08:43:51.032232+0000","last_clean":"2024-09-11T08:43:51.032232+0000","last_became_active":"2024-09-11T08:40:59.754455+0000","last_became_peered":"2024-09-11T08:40:59.754455+0000","last_unstale":"2024-09-11T08:43:51.032232+0000","last_undegraded":"2024-09-11T08:43:51.032232+0000","last_fullsized":"2024-09-11T08:43:51.032232+0000","mapping_epoch":88,"log_start":"105'5900","ondisk_log_start":"105'5900","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6791","last_scrub_stamp":"2024-09-11T08:42:25.722182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:25.722182+0000","objects_scrubbed":6745,"log_size":6291,"log_dups_size":0,"ondisk_log_size":6291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1.8191191760000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213975040,"num_objects":3265,"num_object_clones":0,"num_object_copies":9795,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12191,"num_write_kb":494592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"124'22","reported_seq":140,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.118772+0000","last_change":"2024-09-11T08:43:33.630995+0000","last_active":"2024-09-11T08:43:36.118772+0000","last_peered":"2024-09-11T08:43:36.118772+0000","last_clean":"2024-09-11T08:43:36.118772+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:43:36.118772+0000","last_undegraded":"2024-09-11T08:43:36.118772+0000","last_fullsized":"2024-09-11T08:43:36.118772+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.630901+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:45:03.420729+0000","scrub_duration":0.01477299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":99,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.135935+0000","last_change":"2024-09-11T08:42:35.769631+0000","last_active":"2024-09-11T08:43:36.135935+0000","last_peered":"2024-09-11T08:43:36.135935+0000","last_clean":"2024-09-11T08:43:36.135935+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:43:36.135935+0000","last_undegraded":"2024-09-11T08:43:36.135935+0000","last_fullsized":"2024-09-11T08:43:36.135935+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:35.769558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:35.769558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:44.560878+0000","scrub_duration":0.0096185300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":166,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:42.731403+0000","last_change":"2024-09-11T08:42:43.134695+0000","last_active":"2024-09-11T08:43:42.731403+0000","last_peered":"2024-09-11T08:43:42.731403+0000","last_clean":"2024-09-11T08:43:42.731403+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:43:42.731403+0000","last_undegraded":"2024-09-11T08:43:42.731403+0000","last_fullsized":"2024-09-11T08:43:42.731403+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:42:43.134610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:43.134610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:56.769219+0000","scrub_duration":0.0084874449999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116744+0000","last_change":"2024-09-11T08:42:34.555469+0000","last_active":"2024-09-11T08:43:36.116744+0000","last_peered":"2024-09-11T08:43:36.116744+0000","last_clean":"2024-09-11T08:43:36.116744+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:43:36.116744+0000","last_undegraded":"2024-09-11T08:43:36.116744+0000","last_fullsized":"2024-09-11T08:43:36.116744+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:34.555379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:34.555379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:54.229410+0000","scrub_duration":0.0073682610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"127'12306","reported_seq":12353,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:51.316307+0000","last_change":"2024-09-11T08:40:59.753044+0000","last_active":"2024-09-11T08:43:51.316307+0000","last_peered":"2024-09-11T08:43:51.316307+0000","last_clean":"2024-09-11T08:43:51.316307+0000","last_became_active":"2024-09-11T08:40:59.752731+0000","last_became_peered":"2024-09-11T08:40:59.752731+0000","last_unstale":"2024-09-11T08:43:51.316307+0000","last_undegraded":"2024-09-11T08:43:51.316307+0000","last_fullsized":"2024-09-11T08:43:51.316307+0000","mapping_epoch":88,"log_start":"108'7600","ondisk_log_start":"108'7600","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":4706,"log_dups_size":0,"ondisk_log_size":4706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230162432,"num_objects":3512,"num_object_clones":0,"num_object_copies":10536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12306,"num_write_kb":506176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":129,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.135756+0000","last_change":"2024-09-11T08:41:45.395630+0000","last_active":"2024-09-11T08:43:36.135756+0000","last_peered":"2024-09-11T08:43:36.135756+0000","last_clean":"2024-09-11T08:43:36.135756+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:43:36.135756+0000","last_undegraded":"2024-09-11T08:43:36.135756+0000","last_fullsized":"2024-09-11T08:43:36.135756+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-11T08:41:45.395547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:41:45.395547+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060283000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":99,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.133460+0000","last_change":"2024-09-11T08:41:43.421072+0000","last_active":"2024-09-11T08:43:36.133460+0000","last_peered":"2024-09-11T08:43:36.133460+0000","last_clean":"2024-09-11T08:43:36.133460+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:43:36.133460+0000","last_undegraded":"2024-09-11T08:43:36.133460+0000","last_fullsized":"2024-09-11T08:43:36.133460+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:43.421000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:41:43.421000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015196105,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123521+0000","last_change":"2024-09-11T08:42:30.305683+0000","last_active":"2024-09-11T08:43:36.123521+0000","last_peered":"2024-09-11T08:43:36.123521+0000","last_clean":"2024-09-11T08:43:36.123521+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:43:36.123521+0000","last_undegraded":"2024-09-11T08:43:36.123521+0000","last_fullsized":"2024-09-11T08:43:36.123521+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:30.305585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:30.305585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:49.696009+0000","scrub_duration":0.0023568949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123556+0000","last_change":"2024-09-11T08:43:21.980665+0000","last_active":"2024-09-11T08:43:36.123556+0000","last_peered":"2024-09-11T08:43:36.123556+0000","last_clean":"2024-09-11T08:43:36.123556+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:43:36.123556+0000","last_undegraded":"2024-09-11T08:43:36.123556+0000","last_fullsized":"2024-09-11T08:43:36.123556+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:21.980584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:21.980584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:22.213311+0000","scrub_duration":0.019382835000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"127'12388","reported_seq":12435,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:51.237404+0000","last_change":"2024-09-11T08:40:59.753523+0000","last_active":"2024-09-11T08:43:51.237404+0000","last_peered":"2024-09-11T08:43:51.237404+0000","last_clean":"2024-09-11T08:43:51.237404+0000","last_became_active":"2024-09-11T08:40:59.752898+0000","last_became_peered":"2024-09-11T08:40:59.752898+0000","last_unstale":"2024-09-11T08:43:51.237404+0000","last_undegraded":"2024-09-11T08:43:51.237404+0000","last_fullsized":"2024-09-11T08:43:51.237404+0000","mapping_epoch":88,"log_start":"108'7700","ondisk_log_start":"108'7700","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":4688,"log_dups_size":0,"ondisk_log_size":4688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225837056,"num_objects":3446,"num_object_clones":0,"num_object_copies":10338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12388,"num_write_kb":506688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"86'21","reported_seq":126,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123717+0000","last_change":"2024-09-11T08:43:30.997804+0000","last_active":"2024-09-11T08:43:36.123717+0000","last_peered":"2024-09-11T08:43:36.123717+0000","last_clean":"2024-09-11T08:43:36.123717+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:43:36.123717+0000","last_undegraded":"2024-09-11T08:43:36.123717+0000","last_fullsized":"2024-09-11T08:43:36.123717+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'21","last_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:43:30.997710+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:45.668349+0000","scrub_duration":0.0025287299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.144474+0000","last_change":"2024-09-11T08:43:34.347814+0000","last_active":"2024-09-11T08:43:36.144474+0000","last_peered":"2024-09-11T08:43:36.144474+0000","last_clean":"2024-09-11T08:43:36.144474+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:43:36.144474+0000","last_undegraded":"2024-09-11T08:43:36.144474+0000","last_fullsized":"2024-09-11T08:43:36.144474+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:34.346299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:34.346299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:37.960188+0000","scrub_duration":0.0059519810000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":158,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:42.734187+0000","last_change":"2024-09-11T08:40:51.309316+0000","last_active":"2024-09-11T08:43:42.734187+0000","last_peered":"2024-09-11T08:43:42.734187+0000","last_clean":"2024-09-11T08:43:42.734187+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:43:42.734187+0000","last_undegraded":"2024-09-11T08:43:42.734187+0000","last_fullsized":"2024-09-11T08:43:42.734187+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:40:51.309259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:40:51.309259+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017959670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":103,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.125082+0000","last_change":"2024-09-11T08:43:22.119204+0000","last_active":"2024-09-11T08:43:36.125082+0000","last_peered":"2024-09-11T08:43:36.125082+0000","last_clean":"2024-09-11T08:43:36.125082+0000","last_became_active":"2024-09-11T08:38:45.182724+0000","last_became_peered":"2024-09-11T08:38:45.182724+0000","last_unstale":"2024-09-11T08:43:36.125082+0000","last_undegraded":"2024-09-11T08:43:36.125082+0000","last_fullsized":"2024-09-11T08:43:36.125082+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:22.119108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:22.119108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:38.805197+0000","scrub_duration":0.093065431000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"127'12302","reported_seq":12349,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:51.009813+0000","last_change":"2024-09-11T08:40:59.753707+0000","last_active":"2024-09-11T08:43:51.009813+0000","last_peered":"2024-09-11T08:43:51.009813+0000","last_clean":"2024-09-11T08:43:51.009813+0000","last_became_active":"2024-09-11T08:40:59.753449+0000","last_became_peered":"2024-09-11T08:40:59.753449+0000","last_unstale":"2024-09-11T08:43:51.009813+0000","last_undegraded":"2024-09-11T08:43:51.009813+0000","last_fullsized":"2024-09-11T08:43:51.009813+0000","mapping_epoch":88,"log_start":"108'7500","ondisk_log_start":"108'7500","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:40:58.730067+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222560256,"num_objects":3396,"num_object_clones":0,"num_object_copies":10188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12302,"num_write_kb":502336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"100'15","reported_seq":128,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.603065+0000","last_change":"2024-09-11T08:43:36.602971+0000","last_active":"2024-09-11T08:43:36.603065+0000","last_peered":"2024-09-11T08:43:36.603065+0000","last_clean":"2024-09-11T08:43:36.603065+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:43:36.603065+0000","last_undegraded":"2024-09-11T08:43:36.603065+0000","last_fullsized":"2024-09-11T08:43:36.603065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'15","last_scrub_stamp":"2024-09-11T08:43:36.602876+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:41:56.250393+0000","last_clean_scrub_stamp":"2024-09-11T08:43:36.602876+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:39.522462+0000","scrub_duration":0.0051581880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":100,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.117094+0000","last_change":"2024-09-11T08:42:27.472376+0000","last_active":"2024-09-11T08:43:36.117094+0000","last_peered":"2024-09-11T08:43:36.117094+0000","last_clean":"2024-09-11T08:43:36.117094+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:43:36.117094+0000","last_undegraded":"2024-09-11T08:43:36.117094+0000","last_fullsized":"2024-09-11T08:43:36.117094+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:27.472327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:27.472327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:46.139070+0000","scrub_duration":0.001165318,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":166,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:42.733393+0000","last_change":"2024-09-11T08:42:28.036180+0000","last_active":"2024-09-11T08:43:42.733393+0000","last_peered":"2024-09-11T08:43:42.733393+0000","last_clean":"2024-09-11T08:43:42.733393+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:43:42.733393+0000","last_undegraded":"2024-09-11T08:43:42.733393+0000","last_fullsized":"2024-09-11T08:43:42.733393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:42:28.036083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:28.036083+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:57.370485+0000","scrub_duration":0.0022379549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.135777+0000","last_change":"2024-09-11T08:42:32.112362+0000","last_active":"2024-09-11T08:43:36.135777+0000","last_peered":"2024-09-11T08:43:36.135777+0000","last_clean":"2024-09-11T08:43:36.135777+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:43:36.135777+0000","last_undegraded":"2024-09-11T08:43:36.135777+0000","last_fullsized":"2024-09-11T08:43:36.135777+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:32.112285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:42:32.112285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:39.646427+0000","scrub_duration":0.0073483730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"127'15291","reported_seq":15714,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:51.030804+0000","last_change":"2024-09-11T08:43:19.659282+0000","last_active":"2024-09-11T08:43:51.030804+0000","last_peered":"2024-09-11T08:43:51.030804+0000","last_clean":"2024-09-11T08:43:51.030804+0000","last_became_active":"2024-09-11T08:40:59.753346+0000","last_became_peered":"2024-09-11T08:40:59.753346+0000","last_unstale":"2024-09-11T08:43:51.030804+0000","last_undegraded":"2024-09-11T08:43:51.030804+0000","last_fullsized":"2024-09-11T08:43:51.030804+0000","mapping_epoch":88,"log_start":"113'9500","ondisk_log_start":"113'9500","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"121'13223","last_scrub_stamp":"2024-09-11T08:43:19.659224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:43:19.659224+0000","objects_scrubbed":5482,"log_size":5791,"log_dups_size":0,"ondisk_log_size":5791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:43.964355+0000","scrub_duration":1.9369766209999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":219807744,"num_objects":3354,"num_object_clones":0,"num_object_copies":10062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12324,"num_write_kb":501696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":134,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.143727+0000","last_change":"2024-09-11T08:42:45.105667+0000","last_active":"2024-09-11T08:43:36.143727+0000","last_peered":"2024-09-11T08:43:36.143727+0000","last_clean":"2024-09-11T08:43:36.143727+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:43:36.143727+0000","last_undegraded":"2024-09-11T08:43:36.143727+0000","last_fullsized":"2024-09-11T08:43:36.143727+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:42:45.104549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:42:45.104549+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:14.762641+0000","scrub_duration":0.0073378940000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.119480+0000","last_change":"2024-09-11T08:43:19.502930+0000","last_active":"2024-09-11T08:43:36.119480+0000","last_peered":"2024-09-11T08:43:36.119480+0000","last_clean":"2024-09-11T08:43:36.119480+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:43:36.119480+0000","last_undegraded":"2024-09-11T08:43:36.119480+0000","last_fullsized":"2024-09-11T08:43:36.119480+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:19.502844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:19.502844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:26.390433+0000","scrub_duration":0.0023577619999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":105,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.121847+0000","last_change":"2024-09-11T08:43:23.161380+0000","last_active":"2024-09-11T08:43:36.121847+0000","last_peered":"2024-09-11T08:43:36.121847+0000","last_clean":"2024-09-11T08:43:36.121847+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:43:36.121847+0000","last_undegraded":"2024-09-11T08:43:36.121847+0000","last_fullsized":"2024-09-11T08:43:36.121847+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:23.161300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:43:23.161300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:40.395478+0000","scrub_duration":0.0020977869999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.118884+0000","last_change":"2024-09-11T08:43:31.631898+0000","last_active":"2024-09-11T08:43:36.118884+0000","last_peered":"2024-09-11T08:43:36.118884+0000","last_clean":"2024-09-11T08:43:36.118884+0000","last_became_active":"2024-09-11T08:38:45.182782+0000","last_became_peered":"2024-09-11T08:38:45.182782+0000","last_unstale":"2024-09-11T08:43:36.118884+0000","last_undegraded":"2024-09-11T08:43:36.118884+0000","last_fullsized":"2024-09-11T08:43:36.118884+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:43:31.631810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:52.381803+0000","scrub_duration":0.0074672260000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"127'12213","reported_seq":12581,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:51.047232+0000","last_change":"2024-09-11T08:42:43.178301+0000","last_active":"2024-09-11T08:43:51.047232+0000","last_peered":"2024-09-11T08:43:51.047232+0000","last_clean":"2024-09-11T08:43:51.047232+0000","last_became_active":"2024-09-11T08:40:59.757849+0000","last_became_peered":"2024-09-11T08:40:59.757849+0000","last_unstale":"2024-09-11T08:43:51.047232+0000","last_undegraded":"2024-09-11T08:43:51.047232+0000","last_fullsized":"2024-09-11T08:43:51.047232+0000","mapping_epoch":88,"log_start":"105'5900","ondisk_log_start":"105'5900","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'7880","last_scrub_stamp":"2024-09-11T08:42:43.178246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:43.178246+0000","objects_scrubbed":7738,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:02.385368+0000","scrub_duration":4.2909816630000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222756864,"num_objects":3399,"num_object_clones":0,"num_object_copies":10197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12213,"num_write_kb":499584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":228,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:42.731394+0000","last_change":"2024-09-11T08:42:39.259633+0000","last_active":"2024-09-11T08:43:42.731394+0000","last_peered":"2024-09-11T08:43:42.731394+0000","last_clean":"2024-09-11T08:43:42.731394+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:43:42.731394+0000","last_undegraded":"2024-09-11T08:43:42.731394+0000","last_fullsized":"2024-09-11T08:43:42.731394+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:42:39.259542+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:42:39.259542+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:51.826159+0000","scrub_duration":0.012956343,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":100,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124043+0000","last_change":"2024-09-11T08:42:36.267727+0000","last_active":"2024-09-11T08:43:36.124043+0000","last_peered":"2024-09-11T08:43:36.124043+0000","last_clean":"2024-09-11T08:43:36.124043+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:43:36.124043+0000","last_undegraded":"2024-09-11T08:43:36.124043+0000","last_fullsized":"2024-09-11T08:43:36.124043+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:36.267644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:36.267644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:40.844732+0000","scrub_duration":0.002745029,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":117,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:38.670059+0000","last_change":"2024-09-11T08:43:38.669897+0000","last_active":"2024-09-11T08:43:38.670059+0000","last_peered":"2024-09-11T08:43:38.670059+0000","last_clean":"2024-09-11T08:43:38.670059+0000","last_became_active":"2024-09-11T08:38:38.947270+0000","last_became_peered":"2024-09-11T08:38:38.947270+0000","last_unstale":"2024-09-11T08:43:38.670059+0000","last_undegraded":"2024-09-11T08:43:38.670059+0000","last_fullsized":"2024-09-11T08:43:38.670059+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:43:38.669837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:38.669837+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:45.074560+0000","scrub_duration":0.001723715,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.150756+0000","last_change":"2024-09-11T08:41:52.983114+0000","last_active":"2024-09-11T08:43:36.150756+0000","last_peered":"2024-09-11T08:43:36.150756+0000","last_clean":"2024-09-11T08:43:36.150756+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:43:36.150756+0000","last_undegraded":"2024-09-11T08:43:36.150756+0000","last_fullsized":"2024-09-11T08:43:36.150756+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:52.983038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:41:52.983038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0098101790000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.132674+0000","last_change":"2024-09-11T08:42:49.847720+0000","last_active":"2024-09-11T08:43:36.132674+0000","last_peered":"2024-09-11T08:43:36.132674+0000","last_clean":"2024-09-11T08:43:36.132674+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:43:36.132674+0000","last_undegraded":"2024-09-11T08:43:36.132674+0000","last_fullsized":"2024-09-11T08:43:36.132674+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:49.847548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:49.847548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:07.405718+0000","scrub_duration":0.011241151,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"127'12145","reported_seq":12488,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:51.409061+0000","last_change":"2024-09-11T08:42:53.794983+0000","last_active":"2024-09-11T08:43:51.409061+0000","last_peered":"2024-09-11T08:43:51.409061+0000","last_clean":"2024-09-11T08:43:51.409061+0000","last_became_active":"2024-09-11T08:40:59.756563+0000","last_became_peered":"2024-09-11T08:40:59.756563+0000","last_unstale":"2024-09-11T08:43:51.409061+0000","last_undegraded":"2024-09-11T08:43:51.409061+0000","last_fullsized":"2024-09-11T08:43:51.409061+0000","mapping_epoch":88,"log_start":"107'6500","ondisk_log_start":"107'6500","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'8427","last_scrub_stamp":"2024-09-11T08:42:53.794889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:53.794889+0000","objects_scrubbed":7152,"log_size":5645,"log_dups_size":0,"ondisk_log_size":5645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:02.167218+0000","scrub_duration":2.7347385429999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220266496,"num_objects":3361,"num_object_clones":0,"num_object_copies":10083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12145,"num_write_kb":496192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":135,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.118849+0000","last_change":"2024-09-11T08:42:38.313016+0000","last_active":"2024-09-11T08:43:36.118849+0000","last_peered":"2024-09-11T08:43:36.118849+0000","last_clean":"2024-09-11T08:43:36.118849+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:43:36.118849+0000","last_undegraded":"2024-09-11T08:43:36.118849+0000","last_fullsized":"2024-09-11T08:43:36.118849+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:42:38.312937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:42:38.312937+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.026297491999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.118690+0000","last_change":"2024-09-11T08:43:20.533696+0000","last_active":"2024-09-11T08:43:36.118690+0000","last_peered":"2024-09-11T08:43:36.118690+0000","last_clean":"2024-09-11T08:43:36.118690+0000","last_became_active":"2024-09-11T08:38:38.949019+0000","last_became_peered":"2024-09-11T08:38:38.949019+0000","last_unstale":"2024-09-11T08:43:36.118690+0000","last_undegraded":"2024-09-11T08:43:36.118690+0000","last_fullsized":"2024-09-11T08:43:36.118690+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:20.533619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:20.533619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:27.628937+0000","scrub_duration":0.0026926540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":116,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.126335+0000","last_change":"2024-09-11T08:42:49.606256+0000","last_active":"2024-09-11T08:43:36.126335+0000","last_peered":"2024-09-11T08:43:36.126335+0000","last_clean":"2024-09-11T08:43:36.126335+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:43:36.126335+0000","last_undegraded":"2024-09-11T08:43:36.126335+0000","last_fullsized":"2024-09-11T08:43:36.126335+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:42:49.606178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:49.606178+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:52.569663+0000","scrub_duration":0.0073928789999999998,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.128728+0000","last_change":"2024-09-11T08:43:21.506389+0000","last_active":"2024-09-11T08:43:36.128728+0000","last_peered":"2024-09-11T08:43:36.128728+0000","last_clean":"2024-09-11T08:43:36.128728+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:43:36.128728+0000","last_undegraded":"2024-09-11T08:43:36.128728+0000","last_fullsized":"2024-09-11T08:43:36.128728+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:21.506299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_clean_scrub_stamp":"2024-09-11T08:43:21.506299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:45.821602+0000","scrub_duration":0.0016198600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":103,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.126124+0000","last_change":"2024-09-11T08:43:25.055354+0000","last_active":"2024-09-11T08:43:36.126124+0000","last_peered":"2024-09-11T08:43:36.126124+0000","last_clean":"2024-09-11T08:43:36.126124+0000","last_became_active":"2024-09-11T08:38:45.183582+0000","last_became_peered":"2024-09-11T08:38:45.183582+0000","last_unstale":"2024-09-11T08:43:36.126124+0000","last_undegraded":"2024-09-11T08:43:36.126124+0000","last_fullsized":"2024-09-11T08:43:36.126124+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:25.055268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:25.055268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:40.207781+0000","scrub_duration":0.0032371499999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"127'12356","reported_seq":12715,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:51.032505+0000","last_change":"2024-09-11T08:42:48.186060+0000","last_active":"2024-09-11T08:43:51.032505+0000","last_peered":"2024-09-11T08:43:51.032505+0000","last_clean":"2024-09-11T08:43:51.032505+0000","last_became_active":"2024-09-11T08:40:59.755652+0000","last_became_peered":"2024-09-11T08:40:59.755652+0000","last_unstale":"2024-09-11T08:43:51.032505+0000","last_undegraded":"2024-09-11T08:43:51.032505+0000","last_fullsized":"2024-09-11T08:43:51.032505+0000","mapping_epoch":88,"log_start":"108'7100","ondisk_log_start":"108'7100","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'8307","last_scrub_stamp":"2024-09-11T08:42:48.185997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:48.185997+0000","objects_scrubbed":7567,"log_size":5256,"log_dups_size":0,"ondisk_log_size":5256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:00.620699+0000","scrub_duration":2.2213046419999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225705984,"num_objects":3444,"num_object_clones":0,"num_object_copies":10332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12356,"num_write_kb":505600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":116,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.144318+0000","last_change":"2024-09-11T08:41:56.978529+0000","last_active":"2024-09-11T08:43:36.144318+0000","last_peered":"2024-09-11T08:43:36.144318+0000","last_clean":"2024-09-11T08:43:36.144318+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:43:36.144318+0000","last_undegraded":"2024-09-11T08:43:36.144318+0000","last_fullsized":"2024-09-11T08:43:36.144318+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:41:56.978383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:41:56.978383+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017271161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.136432+0000","last_change":"2024-09-11T08:43:27.359104+0000","last_active":"2024-09-11T08:43:36.136432+0000","last_peered":"2024-09-11T08:43:36.136432+0000","last_clean":"2024-09-11T08:43:36.136432+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:43:36.136432+0000","last_undegraded":"2024-09-11T08:43:36.136432+0000","last_fullsized":"2024-09-11T08:43:36.136432+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:27.359029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:27.359029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:36.281548+0000","scrub_duration":0.002472747,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":422,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:49.509211+0000","last_change":"2024-09-11T08:42:47.044666+0000","last_active":"2024-09-11T08:43:49.509211+0000","last_peered":"2024-09-11T08:43:49.509211+0000","last_clean":"2024-09-11T08:43:49.509211+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:43:49.509211+0000","last_undegraded":"2024-09-11T08:43:49.509211+0000","last_fullsized":"2024-09-11T08:43:49.509211+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:42:47.044598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:47.044598+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:56.754879+0000","scrub_duration":0.086046497999999999,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":305,"num_read_kb":300,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.117465+0000","last_change":"2024-09-11T08:42:36.575430+0000","last_active":"2024-09-11T08:43:36.117465+0000","last_peered":"2024-09-11T08:43:36.117465+0000","last_clean":"2024-09-11T08:43:36.117465+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:43:36.117465+0000","last_undegraded":"2024-09-11T08:43:36.117465+0000","last_fullsized":"2024-09-11T08:43:36.117465+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:36.575347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:36.575347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:42.841995+0000","scrub_duration":0.0078670330000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.119078+0000","last_change":"2024-09-11T08:43:35.596874+0000","last_active":"2024-09-11T08:43:36.119078+0000","last_peered":"2024-09-11T08:43:36.119078+0000","last_clean":"2024-09-11T08:43:36.119078+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:43:36.119078+0000","last_undegraded":"2024-09-11T08:43:36.119078+0000","last_fullsized":"2024-09-11T08:43:36.119078+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:35.596727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:43:35.596727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:45:02.202079+0000","scrub_duration":0.019062624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"127'11948","reported_seq":12289,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:51.361358+0000","last_change":"2024-09-11T08:42:34.330523+0000","last_active":"2024-09-11T08:43:51.361358+0000","last_peered":"2024-09-11T08:43:51.361358+0000","last_clean":"2024-09-11T08:43:51.361358+0000","last_became_active":"2024-09-11T08:40:59.755912+0000","last_became_peered":"2024-09-11T08:40:59.755912+0000","last_unstale":"2024-09-11T08:43:51.361358+0000","last_undegraded":"2024-09-11T08:43:51.361358+0000","last_fullsized":"2024-09-11T08:43:51.361358+0000","mapping_epoch":88,"log_start":"107'6300","ondisk_log_start":"107'6300","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'7230","last_scrub_stamp":"2024-09-11T08:42:34.330327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:34.330327+0000","objects_scrubbed":7114,"log_size":5648,"log_dups_size":0,"ondisk_log_size":5648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"queued for scrub","scrub_duration":4.2856508010000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":221511680,"num_objects":3380,"num_object_clones":0,"num_object_copies":10140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11948,"num_write_kb":490496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":129,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.122299+0000","last_change":"2024-09-11T08:43:26.127930+0000","last_active":"2024-09-11T08:43:36.122299+0000","last_peered":"2024-09-11T08:43:36.122299+0000","last_clean":"2024-09-11T08:43:36.122299+0000","last_became_active":"2024-09-11T08:38:41.040948+0000","last_became_peered":"2024-09-11T08:38:41.040948+0000","last_unstale":"2024-09-11T08:43:36.122299+0000","last_undegraded":"2024-09-11T08:43:36.122299+0000","last_fullsized":"2024-09-11T08:43:36.122299+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:43:26.127412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:26.127412+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:39.494701+0000","scrub_duration":0.0029787630000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":108,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.132035+0000","last_change":"2024-09-11T08:42:28.766183+0000","last_active":"2024-09-11T08:43:36.132035+0000","last_peered":"2024-09-11T08:43:36.132035+0000","last_clean":"2024-09-11T08:43:36.132035+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:43:36.132035+0000","last_undegraded":"2024-09-11T08:43:36.132035+0000","last_fullsized":"2024-09-11T08:43:36.132035+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:42:28.765466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:28.765466+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:55.584403+0000","scrub_duration":0.0053952599999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124997+0000","last_change":"2024-09-11T08:42:23.973455+0000","last_active":"2024-09-11T08:43:36.124997+0000","last_peered":"2024-09-11T08:43:36.124997+0000","last_clean":"2024-09-11T08:43:36.124997+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:43:36.124997+0000","last_undegraded":"2024-09-11T08:43:36.124997+0000","last_fullsized":"2024-09-11T08:43:36.124997+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'79","last_scrub_stamp":"2024-09-11T08:42:23.973378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:42:23.973378+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:49.122950+0000","scrub_duration":0.0076470410000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":105,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.125335+0000","last_change":"2024-09-11T08:43:19.974333+0000","last_active":"2024-09-11T08:43:36.125335+0000","last_peered":"2024-09-11T08:43:36.125335+0000","last_clean":"2024-09-11T08:43:36.125335+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:43:36.125335+0000","last_undegraded":"2024-09-11T08:43:36.125335+0000","last_fullsized":"2024-09-11T08:43:36.125335+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:19.974257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:19.974257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:22.526340+0000","scrub_duration":0.0018788069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.117378+0000","last_change":"2024-09-11T08:43:29.570321+0000","last_active":"2024-09-11T08:43:36.117378+0000","last_peered":"2024-09-11T08:43:36.117378+0000","last_clean":"2024-09-11T08:43:36.117378+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:43:36.117378+0000","last_undegraded":"2024-09-11T08:43:36.117378+0000","last_fullsized":"2024-09-11T08:43:36.117378+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.570246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.570246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:58.852687+0000","scrub_duration":0.0041809610000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"127'12298","reported_seq":12665,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:50.972247+0000","last_change":"2024-09-11T08:42:44.958701+0000","last_active":"2024-09-11T08:43:50.972247+0000","last_peered":"2024-09-11T08:43:50.972247+0000","last_clean":"2024-09-11T08:43:50.972247+0000","last_became_active":"2024-09-11T08:40:59.755894+0000","last_became_peered":"2024-09-11T08:40:59.755894+0000","last_unstale":"2024-09-11T08:43:50.972247+0000","last_undegraded":"2024-09-11T08:43:50.972247+0000","last_fullsized":"2024-09-11T08:43:50.972247+0000","mapping_epoch":88,"log_start":"106'6000","ondisk_log_start":"106'6000","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'8054","last_scrub_stamp":"2024-09-11T08:42:44.958606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:44.958606+0000","objects_scrubbed":7720,"log_size":6298,"log_dups_size":0,"ondisk_log_size":6298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:55.311677+0000","scrub_duration":3.0641676269999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":221970456,"num_objects":3388,"num_object_clones":0,"num_object_copies":10164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3388,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12298,"num_write_kb":501889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":139,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124440+0000","last_change":"2024-09-11T08:43:21.002231+0000","last_active":"2024-09-11T08:43:36.124440+0000","last_peered":"2024-09-11T08:43:36.124440+0000","last_clean":"2024-09-11T08:43:36.124440+0000","last_became_active":"2024-09-11T08:40:16.183867+0000","last_became_peered":"2024-09-11T08:40:16.183867+0000","last_unstale":"2024-09-11T08:43:36.124440+0000","last_undegraded":"2024-09-11T08:43:36.124440+0000","last_fullsized":"2024-09-11T08:43:36.124440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:21.002140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:21.002140+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:25.563338+0000","scrub_duration":0.0025090400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":90,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123790+0000","last_change":"2024-09-11T08:40:54.671894+0000","last_active":"2024-09-11T08:43:36.123790+0000","last_peered":"2024-09-11T08:43:36.123790+0000","last_clean":"2024-09-11T08:43:36.123790+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:43:36.123790+0000","last_undegraded":"2024-09-11T08:43:36.123790+0000","last_fullsized":"2024-09-11T08:43:36.123790+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:40:54.671832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001438541,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.117273+0000","last_change":"2024-09-11T08:42:50.619383+0000","last_active":"2024-09-11T08:43:36.117273+0000","last_peered":"2024-09-11T08:43:36.117273+0000","last_clean":"2024-09-11T08:43:36.117273+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:43:36.117273+0000","last_undegraded":"2024-09-11T08:43:36.117273+0000","last_fullsized":"2024-09-11T08:43:36.117273+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:50.619330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:50.619330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:15.881053+0000","scrub_duration":0.001262085,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.131037+0000","last_change":"2024-09-11T08:42:40.853874+0000","last_active":"2024-09-11T08:43:36.131037+0000","last_peered":"2024-09-11T08:43:36.131037+0000","last_clean":"2024-09-11T08:43:36.131037+0000","last_became_active":"2024-09-11T08:38:45.184410+0000","last_became_peered":"2024-09-11T08:38:45.184410+0000","last_unstale":"2024-09-11T08:43:36.131037+0000","last_undegraded":"2024-09-11T08:43:36.131037+0000","last_fullsized":"2024-09-11T08:43:36.131037+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:40.853782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:40.853782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:41.342555+0000","scrub_duration":0.0045130580000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"127'12463","reported_seq":12802,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:51.137998+0000","last_change":"2024-09-11T08:42:28.551092+0000","last_active":"2024-09-11T08:43:51.137998+0000","last_peered":"2024-09-11T08:43:51.137998+0000","last_clean":"2024-09-11T08:43:51.137998+0000","last_became_active":"2024-09-11T08:40:59.755838+0000","last_became_peered":"2024-09-11T08:40:59.755838+0000","last_unstale":"2024-09-11T08:43:51.137998+0000","last_undegraded":"2024-09-11T08:43:51.137998+0000","last_fullsized":"2024-09-11T08:43:51.137998+0000","mapping_epoch":88,"log_start":"103'5600","ondisk_log_start":"103'5600","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'7153","last_scrub_stamp":"2024-09-11T08:42:28.550888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:28.550888+0000","objects_scrubbed":7055,"log_size":6863,"log_dups_size":0,"ondisk_log_size":6863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"queued for scrub","scrub_duration":4.1207098550000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220135424,"num_objects":3359,"num_object_clones":0,"num_object_copies":10077,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12463,"num_write_kb":506304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":126,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124089+0000","last_change":"2024-09-11T08:41:45.328463+0000","last_active":"2024-09-11T08:43:36.124089+0000","last_peered":"2024-09-11T08:43:36.124089+0000","last_clean":"2024-09-11T08:43:36.124089+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:43:36.124089+0000","last_undegraded":"2024-09-11T08:43:36.124089+0000","last_fullsized":"2024-09-11T08:43:36.124089+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:41:45.328360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:41:45.328360+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048419429999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":109,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.122020+0000","last_change":"2024-09-11T08:42:51.805713+0000","last_active":"2024-09-11T08:43:36.122020+0000","last_peered":"2024-09-11T08:43:36.122020+0000","last_clean":"2024-09-11T08:43:36.122020+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:43:36.122020+0000","last_undegraded":"2024-09-11T08:43:36.122020+0000","last_fullsized":"2024-09-11T08:43:36.122020+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:51.805660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:51.805660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:17.456311+0000","scrub_duration":0.0032519020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.119143+0000","last_change":"2024-09-11T08:42:31.256936+0000","last_active":"2024-09-11T08:43:36.119143+0000","last_peered":"2024-09-11T08:43:36.119143+0000","last_clean":"2024-09-11T08:43:36.119143+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:43:36.119143+0000","last_undegraded":"2024-09-11T08:43:36.119143+0000","last_fullsized":"2024-09-11T08:43:36.119143+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:31.256727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:31.256727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:57.081019+0000","scrub_duration":0.055088536,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.127022+0000","last_change":"2024-09-11T08:43:27.580755+0000","last_active":"2024-09-11T08:43:36.127022+0000","last_peered":"2024-09-11T08:43:36.127022+0000","last_clean":"2024-09-11T08:43:36.127022+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:43:36.127022+0000","last_undegraded":"2024-09-11T08:43:36.127022+0000","last_fullsized":"2024-09-11T08:43:36.127022+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:27.580677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:27.580677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:31.501515+0000","scrub_duration":0.012590228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"127'12245","reported_seq":12608,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:51.397369+0000","last_change":"2024-09-11T08:42:41.749687+0000","last_active":"2024-09-11T08:43:51.397369+0000","last_peered":"2024-09-11T08:43:51.397369+0000","last_clean":"2024-09-11T08:43:51.397369+0000","last_became_active":"2024-09-11T08:40:59.755307+0000","last_became_peered":"2024-09-11T08:40:59.755307+0000","last_unstale":"2024-09-11T08:43:51.397369+0000","last_undegraded":"2024-09-11T08:43:51.397369+0000","last_fullsized":"2024-09-11T08:43:51.397369+0000","mapping_epoch":88,"log_start":"107'6600","ondisk_log_start":"107'6600","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7805","last_scrub_stamp":"2024-09-11T08:42:41.749598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:41.749598+0000","objects_scrubbed":7659,"log_size":5645,"log_dups_size":0,"ondisk_log_size":5645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:05.039544+0000","scrub_duration":5.7357807019999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220135424,"num_objects":3359,"num_object_clones":0,"num_object_copies":10077,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12245,"num_write_kb":499328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":113,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124066+0000","last_change":"2024-09-11T08:43:24.983794+0000","last_active":"2024-09-11T08:43:36.124066+0000","last_peered":"2024-09-11T08:43:36.124066+0000","last_clean":"2024-09-11T08:43:36.124066+0000","last_became_active":"2024-09-11T08:38:40.969348+0000","last_became_peered":"2024-09-11T08:38:40.969348+0000","last_unstale":"2024-09-11T08:43:36.124066+0000","last_undegraded":"2024-09-11T08:43:36.124066+0000","last_fullsized":"2024-09-11T08:43:36.124066+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:43:24.983740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:24.983740+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:38.537961+0000","scrub_duration":0.0069750209999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":99,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.133032+0000","last_change":"2024-09-11T08:43:24.975941+0000","last_active":"2024-09-11T08:43:36.133032+0000","last_peered":"2024-09-11T08:43:36.133032+0000","last_clean":"2024-09-11T08:43:36.133032+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:43:36.133032+0000","last_undegraded":"2024-09-11T08:43:36.133032+0000","last_fullsized":"2024-09-11T08:43:36.133032+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:24.975395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:24.975395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:50.930192+0000","scrub_duration":0.0047857070000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":176,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:42.732098+0000","last_change":"2024-09-11T08:43:22.553542+0000","last_active":"2024-09-11T08:43:42.732098+0000","last_peered":"2024-09-11T08:43:42.732098+0000","last_clean":"2024-09-11T08:43:42.732098+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:43:42.732098+0000","last_undegraded":"2024-09-11T08:43:42.732098+0000","last_fullsized":"2024-09-11T08:43:42.732098+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:43:22.553458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:22.553458+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:47.971884+0000","scrub_duration":0.0089134629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124618+0000","last_change":"2024-09-11T08:42:50.100332+0000","last_active":"2024-09-11T08:43:36.124618+0000","last_peered":"2024-09-11T08:43:36.124618+0000","last_clean":"2024-09-11T08:43:36.124618+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:43:36.124618+0000","last_undegraded":"2024-09-11T08:43:36.124618+0000","last_fullsized":"2024-09-11T08:43:36.124618+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:50.100253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:42:50.100253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:18.100785+0000","scrub_duration":0.013241047000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"127'12431","reported_seq":12859,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:51.131967+0000","last_change":"2024-09-11T08:42:30.989633+0000","last_active":"2024-09-11T08:43:51.131967+0000","last_peered":"2024-09-11T08:43:51.131967+0000","last_clean":"2024-09-11T08:43:51.131967+0000","last_became_active":"2024-09-11T08:40:59.754331+0000","last_became_peered":"2024-09-11T08:40:59.754331+0000","last_unstale":"2024-09-11T08:43:51.131967+0000","last_undegraded":"2024-09-11T08:43:51.131967+0000","last_fullsized":"2024-09-11T08:43:51.131967+0000","mapping_epoch":88,"log_start":"103'5600","ondisk_log_start":"103'5600","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'7272","last_scrub_stamp":"2024-09-11T08:42:30.989546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:30.989546+0000","objects_scrubbed":7197,"log_size":6831,"log_dups_size":0,"ondisk_log_size":6831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2.605776192,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":221184000,"num_objects":3375,"num_object_clones":0,"num_object_copies":10125,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12431,"num_write_kb":505792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":114,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.143561+0000","last_change":"2024-09-11T08:42:35.072141+0000","last_active":"2024-09-11T08:43:36.143561+0000","last_peered":"2024-09-11T08:43:36.143561+0000","last_clean":"2024-09-11T08:43:36.143561+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:43:36.143561+0000","last_undegraded":"2024-09-11T08:43:36.143561+0000","last_fullsized":"2024-09-11T08:43:36.143561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:42:35.072054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:42:35.072054+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:45.379420+0000","scrub_duration":0.022552168000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116863+0000","last_change":"2024-09-11T08:43:24.583745+0000","last_active":"2024-09-11T08:43:36.116863+0000","last_peered":"2024-09-11T08:43:36.116863+0000","last_clean":"2024-09-11T08:43:36.116863+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:43:36.116863+0000","last_undegraded":"2024-09-11T08:43:36.116863+0000","last_fullsized":"2024-09-11T08:43:36.116863+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:24.583665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:43:24.583665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:38.086636+0000","scrub_duration":0.0081591830000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.127671+0000","last_change":"2024-09-11T08:43:25.643701+0000","last_active":"2024-09-11T08:43:36.127671+0000","last_peered":"2024-09-11T08:43:36.127671+0000","last_clean":"2024-09-11T08:43:36.127671+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:43:36.127671+0000","last_undegraded":"2024-09-11T08:43:36.127671+0000","last_fullsized":"2024-09-11T08:43:36.127671+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:25.643625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:25.643625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:53.950125+0000","scrub_duration":0.010514694999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.136781+0000","last_change":"2024-09-11T08:42:27.029550+0000","last_active":"2024-09-11T08:43:36.136781+0000","last_peered":"2024-09-11T08:43:36.136781+0000","last_clean":"2024-09-11T08:43:36.136781+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:43:36.136781+0000","last_undegraded":"2024-09-11T08:43:36.136781+0000","last_fullsized":"2024-09-11T08:43:36.136781+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:27.029300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:27.029300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:41.134838+0000","scrub_duration":0.0029176419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"127'12418","reported_seq":12770,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:51.385032+0000","last_change":"2024-09-11T08:42:34.635136+0000","last_active":"2024-09-11T08:43:51.385032+0000","last_peered":"2024-09-11T08:43:51.385032+0000","last_clean":"2024-09-11T08:43:51.385032+0000","last_became_active":"2024-09-11T08:40:59.755928+0000","last_became_peered":"2024-09-11T08:40:59.755928+0000","last_unstale":"2024-09-11T08:43:51.385032+0000","last_undegraded":"2024-09-11T08:43:51.385032+0000","last_fullsized":"2024-09-11T08:43:51.385032+0000","mapping_epoch":88,"log_start":"108'6800","ondisk_log_start":"108'6800","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'7501","last_scrub_stamp":"2024-09-11T08:42:34.634925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:34.634925+0000","objects_scrubbed":7381,"log_size":5618,"log_dups_size":0,"ondisk_log_size":5618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3.5299405990000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226885632,"num_objects":3462,"num_object_clones":0,"num_object_copies":10386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12418,"num_write_kb":508160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":138,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.121847+0000","last_change":"2024-09-11T08:43:19.146635+0000","last_active":"2024-09-11T08:43:36.121847+0000","last_peered":"2024-09-11T08:43:36.121847+0000","last_clean":"2024-09-11T08:43:36.121847+0000","last_became_active":"2024-09-11T08:38:41.040207+0000","last_became_peered":"2024-09-11T08:38:41.040207+0000","last_unstale":"2024-09-11T08:43:36.121847+0000","last_undegraded":"2024-09-11T08:43:36.121847+0000","last_fullsized":"2024-09-11T08:43:36.121847+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:43:19.146540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:19.146540+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:41.603801+0000","scrub_duration":0.0038449669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":109,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.126467+0000","last_change":"2024-09-11T08:42:51.960897+0000","last_active":"2024-09-11T08:43:36.126467+0000","last_peered":"2024-09-11T08:43:36.126467+0000","last_clean":"2024-09-11T08:43:36.126467+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:43:36.126467+0000","last_undegraded":"2024-09-11T08:43:36.126467+0000","last_fullsized":"2024-09-11T08:43:36.126467+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:51.960847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:51.960847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:11.954924+0000","scrub_duration":0.0017378840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.127344+0000","last_change":"2024-09-11T08:43:20.462355+0000","last_active":"2024-09-11T08:43:36.127344+0000","last_peered":"2024-09-11T08:43:36.127344+0000","last_clean":"2024-09-11T08:43:36.127344+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:43:36.127344+0000","last_undegraded":"2024-09-11T08:43:36.127344+0000","last_fullsized":"2024-09-11T08:43:36.127344+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:20.462184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:20.462184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:25.409093+0000","scrub_duration":0.0037598269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124902+0000","last_change":"2024-09-11T08:42:40.024673+0000","last_active":"2024-09-11T08:43:36.124902+0000","last_peered":"2024-09-11T08:43:36.124902+0000","last_clean":"2024-09-11T08:43:36.124902+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:43:36.124902+0000","last_undegraded":"2024-09-11T08:43:36.124902+0000","last_fullsized":"2024-09-11T08:43:36.124902+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:40.024586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:40.024586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:02.493996+0000","scrub_duration":0.0063827069999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"127'12549","reported_seq":12817,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:51.286006+0000","last_change":"2024-09-11T08:43:25.901583+0000","last_active":"2024-09-11T08:43:51.286006+0000","last_peered":"2024-09-11T08:43:51.286006+0000","last_clean":"2024-09-11T08:43:51.286006+0000","last_became_active":"2024-09-11T08:40:59.753273+0000","last_became_peered":"2024-09-11T08:40:59.753273+0000","last_unstale":"2024-09-11T08:43:51.286006+0000","last_undegraded":"2024-09-11T08:43:51.286006+0000","last_fullsized":"2024-09-11T08:43:51.286006+0000","mapping_epoch":88,"log_start":"109'7900","ondisk_log_start":"109'7900","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"122'10757","last_scrub_stamp":"2024-09-11T08:43:25.901499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:43:25.901499+0000","objects_scrubbed":5291,"log_size":4649,"log_dups_size":0,"ondisk_log_size":4649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:48.183540+0000","scrub_duration":2.8764546480000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223150080,"num_objects":3405,"num_object_clones":0,"num_object_copies":10215,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12549,"num_write_kb":510528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":121,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.123600+0000","last_change":"2024-09-11T08:41:49.328291+0000","last_active":"2024-09-11T08:43:36.123600+0000","last_peered":"2024-09-11T08:43:36.123600+0000","last_clean":"2024-09-11T08:43:36.123600+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:43:36.123600+0000","last_undegraded":"2024-09-11T08:43:36.123600+0000","last_fullsized":"2024-09-11T08:43:36.123600+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:41:49.328238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:41:49.328238+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063967870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":109,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.125686+0000","last_change":"2024-09-11T08:43:17.025878+0000","last_active":"2024-09-11T08:43:36.125686+0000","last_peered":"2024-09-11T08:43:36.125686+0000","last_clean":"2024-09-11T08:43:36.125686+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:43:36.125686+0000","last_undegraded":"2024-09-11T08:43:36.125686+0000","last_fullsized":"2024-09-11T08:43:36.125686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:17.025803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:17.025803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:40.286553+0000","scrub_duration":0.0041414900000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.128113+0000","last_change":"2024-09-11T08:43:17.495291+0000","last_active":"2024-09-11T08:43:36.128113+0000","last_peered":"2024-09-11T08:43:36.128113+0000","last_clean":"2024-09-11T08:43:36.128113+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:43:36.128113+0000","last_undegraded":"2024-09-11T08:43:36.128113+0000","last_fullsized":"2024-09-11T08:43:36.128113+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:17.495214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:17.495214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:25.389508+0000","scrub_duration":0.0045506940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.132773+0000","last_change":"2024-09-11T08:43:28.055111+0000","last_active":"2024-09-11T08:43:36.132773+0000","last_peered":"2024-09-11T08:43:36.132773+0000","last_clean":"2024-09-11T08:43:36.132773+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:43:36.132773+0000","last_undegraded":"2024-09-11T08:43:36.132773+0000","last_fullsized":"2024-09-11T08:43:36.132773+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:28.055031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:28.055031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:56.188615+0000","scrub_duration":0.0082310189999999991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"127'12135","reported_seq":12493,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:51.159104+0000","last_change":"2024-09-11T08:42:39.077974+0000","last_active":"2024-09-11T08:43:51.159104+0000","last_peered":"2024-09-11T08:43:51.159104+0000","last_clean":"2024-09-11T08:43:51.159104+0000","last_became_active":"2024-09-11T08:40:59.753438+0000","last_became_peered":"2024-09-11T08:40:59.753438+0000","last_unstale":"2024-09-11T08:43:51.159104+0000","last_undegraded":"2024-09-11T08:43:51.159104+0000","last_fullsized":"2024-09-11T08:43:51.159104+0000","mapping_epoch":88,"log_start":"108'7100","ondisk_log_start":"108'7100","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7616","last_scrub_stamp":"2024-09-11T08:42:39.077893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:58.730067+0000","last_clean_scrub_stamp":"2024-09-11T08:42:39.077893+0000","objects_scrubbed":7545,"log_size":5035,"log_dups_size":0,"ondisk_log_size":5035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2.8168520839999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225116160,"num_objects":3435,"num_object_clones":0,"num_object_copies":10305,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12135,"num_write_kb":498240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"124'22","reported_seq":140,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116421+0000","last_change":"2024-09-11T08:43:32.625436+0000","last_active":"2024-09-11T08:43:36.116421+0000","last_peered":"2024-09-11T08:43:36.116421+0000","last_clean":"2024-09-11T08:43:36.116421+0000","last_became_active":"2024-09-11T08:38:41.040475+0000","last_became_peered":"2024-09-11T08:38:41.040475+0000","last_unstale":"2024-09-11T08:43:36.116421+0000","last_undegraded":"2024-09-11T08:43:36.116421+0000","last_fullsized":"2024-09-11T08:43:36.116421+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:43:32.625376+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:41.442241+0000","scrub_duration":0.0073394039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":126,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.125561+0000","last_change":"2024-09-11T08:43:27.137539+0000","last_active":"2024-09-11T08:43:36.125561+0000","last_peered":"2024-09-11T08:43:36.125561+0000","last_clean":"2024-09-11T08:43:36.125561+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:43:36.125561+0000","last_undegraded":"2024-09-11T08:43:36.125561+0000","last_fullsized":"2024-09-11T08:43:36.125561+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:43:27.137095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:27.137095+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:49.370638+0000","scrub_duration":0.024287229,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"63'8","reported_seq":175,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:42.731499+0000","last_change":"2024-09-11T08:42:54.045854+0000","last_active":"2024-09-11T08:43:42.731499+0000","last_peered":"2024-09-11T08:43:42.731499+0000","last_clean":"2024-09-11T08:43:42.731499+0000","last_became_active":"2024-09-11T08:38:42.986756+0000","last_became_peered":"2024-09-11T08:38:42.986756+0000","last_unstale":"2024-09-11T08:43:42.731499+0000","last_undegraded":"2024-09-11T08:43:42.731499+0000","last_fullsized":"2024-09-11T08:43:42.731499+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:42:54.045763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:54.045763+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:08.949073+0000","scrub_duration":0.001794164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116501+0000","last_change":"2024-09-11T08:42:24.399593+0000","last_active":"2024-09-11T08:43:36.116501+0000","last_peered":"2024-09-11T08:43:36.116501+0000","last_clean":"2024-09-11T08:43:36.116501+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:43:36.116501+0000","last_undegraded":"2024-09-11T08:43:36.116501+0000","last_fullsized":"2024-09-11T08:43:36.116501+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:24.399510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:24.399510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:39.903679+0000","scrub_duration":0.002003959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":109,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.121866+0000","last_change":"2024-09-11T08:42:37.900470+0000","last_active":"2024-09-11T08:43:36.121866+0000","last_peered":"2024-09-11T08:43:36.121866+0000","last_clean":"2024-09-11T08:43:36.121866+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:43:36.121866+0000","last_undegraded":"2024-09-11T08:43:36.121866+0000","last_fullsized":"2024-09-11T08:43:36.121866+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:37.900396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:55.587871+0000","last_clean_scrub_stamp":"2024-09-11T08:42:37.900396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:44.921779+0000","scrub_duration":0.0037448859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":124,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.133325+0000","last_change":"2024-09-11T08:42:36.813790+0000","last_active":"2024-09-11T08:43:36.133325+0000","last_peered":"2024-09-11T08:43:36.133325+0000","last_clean":"2024-09-11T08:43:36.133325+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:43:36.133325+0000","last_undegraded":"2024-09-11T08:43:36.133325+0000","last_fullsized":"2024-09-11T08:43:36.133325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-11T08:42:36.813701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:42:36.813701+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:44.153537+0000","scrub_duration":0.030877587000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.118425+0000","last_change":"2024-09-11T08:43:18.699360+0000","last_active":"2024-09-11T08:43:36.118425+0000","last_peered":"2024-09-11T08:43:36.118425+0000","last_clean":"2024-09-11T08:43:36.118425+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:43:36.118425+0000","last_undegraded":"2024-09-11T08:43:36.118425+0000","last_fullsized":"2024-09-11T08:43:36.118425+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:18.699275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:18.699275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:36.698947+0000","scrub_duration":0.0041993969999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.126042+0000","last_change":"2024-09-11T08:42:24.944327+0000","last_active":"2024-09-11T08:43:36.126042+0000","last_peered":"2024-09-11T08:43:36.126042+0000","last_clean":"2024-09-11T08:43:36.126042+0000","last_became_active":"2024-09-11T08:38:45.184753+0000","last_became_peered":"2024-09-11T08:38:45.184753+0000","last_unstale":"2024-09-11T08:43:36.126042+0000","last_undegraded":"2024-09-11T08:43:36.126042+0000","last_fullsized":"2024-09-11T08:43:36.126042+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:24.944242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:24.944242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:44.035607+0000","scrub_duration":0.001616784,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":114,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.144399+0000","last_change":"2024-09-11T08:41:59.013658+0000","last_active":"2024-09-11T08:43:36.144399+0000","last_peered":"2024-09-11T08:43:36.144399+0000","last_clean":"2024-09-11T08:43:36.144399+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:43:36.144399+0000","last_undegraded":"2024-09-11T08:43:36.144399+0000","last_fullsized":"2024-09-11T08:43:36.144399+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:41:59.013187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:41:59.013187+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012093042,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":109,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124965+0000","last_change":"2024-09-11T08:42:49.945200+0000","last_active":"2024-09-11T08:43:36.124965+0000","last_peered":"2024-09-11T08:43:36.124965+0000","last_clean":"2024-09-11T08:43:36.124965+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:43:36.124965+0000","last_undegraded":"2024-09-11T08:43:36.124965+0000","last_fullsized":"2024-09-11T08:43:36.124965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:49.945135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:49.945135+0000","last_clean_scrub_stamp":"2024-09-11T08:42:49.945135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:18.914924+0000","scrub_duration":0.0012025040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":95,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.133180+0000","last_change":"2024-09-11T08:42:54.010343+0000","last_active":"2024-09-11T08:43:36.133180+0000","last_peered":"2024-09-11T08:43:36.133180+0000","last_clean":"2024-09-11T08:43:36.133180+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:43:36.133180+0000","last_undegraded":"2024-09-11T08:43:36.133180+0000","last_fullsized":"2024-09-11T08:43:36.133180+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:54.010203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:54.010203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:23.410125+0000","scrub_duration":0.019363425,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.126423+0000","last_change":"2024-09-11T08:43:19.468579+0000","last_active":"2024-09-11T08:43:36.126423+0000","last_peered":"2024-09-11T08:43:36.126423+0000","last_clean":"2024-09-11T08:43:36.126423+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:43:36.126423+0000","last_undegraded":"2024-09-11T08:43:36.126423+0000","last_fullsized":"2024-09-11T08:43:36.126423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:19.468499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_clean_scrub_stamp":"2024-09-11T08:43:19.468499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:43.282595+0000","scrub_duration":0.0015738169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":114,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116986+0000","last_change":"2024-09-11T08:42:25.441884+0000","last_active":"2024-09-11T08:43:36.116986+0000","last_peered":"2024-09-11T08:43:36.116986+0000","last_clean":"2024-09-11T08:43:36.116986+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:43:36.116986+0000","last_undegraded":"2024-09-11T08:43:36.116986+0000","last_fullsized":"2024-09-11T08:43:36.116986+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:42:25.441804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:42:25.441804+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:42.898593+0000","scrub_duration":0.0036498849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":100,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.137158+0000","last_change":"2024-09-11T08:41:43.851759+0000","last_active":"2024-09-11T08:43:36.137158+0000","last_peered":"2024-09-11T08:43:36.137158+0000","last_clean":"2024-09-11T08:43:36.137158+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:43:36.137158+0000","last_undegraded":"2024-09-11T08:43:36.137158+0000","last_fullsized":"2024-09-11T08:43:36.137158+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:43.851663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:41:43.851663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062910420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":165,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:42.734192+0000","last_change":"2024-09-11T08:42:46.918288+0000","last_active":"2024-09-11T08:43:42.734192+0000","last_peered":"2024-09-11T08:43:42.734192+0000","last_clean":"2024-09-11T08:43:42.734192+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:43:42.734192+0000","last_undegraded":"2024-09-11T08:43:42.734192+0000","last_fullsized":"2024-09-11T08:43:42.734192+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:42:46.918041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:46.918041+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:55.837955+0000","scrub_duration":0.0042043150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":84,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124076+0000","last_change":"2024-09-11T08:40:46.709390+0000","last_active":"2024-09-11T08:43:36.124076+0000","last_peered":"2024-09-11T08:43:36.124076+0000","last_clean":"2024-09-11T08:43:36.124076+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:43:36.124076+0000","last_undegraded":"2024-09-11T08:43:36.124076+0000","last_fullsized":"2024-09-11T08:43:36.124076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:40:46.709322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:40:46.709322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017239779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":112,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.133261+0000","last_change":"2024-09-11T08:42:47.929375+0000","last_active":"2024-09-11T08:43:36.133261+0000","last_peered":"2024-09-11T08:43:36.133261+0000","last_clean":"2024-09-11T08:43:36.133261+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:43:36.133261+0000","last_undegraded":"2024-09-11T08:43:36.133261+0000","last_fullsized":"2024-09-11T08:43:36.133261+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:42:47.929291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:42:47.929291+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:06.084342+0000","scrub_duration":0.017023771,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":99,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.125214+0000","last_change":"2024-09-11T08:42:35.039885+0000","last_active":"2024-09-11T08:43:36.125214+0000","last_peered":"2024-09-11T08:43:36.125214+0000","last_clean":"2024-09-11T08:43:36.125214+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:43:36.125214+0000","last_undegraded":"2024-09-11T08:43:36.125214+0000","last_fullsized":"2024-09-11T08:43:36.125214+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:35.039796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:35.039796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:04.620638+0000","scrub_duration":0.012556869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":176,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-11T08:43:42.731306+0000","last_change":"2024-09-11T08:43:28.525724+0000","last_active":"2024-09-11T08:43:42.731306+0000","last_peered":"2024-09-11T08:43:42.731306+0000","last_clean":"2024-09-11T08:43:42.731306+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:43:42.731306+0000","last_undegraded":"2024-09-11T08:43:42.731306+0000","last_fullsized":"2024-09-11T08:43:42.731306+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:43:28.525595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:28.525595+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:52.637367+0000","scrub_duration":0.0046784749999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.136601+0000","last_change":"2024-09-11T08:43:28.384507+0000","last_active":"2024-09-11T08:43:36.136601+0000","last_peered":"2024-09-11T08:43:36.136601+0000","last_clean":"2024-09-11T08:43:36.136601+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:43:36.136601+0000","last_undegraded":"2024-09-11T08:43:36.136601+0000","last_fullsized":"2024-09-11T08:43:36.136601+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:28.384312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:28.384312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:32.247444+0000","scrub_duration":0.010892657,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":117,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:37.137328+0000","last_change":"2024-09-11T08:43:37.137260+0000","last_active":"2024-09-11T08:43:37.137328+0000","last_peered":"2024-09-11T08:43:37.137328+0000","last_clean":"2024-09-11T08:43:37.137328+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:43:37.137328+0000","last_undegraded":"2024-09-11T08:43:37.137328+0000","last_fullsized":"2024-09-11T08:43:37.137328+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:37.137173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:37.137173+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:56.344201+0000","scrub_duration":0.0085039490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":100,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124172+0000","last_change":"2024-09-11T08:42:54.072438+0000","last_active":"2024-09-11T08:43:36.124172+0000","last_peered":"2024-09-11T08:43:36.124172+0000","last_clean":"2024-09-11T08:43:36.124172+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:43:36.124172+0000","last_undegraded":"2024-09-11T08:43:36.124172+0000","last_fullsized":"2024-09-11T08:43:36.124172+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:54.072359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:42:54.072359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:04.188518+0000","scrub_duration":0.00138809,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.119431+0000","last_change":"2024-09-11T08:42:24.194351+0000","last_active":"2024-09-11T08:43:36.119431+0000","last_peered":"2024-09-11T08:43:36.119431+0000","last_clean":"2024-09-11T08:43:36.119431+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:43:36.119431+0000","last_undegraded":"2024-09-11T08:43:36.119431+0000","last_fullsized":"2024-09-11T08:43:36.119431+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:24.194249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:24.194249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0084760800000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.130539+0000","last_change":"2024-09-11T08:43:17.962352+0000","last_active":"2024-09-11T08:43:36.130539+0000","last_peered":"2024-09-11T08:43:36.130539+0000","last_clean":"2024-09-11T08:43:36.130539+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:43:36.130539+0000","last_undegraded":"2024-09-11T08:43:36.130539+0000","last_fullsized":"2024-09-11T08:43:36.130539+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:17.962284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:17.962284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:46.961030+0000","scrub_duration":0.015011818,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":97,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.116319+0000","last_change":"2024-09-11T08:43:20.621567+0000","last_active":"2024-09-11T08:43:36.116319+0000","last_peered":"2024-09-11T08:43:36.116319+0000","last_clean":"2024-09-11T08:43:36.116319+0000","last_became_active":"2024-09-11T08:40:16.181181+0000","last_became_peered":"2024-09-11T08:40:16.181181+0000","last_unstale":"2024-09-11T08:43:36.116319+0000","last_undegraded":"2024-09-11T08:43:36.116319+0000","last_fullsized":"2024-09-11T08:43:36.116319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:43:20.621471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:20.621471+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:50.034800+0000","scrub_duration":0.0039709369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":109,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:37.206228+0000","last_change":"2024-09-11T08:43:37.206161+0000","last_active":"2024-09-11T08:43:37.206228+0000","last_peered":"2024-09-11T08:43:37.206228+0000","last_clean":"2024-09-11T08:43:37.206228+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:43:37.206228+0000","last_undegraded":"2024-09-11T08:43:37.206228+0000","last_fullsized":"2024-09-11T08:43:37.206228+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:37.206096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:37.206096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:41.725411+0000","scrub_duration":0.001561808,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":95,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.132844+0000","last_change":"2024-09-11T08:42:29.790258+0000","last_active":"2024-09-11T08:43:36.132844+0000","last_peered":"2024-09-11T08:43:36.132844+0000","last_clean":"2024-09-11T08:43:36.132844+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:43:36.132844+0000","last_undegraded":"2024-09-11T08:43:36.132844+0000","last_fullsized":"2024-09-11T08:43:36.132844+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:29.790174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:29.790174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:46.828100+0000","scrub_duration":0.016936698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.135549+0000","last_change":"2024-09-11T08:43:29.349431+0000","last_active":"2024-09-11T08:43:36.135549+0000","last_peered":"2024-09-11T08:43:36.135549+0000","last_clean":"2024-09-11T08:43:36.135549+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:43:36.135549+0000","last_undegraded":"2024-09-11T08:43:36.135549+0000","last_fullsized":"2024-09-11T08:43:36.135549+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.349351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.349351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:57.738748+0000","scrub_duration":0.006276294,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.130663+0000","last_change":"2024-09-11T08:40:54.351650+0000","last_active":"2024-09-11T08:43:36.130663+0000","last_peered":"2024-09-11T08:43:36.130663+0000","last_clean":"2024-09-11T08:43:36.130663+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:43:36.130663+0000","last_undegraded":"2024-09-11T08:43:36.130663+0000","last_fullsized":"2024-09-11T08:43:36.130663+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:40:54.351585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:40:54.351585+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021705309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.118462+0000","last_change":"2024-09-11T08:43:35.725634+0000","last_active":"2024-09-11T08:43:36.118462+0000","last_peered":"2024-09-11T08:43:36.118462+0000","last_clean":"2024-09-11T08:43:36.118462+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:43:36.118462+0000","last_undegraded":"2024-09-11T08:43:36.118462+0000","last_fullsized":"2024-09-11T08:43:36.118462+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:35.725556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:35.725556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:42.652307+0000","scrub_duration":0.0043755219999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":105,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124561+0000","last_change":"2024-09-11T08:42:44.949659+0000","last_active":"2024-09-11T08:43:36.124561+0000","last_peered":"2024-09-11T08:43:36.124561+0000","last_clean":"2024-09-11T08:43:36.124561+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:43:36.124561+0000","last_undegraded":"2024-09-11T08:43:36.124561+0000","last_fullsized":"2024-09-11T08:43:36.124561+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:44.949525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:44.949525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:09.104543+0000","scrub_duration":0.001556291,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.132967+0000","last_change":"2024-09-11T08:43:32.115066+0000","last_active":"2024-09-11T08:43:36.132967+0000","last_peered":"2024-09-11T08:43:36.132967+0000","last_clean":"2024-09-11T08:43:36.132967+0000","last_became_active":"2024-09-11T08:38:45.183548+0000","last_became_peered":"2024-09-11T08:38:45.183548+0000","last_unstale":"2024-09-11T08:43:36.132967+0000","last_undegraded":"2024-09-11T08:43:36.132967+0000","last_fullsized":"2024-09-11T08:43:36.132967+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:32.114990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:32.114990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:59.616575+0000","scrub_duration":0.014089513999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":103,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.135685+0000","last_change":"2024-09-11T08:41:36.323880+0000","last_active":"2024-09-11T08:43:36.135685+0000","last_peered":"2024-09-11T08:43:36.135685+0000","last_clean":"2024-09-11T08:43:36.135685+0000","last_became_active":"2024-09-11T08:38:40.973799+0000","last_became_peered":"2024-09-11T08:38:40.973799+0000","last_unstale":"2024-09-11T08:43:36.135685+0000","last_undegraded":"2024-09-11T08:43:36.135685+0000","last_fullsized":"2024-09-11T08:43:36.135685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-11T08:41:36.323786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:41:36.323786+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071016270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":117,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.136462+0000","last_change":"2024-09-11T08:43:17.340312+0000","last_active":"2024-09-11T08:43:36.136462+0000","last_peered":"2024-09-11T08:43:36.136462+0000","last_clean":"2024-09-11T08:43:36.136462+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:43:36.136462+0000","last_undegraded":"2024-09-11T08:43:36.136462+0000","last_fullsized":"2024-09-11T08:43:36.136462+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:43:17.339897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:17.339897+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:36.637359+0000","scrub_duration":0.012677927,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":105,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.122140+0000","last_change":"2024-09-11T08:42:49.803075+0000","last_active":"2024-09-11T08:43:36.122140+0000","last_peered":"2024-09-11T08:43:36.122140+0000","last_clean":"2024-09-11T08:43:36.122140+0000","last_became_active":"2024-09-11T08:38:42.989556+0000","last_became_peered":"2024-09-11T08:38:42.989556+0000","last_unstale":"2024-09-11T08:43:36.122140+0000","last_undegraded":"2024-09-11T08:43:36.122140+0000","last_fullsized":"2024-09-11T08:43:36.122140+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:49.802985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:42:49.802985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:03.649380+0000","scrub_duration":0.0014514,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":103,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.124712+0000","last_change":"2024-09-11T08:42:50.985633+0000","last_active":"2024-09-11T08:43:36.124712+0000","last_peered":"2024-09-11T08:43:36.124712+0000","last_clean":"2024-09-11T08:43:36.124712+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:43:36.124712+0000","last_undegraded":"2024-09-11T08:43:36.124712+0000","last_fullsized":"2024-09-11T08:43:36.124712+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:50.985537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:50.985537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:59.594451+0000","scrub_duration":0.0077766800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":104,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.120244+0000","last_change":"2024-09-11T08:42:32.215521+0000","last_active":"2024-09-11T08:43:36.120244+0000","last_peered":"2024-09-11T08:43:36.120244+0000","last_clean":"2024-09-11T08:43:36.120244+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:43:36.120244+0000","last_undegraded":"2024-09-11T08:43:36.120244+0000","last_fullsized":"2024-09-11T08:43:36.120244+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:42:32.215417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:42:32.215417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053169189999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":99,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.135875+0000","last_change":"2024-09-11T08:41:32.353132+0000","last_active":"2024-09-11T08:43:36.135875+0000","last_peered":"2024-09-11T08:43:36.135875+0000","last_clean":"2024-09-11T08:43:36.135875+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:43:36.135875+0000","last_undegraded":"2024-09-11T08:43:36.135875+0000","last_fullsized":"2024-09-11T08:43:36.135875+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:41:32.353055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:41:32.353055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011132267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":126,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.122406+0000","last_change":"2024-09-11T08:43:31.108299+0000","last_active":"2024-09-11T08:43:36.122406+0000","last_peered":"2024-09-11T08:43:36.122406+0000","last_clean":"2024-09-11T08:43:36.122406+0000","last_became_active":"2024-09-11T08:38:40.974451+0000","last_became_peered":"2024-09-11T08:38:40.974451+0000","last_unstale":"2024-09-11T08:43:36.122406+0000","last_undegraded":"2024-09-11T08:43:36.122406+0000","last_fullsized":"2024-09-11T08:43:36.122406+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:31.108210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:31.108210+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:45:00.896181+0000","scrub_duration":0.0045306019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":106,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-11T08:43:36.144235+0000","last_change":"2024-09-11T08:43:30.336272+0000","last_active":"2024-09-11T08:43:36.144235+0000","last_peered":"2024-09-11T08:43:36.144235+0000","last_clean":"2024-09-11T08:43:36.144235+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:43:36.144235+0000","last_undegraded":"2024-09-11T08:43:36.144235+0000","last_fullsized":"2024-09-11T08:43:36.144235+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:30.336188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:30.336188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:45.754555+0000","scrub_duration":0.0044263979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":3561160728,"num_objects":54340,"num_object_clones":0,"num_object_copies":163020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54340,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":196712,"num_write_kb":8033601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10688147456,"data_stored":10688135240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90079,"ondisk_log_size":90079,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":449,"ondisk_log_size":449,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":305,"num_read_kb":300,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299460,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1407532,"kb_used_data":1097700,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92337620,"statfs":{"total":95995035648,"available":94553722880,"internally_reserved":0,"allocated":1124044800,"data_stored":1123711398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:43:35 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":0.78600000000000003,"15min":0.59699999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.9260000000000002,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":0.309},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.879,"15min":0.63100000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":3.806,"5min":5.9790000000000001,"15min":5.9790000000000001},"last":4.6459999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:43:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":0.93799999999999994,"15min":0.65000000000000002},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":8.5030000000000001,"5min":8.5030000000000001,"15min":8.5030000000000001},"last":1.8129999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":0.78300000000000003,"15min":0.60999999999999999},"min":{"1min":0.30599999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.9830000000000001,"5min":6.593,"15min":6.593},"last":1.329}]},{"osd":2,"last update":"Wed Sep 11 08:43:35 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.78800000000000003,"15min":0.60099999999999998},"min":{"1min":0.28499999999999998,"5min":0.255,"15min":0.255},"max":{"1min":2.2749999999999999,"5min":4.6029999999999998,"15min":4.6029999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.876,"5min":0.69199999999999995,"15min":0.55700000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.6549999999999998,"5min":2.6579999999999999,"15min":2.6579999999999999},"last":1.056}]},{"osd":3,"last update":"Wed Sep 11 08:43:35 2024","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":0.90500000000000003,"15min":0.65100000000000002},"min":{"1min":0.28000000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":7.8609999999999998,"5min":7.8609999999999998,"15min":7.8609999999999998},"last":1.206},{"interface":"front","average":{"1min":2.0369999999999999,"5min":1.0580000000000001,"15min":0.70599999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":6.2080000000000002,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":0.36399999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:43:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.47799999999999998,"15min":0.44},"min":{"1min":0.249,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46300000000000002,"15min":0.435},"min":{"1min":0.28100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70099999999999996,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.28299999999999997}]},{"osd":5,"last update":"Wed Sep 11 08:43:35 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.496,"15min":0.45600000000000002},"min":{"1min":0.26600000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.57399999999999995,"15min":0.48199999999999998},"min":{"1min":0.29099999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.65200000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:43:35 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.59899999999999998,"15min":0.51100000000000001},"min":{"1min":0.317,"5min":0.315,"15min":0.315},"max":{"1min":3.5979999999999999,"5min":3.5979999999999999,"15min":3.5979999999999999},"last":0.628},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.56999999999999995,"15min":0.504},"min":{"1min":0.309,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":0.35499999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528393,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1411748,"kb_used_data":1101916,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92333404,"statfs":{"total":95995035648,"available":94549405696,"internally_reserved":0,"allocated":1128361984,"data_stored":1128036634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":2.3929999999999998,"5min":1.1839999999999999,"15min":0.75},"min":{"1min":0.23599999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":28.302,"5min":28.302,"15min":28.302},"last":2.0640000000000001},{"interface":"front","average":{"1min":2.7890000000000001,"5min":1.097,"15min":0.72099999999999997},"min":{"1min":0.24399999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":28.314,"5min":28.314,"15min":28.314},"last":2.2999999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.69999999999999996,"15min":0.58799999999999997},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.8639999999999999,"5min":4.0839999999999996,"15min":4.0839999999999996},"last":1.9319999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.69599999999999995,"15min":0.57999999999999996},"min":{"1min":0.254,"5min":0.253,"15min":0.253},"max":{"1min":3.4209999999999998,"5min":3.4209999999999998,"15min":3.4209999999999998},"last":0.25600000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.65500000000000003,"15min":0.58699999999999997},"min":{"1min":0.28299999999999997,"5min":0.25,"15min":0.25},"max":{"1min":3.0710000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":1.851},{"interface":"front","average":{"1min":1.01,"5min":0.68300000000000005,"15min":0.56699999999999995},"min":{"1min":0.27200000000000002,"5min":0.251,"15min":0.251},"max":{"1min":3.282,"5min":4.25,"15min":4.25},"last":2.5150000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":0.85899999999999999,"15min":0.63300000000000001},"min":{"1min":0.36899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":8.1869999999999994,"5min":8.1869999999999994,"15min":8.1869999999999994},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.6240000000000001,"5min":0.92400000000000004,"15min":0.66600000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":9.1069999999999993,"5min":9.1069999999999993,"15min":9.1069999999999993},"last":0.224}]},{"osd":4,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42799999999999999,"15min":0.41299999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.214},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.41999999999999998,"15min":0.40999999999999998},"min":{"1min":0.28599999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.27600000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47599999999999998,"15min":0.44400000000000001},"min":{"1min":0.27000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.248},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.46999999999999997,"15min":0.442},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.312}]},{"osd":7,"last update":"Wed Sep 11 08:43:30 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.49199999999999999,"15min":0.44800000000000001},"min":{"1min":0.25900000000000001,"5min":0.247,"15min":0.247},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.495},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.29599999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757325,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1183144,"kb_used_data":873312,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92562008,"statfs":{"total":95995035648,"available":94783496192,"internally_reserved":0,"allocated":894271488,"data_stored":893942271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:43:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":0.81599999999999995,"15min":0.64800000000000002},"min":{"1min":0.20899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":7.3899999999999997,"5min":7.3899999999999997,"15min":7.3899999999999997},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.341,"5min":0.97699999999999998,"15min":0.67800000000000005},"min":{"1min":0.23400000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":6.5789999999999997,"5min":6.5789999999999997,"15min":6.5789999999999997},"last":0.36399999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:43:49 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":0.84499999999999997,"15min":0.66400000000000003},"min":{"1min":0.26100000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":3.3540000000000001,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":0.434},{"interface":"front","average":{"1min":1.4550000000000001,"5min":0.82699999999999996,"15min":0.61699999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":6.6719999999999997,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.25700000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:43:49 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":0.83999999999999997,"15min":0.64200000000000002},"min":{"1min":0.28000000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":3.0230000000000001,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.32700000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":0.72599999999999998,"15min":0.61599999999999999},"min":{"1min":0.29299999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.2519999999999998,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.46800000000000003}]},{"osd":3,"last update":"Wed Sep 11 08:43:49 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":0.89100000000000001,"15min":0.65700000000000003},"min":{"1min":0.28399999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":6.0460000000000003,"5min":6.0460000000000003,"15min":6.0460000000000003},"last":2.9129999999999998},{"interface":"front","average":{"1min":1.5860000000000001,"5min":0.997,"15min":0.70799999999999996},"min":{"1min":0.308,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":7.4379999999999997,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":3.0339999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:43:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.42399999999999999,"15min":0.39000000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.44900000000000001,"15min":0.41299999999999998},"min":{"1min":0.24399999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.51200000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45700000000000002,"15min":0.435},"min":{"1min":0.29299999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0149999999999999,"5min":1.702,"15min":1.702},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.44800000000000001,"15min":0.42699999999999999},"min":{"1min":0.27900000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.86699999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.39700000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:43:28 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46600000000000003,"15min":0.47999999999999998},"min":{"1min":0.32500000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71899999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.497,"15min":0.53300000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.622,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.69299999999999995}]}]},{"osd":4,"up_from":31,"seq":133143986257,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1397032,"kb_used_data":1099680,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92348120,"statfs":{"total":95995035648,"available":94564474880,"internally_reserved":0,"allocated":1126072320,"data_stored":1125741932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:43:27 2024","interfaces":[{"interface":"back","average":{"1min":1.9299999999999999,"5min":2.004,"15min":1.036},"min":{"1min":0.24099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":7.4450000000000003,"5min":89.510000000000005,"15min":89.510000000000005},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.579,"5min":1.964,"15min":1.04},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":3.73,"5min":89.765000000000001,"15min":89.765000000000001},"last":0.67000000000000004}]},{"osd":1,"last update":"Wed Sep 11 08:43:27 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":0.70099999999999996,"15min":0.61199999999999999},"min":{"1min":0.27200000000000002,"5min":0.254,"15min":0.254},"max":{"1min":5.4059999999999997,"5min":5.4059999999999997,"15min":5.4059999999999997},"last":4.0730000000000004},{"interface":"front","average":{"1min":1.2470000000000001,"5min":0.73399999999999999,"15min":0.62},"min":{"1min":0.255,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":3.319,"5min":3.319,"15min":3.319},"last":0.54900000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:43:27 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":0.73299999999999998,"15min":0.63200000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":5.3330000000000002,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":2.883},{"interface":"front","average":{"1min":1.7070000000000001,"5min":0.81499999999999995,"15min":0.66100000000000003},"min":{"1min":0.29799999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":1.0209999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:43:27 2024","interfaces":[{"interface":"back","average":{"1min":2.097,"5min":2.1539999999999999,"15min":1.1100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":8.8919999999999995,"5min":89.881,"15min":89.881},"last":2.9340000000000002},{"interface":"front","average":{"1min":1.861,"5min":2.0609999999999999,"15min":1.0629999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":5.7060000000000004,"5min":89.715000000000003,"15min":89.715000000000003},"last":0.371}]},{"osd":5,"last update":"Wed Sep 11 08:42:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45600000000000002,"15min":0.40400000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.439,"15min":0.41499999999999998},"min":{"1min":0.29099999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.439}]},{"osd":6,"last update":"Wed Sep 11 08:43:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46100000000000002,"15min":0.434},"min":{"1min":0.28000000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.79600000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.53100000000000003,"15min":0.48099999999999998},"min":{"1min":0.30099999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.34000000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:43:31 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.504,"15min":0.48599999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.1339999999999999,"5min":1.194,"15min":1.194},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.504,"15min":0.47699999999999998},"min":{"1min":0.30499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.42599999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149783,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2264256,"kb_used_data":1958584,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91480896,"statfs":{"total":95995035648,"available":93676437504,"internally_reserved":0,"allocated":2005590016,"data_stored":2005236633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":2.5190000000000001,"5min":0.97899999999999998,"15min":0.57899999999999996},"min":{"1min":0.189,"5min":0.184,"15min":0.184},"max":{"1min":10.901999999999999,"5min":10.901999999999999,"15min":10.901999999999999},"last":2.6349999999999998},{"interface":"front","average":{"1min":2.6240000000000001,"5min":1.099,"15min":0.63700000000000001},"min":{"1min":0.51800000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":8.5790000000000006,"5min":8.5790000000000006,"15min":8.5790000000000006},"last":0.85599999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":2.7480000000000002,"5min":1.1160000000000001,"15min":0.64600000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":6.1100000000000003,"5min":8.2050000000000001,"15min":8.2050000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":2.2559999999999998,"5min":0.93999999999999995,"15min":0.59299999999999997},"min":{"1min":0.20300000000000001,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":8.1470000000000002,"5min":8.1470000000000002,"15min":8.1470000000000002},"last":6.327}]},{"osd":2,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":2.1269999999999998,"5min":0.95399999999999996,"15min":0.61099999999999999},"min":{"1min":0.251,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":8.2129999999999992,"5min":8.2129999999999992,"15min":8.2129999999999992},"last":2.4249999999999998},{"interface":"front","average":{"1min":2.319,"5min":1.026,"15min":0.63500000000000001},"min":{"1min":0.52400000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":10.695,"5min":10.695,"15min":10.695},"last":2.3050000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:43:26 2024","interfaces":[{"interface":"back","average":{"1min":2.0150000000000001,"5min":0.97999999999999998,"15min":0.70699999999999996},"min":{"1min":0.27500000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":8.9969999999999999,"5min":8.9969999999999999,"15min":8.9969999999999999},"last":2.4910000000000001},{"interface":"front","average":{"1min":1.6519999999999999,"5min":0.86699999999999999,"15min":0.66500000000000004},"min":{"1min":0.28000000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.9740000000000002,"5min":3.9740000000000002,"15min":3.9740000000000002},"last":2.3319999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:43:43 2024","interfaces":[{"interface":"back","average":{"1min":2.0350000000000001,"5min":1.0760000000000001,"15min":0.73799999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":11.526999999999999,"5min":11.526999999999999,"15min":11.526999999999999},"last":5.9039999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":0.90500000000000003,"15min":0.66900000000000004},"min":{"1min":0.40600000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.9449999999999998,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":2.3639999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:43:11 2024","interfaces":[{"interface":"back","average":{"1min":1.8779999999999999,"5min":0.90700000000000003,"15min":0.70099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":9.0139999999999993,"5min":9.0139999999999993,"15min":9.0139999999999993},"last":2.669},{"interface":"front","average":{"1min":1.8,"5min":0.97899999999999998,"15min":0.70099999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":5.4349999999999996,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.88300000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:43:32 2024","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":1.0740000000000001,"15min":0.77200000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":11.561,"5min":11.561,"15min":11.561},"last":5.8540000000000001},{"interface":"front","average":{"1min":1.635,"5min":0.93600000000000005,"15min":0.70999999999999996},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.008,"5min":4.008,"15min":4.008},"last":2.3889999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1624980,"kb_used_data":1327628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92120172,"statfs":{"total":95995035648,"available":94331056128,"internally_reserved":0,"allocated":1359491072,"data_stored":1359180875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:43:46 2024","interfaces":[{"interface":"back","average":{"1min":3.2389999999999999,"5min":1.3620000000000001,"15min":0.68899999999999995},"min":{"1min":0.159,"5min":0.159,"15min":0.159},"max":{"1min":24.989000000000001,"5min":24.989000000000001,"15min":24.989000000000001},"last":1.052},{"interface":"front","average":{"1min":3.4950000000000001,"5min":1.597,"15min":0.78300000000000003},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":21.905000000000001,"5min":21.905000000000001,"15min":21.905000000000001},"last":4.258}]},{"osd":1,"last update":"Wed Sep 11 08:43:46 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.0009999999999999,"15min":0.58999999999999997},"min":{"1min":0.215,"5min":0.19,"15min":0.19},"max":{"1min":3.6429999999999998,"5min":6.3460000000000001,"15min":6.3460000000000001},"last":1.542},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.069,"15min":0.60499999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":3.919,"5min":8.1270000000000007,"15min":8.1270000000000007},"last":5.702}]},{"osd":3,"last update":"Wed Sep 11 08:43:02 2024","interfaces":[{"interface":"back","average":{"1min":2.0529999999999999,"5min":1.014,"15min":0.625},"min":{"1min":0.42099999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":6.2590000000000003,"5min":6.718,"15min":6.718},"last":5.5270000000000001},{"interface":"front","average":{"1min":2.5419999999999998,"5min":1.101,"15min":0.66000000000000003},"min":{"1min":0.36799999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":6.9870000000000001,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":4.2779999999999996}]},{"osd":4,"last update":"Wed Sep 11 08:43:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":0.93300000000000005,"15min":0.68000000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":6.181,"5min":6.181,"15min":6.181},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":0.89100000000000001,"15min":0.66200000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":6.6660000000000004,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":4.1790000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:43:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":0.93100000000000005,"15min":0.68600000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.7559999999999998,"5min":6.7039999999999997,"15min":6.7039999999999997},"last":4.2380000000000004},{"interface":"front","average":{"1min":1.099,"5min":0.872,"15min":0.66400000000000003},"min":{"1min":0.30199999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.8889999999999998,"5min":5.9530000000000003,"15min":5.9530000000000003},"last":1.575}]},{"osd":6,"last update":"Wed Sep 11 08:43:13 2024","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":0.95899999999999996,"15min":0.70699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":6.2030000000000003,"5min":6.2030000000000003,"15min":6.2030000000000003},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.472,"5min":0.88600000000000001,"15min":0.66700000000000004},"min":{"1min":0.40100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":6.726,"5min":6.726,"15min":6.726},"last":4.2149999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:43:28 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":0.88,"15min":0.68600000000000005},"min":{"1min":0.34200000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.1820000000000004,"5min":5.9909999999999997,"15min":5.9909999999999997},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.135,"5min":0.998,"15min":0.73599999999999999},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.3090000000000002,"5min":6.157,"15min":6.157},"last":1.2370000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1629768,"kb_used_data":1324096,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92115384,"statfs":{"total":95995035648,"available":94326153216,"internally_reserved":0,"allocated":1355874304,"data_stored":1355512132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:43:27 2024","interfaces":[{"interface":"back","average":{"1min":1.774,"5min":2.1160000000000001,"15min":0.91400000000000003},"min":{"1min":0.22800000000000001,"5min":0.183,"15min":0.183},"max":{"1min":6.7919999999999998,"5min":117.01600000000001,"15min":117.01600000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":3.1920000000000002,"5min":2.5009999999999999,"15min":1.0529999999999999},"min":{"1min":0.308,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":9.4830000000000005,"5min":117.62,"15min":117.62},"last":1.571}]},{"osd":2,"last update":"Wed Sep 11 08:43:43 2024","interfaces":[{"interface":"back","average":{"1min":2.4630000000000001,"5min":1.407,"15min":0.72399999999999998},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":9.5719999999999992,"5min":20.628,"15min":20.628},"last":1.544},{"interface":"front","average":{"1min":1.9570000000000001,"5min":1.3,"15min":0.68000000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":5.1689999999999996,"5min":20.148,"15min":20.148},"last":1.6890000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:43:02 2024","interfaces":[{"interface":"back","average":{"1min":2.258,"5min":1.9299999999999999,"15min":0.92800000000000005},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":7.6849999999999996,"5min":116.98,"15min":116.98},"last":0.437},{"interface":"front","average":{"1min":2.681,"5min":2.2080000000000002,"15min":1.026},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":10.379,"5min":117.56699999999999,"15min":117.56699999999999},"last":1.6359999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:43:27 2024","interfaces":[{"interface":"back","average":{"1min":1.788,"5min":0.89000000000000001,"15min":0.65200000000000002},"min":{"1min":0.30599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":11.19,"5min":11.19,"15min":11.19},"last":1.667},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.76600000000000001,"15min":0.63700000000000001},"min":{"1min":0.28499999999999998,"5min":0.251,"15min":0.251},"max":{"1min":4.8310000000000004,"5min":7.7169999999999996,"15min":7.7169999999999996},"last":0.46400000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:42:49 2024","interfaces":[{"interface":"back","average":{"1min":1.869,"5min":0.88700000000000001,"15min":0.68100000000000005},"min":{"1min":0.371,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":8.2569999999999997,"5min":8.2569999999999997,"15min":8.2569999999999997},"last":1.617},{"interface":"front","average":{"1min":1.7509999999999999,"5min":0.81999999999999995,"15min":0.65200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":11.382999999999999,"5min":11.382999999999999,"15min":11.382999999999999},"last":1.708}]},{"osd":6,"last update":"Wed Sep 11 08:43:13 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":0.79500000000000004,"15min":0.65500000000000003},"min":{"1min":0.33100000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.9149999999999991,"5min":8.9149999999999991,"15min":8.9149999999999991},"last":0.503},{"interface":"front","average":{"1min":2.3420000000000001,"5min":1.018,"15min":0.745},"min":{"1min":0.312,"5min":0.31,"15min":0.31},"max":{"1min":9.5199999999999996,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":1.5960000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:43:27 2024","interfaces":[{"interface":"back","average":{"1min":1.8300000000000001,"5min":0.96199999999999997,"15min":0.70599999999999996},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":11.315,"5min":11.315,"15min":11.315},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.085,"5min":0.84599999999999997,"15min":0.67700000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":4.9100000000000001,"5min":8.8800000000000008,"15min":8.8800000000000008},"last":0.52200000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2062772,"kb_used_data":1761260,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91682380,"statfs":{"total":95995035648,"available":93882757120,"internally_reserved":0,"allocated":1803530240,"data_stored":1803188729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:43:20 2024","interfaces":[{"interface":"back","average":{"1min":2.7599999999999998,"5min":1.131,"15min":0.58199999999999996},"min":{"1min":0.215,"5min":0.186,"15min":0.186},"max":{"1min":7.758,"5min":7.758,"15min":7.758},"last":1.982},{"interface":"front","average":{"1min":1.7430000000000001,"5min":0.88,"15min":0.499},"min":{"1min":0.32000000000000001,"5min":0.186,"15min":0.186},"max":{"1min":4.2320000000000002,"5min":4.5359999999999996,"15min":4.5359999999999996},"last":0.46999999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:43:46 2024","interfaces":[{"interface":"back","average":{"1min":2.0750000000000002,"5min":1.383,"15min":0.69199999999999995},"min":{"1min":0.33900000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":6.5709999999999997,"5min":19.940000000000001,"15min":19.940000000000001},"last":0.63},{"interface":"front","average":{"1min":3.1989999999999998,"5min":1.5309999999999999,"15min":0.74299999999999999},"min":{"1min":0.63300000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":8.3049999999999997,"5min":20.757000000000001,"15min":20.757000000000001},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed Sep 11 08:43:02 2024","interfaces":[{"interface":"back","average":{"1min":2.5070000000000001,"5min":1.1319999999999999,"15min":0.64500000000000002},"min":{"1min":0.58799999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":6.9329999999999998,"5min":6.9329999999999998,"15min":6.9329999999999998},"last":0.92000000000000004},{"interface":"front","average":{"1min":2.6059999999999999,"5min":1.1619999999999999,"15min":0.65000000000000002},"min":{"1min":0.27000000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":7.5099999999999998,"5min":7.5099999999999998,"15min":7.5099999999999998},"last":1.9490000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:43:26 2024","interfaces":[{"interface":"back","average":{"1min":2.532,"5min":1.125,"15min":0.72899999999999998},"min":{"1min":0.34000000000000002,"5min":0.25,"15min":0.25},"max":{"1min":7.4690000000000003,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":2.0379999999999998},{"interface":"front","average":{"1min":2.121,"5min":1.054,"15min":0.70199999999999996},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":6.9539999999999997,"5min":6.9539999999999997,"15min":6.9539999999999997},"last":0.59399999999999997}]},{"osd":5,"last update":"Wed Sep 11 08:43:46 2024","interfaces":[{"interface":"back","average":{"1min":2.0419999999999998,"5min":1.1759999999999999,"15min":0.74199999999999999},"min":{"1min":0.34899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":5.7729999999999997,"5min":6.7690000000000001,"15min":6.7690000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":2.1899999999999999,"5min":1.246,"15min":0.78400000000000003},"min":{"1min":0.40200000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":6.0010000000000003,"5min":7.423,"15min":7.423},"last":2.089}]},{"osd":6,"last update":"Wed Sep 11 08:43:08 2024","interfaces":[{"interface":"back","average":{"1min":1.7010000000000001,"5min":0.97499999999999998,"15min":0.68700000000000006},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.8419999999999996,"5min":6.8419999999999996,"15min":6.8419999999999996},"last":0.69599999999999995},{"interface":"front","average":{"1min":2.2090000000000001,"5min":1.089,"15min":0.751},"min":{"1min":0.32500000000000001,"5min":0.311,"15min":0.311},"max":{"1min":7.4489999999999998,"5min":7.4489999999999998,"15min":7.4489999999999998},"last":2.0649999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:43:33 2024","interfaces":[{"interface":"back","average":{"1min":2.6880000000000002,"5min":1.1890000000000001,"15min":0.78100000000000003},"min":{"1min":0.42599999999999999,"5min":0.372,"15min":0.372},"max":{"1min":7.0049999999999999,"5min":7.0049999999999999,"15min":7.0049999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":2.0960000000000001,"5min":1.1830000000000001,"15min":0.76700000000000002},"min":{"1min":0.36399999999999999,"5min":0.307,"15min":0.307},"max":{"1min":5.8410000000000002,"5min":6.8129999999999997,"15min":6.8129999999999997},"last":0.57499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1775243264,"data_stored":1775239192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1341329408,"data_stored":1341325336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1347878912,"data_stored":1347878912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1997275136,"data_stored":1997275136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1125056512,"data_stored":1125056512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":889389056,"data_stored":889389056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1106382848,"data_stored":1106378776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1105592320,"data_stored":1105592320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:43:59.894 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-11T08:43:59.894 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:43:59.894 INFO:tasks.thrashosds.thrasher:Reweighting osd 6 to 0.4565672136789394 2024-09-11T08:43:59.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 6 0.4565672136789394 2024-09-11T08:43:59.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:00.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:00.015 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:00.015 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:00.116 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:00.138 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:00.141 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:00.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:00.142 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:00.243 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:00.297 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:00.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:00.299 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:00.399 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:00.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:00.435 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:00.435 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:00.536 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:00.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:00 smithi092 ceph-mon[29647]: pgmap v205: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T08:44:00.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:00.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:00 smithi092 ceph-mon[41114]: pgmap v205: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T08:44:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:00.571 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:00.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:00.572 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:00 smithi145 ceph-mon[33384]: pgmap v205: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T08:44:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:00.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-11T08:44:00.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:00.778 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:00.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:00.779 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:00.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:00.919 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:00.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:00.920 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:01.021 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:01.046 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:01.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:01.046 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:01.147 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:01.177 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:01.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:01.178 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:01.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:01.322 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:01.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:01.323 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:01.424 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:01.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:44:01.453 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:01.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:01.454 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:01.515 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:01.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:01.567 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:44:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:44:01] "GET /metrics HTTP/1.1" 200 223840 "" "Prometheus/2.43.0" 2024-09-11T08:44:01.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:01.618 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:01.619 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:01.619 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:01.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:01.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:01.776 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:01.877 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:01.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:01.916 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:01.916 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:02.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:02.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:02.060 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:02.060 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:02.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:02.198 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:02.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:02.199 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:02.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:02.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:02.349 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:02.349 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:02.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:02.474 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:02 smithi092 ceph-mon[29647]: pgmap v206: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 928 op/s 2024-09-11T08:44:02.474 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:02 smithi092 ceph-mon[41114]: pgmap v206: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 928 op/s 2024-09-11T08:44:02.496 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:02.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:02.497 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:02.598 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:02.627 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:02.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:02.628 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:02 smithi145 ceph-mon[33384]: pgmap v206: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 928 op/s 2024-09-11T08:44:02.729 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:02.756 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:02.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:02.757 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:02.858 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:02.884 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:02.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:02.885 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:02.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:03.027 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:03.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:03.028 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:03.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:03.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:44:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:44:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:03.164 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:03.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:03.165 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:03.266 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:03.305 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:03.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:03.306 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:03.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:03.436 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:03.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:03.437 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:03.538 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:03.575 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:03.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:03.576 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:03.677 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:03.703 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:03.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:03.704 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:03.804 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:03.830 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:03.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:03.831 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:03.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:03.967 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:03.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:03.968 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:04.069 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:04.092 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:04.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:04.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:04.194 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:04.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:04.216 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:04.217 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:04.318 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:04.345 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:04.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:04.346 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:04.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:04.481 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:04.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:04.485 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:04.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:04 smithi092 ceph-mon[29647]: pgmap v207: 148 pgs: 148 active+clean; 2.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T08:44:04.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:04 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:44:04.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:04 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:44:04.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:04 smithi092 ceph-mon[29647]: osdmap e130: 8 total, 8 up, 8 in 2024-09-11T08:44:04.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:04 smithi092 ceph-mon[41114]: pgmap v207: 148 pgs: 148 active+clean; 2.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T08:44:04.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:04 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:44:04.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:04 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:44:04.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:04 smithi092 ceph-mon[41114]: osdmap e130: 8 total, 8 up, 8 in 2024-09-11T08:44:04.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:04.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:04.628 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:04.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:04 smithi145 ceph-mon[33384]: pgmap v207: 148 pgs: 148 active+clean; 2.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T08:44:04.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:04 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:44:04.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:04 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:44:04.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:04 smithi145 ceph-mon[33384]: osdmap e130: 8 total, 8 up, 8 in 2024-09-11T08:44:04.729 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:04.754 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:04.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:04.755 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:04.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:04.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:04.894 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:04.895 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:04.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:05.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:05.020 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:05.020 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:05.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:05.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:05.164 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:05.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:05.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:05.303 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:05.303 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:05.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:05.440 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:05.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:05.441 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:05.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:05.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:05.590 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:05.691 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:05.715 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:05.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:05.716 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:05.816 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:05.842 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:05.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:05.843 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:05.944 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:05.965 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:05.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:05.966 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:06.067 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:06.097 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:06.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:06.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:06.199 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:06.226 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:06.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:06.227 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:06.304 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:06.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:06.385 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:06.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:06.387 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:06.487 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:06.506 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:06 smithi092 ceph-mon[29647]: pgmap v209: 148 pgs: 148 active+clean; 2.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 663 op/s 2024-09-11T08:44:06.507 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:06 smithi092 ceph-mon[41114]: pgmap v209: 148 pgs: 148 active+clean; 2.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 663 op/s 2024-09-11T08:44:06.512 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:06.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:06.513 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:06.613 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:06.639 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:06.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:06.640 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:06.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:06 smithi145 ceph-mon[33384]: pgmap v209: 148 pgs: 148 active+clean; 2.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 663 op/s 2024-09-11T08:44:06.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:44:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:44:06.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:06.778 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:06.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:06.779 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:06.880 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:06.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:06.907 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:06.907 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:07.008 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:07.047 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:07.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:07.048 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:07.149 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:07.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:07.178 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:07.178 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:07.279 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:07.304 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:07.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:07.305 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:07.406 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:07.449 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:07.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:07.450 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:07.551 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:07.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:07 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:07.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:07 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:07.576 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:07.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:07.577 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:07.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:07 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:07.678 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:07.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:07.705 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:07.706 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:07.806 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:07.847 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:07.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:07.849 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:07.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:07.987 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:07.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:07.988 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:08.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:08.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:08.125 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:08.125 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:08.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:08.263 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:08.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:08.264 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:08.365 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:08.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:08.407 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:08.407 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:08.462 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:08 smithi092 ceph-mon[29647]: pgmap v210: 148 pgs: 148 active+clean; 2.3 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.07k op/s 2024-09-11T08:44:08.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:08 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:44:08.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:08 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:44:08.463 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:08 smithi092 ceph-mon[41114]: pgmap v210: 148 pgs: 148 active+clean; 2.3 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.07k op/s 2024-09-11T08:44:08.463 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:08 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:44:08.463 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:08 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:44:08.507 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:08.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:08.532 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:08.532 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:08.633 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:08.657 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:08.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:08.658 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:08.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:08 smithi145 ceph-mon[33384]: pgmap v210: 148 pgs: 148 active+clean; 2.3 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.07k op/s 2024-09-11T08:44:08.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:08 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:44:08.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:08 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:44:08.759 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:08.787 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:08.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:08.788 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:08.889 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:08.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:08.933 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:08.933 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:09.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:09.078 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:09.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:09.079 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:09.180 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:09.204 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:09.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:09.205 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:09.301 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:44:09.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:09.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:09.365 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:09.365 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:09.466 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:09.495 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:09.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:09.496 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:09.548 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:09 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:44:09.549 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:09 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:44:09.549 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:09 smithi145 ceph-mon[33384]: osdmap e131: 8 total, 8 up, 8 in 2024-09-11T08:44:09.549 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:09 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2998198030' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:44:09.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:09 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:44:09.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:09 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:44:09.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:09 smithi092 ceph-mon[29647]: osdmap e131: 8 total, 8 up, 8 in 2024-09-11T08:44:09.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:09 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2998198030' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:44:09.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:09 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:44:09.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:09 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:44:09.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:09 smithi092 ceph-mon[41114]: osdmap e131: 8 total, 8 up, 8 in 2024-09-11T08:44:09.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:09 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2998198030' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:44:09.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:09.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:09.625 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:09.625 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:09.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:09.764 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:09.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:09.765 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:09.866 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:09.905 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:09.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:09.906 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:10.007 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:10.033 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:10.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:10.034 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:10.135 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:10.160 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:10.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:10.162 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:10.262 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:10.304 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.6 to 0.456567 (74e1) 2024-09-11T08:44:10.305 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:10.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:10.306 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:10.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:10.428 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:10.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:10.432 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:10.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:10.554 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:10.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:10.555 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:10.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:10 smithi092 ceph-mon[29647]: pgmap v212: 148 pgs: 148 active+clean; 2.3 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T08:44:10.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:10 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:44:10.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:10 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2593258870' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 0.4565672136789394}]: dispatch 2024-09-11T08:44:10.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:10 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2998198030' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:44:10.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:10 smithi092 ceph-mon[29647]: osdmap e132: 8 total, 8 up, 8 in 2024-09-11T08:44:10.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:10 smithi092 ceph-mon[29647]: 6.15 scrub starts 2024-09-11T08:44:10.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:10 smithi092 ceph-mon[29647]: 6.15 scrub ok 2024-09-11T08:44:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:10 smithi092 ceph-mon[41114]: pgmap v212: 148 pgs: 148 active+clean; 2.3 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T08:44:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:10 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:44:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:10 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2593258870' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 0.4565672136789394}]: dispatch 2024-09-11T08:44:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:10 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2998198030' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:44:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:10 smithi092 ceph-mon[41114]: osdmap e132: 8 total, 8 up, 8 in 2024-09-11T08:44:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:10 smithi092 ceph-mon[41114]: 6.15 scrub starts 2024-09-11T08:44:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:10 smithi092 ceph-mon[41114]: 6.15 scrub ok 2024-09-11T08:44:10.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:10.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:10 smithi145 ceph-mon[33384]: pgmap v212: 148 pgs: 148 active+clean; 2.3 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T08:44:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:10 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:44:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:10 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2593258870' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 0.4565672136789394}]: dispatch 2024-09-11T08:44:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:10 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2998198030' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:44:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:10 smithi145 ceph-mon[33384]: osdmap e132: 8 total, 8 up, 8 in 2024-09-11T08:44:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:10 smithi145 ceph-mon[33384]: 6.15 scrub starts 2024-09-11T08:44:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:10 smithi145 ceph-mon[33384]: 6.15 scrub ok 2024-09-11T08:44:10.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:10.679 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:10.679 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:10.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:10.794 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:10.795 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:10.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:10.804 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:10.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:10.926 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:10.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:10.927 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:11.028 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:11.050 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:11.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:11.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:11.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:11.174 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:11.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:11.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:11.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:11.297 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:11.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:11.298 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:11.398 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:11.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[29647]: 5.12 scrub starts 2024-09-11T08:44:11.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[29647]: 5.12 scrub ok 2024-09-11T08:44:11.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[29647]: 3.12 scrub starts 2024-09-11T08:44:11.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[29647]: 3.12 scrub ok 2024-09-11T08:44:11.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[29647]: 3.1c scrub starts 2024-09-11T08:44:11.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[29647]: 3.1c scrub ok 2024-09-11T08:44:11.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2593258870' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 0.4565672136789394}]': finished 2024-09-11T08:44:11.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[29647]: osdmap e133: 8 total, 8 up, 8 in 2024-09-11T08:44:11.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[41114]: 5.12 scrub starts 2024-09-11T08:44:11.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[41114]: 5.12 scrub ok 2024-09-11T08:44:11.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[41114]: 3.12 scrub starts 2024-09-11T08:44:11.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[41114]: 3.12 scrub ok 2024-09-11T08:44:11.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[41114]: 3.1c scrub starts 2024-09-11T08:44:11.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[41114]: 3.1c scrub ok 2024-09-11T08:44:11.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2593258870' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 0.4565672136789394}]': finished 2024-09-11T08:44:11.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-mon[41114]: osdmap e133: 8 total, 8 up, 8 in 2024-09-11T08:44:11.420 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:11.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:11.421 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:11.508 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:44:11.508 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:11.511 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:44:11.511 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:11.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:11.526 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:44:11.526 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:11.543 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:44:11.544 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:11.545 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:11.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:11.546 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:11.563 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:44:11.563 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:11.578 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:44:11.578 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:11.596 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:44:11.597 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:11.609 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:44:11.609 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:11.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:11.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:11 smithi145 ceph-mon[33384]: 5.12 scrub starts 2024-09-11T08:44:11.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:11 smithi145 ceph-mon[33384]: 5.12 scrub ok 2024-09-11T08:44:11.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:11 smithi145 ceph-mon[33384]: 3.12 scrub starts 2024-09-11T08:44:11.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:11 smithi145 ceph-mon[33384]: 3.12 scrub ok 2024-09-11T08:44:11.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:11 smithi145 ceph-mon[33384]: 3.1c scrub starts 2024-09-11T08:44:11.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:11 smithi145 ceph-mon[33384]: 3.1c scrub ok 2024-09-11T08:44:11.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:11 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2593258870' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 0.4565672136789394}]': finished 2024-09-11T08:44:11.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:11 smithi145 ceph-mon[33384]: osdmap e133: 8 total, 8 up, 8 in 2024-09-11T08:44:11.669 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:11.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:11.670 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:11.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:11.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:11.793 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:11.794 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:11.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:44:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:44:11] "GET /metrics HTTP/1.1" 200 223840 "" "Prometheus/2.43.0" 2024-09-11T08:44:11.895 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:11.918 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:11.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:11.919 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:12.020 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:12.043 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:12.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:12.044 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:12.145 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:12.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:12.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:12.267 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:12.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:44:12.291 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:12.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:12.292 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:12.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:12.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:12.426 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:12.426 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:12.527 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:12.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[29647]: 4.1e scrub starts 2024-09-11T08:44:12.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[29647]: 4.1e scrub ok 2024-09-11T08:44:12.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[29647]: 3.10 scrub starts 2024-09-11T08:44:12.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[29647]: 3.10 scrub ok 2024-09-11T08:44:12.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[29647]: pgmap v215: 148 pgs: 16 peering, 132 active+clean; 2.3 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T08:44:12.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[29647]: 5.0 scrub starts 2024-09-11T08:44:12.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[29647]: 5.0 scrub ok 2024-09-11T08:44:12.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[29647]: Health check failed: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-11T08:44:12.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[29647]: osdmap e134: 8 total, 8 up, 8 in 2024-09-11T08:44:12.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2156833752' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:44:12.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[41114]: 4.1e scrub starts 2024-09-11T08:44:12.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[41114]: 4.1e scrub ok 2024-09-11T08:44:12.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[41114]: 3.10 scrub starts 2024-09-11T08:44:12.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[41114]: 3.10 scrub ok 2024-09-11T08:44:12.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[41114]: pgmap v215: 148 pgs: 16 peering, 132 active+clean; 2.3 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T08:44:12.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[41114]: 5.0 scrub starts 2024-09-11T08:44:12.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[41114]: 5.0 scrub ok 2024-09-11T08:44:12.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[41114]: Health check failed: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-11T08:44:12.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[41114]: osdmap e134: 8 total, 8 up, 8 in 2024-09-11T08:44:12.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:12 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2156833752' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:44:12.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:12.574 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:12.574 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:12.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:12 smithi145 ceph-mon[33384]: 4.1e scrub starts 2024-09-11T08:44:12.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:12 smithi145 ceph-mon[33384]: 4.1e scrub ok 2024-09-11T08:44:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:12 smithi145 ceph-mon[33384]: 3.10 scrub starts 2024-09-11T08:44:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:12 smithi145 ceph-mon[33384]: 3.10 scrub ok 2024-09-11T08:44:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:12 smithi145 ceph-mon[33384]: pgmap v215: 148 pgs: 16 peering, 132 active+clean; 2.3 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T08:44:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:12 smithi145 ceph-mon[33384]: 5.0 scrub starts 2024-09-11T08:44:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:12 smithi145 ceph-mon[33384]: 5.0 scrub ok 2024-09-11T08:44:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:12 smithi145 ceph-mon[33384]: Health check failed: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-11T08:44:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:12 smithi145 ceph-mon[33384]: osdmap e134: 8 total, 8 up, 8 in 2024-09-11T08:44:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:12 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2156833752' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:44:12.675 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:12.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:12.700 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:12.700 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:12.801 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:12.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:12.828 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:12.828 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:12.929 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:12.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:12.973 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:12.973 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:12.985 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-11T08:44:13.074 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:13.097 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:13.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:13.098 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:13.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:44:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:44:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:13.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:13.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:13.282 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:13.383 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:13.408 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:13.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:13.409 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:13.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:13.546 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:13.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:13.547 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:13.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:13.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:13 smithi145 ceph-mon[33384]: 4.16 scrub starts 2024-09-11T08:44:13.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:13 smithi145 ceph-mon[33384]: 4.16 scrub ok 2024-09-11T08:44:13.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:13 smithi145 ceph-mon[33384]: 4.11 scrub starts 2024-09-11T08:44:13.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:13 smithi145 ceph-mon[33384]: 4.11 scrub ok 2024-09-11T08:44:13.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:13 smithi145 ceph-mon[33384]: osdmap e135: 8 total, 8 up, 8 in 2024-09-11T08:44:13.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:13 smithi145 ceph-mon[33384]: 7.1 starting backfill to osd.1 from (0'0,0'0] MAX to 132'13574 2024-09-11T08:44:13.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:13 smithi145 ceph-mon[33384]: 7.d starting backfill to osd.3 from (0'0,0'0] MAX to 132'13552 2024-09-11T08:44:13.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:13 smithi145 ceph-mon[33384]: 7.7 starting backfill to osd.1 from (0'0,0'0] MAX to 132'13502 2024-09-11T08:44:13.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:13 smithi145 ceph-mon[33384]: 1.0 scrub starts 2024-09-11T08:44:13.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:13 smithi145 ceph-mon[33384]: 1.0 scrub ok 2024-09-11T08:44:13.673 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:13.673 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[41114]: 4.16 scrub starts 2024-09-11T08:44:13.673 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[41114]: 4.16 scrub ok 2024-09-11T08:44:13.674 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[41114]: 4.11 scrub starts 2024-09-11T08:44:13.674 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[41114]: 4.11 scrub ok 2024-09-11T08:44:13.674 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[41114]: osdmap e135: 8 total, 8 up, 8 in 2024-09-11T08:44:13.674 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[41114]: 7.1 starting backfill to osd.1 from (0'0,0'0] MAX to 132'13574 2024-09-11T08:44:13.674 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[41114]: 7.d starting backfill to osd.3 from (0'0,0'0] MAX to 132'13552 2024-09-11T08:44:13.674 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[41114]: 7.7 starting backfill to osd.1 from (0'0,0'0] MAX to 132'13502 2024-09-11T08:44:13.674 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[41114]: 1.0 scrub starts 2024-09-11T08:44:13.674 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[41114]: 1.0 scrub ok 2024-09-11T08:44:13.674 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[29647]: 4.16 scrub starts 2024-09-11T08:44:13.674 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[29647]: 4.16 scrub ok 2024-09-11T08:44:13.674 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[29647]: 4.11 scrub starts 2024-09-11T08:44:13.674 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[29647]: 4.11 scrub ok 2024-09-11T08:44:13.675 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[29647]: osdmap e135: 8 total, 8 up, 8 in 2024-09-11T08:44:13.675 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[29647]: 7.1 starting backfill to osd.1 from (0'0,0'0] MAX to 132'13574 2024-09-11T08:44:13.675 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[29647]: 7.d starting backfill to osd.3 from (0'0,0'0] MAX to 132'13552 2024-09-11T08:44:13.675 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[29647]: 7.7 starting backfill to osd.1 from (0'0,0'0] MAX to 132'13502 2024-09-11T08:44:13.675 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[29647]: 1.0 scrub starts 2024-09-11T08:44:13.675 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:13 smithi092 ceph-mon[29647]: 1.0 scrub ok 2024-09-11T08:44:13.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:13.676 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:13.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:13.824 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:13.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:13.825 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:13.926 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:13.952 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:13.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:13.953 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:14.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:14.098 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:14.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:14.099 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:14.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:14.244 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:14.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:14.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:14.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:14.368 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:14.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:14.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:14.469 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:14.493 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:14.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:14.494 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:14.595 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:14.617 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:14.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:14.618 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:14.620 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:14 smithi092 ceph-mon[41114]: 5.1d scrub starts 2024-09-11T08:44:14.620 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:14 smithi092 ceph-mon[41114]: 5.1d scrub ok 2024-09-11T08:44:14.620 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:14 smithi092 ceph-mon[41114]: 3.1d scrub starts 2024-09-11T08:44:14.620 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:14 smithi092 ceph-mon[41114]: 3.1d scrub ok 2024-09-11T08:44:14.620 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:14 smithi092 ceph-mon[41114]: pgmap v218: 148 pgs: 1 remapped+peering, 20 peering, 127 active+clean; 2.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 1.12k op/s; 3 B/s, 0 objects/s recovering 2024-09-11T08:44:14.620 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:14 smithi092 ceph-mon[41114]: 7.2 scrub starts 2024-09-11T08:44:14.622 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:14 smithi092 ceph-mon[29647]: 5.1d scrub starts 2024-09-11T08:44:14.622 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:14 smithi092 ceph-mon[29647]: 5.1d scrub ok 2024-09-11T08:44:14.622 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:14 smithi092 ceph-mon[29647]: 3.1d scrub starts 2024-09-11T08:44:14.622 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:14 smithi092 ceph-mon[29647]: 3.1d scrub ok 2024-09-11T08:44:14.623 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:14 smithi092 ceph-mon[29647]: pgmap v218: 148 pgs: 1 remapped+peering, 20 peering, 127 active+clean; 2.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 1.12k op/s; 3 B/s, 0 objects/s recovering 2024-09-11T08:44:14.623 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:14 smithi092 ceph-mon[29647]: 7.2 scrub starts 2024-09-11T08:44:14.661 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:14.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:14 smithi145 ceph-mon[33384]: 5.1d scrub starts 2024-09-11T08:44:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:14 smithi145 ceph-mon[33384]: 5.1d scrub ok 2024-09-11T08:44:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:14 smithi145 ceph-mon[33384]: 3.1d scrub starts 2024-09-11T08:44:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:14 smithi145 ceph-mon[33384]: 3.1d scrub ok 2024-09-11T08:44:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:14 smithi145 ceph-mon[33384]: pgmap v218: 148 pgs: 1 remapped+peering, 20 peering, 127 active+clean; 2.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 1.12k op/s; 3 B/s, 0 objects/s recovering 2024-09-11T08:44:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:14 smithi145 ceph-mon[33384]: 7.2 scrub starts 2024-09-11T08:44:14.668 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:14.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:14.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:14.774 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:14.874 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:14.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:14.924 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:14.924 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:15.025 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:15.049 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:15.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:15.050 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:15.151 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:15.176 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:15.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:15.177 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:15.280 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:15.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:15.325 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:15.426 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:15.453 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:15.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:15.454 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:15.469 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:15 smithi092 ceph-mon[29647]: 7.3 scrub starts 2024-09-11T08:44:15.469 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:15 smithi092 ceph-mon[29647]: 7.e scrub starts 2024-09-11T08:44:15.469 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:15 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:15.469 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:15.470 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:15 smithi092 ceph-mon[41114]: 7.3 scrub starts 2024-09-11T08:44:15.470 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:15 smithi092 ceph-mon[41114]: 7.e scrub starts 2024-09-11T08:44:15.470 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:15 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:15.470 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:15.554 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:15.579 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:15.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:15.580 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:15.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:15 smithi145 ceph-mon[33384]: 7.3 scrub starts 2024-09-11T08:44:15.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:15 smithi145 ceph-mon[33384]: 7.e scrub starts 2024-09-11T08:44:15.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:15 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:15.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:15.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:15.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:44:15.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:15.724 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:15.724 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:15.825 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:15.849 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:15.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:15.850 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:15.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:16.000 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:16.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:16.001 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:16.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:16.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:16.149 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:16.250 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:16.276 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:16.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:16.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:16.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:16.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:16.428 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:16.428 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:16.495 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[29647]: 7.2 scrub ok 2024-09-11T08:44:16.495 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[29647]: 5.5 scrub starts 2024-09-11T08:44:16.495 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[29647]: 5.5 scrub ok 2024-09-11T08:44:16.495 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[29647]: 7.c scrub starts 2024-09-11T08:44:16.495 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[29647]: pgmap v219: 148 pgs: 1 remapped+peering, 20 peering, 127 active+clean; 2.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 351 B/s rd, 0 B/s wr, 771 op/s; 2 B/s, 0 objects/s recovering 2024-09-11T08:44:16.495 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[29647]: 7.3 scrub ok 2024-09-11T08:44:16.495 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[29647]: 2.0 scrub starts 2024-09-11T08:44:16.496 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[29647]: 2.0 scrub ok 2024-09-11T08:44:16.496 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[29647]: 7.8 scrub starts 2024-09-11T08:44:16.496 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[29647]: 7.e scrub ok 2024-09-11T08:44:16.496 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[41114]: 7.2 scrub ok 2024-09-11T08:44:16.496 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[41114]: 5.5 scrub starts 2024-09-11T08:44:16.496 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[41114]: 5.5 scrub ok 2024-09-11T08:44:16.496 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[41114]: 7.c scrub starts 2024-09-11T08:44:16.496 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[41114]: pgmap v219: 148 pgs: 1 remapped+peering, 20 peering, 127 active+clean; 2.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 351 B/s rd, 0 B/s wr, 771 op/s; 2 B/s, 0 objects/s recovering 2024-09-11T08:44:16.496 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[41114]: 7.3 scrub ok 2024-09-11T08:44:16.496 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[41114]: 2.0 scrub starts 2024-09-11T08:44:16.496 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[41114]: 2.0 scrub ok 2024-09-11T08:44:16.497 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[41114]: 7.8 scrub starts 2024-09-11T08:44:16.497 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:16 smithi092 ceph-mon[41114]: 7.e scrub ok 2024-09-11T08:44:16.529 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:16.555 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:16.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:16.556 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:16.657 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:16.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:16 smithi145 ceph-mon[33384]: 7.2 scrub ok 2024-09-11T08:44:16.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:16 smithi145 ceph-mon[33384]: 5.5 scrub starts 2024-09-11T08:44:16.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:16 smithi145 ceph-mon[33384]: 5.5 scrub ok 2024-09-11T08:44:16.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:16 smithi145 ceph-mon[33384]: 7.c scrub starts 2024-09-11T08:44:16.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:16 smithi145 ceph-mon[33384]: pgmap v219: 148 pgs: 1 remapped+peering, 20 peering, 127 active+clean; 2.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 351 B/s rd, 0 B/s wr, 771 op/s; 2 B/s, 0 objects/s recovering 2024-09-11T08:44:16.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:16 smithi145 ceph-mon[33384]: 7.3 scrub ok 2024-09-11T08:44:16.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:16 smithi145 ceph-mon[33384]: 2.0 scrub starts 2024-09-11T08:44:16.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:16 smithi145 ceph-mon[33384]: 2.0 scrub ok 2024-09-11T08:44:16.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:16 smithi145 ceph-mon[33384]: 7.8 scrub starts 2024-09-11T08:44:16.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:16 smithi145 ceph-mon[33384]: 7.e scrub ok 2024-09-11T08:44:16.664 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:44:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:44:16.683 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:16.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:16.684 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:16.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:16.822 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:16.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:16.823 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:16.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:16.964 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:16.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:16.965 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:17.066 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:17.092 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:17.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:17.093 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:17.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:17.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:17.233 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:17.233 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:17.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:17.368 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:17.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:17.369 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:17.470 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:17.495 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:17.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:17.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:17.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[29647]: 6.1d scrub starts 2024-09-11T08:44:17.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[29647]: 6.1d scrub ok 2024-09-11T08:44:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[29647]: 7.c scrub ok 2024-09-11T08:44:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[29647]: 4.6 scrub starts 2024-09-11T08:44:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[29647]: 4.6 scrub ok 2024-09-11T08:44:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[29647]: 6.a scrub starts 2024-09-11T08:44:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[29647]: 6.a scrub ok 2024-09-11T08:44:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[29647]: 7.8 scrub ok 2024-09-11T08:44:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[29647]: 6.12 scrub starts 2024-09-11T08:44:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[29647]: 6.12 scrub ok 2024-09-11T08:44:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[29647]: 2.1 scrub starts 2024-09-11T08:44:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[29647]: 2.1 scrub ok 2024-09-11T08:44:17.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[41114]: 6.1d scrub starts 2024-09-11T08:44:17.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[41114]: 6.1d scrub ok 2024-09-11T08:44:17.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[41114]: 7.c scrub ok 2024-09-11T08:44:17.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[41114]: 4.6 scrub starts 2024-09-11T08:44:17.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[41114]: 4.6 scrub ok 2024-09-11T08:44:17.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[41114]: 6.a scrub starts 2024-09-11T08:44:17.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[41114]: 6.a scrub ok 2024-09-11T08:44:17.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[41114]: 7.8 scrub ok 2024-09-11T08:44:17.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:17.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[41114]: 6.12 scrub starts 2024-09-11T08:44:17.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[41114]: 6.12 scrub ok 2024-09-11T08:44:17.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[41114]: 2.1 scrub starts 2024-09-11T08:44:17.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:17 smithi092 ceph-mon[41114]: 2.1 scrub ok 2024-09-11T08:44:17.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:17.625 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:17.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:17.626 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:17.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:17 smithi145 ceph-mon[33384]: 6.1d scrub starts 2024-09-11T08:44:17.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:17 smithi145 ceph-mon[33384]: 6.1d scrub ok 2024-09-11T08:44:17.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:17 smithi145 ceph-mon[33384]: 7.c scrub ok 2024-09-11T08:44:17.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:17 smithi145 ceph-mon[33384]: 4.6 scrub starts 2024-09-11T08:44:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:17 smithi145 ceph-mon[33384]: 4.6 scrub ok 2024-09-11T08:44:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:17 smithi145 ceph-mon[33384]: 6.a scrub starts 2024-09-11T08:44:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:17 smithi145 ceph-mon[33384]: 6.a scrub ok 2024-09-11T08:44:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:17 smithi145 ceph-mon[33384]: 7.8 scrub ok 2024-09-11T08:44:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:17 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:17 smithi145 ceph-mon[33384]: 6.12 scrub starts 2024-09-11T08:44:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:17 smithi145 ceph-mon[33384]: 6.12 scrub ok 2024-09-11T08:44:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:17 smithi145 ceph-mon[33384]: 2.1 scrub starts 2024-09-11T08:44:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:17 smithi145 ceph-mon[33384]: 2.1 scrub ok 2024-09-11T08:44:17.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:17.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:17.780 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:17.780 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:17.881 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:17.906 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:17.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:17.907 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:18.007 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:18.031 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:18.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:18.031 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:18.132 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:18.156 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:18.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:18.157 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:18.258 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:18.281 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:18.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:18.282 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:18.383 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:18.407 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:18.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:18.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:18.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:18 smithi092 ceph-mon[29647]: 6.0 scrub starts 2024-09-11T08:44:18.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:18 smithi092 ceph-mon[29647]: 6.0 scrub ok 2024-09-11T08:44:18.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:18 smithi092 ceph-mon[29647]: 5.7 scrub starts 2024-09-11T08:44:18.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:18 smithi092 ceph-mon[29647]: 5.7 scrub ok 2024-09-11T08:44:18.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:18 smithi092 ceph-mon[29647]: pgmap v220: 148 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 144 active+clean; 1.8 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.19k op/s; 5684/91269 objects misplaced (6.228%); 124 B/s, 8 objects/s recovering 2024-09-11T08:44:18.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:18 smithi092 ceph-mon[29647]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:44:18.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:18 smithi092 ceph-mon[29647]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 4 pgs peering) 2024-09-11T08:44:18.455 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:18 smithi092 ceph-mon[41114]: 6.0 scrub starts 2024-09-11T08:44:18.455 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:18 smithi092 ceph-mon[41114]: 6.0 scrub ok 2024-09-11T08:44:18.455 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:18 smithi092 ceph-mon[41114]: 5.7 scrub starts 2024-09-11T08:44:18.455 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:18 smithi092 ceph-mon[41114]: 5.7 scrub ok 2024-09-11T08:44:18.455 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:18 smithi092 ceph-mon[41114]: pgmap v220: 148 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 144 active+clean; 1.8 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.19k op/s; 5684/91269 objects misplaced (6.228%); 124 B/s, 8 objects/s recovering 2024-09-11T08:44:18.455 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:18 smithi092 ceph-mon[41114]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:44:18.455 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:18 smithi092 ceph-mon[41114]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 4 pgs peering) 2024-09-11T08:44:18.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:18.551 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:44:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:44:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:223270 7.b 7:dc7112cd:::benchmark_data_smithi092_84565_object97742:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-11T08:44:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:18.459072+0000", 2024-09-11T08:44:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0090093399999999994, 2024-09-11T08:44:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090405550000000005, 2024-09-11T08:44:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:44:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:44:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:44:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:44:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 223270 2024-09-11T08:44:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.459072+0000", 2024-09-11T08:44:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.459072+0000", 2024-09-11T08:44:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.459071+0000", 2024-09-11T08:44:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:44:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.459079+0000", 2024-09-11T08:44:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7940000000000003e-06 2024-09-11T08:44:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.459093+0000", 2024-09-11T08:44:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4506e-05 2024-09-11T08:44:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.459100+0000", 2024-09-11T08:44:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7290000000000003e-06 2024-09-11T08:44:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.459146+0000", 2024-09-11T08:44:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6218999999999999e-05 2024-09-11T08:44:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460469+0000", 2024-09-11T08:44:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001323297 2024-09-11T08:44:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:44:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460787+0000", 2024-09-11T08:44:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031739300000000002 2024-09-11T08:44:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.464513+0000", 2024-09-11T08:44:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037265430000000001 2024-09-11T08:44:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:44:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.464542+0000", 2024-09-11T08:44:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8249e-05 2024-09-11T08:44:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:223272 7.d 7:b15c9811:::benchmark_data_smithi092_84565_object97744:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-11T08:44:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:18.459638+0000", 2024-09-11T08:44:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0084437060000000005, 2024-09-11T08:44:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085841719999999993, 2024-09-11T08:44:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:44:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:44:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:44:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:44:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 223272 2024-09-11T08:44:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.459638+0000", 2024-09-11T08:44:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.459638+0000", 2024-09-11T08:44:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.459637+0000", 2024-09-11T08:44:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:44:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.459642+0000", 2024-09-11T08:44:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0880000000000002e-06 2024-09-11T08:44:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.459685+0000", 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3353999999999999e-05 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.459706+0000", 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0313999999999999e-05 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.459863+0000", 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015679800000000001 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.459983+0000", 2024-09-11T08:44:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012089 2024-09-11T08:44:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,3,6", 2024-09-11T08:44:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460104+0000", 2024-09-11T08:44:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012081700000000001 2024-09-11T08:44:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:223273 7.d 7:b7c20175:::benchmark_data_smithi092_84565_object97745:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-11T08:44:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:18.460391+0000", 2024-09-11T08:44:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0076908269999999999, 2024-09-11T08:44:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092607139999999998, 2024-09-11T08:44:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:44:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:44:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:44:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:44:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 223273 2024-09-11T08:44:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460391+0000", 2024-09-11T08:44:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460391+0000", 2024-09-11T08:44:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460390+0000", 2024-09-11T08:44:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:44:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460395+0000", 2024-09-11T08:44:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2329999999999997e-06 2024-09-11T08:44:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460404+0000", 2024-09-11T08:44:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6680000000000007e-06 2024-09-11T08:44:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460409+0000", 2024-09-11T08:44:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1880000000000001e-06 2024-09-11T08:44:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.462473+0000", 2024-09-11T08:44:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020635010000000001 2024-09-11T08:44:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.462585+0000", 2024-09-11T08:44:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011192 2024-09-11T08:44:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,3,6", 2024-09-11T08:44:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.462710+0000", 2024-09-11T08:44:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012513699999999999 2024-09-11T08:44:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:223278 7.b 7:d35f834d:::benchmark_data_smithi092_84565_object97750:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-11T08:44:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:18.460414+0000", 2024-09-11T08:44:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0076671660000000004, 2024-09-11T08:44:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093171179999999992, 2024-09-11T08:44:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:44:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:44:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:44:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:44:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 223278 2024-09-11T08:44:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460414+0000", 2024-09-11T08:44:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460414+0000", 2024-09-11T08:44:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460414+0000", 2024-09-11T08:44:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:44:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460416+0000", 2024-09-11T08:44:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0640000000000001e-06 2024-09-11T08:44:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460421+0000", 2024-09-11T08:44:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9089999999999999e-06 2024-09-11T08:44:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460424+0000", 2024-09-11T08:44:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1920000000000001e-06 2024-09-11T08:44:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.464618+0000", 2024-09-11T08:44:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041942050000000003 2024-09-11T08:44:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.464888+0000", 2024-09-11T08:44:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026943300000000002 2024-09-11T08:44:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:44:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.465068+0000", 2024-09-11T08:44:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018000000000000001 2024-09-11T08:44:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.467809+0000", 2024-09-11T08:44:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027411060000000001 2024-09-11T08:44:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:44:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.467816+0000", 2024-09-11T08:44:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4540000000000001e-06 2024-09-11T08:44:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:223281 7.d 7:be8eeae0:::benchmark_data_smithi092_84565_object97753:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-11T08:44:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:18.460428+0000", 2024-09-11T08:44:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0076536119999999997, 2024-09-11T08:44:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093836590000000008, 2024-09-11T08:44:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:44:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:44:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:44:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:44:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 223281 2024-09-11T08:44:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460428+0000", 2024-09-11T08:44:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460428+0000", 2024-09-11T08:44:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460428+0000", 2024-09-11T08:44:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:44:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460429+0000", 2024-09-11T08:44:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7439999999999999e-06 2024-09-11T08:44:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460434+0000", 2024-09-11T08:44:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5199999999999999e-06 2024-09-11T08:44:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.460437+0000", 2024-09-11T08:44:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.92e-06 2024-09-11T08:44:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.465607+0000", 2024-09-11T08:44:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051701969999999996 2024-09-11T08:44:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.465887+0000", 2024-09-11T08:44:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028003799999999998 2024-09-11T08:44:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,3,6", 2024-09-11T08:44:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.466236+0000", 2024-09-11T08:44:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034918999999999998 2024-09-11T08:44:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.14814.0:223272 7.d e135/134 ondisk, result = 0)", 2024-09-11T08:44:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:18.468061+0000", 2024-09-11T08:44:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.0557e-05, 2024-09-11T08:44:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001822665, 2024-09-11T08:44:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:44:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.468061+0000", 2024-09-11T08:44:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.468061+0000", 2024-09-11T08:44:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.468060+0000", 2024-09-11T08:44:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:44:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.468064+0000", 2024-09-11T08:44:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9400000000000004e-06 2024-09-11T08:44:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.468075+0000", 2024-09-11T08:44:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0872e-05 2024-09-11T08:44:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.468081+0000", 2024-09-11T08:44:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4559999999999999e-06 2024-09-11T08:44:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.469568+0000", 2024-09-11T08:44:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014878179999999999 2024-09-11T08:44:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.469576+0000", 2024-09-11T08:44:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9340000000000006e-06 2024-09-11T08:44:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:223280 7.2 e135/88)", 2024-09-11T08:44:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:18.468169+0000", 2024-09-11T08:44:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4294967295.9999132, 2024-09-11T08:44:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001765693, 2024-09-11T08:44:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-11T08:44:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.468169+0000", 2024-09-11T08:44:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.468169+0000", 2024-09-11T08:44:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.468168+0000", 2024-09-11T08:44:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:44:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.468173+0000", 2024-09-11T08:44:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0839999999999999e-06 2024-09-11T08:44:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.468182+0000", 2024-09-11T08:44:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5930000000000006e-06 2024-09-11T08:44:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:18.468187+0000", 2024-09-11T08:44:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.933e-06 2024-09-11T08:44:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:44:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 7 2024-09-11T08:44:18.584 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:44:18.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:18.585 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:18.585 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:18.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:18 smithi145 ceph-mon[33384]: 6.0 scrub starts 2024-09-11T08:44:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:18 smithi145 ceph-mon[33384]: 6.0 scrub ok 2024-09-11T08:44:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:18 smithi145 ceph-mon[33384]: 5.7 scrub starts 2024-09-11T08:44:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:18 smithi145 ceph-mon[33384]: 5.7 scrub ok 2024-09-11T08:44:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:18 smithi145 ceph-mon[33384]: pgmap v220: 148 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 144 active+clean; 1.8 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.19k op/s; 5684/91269 objects misplaced (6.228%); 124 B/s, 8 objects/s recovering 2024-09-11T08:44:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:18 smithi145 ceph-mon[33384]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:44:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:18 smithi145 ceph-mon[33384]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 4 pgs peering) 2024-09-11T08:44:18.686 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:18.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:18.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:18.829 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:18.866 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:18.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:18.867 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:18.968 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:19.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:19.010 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:19.110 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:19.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:19.137 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:19.137 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:19.238 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:19.263 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:19.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:19.263 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:19.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:19.404 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:19.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:19.405 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:19.506 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:19.540 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:19.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:19.541 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:19.573 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:19.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-11T08:44:19.624 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:44:19.624 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:44:19.624 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-11T08:44:19.624 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T08:44:19.624 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T08:44:19.625 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 config set filestore_inject_stall 3 2024-09-11T08:44:19.642 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:19.656 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:19 smithi092 ceph-mon[29647]: 6.9 scrub starts 2024-09-11T08:44:19.656 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:19 smithi092 ceph-mon[29647]: 6.9 scrub ok 2024-09-11T08:44:19.656 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:19 smithi092 ceph-mon[29647]: 7.5 scrub starts 2024-09-11T08:44:19.656 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:19 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T08:44:19.656 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:19 smithi092 ceph-mon[29647]: 4.1c scrub ok 2024-09-11T08:44:19.656 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:19 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/806909357' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:44:19.656 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:19 smithi092 ceph-mon[29647]: 7.5 scrub ok 2024-09-11T08:44:19.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:19 smithi092 ceph-mon[41114]: 6.9 scrub starts 2024-09-11T08:44:19.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:19 smithi092 ceph-mon[41114]: 6.9 scrub ok 2024-09-11T08:44:19.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:19 smithi092 ceph-mon[41114]: 7.5 scrub starts 2024-09-11T08:44:19.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:19 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T08:44:19.658 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:19 smithi092 ceph-mon[41114]: 4.1c scrub ok 2024-09-11T08:44:19.658 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:19 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/806909357' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:44:19.658 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:19 smithi092 ceph-mon[41114]: 7.5 scrub ok 2024-09-11T08:44:19.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:19 smithi145 ceph-mon[33384]: 6.9 scrub starts 2024-09-11T08:44:19.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:19 smithi145 ceph-mon[33384]: 6.9 scrub ok 2024-09-11T08:44:19.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:19 smithi145 ceph-mon[33384]: 7.5 scrub starts 2024-09-11T08:44:19.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:19 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T08:44:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:19 smithi145 ceph-mon[33384]: 4.1c scrub ok 2024-09-11T08:44:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:19 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/806909357' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:44:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:19 smithi145 ceph-mon[33384]: 7.5 scrub ok 2024-09-11T08:44:19.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:19.709 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:19.709 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:19.775 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:44:19.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:19.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:19.874 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:19.875 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:19.976 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:19.998 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:19.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:19.999 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:20.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:20.141 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:20.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:20.142 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:20.243 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:20.285 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:20.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:20.286 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:20.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:20.425 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:20.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:20.426 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:20.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:20.565 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:20.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[29647]: 4.15 scrub starts 2024-09-11T08:44:20.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[29647]: 4.15 scrub ok 2024-09-11T08:44:20.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[29647]: pgmap v221: 148 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 144 active+clean; 1.8 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 860 op/s; 5684/91269 objects misplaced (6.228%); 105 B/s, 6 objects/s recovering 2024-09-11T08:44:20.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:44:20.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/806909357' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:44:20.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[29647]: osdmap e136: 8 total, 8 up, 8 in 2024-09-11T08:44:20.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[29647]: 6.17 scrub starts 2024-09-11T08:44:20.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[29647]: 6.17 scrub ok 2024-09-11T08:44:20.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[41114]: 4.15 scrub starts 2024-09-11T08:44:20.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[41114]: 4.15 scrub ok 2024-09-11T08:44:20.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[41114]: pgmap v221: 148 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 144 active+clean; 1.8 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 860 op/s; 5684/91269 objects misplaced (6.228%); 105 B/s, 6 objects/s recovering 2024-09-11T08:44:20.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:44:20.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/806909357' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:44:20.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[41114]: osdmap e136: 8 total, 8 up, 8 in 2024-09-11T08:44:20.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[41114]: 6.17 scrub starts 2024-09-11T08:44:20.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:20 smithi092 ceph-mon[41114]: 6.17 scrub ok 2024-09-11T08:44:20.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:20.567 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:20.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:20 smithi145 ceph-mon[33384]: 4.15 scrub starts 2024-09-11T08:44:20.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:20 smithi145 ceph-mon[33384]: 4.15 scrub ok 2024-09-11T08:44:20.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:20 smithi145 ceph-mon[33384]: pgmap v221: 148 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 144 active+clean; 1.8 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 860 op/s; 5684/91269 objects misplaced (6.228%); 105 B/s, 6 objects/s recovering 2024-09-11T08:44:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:20 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:44:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:20 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/806909357' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:44:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:20 smithi145 ceph-mon[33384]: osdmap e136: 8 total, 8 up, 8 in 2024-09-11T08:44:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:20 smithi145 ceph-mon[33384]: 6.17 scrub starts 2024-09-11T08:44:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:20 smithi145 ceph-mon[33384]: 6.17 scrub ok 2024-09-11T08:44:20.668 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:20.694 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:20.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:20.695 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:20.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:20.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:20.838 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:20.838 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:20.939 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:20.966 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:20.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:20.967 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:21.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:21.112 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:21.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:21.113 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:21.213 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:21.254 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:21.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:21.255 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:21.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:21.388 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:21.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:21.389 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:21.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:21.509 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:21.509 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:21.554 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:21.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:21.555 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:21.643 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:21 smithi092 ceph-mon[29647]: 3.5 scrub starts 2024-09-11T08:44:21.643 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:21 smithi092 ceph-mon[29647]: 3.5 scrub ok 2024-09-11T08:44:21.644 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:21 smithi092 ceph-mon[41114]: 3.5 scrub starts 2024-09-11T08:44:21.644 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:21 smithi092 ceph-mon[41114]: 3.5 scrub ok 2024-09-11T08:44:21.644 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:44:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:44:21] "GET /metrics HTTP/1.1" 200 223947 "" "Prometheus/2.43.0" 2024-09-11T08:44:21.656 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:21.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:21 smithi145 ceph-mon[33384]: 3.5 scrub starts 2024-09-11T08:44:21.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:21 smithi145 ceph-mon[33384]: 3.5 scrub ok 2024-09-11T08:44:21.684 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:21.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:21.686 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:21.786 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:21.811 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:21.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:21.812 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:21.912 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:21.938 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:21.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:21.939 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:22.040 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:22.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:22.078 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:22.078 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:22.179 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:22.202 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:22.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:22.203 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:22.304 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:22.345 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:22.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:22.346 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:22.447 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:22.470 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:22.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:22.472 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:22.572 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:22.590 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[29647]: 4.7 scrub starts 2024-09-11T08:44:22.590 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[29647]: 4.7 scrub ok 2024-09-11T08:44:22.590 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[29647]: 3.a scrub starts 2024-09-11T08:44:22.590 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[29647]: 3.a scrub ok 2024-09-11T08:44:22.590 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[29647]: pgmap v223: 148 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 144 active+clean; 1.8 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 554 op/s; 5380/87384 objects misplaced (6.157%); 94 B/s, 5 objects/s recovering 2024-09-11T08:44:22.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[29647]: 4.10 deep-scrub starts 2024-09-11T08:44:22.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[29647]: 4.10 deep-scrub ok 2024-09-11T08:44:22.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[29647]: 3.9 scrub starts 2024-09-11T08:44:22.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[29647]: 3.9 scrub ok 2024-09-11T08:44:22.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[41114]: 4.7 scrub starts 2024-09-11T08:44:22.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[41114]: 4.7 scrub ok 2024-09-11T08:44:22.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[41114]: 3.a scrub starts 2024-09-11T08:44:22.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[41114]: 3.a scrub ok 2024-09-11T08:44:22.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[41114]: pgmap v223: 148 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 144 active+clean; 1.8 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 554 op/s; 5380/87384 objects misplaced (6.157%); 94 B/s, 5 objects/s recovering 2024-09-11T08:44:22.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[41114]: 4.10 deep-scrub starts 2024-09-11T08:44:22.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[41114]: 4.10 deep-scrub ok 2024-09-11T08:44:22.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[41114]: 3.9 scrub starts 2024-09-11T08:44:22.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:22 smithi092 ceph-mon[41114]: 3.9 scrub ok 2024-09-11T08:44:22.597 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:22.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:22.598 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:22.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:22 smithi145 ceph-mon[33384]: 4.7 scrub starts 2024-09-11T08:44:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:22 smithi145 ceph-mon[33384]: 4.7 scrub ok 2024-09-11T08:44:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:22 smithi145 ceph-mon[33384]: 3.a scrub starts 2024-09-11T08:44:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:22 smithi145 ceph-mon[33384]: 3.a scrub ok 2024-09-11T08:44:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:22 smithi145 ceph-mon[33384]: pgmap v223: 148 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 144 active+clean; 1.8 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 554 op/s; 5380/87384 objects misplaced (6.157%); 94 B/s, 5 objects/s recovering 2024-09-11T08:44:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:22 smithi145 ceph-mon[33384]: 4.10 deep-scrub starts 2024-09-11T08:44:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:22 smithi145 ceph-mon[33384]: 4.10 deep-scrub ok 2024-09-11T08:44:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:22 smithi145 ceph-mon[33384]: 3.9 scrub starts 2024-09-11T08:44:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:22 smithi145 ceph-mon[33384]: 3.9 scrub ok 2024-09-11T08:44:22.699 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:22.722 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:22.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:22.723 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:22.734 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:44:22.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:22.871 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:22.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:22.873 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:22.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:23.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:23.018 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:23.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:23.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:23.157 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:23.157 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:23.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:44:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:44:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:23.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:23.292 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:23.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:23.293 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:23.393 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:23.414 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:23.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:23.415 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:23.516 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:23.541 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:23.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:23.542 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:23.643 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:23.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:23 smithi145 ceph-mon[33384]: 7.9 scrub starts 2024-09-11T08:44:23.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:23 smithi145 ceph-mon[33384]: 7.b scrub starts 2024-09-11T08:44:23.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:23 smithi145 ceph-mon[33384]: 7.b scrub ok 2024-09-11T08:44:23.680 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:23.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:23.681 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:23.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:23.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:23 smithi092 ceph-mon[29647]: 7.9 scrub starts 2024-09-11T08:44:23.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:23 smithi092 ceph-mon[29647]: 7.b scrub starts 2024-09-11T08:44:23.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:23 smithi092 ceph-mon[29647]: 7.b scrub ok 2024-09-11T08:44:23.796 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:23 smithi092 ceph-mon[41114]: 7.9 scrub starts 2024-09-11T08:44:23.796 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:23 smithi092 ceph-mon[41114]: 7.b scrub starts 2024-09-11T08:44:23.796 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:23 smithi092 ceph-mon[41114]: 7.b scrub ok 2024-09-11T08:44:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:23.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:23.820 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:23.920 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:23.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:23.945 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:23.945 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:24.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:24.096 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:24.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:24.098 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:24.198 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:24.223 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:24.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:24.224 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:24.325 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:24.349 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:24.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:24.350 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:24.451 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:24.476 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:24.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:24.477 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:24.578 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:24.603 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:24.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:24.604 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:24.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:24 smithi145 ceph-mon[33384]: 7.9 scrub ok 2024-09-11T08:44:24.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:24 smithi145 ceph-mon[33384]: 6.19 scrub starts 2024-09-11T08:44:24.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:24 smithi145 ceph-mon[33384]: 6.19 scrub ok 2024-09-11T08:44:24.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:24 smithi145 ceph-mon[33384]: pgmap v224: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 1.5 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 896 op/s; 4707/75153 objects misplaced (6.263%); 83 B/s, 5 objects/s recovering 2024-09-11T08:44:24.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:24 smithi145 ceph-mon[33384]: 3.18 scrub starts 2024-09-11T08:44:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:24 smithi145 ceph-mon[33384]: 3.18 scrub ok 2024-09-11T08:44:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:24 smithi145 ceph-mon[33384]: 6.8 scrub starts 2024-09-11T08:44:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:24 smithi145 ceph-mon[33384]: 6.8 scrub ok 2024-09-11T08:44:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:24 smithi145 ceph-mon[33384]: 4.4 scrub starts 2024-09-11T08:44:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:24 smithi145 ceph-mon[33384]: 4.4 scrub ok 2024-09-11T08:44:24.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:24.742 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:24.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:24.743 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:24.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[29647]: 7.9 scrub ok 2024-09-11T08:44:24.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[29647]: 6.19 scrub starts 2024-09-11T08:44:24.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[29647]: 6.19 scrub ok 2024-09-11T08:44:24.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[29647]: pgmap v224: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 1.5 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 896 op/s; 4707/75153 objects misplaced (6.263%); 83 B/s, 5 objects/s recovering 2024-09-11T08:44:24.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[29647]: 3.18 scrub starts 2024-09-11T08:44:24.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[29647]: 3.18 scrub ok 2024-09-11T08:44:24.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[29647]: 6.8 scrub starts 2024-09-11T08:44:24.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[29647]: 6.8 scrub ok 2024-09-11T08:44:24.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[29647]: 4.4 scrub starts 2024-09-11T08:44:24.796 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[29647]: 4.4 scrub ok 2024-09-11T08:44:24.796 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[41114]: 7.9 scrub ok 2024-09-11T08:44:24.796 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[41114]: 6.19 scrub starts 2024-09-11T08:44:24.796 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[41114]: 6.19 scrub ok 2024-09-11T08:44:24.796 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[41114]: pgmap v224: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 1.5 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 896 op/s; 4707/75153 objects misplaced (6.263%); 83 B/s, 5 objects/s recovering 2024-09-11T08:44:24.796 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[41114]: 3.18 scrub starts 2024-09-11T08:44:24.797 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[41114]: 3.18 scrub ok 2024-09-11T08:44:24.797 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[41114]: 6.8 scrub starts 2024-09-11T08:44:24.797 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[41114]: 6.8 scrub ok 2024-09-11T08:44:24.797 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[41114]: 4.4 scrub starts 2024-09-11T08:44:24.797 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:24 smithi092 ceph-mon[41114]: 4.4 scrub ok 2024-09-11T08:44:24.844 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:24.870 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:24.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:24.871 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:24.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:24.975 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:44:24.975 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:25.012 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:25.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:25.013 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:25.042 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:44:25.042 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:25.070 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:44:25.070 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:25.098 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:44:25.098 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:25.114 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:25.129 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:44:25.129 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:25.139 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:25.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:25.141 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:25.160 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:44:25.161 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:25.189 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:44:25.189 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:25.219 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:44:25.220 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:25.241 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:25.264 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:25.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:25.265 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:25.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:25.413 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:25.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:25.414 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:25.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:25.546 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:25.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:25.547 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:25.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:25.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:25.696 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:25.796 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:25.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:25 smithi092 ceph-mon[29647]: 4.19 scrub starts 2024-09-11T08:44:25.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:25 smithi092 ceph-mon[29647]: 4.19 scrub ok 2024-09-11T08:44:25.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:25 smithi092 ceph-mon[29647]: 3.13 scrub starts 2024-09-11T08:44:25.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:25 smithi092 ceph-mon[29647]: 3.13 scrub ok 2024-09-11T08:44:25.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:25 smithi092 ceph-mon[29647]: 5.c scrub starts 2024-09-11T08:44:25.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:25 smithi092 ceph-mon[29647]: 5.c scrub ok 2024-09-11T08:44:25.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:25 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2259392011' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:44:25.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:25 smithi092 ceph-mon[41114]: 4.19 scrub starts 2024-09-11T08:44:25.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:25 smithi092 ceph-mon[41114]: 4.19 scrub ok 2024-09-11T08:44:25.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:25 smithi092 ceph-mon[41114]: 3.13 scrub starts 2024-09-11T08:44:25.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:25 smithi092 ceph-mon[41114]: 3.13 scrub ok 2024-09-11T08:44:25.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:25 smithi092 ceph-mon[41114]: 5.c scrub starts 2024-09-11T08:44:25.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:25 smithi092 ceph-mon[41114]: 5.c scrub ok 2024-09-11T08:44:25.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:25 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2259392011' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:44:25.820 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:25.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:25.821 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:25.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:25 smithi145 ceph-mon[33384]: 4.19 scrub starts 2024-09-11T08:44:25.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:25 smithi145 ceph-mon[33384]: 4.19 scrub ok 2024-09-11T08:44:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:25 smithi145 ceph-mon[33384]: 3.13 scrub starts 2024-09-11T08:44:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:25 smithi145 ceph-mon[33384]: 3.13 scrub ok 2024-09-11T08:44:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:25 smithi145 ceph-mon[33384]: 5.c scrub starts 2024-09-11T08:44:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:25 smithi145 ceph-mon[33384]: 5.c scrub ok 2024-09-11T08:44:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:25 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2259392011' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:44:25.922 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:25.947 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:25.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:25.948 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:26.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:26.104 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:26.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:26.105 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:26.205 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:26.239 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:26.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:26.240 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:26.341 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:26.367 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:26.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:26.368 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:26.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:26.505 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:26.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:26.506 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[29647]: 6.d scrub starts 2024-09-11T08:44:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[29647]: 6.d scrub ok 2024-09-11T08:44:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[29647]: pgmap v225: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 1.5 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 896 op/s; 4707/75153 objects misplaced (6.263%); 83 B/s, 5 objects/s recovering 2024-09-11T08:44:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[29647]: 3.7 scrub starts 2024-09-11T08:44:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[29647]: 3.7 scrub ok 2024-09-11T08:44:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[29647]: 5.19 scrub starts 2024-09-11T08:44:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[29647]: 5.19 scrub ok 2024-09-11T08:44:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[29647]: 6.1a scrub starts 2024-09-11T08:44:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[29647]: 6.1a scrub ok 2024-09-11T08:44:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[41114]: 6.d scrub starts 2024-09-11T08:44:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[41114]: 6.d scrub ok 2024-09-11T08:44:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[41114]: pgmap v225: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 1.5 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 896 op/s; 4707/75153 objects misplaced (6.263%); 83 B/s, 5 objects/s recovering 2024-09-11T08:44:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[41114]: 3.7 scrub starts 2024-09-11T08:44:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[41114]: 3.7 scrub ok 2024-09-11T08:44:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[41114]: 5.19 scrub starts 2024-09-11T08:44:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[41114]: 5.19 scrub ok 2024-09-11T08:44:26.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[41114]: 6.1a scrub starts 2024-09-11T08:44:26.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:26 smithi092 ceph-mon[41114]: 6.1a scrub ok 2024-09-11T08:44:26.607 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:26.632 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:26.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:26.633 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:26.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:44:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:44:26.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:26 smithi145 ceph-mon[33384]: 6.d scrub starts 2024-09-11T08:44:26.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:26 smithi145 ceph-mon[33384]: 6.d scrub ok 2024-09-11T08:44:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:26 smithi145 ceph-mon[33384]: pgmap v225: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 1.5 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 896 op/s; 4707/75153 objects misplaced (6.263%); 83 B/s, 5 objects/s recovering 2024-09-11T08:44:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:26 smithi145 ceph-mon[33384]: 3.7 scrub starts 2024-09-11T08:44:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:26 smithi145 ceph-mon[33384]: 3.7 scrub ok 2024-09-11T08:44:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:26 smithi145 ceph-mon[33384]: 5.19 scrub starts 2024-09-11T08:44:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:26 smithi145 ceph-mon[33384]: 5.19 scrub ok 2024-09-11T08:44:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:26 smithi145 ceph-mon[33384]: 6.1a scrub starts 2024-09-11T08:44:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:26 smithi145 ceph-mon[33384]: 6.1a scrub ok 2024-09-11T08:44:26.733 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:26.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:26.761 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:26.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:26.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:26.899 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:27.000 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:27.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:27.025 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:27.026 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:27.126 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:27.151 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:27.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:27.152 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:27.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:27.290 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:27.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:27.291 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:27.392 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:27.415 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:27.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:27.416 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:27.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:27.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:27.549 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:27.550 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:27.650 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:27.687 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:27.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:27.688 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:27.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:27.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:27.810 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:27.811 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:27.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:27 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:27.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:27 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:44:27.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:27 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:44:27.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:27 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:27.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:27 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:44:27.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:27 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:44:27.911 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:27.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:27 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:27.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:27 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:44:27.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:27 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:44:27.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:27.958 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:27.958 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:28.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:28.099 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:28.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:28.101 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:28.108 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:44:28.109 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T08:44:28.109 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:44:28.201 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:28.225 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:28.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:28.226 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:28.326 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:28.365 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:28.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:28.366 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:28.467 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:28.493 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:28.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:28.494 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:28.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[29647]: 7.6 scrub starts 2024-09-11T08:44:28.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[29647]: 5.f deep-scrub starts 2024-09-11T08:44:28.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[29647]: 5.f deep-scrub ok 2024-09-11T08:44:28.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:44:28.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:44:28.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[29647]: pgmap v226: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 1.2 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.06k op/s; 3660/59349 objects misplaced (6.167%) 2024-09-11T08:44:28.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[29647]: osdmap e137: 8 total, 8 up, 8 in 2024-09-11T08:44:28.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[29647]: 4.a scrub starts 2024-09-11T08:44:28.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[29647]: 4.a scrub ok 2024-09-11T08:44:28.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[29647]: 4.1a scrub starts 2024-09-11T08:44:28.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[29647]: 4.1a scrub ok 2024-09-11T08:44:28.520 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[41114]: 7.6 scrub starts 2024-09-11T08:44:28.520 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[41114]: 5.f deep-scrub starts 2024-09-11T08:44:28.520 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[41114]: 5.f deep-scrub ok 2024-09-11T08:44:28.520 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:44:28.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:44:28.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[41114]: pgmap v226: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 1.2 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.06k op/s; 3660/59349 objects misplaced (6.167%) 2024-09-11T08:44:28.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[41114]: osdmap e137: 8 total, 8 up, 8 in 2024-09-11T08:44:28.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[41114]: 4.a scrub starts 2024-09-11T08:44:28.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[41114]: 4.a scrub ok 2024-09-11T08:44:28.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[41114]: 4.1a scrub starts 2024-09-11T08:44:28.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:28 smithi092 ceph-mon[41114]: 4.1a scrub ok 2024-09-11T08:44:28.595 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:28.623 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:28.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:28.624 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:28.725 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:28.751 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:28.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:28.752 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:28 smithi145 ceph-mon[33384]: 7.6 scrub starts 2024-09-11T08:44:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:28 smithi145 ceph-mon[33384]: 5.f deep-scrub starts 2024-09-11T08:44:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:28 smithi145 ceph-mon[33384]: 5.f deep-scrub ok 2024-09-11T08:44:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:28 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:44:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:28 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:44:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:28 smithi145 ceph-mon[33384]: pgmap v226: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 1.2 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.06k op/s; 3660/59349 objects misplaced (6.167%) 2024-09-11T08:44:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:28 smithi145 ceph-mon[33384]: osdmap e137: 8 total, 8 up, 8 in 2024-09-11T08:44:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:28 smithi145 ceph-mon[33384]: 4.a scrub starts 2024-09-11T08:44:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:28 smithi145 ceph-mon[33384]: 4.a scrub ok 2024-09-11T08:44:28.813 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:28 smithi145 ceph-mon[33384]: 4.1a scrub starts 2024-09-11T08:44:28.813 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:28 smithi145 ceph-mon[33384]: 4.1a scrub ok 2024-09-11T08:44:28.852 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:28.879 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:28.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:28.880 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:28.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:29.019 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:29.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:29.020 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:29.121 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:29.147 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:29.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:29.148 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:29.249 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:29.279 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:29.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:29.280 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:29.334 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:29.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:29.436 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:29.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:29.438 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:29.459 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:29 smithi145 ceph-mon[33384]: 7.6 scrub ok 2024-09-11T08:44:29.459 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:29 smithi145 ceph-mon[33384]: 6.1b scrub starts 2024-09-11T08:44:29.459 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:29 smithi145 ceph-mon[33384]: 6.1b scrub ok 2024-09-11T08:44:29.459 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:29 smithi145 ceph-mon[33384]: 5.10 scrub starts 2024-09-11T08:44:29.459 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:29 smithi145 ceph-mon[33384]: 5.10 scrub ok 2024-09-11T08:44:29.460 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:29 smithi145 ceph-mon[33384]: 4.d scrub starts 2024-09-11T08:44:29.460 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:29 smithi145 ceph-mon[33384]: 4.d scrub ok 2024-09-11T08:44:29.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:29.580 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:29.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:29.581 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:29.633 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:29 smithi092 ceph-mon[41114]: 7.6 scrub ok 2024-09-11T08:44:29.634 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:29 smithi092 ceph-mon[41114]: 6.1b scrub starts 2024-09-11T08:44:29.634 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:29 smithi092 ceph-mon[41114]: 6.1b scrub ok 2024-09-11T08:44:29.634 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:29 smithi092 ceph-mon[41114]: 5.10 scrub starts 2024-09-11T08:44:29.634 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:29 smithi092 ceph-mon[41114]: 5.10 scrub ok 2024-09-11T08:44:29.634 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:29 smithi092 ceph-mon[41114]: 4.d scrub starts 2024-09-11T08:44:29.634 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:29 smithi092 ceph-mon[41114]: 4.d scrub ok 2024-09-11T08:44:29.634 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:29 smithi092 ceph-mon[29647]: 7.6 scrub ok 2024-09-11T08:44:29.634 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:29 smithi092 ceph-mon[29647]: 6.1b scrub starts 2024-09-11T08:44:29.634 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:29 smithi092 ceph-mon[29647]: 6.1b scrub ok 2024-09-11T08:44:29.634 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:29 smithi092 ceph-mon[29647]: 5.10 scrub starts 2024-09-11T08:44:29.635 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:29 smithi092 ceph-mon[29647]: 5.10 scrub ok 2024-09-11T08:44:29.635 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:29 smithi092 ceph-mon[29647]: 4.d scrub starts 2024-09-11T08:44:29.635 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:29 smithi092 ceph-mon[29647]: 4.d scrub ok 2024-09-11T08:44:29.682 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:29.710 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:29.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:29.711 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:29.812 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:29.859 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:29.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:29.860 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:29.960 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:29.991 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:29.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:29.992 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:30.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:30.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:30.134 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:30.135 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:30.232 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:44:30.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:30.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:30.313 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:30.314 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:30.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:30.449 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:30.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:30.451 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:30.551 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:30.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:30 smithi092 ceph-mon[29647]: pgmap v228: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 1.2 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.14k op/s; 3660/59349 objects misplaced (6.167%) 2024-09-11T08:44:30.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:30 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:44:30.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:30 smithi092 ceph-mon[29647]: 5.8 scrub ok 2024-09-11T08:44:30.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:30 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:30.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:30.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:30 smithi092 ceph-mon[41114]: pgmap v228: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 1.2 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.14k op/s; 3660/59349 objects misplaced (6.167%) 2024-09-11T08:44:30.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:30 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:44:30.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:30 smithi092 ceph-mon[41114]: 5.8 scrub ok 2024-09-11T08:44:30.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:30 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:30.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:30.578 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:30.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:30.579 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:30.680 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:30.716 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:30.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:30.718 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:30.818 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:30.842 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:30.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:30.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:30.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:30 smithi145 ceph-mon[33384]: pgmap v228: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 1.2 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.14k op/s; 3660/59349 objects misplaced (6.167%) 2024-09-11T08:44:30.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:30 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:44:30.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:30 smithi145 ceph-mon[33384]: 5.8 scrub ok 2024-09-11T08:44:30.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:30 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:30.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:30.945 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:30.971 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:30.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:30.972 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:31.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:31.120 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:31.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:31.121 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:31.222 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:31.262 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:31.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:31.263 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:31.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:31.378 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:44:31.378 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:44:31.419 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:31.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:31.420 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:31.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:31.561 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:31.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:31.563 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:31.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:31 smithi092 ceph-mon[29647]: 5.b scrub starts 2024-09-11T08:44:31.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:31 smithi092 ceph-mon[29647]: 5.b scrub ok 2024-09-11T08:44:31.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:31 smithi092 ceph-mon[29647]: 3.4 scrub starts 2024-09-11T08:44:31.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:31 smithi092 ceph-mon[29647]: 3.4 scrub ok 2024-09-11T08:44:31.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:44:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:44:31] "GET /metrics HTTP/1.1" 200 224038 "" "Prometheus/2.43.0" 2024-09-11T08:44:31.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:31 smithi092 ceph-mon[41114]: 5.b scrub starts 2024-09-11T08:44:31.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:31 smithi092 ceph-mon[41114]: 5.b scrub ok 2024-09-11T08:44:31.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:31 smithi092 ceph-mon[41114]: 3.4 scrub starts 2024-09-11T08:44:31.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:31 smithi092 ceph-mon[41114]: 3.4 scrub ok 2024-09-11T08:44:31.663 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:31.704 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:31.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:31.705 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:31.806 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:31.831 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:31.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:31.832 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:31.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:31 smithi145 ceph-mon[33384]: 5.b scrub starts 2024-09-11T08:44:31.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:31 smithi145 ceph-mon[33384]: 5.b scrub ok 2024-09-11T08:44:31.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:31 smithi145 ceph-mon[33384]: 3.4 scrub starts 2024-09-11T08:44:31.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:31 smithi145 ceph-mon[33384]: 3.4 scrub ok 2024-09-11T08:44:31.933 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:31.958 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:31.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:31.959 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:32.060 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:32.084 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:32.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:32.085 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:32.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:32.205 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:32.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:32.250 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:32.250 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:32.351 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:32.376 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:32.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:32.377 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:32.477 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:32.516 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:32.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:32.517 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:32.618 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:32.655 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:32.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:32.656 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:32.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:32 smithi145 ceph-mon[33384]: pgmap v229: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 1.1 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.06k op/s; 3341/55449 objects misplaced (6.025%) 2024-09-11T08:44:32.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:32 smithi145 ceph-mon[33384]: 3.1e scrub starts 2024-09-11T08:44:32.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:32 smithi145 ceph-mon[33384]: 3.1e scrub ok 2024-09-11T08:44:32.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:32 smithi145 ceph-mon[33384]: 3.d scrub starts 2024-09-11T08:44:32.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:32 smithi145 ceph-mon[33384]: 3.d scrub ok 2024-09-11T08:44:32.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:32 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T08:44:32.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:32 smithi145 ceph-mon[33384]: 6.f scrub ok 2024-09-11T08:44:32.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:32 smithi092 ceph-mon[29647]: pgmap v229: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 1.1 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.06k op/s; 3341/55449 objects misplaced (6.025%) 2024-09-11T08:44:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:32 smithi092 ceph-mon[29647]: 3.1e scrub starts 2024-09-11T08:44:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:32 smithi092 ceph-mon[29647]: 3.1e scrub ok 2024-09-11T08:44:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:32 smithi092 ceph-mon[29647]: 3.d scrub starts 2024-09-11T08:44:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:32 smithi092 ceph-mon[29647]: 3.d scrub ok 2024-09-11T08:44:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:32 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T08:44:32.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:32 smithi092 ceph-mon[29647]: 6.f scrub ok 2024-09-11T08:44:32.770 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:32 smithi092 ceph-mon[41114]: pgmap v229: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 1.1 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.06k op/s; 3341/55449 objects misplaced (6.025%) 2024-09-11T08:44:32.770 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:32 smithi092 ceph-mon[41114]: 3.1e scrub starts 2024-09-11T08:44:32.770 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:32 smithi092 ceph-mon[41114]: 3.1e scrub ok 2024-09-11T08:44:32.770 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:32 smithi092 ceph-mon[41114]: 3.d scrub starts 2024-09-11T08:44:32.770 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:32 smithi092 ceph-mon[41114]: 3.d scrub ok 2024-09-11T08:44:32.770 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:32 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T08:44:32.770 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:32 smithi092 ceph-mon[41114]: 6.f scrub ok 2024-09-11T08:44:32.796 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:32.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:32.797 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:32.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:32.937 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:32.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:32.938 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:33.039 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:33.066 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:33.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:33.067 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:33.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-11T08:44:33.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:44:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:44:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:33.168 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:33.191 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:33.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:33.192 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:33.293 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:33.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:33.341 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:33.342 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:33.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:33.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:33.486 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:33.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:33.587 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:33.611 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:33.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:33.613 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:33.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:33 smithi092 ceph-mon[29647]: 4.b scrub starts 2024-09-11T08:44:33.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:33 smithi092 ceph-mon[29647]: 4.b scrub ok 2024-09-11T08:44:33.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:33 smithi092 ceph-mon[29647]: 3.11 deep-scrub starts 2024-09-11T08:44:33.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:33 smithi092 ceph-mon[29647]: 3.11 deep-scrub ok 2024-09-11T08:44:33.673 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:33 smithi092 ceph-mon[41114]: 4.b scrub starts 2024-09-11T08:44:33.673 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:33 smithi092 ceph-mon[41114]: 4.b scrub ok 2024-09-11T08:44:33.673 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:33 smithi092 ceph-mon[41114]: 3.11 deep-scrub starts 2024-09-11T08:44:33.673 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:33 smithi092 ceph-mon[41114]: 3.11 deep-scrub ok 2024-09-11T08:44:33.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:33.761 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:33.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:33.763 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:33.859 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:44:33.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 0 1 2024-09-11T08:44:33.864 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:33.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:33 smithi145 ceph-mon[33384]: 4.b scrub starts 2024-09-11T08:44:33.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:33 smithi145 ceph-mon[33384]: 4.b scrub ok 2024-09-11T08:44:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:33 smithi145 ceph-mon[33384]: 3.11 deep-scrub starts 2024-09-11T08:44:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:33 smithi145 ceph-mon[33384]: 3.11 deep-scrub ok 2024-09-11T08:44:33.945 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:33.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:33.947 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:34.047 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:34.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:34.073 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:34.074 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:34.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:34.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:34.216 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:34.216 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:34.317 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:34.341 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:34.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:34.343 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:34.444 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:34.470 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:34.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:34.471 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:34.572 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:34.596 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:34.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:34.597 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:34.682 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:34.697 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:34.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:34.721 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:34.722 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:34.748 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:34 smithi092 ceph-mon[29647]: pgmap v230: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 894 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.06k op/s; 2696/43464 objects misplaced (6.203%) 2024-09-11T08:44:34.748 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:34 smithi092 ceph-mon[29647]: 4.0 scrub starts 2024-09-11T08:44:34.748 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:34 smithi092 ceph-mon[29647]: 4.0 scrub ok 2024-09-11T08:44:34.748 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:34 smithi092 ceph-mon[29647]: 3.19 deep-scrub starts 2024-09-11T08:44:34.748 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:34 smithi092 ceph-mon[29647]: 3.19 deep-scrub ok 2024-09-11T08:44:34.748 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:34 smithi092 ceph-mon[41114]: pgmap v230: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 894 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.06k op/s; 2696/43464 objects misplaced (6.203%) 2024-09-11T08:44:34.748 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:34 smithi092 ceph-mon[41114]: 4.0 scrub starts 2024-09-11T08:44:34.749 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:34 smithi092 ceph-mon[41114]: 4.0 scrub ok 2024-09-11T08:44:34.749 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:34 smithi092 ceph-mon[41114]: 3.19 deep-scrub starts 2024-09-11T08:44:34.749 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:34 smithi092 ceph-mon[41114]: 3.19 deep-scrub ok 2024-09-11T08:44:34.822 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:34.845 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:34.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:34.846 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:34 smithi145 ceph-mon[33384]: pgmap v230: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 894 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.06k op/s; 2696/43464 objects misplaced (6.203%) 2024-09-11T08:44:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:34 smithi145 ceph-mon[33384]: 4.0 scrub starts 2024-09-11T08:44:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:34 smithi145 ceph-mon[33384]: 4.0 scrub ok 2024-09-11T08:44:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:34 smithi145 ceph-mon[33384]: 3.19 deep-scrub starts 2024-09-11T08:44:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:34 smithi145 ceph-mon[33384]: 3.19 deep-scrub ok 2024-09-11T08:44:34.947 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:34.988 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:34.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:34.989 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:35.090 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:35.123 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:35.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:35.124 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:35.128 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:35.225 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:35.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:35.255 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:35.255 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:35.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:35.401 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:35.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:35.402 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:35.501 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:44:35.503 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:35.527 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:35.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:35.528 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:35.596 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:35 smithi092 ceph-mon[29647]: 5.3 scrub starts 2024-09-11T08:44:35.596 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:35 smithi092 ceph-mon[29647]: 5.3 scrub ok 2024-09-11T08:44:35.596 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:35 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/425330693' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:44:35.596 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:35 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:44:35.597 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:35 smithi092 ceph-mon[41114]: 5.3 scrub starts 2024-09-11T08:44:35.597 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:35 smithi092 ceph-mon[41114]: 5.3 scrub ok 2024-09-11T08:44:35.597 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:35 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/425330693' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:44:35.597 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:35 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:44:35.629 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:35.668 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:35.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:35.669 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:35.770 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:35.795 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:35.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:35.796 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:35.897 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:35.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:35 smithi145 ceph-mon[33384]: 5.3 scrub starts 2024-09-11T08:44:35.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:35 smithi145 ceph-mon[33384]: 5.3 scrub ok 2024-09-11T08:44:35.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:35 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/425330693' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:44:35.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:35 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:44:35.923 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:35.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:35.924 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:36.025 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:36.049 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:36.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:36.050 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:36.150 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:36.176 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:36.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:36.177 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:36.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:36.315 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:36.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:36.316 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:36.417 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:36.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:36.441 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:36.441 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:36.544 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:36.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:36.578 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:36.578 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:36.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:44:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:44:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:36 smithi145 ceph-mon[33384]: pgmap v231: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 894 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.06k op/s; 2696/43464 objects misplaced (6.203%) 2024-09-11T08:44:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:36 smithi145 ceph-mon[33384]: 4.17 scrub starts 2024-09-11T08:44:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:36 smithi145 ceph-mon[33384]: 4.17 scrub ok 2024-09-11T08:44:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:36 smithi145 ceph-mon[33384]: 5.4 scrub starts 2024-09-11T08:44:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:36 smithi145 ceph-mon[33384]: 5.4 scrub ok 2024-09-11T08:44:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:36 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:44:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:36 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:44:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:36 smithi145 ceph-mon[33384]: osdmap e138: 8 total, 8 up, 8 in 2024-09-11T08:44:36.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:36 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3515787069' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:44:36.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:36.703 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:36.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:36.704 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:36.805 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[29647]: pgmap v231: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 894 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.06k op/s; 2696/43464 objects misplaced (6.203%) 2024-09-11T08:44:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[29647]: 4.17 scrub starts 2024-09-11T08:44:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[29647]: 4.17 scrub ok 2024-09-11T08:44:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[29647]: 5.4 scrub starts 2024-09-11T08:44:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[29647]: 5.4 scrub ok 2024-09-11T08:44:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:44:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:44:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[29647]: osdmap e138: 8 total, 8 up, 8 in 2024-09-11T08:44:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3515787069' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:44:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[41114]: pgmap v231: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 894 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.06k op/s; 2696/43464 objects misplaced (6.203%) 2024-09-11T08:44:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[41114]: 4.17 scrub starts 2024-09-11T08:44:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[41114]: 4.17 scrub ok 2024-09-11T08:44:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[41114]: 5.4 scrub starts 2024-09-11T08:44:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[41114]: 5.4 scrub ok 2024-09-11T08:44:36.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:44:36.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:44:36.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[41114]: osdmap e138: 8 total, 8 up, 8 in 2024-09-11T08:44:36.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:36 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3515787069' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:44:36.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:36.842 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:36.842 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:36.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:36.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:36.983 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:36.983 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:37.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:37.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:37.123 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:37.123 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:37.224 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:37.258 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:37.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:37.259 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:37.360 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:37.384 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:37.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:37.385 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:37.486 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:37.512 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:37.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:37.513 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:37.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:37.635 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:37.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:37.636 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:37.737 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:37.762 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:37.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:37.764 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:37 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:37 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:44:37.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:37 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:44:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:37 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:37 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:44:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:37 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:44:37.865 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:37.889 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:37.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:37.890 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:37.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:37 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:37.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:37 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:44:37.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:37 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:44:37.991 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:38.017 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:38.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:38.018 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:38.084 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:44:38.084 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:44:38.084 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:44:38.084 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:44:38.085 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:44:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:38.085 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:44:38.086 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:44:38.086 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:44:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:38.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:38.158 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:38.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:38.159 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:38.260 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:38.286 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:38.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:38.287 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:38.388 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:38.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:38.438 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:38.438 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:38.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:38.580 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:38.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:38.584 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:38.685 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:38.711 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:38.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:38.712 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:38.813 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:38.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:38 smithi092 ceph-mon[29647]: pgmap v233: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 564 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.09k op/s; 1718/27630 objects misplaced (6.218%) 2024-09-11T08:44:38.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:38 smithi092 ceph-mon[29647]: osdmap e139: 8 total, 8 up, 8 in 2024-09-11T08:44:38.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:38 smithi092 ceph-mon[41114]: pgmap v233: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 564 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.09k op/s; 1718/27630 objects misplaced (6.218%) 2024-09-11T08:44:38.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:38 smithi092 ceph-mon[41114]: osdmap e139: 8 total, 8 up, 8 in 2024-09-11T08:44:38.841 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:38.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:38.846 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:38.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:38 smithi145 ceph-mon[33384]: pgmap v233: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 564 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.09k op/s; 1718/27630 objects misplaced (6.218%) 2024-09-11T08:44:38.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:38 smithi145 ceph-mon[33384]: osdmap e139: 8 total, 8 up, 8 in 2024-09-11T08:44:38.947 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:38.984 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:38.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:38.985 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:39.085 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:39.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:39.110 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:39.110 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:39.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:39.249 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:39.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:39.250 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:39.351 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:39.386 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:39.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:39.387 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:39.488 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:39.515 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:39.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:39.516 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:39.617 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:39.641 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:39.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:39.642 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:39.743 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:39.791 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:39.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:39.792 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:39.893 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:39 smithi092 ceph-mon[29647]: pgmap v235: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 564 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.16k op/s; 1718/27630 objects misplaced (6.218%) 2024-09-11T08:44:39.893 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:39 smithi092 ceph-mon[41114]: pgmap v235: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 564 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.16k op/s; 1718/27630 objects misplaced (6.218%) 2024-09-11T08:44:39.894 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:39.917 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:39.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:39.918 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:40.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:40.060 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:40.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:40.061 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:40.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:39 smithi145 ceph-mon[33384]: pgmap v235: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 564 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.16k op/s; 1718/27630 objects misplaced (6.218%) 2024-09-11T08:44:40.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:40.202 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:40.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:40.203 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:40.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:40.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:40.343 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:40.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:40.486 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:40.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:40.487 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:40.588 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:40.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:40.611 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:40.611 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:40.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:40.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:40.755 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:40.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:40.889 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:44:40.893 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:40.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:40.894 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:40.994 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:41.020 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:41.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:41.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:41.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:40 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:44:41.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:40 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:44:41.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:40 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:44:41.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:40 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:44:41.122 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:41.146 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:41.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:41.147 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:41.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:40 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:44:41.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:40 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:44:41.248 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:41.272 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:41.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:41.273 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:41.374 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:41.413 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:41.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:41.414 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:41.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:41.553 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:41.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:41.554 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:41.654 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:41.681 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:41.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:41.682 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:41.783 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:41.807 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:44:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:44:41] "GET /metrics HTTP/1.1" 200 224038 "" "Prometheus/2.43.0" 2024-09-11T08:44:41.809 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:41.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:41.811 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:41.912 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:41.939 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:41.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:41.940 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:42.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:42.059 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:41 smithi092 ceph-mon[29647]: osdmap e140: 8 total, 8 up, 8 in 2024-09-11T08:44:42.059 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:41 smithi092 ceph-mon[29647]: pgmap v237: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 479 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.11k op/s; 1385/23559 objects misplaced (5.879%) 2024-09-11T08:44:42.060 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:41 smithi092 ceph-mon[41114]: osdmap e140: 8 total, 8 up, 8 in 2024-09-11T08:44:42.060 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:41 smithi092 ceph-mon[41114]: pgmap v237: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 479 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.11k op/s; 1385/23559 objects misplaced (5.879%) 2024-09-11T08:44:42.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:42.108 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:42.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:41 smithi145 ceph-mon[33384]: osdmap e140: 8 total, 8 up, 8 in 2024-09-11T08:44:42.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:41 smithi145 ceph-mon[33384]: pgmap v237: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 479 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.11k op/s; 1385/23559 objects misplaced (5.879%) 2024-09-11T08:44:42.209 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:42.228 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:44:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-11T08:44:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-11T08:44:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:44:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14156.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-11T08:44:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:36:30.992002+0000", 2024-09-11T08:44:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 489.54139628600001, 2024-09-11T08:44:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.51953536199999995, 2024-09-11T08:44:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:44:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14156", 2024-09-11T08:44:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3501902574", 2024-09-11T08:44:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1 2024-09-11T08:44:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992002+0000", 2024-09-11T08:44:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992002+0000", 2024-09-11T08:44:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992000+0000", 2024-09-11T08:44:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:44:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992008+0000", 2024-09-11T08:44:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9279999999999993e-06 2024-09-11T08:44:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992018+0000", 2024-09-11T08:44:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0045e-05 2024-09-11T08:44:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992026+0000", 2024-09-11T08:44:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8520000000000004e-06 2024-09-11T08:44:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992080+0000", 2024-09-11T08:44:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3891999999999998e-05 2024-09-11T08:44:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:44:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:30.992090+0000", 2024-09-11T08:44:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9750000000000002e-06 2024-09-11T08:44:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.510497+0000", 2024-09-11T08:44:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.51840699700000004 2024-09-11T08:44:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.510602+0000", 2024-09-11T08:44:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105664 2024-09-11T08:44:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:44:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.510729+0000", 2024-09-11T08:44:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012635399999999999 2024-09-11T08:44:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:44:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.511254+0000", 2024-09-11T08:44:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052565800000000003 2024-09-11T08:44:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.511424+0000", 2024-09-11T08:44:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169438 2024-09-11T08:44:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.511483+0000", 2024-09-11T08:44:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9095999999999998e-05 2024-09-11T08:44:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.511516+0000", 2024-09-11T08:44:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2801e-05 2024-09-11T08:44:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:36:31.511537+0000", 2024-09-11T08:44:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1376999999999999e-05 2024-09-11T08:44:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:66215 7.4 e99/88 7:223f9d8a:::benchmark_data_smithi092_84565_object66214:head v 99'4092, mlcod=99'4089)", 2024-09-11T08:44:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.170867+0000", 2024-09-11T08:44:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 174.362530846, 2024-09-11T08:44:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103303007, 2024-09-11T08:44:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.170867+0000", 2024-09-11T08:44:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.170867+0000", 2024-09-11T08:44:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.170866+0000", 2024-09-11T08:44:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:44:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.170893+0000", 2024-09-11T08:44:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7039999999999999e-05 2024-09-11T08:44:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.170906+0000", 2024-09-11T08:44:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2996000000000001e-05 2024-09-11T08:44:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.170913+0000", 2024-09-11T08:44:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6139999999999997e-06 2024-09-11T08:44:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.170984+0000", 2024-09-11T08:44:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0864000000000005e-05 2024-09-11T08:44:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.171002+0000", 2024-09-11T08:44:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8230999999999999e-05 2024-09-11T08:44:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.175422+0000", 2024-09-11T08:44:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044200940000000003 2024-09-11T08:44:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274170+0000", 2024-09-11T08:44:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.098748194999999997 2024-09-11T08:44:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:66220 7.0 e99/88 7:0e1c1cb9:::benchmark_data_smithi092_84565_object66219:head v 99'4219, mlcod=99'4217)", 2024-09-11T08:44:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.172989+0000", 2024-09-11T08:44:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 174.360409237, 2024-09-11T08:44:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10152572, 2024-09-11T08:44:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.172989+0000", 2024-09-11T08:44:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.172989+0000", 2024-09-11T08:44:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.172979+0000", 2024-09-11T08:44:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999905 2024-09-11T08:44:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173083+0000", 2024-09-11T08:44:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104143 2024-09-11T08:44:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173095+0000", 2024-09-11T08:44:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1891e-05 2024-09-11T08:44:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173101+0000", 2024-09-11T08:44:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3580000000000002e-06 2024-09-11T08:44:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173156+0000", 2024-09-11T08:44:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4836000000000001e-05 2024-09-11T08:44:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173175+0000", 2024-09-11T08:44:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9369999999999999e-05 2024-09-11T08:44:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.175513+0000", 2024-09-11T08:44:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023384289999999999 2024-09-11T08:44:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274514+0000", 2024-09-11T08:44:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.099001071999999996 2024-09-11T08:44:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:66218 7.4 e99/88 7:2ebb945f:::benchmark_data_smithi092_84565_object66217:head v 99'4093, mlcod=99'4090)", 2024-09-11T08:44:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.173196+0000", 2024-09-11T08:44:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 174.360201617, 2024-09-11T08:44:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101057367, 2024-09-11T08:44:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173196+0000", 2024-09-11T08:44:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173196+0000", 2024-09-11T08:44:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173196+0000", 2024-09-11T08:44:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:44:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173219+0000", 2024-09-11T08:44:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3541999999999999e-05 2024-09-11T08:44:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173230+0000", 2024-09-11T08:44:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0504e-05 2024-09-11T08:44:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173235+0000", 2024-09-11T08:44:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3789999999999997e-06 2024-09-11T08:44:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173403+0000", 2024-09-11T08:44:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016734700000000001 2024-09-11T08:44:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.173419+0000", 2024-09-11T08:44:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6478e-05 2024-09-11T08:44:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274219+0000", 2024-09-11T08:44:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10079982799999999 2024-09-11T08:44:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274254+0000", 2024-09-11T08:44:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4863999999999997e-05 2024-09-11T08:44:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:66214 7.2 e99/88 7:43d51050:::benchmark_data_smithi092_84565_object66213:head v 99'4137, mlcod=99'4135)", 2024-09-11T08:44:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.174111+0000", 2024-09-11T08:44:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 174.35928702300001, 2024-09-11T08:44:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.106495039, 2024-09-11T08:44:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174111+0000", 2024-09-11T08:44:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174111+0000", 2024-09-11T08:44:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174110+0000", 2024-09-11T08:44:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:44:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174138+0000", 2024-09-11T08:44:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7681000000000001e-05 2024-09-11T08:44:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174149+0000", 2024-09-11T08:44:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1185e-05 2024-09-11T08:44:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174155+0000", 2024-09-11T08:44:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6230000000000002e-06 2024-09-11T08:44:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174224+0000", 2024-09-11T08:44:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9218e-05 2024-09-11T08:44:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174241+0000", 2024-09-11T08:44:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7356000000000001e-05 2024-09-11T08:44:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280502+0000", 2024-09-11T08:44:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10626126499999999 2024-09-11T08:44:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280606+0000", 2024-09-11T08:44:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103496 2024-09-11T08:44:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:66225 7.0 e99/88 7:01cb4ea1:::benchmark_data_smithi092_84565_object66224:head v 99'4220, mlcod=99'4217)", 2024-09-11T08:44:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.176937+0000", 2024-09-11T08:44:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 174.356460513, 2024-09-11T08:44:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103855564, 2024-09-11T08:44:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.176937+0000", 2024-09-11T08:44:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.176937+0000", 2024-09-11T08:44:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.176936+0000", 2024-09-11T08:44:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:44:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.176985+0000", 2024-09-11T08:44:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.939e-05 2024-09-11T08:44:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274110+0000", 2024-09-11T08:44:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.097124391000000004 2024-09-11T08:44:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274120+0000", 2024-09-11T08:44:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0302999999999999e-05 2024-09-11T08:44:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274544+0000", 2024-09-11T08:44:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000424151 2024-09-11T08:44:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274563+0000", 2024-09-11T08:44:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.876e-05 2024-09-11T08:44:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280720+0000", 2024-09-11T08:44:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061573000000000001 2024-09-11T08:44:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280793+0000", 2024-09-11T08:44:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2708999999999998e-05 2024-09-11T08:44:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:66217 7.2 e99/88 7:49541d7a:::benchmark_data_smithi092_84565_object66216:head v 99'4138, mlcod=99'4135)", 2024-09-11T08:44:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.178474+0000", 2024-09-11T08:44:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 174.35492344299999, 2024-09-11T08:44:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102289936, 2024-09-11T08:44:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.178474+0000", 2024-09-11T08:44:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.178474+0000", 2024-09-11T08:44:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.178473+0000", 2024-09-11T08:44:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:44:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.178504+0000", 2024-09-11T08:44:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0511999999999999e-05 2024-09-11T08:44:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274100+0000", 2024-09-11T08:44:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.095596001999999999 2024-09-11T08:44:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274117+0000", 2024-09-11T08:44:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6860000000000001e-05 2024-09-11T08:44:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274161+0000", 2024-09-11T08:44:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4122000000000002e-05 2024-09-11T08:44:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274188+0000", 2024-09-11T08:44:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6631999999999998e-05 2024-09-11T08:44:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280611+0000", 2024-09-11T08:44:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064237349999999999 2024-09-11T08:44:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280764+0000", 2024-09-11T08:44:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015312600000000001 2024-09-11T08:44:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:66238 7.9 e99/88 7:939fc3d5:::benchmark_data_smithi092_84565_object66237:head v 99'4158, mlcod=99'4156)", 2024-09-11T08:44:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.188360+0000", 2024-09-11T08:44:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 174.34503812899999, 2024-09-11T08:44:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.088677141000000001, 2024-09-11T08:44:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.188360+0000", 2024-09-11T08:44:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.188360+0000", 2024-09-11T08:44:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.188358+0000", 2024-09-11T08:44:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:44:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.188387+0000", 2024-09-11T08:44:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8728999999999999e-05 2024-09-11T08:44:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274139+0000", 2024-09-11T08:44:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.085752140000000004 2024-09-11T08:44:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274148+0000", 2024-09-11T08:44:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2480000000000002e-06 2024-09-11T08:44:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274183+0000", 2024-09-11T08:44:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4448e-05 2024-09-11T08:44:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.274203+0000", 2024-09-11T08:44:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0472e-05 2024-09-11T08:44:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.276918+0000", 2024-09-11T08:44:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027144780000000002 2024-09-11T08:44:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.277037+0000", 2024-09-11T08:44:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011909300000000001 2024-09-11T08:44:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:11351 7.0 e114/88 7:065d7455:::benchmark_data_smithi092_84565_object85268:head v 114'11318, mlcod=114'11292)", 2024-09-11T08:44:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:00.380775+0000", 2024-09-11T08:44:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.15262323100001, 2024-09-11T08:44:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.090075323999999998, 2024-09-11T08:44:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.380775+0000", 2024-09-11T08:44:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.380775+0000", 2024-09-11T08:44:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.380774+0000", 2024-09-11T08:44:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:44:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.380782+0000", 2024-09-11T08:44:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2819999999999996e-06 2024-09-11T08:44:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.380795+0000", 2024-09-11T08:44:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2320000000000001e-05 2024-09-11T08:44:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.380800+0000", 2024-09-11T08:44:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4140000000000002e-06 2024-09-11T08:44:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.450309+0000", 2024-09-11T08:44:42.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069508728000000006 2024-09-11T08:44:42.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.450328+0000", 2024-09-11T08:44:42.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9626000000000001e-05 2024-09-11T08:44:42.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.470641+0000", 2024-09-11T08:44:42.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020312304 2024-09-11T08:44:42.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.470850+0000", 2024-09-11T08:44:42.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020926699999999999 2024-09-11T08:44:42.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.272 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.272 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:11352 7.0 e114/88 7:065e0608:::benchmark_data_smithi092_84565_object42644:head v 114'11319, mlcod=114'11292)", 2024-09-11T08:44:42.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:00.383061+0000", 2024-09-11T08:44:42.272 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.150337429, 2024-09-11T08:44:42.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.090758011, 2024-09-11T08:44:42.272 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.273 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.383061+0000", 2024-09-11T08:44:42.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.383061+0000", 2024-09-11T08:44:42.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.383059+0000", 2024-09-11T08:44:42.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:44:42.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.383071+0000", 2024-09-11T08:44:42.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2048999999999999e-05 2024-09-11T08:44:42.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.383104+0000", 2024-09-11T08:44:42.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2744999999999999e-05 2024-09-11T08:44:42.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.383112+0000", 2024-09-11T08:44:42.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5429999999999996e-06 2024-09-11T08:44:42.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.453128+0000", 2024-09-11T08:44:42.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.07001599 2024-09-11T08:44:42.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.453144+0000", 2024-09-11T08:44:42.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6725999999999999e-05 2024-09-11T08:44:42.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.473749+0000", 2024-09-11T08:44:42.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020604899999999999 2024-09-11T08:44:42.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.473819+0000", 2024-09-11T08:44:42.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9354e-05 2024-09-11T08:44:42.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.276 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:11353 7.0 e114/88 7:065e7579:::benchmark_data_smithi092_84565_object89724:head v 114'11320, mlcod=114'11292)", 2024-09-11T08:44:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:00.385966+0000", 2024-09-11T08:44:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.14743233599999, 2024-09-11T08:44:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.087933744999999994, 2024-09-11T08:44:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.385966+0000", 2024-09-11T08:44:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.385966+0000", 2024-09-11T08:44:42.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.385965+0000", 2024-09-11T08:44:42.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:44:42.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.385974+0000", 2024-09-11T08:44:42.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0559999999999994e-06 2024-09-11T08:44:42.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.385984+0000", 2024-09-11T08:44:42.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9569999999999997e-06 2024-09-11T08:44:42.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.385989+0000", 2024-09-11T08:44:42.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0919999999999997e-06 2024-09-11T08:44:42.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.455701+0000", 2024-09-11T08:44:42.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069712139000000006 2024-09-11T08:44:42.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.455719+0000", 2024-09-11T08:44:42.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7363e-05 2024-09-11T08:44:42.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.473823+0000", 2024-09-11T08:44:42.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018104331000000001 2024-09-11T08:44:42.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.473899+0000", 2024-09-11T08:44:42.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6463000000000003e-05 2024-09-11T08:44:42.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.281 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.281 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:11354 7.0 e114/88 7:0660af87:::benchmark_data_smithi092_84565_object55546:head v 114'11321, mlcod=114'11292)", 2024-09-11T08:44:42.281 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:00.390139+0000", 2024-09-11T08:44:42.281 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.14325933800001, 2024-09-11T08:44:42.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.086986796000000005, 2024-09-11T08:44:42.281 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.281 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.281 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.390139+0000", 2024-09-11T08:44:42.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.390139+0000", 2024-09-11T08:44:42.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.390138+0000", 2024-09-11T08:44:42.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:44:42.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.390147+0000", 2024-09-11T08:44:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8030000000000003e-06 2024-09-11T08:44:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.390162+0000", 2024-09-11T08:44:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5664000000000001e-05 2024-09-11T08:44:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.390168+0000", 2024-09-11T08:44:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8900000000000004e-06 2024-09-11T08:44:42.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.458321+0000", 2024-09-11T08:44:42.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068152775999999998 2024-09-11T08:44:42.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.458339+0000", 2024-09-11T08:44:42.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7886000000000001e-05 2024-09-11T08:44:42.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.476971+0000", 2024-09-11T08:44:42.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018632266000000001 2024-09-11T08:44:42.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.477125+0000", 2024-09-11T08:44:42.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015410000000000001 2024-09-11T08:44:42.285 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.285 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.285 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.285 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:143095 7.0 e114/88 7:00694a13:::benchmark_data_smithi092_84565_object17567:head v 114'11322, mlcod=114'11292)", 2024-09-11T08:44:42.285 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:00.390174+0000", 2024-09-11T08:44:42.285 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.143224317, 2024-09-11T08:44:42.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.087031933000000006, 2024-09-11T08:44:42.286 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.286 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.286 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.390174+0000", 2024-09-11T08:44:42.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.390174+0000", 2024-09-11T08:44:42.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.390173+0000", 2024-09-11T08:44:42.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:44:42.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.390177+0000", 2024-09-11T08:44:42.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.597e-06 2024-09-11T08:44:42.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.390182+0000", 2024-09-11T08:44:42.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2959999999999998e-06 2024-09-11T08:44:42.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.390185+0000", 2024-09-11T08:44:42.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1080000000000001e-06 2024-09-11T08:44:42.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.461046+0000", 2024-09-11T08:44:42.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.070860880000000001 2024-09-11T08:44:42.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.461095+0000", 2024-09-11T08:44:42.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9382000000000003e-05 2024-09-11T08:44:42.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.477129+0000", 2024-09-11T08:44:42.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016033269999999999 2024-09-11T08:44:42.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.477206+0000", 2024-09-11T08:44:42.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6889000000000002e-05 2024-09-11T08:44:42.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.289 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.290 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:143096 7.0 e114/88 7:057334a6:::benchmark_data_smithi092_84565_object17568:head v 114'11323, mlcod=114'11297)", 2024-09-11T08:44:42.290 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:00.391341+0000", 2024-09-11T08:44:42.290 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.142056744, 2024-09-11T08:44:42.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.086000943999999996, 2024-09-11T08:44:42.290 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.290 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.290 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.391341+0000", 2024-09-11T08:44:42.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.391341+0000", 2024-09-11T08:44:42.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.391341+0000", 2024-09-11T08:44:42.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:44:42.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.391349+0000", 2024-09-11T08:44:42.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8799999999999997e-06 2024-09-11T08:44:42.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.391360+0000", 2024-09-11T08:44:42.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0543000000000001e-05 2024-09-11T08:44:42.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.391365+0000", 2024-09-11T08:44:42.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9840000000000001e-06 2024-09-11T08:44:42.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.464080+0000", 2024-09-11T08:44:42.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.072714650000000006 2024-09-11T08:44:42.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.464138+0000", 2024-09-11T08:44:42.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8119999999999999e-05 2024-09-11T08:44:42.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.477214+0000", 2024-09-11T08:44:42.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013076005999999999 2024-09-11T08:44:42.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.477342+0000", 2024-09-11T08:44:42.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000128441 2024-09-11T08:44:42.294 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.294 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.294 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.294 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:143102 7.0 e114/88 7:0af89bd2:::benchmark_data_smithi092_84565_object17574:head v 114'11324, mlcod=114'11297)", 2024-09-11T08:44:42.294 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:00.394553+0000", 2024-09-11T08:44:42.294 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.138845092, 2024-09-11T08:44:42.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082883027999999997, 2024-09-11T08:44:42.294 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.294 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.294 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.394553+0000", 2024-09-11T08:44:42.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.394553+0000", 2024-09-11T08:44:42.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.394552+0000", 2024-09-11T08:44:42.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:44:42.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.394563+0000", 2024-09-11T08:44:42.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0919e-05 2024-09-11T08:44:42.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.394574+0000", 2024-09-11T08:44:42.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1452e-05 2024-09-11T08:44:42.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.394580+0000", 2024-09-11T08:44:42.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9939999999999997e-06 2024-09-11T08:44:42.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.467540+0000", 2024-09-11T08:44:42.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.072959428000000007 2024-09-11T08:44:42.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.467603+0000", 2024-09-11T08:44:42.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2710000000000001e-05 2024-09-11T08:44:42.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.477348+0000", 2024-09-11T08:44:42.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097452500000000004 2024-09-11T08:44:42.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.477436+0000", 2024-09-11T08:44:42.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8103999999999995e-05 2024-09-11T08:44:42.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.298 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.298 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:143161 7.0 e114/88 7:0ef15c43:::benchmark_data_smithi092_84565_object17633:head v 114'11329, mlcod=114'11308)", 2024-09-11T08:44:42.298 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:00.438539+0000", 2024-09-11T08:44:42.299 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.09485901399999, 2024-09-11T08:44:42.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.083585434, 2024-09-11T08:44:42.299 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.299 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.299 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.438539+0000", 2024-09-11T08:44:42.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.438539+0000", 2024-09-11T08:44:42.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.438538+0000", 2024-09-11T08:44:42.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:44:42.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.438548+0000", 2024-09-11T08:44:42.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.011e-05 2024-09-11T08:44:42.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.438566+0000", 2024-09-11T08:44:42.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7563000000000001e-05 2024-09-11T08:44:42.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.438572+0000", 2024-09-11T08:44:42.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.669e-06 2024-09-11T08:44:42.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.485490+0000", 2024-09-11T08:44:42.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046918287000000003 2024-09-11T08:44:42.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.485541+0000", 2024-09-11T08:44:42.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0674999999999998e-05 2024-09-11T08:44:42.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.522048+0000", 2024-09-11T08:44:42.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036506774999999998 2024-09-11T08:44:42.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:00.522124+0000", 2024-09-11T08:44:42.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6446999999999995e-05 2024-09-11T08:44:42.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.303 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.303 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:12214 7.0 e116/88 7:07f92677:::benchmark_data_smithi092_84565_object34834:head v 116'12181, mlcod=116'12151)", 2024-09-11T08:44:42.303 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:04.407146+0000", 2024-09-11T08:44:42.303 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.126252163000004, 2024-09-11T08:44:42.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082190006999999995, 2024-09-11T08:44:42.303 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.303 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.303 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.407146+0000", 2024-09-11T08:44:42.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.407146+0000", 2024-09-11T08:44:42.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.407145+0000", 2024-09-11T08:44:42.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:44:42.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.407153+0000", 2024-09-11T08:44:42.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4239999999999993e-06 2024-09-11T08:44:42.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.407199+0000", 2024-09-11T08:44:42.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5769000000000002e-05 2024-09-11T08:44:42.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.407208+0000", 2024-09-11T08:44:42.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6160000000000002e-06 2024-09-11T08:44:42.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.463685+0000", 2024-09-11T08:44:42.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056476942000000002 2024-09-11T08:44:42.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.463764+0000", 2024-09-11T08:44:42.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9043999999999997e-05 2024-09-11T08:44:42.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.489182+0000", 2024-09-11T08:44:42.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025418125999999999 2024-09-11T08:44:42.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.489336+0000", 2024-09-11T08:44:42.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154 2024-09-11T08:44:42.307 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.307 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.307 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.307 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:12215 7.0 e116/88 7:07fb2dff:::benchmark_data_smithi092_84565_object104976:head v 116'12182, mlcod=116'12151)", 2024-09-11T08:44:42.308 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:04.413158+0000", 2024-09-11T08:44:42.308 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.120239706000007, 2024-09-11T08:44:42.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.083466291999999997, 2024-09-11T08:44:42.308 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.308 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.308 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.413158+0000", 2024-09-11T08:44:42.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.413158+0000", 2024-09-11T08:44:42.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.413157+0000", 2024-09-11T08:44:42.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:44:42.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.413167+0000", 2024-09-11T08:44:42.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0356e-05 2024-09-11T08:44:42.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.413201+0000", 2024-09-11T08:44:42.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3937999999999999e-05 2024-09-11T08:44:42.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.413209+0000", 2024-09-11T08:44:42.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2509999999999997e-06 2024-09-11T08:44:42.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.467175+0000", 2024-09-11T08:44:42.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053965878000000002 2024-09-11T08:44:42.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.467229+0000", 2024-09-11T08:44:42.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4802999999999997e-05 2024-09-11T08:44:42.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.496497+0000", 2024-09-11T08:44:42.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029267814 2024-09-11T08:44:42.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.496625+0000", 2024-09-11T08:44:42.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000127391 2024-09-11T08:44:42.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.312 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.312 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:146846 7.0 e116/88 7:0bef7b16:::benchmark_data_smithi092_84565_object21318:head v 116'12183, mlcod=116'12163)", 2024-09-11T08:44:42.312 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:04.413216+0000", 2024-09-11T08:44:42.312 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.120182091000004, 2024-09-11T08:44:42.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.083531543, 2024-09-11T08:44:42.312 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.312 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.312 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.413216+0000", 2024-09-11T08:44:42.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.413216+0000", 2024-09-11T08:44:42.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.413215+0000", 2024-09-11T08:44:42.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:44:42.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.413220+0000", 2024-09-11T08:44:42.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5800000000000002e-06 2024-09-11T08:44:42.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.413226+0000", 2024-09-11T08:44:42.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-11T08:44:42.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.413229+0000", 2024-09-11T08:44:42.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0790000000000001e-06 2024-09-11T08:44:42.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.474517+0000", 2024-09-11T08:44:42.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061288271999999998 2024-09-11T08:44:42.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.479767+0000", 2024-09-11T08:44:42.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052494280000000004 2024-09-11T08:44:42.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.496627+0000", 2024-09-11T08:44:42.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016860818 2024-09-11T08:44:42.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:04.496747+0000", 2024-09-11T08:44:42.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011996400000000001 2024-09-11T08:44:42.316 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.316 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.316 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.316 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:165672 7.b 7:da614d45:::benchmark_data_smithi092_84565_object40144:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T08:44:42.317 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:22.012630+0000", 2024-09-11T08:44:42.317 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 78.520767991, 2024-09-11T08:44:42.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103975978, 2024-09-11T08:44:42.317 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:44:42.317 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:44:42.317 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:44:42.317 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:44:42.317 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:44:42.317 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 165672 2024-09-11T08:44:42.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.317 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:44:42.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:44:42.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.012630+0000", 2024-09-11T08:44:42.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:44:42.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.012630+0000", 2024-09-11T08:44:42.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:44:42.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:44:42.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.012628+0000", 2024-09-11T08:44:42.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:44:42.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.012635+0000", 2024-09-11T08:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5649999999999998e-06 2024-09-11T08:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.012651+0000", 2024-09-11T08:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5962999999999999e-05 2024-09-11T08:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.012657+0000", 2024-09-11T08:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8889999999999999e-06 2024-09-11T08:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:44:42.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.012697+0000", 2024-09-11T08:44:42.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0478000000000001e-05 2024-09-11T08:44:42.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:44:42.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.012795+0000", 2024-09-11T08:44:42.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7565999999999994e-05 2024-09-11T08:44:42.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:44:42.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.012913+0000", 2024-09-11T08:44:42.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117626 2024-09-11T08:44:42.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:42.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.015346+0000", 2024-09-11T08:44:42.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024335189999999999 2024-09-11T08:44:42.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:44:42.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.015517+0000", 2024-09-11T08:44:42.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017104199999999999 2024-09-11T08:44:42.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:44:42.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.116373+0000", 2024-09-11T08:44:42.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100855918 2024-09-11T08:44:42.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:44:42.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.116490+0000", 2024-09-11T08:44:42.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117302 2024-09-11T08:44:42.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:44:42.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:44:42.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:44:42.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.116606+0000", 2024-09-11T08:44:42.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115575 2024-09-11T08:44:42.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.322 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:44:42.323 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:44:42.323 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:44:42.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:42.324 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:42.324 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:42.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:42.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:42.472 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:42.473 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:42.573 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:42.597 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:42.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:42.598 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:42.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:42.745 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:42.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:42.747 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:42.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:42.889 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:42.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:42.890 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:42.991 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:43.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T08:44:43.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:43.017 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:43.018 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:43.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:44:43.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:43.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:44:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:44:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:43.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:43.237 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:43.237 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:43.338 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:43.360 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:43.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:43.361 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:43.462 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:43.486 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:43.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:43.487 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:43.588 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:43.611 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:43.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:43.612 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:43.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:43.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:43.771 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:43.771 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:43.786 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:43.872 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:43.896 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:43.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:43.897 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:43.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:44.037 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:44.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:44.038 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:44.139 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:44.160 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:44.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:44.161 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:44.242 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:44 smithi092 ceph-mon[29647]: pgmap v238: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 232 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.39k op/s; 736/11703 objects misplaced (6.289%) 2024-09-11T08:44:44.243 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:44 smithi092 ceph-mon[29647]: osdmap e141: 8 total, 8 up, 8 in 2024-09-11T08:44:44.243 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:44 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2644289852' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T08:44:44.243 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:44 smithi092 ceph-mon[41114]: pgmap v238: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 232 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.39k op/s; 736/11703 objects misplaced (6.289%) 2024-09-11T08:44:44.243 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:44 smithi092 ceph-mon[41114]: osdmap e141: 8 total, 8 up, 8 in 2024-09-11T08:44:44.243 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:44 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2644289852' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T08:44:44.262 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:44.283 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:44.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:44.284 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:44.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:44.407 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:44.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:44.407 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:44.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:44.530 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:44.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:44.531 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:44.540 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:44.540 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:44.632 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:44.656 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:44.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:44.657 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:44 smithi145 ceph-mon[33384]: pgmap v238: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 232 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.39k op/s; 736/11703 objects misplaced (6.289%) 2024-09-11T08:44:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:44 smithi145 ceph-mon[33384]: osdmap e141: 8 total, 8 up, 8 in 2024-09-11T08:44:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:44 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2644289852' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T08:44:44.758 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:44.781 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T08:44:44.784 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:44.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:44.785 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:44.886 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:44.912 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:44.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:44.913 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:45.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:45.036 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:45.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:45.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:45.138 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:45.159 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:45.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:45.160 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:45.261 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:45.282 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:45.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:45.283 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:45.384 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:45.405 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:45.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:45.406 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:45.507 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:45.533 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:45.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:45.534 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:45.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:45.660 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:45.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:45.661 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:45.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:45.792 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:45.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:45.793 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:45.894 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:45.919 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:45.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:45.920 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:46.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:46.060 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:46.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:46.061 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:46.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:45 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2644289852' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T08:44:46.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:45 smithi092 ceph-mon[29647]: osdmap e142: 8 total, 8 up, 8 in 2024-09-11T08:44:46.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:45 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:46.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:46.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:45 smithi092 ceph-mon[29647]: pgmap v241: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 232 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 885 op/s; 736/11703 objects misplaced (6.289%) 2024-09-11T08:44:46.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:45 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2644289852' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T08:44:46.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:45 smithi092 ceph-mon[41114]: osdmap e142: 8 total, 8 up, 8 in 2024-09-11T08:44:46.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:45 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:46.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:46.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:45 smithi092 ceph-mon[41114]: pgmap v241: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 232 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 885 op/s; 736/11703 objects misplaced (6.289%) 2024-09-11T08:44:46.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:45 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2644289852' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T08:44:46.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:45 smithi145 ceph-mon[33384]: osdmap e142: 8 total, 8 up, 8 in 2024-09-11T08:44:46.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:45 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:46.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:46.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:45 smithi145 ceph-mon[33384]: pgmap v241: 148 pgs: 3 active+remapped+backfill_toofull, 145 active+clean; 232 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 885 op/s; 736/11703 objects misplaced (6.289%) 2024-09-11T08:44:46.162 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:46.186 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:46.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:46.187 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:46.288 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:46.311 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:46.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:46.312 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:46.412 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:46.446 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:46.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:46.447 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:46.548 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:46.582 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:46.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:46.583 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:46.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:44:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:44:46.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:46.730 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:46.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:46.731 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:46.736 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Removed 125525 objects 2024-09-11T08:44:46.737 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Clean up completed and total clean up time :125.026 2024-09-11T08:44:46.739 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 1 1 2024-09-11T08:44:46.833 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:46.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:46.855 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:46.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:46.956 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:46.977 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:46.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:46.978 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:47.040 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:46 smithi092 ceph-mon[29647]: osdmap e143: 8 total, 8 up, 8 in 2024-09-11T08:44:47.040 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:46 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:47.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:46 smithi092 ceph-mon[41114]: osdmap e143: 8 total, 8 up, 8 in 2024-09-11T08:44:47.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:46 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:47.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:47.100 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:47.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:47.101 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:47.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:46 smithi145 ceph-mon[33384]: osdmap e143: 8 total, 8 up, 8 in 2024-09-11T08:44:47.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:46 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:47.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:47.222 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:47.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:47.223 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:47.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:47.345 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:47.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:47.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:47.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:47.468 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:47.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:47.469 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:47.570 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:47.591 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:47.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:47.592 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:47.693 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:47.719 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:47.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:47.720 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:47.820 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:47.845 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:47.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:47.846 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:47.947 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:47.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:47.970 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:47.970 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:48.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:47 smithi092 ceph-mon[29647]: pgmap v243: 148 pgs: 148 active+clean; 29 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.20k op/s; 3.7 MiB/s, 59 objects/s recovering 2024-09-11T08:44:48.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:47 smithi092 ceph-mon[41114]: pgmap v243: 148 pgs: 148 active+clean; 29 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.20k op/s; 3.7 MiB/s, 59 objects/s recovering 2024-09-11T08:44:48.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:48.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:48.093 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:48.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:47 smithi145 ceph-mon[33384]: pgmap v243: 148 pgs: 148 active+clean; 29 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.20k op/s; 3.7 MiB/s, 59 objects/s recovering 2024-09-11T08:44:48.194 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:48.219 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:48.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:48.220 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:48.321 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:48.345 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:48.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:48.346 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:48.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:48.470 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:48.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:48.471 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:48.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:48.594 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:48.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:48.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:48.696 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:48.722 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:48.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:48.722 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:48.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:48.847 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:48.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:48.848 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:48.949 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:48.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:48.974 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:48.974 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:49.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:48 smithi092 ceph-mon[29647]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull) 2024-09-11T08:44:49.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:48 smithi092 ceph-mon[41114]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull) 2024-09-11T08:44:49.074 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:49.097 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:49.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:49.099 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:49.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:48 smithi145 ceph-mon[33384]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull) 2024-09-11T08:44:49.199 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:49.223 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:49.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:49.224 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:49.325 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:49.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:49.351 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:49.352 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:49.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:49.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:49.478 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:49.479 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:49.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:49.605 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:49.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:49.605 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:49.706 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:49.732 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:49.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:49.732 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:49.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:49.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:49.860 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:49.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:49.986 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:49.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:49.987 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:50.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:49 smithi092 ceph-mon[29647]: pgmap v244: 148 pgs: 148 active+clean; 29 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 540 op/s; 3.7 MiB/s, 59 objects/s recovering 2024-09-11T08:44:50.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:49 smithi092 ceph-mon[41114]: pgmap v244: 148 pgs: 148 active+clean; 29 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 540 op/s; 3.7 MiB/s, 59 objects/s recovering 2024-09-11T08:44:50.088 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:50.114 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:50.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:50.115 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:50.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:49 smithi145 ceph-mon[33384]: pgmap v244: 148 pgs: 148 active+clean; 29 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 540 op/s; 3.7 MiB/s, 59 objects/s recovering 2024-09-11T08:44:50.216 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:50.244 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:50.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:50.245 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:50.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:50.371 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:50.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:50.372 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:50.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:50.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:50.494 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:50.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:50.615 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:50.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:50.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:50.717 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:50.743 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:50.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:50.744 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:50.814 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:44:50.845 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:50.869 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:50.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:50.870 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:50.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:50.993 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:50.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:50.994 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:51.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:50 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4034612450' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:44:51.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:50 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:44:51.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:50 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4034612450' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:44:51.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:50 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:44:51.095 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:51.116 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:51.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:51.117 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:51.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:50 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4034612450' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:44:51.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:50 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:44:51.218 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:51.230 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-11T08:44:51.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-11T08:44:51.243 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:51.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:51.244 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:51.344 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:51.370 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:51.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:51.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:51.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:51.494 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:51.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:51.495 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:51.595 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:51.617 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:51.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:51.618 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:51.705 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:44:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:44:51] "GET /metrics HTTP/1.1" 200 224046 "" "Prometheus/2.43.0" 2024-09-11T08:44:51.719 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:51.741 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:51.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:51.742 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:51.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:51.866 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:51.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:51.867 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:51.968 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:51.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:51.990 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:51.990 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:52.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:51 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:44:52.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:51 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:44:52.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:51 smithi092 ceph-mon[29647]: osdmap e144: 8 total, 8 up, 8 in 2024-09-11T08:44:52.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:51 smithi092 ceph-mon[29647]: pgmap v246: 148 pgs: 148 active+clean; 17 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 544 op/s; 3.5 MiB/s, 56 objects/s recovering 2024-09-11T08:44:52.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:51 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:44:52.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:51 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:52.019 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:51 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:52.019 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:51 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:44:52.019 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:51 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:44:52.019 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:51 smithi092 ceph-mon[41114]: osdmap e144: 8 total, 8 up, 8 in 2024-09-11T08:44:52.020 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:51 smithi092 ceph-mon[41114]: pgmap v246: 148 pgs: 148 active+clean; 17 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 544 op/s; 3.5 MiB/s, 56 objects/s recovering 2024-09-11T08:44:52.020 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:51 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:44:52.020 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:51 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:52.020 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:51 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:52.039 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:44:52.039 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:44:52.039 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-11T08:44:52.039 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:44:52.091 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:52.112 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:52.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:52.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:52.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:51 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:44:52.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:51 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:44:52.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:51 smithi145 ceph-mon[33384]: osdmap e144: 8 total, 8 up, 8 in 2024-09-11T08:44:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:51 smithi145 ceph-mon[33384]: pgmap v246: 148 pgs: 148 active+clean; 17 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 544 op/s; 3.5 MiB/s, 56 objects/s recovering 2024-09-11T08:44:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:51 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:44:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:51 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:51 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:52.214 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:52.227 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:44:52.227 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:44:52.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:52.236 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:52.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:52.238 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:44:52.238 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:44:52.253 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:44:52.254 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:44:52.268 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:44:52.269 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:44:52.283 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:44:52.284 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:44:52.299 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:44:52.299 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:44:52.314 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:44:52.314 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:44:52.329 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:44:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:44:52.337 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:52.358 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:52.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:52.359 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:52.460 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:52.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:52.483 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:52.483 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:52.583 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:52.604 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:52.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:52.605 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:52.706 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:52.727 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:52.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:52.728 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:52.829 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:52.850 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:52.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:52.851 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:52.919 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:52 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2551599252' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:44:52.919 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:52 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2551599252' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:44:52.952 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:52.989 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:52.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:52.991 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:53.092 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:53.115 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:53.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:53.117 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:53.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:52 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2551599252' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:44:53.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:44:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:44:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:53.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:53.240 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:53.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:53.241 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:53.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:53.363 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:53.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:53.364 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:53.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:53.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:53.486 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:53.487 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:53.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:53.608 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:53.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:53.609 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:53.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:53.731 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:53.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:53.732 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:53.833 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:53.859 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:53.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:53.860 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:53.960 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:53.986 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:53.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:53.987 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:54.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:53 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:54.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:53 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:54.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:53 smithi092 ceph-mon[29647]: pgmap v247: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 464 op/s; 2.8 MiB/s, 44 objects/s recovering 2024-09-11T08:44:54.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:53 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:44:54.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:53 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:44:54.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:53 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:54.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:53 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:54.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:53 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:54.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:53 smithi092 ceph-mon[41114]: pgmap v247: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 464 op/s; 2.8 MiB/s, 44 objects/s recovering 2024-09-11T08:44:54.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:53 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:44:54.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:53 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:44:54.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:53 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:54.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:54.111 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:54.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:54.112 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:54.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-11T08:44:54.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:53 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:54.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:53 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:54.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:53 smithi145 ceph-mon[33384]: pgmap v247: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 464 op/s; 2.8 MiB/s, 44 objects/s recovering 2024-09-11T08:44:54.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:53 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:44:54.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:53 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:44:54.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:53 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:44:54.213 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:54.235 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:54.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:54.237 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:54.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:54.358 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:54.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:54.359 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:54.460 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:54.481 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:54.484 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:54.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:54.485 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:54.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:54.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:54.608 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:54.609 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:54.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:54.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:54.733 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:54.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:54.834 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:54.859 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:54.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:54.860 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:54.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:54.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:54.983 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:55.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:55.104 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:55.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:55.105 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:55.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:55.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:44:55.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:55.228 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:55.228 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:55.329 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:55.355 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:55.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:55.356 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:55.457 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:55.478 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:55.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:55.479 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:55.580 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:55.605 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:55.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:55.606 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:55.699 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:55.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:55.728 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:55.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:55.729 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:55.738 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:44:55.830 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:55.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:55.857 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:55.857 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:55.958 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:55.982 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:55.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:55.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:44:56.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:56.106 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:56.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:56.107 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:56.208 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:56.235 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:56.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:56.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:56.238 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:56 smithi092 ceph-mon[29647]: pgmap v248: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 397 op/s; 2.4 MiB/s, 38 objects/s recovering 2024-09-11T08:44:56.238 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:56 smithi092 ceph-mon[41114]: pgmap v248: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 397 op/s; 2.4 MiB/s, 38 objects/s recovering 2024-09-11T08:44:56.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:56.358 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:56.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:56.359 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:56.460 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:56.485 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:56.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:56.486 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:56.587 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:56.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:56.613 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:56.613 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:56.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:56 smithi145 ceph-mon[33384]: pgmap v248: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 397 op/s; 2.4 MiB/s, 38 objects/s recovering 2024-09-11T08:44:56.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:44:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:44:56.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:56.736 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:56.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:56.737 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:56.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:56.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:56.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:56.960 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:56.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:56.988 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:56.988 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:57.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:44:57.110 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:57.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:57.111 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:44:57.195 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.1 to 1 (10000) 2024-09-11T08:44:57.212 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:57.237 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:57.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:57.238 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:57.338 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:57.364 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:57.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:57.365 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:57.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:57.487 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:57.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:57.488 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:57.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:57 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:57.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:57 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1317152144' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T08:44:57.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:57 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:57.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:57 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1317152144' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T08:44:57.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:57.610 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:57.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:57.611 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:57.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:57 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:57.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:57 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1317152144' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T08:44:57.712 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:57.738 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:57.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:57.739 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:57.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:57.862 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:57.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:57.863 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:57.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:57.987 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:57.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:57.988 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:57.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 2 1 2024-09-11T08:44:58.088 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:58.113 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:58.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:58.114 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:58.215 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:58.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:58.236 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:58.236 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:58.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:58 smithi092 ceph-mon[29647]: pgmap v249: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 47 op/s 2024-09-11T08:44:58.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:58 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1317152144' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T08:44:58.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:58 smithi092 ceph-mon[29647]: osdmap e145: 8 total, 8 up, 8 in 2024-09-11T08:44:58.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:58 smithi092 ceph-mon[41114]: pgmap v249: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 47 op/s 2024-09-11T08:44:58.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:58 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1317152144' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T08:44:58.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:58 smithi092 ceph-mon[41114]: osdmap e145: 8 total, 8 up, 8 in 2024-09-11T08:44:58.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:58.360 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:58.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:58.361 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:58.462 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:58.488 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:58.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:58.489 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:58.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:58 smithi145 ceph-mon[33384]: pgmap v249: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 47 op/s 2024-09-11T08:44:58.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:58 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1317152144' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T08:44:58.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:58 smithi145 ceph-mon[33384]: osdmap e145: 8 total, 8 up, 8 in 2024-09-11T08:44:58.589 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:58.615 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:58.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:58.616 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:58.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:44:58.738 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:58.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:58.739 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:44:58.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:58.861 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:58.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:58.862 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:58.963 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:44:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:58.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:58.986 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:44:59.026 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:59.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:59.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:59.109 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:59.110 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:59.202 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'unique_pool_0' removed 2024-09-11T08:44:59.211 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:59.237 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:59.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:59.238 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:59.339 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:59.363 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:59.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:59.364 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:59.465 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:44:59.499 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:59.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:59.501 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:44:59.552 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:44:59 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2409629796' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:44:59.553 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:44:59 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2409629796' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:44:59.602 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:44:59.624 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:59.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:59.625 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:44:59.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:44:59 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2409629796' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:44:59.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:44:59.749 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:44:59.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:59.750 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:44:59.851 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:44:59.874 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:59.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:59.875 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:44:59.951 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-11T08:44:59.952 INFO:tasks.radosbench:Beginning radosbench... 2024-09-11T08:44:59.953 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_1 2024-09-11T08:44:59.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool create unique_pool_1 16 2024-09-11T08:44:59.959 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:44:59.975 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:44:59.998 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:44:59.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:44:59.999 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:00.099 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:00.122 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:00.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:00.123 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:00.224 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:00.245 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:00.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:00.246 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:00.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:00.369 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:00.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:00.370 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:00.470 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:00.496 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:00.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:00.497 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:00.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:00 smithi092 ceph-mon[29647]: pgmap v251: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 56 op/s 2024-09-11T08:45:00.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:00 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2409629796' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:45:00.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:00 smithi092 ceph-mon[29647]: osdmap e146: 8 total, 8 up, 8 in 2024-09-11T08:45:00.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:00 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:45:00.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:00 smithi092 ceph-mon[41114]: pgmap v251: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 56 op/s 2024-09-11T08:45:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:00 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2409629796' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:45:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:00 smithi092 ceph-mon[41114]: osdmap e146: 8 total, 8 up, 8 in 2024-09-11T08:45:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:00 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:45:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:00.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:00.621 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:00.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:00.622 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:00.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:00 smithi145 ceph-mon[33384]: pgmap v251: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 56 op/s 2024-09-11T08:45:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:00 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2409629796' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:45:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:00 smithi145 ceph-mon[33384]: osdmap e146: 8 total, 8 up, 8 in 2024-09-11T08:45:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:00 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:45:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:00.724 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:00.749 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:00.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:00.750 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:00.851 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:00.876 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:00.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:00.877 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:00.978 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:01.004 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:01.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:01.005 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:01.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:01.128 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:01.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:01.129 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:01.230 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:01.255 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:01.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:01.256 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:01.356 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:01.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:01.382 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:01.382 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:01.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:01.506 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:01.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:01.507 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:01.521 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:45:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:45:01] "GET /metrics HTTP/1.1" 200 221598 "" "Prometheus/2.43.0" 2024-09-11T08:45:01.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:01.630 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:01.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:01.631 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:01.732 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:01.758 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:01.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:01.759 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:01.860 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:01.885 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:01.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:01.888 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:01.989 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:02.010 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:02.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:02.011 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:02.112 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:02.138 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:02.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:02.139 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:02.239 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:02.261 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:02.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:02.262 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:02.363 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:02.389 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:02.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:02.390 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:02.491 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:02.513 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:02 smithi092 ceph-mon[29647]: pgmap v253: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:45:02.514 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:02 smithi092 ceph-mon[41114]: pgmap v253: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:45:02.516 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:02.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:02.517 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:02.618 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:02.643 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:02.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:02.644 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:02 smithi145 ceph-mon[33384]: pgmap v253: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:45:02.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:02.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:02.768 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:02.869 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:02.895 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:02.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:02.896 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:02.997 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:03.022 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:03.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:03.023 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:03.124 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:03.148 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:03.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:03.149 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:03.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:45:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:45:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:03.250 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:03.275 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:03.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:03.276 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:03.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:03.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:03.400 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:03.400 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:03.501 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:03.526 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:03.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:03.527 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:03.628 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:03.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:03.651 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:03.651 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:03.753 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:03.779 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:03.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:03.781 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:03.882 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:03.907 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:03.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:03.908 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:04.009 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:04.035 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:04.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:04.036 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:04.137 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:04.164 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:04.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:04.166 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:04.266 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:04.291 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:04.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:04.292 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:04.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:04.417 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:04.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:04.418 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:04.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:04 smithi092 ceph-mon[29647]: pgmap v254: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:45:04.511 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:04 smithi092 ceph-mon[41114]: pgmap v254: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:45:04.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:04.542 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:04.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:04.543 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:04.644 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:04.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:04 smithi145 ceph-mon[33384]: pgmap v254: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:45:04.670 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:04.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:04.671 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:04.708 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:45:04.708 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:45:04.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:04.793 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:04.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:04.794 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:04.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:04.916 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:04.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:04.917 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:05.017 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:05.043 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:05.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:05.044 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:05.145 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:05.170 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:05.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:05.171 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:05.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:05.294 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:05.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:05.295 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:05.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:05.418 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:05.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:05.419 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:05.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-11T08:45:05.520 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:05.546 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:05.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:05.547 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:05.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:05.670 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:05.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:05.671 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:05.772 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:05.798 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:05.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:05.799 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:05.900 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:05.925 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:05.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:05.926 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:06.027 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:06.053 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:06.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:06.054 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:06.155 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:06.177 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:06.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:06.178 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:06.210 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:45:06.279 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:06.305 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:06.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:06.306 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:06.407 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:06.433 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:06.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:06.434 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:06.534 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:06.560 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:06.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:06.561 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:06.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:06 smithi092 ceph-mon[29647]: pgmap v255: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-11T08:45:06.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:06 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/307334255' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:45:06.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:06 smithi092 ceph-mon[41114]: pgmap v255: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-11T08:45:06.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:06 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/307334255' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:45:06.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:06 smithi145 ceph-mon[33384]: pgmap v255: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-11T08:45:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:06 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/307334255' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:45:06.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:45:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:45:06.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:06.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:06.685 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:06.685 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:06.786 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:06.812 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:06.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:06.813 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:06.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:06.936 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:06.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:06.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:07.038 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:07.063 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:07.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:07.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:07.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:07.187 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:07.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:07.187 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:07.288 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:07.315 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:07.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:07.315 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:07.416 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:07.442 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:07.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:07.443 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:07.443 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:45:07.443 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:45:07.445 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:45:07.445 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:45:07.451 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:07.462 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:45:07.462 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:45:07.478 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:45:07.479 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:45:07.479 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:07 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:45:07.479 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:07 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/307334255' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:45:07.479 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:07 smithi092 ceph-mon[29647]: osdmap e147: 8 total, 8 up, 8 in 2024-09-11T08:45:07.479 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:07 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:07.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:07 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:45:07.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:07 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/307334255' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:45:07.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:07 smithi092 ceph-mon[41114]: osdmap e147: 8 total, 8 up, 8 in 2024-09-11T08:45:07.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:07 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:07.491 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:07.496 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:45:07.496 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:45:07.513 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:45:07.513 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:45:07.529 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:45:07.529 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:45:07.543 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:07.544 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:45:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:45:07.565 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:07.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:07.566 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:07.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:07 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:45:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:07 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/307334255' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:45:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:07 smithi145 ceph-mon[33384]: osdmap e147: 8 total, 8 up, 8 in 2024-09-11T08:45:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:07 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:07.667 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:07.688 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:07.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:07.689 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:07.790 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:07.816 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:07.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:07.817 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:07.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:07.939 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:07.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:07.940 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:08.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:08.062 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:08.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:08.063 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:08.163 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:08.189 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:08.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:08.190 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:08.228 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.2 to 1 (10000) 2024-09-11T08:45:08.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:08.338 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[41114]: 3.1f scrub starts 2024-09-11T08:45:08.338 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[41114]: 3.1f scrub ok 2024-09-11T08:45:08.339 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[41114]: 4.12 scrub starts 2024-09-11T08:45:08.339 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[41114]: 4.12 scrub ok 2024-09-11T08:45:08.339 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[41114]: pgmap v257: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:45:08.339 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[41114]: 5.2 scrub starts 2024-09-11T08:45:08.339 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[41114]: 5.2 scrub ok 2024-09-11T08:45:08.339 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1939452047' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T08:45:08.339 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/188317325' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:45:08.339 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[41114]: 6.14 scrub starts 2024-09-11T08:45:08.339 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[41114]: 6.14 scrub ok 2024-09-11T08:45:08.339 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[41114]: 4.9 scrub starts 2024-09-11T08:45:08.339 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[41114]: 4.9 scrub ok 2024-09-11T08:45:08.340 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[29647]: 3.1f scrub starts 2024-09-11T08:45:08.340 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[29647]: 3.1f scrub ok 2024-09-11T08:45:08.340 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[29647]: 4.12 scrub starts 2024-09-11T08:45:08.340 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[29647]: 4.12 scrub ok 2024-09-11T08:45:08.340 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[29647]: pgmap v257: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:45:08.340 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[29647]: 5.2 scrub starts 2024-09-11T08:45:08.340 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[29647]: 5.2 scrub ok 2024-09-11T08:45:08.340 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1939452047' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T08:45:08.340 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/188317325' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:45:08.340 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[29647]: 6.14 scrub starts 2024-09-11T08:45:08.340 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[29647]: 6.14 scrub ok 2024-09-11T08:45:08.340 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[29647]: 4.9 scrub starts 2024-09-11T08:45:08.340 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:08 smithi092 ceph-mon[29647]: 4.9 scrub ok 2024-09-11T08:45:08.361 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:08.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:08.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:08.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:08.485 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:08.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:08.486 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:08.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:08.607 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:08.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:08.608 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:08.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:08 smithi145 ceph-mon[33384]: 3.1f scrub starts 2024-09-11T08:45:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:08 smithi145 ceph-mon[33384]: 3.1f scrub ok 2024-09-11T08:45:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:08 smithi145 ceph-mon[33384]: 4.12 scrub starts 2024-09-11T08:45:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:08 smithi145 ceph-mon[33384]: 4.12 scrub ok 2024-09-11T08:45:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:08 smithi145 ceph-mon[33384]: pgmap v257: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:45:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:08 smithi145 ceph-mon[33384]: 5.2 scrub starts 2024-09-11T08:45:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:08 smithi145 ceph-mon[33384]: 5.2 scrub ok 2024-09-11T08:45:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:08 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1939452047' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T08:45:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:08 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/188317325' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:45:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:08 smithi145 ceph-mon[33384]: 6.14 scrub starts 2024-09-11T08:45:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:08 smithi145 ceph-mon[33384]: 6.14 scrub ok 2024-09-11T08:45:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:08 smithi145 ceph-mon[33384]: 4.9 scrub starts 2024-09-11T08:45:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:08 smithi145 ceph-mon[33384]: 4.9 scrub ok 2024-09-11T08:45:08.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:08.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:08.730 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:08.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:08.852 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:08.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:08.853 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:08.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:08.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:45:08.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:08.976 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:08.976 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:09.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:09.098 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:09.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:09.099 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:09.200 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:09.226 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:09.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:09.227 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:09.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:09 smithi092 ceph-mon[41114]: 6.1 scrub starts 2024-09-11T08:45:09.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:09 smithi092 ceph-mon[41114]: 6.1 scrub ok 2024-09-11T08:45:09.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:09 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1939452047' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T08:45:09.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:09 smithi092 ceph-mon[41114]: osdmap e148: 8 total, 8 up, 8 in 2024-09-11T08:45:09.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:09 smithi092 ceph-mon[41114]: 5.1f scrub starts 2024-09-11T08:45:09.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:09 smithi092 ceph-mon[41114]: 5.1f scrub ok 2024-09-11T08:45:09.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:09 smithi092 ceph-mon[29647]: 6.1 scrub starts 2024-09-11T08:45:09.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:09 smithi092 ceph-mon[29647]: 6.1 scrub ok 2024-09-11T08:45:09.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:09 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1939452047' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T08:45:09.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:09 smithi092 ceph-mon[29647]: osdmap e148: 8 total, 8 up, 8 in 2024-09-11T08:45:09.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:09 smithi092 ceph-mon[29647]: 5.1f scrub starts 2024-09-11T08:45:09.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:09 smithi092 ceph-mon[29647]: 5.1f scrub ok 2024-09-11T08:45:09.328 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:09.353 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:09.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:09.354 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:09.454 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:09.479 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:09.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:09.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:09.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:09.603 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:09.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:09.604 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:09.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:09 smithi145 ceph-mon[33384]: 6.1 scrub starts 2024-09-11T08:45:09.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:09 smithi145 ceph-mon[33384]: 6.1 scrub ok 2024-09-11T08:45:09.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:09 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1939452047' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T08:45:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:09 smithi145 ceph-mon[33384]: osdmap e148: 8 total, 8 up, 8 in 2024-09-11T08:45:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:09 smithi145 ceph-mon[33384]: 5.1f scrub starts 2024-09-11T08:45:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:09 smithi145 ceph-mon[33384]: 5.1f scrub ok 2024-09-11T08:45:09.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:09.725 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:09.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:09.726 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:09.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:09.848 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:09.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:09.849 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:09.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:09.971 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:09.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:09.972 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:10.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:10.094 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:10.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:10.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:10.195 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:10.221 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:10.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:10.222 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:10.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:10.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:10.346 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:10.346 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:10.447 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:10.459 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:45:10.473 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:10.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:10.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:10.481 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:10 smithi092 ceph-mon[41114]: 5.16 scrub starts 2024-09-11T08:45:10.481 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:10 smithi092 ceph-mon[41114]: 5.16 scrub ok 2024-09-11T08:45:10.481 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:10 smithi092 ceph-mon[41114]: pgmap v259: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:45:10.481 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:10 smithi092 ceph-mon[41114]: 5.6 scrub starts 2024-09-11T08:45:10.482 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:10 smithi092 ceph-mon[41114]: 5.6 scrub ok 2024-09-11T08:45:10.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:10 smithi092 ceph-mon[29647]: 5.16 scrub starts 2024-09-11T08:45:10.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:10 smithi092 ceph-mon[29647]: 5.16 scrub ok 2024-09-11T08:45:10.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:10 smithi092 ceph-mon[29647]: pgmap v259: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:45:10.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:10 smithi092 ceph-mon[29647]: 5.6 scrub starts 2024-09-11T08:45:10.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:10 smithi092 ceph-mon[29647]: 5.6 scrub ok 2024-09-11T08:45:10.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:10.595 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:10.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:10.596 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:10.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:10 smithi145 ceph-mon[33384]: 5.16 scrub starts 2024-09-11T08:45:10.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:10 smithi145 ceph-mon[33384]: 5.16 scrub ok 2024-09-11T08:45:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:10 smithi145 ceph-mon[33384]: pgmap v259: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:45:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:10 smithi145 ceph-mon[33384]: 5.6 scrub starts 2024-09-11T08:45:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:10 smithi145 ceph-mon[33384]: 5.6 scrub ok 2024-09-11T08:45:10.697 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:10.719 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:10.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:10.720 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:10.820 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:10.841 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:10.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:10.842 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:10.942 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:10.964 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:10.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:10.965 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:11.066 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:11.092 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:11.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:11.093 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:11.194 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:11.216 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:11.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:11.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:11.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:11 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T08:45:11.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:11 smithi092 ceph-mon[29647]: 5.d scrub ok 2024-09-11T08:45:11.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:11 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T08:45:11.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:11 smithi092 ceph-mon[41114]: 5.d scrub ok 2024-09-11T08:45:11.318 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:11.342 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:11.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:11.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:11.444 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:11.468 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:11.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:11.469 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:11.570 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:11.575 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:45:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:45:11] "GET /metrics HTTP/1.1" 200 221598 "" "Prometheus/2.43.0" 2024-09-11T08:45:11.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 3 1 2024-09-11T08:45:11.594 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:11.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:11.595 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:11.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:11 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T08:45:11.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:11 smithi145 ceph-mon[33384]: 5.d scrub ok 2024-09-11T08:45:11.695 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:11.719 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:11.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:11.720 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:11.778 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:45:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-11T08:45:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-11T08:45:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:45:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:65018 7.e 7:7bf36b07:::benchmark_data_smithi092_84565_object65017:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:45:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:45.059616+0000", 2024-09-11T08:45:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 206.71631659100001, 2024-09-11T08:45:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.112304603, 2024-09-11T08:45:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 65018 2024-09-11T08:45:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059616+0000", 2024-09-11T08:45:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059616+0000", 2024-09-11T08:45:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059615+0000", 2024-09-11T08:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059639+0000", 2024-09-11T08:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4264999999999999e-05 2024-09-11T08:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059653+0000", 2024-09-11T08:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3691e-05 2024-09-11T08:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059659+0000", 2024-09-11T08:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.263e-06 2024-09-11T08:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062216+0000", 2024-09-11T08:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025571280000000001 2024-09-11T08:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062351+0000", 2024-09-11T08:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013464400000000001 2024-09-11T08:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062504+0000", 2024-09-11T08:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015295099999999999 2024-09-11T08:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.064861+0000", 2024-09-11T08:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023569789999999999 2024-09-11T08:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.069304+0000", 2024-09-11T08:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044433959999999996 2024-09-11T08:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.171603+0000", 2024-09-11T08:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10229874 2024-09-11T08:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.171882+0000", 2024-09-11T08:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027927099999999999 2024-09-11T08:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.171921+0000", 2024-09-11T08:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8711000000000003e-05 2024-09-11T08:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:65021 7.e 7:7eb86f3b:::benchmark_data_smithi092_84565_object65020:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:45:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:45.062298+0000", 2024-09-11T08:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 206.713634963, 2024-09-11T08:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11573973999999999, 2024-09-11T08:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 65021 2024-09-11T08:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062298+0000", 2024-09-11T08:45:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062298+0000", 2024-09-11T08:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062296+0000", 2024-09-11T08:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062323+0000", 2024-09-11T08:45:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6262999999999999e-05 2024-09-11T08:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062337+0000", 2024-09-11T08:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3883000000000001e-05 2024-09-11T08:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062343+0000", 2024-09-11T08:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.438e-06 2024-09-11T08:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.064881+0000", 2024-09-11T08:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025380279999999999 2024-09-11T08:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.065038+0000", 2024-09-11T08:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015697000000000001 2024-09-11T08:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.065171+0000", 2024-09-11T08:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013319499999999999 2024-09-11T08:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.069338+0000", 2024-09-11T08:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041663999999999998 2024-09-11T08:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.069603+0000", 2024-09-11T08:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026567600000000001 2024-09-11T08:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.177874+0000", 2024-09-11T08:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.108271144 2024-09-11T08:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.178003+0000", 2024-09-11T08:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012872900000000001 2024-09-11T08:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.178038+0000", 2024-09-11T08:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4688999999999999e-05 2024-09-11T08:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:122188 7.e 7:78531627:::benchmark_data_smithi092_84565_object122187:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:38.166756+0000", 2024-09-11T08:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 153.60917669899999, 2024-09-11T08:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.122095332, 2024-09-11T08:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122188 2024-09-11T08:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.166756+0000", 2024-09-11T08:45:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.166756+0000", 2024-09-11T08:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.166755+0000", 2024-09-11T08:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.166776+0000", 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1301000000000002e-05 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.166787+0000", 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0686999999999999e-05 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.166792+0000", 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4e-06 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.166850+0000", 2024-09-11T08:45:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7219000000000002e-05 2024-09-11T08:45:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.167337+0000", 2024-09-11T08:45:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048781699999999998 2024-09-11T08:45:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.167481+0000", 2024-09-11T08:45:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014400099999999999 2024-09-11T08:45:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.172215+0000", 2024-09-11T08:45:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047336779999999998 2024-09-11T08:45:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.173225+0000", 2024-09-11T08:45:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001009519 2024-09-11T08:45:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.288575+0000", 2024-09-11T08:45:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11535047900000001 2024-09-11T08:45:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.288718+0000", 2024-09-11T08:45:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014307 2024-09-11T08:45:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.288852+0000", 2024-09-11T08:45:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133609 2024-09-11T08:45:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:122203 7.e 7:778cd3fb:::benchmark_data_smithi092_84565_object122202:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:45:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:38.180983+0000", 2024-09-11T08:45:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 153.59495017200001, 2024-09-11T08:45:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.108224366, 2024-09-11T08:45:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122203 2024-09-11T08:45:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.180983+0000", 2024-09-11T08:45:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.180983+0000", 2024-09-11T08:45:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.180982+0000", 2024-09-11T08:45:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:45:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181008+0000", 2024-09-11T08:45:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6332000000000001e-05 2024-09-11T08:45:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181074+0000", 2024-09-11T08:45:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6341999999999996e-05 2024-09-11T08:45:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181080+0000", 2024-09-11T08:45:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2179999999999996e-06 2024-09-11T08:45:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181512+0000", 2024-09-11T08:45:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00043139699999999998 2024-09-11T08:45:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181772+0000", 2024-09-11T08:45:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026024700000000002 2024-09-11T08:45:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181931+0000", 2024-09-11T08:45:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015916199999999999 2024-09-11T08:45:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.189271+0000", 2024-09-11T08:45:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007340254 2024-09-11T08:45:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.189314+0000", 2024-09-11T08:45:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2324000000000003e-05 2024-09-11T08:45:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.288992+0000", 2024-09-11T08:45:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.099678418000000005 2024-09-11T08:45:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.289106+0000", 2024-09-11T08:45:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113891 2024-09-11T08:45:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.289207+0000", 2024-09-11T08:45:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101184 2024-09-11T08:45:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:122204 7.e 7:7bdd875c:::benchmark_data_smithi092_84565_object122203:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:45:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:38.181101+0000", 2024-09-11T08:45:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 153.59483207599999, 2024-09-11T08:45:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.109219345, 2024-09-11T08:45:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122204 2024-09-11T08:45:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181101+0000", 2024-09-11T08:45:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181101+0000", 2024-09-11T08:45:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181100+0000", 2024-09-11T08:45:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:45:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181121+0000", 2024-09-11T08:45:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0336e-05 2024-09-11T08:45:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181129+0000", 2024-09-11T08:45:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9880000000000004e-06 2024-09-11T08:45:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181133+0000", 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4800000000000003e-06 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.185885+0000", 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047516800000000003 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.185987+0000", 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010206599999999999 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.186113+0000", 2024-09-11T08:45:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012586499999999999 2024-09-11T08:45:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.189355+0000", 2024-09-11T08:45:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032418759999999999 2024-09-11T08:45:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.191836+0000", 2024-09-11T08:45:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002481254 2024-09-11T08:45:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.289325+0000", 2024-09-11T08:45:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.097488915999999995 2024-09-11T08:45:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.290282+0000", 2024-09-11T08:45:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00095695399999999999 2024-09-11T08:45:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.290320+0000", 2024-09-11T08:45:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8665999999999999e-05 2024-09-11T08:45:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:122211 7.e 7:7644088b:::benchmark_data_smithi092_84565_object122210:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:45:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:38.187420+0000", 2024-09-11T08:45:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 153.58851309799999, 2024-09-11T08:45:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103023819, 2024-09-11T08:45:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122211 2024-09-11T08:45:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.187420+0000", 2024-09-11T08:45:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.187420+0000", 2024-09-11T08:45:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.187419+0000", 2024-09-11T08:45:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:45:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.187437+0000", 2024-09-11T08:45:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8065999999999999e-05 2024-09-11T08:45:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.187447+0000", 2024-09-11T08:45:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7000000000000003e-06 2024-09-11T08:45:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.187451+0000", 2024-09-11T08:45:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4220000000000002e-06 2024-09-11T08:45:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.189375+0000", 2024-09-11T08:45:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019236629999999999 2024-09-11T08:45:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.189444+0000", 2024-09-11T08:45:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9036999999999996e-05 2024-09-11T08:45:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.189522+0000", 2024-09-11T08:45:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8615000000000005e-05 2024-09-11T08:45:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.191958+0000", 2024-09-11T08:45:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024359690000000001 2024-09-11T08:45:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.192499+0000", 2024-09-11T08:45:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054097100000000001 2024-09-11T08:45:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.290364+0000", 2024-09-11T08:45:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.097864958000000002 2024-09-11T08:45:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.290406+0000", 2024-09-11T08:45:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1990999999999998e-05 2024-09-11T08:45:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.290444+0000", 2024-09-11T08:45:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.748e-05 2024-09-11T08:45:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:122815 7.e 7:7bbdcb6b:::benchmark_data_smithi092_84565_object122814:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:45:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:38.925138+0000", 2024-09-11T08:45:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 152.85079485899999, 2024-09-11T08:45:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.110102964, 2024-09-11T08:45:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122815 2024-09-11T08:45:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.925138+0000", 2024-09-11T08:45:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.925138+0000", 2024-09-11T08:45:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.925138+0000", 2024-09-11T08:45:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:45:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.925160+0000", 2024-09-11T08:45:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2076e-05 2024-09-11T08:45:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.925167+0000", 2024-09-11T08:45:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2259999999999999e-06 2024-09-11T08:45:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.925171+0000", 2024-09-11T08:45:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9160000000000003e-06 2024-09-11T08:45:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.926590+0000", 2024-09-11T08:45:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014194100000000001 2024-09-11T08:45:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.926701+0000", 2024-09-11T08:45:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110402 2024-09-11T08:45:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.927767+0000", 2024-09-11T08:45:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010665869999999999 2024-09-11T08:45:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.930973+0000", 2024-09-11T08:45:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00320587 2024-09-11T08:45:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.931384+0000", 2024-09-11T08:45:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00041067099999999999 2024-09-11T08:45:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:39.035160+0000", 2024-09-11T08:45:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10377653000000001 2024-09-11T08:45:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:39.035207+0000", 2024-09-11T08:45:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6576000000000003e-05 2024-09-11T08:45:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:39.035241+0000", 2024-09-11T08:45:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4161e-05 2024-09-11T08:45:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:149662 7.e 7:7558e195:::benchmark_data_smithi092_84565_object24134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:45:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:06.838684+0000", 2024-09-11T08:45:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 124.937249267, 2024-09-11T08:45:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11249593500000001, 2024-09-11T08:45:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 149662 2024-09-11T08:45:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.838684+0000", 2024-09-11T08:45:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.838684+0000", 2024-09-11T08:45:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.838683+0000", 2024-09-11T08:45:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:45:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.838689+0000", 2024-09-11T08:45:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5949999999999998e-06 2024-09-11T08:45:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.838698+0000", 2024-09-11T08:45:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6949999999999995e-06 2024-09-11T08:45:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.838703+0000", 2024-09-11T08:45:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1950000000000002e-06 2024-09-11T08:45:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.839996+0000", 2024-09-11T08:45:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012930350000000001 2024-09-11T08:45:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.840087+0000", 2024-09-11T08:45:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0987000000000001e-05 2024-09-11T08:45:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.840184+0000", 2024-09-11T08:45:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6106000000000004e-05 2024-09-11T08:45:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850972+0000", 2024-09-11T08:45:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010788904 2024-09-11T08:45:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.851257+0000", 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028416200000000002 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.951095+0000", 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.099838475999999995 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.951140+0000", 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4660000000000003e-05 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.951180+0000", 2024-09-11T08:45:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0005000000000001e-05 2024-09-11T08:45:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:149668 7.e 7:72833b7f:::benchmark_data_smithi092_84565_object24140:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:45:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:06.842304+0000", 2024-09-11T08:45:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 124.933628622, 2024-09-11T08:45:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.115108186, 2024-09-11T08:45:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.828 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 149668 2024-09-11T08:45:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.842304+0000", 2024-09-11T08:45:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.842304+0000", 2024-09-11T08:45:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.842303+0000", 2024-09-11T08:45:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:45:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.842308+0000", 2024-09-11T08:45:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1900000000000003e-06 2024-09-11T08:45:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.842321+0000", 2024-09-11T08:45:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2679e-05 2024-09-11T08:45:11.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.842327+0000", 2024-09-11T08:45:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8799999999999996e-06 2024-09-11T08:45:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.847003+0000", 2024-09-11T08:45:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046758160000000002 2024-09-11T08:45:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.847105+0000", 2024-09-11T08:45:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010242999999999999 2024-09-11T08:45:11.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.847211+0000", 2024-09-11T08:45:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105655 2024-09-11T08:45:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.832 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.851306+0000", 2024-09-11T08:45:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040949929999999999 2024-09-11T08:45:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.854970+0000", 2024-09-11T08:45:11.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003663805 2024-09-11T08:45:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.957107+0000", 2024-09-11T08:45:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102137193 2024-09-11T08:45:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.957267+0000", 2024-09-11T08:45:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160106 2024-09-11T08:45:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.957413+0000", 2024-09-11T08:45:11.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014575100000000001 2024-09-11T08:45:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:149675 7.e 7:78c0dd9b:::benchmark_data_smithi092_84565_object24147:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:45:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:06.849986+0000", 2024-09-11T08:45:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 124.925947111, 2024-09-11T08:45:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11731897400000001, 2024-09-11T08:45:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 149675 2024-09-11T08:45:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.849986+0000", 2024-09-11T08:45:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.849986+0000", 2024-09-11T08:45:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.849985+0000", 2024-09-11T08:45:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:45:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.849989+0000", 2024-09-11T08:45:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5419999999999999e-06 2024-09-11T08:45:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.849997+0000", 2024-09-11T08:45:11.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0150000000000003e-06 2024-09-11T08:45:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850001+0000", 2024-09-11T08:45:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9509999999999999e-06 2024-09-11T08:45:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.851332+0000", 2024-09-11T08:45:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013308560000000001 2024-09-11T08:45:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.851428+0000", 2024-09-11T08:45:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6181999999999995e-05 2024-09-11T08:45:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.851640+0000", 2024-09-11T08:45:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021141000000000001 2024-09-11T08:45:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.951068+0000", 2024-09-11T08:45:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.099427984999999997 2024-09-11T08:45:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.951212+0000", 2024-09-11T08:45:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143979 2024-09-11T08:45:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.966788+0000", 2024-09-11T08:45:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01557618 2024-09-11T08:45:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.966972+0000", 2024-09-11T08:45:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018452400000000001 2024-09-11T08:45:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.967305+0000", 2024-09-11T08:45:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033248799999999998 2024-09-11T08:45:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:149676 7.e 7:74140a03:::benchmark_data_smithi092_84565_object24148:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:45:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:06.850577+0000", 2024-09-11T08:45:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 124.925355752, 2024-09-11T08:45:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11728794100000001, 2024-09-11T08:45:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.841 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 149676 2024-09-11T08:45:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850577+0000", 2024-09-11T08:45:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850577+0000", 2024-09-11T08:45:11.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850576+0000", 2024-09-11T08:45:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:45:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850582+0000", 2024-09-11T08:45:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3079999999999998e-06 2024-09-11T08:45:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850593+0000", 2024-09-11T08:45:11.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1253000000000001e-05 2024-09-11T08:45:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850599+0000", 2024-09-11T08:45:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.609e-06 2024-09-11T08:45:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.855054+0000", 2024-09-11T08:45:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044557570000000003 2024-09-11T08:45:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.855255+0000", 2024-09-11T08:45:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020054099999999999 2024-09-11T08:45:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.855366+0000", 2024-09-11T08:45:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111146 2024-09-11T08:45:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.951218+0000", 2024-09-11T08:45:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.095852437999999998 2024-09-11T08:45:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.951392+0000", 2024-09-11T08:45:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017315700000000001 2024-09-11T08:45:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.967493+0000", 2024-09-11T08:45:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016101358999999999 2024-09-11T08:45:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.967702+0000", 2024-09-11T08:45:11.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208851 2024-09-11T08:45:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.967865+0000", 2024-09-11T08:45:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016345199999999999 2024-09-11T08:45:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:149685 7.8 7:17ce0a5b:::benchmark_data_smithi092_84565_object24157:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:45:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:06.857814+0000", 2024-09-11T08:45:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 124.918119361, 2024-09-11T08:45:11.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10512039099999999, 2024-09-11T08:45:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 149685 2024-09-11T08:45:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.857814+0000", 2024-09-11T08:45:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.857814+0000", 2024-09-11T08:45:11.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.857813+0000", 2024-09-11T08:45:11.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:45:11.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.857818+0000", 2024-09-11T08:45:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8289999999999996e-06 2024-09-11T08:45:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.950984+0000", 2024-09-11T08:45:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.093165534999999994 2024-09-11T08:45:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.950997+0000", 2024-09-11T08:45:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3461e-05 2024-09-11T08:45:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.954783+0000", 2024-09-11T08:45:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037859780000000002 2024-09-11T08:45:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.954893+0000", 2024-09-11T08:45:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109236 2024-09-11T08:45:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,7", 2024-09-11T08:45:11.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.954996+0000", 2024-09-11T08:45:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010373 2024-09-11T08:45:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.958806+0000", 2024-09-11T08:45:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003809729 2024-09-11T08:45:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.962452+0000", 2024-09-11T08:45:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003645676 2024-09-11T08:45:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.962839+0000", 2024-09-11T08:45:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038698200000000002 2024-09-11T08:45:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.962893+0000", 2024-09-11T08:45:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4413999999999998e-05 2024-09-11T08:45:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.962934+0000", 2024-09-11T08:45:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1031000000000001e-05 2024-09-11T08:45:11.853 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:150318 7.e 7:75745c33:::benchmark_data_smithi092_84565_object24790:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:45:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:07.591034+0000", 2024-09-11T08:45:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 124.184899234, 2024-09-11T08:45:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20382811000000001, 2024-09-11T08:45:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.854 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 150318 2024-09-11T08:45:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591034+0000", 2024-09-11T08:45:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591034+0000", 2024-09-11T08:45:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591032+0000", 2024-09-11T08:45:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:45:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591039+0000", 2024-09-11T08:45:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4189999999999999e-06 2024-09-11T08:45:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591053+0000", 2024-09-11T08:45:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4749000000000001e-05 2024-09-11T08:45:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591059+0000", 2024-09-11T08:45:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.604e-06 2024-09-11T08:45:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591198+0000", 2024-09-11T08:45:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013907700000000001 2024-09-11T08:45:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591470+0000", 2024-09-11T08:45:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027224200000000001 2024-09-11T08:45:11.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.592889+0000", 2024-09-11T08:45:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00141906 2024-09-11T08:45:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.598327+0000", 2024-09-11T08:45:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054377890000000002 2024-09-11T08:45:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.600216+0000", 2024-09-11T08:45:11.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018887369999999999 2024-09-11T08:45:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.794706+0000", 2024-09-11T08:45:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.194490462 2024-09-11T08:45:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.794805+0000", 2024-09-11T08:45:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8669e-05 2024-09-11T08:45:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.794862+0000", 2024-09-11T08:45:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6833000000000003e-05 2024-09-11T08:45:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:150321 7.e 7:7128a19a:::benchmark_data_smithi092_84565_object24793:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:45:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:07.594313+0000", 2024-09-11T08:45:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 124.18161995, 2024-09-11T08:45:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20085424599999999, 2024-09-11T08:45:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 150321 2024-09-11T08:45:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.594313+0000", 2024-09-11T08:45:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.594313+0000", 2024-09-11T08:45:11.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.594311+0000", 2024-09-11T08:45:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:45:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.594318+0000", 2024-09-11T08:45:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-11T08:45:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.594329+0000", 2024-09-11T08:45:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1574e-05 2024-09-11T08:45:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.594335+0000", 2024-09-11T08:45:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6690000000000003e-06 2024-09-11T08:45:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.595742+0000", 2024-09-11T08:45:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001407373 2024-09-11T08:45:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.595820+0000", 2024-09-11T08:45:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7473999999999997e-05 2024-09-11T08:45:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.595905+0000", 2024-09-11T08:45:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5161999999999995e-05 2024-09-11T08:45:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.598524+0000", 2024-09-11T08:45:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026189059999999998 2024-09-11T08:45:11.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.600433+0000", 2024-09-11T08:45:11.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019096569999999999 2024-09-11T08:45:11.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.795062+0000", 2024-09-11T08:45:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19462868899999999 2024-09-11T08:45:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.795133+0000", 2024-09-11T08:45:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1321000000000004e-05 2024-09-11T08:45:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.795167+0000", 2024-09-11T08:45:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4019000000000001e-05 2024-09-11T08:45:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:165644 7.e 7:72bc5608:::benchmark_data_smithi092_84565_object40116:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T08:45:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:21.990537+0000", 2024-09-11T08:45:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 109.785396233, 2024-09-11T08:45:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.120823112, 2024-09-11T08:45:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.867 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 165644 2024-09-11T08:45:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990537+0000", 2024-09-11T08:45:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990537+0000", 2024-09-11T08:45:11.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990536+0000", 2024-09-11T08:45:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:45:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990542+0000", 2024-09-11T08:45:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.925e-06 2024-09-11T08:45:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990558+0000", 2024-09-11T08:45:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6025e-05 2024-09-11T08:45:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990564+0000", 2024-09-11T08:45:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1199999999999999e-06 2024-09-11T08:45:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.001643+0000", 2024-09-11T08:45:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011079435 2024-09-11T08:45:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.002405+0000", 2024-09-11T08:45:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00076212999999999997 2024-09-11T08:45:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.002507+0000", 2024-09-11T08:45:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010155800000000001 2024-09-11T08:45:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.012490+0000", 2024-09-11T08:45:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099833030000000007 2024-09-11T08:45:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.017752+0000", 2024-09-11T08:45:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052617610000000002 2024-09-11T08:45:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.111298+0000", 2024-09-11T08:45:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.093546291000000004 2024-09-11T08:45:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.111332+0000", 2024-09-11T08:45:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4039999999999999e-05 2024-09-11T08:45:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.111360+0000", 2024-09-11T08:45:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7724000000000001e-05 2024-09-11T08:45:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:165648 7.e 7:7cd3e667:::benchmark_data_smithi092_84565_object40120:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T08:45:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:21.990836+0000", 2024-09-11T08:45:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 109.785096831, 2024-09-11T08:45:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.13651323500000001, 2024-09-11T08:45:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 165648 2024-09-11T08:45:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990836+0000", 2024-09-11T08:45:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990836+0000", 2024-09-11T08:45:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990823+0000", 2024-09-11T08:45:11.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999866 2024-09-11T08:45:11.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990842+0000", 2024-09-11T08:45:11.876 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:11.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8722000000000001e-05 2024-09-11T08:45:11.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990857+0000", 2024-09-11T08:45:11.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5017999999999999e-05 2024-09-11T08:45:11.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990871+0000", 2024-09-11T08:45:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4874e-05 2024-09-11T08:45:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.005959+0000", 2024-09-11T08:45:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015087233 2024-09-11T08:45:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.008192+0000", 2024-09-11T08:45:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022335369999999998 2024-09-11T08:45:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.008315+0000", 2024-09-11T08:45:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012303499999999999 2024-09-11T08:45:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.012524+0000", 2024-09-11T08:45:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042090540000000003 2024-09-11T08:45:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.017757+0000", 2024-09-11T08:45:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052328549999999998 2024-09-11T08:45:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127227+0000", 2024-09-11T08:45:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.109469724 2024-09-11T08:45:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127288+0000", 2024-09-11T08:45:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0661000000000001e-05 2024-09-11T08:45:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127349+0000", 2024-09-11T08:45:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1885999999999997e-05 2024-09-11T08:45:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:165654 7.e 7:79193999:::benchmark_data_smithi092_84565_object40126:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T08:45:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:21.993884+0000", 2024-09-11T08:45:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 109.78204948600001, 2024-09-11T08:45:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.13364512100000001, 2024-09-11T08:45:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 165654 2024-09-11T08:45:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.993884+0000", 2024-09-11T08:45:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.993884+0000", 2024-09-11T08:45:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.993882+0000", 2024-09-11T08:45:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:45:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.993888+0000", 2024-09-11T08:45:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7699999999999998e-06 2024-09-11T08:45:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.993902+0000", 2024-09-11T08:45:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4158999999999999e-05 2024-09-11T08:45:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.993908+0000", 2024-09-11T08:45:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3940000000000004e-06 2024-09-11T08:45:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.012627+0000", 2024-09-11T08:45:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018719448 2024-09-11T08:45:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.014391+0000", 2024-09-11T08:45:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001763861 2024-09-11T08:45:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.014513+0000", 2024-09-11T08:45:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012184999999999999 2024-09-11T08:45:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.020541+0000", 2024-09-11T08:45:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060278110000000001 2024-09-11T08:45:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.022550+0000", 2024-09-11T08:45:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020094650000000002 2024-09-11T08:45:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127411+0000", 2024-09-11T08:45:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10486058299999999 2024-09-11T08:45:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127488+0000", 2024-09-11T08:45:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7458000000000003e-05 2024-09-11T08:45:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127529+0000", 2024-09-11T08:45:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0655999999999998e-05 2024-09-11T08:45:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:165664 7.e 7:7c58b4a4:::benchmark_data_smithi092_84565_object40136:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T08:45:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:22.006095+0000", 2024-09-11T08:45:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 109.769837885, 2024-09-11T08:45:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12157319699999999, 2024-09-11T08:45:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 165664 2024-09-11T08:45:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006095+0000", 2024-09-11T08:45:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006095+0000", 2024-09-11T08:45:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006095+0000", 2024-09-11T08:45:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:45:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006099+0000", 2024-09-11T08:45:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3479999999999997e-06 2024-09-11T08:45:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006119+0000", 2024-09-11T08:45:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0120999999999999e-05 2024-09-11T08:45:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006125+0000", 2024-09-11T08:45:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8950000000000003e-06 2024-09-11T08:45:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.017785+0000", 2024-09-11T08:45:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011659653000000001 2024-09-11T08:45:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.017884+0000", 2024-09-11T08:45:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9389000000000007e-05 2024-09-11T08:45:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.017993+0000", 2024-09-11T08:45:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109109 2024-09-11T08:45:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.022566+0000", 2024-09-11T08:45:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045726350000000002 2024-09-11T08:45:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.023501+0000", 2024-09-11T08:45:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093503 2024-09-11T08:45:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127566+0000", 2024-09-11T08:45:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104065012 2024-09-11T08:45:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127635+0000", 2024-09-11T08:45:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9608000000000002e-05 2024-09-11T08:45:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127668+0000", 2024-09-11T08:45:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3065000000000003e-05 2024-09-11T08:45:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:165665 7.e 7:72a055f1:::benchmark_data_smithi092_84565_object40137:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T08:45:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:22.006131+0000", 2024-09-11T08:45:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 109.76980159999999, 2024-09-11T08:45:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.121650015, 2024-09-11T08:45:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 165665 2024-09-11T08:45:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006131+0000", 2024-09-11T08:45:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006131+0000", 2024-09-11T08:45:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006131+0000", 2024-09-11T08:45:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:45:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006134+0000", 2024-09-11T08:45:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0259999999999999e-06 2024-09-11T08:45:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006147+0000", 2024-09-11T08:45:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.294e-05 2024-09-11T08:45:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006158+0000", 2024-09-11T08:45:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1144999999999999e-05 2024-09-11T08:45:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.020564+0000", 2024-09-11T08:45:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014405545000000001 2024-09-11T08:45:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.020641+0000", 2024-09-11T08:45:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7546999999999995e-05 2024-09-11T08:45:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.020718+0000", 2024-09-11T08:45:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6656000000000006e-05 2024-09-11T08:45:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.022668+0000", 2024-09-11T08:45:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001950476 2024-09-11T08:45:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.023605+0000", 2024-09-11T08:45:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093641799999999995 2024-09-11T08:45:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127714+0000", 2024-09-11T08:45:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104109722 2024-09-11T08:45:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127751+0000", 2024-09-11T08:45:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6189999999999997e-05 2024-09-11T08:45:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127781+0000", 2024-09-11T08:45:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0880000000000002e-05 2024-09-11T08:45:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:242111 7.e 7:70cafa55:::benchmark_data_smithi092_84565_object116583:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-11T08:45:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:36.320509+0000", 2024-09-11T08:45:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.455423787999997, 2024-09-11T08:45:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.110695035, 2024-09-11T08:45:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:45:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:45:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:45:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:45:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:45:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 242111 2024-09-11T08:45:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:45:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:45:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320509+0000", 2024-09-11T08:45:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:45:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320509+0000", 2024-09-11T08:45:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:45:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:45:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320508+0000", 2024-09-11T08:45:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:45:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:45:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320514+0000", 2024-09-11T08:45:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1959999999999996e-06 2024-09-11T08:45:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:45:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320526+0000", 2024-09-11T08:45:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2131e-05 2024-09-11T08:45:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:45:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320533+0000", 2024-09-11T08:45:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0700000000000001e-06 2024-09-11T08:45:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:45:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320708+0000", 2024-09-11T08:45:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017503500000000001 2024-09-11T08:45:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:45:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320810+0000", 2024-09-11T08:45:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101578 2024-09-11T08:45:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:45:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320920+0000", 2024-09-11T08:45:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110511 2024-09-11T08:45:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:45:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.323603+0000", 2024-09-11T08:45:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002682612 2024-09-11T08:45:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.323679+0000", 2024-09-11T08:45:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6241000000000004e-05 2024-09-11T08:45:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:45:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430763+0000", 2024-09-11T08:45:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10708406400000001 2024-09-11T08:45:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:45:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430956+0000", 2024-09-11T08:45:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000192497 2024-09-11T08:45:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:45:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:45:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:45:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.431204+0000", 2024-09-11T08:45:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024852899999999999 2024-09-11T08:45:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:45:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:45:11.906 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:45:11.906 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:11.930 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:11.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:11.931 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:12.031 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:12.056 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:12.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:12.057 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:12.158 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:12.182 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:12.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:12.183 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:12.283 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:12.308 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:12.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:12.309 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:12.410 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:12.434 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:12.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:12.436 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:12.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:12 smithi092 ceph-mon[29647]: 3.c scrub starts 2024-09-11T08:45:12.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:12 smithi092 ceph-mon[29647]: 3.c scrub ok 2024-09-11T08:45:12.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:12 smithi092 ceph-mon[29647]: pgmap v260: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:12.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:12 smithi092 ceph-mon[29647]: 5.9 scrub starts 2024-09-11T08:45:12.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:12 smithi092 ceph-mon[29647]: 5.9 scrub ok 2024-09-11T08:45:12.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:12 smithi092 ceph-mon[41114]: 3.c scrub starts 2024-09-11T08:45:12.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:12 smithi092 ceph-mon[41114]: 3.c scrub ok 2024-09-11T08:45:12.522 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:12 smithi092 ceph-mon[41114]: pgmap v260: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:12.522 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:12 smithi092 ceph-mon[41114]: 5.9 scrub starts 2024-09-11T08:45:12.522 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:12 smithi092 ceph-mon[41114]: 5.9 scrub ok 2024-09-11T08:45:12.536 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:12.560 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:12.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:12.561 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:12.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:12 smithi145 ceph-mon[33384]: 3.c scrub starts 2024-09-11T08:45:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:12 smithi145 ceph-mon[33384]: 3.c scrub ok 2024-09-11T08:45:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:12 smithi145 ceph-mon[33384]: pgmap v260: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:12 smithi145 ceph-mon[33384]: 5.9 scrub starts 2024-09-11T08:45:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:12 smithi145 ceph-mon[33384]: 5.9 scrub ok 2024-09-11T08:45:12.684 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:12.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:12.686 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:12.787 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:12.811 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:12.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:12.812 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:12.913 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:12.938 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:12.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:12.939 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:13.039 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:13.063 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:13.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:13.065 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:13.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:45:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:45:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:13.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:13.188 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:13.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:13.189 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:13.290 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:13.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:13.315 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:13.315 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:13.416 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:13.437 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:13.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:13.438 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:13.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[29647]: 4.3 scrub starts 2024-09-11T08:45:13.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[29647]: 4.3 scrub ok 2024-09-11T08:45:13.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[29647]: 6.5 scrub starts 2024-09-11T08:45:13.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[29647]: 6.5 scrub ok 2024-09-11T08:45:13.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[29647]: 5.a scrub starts 2024-09-11T08:45:13.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[29647]: 5.a scrub ok 2024-09-11T08:45:13.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[29647]: 4.18 scrub starts 2024-09-11T08:45:13.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[29647]: 4.18 scrub ok 2024-09-11T08:45:13.453 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[41114]: 4.3 scrub starts 2024-09-11T08:45:13.453 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[41114]: 4.3 scrub ok 2024-09-11T08:45:13.453 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[41114]: 6.5 scrub starts 2024-09-11T08:45:13.453 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[41114]: 6.5 scrub ok 2024-09-11T08:45:13.453 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[41114]: 5.a scrub starts 2024-09-11T08:45:13.453 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[41114]: 5.a scrub ok 2024-09-11T08:45:13.453 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[41114]: 4.18 scrub starts 2024-09-11T08:45:13.453 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:13 smithi092 ceph-mon[41114]: 4.18 scrub ok 2024-09-11T08:45:13.539 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:13.563 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:13.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:13.564 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:13.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:13 smithi145 ceph-mon[33384]: 4.3 scrub starts 2024-09-11T08:45:13.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:13 smithi145 ceph-mon[33384]: 4.3 scrub ok 2024-09-11T08:45:13.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:13 smithi145 ceph-mon[33384]: 6.5 scrub starts 2024-09-11T08:45:13.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:13 smithi145 ceph-mon[33384]: 6.5 scrub ok 2024-09-11T08:45:13.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:13 smithi145 ceph-mon[33384]: 5.a scrub starts 2024-09-11T08:45:13.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:13 smithi145 ceph-mon[33384]: 5.a scrub ok 2024-09-11T08:45:13.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:13 smithi145 ceph-mon[33384]: 4.18 scrub starts 2024-09-11T08:45:13.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:13 smithi145 ceph-mon[33384]: 4.18 scrub ok 2024-09-11T08:45:13.664 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:13.687 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:13.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:13.688 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:13.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:13.810 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:13.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:13.811 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:13.912 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:13.935 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:13.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:13.936 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:14.037 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:14.061 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:14.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:14.063 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:14.163 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:14.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:14.186 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:14.187 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:14.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-11T08:45:14.250 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'unique_pool_1' created 2024-09-11T08:45:14.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:14.310 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:14.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:14.312 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:14.412 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:14.439 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:14.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:14.440 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:14.540 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:14.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[29647]: 4.2 scrub starts 2024-09-11T08:45:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[29647]: 4.2 scrub ok 2024-09-11T08:45:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[29647]: pgmap v261: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[29647]: 3.8 scrub starts 2024-09-11T08:45:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[29647]: 3.8 scrub ok 2024-09-11T08:45:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[29647]: 5.1a scrub starts 2024-09-11T08:45:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[29647]: 5.1a scrub ok 2024-09-11T08:45:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2136637171' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-11T08:45:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-11T08:45:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[41114]: 4.2 scrub starts 2024-09-11T08:45:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[41114]: 4.2 scrub ok 2024-09-11T08:45:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[41114]: pgmap v261: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[41114]: 3.8 scrub starts 2024-09-11T08:45:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[41114]: 3.8 scrub ok 2024-09-11T08:45:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[41114]: 5.1a scrub starts 2024-09-11T08:45:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[41114]: 5.1a scrub ok 2024-09-11T08:45:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2136637171' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-11T08:45:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:14 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-11T08:45:14.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:14.568 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:14.568 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:14 smithi145 ceph-mon[33384]: 4.2 scrub starts 2024-09-11T08:45:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:14 smithi145 ceph-mon[33384]: 4.2 scrub ok 2024-09-11T08:45:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:14 smithi145 ceph-mon[33384]: pgmap v261: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:14 smithi145 ceph-mon[33384]: 3.8 scrub starts 2024-09-11T08:45:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:14 smithi145 ceph-mon[33384]: 3.8 scrub ok 2024-09-11T08:45:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:14 smithi145 ceph-mon[33384]: 5.1a scrub starts 2024-09-11T08:45:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:14 smithi145 ceph-mon[33384]: 5.1a scrub ok 2024-09-11T08:45:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:14 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2136637171' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-11T08:45:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:14 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-11T08:45:14.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:14.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:14.691 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:14.691 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:14.792 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:14.813 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:14.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:14.814 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:14.915 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:14.944 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:14.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:14.945 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:15.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:15.068 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:15.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:15.069 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:15.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:15.193 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:15.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:15.194 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:15.294 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:15.322 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:15.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:15.323 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:15.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:15.445 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:15.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:15.446 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:15.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:45:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:45:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[29647]: 5.13 scrub starts 2024-09-11T08:45:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[29647]: 5.13 scrub ok 2024-09-11T08:45:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-11T08:45:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[29647]: osdmap e149: 8 total, 8 up, 8 in 2024-09-11T08:45:15.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[29647]: 3.e scrub starts 2024-09-11T08:45:15.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[29647]: 3.e scrub ok 2024-09-11T08:45:15.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:15.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[29647]: 3.3 scrub starts 2024-09-11T08:45:15.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[29647]: 3.3 scrub ok 2024-09-11T08:45:15.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:45:15.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:45:15.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[41114]: 5.13 scrub starts 2024-09-11T08:45:15.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[41114]: 5.13 scrub ok 2024-09-11T08:45:15.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-11T08:45:15.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[41114]: osdmap e149: 8 total, 8 up, 8 in 2024-09-11T08:45:15.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[41114]: 3.e scrub starts 2024-09-11T08:45:15.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[41114]: 3.e scrub ok 2024-09-11T08:45:15.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:15.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[41114]: 3.3 scrub starts 2024-09-11T08:45:15.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:15 smithi092 ceph-mon[41114]: 3.3 scrub ok 2024-09-11T08:45:15.568 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:15.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:15.569 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:15.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:15 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:45:15.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:15 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:45:15.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:15 smithi145 ceph-mon[33384]: 5.13 scrub starts 2024-09-11T08:45:15.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:15 smithi145 ceph-mon[33384]: 5.13 scrub ok 2024-09-11T08:45:15.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:15 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-11T08:45:15.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:15 smithi145 ceph-mon[33384]: osdmap e149: 8 total, 8 up, 8 in 2024-09-11T08:45:15.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:15 smithi145 ceph-mon[33384]: 3.e scrub starts 2024-09-11T08:45:15.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:15 smithi145 ceph-mon[33384]: 3.e scrub ok 2024-09-11T08:45:15.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:15.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:15 smithi145 ceph-mon[33384]: 3.3 scrub starts 2024-09-11T08:45:15.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:15 smithi145 ceph-mon[33384]: 3.3 scrub ok 2024-09-11T08:45:15.670 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:15.695 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:15.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:15.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:15.797 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:15.806 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:15.822 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:15.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:15.823 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:15.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:15.945 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:15.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:15.946 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:16.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:16.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:16.069 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:16.069 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:16.170 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:16.195 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:16.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:16.196 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:16.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2024-09-11T08:45:16.296 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:16.322 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:16.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:16.323 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:16.423 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:16.448 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:16.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:16.450 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:16.478 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[29647]: 3.16 scrub starts 2024-09-11T08:45:16.478 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[29647]: 3.16 scrub ok 2024-09-11T08:45:16.479 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[29647]: 6.16 scrub starts 2024-09-11T08:45:16.479 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[29647]: 6.16 scrub ok 2024-09-11T08:45:16.479 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[29647]: 3.0 scrub starts 2024-09-11T08:45:16.479 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[29647]: 3.0 scrub ok 2024-09-11T08:45:16.479 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[29647]: pgmap v263: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:45:16.479 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[29647]: 6.c scrub starts 2024-09-11T08:45:16.479 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[29647]: 6.c scrub ok 2024-09-11T08:45:16.479 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[29647]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:45:16.479 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[29647]: osdmap e150: 8 total, 8 up, 8 in 2024-09-11T08:45:16.479 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[29647]: 3.15 scrub starts 2024-09-11T08:45:16.479 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[29647]: 3.15 scrub ok 2024-09-11T08:45:16.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[41114]: 3.16 scrub starts 2024-09-11T08:45:16.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[41114]: 3.16 scrub ok 2024-09-11T08:45:16.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[41114]: 6.16 scrub starts 2024-09-11T08:45:16.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[41114]: 6.16 scrub ok 2024-09-11T08:45:16.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[41114]: 3.0 scrub starts 2024-09-11T08:45:16.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[41114]: 3.0 scrub ok 2024-09-11T08:45:16.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[41114]: pgmap v263: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:45:16.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[41114]: 6.c scrub starts 2024-09-11T08:45:16.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[41114]: 6.c scrub ok 2024-09-11T08:45:16.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[41114]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:45:16.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[41114]: osdmap e150: 8 total, 8 up, 8 in 2024-09-11T08:45:16.480 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[41114]: 3.15 scrub starts 2024-09-11T08:45:16.481 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:16 smithi092 ceph-mon[41114]: 3.15 scrub ok 2024-09-11T08:45:16.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:16.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:16.574 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:16.574 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:16.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:45:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:45:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:16 smithi145 ceph-mon[33384]: 3.16 scrub starts 2024-09-11T08:45:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:16 smithi145 ceph-mon[33384]: 3.16 scrub ok 2024-09-11T08:45:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:16 smithi145 ceph-mon[33384]: 6.16 scrub starts 2024-09-11T08:45:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:16 smithi145 ceph-mon[33384]: 6.16 scrub ok 2024-09-11T08:45:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:16 smithi145 ceph-mon[33384]: 3.0 scrub starts 2024-09-11T08:45:16.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:16 smithi145 ceph-mon[33384]: 3.0 scrub ok 2024-09-11T08:45:16.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:16 smithi145 ceph-mon[33384]: pgmap v263: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:45:16.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:16 smithi145 ceph-mon[33384]: 6.c scrub starts 2024-09-11T08:45:16.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:16 smithi145 ceph-mon[33384]: 6.c scrub ok 2024-09-11T08:45:16.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:16 smithi145 ceph-mon[33384]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:45:16.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:16 smithi145 ceph-mon[33384]: osdmap e150: 8 total, 8 up, 8 in 2024-09-11T08:45:16.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:16 smithi145 ceph-mon[33384]: 3.15 scrub starts 2024-09-11T08:45:16.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:16 smithi145 ceph-mon[33384]: 3.15 scrub ok 2024-09-11T08:45:16.675 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:16.700 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:16.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:16.701 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:16.802 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:16.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:16.828 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:16.829 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:16.892 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:45:16.892 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:45:16.894 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:45:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:45:16.909 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:45:16.909 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:45:16.924 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:45:16.924 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:45:16.929 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:16.940 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:45:16.940 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:45:16.954 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:16.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:16.955 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:16.955 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:45:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:45:16.971 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:45:16.971 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:45:16.986 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:45:16.986 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:45:17.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:17.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:17.079 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:17.079 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:17.180 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:17.204 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:17.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:17.205 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:17.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:17.327 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:17.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:17.328 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:17.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:17.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:17.451 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:17.452 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:17.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:17.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:17 smithi092 ceph-mon[41114]: 4.1d scrub starts 2024-09-11T08:45:17.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:17 smithi092 ceph-mon[41114]: 4.1d scrub ok 2024-09-11T08:45:17.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:17 smithi092 ceph-mon[41114]: osdmap e151: 8 total, 8 up, 8 in 2024-09-11T08:45:17.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:17 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:17.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:17 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2635527308' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:45:17.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:17 smithi092 ceph-mon[41114]: 5.1 scrub starts 2024-09-11T08:45:17.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:17 smithi092 ceph-mon[41114]: 5.1 scrub ok 2024-09-11T08:45:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:17 smithi092 ceph-mon[29647]: 4.1d scrub starts 2024-09-11T08:45:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:17 smithi092 ceph-mon[29647]: 4.1d scrub ok 2024-09-11T08:45:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:17 smithi092 ceph-mon[29647]: osdmap e151: 8 total, 8 up, 8 in 2024-09-11T08:45:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:17 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:17 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2635527308' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:45:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:17 smithi092 ceph-mon[29647]: 5.1 scrub starts 2024-09-11T08:45:17.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:17 smithi092 ceph-mon[29647]: 5.1 scrub ok 2024-09-11T08:45:17.574 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:17.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:17.575 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:17 smithi145 ceph-mon[33384]: 4.1d scrub starts 2024-09-11T08:45:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:17 smithi145 ceph-mon[33384]: 4.1d scrub ok 2024-09-11T08:45:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:17 smithi145 ceph-mon[33384]: osdmap e151: 8 total, 8 up, 8 in 2024-09-11T08:45:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:17 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:17 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2635527308' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:45:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:17 smithi145 ceph-mon[33384]: 5.1 scrub starts 2024-09-11T08:45:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:17 smithi145 ceph-mon[33384]: 5.1 scrub ok 2024-09-11T08:45:17.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:17.697 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:17.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:17.698 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:17.799 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:17.825 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:17.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:17.826 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:17.927 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:17.952 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:17.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:17.953 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:17.973 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:18.054 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:18.075 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:18.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:18.076 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:18.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:18.199 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:18.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:18.200 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:18.274 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:45:18.301 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:18.327 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:18.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:18.328 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:18.429 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:18.454 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:18.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:18.456 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:18.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:18 smithi092 ceph-mon[29647]: pgmap v266: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:18.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:18 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2564722223' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:45:18.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:18 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:45:18.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:18 smithi092 ceph-mon[29647]: 6.3 scrub starts 2024-09-11T08:45:18.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:18 smithi092 ceph-mon[29647]: 6.3 scrub ok 2024-09-11T08:45:18.459 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:18 smithi092 ceph-mon[41114]: pgmap v266: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:18.459 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:18 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2564722223' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:45:18.459 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:18 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:45:18.459 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:18 smithi092 ceph-mon[41114]: 6.3 scrub starts 2024-09-11T08:45:18.459 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:18 smithi092 ceph-mon[41114]: 6.3 scrub ok 2024-09-11T08:45:18.557 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:18.583 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:18.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:18.584 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:18.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:18 smithi145 ceph-mon[33384]: pgmap v266: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:18 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2564722223' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:45:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:18 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:45:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:18 smithi145 ceph-mon[33384]: 6.3 scrub starts 2024-09-11T08:45:18.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:18 smithi145 ceph-mon[33384]: 6.3 scrub ok 2024-09-11T08:45:18.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:18.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:18.708 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:18.708 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:18.809 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:18.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:18.834 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:18.835 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:18.935 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:18.957 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:18.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:18.958 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:19.059 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:19.085 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:19.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:19.085 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:19.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:19.208 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:19.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:19.209 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:19.310 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:19.337 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:19.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:19.338 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:19.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:19.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:19.462 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:19.563 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:19.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:19 smithi092 ceph-mon[29647]: 6.4 scrub starts 2024-09-11T08:45:19.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:19 smithi092 ceph-mon[29647]: 6.4 scrub ok 2024-09-11T08:45:19.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:19 smithi092 ceph-mon[29647]: 3.2 scrub starts 2024-09-11T08:45:19.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:19 smithi092 ceph-mon[29647]: 3.2 scrub ok 2024-09-11T08:45:19.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:19 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:45:19.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:19 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:45:19.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:19 smithi092 ceph-mon[29647]: osdmap e152: 8 total, 8 up, 8 in 2024-09-11T08:45:19.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:19 smithi092 ceph-mon[41114]: 6.4 scrub starts 2024-09-11T08:45:19.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:19 smithi092 ceph-mon[41114]: 6.4 scrub ok 2024-09-11T08:45:19.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:19 smithi092 ceph-mon[41114]: 3.2 scrub starts 2024-09-11T08:45:19.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:19 smithi092 ceph-mon[41114]: 3.2 scrub ok 2024-09-11T08:45:19.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:19 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:45:19.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:19 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:45:19.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:19 smithi092 ceph-mon[41114]: osdmap e152: 8 total, 8 up, 8 in 2024-09-11T08:45:19.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:19.584 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:19.585 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:19.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:19 smithi145 ceph-mon[33384]: 6.4 scrub starts 2024-09-11T08:45:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:19 smithi145 ceph-mon[33384]: 6.4 scrub ok 2024-09-11T08:45:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:19 smithi145 ceph-mon[33384]: 3.2 scrub starts 2024-09-11T08:45:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:19 smithi145 ceph-mon[33384]: 3.2 scrub ok 2024-09-11T08:45:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:19 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:45:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:19 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:45:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:19 smithi145 ceph-mon[33384]: osdmap e152: 8 total, 8 up, 8 in 2024-09-11T08:45:19.685 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:19.711 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:19.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:19.712 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:19.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:19.834 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:19.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:19.835 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:19.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:45:19.936 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:19.962 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:19.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:19.963 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:20.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:20.085 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:20.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:20.086 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:20.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:20.208 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:20.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:20.209 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:20.310 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:20.336 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:20.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:20.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:20.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:20.459 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:20.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:20.460 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:20.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:20.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:20 smithi092 ceph-mon[29647]: 4.f scrub starts 2024-09-11T08:45:20.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:20 smithi092 ceph-mon[29647]: 4.f scrub ok 2024-09-11T08:45:20.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:20 smithi092 ceph-mon[29647]: pgmap v268: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:45:20.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:20 smithi092 ceph-mon[29647]: 6.1c scrub starts 2024-09-11T08:45:20.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:20 smithi092 ceph-mon[29647]: 6.1c scrub ok 2024-09-11T08:45:20.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:20 smithi092 ceph-mon[41114]: 4.f scrub starts 2024-09-11T08:45:20.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:20 smithi092 ceph-mon[41114]: 4.f scrub ok 2024-09-11T08:45:20.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:20 smithi092 ceph-mon[41114]: pgmap v268: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:45:20.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:20 smithi092 ceph-mon[41114]: 6.1c scrub starts 2024-09-11T08:45:20.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:20 smithi092 ceph-mon[41114]: 6.1c scrub ok 2024-09-11T08:45:20.582 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:20.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:20.583 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:20 smithi145 ceph-mon[33384]: 4.f scrub starts 2024-09-11T08:45:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:20 smithi145 ceph-mon[33384]: 4.f scrub ok 2024-09-11T08:45:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:20 smithi145 ceph-mon[33384]: pgmap v268: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:45:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:20 smithi145 ceph-mon[33384]: 6.1c scrub starts 2024-09-11T08:45:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:20 smithi145 ceph-mon[33384]: 6.1c scrub ok 2024-09-11T08:45:20.683 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:20.709 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:20.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:20.712 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:20.813 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:20.838 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:20.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:20.839 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:20.939 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:20.964 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:20.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:20.965 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:21.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:21.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:21.089 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:21.089 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:21.190 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:21.213 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:21.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:21.214 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:21.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:21.336 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:21.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:21.337 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:21.386 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:21.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:21.459 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:21.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:21.460 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:21.561 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:21.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:45:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:45:21] "GET /metrics HTTP/1.1" 200 223854 "" "Prometheus/2.43.0" 2024-09-11T08:45:21.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:21 smithi092 ceph-mon[29647]: 5.14 scrub starts 2024-09-11T08:45:21.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:21 smithi092 ceph-mon[29647]: 5.14 scrub ok 2024-09-11T08:45:21.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:21 smithi092 ceph-mon[29647]: 3.f scrub starts 2024-09-11T08:45:21.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:21 smithi092 ceph-mon[29647]: 3.f scrub ok 2024-09-11T08:45:21.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:21 smithi092 ceph-mon[29647]: 3.1 scrub starts 2024-09-11T08:45:21.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:21 smithi092 ceph-mon[29647]: 3.1 scrub ok 2024-09-11T08:45:21.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:21 smithi092 ceph-mon[41114]: 5.14 scrub starts 2024-09-11T08:45:21.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:21 smithi092 ceph-mon[41114]: 5.14 scrub ok 2024-09-11T08:45:21.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:21 smithi092 ceph-mon[41114]: 3.f scrub starts 2024-09-11T08:45:21.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:21 smithi092 ceph-mon[41114]: 3.f scrub ok 2024-09-11T08:45:21.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:21 smithi092 ceph-mon[41114]: 3.1 scrub starts 2024-09-11T08:45:21.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:21 smithi092 ceph-mon[41114]: 3.1 scrub ok 2024-09-11T08:45:21.586 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:21.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:21.587 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:21.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:21 smithi145 ceph-mon[33384]: 5.14 scrub starts 2024-09-11T08:45:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:21 smithi145 ceph-mon[33384]: 5.14 scrub ok 2024-09-11T08:45:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:21 smithi145 ceph-mon[33384]: 3.f scrub starts 2024-09-11T08:45:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:21 smithi145 ceph-mon[33384]: 3.f scrub ok 2024-09-11T08:45:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:21 smithi145 ceph-mon[33384]: 3.1 scrub starts 2024-09-11T08:45:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:21 smithi145 ceph-mon[33384]: 3.1 scrub ok 2024-09-11T08:45:21.689 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:21.691 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:45:21.714 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:21.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:21.715 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:21.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:21.837 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:21.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:21.838 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:21.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:21.960 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:21.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:21.961 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:22.062 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:22.085 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:22.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:22.086 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:22.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:22.208 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:22.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:22.209 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T08:45:22.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:22.331 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:22.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:22.332 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:22.432 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:22.459 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:22.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:22.459 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:22.560 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:45:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:45:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[29647]: 4.14 deep-scrub starts 2024-09-11T08:45:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[29647]: 4.14 deep-scrub ok 2024-09-11T08:45:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[29647]: pgmap v269: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:45:22.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[29647]: osdmap e153: 8 total, 8 up, 8 in 2024-09-11T08:45:22.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[29647]: 2.0 scrub starts 2024-09-11T08:45:22.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[29647]: 2.0 scrub ok 2024-09-11T08:45:22.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2141164977' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T08:45:22.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T08:45:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:45:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:45:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[41114]: 4.14 deep-scrub starts 2024-09-11T08:45:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[41114]: 4.14 deep-scrub ok 2024-09-11T08:45:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[41114]: pgmap v269: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:45:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[41114]: osdmap e153: 8 total, 8 up, 8 in 2024-09-11T08:45:22.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[41114]: 2.0 scrub starts 2024-09-11T08:45:22.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[41114]: 2.0 scrub ok 2024-09-11T08:45:22.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2141164977' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T08:45:22.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:22 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T08:45:22.586 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:22.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:22.587 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:22.619 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:45:22.620 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:45:22.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:22 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:45:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:22 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:45:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:22 smithi145 ceph-mon[33384]: 4.14 deep-scrub starts 2024-09-11T08:45:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:22 smithi145 ceph-mon[33384]: 4.14 deep-scrub ok 2024-09-11T08:45:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:22 smithi145 ceph-mon[33384]: pgmap v269: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:45:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:22 smithi145 ceph-mon[33384]: osdmap e153: 8 total, 8 up, 8 in 2024-09-11T08:45:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:22 smithi145 ceph-mon[33384]: 2.0 scrub starts 2024-09-11T08:45:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:22 smithi145 ceph-mon[33384]: 2.0 scrub ok 2024-09-11T08:45:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:22 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2141164977' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T08:45:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:22 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T08:45:22.688 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:22.709 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:22.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:22.710 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:22.811 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:22.836 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:22.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:22.837 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:22.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:22.960 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:22.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:22.961 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:23.062 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:23.087 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:23.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:23.089 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:23.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:45:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:45:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:23.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:23.211 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:23.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:23.212 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:23.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:23.335 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:23.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:23.336 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:23.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-11T08:45:23.370 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:23.374 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:23.437 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:23.458 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:23.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:23.459 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:23.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:23.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[41114]: 3.17 scrub starts 2024-09-11T08:45:23.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[41114]: 3.17 scrub ok 2024-09-11T08:45:23.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[41114]: 5.12 scrub starts 2024-09-11T08:45:23.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[41114]: 5.12 scrub ok 2024-09-11T08:45:23.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[41114]: 6.6 scrub starts 2024-09-11T08:45:23.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[41114]: 6.6 scrub ok 2024-09-11T08:45:23.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[41114]: 5.1c scrub starts 2024-09-11T08:45:23.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[41114]: 5.1c scrub ok 2024-09-11T08:45:23.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T08:45:23.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[41114]: osdmap e154: 8 total, 8 up, 8 in 2024-09-11T08:45:23.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[41114]: 5.11 scrub starts 2024-09-11T08:45:23.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[41114]: 5.11 scrub ok 2024-09-11T08:45:23.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[29647]: 3.17 scrub starts 2024-09-11T08:45:23.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[29647]: 3.17 scrub ok 2024-09-11T08:45:23.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[29647]: 5.12 scrub starts 2024-09-11T08:45:23.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[29647]: 5.12 scrub ok 2024-09-11T08:45:23.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[29647]: 6.6 scrub starts 2024-09-11T08:45:23.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[29647]: 6.6 scrub ok 2024-09-11T08:45:23.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[29647]: 5.1c scrub starts 2024-09-11T08:45:23.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[29647]: 5.1c scrub ok 2024-09-11T08:45:23.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T08:45:23.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[29647]: osdmap e154: 8 total, 8 up, 8 in 2024-09-11T08:45:23.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[29647]: 5.11 scrub starts 2024-09-11T08:45:23.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:23 smithi092 ceph-mon[29647]: 5.11 scrub ok 2024-09-11T08:45:23.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:23.582 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:23.582 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:23.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:23 smithi145 ceph-mon[33384]: 3.17 scrub starts 2024-09-11T08:45:23.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:23 smithi145 ceph-mon[33384]: 3.17 scrub ok 2024-09-11T08:45:23.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:23 smithi145 ceph-mon[33384]: 5.12 scrub starts 2024-09-11T08:45:23.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:23 smithi145 ceph-mon[33384]: 5.12 scrub ok 2024-09-11T08:45:23.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:23 smithi145 ceph-mon[33384]: 6.6 scrub starts 2024-09-11T08:45:23.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:23 smithi145 ceph-mon[33384]: 6.6 scrub ok 2024-09-11T08:45:23.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:23 smithi145 ceph-mon[33384]: 5.1c scrub starts 2024-09-11T08:45:23.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:23 smithi145 ceph-mon[33384]: 5.1c scrub ok 2024-09-11T08:45:23.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:23 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T08:45:23.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:23 smithi145 ceph-mon[33384]: osdmap e154: 8 total, 8 up, 8 in 2024-09-11T08:45:23.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:23 smithi145 ceph-mon[33384]: 5.11 scrub starts 2024-09-11T08:45:23.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:23 smithi145 ceph-mon[33384]: 5.11 scrub ok 2024-09-11T08:45:23.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:23.705 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:23.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:23.706 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:23.807 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:23.832 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:23.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:23.833 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:23.934 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:23.959 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:23.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:23.960 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:23.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 4 1 2024-09-11T08:45:24.060 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:24.082 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:24.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:24.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:24.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:24.206 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:24.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:24.207 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:24.308 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:24.329 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:24.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:24.330 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:24.431 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:24.458 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:24.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:24.459 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:24.559 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:24.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:24 smithi092 ceph-mon[29647]: pgmap v272: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:24.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:24 smithi092 ceph-mon[29647]: 5.17 scrub starts 2024-09-11T08:45:24.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:24 smithi092 ceph-mon[29647]: 5.17 scrub ok 2024-09-11T08:45:24.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:24 smithi092 ceph-mon[29647]: 6.12 scrub starts 2024-09-11T08:45:24.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:24 smithi092 ceph-mon[29647]: 6.12 scrub ok 2024-09-11T08:45:24.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:24 smithi092 ceph-mon[29647]: 6.1e scrub starts 2024-09-11T08:45:24.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:24 smithi092 ceph-mon[29647]: 6.1e scrub ok 2024-09-11T08:45:24.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:24 smithi092 ceph-mon[41114]: pgmap v272: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:24.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:24 smithi092 ceph-mon[41114]: 5.17 scrub starts 2024-09-11T08:45:24.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:24 smithi092 ceph-mon[41114]: 5.17 scrub ok 2024-09-11T08:45:24.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:24 smithi092 ceph-mon[41114]: 6.12 scrub starts 2024-09-11T08:45:24.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:24 smithi092 ceph-mon[41114]: 6.12 scrub ok 2024-09-11T08:45:24.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:24 smithi092 ceph-mon[41114]: 6.1e scrub starts 2024-09-11T08:45:24.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:24 smithi092 ceph-mon[41114]: 6.1e scrub ok 2024-09-11T08:45:24.585 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:24.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:24.586 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:24 smithi145 ceph-mon[33384]: pgmap v272: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:24 smithi145 ceph-mon[33384]: 5.17 scrub starts 2024-09-11T08:45:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:24 smithi145 ceph-mon[33384]: 5.17 scrub ok 2024-09-11T08:45:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:24 smithi145 ceph-mon[33384]: 6.12 scrub starts 2024-09-11T08:45:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:24 smithi145 ceph-mon[33384]: 6.12 scrub ok 2024-09-11T08:45:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:24 smithi145 ceph-mon[33384]: 6.1e scrub starts 2024-09-11T08:45:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:24 smithi145 ceph-mon[33384]: 6.1e scrub ok 2024-09-11T08:45:24.687 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:24.713 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:24.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:24.714 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:24.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:24.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:24.838 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:24.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:24.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:24.961 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:24.961 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:25.062 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:25.088 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:25.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:25.088 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:25.189 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:25.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:25.216 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:25.216 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:25.317 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:25.339 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:25.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:25.340 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:25.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:25.462 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:25.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:25.463 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:25.564 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:25.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:25 smithi092 ceph-mon[29647]: 6.1d scrub starts 2024-09-11T08:45:25.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:25 smithi092 ceph-mon[29647]: 6.1d scrub ok 2024-09-11T08:45:25.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:25 smithi092 ceph-mon[29647]: 6.1f scrub starts 2024-09-11T08:45:25.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:25 smithi092 ceph-mon[29647]: 6.1f scrub ok 2024-09-11T08:45:25.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:25 smithi092 ceph-mon[41114]: 6.1d scrub starts 2024-09-11T08:45:25.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:25 smithi092 ceph-mon[41114]: 6.1d scrub ok 2024-09-11T08:45:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:25 smithi092 ceph-mon[41114]: 6.1f scrub starts 2024-09-11T08:45:25.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:25 smithi092 ceph-mon[41114]: 6.1f scrub ok 2024-09-11T08:45:25.591 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:25.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:25.592 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:25.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:25 smithi145 ceph-mon[33384]: 6.1d scrub starts 2024-09-11T08:45:25.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:25 smithi145 ceph-mon[33384]: 6.1d scrub ok 2024-09-11T08:45:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:25 smithi145 ceph-mon[33384]: 6.1f scrub starts 2024-09-11T08:45:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:25 smithi145 ceph-mon[33384]: 6.1f scrub ok 2024-09-11T08:45:25.692 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:25.717 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:25.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:25.718 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:25.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:25.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:25.842 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:25.944 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:25.969 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:25.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:25.970 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:26.071 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:26.097 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:26.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:26.097 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:26.198 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:26.224 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:26.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:26.225 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:26.307 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:26.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:26.346 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:26.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:26.348 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:26.449 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:26.475 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:26.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:26.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:26 smithi092 ceph-mon[29647]: pgmap v273: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 745 B/s rd, 0 op/s 2024-09-11T08:45:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:26 smithi092 ceph-mon[29647]: 3.a deep-scrub starts 2024-09-11T08:45:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:26 smithi092 ceph-mon[29647]: 3.a deep-scrub ok 2024-09-11T08:45:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:26 smithi092 ceph-mon[29647]: 6.15 scrub starts 2024-09-11T08:45:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:26 smithi092 ceph-mon[29647]: 6.15 scrub ok 2024-09-11T08:45:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:26 smithi092 ceph-mon[29647]: 5.e deep-scrub starts 2024-09-11T08:45:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:26 smithi092 ceph-mon[29647]: 5.e deep-scrub ok 2024-09-11T08:45:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:26 smithi092 ceph-mon[41114]: pgmap v273: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 745 B/s rd, 0 op/s 2024-09-11T08:45:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:26 smithi092 ceph-mon[41114]: 3.a deep-scrub starts 2024-09-11T08:45:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:26 smithi092 ceph-mon[41114]: 3.a deep-scrub ok 2024-09-11T08:45:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:26 smithi092 ceph-mon[41114]: 6.15 scrub starts 2024-09-11T08:45:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:26 smithi092 ceph-mon[41114]: 6.15 scrub ok 2024-09-11T08:45:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:26 smithi092 ceph-mon[41114]: 5.e deep-scrub starts 2024-09-11T08:45:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:26 smithi092 ceph-mon[41114]: 5.e deep-scrub ok 2024-09-11T08:45:26.577 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:26.602 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:26.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:26.603 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:26.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:45:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:45:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:26 smithi145 ceph-mon[33384]: pgmap v273: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 745 B/s rd, 0 op/s 2024-09-11T08:45:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:26 smithi145 ceph-mon[33384]: 3.a deep-scrub starts 2024-09-11T08:45:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:26 smithi145 ceph-mon[33384]: 3.a deep-scrub ok 2024-09-11T08:45:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:26 smithi145 ceph-mon[33384]: 6.15 scrub starts 2024-09-11T08:45:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:26 smithi145 ceph-mon[33384]: 6.15 scrub ok 2024-09-11T08:45:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:26 smithi145 ceph-mon[33384]: 5.e deep-scrub starts 2024-09-11T08:45:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:26 smithi145 ceph-mon[33384]: 5.e deep-scrub ok 2024-09-11T08:45:26.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:26.726 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:26.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:26.727 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:26.828 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:26.853 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:26.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:26.854 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:26.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:26.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:26.977 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:27.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:27.100 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:27.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:27.101 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:27.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:27.224 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:27.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:27.225 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:27.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:27.346 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:27.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:27.348 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:27.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:27 smithi092 ceph-mon[29647]: 4.1f scrub starts 2024-09-11T08:45:27.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:27 smithi092 ceph-mon[29647]: 4.1f scrub ok 2024-09-11T08:45:27.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:27 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:27.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:27 smithi092 ceph-mon[29647]: 3.14 scrub starts 2024-09-11T08:45:27.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:27 smithi092 ceph-mon[29647]: 3.14 scrub ok 2024-09-11T08:45:27.406 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:27 smithi092 ceph-mon[41114]: 4.1f scrub starts 2024-09-11T08:45:27.406 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:27 smithi092 ceph-mon[41114]: 4.1f scrub ok 2024-09-11T08:45:27.406 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:27 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:27.406 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:27 smithi092 ceph-mon[41114]: 3.14 scrub starts 2024-09-11T08:45:27.406 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:27 smithi092 ceph-mon[41114]: 3.14 scrub ok 2024-09-11T08:45:27.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:27.470 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:27.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:27.471 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:27.572 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:27.598 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:27.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:27.599 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:27.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:27 smithi145 ceph-mon[33384]: 4.1f scrub starts 2024-09-11T08:45:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:27 smithi145 ceph-mon[33384]: 4.1f scrub ok 2024-09-11T08:45:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:27 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:27 smithi145 ceph-mon[33384]: 3.14 scrub starts 2024-09-11T08:45:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:27 smithi145 ceph-mon[33384]: 3.14 scrub ok 2024-09-11T08:45:27.699 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:27.725 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:27.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:27.726 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:27.827 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:27.852 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:27.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:27.853 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:27.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:27.975 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:27.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:27.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:28.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:28.098 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:28.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:28.099 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:28.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:28.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:28.222 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:28.308 INFO:teuthology.orchestra.run.smithi092.stderr:enabled application 'rados' on pool 'unique_pool_1' 2024-09-11T08:45:28.323 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:28.349 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:28.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:28.350 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:28.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:28.472 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:28.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:28.473 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:28.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:28 smithi145 ceph-mon[33384]: 4.4 scrub starts 2024-09-11T08:45:28.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:28 smithi145 ceph-mon[33384]: 4.4 scrub ok 2024-09-11T08:45:28.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:28 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:45:28.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:28 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:45:28.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:28 smithi145 ceph-mon[33384]: pgmap v274: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:28.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:28 smithi145 ceph-mon[33384]: osdmap e155: 8 total, 8 up, 8 in 2024-09-11T08:45:28.564 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:28 smithi145 ceph-mon[33384]: 3.13 deep-scrub starts 2024-09-11T08:45:28.564 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:28 smithi145 ceph-mon[33384]: 3.13 deep-scrub ok 2024-09-11T08:45:28.564 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:28 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/568717018' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:28.564 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:28 smithi145 ceph-mon[33384]: 3.1a scrub starts 2024-09-11T08:45:28.564 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:28 smithi145 ceph-mon[33384]: 3.1a scrub ok 2024-09-11T08:45:28.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[41114]: 4.4 scrub starts 2024-09-11T08:45:28.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[41114]: 4.4 scrub ok 2024-09-11T08:45:28.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:45:28.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:45:28.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[41114]: pgmap v274: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:28.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[41114]: osdmap e155: 8 total, 8 up, 8 in 2024-09-11T08:45:28.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[41114]: 3.13 deep-scrub starts 2024-09-11T08:45:28.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[41114]: 3.13 deep-scrub ok 2024-09-11T08:45:28.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/568717018' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:28.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[41114]: 3.1a scrub starts 2024-09-11T08:45:28.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[41114]: 3.1a scrub ok 2024-09-11T08:45:28.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[29647]: 4.4 scrub starts 2024-09-11T08:45:28.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[29647]: 4.4 scrub ok 2024-09-11T08:45:28.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:45:28.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:45:28.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[29647]: pgmap v274: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:28.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[29647]: osdmap e155: 8 total, 8 up, 8 in 2024-09-11T08:45:28.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[29647]: 3.13 deep-scrub starts 2024-09-11T08:45:28.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[29647]: 3.13 deep-scrub ok 2024-09-11T08:45:28.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/568717018' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:45:28.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[29647]: 3.1a scrub starts 2024-09-11T08:45:28.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:28 smithi092 ceph-mon[29647]: 3.1a scrub ok 2024-09-11T08:45:28.574 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:28.599 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:28.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:28.600 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:28.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:28.722 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:28.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:28.723 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:28.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:28.872 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:28.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:28.873 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:28.974 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:29.000 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:29.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:29.001 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:29.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:29.123 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:29.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:29.124 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:29.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:29.247 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:29.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:29.248 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:29.348 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:29.374 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:29.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:29.375 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:29.380 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[41114]: 4.7 scrub starts 2024-09-11T08:45:29.380 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[41114]: 4.7 scrub ok 2024-09-11T08:45:29.380 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[41114]: 3.10 deep-scrub starts 2024-09-11T08:45:29.380 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[41114]: 3.10 deep-scrub ok 2024-09-11T08:45:29.380 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[41114]: 6.11 scrub starts 2024-09-11T08:45:29.380 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[41114]: 6.11 scrub ok 2024-09-11T08:45:29.380 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/568717018' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:45:29.380 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[41114]: osdmap e156: 8 total, 8 up, 8 in 2024-09-11T08:45:29.380 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[29647]: 4.7 scrub starts 2024-09-11T08:45:29.380 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[29647]: 4.7 scrub ok 2024-09-11T08:45:29.381 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[29647]: 3.10 deep-scrub starts 2024-09-11T08:45:29.381 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[29647]: 3.10 deep-scrub ok 2024-09-11T08:45:29.381 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[29647]: 6.11 scrub starts 2024-09-11T08:45:29.381 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[29647]: 6.11 scrub ok 2024-09-11T08:45:29.381 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/568717018' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:45:29.381 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:29 smithi092 ceph-mon[29647]: osdmap e156: 8 total, 8 up, 8 in 2024-09-11T08:45:29.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:29.497 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:29.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:29.498 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:29.501 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:45:29.501 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:45:29.501 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:45:29.501 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:45:29.502 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:45:29.502 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:45:29.502 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:45:29.502 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:45:29.502 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:45:29.502 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:45:29.502 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:45:29.502 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:45:29.502 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:45:29.503 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:45:29.503 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:45:29.503 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:45:29.599 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:29.626 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:29.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:29.627 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:29.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:29 smithi145 ceph-mon[33384]: 4.7 scrub starts 2024-09-11T08:45:29.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:29 smithi145 ceph-mon[33384]: 4.7 scrub ok 2024-09-11T08:45:29.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:29 smithi145 ceph-mon[33384]: 3.10 deep-scrub starts 2024-09-11T08:45:29.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:29 smithi145 ceph-mon[33384]: 3.10 deep-scrub ok 2024-09-11T08:45:29.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:29 smithi145 ceph-mon[33384]: 6.11 scrub starts 2024-09-11T08:45:29.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:29 smithi145 ceph-mon[33384]: 6.11 scrub ok 2024-09-11T08:45:29.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:29 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/568717018' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:45:29.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:29 smithi145 ceph-mon[33384]: osdmap e156: 8 total, 8 up, 8 in 2024-09-11T08:45:29.727 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:29.752 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:29.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:29.753 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:29.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:29.876 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:29.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:29.877 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:29.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:29.999 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:30.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:30.000 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:30.101 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:30.109 DEBUG:teuthology.orchestra.run.smithi092:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_1 bench 90 write' 2024-09-11T08:45:30.112 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-11T08:45:30.112 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-11T08:45:30.127 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:30.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:30.128 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:30.156 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:hints = 1 2024-09-11T08:45:30.157 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-11T08:45:30.157 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Object prefix: benchmark_data_smithi092_117541 2024-09-11T08:45:30.157 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:45:30.157 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 0 0 0 0 0 0 - 0 2024-09-11T08:45:30.229 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:30.264 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:30.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:30.265 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:30.341 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:30.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:30.404 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:30.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:30.405 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:30.506 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:30.536 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:30.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:30.537 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:30.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[29647]: 5.15 scrub starts 2024-09-11T08:45:30.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[29647]: 5.15 scrub ok 2024-09-11T08:45:30.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[29647]: pgmap v277: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 745 B/s rd, 0 op/s 2024-09-11T08:45:30.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:45:30.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:45:30.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3193072855' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:45:30.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[29647]: 1.0 scrub starts 2024-09-11T08:45:30.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[29647]: 1.0 scrub ok 2024-09-11T08:45:30.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[29647]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:45:30.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:45:30.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[29647]: osdmap e157: 8 total, 8 up, 8 in 2024-09-11T08:45:30.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:45:30.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:30.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[41114]: 5.15 scrub starts 2024-09-11T08:45:30.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[41114]: 5.15 scrub ok 2024-09-11T08:45:30.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[41114]: pgmap v277: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 745 B/s rd, 0 op/s 2024-09-11T08:45:30.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:45:30.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:45:30.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3193072855' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:45:30.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[41114]: 1.0 scrub starts 2024-09-11T08:45:30.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[41114]: 1.0 scrub ok 2024-09-11T08:45:30.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[41114]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:45:30.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:45:30.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[41114]: osdmap e157: 8 total, 8 up, 8 in 2024-09-11T08:45:30.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:45:30.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:30.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:30.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:30 smithi145 ceph-mon[33384]: 5.15 scrub starts 2024-09-11T08:45:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:30 smithi145 ceph-mon[33384]: 5.15 scrub ok 2024-09-11T08:45:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:30 smithi145 ceph-mon[33384]: pgmap v277: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 745 B/s rd, 0 op/s 2024-09-11T08:45:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:30 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:45:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:30 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:45:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:30 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3193072855' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:45:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:30 smithi145 ceph-mon[33384]: 1.0 scrub starts 2024-09-11T08:45:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:30 smithi145 ceph-mon[33384]: 1.0 scrub ok 2024-09-11T08:45:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:30 smithi145 ceph-mon[33384]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:45:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:30 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:45:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:30 smithi145 ceph-mon[33384]: osdmap e157: 8 total, 8 up, 8 in 2024-09-11T08:45:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:30 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:45:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:30.677 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:30.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:30.678 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:30.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:30.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:30.821 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:30.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:30.962 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:30.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:30.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:31.064 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:31.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:31.097 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:31.098 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:31.157 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 1 16 3759 3743 233.927 233.938 0.00485353 0.00425725 2024-09-11T08:45:31.199 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:31.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:31.231 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:31.232 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:31.332 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:31.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:31.374 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:31.374 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:31.375 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:31 smithi092 ceph-mon[29647]: 3.6 scrub starts 2024-09-11T08:45:31.375 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:31 smithi092 ceph-mon[29647]: 6.e scrub starts 2024-09-11T08:45:31.375 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:31 smithi092 ceph-mon[29647]: 6.e scrub ok 2024-09-11T08:45:31.375 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:31 smithi092 ceph-mon[41114]: 3.6 scrub starts 2024-09-11T08:45:31.375 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:31 smithi092 ceph-mon[41114]: 6.e scrub starts 2024-09-11T08:45:31.375 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:31 smithi092 ceph-mon[41114]: 6.e scrub ok 2024-09-11T08:45:31.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:31.511 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:31.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:31.512 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:31.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:31.654 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:31.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:31.655 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:31.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:31 smithi145 ceph-mon[33384]: 3.6 scrub starts 2024-09-11T08:45:31.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:31 smithi145 ceph-mon[33384]: 6.e scrub starts 2024-09-11T08:45:31.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:31 smithi145 ceph-mon[33384]: 6.e scrub ok 2024-09-11T08:45:31.756 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:31.787 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:31.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:31.788 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:31.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:45:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:45:31] "GET /metrics HTTP/1.1" 200 223912 "" "Prometheus/2.43.0" 2024-09-11T08:45:31.889 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:31.930 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:31.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:31.932 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:32.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:32.076 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:32.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:32.077 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:32.157 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 2 16 7167 7151 223.454 213 0.00446524 0.00446449 2024-09-11T08:45:32.178 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:32.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:32.219 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:32.219 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:32.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:32.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:45:32.360 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:32.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:32.361 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:32.395 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:32 smithi092 ceph-mon[29647]: 3.6 scrub ok 2024-09-11T08:45:32.395 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:32 smithi092 ceph-mon[29647]: pgmap v279: 148 pgs: 148 active+clean; 84 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 14 MiB/s wr, 224 op/s 2024-09-11T08:45:32.395 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:32 smithi092 ceph-mon[29647]: 6.18 scrub starts 2024-09-11T08:45:32.396 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:32 smithi092 ceph-mon[29647]: 6.18 scrub ok 2024-09-11T08:45:32.396 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:32 smithi092 ceph-mon[41114]: 3.6 scrub ok 2024-09-11T08:45:32.396 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:32 smithi092 ceph-mon[41114]: pgmap v279: 148 pgs: 148 active+clean; 84 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 14 MiB/s wr, 224 op/s 2024-09-11T08:45:32.396 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:32 smithi092 ceph-mon[41114]: 6.18 scrub starts 2024-09-11T08:45:32.396 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:32 smithi092 ceph-mon[41114]: 6.18 scrub ok 2024-09-11T08:45:32.462 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:32.493 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:32.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:32.494 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:32.594 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:32.625 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:32.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:32.626 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:32.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:32 smithi145 ceph-mon[33384]: 3.6 scrub ok 2024-09-11T08:45:32.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:32 smithi145 ceph-mon[33384]: pgmap v279: 148 pgs: 148 active+clean; 84 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 14 MiB/s wr, 224 op/s 2024-09-11T08:45:32.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:32 smithi145 ceph-mon[33384]: 6.18 scrub starts 2024-09-11T08:45:32.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:32 smithi145 ceph-mon[33384]: 6.18 scrub ok 2024-09-11T08:45:32.727 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:32.756 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:32.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:32.758 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:32.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:32.897 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:32.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:32.898 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:32.999 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:33.031 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:33.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:33.032 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:33.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:33.154 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:33.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:33.155 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:33.159 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 3 16 10099 10083 210.047 183.25 0.00223848 0.00468338 2024-09-11T08:45:33.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:45:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:45:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:33.257 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:33.293 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:33.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:33.294 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:33.395 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:33.424 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:33.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:33.425 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:33.526 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:33.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:33.558 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:33.558 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:33.649 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:33 smithi092 ceph-mon[29647]: 3.b scrub starts 2024-09-11T08:45:33.649 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:33 smithi092 ceph-mon[29647]: 3.b scrub ok 2024-09-11T08:45:33.649 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:33 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2145886087' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:45:33.649 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:33 smithi092 ceph-mon[29647]: 2.1 scrub starts 2024-09-11T08:45:33.649 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:33 smithi092 ceph-mon[29647]: 2.1 scrub ok 2024-09-11T08:45:33.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:33 smithi092 ceph-mon[41114]: 3.b scrub starts 2024-09-11T08:45:33.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:33 smithi092 ceph-mon[41114]: 3.b scrub ok 2024-09-11T08:45:33.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:33 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2145886087' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:45:33.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:33 smithi092 ceph-mon[41114]: 2.1 scrub starts 2024-09-11T08:45:33.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:33 smithi092 ceph-mon[41114]: 2.1 scrub ok 2024-09-11T08:45:33.659 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:33 smithi145 ceph-mon[33384]: 3.b scrub starts 2024-09-11T08:45:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:33 smithi145 ceph-mon[33384]: 3.b scrub ok 2024-09-11T08:45:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:33 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2145886087' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:45:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:33 smithi145 ceph-mon[33384]: 2.1 scrub starts 2024-09-11T08:45:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:33 smithi145 ceph-mon[33384]: 2.1 scrub ok 2024-09-11T08:45:33.707 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:33.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:33.708 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:33.774 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:45:33.809 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:33.842 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:33.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:33.843 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:33.944 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:33.982 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:33.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:33.983 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:34.084 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:34.114 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:34.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:34.115 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:34.157 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 4 16 13131 13115 204.906 189.5 0.00446482 0.00487381 2024-09-11T08:45:34.216 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:34.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:34.258 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:34.258 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:34.359 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:34.390 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:34.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:34.391 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:34.414 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:34 smithi092 ceph-mon[29647]: pgmap v280: 148 pgs: 148 active+clean; 263 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 44 MiB/s wr, 700 op/s 2024-09-11T08:45:34.414 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:34 smithi092 ceph-mon[29647]: 6.b deep-scrub starts 2024-09-11T08:45:34.414 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:34 smithi092 ceph-mon[29647]: 6.b deep-scrub ok 2024-09-11T08:45:34.414 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:34 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:45:34.414 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:34 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2145886087' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:45:34.414 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:34 smithi092 ceph-mon[29647]: osdmap e158: 8 total, 8 up, 8 in 2024-09-11T08:45:34.414 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:34 smithi092 ceph-mon[41114]: pgmap v280: 148 pgs: 148 active+clean; 263 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 44 MiB/s wr, 700 op/s 2024-09-11T08:45:34.414 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:34 smithi092 ceph-mon[41114]: 6.b deep-scrub starts 2024-09-11T08:45:34.415 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:34 smithi092 ceph-mon[41114]: 6.b deep-scrub ok 2024-09-11T08:45:34.415 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:34 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:45:34.415 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:34 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2145886087' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:45:34.415 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:34 smithi092 ceph-mon[41114]: osdmap e158: 8 total, 8 up, 8 in 2024-09-11T08:45:34.492 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:34.524 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:34.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:34.525 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:34.626 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:34.657 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:34.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:34.658 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:34.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:34 smithi145 ceph-mon[33384]: pgmap v280: 148 pgs: 148 active+clean; 263 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 44 MiB/s wr, 700 op/s 2024-09-11T08:45:34.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:34 smithi145 ceph-mon[33384]: 6.b deep-scrub starts 2024-09-11T08:45:34.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:34 smithi145 ceph-mon[33384]: 6.b deep-scrub ok 2024-09-11T08:45:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:34 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:45:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:34 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2145886087' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:45:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:34 smithi145 ceph-mon[33384]: osdmap e158: 8 total, 8 up, 8 in 2024-09-11T08:45:34.759 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:34.790 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:34.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:34.791 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:34.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:34.928 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:34.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:34.929 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:35.030 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:35.080 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:35.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:35.081 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:35.158 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 5 15 16123 16108 201.306 187.062 0.00403398 0.0049609 2024-09-11T08:45:35.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:35.220 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:35.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:35.221 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:35.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:35.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:35.360 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:35.461 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:35.486 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:35.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:35.487 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:35.588 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:35.619 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:35.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:35.620 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:35.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:35.758 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:35.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:35.759 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:35.860 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:35.888 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:35.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:35.889 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:35.990 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:36.018 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:36.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:36.019 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:36.066 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:36.113 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:45:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:45:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:45:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:45:36.113 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:45:36.120 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:36.147 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:36.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:36.148 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:36.158 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 6 16 18718 18702 194.774 162.125 0.0047132 0.00512694 2024-09-11T08:45:36.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:36.291 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:36.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:36.292 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T08:45:36.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:36.431 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:36.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:36.433 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:36.457 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:45:36.461 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:36 smithi092 ceph-mon[29647]: pgmap v282: 148 pgs: 148 active+clean; 263 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 749 B/s rd, 38 MiB/s wr, 615 op/s 2024-09-11T08:45:36.461 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:36 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3920074398' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T08:45:36.461 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:36 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T08:45:36.462 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:36 smithi092 ceph-mon[41114]: pgmap v282: 148 pgs: 148 active+clean; 263 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 749 B/s rd, 38 MiB/s wr, 615 op/s 2024-09-11T08:45:36.462 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:36 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3920074398' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T08:45:36.462 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:36 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T08:45:36.533 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:36.561 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:36.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:36.562 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:36.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:45:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:45:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:36 smithi145 ceph-mon[33384]: pgmap v282: 148 pgs: 148 active+clean; 263 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 749 B/s rd, 38 MiB/s wr, 615 op/s 2024-09-11T08:45:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:36 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3920074398' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T08:45:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:36 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T08:45:36.663 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:36.704 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:36.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:36.706 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:36.807 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:36.837 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:36.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:36.838 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:36.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:36.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:36.981 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:36.982 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:37.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-11T08:45:37.082 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:37.110 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:37.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:37.111 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:37.158 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 7 16 21228 21212 189.357 156.875 0.00562727 0.00527474 2024-09-11T08:45:37.212 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:37.240 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:37.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:37.241 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:37.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:37.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:37.392 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:37.392 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:37.493 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:37.533 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:37.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:37.534 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:37.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:37 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T08:45:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:37 smithi145 ceph-mon[33384]: osdmap e159: 8 total, 8 up, 8 in 2024-09-11T08:45:37.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:37 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:37.672 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:37.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:37.673 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:37.774 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:37.804 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:37.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:37.805 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:37 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T08:45:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:37 smithi092 ceph-mon[29647]: osdmap e159: 8 total, 8 up, 8 in 2024-09-11T08:45:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:37 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:37 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T08:45:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:37 smithi092 ceph-mon[41114]: osdmap e159: 8 total, 8 up, 8 in 2024-09-11T08:45:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:37 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:37.906 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:37.935 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:37.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:37.936 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:38.025 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 5 1 2024-09-11T08:45:38.036 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:38.065 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:38.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:38.066 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:38.159 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 8 16 23715 23699 185.114 155.438 0.00421665 0.00539608 2024-09-11T08:45:38.167 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:38.196 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:38.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:38.197 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:38.298 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:38.340 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:38.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:38.341 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:38.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:38.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:38.482 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:38.583 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:38.625 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:38.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:38.626 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:38.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:38 smithi145 ceph-mon[33384]: pgmap v284: 148 pgs: 148 active+clean; 1.2 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.4 KiB/s rd, 162 MiB/s wr, 2.59k op/s 2024-09-11T08:45:38.727 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:38.759 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:38.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:38.760 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:38.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:38 smithi092 ceph-mon[29647]: pgmap v284: 148 pgs: 148 active+clean; 1.2 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.4 KiB/s rd, 162 MiB/s wr, 2.59k op/s 2024-09-11T08:45:38.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:38 smithi092 ceph-mon[41114]: pgmap v284: 148 pgs: 148 active+clean; 1.2 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.4 KiB/s rd, 162 MiB/s wr, 2.59k op/s 2024-09-11T08:45:38.861 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:38.889 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:38.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:38.890 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:38.991 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:39.011 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:39.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:39.012 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:39.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:39.152 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:39.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:39.153 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:39.158 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 9 16 25965 25949 180.17 140.625 0.00536373 0.00554434 2024-09-11T08:45:39.254 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:39.276 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:39.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:39.277 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:39.378 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:39.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:39.408 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:39.408 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:39.509 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:39.541 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:39.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:39.542 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:39.643 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:39.671 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:39.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:39.672 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:39.685 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:39.730 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:39.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:39.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:39.824 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:39.825 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:39.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:39.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:39.967 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:39.968 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:40.069 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:40.099 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:40.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:40.100 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:40.159 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 10 16 28191 28175 176.064 139.125 0.0040112 0.00567391 2024-09-11T08:45:40.201 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:40.234 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:40.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:40.235 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:40.336 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:40.367 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:40.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:40.368 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:40.468 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:40.495 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:40.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:40.496 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:40.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:40.629 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:40.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:40.630 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:40.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:40 smithi145 ceph-mon[33384]: pgmap v285: 148 pgs: 148 active+clean; 1.2 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 138 MiB/s wr, 2.21k op/s 2024-09-11T08:45:40.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:40.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:40.765 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:40.766 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:40.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:40 smithi092 ceph-mon[29647]: pgmap v285: 148 pgs: 148 active+clean; 1.2 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 138 MiB/s wr, 2.21k op/s 2024-09-11T08:45:40.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:40 smithi092 ceph-mon[41114]: pgmap v285: 148 pgs: 148 active+clean; 1.2 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 138 MiB/s wr, 2.21k op/s 2024-09-11T08:45:40.867 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:40.899 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:40.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:40.900 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:41.001 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:41.039 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:41.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:41.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:41.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:41.159 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 11 16 30525 30509 173.319 145.875 0.0044785 0.0057642 2024-09-11T08:45:41.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:41.171 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:41.272 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:41.303 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:41.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:41.304 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:41.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:41.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:41.435 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:41.536 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:41.568 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:41.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:41.569 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:41 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:45:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:41 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:45:41.670 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:41.699 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:41.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:41.701 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:41.802 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:41.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:41 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:45:41.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:41 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:45:41.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:41 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:45:41.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:41 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:45:41.816 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:45:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:45:41] "GET /metrics HTTP/1.1" 200 223912 "" "Prometheus/2.43.0" 2024-09-11T08:45:41.833 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:41.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:41.834 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:41.935 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:41.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:41.973 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:41.974 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:42.074 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:42.095 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:42.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:42.096 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:42.159 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 12 16 32225 32209 167.729 106.25 0.00185478 0.00591412 2024-09-11T08:45:42.197 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:42.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:42.231 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:42.231 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:42.333 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:42.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:42.355 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:42.356 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:42.457 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:42.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:42.489 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:42.490 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:42.591 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:42.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:42.616 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:42.616 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:42 smithi145 ceph-mon[33384]: pgmap v286: 148 pgs: 148 active+clean; 1.4 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 151 MiB/s wr, 2.41k op/s 2024-09-11T08:45:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:42 smithi145 ceph-mon[33384]: osdmap e160: 8 total, 8 up, 8 in 2024-09-11T08:45:42.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:42.736 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:42 smithi092 ceph-mon[29647]: pgmap v286: 148 pgs: 148 active+clean; 1.4 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 151 MiB/s wr, 2.41k op/s 2024-09-11T08:45:42.736 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:42 smithi092 ceph-mon[29647]: osdmap e160: 8 total, 8 up, 8 in 2024-09-11T08:45:42.737 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:42 smithi092 ceph-mon[41114]: pgmap v286: 148 pgs: 148 active+clean; 1.4 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 151 MiB/s wr, 2.41k op/s 2024-09-11T08:45:42.737 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:42 smithi092 ceph-mon[41114]: osdmap e160: 8 total, 8 up, 8 in 2024-09-11T08:45:42.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:42.756 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:42.756 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:42.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:42.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:42.895 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:42.895 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:42.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:43.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:43.054 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:43.054 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:43.155 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:43.159 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 13 16 33904 33888 162.898 104.938 0.00807523 0.00613245 2024-09-11T08:45:43.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:45:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:45:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:43.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:43.197 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:43.197 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:43.298 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:43.324 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:43.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:43.325 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:43.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:43.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:43.465 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:43.466 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:43.566 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:43.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:43.605 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:43.706 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:43.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:43.747 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:43.849 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:43.886 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:43.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:43.887 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:43.987 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:44.015 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:44.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:44.016 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:44.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:44.155 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:44.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:44.200 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:44.200 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 14 15 35686 35671 159.221 111.438 0.00422587 0.00627465 2024-09-11T08:45:44.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:44.340 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:44.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:44.343 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:44.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:44.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:44.482 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:44.482 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:44.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:44 smithi092 ceph-mon[29647]: pgmap v288: 148 pgs: 148 active+clean; 1.9 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 207 MiB/s wr, 3.31k op/s 2024-09-11T08:45:44.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:44 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:45:44.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:44 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:45:44.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:44 smithi092 ceph-mon[41114]: pgmap v288: 148 pgs: 148 active+clean; 1.9 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 207 MiB/s wr, 3.31k op/s 2024-09-11T08:45:44.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:44 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:45:44.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:44 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:45:44.583 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:44.612 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:44.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:44.613 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:44 smithi145 ceph-mon[33384]: pgmap v288: 148 pgs: 148 active+clean; 1.9 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 207 MiB/s wr, 3.31k op/s 2024-09-11T08:45:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:44 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:45:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:44 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:45:44.714 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:44.740 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:44.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:44.742 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:44.811 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:45:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:45:44.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:44.846 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:45:44.846 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:45:44.876 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:45:44.877 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:45:44.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:44.884 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:44.885 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:44.907 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:45:44.908 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:45:44.940 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:45:44.940 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:45:44.972 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:45:44.973 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:45:44.985 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:45.008 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:45:45.008 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:45:45.015 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:45.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:45.016 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:45.040 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:45:45.040 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:45:45.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:45.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:45.152 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:45.153 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:45.159 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 15 16 37459 37443 155.99 110.75 0.00475349 0.00640496 2024-09-11T08:45:45.254 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:45.284 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:45.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:45.285 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:45.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:45.436 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:45.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:45.437 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:45.538 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:45.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:45 smithi092 ceph-mon[29647]: osdmap e161: 8 total, 8 up, 8 in 2024-09-11T08:45:45.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:45 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2336533264' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:45:45.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:45 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:45:45.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:45 smithi092 ceph-mon[41114]: osdmap e161: 8 total, 8 up, 8 in 2024-09-11T08:45:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:45 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2336533264' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:45:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:45 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:45:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:45.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:45.567 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:45.567 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:45.666 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:45 smithi145 ceph-mon[33384]: osdmap e161: 8 total, 8 up, 8 in 2024-09-11T08:45:45.666 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:45 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2336533264' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:45:45.666 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:45 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:45:45.666 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:45.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:45.706 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:45.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:45.707 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:45.808 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:45.842 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:45.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:45.843 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:45.944 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:45.971 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:45.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:45.973 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:46.073 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:46.100 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:46.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:46.101 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:46.160 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 16 16 39459 39443 154.048 125 0.00585336 0.00648674 2024-09-11T08:45:46.201 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:46.228 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:46.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:46.229 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:46.330 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:46.358 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:46.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:46.359 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:46.460 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:46.495 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:46.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:46.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:46.501 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:46 smithi092 ceph-mon[29647]: pgmap v290: 148 pgs: 148 active+clean; 1.9 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 91 MiB/s wr, 1.45k op/s 2024-09-11T08:45:46.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:46 smithi092 ceph-mon[41114]: pgmap v290: 148 pgs: 148 active+clean; 1.9 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 91 MiB/s wr, 1.45k op/s 2024-09-11T08:45:46.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:46.635 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:46.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:46.636 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:46.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:46 smithi145 ceph-mon[33384]: pgmap v290: 148 pgs: 148 active+clean; 1.9 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 91 MiB/s wr, 1.45k op/s 2024-09-11T08:45:46.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:45:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:45:46.736 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:46.764 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:46.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:46.766 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:46.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:46.904 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:46.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:46.906 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:47.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:47.067 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:47.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:47.068 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:47.161 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 17 15 41417 41402 152.178 122.438 0.00518884 0.00656576 2024-09-11T08:45:47.168 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:47.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:47.201 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:47.201 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:47.302 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:47.331 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:47.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:47.332 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:47.433 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:47.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:47.468 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:47.468 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:47.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:47.574 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:47 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:47.575 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:47 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:47.611 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:47.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:47.612 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:47.713 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:47.741 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:47.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:47.742 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:47.829 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:45:47.830 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:45:47.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:47.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:47.886 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:47.886 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:47.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:47 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:47.987 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:48.016 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:48.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:48.017 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:48.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:48.140 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:48.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:48.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:48.161 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 18 16 43110 43094 149.599 105.75 0.00258565 0.00665666 2024-09-11T08:45:48.243 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:48.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:48.279 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:48.280 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:48.381 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:48.412 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:48.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:48.413 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:48.514 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:48.543 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:45:48.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:48.557 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:48.558 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:48.659 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:48.685 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:48.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:48.686 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:48.787 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:48.808 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:48.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:48.809 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:48.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:48 smithi092 ceph-mon[29647]: pgmap v291: 148 pgs: 148 active+clean; 2.4 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 162 MiB/s wr, 2.59k op/s 2024-09-11T08:45:48.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:48 smithi092 ceph-mon[41114]: pgmap v291: 148 pgs: 148 active+clean; 2.4 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 162 MiB/s wr, 2.59k op/s 2024-09-11T08:45:48.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:48.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:48 smithi145 ceph-mon[33384]: pgmap v291: 148 pgs: 148 active+clean; 2.4 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 162 MiB/s wr, 2.59k op/s 2024-09-11T08:45:48.952 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:48.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:48.953 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:49.054 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:49.080 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:49.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:49.081 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:49.161 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 19 16 44674 44658 146.869 97.75 0.00919951 0.00680363 2024-09-11T08:45:49.182 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:49.202 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:49.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:49.203 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:49.304 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:49.333 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:49.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:49.334 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:49.435 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:49.470 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:49.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:49.471 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:49.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:49.606 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:49.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:49.608 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:49.708 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:49.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:49.745 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:49.746 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:49.846 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:49.874 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:49.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:49.875 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:49.920 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:49 smithi092 ceph-mon[29647]: pgmap v292: 148 pgs: 148 active+clean; 2.4 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 127 MiB/s wr, 2.03k op/s 2024-09-11T08:45:49.920 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:49 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:45:49.920 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:49 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:45:49.920 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:49 smithi092 ceph-mon[41114]: pgmap v292: 148 pgs: 148 active+clean; 2.4 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 127 MiB/s wr, 2.03k op/s 2024-09-11T08:45:49.921 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:49 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:45:49.921 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:49 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:45:49.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:50.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:50.007 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:50.007 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:50.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:50.139 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:50.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:50.140 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:50.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:49 smithi145 ceph-mon[33384]: pgmap v292: 148 pgs: 148 active+clean; 2.4 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 127 MiB/s wr, 2.03k op/s 2024-09-11T08:45:50.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:49 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:45:50.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:49 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:45:50.162 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:45:50.162823+0000 min lat: 0.00108577 max lat: 0.11518 avg lat: 0.00692083 2024-09-11T08:45:50.162 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:45:50.162 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 20 16 46228 46212 144.382 97.125 0.00551881 0.00692083 2024-09-11T08:45:50.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:50.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:50.274 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:50.274 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:50.375 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:50.406 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:50.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:50.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:50.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:50.549 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:50.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:50.551 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:50.651 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:50.676 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:50.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:50.677 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:50.778 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:50.808 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:50.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:50.809 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:50.829 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:45:50.829 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:50 smithi092 ceph-mon[29647]: osdmap e162: 8 total, 8 up, 8 in 2024-09-11T08:45:50.829 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:50 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/320945947' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:45:50.829 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:50 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:45:50.830 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:50 smithi092 ceph-mon[41114]: osdmap e162: 8 total, 8 up, 8 in 2024-09-11T08:45:50.830 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:50 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/320945947' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:45:50.830 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:50 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:45:50.910 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:50.938 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:50.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:50.939 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:51.040 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:51.079 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:51.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:51.080 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:51.161 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 21 16 47907 47891 142.503 104.938 0.00360208 0.00701211 2024-09-11T08:45:51.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:50 smithi145 ceph-mon[33384]: osdmap e162: 8 total, 8 up, 8 in 2024-09-11T08:45:51.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:50 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/320945947' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:45:51.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:50 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:45:51.181 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:51.212 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:51.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:51.213 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:51.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:51.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:51.351 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:51.452 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:51.478 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:51.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:51.479 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:51.580 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:51.582 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-11T08:45:51.613 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:51.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:51.614 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:51.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:51.753 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:51.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:51.754 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:51.807 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:45:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:45:51] "GET /metrics HTTP/1.1" 200 224126 "" "Prometheus/2.43.0" 2024-09-11T08:45:51.841 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.5 to 1 (10000) 2024-09-11T08:45:51.855 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:51.891 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:51.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:51.892 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:51 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:45:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:51 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:45:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:51 smithi145 ceph-mon[33384]: osdmap e163: 8 total, 8 up, 8 in 2024-09-11T08:45:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:51 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/355647437' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T08:45:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:51 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T08:45:51.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:51 smithi145 ceph-mon[33384]: pgmap v295: 148 pgs: 148 active+clean; 2.6 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.5 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-11T08:45:51.957 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:45:51.993 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:52.026 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:52.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:52.028 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:52.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:51 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:45:52.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:51 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:45:52.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:51 smithi092 ceph-mon[29647]: osdmap e163: 8 total, 8 up, 8 in 2024-09-11T08:45:52.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:51 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/355647437' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T08:45:52.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:51 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T08:45:52.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:51 smithi092 ceph-mon[29647]: pgmap v295: 148 pgs: 148 active+clean; 2.6 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.5 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-11T08:45:52.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:51 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:45:52.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:51 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:45:52.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:51 smithi092 ceph-mon[41114]: osdmap e163: 8 total, 8 up, 8 in 2024-09-11T08:45:52.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:51 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/355647437' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T08:45:52.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:51 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T08:45:52.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:51 smithi092 ceph-mon[41114]: pgmap v295: 148 pgs: 148 active+clean; 2.6 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.5 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-11T08:45:52.129 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:52.162 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 22 16 49679 49663 141.06 110.75 0.00847076 0.00708404 2024-09-11T08:45:52.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:52.166 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:52.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:52.267 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:52.300 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:52.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:52.301 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:52.402 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:52.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:52.433 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:52.433 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:52.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:52.571 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:52.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:52.572 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:52.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:52.695 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:52.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:52.696 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:52.797 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:52.826 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:52.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:52.827 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:52.928 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:52.961 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:52.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:52.962 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:53.041 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:52 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T08:45:53.041 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:52 smithi092 ceph-mon[29647]: osdmap e164: 8 total, 8 up, 8 in 2024-09-11T08:45:53.042 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:52 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T08:45:53.042 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:52 smithi092 ceph-mon[41114]: osdmap e164: 8 total, 8 up, 8 in 2024-09-11T08:45:53.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:53.086 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 6 1 2024-09-11T08:45:53.095 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:45:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:45:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:53.095 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:52 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T08:45:53.095 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:52 smithi145 ceph-mon[33384]: osdmap e164: 8 total, 8 up, 8 in 2024-09-11T08:45:53.108 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:53.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:53.109 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:53.133 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:53.162 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 23 16 51254 51238 139.204 98.4375 0.0172551 0.0071779 2024-09-11T08:45:53.209 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:53.236 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:53.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:53.237 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:53.338 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:53.395 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:53.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:53.396 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:53.497 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:53.540 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:53.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:53.541 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:53.642 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:53.670 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:53.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:53.671 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:53.772 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:53.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:53.819 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:53.819 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:53.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:53.951 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:53 smithi092 ceph-mon[29647]: pgmap v297: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 87 MiB/s wr, 1.40k op/s 2024-09-11T08:45:53.951 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:53 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:53.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:53 smithi092 ceph-mon[29647]: osdmap e165: 8 total, 8 up, 8 in 2024-09-11T08:45:53.952 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:53 smithi092 ceph-mon[41114]: pgmap v297: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 87 MiB/s wr, 1.40k op/s 2024-09-11T08:45:53.952 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:53 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:53.952 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:53 smithi092 ceph-mon[41114]: osdmap e165: 8 total, 8 up, 8 in 2024-09-11T08:45:53.954 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:53.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:53.955 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:54.056 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:54.084 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:54.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:54.086 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:54.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:53 smithi145 ceph-mon[33384]: pgmap v297: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 87 MiB/s wr, 1.40k op/s 2024-09-11T08:45:54.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:53 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:54.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:53 smithi145 ceph-mon[33384]: osdmap e165: 8 total, 8 up, 8 in 2024-09-11T08:45:54.162 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 24 16 52623 52607 136.969 85.5625 0.00993656 0.00729634 2024-09-11T08:45:54.186 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:54.218 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:54.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:54.219 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:54.320 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:54.344 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:54.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:54.345 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:54.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:45:54.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:54.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:54.495 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:54.495 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:54.581 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:54.596 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:54.620 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:54.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:54.622 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:54.723 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:54.750 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:54.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:54.751 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:54.852 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:54.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:54.891 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:54.891 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:54.993 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:55.007 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:54 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:55.007 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:54 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:45:55.007 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:54 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:45:55.007 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:54 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:55.007 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:54 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:45:55.007 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:54 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:45:55.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:55.023 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:55.023 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:55.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:55.152 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:54 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:55.152 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:54 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:45:55.152 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:54 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:45:55.166 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 25 16 54047 54031 135.036 89 0.00784907 0.00739995 2024-09-11T08:45:55.174 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:55.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:55.176 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:55.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:55.319 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:55.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:55.320 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:55.421 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:55.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:55.454 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:55.454 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:55.555 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:55.583 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:55.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:55.584 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:55.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:55.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:55.725 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:55.725 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:55.826 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:55.844 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:45:55.844 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:45:55.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:55.865 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:55.866 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:55.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:56.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:56.004 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:56.105 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:56.134 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:56.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:56.135 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:56.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:55 smithi145 ceph-mon[33384]: pgmap v299: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 958 B/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-11T08:45:56.165 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 26 16 55679 55663 133.765 102 0.0061071 0.00747045 2024-09-11T08:45:56.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:56.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:56.275 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:56.275 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:56.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:55 smithi092 ceph-mon[29647]: pgmap v299: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 958 B/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-11T08:45:56.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:55 smithi092 ceph-mon[41114]: pgmap v299: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 958 B/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-11T08:45:56.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:56.409 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:56.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:56.410 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:56.511 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:56.532 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:56.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:56.533 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:56.546 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:45:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:45:56.586 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-11T08:45:56.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:56.674 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:56.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:56.675 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:56.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:56.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:56.810 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:56.810 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:56.911 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:56.946 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:56.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:56.947 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:56.993 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:45:57.049 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:57.073 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:57.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:57.074 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:57.165 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 27 16 57256 57240 132.461 98.5625 0.0117223 0.00754458 2024-09-11T08:45:57.172 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:56 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:57.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:57.200 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:56 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:57.200 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:56 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:57.213 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:57.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:57.214 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:57.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:57.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:57.353 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:57.353 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:57.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:57.492 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:57.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:57.493 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:57.594 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:57.621 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:57.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:57.622 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:57.723 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:57.775 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:57.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:57.776 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:57.877 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:57.902 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:57.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:57.903 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:58.004 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:58.026 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:45:58.027 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:45:58.031 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:58.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:58.032 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:58.064 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:45:58.065 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:45:58.094 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:45:58.095 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:45:58.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:58.139 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:45:58.140 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:45:58.166 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 28 16 58729 58713 131.013 92.0625 0.0141527 0.00762733 2024-09-11T08:45:58.167 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:45:58.167 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:45:58.176 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:58.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:58.177 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:58.198 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:45:58.198 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:45:58.199 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:57 smithi092 ceph-mon[29647]: pgmap v300: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.6 KiB/s rd, 160 MiB/s wr, 2.55k op/s 2024-09-11T08:45:58.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:57 smithi092 ceph-mon[41114]: pgmap v300: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.6 KiB/s rd, 160 MiB/s wr, 2.55k op/s 2024-09-11T08:45:58.229 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:45:58.229 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:45:58.262 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:45:58.262 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:45:58.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:45:58.292 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:57 smithi145 ceph-mon[33384]: pgmap v300: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.6 KiB/s rd, 160 MiB/s wr, 2.55k op/s 2024-09-11T08:45:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:58.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:58.319 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:45:58.420 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:58.463 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:58.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:58.464 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:45:58.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:45:58.606 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:58.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:58.607 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:45:58.708 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:58.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:58.741 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:58.741 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:58.819 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:45:58.841 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:58.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:58.873 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:58.873 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:58.974 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:59.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:59.003 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:59.003 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:59.012 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:58 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1275785558' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:45:59.012 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:58 smithi145 ceph-mon[33384]: osdmap e166: 8 total, 8 up, 8 in 2024-09-11T08:45:59.104 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:45:59.142 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:59.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:59.143 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:45:59.167 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 29 16 60236 60220 129.743 94.1875 0.0134515 0.00770266 2024-09-11T08:45:59.243 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:45:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:59.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:59.278 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:45:59.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:58 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1275785558' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:45:59.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:58 smithi092 ceph-mon[29647]: osdmap e166: 8 total, 8 up, 8 in 2024-09-11T08:45:59.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:58 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1275785558' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:45:59.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:58 smithi092 ceph-mon[41114]: osdmap e166: 8 total, 8 up, 8 in 2024-09-11T08:45:59.328 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:45:59.329 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T08:45:59.329 INFO:teuthology.orchestra.run.smithi145.stdout: "complaint_time": 30, 2024-09-11T08:45:59.329 INFO:teuthology.orchestra.run.smithi145.stdout: "num_blocked_ops": 0 2024-09-11T08:45:59.329 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:45:59.379 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:59.408 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:59.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:59.410 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:59.510 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:45:59.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:59.546 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:59.547 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:45:59.647 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:45:59.682 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:59.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:59.683 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:45:59.784 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:45:59.814 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:45:59.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:59.815 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:45:59.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:45:59.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:45:59.958 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:45:59.958 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:00.028 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-11T08:46:00.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:00.096 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:00.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:00.097 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:00.167 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 30 15 61643 61628 128.352 88 0.0119849 0.00778575 2024-09-11T08:46:00.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:00.244 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:00.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:00.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:00.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:45:59 smithi092 ceph-mon[29647]: pgmap v302: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 700 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-11T08:46:00.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:45:59 smithi092 ceph-mon[41114]: pgmap v302: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 700 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-11T08:46:00.346 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:00.400 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:00.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:00.402 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:00.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:45:59 smithi145 ceph-mon[33384]: pgmap v302: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 700 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-11T08:46:00.432 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:46:00.503 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:00.540 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:00.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:00.541 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:00.642 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:00.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:00.678 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:00.678 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:00.780 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:00.811 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:00.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:00.812 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:00.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:00.947 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:00.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:00.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:00.965 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.6 to 1 (10000) 2024-09-11T08:46:01.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:01.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:01.082 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:01.082 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:01.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:01.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:00 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1958132583' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T08:46:01.166 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 31 16 62948 62932 126.84 81.5 0.00242502 0.00784869 2024-09-11T08:46:01.184 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:01.206 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:01.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:01.207 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:01.308 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:01.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:01.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:00 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1958132583' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T08:46:01.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:01.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:00 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1958132583' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T08:46:01.330 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:01.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:01.331 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:01.398 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:46:01.432 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:01.454 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:01.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:01.455 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:01.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:01.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:01.579 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:01.579 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:01.680 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:01.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:01.703 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:01.784 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:46:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:46:01] "GET /metrics HTTP/1.1" 200 224183 "" "Prometheus/2.43.0" 2024-09-11T08:46:01.803 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:01.824 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:01.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:01.825 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:01.926 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:01.947 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:01.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:01.948 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:02.049 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:01 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1958132583' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T08:46:02.049 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:01 smithi145 ceph-mon[33384]: osdmap e167: 8 total, 8 up, 8 in 2024-09-11T08:46:02.049 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:01 smithi145 ceph-mon[33384]: pgmap v304: 148 pgs: 16 peering, 132 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 86 MiB/s wr, 1.36k op/s 2024-09-11T08:46:02.049 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:02.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:01 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1958132583' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T08:46:02.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:01 smithi092 ceph-mon[29647]: osdmap e167: 8 total, 8 up, 8 in 2024-09-11T08:46:02.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:01 smithi092 ceph-mon[29647]: pgmap v304: 148 pgs: 16 peering, 132 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 86 MiB/s wr, 1.36k op/s 2024-09-11T08:46:02.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:01 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1958132583' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T08:46:02.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:01 smithi092 ceph-mon[41114]: osdmap e167: 8 total, 8 up, 8 in 2024-09-11T08:46:02.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:01 smithi092 ceph-mon[41114]: pgmap v304: 148 pgs: 16 peering, 132 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 86 MiB/s wr, 1.36k op/s 2024-09-11T08:46:02.070 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:02.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:02.072 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:02.167 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 32 16 62948 62932 122.877 0 - 0.00784869 2024-09-11T08:46:02.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:02.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:02.195 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:02.195 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:02.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:02.317 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:02.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:02.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:02.334 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:46:02.334 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T08:46:02.335 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T08:46:02.335 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:46:02.335 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.335 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:63550 7.1 7:839dabea:::benchmark_data_smithi092_84565_object63549:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:46:02.335 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:43.960658+0000", 2024-09-11T08:46:02.335 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 258.35133540300001, 2024-09-11T08:46:02.335 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.090364671999999993, 2024-09-11T08:46:02.335 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.335 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.335 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.335 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.335 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.335 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 63550 2024-09-11T08:46:02.336 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.336 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.336 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.336 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.336 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960658+0000", 2024-09-11T08:46:02.336 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.336 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.336 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.336 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.336 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960658+0000", 2024-09-11T08:46:02.336 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.336 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.336 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.336 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.337 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960657+0000", 2024-09-11T08:46:02.337 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:02.337 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.337 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.337 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.337 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960724+0000", 2024-09-11T08:46:02.337 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6901000000000002e-05 2024-09-11T08:46:02.337 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.337 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.337 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.337 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960734+0000", 2024-09-11T08:46:02.337 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0206e-05 2024-09-11T08:46:02.337 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.337 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.338 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.338 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960738+0000", 2024-09-11T08:46:02.338 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8500000000000004e-06 2024-09-11T08:46:02.338 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.338 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.338 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.338 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960767+0000", 2024-09-11T08:46:02.338 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.8841999999999999e-05 2024-09-11T08:46:02.338 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.338 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.338 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.338 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960844+0000", 2024-09-11T08:46:02.338 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7350999999999999e-05 2024-09-11T08:46:02.339 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.339 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.339 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:46:02.339 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.960925+0000", 2024-09-11T08:46:02.339 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.0576999999999998e-05 2024-09-11T08:46:02.339 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.339 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.339 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.339 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963102+0000", 2024-09-11T08:46:02.339 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0021772459999999999 2024-09-11T08:46:02.339 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.339 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.339 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.339 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963265+0000", 2024-09-11T08:46:02.340 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00016236399999999999 2024-09-11T08:46:02.340 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.340 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.340 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.340 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:44.050915+0000", 2024-09-11T08:46:02.340 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.087650790000000006 2024-09-11T08:46:02.340 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.340 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.340 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.340 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:44.050982+0000", 2024-09-11T08:46:02.340 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6919000000000001e-05 2024-09-11T08:46:02.340 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.340 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.341 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.341 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:44.051023+0000", 2024-09-11T08:46:02.341 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0818000000000001e-05 2024-09-11T08:46:02.341 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.341 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.341 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.341 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.341 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.341 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:63554 7.1 7:824a3d8d:::benchmark_data_smithi092_84565_object63553:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:46:02.341 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:43.963333+0000", 2024-09-11T08:46:02.341 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 258.348661096, 2024-09-11T08:46:02.341 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.087833008000000004, 2024-09-11T08:46:02.341 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.342 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.342 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.342 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.342 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.342 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 63554 2024-09-11T08:46:02.342 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.342 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.342 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.342 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.342 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963333+0000", 2024-09-11T08:46:02.342 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.342 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.342 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.342 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.342 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963333+0000", 2024-09-11T08:46:02.343 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.343 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.343 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.343 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.343 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963332+0000", 2024-09-11T08:46:02.343 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:02.343 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.343 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.343 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.343 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963380+0000", 2024-09-11T08:46:02.343 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8263999999999997e-05 2024-09-11T08:46:02.343 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.343 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.343 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.343 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963395+0000", 2024-09-11T08:46:02.344 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4732999999999999e-05 2024-09-11T08:46:02.344 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.344 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.344 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.344 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963410+0000", 2024-09-11T08:46:02.344 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5902000000000001e-05 2024-09-11T08:46:02.344 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.344 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.344 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.344 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963458+0000", 2024-09-11T08:46:02.344 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7958999999999999e-05 2024-09-11T08:46:02.344 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.344 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.344 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.345 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963595+0000", 2024-09-11T08:46:02.345 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013652799999999999 2024-09-11T08:46:02.345 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.345 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.345 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,6", 2024-09-11T08:46:02.345 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.963727+0000", 2024-09-11T08:46:02.345 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000132127 2024-09-11T08:46:02.345 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.345 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.345 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.345 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.965814+0000", 2024-09-11T08:46:02.345 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0020865129999999999 2024-09-11T08:46:02.345 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.346 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.346 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.346 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:43.966175+0000", 2024-09-11T08:46:02.346 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00036101699999999999 2024-09-11T08:46:02.346 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.346 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.346 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.346 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:44.051074+0000", 2024-09-11T08:46:02.346 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.084899530000000001 2024-09-11T08:46:02.346 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.346 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.346 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.346 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:44.051132+0000", 2024-09-11T08:46:02.346 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7462e-05 2024-09-11T08:46:02.347 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.347 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.347 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.347 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:44.051166+0000", 2024-09-11T08:46:02.347 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4175999999999999e-05 2024-09-11T08:46:02.347 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.347 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.347 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.347 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.347 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.347 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249614 7.1 7:8187506c:::benchmark_data_smithi092_84565_object124086:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-11T08:46:02.347 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:43.754828+0000", 2024-09-11T08:46:02.347 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.557166289999998, 2024-09-11T08:46:02.347 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.063092063, 2024-09-11T08:46:02.348 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.348 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:46:02.348 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.348 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.348 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.348 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249614 2024-09-11T08:46:02.348 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.348 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.348 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.348 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.348 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.754828+0000", 2024-09-11T08:46:02.348 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.348 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.349 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.349 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.349 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.754828+0000", 2024-09-11T08:46:02.349 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.349 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.349 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.349 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.349 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.754827+0000", 2024-09-11T08:46:02.349 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:02.349 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.349 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.349 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.349 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.754831+0000", 2024-09-11T08:46:02.350 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.843e-06 2024-09-11T08:46:02.350 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.350 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.350 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.350 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.754843+0000", 2024-09-11T08:46:02.350 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1456e-05 2024-09-11T08:46:02.350 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.350 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.350 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.351 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.754847+0000", 2024-09-11T08:46:02.351 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3889999999999998e-06 2024-09-11T08:46:02.351 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.351 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.351 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.351 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.754877+0000", 2024-09-11T08:46:02.351 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9258999999999999e-05 2024-09-11T08:46:02.351 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.351 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.351 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.351 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.754954+0000", 2024-09-11T08:46:02.351 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7341000000000004e-05 2024-09-11T08:46:02.351 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.351 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.352 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3,6", 2024-09-11T08:46:02.352 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.755042+0000", 2024-09-11T08:46:02.352 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8447000000000002e-05 2024-09-11T08:46:02.352 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.352 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.352 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.352 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.757293+0000", 2024-09-11T08:46:02.352 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0022502260000000001 2024-09-11T08:46:02.352 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.352 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.352 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.352 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.757602+0000", 2024-09-11T08:46:02.353 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00030942099999999999 2024-09-11T08:46:02.353 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.353 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.353 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.353 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.760347+0000", 2024-09-11T08:46:02.353 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.002744692 2024-09-11T08:46:02.353 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.353 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.353 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.353 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.787657+0000", 2024-09-11T08:46:02.353 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.027310582 2024-09-11T08:46:02.353 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.353 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.354 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.354 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.787667+0000", 2024-09-11T08:46:02.354 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0025e-05 2024-09-11T08:46:02.354 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.354 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.354 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.354 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.817868+0000", 2024-09-11T08:46:02.354 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0302006509999999 2024-09-11T08:46:02.354 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.354 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.354 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.354 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.817920+0000", 2024-09-11T08:46:02.354 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1745e-05 2024-09-11T08:46:02.354 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.355 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.355 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.355 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.355 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.355 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249643 7.1 7:88946354:::benchmark_data_smithi092_84565_object124115:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T08:46:02.355 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:43.796778+0000", 2024-09-11T08:46:02.355 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.515215699999999, 2024-09-11T08:46:02.355 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0273323110000001, 2024-09-11T08:46:02.355 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.355 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.355 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.355 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.355 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.355 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249643 2024-09-11T08:46:02.356 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.356 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.356 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.356 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.356 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.796778+0000", 2024-09-11T08:46:02.356 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.356 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.356 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.356 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.356 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.796778+0000", 2024-09-11T08:46:02.356 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.356 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.357 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.357 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.357 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.796777+0000", 2024-09-11T08:46:02.357 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:02.357 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.357 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.357 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.357 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.796783+0000", 2024-09-11T08:46:02.357 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0179999999999998e-06 2024-09-11T08:46:02.357 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.357 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.357 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.357 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.796795+0000", 2024-09-11T08:46:02.357 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1788e-05 2024-09-11T08:46:02.357 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.358 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.358 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.358 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.796801+0000", 2024-09-11T08:46:02.358 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7749999999999998e-06 2024-09-11T08:46:02.358 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.358 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.358 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.358 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.796847+0000", 2024-09-11T08:46:02.358 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6694999999999999e-05 2024-09-11T08:46:02.358 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.358 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.358 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.358 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.796855+0000", 2024-09-11T08:46:02.358 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6009999999999997e-06 2024-09-11T08:46:02.359 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.359 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.359 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.359 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.818222+0000", 2024-09-11T08:46:02.359 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.021366757 2024-09-11T08:46:02.359 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.359 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.359 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.359 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.818329+0000", 2024-09-11T08:46:02.359 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000107136 2024-09-11T08:46:02.359 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.359 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.359 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:46:02.359 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.818433+0000", 2024-09-11T08:46:02.360 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000103769 2024-09-11T08:46:02.360 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.360 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.360 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.360 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.822619+0000", 2024-09-11T08:46:02.360 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0041864090000000003 2024-09-11T08:46:02.360 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.360 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.360 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.360 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.822731+0000", 2024-09-11T08:46:02.360 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000112476 2024-09-11T08:46:02.360 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.360 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.361 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.361 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.824042+0000", 2024-09-11T08:46:02.361 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0013100760000000001 2024-09-11T08:46:02.361 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.361 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.361 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.361 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.824089+0000", 2024-09-11T08:46:02.361 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7071999999999999e-05 2024-09-11T08:46:02.361 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.361 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.361 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.361 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.824110+0000", 2024-09-11T08:46:02.361 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.1905000000000001e-05 2024-09-11T08:46:02.361 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.362 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.362 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.362 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.362 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.362 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249690 7.1 7:80f7d765:::benchmark_data_smithi092_84565_object124162:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T08:46:02.362 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:43.882465+0000", 2024-09-11T08:46:02.362 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.429528798999996, 2024-09-11T08:46:02.362 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.94306494900000004, 2024-09-11T08:46:02.362 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.362 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.362 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.362 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.362 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.362 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249690 2024-09-11T08:46:02.363 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.363 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.363 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.363 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.363 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.882465+0000", 2024-09-11T08:46:02.363 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.363 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.363 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.363 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.363 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.882465+0000", 2024-09-11T08:46:02.363 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.363 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.363 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.363 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.364 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.882463+0000", 2024-09-11T08:46:02.364 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:46:02.364 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.364 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.364 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.364 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.882471+0000", 2024-09-11T08:46:02.364 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7700000000000001e-06 2024-09-11T08:46:02.364 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.364 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.364 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.364 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.882488+0000", 2024-09-11T08:46:02.364 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6714999999999998e-05 2024-09-11T08:46:02.364 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.364 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.365 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.365 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.882494+0000", 2024-09-11T08:46:02.365 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5649999999999998e-06 2024-09-11T08:46:02.365 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.365 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.365 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.365 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.882536+0000", 2024-09-11T08:46:02.365 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1774999999999999e-05 2024-09-11T08:46:02.365 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.365 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.365 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.365 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.882543+0000", 2024-09-11T08:46:02.365 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.3490000000000003e-06 2024-09-11T08:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.820935+0000", 2024-09-11T08:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.93839202799999999 2024-09-11T08:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.821039+0000", 2024-09-11T08:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000103187 2024-09-11T08:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.821101+0000", 2024-09-11T08:46:02.367 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.245e-05 2024-09-11T08:46:02.367 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.367 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.367 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.367 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.825449+0000", 2024-09-11T08:46:02.367 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0043476740000000002 2024-09-11T08:46:02.367 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.367 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.367 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.367 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.825459+0000", 2024-09-11T08:46:02.367 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0224000000000001e-05 2024-09-11T08:46:02.367 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.367 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.367 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.368 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.825481+0000", 2024-09-11T08:46:02.368 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.2325999999999999e-05 2024-09-11T08:46:02.368 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.368 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.368 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.368 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.825509+0000", 2024-09-11T08:46:02.368 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7613000000000001e-05 2024-09-11T08:46:02.368 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.368 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.368 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.368 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.825530+0000", 2024-09-11T08:46:02.368 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.1233000000000002e-05 2024-09-11T08:46:02.368 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.368 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.369 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.369 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.369 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.369 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249695 7.1 7:8ccede20:::benchmark_data_smithi092_84565_object124167:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T08:46:02.369 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:43.889057+0000", 2024-09-11T08:46:02.369 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.422936973000006, 2024-09-11T08:46:02.369 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.94120274900000001, 2024-09-11T08:46:02.369 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.369 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.369 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.369 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.369 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.369 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249695 2024-09-11T08:46:02.370 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.370 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.370 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.370 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.370 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.889057+0000", 2024-09-11T08:46:02.370 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.370 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.370 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.370 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.370 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.889057+0000", 2024-09-11T08:46:02.370 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.370 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.370 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.370 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.371 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.889055+0000", 2024-09-11T08:46:02.371 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:02.371 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.371 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.371 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.371 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.889062+0000", 2024-09-11T08:46:02.371 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5579999999999997e-06 2024-09-11T08:46:02.371 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.371 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.371 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.371 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.889077+0000", 2024-09-11T08:46:02.371 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5014000000000001e-05 2024-09-11T08:46:02.371 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.371 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.372 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.372 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.889083+0000", 2024-09-11T08:46:02.372 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3199999999999996e-06 2024-09-11T08:46:02.372 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.372 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.372 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.372 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.889128+0000", 2024-09-11T08:46:02.372 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5235000000000003e-05 2024-09-11T08:46:02.372 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.372 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.372 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.372 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.889137+0000", 2024-09-11T08:46:02.372 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.4030000000000007e-06 2024-09-11T08:46:02.373 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.373 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.373 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.373 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.822786+0000", 2024-09-11T08:46:02.373 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.93364916200000003 2024-09-11T08:46:02.373 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.373 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.373 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.373 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.822859+0000", 2024-09-11T08:46:02.373 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.3299000000000004e-05 2024-09-11T08:46:02.373 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.373 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.373 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:46:02.373 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.822920+0000", 2024-09-11T08:46:02.373 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0816999999999997e-05 2024-09-11T08:46:02.374 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.374 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.374 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.374 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.826748+0000", 2024-09-11T08:46:02.374 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003827946 2024-09-11T08:46:02.374 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.374 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.374 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.374 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.827900+0000", 2024-09-11T08:46:02.375 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001151851 2024-09-11T08:46:02.375 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.375 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.375 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.375 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.830181+0000", 2024-09-11T08:46:02.375 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0022809969999999999 2024-09-11T08:46:02.375 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.375 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.375 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.375 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.830226+0000", 2024-09-11T08:46:02.375 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5187e-05 2024-09-11T08:46:02.375 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.376 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.376 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.376 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.830260+0000", 2024-09-11T08:46:02.376 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3568000000000001e-05 2024-09-11T08:46:02.376 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.376 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.376 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.376 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.376 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.376 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249704 7.1 7:834f332e:::benchmark_data_smithi092_84565_object124176:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T08:46:02.376 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:43.902550+0000", 2024-09-11T08:46:02.376 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.409444033, 2024-09-11T08:46:02.376 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.92786211900000004, 2024-09-11T08:46:02.377 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.377 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.377 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.377 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.377 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.377 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249704 2024-09-11T08:46:02.377 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.377 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.377 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.377 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.377 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.902550+0000", 2024-09-11T08:46:02.377 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.377 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.377 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.378 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.378 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.902550+0000", 2024-09-11T08:46:02.378 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.378 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.378 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.378 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.378 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.902548+0000", 2024-09-11T08:46:02.378 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:02.378 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.378 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.378 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.378 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.902556+0000", 2024-09-11T08:46:02.378 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.8120000000000008e-06 2024-09-11T08:46:02.378 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.379 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.379 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.379 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.902572+0000", 2024-09-11T08:46:02.379 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5491000000000001e-05 2024-09-11T08:46:02.379 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.379 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.379 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.379 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.902580+0000", 2024-09-11T08:46:02.379 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.977e-06 2024-09-11T08:46:02.379 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.379 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.379 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.379 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.902611+0000", 2024-09-11T08:46:02.379 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0902000000000003e-05 2024-09-11T08:46:02.379 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.380 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.380 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.380 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.902617+0000", 2024-09-11T08:46:02.380 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9649999999999996e-06 2024-09-11T08:46:02.380 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.380 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.380 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.380 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.824147+0000", 2024-09-11T08:46:02.380 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.92153080600000004 2024-09-11T08:46:02.380 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.380 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.380 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.380 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.824219+0000", 2024-09-11T08:46:02.380 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1149000000000006e-05 2024-09-11T08:46:02.380 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.381 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.381 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:46:02.381 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.824296+0000", 2024-09-11T08:46:02.381 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7220999999999998e-05 2024-09-11T08:46:02.381 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.381 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.381 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.381 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.827906+0000", 2024-09-11T08:46:02.381 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0036105550000000001 2024-09-11T08:46:02.381 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.381 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.381 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.381 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.830308+0000", 2024-09-11T08:46:02.381 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0024020600000000001 2024-09-11T08:46:02.382 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.382 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.382 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.382 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.830340+0000", 2024-09-11T08:46:02.382 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1819000000000001e-05 2024-09-11T08:46:02.382 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.382 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.382 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.382 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.830379+0000", 2024-09-11T08:46:02.382 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8983000000000003e-05 2024-09-11T08:46:02.382 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.382 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.382 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.383 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.830412+0000", 2024-09-11T08:46:02.383 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2767e-05 2024-09-11T08:46:02.383 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.383 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.383 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.383 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.383 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.383 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249726 7.1 7:8866147b:::benchmark_data_smithi092_84565_object124198:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T08:46:02.383 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:43.930701+0000", 2024-09-11T08:46:02.383 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.381293331999998, 2024-09-11T08:46:02.383 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90838300800000005, 2024-09-11T08:46:02.383 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.383 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.383 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.383 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.384 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.384 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249726 2024-09-11T08:46:02.384 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.384 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.384 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.384 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.384 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.930701+0000", 2024-09-11T08:46:02.384 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.385 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.385 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.385 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.385 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.930701+0000", 2024-09-11T08:46:02.385 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.385 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.385 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.385 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.385 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.930699+0000", 2024-09-11T08:46:02.385 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:02.385 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.385 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.385 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.385 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.930706+0000", 2024-09-11T08:46:02.386 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1489999999999997e-06 2024-09-11T08:46:02.386 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.386 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.386 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.386 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.930721+0000", 2024-09-11T08:46:02.386 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4506e-05 2024-09-11T08:46:02.386 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.386 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.386 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.386 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.930727+0000", 2024-09-11T08:46:02.386 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2029999999999997e-06 2024-09-11T08:46:02.386 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.387 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.387 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.387 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.930770+0000", 2024-09-11T08:46:02.387 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3226000000000003e-05 2024-09-11T08:46:02.387 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.387 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.387 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.387 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.930778+0000", 2024-09-11T08:46:02.387 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2819999999999996e-06 2024-09-11T08:46:02.387 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.387 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.387 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.387 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.825564+0000", 2024-09-11T08:46:02.387 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89478615399999994 2024-09-11T08:46:02.388 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.388 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.388 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.388 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.825636+0000", 2024-09-11T08:46:02.388 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1895000000000002e-05 2024-09-11T08:46:02.388 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.388 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.388 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:46:02.388 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.825699+0000", 2024-09-11T08:46:02.388 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2229000000000003e-05 2024-09-11T08:46:02.388 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.388 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.388 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.389 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.830139+0000", 2024-09-11T08:46:02.389 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0044403029999999996 2024-09-11T08:46:02.389 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.389 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.389 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.389 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.835813+0000", 2024-09-11T08:46:02.389 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0056737630000000001 2024-09-11T08:46:02.389 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.389 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.389 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.389 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.838992+0000", 2024-09-11T08:46:02.389 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0031790550000000001 2024-09-11T08:46:02.389 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.389 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839043+0000", 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1164e-05 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839084+0000", 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0734e-05 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249734 7.1 7:80e68765:::benchmark_data_smithi092_84565_object124206:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:43.940359+0000", 2024-09-11T08:46:02.390 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.371634506000007, 2024-09-11T08:46:02.391 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89888959999999996, 2024-09-11T08:46:02.391 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.391 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.391 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.391 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.391 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.391 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249734 2024-09-11T08:46:02.391 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.391 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.391 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.391 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.391 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.940359+0000", 2024-09-11T08:46:02.391 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.392 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.392 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.392 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.392 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.940359+0000", 2024-09-11T08:46:02.392 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.392 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.392 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.392 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.392 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.940357+0000", 2024-09-11T08:46:02.392 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999976 2024-09-11T08:46:02.392 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.392 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.392 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.392 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.940370+0000", 2024-09-11T08:46:02.393 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.295e-05 2024-09-11T08:46:02.393 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.393 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.393 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.393 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.940398+0000", 2024-09-11T08:46:02.393 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7855999999999999e-05 2024-09-11T08:46:02.393 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.393 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.393 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.393 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.940409+0000", 2024-09-11T08:46:02.393 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1545999999999999e-05 2024-09-11T08:46:02.393 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.393 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.393 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.940492+0000", 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2095000000000006e-05 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.940507+0000", 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5480000000000001e-05 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.826781+0000", 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.88627352800000003 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.394 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.826843+0000", 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2261999999999994e-05 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.826913+0000", 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.0570999999999999e-05 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.830314+0000", 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003400658 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.838936+0000", 2024-09-11T08:46:02.395 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0086218890000000006 2024-09-11T08:46:02.396 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.396 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.396 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.396 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839171+0000", 2024-09-11T08:46:02.396 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00023492700000000001 2024-09-11T08:46:02.396 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.396 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.396 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.396 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839215+0000", 2024-09-11T08:46:02.396 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3779999999999998e-05 2024-09-11T08:46:02.396 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.396 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.396 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.396 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839249+0000", 2024-09-11T08:46:02.396 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4320999999999999e-05 2024-09-11T08:46:02.397 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.397 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.397 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.397 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.397 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.397 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249740 7.1 7:81465edd:::benchmark_data_smithi092_84565_object124212:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T08:46:02.397 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:43.948322+0000", 2024-09-11T08:46:02.397 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.363671549000003, 2024-09-11T08:46:02.397 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89109624899999995, 2024-09-11T08:46:02.397 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.397 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.397 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.397 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.398 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.398 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249740 2024-09-11T08:46:02.398 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.398 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.398 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.398 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.398 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948322+0000", 2024-09-11T08:46:02.398 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.398 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.398 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.398 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.398 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948322+0000", 2024-09-11T08:46:02.398 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.399 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.399 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.399 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.399 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948321+0000", 2024-09-11T08:46:02.399 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:02.399 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.399 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.399 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.399 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948327+0000", 2024-09-11T08:46:02.399 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7370000000000003e-06 2024-09-11T08:46:02.399 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.399 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.399 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.400 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948338+0000", 2024-09-11T08:46:02.400 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0951999999999999e-05 2024-09-11T08:46:02.400 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.400 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.400 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.400 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948342+0000", 2024-09-11T08:46:02.400 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4689999999999999e-06 2024-09-11T08:46:02.400 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.400 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.400 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.400 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948381+0000", 2024-09-11T08:46:02.400 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8349000000000002e-05 2024-09-11T08:46:02.400 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.401 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.401 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.401 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948386+0000", 2024-09-11T08:46:02.401 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8890000000000001e-06 2024-09-11T08:46:02.401 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.401 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.401 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.401 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.827939+0000", 2024-09-11T08:46:02.401 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.87955356100000004 2024-09-11T08:46:02.401 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.401 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.401 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.401 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.828020+0000", 2024-09-11T08:46:02.402 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.0891999999999997e-05 2024-09-11T08:46:02.402 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.402 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.402 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:46:02.402 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.828082+0000", 2024-09-11T08:46:02.402 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2213999999999998e-05 2024-09-11T08:46:02.402 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.402 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.402 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.402 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.830424+0000", 2024-09-11T08:46:02.402 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0023414809999999999 2024-09-11T08:46:02.402 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.402 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.402 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.402 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839130+0000", 2024-09-11T08:46:02.403 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0087064059999999999 2024-09-11T08:46:02.403 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.403 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.403 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.403 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839343+0000", 2024-09-11T08:46:02.403 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00021247799999999999 2024-09-11T08:46:02.403 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.403 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.403 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.403 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839383+0000", 2024-09-11T08:46:02.403 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0488000000000003e-05 2024-09-11T08:46:02.403 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.403 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.404 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.404 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839419+0000", 2024-09-11T08:46:02.404 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5522000000000003e-05 2024-09-11T08:46:02.404 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.404 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.404 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.404 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.404 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.405 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249742 7.1 7:8020985c:::benchmark_data_smithi092_84565_object124214:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T08:46:02.405 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:43.948349+0000", 2024-09-11T08:46:02.405 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.363645300000002, 2024-09-11T08:46:02.405 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.891260312, 2024-09-11T08:46:02.406 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.406 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.406 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.406 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.406 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.406 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249742 2024-09-11T08:46:02.406 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.406 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.406 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.406 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.406 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948349+0000", 2024-09-11T08:46:02.406 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.407 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.407 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.407 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948349+0000", 2024-09-11T08:46:02.407 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.407 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.407 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.407 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948348+0000", 2024-09-11T08:46:02.407 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:46:02.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.407 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.407 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948351+0000", 2024-09-11T08:46:02.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5469999999999999e-06 2024-09-11T08:46:02.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948358+0000", 2024-09-11T08:46:02.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.092e-06 2024-09-11T08:46:02.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948362+0000", 2024-09-11T08:46:02.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7790000000000002e-06 2024-09-11T08:46:02.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.409 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948408+0000", 2024-09-11T08:46:02.409 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6285999999999997e-05 2024-09-11T08:46:02.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.409 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.948416+0000", 2024-09-11T08:46:02.409 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.5199999999999997e-06 2024-09-11T08:46:02.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.409 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.829055+0000", 2024-09-11T08:46:02.409 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.88063833000000002 2024-09-11T08:46:02.410 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.410 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.410 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.410 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.829143+0000", 2024-09-11T08:46:02.410 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8725000000000002e-05 2024-09-11T08:46:02.410 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.410 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.410 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:46:02.410 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.829212+0000", 2024-09-11T08:46:02.410 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8800999999999994e-05 2024-09-11T08:46:02.410 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.410 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.410 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.410 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.834025+0000", 2024-09-11T08:46:02.410 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0048128140000000003 2024-09-11T08:46:02.411 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.411 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.411 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.411 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839302+0000", 2024-09-11T08:46:02.411 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0052773330000000004 2024-09-11T08:46:02.411 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.411 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.411 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.411 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839545+0000", 2024-09-11T08:46:02.411 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00024252799999999999 2024-09-11T08:46:02.411 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.411 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.411 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.411 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839584+0000", 2024-09-11T08:46:02.412 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9026999999999998e-05 2024-09-11T08:46:02.412 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.412 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.412 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.412 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839609+0000", 2024-09-11T08:46:02.412 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4989000000000001e-05 2024-09-11T08:46:02.412 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.412 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.412 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.412 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.412 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.412 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249745 7.1 7:8cfd147a:::benchmark_data_smithi092_84565_object124217:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T08:46:02.412 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:43.954461+0000", 2024-09-11T08:46:02.412 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.357533083000007, 2024-09-11T08:46:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89663142900000004, 2024-09-11T08:46:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249745 2024-09-11T08:46:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.954461+0000", 2024-09-11T08:46:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.414 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.414 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.954461+0000", 2024-09-11T08:46:02.414 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.414 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.414 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.414 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.414 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.954459+0000", 2024-09-11T08:46:02.414 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:46:02.414 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.414 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.414 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.414 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.954471+0000", 2024-09-11T08:46:02.414 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2099999999999999e-05 2024-09-11T08:46:02.414 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.415 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.415 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.415 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.954499+0000", 2024-09-11T08:46:02.415 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7835999999999999e-05 2024-09-11T08:46:02.415 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.415 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.415 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.415 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.954510+0000", 2024-09-11T08:46:02.415 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1173e-05 2024-09-11T08:46:02.415 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.415 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.415 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.416 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.954623+0000", 2024-09-11T08:46:02.416 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000112518 2024-09-11T08:46:02.416 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.416 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.416 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.416 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.954642+0000", 2024-09-11T08:46:02.416 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.9358999999999999e-05 2024-09-11T08:46:02.416 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.416 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.416 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.416 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.830438+0000", 2024-09-11T08:46:02.416 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.87579660500000001 2024-09-11T08:46:02.416 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.417 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.417 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.417 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.830540+0000", 2024-09-11T08:46:02.417 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010136699999999999 2024-09-11T08:46:02.417 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.417 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.417 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:46:02.417 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.830654+0000", 2024-09-11T08:46:02.417 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011401299999999999 2024-09-11T08:46:02.417 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.417 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.417 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.418 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.835821+0000", 2024-09-11T08:46:02.418 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.005166655 2024-09-11T08:46:02.418 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.418 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.418 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.418 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839483+0000", 2024-09-11T08:46:02.418 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0036622790000000001 2024-09-11T08:46:02.418 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.418 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.418 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.418 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851042+0000", 2024-09-11T08:46:02.418 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011559674000000001 2024-09-11T08:46:02.418 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.418 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.419 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.419 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851069+0000", 2024-09-11T08:46:02.419 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.6855999999999998e-05 2024-09-11T08:46:02.419 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.419 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.419 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.419 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851092+0000", 2024-09-11T08:46:02.419 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.2897000000000001e-05 2024-09-11T08:46:02.419 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.419 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.419 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.419 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.419 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.420 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249768 7.1 7:866c4bd8:::benchmark_data_smithi092_84565_object124240:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T08:46:02.420 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:43.983289+0000", 2024-09-11T08:46:02.420 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.328705225999997, 2024-09-11T08:46:02.420 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.86788239700000003, 2024-09-11T08:46:02.420 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.420 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.420 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.420 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.420 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.420 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249768 2024-09-11T08:46:02.420 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.420 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.420 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.421 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.421 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.983289+0000", 2024-09-11T08:46:02.421 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.421 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.421 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.421 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.421 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.983289+0000", 2024-09-11T08:46:02.421 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.421 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.421 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.421 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.421 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.983287+0000", 2024-09-11T08:46:02.422 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:02.422 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.422 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.422 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.422 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.983295+0000", 2024-09-11T08:46:02.422 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7540000000000006e-06 2024-09-11T08:46:02.422 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.422 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.422 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.422 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.983310+0000", 2024-09-11T08:46:02.422 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4856e-05 2024-09-11T08:46:02.422 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.422 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.423 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.423 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.983316+0000", 2024-09-11T08:46:02.423 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4819999999999999e-06 2024-09-11T08:46:02.423 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.423 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.423 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.423 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.983352+0000", 2024-09-11T08:46:02.423 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5497999999999998e-05 2024-09-11T08:46:02.423 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.423 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.424 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.424 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:43.983358+0000", 2024-09-11T08:46:02.424 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8290000000000002e-06 2024-09-11T08:46:02.424 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.424 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.424 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.424 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.832031+0000", 2024-09-11T08:46:02.424 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.84867219699999996 2024-09-11T08:46:02.424 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.424 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.424 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.424 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.832136+0000", 2024-09-11T08:46:02.424 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000105264 2024-09-11T08:46:02.425 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.425 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.425 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:46:02.425 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.832249+0000", 2024-09-11T08:46:02.425 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000112939 2024-09-11T08:46:02.425 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.425 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.425 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.425 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.838888+0000", 2024-09-11T08:46:02.425 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0066395050000000004 2024-09-11T08:46:02.425 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.425 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.425 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.425 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839662+0000", 2024-09-11T08:46:02.425 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00077381800000000001 2024-09-11T08:46:02.426 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.426 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.426 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.426 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851125+0000", 2024-09-11T08:46:02.426 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011462704000000001 2024-09-11T08:46:02.426 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.426 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.426 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.426 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851152+0000", 2024-09-11T08:46:02.426 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.6829999999999999e-05 2024-09-11T08:46:02.426 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.426 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.426 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851171+0000", 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.9343000000000001e-05 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249809 7.1 7:85e34ef7:::benchmark_data_smithi092_84565_object124281:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:44.039118+0000", 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.272875687999999, 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.81212616299999996, 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.427 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.428 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249809 2024-09-11T08:46:02.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.428 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.428 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.428 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.039118+0000", 2024-09-11T08:46:02.428 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.428 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.428 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.039118+0000", 2024-09-11T08:46:02.428 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.428 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.039117+0000", 2024-09-11T08:46:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.039123+0000", 2024-09-11T08:46:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3470000000000004e-06 2024-09-11T08:46:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.039139+0000", 2024-09-11T08:46:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6291e-05 2024-09-11T08:46:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.429 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.430 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.039145+0000", 2024-09-11T08:46:02.430 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8880000000000002e-06 2024-09-11T08:46:02.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.430 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.430 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.430 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.039182+0000", 2024-09-11T08:46:02.430 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6959000000000003e-05 2024-09-11T08:46:02.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.430 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.430 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.430 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.039188+0000", 2024-09-11T08:46:02.430 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0000000000000002e-06 2024-09-11T08:46:02.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.430 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.430 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.834123+0000", 2024-09-11T08:46:02.431 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.79493499700000003 2024-09-11T08:46:02.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.431 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.431 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.834270+0000", 2024-09-11T08:46:02.431 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014673099999999999 2024-09-11T08:46:02.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.431 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.431 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:46:02.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.834392+0000", 2024-09-11T08:46:02.431 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000122421 2024-09-11T08:46:02.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.431 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.432 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.432 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839094+0000", 2024-09-11T08:46:02.432 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00470135 2024-09-11T08:46:02.432 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.432 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.432 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.432 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.841682+0000", 2024-09-11T08:46:02.432 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0025878149999999998 2024-09-11T08:46:02.432 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.432 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.432 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.432 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851202+0000", 2024-09-11T08:46:02.432 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0095206279999999997 2024-09-11T08:46:02.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.433 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.433 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.433 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851226+0000", 2024-09-11T08:46:02.433 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.332e-05 2024-09-11T08:46:02.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.433 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.433 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.433 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851244+0000", 2024-09-11T08:46:02.433 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.8824000000000001e-05 2024-09-11T08:46:02.433 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.433 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.433 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.434 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.434 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249833 7.1 7:865bb261:::benchmark_data_smithi092_84565_object124305:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T08:46:02.434 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:44.072971+0000", 2024-09-11T08:46:02.434 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.239023032999995, 2024-09-11T08:46:02.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.778342635, 2024-09-11T08:46:02.434 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.434 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.434 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.434 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.434 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.434 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249833 2024-09-11T08:46:02.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.435 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.435 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.435 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.072971+0000", 2024-09-11T08:46:02.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.435 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.435 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.435 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.072971+0000", 2024-09-11T08:46:02.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.435 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.435 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.435 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.436 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.072969+0000", 2024-09-11T08:46:02.436 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:02.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.436 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.436 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.436 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.072976+0000", 2024-09-11T08:46:02.436 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6059999999999999e-06 2024-09-11T08:46:02.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.436 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.436 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.436 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.072991+0000", 2024-09-11T08:46:02.436 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4627000000000001e-05 2024-09-11T08:46:02.437 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.437 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.437 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.437 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.072996+0000", 2024-09-11T08:46:02.437 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6579999999999999e-06 2024-09-11T08:46:02.437 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.437 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.437 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.437 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.073038+0000", 2024-09-11T08:46:02.437 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1513e-05 2024-09-11T08:46:02.437 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.437 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.437 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.437 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.073046+0000", 2024-09-11T08:46:02.438 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.858e-06 2024-09-11T08:46:02.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.438 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.438 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.438 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.835858+0000", 2024-09-11T08:46:02.438 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.76281221899999996 2024-09-11T08:46:02.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.438 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.438 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.438 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.835952+0000", 2024-09-11T08:46:02.438 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.4609000000000002e-05 2024-09-11T08:46:02.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.438 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:46:02.439 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.836052+0000", 2024-09-11T08:46:02.439 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.9296999999999995e-05 2024-09-11T08:46:02.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.439 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.439 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839100+0000", 2024-09-11T08:46:02.439 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003047892 2024-09-11T08:46:02.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.439 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.439 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.841712+0000", 2024-09-11T08:46:02.439 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0026126589999999998 2024-09-11T08:46:02.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.440 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.440 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851272+0000", 2024-09-11T08:46:02.440 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0095602280000000005 2024-09-11T08:46:02.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.440 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.440 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851297+0000", 2024-09-11T08:46:02.440 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4249000000000001e-05 2024-09-11T08:46:02.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.440 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.440 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851313+0000", 2024-09-11T08:46:02.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.679e-05 2024-09-11T08:46:02.441 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.441 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.441 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.441 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.441 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.441 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249839 7.1 7:81d8e1e4:::benchmark_data_smithi092_84565_object124311:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T08:46:02.441 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:44.079953+0000", 2024-09-11T08:46:02.441 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.232040780000005, 2024-09-11T08:46:02.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.77142149800000004, 2024-09-11T08:46:02.441 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.441 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.441 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.442 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.442 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.442 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249839 2024-09-11T08:46:02.442 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.442 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.442 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.442 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.442 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.079953+0000", 2024-09-11T08:46:02.442 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.442 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.442 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.442 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.442 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.079953+0000", 2024-09-11T08:46:02.442 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.443 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.443 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.443 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.443 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.079952+0000", 2024-09-11T08:46:02.443 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:02.443 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.443 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.443 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.443 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.079969+0000", 2024-09-11T08:46:02.443 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6934000000000001e-05 2024-09-11T08:46:02.443 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.443 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.443 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.443 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.079981+0000", 2024-09-11T08:46:02.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2347999999999999e-05 2024-09-11T08:46:02.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.444 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.079986+0000", 2024-09-11T08:46:02.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5800000000000002e-06 2024-09-11T08:46:02.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.444 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.080030+0000", 2024-09-11T08:46:02.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4329000000000002e-05 2024-09-11T08:46:02.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.445 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.080038+0000", 2024-09-11T08:46:02.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7270000000000007e-06 2024-09-11T08:46:02.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.445 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.445 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.837551+0000", 2024-09-11T08:46:02.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.75751333200000004 2024-09-11T08:46:02.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.445 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.445 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.837627+0000", 2024-09-11T08:46:02.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.5838000000000001e-05 2024-09-11T08:46:02.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:46:02.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.837698+0000", 2024-09-11T08:46:02.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.0961e-05 2024-09-11T08:46:02.446 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839258+0000", 2024-09-11T08:46:02.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001560606 2024-09-11T08:46:02.446 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.845737+0000", 2024-09-11T08:46:02.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0064787849999999999 2024-09-11T08:46:02.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851335+0000", 2024-09-11T08:46:02.447 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0055976419999999999 2024-09-11T08:46:02.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851358+0000", 2024-09-11T08:46:02.447 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3002000000000001e-05 2024-09-11T08:46:02.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.448 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851375+0000", 2024-09-11T08:46:02.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6688999999999999e-05 2024-09-11T08:46:02.448 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.448 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.448 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.448 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.448 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.448 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249844 7.1 7:8853c3b6:::benchmark_data_smithi092_84565_object124316:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T08:46:02.448 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:44.086966+0000", 2024-09-11T08:46:02.448 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.225027502000003, 2024-09-11T08:46:02.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.76464301899999998, 2024-09-11T08:46:02.448 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.448 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.449 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.449 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.449 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.449 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249844 2024-09-11T08:46:02.449 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.449 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.449 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.449 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.449 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.086966+0000", 2024-09-11T08:46:02.449 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.449 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.449 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.449 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.086966+0000", 2024-09-11T08:46:02.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.086965+0000", 2024-09-11T08:46:02.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:02.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.086970+0000", 2024-09-11T08:46:02.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0679999999999996e-06 2024-09-11T08:46:02.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.086981+0000", 2024-09-11T08:46:02.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0835999999999999e-05 2024-09-11T08:46:02.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.086986+0000", 2024-09-11T08:46:02.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.313e-06 2024-09-11T08:46:02.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.087026+0000", 2024-09-11T08:46:02.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0649999999999999e-05 2024-09-11T08:46:02.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.087035+0000", 2024-09-11T08:46:02.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.7909999999999993e-06 2024-09-11T08:46:02.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839693+0000", 2024-09-11T08:46:02.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.75265790700000001 2024-09-11T08:46:02.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839802+0000", 2024-09-11T08:46:02.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010938399999999999 2024-09-11T08:46:02.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:46:02.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.839917+0000", 2024-09-11T08:46:02.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000115055 2024-09-11T08:46:02.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.844116+0000", 2024-09-11T08:46:02.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0041989430000000001 2024-09-11T08:46:02.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.845799+0000", 2024-09-11T08:46:02.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001683033 2024-09-11T08:46:02.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851528+0000", 2024-09-11T08:46:02.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0057286389999999998 2024-09-11T08:46:02.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.455 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.455 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851572+0000", 2024-09-11T08:46:02.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4376000000000003e-05 2024-09-11T08:46:02.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.455 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.455 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851609+0000", 2024-09-11T08:46:02.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7088000000000001e-05 2024-09-11T08:46:02.455 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.455 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.455 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.455 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:249845 7.1 7:866c97dd:::benchmark_data_smithi092_84565_object124317:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T08:46:02.456 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:44:44.087676+0000", 2024-09-11T08:46:02.456 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.224317912000004, 2024-09-11T08:46:02.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.76406514800000003, 2024-09-11T08:46:02.456 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.456 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.456 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.456 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:02.456 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:02.456 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 249845 2024-09-11T08:46:02.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.456 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.456 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.087676+0000", 2024-09-11T08:46:02.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.087676+0000", 2024-09-11T08:46:02.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.087676+0000", 2024-09-11T08:46:02.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:46:02.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.087678+0000", 2024-09-11T08:46:02.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4739999999999999e-06 2024-09-11T08:46:02.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.087683+0000", 2024-09-11T08:46:02.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3140000000000003e-06 2024-09-11T08:46:02.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.087703+0000", 2024-09-11T08:46:02.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.9868e-05 2024-09-11T08:46:02.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.087729+0000", 2024-09-11T08:46:02.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.6129999999999999e-05 2024-09-11T08:46:02.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:02.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.087734+0000", 2024-09-11T08:46:02.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4410000000000003e-06 2024-09-11T08:46:02.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.841798+0000", 2024-09-11T08:46:02.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.75406421499999998 2024-09-11T08:46:02.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.841895+0000", 2024-09-11T08:46:02.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.6786999999999993e-05 2024-09-11T08:46:02.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:46:02.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.842002+0000", 2024-09-11T08:46:02.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000106866 2024-09-11T08:46:02.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.844551+0000", 2024-09-11T08:46:02.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0025489689999999999 2024-09-11T08:46:02.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.845861+0000", 2024-09-11T08:46:02.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001310395 2024-09-11T08:46:02.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851654+0000", 2024-09-11T08:46:02.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0057932859999999999 2024-09-11T08:46:02.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851702+0000", 2024-09-11T08:46:02.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7521999999999997e-05 2024-09-11T08:46:02.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:44:44.851741+0000", 2024-09-11T08:46:02.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9289999999999998e-05 2024-09-11T08:46:02.462 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.462 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.462 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.463 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:25718 8.2 8:4be854d1:::benchmark_data_smithi092_117541_object25717:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159)", 2024-09-11T08:46:02.463 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:45:38.979328+0000", 2024-09-11T08:46:02.463 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 23.332665385999999, 2024-09-11T08:46:02.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.093480158999999993, 2024-09-11T08:46:02.463 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.463 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.463 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.463 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:46:02.463 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:46:02.463 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 25718 2024-09-11T08:46:02.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.463 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.979328+0000", 2024-09-11T08:46:02.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.979328+0000", 2024-09-11T08:46:02.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.979327+0000", 2024-09-11T08:46:02.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:46:02.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.979376+0000", 2024-09-11T08:46:02.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9385999999999998e-05 2024-09-11T08:46:02.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.979388+0000", 2024-09-11T08:46:02.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1833e-05 2024-09-11T08:46:02.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.979394+0000", 2024-09-11T08:46:02.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6330000000000001e-06 2024-09-11T08:46:02.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.466 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.979425+0000", 2024-09-11T08:46:02.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1262e-05 2024-09-11T08:46:02.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.466 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.979508+0000", 2024-09-11T08:46:02.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2829000000000003e-05 2024-09-11T08:46:02.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:46:02.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.979614+0000", 2024-09-11T08:46:02.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000105778 2024-09-11T08:46:02.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.467 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.980490+0000", 2024-09-11T08:46:02.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00087635299999999996 2024-09-11T08:46:02.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.467 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.980762+0000", 2024-09-11T08:46:02.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00027218199999999999 2024-09-11T08:46:02.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:39.072721+0000", 2024-09-11T08:46:02.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.091959307000000004 2024-09-11T08:46:02.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:39.072771+0000", 2024-09-11T08:46:02.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9642999999999999e-05 2024-09-11T08:46:02.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:39.072809+0000", 2024-09-11T08:46:02.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7626000000000002e-05 2024-09-11T08:46:02.469 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.469 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.469 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.469 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:25731 8.2 8:442cb9b7:::benchmark_data_smithi092_117541_object25730:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159)", 2024-09-11T08:46:02.469 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:45:38.984772+0000", 2024-09-11T08:46:02.469 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 23.327222012, 2024-09-11T08:46:02.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.088168152999999999, 2024-09-11T08:46:02.469 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:02.469 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:02.469 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:02.470 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:46:02.470 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:46:02.470 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 25731 2024-09-11T08:46:02.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.470 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:02.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:02.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.984772+0000", 2024-09-11T08:46:02.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:02.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.984772+0000", 2024-09-11T08:46:02.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:02.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:02.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.984770+0000", 2024-09-11T08:46:02.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:02.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:02.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.984829+0000", 2024-09-11T08:46:02.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8822e-05 2024-09-11T08:46:02.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:02.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.984844+0000", 2024-09-11T08:46:02.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5213e-05 2024-09-11T08:46:02.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:02.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.984849+0000", 2024-09-11T08:46:02.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6249999999999998e-06 2024-09-11T08:46:02.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:02.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.984892+0000", 2024-09-11T08:46:02.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2849000000000003e-05 2024-09-11T08:46:02.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:02.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.985020+0000", 2024-09-11T08:46:02.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012812000000000001 2024-09-11T08:46:02.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:46:02.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.985163+0000", 2024-09-11T08:46:02.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014335300000000001 2024-09-11T08:46:02.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:02.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.986344+0000", 2024-09-11T08:46:02.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0011807790000000001 2024-09-11T08:46:02.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:38.986514+0000", 2024-09-11T08:46:02.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000169371 2024-09-11T08:46:02.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:02.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:39.072866+0000", 2024-09-11T08:46:02.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.086352132999999998 2024-09-11T08:46:02.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:02.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:39.072906+0000", 2024-09-11T08:46:02.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0095999999999997e-05 2024-09-11T08:46:02.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:02.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:02.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:02.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:45:39.072940+0000", 2024-09-11T08:46:02.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4267000000000002e-05 2024-09-11T08:46:02.475 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.475 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.475 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.475 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:02.475 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:02.475 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:46:02.476 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:02.493 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 7 1 2024-09-11T08:46:02.497 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:02.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:02.498 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:02.520 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:46:02.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:02.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:02.620 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:02.621 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:02.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:02.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:02.743 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:02.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:02.844 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:02.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:02.866 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:02.866 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:02.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:46:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:46:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:46:02.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:46:03.002 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:03.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:03.003 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:03.091 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:02 smithi092 ceph-mon[29647]: osdmap e168: 8 total, 8 up, 8 in 2024-09-11T08:46:03.091 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:02 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/424441494' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:46:03.091 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:02 smithi092 ceph-mon[41114]: osdmap e168: 8 total, 8 up, 8 in 2024-09-11T08:46:03.091 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:02 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/424441494' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:46:03.103 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:03.125 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:03.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:03.126 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:03.167 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 33 16 62948 62932 119.154 0 - 0.00784869 2024-09-11T08:46:03.213 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-11T08:46:03.227 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:03.236 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:02 smithi145 ceph-mon[33384]: osdmap e168: 8 total, 8 up, 8 in 2024-09-11T08:46:03.236 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:02 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/424441494' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:46:03.262 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:03.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:03.263 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:03.363 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:03.394 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:03.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:03.395 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:03.496 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:03.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:03.541 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:03.541 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:03.572 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:46:03.642 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:03.678 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:03.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:03.679 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:03.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:03.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:03.819 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:03.920 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:03.954 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:03.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:03.955 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:04.056 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:04.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:03 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:46:04.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:03 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/424441494' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:46:04.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:03 smithi092 ceph-mon[29647]: osdmap e169: 8 total, 8 up, 8 in 2024-09-11T08:46:04.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:03 smithi092 ceph-mon[29647]: 6.2 scrub starts 2024-09-11T08:46:04.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:03 smithi092 ceph-mon[29647]: 6.2 scrub ok 2024-09-11T08:46:04.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:03 smithi092 ceph-mon[29647]: pgmap v307: 148 pgs: 25 peering, 123 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 95 MiB/s wr, 1.51k op/s 2024-09-11T08:46:04.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:03 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T08:46:04.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:03 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:46:04.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:03 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/424441494' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:46:04.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:03 smithi092 ceph-mon[41114]: osdmap e169: 8 total, 8 up, 8 in 2024-09-11T08:46:04.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:03 smithi092 ceph-mon[41114]: 6.2 scrub starts 2024-09-11T08:46:04.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:03 smithi092 ceph-mon[41114]: 6.2 scrub ok 2024-09-11T08:46:04.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:03 smithi092 ceph-mon[41114]: pgmap v307: 148 pgs: 25 peering, 123 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 95 MiB/s wr, 1.51k op/s 2024-09-11T08:46:04.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:03 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T08:46:04.079 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:04.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:04.080 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:04.167 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 34 16 63983 63967 117.552 21.5625 0.00663512 0.00850082 2024-09-11T08:46:04.180 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:04.207 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:04.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:04.208 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:04.226 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:03 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:46:04.226 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:03 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/424441494' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:46:04.226 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:03 smithi145 ceph-mon[33384]: osdmap e169: 8 total, 8 up, 8 in 2024-09-11T08:46:04.226 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:03 smithi145 ceph-mon[33384]: 6.2 scrub starts 2024-09-11T08:46:04.226 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:03 smithi145 ceph-mon[33384]: 6.2 scrub ok 2024-09-11T08:46:04.226 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:03 smithi145 ceph-mon[33384]: pgmap v307: 148 pgs: 25 peering, 123 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 95 MiB/s wr, 1.51k op/s 2024-09-11T08:46:04.226 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:03 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T08:46:04.309 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:04.339 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:04.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:04.340 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:04.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:04.478 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:04.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:04.479 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:04.580 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:04.612 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:04.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:04.613 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:04.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:04.749 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:04.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:04.750 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:04.850 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:04.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:04.872 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:04.873 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:04.973 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:05.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:05.009 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:05.009 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:05.110 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:05.144 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:05.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:05.146 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:05.167 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 35 16 65117 65101 116.219 70.875 0.0491692 0.00859232 2024-09-11T08:46:05.247 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:05.282 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:05.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:05.284 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:05.384 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:05.421 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:05.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:05.422 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:05.523 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:05.552 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:05.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:05.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:05.654 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:05.686 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:05.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:05.687 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:05.693 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:46:05.693 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:46:05.693 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.693 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15126.0:65698 8.9 e169/149 8:973a9bf7:::benchmark_data_smithi092_117541_object65697:head v 169'4133, mlcod=169'4130)", 2024-09-11T08:46:05.693 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:05.686618+0000", 2024-09-11T08:46:05.694 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.0060217559999999996, 2024-09-11T08:46:05.694 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0060508749999999998, 2024-09-11T08:46:05.694 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:05.694 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T08:46:05.694 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:05.694 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.694 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:05.694 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.686618+0000", 2024-09-11T08:46:05.694 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:05.694 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.694 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.694 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:05.694 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.686618+0000", 2024-09-11T08:46:05.694 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:05.695 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.695 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.695 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:05.695 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.686617+0000", 2024-09-11T08:46:05.695 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:05.695 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.695 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.695 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:05.695 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.686641+0000", 2024-09-11T08:46:05.695 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.323e-05 2024-09-11T08:46:05.695 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.686653+0000", 2024-09-11T08:46:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2717e-05 2024-09-11T08:46:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.686659+0000", 2024-09-11T08:46:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6219999999999997e-06 2024-09-11T08:46:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.686696+0000", 2024-09-11T08:46:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6698e-05 2024-09-11T08:46:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.686715+0000", 2024-09-11T08:46:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.9313e-05 2024-09-11T08:46:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:65701 8.8 8:1c68c88c:::benchmark_data_smithi092_117541_object65700:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-11T08:46:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:05.688100+0000", 2024-09-11T08:46:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.0045398670000000004, 2024-09-11T08:46:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0046400010000000004, 2024-09-11T08:46:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:46:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:46:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:46:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 65701 2024-09-11T08:46:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.688100+0000", 2024-09-11T08:46:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.688100+0000", 2024-09-11T08:46:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.688099+0000", 2024-09-11T08:46:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.688149+0000", 2024-09-11T08:46:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9644000000000001e-05 2024-09-11T08:46:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.688160+0000", 2024-09-11T08:46:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1629e-05 2024-09-11T08:46:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.688165+0000", 2024-09-11T08:46:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2610000000000004e-06 2024-09-11T08:46:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.688202+0000", 2024-09-11T08:46:05.701 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7258999999999997e-05 2024-09-11T08:46:05.701 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.701 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.701 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:05.701 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.688337+0000", 2024-09-11T08:46:05.701 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013512499999999999 2024-09-11T08:46:05.701 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.701 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.701 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,7", 2024-09-11T08:46:05.701 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.688488+0000", 2024-09-11T08:46:05.701 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015063899999999999 2024-09-11T08:46:05.701 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15126.0:65699 8.9 e169/149 8:9e8671cf:::benchmark_data_smithi092_117541_object65698:head v 169'4134, mlcod=169'4130)", 2024-09-11T08:46:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:05.689335+0000", 2024-09-11T08:46:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.0033048109999999999, 2024-09-11T08:46:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0034732069999999999, 2024-09-11T08:46:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T08:46:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.689335+0000", 2024-09-11T08:46:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:05.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.703 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.703 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:05.703 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.689335+0000", 2024-09-11T08:46:05.703 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:05.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.703 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.703 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:05.703 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.689334+0000", 2024-09-11T08:46:05.703 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:05.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.703 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.703 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:05.703 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.689359+0000", 2024-09-11T08:46:05.704 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4868999999999999e-05 2024-09-11T08:46:05.704 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.704 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.704 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:05.704 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.689373+0000", 2024-09-11T08:46:05.704 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3954999999999999e-05 2024-09-11T08:46:05.704 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.704 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.704 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:05.704 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.689378+0000", 2024-09-11T08:46:05.704 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9200000000000003e-06 2024-09-11T08:46:05.704 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.704 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.705 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:05.705 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.689433+0000", 2024-09-11T08:46:05.705 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5254000000000003e-05 2024-09-11T08:46:05.705 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.705 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.705 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:05.705 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.689450+0000", 2024-09-11T08:46:05.705 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.713e-05 2024-09-11T08:46:05.705 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:05.705 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:05.705 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:05.705 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.705 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.705 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15126.0:65700 8.a e169/149 8:5fe24d66:::benchmark_data_smithi092_117541_object65699:head v 169'4259, mlcod=169'4257)", 2024-09-11T08:46:05.705 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:05.691657+0000", 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.00098323799999999999, 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0012036379999999999, 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.691657+0000", 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.691657+0000", 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.706 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.707 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:05.707 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.691655+0000", 2024-09-11T08:46:05.707 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:05.707 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.707 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.707 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:05.707 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.691688+0000", 2024-09-11T08:46:05.707 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2271999999999999e-05 2024-09-11T08:46:05.707 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.708 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.708 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:05.708 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.691708+0000", 2024-09-11T08:46:05.708 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0373999999999998e-05 2024-09-11T08:46:05.708 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.708 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.708 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:05.708 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.691715+0000", 2024-09-11T08:46:05.708 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8140000000000003e-06 2024-09-11T08:46:05.708 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.708 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.708 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:05.708 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.691774+0000", 2024-09-11T08:46:05.708 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9382999999999997e-05 2024-09-11T08:46:05.709 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.709 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.709 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:05.709 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.691793+0000", 2024-09-11T08:46:05.709 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.8961999999999999e-05 2024-09-11T08:46:05.709 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:05.709 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:05.709 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:05.709 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.709 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.709 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop_reply(client.15126.0:65701 8.8 e169/149)", 2024-09-11T08:46:05.709 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:05.692034+0000", 2024-09-11T08:46:05.709 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.00060549599999999996, 2024-09-11T08:46:05.710 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00087648200000000004, 2024-09-11T08:46:05.710 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:05.710 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "queued for pg", 2024-09-11T08:46:05.710 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:05.710 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.710 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:05.710 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.692034+0000", 2024-09-11T08:46:05.710 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:05.710 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.710 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.710 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:05.710 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.692034+0000", 2024-09-11T08:46:05.710 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:05.710 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.710 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.711 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:05.711 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.692034+0000", 2024-09-11T08:46:05.711 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:46:05.711 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.711 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.711 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:05.711 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.692038+0000", 2024-09-11T08:46:05.711 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7129999999999996e-06 2024-09-11T08:46:05.711 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.711 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.711 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:05.711 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.692051+0000", 2024-09-11T08:46:05.711 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2864000000000001e-05 2024-09-11T08:46:05.712 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.712 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.712 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:05.712 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.692056+0000", 2024-09-11T08:46:05.712 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7809999999999996e-06 2024-09-11T08:46:05.712 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:05.712 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:05.712 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:05.712 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.712 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.712 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop_reply(client.15126.0:65701 8.8 e169/149)", 2024-09-11T08:46:05.712 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:05.692126+0000", 2024-09-11T08:46:05.712 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.00051390699999999997, 2024-09-11T08:46:05.713 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00082393400000000002, 2024-09-11T08:46:05.713 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:05.713 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "queued for pg", 2024-09-11T08:46:05.713 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:05.713 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.713 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:05.713 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.692126+0000", 2024-09-11T08:46:05.713 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:05.713 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.713 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.713 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:05.713 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.692126+0000", 2024-09-11T08:46:05.713 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:05.713 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.714 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.714 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:05.714 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.692125+0000", 2024-09-11T08:46:05.714 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:05.714 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.714 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.714 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:05.714 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.692131+0000", 2024-09-11T08:46:05.714 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9599999999999997e-06 2024-09-11T08:46:05.714 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.714 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.714 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:05.714 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.692142+0000", 2024-09-11T08:46:05.714 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1402e-05 2024-09-11T08:46:05.715 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:05.715 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:05.715 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:05.715 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:05.692148+0000", 2024-09-11T08:46:05.715 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.575e-06 2024-09-11T08:46:05.715 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:05.715 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:05.715 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:05.715 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:05.715 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2024-09-11T08:46:05.715 INFO:teuthology.orchestra.run.smithi145.stdout: "num_ops": 6 2024-09-11T08:46:05.715 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:46:05.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:05.822 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:05.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:05.823 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:05.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:05.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:05.963 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:05.963 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:06.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:06.098 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:06.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:06.099 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:06.167 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 36 16 66295 66279 115.036 73.625 0.00769216 0.00868735 2024-09-11T08:46:06.201 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:06.229 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:06.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:06.230 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:06.331 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:06.359 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:06.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:06.360 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:06.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:06 smithi145 ceph-mon[33384]: pgmap v308: 148 pgs: 25 peering, 123 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 69 MiB/s wr, 1.09k op/s 2024-09-11T08:46:06.461 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:06.489 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:06 smithi092 ceph-mon[29647]: pgmap v308: 148 pgs: 25 peering, 123 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 69 MiB/s wr, 1.09k op/s 2024-09-11T08:46:06.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:06 smithi092 ceph-mon[41114]: pgmap v308: 148 pgs: 25 peering, 123 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 69 MiB/s wr, 1.09k op/s 2024-09-11T08:46:06.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:06.504 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:06.504 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:06.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:06.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:06.642 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:06.642 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:06.743 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:06.786 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:06.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:06.787 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:06.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:46:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:46:06.850 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-11T08:46:06.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:06.930 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:06.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:06.931 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:07.032 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:07.071 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:07.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:07.072 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:07.167 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 37 16 67485 67469 113.937 74.375 0.0164985 0.00877086 2024-09-11T08:46:07.173 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:07.213 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:07.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:07.214 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:07.236 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:46:07.316 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:07.348 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:07.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:07.349 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:07.368 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:07 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:46:07.368 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:07 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:46:07.368 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:07 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:07.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:07 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:46:07.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:07 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:46:07.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:07 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:07.384 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:46:07.445 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:46:07.445 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:46:07.450 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:07.475 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:46:07.475 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:46:07.479 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:07.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:07.480 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:07.512 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:46:07.512 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:46:07.536 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:07 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:46:07.536 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:07 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:46:07.537 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:07 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:07.541 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:46:07.542 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:46:07.571 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:46:07.571 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:46:07.581 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:07.604 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:46:07.604 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:46:07.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:07.613 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:07.613 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:07.637 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:46:07.638 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:46:07.668 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:46:07.668 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:46:07.714 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:07.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:07.741 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:07.741 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:07.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:07.879 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:07.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:07.880 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:07.981 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:08.010 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:08.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:08.011 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:08.112 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:08.145 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:08.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:08.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:08.167 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 38 16 68605 68589 112.781 70 0.0394986 0.0088608 2024-09-11T08:46:08.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:08.289 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:08.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:08.290 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:08.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:08 smithi092 ceph-mon[29647]: pgmap v309: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.6 KiB/s rd, 107 MiB/s wr, 1.71k op/s; 7610/200040 objects misplaced (3.804%); 1.6 MiB/s, 31 objects/s recovering 2024-09-11T08:46:08.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:08 smithi092 ceph-mon[29647]: osdmap e170: 8 total, 8 up, 8 in 2024-09-11T08:46:08.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:08 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T08:46:08.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:08 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2820819588' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:46:08.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:08 smithi092 ceph-mon[41114]: pgmap v309: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.6 KiB/s rd, 107 MiB/s wr, 1.71k op/s; 7610/200040 objects misplaced (3.804%); 1.6 MiB/s, 31 objects/s recovering 2024-09-11T08:46:08.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:08 smithi092 ceph-mon[41114]: osdmap e170: 8 total, 8 up, 8 in 2024-09-11T08:46:08.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:08 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T08:46:08.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:08 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2820819588' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:46:08.390 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:08.420 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:08.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:08.421 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:08.503 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:08 smithi145 ceph-mon[33384]: pgmap v309: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.6 KiB/s rd, 107 MiB/s wr, 1.71k op/s; 7610/200040 objects misplaced (3.804%); 1.6 MiB/s, 31 objects/s recovering 2024-09-11T08:46:08.503 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:08 smithi145 ceph-mon[33384]: osdmap e170: 8 total, 8 up, 8 in 2024-09-11T08:46:08.504 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:08 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T08:46:08.504 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:08 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2820819588' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:46:08.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:08.582 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:08.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:08.583 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:08.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:08.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:08.721 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:08.721 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:08.822 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:08.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:46:08.856 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:08.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:08.857 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:08.958 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:08.991 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:08.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:08.992 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:09.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:09.132 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:46:09.132 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:46:09.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:09.142 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:09.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:09.167 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 39 16 69687 69671 111.622 67.625 0.0053467 0.008953 2024-09-11T08:46:09.243 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:09.273 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:09.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:09.274 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:09.375 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:09.411 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:09.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:09.412 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:09.513 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:09.546 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:09.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:09.547 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:09.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:09.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:09.704 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:09.705 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:09.805 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:09.834 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:09.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:09.835 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:09.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:09.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:09.987 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:09.987 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:10.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:09 smithi092 ceph-mon[29647]: 2.2 scrub starts 2024-09-11T08:46:10.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:09 smithi092 ceph-mon[29647]: osdmap e171: 8 total, 8 up, 8 in 2024-09-11T08:46:10.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:09 smithi092 ceph-mon[29647]: pgmap v312: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 8.3 KiB/s rd, 40 MiB/s wr, 664 op/s; 7610/200040 objects misplaced (3.804%); 1.6 MiB/s, 35 objects/s recovering 2024-09-11T08:46:10.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:09 smithi092 ceph-mon[41114]: 2.2 scrub starts 2024-09-11T08:46:10.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:09 smithi092 ceph-mon[41114]: osdmap e171: 8 total, 8 up, 8 in 2024-09-11T08:46:10.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:09 smithi092 ceph-mon[41114]: pgmap v312: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 8.3 KiB/s rd, 40 MiB/s wr, 664 op/s; 7610/200040 objects misplaced (3.804%); 1.6 MiB/s, 35 objects/s recovering 2024-09-11T08:46:10.088 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:10.122 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:10.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:10.123 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:10.134 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:09 smithi145 ceph-mon[33384]: 2.2 scrub starts 2024-09-11T08:46:10.134 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:09 smithi145 ceph-mon[33384]: osdmap e171: 8 total, 8 up, 8 in 2024-09-11T08:46:10.134 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:09 smithi145 ceph-mon[33384]: pgmap v312: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 8.3 KiB/s rd, 40 MiB/s wr, 664 op/s; 7610/200040 objects misplaced (3.804%); 1.6 MiB/s, 35 objects/s recovering 2024-09-11T08:46:10.168 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:46:10.169161+0000 min lat: 0.00108577 max lat: 2.31233 avg lat: 0.00904014 2024-09-11T08:46:10.168 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:46:10.168 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 40 15 70784 70769 110.547 68.625 0.0124186 0.00904014 2024-09-11T08:46:10.224 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:10.234 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:46:10.249 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:10.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:10.250 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:10.351 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:10.379 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:10.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:10.380 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:10.481 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:10.503 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:46:10.508 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-11T08:46:10.512 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:10.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:10.514 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:10.615 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:10.648 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:10.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:10.649 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:10.750 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:10.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:10.789 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:10.789 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:10.832 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.7 to 1 (10000) 2024-09-11T08:46:10.866 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:46:10.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:10.926 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:10.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:10.927 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:11.028 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:11.062 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:11.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:11.063 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:11.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:10 smithi092 ceph-mon[29647]: 5.1b scrub starts 2024-09-11T08:46:11.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:10 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/314584437' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T08:46:11.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:10 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T08:46:11.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:10 smithi092 ceph-mon[41114]: 5.1b scrub starts 2024-09-11T08:46:11.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:10 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/314584437' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T08:46:11.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:10 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T08:46:11.155 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:10 smithi145 ceph-mon[33384]: 5.1b scrub starts 2024-09-11T08:46:11.155 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:10 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/314584437' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T08:46:11.156 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:10 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T08:46:11.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:11.167 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 41 15 71780 71765 109.369 62.25 0.00737188 0.00913718 2024-09-11T08:46:11.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:11.200 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:11.200 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:11.301 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:11.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:11.338 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:11.338 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:11.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:11.476 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:11.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:11.477 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:11.578 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:11.614 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:11.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:11.615 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:11.716 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:11.741 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:11.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:11.742 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:11.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:46:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:46:11] "GET /metrics HTTP/1.1" 200 224183 "" "Prometheus/2.43.0" 2024-09-11T08:46:11.843 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:11.875 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:11.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:11.877 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:11.977 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:12.006 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:12.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:12.007 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:12.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:12.120 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:11 smithi092 ceph-mon[29647]: 6.7 scrub starts 2024-09-11T08:46:12.120 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:11 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T08:46:12.120 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:11 smithi092 ceph-mon[29647]: osdmap e172: 8 total, 8 up, 8 in 2024-09-11T08:46:12.120 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:11 smithi092 ceph-mon[29647]: pgmap v314: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 9.3 KiB/s rd, 59 MiB/s wr, 967 op/s; 7610/205131 objects misplaced (3.710%); 1.7 MiB/s, 36 objects/s recovering 2024-09-11T08:46:12.121 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:11 smithi092 ceph-mon[41114]: 6.7 scrub starts 2024-09-11T08:46:12.121 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:11 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T08:46:12.121 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:11 smithi092 ceph-mon[41114]: osdmap e172: 8 total, 8 up, 8 in 2024-09-11T08:46:12.121 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:11 smithi092 ceph-mon[41114]: pgmap v314: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 9.3 KiB/s rd, 59 MiB/s wr, 967 op/s; 7610/205131 objects misplaced (3.710%); 1.7 MiB/s, 36 objects/s recovering 2024-09-11T08:46:12.146 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:12.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:12.147 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:12.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:11 smithi145 ceph-mon[33384]: 6.7 scrub starts 2024-09-11T08:46:12.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:11 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T08:46:12.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:11 smithi145 ceph-mon[33384]: osdmap e172: 8 total, 8 up, 8 in 2024-09-11T08:46:12.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:11 smithi145 ceph-mon[33384]: pgmap v314: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 9.3 KiB/s rd, 59 MiB/s wr, 967 op/s; 7610/205131 objects misplaced (3.710%); 1.7 MiB/s, 36 objects/s recovering 2024-09-11T08:46:12.168 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 42 16 72902 72886 108.434 70.0625 0.0152007 0.00921023 2024-09-11T08:46:12.248 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:12.277 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:12.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:12.278 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:12.379 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:12.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:12.407 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:12.408 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:12.508 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:12.544 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:12.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:12.545 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:12.646 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:12.681 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:12.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:12.682 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:12.783 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:12.819 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:12.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:12.820 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:12.884 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:46:12.885 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:46:12.892 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:12 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:46:12.893 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:12 smithi145 ceph-mon[33384]: 4.8 scrub starts 2024-09-11T08:46:12.893 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:46:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:46:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:46:12.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:12.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:12.958 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:12.958 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:13.059 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:13.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:12 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:46:13.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:12 smithi092 ceph-mon[29647]: 4.8 scrub starts 2024-09-11T08:46:13.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:12 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:46:13.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:12 smithi092 ceph-mon[41114]: 4.8 scrub starts 2024-09-11T08:46:13.090 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:13.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:13.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:13.168 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 43 16 73685 73669 107.05 48.9375 0.00630795 0.00933559 2024-09-11T08:46:13.191 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:13.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:13.213 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:13.214 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:13.314 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:13.339 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:13.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:13.340 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:13.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:13.482 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:13.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:13.483 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:13.584 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:13.613 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:13.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:13.614 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:13.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:13.732 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-11T08:46:13.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:13.753 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:13.753 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:13.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:13.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:13.900 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:14.001 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:14.044 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:14.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:14.045 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:13 smithi092 ceph-mon[29647]: 5.b scrub starts 2024-09-11T08:46:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:13 smithi092 ceph-mon[29647]: 5.7 scrub starts 2024-09-11T08:46:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:13 smithi092 ceph-mon[29647]: pgmap v315: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 57 MiB/s wr, 913 op/s; 7333/216474 objects misplaced (3.387%); 2.9 MiB/s, 46 objects/s recovering 2024-09-11T08:46:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:13 smithi092 ceph-mon[29647]: osdmap e173: 8 total, 8 up, 8 in 2024-09-11T08:46:14.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:13 smithi092 ceph-mon[41114]: 5.b scrub starts 2024-09-11T08:46:14.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:13 smithi092 ceph-mon[41114]: 5.7 scrub starts 2024-09-11T08:46:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:13 smithi092 ceph-mon[41114]: pgmap v315: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 57 MiB/s wr, 913 op/s; 7333/216474 objects misplaced (3.387%); 2.9 MiB/s, 46 objects/s recovering 2024-09-11T08:46:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:13 smithi092 ceph-mon[41114]: osdmap e173: 8 total, 8 up, 8 in 2024-09-11T08:46:14.076 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:46:14.146 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:14.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:13 smithi145 ceph-mon[33384]: 5.b scrub starts 2024-09-11T08:46:14.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:13 smithi145 ceph-mon[33384]: 5.7 scrub starts 2024-09-11T08:46:14.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:13 smithi145 ceph-mon[33384]: pgmap v315: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 57 MiB/s wr, 913 op/s; 7333/216474 objects misplaced (3.387%); 2.9 MiB/s, 46 objects/s recovering 2024-09-11T08:46:14.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:13 smithi145 ceph-mon[33384]: osdmap e173: 8 total, 8 up, 8 in 2024-09-11T08:46:14.168 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 44 16 74541 74525 105.833 53.5 0.00435066 0.00944088 2024-09-11T08:46:14.171 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:14.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:14.172 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:14.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:14.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:14.307 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:14.307 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:14.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:14.443 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:14.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:14.445 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:14.546 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:14.573 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:14.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:14.574 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:14.675 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:14.706 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:14.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:14.707 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:14.808 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:14.838 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:14.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:14.839 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:14.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:14.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:14.983 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:14.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:15.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:15.124 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:15.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:15.126 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:15.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:14 smithi145 ceph-mon[33384]: 5.6 scrub starts 2024-09-11T08:46:15.168 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 45 16 75557 75541 104.892 63.5 0.0365486 0.00952845 2024-09-11T08:46:15.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:15.259 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:14 smithi092 ceph-mon[41114]: 5.6 scrub starts 2024-09-11T08:46:15.259 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:14 smithi092 ceph-mon[29647]: 5.6 scrub starts 2024-09-11T08:46:15.269 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:15.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:15.271 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:15.371 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:15.413 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:15.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:15.414 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:15.514 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:15.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:15.550 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:15.550 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:15.651 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:15.683 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:15.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:15.684 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:15.742 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T08:46:15.742 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:46:15.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:15.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:15.843 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:15.843 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:15.944 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:15.953 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:15.953 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:15 smithi092 ceph-mon[41114]: pgmap v317: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 805 B/s rd, 54 MiB/s wr, 863 op/s; 7333/216474 objects misplaced (3.387%); 2.7 MiB/s, 43 objects/s recovering 2024-09-11T08:46:15.954 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:15.954 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:15 smithi092 ceph-mon[29647]: pgmap v317: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 805 B/s rd, 54 MiB/s wr, 863 op/s; 7333/216474 objects misplaced (3.387%); 2.7 MiB/s, 43 objects/s recovering 2024-09-11T08:46:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:15.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:15.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:16.097 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:16.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:16.119 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:16.119 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:16.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:15 smithi145 ceph-mon[33384]: pgmap v317: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 805 B/s rd, 54 MiB/s wr, 863 op/s; 7333/216474 objects misplaced (3.387%); 2.7 MiB/s, 43 objects/s recovering 2024-09-11T08:46:16.168 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 46 16 76585 76569 104.009 64.25 0.0243931 0.00960925 2024-09-11T08:46:16.221 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:16.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:16.259 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:16.259 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:16.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:16.398 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:16.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:16.399 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:16.499 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:16.509 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:46:16.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:16.529 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:16.530 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:16.549 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:46:16.550 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:46:16.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:46:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:46:16.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:16.678 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:16.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:16.679 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:16.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:16.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:16.833 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:16.933 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:16.963 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:16.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:16.964 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:17.065 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:17.069 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:16 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T08:46:17.069 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:16 smithi092 ceph-mon[41114]: 4.15 scrub starts 2024-09-11T08:46:17.069 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:16 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:17.069 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:16 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T08:46:17.069 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:16 smithi092 ceph-mon[29647]: 4.15 scrub starts 2024-09-11T08:46:17.069 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:16 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:17.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:17.096 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:17.096 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:17.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:16 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T08:46:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:16 smithi145 ceph-mon[33384]: 4.15 scrub starts 2024-09-11T08:46:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:16 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:17.168 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 47 16 77627 77611 103.182 65.125 0.0115211 0.00968616 2024-09-11T08:46:17.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:17.244 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:17.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:17.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:17.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:17.382 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-11T08:46:17.395 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:17.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:17.396 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:17.497 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:17.537 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:17.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:17.538 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:17.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:17.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:17.675 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:17.719 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:46:17.776 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:17.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:17.803 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:17.803 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:17.892 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:46:17.904 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:17.909 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:46:17.929 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:17.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:17.930 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:18.031 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:18.068 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:18.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:18.069 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:18.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:17 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T08:46:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:17 smithi145 ceph-mon[33384]: pgmap v318: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.28k op/s; 7058/230880 objects misplaced (3.057%); 4.3 MiB/s, 68 objects/s recovering 2024-09-11T08:46:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:17 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3280737869' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:46:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:17 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:46:18.168 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 48 16 78616 78600 102.319 61.8125 0.00401872 0.00976748 2024-09-11T08:46:18.171 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:18.173 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:17 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T08:46:18.173 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:17 smithi092 ceph-mon[29647]: pgmap v318: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.28k op/s; 7058/230880 objects misplaced (3.057%); 4.3 MiB/s, 68 objects/s recovering 2024-09-11T08:46:18.173 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:17 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3280737869' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:46:18.173 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:17 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:46:18.173 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:17 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T08:46:18.173 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:17 smithi092 ceph-mon[41114]: pgmap v318: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.28k op/s; 7058/230880 objects misplaced (3.057%); 4.3 MiB/s, 68 objects/s recovering 2024-09-11T08:46:18.173 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:17 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3280737869' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:46:18.173 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:17 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:46:18.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:18.216 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:18.216 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:18.317 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:18.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:18.356 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:18.456 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:18.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:18.495 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:18.495 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:18.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:18.637 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:18.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:18.638 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:18.738 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:18.776 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:18.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:18.777 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:18.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:18.911 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:18.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:18.912 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:19.013 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:19.018 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:18 smithi145 ceph-mon[33384]: 3.1d scrub starts 2024-09-11T08:46:19.018 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:18 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:46:19.018 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:18 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:46:19.018 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:18 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:46:19.018 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:18 smithi145 ceph-mon[33384]: osdmap e174: 8 total, 8 up, 8 in 2024-09-11T08:46:19.019 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:18 smithi145 ceph-mon[33384]: osdmap e175: 8 total, 8 up, 8 in 2024-09-11T08:46:19.051 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:19.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:19.052 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:19.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:19.168 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 49 16 79651 79635 101.551 64.6875 0.0111364 0.00984133 2024-09-11T08:46:19.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:19.202 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:19.203 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:19.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:18 smithi092 ceph-mon[29647]: 3.1d scrub starts 2024-09-11T08:46:19.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:18 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:46:19.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:18 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:46:19.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:18 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:46:19.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:18 smithi092 ceph-mon[29647]: osdmap e174: 8 total, 8 up, 8 in 2024-09-11T08:46:19.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:18 smithi092 ceph-mon[29647]: osdmap e175: 8 total, 8 up, 8 in 2024-09-11T08:46:19.214 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:18 smithi092 ceph-mon[41114]: 3.1d scrub starts 2024-09-11T08:46:19.215 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:18 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:46:19.215 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:18 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:46:19.215 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:18 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:46:19.215 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:18 smithi092 ceph-mon[41114]: osdmap e174: 8 total, 8 up, 8 in 2024-09-11T08:46:19.215 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:18 smithi092 ceph-mon[41114]: osdmap e175: 8 total, 8 up, 8 in 2024-09-11T08:46:19.304 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:19.334 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:19.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:19.335 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:19.435 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:19.465 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:19.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:19.466 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:19.567 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:19.599 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:19.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:19.600 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:19.678 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:46:19.678 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:46:19.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:19.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:19.738 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:19.839 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:19.874 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:19.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:19.875 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:19.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:20.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:20.027 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:20.027 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:20.128 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:20.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:19 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T08:46:20.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:19 smithi145 ceph-mon[33384]: pgmap v321: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 50 MiB/s wr, 800 op/s; 7058/230880 objects misplaced (3.057%); 2.9 MiB/s, 45 objects/s recovering 2024-09-11T08:46:20.163 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:46:20.163 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:46:20.164 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:19 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T08:46:20.164 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:19 smithi092 ceph-mon[29647]: pgmap v321: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 50 MiB/s wr, 800 op/s; 7058/230880 objects misplaced (3.057%); 2.9 MiB/s, 45 objects/s recovering 2024-09-11T08:46:20.164 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:19 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T08:46:20.164 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:19 smithi092 ceph-mon[41114]: pgmap v321: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 50 MiB/s wr, 800 op/s; 7058/230880 objects misplaced (3.057%); 2.9 MiB/s, 45 objects/s recovering 2024-09-11T08:46:20.169 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 50 16 80705 80689 100.838 65.875 0.0106343 0.00991019 2024-09-11T08:46:20.173 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:20.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:20.174 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:20.199 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:46:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:46:20.232 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:46:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:46:20.265 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:46:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:46:20.275 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:20.292 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:46:20.292 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:46:20.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:20.311 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:20.311 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:20.320 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:46:20.321 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:46:20.353 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:46:20.354 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:46:20.384 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:46:20.384 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:46:20.398 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-11T08:46:20.412 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:20.470 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:20.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:20.472 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:20.573 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:20.606 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:20.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:20.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:20.708 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:20.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:20.744 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:20.745 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:20.780 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:46:20.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:20.882 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:20.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:20.883 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:20.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:21.022 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:21.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:21.023 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:21.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:21.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:20 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4100894107' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:46:21.163 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:21.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:21.164 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:21.169 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 51 16 81787 81771 100.187 67.625 0.00872247 0.00997613 2024-09-11T08:46:21.197 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:20 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4100894107' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:46:21.197 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:20 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4100894107' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:46:21.265 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:21.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:21.298 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:21.298 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:21.399 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:21.432 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:21.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:21.433 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:21.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:21.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:46:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:46:21] "GET /metrics HTTP/1.1" 200 224241 "" "Prometheus/2.43.0" 2024-09-11T08:46:21.570 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:21.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:21.571 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:21.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:21.708 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:21.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:21.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:21.810 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:21.844 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:21.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:21.846 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:21.946 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:21.980 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:21.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:21.981 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:22.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:22.130 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:22.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:22.132 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:22.169 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 52 16 82843 82827 99.5291 66 0.00585916 0.0100413 2024-09-11T08:46:22.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:22.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:46:22.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:22.273 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:22.274 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:22.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:21 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:46:22.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:21 smithi092 ceph-mon[29647]: pgmap v322: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 55 MiB/s wr, 883 op/s; 7058/235917 objects misplaced (2.992%); 2.3 MiB/s, 37 objects/s recovering 2024-09-11T08:46:22.312 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:21 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:46:22.312 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:21 smithi092 ceph-mon[41114]: pgmap v322: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 55 MiB/s wr, 883 op/s; 7058/235917 objects misplaced (2.992%); 2.3 MiB/s, 37 objects/s recovering 2024-09-11T08:46:22.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:22.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:21 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:46:22.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:21 smithi145 ceph-mon[33384]: pgmap v322: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 55 MiB/s wr, 883 op/s; 7058/235917 objects misplaced (2.992%); 2.3 MiB/s, 37 objects/s recovering 2024-09-11T08:46:22.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:22.418 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:22.418 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:22.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:22.548 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:22.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:22.549 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:22.650 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:22.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:22.698 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:22.698 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:22.799 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:22.829 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:22.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:22.830 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:22.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:22.967 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:22.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:22.968 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:22.991 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:46:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:46:23.055 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:46:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:46:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:46:23.056 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:22 smithi145 ceph-mon[33384]: 5.b scrub starts 2024-09-11T08:46:23.056 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:22 smithi145 ceph-mon[33384]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:46:23.069 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:23.105 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:23.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:23.106 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:23.169 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 53 16 83708 83692 98.6712 54.0625 0.0208877 0.0101297 2024-09-11T08:46:23.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:23.242 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:23.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:23.243 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:23.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:22 smithi092 ceph-mon[29647]: 5.b scrub starts 2024-09-11T08:46:23.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:22 smithi092 ceph-mon[29647]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:46:23.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:22 smithi092 ceph-mon[41114]: 5.b scrub starts 2024-09-11T08:46:23.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:22 smithi092 ceph-mon[41114]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:46:23.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:23.367 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:23.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:23.368 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:23.469 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:23.497 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:23.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:23.498 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:23.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:23.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:23.638 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:23.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:46:23.699 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:46:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T08:46:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T08:46:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:46:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:115418 7.7 7:eb0976f3:::benchmark_data_smithi092_84565_object115417:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:46:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:31.169655+0000", 2024-09-11T08:46:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 232.525807765, 2024-09-11T08:46:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.116402226, 2024-09-11T08:46:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 115418 2024-09-11T08:46:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.169655+0000", 2024-09-11T08:46:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.169655+0000", 2024-09-11T08:46:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.169653+0000", 2024-09-11T08:46:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:46:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.169724+0000", 2024-09-11T08:46:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.0636999999999995e-05 2024-09-11T08:46:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.169740+0000", 2024-09-11T08:46:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6290000000000002e-05 2024-09-11T08:46:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.169747+0000", 2024-09-11T08:46:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6239999999999996e-06 2024-09-11T08:46:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.170207+0000", 2024-09-11T08:46:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00046066800000000001 2024-09-11T08:46:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.170323+0000", 2024-09-11T08:46:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000115939 2024-09-11T08:46:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,3", 2024-09-11T08:46:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.170468+0000", 2024-09-11T08:46:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000144876 2024-09-11T08:46:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.173268+0000", 2024-09-11T08:46:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0028000450000000001 2024-09-11T08:46:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.182445+0000", 2024-09-11T08:46:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0091763790000000001 2024-09-11T08:46:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.285955+0000", 2024-09-11T08:46:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.103510519 2024-09-11T08:46:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.286011+0000", 2024-09-11T08:46:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5918000000000002e-05 2024-09-11T08:46:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.286057+0000", 2024-09-11T08:46:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6059000000000001e-05 2024-09-11T08:46:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:115425 7.7 7:e4dc9892:::benchmark_data_smithi092_84565_object115424:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:46:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:31.177874+0000", 2024-09-11T08:46:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 232.51758897799999, 2024-09-11T08:46:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.11603140200000001, 2024-09-11T08:46:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 115425 2024-09-11T08:46:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177874+0000", 2024-09-11T08:46:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177874+0000", 2024-09-11T08:46:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177873+0000", 2024-09-11T08:46:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177919+0000", 2024-09-11T08:46:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6035000000000002e-05 2024-09-11T08:46:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177930+0000", 2024-09-11T08:46:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1321999999999999e-05 2024-09-11T08:46:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.177935+0000", 2024-09-11T08:46:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5029999999999999e-06 2024-09-11T08:46:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.179993+0000", 2024-09-11T08:46:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0020583189999999999 2024-09-11T08:46:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.180067+0000", 2024-09-11T08:46:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4243000000000001e-05 2024-09-11T08:46:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,3", 2024-09-11T08:46:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.180152+0000", 2024-09-11T08:46:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.4531999999999996e-05 2024-09-11T08:46:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.181892+0000", 2024-09-11T08:46:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0017399970000000001 2024-09-11T08:46:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.185772+0000", 2024-09-11T08:46:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0038804880000000001 2024-09-11T08:46:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.293815+0000", 2024-09-11T08:46:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.108043282 2024-09-11T08:46:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.293870+0000", 2024-09-11T08:46:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4389999999999999e-05 2024-09-11T08:46:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.293905+0000", 2024-09-11T08:46:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5357e-05 2024-09-11T08:46:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122670 7.7 7:e7a085d3:::benchmark_data_smithi092_84565_object122669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:46:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.707082+0000", 2024-09-11T08:46:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 224.98838048499999, 2024-09-11T08:46:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.114477678, 2024-09-11T08:46:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122670 2024-09-11T08:46:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707082+0000", 2024-09-11T08:46:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707082+0000", 2024-09-11T08:46:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707072+0000", 2024-09-11T08:46:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999895 2024-09-11T08:46:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.718 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.718 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.718 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707134+0000", 2024-09-11T08:46:23.718 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2637000000000004e-05 2024-09-11T08:46:23.718 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.718 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.718 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.718 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707148+0000", 2024-09-11T08:46:23.719 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3801999999999999e-05 2024-09-11T08:46:23.719 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.719 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.719 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.719 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707153+0000", 2024-09-11T08:46:23.719 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1939999999999997e-06 2024-09-11T08:46:23.719 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.719 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.719 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.719 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707202+0000", 2024-09-11T08:46:23.719 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8974000000000002e-05 2024-09-11T08:46:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707333+0000", 2024-09-11T08:46:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013053300000000001 2024-09-11T08:46:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,3", 2024-09-11T08:46:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707436+0000", 2024-09-11T08:46:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010262700000000001 2024-09-11T08:46:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.710596+0000", 2024-09-11T08:46:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0031604189999999998 2024-09-11T08:46:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.809030+0000", 2024-09-11T08:46:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.098434113000000004 2024-09-11T08:46:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.821512+0000", 2024-09-11T08:46:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.012481628999999999 2024-09-11T08:46:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.821539+0000", 2024-09-11T08:46:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7396e-05 2024-09-11T08:46:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.821560+0000", 2024-09-11T08:46:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0849e-05 2024-09-11T08:46:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122672 7.c 7:30125efa:::benchmark_data_smithi092_84565_object122671:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:46:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.707596+0000", 2024-09-11T08:46:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 224.98786657400001, 2024-09-11T08:46:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.113226995, 2024-09-11T08:46:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122672 2024-09-11T08:46:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707596+0000", 2024-09-11T08:46:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707596+0000", 2024-09-11T08:46:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707595+0000", 2024-09-11T08:46:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707624+0000", 2024-09-11T08:46:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.8662000000000001e-05 2024-09-11T08:46:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707638+0000", 2024-09-11T08:46:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3611000000000001e-05 2024-09-11T08:46:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707642+0000", 2024-09-11T08:46:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6689999999999997e-06 2024-09-11T08:46:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707681+0000", 2024-09-11T08:46:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8810000000000003e-05 2024-09-11T08:46:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707776+0000", 2024-09-11T08:46:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.4856000000000001e-05 2024-09-11T08:46:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,7", 2024-09-11T08:46:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707900+0000", 2024-09-11T08:46:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000124156 2024-09-11T08:46:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.710823+0000", 2024-09-11T08:46:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0029230459999999999 2024-09-11T08:46:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.711721+0000", 2024-09-11T08:46:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000897385 2024-09-11T08:46:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.820742+0000", 2024-09-11T08:46:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10902121300000001 2024-09-11T08:46:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.820787+0000", 2024-09-11T08:46:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5170000000000003e-05 2024-09-11T08:46:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.820823+0000", 2024-09-11T08:46:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6226999999999997e-05 2024-09-11T08:46:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:149681 7.d 7:b6ab2c94:::benchmark_data_smithi092_84565_object24153:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:46:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:06.851807+0000", 2024-09-11T08:46:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 196.843655321, 2024-09-11T08:46:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.115632797, 2024-09-11T08:46:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 149681 2024-09-11T08:46:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.851807+0000", 2024-09-11T08:46:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.851807+0000", 2024-09-11T08:46:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.851807+0000", 2024-09-11T08:46:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.851811+0000", 2024-09-11T08:46:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6879999999999998e-06 2024-09-11T08:46:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.851822+0000", 2024-09-11T08:46:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0293e-05 2024-09-11T08:46:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.851827+0000", 2024-09-11T08:46:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4029999999999998e-06 2024-09-11T08:46:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.851972+0000", 2024-09-11T08:46:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014493700000000001 2024-09-11T08:46:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852343+0000", 2024-09-11T08:46:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00037107900000000002 2024-09-11T08:46:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:46:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852440+0000", 2024-09-11T08:46:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.7342000000000004e-05 2024-09-11T08:46:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.860532+0000", 2024-09-11T08:46:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0080917650000000008 2024-09-11T08:46:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.860574+0000", 2024-09-11T08:46:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1653999999999998e-05 2024-09-11T08:46:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.967384+0000", 2024-09-11T08:46:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.106809742 2024-09-11T08:46:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.967416+0000", 2024-09-11T08:46:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2274000000000003e-05 2024-09-11T08:46:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.967440+0000", 2024-09-11T08:46:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4416000000000001e-05 2024-09-11T08:46:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:149682 7.d 7:b1cf1844:::benchmark_data_smithi092_84565_object24154:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:46:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:06.852894+0000", 2024-09-11T08:46:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 196.84256879399999, 2024-09-11T08:46:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.114631739, 2024-09-11T08:46:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 149682 2024-09-11T08:46:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852894+0000", 2024-09-11T08:46:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852894+0000", 2024-09-11T08:46:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852893+0000", 2024-09-11T08:46:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852898+0000", 2024-09-11T08:46:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.2000000000000002e-06 2024-09-11T08:46:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852912+0000", 2024-09-11T08:46:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3471e-05 2024-09-11T08:46:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.852918+0000", 2024-09-11T08:46:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8760000000000002e-06 2024-09-11T08:46:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.857394+0000", 2024-09-11T08:46:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0044762539999999998 2024-09-11T08:46:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.857536+0000", 2024-09-11T08:46:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000142022 2024-09-11T08:46:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:46:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.857661+0000", 2024-09-11T08:46:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000125069 2024-09-11T08:46:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.860677+0000", 2024-09-11T08:46:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003015804 2024-09-11T08:46:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.861044+0000", 2024-09-11T08:46:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00036762900000000001 2024-09-11T08:46:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.967467+0000", 2024-09-11T08:46:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.106422302 2024-09-11T08:46:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.967496+0000", 2024-09-11T08:46:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.8878999999999999e-05 2024-09-11T08:46:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:06.967526+0000", 2024-09-11T08:46:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0128999999999999e-05 2024-09-11T08:46:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:150198 7.7 7:e4f1bebe:::benchmark_data_smithi092_84565_object24670:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:46:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:07.404760+0000", 2024-09-11T08:46:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 196.29070234, 2024-09-11T08:46:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.117250244, 2024-09-11T08:46:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 150198 2024-09-11T08:46:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404760+0000", 2024-09-11T08:46:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404760+0000", 2024-09-11T08:46:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404759+0000", 2024-09-11T08:46:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404767+0000", 2024-09-11T08:46:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7179999999999996e-06 2024-09-11T08:46:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404781+0000", 2024-09-11T08:46:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4462e-05 2024-09-11T08:46:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404787+0000", 2024-09-11T08:46:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4720000000000003e-06 2024-09-11T08:46:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404832+0000", 2024-09-11T08:46:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5782999999999999e-05 2024-09-11T08:46:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.404977+0000", 2024-09-11T08:46:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014502100000000001 2024-09-11T08:46:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,3", 2024-09-11T08:46:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.405110+0000", 2024-09-11T08:46:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013258700000000001 2024-09-11T08:46:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.408834+0000", 2024-09-11T08:46:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003723933 2024-09-11T08:46:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.409512+0000", 2024-09-11T08:46:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00067832799999999996 2024-09-11T08:46:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.521917+0000", 2024-09-11T08:46:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.11240472999999999 2024-09-11T08:46:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.752 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.752 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:23.752 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.521975+0000", 2024-09-11T08:46:23.752 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7726999999999998e-05 2024-09-11T08:46:23.752 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.752 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.752 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.752 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.522011+0000", 2024-09-11T08:46:23.752 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5930000000000003e-05 2024-09-11T08:46:23.752 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.752 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.752 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.752 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:150331 7.d 7:b45ac87d:::benchmark_data_smithi092_84565_object24803:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:46:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:07.601826+0000", 2024-09-11T08:46:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 196.09363630799999, 2024-09-11T08:46:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.199883954, 2024-09-11T08:46:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 150331 2024-09-11T08:46:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.601826+0000", 2024-09-11T08:46:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.601826+0000", 2024-09-11T08:46:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.601826+0000", 2024-09-11T08:46:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:46:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.601829+0000", 2024-09-11T08:46:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2940000000000001e-06 2024-09-11T08:46:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.601838+0000", 2024-09-11T08:46:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.6319999999999997e-06 2024-09-11T08:46:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.601842+0000", 2024-09-11T08:46:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0339999999999998e-06 2024-09-11T08:46:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.601890+0000", 2024-09-11T08:46:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8466e-05 2024-09-11T08:46:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.602002+0000", 2024-09-11T08:46:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000112199 2024-09-11T08:46:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:46:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.602104+0000", 2024-09-11T08:46:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000101413 2024-09-11T08:46:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.605993+0000", 2024-09-11T08:46:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0038893019999999999 2024-09-11T08:46:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.758 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.606054+0000", 2024-09-11T08:46:23.758 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0368999999999997e-05 2024-09-11T08:46:23.758 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.758 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.758 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.758 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801629+0000", 2024-09-11T08:46:23.758 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.195575569 2024-09-11T08:46:23.758 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.758 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.758 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:23.758 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801670+0000", 2024-09-11T08:46:23.758 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1189999999999997e-05 2024-09-11T08:46:23.758 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.758 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.759 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.759 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801710+0000", 2024-09-11T08:46:23.759 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0058999999999997e-05 2024-09-11T08:46:23.759 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.759 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.759 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.759 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.759 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.759 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:150339 7.d 7:b48099e1:::benchmark_data_smithi092_84565_object24811:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:46:23.759 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:07.610231+0000", 2024-09-11T08:46:23.759 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 196.08523191899999, 2024-09-11T08:46:23.759 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.19162894, 2024-09-11T08:46:23.759 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.760 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:23.760 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.760 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:23.760 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:23.760 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 150339 2024-09-11T08:46:23.760 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.760 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.760 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.760 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.760 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610231+0000", 2024-09-11T08:46:23.760 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.760 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.761 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.761 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.761 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610231+0000", 2024-09-11T08:46:23.761 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.761 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.761 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.761 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.761 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610230+0000", 2024-09-11T08:46:23.761 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:23.761 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.761 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.761 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.761 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610235+0000", 2024-09-11T08:46:23.761 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3020000000000002e-06 2024-09-11T08:46:23.762 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.762 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.762 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.762 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610246+0000", 2024-09-11T08:46:23.762 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0465999999999999e-05 2024-09-11T08:46:23.762 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.762 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.762 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.762 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610250+0000", 2024-09-11T08:46:23.762 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2379999999999999e-06 2024-09-11T08:46:23.762 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.762 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.762 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.763 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610325+0000", 2024-09-11T08:46:23.763 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4641000000000006e-05 2024-09-11T08:46:23.763 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.763 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.763 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:23.763 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610514+0000", 2024-09-11T08:46:23.763 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00018959599999999999 2024-09-11T08:46:23.764 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.764 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.764 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:46:23.764 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.610635+0000", 2024-09-11T08:46:23.764 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012111499999999999 2024-09-11T08:46:23.764 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.764 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.765 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:23.765 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.616121+0000", 2024-09-11T08:46:23.765 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0054855390000000002 2024-09-11T08:46:23.765 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.765 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.766 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.766 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.616739+0000", 2024-09-11T08:46:23.766 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00061781100000000001 2024-09-11T08:46:23.766 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.766 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.766 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.766 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801782+0000", 2024-09-11T08:46:23.766 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.185043292 2024-09-11T08:46:23.766 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.766 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.766 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:23.766 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801827+0000", 2024-09-11T08:46:23.767 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5318000000000002e-05 2024-09-11T08:46:23.767 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.767 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.767 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.767 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801860+0000", 2024-09-11T08:46:23.767 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2393e-05 2024-09-11T08:46:23.767 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.767 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.767 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.767 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.767 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.767 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:150344 7.d 7:bc2fa16d:::benchmark_data_smithi092_84565_object24816:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:46:23.767 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:07.613435+0000", 2024-09-11T08:46:23.767 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 196.082027307, 2024-09-11T08:46:23.768 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.18853381399999999, 2024-09-11T08:46:23.768 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.768 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:46:23.768 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.768 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:46:23.768 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:46:23.768 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 150344 2024-09-11T08:46:23.768 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.768 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.768 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.768 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.768 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613435+0000", 2024-09-11T08:46:23.768 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.769 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.769 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.769 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.769 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613435+0000", 2024-09-11T08:46:23.769 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.769 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.769 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.769 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.769 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613435+0000", 2024-09-11T08:46:23.769 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:23.769 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.769 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.770 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.770 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613439+0000", 2024-09-11T08:46:23.770 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2139999999999998e-06 2024-09-11T08:46:23.770 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.770 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.770 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.770 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613446+0000", 2024-09-11T08:46:23.770 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.3259999999999998e-06 2024-09-11T08:46:23.770 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.770 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.771 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.771 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613449+0000", 2024-09-11T08:46:23.771 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2480000000000001e-06 2024-09-11T08:46:23.771 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.771 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.771 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.771 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613623+0000", 2024-09-11T08:46:23.771 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00017342599999999999 2024-09-11T08:46:23.771 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.771 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.772 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:46:23.772 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613752+0000", 2024-09-11T08:46:23.772 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000129467 2024-09-11T08:46:23.772 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.772 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.772 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:46:23.772 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.613849+0000", 2024-09-11T08:46:23.772 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.6953999999999994e-05 2024-09-11T08:46:23.772 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.772 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.772 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:46:23.773 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.616228+0000", 2024-09-11T08:46:23.773 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0023789060000000001 2024-09-11T08:46:23.773 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.773 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.773 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.773 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.616867+0000", 2024-09-11T08:46:23.773 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00063850099999999998 2024-09-11T08:46:23.773 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.773 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.773 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:46:23.773 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801899+0000", 2024-09-11T08:46:23.773 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.185032274 2024-09-11T08:46:23.774 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.774 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.774 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:46:23.774 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801935+0000", 2024-09-11T08:46:23.774 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5729000000000003e-05 2024-09-11T08:46:23.774 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.774 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.774 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.774 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.801969+0000", 2024-09-11T08:46:23.775 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4638000000000003e-05 2024-09-11T08:46:23.775 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.775 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.775 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.775 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.775 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.775 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:62827 8.5 8.fa8961b5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-11T08:46:23.775 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:00.991662+0000", 2024-09-11T08:46:23.775 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 22.703801166000002, 2024-09-11T08:46:23.775 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.98177320800000001, 2024-09-11T08:46:23.775 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.775 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:46:23.775 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.776 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:46:23.776 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:46:23.776 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 62827 2024-09-11T08:46:23.776 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.776 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.776 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.776 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.776 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:00.991662+0000", 2024-09-11T08:46:23.776 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.776 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.777 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.777 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.777 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:00.991662+0000", 2024-09-11T08:46:23.777 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.777 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.777 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.777 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.777 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:00.991660+0000", 2024-09-11T08:46:23.777 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:23.777 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.778 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.778 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.778 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:00.991690+0000", 2024-09-11T08:46:23.778 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9788999999999999e-05 2024-09-11T08:46:23.778 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.778 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.778 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.778 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:00.991706+0000", 2024-09-11T08:46:23.778 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6543000000000001e-05 2024-09-11T08:46:23.779 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.779 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.779 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.779 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:00.991712+0000", 2024-09-11T08:46:23.779 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8690000000000001e-06 2024-09-11T08:46:23.779 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.779 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.779 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.779 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:00.991764+0000", 2024-09-11T08:46:23.779 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1959999999999997e-05 2024-09-11T08:46:23.780 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.780 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.780 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:23.780 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:00.991772+0000", 2024-09-11T08:46:23.780 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.0269999999999995e-06 2024-09-11T08:46:23.780 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.780 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.780 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.781 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973425+0000", 2024-09-11T08:46:23.781 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.981652992 2024-09-11T08:46:23.781 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.781 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.781 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.781 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973435+0000", 2024-09-11T08:46:23.781 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.6840000000000008e-06 2024-09-11T08:46:23.781 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.781 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.781 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.782 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.782 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.782 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:62854 8.5 8.a0a6fa75 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-11T08:46:23.782 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:01.018362+0000", 2024-09-11T08:46:23.782 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 22.677100448000001, 2024-09-11T08:46:23.782 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.955098373, 2024-09-11T08:46:23.782 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.782 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:46:23.782 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.783 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:46:23.783 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:46:23.783 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 62854 2024-09-11T08:46:23.783 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.783 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.783 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.783 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.783 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.018362+0000", 2024-09-11T08:46:23.783 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.783 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.783 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.783 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.783 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.018362+0000", 2024-09-11T08:46:23.784 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.784 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.784 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.784 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.784 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.018361+0000", 2024-09-11T08:46:23.784 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:23.784 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.784 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.784 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.784 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.018412+0000", 2024-09-11T08:46:23.784 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0992000000000002e-05 2024-09-11T08:46:23.784 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.785 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.785 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.785 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.018423+0000", 2024-09-11T08:46:23.785 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0695e-05 2024-09-11T08:46:23.785 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.785 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.785 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.785 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.018428+0000", 2024-09-11T08:46:23.785 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4240000000000001e-06 2024-09-11T08:46:23.785 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.785 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.785 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.785 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.018465+0000", 2024-09-11T08:46:23.786 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6353999999999998e-05 2024-09-11T08:46:23.786 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.786 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.786 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:23.786 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.018470+0000", 2024-09-11T08:46:23.786 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1619999999999999e-06 2024-09-11T08:46:23.786 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.786 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.786 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.786 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973455+0000", 2024-09-11T08:46:23.786 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.95498487300000001 2024-09-11T08:46:23.786 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.786 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.787 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.787 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973461+0000", 2024-09-11T08:46:23.787 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0050000000000001e-06 2024-09-11T08:46:23.787 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.787 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.787 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.787 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.787 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.787 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:62857 8.5 8.3c0017f5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-11T08:46:23.787 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:01.019382+0000", 2024-09-11T08:46:23.787 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 22.676080617, 2024-09-11T08:46:23.787 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.95410566399999996, 2024-09-11T08:46:23.787 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.787 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:46:23.788 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.788 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:46:23.788 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:46:23.788 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 62857 2024-09-11T08:46:23.788 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.788 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.788 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.788 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.788 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.019382+0000", 2024-09-11T08:46:23.789 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.789 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.789 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.789 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.789 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.019382+0000", 2024-09-11T08:46:23.789 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.789 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.789 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.789 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.790 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.019382+0000", 2024-09-11T08:46:23.790 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:46:23.790 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.790 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.790 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.790 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.019560+0000", 2024-09-11T08:46:23.790 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000178474 2024-09-11T08:46:23.790 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.790 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.790 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.790 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.019576+0000", 2024-09-11T08:46:23.790 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5767999999999999e-05 2024-09-11T08:46:23.791 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.791 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.791 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.791 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.019582+0000", 2024-09-11T08:46:23.791 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1489999999999999e-06 2024-09-11T08:46:23.791 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.791 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.791 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.791 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.019624+0000", 2024-09-11T08:46:23.791 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1764000000000002e-05 2024-09-11T08:46:23.791 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.791 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.791 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:23.791 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.019631+0000", 2024-09-11T08:46:23.792 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.8050000000000007e-06 2024-09-11T08:46:23.792 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.792 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.792 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.792 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973482+0000", 2024-09-11T08:46:23.792 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.95385026699999997 2024-09-11T08:46:23.792 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.792 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.792 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.792 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973488+0000", 2024-09-11T08:46:23.792 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0379999999999996e-06 2024-09-11T08:46:23.792 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.792 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.793 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.793 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.793 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.793 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:62867 8.5 8.445913a5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-11T08:46:23.793 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:01.027771+0000", 2024-09-11T08:46:23.793 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 22.667691284, 2024-09-11T08:46:23.793 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.945739569, 2024-09-11T08:46:23.793 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.793 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:46:23.793 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.793 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:46:23.793 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:46:23.793 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 62867 2024-09-11T08:46:23.793 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.794 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.794 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.794 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.794 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.027771+0000", 2024-09-11T08:46:23.794 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.794 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.794 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.794 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.794 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.027771+0000", 2024-09-11T08:46:23.794 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.794 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.794 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.794 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.794 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.027770+0000", 2024-09-11T08:46:23.795 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:23.795 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.795 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.795 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.795 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.027835+0000", 2024-09-11T08:46:23.795 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4974999999999994e-05 2024-09-11T08:46:23.795 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.795 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.795 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.795 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.027848+0000", 2024-09-11T08:46:23.795 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2673999999999999e-05 2024-09-11T08:46:23.795 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.795 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.796 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.796 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.027854+0000", 2024-09-11T08:46:23.796 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5280000000000003e-06 2024-09-11T08:46:23.796 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.796 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.796 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.796 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.027944+0000", 2024-09-11T08:46:23.796 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.0134000000000001e-05 2024-09-11T08:46:23.796 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.796 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.796 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:23.796 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.027951+0000", 2024-09-11T08:46:23.796 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.3350000000000001e-06 2024-09-11T08:46:23.796 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.797 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.797 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.797 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973505+0000", 2024-09-11T08:46:23.797 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.94555393600000004 2024-09-11T08:46:23.797 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.797 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.797 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.797 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973511+0000", 2024-09-11T08:46:23.797 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.947e-06 2024-09-11T08:46:23.797 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.797 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.797 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.797 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.797 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.798 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:62869 8.5 8.66c1cab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-11T08:46:23.798 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:01.029323+0000", 2024-09-11T08:46:23.798 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 22.666139759, 2024-09-11T08:46:23.798 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.94421028799999995, 2024-09-11T08:46:23.798 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.798 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:46:23.798 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.798 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:46:23.798 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:46:23.798 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 62869 2024-09-11T08:46:23.798 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.798 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.798 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.798 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.798 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.029323+0000", 2024-09-11T08:46:23.799 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.799 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.799 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.799 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.799 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:23.800 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.029323+0000", 2024-09-11T08:46:23.800 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.800 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.801 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.801 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.801 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.029322+0000", 2024-09-11T08:46:23.801 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:46:23.801 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.801 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.801 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.801 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.029384+0000", 2024-09-11T08:46:23.801 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1948999999999999e-05 2024-09-11T08:46:23.801 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.801 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.801 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.801 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.029397+0000", 2024-09-11T08:46:23.801 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3322000000000001e-05 2024-09-11T08:46:23.802 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.802 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.802 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.802 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.029403+0000", 2024-09-11T08:46:23.802 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.604e-06 2024-09-11T08:46:23.802 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.802 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.802 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.802 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.029471+0000", 2024-09-11T08:46:23.802 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.7817999999999993e-05 2024-09-11T08:46:23.802 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.802 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.802 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:23.803 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.029476+0000", 2024-09-11T08:46:23.803 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3539999999999999e-06 2024-09-11T08:46:23.803 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.803 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.803 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.803 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973527+0000", 2024-09-11T08:46:23.803 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.94405135900000003 2024-09-11T08:46:23.803 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.803 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.803 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.803 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973533+0000", 2024-09-11T08:46:23.803 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9529999999999996e-06 2024-09-11T08:46:23.803 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.803 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.804 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.804 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.804 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.804 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:62876 8.5 8.f62c7fc5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-11T08:46:23.804 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:01.035170+0000", 2024-09-11T08:46:23.804 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 22.66029224, 2024-09-11T08:46:23.804 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.93838940500000001, 2024-09-11T08:46:23.804 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.804 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:46:23.804 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.804 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:46:23.804 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:46:23.804 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 62876 2024-09-11T08:46:23.805 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.805 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.805 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.805 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.805 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.035170+0000", 2024-09-11T08:46:23.805 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.805 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.805 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.805 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.805 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.035170+0000", 2024-09-11T08:46:23.805 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.805 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.805 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.806 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.806 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.035169+0000", 2024-09-11T08:46:23.806 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:23.806 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.806 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.806 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.806 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.035324+0000", 2024-09-11T08:46:23.806 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000154831 2024-09-11T08:46:23.806 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.806 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.806 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.806 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.035341+0000", 2024-09-11T08:46:23.806 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7104000000000001e-05 2024-09-11T08:46:23.806 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.807 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.807 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.807 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.035348+0000", 2024-09-11T08:46:23.807 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.0740000000000004e-06 2024-09-11T08:46:23.807 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.807 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.807 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.807 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.035398+0000", 2024-09-11T08:46:23.807 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9904000000000002e-05 2024-09-11T08:46:23.807 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.807 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.807 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:23.807 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.035406+0000", 2024-09-11T08:46:23.807 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.0099999999999995e-06 2024-09-11T08:46:23.808 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.808 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.808 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.808 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973554+0000", 2024-09-11T08:46:23.808 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.938148019 2024-09-11T08:46:23.808 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.808 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.808 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.808 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973560+0000", 2024-09-11T08:46:23.808 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7509999999999997e-06 2024-09-11T08:46:23.808 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.808 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.809 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.809 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.809 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.809 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:62883 8.5 8.e7972805 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-11T08:46:23.809 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:01.039677+0000", 2024-09-11T08:46:23.809 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 22.655785821999999, 2024-09-11T08:46:23.809 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.93391084800000002, 2024-09-11T08:46:23.809 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.809 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:46:23.809 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.809 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:46:23.810 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:46:23.810 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 62883 2024-09-11T08:46:23.810 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.810 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.810 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.810 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.810 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.039677+0000", 2024-09-11T08:46:23.810 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.810 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.811 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.811 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.811 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.039677+0000", 2024-09-11T08:46:23.811 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.811 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.811 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.811 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.811 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.039676+0000", 2024-09-11T08:46:23.811 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:23.811 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.811 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.811 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.811 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.039740+0000", 2024-09-11T08:46:23.812 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4189999999999994e-05 2024-09-11T08:46:23.812 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.812 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.812 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.812 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.039752+0000", 2024-09-11T08:46:23.812 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.203e-05 2024-09-11T08:46:23.812 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.812 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.812 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.813 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.039756+0000", 2024-09-11T08:46:23.813 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1119999999999998e-06 2024-09-11T08:46:23.813 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.813 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.813 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.813 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.039797+0000", 2024-09-11T08:46:23.813 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1418999999999998e-05 2024-09-11T08:46:23.813 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.813 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.813 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:23.813 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.039805+0000", 2024-09-11T08:46:23.813 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7689999999999996e-06 2024-09-11T08:46:23.813 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.814 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.814 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.814 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973581+0000", 2024-09-11T08:46:23.814 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.93377587900000003 2024-09-11T08:46:23.814 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.814 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.814 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.814 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973588+0000", 2024-09-11T08:46:23.814 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.669e-06 2024-09-11T08:46:23.814 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.815 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.815 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.815 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.815 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.815 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:62909 8.5 8.8c9299a5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-11T08:46:23.815 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:01.059953+0000", 2024-09-11T08:46:23.815 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 22.635509673000001, 2024-09-11T08:46:23.815 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.91365951499999998, 2024-09-11T08:46:23.815 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.815 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:46:23.815 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.815 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:46:23.815 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:46:23.816 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 62909 2024-09-11T08:46:23.816 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.816 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.816 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.816 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.816 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.059953+0000", 2024-09-11T08:46:23.816 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.816 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.816 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.817 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.817 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.059953+0000", 2024-09-11T08:46:23.817 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.817 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.817 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.817 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.817 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.059951+0000", 2024-09-11T08:46:23.817 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:23.817 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.817 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.817 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.818 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.060006+0000", 2024-09-11T08:46:23.818 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4280999999999997e-05 2024-09-11T08:46:23.818 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.818 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.818 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.818 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.060020+0000", 2024-09-11T08:46:23.818 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3797e-05 2024-09-11T08:46:23.818 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.818 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.819 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.819 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.060025+0000", 2024-09-11T08:46:23.819 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8100000000000003e-06 2024-09-11T08:46:23.819 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.819 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.819 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.819 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.060069+0000", 2024-09-11T08:46:23.819 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3210000000000001e-05 2024-09-11T08:46:23.819 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.819 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.820 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:23.820 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.060075+0000", 2024-09-11T08:46:23.820 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3589999999999996e-06 2024-09-11T08:46:23.820 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.820 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.820 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.820 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973607+0000", 2024-09-11T08:46:23.820 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.91353201299999998 2024-09-11T08:46:23.820 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.821 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.821 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.821 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973613+0000", 2024-09-11T08:46:23.821 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.614e-06 2024-09-11T08:46:23.821 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.821 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.821 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.822 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.822 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.822 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:62913 8.5 8.962622d5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-11T08:46:23.822 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:01.062088+0000", 2024-09-11T08:46:23.822 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 22.633374841999998, 2024-09-11T08:46:23.822 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.911548409, 2024-09-11T08:46:23.822 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.822 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:46:23.822 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.822 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:46:23.822 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:46:23.823 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 62913 2024-09-11T08:46:23.823 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.823 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.823 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.823 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.823 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.062088+0000", 2024-09-11T08:46:23.823 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.823 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.823 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.823 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.823 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.062088+0000", 2024-09-11T08:46:23.823 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.823 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.823 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.823 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.824 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.062087+0000", 2024-09-11T08:46:23.824 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:46:23.824 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.824 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.824 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.824 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.062136+0000", 2024-09-11T08:46:23.824 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8322999999999998e-05 2024-09-11T08:46:23.824 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.824 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.824 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.824 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.062145+0000", 2024-09-11T08:46:23.824 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.5419999999999994e-06 2024-09-11T08:46:23.824 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.062148+0000", 2024-09-11T08:46:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0170000000000001e-06 2024-09-11T08:46:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.062187+0000", 2024-09-11T08:46:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8294999999999998e-05 2024-09-11T08:46:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.062193+0000", 2024-09-11T08:46:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1310000000000003e-06 2024-09-11T08:46:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.826 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.826 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.826 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973630+0000", 2024-09-11T08:46:23.826 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.91143700900000002 2024-09-11T08:46:23.826 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.826 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.826 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.826 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973636+0000", 2024-09-11T08:46:23.826 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5860000000000001e-06 2024-09-11T08:46:23.826 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.826 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.826 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.826 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.826 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.827 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:62948 8.5 8.ed8236a5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-11T08:46:23.827 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:46:01.112385+0000", 2024-09-11T08:46:23.827 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 22.583077932999998, 2024-09-11T08:46:23.827 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.86127368400000004, 2024-09-11T08:46:23.827 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:46:23.827 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:46:23.827 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:46:23.827 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:46:23.827 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:46:23.827 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 62948 2024-09-11T08:46:23.827 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.827 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:46:23.827 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.827 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:46:23.828 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.112385+0000", 2024-09-11T08:46:23.828 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.828 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.828 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.828 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:46:23.828 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.112385+0000", 2024-09-11T08:46:23.828 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:46:23.828 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.828 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.828 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:46:23.828 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.112384+0000", 2024-09-11T08:46:23.828 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:46:23.828 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.828 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.828 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:46:23.829 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.112432+0000", 2024-09-11T08:46:23.829 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8151e-05 2024-09-11T08:46:23.829 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.829 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.829 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:46:23.829 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.112443+0000", 2024-09-11T08:46:23.829 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1735e-05 2024-09-11T08:46:23.829 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.829 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.829 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:46:23.829 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.112449+0000", 2024-09-11T08:46:23.829 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4999999999999999e-06 2024-09-11T08:46:23.829 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.829 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.830 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.830 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.112486+0000", 2024-09-11T08:46:23.830 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7018999999999999e-05 2024-09-11T08:46:23.830 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.830 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.830 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:46:23.830 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.112491+0000", 2024-09-11T08:46:23.830 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7650000000000001e-06 2024-09-11T08:46:23.830 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.830 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.830 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:46:23.830 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973652+0000", 2024-09-11T08:46:23.830 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.86116174700000003 2024-09-11T08:46:23.830 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:46:23.830 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:46:23.831 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:46:23.831 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:46:01.973658+0000", 2024-09-11T08:46:23.831 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.049e-06 2024-09-11T08:46:23.831 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.831 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.831 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.831 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:46:23.831 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:46:23.831 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:46:23.847 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:23.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:23.849 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:23.949 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:23.977 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:23.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:23.978 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:24.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:23 smithi092 ceph-mon[29647]: 3.1c deep-scrub starts 2024-09-11T08:46:24.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:23 smithi092 ceph-mon[29647]: pgmap v323: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s; 6781/246504 objects misplaced (2.751%); 4.3 MiB/s, 68 objects/s recovering 2024-09-11T08:46:24.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:23 smithi092 ceph-mon[41114]: 3.1c deep-scrub starts 2024-09-11T08:46:24.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:23 smithi092 ceph-mon[41114]: pgmap v323: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s; 6781/246504 objects misplaced (2.751%); 4.3 MiB/s, 68 objects/s recovering 2024-09-11T08:46:24.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:24.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:24.119 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:24.120 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:24.169 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 54 16 84661 84645 97.947 59.5625 0.00868868 0.0102037 2024-09-11T08:46:24.208 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:23 smithi145 ceph-mon[33384]: 3.1c deep-scrub starts 2024-09-11T08:46:24.208 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:23 smithi145 ceph-mon[33384]: pgmap v323: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s; 6781/246504 objects misplaced (2.751%); 4.3 MiB/s, 68 objects/s recovering 2024-09-11T08:46:24.221 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:24.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:24.254 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:24.254 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:24.355 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:24.389 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:24.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:24.390 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:24.491 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:24.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:24.534 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:24.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:24.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:24.686 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:24.713 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T08:46:24.787 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:24.827 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:24.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:24.828 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:24.929 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:24.971 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:24.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:24.972 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:25.073 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:25.117 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:46:25.128 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:25.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:25.129 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:25.169 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 55 16 85646 85630 97.2855 61.5625 0.0124571 0.0102734 2024-09-11T08:46:25.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:25.255 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:24 smithi145 ceph-mon[33384]: 5.6 scrub starts 2024-09-11T08:46:25.255 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:24 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:46:25.255 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:24 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:46:25.258 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:25.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:25.259 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:25.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:24 smithi092 ceph-mon[29647]: 5.6 scrub starts 2024-09-11T08:46:25.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:24 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:46:25.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:24 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:46:25.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:24 smithi092 ceph-mon[41114]: 5.6 scrub starts 2024-09-11T08:46:25.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:24 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:46:25.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:24 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:46:25.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:25.395 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:25.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:25.396 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:25.497 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:25.533 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:25.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:25.534 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:25.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:25.672 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:25.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:25.673 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:25.766 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-11T08:46:21.157245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5141436636,"num_objects":78639,"num_object_clones":0,"num_object_copies":235917,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7058,"num_objects_unfound":0,"num_objects_dirty":78639,"num_whiteouts":0,"num_read":1305,"num_read_kb":1162,"num_write":78986,"num_write_kb":5022023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":815,"num_bytes_recovered":47188112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78759,"ondisk_log_size":78759,"up":444,"acting":443,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":446,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":17618704,"kb_used_data":15198276,"kb_used_omap":13,"kb_used_meta":2420338,"kb_avail":732342512,"statfs":{"total":767960285184,"available":749918732288,"internally_reserved":0,"allocated":15563034624,"data_stored":15559883788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2478426816},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":424738816,"num_objects":6481,"num_object_clones":0,"num_object_copies":19443,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-275,"num_objects_unfound":0,"num_objects_dirty":6481,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6481,"num_write_kb":414784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":275,"num_bytes_recovered":18022400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.348632"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":166,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.845607+0000","last_change":"2024-09-11T08:45:06.574593+0000","last_active":"2024-09-11T08:46:10.845607+0000","last_peered":"2024-09-11T08:46:10.845607+0000","last_clean":"2024-09-11T08:46:10.845607+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:46:10.845607+0000","last_undegraded":"2024-09-11T08:46:10.845607+0000","last_fullsized":"2024-09-11T08:46:10.845607+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:06.574539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.574539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:21.697319+0000","scrub_duration":0.0028477630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":216,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.903368+0000","last_change":"2024-09-11T08:46:01.991434+0000","last_active":"2024-09-11T08:46:17.903368+0000","last_peered":"2024-09-11T08:46:17.903368+0000","last_clean":"2024-09-11T08:46:17.903368+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:46:17.903368+0000","last_undegraded":"2024-09-11T08:46:17.903368+0000","last_fullsized":"2024-09-11T08:46:17.903368+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:45:13.113315+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.113315+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:19.047302+0000","scrub_duration":0.0018068470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":162,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.846671+0000","last_change":"2024-09-11T08:44:25.796953+0000","last_active":"2024-09-11T08:46:10.846671+0000","last_peered":"2024-09-11T08:46:10.846671+0000","last_clean":"2024-09-11T08:46:10.846671+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:46:10.846671+0000","last_undegraded":"2024-09-11T08:46:10.846671+0000","last_fullsized":"2024-09-11T08:46:10.846671+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:25.796699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:25.796699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025405829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":142,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.904672+0000","last_change":"2024-09-11T08:44:26.005322+0000","last_active":"2024-09-11T08:46:17.904672+0000","last_peered":"2024-09-11T08:46:17.904672+0000","last_clean":"2024-09-11T08:46:17.904672+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:46:17.904672+0000","last_undegraded":"2024-09-11T08:46:17.904672+0000","last_fullsized":"2024-09-11T08:46:17.904672+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.005270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.005270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038675070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":169,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.832880+0000","last_change":"2024-09-11T08:46:01.984446+0000","last_active":"2024-09-11T08:46:10.832880+0000","last_peered":"2024-09-11T08:46:10.832880+0000","last_clean":"2024-09-11T08:46:10.832880+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:46:10.832880+0000","last_undegraded":"2024-09-11T08:46:10.832880+0000","last_fullsized":"2024-09-11T08:46:10.832880+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:28.356172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.356172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029955870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":175,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.834547+0000","last_change":"2024-09-11T08:46:01.981971+0000","last_active":"2024-09-11T08:46:10.834547+0000","last_peered":"2024-09-11T08:46:10.834547+0000","last_clean":"2024-09-11T08:46:10.834547+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:46:10.834547+0000","last_undegraded":"2024-09-11T08:46:10.834547+0000","last_fullsized":"2024-09-11T08:46:10.834547+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.330642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.330642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0087830849999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":185,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.833186+0000","last_change":"2024-09-11T08:44:24.344328+0000","last_active":"2024-09-11T08:46:10.833186+0000","last_peered":"2024-09-11T08:46:10.833186+0000","last_clean":"2024-09-11T08:46:10.833186+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:46:10.833186+0000","last_undegraded":"2024-09-11T08:46:10.833186+0000","last_fullsized":"2024-09-11T08:46:10.833186+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:44:24.344127+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.344127+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010996336000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":144,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.903296+0000","last_change":"2024-09-11T08:43:28.988751+0000","last_active":"2024-09-11T08:46:17.903296+0000","last_peered":"2024-09-11T08:46:17.903296+0000","last_clean":"2024-09-11T08:46:17.903296+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:46:17.903296+0000","last_undegraded":"2024-09-11T08:46:17.903296+0000","last_fullsized":"2024-09-11T08:46:17.903296+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:43:28.988697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00375019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":181,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.902145+0000","last_change":"2024-09-11T08:46:17.669115+0000","last_active":"2024-09-11T08:46:17.902145+0000","last_peered":"2024-09-11T08:46:17.902145+0000","last_clean":"2024-09-11T08:46:17.902145+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:46:17.902145+0000","last_undegraded":"2024-09-11T08:46:17.902145+0000","last_fullsized":"2024-09-11T08:46:17.902145+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:12.878132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.878132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077623500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":176,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.906952+0000","last_change":"2024-09-11T08:44:27.963772+0000","last_active":"2024-09-11T08:46:17.906952+0000","last_peered":"2024-09-11T08:46:17.906952+0000","last_clean":"2024-09-11T08:46:17.906952+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:46:17.906952+0000","last_undegraded":"2024-09-11T08:46:17.906952+0000","last_fullsized":"2024-09-11T08:46:17.906952+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:44:27.963689+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:44:27.963689+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011544877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":157,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.900141+0000","last_change":"2024-09-11T08:46:09.691657+0000","last_active":"2024-09-11T08:46:17.900141+0000","last_peered":"2024-09-11T08:46:17.900141+0000","last_clean":"2024-09-11T08:46:17.900141+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:46:17.900141+0000","last_undegraded":"2024-09-11T08:46:17.900141+0000","last_fullsized":"2024-09-11T08:46:17.900141+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:33.644869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.644869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020832210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":160,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.845447+0000","last_change":"2024-09-11T08:45:31.457775+0000","last_active":"2024-09-11T08:46:10.845447+0000","last_peered":"2024-09-11T08:46:10.845447+0000","last_clean":"2024-09-11T08:46:10.845447+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:46:10.845447+0000","last_undegraded":"2024-09-11T08:46:10.845447+0000","last_fullsized":"2024-09-11T08:46:10.845447+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.457652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.457652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:45.911145+0000","scrub_duration":0.006759045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":158,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.899928+0000","last_change":"2024-09-11T08:44:09.871734+0000","last_active":"2024-09-11T08:46:17.899928+0000","last_peered":"2024-09-11T08:46:17.899928+0000","last_clean":"2024-09-11T08:46:17.899928+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:46:17.899928+0000","last_undegraded":"2024-09-11T08:46:17.899928+0000","last_fullsized":"2024-09-11T08:46:17.899928+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:09.871637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:09.871637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029269349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":156,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.903987+0000","last_change":"2024-09-11T08:43:26.983719+0000","last_active":"2024-09-11T08:46:17.903987+0000","last_peered":"2024-09-11T08:46:17.903987+0000","last_clean":"2024-09-11T08:46:17.903987+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:46:17.903987+0000","last_undegraded":"2024-09-11T08:46:17.903987+0000","last_fullsized":"2024-09-11T08:46:17.903987+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:43:26.983587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:26.983587+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082679390000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":163,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.905719+0000","last_change":"2024-09-11T08:45:13.320968+0000","last_active":"2024-09-11T08:46:17.905719+0000","last_peered":"2024-09-11T08:46:17.905719+0000","last_clean":"2024-09-11T08:46:17.905719+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:46:17.905719+0000","last_undegraded":"2024-09-11T08:46:17.905719+0000","last_fullsized":"2024-09-11T08:46:17.905719+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:13.320898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.320898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:24.405222+0000","scrub_duration":0.0013000240000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":152,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.900079+0000","last_change":"2024-09-11T08:44:22.850766+0000","last_active":"2024-09-11T08:46:17.900079+0000","last_peered":"2024-09-11T08:46:17.900079+0000","last_clean":"2024-09-11T08:46:17.900079+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:46:17.900079+0000","last_undegraded":"2024-09-11T08:46:17.900079+0000","last_fullsized":"2024-09-11T08:46:17.900079+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.850684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.850684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058755379999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":172,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.907553+0000","last_change":"2024-09-11T08:46:01.978488+0000","last_active":"2024-09-11T08:46:17.907553+0000","last_peered":"2024-09-11T08:46:17.907553+0000","last_clean":"2024-09-11T08:46:17.907553+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:46:17.907553+0000","last_undegraded":"2024-09-11T08:46:17.907553+0000","last_fullsized":"2024-09-11T08:46:17.907553+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:24.197149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.197149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:48.966732+0000","scrub_duration":0.001510267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":156,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.846696+0000","last_change":"2024-09-11T08:43:33.346941+0000","last_active":"2024-09-11T08:46:10.846696+0000","last_peered":"2024-09-11T08:46:10.846696+0000","last_clean":"2024-09-11T08:46:10.846696+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:46:10.846696+0000","last_undegraded":"2024-09-11T08:46:10.846696+0000","last_fullsized":"2024-09-11T08:46:10.846696+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:33.346856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.346856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053605370000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":199,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.846361+0000","last_change":"2024-09-11T08:46:07.378561+0000","last_active":"2024-09-11T08:46:10.846361+0000","last_peered":"2024-09-11T08:46:10.846361+0000","last_clean":"2024-09-11T08:46:10.846361+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:46:10.846361+0000","last_undegraded":"2024-09-11T08:46:10.846361+0000","last_fullsized":"2024-09-11T08:46:10.846361+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:44:18.678748+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:44:18.678748+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016068020999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":162,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.848749+0000","last_change":"2024-09-11T08:44:12.518229+0000","last_active":"2024-09-11T08:46:10.848749+0000","last_peered":"2024-09-11T08:46:10.848749+0000","last_clean":"2024-09-11T08:46:10.848749+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:46:10.848749+0000","last_undegraded":"2024-09-11T08:46:10.848749+0000","last_fullsized":"2024-09-11T08:46:10.848749+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:12.518151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.518151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013208541000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":178,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.836182+0000","last_change":"2024-09-11T08:46:01.998163+0000","last_active":"2024-09-11T08:46:10.836182+0000","last_peered":"2024-09-11T08:46:10.836182+0000","last_clean":"2024-09-11T08:46:10.836182+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:46:10.836182+0000","last_undegraded":"2024-09-11T08:46:10.836182+0000","last_fullsized":"2024-09-11T08:46:10.836182+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:24.223237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.223237+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:45.841836+0000","scrub_duration":0.0019917720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":158,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.907022+0000","last_change":"2024-09-11T08:45:28.194198+0000","last_active":"2024-09-11T08:46:17.907022+0000","last_peered":"2024-09-11T08:46:17.907022+0000","last_clean":"2024-09-11T08:46:17.907022+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:46:17.907022+0000","last_undegraded":"2024-09-11T08:46:17.907022+0000","last_fullsized":"2024-09-11T08:46:17.907022+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.194095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.194095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:49.129673+0000","scrub_duration":0.001559936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":176,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.834101+0000","last_change":"2024-09-11T08:45:16.162671+0000","last_active":"2024-09-11T08:46:10.834101+0000","last_peered":"2024-09-11T08:46:10.834101+0000","last_clean":"2024-09-11T08:46:10.834101+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:46:10.834101+0000","last_undegraded":"2024-09-11T08:46:10.834101+0000","last_fullsized":"2024-09-11T08:46:10.834101+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:16.162616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:16.162616+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:16.300194+0000","scrub_duration":0.001890833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":154,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.907532+0000","last_change":"2024-09-11T08:45:22.244097+0000","last_active":"2024-09-11T08:46:17.907532+0000","last_peered":"2024-09-11T08:46:17.907532+0000","last_clean":"2024-09-11T08:46:17.907532+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:46:17.907532+0000","last_undegraded":"2024-09-11T08:46:17.907532+0000","last_fullsized":"2024-09-11T08:46:17.907532+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.244038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.244038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:44.607127+0000","scrub_duration":0.0015252620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":162,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.906097+0000","last_change":"2024-09-11T08:45:19.335163+0000","last_active":"2024-09-11T08:46:17.906097+0000","last_peered":"2024-09-11T08:46:17.906097+0000","last_clean":"2024-09-11T08:46:17.906097+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:46:17.906097+0000","last_undegraded":"2024-09-11T08:46:17.906097+0000","last_fullsized":"2024-09-11T08:46:17.906097+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:19.335065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:19.335065+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:49.064118+0000","scrub_duration":0.0026052929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":166,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.846760+0000","last_change":"2024-09-11T08:44:33.758098+0000","last_active":"2024-09-11T08:46:10.846760+0000","last_peered":"2024-09-11T08:46:10.846760+0000","last_clean":"2024-09-11T08:46:10.846760+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:46:10.846760+0000","last_undegraded":"2024-09-11T08:46:10.846760+0000","last_fullsized":"2024-09-11T08:46:10.846760+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:44:33.758018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025150559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":169,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.847179+0000","last_change":"2024-09-11T08:44:10.704528+0000","last_active":"2024-09-11T08:46:10.847179+0000","last_peered":"2024-09-11T08:46:10.847179+0000","last_clean":"2024-09-11T08:46:10.847179+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:46:10.847179+0000","last_undegraded":"2024-09-11T08:46:10.847179+0000","last_fullsized":"2024-09-11T08:46:10.847179+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:44:10.704471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:10.704471+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020557259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":266,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.924645+0000","last_change":"2024-09-11T08:45:08.310590+0000","last_active":"2024-09-11T08:46:17.924645+0000","last_peered":"2024-09-11T08:46:17.924645+0000","last_clean":"2024-09-11T08:46:17.924645+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:46:17.924645+0000","last_undegraded":"2024-09-11T08:46:17.924645+0000","last_fullsized":"2024-09-11T08:46:17.924645+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:08.310527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.310527+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021434650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"173'4705","reported_seq":4737,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T08:46:16.247898+0000","last_change":"2024-09-11T08:45:15.261892+0000","last_active":"2024-09-11T08:46:16.247898+0000","last_peered":"2024-09-11T08:46:16.247898+0000","last_clean":"2024-09-11T08:46:16.247898+0000","last_became_active":"2024-09-11T08:45:15.261608+0000","last_became_peered":"2024-09-11T08:45:15.261608+0000","last_unstale":"2024-09-11T08:46:16.247898+0000","last_undegraded":"2024-09-11T08:46:16.247898+0000","last_fullsized":"2024-09-11T08:46:16.247898+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4705,"log_dups_size":0,"ondisk_log_size":4705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:40.441095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308346880,"num_objects":4705,"num_object_clones":0,"num_object_copies":14115,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4705,"num_write_kb":301120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"124'22","reported_seq":196,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.849123+0000","last_change":"2024-09-11T08:45:18.777528+0000","last_active":"2024-09-11T08:46:10.849123+0000","last_peered":"2024-09-11T08:46:10.849123+0000","last_clean":"2024-09-11T08:46:10.849123+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:46:10.849123+0000","last_undegraded":"2024-09-11T08:46:10.849123+0000","last_fullsized":"2024-09-11T08:46:10.849123+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:45:18.777466+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.777466+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:47.682019+0000","scrub_duration":0.0045440580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":155,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.836239+0000","last_change":"2024-09-11T08:45:13.214007+0000","last_active":"2024-09-11T08:46:10.836239+0000","last_peered":"2024-09-11T08:46:10.836239+0000","last_clean":"2024-09-11T08:46:10.836239+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:46:10.836239+0000","last_undegraded":"2024-09-11T08:46:10.836239+0000","last_fullsized":"2024-09-11T08:46:10.836239+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:13.213922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.213922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016325280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"154'9","reported_seq":255,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.907640+0000","last_change":"2024-09-11T08:45:26.208887+0000","last_active":"2024-09-11T08:46:17.907640+0000","last_peered":"2024-09-11T08:46:17.907640+0000","last_clean":"2024-09-11T08:46:17.907640+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:46:17.907640+0000","last_undegraded":"2024-09-11T08:46:17.907640+0000","last_fullsized":"2024-09-11T08:46:17.907640+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'9","last_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.208818+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:39.597635+0000","scrub_duration":0.0028933230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":152,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.900544+0000","last_change":"2024-09-11T08:44:24.813906+0000","last_active":"2024-09-11T08:46:17.900544+0000","last_peered":"2024-09-11T08:46:17.900544+0000","last_clean":"2024-09-11T08:46:17.900544+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:46:17.900544+0000","last_undegraded":"2024-09-11T08:46:17.900544+0000","last_fullsized":"2024-09-11T08:46:17.900544+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:24.813857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.813857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01118625,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"173'4825","reported_seq":4857,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T08:46:16.464520+0000","last_change":"2024-09-11T08:45:15.297710+0000","last_active":"2024-09-11T08:46:16.464520+0000","last_peered":"2024-09-11T08:46:16.464520+0000","last_clean":"2024-09-11T08:46:16.464520+0000","last_became_active":"2024-09-11T08:45:15.297378+0000","last_became_peered":"2024-09-11T08:45:15.297378+0000","last_unstale":"2024-09-11T08:46:16.464520+0000","last_undegraded":"2024-09-11T08:46:16.464520+0000","last_fullsized":"2024-09-11T08:46:16.464520+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4825,"log_dups_size":0,"ondisk_log_size":4825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:23.893934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316211200,"num_objects":4825,"num_object_clones":0,"num_object_copies":14475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4825,"num_write_kb":308800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":185,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.836035+0000","last_change":"2024-09-11T08:44:33.387066+0000","last_active":"2024-09-11T08:46:10.836035+0000","last_peered":"2024-09-11T08:46:10.836035+0000","last_clean":"2024-09-11T08:46:10.836035+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:46:10.836035+0000","last_undegraded":"2024-09-11T08:46:10.836035+0000","last_fullsized":"2024-09-11T08:46:10.836035+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-11T08:44:33.386972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:33.386972+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022646007999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":155,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.840981+0000","last_change":"2024-09-11T08:44:25.327692+0000","last_active":"2024-09-11T08:46:10.840981+0000","last_peered":"2024-09-11T08:46:10.840981+0000","last_clean":"2024-09-11T08:46:10.840981+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:46:10.840981+0000","last_undegraded":"2024-09-11T08:46:10.840981+0000","last_fullsized":"2024-09-11T08:46:10.840981+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:25.327553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:25.327553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016327754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":154,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.903476+0000","last_change":"2024-09-11T08:45:17.150827+0000","last_active":"2024-09-11T08:46:17.903476+0000","last_peered":"2024-09-11T08:46:17.903476+0000","last_clean":"2024-09-11T08:46:17.903476+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:46:17.903476+0000","last_undegraded":"2024-09-11T08:46:17.903476+0000","last_fullsized":"2024-09-11T08:46:17.903476+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.150767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.150767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:19.036127+0000","scrub_duration":0.001327707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":152,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.904785+0000","last_change":"2024-09-11T08:46:03.144149+0000","last_active":"2024-09-11T08:46:17.904785+0000","last_peered":"2024-09-11T08:46:17.904785+0000","last_clean":"2024-09-11T08:46:17.904785+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:46:17.904785+0000","last_undegraded":"2024-09-11T08:46:17.904785+0000","last_fullsized":"2024-09-11T08:46:17.904785+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:46:03.144096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:46:03.144096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:47:11.742114+0000","scrub_duration":0.001497151,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"175'5082","reported_seq":5115,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T08:46:21.105411+0000","last_change":"2024-09-11T08:45:15.627562+0000","last_active":"2024-09-11T08:46:21.105411+0000","last_peered":"2024-09-11T08:46:21.105411+0000","last_clean":"2024-09-11T08:46:21.105411+0000","last_became_active":"2024-09-11T08:45:15.627210+0000","last_became_peered":"2024-09-11T08:45:15.627210+0000","last_unstale":"2024-09-11T08:46:21.105411+0000","last_undegraded":"2024-09-11T08:46:21.105411+0000","last_fullsized":"2024-09-11T08:46:21.105411+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5082,"log_dups_size":0,"ondisk_log_size":5082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:25.506499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":333053952,"num_objects":5082,"num_object_clones":0,"num_object_copies":15246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5082,"num_write_kb":325248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"86'21","reported_seq":174,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.903802+0000","last_change":"2024-09-11T08:43:30.997804+0000","last_active":"2024-09-11T08:46:17.903802+0000","last_peered":"2024-09-11T08:46:17.903802+0000","last_clean":"2024-09-11T08:46:17.903802+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:46:17.903802+0000","last_undegraded":"2024-09-11T08:46:17.903802+0000","last_fullsized":"2024-09-11T08:46:17.903802+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'21","last_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:43:30.997710+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025287299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":166,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.847406+0000","last_change":"2024-09-11T08:45:30.475446+0000","last_active":"2024-09-11T08:46:10.847406+0000","last_peered":"2024-09-11T08:46:10.847406+0000","last_clean":"2024-09-11T08:46:10.847406+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:46:10.847406+0000","last_undegraded":"2024-09-11T08:46:10.847406+0000","last_fullsized":"2024-09-11T08:46:10.847406+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:30.475365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:30.475365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:37.212788+0000","scrub_duration":0.010690405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":243,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:12.761937+0000","last_change":"2024-09-11T08:44:11.198993+0000","last_active":"2024-09-11T08:46:12.761937+0000","last_peered":"2024-09-11T08:46:12.761937+0000","last_clean":"2024-09-11T08:46:12.761937+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:46:12.761937+0000","last_undegraded":"2024-09-11T08:46:12.761937+0000","last_fullsized":"2024-09-11T08:46:12.761937+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:44:11.198936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:11.198936+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019660649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":181,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.905816+0000","last_change":"2024-09-11T08:46:01.983654+0000","last_active":"2024-09-11T08:46:17.905816+0000","last_peered":"2024-09-11T08:46:17.905816+0000","last_clean":"2024-09-11T08:46:17.905816+0000","last_became_active":"2024-09-11T08:46:01.983335+0000","last_became_peered":"2024-09-11T08:46:01.983335+0000","last_unstale":"2024-09-11T08:46:17.905816+0000","last_undegraded":"2024-09-11T08:46:17.905816+0000","last_fullsized":"2024-09-11T08:46:17.905816+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.303194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.303194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:39.755953+0000","scrub_duration":0.001492431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"173'4768","reported_seq":4800,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T08:46:16.567813+0000","last_change":"2024-09-11T08:45:15.626487+0000","last_active":"2024-09-11T08:46:16.567813+0000","last_peered":"2024-09-11T08:46:16.567813+0000","last_clean":"2024-09-11T08:46:16.567813+0000","last_became_active":"2024-09-11T08:45:15.626210+0000","last_became_peered":"2024-09-11T08:45:15.626210+0000","last_unstale":"2024-09-11T08:46:16.567813+0000","last_undegraded":"2024-09-11T08:46:16.567813+0000","last_fullsized":"2024-09-11T08:46:16.567813+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4768,"log_dups_size":0,"ondisk_log_size":4768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":312475648,"num_objects":4768,"num_object_clones":0,"num_object_copies":14304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4768,"num_write_kb":305152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"100'15","reported_seq":184,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.846265+0000","last_change":"2024-09-11T08:45:12.769299+0000","last_active":"2024-09-11T08:46:10.846265+0000","last_peered":"2024-09-11T08:46:10.846265+0000","last_clean":"2024-09-11T08:46:10.846265+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:46:10.846265+0000","last_undegraded":"2024-09-11T08:46:10.846265+0000","last_fullsized":"2024-09-11T08:46:10.846265+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'15","last_scrub_stamp":"2024-09-11T08:45:12.769239+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:41:56.250393+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.769239+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:29.645624+0000","scrub_duration":0.0026298300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":158,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.902227+0000","last_change":"2024-09-11T08:44:19.905540+0000","last_active":"2024-09-11T08:46:17.902227+0000","last_peered":"2024-09-11T08:46:17.902227+0000","last_clean":"2024-09-11T08:46:17.902227+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:46:17.902227+0000","last_undegraded":"2024-09-11T08:46:17.902227+0000","last_fullsized":"2024-09-11T08:46:17.902227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:19.905459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:19.905459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015766237999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":251,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:12.762376+0000","last_change":"2024-09-11T08:44:34.630457+0000","last_active":"2024-09-11T08:46:12.762376+0000","last_peered":"2024-09-11T08:46:12.762376+0000","last_clean":"2024-09-11T08:46:12.762376+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:46:12.762376+0000","last_undegraded":"2024-09-11T08:46:12.762376+0000","last_fullsized":"2024-09-11T08:46:12.762376+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:44:34.630191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:34.630191+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089275420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":150,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.845587+0000","last_change":"2024-09-11T08:44:16.681124+0000","last_active":"2024-09-11T08:46:10.845587+0000","last_peered":"2024-09-11T08:46:10.845587+0000","last_clean":"2024-09-11T08:46:10.845587+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:46:10.845587+0000","last_undegraded":"2024-09-11T08:46:10.845587+0000","last_fullsized":"2024-09-11T08:46:10.845587+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:16.681036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:44:16.681036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047481509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"173'4803","reported_seq":4833,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T08:46:16.509514+0000","last_change":"2024-09-11T08:45:15.627960+0000","last_active":"2024-09-11T08:46:16.509514+0000","last_peered":"2024-09-11T08:46:16.509514+0000","last_clean":"2024-09-11T08:46:16.509514+0000","last_became_active":"2024-09-11T08:45:15.627639+0000","last_became_peered":"2024-09-11T08:45:15.627639+0000","last_unstale":"2024-09-11T08:46:16.509514+0000","last_undegraded":"2024-09-11T08:46:16.509514+0000","last_fullsized":"2024-09-11T08:46:16.509514+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:29.073666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314769408,"num_objects":4803,"num_object_clones":0,"num_object_copies":14409,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4803,"num_write_kb":307392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":190,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.846568+0000","last_change":"2024-09-11T08:45:11.576184+0000","last_active":"2024-09-11T08:46:10.846568+0000","last_peered":"2024-09-11T08:46:10.846568+0000","last_clean":"2024-09-11T08:46:10.846568+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:46:10.846568+0000","last_undegraded":"2024-09-11T08:46:10.846568+0000","last_fullsized":"2024-09-11T08:46:10.846568+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:45:11.576085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.576085+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:25.731897+0000","scrub_duration":0.002366754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":166,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.848415+0000","last_change":"2024-09-11T08:44:30.802099+0000","last_active":"2024-09-11T08:46:10.848415+0000","last_peered":"2024-09-11T08:46:10.848415+0000","last_clean":"2024-09-11T08:46:10.848415+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:46:10.848415+0000","last_undegraded":"2024-09-11T08:46:10.848415+0000","last_fullsized":"2024-09-11T08:46:10.848415+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:30.800097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:30.800097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071432789999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":163,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.923986+0000","last_change":"2024-09-11T08:45:07.264895+0000","last_active":"2024-09-11T08:46:17.923986+0000","last_peered":"2024-09-11T08:46:17.923986+0000","last_clean":"2024-09-11T08:46:17.923986+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:46:17.923986+0000","last_undegraded":"2024-09-11T08:46:17.923986+0000","last_fullsized":"2024-09-11T08:46:17.923986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.264822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.264822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:32.085785+0000","scrub_duration":0.0014425250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":180,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.846628+0000","last_change":"2024-09-11T08:46:01.985315+0000","last_active":"2024-09-11T08:46:10.846628+0000","last_peered":"2024-09-11T08:46:10.846628+0000","last_clean":"2024-09-11T08:46:10.846628+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:46:10.846628+0000","last_undegraded":"2024-09-11T08:46:10.846628+0000","last_fullsized":"2024-09-11T08:46:10.846628+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.805220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.805220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:22.738957+0000","scrub_duration":0.0012010009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"175'5140","reported_seq":5174,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T08:46:21.111254+0000","last_change":"2024-09-11T08:45:15.298396+0000","last_active":"2024-09-11T08:46:21.111254+0000","last_peered":"2024-09-11T08:46:21.111254+0000","last_clean":"2024-09-11T08:46:21.111254+0000","last_became_active":"2024-09-11T08:45:15.298053+0000","last_became_peered":"2024-09-11T08:45:15.298053+0000","last_unstale":"2024-09-11T08:46:21.111254+0000","last_undegraded":"2024-09-11T08:46:21.111254+0000","last_fullsized":"2024-09-11T08:46:21.111254+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5140,"log_dups_size":0,"ondisk_log_size":5140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:36.450006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":336855040,"num_objects":5140,"num_object_clones":0,"num_object_copies":15420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5140,"num_write_kb":328960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":323,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:12.762897+0000","last_change":"2024-09-11T08:45:26.666261+0000","last_active":"2024-09-11T08:46:12.762897+0000","last_peered":"2024-09-11T08:46:12.762897+0000","last_clean":"2024-09-11T08:46:12.762897+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:46:12.762897+0000","last_undegraded":"2024-09-11T08:46:12.762897+0000","last_fullsized":"2024-09-11T08:46:12.762897+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:45:26.666199+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.666199+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:53.748755+0000","scrub_duration":0.0020004279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":158,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.904214+0000","last_change":"2024-09-11T08:45:15.130637+0000","last_active":"2024-09-11T08:46:17.904214+0000","last_peered":"2024-09-11T08:46:17.904214+0000","last_clean":"2024-09-11T08:46:17.904214+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:46:17.904214+0000","last_undegraded":"2024-09-11T08:46:17.904214+0000","last_fullsized":"2024-09-11T08:46:17.904214+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.130572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.130572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019779749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":196,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.901871+0000","last_change":"2024-09-11T08:46:08.737156+0000","last_active":"2024-09-11T08:46:17.901871+0000","last_peered":"2024-09-11T08:46:17.901871+0000","last_clean":"2024-09-11T08:46:17.901871+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:46:17.901871+0000","last_undegraded":"2024-09-11T08:46:17.901871+0000","last_fullsized":"2024-09-11T08:46:17.901871+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:43:38.669837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:38.669837+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001723715,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":152,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.848343+0000","last_change":"2024-09-11T08:44:14.679790+0000","last_active":"2024-09-11T08:46:10.848343+0000","last_peered":"2024-09-11T08:46:10.848343+0000","last_clean":"2024-09-11T08:46:10.848343+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:46:10.848343+0000","last_undegraded":"2024-09-11T08:46:10.848343+0000","last_fullsized":"2024-09-11T08:46:10.848343+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:14.679598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:14.679598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011950855999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":149,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.834327+0000","last_change":"2024-09-11T08:45:22.169436+0000","last_active":"2024-09-11T08:46:10.834327+0000","last_peered":"2024-09-11T08:46:10.834327+0000","last_clean":"2024-09-11T08:46:10.834327+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:46:10.834327+0000","last_undegraded":"2024-09-11T08:46:10.834327+0000","last_fullsized":"2024-09-11T08:46:10.834327+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.169367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.169367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:43.728179+0000","scrub_duration":0.005349693,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"173'4809","reported_seq":4841,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T08:46:16.481799+0000","last_change":"2024-09-11T08:45:15.261656+0000","last_active":"2024-09-11T08:46:16.481799+0000","last_peered":"2024-09-11T08:46:16.481799+0000","last_clean":"2024-09-11T08:46:16.481799+0000","last_became_active":"2024-09-11T08:45:15.261370+0000","last_became_peered":"2024-09-11T08:45:15.261370+0000","last_unstale":"2024-09-11T08:46:16.481799+0000","last_undegraded":"2024-09-11T08:46:16.481799+0000","last_fullsized":"2024-09-11T08:46:16.481799+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:22.661111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":315162624,"num_objects":4809,"num_object_clones":0,"num_object_copies":14427,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4809,"num_write_kb":307776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":201,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.849192+0000","last_change":"2024-09-11T08:45:27.666590+0000","last_active":"2024-09-11T08:46:10.849192+0000","last_peered":"2024-09-11T08:46:10.849192+0000","last_clean":"2024-09-11T08:46:10.849192+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:46:10.849192+0000","last_undegraded":"2024-09-11T08:46:10.849192+0000","last_fullsized":"2024-09-11T08:46:10.849192+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:45:27.666502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.666502+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:44.189565+0000","scrub_duration":0.0023582730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":186,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.846697+0000","last_change":"2024-09-11T08:46:01.982031+0000","last_active":"2024-09-11T08:46:10.846697+0000","last_peered":"2024-09-11T08:46:10.846697+0000","last_clean":"2024-09-11T08:46:10.846697+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:46:10.846697+0000","last_undegraded":"2024-09-11T08:46:10.846697+0000","last_fullsized":"2024-09-11T08:46:10.846697+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.775804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.775804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:39.033860+0000","scrub_duration":0.0011899289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":182,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.844773+0000","last_change":"2024-09-11T08:45:32.632342+0000","last_active":"2024-09-11T08:46:10.844773+0000","last_peered":"2024-09-11T08:46:10.844773+0000","last_clean":"2024-09-11T08:46:10.844773+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:46:10.844773+0000","last_undegraded":"2024-09-11T08:46:10.844773+0000","last_fullsized":"2024-09-11T08:46:10.844773+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:45:32.632242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:32.632242+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:47:00.544909+0000","scrub_duration":0.0049221250000000003,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":165,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:13.322922+0000","last_change":"2024-09-11T08:46:13.322816+0000","last_active":"2024-09-11T08:46:13.322922+0000","last_peered":"2024-09-11T08:46:13.322922+0000","last_clean":"2024-09-11T08:46:13.322922+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:46:13.322922+0000","last_undegraded":"2024-09-11T08:46:13.322922+0000","last_fullsized":"2024-09-11T08:46:13.322922+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:09.638694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_clean_scrub_stamp":"2024-09-11T08:45:09.638694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00151615,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":181,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.910157+0000","last_change":"2024-09-11T08:46:01.984350+0000","last_active":"2024-09-11T08:46:17.910157+0000","last_peered":"2024-09-11T08:46:17.910157+0000","last_clean":"2024-09-11T08:46:17.910157+0000","last_became_active":"2024-09-11T08:46:01.983095+0000","last_became_peered":"2024-09-11T08:46:01.983095+0000","last_unstale":"2024-09-11T08:46:17.910157+0000","last_undegraded":"2024-09-11T08:46:17.910157+0000","last_fullsized":"2024-09-11T08:46:17.910157+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:12.278185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.278185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:33.187257+0000","scrub_duration":0.0015184790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"173'4996","reported_seq":5028,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T08:46:16.292539+0000","last_change":"2024-09-11T08:45:15.627633+0000","last_active":"2024-09-11T08:46:16.292539+0000","last_peered":"2024-09-11T08:46:16.292539+0000","last_clean":"2024-09-11T08:46:16.292539+0000","last_became_active":"2024-09-11T08:45:15.627360+0000","last_became_peered":"2024-09-11T08:45:15.627360+0000","last_unstale":"2024-09-11T08:46:16.292539+0000","last_undegraded":"2024-09-11T08:46:16.292539+0000","last_fullsized":"2024-09-11T08:46:16.292539+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4996,"log_dups_size":0,"ondisk_log_size":4996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:26.238941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327417856,"num_objects":4996,"num_object_clones":0,"num_object_copies":14988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4996,"num_write_kb":319744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":172,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.847267+0000","last_change":"2024-09-11T08:44:15.699227+0000","last_active":"2024-09-11T08:46:10.847267+0000","last_peered":"2024-09-11T08:46:10.847267+0000","last_clean":"2024-09-11T08:46:10.847267+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:46:10.847267+0000","last_undegraded":"2024-09-11T08:46:10.847267+0000","last_fullsized":"2024-09-11T08:46:10.847267+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:44:15.699067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:15.699067+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014313306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":166,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.851706+0000","last_change":"2024-09-11T08:45:20.455183+0000","last_active":"2024-09-11T08:46:10.851706+0000","last_peered":"2024-09-11T08:46:10.851706+0000","last_clean":"2024-09-11T08:46:10.851706+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:46:10.851706+0000","last_undegraded":"2024-09-11T08:46:10.851706+0000","last_fullsized":"2024-09-11T08:46:10.851706+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:20.455081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.455081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:35.877174+0000","scrub_duration":0.0016372450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":640,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T08:46:19.700373+0000","last_change":"2024-09-11T08:45:21.370742+0000","last_active":"2024-09-11T08:46:19.700373+0000","last_peered":"2024-09-11T08:46:19.700373+0000","last_clean":"2024-09-11T08:46:19.700373+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:46:19.700373+0000","last_undegraded":"2024-09-11T08:46:19.700373+0000","last_fullsized":"2024-09-11T08:46:19.700373+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:45:21.370673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.370673+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:36.160525+0000","scrub_duration":0.0017307240000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":455,"num_read_kb":450,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":157,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.907603+0000","last_change":"2024-09-11T08:46:12.673215+0000","last_active":"2024-09-11T08:46:17.907603+0000","last_peered":"2024-09-11T08:46:17.907603+0000","last_clean":"2024-09-11T08:46:17.907603+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:46:17.907603+0000","last_undegraded":"2024-09-11T08:46:17.907603+0000","last_fullsized":"2024-09-11T08:46:17.907603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:16.924404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:16.924404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024072690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":160,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.845736+0000","last_change":"2024-09-11T08:45:17.819757+0000","last_active":"2024-09-11T08:46:10.845736+0000","last_peered":"2024-09-11T08:46:10.845736+0000","last_clean":"2024-09-11T08:46:10.845736+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:46:10.845736+0000","last_undegraded":"2024-09-11T08:46:10.845736+0000","last_fullsized":"2024-09-11T08:46:10.845736+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.819707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.819707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:27.560030+0000","scrub_duration":0.0012611440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"175'5156","reported_seq":5190,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T08:46:21.026620+0000","last_change":"2024-09-11T08:45:15.262738+0000","last_active":"2024-09-11T08:46:21.026620+0000","last_peered":"2024-09-11T08:46:21.026620+0000","last_clean":"2024-09-11T08:46:21.026620+0000","last_became_active":"2024-09-11T08:45:15.262440+0000","last_became_peered":"2024-09-11T08:45:15.262440+0000","last_unstale":"2024-09-11T08:46:21.026620+0000","last_undegraded":"2024-09-11T08:46:21.026620+0000","last_fullsized":"2024-09-11T08:46:21.026620+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5156,"log_dups_size":0,"ondisk_log_size":5156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:40.707557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":337903616,"num_objects":5156,"num_object_clones":0,"num_object_copies":15468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5156,"num_write_kb":329984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":33,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.924606+0000","last_change":"2024-09-11T08:46:01.996394+0000","last_active":"2024-09-11T08:46:17.924606+0000","last_peered":"2024-09-11T08:46:17.924606+0000","last_clean":"2024-09-11T08:46:17.924606+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:46:17.924606+0000","last_undegraded":"2024-09-11T08:46:17.924606+0000","last_fullsized":"2024-09-11T08:46:17.924606+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:43:26.127412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:26.127412+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":164,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.833411+0000","last_change":"2024-09-11T08:45:18.127946+0000","last_active":"2024-09-11T08:46:10.833411+0000","last_peered":"2024-09-11T08:46:10.833411+0000","last_clean":"2024-09-11T08:46:10.833411+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:46:10.833411+0000","last_undegraded":"2024-09-11T08:46:10.833411+0000","last_fullsized":"2024-09-11T08:46:10.833411+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:18.127857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.127857+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:47.514774+0000","scrub_duration":0.0018334099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":211,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.905485+0000","last_change":"2024-09-11T08:45:29.524681+0000","last_active":"2024-09-11T08:46:17.905485+0000","last_peered":"2024-09-11T08:46:17.905485+0000","last_clean":"2024-09-11T08:46:17.905485+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:46:17.905485+0000","last_undegraded":"2024-09-11T08:46:17.905485+0000","last_fullsized":"2024-09-11T08:46:17.905485+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'79","last_scrub_stamp":"2024-09-11T08:45:29.524608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:45:29.524608+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:33.259103+0000","scrub_duration":0.002308867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":163,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.906257+0000","last_change":"2024-09-11T08:44:35.441051+0000","last_active":"2024-09-11T08:46:17.906257+0000","last_peered":"2024-09-11T08:46:17.906257+0000","last_clean":"2024-09-11T08:46:17.906257+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:46:17.906257+0000","last_undegraded":"2024-09-11T08:46:17.906257+0000","last_fullsized":"2024-09-11T08:46:17.906257+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:35.440995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:35.440995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0080921540000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":155,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.907464+0000","last_change":"2024-09-11T08:46:10.685628+0000","last_active":"2024-09-11T08:46:17.907464+0000","last_peered":"2024-09-11T08:46:17.907464+0000","last_clean":"2024-09-11T08:46:17.907464+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:46:17.907464+0000","last_undegraded":"2024-09-11T08:46:17.907464+0000","last_fullsized":"2024-09-11T08:46:17.907464+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.570246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.570246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041809610000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"175'5154","reported_seq":5188,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T08:46:21.026069+0000","last_change":"2024-09-11T08:45:15.263087+0000","last_active":"2024-09-11T08:46:21.026069+0000","last_peered":"2024-09-11T08:46:21.026069+0000","last_clean":"2024-09-11T08:46:21.026069+0000","last_became_active":"2024-09-11T08:45:15.262698+0000","last_became_peered":"2024-09-11T08:45:15.262698+0000","last_unstale":"2024-09-11T08:46:21.026069+0000","last_undegraded":"2024-09-11T08:46:21.026069+0000","last_fullsized":"2024-09-11T08:46:21.026069+0000","mapping_epoch":149,"log_start":"157'300","ondisk_log_start":"157'300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:26.678718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":337772544,"num_objects":5154,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5154,"num_write_kb":329856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":231,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.907710+0000","last_change":"2024-09-11T08:46:01.997857+0000","last_active":"2024-09-11T08:46:17.907710+0000","last_peered":"2024-09-11T08:46:17.907710+0000","last_clean":"2024-09-11T08:46:17.907710+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:46:17.907710+0000","last_undegraded":"2024-09-11T08:46:17.907710+0000","last_fullsized":"2024-09-11T08:46:17.907710+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:21.002140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:21.002140+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025090400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":148,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.903775+0000","last_change":"2024-09-11T08:44:22.010602+0000","last_active":"2024-09-11T08:46:17.903775+0000","last_peered":"2024-09-11T08:46:17.903775+0000","last_clean":"2024-09-11T08:46:17.903775+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:46:17.903775+0000","last_undegraded":"2024-09-11T08:46:17.903775+0000","last_fullsized":"2024-09-11T08:46:17.903775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.010528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.010528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047628439999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":164,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.907327+0000","last_change":"2024-09-11T08:44:26.726624+0000","last_active":"2024-09-11T08:46:17.907327+0000","last_peered":"2024-09-11T08:46:17.907327+0000","last_clean":"2024-09-11T08:46:17.907327+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:46:17.907327+0000","last_undegraded":"2024-09-11T08:46:17.907327+0000","last_fullsized":"2024-09-11T08:46:17.907327+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.726573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00117395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":169,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.836261+0000","last_change":"2024-09-11T08:46:01.988315+0000","last_active":"2024-09-11T08:46:10.836261+0000","last_peered":"2024-09-11T08:46:10.836261+0000","last_clean":"2024-09-11T08:46:10.836261+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:46:10.836261+0000","last_undegraded":"2024-09-11T08:46:10.836261+0000","last_fullsized":"2024-09-11T08:46:10.836261+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.167071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.167071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:44.717884+0000","scrub_duration":0.001796483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"173'4739","reported_seq":4771,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T08:46:16.209987+0000","last_change":"2024-09-11T08:45:15.626015+0000","last_active":"2024-09-11T08:46:16.209987+0000","last_peered":"2024-09-11T08:46:16.209987+0000","last_clean":"2024-09-11T08:46:16.209987+0000","last_became_active":"2024-09-11T08:45:15.625726+0000","last_became_peered":"2024-09-11T08:45:15.625726+0000","last_unstale":"2024-09-11T08:46:16.209987+0000","last_undegraded":"2024-09-11T08:46:16.209987+0000","last_fullsized":"2024-09-11T08:46:16.209987+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4739,"log_dups_size":0,"ondisk_log_size":4739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:36.587803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310575104,"num_objects":4739,"num_object_clones":0,"num_object_copies":14217,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4739,"num_write_kb":303296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":184,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.907707+0000","last_change":"2024-09-11T08:44:28.940078+0000","last_active":"2024-09-11T08:46:17.907707+0000","last_peered":"2024-09-11T08:46:17.907707+0000","last_clean":"2024-09-11T08:46:17.907707+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:46:17.907707+0000","last_undegraded":"2024-09-11T08:46:17.907707+0000","last_fullsized":"2024-09-11T08:46:17.907707+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:44:28.940023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.940023+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012856796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":177,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.924095+0000","last_change":"2024-09-11T08:45:25.273200+0000","last_active":"2024-09-11T08:46:17.924095+0000","last_peered":"2024-09-11T08:46:17.924095+0000","last_clean":"2024-09-11T08:46:17.924095+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:46:17.924095+0000","last_undegraded":"2024-09-11T08:46:17.924095+0000","last_fullsized":"2024-09-11T08:46:17.924095+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.273134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:35.226154+0000","scrub_duration":0.0014307370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":162,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.845811+0000","last_change":"2024-09-11T08:44:24.816974+0000","last_active":"2024-09-11T08:46:10.845811+0000","last_peered":"2024-09-11T08:46:10.845811+0000","last_clean":"2024-09-11T08:46:10.845811+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:46:10.845811+0000","last_undegraded":"2024-09-11T08:46:10.845811+0000","last_fullsized":"2024-09-11T08:46:10.845811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:24.816890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.816890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086069140000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":163,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.844844+0000","last_change":"2024-09-11T08:46:03.322747+0000","last_active":"2024-09-11T08:46:10.844844+0000","last_peered":"2024-09-11T08:46:10.844844+0000","last_clean":"2024-09-11T08:46:10.844844+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:46:10.844844+0000","last_undegraded":"2024-09-11T08:46:10.844844+0000","last_fullsized":"2024-09-11T08:46:10.844844+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.756166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.756166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059059109999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"173'4849","reported_seq":4881,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T08:46:16.535110+0000","last_change":"2024-09-11T08:45:15.260658+0000","last_active":"2024-09-11T08:46:16.535110+0000","last_peered":"2024-09-11T08:46:16.535110+0000","last_clean":"2024-09-11T08:46:16.535110+0000","last_became_active":"2024-09-11T08:45:15.260383+0000","last_became_peered":"2024-09-11T08:45:15.260383+0000","last_unstale":"2024-09-11T08:46:16.535110+0000","last_undegraded":"2024-09-11T08:46:16.535110+0000","last_fullsized":"2024-09-11T08:46:16.535110+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4849,"log_dups_size":0,"ondisk_log_size":4849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:30.230298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":317784064,"num_objects":4849,"num_object_clones":0,"num_object_copies":14547,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4849,"num_write_kb":310336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":205,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.907620+0000","last_change":"2024-09-11T08:46:02.002847+0000","last_active":"2024-09-11T08:46:17.907620+0000","last_peered":"2024-09-11T08:46:17.907620+0000","last_clean":"2024-09-11T08:46:17.907620+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:46:17.907620+0000","last_undegraded":"2024-09-11T08:46:17.907620+0000","last_fullsized":"2024-09-11T08:46:17.907620+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:43:24.983740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:24.983740+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069750209999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":155,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.837233+0000","last_change":"2024-09-11T08:45:32.219463+0000","last_active":"2024-09-11T08:46:10.837233+0000","last_peered":"2024-09-11T08:46:10.837233+0000","last_clean":"2024-09-11T08:46:10.837233+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:46:10.837233+0000","last_undegraded":"2024-09-11T08:46:10.837233+0000","last_fullsized":"2024-09-11T08:46:10.837233+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:32.219374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:32.219374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:49.435616+0000","scrub_duration":0.0069622119999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":264,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T08:46:15.335250+0000","last_change":"2024-09-11T08:46:15.335250+0000","last_active":"2024-09-11T08:46:15.335250+0000","last_peered":"2024-09-11T08:46:15.335250+0000","last_clean":"2024-09-11T08:46:15.335250+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:46:15.335250+0000","last_undegraded":"2024-09-11T08:46:15.335250+0000","last_fullsized":"2024-09-11T08:46:15.335250+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:10.659079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:10.659079+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017208200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":152,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.907418+0000","last_change":"2024-09-11T08:45:31.196687+0000","last_active":"2024-09-11T08:46:17.907418+0000","last_peered":"2024-09-11T08:46:17.907418+0000","last_clean":"2024-09-11T08:46:17.907418+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:46:17.907418+0000","last_undegraded":"2024-09-11T08:46:17.907418+0000","last_fullsized":"2024-09-11T08:46:17.907418+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.196599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.196599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:41.921593+0000","scrub_duration":0.0025248190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"173'4829","reported_seq":5451,"reported_epoch":173,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T08:46:16.290728+0000","last_change":"2024-09-11T08:46:03.286008+0000","last_active":"2024-09-11T08:46:16.290728+0000","last_peered":"2024-09-11T08:46:16.290728+0000","last_clean":"2024-09-11T08:46:00.945841+0000","last_became_active":"2024-09-11T08:46:03.248256+0000","last_became_peered":"2024-09-11T08:46:03.248256+0000","last_unstale":"2024-09-11T08:46:16.290728+0000","last_undegraded":"2024-09-11T08:46:16.290728+0000","last_fullsized":"2024-09-11T08:46:02.976155+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4829,"log_dups_size":0,"ondisk_log_size":4829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:39.697319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316473344,"num_objects":4829,"num_object_clones":0,"num_object_copies":14487,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3639,"num_objects_unfound":0,"num_objects_dirty":4829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4829,"num_write_kb":309056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":19005440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3],"avail_no_missing":["1","2","3"],"object_location_counts":[{"shards":"1,2,3","objects":3639},{"shards":"1,3","objects":1190}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":170,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.846512+0000","last_change":"2024-09-11T08:44:31.662275+0000","last_active":"2024-09-11T08:46:10.846512+0000","last_peered":"2024-09-11T08:46:10.846512+0000","last_clean":"2024-09-11T08:46:10.846512+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:46:10.846512+0000","last_undegraded":"2024-09-11T08:46:10.846512+0000","last_fullsized":"2024-09-11T08:46:10.846512+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:31.662190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.662190+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040714619999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":168,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.900056+0000","last_change":"2024-09-11T08:45:10.607400+0000","last_active":"2024-09-11T08:46:17.900056+0000","last_peered":"2024-09-11T08:46:17.900056+0000","last_clean":"2024-09-11T08:46:17.900056+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:46:17.900056+0000","last_undegraded":"2024-09-11T08:46:17.900056+0000","last_fullsized":"2024-09-11T08:46:17.900056+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:10.607288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:45:10.607288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:22.727949+0000","scrub_duration":0.0016158909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":162,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.843788+0000","last_change":"2024-09-11T08:45:12.646009+0000","last_active":"2024-09-11T08:46:10.843788+0000","last_peered":"2024-09-11T08:46:10.843788+0000","last_clean":"2024-09-11T08:46:10.843788+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:46:10.843788+0000","last_undegraded":"2024-09-11T08:46:10.843788+0000","last_fullsized":"2024-09-11T08:46:10.843788+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:12.645956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.645956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:31.984766+0000","scrub_duration":0.0014834379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":150,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.851607+0000","last_change":"2024-09-11T08:44:17.731558+0000","last_active":"2024-09-11T08:46:10.851607+0000","last_peered":"2024-09-11T08:46:10.851607+0000","last_clean":"2024-09-11T08:46:10.851607+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:46:10.851607+0000","last_undegraded":"2024-09-11T08:46:10.851607+0000","last_fullsized":"2024-09-11T08:46:10.851607+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:17.731489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:17.731489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002184517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"173'4713","reported_seq":4745,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T08:46:16.533948+0000","last_change":"2024-09-11T08:45:15.297737+0000","last_active":"2024-09-11T08:46:16.533948+0000","last_peered":"2024-09-11T08:46:16.533948+0000","last_clean":"2024-09-11T08:46:16.533948+0000","last_became_active":"2024-09-11T08:45:15.297420+0000","last_became_peered":"2024-09-11T08:45:15.297420+0000","last_unstale":"2024-09-11T08:46:16.533948+0000","last_undegraded":"2024-09-11T08:46:16.533948+0000","last_fullsized":"2024-09-11T08:46:16.533948+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4713,"log_dups_size":0,"ondisk_log_size":4713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:16.663115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308871168,"num_objects":4713,"num_object_clones":0,"num_object_copies":14139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":301632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":37,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.924015+0000","last_change":"2024-09-11T08:46:01.997537+0000","last_active":"2024-09-11T08:46:17.924015+0000","last_peered":"2024-09-11T08:46:17.924015+0000","last_clean":"2024-09-11T08:46:17.924015+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:46:17.924015+0000","last_undegraded":"2024-09-11T08:46:17.924015+0000","last_fullsized":"2024-09-11T08:46:17.924015+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:44:27.810879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:27.810879+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001826121,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":167,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.910468+0000","last_change":"2024-09-11T08:44:31.494470+0000","last_active":"2024-09-11T08:46:17.910468+0000","last_peered":"2024-09-11T08:46:17.910468+0000","last_clean":"2024-09-11T08:46:17.910468+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:46:17.910468+0000","last_undegraded":"2024-09-11T08:46:17.910468+0000","last_fullsized":"2024-09-11T08:46:17.910468+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.494397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.494397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021588290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":165,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:12.347961+0000","last_change":"2024-09-11T08:46:12.347923+0000","last_active":"2024-09-11T08:46:12.347961+0000","last_peered":"2024-09-11T08:46:12.347961+0000","last_clean":"2024-09-11T08:46:12.347961+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:46:12.347961+0000","last_undegraded":"2024-09-11T08:46:12.347961+0000","last_fullsized":"2024-09-11T08:46:12.347961+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:30.717364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:30.717364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044727940000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":151,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.905400+0000","last_change":"2024-09-11T08:44:23.367862+0000","last_active":"2024-09-11T08:46:17.905400+0000","last_peered":"2024-09-11T08:46:17.905400+0000","last_clean":"2024-09-11T08:46:17.905400+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:46:17.905400+0000","last_undegraded":"2024-09-11T08:46:17.905400+0000","last_fullsized":"2024-09-11T08:46:17.905400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:23.367786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:23.367786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030117270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"173'4718","reported_seq":5600,"reported_epoch":173,"state":"active+recovering+remapped","last_fresh":"2024-09-11T08:46:16.559413+0000","last_change":"2024-09-11T08:46:03.325028+0000","last_active":"2024-09-11T08:46:16.559413+0000","last_peered":"2024-09-11T08:46:16.559413+0000","last_clean":"2024-09-11T08:46:00.945574+0000","last_became_active":"2024-09-11T08:46:03.262389+0000","last_became_peered":"2024-09-11T08:46:03.262389+0000","last_unstale":"2024-09-11T08:46:16.559413+0000","last_undegraded":"2024-09-11T08:46:16.559413+0000","last_fullsized":"2024-09-11T08:46:16.559413+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4718,"log_dups_size":0,"ondisk_log_size":4718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:41.717967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":309198848,"num_objects":4718,"num_object_clones":0,"num_object_copies":14154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3419,"num_objects_unfound":0,"num_objects_dirty":4718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4718,"num_write_kb":301952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":421,"num_bytes_recovered":27590656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[0,4,7],"avail_no_missing":["0","4","7"],"object_location_counts":[{"shards":"0,4,7","objects":1299}],"blocked_by":[],"up_primary":6,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":179,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.903699+0000","last_change":"2024-09-11T08:45:08.140910+0000","last_active":"2024-09-11T08:46:17.903699+0000","last_peered":"2024-09-11T08:46:17.903699+0000","last_clean":"2024-09-11T08:46:17.903699+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:46:17.903699+0000","last_undegraded":"2024-09-11T08:46:17.903699+0000","last_fullsized":"2024-09-11T08:46:17.903699+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:08.140839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.140839+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:29.508166+0000","scrub_duration":0.003679547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":167,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.906830+0000","last_change":"2024-09-11T08:45:14.322786+0000","last_active":"2024-09-11T08:46:17.906830+0000","last_peered":"2024-09-11T08:46:17.906830+0000","last_clean":"2024-09-11T08:46:17.906830+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:46:17.906830+0000","last_undegraded":"2024-09-11T08:46:17.906830+0000","last_fullsized":"2024-09-11T08:46:17.906830+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.322217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.322217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:34.362792+0000","scrub_duration":0.0014821319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":165,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:11.362174+0000","last_change":"2024-09-11T08:46:11.362152+0000","last_active":"2024-09-11T08:46:11.362174+0000","last_peered":"2024-09-11T08:46:11.362174+0000","last_clean":"2024-09-11T08:46:11.362174+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:46:11.362174+0000","last_undegraded":"2024-09-11T08:46:11.362174+0000","last_fullsized":"2024-09-11T08:46:11.362174+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:29.692933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:29.692933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007324077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":149,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.836884+0000","last_change":"2024-09-11T08:45:33.228265+0000","last_active":"2024-09-11T08:46:10.836884+0000","last_peered":"2024-09-11T08:46:10.836884+0000","last_clean":"2024-09-11T08:46:10.836884+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:46:10.836884+0000","last_undegraded":"2024-09-11T08:46:10.836884+0000","last_fullsized":"2024-09-11T08:46:10.836884+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:45:33.228082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:47.790589+0000","scrub_duration":0.0064426500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"175'5157","reported_seq":5190,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T08:46:21.117819+0000","last_change":"2024-09-11T08:45:15.626107+0000","last_active":"2024-09-11T08:46:21.117819+0000","last_peered":"2024-09-11T08:46:21.117819+0000","last_clean":"2024-09-11T08:46:21.117819+0000","last_became_active":"2024-09-11T08:45:15.625778+0000","last_became_peered":"2024-09-11T08:45:15.625778+0000","last_unstale":"2024-09-11T08:46:21.117819+0000","last_undegraded":"2024-09-11T08:46:21.117819+0000","last_fullsized":"2024-09-11T08:46:21.117819+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5157,"log_dups_size":0,"ondisk_log_size":5157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:25.478875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":337969152,"num_objects":5157,"num_object_clones":0,"num_object_copies":15471,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5157,"num_write_kb":330048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"124'22","reported_seq":243,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.900442+0000","last_change":"2024-09-11T08:46:11.689219+0000","last_active":"2024-09-11T08:46:17.900442+0000","last_peered":"2024-09-11T08:46:17.900442+0000","last_clean":"2024-09-11T08:46:17.900442+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:46:17.900442+0000","last_undegraded":"2024-09-11T08:46:17.900442+0000","last_fullsized":"2024-09-11T08:46:17.900442+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:43:32.625376+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073394039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":184,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.906795+0000","last_change":"2024-09-11T08:45:20.372571+0000","last_active":"2024-09-11T08:46:17.906795+0000","last_peered":"2024-09-11T08:46:17.906795+0000","last_clean":"2024-09-11T08:46:17.906795+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:46:17.906795+0000","last_undegraded":"2024-09-11T08:46:17.906795+0000","last_fullsized":"2024-09-11T08:46:17.906795+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:45:20.372454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.372454+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:36.060441+0000","scrub_duration":0.0019547269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"168'10","reported_seq":291,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.905369+0000","last_change":"2024-09-11T08:46:01.988356+0000","last_active":"2024-09-11T08:46:17.905369+0000","last_peered":"2024-09-11T08:46:17.905369+0000","last_clean":"2024-09-11T08:46:17.905369+0000","last_became_active":"2024-09-11T08:46:01.983055+0000","last_became_peered":"2024-09-11T08:46:01.983055+0000","last_unstale":"2024-09-11T08:46:17.905369+0000","last_undegraded":"2024-09-11T08:46:17.905369+0000","last_fullsized":"2024-09-11T08:46:17.905369+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"135'9","last_scrub_stamp":"2024-09-11T08:45:11.298652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.298652+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:32.118049+0000","scrub_duration":0.001669032,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":152,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.901434+0000","last_change":"2024-09-11T08:44:15.888470+0000","last_active":"2024-09-11T08:46:17.901434+0000","last_peered":"2024-09-11T08:46:17.901434+0000","last_clean":"2024-09-11T08:46:17.901434+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:46:17.901434+0000","last_undegraded":"2024-09-11T08:46:17.901434+0000","last_fullsized":"2024-09-11T08:46:17.901434+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:15.888394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:15.888394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003201476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":177,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.923645+0000","last_change":"2024-09-11T08:45:28.247838+0000","last_active":"2024-09-11T08:46:17.923645+0000","last_peered":"2024-09-11T08:46:17.923645+0000","last_clean":"2024-09-11T08:46:17.923645+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:46:17.923645+0000","last_undegraded":"2024-09-11T08:46:17.923645+0000","last_fullsized":"2024-09-11T08:46:17.923645+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.247771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:31.258653+0000","scrub_duration":0.0015413499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":180,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.838952+0000","last_change":"2024-09-11T08:44:35.390285+0000","last_active":"2024-09-11T08:46:10.838952+0000","last_peered":"2024-09-11T08:46:10.838952+0000","last_clean":"2024-09-11T08:46:10.838952+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:46:10.838952+0000","last_undegraded":"2024-09-11T08:46:10.838952+0000","last_fullsized":"2024-09-11T08:46:10.838952+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-11T08:44:35.390134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:35.390134+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017427033000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":164,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.899818+0000","last_change":"2024-09-11T08:45:08.675251+0000","last_active":"2024-09-11T08:46:17.899818+0000","last_peered":"2024-09-11T08:46:17.899818+0000","last_clean":"2024-09-11T08:46:17.899818+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:46:17.899818+0000","last_undegraded":"2024-09-11T08:46:17.899818+0000","last_fullsized":"2024-09-11T08:46:17.899818+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:08.675179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.675179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:37.735413+0000","scrub_duration":0.003515392,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":181,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.910076+0000","last_change":"2024-09-11T08:46:01.983864+0000","last_active":"2024-09-11T08:46:17.910076+0000","last_peered":"2024-09-11T08:46:17.910076+0000","last_clean":"2024-09-11T08:46:17.910076+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:46:17.910076+0000","last_undegraded":"2024-09-11T08:46:17.910076+0000","last_fullsized":"2024-09-11T08:46:17.910076+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:25.344348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.344348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:46.305052+0000","scrub_duration":0.0013849730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":170,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.847340+0000","last_change":"2024-09-11T08:44:11.687456+0000","last_active":"2024-09-11T08:46:10.847340+0000","last_peered":"2024-09-11T08:46:10.847340+0000","last_clean":"2024-09-11T08:46:10.847340+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:46:10.847340+0000","last_undegraded":"2024-09-11T08:46:10.847340+0000","last_fullsized":"2024-09-11T08:46:10.847340+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:11.687402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:11.687402+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018414239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":167,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.905610+0000","last_change":"2024-09-11T08:44:32.483521+0000","last_active":"2024-09-11T08:46:17.905610+0000","last_peered":"2024-09-11T08:46:17.905610+0000","last_clean":"2024-09-11T08:46:17.905610+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:46:17.905610+0000","last_undegraded":"2024-09-11T08:46:17.905610+0000","last_fullsized":"2024-09-11T08:46:17.905610+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:44:32.483439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013511770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":151,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.837838+0000","last_change":"2024-09-11T08:45:23.213642+0000","last_active":"2024-09-11T08:46:10.837838+0000","last_peered":"2024-09-11T08:46:10.837838+0000","last_clean":"2024-09-11T08:46:10.837838+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:46:10.837838+0000","last_undegraded":"2024-09-11T08:46:10.837838+0000","last_fullsized":"2024-09-11T08:46:10.837838+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.213592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.213592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:50.258501+0000","scrub_duration":0.001600473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":160,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.843483+0000","last_change":"2024-09-11T08:45:07.618618+0000","last_active":"2024-09-11T08:46:10.843483+0000","last_peered":"2024-09-11T08:46:10.843483+0000","last_clean":"2024-09-11T08:46:10.843483+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:46:10.843483+0000","last_undegraded":"2024-09-11T08:46:10.843483+0000","last_fullsized":"2024-09-11T08:46:10.843483+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.618564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.618564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:37.053995+0000","scrub_duration":0.0016216729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":175,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.902128+0000","last_change":"2024-09-11T08:46:15.658863+0000","last_active":"2024-09-11T08:46:17.902128+0000","last_peered":"2024-09-11T08:46:17.902128+0000","last_clean":"2024-09-11T08:46:17.902128+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:46:17.902128+0000","last_undegraded":"2024-09-11T08:46:17.902128+0000","last_fullsized":"2024-09-11T08:46:17.902128+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:18.928481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:18.928481+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047130310000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":156,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.845598+0000","last_change":"2024-09-11T08:44:09.678084+0000","last_active":"2024-09-11T08:46:10.845598+0000","last_peered":"2024-09-11T08:46:10.845598+0000","last_clean":"2024-09-11T08:46:10.845598+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:46:10.845598+0000","last_undegraded":"2024-09-11T08:46:10.845598+0000","last_fullsized":"2024-09-11T08:46:10.845598+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:09.677991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:09.677991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066552809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":250,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:12.762038+0000","last_change":"2024-09-11T08:45:20.119727+0000","last_active":"2024-09-11T08:46:12.762038+0000","last_peered":"2024-09-11T08:46:12.762038+0000","last_clean":"2024-09-11T08:46:12.762038+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:46:12.762038+0000","last_undegraded":"2024-09-11T08:46:12.762038+0000","last_fullsized":"2024-09-11T08:46:12.762038+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:20.119626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.119626+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:49.101497+0000","scrub_duration":0.0018087660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":142,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.907391+0000","last_change":"2024-09-11T08:44:20.017341+0000","last_active":"2024-09-11T08:46:17.907391+0000","last_peered":"2024-09-11T08:46:17.907391+0000","last_clean":"2024-09-11T08:46:17.907391+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:46:17.907391+0000","last_undegraded":"2024-09-11T08:46:17.907391+0000","last_fullsized":"2024-09-11T08:46:17.907391+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:20.017261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:20.017261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001558369,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"152'15","reported_seq":173,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.836689+0000","last_change":"2024-09-11T08:45:21.132138+0000","last_active":"2024-09-11T08:46:10.836689+0000","last_peered":"2024-09-11T08:46:10.836689+0000","last_clean":"2024-09-11T08:46:10.836689+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:46:10.836689+0000","last_undegraded":"2024-09-11T08:46:10.836689+0000","last_fullsized":"2024-09-11T08:46:10.836689+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'15","last_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.132083+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:32.804028+0000","scrub_duration":0.003984156,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":167,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.906034+0000","last_change":"2024-09-11T08:45:27.436673+0000","last_active":"2024-09-11T08:46:17.906034+0000","last_peered":"2024-09-11T08:46:17.906034+0000","last_clean":"2024-09-11T08:46:17.906034+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:46:17.906034+0000","last_undegraded":"2024-09-11T08:46:17.906034+0000","last_fullsized":"2024-09-11T08:46:17.906034+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.436552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:30.103656+0000","scrub_duration":0.001660176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":264,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.901978+0000","last_change":"2024-09-11T08:45:28.692173+0000","last_active":"2024-09-11T08:46:17.901978+0000","last_peered":"2024-09-11T08:46:17.901978+0000","last_clean":"2024-09-11T08:46:17.901978+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:46:17.901978+0000","last_undegraded":"2024-09-11T08:46:17.901978+0000","last_fullsized":"2024-09-11T08:46:17.901978+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:28.692100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.692100+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:37.432189+0000","scrub_duration":0.0019579020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":160,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.846883+0000","last_change":"2024-09-11T08:45:14.581287+0000","last_active":"2024-09-11T08:46:10.846883+0000","last_peered":"2024-09-11T08:46:10.846883+0000","last_clean":"2024-09-11T08:46:10.846883+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:46:10.846883+0000","last_undegraded":"2024-09-11T08:46:10.846883+0000","last_fullsized":"2024-09-11T08:46:10.846883+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.581231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.581231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:41.692088+0000","scrub_duration":0.001580932,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":165,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.904036+0000","last_change":"2024-09-11T08:43:37.137260+0000","last_active":"2024-09-11T08:46:17.904036+0000","last_peered":"2024-09-11T08:46:17.904036+0000","last_clean":"2024-09-11T08:46:17.904036+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:46:17.904036+0000","last_undegraded":"2024-09-11T08:46:17.904036+0000","last_fullsized":"2024-09-11T08:46:17.904036+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:37.137173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:37.137173+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085039490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":158,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.907820+0000","last_change":"2024-09-11T08:45:27.198021+0000","last_active":"2024-09-11T08:46:17.907820+0000","last_peered":"2024-09-11T08:46:17.907820+0000","last_clean":"2024-09-11T08:46:17.907820+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:46:17.907820+0000","last_undegraded":"2024-09-11T08:46:17.907820+0000","last_fullsized":"2024-09-11T08:46:17.907820+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.197944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.197944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:39.065394+0000","scrub_duration":0.0015230210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":162,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.846190+0000","last_change":"2024-09-11T08:45:21.736656+0000","last_active":"2024-09-11T08:46:10.846190+0000","last_peered":"2024-09-11T08:46:10.846190+0000","last_clean":"2024-09-11T08:46:10.846190+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:46:10.846190+0000","last_undegraded":"2024-09-11T08:46:10.846190+0000","last_fullsized":"2024-09-11T08:46:10.846190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:21.736573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.736573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:27.195162+0000","scrub_duration":0.0020360389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":149,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.832863+0000","last_change":"2024-09-11T08:45:28.259156+0000","last_active":"2024-09-11T08:46:10.832863+0000","last_peered":"2024-09-11T08:46:10.832863+0000","last_clean":"2024-09-11T08:46:10.832863+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:46:10.832863+0000","last_undegraded":"2024-09-11T08:46:10.832863+0000","last_fullsized":"2024-09-11T08:46:10.832863+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.259103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.259103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:47.651285+0000","scrub_duration":0.0015518800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":195,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.899498+0000","last_change":"2024-09-11T08:46:01.993547+0000","last_active":"2024-09-11T08:46:17.899498+0000","last_peered":"2024-09-11T08:46:17.899498+0000","last_clean":"2024-09-11T08:46:17.899498+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:46:17.899498+0000","last_undegraded":"2024-09-11T08:46:17.899498+0000","last_fullsized":"2024-09-11T08:46:17.899498+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:45:06.657912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.657912+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:34.933163+0000","scrub_duration":0.0023744109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":167,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.910344+0000","last_change":"2024-09-11T08:45:15.298564+0000","last_active":"2024-09-11T08:46:17.910344+0000","last_peered":"2024-09-11T08:46:17.910344+0000","last_clean":"2024-09-11T08:46:17.910344+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:46:17.910344+0000","last_undegraded":"2024-09-11T08:46:17.910344+0000","last_fullsized":"2024-09-11T08:46:17.910344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.298476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.298476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:25.723721+0000","scrub_duration":0.0033059629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":151,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.836962+0000","last_change":"2024-09-11T08:45:14.165632+0000","last_active":"2024-09-11T08:46:10.836962+0000","last_peered":"2024-09-11T08:46:10.836962+0000","last_clean":"2024-09-11T08:46:10.836962+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:46:10.836962+0000","last_undegraded":"2024-09-11T08:46:10.836962+0000","last_fullsized":"2024-09-11T08:46:10.836962+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.165576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.165576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:31.726848+0000","scrub_duration":0.0012147639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":150,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.845316+0000","last_change":"2024-09-11T08:43:29.349431+0000","last_active":"2024-09-11T08:46:10.845316+0000","last_peered":"2024-09-11T08:46:10.845316+0000","last_clean":"2024-09-11T08:46:10.845316+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:46:10.845316+0000","last_undegraded":"2024-09-11T08:46:10.845316+0000","last_fullsized":"2024-09-11T08:46:10.845316+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.349351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.349351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006276294,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":162,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.832981+0000","last_change":"2024-09-11T08:44:12.198399+0000","last_active":"2024-09-11T08:46:10.832981+0000","last_peered":"2024-09-11T08:46:10.832981+0000","last_clean":"2024-09-11T08:46:10.832981+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:46:10.832981+0000","last_undegraded":"2024-09-11T08:46:10.832981+0000","last_fullsized":"2024-09-11T08:46:10.832981+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:44:12.198335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.198335+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020517040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":168,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.907282+0000","last_change":"2024-09-11T08:45:14.574739+0000","last_active":"2024-09-11T08:46:17.907282+0000","last_peered":"2024-09-11T08:46:17.907282+0000","last_clean":"2024-09-11T08:46:17.907282+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:46:17.907282+0000","last_undegraded":"2024-09-11T08:46:17.907282+0000","last_fullsized":"2024-09-11T08:46:17.907282+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.574678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.574678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:25.976833+0000","scrub_duration":0.0013819469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":163,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.905165+0000","last_change":"2024-09-11T08:44:28.484120+0000","last_active":"2024-09-11T08:46:17.905165+0000","last_peered":"2024-09-11T08:46:17.905165+0000","last_clean":"2024-09-11T08:46:17.905165+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:46:17.905165+0000","last_undegraded":"2024-09-11T08:46:17.905165+0000","last_fullsized":"2024-09-11T08:46:17.905165+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:28.484047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.484047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001479172,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":159,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.837132+0000","last_change":"2024-09-11T08:46:01.984620+0000","last_active":"2024-09-11T08:46:10.837132+0000","last_peered":"2024-09-11T08:46:10.837132+0000","last_clean":"2024-09-11T08:46:10.837132+0000","last_became_active":"2024-09-11T08:46:01.984167+0000","last_became_peered":"2024-09-11T08:46:01.984167+0000","last_unstale":"2024-09-11T08:46:10.837132+0000","last_undegraded":"2024-09-11T08:46:10.837132+0000","last_fullsized":"2024-09-11T08:46:10.837132+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:32.114990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:32.114990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014089513999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"136'6","reported_seq":193,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.837388+0000","last_change":"2024-09-11T08:46:01.996138+0000","last_active":"2024-09-11T08:46:10.837388+0000","last_peered":"2024-09-11T08:46:10.837388+0000","last_clean":"2024-09-11T08:46:10.837388+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:46:10.837388+0000","last_undegraded":"2024-09-11T08:46:10.837388+0000","last_fullsized":"2024-09-11T08:46:10.837388+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'6","last_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:44:21.302870+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012512683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":173,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.845691+0000","last_change":"2024-09-11T08:45:21.481725+0000","last_active":"2024-09-11T08:46:10.845691+0000","last_peered":"2024-09-11T08:46:10.845691+0000","last_clean":"2024-09-11T08:46:10.845691+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:46:10.845691+0000","last_undegraded":"2024-09-11T08:46:10.845691+0000","last_fullsized":"2024-09-11T08:46:10.845691+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:21.481634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.481634+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:29.627943+0000","scrub_duration":0.0025664590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":17,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.923771+0000","last_change":"2024-09-11T08:46:01.979915+0000","last_active":"2024-09-11T08:46:17.923771+0000","last_peered":"2024-09-11T08:46:17.923771+0000","last_clean":"2024-09-11T08:46:17.923771+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:46:17.923771+0000","last_undegraded":"2024-09-11T08:46:17.923771+0000","last_fullsized":"2024-09-11T08:46:17.923771+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.196277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.196277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:37.455059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":171,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.905369+0000","last_change":"2024-09-11T08:45:23.348327+0000","last_active":"2024-09-11T08:46:17.905369+0000","last_peered":"2024-09-11T08:46:17.905369+0000","last_clean":"2024-09-11T08:46:17.905369+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:46:17.905369+0000","last_undegraded":"2024-09-11T08:46:17.905369+0000","last_fullsized":"2024-09-11T08:46:17.905369+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.348275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.348275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:38.671979+0000","scrub_duration":0.0035843659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":170,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.848544+0000","last_change":"2024-09-11T08:45:23.674163+0000","last_active":"2024-09-11T08:46:10.848544+0000","last_peered":"2024-09-11T08:46:10.848544+0000","last_clean":"2024-09-11T08:46:10.848544+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:46:10.848544+0000","last_undegraded":"2024-09-11T08:46:10.848544+0000","last_fullsized":"2024-09-11T08:46:10.848544+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.674102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.674102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:44.657753+0000","scrub_duration":0.001440293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":155,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.836156+0000","last_change":"2024-09-11T08:44:23.320368+0000","last_active":"2024-09-11T08:46:10.836156+0000","last_peered":"2024-09-11T08:46:10.836156+0000","last_clean":"2024-09-11T08:46:10.836156+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:46:10.836156+0000","last_undegraded":"2024-09-11T08:46:10.836156+0000","last_fullsized":"2024-09-11T08:46:10.836156+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:23.320301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:23.320301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017133368999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":29,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-11T08:46:17.924467+0000","last_change":"2024-09-11T08:46:02.002471+0000","last_active":"2024-09-11T08:46:17.924467+0000","last_peered":"2024-09-11T08:46:17.924467+0000","last_clean":"2024-09-11T08:46:17.924467+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:46:17.924467+0000","last_undegraded":"2024-09-11T08:46:17.924467+0000","last_fullsized":"2024-09-11T08:46:17.924467+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:45:25.620635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.620635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:30.539650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":152,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-11T08:46:10.847109+0000","last_change":"2024-09-11T08:43:30.336272+0000","last_active":"2024-09-11T08:46:10.847109+0000","last_peered":"2024-09-11T08:46:10.847109+0000","last_clean":"2024-09-11T08:46:10.847109+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:46:10.847109+0000","last_undegraded":"2024-09-11T08:46:10.847109+0000","last_fullsized":"2024-09-11T08:46:10.847109+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:30.336188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:30.336188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044263979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":5140840448,"num_objects":78443,"num_object_clones":0,"num_object_copies":235329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7058,"num_objects_unfound":0,"num_objects_dirty":78443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78443,"num_write_kb":5020352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":711,"num_bytes_recovered":46596096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15682502656,"data_stored":15682502656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78143,"ondisk_log_size":78143,"up":48,"acting":47,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":1584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":456,"ondisk_log_size":456,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":455,"num_read_kb":450,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299490,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1907328,"kb_used_data":1609972,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91837824,"statfs":{"total":95995035648,"available":94041931776,"internally_reserved":0,"allocated":1648611328,"data_stored":1648214114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.89900000000000002,"15min":0.63300000000000001},"min":{"1min":0.34499999999999997,"5min":0.22,"15min":0.22},"max":{"1min":1.2,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":1.022,"15min":0.68000000000000005},"min":{"1min":0.36699999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.72099999999999997,"5min":5.9790000000000001,"15min":5.9790000000000001},"last":1.7589999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":1.056,"15min":0.68999999999999995},"min":{"1min":0.25900000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.238,"5min":8.5030000000000001,"15min":8.5030000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.88300000000000001,"15min":0.64300000000000002},"min":{"1min":0.33000000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.747,"5min":6.593,"15min":6.593},"last":3.097}]},{"osd":2,"last update":"Wed Sep 11 08:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.97499999999999998,"15min":0.66200000000000003},"min":{"1min":0.373,"5min":0.255,"15min":0.255},"max":{"1min":1.1539999999999999,"5min":4.6029999999999998,"15min":4.6029999999999998},"last":2.1589999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.96299999999999997,"15min":0.65000000000000002},"min":{"1min":0.30099999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.0059999999999998,"5min":5.7990000000000004,"15min":5.7990000000000004},"last":2.8090000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.0780000000000001,"15min":0.70699999999999996},"min":{"1min":0.38800000000000001,"5min":0.25,"15min":0.25},"max":{"1min":2.7269999999999999,"5min":7.8609999999999998,"15min":7.8609999999999998},"last":2.0059999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.2350000000000001,"15min":0.76500000000000001},"min":{"1min":0.318,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.5940000000000001,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":2.8370000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48699999999999999,"15min":0.44400000000000001},"min":{"1min":0.29299999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.66100000000000003,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.249},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.441},"min":{"1min":0.36099999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.048,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.29599999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.49299999999999999,"15min":0.45800000000000002},"min":{"1min":0.29399999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.2649999999999999,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.432},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60099999999999998,"15min":0.495},"min":{"1min":0.39100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82999999999999996,"5min":2.806,"15min":2.806},"last":2.496}]},{"osd":6,"last update":"Wed Sep 11 08:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.627,"15min":0.52300000000000002},"min":{"1min":0.38100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.72499999999999998,"5min":3.5979999999999999,"15min":3.5979999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.51500000000000001},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.1309999999999998,"5min":2.633,"15min":2.633},"last":0.34100000000000003}]}]},{"osd":6,"up_from":45,"seq":193273528423,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1465968,"kb_used_data":1164456,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":92279184,"statfs":{"total":95995035648,"available":94493884416,"internally_reserved":0,"allocated":1192402944,"data_stored":1192017854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":1.4399999999999999,"15min":0.83099999999999996},"min":{"1min":0.222,"5min":0.217,"15min":0.217},"max":{"1min":2.0049999999999999,"5min":28.302,"15min":28.302},"last":1.956},{"interface":"front","average":{"1min":0.82199999999999995,"5min":1.4390000000000001,"15min":0.83099999999999996},"min":{"1min":0.29899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":2.5750000000000002,"5min":28.314,"15min":28.314},"last":1.825}]},{"osd":1,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.1930000000000001,"15min":0.747},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.4649999999999999,"5min":12.586,"15min":12.586},"last":1.5700000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.077,"15min":0.70699999999999996},"min":{"1min":0.25700000000000001,"5min":0.253,"15min":0.253},"max":{"1min":3.2269999999999999,"5min":10.834,"15min":10.834},"last":2.4340000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.92500000000000004,"15min":0.67300000000000004},"min":{"1min":0.27700000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.82299999999999995,"5min":14.313000000000001,"15min":14.313000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.95899999999999996,"15min":0.65900000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":2.4660000000000002,"5min":6.5979999999999999,"15min":6.5979999999999999},"last":2.4660000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1.2529999999999999,"15min":0.76200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.2130000000000001,"5min":11.433,"15min":11.433},"last":3.2130000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":1.1060000000000001,"15min":0.72799999999999998},"min":{"1min":0.311,"5min":0.224,"15min":0.224},"max":{"1min":1.194,"5min":9.1069999999999993,"15min":9.1069999999999993},"last":0.318}]},{"osd":4,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.47799999999999998,"15min":0.433},"min":{"1min":0.25600000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.577,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.44600000000000001,"15min":0.42299999999999999},"min":{"1min":0.28299999999999997,"5min":0.27300000000000002,"15min":0.26400000000000001},"max":{"1min":1.1990000000000001,"5min":1.361,"15min":1.361},"last":0.40100000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.53300000000000003,"15min":0.47099999999999997},"min":{"1min":0.32700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":1.246,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.505,"15min":0.46000000000000002},"min":{"1min":0.30199999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.2609999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.373}]},{"osd":7,"last update":"Wed Sep 11 08:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50900000000000001,"15min":0.46000000000000002},"min":{"1min":0.39500000000000002,"5min":0.25900000000000001,"15min":0.247},"max":{"1min":0.66800000000000004,"5min":1.706,"15min":1.706},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52900000000000003,"15min":0.502},"min":{"1min":0.34499999999999997,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.68999999999999995,"5min":1.3,"15min":1.3},"last":0.55700000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757355,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2921344,"kb_used_data":2615664,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":90823808,"statfs":{"total":95995035648,"available":93003579392,"internally_reserved":0,"allocated":2678439936,"data_stored":2678046907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89000000000000001,"15min":0.67000000000000004},"min":{"1min":0.40100000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":2.637,"5min":7.3899999999999997,"15min":7.3899999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.1080000000000001,"15min":0.72099999999999997},"min":{"1min":0.28299999999999997,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":3.1320000000000001,"5min":6.5789999999999997,"15min":6.5789999999999997},"last":0.86699999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":1.02,"15min":0.71599999999999997},"min":{"1min":0.312,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.0270000000000001,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.95999999999999996,"15min":0.66200000000000003},"min":{"1min":0.32000000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.7370000000000001,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":1.8120000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.98799999999999999,"15min":0.68999999999999995},"min":{"1min":0.33500000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.0310000000000001,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":1.387},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.89600000000000002,"15min":0.67000000000000004},"min":{"1min":0.312,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.306,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":1.75}]},{"osd":3,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.98499999999999999,"15min":0.68600000000000005},"min":{"1min":0.35499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.284,"5min":6.0460000000000003,"15min":6.0460000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":1.1850000000000001,"15min":0.77100000000000002},"min":{"1min":0.46400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.0539999999999998,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":0.64600000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.44500000000000001,"15min":0.40200000000000002},"min":{"1min":0.27200000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.3540000000000001,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.8340000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.48799999999999999,"15min":0.43099999999999999},"min":{"1min":0.34799999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.1899999999999999,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.499}]},{"osd":6,"last update":"Wed Sep 11 08:45:20 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.46600000000000003,"15min":0.44},"min":{"1min":0.314,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.89100000000000001,"5min":1.702,"15min":1.702},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47299999999999998,"15min":0.439},"min":{"1min":0.27200000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.73399999999999999,"5min":2.319,"15min":2.319},"last":1.893}]},{"osd":7,"last update":"Wed Sep 11 08:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47999999999999998,"15min":0.48499999999999999},"min":{"1min":0.33800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66100000000000003,"5min":1.097,"15min":1.097},"last":1.7829999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.51200000000000001,"15min":0.53700000000000003},"min":{"1min":0.308,"5min":0.307,"15min":0.307},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.61299999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986287,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1930296,"kb_used_data":1628784,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91814856,"statfs":{"total":95995035648,"available":94018412544,"internally_reserved":0,"allocated":1667874816,"data_stored":1667480616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":2.6179999999999999,"15min":1.2370000000000001},"min":{"1min":0.23000000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":2.2959999999999998,"5min":89.510000000000005,"15min":89.510000000000005},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.046,"5min":2.706,"15min":1.2789999999999999},"min":{"1min":0.27500000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":5.399,"5min":89.765000000000001,"15min":89.765000000000001},"last":1.294}]},{"osd":1,"last update":"Wed Sep 11 08:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.1379999999999999,"15min":0.755},"min":{"1min":0.28599999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":4.8159999999999998,"5min":18.596,"15min":18.596},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.99299999999999999,"15min":0.70399999999999996},"min":{"1min":0.25700000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.0490000000000004,"5min":18.754000000000001,"15min":18.754000000000001},"last":1.633}]},{"osd":2,"last update":"Wed Sep 11 08:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.83199999999999996,"15min":0.65800000000000003},"min":{"1min":0.29499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":3.0960000000000001,"5min":5.3330000000000002,"15min":5.3330000000000002},"last":1.661},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.85399999999999998,"15min":0.66700000000000004},"min":{"1min":0.27500000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.7789999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":1.3149999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:45:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":2.9980000000000002,"15min":1.381},"min":{"1min":0.39200000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":7.5289999999999999,"5min":89.881,"15min":89.881},"last":1.853},{"interface":"front","average":{"1min":0.96499999999999997,"5min":2.7530000000000001,"15min":1.29},"min":{"1min":0.38600000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":8.7059999999999995,"5min":89.715000000000003,"15min":89.715000000000003},"last":1.544}]},{"osd":5,"last update":"Wed Sep 11 08:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.48699999999999999,"15min":0.43099999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":1.5169999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.46800000000000003,"15min":0.434},"min":{"1min":0.307,"5min":0.24199999999999999,"15min":0.23699999999999999},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.6819999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:45:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47399999999999998,"15min":0.44},"min":{"1min":0.309,"5min":0.26700000000000002,"15min":0.248},"max":{"1min":0.73799999999999999,"5min":1.792,"15min":1.792},"last":1.26},{"interface":"front","average":{"1min":0.5,"5min":0.53900000000000003,"15min":0.48699999999999999},"min":{"1min":0.33000000000000002,"5min":0.28100000000000003,"15min":0.26400000000000001},"max":{"1min":0.75600000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:45:38 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.505,"15min":0.48999999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.72099999999999997,"5min":1.1339999999999999,"15min":1.194},"last":1.7310000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51800000000000002,"15min":0.48599999999999999},"min":{"1min":0.33200000000000002,"5min":0.30499999999999999,"15min":0.28799999999999998},"max":{"1min":1.083,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":2.355}]}]},{"osd":3,"up_from":26,"seq":111669149812,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2145464,"kb_used_data":1848112,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91599688,"statfs":{"total":95995035648,"available":93798080512,"internally_reserved":0,"allocated":1892466688,"data_stored":1892057759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:45:16 2024","interfaces":[{"interface":"back","average":{"1min":2.0059999999999998,"5min":1.5680000000000001,"15min":0.77000000000000002},"min":{"1min":0.19500000000000001,"5min":0.189,"15min":0.184},"max":{"1min":7.4530000000000003,"5min":10.901999999999999,"15min":10.901999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":2.3220000000000001,"5min":1.944,"15min":0.91300000000000003},"min":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23200000000000001},"max":{"1min":9.6820000000000004,"5min":9.6820000000000004,"15min":9.6820000000000004},"last":0.30499999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:45:16 2024","interfaces":[{"interface":"back","average":{"1min":1.9670000000000001,"5min":1.8149999999999999,"15min":0.872},"min":{"1min":0.26600000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":9.5619999999999994,"5min":9.5619999999999994,"15min":9.5619999999999994},"last":115.601},{"interface":"front","average":{"1min":1.802,"5min":1.679,"15min":0.83399999999999996},"min":{"1min":0.23300000000000001,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":6.2290000000000001,"5min":9.0239999999999991,"15min":9.0239999999999991},"last":115.474}]},{"osd":2,"last update":"Wed Sep 11 08:45:16 2024","interfaces":[{"interface":"back","average":{"1min":2.0710000000000002,"5min":1.7310000000000001,"15min":0.86599999999999999},"min":{"1min":0.26400000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":6.2969999999999997,"5min":8.9510000000000005,"15min":8.9510000000000005},"last":0.32400000000000001},{"interface":"front","average":{"1min":1.8360000000000001,"5min":1.704,"15min":0.85499999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23599999999999999},"max":{"1min":10.170999999999999,"5min":10.695,"15min":10.695},"last":0.33700000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:45:28 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.2789999999999999,"15min":0.80500000000000005},"min":{"1min":0.33000000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":6.1609999999999996,"5min":8.9969999999999999,"15min":8.9969999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":1.018,"5min":1.155,"15min":0.75600000000000001},"min":{"1min":0.34200000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":6.3369999999999997,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":0.432}]},{"osd":5,"last update":"Wed Sep 11 08:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":1.3759999999999999,"15min":0.83799999999999997},"min":{"1min":0.34300000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.9079999999999999,"5min":11.526999999999999,"15min":11.526999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.2350000000000001,"15min":0.77900000000000003},"min":{"1min":0.38600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.6000000000000001,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":0.40300000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:45:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4690000000000001,"15min":0.88700000000000001},"min":{"1min":0.311,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":6.1980000000000004,"5min":11.494999999999999,"15min":11.494999999999999},"last":0.497},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.482,"15min":0.86899999999999999},"min":{"1min":0.41099999999999998,"5min":0.311,"15min":0.311},"max":{"1min":7.7249999999999996,"5min":7.7249999999999996,"15min":7.7249999999999996},"last":0.38800000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.4279999999999999,"15min":0.88600000000000001},"min":{"1min":0.37,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92900000000000005,"5min":11.561,"15min":11.561},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":1.25,"15min":0.81399999999999995},"min":{"1min":0.372,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.9380000000000002,"5min":6.3170000000000002,"15min":6.3170000000000002},"last":0.59199999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411448,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2376844,"kb_used_data":2075332,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91368308,"statfs":{"total":95995035648,"available":93561147392,"internally_reserved":0,"allocated":2125139968,"data_stored":2124774225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.954,"15min":0.88900000000000001},"min":{"1min":0.20699999999999999,"5min":0.159,"15min":0.159},"max":{"1min":8.2059999999999995,"5min":24.989000000000001,"15min":24.989000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.97999999999999998,"5min":2.3220000000000001,"15min":1.024},"min":{"1min":0.23599999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":7.0170000000000003,"5min":24.803999999999998,"15min":24.803999999999998},"last":0.309}]},{"osd":1,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.385,"15min":0.71699999999999997},"min":{"1min":0.25800000000000001,"5min":0.19,"15min":0.19},"max":{"1min":8.3100000000000005,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.631,"15min":0.79600000000000004},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":4.9260000000000002,"5min":9.3119999999999994,"15min":9.3119999999999994},"last":1.401}]},{"osd":3,"last update":"Wed Sep 11 08:46:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.895,"15min":0.91400000000000003},"min":{"1min":0.26800000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":8.0350000000000001,"5min":9.5129999999999999,"15min":9.5129999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.611,"5min":1.8680000000000001,"15min":0.91400000000000003},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.26800000000000002},"max":{"1min":5.1310000000000002,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.40999999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:45:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":1.1659999999999999,"15min":0.76000000000000001},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.3069999999999999,"5min":6.633,"15min":6.633},"last":0.498},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.131,"15min":0.747},"min":{"1min":0.372,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":6.9320000000000004,"5min":6.9320000000000004,"15min":6.9320000000000004},"last":0.35499999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.1930000000000001,"15min":0.77600000000000002},"min":{"1min":0.38700000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":4.8949999999999996,"5min":6.9550000000000001,"15min":6.9550000000000001},"last":0.439},{"interface":"front","average":{"1min":0.70499999999999996,"5min":1,"15min":0.70599999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.0419999999999998,"5min":5.9530000000000003,"15min":5.9530000000000003},"last":0.84899999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":1.1910000000000001,"15min":0.78800000000000003},"min":{"1min":0.33600000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.5819999999999999,"5min":6.6609999999999996,"15min":6.6609999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.181,"5min":1.139,"15min":0.755},"min":{"1min":0.36199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":6.9779999999999998,"5min":6.9779999999999998,"15min":6.9779999999999998},"last":0.38500000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:45:29 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":1.008,"15min":0.72699999999999998},"min":{"1min":0.38300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.90800000000000003,"5min":5.9909999999999997,"15min":5.9909999999999997},"last":0.875},{"interface":"front","average":{"1min":0.77900000000000003,"5min":1.226,"15min":0.81200000000000006},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.3519999999999999,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.56000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2446268,"kb_used_data":2136436,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91298884,"statfs":{"total":95995035648,"available":93490057216,"internally_reserved":0,"allocated":2187710464,"data_stored":2187296818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:45:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":2.7410000000000001,"15min":1.125},"min":{"1min":0.192,"5min":0.183,"15min":0.183},"max":{"1min":4.2779999999999996,"5min":117.01600000000001,"15min":117.01600000000001},"last":1.018},{"interface":"front","average":{"1min":0.63700000000000001,"5min":3.3519999999999999,"15min":1.343},"min":{"1min":0.21299999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":3.5870000000000002,"5min":117.62,"15min":117.62},"last":0.309}]},{"osd":2,"last update":"Wed Sep 11 08:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":1.7709999999999999,"15min":0.84299999999999997},"min":{"1min":0.21099999999999999,"5min":0.191,"15min":0.191},"max":{"1min":2.4180000000000001,"5min":20.628,"15min":20.628},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.696,"15min":0.81000000000000005},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":4.4779999999999998,"5min":20.148,"15min":20.148},"last":0.67900000000000005}]},{"osd":3,"last update":"Wed Sep 11 08:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":2.5369999999999999,"15min":1.1180000000000001},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":2.2730000000000001,"5min":116.98,"15min":116.98},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.99299999999999999,"5min":2.891,"15min":1.25},"min":{"1min":0.26400000000000001,"5min":0.22,"15min":0.22},"max":{"1min":4.2679999999999998,"5min":117.56699999999999,"15min":117.56699999999999},"last":0.27100000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:45:29 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":1.115,"15min":0.72399999999999998},"min":{"1min":0.35799999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.9430000000000001,"5min":11.19,"15min":11.19},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.94799999999999995,"15min":0.69099999999999995},"min":{"1min":0.39800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":2.0699999999999998,"5min":7.7169999999999996,"15min":7.7169999999999996},"last":1.885}]},{"osd":5,"last update":"Wed Sep 11 08:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":2.5139999999999998,"15min":1.218},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.2280000000000002,"5min":103.337,"15min":103.337},"last":0.34200000000000003},{"interface":"front","average":{"1min":1.0229999999999999,"5min":2.431,"15min":1.1850000000000001},"min":{"1min":0.38300000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.7829999999999999,"5min":103.163,"15min":103.163},"last":1.167}]},{"osd":6,"last update":"Wed Sep 11 08:45:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.96599999999999997,"15min":0.70799999999999996},"min":{"1min":0.34300000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":3.0600000000000001,"5min":8.9149999999999991,"15min":8.9149999999999991},"last":1.9299999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.292,"15min":0.83399999999999996},"min":{"1min":0.47099999999999997,"5min":0.31,"15min":0.31},"max":{"1min":3.8929999999999998,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":0.36299999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:45:29 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":1.1719999999999999,"15min":0.77500000000000002},"min":{"1min":0.38200000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.0059999999999998,"5min":11.315,"15min":11.315},"last":1.103},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.997,"15min":0.72599999999999998},"min":{"1min":0.42699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":2.0950000000000002,"5min":8.8800000000000008,"15min":8.8800000000000008},"last":2.665}]}]},{"osd":0,"up_from":8,"seq":34359738497,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2425192,"kb_used_data":2119520,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91319960,"statfs":{"total":95995035648,"available":93511639040,"internally_reserved":0,"allocated":2170388480,"data_stored":2169995495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":1.627,"15min":0.752},"min":{"1min":0.188,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":4.9400000000000004,"5min":7.758,"15min":7.758},"last":1.369},{"interface":"front","average":{"1min":0.72799999999999998,"5min":1.3260000000000001,"15min":0.65500000000000003},"min":{"1min":0.20300000000000001,"5min":0.186,"15min":0.186},"max":{"1min":2.9590000000000001,"5min":6.9969999999999999,"15min":6.9969999999999999},"last":1.4550000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.762,"15min":0.81999999999999995},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":3.5670000000000002,"5min":19.940000000000001,"15min":19.940000000000001},"last":1.603},{"interface":"front","average":{"1min":0.76000000000000001,"5min":1.968,"15min":0.88800000000000001},"min":{"1min":0.23599999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":3.2650000000000001,"5min":20.757000000000001,"15min":20.757000000000001},"last":0.749}]},{"osd":3,"last update":"Wed Sep 11 08:46:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":2.2360000000000002,"15min":1.0069999999999999},"min":{"1min":0.23100000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":3.927,"5min":11.951000000000001,"15min":11.951000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.512,"5min":2.0009999999999999,"15min":0.92400000000000004},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.746,"5min":7.5099999999999998,"15min":7.5099999999999998},"last":0.45300000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:45:29 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":1.3069999999999999,"15min":0.78800000000000003},"min":{"1min":0.374,"5min":0.25,"15min":0.25},"max":{"1min":3.3439999999999999,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":1.272,"15min":0.77400000000000002},"min":{"1min":0.38,"5min":0.255,"15min":0.255},"max":{"1min":2.52,"5min":6.9539999999999997,"15min":6.9539999999999997},"last":2.0760000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.474,"15min":0.84299999999999997},"min":{"1min":0.36099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.4809999999999999,"5min":7.4009999999999998,"15min":7.4009999999999998},"last":1.881},{"interface":"front","average":{"1min":0.77400000000000002,"5min":1.407,"15min":0.83599999999999997},"min":{"1min":0.38700000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.2930000000000001,"5min":7.423,"15min":7.423},"last":0.66200000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:46:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.615,"15min":0.90000000000000002},"min":{"1min":0.372,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.609,"5min":7.4240000000000004,"15min":7.4240000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.6220000000000001,"15min":0.92300000000000004},"min":{"1min":0.41499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":3.6840000000000002,"5min":7.4489999999999998,"15min":7.4489999999999998},"last":1.897}]},{"osd":7,"last update":"Wed Sep 11 08:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":1.4259999999999999,"15min":0.85899999999999999},"min":{"1min":0.40699999999999997,"5min":0.313,"15min":0.313},"max":{"1min":3.2650000000000001,"5min":7.0049999999999999,"15min":7.0049999999999999},"last":1.913},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.4530000000000001,"15min":0.85699999999999998},"min":{"1min":0.44,"5min":0.307,"15min":0.307},"max":{"1min":1.127,"5min":7.4450000000000003,"15min":7.4450000000000003},"last":0.5}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2195193856,"data_stored":2195193856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2206793728,"data_stored":2206793728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2145452032,"data_stored":2145452032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1898119168,"data_stored":1898119168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1667366912,"data_stored":1667366912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2698248192,"data_stored":2698248192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1212088320,"data_stored":1212088320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1659240448,"data_stored":1659240448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:46:25.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:46:25.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:25.830 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:46:25.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:25.842 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:25.842 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:25.869 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:46:25.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:25.973 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:25 smithi092 ceph-mon[41114]: osdmap e176: 8 total, 8 up, 8 in 2024-09-11T08:46:25.974 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:25 smithi092 ceph-mon[41114]: pgmap v325: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 705 B/s rd, 45 MiB/s wr, 718 op/s; 6781/246504 objects misplaced (2.751%); 2.4 MiB/s, 38 objects/s recovering 2024-09-11T08:46:25.974 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:25 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T08:46:25.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:25 smithi092 ceph-mon[29647]: osdmap e176: 8 total, 8 up, 8 in 2024-09-11T08:46:25.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:25 smithi092 ceph-mon[29647]: pgmap v325: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 705 B/s rd, 45 MiB/s wr, 718 op/s; 6781/246504 objects misplaced (2.751%); 2.4 MiB/s, 38 objects/s recovering 2024-09-11T08:46:25.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:25 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T08:46:25.986 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:25.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:25.987 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:26.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:26.131 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:26.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:26.132 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:26.169 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 56 16 86541 86525 96.5471 55.9375 0.0122105 0.0103508 2024-09-11T08:46:26.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:26.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:26.279 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:26.279 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:26.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:26.395 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:25 smithi145 ceph-mon[33384]: osdmap e176: 8 total, 8 up, 8 in 2024-09-11T08:46:26.396 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:25 smithi145 ceph-mon[33384]: pgmap v325: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 705 B/s rd, 45 MiB/s wr, 718 op/s; 6781/246504 objects misplaced (2.751%); 2.4 MiB/s, 38 objects/s recovering 2024-09-11T08:46:26.396 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:25 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T08:46:26.417 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:26.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:26.418 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:26.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:26.557 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:26.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:26.558 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:26.659 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:26.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:46:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:46:26.706 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:26.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:26.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:26.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:26.847 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:26.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:26.847 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:26.948 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:26.980 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:26.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:26.981 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:27.082 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:27.119 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:27.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:27.120 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:27.169 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 57 16 87561 87545 95.9717 63.75 0.00810912 0.0104141 2024-09-11T08:46:27.221 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:27.256 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:27.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:27.257 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:27.266 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:26 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:27.266 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:26 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T08:46:27.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:26 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:27.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:26 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T08:46:27.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:26 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:27.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:26 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T08:46:27.358 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:27.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:27.398 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:27.398 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:27.417 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:46:27.417 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T08:46:27.417 INFO:teuthology.orchestra.run.smithi145.stdout: "num_ops": 0 2024-09-11T08:46:27.417 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:46:27.499 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:27.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:27.527 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:27.528 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:27.628 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:27.660 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:27.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:27.661 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:27.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:27.798 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:27.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:27.799 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:27.900 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:27.932 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:27.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:27.933 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:27.975 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:46:28.034 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:28.062 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:28.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:28.063 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:28.131 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:27 smithi145 ceph-mon[33384]: pgmap v326: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.19k op/s; 6508/260754 objects misplaced (2.496%); 4.1 MiB/s, 65 objects/s recovering 2024-09-11T08:46:28.132 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:27 smithi145 ceph-mon[33384]: 6.a scrub starts 2024-09-11T08:46:28.164 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:28.169 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 58 16 88628 88612 95.4663 66.6875 0.0177455 0.0104705 2024-09-11T08:46:28.170 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-11T08:46:28.194 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:28.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:28.195 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:28.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:27 smithi092 ceph-mon[29647]: pgmap v326: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.19k op/s; 6508/260754 objects misplaced (2.496%); 4.1 MiB/s, 65 objects/s recovering 2024-09-11T08:46:28.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:27 smithi092 ceph-mon[29647]: 6.a scrub starts 2024-09-11T08:46:28.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:27 smithi092 ceph-mon[41114]: pgmap v326: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.19k op/s; 6508/260754 objects misplaced (2.496%); 4.1 MiB/s, 65 objects/s recovering 2024-09-11T08:46:28.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:27 smithi092 ceph-mon[41114]: 6.a scrub starts 2024-09-11T08:46:28.296 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:28.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:28.327 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:28.328 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:28.428 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:28.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:28.475 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:28.476 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:28.570 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:46:28.577 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:28.610 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:28.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:28.611 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:28.712 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:28.754 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:28.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:28.756 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:28.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:28.891 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:28.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:28.892 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:28.993 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:29.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:29.032 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:29.033 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:29.133 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:29.170 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 59 15 89641 89626 94.9224 63.375 0.00949259 0.0105297 2024-09-11T08:46:29.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:29.178 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:29.178 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:29.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:29.319 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:29.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:29.320 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:29.421 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:29.458 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:29.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:29.459 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:29.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:29.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:29.629 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:29.629 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:29.730 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:29.759 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:29.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:29.761 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:29.861 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:29.891 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:46:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:46:29.897 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:29.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:29.898 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:29.925 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:46:29.925 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:46:29.958 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:46:29.959 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:46:29.989 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:46:29.989 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:46:29.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:30.025 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:46:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:46:30.039 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:30.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:30.040 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:30.054 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:46:30.054 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:46:30.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:29 smithi092 ceph-mon[29647]: osdmap e177: 8 total, 8 up, 8 in 2024-09-11T08:46:30.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:29 smithi092 ceph-mon[29647]: pgmap v328: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 65 MiB/s wr, 1.03k op/s; 6508/260754 objects misplaced (2.496%); 4.3 MiB/s, 68 objects/s recovering 2024-09-11T08:46:30.055 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:29 smithi092 ceph-mon[41114]: osdmap e177: 8 total, 8 up, 8 in 2024-09-11T08:46:30.055 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:29 smithi092 ceph-mon[41114]: pgmap v328: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 65 MiB/s wr, 1.03k op/s; 6508/260754 objects misplaced (2.496%); 4.3 MiB/s, 68 objects/s recovering 2024-09-11T08:46:30.075 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:29 smithi145 ceph-mon[33384]: osdmap e177: 8 total, 8 up, 8 in 2024-09-11T08:46:30.076 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:29 smithi145 ceph-mon[33384]: pgmap v328: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 65 MiB/s wr, 1.03k op/s; 6508/260754 objects misplaced (2.496%); 4.3 MiB/s, 68 objects/s recovering 2024-09-11T08:46:30.088 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:46:30.088 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:46:30.116 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:46:30.117 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:46:30.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:30.170 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:46:30.171774+0000 min lat: 0.00108577 max lat: 2.31233 avg lat: 0.0105939 2024-09-11T08:46:30.170 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:46:30.170 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 60 16 90608 90592 94.346 60.375 0.0630615 0.0105939 2024-09-11T08:46:30.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:30.181 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:30.181 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:30.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:30.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:30.379 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:30.379 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:30.480 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:30.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:30.530 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:30.530 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:30.581 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:46:30.581 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:46:30.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:30.673 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:30.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:30.675 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:30.775 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:30.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:30.810 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:30.811 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:30.911 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:30.941 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:30.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:30.942 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:31.043 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:31.071 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:31.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:31.072 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:31.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:30 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2767281153' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:46:31.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:30 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:31.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:31.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:30 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:46:31.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:30 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:46:31.170 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 61 16 91591 91575 93.8065 61.4375 0.0111828 0.0106544 2024-09-11T08:46:31.173 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:31.220 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:31.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:31.222 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:31.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:30 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2767281153' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:46:31.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:30 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:31.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:31.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:30 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:46:31.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:30 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:46:31.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:30 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2767281153' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:46:31.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:30 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:31.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:31.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:30 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:46:31.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:30 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:46:31.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:31.364 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:31.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:31.366 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:31.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:31.500 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:31.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:31.501 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:31.546 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-11T08:46:31.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:31.641 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:31.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:31.642 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:31.743 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:31.776 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:31.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:31.777 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:31.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:46:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:46:31] "GET /metrics HTTP/1.1" 200 224275 "" "Prometheus/2.43.0" 2024-09-11T08:46:31.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:31.883 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:46:31.901 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:31.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:31.902 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:32.003 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:32.030 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:32.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:32.031 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:32.088 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:46:32.091 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:31 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T08:46:32.091 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:31 smithi092 ceph-mon[29647]: 3.5 scrub starts 2024-09-11T08:46:32.091 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:31 smithi092 ceph-mon[29647]: osdmap e178: 8 total, 8 up, 8 in 2024-09-11T08:46:32.091 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:31 smithi092 ceph-mon[29647]: pgmap v330: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.4 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.6 KiB/s rd, 63 MiB/s wr, 1.02k op/s; 6508/265410 objects misplaced (2.452%); 2.7 MiB/s, 43 objects/s recovering 2024-09-11T08:46:32.091 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:31 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2130120083' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:46:32.091 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:31 smithi092 ceph-mon[29647]: 6.19 scrub starts 2024-09-11T08:46:32.092 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:31 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T08:46:32.092 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:31 smithi092 ceph-mon[41114]: 3.5 scrub starts 2024-09-11T08:46:32.092 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:31 smithi092 ceph-mon[41114]: osdmap e178: 8 total, 8 up, 8 in 2024-09-11T08:46:32.092 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:31 smithi092 ceph-mon[41114]: pgmap v330: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.4 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.6 KiB/s rd, 63 MiB/s wr, 1.02k op/s; 6508/265410 objects misplaced (2.452%); 2.7 MiB/s, 43 objects/s recovering 2024-09-11T08:46:32.092 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:31 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2130120083' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:46:32.092 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:31 smithi092 ceph-mon[41114]: 6.19 scrub starts 2024-09-11T08:46:32.132 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:32.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:31 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T08:46:32.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:31 smithi145 ceph-mon[33384]: 3.5 scrub starts 2024-09-11T08:46:32.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:31 smithi145 ceph-mon[33384]: osdmap e178: 8 total, 8 up, 8 in 2024-09-11T08:46:32.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:31 smithi145 ceph-mon[33384]: pgmap v330: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.4 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.6 KiB/s rd, 63 MiB/s wr, 1.02k op/s; 6508/265410 objects misplaced (2.452%); 2.7 MiB/s, 43 objects/s recovering 2024-09-11T08:46:32.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:31 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2130120083' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:46:32.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:31 smithi145 ceph-mon[33384]: 6.19 scrub starts 2024-09-11T08:46:32.166 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:32.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:32.167 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:32.170 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 62 16 92484 92468 93.1937 55.8125 0.011622 0.010725 2024-09-11T08:46:32.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:32.306 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:32.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:32.307 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:32.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:32.445 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:32.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:32.446 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:32.547 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:32.580 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:32.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:32.581 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:32.682 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:32.714 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:32.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:32.715 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:32.815 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:32.845 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:32.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:32.846 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:32.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:32.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:32.980 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:33.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:33.114 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:33.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:33.115 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:33.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:46:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:46:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:46:33.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:32 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:46:33.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:32 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2130120083' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:46:33.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:32 smithi145 ceph-mon[33384]: osdmap e179: 8 total, 8 up, 8 in 2024-09-11T08:46:33.170 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 63 16 93408 93392 92.6311 57.75 0.0173326 0.0107906 2024-09-11T08:46:33.216 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:33.249 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:33.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:33.250 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:33.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:32 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:46:33.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:32 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2130120083' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:46:33.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:32 smithi092 ceph-mon[29647]: osdmap e179: 8 total, 8 up, 8 in 2024-09-11T08:46:33.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:32 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:46:33.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:32 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2130120083' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:46:33.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:32 smithi092 ceph-mon[41114]: osdmap e179: 8 total, 8 up, 8 in 2024-09-11T08:46:33.351 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:33.380 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:33.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:33.381 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:33.482 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:33.506 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:33.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:33.508 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:33.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:33.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:33.647 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:33.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:33.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:33.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:33.784 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:33.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:33.920 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:33.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:33.921 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:34.022 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:34.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:34.055 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:34.056 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:34.108 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:33 smithi145 ceph-mon[33384]: pgmap v332: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 52 MiB/s wr, 835 op/s; 6211/275784 objects misplaced (2.252%); 3.1 MiB/s, 49 objects/s recovering 2024-09-11T08:46:34.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:34.170 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 64 16 94431 94415 92.1827 63.9375 0.0264832 0.0108426 2024-09-11T08:46:34.198 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:34.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:34.199 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:34.199 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:33 smithi092 ceph-mon[29647]: pgmap v332: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 52 MiB/s wr, 835 op/s; 6211/275784 objects misplaced (2.252%); 3.1 MiB/s, 49 objects/s recovering 2024-09-11T08:46:34.200 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:33 smithi092 ceph-mon[41114]: pgmap v332: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 52 MiB/s wr, 835 op/s; 6211/275784 objects misplaced (2.252%); 3.1 MiB/s, 49 objects/s recovering 2024-09-11T08:46:34.238 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:46:34.238 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:46:34.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:34.335 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:34.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:34.336 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:34.437 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:34.478 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:34.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:34.479 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:34.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:34.616 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:34.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:34.617 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:34.718 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:34.752 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:34.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:34.753 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:34.854 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:34.888 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:34.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:34.889 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:34.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-11T08:46:34.991 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:35.018 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:35.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:35.020 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:35.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:35.162 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:35.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:35.163 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:35.170 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 65 16 95495 95479 91.7876 66.5 0.00477306 0.0108876 2024-09-11T08:46:35.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:35.344 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:35.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:35.346 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:35.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:35.482 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:35.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:35.484 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:35.584 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:35.617 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:35.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:35.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:35.719 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:35.744 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:35.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:35.745 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:35.846 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:35.873 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:35.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:35.874 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:35.975 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:36.007 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:36.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:36.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:36.109 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:36.137 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:36.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:36.138 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:36.171 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 66 16 96555 96539 91.4006 66.25 0.00872865 0.0109362 2024-09-11T08:46:36.239 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:36.265 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:36.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:36.266 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:36.289 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:46:36.367 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:36.409 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:36.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:36.410 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:36.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:36 smithi092 ceph-mon[29647]: pgmap v333: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 805 B/s rd, 49 MiB/s wr, 789 op/s; 6211/275784 objects misplaced (2.252%); 2.9 MiB/s, 46 objects/s recovering 2024-09-11T08:46:36.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:36 smithi092 ceph-mon[41114]: pgmap v333: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 805 B/s rd, 49 MiB/s wr, 789 op/s; 6211/275784 objects misplaced (2.252%); 2.9 MiB/s, 46 objects/s recovering 2024-09-11T08:46:36.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:36.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:36.549 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:36.650 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:36.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:36 smithi145 ceph-mon[33384]: pgmap v333: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 805 B/s rd, 49 MiB/s wr, 789 op/s; 6211/275784 objects misplaced (2.252%); 2.9 MiB/s, 46 objects/s recovering 2024-09-11T08:46:36.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:46:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:46:36.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:36.692 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:36.692 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:36.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:36.841 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:36.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:36.842 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:36.943 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:36.973 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:36.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:36.974 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:37.075 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:37.104 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:37.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:37.105 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:37.171 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 67 16 97513 97497 90.9301 59.875 0.0289778 0.0109922 2024-09-11T08:46:37.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:37.246 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:37.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:37.247 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:37.262 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:46:37.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:37.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:37.402 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:37.402 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:37.413 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:37 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:46:37.413 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:37 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:46:37.413 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:37 smithi092 ceph-mon[29647]: osdmap e180: 8 total, 8 up, 8 in 2024-09-11T08:46:37.413 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:37 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:37.414 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:37 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:46:37.414 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:37 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:46:37.414 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:37 smithi092 ceph-mon[41114]: osdmap e180: 8 total, 8 up, 8 in 2024-09-11T08:46:37.414 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:37 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:37.503 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:37.555 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:37.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:37.557 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:37.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:37 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:46:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:37 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:46:37.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:37 smithi145 ceph-mon[33384]: osdmap e180: 8 total, 8 up, 8 in 2024-09-11T08:46:37.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:37 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:37.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:37.699 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:37.699 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:37.733 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:46:37.801 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:37.830 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:37.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:37.831 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:37.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:37.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:37.975 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:38.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:38.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:38.135 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:38.135 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:38.171 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 68 16 98411 98395 90.418 56.125 0.0123105 0.0110547 2024-09-11T08:46:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:46:38.186 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:46:38.235 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:38.264 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:38.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:38.265 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:38.366 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:38.394 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:38.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:38.395 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:38.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:38.531 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:38 smithi092 ceph-mon[41114]: pgmap v335: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.6 KiB/s rd, 100 MiB/s wr, 1.59k op/s; 5905/290754 objects misplaced (2.031%); 6.2 MiB/s, 99 objects/s recovering 2024-09-11T08:46:38.531 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:38 smithi092 ceph-mon[41114]: from='client.24955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:46:38.531 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:38 smithi092 ceph-mon[29647]: pgmap v335: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.6 KiB/s rd, 100 MiB/s wr, 1.59k op/s; 5905/290754 objects misplaced (2.031%); 6.2 MiB/s, 99 objects/s recovering 2024-09-11T08:46:38.531 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:38 smithi092 ceph-mon[29647]: from='client.24955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:46:38.532 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:38.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:38.533 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:38.634 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:38.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:38 smithi145 ceph-mon[33384]: pgmap v335: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.6 KiB/s rd, 100 MiB/s wr, 1.59k op/s; 5905/290754 objects misplaced (2.031%); 6.2 MiB/s, 99 objects/s recovering 2024-09-11T08:46:38.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:38 smithi145 ceph-mon[33384]: from='client.24955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:46:38.664 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:38.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:38.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:38.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:38.813 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:38.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:38.814 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:38.915 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:38.944 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:38.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:38.945 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:39.043 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-11T08:46:37.163519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6339369180,"num_objects":96918,"num_object_clones":0,"num_object_copies":290754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5905,"num_objects_unfound":0,"num_objects_dirty":96918,"num_whiteouts":0,"num_read":1320,"num_read_kb":1177,"num_write":97265,"num_write_kb":6191879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1968,"num_bytes_recovered":122751120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85038,"ondisk_log_size":85038,"up":444,"acting":443,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":446,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":21131508,"kb_used_data":18710728,"kb_used_omap":13,"kb_used_meta":2420338,"kb_avail":728829708,"statfs":{"total":767960285184,"available":746321620992,"internally_reserved":0,"allocated":19159785472,"data_stored":19156591385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2478426816},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":635371520,"num_objects":9695,"num_object_clones":0,"num_object_copies":29085,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-603,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9695,"num_write_kb":620480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":603,"num_bytes_recovered":39518208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.081453"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":173,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.093103+0000","last_change":"2024-09-11T08:45:06.574593+0000","last_active":"2024-09-11T08:46:32.093103+0000","last_peered":"2024-09-11T08:46:32.093103+0000","last_clean":"2024-09-11T08:46:32.093103+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:46:32.093103+0000","last_undegraded":"2024-09-11T08:46:32.093103+0000","last_fullsized":"2024-09-11T08:46:32.093103+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:06.574539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.574539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028477630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":220,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.095054+0000","last_change":"2024-09-11T08:46:01.991434+0000","last_active":"2024-09-11T08:46:31.095054+0000","last_peered":"2024-09-11T08:46:31.095054+0000","last_clean":"2024-09-11T08:46:31.095054+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:46:31.095054+0000","last_undegraded":"2024-09-11T08:46:31.095054+0000","last_fullsized":"2024-09-11T08:46:31.095054+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:45:13.113315+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.113315+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018068470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":169,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.106867+0000","last_change":"2024-09-11T08:44:25.796953+0000","last_active":"2024-09-11T08:46:32.106867+0000","last_peered":"2024-09-11T08:46:32.106867+0000","last_clean":"2024-09-11T08:46:32.106867+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:46:32.106867+0000","last_undegraded":"2024-09-11T08:46:32.106867+0000","last_fullsized":"2024-09-11T08:46:32.106867+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:25.796699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:25.796699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025405829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":146,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.095777+0000","last_change":"2024-09-11T08:44:26.005322+0000","last_active":"2024-09-11T08:46:31.095777+0000","last_peered":"2024-09-11T08:46:31.095777+0000","last_clean":"2024-09-11T08:46:31.095777+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:46:31.095777+0000","last_undegraded":"2024-09-11T08:46:31.095777+0000","last_fullsized":"2024-09-11T08:46:31.095777+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.005270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.005270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038675070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":176,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.106834+0000","last_change":"2024-09-11T08:46:01.984446+0000","last_active":"2024-09-11T08:46:32.106834+0000","last_peered":"2024-09-11T08:46:32.106834+0000","last_clean":"2024-09-11T08:46:32.106834+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:46:32.106834+0000","last_undegraded":"2024-09-11T08:46:32.106834+0000","last_fullsized":"2024-09-11T08:46:32.106834+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:28.356172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.356172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029955870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":182,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.108932+0000","last_change":"2024-09-11T08:46:01.981971+0000","last_active":"2024-09-11T08:46:32.108932+0000","last_peered":"2024-09-11T08:46:32.108932+0000","last_clean":"2024-09-11T08:46:32.108932+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:46:32.108932+0000","last_undegraded":"2024-09-11T08:46:32.108932+0000","last_fullsized":"2024-09-11T08:46:32.108932+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.330642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.330642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0087830849999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":192,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.094670+0000","last_change":"2024-09-11T08:44:24.344328+0000","last_active":"2024-09-11T08:46:32.094670+0000","last_peered":"2024-09-11T08:46:32.094670+0000","last_clean":"2024-09-11T08:46:32.094670+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:46:32.094670+0000","last_undegraded":"2024-09-11T08:46:32.094670+0000","last_fullsized":"2024-09-11T08:46:32.094670+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:44:24.344127+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.344127+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010996336000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":148,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.095047+0000","last_change":"2024-09-11T08:43:28.988751+0000","last_active":"2024-09-11T08:46:31.095047+0000","last_peered":"2024-09-11T08:46:31.095047+0000","last_clean":"2024-09-11T08:46:31.095047+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:46:31.095047+0000","last_undegraded":"2024-09-11T08:46:31.095047+0000","last_fullsized":"2024-09-11T08:46:31.095047+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:43:28.988697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00375019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":185,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.080250+0000","last_change":"2024-09-11T08:46:17.669115+0000","last_active":"2024-09-11T08:46:31.080250+0000","last_peered":"2024-09-11T08:46:31.080250+0000","last_clean":"2024-09-11T08:46:31.080250+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:46:31.080250+0000","last_undegraded":"2024-09-11T08:46:31.080250+0000","last_fullsized":"2024-09-11T08:46:31.080250+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:12.878132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.878132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077623500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":180,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.095979+0000","last_change":"2024-09-11T08:44:27.963772+0000","last_active":"2024-09-11T08:46:31.095979+0000","last_peered":"2024-09-11T08:46:31.095979+0000","last_clean":"2024-09-11T08:46:31.095979+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:46:31.095979+0000","last_undegraded":"2024-09-11T08:46:31.095979+0000","last_fullsized":"2024-09-11T08:46:31.095979+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:44:27.963689+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:44:27.963689+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011544877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":161,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.080140+0000","last_change":"2024-09-11T08:46:09.691657+0000","last_active":"2024-09-11T08:46:31.080140+0000","last_peered":"2024-09-11T08:46:31.080140+0000","last_clean":"2024-09-11T08:46:31.080140+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:46:31.080140+0000","last_undegraded":"2024-09-11T08:46:31.080140+0000","last_fullsized":"2024-09-11T08:46:31.080140+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:33.644869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.644869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020832210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":167,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.091034+0000","last_change":"2024-09-11T08:45:31.457775+0000","last_active":"2024-09-11T08:46:32.091034+0000","last_peered":"2024-09-11T08:46:32.091034+0000","last_clean":"2024-09-11T08:46:32.091034+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:46:32.091034+0000","last_undegraded":"2024-09-11T08:46:32.091034+0000","last_fullsized":"2024-09-11T08:46:32.091034+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.457652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.457652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:45.911145+0000","scrub_duration":0.006759045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":165,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.079992+0000","last_change":"2024-09-11T08:46:22.625087+0000","last_active":"2024-09-11T08:46:31.079992+0000","last_peered":"2024-09-11T08:46:31.079992+0000","last_clean":"2024-09-11T08:46:31.079992+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:46:31.079992+0000","last_undegraded":"2024-09-11T08:46:31.079992+0000","last_fullsized":"2024-09-11T08:46:31.079992+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:09.871637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:09.871637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029269349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":160,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.097352+0000","last_change":"2024-09-11T08:43:26.983719+0000","last_active":"2024-09-11T08:46:31.097352+0000","last_peered":"2024-09-11T08:46:31.097352+0000","last_clean":"2024-09-11T08:46:31.097352+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:46:31.097352+0000","last_undegraded":"2024-09-11T08:46:31.097352+0000","last_fullsized":"2024-09-11T08:46:31.097352+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:43:26.983587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:26.983587+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082679390000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":167,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.086533+0000","last_change":"2024-09-11T08:45:13.320968+0000","last_active":"2024-09-11T08:46:31.086533+0000","last_peered":"2024-09-11T08:46:31.086533+0000","last_clean":"2024-09-11T08:46:31.086533+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:46:31.086533+0000","last_undegraded":"2024-09-11T08:46:31.086533+0000","last_fullsized":"2024-09-11T08:46:31.086533+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:13.320898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.320898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013000240000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":159,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.564545+0000","last_change":"2024-09-11T08:46:31.564511+0000","last_active":"2024-09-11T08:46:31.564545+0000","last_peered":"2024-09-11T08:46:31.564545+0000","last_clean":"2024-09-11T08:46:31.564545+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:46:31.564545+0000","last_undegraded":"2024-09-11T08:46:31.564545+0000","last_fullsized":"2024-09-11T08:46:31.564545+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.850684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.850684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058755379999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":176,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.097290+0000","last_change":"2024-09-11T08:46:01.978488+0000","last_active":"2024-09-11T08:46:31.097290+0000","last_peered":"2024-09-11T08:46:31.097290+0000","last_clean":"2024-09-11T08:46:31.097290+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:46:31.097290+0000","last_undegraded":"2024-09-11T08:46:31.097290+0000","last_fullsized":"2024-09-11T08:46:31.097290+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:24.197149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.197149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:48.966732+0000","scrub_duration":0.001510267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":163,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.091850+0000","last_change":"2024-09-11T08:43:33.346941+0000","last_active":"2024-09-11T08:46:32.091850+0000","last_peered":"2024-09-11T08:46:32.091850+0000","last_clean":"2024-09-11T08:46:32.091850+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:46:32.091850+0000","last_undegraded":"2024-09-11T08:46:32.091850+0000","last_fullsized":"2024-09-11T08:46:32.091850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:33.346856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.346856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053605370000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":211,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.089900+0000","last_change":"2024-09-11T08:46:30.359417+0000","last_active":"2024-09-11T08:46:31.089900+0000","last_peered":"2024-09-11T08:46:31.089900+0000","last_clean":"2024-09-11T08:46:31.089900+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:46:31.089900+0000","last_undegraded":"2024-09-11T08:46:31.089900+0000","last_fullsized":"2024-09-11T08:46:31.089900+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:44:18.678748+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:44:18.678748+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016068020999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":169,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.101855+0000","last_change":"2024-09-11T08:44:12.518229+0000","last_active":"2024-09-11T08:46:32.101855+0000","last_peered":"2024-09-11T08:46:32.101855+0000","last_clean":"2024-09-11T08:46:32.101855+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:46:32.101855+0000","last_undegraded":"2024-09-11T08:46:32.101855+0000","last_fullsized":"2024-09-11T08:46:32.101855+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:12.518151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.518151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013208541000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":185,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.108080+0000","last_change":"2024-09-11T08:46:01.998163+0000","last_active":"2024-09-11T08:46:32.108080+0000","last_peered":"2024-09-11T08:46:32.108080+0000","last_clean":"2024-09-11T08:46:32.108080+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:46:32.108080+0000","last_undegraded":"2024-09-11T08:46:32.108080+0000","last_fullsized":"2024-09-11T08:46:32.108080+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:24.223237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.223237+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:45.841836+0000","scrub_duration":0.0019917720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":162,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.096051+0000","last_change":"2024-09-11T08:45:28.194198+0000","last_active":"2024-09-11T08:46:31.096051+0000","last_peered":"2024-09-11T08:46:31.096051+0000","last_clean":"2024-09-11T08:46:31.096051+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:46:31.096051+0000","last_undegraded":"2024-09-11T08:46:31.096051+0000","last_fullsized":"2024-09-11T08:46:31.096051+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.194095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.194095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:49.129673+0000","scrub_duration":0.001559936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":183,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.106554+0000","last_change":"2024-09-11T08:45:16.162671+0000","last_active":"2024-09-11T08:46:32.106554+0000","last_peered":"2024-09-11T08:46:32.106554+0000","last_clean":"2024-09-11T08:46:32.106554+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:46:32.106554+0000","last_undegraded":"2024-09-11T08:46:32.106554+0000","last_fullsized":"2024-09-11T08:46:32.106554+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:16.162616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:16.162616+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001890833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":158,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.098505+0000","last_change":"2024-09-11T08:45:22.244097+0000","last_active":"2024-09-11T08:46:31.098505+0000","last_peered":"2024-09-11T08:46:31.098505+0000","last_clean":"2024-09-11T08:46:31.098505+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:46:31.098505+0000","last_undegraded":"2024-09-11T08:46:31.098505+0000","last_fullsized":"2024-09-11T08:46:31.098505+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.244038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.244038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:44.607127+0000","scrub_duration":0.0015252620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":166,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.086552+0000","last_change":"2024-09-11T08:45:19.335163+0000","last_active":"2024-09-11T08:46:31.086552+0000","last_peered":"2024-09-11T08:46:31.086552+0000","last_clean":"2024-09-11T08:46:31.086552+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:46:31.086552+0000","last_undegraded":"2024-09-11T08:46:31.086552+0000","last_fullsized":"2024-09-11T08:46:31.086552+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:19.335065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:19.335065+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:49.064118+0000","scrub_duration":0.0026052929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":173,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.106993+0000","last_change":"2024-09-11T08:44:33.758098+0000","last_active":"2024-09-11T08:46:32.106993+0000","last_peered":"2024-09-11T08:46:32.106993+0000","last_clean":"2024-09-11T08:46:32.106993+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:46:32.106993+0000","last_undegraded":"2024-09-11T08:46:32.106993+0000","last_fullsized":"2024-09-11T08:46:32.106993+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:44:33.758018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025150559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":176,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.095704+0000","last_change":"2024-09-11T08:44:10.704528+0000","last_active":"2024-09-11T08:46:32.095704+0000","last_peered":"2024-09-11T08:46:32.095704+0000","last_clean":"2024-09-11T08:46:32.095704+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:46:32.095704+0000","last_undegraded":"2024-09-11T08:46:32.095704+0000","last_fullsized":"2024-09-11T08:46:32.095704+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:44:10.704471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:10.704471+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020557259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":273,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.097974+0000","last_change":"2024-09-11T08:45:08.310590+0000","last_active":"2024-09-11T08:46:32.097974+0000","last_peered":"2024-09-11T08:46:32.097974+0000","last_clean":"2024-09-11T08:46:32.097974+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:46:32.097974+0000","last_undegraded":"2024-09-11T08:46:32.097974+0000","last_fullsized":"2024-09-11T08:46:32.097974+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:08.310527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.310527+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021434650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"180'5951","reported_seq":5990,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-11T08:46:36.269984+0000","last_change":"2024-09-11T08:45:15.261892+0000","last_active":"2024-09-11T08:46:36.269984+0000","last_peered":"2024-09-11T08:46:36.269984+0000","last_clean":"2024-09-11T08:46:36.269984+0000","last_became_active":"2024-09-11T08:45:15.261608+0000","last_became_peered":"2024-09-11T08:45:15.261608+0000","last_unstale":"2024-09-11T08:46:36.269984+0000","last_undegraded":"2024-09-11T08:46:36.269984+0000","last_fullsized":"2024-09-11T08:46:36.269984+0000","mapping_epoch":149,"log_start":"158'1000","ondisk_log_start":"158'1000","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4951,"log_dups_size":0,"ondisk_log_size":4951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:40.441095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":390004736,"num_objects":5951,"num_object_clones":0,"num_object_copies":17853,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5951,"num_write_kb":380864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"124'22","reported_seq":203,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.104157+0000","last_change":"2024-09-11T08:45:18.777528+0000","last_active":"2024-09-11T08:46:32.104157+0000","last_peered":"2024-09-11T08:46:32.104157+0000","last_clean":"2024-09-11T08:46:32.104157+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:46:32.104157+0000","last_undegraded":"2024-09-11T08:46:32.104157+0000","last_fullsized":"2024-09-11T08:46:32.104157+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:45:18.777466+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.777466+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:47.682019+0000","scrub_duration":0.0045440580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":162,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.096811+0000","last_change":"2024-09-11T08:45:13.214007+0000","last_active":"2024-09-11T08:46:32.096811+0000","last_peered":"2024-09-11T08:46:32.096811+0000","last_clean":"2024-09-11T08:46:32.096811+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:46:32.096811+0000","last_undegraded":"2024-09-11T08:46:32.096811+0000","last_fullsized":"2024-09-11T08:46:32.096811+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:13.213922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.213922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016325280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"154'9","reported_seq":262,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.088503+0000","last_change":"2024-09-11T08:45:26.208887+0000","last_active":"2024-09-11T08:46:32.088503+0000","last_peered":"2024-09-11T08:46:32.088503+0000","last_clean":"2024-09-11T08:46:32.088503+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:46:32.088503+0000","last_undegraded":"2024-09-11T08:46:32.088503+0000","last_fullsized":"2024-09-11T08:46:32.088503+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'9","last_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.208818+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:39.597635+0000","scrub_duration":0.0028933230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":156,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.079762+0000","last_change":"2024-09-11T08:44:24.813906+0000","last_active":"2024-09-11T08:46:31.079762+0000","last_peered":"2024-09-11T08:46:31.079762+0000","last_clean":"2024-09-11T08:46:31.079762+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:46:31.079762+0000","last_undegraded":"2024-09-11T08:46:31.079762+0000","last_fullsized":"2024-09-11T08:46:31.079762+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:24.813857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.813857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01118625,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"180'6074","reported_seq":6113,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-11T08:46:36.586700+0000","last_change":"2024-09-11T08:45:15.297710+0000","last_active":"2024-09-11T08:46:36.586700+0000","last_peered":"2024-09-11T08:46:36.586700+0000","last_clean":"2024-09-11T08:46:36.586700+0000","last_became_active":"2024-09-11T08:45:15.297378+0000","last_became_peered":"2024-09-11T08:45:15.297378+0000","last_unstale":"2024-09-11T08:46:36.586700+0000","last_undegraded":"2024-09-11T08:46:36.586700+0000","last_fullsized":"2024-09-11T08:46:36.586700+0000","mapping_epoch":149,"log_start":"157'400","ondisk_log_start":"157'400","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5674,"log_dups_size":0,"ondisk_log_size":5674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398065664,"num_objects":6074,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6074,"num_write_kb":388736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":192,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.096614+0000","last_change":"2024-09-11T08:44:33.387066+0000","last_active":"2024-09-11T08:46:32.096614+0000","last_peered":"2024-09-11T08:46:32.096614+0000","last_clean":"2024-09-11T08:46:32.096614+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:46:32.096614+0000","last_undegraded":"2024-09-11T08:46:32.096614+0000","last_fullsized":"2024-09-11T08:46:32.096614+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-11T08:44:33.386972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:33.386972+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022646007999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":162,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.108435+0000","last_change":"2024-09-11T08:44:25.327692+0000","last_active":"2024-09-11T08:46:32.108435+0000","last_peered":"2024-09-11T08:46:32.108435+0000","last_clean":"2024-09-11T08:46:32.108435+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:46:32.108435+0000","last_undegraded":"2024-09-11T08:46:32.108435+0000","last_fullsized":"2024-09-11T08:46:32.108435+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:25.327553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:25.327553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016327754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":158,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.095155+0000","last_change":"2024-09-11T08:45:17.150827+0000","last_active":"2024-09-11T08:46:31.095155+0000","last_peered":"2024-09-11T08:46:31.095155+0000","last_clean":"2024-09-11T08:46:31.095155+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:46:31.095155+0000","last_undegraded":"2024-09-11T08:46:31.095155+0000","last_fullsized":"2024-09-11T08:46:31.095155+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.150767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.150767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001327707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":156,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.095896+0000","last_change":"2024-09-11T08:46:03.144149+0000","last_active":"2024-09-11T08:46:31.095896+0000","last_peered":"2024-09-11T08:46:31.095896+0000","last_clean":"2024-09-11T08:46:31.095896+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:46:31.095896+0000","last_undegraded":"2024-09-11T08:46:31.095896+0000","last_fullsized":"2024-09-11T08:46:31.095896+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:46:03.144096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:46:03.144096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:47:11.742114+0000","scrub_duration":0.001497151,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"179'6004","reported_seq":6042,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:36.104262+0000","last_change":"2024-09-11T08:45:15.627562+0000","last_active":"2024-09-11T08:46:36.104262+0000","last_peered":"2024-09-11T08:46:36.104262+0000","last_clean":"2024-09-11T08:46:36.104262+0000","last_became_active":"2024-09-11T08:45:15.627210+0000","last_became_peered":"2024-09-11T08:45:15.627210+0000","last_unstale":"2024-09-11T08:46:36.104262+0000","last_undegraded":"2024-09-11T08:46:36.104262+0000","last_fullsized":"2024-09-11T08:46:36.104262+0000","mapping_epoch":149,"log_start":"157'600","ondisk_log_start":"157'600","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5404,"log_dups_size":0,"ondisk_log_size":5404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393478144,"num_objects":6004,"num_object_clones":0,"num_object_copies":18012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6004,"num_write_kb":384256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"86'21","reported_seq":178,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.095498+0000","last_change":"2024-09-11T08:43:30.997804+0000","last_active":"2024-09-11T08:46:31.095498+0000","last_peered":"2024-09-11T08:46:31.095498+0000","last_clean":"2024-09-11T08:46:31.095498+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:46:31.095498+0000","last_undegraded":"2024-09-11T08:46:31.095498+0000","last_fullsized":"2024-09-11T08:46:31.095498+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'21","last_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:43:30.997710+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025287299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":173,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.095988+0000","last_change":"2024-09-11T08:45:30.475446+0000","last_active":"2024-09-11T08:46:32.095988+0000","last_peered":"2024-09-11T08:46:32.095988+0000","last_clean":"2024-09-11T08:46:32.095988+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:46:32.095988+0000","last_undegraded":"2024-09-11T08:46:32.095988+0000","last_fullsized":"2024-09-11T08:46:32.095988+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:30.475365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:30.475365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:37.212788+0000","scrub_duration":0.010690405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":254,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.764393+0000","last_change":"2024-09-11T08:44:11.198993+0000","last_active":"2024-09-11T08:46:32.764393+0000","last_peered":"2024-09-11T08:46:32.764393+0000","last_clean":"2024-09-11T08:46:32.764393+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:46:32.764393+0000","last_undegraded":"2024-09-11T08:46:32.764393+0000","last_fullsized":"2024-09-11T08:46:32.764393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:44:11.198936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:11.198936+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019660649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":185,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.086754+0000","last_change":"2024-09-11T08:46:01.983654+0000","last_active":"2024-09-11T08:46:31.086754+0000","last_peered":"2024-09-11T08:46:31.086754+0000","last_clean":"2024-09-11T08:46:31.086754+0000","last_became_active":"2024-09-11T08:46:01.983335+0000","last_became_peered":"2024-09-11T08:46:01.983335+0000","last_unstale":"2024-09-11T08:46:31.086754+0000","last_undegraded":"2024-09-11T08:46:31.086754+0000","last_fullsized":"2024-09-11T08:46:31.086754+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.303194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.303194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:39.755953+0000","scrub_duration":0.001492431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"180'6011","reported_seq":6050,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-11T08:46:36.589899+0000","last_change":"2024-09-11T08:45:15.626487+0000","last_active":"2024-09-11T08:46:36.589899+0000","last_peered":"2024-09-11T08:46:36.589899+0000","last_clean":"2024-09-11T08:46:36.589899+0000","last_became_active":"2024-09-11T08:45:15.626210+0000","last_became_peered":"2024-09-11T08:45:15.626210+0000","last_unstale":"2024-09-11T08:46:36.589899+0000","last_undegraded":"2024-09-11T08:46:36.589899+0000","last_fullsized":"2024-09-11T08:46:36.589899+0000","mapping_epoch":149,"log_start":"157'300","ondisk_log_start":"157'300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5711,"log_dups_size":0,"ondisk_log_size":5711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393936896,"num_objects":6011,"num_object_clones":0,"num_object_copies":18033,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6011,"num_write_kb":384704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"100'15","reported_seq":191,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.113981+0000","last_change":"2024-09-11T08:45:12.769299+0000","last_active":"2024-09-11T08:46:32.113981+0000","last_peered":"2024-09-11T08:46:32.113981+0000","last_clean":"2024-09-11T08:46:32.113981+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:46:32.113981+0000","last_undegraded":"2024-09-11T08:46:32.113981+0000","last_fullsized":"2024-09-11T08:46:32.113981+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'15","last_scrub_stamp":"2024-09-11T08:45:12.769239+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:41:56.250393+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.769239+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026298300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":165,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.080050+0000","last_change":"2024-09-11T08:46:30.615598+0000","last_active":"2024-09-11T08:46:31.080050+0000","last_peered":"2024-09-11T08:46:31.080050+0000","last_clean":"2024-09-11T08:46:31.080050+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:46:31.080050+0000","last_undegraded":"2024-09-11T08:46:31.080050+0000","last_fullsized":"2024-09-11T08:46:31.080050+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:19.905459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:19.905459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015766237999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":262,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.767522+0000","last_change":"2024-09-11T08:44:34.630457+0000","last_active":"2024-09-11T08:46:32.767522+0000","last_peered":"2024-09-11T08:46:32.767522+0000","last_clean":"2024-09-11T08:46:32.767522+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:46:32.767522+0000","last_undegraded":"2024-09-11T08:46:32.767522+0000","last_fullsized":"2024-09-11T08:46:32.767522+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:44:34.630191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:34.630191+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089275420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":157,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.092858+0000","last_change":"2024-09-11T08:44:16.681124+0000","last_active":"2024-09-11T08:46:32.092858+0000","last_peered":"2024-09-11T08:46:32.092858+0000","last_clean":"2024-09-11T08:46:32.092858+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:46:32.092858+0000","last_undegraded":"2024-09-11T08:46:32.092858+0000","last_fullsized":"2024-09-11T08:46:32.092858+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:16.681036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:44:16.681036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047481509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"180'6039","reported_seq":6078,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-11T08:46:36.555928+0000","last_change":"2024-09-11T08:45:15.627960+0000","last_active":"2024-09-11T08:46:36.555928+0000","last_peered":"2024-09-11T08:46:36.555928+0000","last_clean":"2024-09-11T08:46:36.555928+0000","last_became_active":"2024-09-11T08:45:15.627639+0000","last_became_peered":"2024-09-11T08:45:15.627639+0000","last_unstale":"2024-09-11T08:46:36.555928+0000","last_undegraded":"2024-09-11T08:46:36.555928+0000","last_fullsized":"2024-09-11T08:46:36.555928+0000","mapping_epoch":149,"log_start":"159'1200","ondisk_log_start":"159'1200","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":395771904,"num_objects":6039,"num_object_clones":0,"num_object_copies":18117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6039,"num_write_kb":386496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":197,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.091655+0000","last_change":"2024-09-11T08:45:11.576184+0000","last_active":"2024-09-11T08:46:32.091655+0000","last_peered":"2024-09-11T08:46:32.091655+0000","last_clean":"2024-09-11T08:46:32.091655+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:46:32.091655+0000","last_undegraded":"2024-09-11T08:46:32.091655+0000","last_fullsized":"2024-09-11T08:46:32.091655+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:45:11.576085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.576085+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002366754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":173,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.102511+0000","last_change":"2024-09-11T08:44:30.802099+0000","last_active":"2024-09-11T08:46:32.102511+0000","last_peered":"2024-09-11T08:46:32.102511+0000","last_clean":"2024-09-11T08:46:32.102511+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:46:32.102511+0000","last_undegraded":"2024-09-11T08:46:32.102511+0000","last_fullsized":"2024-09-11T08:46:32.102511+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:30.800097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:30.800097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071432789999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":167,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.087073+0000","last_change":"2024-09-11T08:45:07.264895+0000","last_active":"2024-09-11T08:46:31.087073+0000","last_peered":"2024-09-11T08:46:31.087073+0000","last_clean":"2024-09-11T08:46:31.087073+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:46:31.087073+0000","last_undegraded":"2024-09-11T08:46:31.087073+0000","last_fullsized":"2024-09-11T08:46:31.087073+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.264822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.264822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014425250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":187,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.107751+0000","last_change":"2024-09-11T08:46:01.985315+0000","last_active":"2024-09-11T08:46:32.107751+0000","last_peered":"2024-09-11T08:46:32.107751+0000","last_clean":"2024-09-11T08:46:32.107751+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:46:32.107751+0000","last_undegraded":"2024-09-11T08:46:32.107751+0000","last_fullsized":"2024-09-11T08:46:32.107751+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.805220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.805220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012010009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"179'6027","reported_seq":6065,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:36.113560+0000","last_change":"2024-09-11T08:45:15.298396+0000","last_active":"2024-09-11T08:46:36.113560+0000","last_peered":"2024-09-11T08:46:36.113560+0000","last_clean":"2024-09-11T08:46:36.113560+0000","last_became_active":"2024-09-11T08:45:15.298053+0000","last_became_peered":"2024-09-11T08:45:15.298053+0000","last_unstale":"2024-09-11T08:46:36.113560+0000","last_undegraded":"2024-09-11T08:46:36.113560+0000","last_fullsized":"2024-09-11T08:46:36.113560+0000","mapping_epoch":149,"log_start":"157'600","ondisk_log_start":"157'600","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5427,"log_dups_size":0,"ondisk_log_size":5427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:36.450006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":394985472,"num_objects":6027,"num_object_clones":0,"num_object_copies":18081,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6027,"num_write_kb":385728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":334,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.764731+0000","last_change":"2024-09-11T08:45:26.666261+0000","last_active":"2024-09-11T08:46:32.764731+0000","last_peered":"2024-09-11T08:46:32.764731+0000","last_clean":"2024-09-11T08:46:32.764731+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:46:32.764731+0000","last_undegraded":"2024-09-11T08:46:32.764731+0000","last_fullsized":"2024-09-11T08:46:32.764731+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:45:26.666199+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.666199+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:53.748755+0000","scrub_duration":0.0020004279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":162,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.097609+0000","last_change":"2024-09-11T08:45:15.130637+0000","last_active":"2024-09-11T08:46:31.097609+0000","last_peered":"2024-09-11T08:46:31.097609+0000","last_clean":"2024-09-11T08:46:31.097609+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:46:31.097609+0000","last_undegraded":"2024-09-11T08:46:31.097609+0000","last_fullsized":"2024-09-11T08:46:31.097609+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.130572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.130572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019779749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":200,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.080614+0000","last_change":"2024-09-11T08:46:08.737156+0000","last_active":"2024-09-11T08:46:31.080614+0000","last_peered":"2024-09-11T08:46:31.080614+0000","last_clean":"2024-09-11T08:46:31.080614+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:46:31.080614+0000","last_undegraded":"2024-09-11T08:46:31.080614+0000","last_fullsized":"2024-09-11T08:46:31.080614+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:43:38.669837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:38.669837+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001723715,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":159,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.101194+0000","last_change":"2024-09-11T08:44:14.679790+0000","last_active":"2024-09-11T08:46:32.101194+0000","last_peered":"2024-09-11T08:46:32.101194+0000","last_clean":"2024-09-11T08:46:32.101194+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:46:32.101194+0000","last_undegraded":"2024-09-11T08:46:32.101194+0000","last_fullsized":"2024-09-11T08:46:32.101194+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:14.679598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:14.679598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011950855999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":156,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.107544+0000","last_change":"2024-09-11T08:45:22.169436+0000","last_active":"2024-09-11T08:46:32.107544+0000","last_peered":"2024-09-11T08:46:32.107544+0000","last_clean":"2024-09-11T08:46:32.107544+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:46:32.107544+0000","last_undegraded":"2024-09-11T08:46:32.107544+0000","last_fullsized":"2024-09-11T08:46:32.107544+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.169367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.169367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:43.728179+0000","scrub_duration":0.005349693,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"180'6039","reported_seq":6078,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-11T08:46:36.541257+0000","last_change":"2024-09-11T08:45:15.261656+0000","last_active":"2024-09-11T08:46:36.541257+0000","last_peered":"2024-09-11T08:46:36.541257+0000","last_clean":"2024-09-11T08:46:36.541257+0000","last_became_active":"2024-09-11T08:45:15.261370+0000","last_became_peered":"2024-09-11T08:45:15.261370+0000","last_unstale":"2024-09-11T08:46:36.541257+0000","last_undegraded":"2024-09-11T08:46:36.541257+0000","last_fullsized":"2024-09-11T08:46:36.541257+0000","mapping_epoch":149,"log_start":"159'1200","ondisk_log_start":"159'1200","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":395771904,"num_objects":6039,"num_object_clones":0,"num_object_copies":18117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6039,"num_write_kb":386496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":208,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.104233+0000","last_change":"2024-09-11T08:45:27.666590+0000","last_active":"2024-09-11T08:46:32.104233+0000","last_peered":"2024-09-11T08:46:32.104233+0000","last_clean":"2024-09-11T08:46:32.104233+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:46:32.104233+0000","last_undegraded":"2024-09-11T08:46:32.104233+0000","last_fullsized":"2024-09-11T08:46:32.104233+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:45:27.666502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.666502+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:44.189565+0000","scrub_duration":0.0023582730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":193,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.102112+0000","last_change":"2024-09-11T08:46:01.982031+0000","last_active":"2024-09-11T08:46:32.102112+0000","last_peered":"2024-09-11T08:46:32.102112+0000","last_clean":"2024-09-11T08:46:32.102112+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:46:32.102112+0000","last_undegraded":"2024-09-11T08:46:32.102112+0000","last_fullsized":"2024-09-11T08:46:32.102112+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.775804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.775804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:39.033860+0000","scrub_duration":0.0011899289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":188,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.089727+0000","last_change":"2024-09-11T08:45:32.632342+0000","last_active":"2024-09-11T08:46:31.089727+0000","last_peered":"2024-09-11T08:46:31.089727+0000","last_clean":"2024-09-11T08:46:31.089727+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:46:31.089727+0000","last_undegraded":"2024-09-11T08:46:31.089727+0000","last_fullsized":"2024-09-11T08:46:31.089727+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:45:32.632242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:32.632242+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:47:00.544909+0000","scrub_duration":0.0049221250000000003,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":174,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.090365+0000","last_change":"2024-09-11T08:46:23.414859+0000","last_active":"2024-09-11T08:46:31.090365+0000","last_peered":"2024-09-11T08:46:31.090365+0000","last_clean":"2024-09-11T08:46:31.090365+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:46:31.090365+0000","last_undegraded":"2024-09-11T08:46:31.090365+0000","last_fullsized":"2024-09-11T08:46:31.090365+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:09.638694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_clean_scrub_stamp":"2024-09-11T08:45:09.638694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00151615,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":185,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.087673+0000","last_change":"2024-09-11T08:46:01.984350+0000","last_active":"2024-09-11T08:46:31.087673+0000","last_peered":"2024-09-11T08:46:31.087673+0000","last_clean":"2024-09-11T08:46:31.087673+0000","last_became_active":"2024-09-11T08:46:01.983095+0000","last_became_peered":"2024-09-11T08:46:01.983095+0000","last_unstale":"2024-09-11T08:46:31.087673+0000","last_undegraded":"2024-09-11T08:46:31.087673+0000","last_fullsized":"2024-09-11T08:46:31.087673+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:12.278185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.278185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:33.187257+0000","scrub_duration":0.0015184790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"180'6269","reported_seq":6308,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-11T08:46:36.324684+0000","last_change":"2024-09-11T08:45:15.627633+0000","last_active":"2024-09-11T08:46:36.324684+0000","last_peered":"2024-09-11T08:46:36.324684+0000","last_clean":"2024-09-11T08:46:36.324684+0000","last_became_active":"2024-09-11T08:45:15.627360+0000","last_became_peered":"2024-09-11T08:45:15.627360+0000","last_unstale":"2024-09-11T08:46:36.324684+0000","last_undegraded":"2024-09-11T08:46:36.324684+0000","last_fullsized":"2024-09-11T08:46:36.324684+0000","mapping_epoch":149,"log_start":"159'1300","ondisk_log_start":"159'1300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4969,"log_dups_size":0,"ondisk_log_size":4969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410845184,"num_objects":6269,"num_object_clones":0,"num_object_copies":18807,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6269,"num_write_kb":401216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":179,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.097422+0000","last_change":"2024-09-11T08:44:15.699227+0000","last_active":"2024-09-11T08:46:32.097422+0000","last_peered":"2024-09-11T08:46:32.097422+0000","last_clean":"2024-09-11T08:46:32.097422+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:46:32.097422+0000","last_undegraded":"2024-09-11T08:46:32.097422+0000","last_fullsized":"2024-09-11T08:46:32.097422+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:44:15.699067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:15.699067+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014313306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":173,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.097403+0000","last_change":"2024-09-11T08:45:20.455183+0000","last_active":"2024-09-11T08:46:32.097403+0000","last_peered":"2024-09-11T08:46:32.097403+0000","last_clean":"2024-09-11T08:46:32.097403+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:46:32.097403+0000","last_undegraded":"2024-09-11T08:46:32.097403+0000","last_fullsized":"2024-09-11T08:46:32.097403+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:20.455081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.455081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016372450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":659,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:34.727796+0000","last_change":"2024-09-11T08:45:21.370742+0000","last_active":"2024-09-11T08:46:34.727796+0000","last_peered":"2024-09-11T08:46:34.727796+0000","last_clean":"2024-09-11T08:46:34.727796+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:46:34.727796+0000","last_undegraded":"2024-09-11T08:46:34.727796+0000","last_fullsized":"2024-09-11T08:46:34.727796+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:45:21.370673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.370673+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:36.160525+0000","scrub_duration":0.0017307240000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":470,"num_read_kb":465,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":161,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.080351+0000","last_change":"2024-09-11T08:46:12.673215+0000","last_active":"2024-09-11T08:46:31.080351+0000","last_peered":"2024-09-11T08:46:31.080351+0000","last_clean":"2024-09-11T08:46:31.080351+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:46:31.080351+0000","last_undegraded":"2024-09-11T08:46:31.080351+0000","last_fullsized":"2024-09-11T08:46:31.080351+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:16.924404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:16.924404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024072690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":167,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.101441+0000","last_change":"2024-09-11T08:45:17.819757+0000","last_active":"2024-09-11T08:46:32.101441+0000","last_peered":"2024-09-11T08:46:32.101441+0000","last_clean":"2024-09-11T08:46:32.101441+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:46:32.101441+0000","last_undegraded":"2024-09-11T08:46:32.101441+0000","last_fullsized":"2024-09-11T08:46:32.101441+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.819707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.819707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012611440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"179'6106","reported_seq":6144,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:36.074728+0000","last_change":"2024-09-11T08:45:15.262738+0000","last_active":"2024-09-11T08:46:36.074728+0000","last_peered":"2024-09-11T08:46:36.074728+0000","last_clean":"2024-09-11T08:46:36.074728+0000","last_became_active":"2024-09-11T08:45:15.262440+0000","last_became_peered":"2024-09-11T08:45:15.262440+0000","last_unstale":"2024-09-11T08:46:36.074728+0000","last_undegraded":"2024-09-11T08:46:36.074728+0000","last_fullsized":"2024-09-11T08:46:36.074728+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":6106,"log_dups_size":0,"ondisk_log_size":6106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:40.707557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":400162816,"num_objects":6106,"num_object_clones":0,"num_object_copies":18318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6106,"num_write_kb":390784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":37,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.085260+0000","last_change":"2024-09-11T08:46:01.996394+0000","last_active":"2024-09-11T08:46:31.085260+0000","last_peered":"2024-09-11T08:46:31.085260+0000","last_clean":"2024-09-11T08:46:31.085260+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:46:31.085260+0000","last_undegraded":"2024-09-11T08:46:31.085260+0000","last_fullsized":"2024-09-11T08:46:31.085260+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:43:26.127412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:26.127412+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":171,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.105687+0000","last_change":"2024-09-11T08:45:18.127946+0000","last_active":"2024-09-11T08:46:32.105687+0000","last_peered":"2024-09-11T08:46:32.105687+0000","last_clean":"2024-09-11T08:46:32.105687+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:46:32.105687+0000","last_undegraded":"2024-09-11T08:46:32.105687+0000","last_fullsized":"2024-09-11T08:46:32.105687+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:18.127857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.127857+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:47.514774+0000","scrub_duration":0.0018334099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":215,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.086942+0000","last_change":"2024-09-11T08:45:29.524681+0000","last_active":"2024-09-11T08:46:31.086942+0000","last_peered":"2024-09-11T08:46:31.086942+0000","last_clean":"2024-09-11T08:46:31.086942+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:46:31.086942+0000","last_undegraded":"2024-09-11T08:46:31.086942+0000","last_fullsized":"2024-09-11T08:46:31.086942+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'79","last_scrub_stamp":"2024-09-11T08:45:29.524608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:45:29.524608+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:33.259103+0000","scrub_duration":0.002308867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":167,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.087938+0000","last_change":"2024-09-11T08:44:35.441051+0000","last_active":"2024-09-11T08:46:31.087938+0000","last_peered":"2024-09-11T08:46:31.087938+0000","last_clean":"2024-09-11T08:46:31.087938+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:46:31.087938+0000","last_undegraded":"2024-09-11T08:46:31.087938+0000","last_fullsized":"2024-09-11T08:46:31.087938+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:35.440995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:35.440995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0080921540000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":159,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.080223+0000","last_change":"2024-09-11T08:46:10.685628+0000","last_active":"2024-09-11T08:46:31.080223+0000","last_peered":"2024-09-11T08:46:31.080223+0000","last_clean":"2024-09-11T08:46:31.080223+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:46:31.080223+0000","last_undegraded":"2024-09-11T08:46:31.080223+0000","last_fullsized":"2024-09-11T08:46:31.080223+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.570246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.570246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041809610000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"179'6104","reported_seq":6142,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:36.068747+0000","last_change":"2024-09-11T08:45:15.263087+0000","last_active":"2024-09-11T08:46:36.068747+0000","last_peered":"2024-09-11T08:46:36.068747+0000","last_clean":"2024-09-11T08:46:36.068747+0000","last_became_active":"2024-09-11T08:45:15.262698+0000","last_became_peered":"2024-09-11T08:45:15.262698+0000","last_unstale":"2024-09-11T08:46:36.068747+0000","last_undegraded":"2024-09-11T08:46:36.068747+0000","last_fullsized":"2024-09-11T08:46:36.068747+0000","mapping_epoch":149,"log_start":"159'1300","ondisk_log_start":"159'1300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4804,"log_dups_size":0,"ondisk_log_size":4804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":400031744,"num_objects":6104,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6104,"num_write_kb":390656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":235,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.097363+0000","last_change":"2024-09-11T08:46:01.997857+0000","last_active":"2024-09-11T08:46:31.097363+0000","last_peered":"2024-09-11T08:46:31.097363+0000","last_clean":"2024-09-11T08:46:31.097363+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:46:31.097363+0000","last_undegraded":"2024-09-11T08:46:31.097363+0000","last_fullsized":"2024-09-11T08:46:31.097363+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:21.002140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:21.002140+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025090400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":152,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.095621+0000","last_change":"2024-09-11T08:44:22.010602+0000","last_active":"2024-09-11T08:46:31.095621+0000","last_peered":"2024-09-11T08:46:31.095621+0000","last_clean":"2024-09-11T08:46:31.095621+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:46:31.095621+0000","last_undegraded":"2024-09-11T08:46:31.095621+0000","last_fullsized":"2024-09-11T08:46:31.095621+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.010528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.010528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047628439999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":168,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.079991+0000","last_change":"2024-09-11T08:44:26.726624+0000","last_active":"2024-09-11T08:46:31.079991+0000","last_peered":"2024-09-11T08:46:31.079991+0000","last_clean":"2024-09-11T08:46:31.079991+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:46:31.079991+0000","last_undegraded":"2024-09-11T08:46:31.079991+0000","last_fullsized":"2024-09-11T08:46:31.079991+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.726573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00117395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":176,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.106760+0000","last_change":"2024-09-11T08:46:01.988315+0000","last_active":"2024-09-11T08:46:32.106760+0000","last_peered":"2024-09-11T08:46:32.106760+0000","last_clean":"2024-09-11T08:46:32.106760+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:46:32.106760+0000","last_undegraded":"2024-09-11T08:46:32.106760+0000","last_fullsized":"2024-09-11T08:46:32.106760+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.167071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.167071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:44.717884+0000","scrub_duration":0.001796483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"180'5947","reported_seq":5986,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-11T08:46:36.330941+0000","last_change":"2024-09-11T08:45:15.626015+0000","last_active":"2024-09-11T08:46:36.330941+0000","last_peered":"2024-09-11T08:46:36.330941+0000","last_clean":"2024-09-11T08:46:36.330941+0000","last_became_active":"2024-09-11T08:45:15.625726+0000","last_became_peered":"2024-09-11T08:45:15.625726+0000","last_unstale":"2024-09-11T08:46:36.330941+0000","last_undegraded":"2024-09-11T08:46:36.330941+0000","last_fullsized":"2024-09-11T08:46:36.330941+0000","mapping_epoch":149,"log_start":"158'1000","ondisk_log_start":"158'1000","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:36.587803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389742592,"num_objects":5947,"num_object_clones":0,"num_object_copies":17841,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5947,"num_write_kb":380608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":188,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.096313+0000","last_change":"2024-09-11T08:44:28.940078+0000","last_active":"2024-09-11T08:46:31.096313+0000","last_peered":"2024-09-11T08:46:31.096313+0000","last_clean":"2024-09-11T08:46:31.096313+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:46:31.096313+0000","last_undegraded":"2024-09-11T08:46:31.096313+0000","last_fullsized":"2024-09-11T08:46:31.096313+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:44:28.940023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.940023+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012856796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":181,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.087189+0000","last_change":"2024-09-11T08:45:25.273200+0000","last_active":"2024-09-11T08:46:31.087189+0000","last_peered":"2024-09-11T08:46:31.087189+0000","last_clean":"2024-09-11T08:46:31.087189+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:46:31.087189+0000","last_undegraded":"2024-09-11T08:46:31.087189+0000","last_fullsized":"2024-09-11T08:46:31.087189+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.273134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:35.226154+0000","scrub_duration":0.0014307370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":169,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.102285+0000","last_change":"2024-09-11T08:44:24.816974+0000","last_active":"2024-09-11T08:46:32.102285+0000","last_peered":"2024-09-11T08:46:32.102285+0000","last_clean":"2024-09-11T08:46:32.102285+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:46:32.102285+0000","last_undegraded":"2024-09-11T08:46:32.102285+0000","last_fullsized":"2024-09-11T08:46:32.102285+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:24.816890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.816890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086069140000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":175,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.089919+0000","last_change":"2024-09-11T08:46:26.461698+0000","last_active":"2024-09-11T08:46:31.089919+0000","last_peered":"2024-09-11T08:46:31.089919+0000","last_clean":"2024-09-11T08:46:31.089919+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:46:31.089919+0000","last_undegraded":"2024-09-11T08:46:31.089919+0000","last_fullsized":"2024-09-11T08:46:31.089919+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.756166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.756166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059059109999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"180'6104","reported_seq":6143,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-11T08:46:36.428406+0000","last_change":"2024-09-11T08:45:15.260658+0000","last_active":"2024-09-11T08:46:36.428406+0000","last_peered":"2024-09-11T08:46:36.428406+0000","last_clean":"2024-09-11T08:46:36.428406+0000","last_became_active":"2024-09-11T08:45:15.260383+0000","last_became_peered":"2024-09-11T08:45:15.260383+0000","last_unstale":"2024-09-11T08:46:36.428406+0000","last_undegraded":"2024-09-11T08:46:36.428406+0000","last_fullsized":"2024-09-11T08:46:36.428406+0000","mapping_epoch":149,"log_start":"159'1200","ondisk_log_start":"159'1200","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4904,"log_dups_size":0,"ondisk_log_size":4904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":400031744,"num_objects":6104,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6104,"num_write_kb":390656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":209,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.098550+0000","last_change":"2024-09-11T08:46:02.002847+0000","last_active":"2024-09-11T08:46:31.098550+0000","last_peered":"2024-09-11T08:46:31.098550+0000","last_clean":"2024-09-11T08:46:31.098550+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:46:31.098550+0000","last_undegraded":"2024-09-11T08:46:31.098550+0000","last_fullsized":"2024-09-11T08:46:31.098550+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:43:24.983740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:24.983740+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069750209999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":162,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.107133+0000","last_change":"2024-09-11T08:45:32.219463+0000","last_active":"2024-09-11T08:46:32.107133+0000","last_peered":"2024-09-11T08:46:32.107133+0000","last_clean":"2024-09-11T08:46:32.107133+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:46:32.107133+0000","last_undegraded":"2024-09-11T08:46:32.107133+0000","last_fullsized":"2024-09-11T08:46:32.107133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:32.219374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:32.219374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:49.435616+0000","scrub_duration":0.0069622119999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":277,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.105711+0000","last_change":"2024-09-11T08:46:25.415023+0000","last_active":"2024-09-11T08:46:32.105711+0000","last_peered":"2024-09-11T08:46:32.105711+0000","last_clean":"2024-09-11T08:46:32.105711+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:46:32.105711+0000","last_undegraded":"2024-09-11T08:46:32.105711+0000","last_fullsized":"2024-09-11T08:46:32.105711+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:10.659079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:10.659079+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017208200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":156,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.097248+0000","last_change":"2024-09-11T08:45:31.196687+0000","last_active":"2024-09-11T08:46:31.097248+0000","last_peered":"2024-09-11T08:46:31.097248+0000","last_clean":"2024-09-11T08:46:31.097248+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:46:31.097248+0000","last_undegraded":"2024-09-11T08:46:31.097248+0000","last_fullsized":"2024-09-11T08:46:31.097248+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.196599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.196599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:41.921593+0000","scrub_duration":0.0025248190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"180'6100","reported_seq":7719,"reported_epoch":180,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T08:46:36.331807+0000","last_change":"2024-09-11T08:46:03.286008+0000","last_active":"2024-09-11T08:46:36.331807+0000","last_peered":"2024-09-11T08:46:36.331807+0000","last_clean":"2024-09-11T08:46:00.945841+0000","last_became_active":"2024-09-11T08:46:03.248256+0000","last_became_peered":"2024-09-11T08:46:03.248256+0000","last_unstale":"2024-09-11T08:46:36.331807+0000","last_undegraded":"2024-09-11T08:46:36.331807+0000","last_fullsized":"2024-09-11T08:46:02.976155+0000","mapping_epoch":168,"log_start":"158'1100","ondisk_log_start":"158'1100","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5000,"log_dups_size":0,"ondisk_log_size":5000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:39.697319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":399769600,"num_objects":6100,"num_object_clones":0,"num_object_copies":18300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3144,"num_objects_unfound":0,"num_objects_dirty":6100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6100,"num_write_kb":390400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":785,"num_bytes_recovered":51445760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3],"avail_no_missing":["1","2","3"],"object_location_counts":[{"shards":"1,2,3","objects":3144},{"shards":"1,3","objects":2956}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":177,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.091590+0000","last_change":"2024-09-11T08:44:31.662275+0000","last_active":"2024-09-11T08:46:32.091590+0000","last_peered":"2024-09-11T08:46:32.091590+0000","last_clean":"2024-09-11T08:46:32.091590+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:46:32.091590+0000","last_undegraded":"2024-09-11T08:46:32.091590+0000","last_fullsized":"2024-09-11T08:46:32.091590+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:31.662190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.662190+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040714619999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":172,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.080114+0000","last_change":"2024-09-11T08:45:10.607400+0000","last_active":"2024-09-11T08:46:31.080114+0000","last_peered":"2024-09-11T08:46:31.080114+0000","last_clean":"2024-09-11T08:46:31.080114+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:46:31.080114+0000","last_undegraded":"2024-09-11T08:46:31.080114+0000","last_fullsized":"2024-09-11T08:46:31.080114+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:10.607288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:45:10.607288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016158909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":168,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.090258+0000","last_change":"2024-09-11T08:45:12.646009+0000","last_active":"2024-09-11T08:46:31.090258+0000","last_peered":"2024-09-11T08:46:31.090258+0000","last_clean":"2024-09-11T08:46:31.090258+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:46:31.090258+0000","last_undegraded":"2024-09-11T08:46:31.090258+0000","last_fullsized":"2024-09-11T08:46:31.090258+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:12.645956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.645956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014834379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":157,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.097318+0000","last_change":"2024-09-11T08:44:17.731558+0000","last_active":"2024-09-11T08:46:32.097318+0000","last_peered":"2024-09-11T08:46:32.097318+0000","last_clean":"2024-09-11T08:46:32.097318+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:46:32.097318+0000","last_undegraded":"2024-09-11T08:46:32.097318+0000","last_fullsized":"2024-09-11T08:46:32.097318+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:17.731489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:17.731489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002184517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"180'5931","reported_seq":5970,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-11T08:46:36.559236+0000","last_change":"2024-09-11T08:45:15.297737+0000","last_active":"2024-09-11T08:46:36.559236+0000","last_peered":"2024-09-11T08:46:36.559236+0000","last_clean":"2024-09-11T08:46:36.559236+0000","last_became_active":"2024-09-11T08:45:15.297420+0000","last_became_peered":"2024-09-11T08:45:15.297420+0000","last_unstale":"2024-09-11T08:46:36.559236+0000","last_undegraded":"2024-09-11T08:46:36.559236+0000","last_fullsized":"2024-09-11T08:46:36.559236+0000","mapping_epoch":149,"log_start":"157'200","ondisk_log_start":"157'200","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5731,"log_dups_size":0,"ondisk_log_size":5731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388694016,"num_objects":5931,"num_object_clones":0,"num_object_copies":17793,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5931,"num_write_kb":379584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":41,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.087102+0000","last_change":"2024-09-11T08:46:01.997537+0000","last_active":"2024-09-11T08:46:31.087102+0000","last_peered":"2024-09-11T08:46:31.087102+0000","last_clean":"2024-09-11T08:46:31.087102+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:46:31.087102+0000","last_undegraded":"2024-09-11T08:46:31.087102+0000","last_fullsized":"2024-09-11T08:46:31.087102+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:44:27.810879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:27.810879+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001826121,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":171,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.088001+0000","last_change":"2024-09-11T08:44:31.494470+0000","last_active":"2024-09-11T08:46:31.088001+0000","last_peered":"2024-09-11T08:46:31.088001+0000","last_clean":"2024-09-11T08:46:31.088001+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:46:31.088001+0000","last_undegraded":"2024-09-11T08:46:31.088001+0000","last_fullsized":"2024-09-11T08:46:31.088001+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.494397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.494397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021588290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":174,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.089313+0000","last_change":"2024-09-11T08:46:21.348979+0000","last_active":"2024-09-11T08:46:31.089313+0000","last_peered":"2024-09-11T08:46:31.089313+0000","last_clean":"2024-09-11T08:46:31.089313+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:46:31.089313+0000","last_undegraded":"2024-09-11T08:46:31.089313+0000","last_fullsized":"2024-09-11T08:46:31.089313+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:30.717364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:30.717364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044727940000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":155,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.086870+0000","last_change":"2024-09-11T08:44:23.367862+0000","last_active":"2024-09-11T08:46:31.086870+0000","last_peered":"2024-09-11T08:46:31.086870+0000","last_clean":"2024-09-11T08:46:31.086870+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:46:31.086870+0000","last_undegraded":"2024-09-11T08:46:31.086870+0000","last_fullsized":"2024-09-11T08:46:31.086870+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:23.367786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:23.367786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030117270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"180'5973","reported_seq":8178,"reported_epoch":180,"state":"active+recovering+remapped","last_fresh":"2024-09-11T08:46:36.590095+0000","last_change":"2024-09-11T08:46:03.325028+0000","last_active":"2024-09-11T08:46:36.590095+0000","last_peered":"2024-09-11T08:46:36.590095+0000","last_clean":"2024-09-11T08:46:00.945574+0000","last_became_active":"2024-09-11T08:46:03.262389+0000","last_became_peered":"2024-09-11T08:46:03.262389+0000","last_unstale":"2024-09-11T08:46:36.590095+0000","last_undegraded":"2024-09-11T08:46:36.590095+0000","last_fullsized":"2024-09-11T08:46:36.590095+0000","mapping_epoch":168,"log_start":"157'300","ondisk_log_start":"157'300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5673,"log_dups_size":0,"ondisk_log_size":5673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:41.717967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":391446528,"num_objects":5973,"num_object_clones":0,"num_object_copies":17919,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2761,"num_objects_unfound":0,"num_objects_dirty":5973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5973,"num_write_kb":382272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1079,"num_bytes_recovered":70713344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[0,4,7],"avail_no_missing":["0","4","7"],"object_location_counts":[{"shards":"0,4,7","objects":3212}],"blocked_by":[],"up_primary":6,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":183,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.095352+0000","last_change":"2024-09-11T08:45:08.140910+0000","last_active":"2024-09-11T08:46:31.095352+0000","last_peered":"2024-09-11T08:46:31.095352+0000","last_clean":"2024-09-11T08:46:31.095352+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:46:31.095352+0000","last_undegraded":"2024-09-11T08:46:31.095352+0000","last_fullsized":"2024-09-11T08:46:31.095352+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:08.140839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.140839+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003679547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":171,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.087848+0000","last_change":"2024-09-11T08:45:14.322786+0000","last_active":"2024-09-11T08:46:31.087848+0000","last_peered":"2024-09-11T08:46:31.087848+0000","last_clean":"2024-09-11T08:46:31.087848+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:46:31.087848+0000","last_undegraded":"2024-09-11T08:46:31.087848+0000","last_fullsized":"2024-09-11T08:46:31.087848+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.322217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.322217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:34.362792+0000","scrub_duration":0.0014821319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":174,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.088934+0000","last_change":"2024-09-11T08:46:20.339514+0000","last_active":"2024-09-11T08:46:31.088934+0000","last_peered":"2024-09-11T08:46:31.088934+0000","last_clean":"2024-09-11T08:46:31.088934+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:46:31.088934+0000","last_undegraded":"2024-09-11T08:46:31.088934+0000","last_fullsized":"2024-09-11T08:46:31.088934+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:29.692933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:29.692933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007324077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":156,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.106901+0000","last_change":"2024-09-11T08:45:33.228265+0000","last_active":"2024-09-11T08:46:32.106901+0000","last_peered":"2024-09-11T08:46:32.106901+0000","last_clean":"2024-09-11T08:46:32.106901+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:46:32.106901+0000","last_undegraded":"2024-09-11T08:46:32.106901+0000","last_fullsized":"2024-09-11T08:46:32.106901+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:45:33.228082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:47.790589+0000","scrub_duration":0.0064426500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"179'6043","reported_seq":6081,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:36.082914+0000","last_change":"2024-09-11T08:45:15.626107+0000","last_active":"2024-09-11T08:46:36.082914+0000","last_peered":"2024-09-11T08:46:36.082914+0000","last_clean":"2024-09-11T08:46:36.082914+0000","last_became_active":"2024-09-11T08:45:15.625778+0000","last_became_peered":"2024-09-11T08:45:15.625778+0000","last_unstale":"2024-09-11T08:46:36.082914+0000","last_undegraded":"2024-09-11T08:46:36.082914+0000","last_fullsized":"2024-09-11T08:46:36.082914+0000","mapping_epoch":149,"log_start":"157'600","ondisk_log_start":"157'600","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5443,"log_dups_size":0,"ondisk_log_size":5443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":396034048,"num_objects":6043,"num_object_clones":0,"num_object_copies":18129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6043,"num_write_kb":386752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"124'22","reported_seq":247,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.079624+0000","last_change":"2024-09-11T08:46:11.689219+0000","last_active":"2024-09-11T08:46:31.079624+0000","last_peered":"2024-09-11T08:46:31.079624+0000","last_clean":"2024-09-11T08:46:31.079624+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:46:31.079624+0000","last_undegraded":"2024-09-11T08:46:31.079624+0000","last_fullsized":"2024-09-11T08:46:31.079624+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:43:32.625376+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073394039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":188,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.087795+0000","last_change":"2024-09-11T08:45:20.372571+0000","last_active":"2024-09-11T08:46:31.087795+0000","last_peered":"2024-09-11T08:46:31.087795+0000","last_clean":"2024-09-11T08:46:31.087795+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:46:31.087795+0000","last_undegraded":"2024-09-11T08:46:31.087795+0000","last_fullsized":"2024-09-11T08:46:31.087795+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:45:20.372454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.372454+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:36.060441+0000","scrub_duration":0.0019547269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"168'10","reported_seq":298,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.083495+0000","last_change":"2024-09-11T08:46:01.988356+0000","last_active":"2024-09-11T08:46:32.083495+0000","last_peered":"2024-09-11T08:46:32.083495+0000","last_clean":"2024-09-11T08:46:32.083495+0000","last_became_active":"2024-09-11T08:46:01.983055+0000","last_became_peered":"2024-09-11T08:46:01.983055+0000","last_unstale":"2024-09-11T08:46:32.083495+0000","last_undegraded":"2024-09-11T08:46:32.083495+0000","last_fullsized":"2024-09-11T08:46:32.083495+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"135'9","last_scrub_stamp":"2024-09-11T08:45:11.298652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.298652+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001669032,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":159,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.080414+0000","last_change":"2024-09-11T08:46:27.611148+0000","last_active":"2024-09-11T08:46:31.080414+0000","last_peered":"2024-09-11T08:46:31.080414+0000","last_clean":"2024-09-11T08:46:31.080414+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:46:31.080414+0000","last_undegraded":"2024-09-11T08:46:31.080414+0000","last_fullsized":"2024-09-11T08:46:31.080414+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:15.888394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:15.888394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003201476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":181,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.084247+0000","last_change":"2024-09-11T08:45:28.247838+0000","last_active":"2024-09-11T08:46:31.084247+0000","last_peered":"2024-09-11T08:46:31.084247+0000","last_clean":"2024-09-11T08:46:31.084247+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:46:31.084247+0000","last_undegraded":"2024-09-11T08:46:31.084247+0000","last_fullsized":"2024-09-11T08:46:31.084247+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.247771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015413499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":187,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.108243+0000","last_change":"2024-09-11T08:44:35.390285+0000","last_active":"2024-09-11T08:46:32.108243+0000","last_peered":"2024-09-11T08:46:32.108243+0000","last_clean":"2024-09-11T08:46:32.108243+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:46:32.108243+0000","last_undegraded":"2024-09-11T08:46:32.108243+0000","last_fullsized":"2024-09-11T08:46:32.108243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-11T08:44:35.390134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:35.390134+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017427033000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":168,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.079964+0000","last_change":"2024-09-11T08:45:08.675251+0000","last_active":"2024-09-11T08:46:31.079964+0000","last_peered":"2024-09-11T08:46:31.079964+0000","last_clean":"2024-09-11T08:46:31.079964+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:46:31.079964+0000","last_undegraded":"2024-09-11T08:46:31.079964+0000","last_fullsized":"2024-09-11T08:46:31.079964+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:08.675179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.675179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:37.735413+0000","scrub_duration":0.003515392,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":185,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.087593+0000","last_change":"2024-09-11T08:46:01.983864+0000","last_active":"2024-09-11T08:46:31.087593+0000","last_peered":"2024-09-11T08:46:31.087593+0000","last_clean":"2024-09-11T08:46:31.087593+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:46:31.087593+0000","last_undegraded":"2024-09-11T08:46:31.087593+0000","last_fullsized":"2024-09-11T08:46:31.087593+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:25.344348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.344348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:46.305052+0000","scrub_duration":0.0013849730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":177,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.095913+0000","last_change":"2024-09-11T08:44:11.687456+0000","last_active":"2024-09-11T08:46:32.095913+0000","last_peered":"2024-09-11T08:46:32.095913+0000","last_clean":"2024-09-11T08:46:32.095913+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:46:32.095913+0000","last_undegraded":"2024-09-11T08:46:32.095913+0000","last_fullsized":"2024-09-11T08:46:32.095913+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:11.687402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:11.687402+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018414239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":171,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.086718+0000","last_change":"2024-09-11T08:44:32.483521+0000","last_active":"2024-09-11T08:46:31.086718+0000","last_peered":"2024-09-11T08:46:31.086718+0000","last_clean":"2024-09-11T08:46:31.086718+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:46:31.086718+0000","last_undegraded":"2024-09-11T08:46:31.086718+0000","last_fullsized":"2024-09-11T08:46:31.086718+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:44:32.483439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013511770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":158,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.108139+0000","last_change":"2024-09-11T08:45:23.213642+0000","last_active":"2024-09-11T08:46:32.108139+0000","last_peered":"2024-09-11T08:46:32.108139+0000","last_clean":"2024-09-11T08:46:32.108139+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:46:32.108139+0000","last_undegraded":"2024-09-11T08:46:32.108139+0000","last_fullsized":"2024-09-11T08:46:32.108139+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.213592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.213592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:50.258501+0000","scrub_duration":0.001600473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":166,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.089481+0000","last_change":"2024-09-11T08:45:07.618618+0000","last_active":"2024-09-11T08:46:31.089481+0000","last_peered":"2024-09-11T08:46:31.089481+0000","last_clean":"2024-09-11T08:46:31.089481+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:46:31.089481+0000","last_undegraded":"2024-09-11T08:46:31.089481+0000","last_fullsized":"2024-09-11T08:46:31.089481+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.618564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.618564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:37.053995+0000","scrub_duration":0.0016216729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":179,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.079973+0000","last_change":"2024-09-11T08:46:15.658863+0000","last_active":"2024-09-11T08:46:31.079973+0000","last_peered":"2024-09-11T08:46:31.079973+0000","last_clean":"2024-09-11T08:46:31.079973+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:46:31.079973+0000","last_undegraded":"2024-09-11T08:46:31.079973+0000","last_fullsized":"2024-09-11T08:46:31.079973+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:18.928481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:18.928481+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047130310000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":163,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.093451+0000","last_change":"2024-09-11T08:44:09.678084+0000","last_active":"2024-09-11T08:46:32.093451+0000","last_peered":"2024-09-11T08:46:32.093451+0000","last_clean":"2024-09-11T08:46:32.093451+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:46:32.093451+0000","last_undegraded":"2024-09-11T08:46:32.093451+0000","last_fullsized":"2024-09-11T08:46:32.093451+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:09.677991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:09.677991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066552809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":261,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.764597+0000","last_change":"2024-09-11T08:45:20.119727+0000","last_active":"2024-09-11T08:46:32.764597+0000","last_peered":"2024-09-11T08:46:32.764597+0000","last_clean":"2024-09-11T08:46:32.764597+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:46:32.764597+0000","last_undegraded":"2024-09-11T08:46:32.764597+0000","last_fullsized":"2024-09-11T08:46:32.764597+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:20.119626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.119626+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:49.101497+0000","scrub_duration":0.0018087660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":146,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.096181+0000","last_change":"2024-09-11T08:44:20.017341+0000","last_active":"2024-09-11T08:46:31.096181+0000","last_peered":"2024-09-11T08:46:31.096181+0000","last_clean":"2024-09-11T08:46:31.096181+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:46:31.096181+0000","last_undegraded":"2024-09-11T08:46:31.096181+0000","last_fullsized":"2024-09-11T08:46:31.096181+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:20.017261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:20.017261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001558369,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"152'15","reported_seq":180,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.107173+0000","last_change":"2024-09-11T08:45:21.132138+0000","last_active":"2024-09-11T08:46:32.107173+0000","last_peered":"2024-09-11T08:46:32.107173+0000","last_clean":"2024-09-11T08:46:32.107173+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:46:32.107173+0000","last_undegraded":"2024-09-11T08:46:32.107173+0000","last_fullsized":"2024-09-11T08:46:32.107173+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'15","last_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.132083+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003984156,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":171,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.086996+0000","last_change":"2024-09-11T08:45:27.436673+0000","last_active":"2024-09-11T08:46:31.086996+0000","last_peered":"2024-09-11T08:46:31.086996+0000","last_clean":"2024-09-11T08:46:31.086996+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:46:31.086996+0000","last_undegraded":"2024-09-11T08:46:31.086996+0000","last_fullsized":"2024-09-11T08:46:31.086996+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.436552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001660176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":271,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.091797+0000","last_change":"2024-09-11T08:45:28.692173+0000","last_active":"2024-09-11T08:46:32.091797+0000","last_peered":"2024-09-11T08:46:32.091797+0000","last_clean":"2024-09-11T08:46:32.091797+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:46:32.091797+0000","last_undegraded":"2024-09-11T08:46:32.091797+0000","last_fullsized":"2024-09-11T08:46:32.091797+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:28.692100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.692100+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:37.432189+0000","scrub_duration":0.0019579020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":167,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.094594+0000","last_change":"2024-09-11T08:45:14.581287+0000","last_active":"2024-09-11T08:46:32.094594+0000","last_peered":"2024-09-11T08:46:32.094594+0000","last_clean":"2024-09-11T08:46:32.094594+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:46:32.094594+0000","last_undegraded":"2024-09-11T08:46:32.094594+0000","last_fullsized":"2024-09-11T08:46:32.094594+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.581231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.581231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:41.692088+0000","scrub_duration":0.001580932,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":169,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.097436+0000","last_change":"2024-09-11T08:43:37.137260+0000","last_active":"2024-09-11T08:46:31.097436+0000","last_peered":"2024-09-11T08:46:31.097436+0000","last_clean":"2024-09-11T08:46:31.097436+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:46:31.097436+0000","last_undegraded":"2024-09-11T08:46:31.097436+0000","last_fullsized":"2024-09-11T08:46:31.097436+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:37.137173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:37.137173+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085039490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":162,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.098581+0000","last_change":"2024-09-11T08:45:27.198021+0000","last_active":"2024-09-11T08:46:31.098581+0000","last_peered":"2024-09-11T08:46:31.098581+0000","last_clean":"2024-09-11T08:46:31.098581+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:46:31.098581+0000","last_undegraded":"2024-09-11T08:46:31.098581+0000","last_fullsized":"2024-09-11T08:46:31.098581+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.197944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.197944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:39.065394+0000","scrub_duration":0.0015230210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":169,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.113864+0000","last_change":"2024-09-11T08:45:21.736656+0000","last_active":"2024-09-11T08:46:32.113864+0000","last_peered":"2024-09-11T08:46:32.113864+0000","last_clean":"2024-09-11T08:46:32.113864+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:46:32.113864+0000","last_undegraded":"2024-09-11T08:46:32.113864+0000","last_fullsized":"2024-09-11T08:46:32.113864+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:21.736573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.736573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020360389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":156,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.094116+0000","last_change":"2024-09-11T08:45:28.259156+0000","last_active":"2024-09-11T08:46:32.094116+0000","last_peered":"2024-09-11T08:46:32.094116+0000","last_clean":"2024-09-11T08:46:32.094116+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:46:32.094116+0000","last_undegraded":"2024-09-11T08:46:32.094116+0000","last_fullsized":"2024-09-11T08:46:32.094116+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.259103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.259103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:47.651285+0000","scrub_duration":0.0015518800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":199,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.080303+0000","last_change":"2024-09-11T08:46:01.993547+0000","last_active":"2024-09-11T08:46:31.080303+0000","last_peered":"2024-09-11T08:46:31.080303+0000","last_clean":"2024-09-11T08:46:31.080303+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:46:31.080303+0000","last_undegraded":"2024-09-11T08:46:31.080303+0000","last_fullsized":"2024-09-11T08:46:31.080303+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:45:06.657912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.657912+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:34.933163+0000","scrub_duration":0.0023744109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":171,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.087896+0000","last_change":"2024-09-11T08:45:15.298564+0000","last_active":"2024-09-11T08:46:31.087896+0000","last_peered":"2024-09-11T08:46:31.087896+0000","last_clean":"2024-09-11T08:46:31.087896+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:46:31.087896+0000","last_undegraded":"2024-09-11T08:46:31.087896+0000","last_fullsized":"2024-09-11T08:46:31.087896+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.298476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.298476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033059629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":158,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.106959+0000","last_change":"2024-09-11T08:45:14.165632+0000","last_active":"2024-09-11T08:46:32.106959+0000","last_peered":"2024-09-11T08:46:32.106959+0000","last_clean":"2024-09-11T08:46:32.106959+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:46:32.106959+0000","last_undegraded":"2024-09-11T08:46:32.106959+0000","last_fullsized":"2024-09-11T08:46:32.106959+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.165576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.165576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012147639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":157,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.090920+0000","last_change":"2024-09-11T08:43:29.349431+0000","last_active":"2024-09-11T08:46:32.090920+0000","last_peered":"2024-09-11T08:46:32.090920+0000","last_clean":"2024-09-11T08:46:32.090920+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:46:32.090920+0000","last_undegraded":"2024-09-11T08:46:32.090920+0000","last_fullsized":"2024-09-11T08:46:32.090920+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.349351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.349351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006276294,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":169,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.094431+0000","last_change":"2024-09-11T08:44:12.198399+0000","last_active":"2024-09-11T08:46:32.094431+0000","last_peered":"2024-09-11T08:46:32.094431+0000","last_clean":"2024-09-11T08:46:32.094431+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:46:32.094431+0000","last_undegraded":"2024-09-11T08:46:32.094431+0000","last_fullsized":"2024-09-11T08:46:32.094431+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:44:12.198335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.198335+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020517040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":172,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.079894+0000","last_change":"2024-09-11T08:45:14.574739+0000","last_active":"2024-09-11T08:46:31.079894+0000","last_peered":"2024-09-11T08:46:31.079894+0000","last_clean":"2024-09-11T08:46:31.079894+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:46:31.079894+0000","last_undegraded":"2024-09-11T08:46:31.079894+0000","last_fullsized":"2024-09-11T08:46:31.079894+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.574678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.574678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013819469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":167,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.086333+0000","last_change":"2024-09-11T08:44:28.484120+0000","last_active":"2024-09-11T08:46:31.086333+0000","last_peered":"2024-09-11T08:46:31.086333+0000","last_clean":"2024-09-11T08:46:31.086333+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:46:31.086333+0000","last_undegraded":"2024-09-11T08:46:31.086333+0000","last_fullsized":"2024-09-11T08:46:31.086333+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:28.484047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.484047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001479172,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":166,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.107072+0000","last_change":"2024-09-11T08:46:01.984620+0000","last_active":"2024-09-11T08:46:32.107072+0000","last_peered":"2024-09-11T08:46:32.107072+0000","last_clean":"2024-09-11T08:46:32.107072+0000","last_became_active":"2024-09-11T08:46:01.984167+0000","last_became_peered":"2024-09-11T08:46:01.984167+0000","last_unstale":"2024-09-11T08:46:32.107072+0000","last_undegraded":"2024-09-11T08:46:32.107072+0000","last_fullsized":"2024-09-11T08:46:32.107072+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:32.114990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:32.114990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014089513999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"136'6","reported_seq":200,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.096536+0000","last_change":"2024-09-11T08:46:01.996138+0000","last_active":"2024-09-11T08:46:32.096536+0000","last_peered":"2024-09-11T08:46:32.096536+0000","last_clean":"2024-09-11T08:46:32.096536+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:46:32.096536+0000","last_undegraded":"2024-09-11T08:46:32.096536+0000","last_fullsized":"2024-09-11T08:46:32.096536+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'6","last_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:44:21.302870+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012512683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":180,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.093169+0000","last_change":"2024-09-11T08:45:21.481725+0000","last_active":"2024-09-11T08:46:32.093169+0000","last_peered":"2024-09-11T08:46:32.093169+0000","last_clean":"2024-09-11T08:46:32.093169+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:46:32.093169+0000","last_undegraded":"2024-09-11T08:46:32.093169+0000","last_fullsized":"2024-09-11T08:46:32.093169+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:21.481634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.481634+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025664590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":21,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.085593+0000","last_change":"2024-09-11T08:46:01.979915+0000","last_active":"2024-09-11T08:46:31.085593+0000","last_peered":"2024-09-11T08:46:31.085593+0000","last_clean":"2024-09-11T08:46:31.085593+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:46:31.085593+0000","last_undegraded":"2024-09-11T08:46:31.085593+0000","last_fullsized":"2024-09-11T08:46:31.085593+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.196277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.196277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:37.455059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":175,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.086463+0000","last_change":"2024-09-11T08:45:23.348327+0000","last_active":"2024-09-11T08:46:31.086463+0000","last_peered":"2024-09-11T08:46:31.086463+0000","last_clean":"2024-09-11T08:46:31.086463+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:46:31.086463+0000","last_undegraded":"2024-09-11T08:46:31.086463+0000","last_fullsized":"2024-09-11T08:46:31.086463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.348275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.348275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:38.671979+0000","scrub_duration":0.0035843659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":177,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.101670+0000","last_change":"2024-09-11T08:45:23.674163+0000","last_active":"2024-09-11T08:46:32.101670+0000","last_peered":"2024-09-11T08:46:32.101670+0000","last_clean":"2024-09-11T08:46:32.101670+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:46:32.101670+0000","last_undegraded":"2024-09-11T08:46:32.101670+0000","last_fullsized":"2024-09-11T08:46:32.101670+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.674102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.674102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:44.657753+0000","scrub_duration":0.001440293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":162,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.096735+0000","last_change":"2024-09-11T08:44:23.320368+0000","last_active":"2024-09-11T08:46:32.096735+0000","last_peered":"2024-09-11T08:46:32.096735+0000","last_clean":"2024-09-11T08:46:32.096735+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:46:32.096735+0000","last_undegraded":"2024-09-11T08:46:32.096735+0000","last_fullsized":"2024-09-11T08:46:32.096735+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:23.320301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:23.320301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017133368999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":33,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-11T08:46:31.085513+0000","last_change":"2024-09-11T08:46:02.002471+0000","last_active":"2024-09-11T08:46:31.085513+0000","last_peered":"2024-09-11T08:46:31.085513+0000","last_clean":"2024-09-11T08:46:31.085513+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:46:31.085513+0000","last_undegraded":"2024-09-11T08:46:31.085513+0000","last_fullsized":"2024-09-11T08:46:31.085513+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:45:25.620635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.620635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":159,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-11T08:46:32.095634+0000","last_change":"2024-09-11T08:43:30.336272+0000","last_active":"2024-09-11T08:46:32.095634+0000","last_peered":"2024-09-11T08:46:32.095634+0000","last_clean":"2024-09-11T08:46:32.095634+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:46:32.095634+0000","last_undegraded":"2024-09-11T08:46:32.095634+0000","last_fullsized":"2024-09-11T08:46:32.095634+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:30.336188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:30.336188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044263979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":6338772992,"num_objects":96722,"num_object_clones":0,"num_object_copies":290166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5905,"num_objects_unfound":0,"num_objects_dirty":96722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96722,"num_write_kb":6190208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1864,"num_bytes_recovered":122159104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19259588608,"data_stored":19259588608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84422,"ondisk_log_size":84422,"up":48,"acting":47,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":1584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":456,"ondisk_log_size":456,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":470,"num_read_kb":465,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299493,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2205228,"kb_used_data":1907876,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91539924,"statfs":{"total":95995035648,"available":93736882176,"internally_reserved":0,"allocated":1953665024,"data_stored":1953251378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.89900000000000002,"15min":0.63300000000000001},"min":{"1min":0.34499999999999997,"5min":0.22,"15min":0.22},"max":{"1min":1.2,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":1.022,"15min":0.68000000000000005},"min":{"1min":0.36699999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.72099999999999997,"5min":5.9790000000000001,"15min":5.9790000000000001},"last":0.58199999999999996}]},{"osd":1,"last update":"Wed Sep 11 08:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":1.056,"15min":0.68999999999999995},"min":{"1min":0.25900000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.238,"5min":8.5030000000000001,"15min":8.5030000000000001},"last":0.502},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.88300000000000001,"15min":0.64300000000000002},"min":{"1min":0.33000000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.747,"5min":6.593,"15min":6.593},"last":0.36599999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.97499999999999998,"15min":0.66200000000000003},"min":{"1min":0.373,"5min":0.255,"15min":0.255},"max":{"1min":1.1539999999999999,"5min":4.6029999999999998,"15min":4.6029999999999998},"last":0.316},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.96299999999999997,"15min":0.65000000000000002},"min":{"1min":0.30099999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.0059999999999998,"5min":5.7990000000000004,"15min":5.7990000000000004},"last":0.79100000000000004}]},{"osd":3,"last update":"Wed Sep 11 08:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.0780000000000001,"15min":0.70699999999999996},"min":{"1min":0.38800000000000001,"5min":0.25,"15min":0.25},"max":{"1min":2.7269999999999999,"5min":7.8609999999999998,"15min":7.8609999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.2350000000000001,"15min":0.76500000000000001},"min":{"1min":0.318,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.5940000000000001,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.40999999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48699999999999999,"15min":0.44400000000000001},"min":{"1min":0.29299999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.66100000000000003,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.441},"min":{"1min":0.36099999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.048,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.34999999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.49299999999999999,"15min":0.45800000000000002},"min":{"1min":0.29399999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.2649999999999999,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60099999999999998,"15min":0.495},"min":{"1min":0.39100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82999999999999996,"5min":2.806,"15min":2.806},"last":0.45900000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.627,"15min":0.52300000000000002},"min":{"1min":0.38100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.72499999999999998,"5min":3.5979999999999999,"15min":3.5979999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.51500000000000001},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.1309999999999998,"5min":2.633,"15min":2.633},"last":0.47499999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528426,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1826720,"kb_used_data":1521048,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91918432,"statfs":{"total":95995035648,"available":94124474368,"internally_reserved":0,"allocated":1557553152,"data_stored":1557151630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":1.4399999999999999,"15min":0.83099999999999996},"min":{"1min":0.222,"5min":0.217,"15min":0.217},"max":{"1min":2.0049999999999999,"5min":28.302,"15min":28.302},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":1.4390000000000001,"15min":0.83099999999999996},"min":{"1min":0.29899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":2.5750000000000002,"5min":28.314,"15min":28.314},"last":1.3160000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.1930000000000001,"15min":0.747},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.4649999999999999,"5min":12.586,"15min":12.586},"last":3.3380000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.077,"15min":0.70699999999999996},"min":{"1min":0.25700000000000001,"5min":0.253,"15min":0.253},"max":{"1min":3.2269999999999999,"5min":10.834,"15min":10.834},"last":0.93700000000000006}]},{"osd":2,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.92500000000000004,"15min":0.67300000000000004},"min":{"1min":0.27700000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.82299999999999995,"5min":14.313000000000001,"15min":14.313000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.95899999999999996,"15min":0.65900000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":2.4660000000000002,"5min":6.5979999999999999,"15min":6.5979999999999999},"last":0.58699999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1.2529999999999999,"15min":0.76200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.2130000000000001,"5min":11.433,"15min":11.433},"last":3.367},{"interface":"front","average":{"1min":0.54800000000000004,"5min":1.1060000000000001,"15min":0.72799999999999998},"min":{"1min":0.311,"5min":0.224,"15min":0.224},"max":{"1min":1.194,"5min":9.1069999999999993,"15min":9.1069999999999993},"last":2.2949999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.47799999999999998,"15min":0.433},"min":{"1min":0.25600000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.577,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.2809999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.44600000000000001,"15min":0.42299999999999999},"min":{"1min":0.28299999999999997,"5min":0.27300000000000002,"15min":0.26400000000000001},"max":{"1min":1.1990000000000001,"5min":1.361,"15min":1.361},"last":2.0129999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.53300000000000003,"15min":0.47099999999999997},"min":{"1min":0.32700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":1.246,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":3.097},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.505,"15min":0.46000000000000002},"min":{"1min":0.30199999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.2609999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":2.2200000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:46:34 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.61499999999999999,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.25900000000000001,"15min":0.247},"max":{"1min":7.327,"5min":7.327,"15min":7.327},"last":7.327},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53100000000000003,"15min":0.505},"min":{"1min":0.373,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.95899999999999996,"5min":1.3,"15min":1.3},"last":0.95899999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757358,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3392360,"kb_used_data":3090848,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":90352792,"statfs":{"total":95995035648,"available":92521259008,"internally_reserved":0,"allocated":3165028352,"data_stored":3164618891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89000000000000001,"15min":0.67000000000000004},"min":{"1min":0.40100000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":2.637,"5min":7.3899999999999997,"15min":7.3899999999999997},"last":3.1850000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.1080000000000001,"15min":0.72099999999999997},"min":{"1min":0.28299999999999997,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":3.1320000000000001,"5min":6.5789999999999997,"15min":6.5789999999999997},"last":3.3799999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":1.02,"15min":0.71599999999999997},"min":{"1min":0.312,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.0270000000000001,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.95999999999999996,"15min":0.66200000000000003},"min":{"1min":0.32000000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.7370000000000001,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":1.6859999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.98799999999999999,"15min":0.68999999999999995},"min":{"1min":0.33500000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.0310000000000001,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":4.0919999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.89600000000000002,"15min":0.67000000000000004},"min":{"1min":0.312,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.306,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":1.026}]},{"osd":3,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.98499999999999999,"15min":0.68600000000000005},"min":{"1min":0.35499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.284,"5min":6.0460000000000003,"15min":6.0460000000000003},"last":3.0499999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":1.1850000000000001,"15min":0.77100000000000002},"min":{"1min":0.46400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.0539999999999998,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":4.415}]},{"osd":4,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.44500000000000001,"15min":0.40200000000000002},"min":{"1min":0.27200000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.3540000000000001,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.48799999999999999,"15min":0.43099999999999999},"min":{"1min":0.34799999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.1899999999999999,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.40699999999999997}]},{"osd":6,"last update":"Wed Sep 11 08:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.52600000000000002,"15min":0.46300000000000002},"min":{"1min":0.32200000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":3.4329999999999998,"5min":3.4329999999999998,"15min":3.4329999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.51800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":1.893,"5min":2.319,"15min":2.319},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:46:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.51700000000000002,"15min":0.496},"min":{"1min":0.36299999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.307,"15min":0.307},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.65700000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986290,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2223784,"kb_used_data":1922272,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91521368,"statfs":{"total":95995035648,"available":93717880832,"internally_reserved":0,"allocated":1968406528,"data_stored":1967995896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":2.6720000000000002,"15min":1.2509999999999999},"min":{"1min":0.23999999999999999,"5min":0.218,"15min":0.20399999999999999},"max":{"1min":1.7090000000000001,"5min":89.510000000000005,"15min":89.510000000000005},"last":1.2809999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":2.7280000000000002,"15min":1.2889999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.9060000000000001,"5min":89.765000000000001,"15min":89.765000000000001},"last":0.34899999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.228,"15min":0.78100000000000003},"min":{"1min":0.28799999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.0960000000000001,"5min":18.596,"15min":18.596},"last":0.47199999999999998},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.2070000000000001,"15min":0.77300000000000002},"min":{"1min":0.26100000000000001,"5min":0.24399999999999999,"15min":0.23599999999999999},"max":{"1min":6.3300000000000001,"5min":18.754000000000001,"15min":18.754000000000001},"last":0.875}]},{"osd":2,"last update":"Wed Sep 11 08:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.92700000000000005,"15min":0.68500000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":5.6660000000000004,"5min":5.6660000000000004,"15min":5.6660000000000004},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.91800000000000004,"15min":0.68200000000000005},"min":{"1min":0.27500000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":3.0030000000000001,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.30599999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":3.0590000000000002,"15min":1.4039999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":2.9820000000000002,"5min":89.881,"15min":89.881},"last":0.37},{"interface":"front","average":{"1min":1.3480000000000001,"5min":2.9049999999999998,"15min":1.343},"min":{"1min":0.32300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":7.8330000000000002,"5min":89.715000000000003,"15min":89.715000000000003},"last":0.80600000000000005}]},{"osd":5,"last update":"Wed Sep 11 08:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.48699999999999999,"15min":0.43099999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.46800000000000003,"15min":0.434},"min":{"1min":0.307,"5min":0.24199999999999999,"15min":0.23699999999999999},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":0.43099999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:46:27 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.48899999999999999,"15min":0.45300000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":2.9390000000000001,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.57199999999999995,"15min":0.503},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":2.371,"5min":2.371,"15min":2.371},"last":0.63800000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:45:38 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.505,"15min":0.48999999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.72099999999999997,"5min":1.1339999999999999,"15min":1.194},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51800000000000002,"15min":0.48599999999999999},"min":{"1min":0.33200000000000002,"5min":0.30499999999999999,"15min":0.28799999999999998},"max":{"1min":1.083,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.44600000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149816,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2630720,"kb_used_data":2325048,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91114432,"statfs":{"total":95995035648,"available":93301178368,"internally_reserved":0,"allocated":2380849152,"data_stored":2380415525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.6799999999999999,"15min":0.80200000000000005},"min":{"1min":0.20699999999999999,"5min":0.189,"15min":0.184},"max":{"1min":2.7040000000000002,"5min":10.901999999999999,"15min":10.901999999999999},"last":2.508},{"interface":"front","average":{"1min":0.98099999999999998,"5min":2.0720000000000001,"15min":0.95099999999999996},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":4.6760000000000002,"5min":9.6820000000000004,"15min":9.6820000000000004},"last":4.9349999999999996}]},{"osd":1,"last update":"Wed Sep 11 08:46:17 2024","interfaces":[{"interface":"back","average":{"1min":6.6299999999999999,"5min":3.0699999999999998,"15min":1.2869999999999999},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":115.601,"5min":115.601,"15min":115.601},"last":7.3259999999999996},{"interface":"front","average":{"1min":6.8380000000000001,"5min":2.9790000000000001,"15min":1.262},"min":{"1min":0.251,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":115.474,"5min":115.474,"15min":115.474},"last":8.4420000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:46:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.875,"15min":0.90900000000000003},"min":{"1min":0.32400000000000001,"5min":0.251,"15min":0.23200000000000001},"max":{"1min":2.8109999999999999,"5min":8.9510000000000005,"15min":8.9510000000000005},"last":8.2699999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":1.7909999999999999,"15min":0.876},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.1749999999999998,"5min":10.695,"15min":10.695},"last":2.5270000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:46:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.4279999999999999,"15min":0.85599999999999998},"min":{"1min":0.28599999999999998,"5min":0.27000000000000002,"15min":0.26700000000000002},"max":{"1min":5.5460000000000003,"5min":8.9969999999999999,"15min":8.9969999999999999},"last":8.3279999999999994},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.2450000000000001,"15min":0.78200000000000003},"min":{"1min":0.34599999999999997,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.7599999999999998,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":2.4870000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":1.3759999999999999,"15min":0.83799999999999997},"min":{"1min":0.34300000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.9079999999999999,"5min":11.526999999999999,"15min":11.526999999999999},"last":8.3599999999999994},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.2350000000000001,"15min":0.77900000000000003},"min":{"1min":0.38600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.6000000000000001,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":2.4369999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:46:17 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.5840000000000001,"15min":0.92200000000000004},"min":{"1min":0.45700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.9649999999999999,"5min":11.494999999999999,"15min":11.494999999999999},"last":8.4160000000000004},{"interface":"front","average":{"1min":1.171,"5min":1.605,"15min":0.91000000000000003},"min":{"1min":0.314,"5min":0.311,"15min":0.311},"max":{"1min":4.6310000000000002,"5min":7.7249999999999996,"15min":7.7249999999999996},"last":0.63500000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.4279999999999999,"15min":0.88600000000000001},"min":{"1min":0.37,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92900000000000005,"5min":11.561,"15min":11.561},"last":8.3879999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":1.25,"15min":0.81399999999999995},"min":{"1min":0.372,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.9380000000000002,"5min":6.3170000000000002,"15min":6.3170000000000002},"last":2.4660000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411452,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2850836,"kb_used_data":2553484,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":90894316,"statfs":{"total":95995035648,"available":93075779584,"internally_reserved":0,"allocated":2614767616,"data_stored":2614377175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.954,"15min":0.88900000000000001},"min":{"1min":0.20699999999999999,"5min":0.159,"15min":0.159},"max":{"1min":8.2059999999999995,"5min":24.989000000000001,"15min":24.989000000000001},"last":34.713999999999999},{"interface":"front","average":{"1min":0.97999999999999998,"5min":2.3220000000000001,"15min":1.024},"min":{"1min":0.23599999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":7.0170000000000003,"5min":24.803999999999998,"15min":24.803999999999998},"last":34.399000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.385,"15min":0.71699999999999997},"min":{"1min":0.25800000000000001,"5min":0.19,"15min":0.19},"max":{"1min":8.3100000000000005,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":34.771000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.631,"15min":0.79600000000000004},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":4.9260000000000002,"5min":9.3119999999999994,"15min":9.3119999999999994},"last":35.209000000000003}]},{"osd":3,"last update":"Wed Sep 11 08:46:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.895,"15min":0.91400000000000003},"min":{"1min":0.26800000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":8.0350000000000001,"5min":9.5129999999999999,"15min":9.5129999999999999},"last":34.75},{"interface":"front","average":{"1min":1.611,"5min":1.8680000000000001,"15min":0.91400000000000003},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.26800000000000002},"max":{"1min":5.1310000000000002,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":32.802}]},{"osd":4,"last update":"Wed Sep 11 08:46:26 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.381,"15min":0.82699999999999996},"min":{"1min":0.29799999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":8.0820000000000007,"5min":8.0820000000000007,"15min":8.0820000000000007},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.304,"15min":0.80200000000000005},"min":{"1min":0.35499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":4.851,"5min":6.9320000000000004,"15min":6.9320000000000004},"last":1.101}]},{"osd":5,"last update":"Wed Sep 11 08:45:51 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.1930000000000001,"15min":0.77600000000000002},"min":{"1min":0.38700000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":4.8949999999999996,"5min":6.9550000000000001,"15min":6.9550000000000001},"last":1.145},{"interface":"front","average":{"1min":0.70499999999999996,"5min":1,"15min":0.70599999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.0419999999999998,"5min":5.9530000000000003,"15min":5.9530000000000003},"last":0.47299999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:46:17 2024","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.4079999999999999,"15min":0.85699999999999998},"min":{"1min":0.38700000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":8.1120000000000001,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":1.089},{"interface":"front","average":{"1min":1.242,"5min":1.2809999999999999,"15min":0.80000000000000004},"min":{"1min":0.38500000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.8290000000000002,"5min":6.9779999999999998,"15min":6.9779999999999998},"last":0.51000000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:46:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.137,"15min":0.76400000000000001},"min":{"1min":0.32500000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.9489999999999998,"5min":5.9909999999999997,"15min":5.9909999999999997},"last":1.218},{"interface":"front","average":{"1min":1.607,"5min":1.444,"15min":0.878},"min":{"1min":0.42099999999999999,"5min":0.371,"15min":0.36099999999999999},"max":{"1min":8.1400000000000006,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.48799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574977,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2993284,"kb_used_data":2687612,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":90751868,"statfs":{"total":95995035648,"available":92929912832,"internally_reserved":0,"allocated":2752114688,"data_stored":2751676344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:31 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":2.8929999999999998,"15min":1.1879999999999999},"min":{"1min":0.20499999999999999,"5min":0.183,"15min":0.183},"max":{"1min":4.8319999999999999,"5min":117.01600000000001,"15min":117.01600000000001},"last":0.23799999999999999},{"interface":"front","average":{"1min":1.915,"5min":3.6200000000000001,"15min":1.4490000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":6.7370000000000001,"5min":117.62,"15min":117.62},"last":0.54400000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:45:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":1.7709999999999999,"15min":0.84299999999999997},"min":{"1min":0.21099999999999999,"5min":0.191,"15min":0.191},"max":{"1min":2.4180000000000001,"5min":20.628,"15min":20.628},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.696,"15min":0.81000000000000005},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":4.4779999999999998,"5min":20.148,"15min":20.148},"last":0.35099999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":2.5369999999999999,"15min":1.1180000000000001},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":2.2730000000000001,"5min":116.98,"15min":116.98},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.99299999999999999,"5min":2.891,"15min":1.25},"min":{"1min":0.26400000000000001,"5min":0.22,"15min":0.22},"max":{"1min":4.2679999999999998,"5min":117.56699999999999,"15min":117.56699999999999},"last":0.69299999999999995}]},{"osd":4,"last update":"Wed Sep 11 08:46:31 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.1899999999999999,"15min":0.75800000000000001},"min":{"1min":0.38400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.4780000000000002,"5min":11.19,"15min":11.19},"last":0.38400000000000001},{"interface":"front","average":{"1min":1.262,"5min":1.0880000000000001,"15min":0.73699999999999999},"min":{"1min":0.32600000000000001,"5min":0.251,"15min":0.251},"max":{"1min":5.056,"5min":7.7169999999999996,"15min":7.7169999999999996},"last":0.42799999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":2.5139999999999998,"15min":1.218},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.2280000000000002,"5min":103.337,"15min":103.337},"last":0.625},{"interface":"front","average":{"1min":1.0229999999999999,"5min":2.431,"15min":1.1850000000000001},"min":{"1min":0.38300000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.7829999999999999,"5min":103.163,"15min":103.163},"last":0.56200000000000006}]},{"osd":6,"last update":"Wed Sep 11 08:46:25 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0529999999999999,"15min":0.73799999999999999},"min":{"1min":0.34100000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.2410000000000001,"5min":8.9149999999999991,"15min":8.9149999999999991},"last":0.51800000000000002},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.3740000000000001,"15min":0.86099999999999999},"min":{"1min":0.36299999999999999,"5min":0.31,"15min":0.31},"max":{"1min":3.2639999999999998,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":3.2229999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:46:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.2450000000000001,"15min":0.81100000000000005},"min":{"1min":0.38400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":3.7519999999999998,"5min":11.315,"15min":11.315},"last":0.40999999999999998},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.1000000000000001,"15min":0.77000000000000002},"min":{"1min":0.376,"5min":0.315,"15min":0.315},"max":{"1min":4.8739999999999997,"5min":8.8800000000000008,"15min":8.8800000000000008},"last":0.47599999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738501,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3008576,"kb_used_data":2702540,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":90736576,"statfs":{"total":95995035648,"available":92914253824,"internally_reserved":0,"allocated":2767400960,"data_stored":2767104546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:46:28 2024","interfaces":[{"interface":"back","average":{"1min":2,"5min":1.9610000000000001,"15min":0.86599999999999999},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":8.3900000000000006,"5min":8.3900000000000006,"15min":8.3900000000000006},"last":3.754},{"interface":"front","average":{"1min":2.431,"5min":1.712,"15min":0.79700000000000004},"min":{"1min":0.30599999999999999,"5min":0.186,"15min":0.186},"max":{"1min":12.532,"5min":12.532,"15min":12.532},"last":2.6739999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.762,"15min":0.81999999999999995},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":3.5670000000000002,"5min":19.940000000000001,"15min":19.940000000000001},"last":2.6989999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":1.968,"15min":0.88800000000000001},"min":{"1min":0.23599999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":3.2650000000000001,"5min":20.757000000000001,"15min":20.757000000000001},"last":2.605}]},{"osd":3,"last update":"Wed Sep 11 08:46:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":2.2360000000000002,"15min":1.0069999999999999},"min":{"1min":0.23100000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":3.927,"5min":11.951000000000001,"15min":11.951000000000001},"last":4.5709999999999997},{"interface":"front","average":{"1min":1.512,"5min":2.0009999999999999,"15min":0.92400000000000004},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.746,"5min":7.5099999999999998,"15min":7.5099999999999998},"last":3.7349999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:46:32 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.47,"15min":0.85199999999999998},"min":{"1min":0.30599999999999999,"5min":0.25,"15min":0.25},"max":{"1min":4.8490000000000002,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":3.6600000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.3759999999999999,"15min":0.82299999999999995},"min":{"1min":0.27600000000000002,"5min":0.255,"15min":0.255},"max":{"1min":3.9860000000000002,"5min":6.9539999999999997,"15min":6.9539999999999997},"last":2.7269999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.474,"15min":0.84299999999999997},"min":{"1min":0.36099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.4809999999999999,"5min":7.4009999999999998,"15min":7.4009999999999998},"last":2.8180000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":1.407,"15min":0.83599999999999997},"min":{"1min":0.38700000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.2930000000000001,"5min":7.423,"15min":7.423},"last":3.6920000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:46:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.615,"15min":0.90000000000000002},"min":{"1min":0.372,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.609,"5min":7.4240000000000004,"15min":7.4240000000000004},"last":2.8399999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.6220000000000001,"15min":0.92300000000000004},"min":{"1min":0.41499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":3.6840000000000002,"5min":7.4489999999999998,"15min":7.4489999999999998},"last":3.714}]},{"osd":7,"last update":"Wed Sep 11 08:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":1.4259999999999999,"15min":0.85899999999999999},"min":{"1min":0.40699999999999997,"5min":0.313,"15min":0.313},"max":{"1min":3.2650000000000001,"5min":7.0049999999999999,"15min":7.0049999999999999},"last":2.7490000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.4530000000000001,"15min":0.85699999999999998},"min":{"1min":0.44,"5min":0.307,"15min":0.307},"max":{"1min":1.127,"5min":7.4450000000000003,"15min":7.4450000000000003},"last":2.8599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2768175104,"data_stored":2768175104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2779578368,"data_stored":2779578368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2634809344,"data_stored":2634809344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2387869696,"data_stored":2387869696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1970470912,"data_stored":1970470912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3183411200,"data_stored":3183411200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1575419904,"data_stored":1575419904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1959854080,"data_stored":1959854080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:46:39.051 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph status --format=json 2024-09-11T08:46:39.052 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:39.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:39.109 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:39.110 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:39.171 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 69 16 99448 99432 90.0469 64.8125 0.0114172 0.0110999 2024-09-11T08:46:39.211 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:39.240 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:39.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:39.241 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:39.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:39.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:39.382 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:39.382 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:39.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:39.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:39.537 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:39.537 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:39.539 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:46:39.544 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:46:39.584 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:46:39.584 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:46:39.638 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:39.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:39.671 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:39.671 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:39.772 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:39.801 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:39.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:39.803 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:39.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:39.941 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:39.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:39.942 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:40.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:40.074 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:40.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:40.076 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:40.150 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:39 smithi145 ceph-mon[33384]: pgmap v336: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.06k op/s; 5905/290754 objects misplaced (2.031%); 4.7 MiB/s, 75 objects/s recovering 2024-09-11T08:46:40.171 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 70 16 100429 100413 89.6364 61.3125 0.0092209 0.011151 2024-09-11T08:46:40.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:40.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:40.214 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:40.215 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:40.232 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:39 smithi092 ceph-mon[29647]: pgmap v336: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.06k op/s; 5905/290754 objects misplaced (2.031%); 4.7 MiB/s, 75 objects/s recovering 2024-09-11T08:46:40.232 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:39 smithi092 ceph-mon[41114]: pgmap v336: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.06k op/s; 5905/290754 objects misplaced (2.031%); 4.7 MiB/s, 75 objects/s recovering 2024-09-11T08:46:40.315 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:40.341 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:40.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:40.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:40.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:40.482 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:40.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:40.483 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:40.584 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:40.612 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:40.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:40.613 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:40.714 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:40.747 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:40.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:40.748 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:40.849 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:40.885 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:40.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:40.886 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:40.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:41.027 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:41.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:41.028 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:41.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:41.170 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:41.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:41.171 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:41.172 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 71 15 101436 101421 89.2612 63 0.0103183 0.0111972 2024-09-11T08:46:41.272 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:41.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:41.307 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:41.307 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:41.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:41.449 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:41.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:41.450 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:41.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:41.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:41.578 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:41.578 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:41.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:41.716 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:41.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:41.717 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:41.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:46:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:46:41] "GET /metrics HTTP/1.1" 200 224275 "" "Prometheus/2.43.0" 2024-09-11T08:46:41.819 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:41.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:41.853 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:41.853 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:41.954 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:41.980 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:41.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:41.981 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:42.082 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:42.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:42.119 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:42.171 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 72 16 102481 102465 88.9276 65.25 0.00779309 0.0112401 2024-09-11T08:46:42.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:42.251 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:42.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:42.252 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:42.353 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:42.382 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:42.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:42.384 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:42.485 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:42.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:42.515 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:42.515 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:42.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:42 smithi092 ceph-mon[29647]: pgmap v337: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.0 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.1 KiB/s rd, 45 MiB/s wr, 719 op/s; 5905/295350 objects misplaced (1.999%); 2.1 MiB/s, 33 objects/s recovering 2024-09-11T08:46:42.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:42 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:46:42.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:42 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:46:42.520 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:42 smithi092 ceph-mon[41114]: pgmap v337: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.0 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.1 KiB/s rd, 45 MiB/s wr, 719 op/s; 5905/295350 objects misplaced (1.999%); 2.1 MiB/s, 33 objects/s recovering 2024-09-11T08:46:42.520 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:42 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:46:42.520 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:42 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:46:42.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:42.659 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:42.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:42.660 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:42 smithi145 ceph-mon[33384]: pgmap v337: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.0 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.1 KiB/s rd, 45 MiB/s wr, 719 op/s; 5905/295350 objects misplaced (1.999%); 2.1 MiB/s, 33 objects/s recovering 2024-09-11T08:46:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:42 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:46:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:42 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:46:42.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:42.797 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:42.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:42.798 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:42.899 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:42.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:42.921 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:42.921 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:43.022 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:43.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:43.051 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:43.051 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:43.152 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:43.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:46:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:46:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:46:43.171 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 73 16 103378 103362 88.4774 56.0625 0.0143994 0.0112969 2024-09-11T08:46:43.176 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:43.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:43.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:43.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-11T08:46:43.281 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:43.306 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:43.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:43.307 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:43.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:43.476 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:43.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:43.477 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:43.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:43 smithi092 ceph-mon[29647]: osdmap e181: 8 total, 8 up, 8 in 2024-09-11T08:46:43.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:43 smithi092 ceph-mon[41114]: osdmap e181: 8 total, 8 up, 8 in 2024-09-11T08:46:43.578 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:43.603 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:43.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:43.605 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:43.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:43 smithi145 ceph-mon[33384]: osdmap e181: 8 total, 8 up, 8 in 2024-09-11T08:46:43.705 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:43.732 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:43.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:43.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:43.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:43.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:43.873 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:43.873 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:43.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:44.007 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:44.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:44.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:44.109 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:44.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:44.140 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:44.140 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:44.172 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 74 16 104263 104247 88.0292 55.3125 0.0144385 0.0113553 2024-09-11T08:46:44.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:44.281 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:44.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:44.282 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:44.359 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:44 smithi092 ceph-mon[29647]: pgmap v339: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.24k op/s; 5586/305574 objects misplaced (1.828%); 4.9 MiB/s, 78 objects/s recovering 2024-09-11T08:46:44.360 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:44 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:46:44.360 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:44 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:46:44.360 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:44 smithi092 ceph-mon[41114]: pgmap v339: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.24k op/s; 5586/305574 objects misplaced (1.828%); 4.9 MiB/s, 78 objects/s recovering 2024-09-11T08:46:44.360 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:44 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:46:44.360 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:44 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:46:44.383 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:44.408 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:44.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:44.409 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:44.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:44.548 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:44.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:44.549 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:44.650 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:44 smithi145 ceph-mon[33384]: pgmap v339: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.24k op/s; 5586/305574 objects misplaced (1.828%); 4.9 MiB/s, 78 objects/s recovering 2024-09-11T08:46:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:44 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:46:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:44 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:46:44.683 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:44.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:44.684 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:44.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:44.820 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:44.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:44.822 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:44.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:44.962 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:44.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:44.963 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:45.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:45.107 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:45.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:45.108 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:45.172 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 75 16 105271 105255 87.6955 63 0.00787906 0.0113985 2024-09-11T08:46:45.209 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:45.236 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:45.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:45.237 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:45.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:45 smithi092 ceph-mon[29647]: osdmap e182: 8 total, 8 up, 8 in 2024-09-11T08:46:45.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:45 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:45.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:45.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:45 smithi092 ceph-mon[41114]: osdmap e182: 8 total, 8 up, 8 in 2024-09-11T08:46:45.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:45 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:45.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:45.338 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:45.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:45.379 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:45.379 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:45.480 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:45.510 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:46:45.513 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:45.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:45.514 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:45.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:45.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:45.660 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:45.660 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:45.665 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:45 smithi145 ceph-mon[33384]: osdmap e182: 8 total, 8 up, 8 in 2024-09-11T08:46:45.665 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:45 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:46:45.665 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:45.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:45.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:45.798 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:45.899 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:45.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:45.926 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:45.926 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:46.027 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:46.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:46.066 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:46.167 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:46.172 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 76 16 106223 106207 87.3245 59.5 0.015718 0.0114469 2024-09-11T08:46:46.196 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:46.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:46.197 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:46.280 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:46:46.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:46.298 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:46 smithi092 ceph-mon[29647]: pgmap v341: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 39 MiB/s wr, 617 op/s; 5586/305574 objects misplaced (1.828%); 2.5 MiB/s, 39 objects/s recovering 2024-09-11T08:46:46.298 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:46 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1516789989' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:46:46.299 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:46 smithi092 ceph-mon[41114]: pgmap v341: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 39 MiB/s wr, 617 op/s; 5586/305574 objects misplaced (1.828%); 2.5 MiB/s, 39 objects/s recovering 2024-09-11T08:46:46.299 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:46 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1516789989' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:46:46.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:46.330 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:46.330 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:46.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:46.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:46.468 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:46.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:46.604 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:46.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:46.605 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:46 smithi145 ceph-mon[33384]: pgmap v341: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 39 MiB/s wr, 617 op/s; 5586/305574 objects misplaced (1.828%); 2.5 MiB/s, 39 objects/s recovering 2024-09-11T08:46:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:46 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1516789989' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:46:46.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:46:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:46:46.706 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:46.752 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:46.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:46.754 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:46.855 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:46.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:46.895 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:46.895 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:46.996 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:47.021 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:47.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:47.022 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:47.123 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:47.153 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:47.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:47.157 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:47.172 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 77 16 107277 107261 87.0458 65.875 0.0145056 0.0114831 2024-09-11T08:46:47.258 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:47.289 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:47.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:47.290 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:47.391 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:47.429 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:47.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:47.430 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:47.531 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:47.563 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:47.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:47.564 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:47.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:47 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:46:47.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:47 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1516789989' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:46:47.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:47 smithi092 ceph-mon[29647]: osdmap e183: 8 total, 8 up, 8 in 2024-09-11T08:46:47.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:47 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:47.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:47 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:46:47.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:47 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1516789989' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:46:47.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:47 smithi092 ceph-mon[41114]: osdmap e183: 8 total, 8 up, 8 in 2024-09-11T08:46:47.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:47 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:47 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:46:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:47 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1516789989' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:46:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:47 smithi145 ceph-mon[33384]: osdmap e183: 8 total, 8 up, 8 in 2024-09-11T08:46:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:47 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:47.666 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:47.672 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:46:47.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:47.695 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:47.695 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:47.796 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:47.825 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:47.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:47.826 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:47.926 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:47.964 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:47.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:47.965 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:48.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:48.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:48.115 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:48.172 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 78 16 108334 108318 86.7763 66.0625 0.0174832 0.0115189 2024-09-11T08:46:48.216 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:48.255 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:48.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:48.256 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:48.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:48.363 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:48 smithi092 ceph-mon[29647]: pgmap v343: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 85 MiB/s wr, 1.36k op/s; 5261/319899 objects misplaced (1.645%); 6.7 MiB/s, 107 objects/s recovering 2024-09-11T08:46:48.364 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:48 smithi092 ceph-mon[41114]: pgmap v343: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 85 MiB/s wr, 1.36k op/s; 5261/319899 objects misplaced (1.645%); 6.7 MiB/s, 107 objects/s recovering 2024-09-11T08:46:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:48.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:48.397 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:48.498 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:48.526 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:48.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:48.527 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:48.628 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:48.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:48 smithi145 ceph-mon[33384]: pgmap v343: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 85 MiB/s wr, 1.36k op/s; 5261/319899 objects misplaced (1.645%); 6.7 MiB/s, 107 objects/s recovering 2024-09-11T08:46:48.673 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:48.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:48.674 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:48.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:48.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:48.822 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:48.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:48.924 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:46:48.925 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:46:48.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:48.964 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:48.964 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:48.966 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:46:48.966 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:46:49.001 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:46:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:46:49.033 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:46:49.034 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:46:49.065 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:49.071 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:46:49.072 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:46:49.097 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:49.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:49.098 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:49.105 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:46:49.105 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:46:49.137 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:46:49.137 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:46:49.172 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 79 16 109313 109297 86.4524 61.1875 0.0148086 0.0115613 2024-09-11T08:46:49.175 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:46:49.175 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:46:49.199 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:49.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:49.229 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:49.229 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:49.330 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:49.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:49.359 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:49.359 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:49.460 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:49.503 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:49.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:49.504 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:49.605 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:49.630 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:49.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:49.632 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:49.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:46:49.733 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:49.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:49.790 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:49.790 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:49.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:49.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:49.946 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:50.014 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:49 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/98143211' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:46:50.014 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:49 smithi092 ceph-mon[29647]: pgmap v344: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 739 B/s rd, 43 MiB/s wr, 690 op/s; 5261/319899 objects misplaced (1.645%); 2.9 MiB/s, 46 objects/s recovering 2024-09-11T08:46:50.015 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:49 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/98143211' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:46:50.015 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:49 smithi092 ceph-mon[41114]: pgmap v344: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 739 B/s rd, 43 MiB/s wr, 690 op/s; 5261/319899 objects misplaced (1.645%); 2.9 MiB/s, 46 objects/s recovering 2024-09-11T08:46:50.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:50.103 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:50.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:50.104 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:50.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:49 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/98143211' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:46:50.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:49 smithi145 ceph-mon[33384]: pgmap v344: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 739 B/s rd, 43 MiB/s wr, 690 op/s; 5261/319899 objects misplaced (1.645%); 2.9 MiB/s, 46 objects/s recovering 2024-09-11T08:46:50.172 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:46:50.174007+0000 min lat: 0.00108577 max lat: 2.31233 avg lat: 0.0116069 2024-09-11T08:46:50.173 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:46:50.173 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 80 16 110268 110252 86.1178 59.6875 0.00887443 0.0116069 2024-09-11T08:46:50.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:50.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:50.240 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:50.341 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:50.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:50.376 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:50.377 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:50.478 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:50.504 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:50.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:50.505 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:50.606 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:50.657 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:50.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:50.658 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:50.759 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:50.786 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:50.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:50.787 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:50.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:50.936 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:50.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:50.937 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:51.038 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:51.063 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:51.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:51.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:51.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:51.173 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 81 16 111243 111227 85.8063 60.9375 0.0153741 0.011648 2024-09-11T08:46:51.180 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:46:51.180 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"211625d4-7018-11ef-bcea-c7b262605968","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":705,"monmap":{"epoch":3,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":183,"num_osds":8,"num_up_osds":8,"osd_up_since":1726043903,"num_in_osds":8,"osd_in_since":1726043886,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":146},{"state_name":"active+recovering+remapped","count":1},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":148,"num_pools":7,"num_objects":106633,"data_bytes":6976051420,"bytes_used":23639314432,"bytes_avail":744320970752,"bytes_total":767960285184,"misplaced_objects":5261,"misplaced_total":319899,"misplaced_ratio":0.016445815710583653,"recovering_objects_per_sec":46,"recovering_bytes_per_sec":3075057,"recovering_keys_per_sec":0,"num_objects_recovered":325,"num_bytes_recovered":21299200,"num_keys_recovered":0,"read_bytes_sec":739,"write_bytes_sec":45179690,"read_op_per_sec":0,"write_op_per_sec":689},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.92:8443/","prometheus":"http://172.21.15.92:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-11T08:40:17.013941+0000","services":{"rgw":{"daemons":{"summary":"","24352":{"start_epoch":12,"start_stamp":"2024-09-11T08:38:46.107187+0000","gid":24352,"addr":"172.21.15.92:0/1175084397","metadata":{"arch":"x86_64","ceph_release":"quincy","ceph_version":"ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)","ceph_version_short":"17.2.7-1623-gd0e6828a","container_hostname":"smithi092","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40","cpu":"Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi092","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024","kernel_version":"5.14.0-505.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32487284","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"02abb3ce-34bc-4920-b6fd-8ad4b7819694","zone_name":"default","zonegroup_id":"8abc369a-51df-4e8f-b783-2e835134b342","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T08:46:51.207 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:51.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:51.208 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:51.309 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:51.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:51 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3556495347' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-11T08:46:51.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:51 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3556495347' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-11T08:46:51.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:51.354 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:51.354 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:51.455 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:51.492 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:51.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:51.493 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:51.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:51.634 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:51.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:51.636 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:51.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:51 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3556495347' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-11T08:46:51.710 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:46:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:46:51] "GET /metrics HTTP/1.1" 200 224282 "" "Prometheus/2.43.0" 2024-09-11T08:46:51.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:51.780 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:51.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:51.781 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:51.882 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:51.921 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:51.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:51.922 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:52.023 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:52.062 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:52.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:52.063 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:52.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:52.173 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 82 16 112299 112283 85.5648 66 0.00695952 0.0116817 2024-09-11T08:46:52.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:52.200 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:52.200 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:52.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:52.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:52.339 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:52.339 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:52.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:52.480 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:52.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:52.481 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:52.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:52 smithi092 ceph-mon[29647]: pgmap v345: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.6 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 MiB/s wr, 799 op/s; 5261/324744 objects misplaced (1.620%); 2.5 MiB/s, 40 objects/s recovering 2024-09-11T08:46:52.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:52 smithi092 ceph-mon[41114]: pgmap v345: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.6 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 MiB/s wr, 799 op/s; 5261/324744 objects misplaced (1.620%); 2.5 MiB/s, 40 objects/s recovering 2024-09-11T08:46:52.582 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:52.616 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:52.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:52.617 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:52 smithi145 ceph-mon[33384]: pgmap v345: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.6 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 MiB/s wr, 799 op/s; 5261/324744 objects misplaced (1.620%); 2.5 MiB/s, 40 objects/s recovering 2024-09-11T08:46:52.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:52.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:52.764 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:52.764 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:52.865 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:52.896 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:52.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:52.897 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:52.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:53.042 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:53.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:53.044 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:53.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:53.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:46:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:46:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:46:53.173 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 83 16 113393 113377 85.3576 68.375 0.0244827 0.0117107 2024-09-11T08:46:53.184 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:53.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:53.185 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:53.286 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:53.300 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-11T08:46:53.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:46:53.316 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:53.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:53.317 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:53.418 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:53.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:53.468 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:53.468 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:53.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:53.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:46:53.583 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:46:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:46:53.591 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:46:53.591 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:46:53.591 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:46:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:46:53.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:53.643 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:53.643 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:53.744 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:53.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:53.772 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:53.772 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:53.873 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:53.902 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:53.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:53.903 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:54.003 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:54.047 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:54.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:54.048 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:54.149 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:54.173 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 84 16 114303 114287 85.0185 56.875 0.0112973 0.0117562 2024-09-11T08:46:54.176 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:54.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:54.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:54.277 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:54.305 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:54.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:54.308 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:54.408 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:54.435 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:54.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:54.436 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:54.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-11T08:46:54.494 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:54 smithi092 ceph-mon[29647]: pgmap v346: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 69 MiB/s wr, 1.10k op/s; 4920/335001 objects misplaced (1.469%); 4.7 MiB/s, 74 objects/s recovering 2024-09-11T08:46:54.494 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:54 smithi092 ceph-mon[41114]: pgmap v346: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 69 MiB/s wr, 1.10k op/s; 4920/335001 objects misplaced (1.469%); 4.7 MiB/s, 74 objects/s recovering 2024-09-11T08:46:54.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:54.579 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:54.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:54.580 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:54.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:54 smithi145 ceph-mon[33384]: pgmap v346: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 69 MiB/s wr, 1.10k op/s; 4920/335001 objects misplaced (1.469%); 4.7 MiB/s, 74 objects/s recovering 2024-09-11T08:46:54.681 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:54.711 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:54.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:54.712 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:54.813 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:54.838 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:54.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:54.839 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:54.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:54.981 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:54.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:54.982 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:55.083 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:55.116 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:55.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:55.117 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:55.173 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 85 16 115273 115257 84.7315 60.625 0.0149961 0.0117969 2024-09-11T08:46:55.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:55.255 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:55.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:55.256 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:55.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:55.412 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:55.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:55.413 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:55.514 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:55.540 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:55.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:55.541 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:55.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:55 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:46:55.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:55 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:46:55.641 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:55.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:55 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:46:55.673 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:55.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:55.674 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:55.775 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:55.799 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:55.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:55.800 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:55.901 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:55.934 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:55.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:55.936 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:56.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:56.057 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:56.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:56.058 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:56.159 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:56.173 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 86 16 116214 116198 84.4301 58.8125 0.0151403 0.0118382 2024-09-11T08:46:56.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:56.195 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:56.195 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:56.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:56.331 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:56.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:56.333 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:56.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:56.470 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:56.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:56.471 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:56.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:56 smithi092 ceph-mon[29647]: pgmap v347: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 61 MiB/s wr, 981 op/s; 4920/335001 objects misplaced (1.469%); 4.2 MiB/s, 66 objects/s recovering 2024-09-11T08:46:56.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:56 smithi092 ceph-mon[41114]: pgmap v347: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 61 MiB/s wr, 981 op/s; 4920/335001 objects misplaced (1.469%); 4.2 MiB/s, 66 objects/s recovering 2024-09-11T08:46:56.572 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:56.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:56.605 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:56.605 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:56.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:56 smithi145 ceph-mon[33384]: pgmap v347: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 61 MiB/s wr, 981 op/s; 4920/335001 objects misplaced (1.469%); 4.2 MiB/s, 66 objects/s recovering 2024-09-11T08:46:56.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:46:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:46:56.706 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:56.734 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:56.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:56.735 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:56.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:56.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:56.858 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:56.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:56.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:56.998 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:57.099 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:57.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:57.122 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:57.122 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:57.174 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 87 16 117153 117137 84.1342 58.6875 0.0162412 0.0118809 2024-09-11T08:46:57.222 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:46:57.263 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:57.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:57.264 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:46:57.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:57 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:57.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:57 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:57.365 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:57.402 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:57.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:57.403 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:57.504 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:57.525 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:57.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:57.526 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:57.627 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:57.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:57.655 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:57.655 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:57 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:57.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:57.792 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:57.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:57.794 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:57.894 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:57.926 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:57.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:57.927 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:46:58.028 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:58.066 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:58.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:58.067 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:58.168 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:46:58.173 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 88 16 118149 118133 83.8855 62.25 0.0133252 0.0119155 2024-09-11T08:46:58.198 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:58.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:58.199 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:46:58.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:58.321 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:58.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:58.322 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:58.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:58.456 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:58.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:58.457 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:58.558 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:58.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:46:58 smithi145 ceph-mon[33384]: pgmap v348: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 7.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 940 B/s rd, 76 MiB/s wr, 1.22k op/s; 4568/349926 objects misplaced (1.305%); 5.8 MiB/s, 93 objects/s recovering 2024-09-11T08:46:58.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:46:58 smithi092 ceph-mon[29647]: pgmap v348: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 7.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 940 B/s rd, 76 MiB/s wr, 1.22k op/s; 4568/349926 objects misplaced (1.305%); 5.8 MiB/s, 93 objects/s recovering 2024-09-11T08:46:58.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:46:58 smithi092 ceph-mon[41114]: pgmap v348: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 7.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 940 B/s rd, 76 MiB/s wr, 1.22k op/s; 4568/349926 objects misplaced (1.305%); 5.8 MiB/s, 93 objects/s recovering 2024-09-11T08:46:58.579 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:58.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:58.580 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:58.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:58.713 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:58.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:58.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:58.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:58.858 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:58.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:58.860 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:58.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:46:58.983 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:58.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:58.984 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:46:59.085 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:59.117 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:59.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:59.118 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:59.174 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 89 16 119039 119023 83.5679 55.625 0.00323899 0.0119592 2024-09-11T08:46:59.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:46:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:59.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:59.258 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:46:59.359 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:59.395 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:59.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:59.396 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:59.497 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:46:59.530 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:59.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:59.531 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:46:59.632 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:46:59.670 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:59.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:59.671 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:46:59.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:46:59.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:59.808 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:46:59.808 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:46:59.909 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:46:59.944 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:46:59.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:46:59.945 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:00.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:00.090 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:00.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:00.091 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:00.174 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Total time run: 90.0101 2024-09-11T08:47:00.174 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Total writes made: 119958 2024-09-11T08:47:00.174 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Write size: 65536 2024-09-11T08:47:00.174 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Object size: 65536 2024-09-11T08:47:00.174 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Bandwidth (MB/sec): 83.2949 2024-09-11T08:47:00.174 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Stddev Bandwidth: 41.3534 2024-09-11T08:47:00.174 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Max bandwidth (MB/sec): 233.938 2024-09-11T08:47:00.175 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Min bandwidth (MB/sec): 0 2024-09-11T08:47:00.175 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Average IOPS: 1332 2024-09-11T08:47:00.175 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Stddev IOPS: 661.655 2024-09-11T08:47:00.175 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Max IOPS: 3743 2024-09-11T08:47:00.175 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Min IOPS: 0 2024-09-11T08:47:00.175 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Average Latency(s): 0.0120013 2024-09-11T08:47:00.175 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Stddev Latency(s): 0.0278353 2024-09-11T08:47:00.175 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Max latency(s): 2.31233 2024-09-11T08:47:00.175 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Min latency(s): 0.00108577 2024-09-11T08:47:00.181 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Cleaning up (deleting benchmark objects) 2024-09-11T08:47:00.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:00.224 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:00.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:00.225 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:00.312 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:00.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:00.326 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:00.326 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:00.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:00.377 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:00.378 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:00.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:00.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:00 smithi092 ceph-mon[41114]: pgmap v349: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 7.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 52 MiB/s wr, 834 op/s; 4568/349926 objects misplaced (1.305%); 3.6 MiB/s, 57 objects/s recovering 2024-09-11T08:47:00.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:00.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:00 smithi092 ceph-mon[29647]: pgmap v349: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 7.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 52 MiB/s wr, 834 op/s; 4568/349926 objects misplaced (1.305%); 3.6 MiB/s, 57 objects/s recovering 2024-09-11T08:47:00.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:00.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:00.530 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:00.531 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:00.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:00.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:00 smithi145 ceph-mon[33384]: pgmap v349: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 7.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 52 MiB/s wr, 834 op/s; 4568/349926 objects misplaced (1.305%); 3.6 MiB/s, 57 objects/s recovering 2024-09-11T08:47:00.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:00.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:00.680 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:00.680 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:00.781 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:00.816 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:00.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:00.817 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:00.917 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:00.944 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:01.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:01.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:01.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:01.154 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:01.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:01.155 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:01.255 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:01.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:01.284 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:01.284 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:01.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:01 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:47:01.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:01 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:47:01.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:01 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:01.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:01 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:47:01.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:01 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:47:01.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:01 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:01.385 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:01.411 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:01.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:01.412 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:01.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:01.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:01.562 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:01.563 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:01.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:01 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:47:01.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:01 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:47:01.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:01 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:01.663 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:01.704 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:01.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:01.705 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:01.806 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:01.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:47:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:47:01] "GET /metrics HTTP/1.1" 200 224289 "" "Prometheus/2.43.0" 2024-09-11T08:47:01.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:01.847 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:01.847 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:01.948 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:01.994 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:01.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:01.995 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:02.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:02.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:02.137 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:02.137 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:02.238 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:02.268 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:02.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:02.269 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:02.371 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:02.399 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:02.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:02.400 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:02.501 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:02.533 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:02.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:02.534 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:02.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:02 smithi092 ceph-mon[29647]: pgmap v350: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.3 KiB/s rd, 58 MiB/s wr, 957 op/s; 4568/352671 objects misplaced (1.295%); 3.6 MiB/s, 57 objects/s recovering 2024-09-11T08:47:02.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:02 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2547165081' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:47:02.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:02 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:47:02.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:02 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:47:02.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:02 smithi092 ceph-mon[41114]: pgmap v350: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.3 KiB/s rd, 58 MiB/s wr, 957 op/s; 4568/352671 objects misplaced (1.295%); 3.6 MiB/s, 57 objects/s recovering 2024-09-11T08:47:02.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:02 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2547165081' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:47:02.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:02 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:47:02.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:02 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:47:02.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:02.650 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:47:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:02 smithi145 ceph-mon[33384]: pgmap v350: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.3 KiB/s rd, 58 MiB/s wr, 957 op/s; 4568/352671 objects misplaced (1.295%); 3.6 MiB/s, 57 objects/s recovering 2024-09-11T08:47:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:02 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2547165081' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:47:02.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:02 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:47:02.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:02 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:47:02.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:02.680 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:02.680 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:02.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:02.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:02.818 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:02.919 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:02.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:02.958 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:02.958 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:03.059 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:03.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:03.090 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:03.091 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:03.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:47:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:47:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:47:03.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:03.229 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:03.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:03.230 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:03.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:03.368 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:03.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:03.369 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:03.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:03.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:03.503 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:03.503 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:03.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:03 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:47:03.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:03 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2547165081' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:47:03.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:03 smithi092 ceph-mon[29647]: osdmap e184: 8 total, 8 up, 8 in 2024-09-11T08:47:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:03 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:47:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:03 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2547165081' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:47:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:03 smithi092 ceph-mon[41114]: osdmap e184: 8 total, 8 up, 8 in 2024-09-11T08:47:03.604 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:03.634 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:03.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:03.635 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:03.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:03 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:47:03.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:03 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2547165081' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:47:03.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:03 smithi145 ceph-mon[33384]: osdmap e184: 8 total, 8 up, 8 in 2024-09-11T08:47:03.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:03.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:03.779 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:03.779 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:03.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:03.913 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:03.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:03.914 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:04.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:04.054 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:04.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:04.055 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:04.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:04.190 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:04.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:04.192 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:04.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:04.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:04.332 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:04.332 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:04.433 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:04.460 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:04.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:04.461 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:04.562 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:04.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:04 smithi092 ceph-mon[29647]: 2.2 scrub starts 2024-09-11T08:47:04.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:04 smithi092 ceph-mon[29647]: pgmap v352: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 53 MiB/s wr, 972 op/s; 4228/356790 objects misplaced (1.185%); 4.3 MiB/s, 69 objects/s recovering 2024-09-11T08:47:04.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:04 smithi092 ceph-mon[29647]: Health check failed: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-09-11T08:47:04.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:04 smithi092 ceph-mon[41114]: 2.2 scrub starts 2024-09-11T08:47:04.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:04 smithi092 ceph-mon[41114]: pgmap v352: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 53 MiB/s wr, 972 op/s; 4228/356790 objects misplaced (1.185%); 4.3 MiB/s, 69 objects/s recovering 2024-09-11T08:47:04.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:04 smithi092 ceph-mon[41114]: Health check failed: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-09-11T08:47:04.600 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:04.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:04.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:04.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:04 smithi145 ceph-mon[33384]: 2.2 scrub starts 2024-09-11T08:47:04.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:04 smithi145 ceph-mon[33384]: pgmap v352: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 53 MiB/s wr, 972 op/s; 4228/356790 objects misplaced (1.185%); 4.3 MiB/s, 69 objects/s recovering 2024-09-11T08:47:04.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:04 smithi145 ceph-mon[33384]: Health check failed: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-09-11T08:47:04.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:04.742 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:04.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:04.743 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:04.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:04.876 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:04.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:04.877 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:04.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:05.001 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:05.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:05.002 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:05.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:05.142 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:05.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:05.144 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:05.245 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:05.287 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:05.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:05.289 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:05.389 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:05.428 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:05.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:05.429 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:05.530 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:05.559 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:05.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:05.560 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:05.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:05 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T08:47:05.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:05 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T08:47:05.661 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:05.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:05 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T08:47:05.686 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:05.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:05.688 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:05.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:05.817 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:05.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:05.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:05.919 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:05.948 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:05.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:05.950 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:06.050 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:06.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:06.081 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:06.081 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:06.182 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:06.211 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:06.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:06.212 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:06.313 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:06.339 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:06.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:06.340 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:06.441 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:06.475 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:06.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:06.476 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:06.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:06 smithi092 ceph-mon[29647]: pgmap v353: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 53 MiB/s wr, 972 op/s; 4228/356790 objects misplaced (1.185%); 4.3 MiB/s, 69 objects/s recovering 2024-09-11T08:47:06.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:06 smithi092 ceph-mon[41114]: pgmap v353: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 53 MiB/s wr, 972 op/s; 4228/356790 objects misplaced (1.185%); 4.3 MiB/s, 69 objects/s recovering 2024-09-11T08:47:06.577 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:06.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:06.618 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:06.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:06 smithi145 ceph-mon[33384]: pgmap v353: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 53 MiB/s wr, 972 op/s; 4228/356790 objects misplaced (1.185%); 4.3 MiB/s, 69 objects/s recovering 2024-09-11T08:47:06.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:47:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:47:06.718 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:06.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:06.748 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:06.749 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:06.849 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:06.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:06.886 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:06.887 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:06.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:07.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:07.026 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:07.026 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:07.127 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:07.154 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:07.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:07.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:07.256 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:07.281 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:07.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:07.281 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:07.382 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:07.412 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:07.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:07.413 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:07.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:07.549 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:07.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:07.550 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:07.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:07 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:47:07.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:07 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:47:07.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:07 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:07.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:07 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:47:07.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:07 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:47:07.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:07 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:07.651 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:07.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:07 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:47:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:07 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:47:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:07 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:07.672 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:07.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:07.673 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:07.774 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:07.810 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:07.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:07.811 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:07.912 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:07.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:07.945 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:07.945 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:08.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:08.085 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:08.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:08.086 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:08.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:08.223 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:08.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:08.224 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:08.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:08.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:08.357 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:08.357 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:08.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:08.488 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:08.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:08.489 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:08.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:08 smithi092 ceph-mon[41114]: pgmap v354: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 22 MiB/s wr, 1.01k op/s; 3829/340608 objects misplaced (1.124%); 4.6 MiB/s, 73 objects/s recovering 2024-09-11T08:47:08.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:08 smithi092 ceph-mon[41114]: osdmap e185: 8 total, 8 up, 8 in 2024-09-11T08:47:08.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:08 smithi092 ceph-mon[29647]: pgmap v354: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 22 MiB/s wr, 1.01k op/s; 3829/340608 objects misplaced (1.124%); 4.6 MiB/s, 73 objects/s recovering 2024-09-11T08:47:08.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:08 smithi092 ceph-mon[29647]: osdmap e185: 8 total, 8 up, 8 in 2024-09-11T08:47:08.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:08.622 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:08.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:08.623 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:08.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:08 smithi145 ceph-mon[33384]: pgmap v354: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 22 MiB/s wr, 1.01k op/s; 3829/340608 objects misplaced (1.124%); 4.6 MiB/s, 73 objects/s recovering 2024-09-11T08:47:08.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:08 smithi145 ceph-mon[33384]: osdmap e185: 8 total, 8 up, 8 in 2024-09-11T08:47:08.724 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:08.757 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:08.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:08.758 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:08.859 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:08.889 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:08.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:08.890 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:08.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:09.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:09.020 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:09.021 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:09.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:09.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:09.158 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:09.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:09.297 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:09.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:09.298 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:09.399 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:09.425 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:09.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:09.427 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:09.527 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:09.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:09.564 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:09.665 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:09.695 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:09.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:09.696 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:09.797 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:09.822 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:09.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:09.823 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:09.924 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:09.951 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:09.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:09.952 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:10.004 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:09 smithi092 ceph-mon[29647]: pgmap v356: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 18 MiB/s wr, 1.08k op/s; 3829/340608 objects misplaced (1.124%); 5.8 MiB/s, 92 objects/s recovering 2024-09-11T08:47:10.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:09 smithi092 ceph-mon[41114]: pgmap v356: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 18 MiB/s wr, 1.08k op/s; 3829/340608 objects misplaced (1.124%); 5.8 MiB/s, 92 objects/s recovering 2024-09-11T08:47:10.053 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:10.083 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:10.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:10.084 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:10.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:09 smithi145 ceph-mon[33384]: pgmap v356: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 18 MiB/s wr, 1.08k op/s; 3829/340608 objects misplaced (1.124%); 5.8 MiB/s, 92 objects/s recovering 2024-09-11T08:47:10.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:10.223 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:10.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:10.224 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:10.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:10.361 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:10.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:10.362 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:10.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:10.501 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:10.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:10.502 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:10.603 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:10.632 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:10.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:10.633 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:10.734 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:10.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:47:10.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:47:10.744 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:47:10.744 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:47:10.768 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:10.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:10.769 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:10.779 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:47:10.779 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:47:10.813 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:47:10.813 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:47:10.846 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:47:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:47:10.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:10.882 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:47:10.882 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:47:10.917 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:10.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:10.918 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:10.927 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:47:10.927 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:47:10.928 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:10 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T08:47:10.928 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:10 smithi092 ceph-mon[29647]: 5.1b scrub starts 2024-09-11T08:47:10.928 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:10 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/140902105' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:47:10.928 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:10 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T08:47:10.928 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:10 smithi092 ceph-mon[41114]: 5.1b scrub starts 2024-09-11T08:47:10.928 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:10 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/140902105' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:47:10.962 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:47:10.963 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:47:11.013 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:47:11.014 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:47:11.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:11.062 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:11.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:11.063 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:11.137 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:47:11.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:10 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T08:47:11.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:10 smithi145 ceph-mon[33384]: 5.1b scrub starts 2024-09-11T08:47:11.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:10 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/140902105' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:47:11.164 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:11.193 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:11.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:11.194 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:11.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:11.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:11.333 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:11.333 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:11.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:11.485 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:11.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:11.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:11.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:47:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:47:11] "GET /metrics HTTP/1.1" 200 224289 "" "Prometheus/2.43.0" 2024-09-11T08:47:11.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:11.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:11.619 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:11.620 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:11.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:11.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:11.754 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:11.855 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:11.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:11.885 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:11.886 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:11.986 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:12.015 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:12.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:12.016 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:12.117 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:12.146 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:12.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:12.147 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:12.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:11 smithi145 ceph-mon[33384]: pgmap v357: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 843 op/s; 3829/335250 objects misplaced (1.142%); 2.9 MiB/s, 46 objects/s recovering 2024-09-11T08:47:12.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:12.285 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:12.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:12.286 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:12.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:11 smithi092 ceph-mon[29647]: pgmap v357: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 843 op/s; 3829/335250 objects misplaced (1.142%); 2.9 MiB/s, 46 objects/s recovering 2024-09-11T08:47:12.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:11 smithi092 ceph-mon[41114]: pgmap v357: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 843 op/s; 3829/335250 objects misplaced (1.142%); 2.9 MiB/s, 46 objects/s recovering 2024-09-11T08:47:12.387 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:12.413 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:12.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:12.414 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:12.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:12.552 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:12.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:12.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:12.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:12.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:12.696 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:12.696 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:12.798 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:12.828 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:12.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:12.829 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:12.930 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:12.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:12.966 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:12.966 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:13.019 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:47:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:47:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:47:13.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:13.110 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:13.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:13.111 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:13.212 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:13.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:13.253 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:13.253 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:13.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:13 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:47:13.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:13 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:47:13.354 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:13.381 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:13.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:13.382 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:13.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:13 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:47:13.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-11T08:47:13.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:13.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:13.536 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:13.536 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:13.637 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:13.660 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:13.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:13.661 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:13.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:13.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:13.817 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:13.817 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:13.919 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:13.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:13.970 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:14.071 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:14.096 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:14.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:14.097 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:14.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:14.238 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:14.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:14.239 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:14.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:14 smithi092 ceph-mon[29647]: 6.7 scrub starts 2024-09-11T08:47:14.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:14 smithi092 ceph-mon[29647]: pgmap v358: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.6 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s; 3396/324240 objects misplaced (1.047%); 5.2 MiB/s, 83 objects/s recovering 2024-09-11T08:47:14.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:14 smithi092 ceph-mon[41114]: 6.7 scrub starts 2024-09-11T08:47:14.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:14 smithi092 ceph-mon[41114]: pgmap v358: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.6 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s; 3396/324240 objects misplaced (1.047%); 5.2 MiB/s, 83 objects/s recovering 2024-09-11T08:47:14.340 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:14.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:14.379 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:14.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:14 smithi145 ceph-mon[33384]: 6.7 scrub starts 2024-09-11T08:47:14.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:14 smithi145 ceph-mon[33384]: pgmap v358: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.6 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s; 3396/324240 objects misplaced (1.047%); 5.2 MiB/s, 83 objects/s recovering 2024-09-11T08:47:14.480 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:14.509 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:14.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:14.510 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:14.610 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:14.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:14.654 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:14.654 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:14.755 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:14.782 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:14.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:14.783 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:14.884 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:14.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:14.936 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:14.936 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:15.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:15.086 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:15.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:15.087 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:15.188 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:15.215 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:15.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:15.216 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:15.290 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:15 smithi092 ceph-mon[29647]: 4.8 scrub starts 2024-09-11T08:47:15.290 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:15 smithi092 ceph-mon[29647]: 5.b scrub starts 2024-09-11T08:47:15.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:15 smithi092 ceph-mon[41114]: 4.8 scrub starts 2024-09-11T08:47:15.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:15 smithi092 ceph-mon[41114]: 5.b scrub starts 2024-09-11T08:47:15.318 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:15.343 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:15.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:15.344 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:15.363 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:15.388 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:47:15.393 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:47:15.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:15 smithi145 ceph-mon[33384]: 4.8 scrub starts 2024-09-11T08:47:15.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:15 smithi145 ceph-mon[33384]: 5.b scrub starts 2024-09-11T08:47:15.446 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:15.476 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:15.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:15.477 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:15.577 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:15.616 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:15.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:15.617 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:15.718 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:15.754 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:15.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:15.755 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:15.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:15.891 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:15.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:15.892 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:15.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:47:15.993 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:16.025 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:16.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:16.026 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:16.126 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:16.158 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:16.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:16.159 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:16.260 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:16.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:16.296 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:16.296 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:16.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:16 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:16.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:16 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:16.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:16 smithi092 ceph-mon[29647]: pgmap v359: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.6 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s; 3396/324240 objects misplaced (1.047%); 5.2 MiB/s, 83 objects/s recovering 2024-09-11T08:47:16.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:16 smithi092 ceph-mon[29647]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:47:16.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:16 smithi092 ceph-mon[29647]: 5.6 scrub starts 2024-09-11T08:47:16.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:16 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:16.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:16 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:16.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:16 smithi092 ceph-mon[41114]: pgmap v359: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.6 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s; 3396/324240 objects misplaced (1.047%); 5.2 MiB/s, 83 objects/s recovering 2024-09-11T08:47:16.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:16 smithi092 ceph-mon[41114]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:47:16.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:16 smithi092 ceph-mon[41114]: 5.6 scrub starts 2024-09-11T08:47:16.397 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:16.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:16 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:16.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:16 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:16.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:16 smithi145 ceph-mon[33384]: pgmap v359: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.6 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s; 3396/324240 objects misplaced (1.047%); 5.2 MiB/s, 83 objects/s recovering 2024-09-11T08:47:16.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:16 smithi145 ceph-mon[33384]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:47:16.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:16 smithi145 ceph-mon[33384]: 5.6 scrub starts 2024-09-11T08:47:16.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:16.448 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:16.448 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:16.534 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-11T08:47:15.179601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7070816500,"num_objects":108080,"num_object_clones":0,"num_object_copies":324240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3396,"num_objects_unfound":0,"num_objects_dirty":108080,"num_whiteouts":0,"num_read":1356,"num_read_kb":1213,"num_write":132577,"num_write_kb":7678984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4477,"num_bytes_recovered":287180944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85350,"ondisk_log_size":85350,"up":444,"acting":443,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":446,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":23734928,"kb_used_data":21293584,"kb_used_omap":13,"kb_used_meta":2441138,"kb_avail":726226288,"statfs":{"total":767960285184,"available":743655718912,"internally_reserved":0,"allocated":21804630016,"data_stored":21802142436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2499726016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,5,6,3,1,1,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-711065600,"num_objects":-10850,"num_object_clones":0,"num_object_copies":-32550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-832,"num_objects_unfound":0,"num_objects_dirty":-10850,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10850,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":832,"num_bytes_recovered":54525952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.007148"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":178,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.664743+0000","last_change":"2024-09-11T08:45:06.574593+0000","last_active":"2024-09-11T08:47:02.664743+0000","last_peered":"2024-09-11T08:47:02.664743+0000","last_clean":"2024-09-11T08:47:02.664743+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:47:02.664743+0000","last_undegraded":"2024-09-11T08:47:02.664743+0000","last_fullsized":"2024-09-11T08:47:02.664743+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:06.574539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.574539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028477630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":226,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654433+0000","last_change":"2024-09-11T08:46:01.991434+0000","last_active":"2024-09-11T08:47:02.654433+0000","last_peered":"2024-09-11T08:47:02.654433+0000","last_clean":"2024-09-11T08:47:02.654433+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:47:02.654433+0000","last_undegraded":"2024-09-11T08:47:02.654433+0000","last_fullsized":"2024-09-11T08:47:02.654433+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:45:13.113315+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.113315+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018068470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":174,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653607+0000","last_change":"2024-09-11T08:44:25.796953+0000","last_active":"2024-09-11T08:47:02.653607+0000","last_peered":"2024-09-11T08:47:02.653607+0000","last_clean":"2024-09-11T08:47:02.653607+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:47:02.653607+0000","last_undegraded":"2024-09-11T08:47:02.653607+0000","last_fullsized":"2024-09-11T08:47:02.653607+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:25.796699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:25.796699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025405829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":152,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654874+0000","last_change":"2024-09-11T08:44:26.005322+0000","last_active":"2024-09-11T08:47:02.654874+0000","last_peered":"2024-09-11T08:47:02.654874+0000","last_clean":"2024-09-11T08:47:02.654874+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:47:02.654874+0000","last_undegraded":"2024-09-11T08:47:02.654874+0000","last_fullsized":"2024-09-11T08:47:02.654874+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.005270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.005270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038675070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":181,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.662755+0000","last_change":"2024-09-11T08:46:01.984446+0000","last_active":"2024-09-11T08:47:02.662755+0000","last_peered":"2024-09-11T08:47:02.662755+0000","last_clean":"2024-09-11T08:47:02.662755+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:47:02.662755+0000","last_undegraded":"2024-09-11T08:47:02.662755+0000","last_fullsized":"2024-09-11T08:47:02.662755+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:28.356172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.356172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029955870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":187,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.665725+0000","last_change":"2024-09-11T08:46:01.981971+0000","last_active":"2024-09-11T08:47:02.665725+0000","last_peered":"2024-09-11T08:47:02.665725+0000","last_clean":"2024-09-11T08:47:02.665725+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:47:02.665725+0000","last_undegraded":"2024-09-11T08:47:02.665725+0000","last_fullsized":"2024-09-11T08:47:02.665725+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.330642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.330642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0087830849999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":197,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.664347+0000","last_change":"2024-09-11T08:44:24.344328+0000","last_active":"2024-09-11T08:47:02.664347+0000","last_peered":"2024-09-11T08:47:02.664347+0000","last_clean":"2024-09-11T08:47:02.664347+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:47:02.664347+0000","last_undegraded":"2024-09-11T08:47:02.664347+0000","last_fullsized":"2024-09-11T08:47:02.664347+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:44:24.344127+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.344127+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010996336000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":154,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654424+0000","last_change":"2024-09-11T08:43:28.988751+0000","last_active":"2024-09-11T08:47:02.654424+0000","last_peered":"2024-09-11T08:47:02.654424+0000","last_clean":"2024-09-11T08:47:02.654424+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:47:02.654424+0000","last_undegraded":"2024-09-11T08:47:02.654424+0000","last_fullsized":"2024-09-11T08:47:02.654424+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:43:28.988697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00375019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":191,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.650076+0000","last_change":"2024-09-11T08:46:17.669115+0000","last_active":"2024-09-11T08:47:02.650076+0000","last_peered":"2024-09-11T08:47:02.650076+0000","last_clean":"2024-09-11T08:47:02.650076+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:47:02.650076+0000","last_undegraded":"2024-09-11T08:47:02.650076+0000","last_fullsized":"2024-09-11T08:47:02.650076+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:12.878132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.878132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077623500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":186,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655020+0000","last_change":"2024-09-11T08:44:27.963772+0000","last_active":"2024-09-11T08:47:02.655020+0000","last_peered":"2024-09-11T08:47:02.655020+0000","last_clean":"2024-09-11T08:47:02.655020+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:47:02.655020+0000","last_undegraded":"2024-09-11T08:47:02.655020+0000","last_fullsized":"2024-09-11T08:47:02.655020+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:44:27.963689+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:44:27.963689+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011544877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":170,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:09.804752+0000","last_change":"2024-09-11T08:47:09.804752+0000","last_active":"2024-09-11T08:47:09.804752+0000","last_peered":"2024-09-11T08:47:09.804752+0000","last_clean":"2024-09-11T08:47:09.804752+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:47:09.804752+0000","last_undegraded":"2024-09-11T08:47:09.804752+0000","last_fullsized":"2024-09-11T08:47:09.804752+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:33.644869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.644869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020832210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":172,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.663449+0000","last_change":"2024-09-11T08:45:31.457775+0000","last_active":"2024-09-11T08:47:02.663449+0000","last_peered":"2024-09-11T08:47:02.663449+0000","last_clean":"2024-09-11T08:47:02.663449+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:47:02.663449+0000","last_undegraded":"2024-09-11T08:47:02.663449+0000","last_fullsized":"2024-09-11T08:47:02.663449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.457652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.457652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006759045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":171,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.650405+0000","last_change":"2024-09-11T08:46:22.625087+0000","last_active":"2024-09-11T08:47:02.650405+0000","last_peered":"2024-09-11T08:47:02.650405+0000","last_clean":"2024-09-11T08:47:02.650405+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:47:02.650405+0000","last_undegraded":"2024-09-11T08:47:02.650405+0000","last_fullsized":"2024-09-11T08:47:02.650405+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:09.871637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:09.871637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029269349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":166,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655235+0000","last_change":"2024-09-11T08:43:26.983719+0000","last_active":"2024-09-11T08:47:02.655235+0000","last_peered":"2024-09-11T08:47:02.655235+0000","last_clean":"2024-09-11T08:47:02.655235+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:47:02.655235+0000","last_undegraded":"2024-09-11T08:47:02.655235+0000","last_fullsized":"2024-09-11T08:47:02.655235+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:43:26.983587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:26.983587+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082679390000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":173,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668371+0000","last_change":"2024-09-11T08:45:13.320968+0000","last_active":"2024-09-11T08:47:02.668371+0000","last_peered":"2024-09-11T08:47:02.668371+0000","last_clean":"2024-09-11T08:47:02.668371+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:47:02.668371+0000","last_undegraded":"2024-09-11T08:47:02.668371+0000","last_fullsized":"2024-09-11T08:47:02.668371+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:13.320898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.320898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013000240000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":165,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.650587+0000","last_change":"2024-09-11T08:46:31.564511+0000","last_active":"2024-09-11T08:47:02.650587+0000","last_peered":"2024-09-11T08:47:02.650587+0000","last_clean":"2024-09-11T08:47:02.650587+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:47:02.650587+0000","last_undegraded":"2024-09-11T08:47:02.650587+0000","last_fullsized":"2024-09-11T08:47:02.650587+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.850684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.850684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058755379999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":182,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654895+0000","last_change":"2024-09-11T08:46:01.978488+0000","last_active":"2024-09-11T08:47:02.654895+0000","last_peered":"2024-09-11T08:47:02.654895+0000","last_clean":"2024-09-11T08:47:02.654895+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:47:02.654895+0000","last_undegraded":"2024-09-11T08:47:02.654895+0000","last_fullsized":"2024-09-11T08:47:02.654895+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:24.197149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.197149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001510267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":168,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.664380+0000","last_change":"2024-09-11T08:43:33.346941+0000","last_active":"2024-09-11T08:47:02.664380+0000","last_peered":"2024-09-11T08:47:02.664380+0000","last_clean":"2024-09-11T08:47:02.664380+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:47:02.664380+0000","last_undegraded":"2024-09-11T08:47:02.664380+0000","last_fullsized":"2024-09-11T08:47:02.664380+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:33.346856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.346856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053605370000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":220,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:09.601761+0000","last_change":"2024-09-11T08:47:09.601761+0000","last_active":"2024-09-11T08:47:09.601761+0000","last_peered":"2024-09-11T08:47:09.601761+0000","last_clean":"2024-09-11T08:47:09.601761+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:47:09.601761+0000","last_undegraded":"2024-09-11T08:47:09.601761+0000","last_fullsized":"2024-09-11T08:47:09.601761+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:44:18.678748+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:44:18.678748+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016068020999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":174,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.658568+0000","last_change":"2024-09-11T08:44:12.518229+0000","last_active":"2024-09-11T08:47:02.658568+0000","last_peered":"2024-09-11T08:47:02.658568+0000","last_clean":"2024-09-11T08:47:02.658568+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:47:02.658568+0000","last_undegraded":"2024-09-11T08:47:02.658568+0000","last_fullsized":"2024-09-11T08:47:02.658568+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:12.518151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.518151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013208541000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":190,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.666057+0000","last_change":"2024-09-11T08:46:01.998163+0000","last_active":"2024-09-11T08:47:02.666057+0000","last_peered":"2024-09-11T08:47:02.666057+0000","last_clean":"2024-09-11T08:47:02.666057+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:47:02.666057+0000","last_undegraded":"2024-09-11T08:47:02.666057+0000","last_fullsized":"2024-09-11T08:47:02.666057+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:24.223237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.223237+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019917720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":168,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655123+0000","last_change":"2024-09-11T08:45:28.194198+0000","last_active":"2024-09-11T08:47:02.655123+0000","last_peered":"2024-09-11T08:47:02.655123+0000","last_clean":"2024-09-11T08:47:02.655123+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:47:02.655123+0000","last_undegraded":"2024-09-11T08:47:02.655123+0000","last_fullsized":"2024-09-11T08:47:02.655123+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.194095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.194095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001559936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":188,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.665267+0000","last_change":"2024-09-11T08:45:16.162671+0000","last_active":"2024-09-11T08:47:02.665267+0000","last_peered":"2024-09-11T08:47:02.665267+0000","last_clean":"2024-09-11T08:47:02.665267+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:47:02.665267+0000","last_undegraded":"2024-09-11T08:47:02.665267+0000","last_fullsized":"2024-09-11T08:47:02.665267+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:16.162616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:16.162616+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001890833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":164,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654730+0000","last_change":"2024-09-11T08:45:22.244097+0000","last_active":"2024-09-11T08:47:02.654730+0000","last_peered":"2024-09-11T08:47:02.654730+0000","last_clean":"2024-09-11T08:47:02.654730+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:47:02.654730+0000","last_undegraded":"2024-09-11T08:47:02.654730+0000","last_fullsized":"2024-09-11T08:47:02.654730+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.244038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.244038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015252620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":172,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668897+0000","last_change":"2024-09-11T08:45:19.335163+0000","last_active":"2024-09-11T08:47:02.668897+0000","last_peered":"2024-09-11T08:47:02.668897+0000","last_clean":"2024-09-11T08:47:02.668897+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:47:02.668897+0000","last_undegraded":"2024-09-11T08:47:02.668897+0000","last_fullsized":"2024-09-11T08:47:02.668897+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:19.335065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:19.335065+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026052929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":178,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654288+0000","last_change":"2024-09-11T08:44:33.758098+0000","last_active":"2024-09-11T08:47:02.654288+0000","last_peered":"2024-09-11T08:47:02.654288+0000","last_clean":"2024-09-11T08:47:02.654288+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:47:02.654288+0000","last_undegraded":"2024-09-11T08:47:02.654288+0000","last_fullsized":"2024-09-11T08:47:02.654288+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:44:33.758018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025150559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":181,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.667250+0000","last_change":"2024-09-11T08:44:10.704528+0000","last_active":"2024-09-11T08:47:02.667250+0000","last_peered":"2024-09-11T08:47:02.667250+0000","last_clean":"2024-09-11T08:47:02.667250+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:47:02.667250+0000","last_undegraded":"2024-09-11T08:47:02.667250+0000","last_fullsized":"2024-09-11T08:47:02.667250+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:44:10.704471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:10.704471+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020557259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":286,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.318899+0000","last_change":"2024-09-11T08:45:08.310590+0000","last_active":"2024-09-11T08:47:07.318899+0000","last_peered":"2024-09-11T08:47:07.318899+0000","last_clean":"2024-09-11T08:47:07.318899+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:47:07.318899+0000","last_undegraded":"2024-09-11T08:47:07.318899+0000","last_fullsized":"2024-09-11T08:47:07.318899+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:08.310527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.310527+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021434650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"185'8112","reported_seq":8156,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:11.344542+0000","last_change":"2024-09-11T08:45:15.261892+0000","last_active":"2024-09-11T08:47:11.344542+0000","last_peered":"2024-09-11T08:47:11.344542+0000","last_clean":"2024-09-11T08:47:11.344542+0000","last_became_active":"2024-09-11T08:45:15.261608+0000","last_became_peered":"2024-09-11T08:45:15.261608+0000","last_unstale":"2024-09-11T08:47:11.344542+0000","last_undegraded":"2024-09-11T08:47:11.344542+0000","last_fullsized":"2024-09-11T08:47:11.344542+0000","mapping_epoch":149,"log_start":"164'3100","ondisk_log_start":"164'3100","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5012,"log_dups_size":0,"ondisk_log_size":5012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":441450496,"num_objects":6736,"num_object_clones":0,"num_object_copies":20208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8112,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"124'22","reported_seq":208,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.662613+0000","last_change":"2024-09-11T08:45:18.777528+0000","last_active":"2024-09-11T08:47:02.662613+0000","last_peered":"2024-09-11T08:47:02.662613+0000","last_clean":"2024-09-11T08:47:02.662613+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:47:02.662613+0000","last_undegraded":"2024-09-11T08:47:02.662613+0000","last_fullsized":"2024-09-11T08:47:02.662613+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:45:18.777466+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.777466+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045440580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.656745+0000","last_change":"2024-09-11T08:45:13.214007+0000","last_active":"2024-09-11T08:47:02.656745+0000","last_peered":"2024-09-11T08:47:02.656745+0000","last_clean":"2024-09-11T08:47:02.656745+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:47:02.656745+0000","last_undegraded":"2024-09-11T08:47:02.656745+0000","last_fullsized":"2024-09-11T08:47:02.656745+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:13.213922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.213922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016325280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"154'9","reported_seq":275,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.307877+0000","last_change":"2024-09-11T08:45:26.208887+0000","last_active":"2024-09-11T08:47:07.307877+0000","last_peered":"2024-09-11T08:47:07.307877+0000","last_clean":"2024-09-11T08:47:07.307877+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:47:07.307877+0000","last_undegraded":"2024-09-11T08:47:07.307877+0000","last_fullsized":"2024-09-11T08:47:07.307877+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'9","last_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.208818+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028933230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.649765+0000","last_change":"2024-09-11T08:44:24.813906+0000","last_active":"2024-09-11T08:47:02.649765+0000","last_peered":"2024-09-11T08:47:02.649765+0000","last_clean":"2024-09-11T08:47:02.649765+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:47:02.649765+0000","last_undegraded":"2024-09-11T08:47:02.649765+0000","last_fullsized":"2024-09-11T08:47:02.649765+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:24.813857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.813857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01118625,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"185'8318","reported_seq":8362,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:11.677124+0000","last_change":"2024-09-11T08:45:15.297710+0000","last_active":"2024-09-11T08:47:11.677124+0000","last_peered":"2024-09-11T08:47:11.677124+0000","last_clean":"2024-09-11T08:47:11.677124+0000","last_became_active":"2024-09-11T08:45:15.297378+0000","last_became_peered":"2024-09-11T08:45:15.297378+0000","last_unstale":"2024-09-11T08:47:11.677124+0000","last_undegraded":"2024-09-11T08:47:11.677124+0000","last_fullsized":"2024-09-11T08:47:11.677124+0000","mapping_epoch":149,"log_start":"161'2600","ondisk_log_start":"161'2600","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5718,"log_dups_size":0,"ondisk_log_size":5718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":444334080,"num_objects":6780,"num_object_clones":0,"num_object_copies":20340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8318,"num_write_kb":483136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":197,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.656605+0000","last_change":"2024-09-11T08:44:33.387066+0000","last_active":"2024-09-11T08:47:02.656605+0000","last_peered":"2024-09-11T08:47:02.656605+0000","last_clean":"2024-09-11T08:47:02.656605+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:47:02.656605+0000","last_undegraded":"2024-09-11T08:47:02.656605+0000","last_fullsized":"2024-09-11T08:47:02.656605+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-11T08:44:33.386972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:33.386972+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022646007999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.666419+0000","last_change":"2024-09-11T08:44:25.327692+0000","last_active":"2024-09-11T08:47:02.666419+0000","last_peered":"2024-09-11T08:47:02.666419+0000","last_clean":"2024-09-11T08:47:02.666419+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:47:02.666419+0000","last_undegraded":"2024-09-11T08:47:02.666419+0000","last_fullsized":"2024-09-11T08:47:02.666419+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:25.327553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:25.327553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016327754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":164,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654486+0000","last_change":"2024-09-11T08:45:17.150827+0000","last_active":"2024-09-11T08:47:02.654486+0000","last_peered":"2024-09-11T08:47:02.654486+0000","last_clean":"2024-09-11T08:47:02.654486+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:47:02.654486+0000","last_undegraded":"2024-09-11T08:47:02.654486+0000","last_fullsized":"2024-09-11T08:47:02.654486+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.150767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.150767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001327707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654978+0000","last_change":"2024-09-11T08:46:03.144149+0000","last_active":"2024-09-11T08:47:02.654978+0000","last_peered":"2024-09-11T08:47:02.654978+0000","last_clean":"2024-09-11T08:47:02.654978+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:47:02.654978+0000","last_undegraded":"2024-09-11T08:47:02.654978+0000","last_fullsized":"2024-09-11T08:47:02.654978+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:46:03.144096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:46:03.144096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:47:11.742114+0000","scrub_duration":0.001497151,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"185'8185","reported_seq":8230,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:11.133511+0000","last_change":"2024-09-11T08:45:15.627562+0000","last_active":"2024-09-11T08:47:11.133511+0000","last_peered":"2024-09-11T08:47:11.133511+0000","last_clean":"2024-09-11T08:47:11.133511+0000","last_became_active":"2024-09-11T08:45:15.627210+0000","last_became_peered":"2024-09-11T08:45:15.627210+0000","last_unstale":"2024-09-11T08:47:11.133511+0000","last_undegraded":"2024-09-11T08:47:11.133511+0000","last_fullsized":"2024-09-11T08:47:11.133511+0000","mapping_epoch":149,"log_start":"161'2800","ondisk_log_start":"161'2800","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5385,"log_dups_size":0,"ondisk_log_size":5385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":435945496,"num_objects":6653,"num_object_clones":0,"num_object_copies":19959,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6653,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8185,"num_write_kb":474753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"86'21","reported_seq":184,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654628+0000","last_change":"2024-09-11T08:43:30.997804+0000","last_active":"2024-09-11T08:47:02.654628+0000","last_peered":"2024-09-11T08:47:02.654628+0000","last_clean":"2024-09-11T08:47:02.654628+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:47:02.654628+0000","last_undegraded":"2024-09-11T08:47:02.654628+0000","last_fullsized":"2024-09-11T08:47:02.654628+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'21","last_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:43:30.997710+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025287299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":178,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.667478+0000","last_change":"2024-09-11T08:45:30.475446+0000","last_active":"2024-09-11T08:47:02.667478+0000","last_peered":"2024-09-11T08:47:02.667478+0000","last_clean":"2024-09-11T08:47:02.667478+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:47:02.667478+0000","last_undegraded":"2024-09-11T08:47:02.667478+0000","last_fullsized":"2024-09-11T08:47:02.667478+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:30.475365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:30.475365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010690405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":266,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.317654+0000","last_change":"2024-09-11T08:44:11.198993+0000","last_active":"2024-09-11T08:47:07.317654+0000","last_peered":"2024-09-11T08:47:07.317654+0000","last_clean":"2024-09-11T08:47:07.317654+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:47:07.317654+0000","last_undegraded":"2024-09-11T08:47:07.317654+0000","last_fullsized":"2024-09-11T08:47:07.317654+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:44:11.198936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:11.198936+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019660649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":191,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668525+0000","last_change":"2024-09-11T08:46:01.983654+0000","last_active":"2024-09-11T08:47:02.668525+0000","last_peered":"2024-09-11T08:47:02.668525+0000","last_clean":"2024-09-11T08:47:02.668525+0000","last_became_active":"2024-09-11T08:46:01.983335+0000","last_became_peered":"2024-09-11T08:46:01.983335+0000","last_unstale":"2024-09-11T08:47:02.668525+0000","last_undegraded":"2024-09-11T08:47:02.668525+0000","last_fullsized":"2024-09-11T08:47:02.668525+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.303194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.303194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001492431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"185'8284","reported_seq":8327,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:11.704586+0000","last_change":"2024-09-11T08:45:15.626487+0000","last_active":"2024-09-11T08:47:11.704586+0000","last_peered":"2024-09-11T08:47:11.704586+0000","last_clean":"2024-09-11T08:47:11.704586+0000","last_became_active":"2024-09-11T08:45:15.626210+0000","last_became_peered":"2024-09-11T08:45:15.626210+0000","last_unstale":"2024-09-11T08:47:11.704586+0000","last_undegraded":"2024-09-11T08:47:11.704586+0000","last_fullsized":"2024-09-11T08:47:11.704586+0000","mapping_epoch":149,"log_start":"161'2600","ondisk_log_start":"161'2600","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5684,"log_dups_size":0,"ondisk_log_size":5684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":434372608,"num_objects":6628,"num_object_clones":0,"num_object_copies":19884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8284,"num_write_kb":477184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"100'15","reported_seq":196,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653556+0000","last_change":"2024-09-11T08:45:12.769299+0000","last_active":"2024-09-11T08:47:02.653556+0000","last_peered":"2024-09-11T08:47:02.653556+0000","last_clean":"2024-09-11T08:47:02.653556+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:47:02.653556+0000","last_undegraded":"2024-09-11T08:47:02.653556+0000","last_fullsized":"2024-09-11T08:47:02.653556+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'15","last_scrub_stamp":"2024-09-11T08:45:12.769239+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:41:56.250393+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.769239+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026298300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":171,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.650117+0000","last_change":"2024-09-11T08:46:30.615598+0000","last_active":"2024-09-11T08:47:02.650117+0000","last_peered":"2024-09-11T08:47:02.650117+0000","last_clean":"2024-09-11T08:47:02.650117+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:47:02.650117+0000","last_undegraded":"2024-09-11T08:47:02.650117+0000","last_fullsized":"2024-09-11T08:47:02.650117+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:19.905459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:19.905459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015766237999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":274,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.314222+0000","last_change":"2024-09-11T08:44:34.630457+0000","last_active":"2024-09-11T08:47:07.314222+0000","last_peered":"2024-09-11T08:47:07.314222+0000","last_clean":"2024-09-11T08:47:07.314222+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:47:07.314222+0000","last_undegraded":"2024-09-11T08:47:07.314222+0000","last_fullsized":"2024-09-11T08:47:07.314222+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:44:34.630191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:34.630191+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089275420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.663599+0000","last_change":"2024-09-11T08:44:16.681124+0000","last_active":"2024-09-11T08:47:02.663599+0000","last_peered":"2024-09-11T08:47:02.663599+0000","last_clean":"2024-09-11T08:47:02.663599+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:47:02.663599+0000","last_undegraded":"2024-09-11T08:47:02.663599+0000","last_fullsized":"2024-09-11T08:47:02.663599+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:16.681036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:44:16.681036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047481509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"185'8224","reported_seq":8267,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:11.537398+0000","last_change":"2024-09-11T08:45:15.627960+0000","last_active":"2024-09-11T08:47:11.537398+0000","last_peered":"2024-09-11T08:47:11.537398+0000","last_clean":"2024-09-11T08:47:11.537398+0000","last_became_active":"2024-09-11T08:45:15.627639+0000","last_became_peered":"2024-09-11T08:45:15.627639+0000","last_unstale":"2024-09-11T08:47:11.537398+0000","last_undegraded":"2024-09-11T08:47:11.537398+0000","last_fullsized":"2024-09-11T08:47:11.537398+0000","mapping_epoch":149,"log_start":"165'3300","ondisk_log_start":"165'3300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4924,"log_dups_size":0,"ondisk_log_size":4924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":439353344,"num_objects":6704,"num_object_clones":0,"num_object_copies":20112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8224,"num_write_kb":477696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":202,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.664181+0000","last_change":"2024-09-11T08:45:11.576184+0000","last_active":"2024-09-11T08:47:02.664181+0000","last_peered":"2024-09-11T08:47:02.664181+0000","last_clean":"2024-09-11T08:47:02.664181+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:47:02.664181+0000","last_undegraded":"2024-09-11T08:47:02.664181+0000","last_fullsized":"2024-09-11T08:47:02.664181+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:45:11.576085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.576085+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002366754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":178,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653652+0000","last_change":"2024-09-11T08:44:30.802099+0000","last_active":"2024-09-11T08:47:02.653652+0000","last_peered":"2024-09-11T08:47:02.653652+0000","last_clean":"2024-09-11T08:47:02.653652+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:47:02.653652+0000","last_undegraded":"2024-09-11T08:47:02.653652+0000","last_fullsized":"2024-09-11T08:47:02.653652+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:30.800097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:30.800097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071432789999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":173,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653048+0000","last_change":"2024-09-11T08:45:07.264895+0000","last_active":"2024-09-11T08:47:02.653048+0000","last_peered":"2024-09-11T08:47:02.653048+0000","last_clean":"2024-09-11T08:47:02.653048+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:47:02.653048+0000","last_undegraded":"2024-09-11T08:47:02.653048+0000","last_fullsized":"2024-09-11T08:47:02.653048+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.264822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.264822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014425250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":192,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653542+0000","last_change":"2024-09-11T08:46:01.985315+0000","last_active":"2024-09-11T08:47:02.653542+0000","last_peered":"2024-09-11T08:47:02.653542+0000","last_clean":"2024-09-11T08:47:02.653542+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:47:02.653542+0000","last_undegraded":"2024-09-11T08:47:02.653542+0000","last_fullsized":"2024-09-11T08:47:02.653542+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.805220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.805220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012010009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"185'8257","reported_seq":8301,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:11.115660+0000","last_change":"2024-09-11T08:45:15.298396+0000","last_active":"2024-09-11T08:47:11.115660+0000","last_peered":"2024-09-11T08:47:11.115660+0000","last_clean":"2024-09-11T08:47:11.115660+0000","last_became_active":"2024-09-11T08:45:15.298053+0000","last_became_peered":"2024-09-11T08:45:15.298053+0000","last_unstale":"2024-09-11T08:47:11.115660+0000","last_undegraded":"2024-09-11T08:47:11.115660+0000","last_fullsized":"2024-09-11T08:47:11.115660+0000","mapping_epoch":149,"log_start":"161'2800","ondisk_log_start":"161'2800","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5457,"log_dups_size":0,"ondisk_log_size":5457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":444399616,"num_objects":6781,"num_object_clones":0,"num_object_copies":20343,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8257,"num_write_kb":481216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":346,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313523+0000","last_change":"2024-09-11T08:45:26.666261+0000","last_active":"2024-09-11T08:47:07.313523+0000","last_peered":"2024-09-11T08:47:07.313523+0000","last_clean":"2024-09-11T08:47:07.313523+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:47:07.313523+0000","last_undegraded":"2024-09-11T08:47:07.313523+0000","last_fullsized":"2024-09-11T08:47:07.313523+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:45:26.666199+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.666199+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020004279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":168,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655579+0000","last_change":"2024-09-11T08:45:15.130637+0000","last_active":"2024-09-11T08:47:02.655579+0000","last_peered":"2024-09-11T08:47:02.655579+0000","last_clean":"2024-09-11T08:47:02.655579+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:47:02.655579+0000","last_undegraded":"2024-09-11T08:47:02.655579+0000","last_fullsized":"2024-09-11T08:47:02.655579+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.130572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.130572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019779749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":209,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.718136+0000","last_change":"2024-09-11T08:47:02.718120+0000","last_active":"2024-09-11T08:47:02.718136+0000","last_peered":"2024-09-11T08:47:02.718136+0000","last_clean":"2024-09-11T08:47:02.718136+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:47:02.718136+0000","last_undegraded":"2024-09-11T08:47:02.718136+0000","last_fullsized":"2024-09-11T08:47:02.718136+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:43:38.669837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:38.669837+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001723715,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":164,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669952+0000","last_change":"2024-09-11T08:44:14.679790+0000","last_active":"2024-09-11T08:47:02.669952+0000","last_peered":"2024-09-11T08:47:02.669952+0000","last_clean":"2024-09-11T08:47:02.669952+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:47:02.669952+0000","last_undegraded":"2024-09-11T08:47:02.669952+0000","last_fullsized":"2024-09-11T08:47:02.669952+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:14.679598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:14.679598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011950855999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":161,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.665502+0000","last_change":"2024-09-11T08:45:22.169436+0000","last_active":"2024-09-11T08:47:02.665502+0000","last_peered":"2024-09-11T08:47:02.665502+0000","last_clean":"2024-09-11T08:47:02.665502+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:47:02.665502+0000","last_undegraded":"2024-09-11T08:47:02.665502+0000","last_fullsized":"2024-09-11T08:47:02.665502+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.169367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.169367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005349693,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"185'8225","reported_seq":8269,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:11.571808+0000","last_change":"2024-09-11T08:45:15.261656+0000","last_active":"2024-09-11T08:47:11.571808+0000","last_peered":"2024-09-11T08:47:11.571808+0000","last_clean":"2024-09-11T08:47:11.571808+0000","last_became_active":"2024-09-11T08:45:15.261370+0000","last_became_peered":"2024-09-11T08:45:15.261370+0000","last_unstale":"2024-09-11T08:47:11.571808+0000","last_undegraded":"2024-09-11T08:47:11.571808+0000","last_fullsized":"2024-09-11T08:47:11.571808+0000","mapping_epoch":149,"log_start":"165'3300","ondisk_log_start":"165'3300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4925,"log_dups_size":0,"ondisk_log_size":4925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":443482112,"num_objects":6767,"num_object_clones":0,"num_object_copies":20301,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8225,"num_write_kb":479744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":213,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.662694+0000","last_change":"2024-09-11T08:45:27.666590+0000","last_active":"2024-09-11T08:47:02.662694+0000","last_peered":"2024-09-11T08:47:02.662694+0000","last_clean":"2024-09-11T08:47:02.662694+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:47:02.662694+0000","last_undegraded":"2024-09-11T08:47:02.662694+0000","last_fullsized":"2024-09-11T08:47:02.662694+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:45:27.666502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.666502+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023582730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":198,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.658846+0000","last_change":"2024-09-11T08:46:01.982031+0000","last_active":"2024-09-11T08:47:02.658846+0000","last_peered":"2024-09-11T08:47:02.658846+0000","last_clean":"2024-09-11T08:47:02.658846+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:47:02.658846+0000","last_undegraded":"2024-09-11T08:47:02.658846+0000","last_fullsized":"2024-09-11T08:47:02.658846+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.775804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.775804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011899289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":194,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.658005+0000","last_change":"2024-09-11T08:45:32.632342+0000","last_active":"2024-09-11T08:47:02.658005+0000","last_peered":"2024-09-11T08:47:02.658005+0000","last_clean":"2024-09-11T08:47:02.658005+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:47:02.658005+0000","last_undegraded":"2024-09-11T08:47:02.658005+0000","last_fullsized":"2024-09-11T08:47:02.658005+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:45:32.632242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:32.632242+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049221250000000003,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":180,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.661407+0000","last_change":"2024-09-11T08:46:23.414859+0000","last_active":"2024-09-11T08:47:02.661407+0000","last_peered":"2024-09-11T08:47:02.661407+0000","last_clean":"2024-09-11T08:47:02.661407+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:47:02.661407+0000","last_undegraded":"2024-09-11T08:47:02.661407+0000","last_fullsized":"2024-09-11T08:47:02.661407+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:09.638694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_clean_scrub_stamp":"2024-09-11T08:45:09.638694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00151615,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":191,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669163+0000","last_change":"2024-09-11T08:46:01.984350+0000","last_active":"2024-09-11T08:47:02.669163+0000","last_peered":"2024-09-11T08:47:02.669163+0000","last_clean":"2024-09-11T08:47:02.669163+0000","last_became_active":"2024-09-11T08:46:01.983095+0000","last_became_peered":"2024-09-11T08:46:01.983095+0000","last_unstale":"2024-09-11T08:47:02.669163+0000","last_undegraded":"2024-09-11T08:47:02.669163+0000","last_fullsized":"2024-09-11T08:47:02.669163+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:12.278185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.278185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015184790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"185'8387","reported_seq":8431,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:11.397478+0000","last_change":"2024-09-11T08:45:15.627633+0000","last_active":"2024-09-11T08:47:11.397478+0000","last_peered":"2024-09-11T08:47:11.397478+0000","last_clean":"2024-09-11T08:47:11.397478+0000","last_became_active":"2024-09-11T08:45:15.627360+0000","last_became_peered":"2024-09-11T08:45:15.627360+0000","last_unstale":"2024-09-11T08:47:11.397478+0000","last_undegraded":"2024-09-11T08:47:11.397478+0000","last_fullsized":"2024-09-11T08:47:11.397478+0000","mapping_epoch":149,"log_start":"165'3400","ondisk_log_start":"165'3400","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4987,"log_dups_size":0,"ondisk_log_size":4987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":455016448,"num_objects":6943,"num_object_clones":0,"num_object_copies":20829,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8387,"num_write_kb":490560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":184,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.667332+0000","last_change":"2024-09-11T08:44:15.699227+0000","last_active":"2024-09-11T08:47:02.667332+0000","last_peered":"2024-09-11T08:47:02.667332+0000","last_clean":"2024-09-11T08:47:02.667332+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:47:02.667332+0000","last_undegraded":"2024-09-11T08:47:02.667332+0000","last_fullsized":"2024-09-11T08:47:02.667332+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:44:15.699067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:15.699067+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014313306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":178,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.664748+0000","last_change":"2024-09-11T08:45:20.455183+0000","last_active":"2024-09-11T08:47:02.664748+0000","last_peered":"2024-09-11T08:47:02.664748+0000","last_clean":"2024-09-11T08:47:02.664748+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:47:02.664748+0000","last_undegraded":"2024-09-11T08:47:02.664748+0000","last_fullsized":"2024-09-11T08:47:02.664748+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:20.455081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.455081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016372450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":700,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:09.775316+0000","last_change":"2024-09-11T08:45:21.370742+0000","last_active":"2024-09-11T08:47:09.775316+0000","last_peered":"2024-09-11T08:47:09.775316+0000","last_clean":"2024-09-11T08:47:09.775316+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:47:09.775316+0000","last_undegraded":"2024-09-11T08:47:09.775316+0000","last_fullsized":"2024-09-11T08:47:09.775316+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:45:21.370673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.370673+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017307240000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":505,"num_read_kb":500,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.649941+0000","last_change":"2024-09-11T08:46:12.673215+0000","last_active":"2024-09-11T08:47:02.649941+0000","last_peered":"2024-09-11T08:47:02.649941+0000","last_clean":"2024-09-11T08:47:02.649941+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:47:02.649941+0000","last_undegraded":"2024-09-11T08:47:02.649941+0000","last_fullsized":"2024-09-11T08:47:02.649941+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:16.924404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:16.924404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024072690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":172,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653349+0000","last_change":"2024-09-11T08:45:17.819757+0000","last_active":"2024-09-11T08:47:02.653349+0000","last_peered":"2024-09-11T08:47:02.653349+0000","last_clean":"2024-09-11T08:47:02.653349+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:47:02.653349+0000","last_undegraded":"2024-09-11T08:47:02.653349+0000","last_fullsized":"2024-09-11T08:47:02.653349+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.819707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.819707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012611440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"185'8390","reported_seq":8434,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:11.076853+0000","last_change":"2024-09-11T08:45:15.262738+0000","last_active":"2024-09-11T08:47:11.076853+0000","last_peered":"2024-09-11T08:47:11.076853+0000","last_clean":"2024-09-11T08:47:11.076853+0000","last_became_active":"2024-09-11T08:45:15.262440+0000","last_became_peered":"2024-09-11T08:45:15.262440+0000","last_unstale":"2024-09-11T08:47:11.076853+0000","last_undegraded":"2024-09-11T08:47:11.076853+0000","last_fullsized":"2024-09-11T08:47:11.076853+0000","mapping_epoch":149,"log_start":"160'2100","ondisk_log_start":"160'2100","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":6290,"log_dups_size":0,"ondisk_log_size":6290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":448266240,"num_objects":6840,"num_object_clones":0,"num_object_copies":20520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8390,"num_write_kb":487360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":43,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.657854+0000","last_change":"2024-09-11T08:46:01.996394+0000","last_active":"2024-09-11T08:47:02.657854+0000","last_peered":"2024-09-11T08:47:02.657854+0000","last_clean":"2024-09-11T08:47:02.657854+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:47:02.657854+0000","last_undegraded":"2024-09-11T08:47:02.657854+0000","last_fullsized":"2024-09-11T08:47:02.657854+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:43:26.127412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:26.127412+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":176,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.664982+0000","last_change":"2024-09-11T08:45:18.127946+0000","last_active":"2024-09-11T08:47:02.664982+0000","last_peered":"2024-09-11T08:47:02.664982+0000","last_clean":"2024-09-11T08:47:02.664982+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:47:02.664982+0000","last_undegraded":"2024-09-11T08:47:02.664982+0000","last_fullsized":"2024-09-11T08:47:02.664982+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:18.127857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.127857+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018334099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":221,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669998+0000","last_change":"2024-09-11T08:45:29.524681+0000","last_active":"2024-09-11T08:47:02.669998+0000","last_peered":"2024-09-11T08:47:02.669998+0000","last_clean":"2024-09-11T08:47:02.669998+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:47:02.669998+0000","last_undegraded":"2024-09-11T08:47:02.669998+0000","last_fullsized":"2024-09-11T08:47:02.669998+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'79","last_scrub_stamp":"2024-09-11T08:45:29.524608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:45:29.524608+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002308867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":173,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668726+0000","last_change":"2024-09-11T08:44:35.441051+0000","last_active":"2024-09-11T08:47:02.668726+0000","last_peered":"2024-09-11T08:47:02.668726+0000","last_clean":"2024-09-11T08:47:02.668726+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:47:02.668726+0000","last_undegraded":"2024-09-11T08:47:02.668726+0000","last_fullsized":"2024-09-11T08:47:02.668726+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:35.440995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:35.440995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0080921540000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":165,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.649820+0000","last_change":"2024-09-11T08:46:10.685628+0000","last_active":"2024-09-11T08:47:02.649820+0000","last_peered":"2024-09-11T08:47:02.649820+0000","last_clean":"2024-09-11T08:47:02.649820+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:47:02.649820+0000","last_undegraded":"2024-09-11T08:47:02.649820+0000","last_fullsized":"2024-09-11T08:47:02.649820+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.570246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.570246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041809610000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"185'8352","reported_seq":8396,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:11.060085+0000","last_change":"2024-09-11T08:45:15.263087+0000","last_active":"2024-09-11T08:47:11.060085+0000","last_peered":"2024-09-11T08:47:11.060085+0000","last_clean":"2024-09-11T08:47:11.060085+0000","last_became_active":"2024-09-11T08:45:15.262698+0000","last_became_peered":"2024-09-11T08:45:15.262698+0000","last_unstale":"2024-09-11T08:47:11.060085+0000","last_undegraded":"2024-09-11T08:47:11.060085+0000","last_fullsized":"2024-09-11T08:47:11.060085+0000","mapping_epoch":149,"log_start":"165'3500","ondisk_log_start":"165'3500","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4852,"log_dups_size":0,"ondisk_log_size":4852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":447741952,"num_objects":6832,"num_object_clones":0,"num_object_copies":20496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8352,"num_write_kb":485888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":241,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655060+0000","last_change":"2024-09-11T08:46:01.997857+0000","last_active":"2024-09-11T08:47:02.655060+0000","last_peered":"2024-09-11T08:47:02.655060+0000","last_clean":"2024-09-11T08:47:02.655060+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:47:02.655060+0000","last_undegraded":"2024-09-11T08:47:02.655060+0000","last_fullsized":"2024-09-11T08:47:02.655060+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:21.002140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:21.002140+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025090400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":158,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654692+0000","last_change":"2024-09-11T08:44:22.010602+0000","last_active":"2024-09-11T08:47:02.654692+0000","last_peered":"2024-09-11T08:47:02.654692+0000","last_clean":"2024-09-11T08:47:02.654692+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:47:02.654692+0000","last_undegraded":"2024-09-11T08:47:02.654692+0000","last_fullsized":"2024-09-11T08:47:02.654692+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.010528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.010528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047628439999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":174,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.649688+0000","last_change":"2024-09-11T08:44:26.726624+0000","last_active":"2024-09-11T08:47:02.649688+0000","last_peered":"2024-09-11T08:47:02.649688+0000","last_clean":"2024-09-11T08:47:02.649688+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:47:02.649688+0000","last_undegraded":"2024-09-11T08:47:02.649688+0000","last_fullsized":"2024-09-11T08:47:02.649688+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.726573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00117395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":181,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.664994+0000","last_change":"2024-09-11T08:46:01.988315+0000","last_active":"2024-09-11T08:47:02.664994+0000","last_peered":"2024-09-11T08:47:02.664994+0000","last_clean":"2024-09-11T08:47:02.664994+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:47:02.664994+0000","last_undegraded":"2024-09-11T08:47:02.664994+0000","last_fullsized":"2024-09-11T08:47:02.664994+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.167071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.167071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001796483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"185'8146","reported_seq":8190,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:11.344660+0000","last_change":"2024-09-11T08:45:15.626015+0000","last_active":"2024-09-11T08:47:11.344660+0000","last_peered":"2024-09-11T08:47:11.344660+0000","last_clean":"2024-09-11T08:47:11.344660+0000","last_became_active":"2024-09-11T08:45:15.625726+0000","last_became_peered":"2024-09-11T08:45:15.625726+0000","last_unstale":"2024-09-11T08:47:11.344660+0000","last_undegraded":"2024-09-11T08:47:11.344660+0000","last_fullsized":"2024-09-11T08:47:11.344660+0000","mapping_epoch":149,"log_start":"164'3200","ondisk_log_start":"164'3200","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4946,"log_dups_size":0,"ondisk_log_size":4946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":434503680,"num_objects":6630,"num_object_clones":0,"num_object_copies":19890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8146,"num_write_kb":472832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":194,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655631+0000","last_change":"2024-09-11T08:44:28.940078+0000","last_active":"2024-09-11T08:47:02.655631+0000","last_peered":"2024-09-11T08:47:02.655631+0000","last_clean":"2024-09-11T08:47:02.655631+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:47:02.655631+0000","last_undegraded":"2024-09-11T08:47:02.655631+0000","last_fullsized":"2024-09-11T08:47:02.655631+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:44:28.940023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.940023+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012856796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":187,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653279+0000","last_change":"2024-09-11T08:45:25.273200+0000","last_active":"2024-09-11T08:47:02.653279+0000","last_peered":"2024-09-11T08:47:02.653279+0000","last_clean":"2024-09-11T08:47:02.653279+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:47:02.653279+0000","last_undegraded":"2024-09-11T08:47:02.653279+0000","last_fullsized":"2024-09-11T08:47:02.653279+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.273134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014307370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":174,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653420+0000","last_change":"2024-09-11T08:44:24.816974+0000","last_active":"2024-09-11T08:47:02.653420+0000","last_peered":"2024-09-11T08:47:02.653420+0000","last_clean":"2024-09-11T08:47:02.653420+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:47:02.653420+0000","last_undegraded":"2024-09-11T08:47:02.653420+0000","last_fullsized":"2024-09-11T08:47:02.653420+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:24.816890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.816890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086069140000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":184,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:04.543421+0000","last_change":"2024-09-11T08:47:04.543347+0000","last_active":"2024-09-11T08:47:04.543421+0000","last_peered":"2024-09-11T08:47:04.543421+0000","last_clean":"2024-09-11T08:47:04.543421+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:47:04.543421+0000","last_undegraded":"2024-09-11T08:47:04.543421+0000","last_fullsized":"2024-09-11T08:47:04.543421+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.756166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.756166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059059109999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"185'8274","reported_seq":8318,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:11.564137+0000","last_change":"2024-09-11T08:45:15.260658+0000","last_active":"2024-09-11T08:47:11.564137+0000","last_peered":"2024-09-11T08:47:11.564137+0000","last_clean":"2024-09-11T08:47:11.564137+0000","last_became_active":"2024-09-11T08:45:15.260383+0000","last_became_peered":"2024-09-11T08:45:15.260383+0000","last_unstale":"2024-09-11T08:47:11.564137+0000","last_undegraded":"2024-09-11T08:47:11.564137+0000","last_fullsized":"2024-09-11T08:47:11.564137+0000","mapping_epoch":149,"log_start":"165'3400","ondisk_log_start":"165'3400","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4874,"log_dups_size":0,"ondisk_log_size":4874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":441450496,"num_objects":6736,"num_object_clones":0,"num_object_copies":20208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8274,"num_write_kb":480320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":215,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655091+0000","last_change":"2024-09-11T08:46:02.002847+0000","last_active":"2024-09-11T08:47:02.655091+0000","last_peered":"2024-09-11T08:47:02.655091+0000","last_clean":"2024-09-11T08:47:02.655091+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:47:02.655091+0000","last_undegraded":"2024-09-11T08:47:02.655091+0000","last_fullsized":"2024-09-11T08:47:02.655091+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:43:24.983740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:24.983740+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069750209999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.666306+0000","last_change":"2024-09-11T08:45:32.219463+0000","last_active":"2024-09-11T08:47:02.666306+0000","last_peered":"2024-09-11T08:47:02.666306+0000","last_clean":"2024-09-11T08:47:02.666306+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:47:02.666306+0000","last_undegraded":"2024-09-11T08:47:02.666306+0000","last_fullsized":"2024-09-11T08:47:02.666306+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:32.219374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:32.219374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069622119999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":290,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.323291+0000","last_change":"2024-09-11T08:46:25.415023+0000","last_active":"2024-09-11T08:47:07.323291+0000","last_peered":"2024-09-11T08:47:07.323291+0000","last_clean":"2024-09-11T08:47:07.323291+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:47:07.323291+0000","last_undegraded":"2024-09-11T08:47:07.323291+0000","last_fullsized":"2024-09-11T08:47:07.323291+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:10.659079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:10.659079+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017208200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654814+0000","last_change":"2024-09-11T08:45:31.196687+0000","last_active":"2024-09-11T08:47:02.654814+0000","last_peered":"2024-09-11T08:47:02.654814+0000","last_clean":"2024-09-11T08:47:02.654814+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:47:02.654814+0000","last_undegraded":"2024-09-11T08:47:02.654814+0000","last_fullsized":"2024-09-11T08:47:02.654814+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.196599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.196599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025248190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"185'8373","reported_seq":12196,"reported_epoch":185,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T08:47:11.396867+0000","last_change":"2024-09-11T08:46:03.286008+0000","last_active":"2024-09-11T08:47:11.396867+0000","last_peered":"2024-09-11T08:47:11.396867+0000","last_clean":"2024-09-11T08:46:00.945841+0000","last_became_active":"2024-09-11T08:46:03.248256+0000","last_became_peered":"2024-09-11T08:46:03.248256+0000","last_unstale":"2024-09-11T08:47:11.396867+0000","last_undegraded":"2024-09-11T08:47:11.396867+0000","last_fullsized":"2024-09-11T08:46:02.976155+0000","mapping_epoch":168,"log_start":"165'3400","ondisk_log_start":"165'3400","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4973,"log_dups_size":0,"ondisk_log_size":4973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":448462848,"num_objects":6843,"num_object_clones":0,"num_object_copies":20529,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2045,"num_objects_unfound":0,"num_objects_dirty":6843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8373,"num_write_kb":486912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1884,"num_bytes_recovered":123469824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3],"avail_no_missing":["1","2","3"],"object_location_counts":[{"shards":"1,2,3","objects":2045},{"shards":"1,3","objects":4798}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":182,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.664103+0000","last_change":"2024-09-11T08:44:31.662275+0000","last_active":"2024-09-11T08:47:02.664103+0000","last_peered":"2024-09-11T08:47:02.664103+0000","last_clean":"2024-09-11T08:47:02.664103+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:47:02.664103+0000","last_undegraded":"2024-09-11T08:47:02.664103+0000","last_fullsized":"2024-09-11T08:47:02.664103+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:31.662190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.662190+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040714619999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":178,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.650496+0000","last_change":"2024-09-11T08:45:10.607400+0000","last_active":"2024-09-11T08:47:02.650496+0000","last_peered":"2024-09-11T08:47:02.650496+0000","last_clean":"2024-09-11T08:47:02.650496+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:47:02.650496+0000","last_undegraded":"2024-09-11T08:47:02.650496+0000","last_fullsized":"2024-09-11T08:47:02.650496+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:10.607288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:45:10.607288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016158909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":174,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.658667+0000","last_change":"2024-09-11T08:45:12.646009+0000","last_active":"2024-09-11T08:47:02.658667+0000","last_peered":"2024-09-11T08:47:02.658667+0000","last_clean":"2024-09-11T08:47:02.658667+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:47:02.658667+0000","last_undegraded":"2024-09-11T08:47:02.658667+0000","last_fullsized":"2024-09-11T08:47:02.658667+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:12.645956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.645956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014834379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.664672+0000","last_change":"2024-09-11T08:44:17.731558+0000","last_active":"2024-09-11T08:47:02.664672+0000","last_peered":"2024-09-11T08:47:02.664672+0000","last_clean":"2024-09-11T08:47:02.664672+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:47:02.664672+0000","last_undegraded":"2024-09-11T08:47:02.664672+0000","last_fullsized":"2024-09-11T08:47:02.664672+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:17.731489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:17.731489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002184517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"185'8074","reported_seq":8118,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:11.666847+0000","last_change":"2024-09-11T08:45:15.297737+0000","last_active":"2024-09-11T08:47:11.666847+0000","last_peered":"2024-09-11T08:47:11.666847+0000","last_clean":"2024-09-11T08:47:11.666847+0000","last_became_active":"2024-09-11T08:45:15.297420+0000","last_became_peered":"2024-09-11T08:45:15.297420+0000","last_unstale":"2024-09-11T08:47:11.666847+0000","last_undegraded":"2024-09-11T08:47:11.666847+0000","last_fullsized":"2024-09-11T08:47:11.666847+0000","mapping_epoch":149,"log_start":"161'2400","ondisk_log_start":"161'2400","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5674,"log_dups_size":0,"ondisk_log_size":5674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":431095808,"num_objects":6578,"num_object_clones":0,"num_object_copies":19734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8074,"num_write_kb":468864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":47,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653113+0000","last_change":"2024-09-11T08:46:01.997537+0000","last_active":"2024-09-11T08:47:02.653113+0000","last_peered":"2024-09-11T08:47:02.653113+0000","last_clean":"2024-09-11T08:47:02.653113+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:47:02.653113+0000","last_undegraded":"2024-09-11T08:47:02.653113+0000","last_fullsized":"2024-09-11T08:47:02.653113+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:44:27.810879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:27.810879+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001826121,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":177,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669416+0000","last_change":"2024-09-11T08:44:31.494470+0000","last_active":"2024-09-11T08:47:02.669416+0000","last_peered":"2024-09-11T08:47:02.669416+0000","last_clean":"2024-09-11T08:47:02.669416+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:47:02.669416+0000","last_undegraded":"2024-09-11T08:47:02.669416+0000","last_fullsized":"2024-09-11T08:47:02.669416+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.494397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.494397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021588290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":180,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.657385+0000","last_change":"2024-09-11T08:46:21.348979+0000","last_active":"2024-09-11T08:47:02.657385+0000","last_peered":"2024-09-11T08:47:02.657385+0000","last_clean":"2024-09-11T08:47:02.657385+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:47:02.657385+0000","last_undegraded":"2024-09-11T08:47:02.657385+0000","last_fullsized":"2024-09-11T08:47:02.657385+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:30.717364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:30.717364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044727940000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":161,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669976+0000","last_change":"2024-09-11T08:44:23.367862+0000","last_active":"2024-09-11T08:47:02.669976+0000","last_peered":"2024-09-11T08:47:02.669976+0000","last_clean":"2024-09-11T08:47:02.669976+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:47:02.669976+0000","last_undegraded":"2024-09-11T08:47:02.669976+0000","last_fullsized":"2024-09-11T08:47:02.669976+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:23.367786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:23.367786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030117270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"185'8161","reported_seq":13191,"reported_epoch":185,"state":"active+recovering+remapped","last_fresh":"2024-09-11T08:47:11.710874+0000","last_change":"2024-09-11T08:46:03.325028+0000","last_active":"2024-09-11T08:47:11.710874+0000","last_peered":"2024-09-11T08:47:11.710874+0000","last_clean":"2024-09-11T08:46:00.945574+0000","last_became_active":"2024-09-11T08:46:03.262389+0000","last_became_peered":"2024-09-11T08:46:03.262389+0000","last_unstale":"2024-09-11T08:47:11.710874+0000","last_undegraded":"2024-09-11T08:47:11.710874+0000","last_fullsized":"2024-09-11T08:47:11.710874+0000","mapping_epoch":168,"log_start":"161'2500","ondisk_log_start":"161'2500","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5661,"log_dups_size":0,"ondisk_log_size":5661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432340992,"num_objects":6597,"num_object_clones":0,"num_object_copies":19791,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1351,"num_objects_unfound":0,"num_objects_dirty":6597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8161,"num_write_kb":472256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2489,"num_bytes_recovered":163119104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[0,4,7],"avail_no_missing":["0","4","7"],"object_location_counts":[{"shards":"0,4,7","objects":5246}],"blocked_by":[],"up_primary":6,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":189,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654574+0000","last_change":"2024-09-11T08:45:08.140910+0000","last_active":"2024-09-11T08:47:02.654574+0000","last_peered":"2024-09-11T08:47:02.654574+0000","last_clean":"2024-09-11T08:47:02.654574+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:47:02.654574+0000","last_undegraded":"2024-09-11T08:47:02.654574+0000","last_fullsized":"2024-09-11T08:47:02.654574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:08.140839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.140839+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003679547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":177,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669047+0000","last_change":"2024-09-11T08:45:14.322786+0000","last_active":"2024-09-11T08:47:02.669047+0000","last_peered":"2024-09-11T08:47:02.669047+0000","last_clean":"2024-09-11T08:47:02.669047+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:47:02.669047+0000","last_undegraded":"2024-09-11T08:47:02.669047+0000","last_fullsized":"2024-09-11T08:47:02.669047+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.322217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.322217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014821319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":180,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.657217+0000","last_change":"2024-09-11T08:46:20.339514+0000","last_active":"2024-09-11T08:47:02.657217+0000","last_peered":"2024-09-11T08:47:02.657217+0000","last_clean":"2024-09-11T08:47:02.657217+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:47:02.657217+0000","last_undegraded":"2024-09-11T08:47:02.657217+0000","last_fullsized":"2024-09-11T08:47:02.657217+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:29.692933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:29.692933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007324077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":161,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.665934+0000","last_change":"2024-09-11T08:45:33.228265+0000","last_active":"2024-09-11T08:47:02.665934+0000","last_peered":"2024-09-11T08:47:02.665934+0000","last_clean":"2024-09-11T08:47:02.665934+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:47:02.665934+0000","last_undegraded":"2024-09-11T08:47:02.665934+0000","last_fullsized":"2024-09-11T08:47:02.665934+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:45:33.228082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064426500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"185'8272","reported_seq":8316,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:11.112271+0000","last_change":"2024-09-11T08:45:15.626107+0000","last_active":"2024-09-11T08:47:11.112271+0000","last_peered":"2024-09-11T08:47:11.112271+0000","last_clean":"2024-09-11T08:47:11.112271+0000","last_became_active":"2024-09-11T08:45:15.625778+0000","last_became_peered":"2024-09-11T08:45:15.625778+0000","last_unstale":"2024-09-11T08:47:11.112271+0000","last_undegraded":"2024-09-11T08:47:11.112271+0000","last_fullsized":"2024-09-11T08:47:11.112271+0000","mapping_epoch":149,"log_start":"162'2900","ondisk_log_start":"162'2900","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5372,"log_dups_size":0,"ondisk_log_size":5372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":448004096,"num_objects":6836,"num_object_clones":0,"num_object_copies":20508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8272,"num_write_kb":483456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"124'22","reported_seq":253,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.649178+0000","last_change":"2024-09-11T08:46:11.689219+0000","last_active":"2024-09-11T08:47:02.649178+0000","last_peered":"2024-09-11T08:47:02.649178+0000","last_clean":"2024-09-11T08:47:02.649178+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:47:02.649178+0000","last_undegraded":"2024-09-11T08:47:02.649178+0000","last_fullsized":"2024-09-11T08:47:02.649178+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:43:32.625376+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073394039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":194,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668965+0000","last_change":"2024-09-11T08:45:20.372571+0000","last_active":"2024-09-11T08:47:02.668965+0000","last_peered":"2024-09-11T08:47:02.668965+0000","last_clean":"2024-09-11T08:47:02.668965+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:47:02.668965+0000","last_undegraded":"2024-09-11T08:47:02.668965+0000","last_fullsized":"2024-09-11T08:47:02.668965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:45:20.372454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.372454+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019547269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"168'10","reported_seq":311,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.311229+0000","last_change":"2024-09-11T08:46:01.988356+0000","last_active":"2024-09-11T08:47:07.311229+0000","last_peered":"2024-09-11T08:47:07.311229+0000","last_clean":"2024-09-11T08:47:07.311229+0000","last_became_active":"2024-09-11T08:46:01.983055+0000","last_became_peered":"2024-09-11T08:46:01.983055+0000","last_unstale":"2024-09-11T08:47:07.311229+0000","last_undegraded":"2024-09-11T08:47:07.311229+0000","last_fullsized":"2024-09-11T08:47:07.311229+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"135'9","last_scrub_stamp":"2024-09-11T08:45:11.298652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.298652+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001669032,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":165,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.649746+0000","last_change":"2024-09-11T08:46:27.611148+0000","last_active":"2024-09-11T08:47:02.649746+0000","last_peered":"2024-09-11T08:47:02.649746+0000","last_clean":"2024-09-11T08:47:02.649746+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:47:02.649746+0000","last_undegraded":"2024-09-11T08:47:02.649746+0000","last_fullsized":"2024-09-11T08:47:02.649746+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:15.888394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:15.888394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003201476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":187,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.652477+0000","last_change":"2024-09-11T08:45:28.247838+0000","last_active":"2024-09-11T08:47:02.652477+0000","last_peered":"2024-09-11T08:47:02.652477+0000","last_clean":"2024-09-11T08:47:02.652477+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:47:02.652477+0000","last_undegraded":"2024-09-11T08:47:02.652477+0000","last_fullsized":"2024-09-11T08:47:02.652477+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.247771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015413499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":192,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.666240+0000","last_change":"2024-09-11T08:44:35.390285+0000","last_active":"2024-09-11T08:47:02.666240+0000","last_peered":"2024-09-11T08:47:02.666240+0000","last_clean":"2024-09-11T08:47:02.666240+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:47:02.666240+0000","last_undegraded":"2024-09-11T08:47:02.666240+0000","last_fullsized":"2024-09-11T08:47:02.666240+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-11T08:44:35.390134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:35.390134+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017427033000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":174,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.649474+0000","last_change":"2024-09-11T08:45:08.675251+0000","last_active":"2024-09-11T08:47:02.649474+0000","last_peered":"2024-09-11T08:47:02.649474+0000","last_clean":"2024-09-11T08:47:02.649474+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:47:02.649474+0000","last_undegraded":"2024-09-11T08:47:02.649474+0000","last_fullsized":"2024-09-11T08:47:02.649474+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:08.675179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.675179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003515392,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":191,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669102+0000","last_change":"2024-09-11T08:46:01.983864+0000","last_active":"2024-09-11T08:47:02.669102+0000","last_peered":"2024-09-11T08:47:02.669102+0000","last_clean":"2024-09-11T08:47:02.669102+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:47:02.669102+0000","last_undegraded":"2024-09-11T08:47:02.669102+0000","last_fullsized":"2024-09-11T08:47:02.669102+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:25.344348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.344348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013849730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":182,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.667408+0000","last_change":"2024-09-11T08:44:11.687456+0000","last_active":"2024-09-11T08:47:02.667408+0000","last_peered":"2024-09-11T08:47:02.667408+0000","last_clean":"2024-09-11T08:47:02.667408+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:47:02.667408+0000","last_undegraded":"2024-09-11T08:47:02.667408+0000","last_fullsized":"2024-09-11T08:47:02.667408+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:11.687402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:11.687402+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018414239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":177,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668600+0000","last_change":"2024-09-11T08:44:32.483521+0000","last_active":"2024-09-11T08:47:02.668600+0000","last_peered":"2024-09-11T08:47:02.668600+0000","last_clean":"2024-09-11T08:47:02.668600+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:47:02.668600+0000","last_undegraded":"2024-09-11T08:47:02.668600+0000","last_fullsized":"2024-09-11T08:47:02.668600+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:44:32.483439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013511770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":163,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.666127+0000","last_change":"2024-09-11T08:45:23.213642+0000","last_active":"2024-09-11T08:47:02.666127+0000","last_peered":"2024-09-11T08:47:02.666127+0000","last_clean":"2024-09-11T08:47:02.666127+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:47:02.666127+0000","last_undegraded":"2024-09-11T08:47:02.666127+0000","last_fullsized":"2024-09-11T08:47:02.666127+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.213592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.213592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001600473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":172,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.657859+0000","last_change":"2024-09-11T08:45:07.618618+0000","last_active":"2024-09-11T08:47:02.657859+0000","last_peered":"2024-09-11T08:47:02.657859+0000","last_clean":"2024-09-11T08:47:02.657859+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:47:02.657859+0000","last_undegraded":"2024-09-11T08:47:02.657859+0000","last_fullsized":"2024-09-11T08:47:02.657859+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.618564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.618564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016216729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":185,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.650043+0000","last_change":"2024-09-11T08:46:15.658863+0000","last_active":"2024-09-11T08:47:02.650043+0000","last_peered":"2024-09-11T08:47:02.650043+0000","last_clean":"2024-09-11T08:47:02.650043+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:47:02.650043+0000","last_undegraded":"2024-09-11T08:47:02.650043+0000","last_fullsized":"2024-09-11T08:47:02.650043+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:18.928481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:18.928481+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047130310000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":168,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.664357+0000","last_change":"2024-09-11T08:44:09.678084+0000","last_active":"2024-09-11T08:47:02.664357+0000","last_peered":"2024-09-11T08:47:02.664357+0000","last_clean":"2024-09-11T08:47:02.664357+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:47:02.664357+0000","last_undegraded":"2024-09-11T08:47:02.664357+0000","last_fullsized":"2024-09-11T08:47:02.664357+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:09.677991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:09.677991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066552809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":273,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.311321+0000","last_change":"2024-09-11T08:45:20.119727+0000","last_active":"2024-09-11T08:47:07.311321+0000","last_peered":"2024-09-11T08:47:07.311321+0000","last_clean":"2024-09-11T08:47:07.311321+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:47:07.311321+0000","last_undegraded":"2024-09-11T08:47:07.311321+0000","last_fullsized":"2024-09-11T08:47:07.311321+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:20.119626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.119626+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018087660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":152,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655729+0000","last_change":"2024-09-11T08:44:20.017341+0000","last_active":"2024-09-11T08:47:02.655729+0000","last_peered":"2024-09-11T08:47:02.655729+0000","last_clean":"2024-09-11T08:47:02.655729+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:47:02.655729+0000","last_undegraded":"2024-09-11T08:47:02.655729+0000","last_fullsized":"2024-09-11T08:47:02.655729+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:20.017261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:20.017261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001558369,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"152'15","reported_seq":185,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.665436+0000","last_change":"2024-09-11T08:45:21.132138+0000","last_active":"2024-09-11T08:47:02.665436+0000","last_peered":"2024-09-11T08:47:02.665436+0000","last_clean":"2024-09-11T08:47:02.665436+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:47:02.665436+0000","last_undegraded":"2024-09-11T08:47:02.665436+0000","last_fullsized":"2024-09-11T08:47:02.665436+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'15","last_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.132083+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003984156,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":177,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668724+0000","last_change":"2024-09-11T08:45:27.436673+0000","last_active":"2024-09-11T08:47:02.668724+0000","last_peered":"2024-09-11T08:47:02.668724+0000","last_clean":"2024-09-11T08:47:02.668724+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:47:02.668724+0000","last_undegraded":"2024-09-11T08:47:02.668724+0000","last_fullsized":"2024-09-11T08:47:02.668724+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.436552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001660176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":284,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.306249+0000","last_change":"2024-09-11T08:45:28.692173+0000","last_active":"2024-09-11T08:47:07.306249+0000","last_peered":"2024-09-11T08:47:07.306249+0000","last_clean":"2024-09-11T08:47:07.306249+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:47:07.306249+0000","last_undegraded":"2024-09-11T08:47:07.306249+0000","last_fullsized":"2024-09-11T08:47:07.306249+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:28.692100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.692100+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019579020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":172,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.666955+0000","last_change":"2024-09-11T08:45:14.581287+0000","last_active":"2024-09-11T08:47:02.666955+0000","last_peered":"2024-09-11T08:47:02.666955+0000","last_clean":"2024-09-11T08:47:02.666955+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:47:02.666955+0000","last_undegraded":"2024-09-11T08:47:02.666955+0000","last_fullsized":"2024-09-11T08:47:02.666955+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.581231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.581231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001580932,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":175,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655317+0000","last_change":"2024-09-11T08:43:37.137260+0000","last_active":"2024-09-11T08:47:02.655317+0000","last_peered":"2024-09-11T08:47:02.655317+0000","last_clean":"2024-09-11T08:47:02.655317+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:47:02.655317+0000","last_undegraded":"2024-09-11T08:47:02.655317+0000","last_fullsized":"2024-09-11T08:47:02.655317+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:37.137173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:37.137173+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085039490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":168,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654928+0000","last_change":"2024-09-11T08:45:27.198021+0000","last_active":"2024-09-11T08:47:02.654928+0000","last_peered":"2024-09-11T08:47:02.654928+0000","last_clean":"2024-09-11T08:47:02.654928+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:47:02.654928+0000","last_undegraded":"2024-09-11T08:47:02.654928+0000","last_fullsized":"2024-09-11T08:47:02.654928+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.197944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.197944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015230210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":174,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653435+0000","last_change":"2024-09-11T08:45:21.736656+0000","last_active":"2024-09-11T08:47:02.653435+0000","last_peered":"2024-09-11T08:47:02.653435+0000","last_clean":"2024-09-11T08:47:02.653435+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:47:02.653435+0000","last_undegraded":"2024-09-11T08:47:02.653435+0000","last_fullsized":"2024-09-11T08:47:02.653435+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:21.736573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.736573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020360389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":161,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.656226+0000","last_change":"2024-09-11T08:45:28.259156+0000","last_active":"2024-09-11T08:47:02.656226+0000","last_peered":"2024-09-11T08:47:02.656226+0000","last_clean":"2024-09-11T08:47:02.656226+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:47:02.656226+0000","last_undegraded":"2024-09-11T08:47:02.656226+0000","last_fullsized":"2024-09-11T08:47:02.656226+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.259103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.259103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015518800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":205,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.649557+0000","last_change":"2024-09-11T08:46:01.993547+0000","last_active":"2024-09-11T08:47:02.649557+0000","last_peered":"2024-09-11T08:47:02.649557+0000","last_clean":"2024-09-11T08:47:02.649557+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:47:02.649557+0000","last_undegraded":"2024-09-11T08:47:02.649557+0000","last_fullsized":"2024-09-11T08:47:02.649557+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:45:06.657912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.657912+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023744109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":177,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669312+0000","last_change":"2024-09-11T08:45:15.298564+0000","last_active":"2024-09-11T08:47:02.669312+0000","last_peered":"2024-09-11T08:47:02.669312+0000","last_clean":"2024-09-11T08:47:02.669312+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:47:02.669312+0000","last_undegraded":"2024-09-11T08:47:02.669312+0000","last_fullsized":"2024-09-11T08:47:02.669312+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.298476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.298476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033059629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":163,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.666000+0000","last_change":"2024-09-11T08:45:14.165632+0000","last_active":"2024-09-11T08:47:02.666000+0000","last_peered":"2024-09-11T08:47:02.666000+0000","last_clean":"2024-09-11T08:47:02.666000+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:47:02.666000+0000","last_undegraded":"2024-09-11T08:47:02.666000+0000","last_fullsized":"2024-09-11T08:47:02.666000+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.165576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.165576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012147639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.663503+0000","last_change":"2024-09-11T08:43:29.349431+0000","last_active":"2024-09-11T08:47:02.663503+0000","last_peered":"2024-09-11T08:47:02.663503+0000","last_clean":"2024-09-11T08:47:02.663503+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:47:02.663503+0000","last_undegraded":"2024-09-11T08:47:02.663503+0000","last_fullsized":"2024-09-11T08:47:02.663503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.349351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.349351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006276294,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":174,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.664115+0000","last_change":"2024-09-11T08:44:12.198399+0000","last_active":"2024-09-11T08:47:02.664115+0000","last_peered":"2024-09-11T08:47:02.664115+0000","last_clean":"2024-09-11T08:47:02.664115+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:47:02.664115+0000","last_undegraded":"2024-09-11T08:47:02.664115+0000","last_fullsized":"2024-09-11T08:47:02.664115+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:44:12.198335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.198335+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020517040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":178,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.649548+0000","last_change":"2024-09-11T08:45:14.574739+0000","last_active":"2024-09-11T08:47:02.649548+0000","last_peered":"2024-09-11T08:47:02.649548+0000","last_clean":"2024-09-11T08:47:02.649548+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:47:02.649548+0000","last_undegraded":"2024-09-11T08:47:02.649548+0000","last_fullsized":"2024-09-11T08:47:02.649548+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.574678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.574678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013819469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":173,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669807+0000","last_change":"2024-09-11T08:44:28.484120+0000","last_active":"2024-09-11T08:47:02.669807+0000","last_peered":"2024-09-11T08:47:02.669807+0000","last_clean":"2024-09-11T08:47:02.669807+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:47:02.669807+0000","last_undegraded":"2024-09-11T08:47:02.669807+0000","last_fullsized":"2024-09-11T08:47:02.669807+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:28.484047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.484047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001479172,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":171,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.666236+0000","last_change":"2024-09-11T08:46:01.984620+0000","last_active":"2024-09-11T08:47:02.666236+0000","last_peered":"2024-09-11T08:47:02.666236+0000","last_clean":"2024-09-11T08:47:02.666236+0000","last_became_active":"2024-09-11T08:46:01.984167+0000","last_became_peered":"2024-09-11T08:46:01.984167+0000","last_unstale":"2024-09-11T08:47:02.666236+0000","last_undegraded":"2024-09-11T08:47:02.666236+0000","last_fullsized":"2024-09-11T08:47:02.666236+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:32.114990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:32.114990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014089513999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"136'6","reported_seq":205,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.656436+0000","last_change":"2024-09-11T08:46:01.996138+0000","last_active":"2024-09-11T08:47:02.656436+0000","last_peered":"2024-09-11T08:47:02.656436+0000","last_clean":"2024-09-11T08:47:02.656436+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:47:02.656436+0000","last_undegraded":"2024-09-11T08:47:02.656436+0000","last_fullsized":"2024-09-11T08:47:02.656436+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'6","last_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:44:21.302870+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012512683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":185,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.664805+0000","last_change":"2024-09-11T08:45:21.481725+0000","last_active":"2024-09-11T08:47:02.664805+0000","last_peered":"2024-09-11T08:47:02.664805+0000","last_clean":"2024-09-11T08:47:02.664805+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:47:02.664805+0000","last_undegraded":"2024-09-11T08:47:02.664805+0000","last_fullsized":"2024-09-11T08:47:02.664805+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:21.481634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.481634+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025664590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":27,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.652651+0000","last_change":"2024-09-11T08:46:01.979915+0000","last_active":"2024-09-11T08:47:02.652651+0000","last_peered":"2024-09-11T08:47:02.652651+0000","last_clean":"2024-09-11T08:47:02.652651+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:47:02.652651+0000","last_undegraded":"2024-09-11T08:47:02.652651+0000","last_fullsized":"2024-09-11T08:47:02.652651+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.196277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.196277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":181,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668351+0000","last_change":"2024-09-11T08:45:23.348327+0000","last_active":"2024-09-11T08:47:02.668351+0000","last_peered":"2024-09-11T08:47:02.668351+0000","last_clean":"2024-09-11T08:47:02.668351+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:47:02.668351+0000","last_undegraded":"2024-09-11T08:47:02.668351+0000","last_fullsized":"2024-09-11T08:47:02.668351+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.348275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.348275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035843659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":182,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653338+0000","last_change":"2024-09-11T08:45:23.674163+0000","last_active":"2024-09-11T08:47:02.653338+0000","last_peered":"2024-09-11T08:47:02.653338+0000","last_clean":"2024-09-11T08:47:02.653338+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:47:02.653338+0000","last_undegraded":"2024-09-11T08:47:02.653338+0000","last_fullsized":"2024-09-11T08:47:02.653338+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.674102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.674102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001440293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.656677+0000","last_change":"2024-09-11T08:44:23.320368+0000","last_active":"2024-09-11T08:47:02.656677+0000","last_peered":"2024-09-11T08:47:02.656677+0000","last_clean":"2024-09-11T08:47:02.656677+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:47:02.656677+0000","last_undegraded":"2024-09-11T08:47:02.656677+0000","last_fullsized":"2024-09-11T08:47:02.656677+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:23.320301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:23.320301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017133368999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":39,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653769+0000","last_change":"2024-09-11T08:46:02.002471+0000","last_active":"2024-09-11T08:47:02.653769+0000","last_peered":"2024-09-11T08:47:02.653769+0000","last_clean":"2024-09-11T08:47:02.653769+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:47:02.653769+0000","last_undegraded":"2024-09-11T08:47:02.653769+0000","last_fullsized":"2024-09-11T08:47:02.653769+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:45:25.620635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.620635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":164,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.667175+0000","last_change":"2024-09-11T08:43:30.336272+0000","last_active":"2024-09-11T08:47:02.667175+0000","last_peered":"2024-09-11T08:47:02.667175+0000","last_clean":"2024-09-11T08:47:02.667175+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:47:02.667175+0000","last_undegraded":"2024-09-11T08:47:02.667175+0000","last_fullsized":"2024-09-11T08:47:02.667175+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:30.336188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:30.336188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044263979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":7070220312,"num_objects":107884,"num_object_clones":0,"num_object_copies":323652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3396,"num_objects_unfound":0,"num_objects_dirty":107884,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132034,"num_write_kb":7677313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":286588928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21698785280,"data_stored":21698773064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84734,"ondisk_log_size":84734,"up":48,"acting":47,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":1584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":456,"ondisk_log_size":456,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":505,"num_read_kb":500,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299500,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2468804,"kb_used_data":2158972,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91276348,"statfs":{"total":95995035648,"available":93466980352,"internally_reserved":0,"allocated":2210787328,"data_stored":2210474609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":2.726,"5min":1.3240000000000001,"15min":0.78200000000000003},"min":{"1min":0.315,"5min":0.22,"15min":0.22},"max":{"1min":32.171999999999997,"5min":32.171999999999997,"15min":32.171999999999997},"last":1.4179999999999999},{"interface":"front","average":{"1min":2.734,"5min":1.427,"15min":0.82799999999999996},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":32.149999999999999,"5min":32.149999999999999,"15min":32.149999999999999},"last":1.4339999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":2.891,"5min":1.5149999999999999,"15min":0.84899999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":33.692999999999998,"5min":33.692999999999998,"15min":33.692999999999998},"last":2.1110000000000002},{"interface":"front","average":{"1min":2.851,"5min":1.343,"15min":0.79800000000000004},"min":{"1min":0.32100000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":35.591000000000001,"5min":35.591000000000001,"15min":35.591000000000001},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.145,"15min":0.72299999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":4.7649999999999997,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.0660000000000001,"15min":0.69499999999999995},"min":{"1min":0.33900000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.609,"5min":5.7990000000000004,"15min":5.7990000000000004},"last":1.381}]},{"osd":3,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.179,"15min":0.74299999999999999},"min":{"1min":0.33000000000000002,"5min":0.25,"15min":0.25},"max":{"1min":2.4849999999999999,"5min":7.8609999999999998,"15min":7.8609999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.2789999999999999,"15min":0.79400000000000004},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.8370000000000002,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":2.323}]},{"osd":4,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.499,"15min":0.44900000000000001},"min":{"1min":0.24099999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.6020000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.751},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48699999999999999,"15min":0.44400000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.804,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.56699999999999995}]},{"osd":5,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.51600000000000001,"15min":0.46800000000000003},"min":{"1min":0.32000000000000001,"5min":0.255,"15min":0.23300000000000001},"max":{"1min":1.6399999999999999,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.6399999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.622,"15min":0.51000000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.496,"5min":2.806,"15min":2.806},"last":0.78400000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64400000000000002,"15min":0.53300000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.315},"max":{"1min":2.0259999999999998,"5min":3.5979999999999999,"15min":3.5979999999999999},"last":0.872},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.52300000000000002},"min":{"1min":0.30499999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.0990000000000002,"5min":2.633,"15min":2.633},"last":0.59499999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528433,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2261108,"kb_used_data":1951276,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91484044,"statfs":{"total":95995035648,"available":93679661056,"internally_reserved":0,"allocated":1998106624,"data_stored":1997810101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":1.4399999999999999,"15min":0.83099999999999996},"min":{"1min":0.222,"5min":0.217,"15min":0.217},"max":{"1min":2.0049999999999999,"5min":28.302,"15min":28.302},"last":3.2480000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":1.4390000000000001,"15min":0.83099999999999996},"min":{"1min":0.29899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":2.5750000000000002,"5min":28.314,"15min":28.314},"last":1.2150000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.1930000000000001,"15min":0.747},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.4649999999999999,"5min":12.586,"15min":12.586},"last":4.1749999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.077,"15min":0.70699999999999996},"min":{"1min":0.25700000000000001,"5min":0.253,"15min":0.253},"max":{"1min":3.2269999999999999,"5min":10.834,"15min":10.834},"last":9.5370000000000008}]},{"osd":2,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.92500000000000004,"15min":0.67300000000000004},"min":{"1min":0.27700000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.82299999999999995,"5min":14.313000000000001,"15min":14.313000000000001},"last":0.377},{"interface":"front","average":{"1min":0.754,"5min":0.95899999999999996,"15min":0.65900000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":2.4660000000000002,"5min":6.5979999999999999,"15min":6.5979999999999999},"last":0.46700000000000003}]},{"osd":3,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1.2529999999999999,"15min":0.76200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.2130000000000001,"5min":11.433,"15min":11.433},"last":1.286},{"interface":"front","average":{"1min":0.54800000000000004,"5min":1.1060000000000001,"15min":0.72799999999999998},"min":{"1min":0.311,"5min":0.224,"15min":0.224},"max":{"1min":1.194,"5min":9.1069999999999993,"15min":9.1069999999999993},"last":0.69899999999999995}]},{"osd":4,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.47799999999999998,"15min":0.433},"min":{"1min":0.25600000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.577,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.44600000000000001,"15min":0.42299999999999999},"min":{"1min":0.28299999999999997,"5min":0.27300000000000002,"15min":0.26400000000000001},"max":{"1min":1.1990000000000001,"5min":1.361,"15min":1.361},"last":0.54500000000000004}]},{"osd":5,"last update":"Wed Sep 11 08:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.53300000000000003,"15min":0.47099999999999997},"min":{"1min":0.32700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":1.246,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.505,"15min":0.46000000000000002},"min":{"1min":0.30199999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.2609999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.42599999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:46:34 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.61499999999999999,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.25900000000000001,"15min":0.247},"max":{"1min":7.327,"5min":7.327,"15min":7.327},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53100000000000003,"15min":0.505},"min":{"1min":0.373,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.95899999999999996,"5min":1.3,"15min":1.3},"last":0.59699999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757365,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3794112,"kb_used_data":3492600,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":89951040,"statfs":{"total":95995035648,"available":92109864960,"internally_reserved":0,"allocated":3576422400,"data_stored":3576113866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.0509999999999999,"15min":0.72799999999999998},"min":{"1min":0.23100000000000001,"5min":0.20899999999999999,"15min":0.20599999999999999},"max":{"1min":3.1850000000000001,"5min":7.3899999999999997,"15min":7.3899999999999997},"last":0.41099999999999998},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.226,"15min":0.77300000000000002},"min":{"1min":0.245,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":3.3799999999999999,"5min":6.5789999999999997,"15min":6.5789999999999997},"last":0.58699999999999997}]},{"osd":1,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.0549999999999999,"15min":0.755},"min":{"1min":0.23799999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":3.8029999999999999,"5min":3.8029999999999999,"15min":4.8419999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.0840000000000001,"15min":0.71599999999999997},"min":{"1min":0.35599999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.2120000000000002,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.54100000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.2010000000000001,"15min":0.76400000000000001},"min":{"1min":0.29499999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.0919999999999996,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.089,"5min":0.98399999999999999,"15min":0.70499999999999996},"min":{"1min":0.38500000000000001,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":3.2509999999999999,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.42799999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.0429999999999999,"15min":0.73499999999999999},"min":{"1min":0.35399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.6429999999999998,"5min":6.0460000000000003,"15min":6.0460000000000003},"last":1.901},{"interface":"front","average":{"1min":1.429,"5min":1.2889999999999999,"15min":0.82799999999999996},"min":{"1min":0.49199999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.415,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":0.86799999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.54800000000000004,"15min":0.44},"min":{"1min":0.27900000000000003,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":3.645,"5min":3.645,"15min":3.645},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.52200000000000002,"15min":0.44700000000000001},"min":{"1min":0.25800000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.3079999999999998,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.36499999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.52600000000000002,"15min":0.46300000000000002},"min":{"1min":0.32200000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":3.4329999999999998,"5min":3.4329999999999998,"15min":3.4329999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.51800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":1.893,"5min":2.319,"15min":2.319},"last":0.47099999999999997}]},{"osd":7,"last update":"Wed Sep 11 08:46:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.51700000000000002,"15min":0.496},"min":{"1min":0.36299999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.307,"15min":0.307},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.443}]}]},{"osd":4,"up_from":31,"seq":133143986297,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2461052,"kb_used_data":2163700,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91284100,"statfs":{"total":95995035648,"available":93474918400,"internally_reserved":0,"allocated":2215628800,"data_stored":2215323167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":2.6720000000000002,"15min":1.2509999999999999},"min":{"1min":0.23999999999999999,"5min":0.218,"15min":0.20399999999999999},"max":{"1min":1.7090000000000001,"5min":89.510000000000005,"15min":89.510000000000005},"last":2.016},{"interface":"front","average":{"1min":0.72799999999999998,"5min":2.7280000000000002,"15min":1.2889999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.9060000000000001,"5min":89.765000000000001,"15min":89.765000000000001},"last":0.23999999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.228,"15min":0.78100000000000003},"min":{"1min":0.28799999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.0960000000000001,"5min":18.596,"15min":18.596},"last":1.044},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.2070000000000001,"15min":0.77300000000000002},"min":{"1min":0.26100000000000001,"5min":0.24399999999999999,"15min":0.23599999999999999},"max":{"1min":6.3300000000000001,"5min":18.754000000000001,"15min":18.754000000000001},"last":2.9089999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.92700000000000005,"15min":0.68500000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":5.6660000000000004,"5min":5.6660000000000004,"15min":5.6660000000000004},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.91800000000000004,"15min":0.68200000000000005},"min":{"1min":0.27500000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":3.0030000000000001,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.58299999999999996}]},{"osd":3,"last update":"Wed Sep 11 08:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":3.0590000000000002,"15min":1.4039999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":2.9820000000000002,"5min":89.881,"15min":89.881},"last":2.234},{"interface":"front","average":{"1min":1.3480000000000001,"5min":2.9049999999999998,"15min":1.343},"min":{"1min":0.32300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":7.8330000000000002,"5min":89.715000000000003,"15min":89.715000000000003},"last":1.659}]},{"osd":5,"last update":"Wed Sep 11 08:47:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.53300000000000003,"15min":0.45000000000000001},"min":{"1min":0.29799999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":2.2389999999999999,"5min":2.516,"15min":2.516},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.54800000000000004,"15min":0.46100000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.23699999999999999},"max":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.377}]},{"osd":6,"last update":"Wed Sep 11 08:46:27 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.48899999999999999,"15min":0.45300000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":2.9390000000000001,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.57199999999999995,"15min":0.503},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":2.371,"5min":2.371,"15min":2.371},"last":0.496}]},{"osd":7,"last update":"Wed Sep 11 08:46:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.505,"15min":0.495},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.54400000000000004,"15min":0.502},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.28799999999999998},"max":{"1min":2.9609999999999999,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":0.41999999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149823,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2922900,"kb_used_data":2617228,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":90822252,"statfs":{"total":95995035648,"available":93001986048,"internally_reserved":0,"allocated":2680041472,"data_stored":2679712844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.6799999999999999,"15min":0.80200000000000005},"min":{"1min":0.20699999999999999,"5min":0.189,"15min":0.184},"max":{"1min":2.7040000000000002,"5min":10.901999999999999,"15min":10.901999999999999},"last":2.5430000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":2.0720000000000001,"15min":0.95099999999999996},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":4.6760000000000002,"5min":9.6820000000000004,"15min":9.6820000000000004},"last":0.17000000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:46:17 2024","interfaces":[{"interface":"back","average":{"1min":6.6299999999999999,"5min":3.0699999999999998,"15min":1.2869999999999999},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":115.601,"5min":115.601,"15min":115.601},"last":3.6160000000000001},{"interface":"front","average":{"1min":6.8380000000000001,"5min":2.9790000000000001,"15min":1.262},"min":{"1min":0.251,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":115.474,"5min":115.474,"15min":115.474},"last":0.30399999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:46:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.875,"15min":0.90900000000000003},"min":{"1min":0.32400000000000001,"5min":0.251,"15min":0.23200000000000001},"max":{"1min":2.8109999999999999,"5min":8.9510000000000005,"15min":8.9510000000000005},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":1.7909999999999999,"15min":0.876},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.1749999999999998,"5min":10.695,"15min":10.695},"last":2.5070000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:46:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.4279999999999999,"15min":0.85599999999999998},"min":{"1min":0.28599999999999998,"5min":0.27000000000000002,"15min":0.26700000000000002},"max":{"1min":5.5460000000000003,"5min":8.9969999999999999,"15min":8.9969999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.2450000000000001,"15min":0.78200000000000003},"min":{"1min":0.34599999999999997,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.7599999999999998,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":2.5649999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:46:49 2024","interfaces":[{"interface":"back","average":{"1min":1.843,"5min":1.6020000000000001,"15min":0.92300000000000004},"min":{"1min":0.29799999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":8.3599999999999994,"5min":11.526999999999999,"15min":11.526999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.296,"15min":0.81999999999999995},"min":{"1min":0.40300000000000002,"5min":0.30599999999999999,"15min":0.30499999999999999},"max":{"1min":3.7949999999999999,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":2.609}]},{"osd":6,"last update":"Wed Sep 11 08:46:17 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.5840000000000001,"15min":0.92200000000000004},"min":{"1min":0.45700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.9649999999999999,"5min":11.494999999999999,"15min":11.494999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.171,"5min":1.605,"15min":0.91000000000000003},"min":{"1min":0.314,"5min":0.311,"15min":0.311},"max":{"1min":4.6310000000000002,"5min":7.7249999999999996,"15min":7.7249999999999996},"last":3.5249999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:46:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.597,"15min":0.94899999999999995},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":8.3879999999999999,"5min":11.561,"15min":11.561},"last":0.754},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.343,"15min":0.85299999999999998},"min":{"1min":0.40000000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":5.6280000000000001,"5min":6.3170000000000002,"15min":6.3170000000000002},"last":2.5859999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411459,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3155756,"kb_used_data":2858404,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":90589396,"statfs":{"total":95995035648,"available":92763541504,"internally_reserved":0,"allocated":2927005696,"data_stored":2926716182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:54 2024","interfaces":[{"interface":"back","average":{"1min":4.9740000000000002,"5min":2.7919999999999998,"15min":1.1970000000000001},"min":{"1min":0.16500000000000001,"5min":0.159,"15min":0.159},"max":{"1min":34.713999999999999,"5min":34.713999999999999,"15min":34.713999999999999},"last":2.8439999999999999},{"interface":"front","average":{"1min":4.2430000000000003,"5min":2.9350000000000001,"15min":1.282},"min":{"1min":0.26200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":34.399000000000001,"5min":34.399000000000001,"15min":34.399000000000001},"last":3.0659999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:46:54 2024","interfaces":[{"interface":"back","average":{"1min":4.6509999999999998,"5min":2.1560000000000001,"15min":1.0009999999999999},"min":{"1min":0.28299999999999997,"5min":0.215,"15min":0.19},"max":{"1min":34.771000000000001,"5min":34.771000000000001,"15min":34.771000000000001},"last":3.1099999999999999},{"interface":"front","average":{"1min":4.0880000000000001,"5min":2.2639999999999998,"15min":1.044},"min":{"1min":0.252,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":35.209000000000003,"5min":35.209000000000003,"15min":35.209000000000003},"last":3.02}]},{"osd":3,"last update":"Wed Sep 11 08:47:10 2024","interfaces":[{"interface":"back","average":{"1min":4.2190000000000003,"5min":2.3809999999999998,"15min":1.167},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":34.75,"5min":34.75,"15min":34.75},"last":3.702},{"interface":"front","average":{"1min":3.948,"5min":2.3239999999999998,"15min":1.1479999999999999},"min":{"1min":0.40999999999999998,"5min":0.27200000000000002,"15min":0.26800000000000002},"max":{"1min":32.802,"5min":32.802,"15min":32.802},"last":3.0419999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:46:26 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.381,"15min":0.82699999999999996},"min":{"1min":0.29799999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":8.0820000000000007,"5min":8.0820000000000007,"15min":8.0820000000000007},"last":2.8820000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.304,"15min":0.80200000000000005},"min":{"1min":0.35499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":4.851,"5min":6.9320000000000004,"15min":6.9320000000000004},"last":2.996}]},{"osd":5,"last update":"Wed Sep 11 08:46:54 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.296,"15min":0.81899999999999995},"min":{"1min":0.439,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.8039999999999998,"5min":6.9550000000000001,"15min":6.9550000000000001},"last":2.9710000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0900000000000001,"15min":0.74199999999999999},"min":{"1min":0.29199999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.9940000000000002,"5min":5.9530000000000003,"15min":5.9530000000000003},"last":3.165}]},{"osd":6,"last update":"Wed Sep 11 08:46:17 2024","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.4079999999999999,"15min":0.85699999999999998},"min":{"1min":0.38700000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":8.1120000000000001,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":2.9060000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.2809999999999999,"15min":0.80000000000000004},"min":{"1min":0.38500000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.8290000000000002,"5min":6.9779999999999998,"15min":6.9779999999999998},"last":2.931}]},{"osd":7,"last update":"Wed Sep 11 08:46:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.137,"15min":0.76400000000000001},"min":{"1min":0.32500000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.9489999999999998,"5min":5.9909999999999997,"15min":5.9909999999999997},"last":3.1400000000000001},{"interface":"front","average":{"1min":1.607,"5min":1.444,"15min":0.878},"min":{"1min":0.42099999999999999,"5min":0.371,"15min":0.36099999999999999},"max":{"1min":8.1400000000000006,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":3.081}]}]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3362904,"kb_used_data":3052944,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":90382248,"statfs":{"total":95995035648,"available":92551421952,"internally_reserved":0,"allocated":3126214656,"data_stored":3125881311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,0,0,1,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:31 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":2.8929999999999998,"15min":1.1879999999999999},"min":{"1min":0.20499999999999999,"5min":0.183,"15min":0.183},"max":{"1min":4.8319999999999999,"5min":117.01600000000001,"15min":117.01600000000001},"last":1.304},{"interface":"front","average":{"1min":1.915,"5min":3.6200000000000001,"15min":1.4490000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":6.7370000000000001,"5min":117.62,"15min":117.62},"last":0.41099999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:46:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.694,"15min":0.90200000000000002},"min":{"1min":0.22800000000000001,"5min":0.191,"15min":0.191},"max":{"1min":4.6379999999999999,"5min":9.5719999999999992,"15min":20.628},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.647,"15min":0.86899999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":7.4080000000000004,"5min":11.461,"15min":20.148},"last":0.46999999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":2.5369999999999999,"15min":1.1180000000000001},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":2.2730000000000001,"5min":116.98,"15min":116.98},"last":4.6299999999999999},{"interface":"front","average":{"1min":0.99299999999999999,"5min":2.891,"15min":1.25},"min":{"1min":0.26400000000000001,"5min":0.22,"15min":0.22},"max":{"1min":4.2679999999999998,"5min":117.56699999999999,"15min":117.56699999999999},"last":0.434}]},{"osd":4,"last update":"Wed Sep 11 08:46:31 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.1899999999999999,"15min":0.75800000000000001},"min":{"1min":0.38400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.4780000000000002,"5min":11.19,"15min":11.19},"last":0.34799999999999998},{"interface":"front","average":{"1min":1.262,"5min":1.0880000000000001,"15min":0.73699999999999999},"min":{"1min":0.32600000000000001,"5min":0.251,"15min":0.251},"max":{"1min":5.056,"5min":7.7169999999999996,"15min":7.7169999999999996},"last":0.312}]},{"osd":5,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":2.6240000000000001,"15min":1.2769999999999999},"min":{"1min":0.34200000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.6680000000000001,"5min":103.337,"15min":103.337},"last":0.51900000000000002},{"interface":"front","average":{"1min":1.05,"5min":2.4980000000000002,"15min":1.2170000000000001},"min":{"1min":0.41599999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.6269999999999998,"5min":103.163,"15min":103.163},"last":0.39000000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:46:25 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0529999999999999,"15min":0.73799999999999999},"min":{"1min":0.34100000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.2410000000000001,"5min":8.9149999999999991,"15min":8.9149999999999991},"last":0.36899999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.3740000000000001,"15min":0.86099999999999999},"min":{"1min":0.36299999999999999,"5min":0.31,"15min":0.31},"max":{"1min":3.2639999999999998,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":0.45300000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:46:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.2450000000000001,"15min":0.81100000000000005},"min":{"1min":0.38400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":3.7519999999999998,"5min":11.315,"15min":11.315},"last":0.50700000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.1000000000000001,"15min":0.77000000000000002},"min":{"1min":0.376,"5min":0.315,"15min":0.315},"max":{"1min":4.8739999999999997,"5min":8.8800000000000008,"15min":8.8800000000000008},"last":0.39800000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738508,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3308292,"kb_used_data":2998460,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":90436860,"statfs":{"total":95995035648,"available":92607344640,"internally_reserved":0,"allocated":3070423040,"data_stored":3070110356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:46:28 2024","interfaces":[{"interface":"back","average":{"1min":2,"5min":1.9610000000000001,"15min":0.86599999999999999},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":8.3900000000000006,"5min":8.3900000000000006,"15min":8.3900000000000006},"last":0.83299999999999996},{"interface":"front","average":{"1min":2.431,"5min":1.712,"15min":0.79700000000000004},"min":{"1min":0.30599999999999999,"5min":0.186,"15min":0.186},"max":{"1min":12.532,"5min":12.532,"15min":12.532},"last":0.48799999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:46:53 2024","interfaces":[{"interface":"back","average":{"1min":2.4830000000000001,"5min":1.843,"15min":0.96299999999999997},"min":{"1min":0.32800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":12.481,"5min":12.481,"15min":19.940000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":2.1240000000000001,"5min":1.9410000000000001,"15min":1.0069999999999999},"min":{"1min":0.39800000000000002,"5min":0.23599999999999999,"15min":0.22700000000000001},"max":{"1min":6.6479999999999997,"5min":8.3049999999999997,"15min":20.757000000000001},"last":0.35999999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:47:09 2024","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":2.1749999999999998,"15min":1.0960000000000001},"min":{"1min":0.312,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.5709999999999997,"5min":11.951000000000001,"15min":11.951000000000001},"last":0.312},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.919,"15min":1.0109999999999999},"min":{"1min":0.28100000000000003,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":7.6399999999999997,"5min":7.6399999999999997,"15min":7.6399999999999997},"last":0.33000000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:46:32 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.47,"15min":0.85199999999999998},"min":{"1min":0.30599999999999999,"5min":0.25,"15min":0.25},"max":{"1min":4.8490000000000002,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":3.6909999999999998},{"interface":"front","average":{"1min":1.252,"5min":1.3759999999999999,"15min":0.82299999999999995},"min":{"1min":0.27600000000000002,"5min":0.255,"15min":0.255},"max":{"1min":3.9860000000000002,"5min":6.9539999999999997,"15min":6.9539999999999997},"last":0.44700000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:46:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.5369999999999999,"15min":0.91800000000000004},"min":{"1min":0.374,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.887,"5min":7.4009999999999998,"15min":7.4009999999999998},"last":0.5},{"interface":"front","average":{"1min":1.754,"5min":1.577,"15min":0.91600000000000004},"min":{"1min":0.32700000000000001,"5min":0.30499999999999999,"15min":0.28299999999999997},"max":{"1min":4.5970000000000004,"5min":7.423,"15min":7.423},"last":0.51700000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:46:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.615,"15min":0.90000000000000002},"min":{"1min":0.372,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.609,"5min":7.4240000000000004,"15min":7.4240000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.6220000000000001,"15min":0.92300000000000004},"min":{"1min":0.41499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":3.6840000000000002,"5min":7.4489999999999998,"15min":7.4489999999999998},"last":0.63300000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":1.716,"5min":1.6000000000000001,"15min":0.93500000000000005},"min":{"1min":0.32700000000000001,"5min":0.313,"15min":0.313},"max":{"1min":7.4900000000000002,"5min":7.4900000000000002,"15min":7.4900000000000002},"last":0.624},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.6180000000000001,"15min":0.92400000000000004},"min":{"1min":0.41499999999999998,"5min":0.307,"15min":0.307},"max":{"1min":3.8279999999999998,"5min":7.4450000000000003,"15min":7.4450000000000003},"last":0.78000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3066429440,"data_stored":3066429440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3105685504,"data_stored":3105685504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2908688384,"data_stored":2908684312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2659450880,"data_stored":2659450880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2214526976,"data_stored":2214526976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3565555712,"data_stored":3565551640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1983840256,"data_stored":1983840256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2194608128,"data_stored":2194604056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:47:16.593 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:16.630 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:16.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:16.631 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:16.731 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:16.765 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:16.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:16.766 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:16.866 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:16.897 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:16.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:16.898 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:16.912 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:47:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:47:16.964 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:16.998 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:17.023 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:17.025 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:17.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:17.026 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:17.127 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:17.156 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:17.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:17.157 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:17.258 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:17.289 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:17.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:17.290 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:17.389 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:17 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:17.389 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:17 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:17.391 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:17.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:17 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:17.426 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:17.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:17.427 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:17.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:17.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:17.565 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:17.565 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:17.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:17.697 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:17.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:17.698 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:17.799 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:17.834 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:17.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:17.835 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:17.936 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:17.967 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:17.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:17.967 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:18.068 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:18.095 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:18.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:18.096 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:18.196 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:18.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:18.228 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:18.229 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:18.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:18 smithi092 ceph-mon[29647]: pgmap v360: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.3 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s; 2927/308586 objects misplaced (0.949%); 5.6 MiB/s, 90 objects/s recovering 2024-09-11T08:47:18.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:18 smithi092 ceph-mon[41114]: pgmap v360: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.3 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s; 2927/308586 objects misplaced (0.949%); 5.6 MiB/s, 90 objects/s recovering 2024-09-11T08:47:18.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:18.365 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:18.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:18.366 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:18.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:18 smithi145 ceph-mon[33384]: pgmap v360: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.3 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s; 2927/308586 objects misplaced (0.949%); 5.6 MiB/s, 90 objects/s recovering 2024-09-11T08:47:18.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:18.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:18.509 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:18.509 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:18.610 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:18.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:18.647 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:18.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:18.780 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:18.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:18.781 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:18.882 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:18.908 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:18.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:18.909 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:19.009 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:19.035 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:19.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:19.036 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:19.136 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:19.167 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:19.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:19.168 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:19.270 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:19.309 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:19.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:19.310 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:19.410 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:19.444 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:19.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:19.445 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:19.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:19.575 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:19.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:19.576 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:19.595 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:47:19.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:19.717 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:19.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:19.718 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:19.819 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:19.844 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:19.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:19.845 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:19.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:19.990 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:19.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:19.991 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:20.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:20.130 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:20.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:20.131 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:20.232 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:20.263 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:20.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:20.265 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:20.365 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:20.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:20.409 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:20.410 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:20.510 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:20.541 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:20.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:20.542 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:20.559 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:20 smithi092 ceph-mon[29647]: pgmap v361: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.3 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 862 B/s rd, 0 B/s wr, 899 op/s; 2927/308586 objects misplaced (0.949%); 4.7 MiB/s, 75 objects/s recovering 2024-09-11T08:47:20.559 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:20 smithi092 ceph-mon[41114]: pgmap v361: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.3 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 862 B/s rd, 0 B/s wr, 899 op/s; 2927/308586 objects misplaced (0.949%); 4.7 MiB/s, 75 objects/s recovering 2024-09-11T08:47:20.643 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:20.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:20 smithi145 ceph-mon[33384]: pgmap v361: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.3 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 862 B/s rd, 0 B/s wr, 899 op/s; 2927/308586 objects misplaced (0.949%); 4.7 MiB/s, 75 objects/s recovering 2024-09-11T08:47:20.672 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:20.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:20.673 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:20.774 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:20.807 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:20.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:20.809 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:20.909 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:20.964 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:20.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:20.965 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:21.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:21.112 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:21.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:21.114 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:21.215 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:21.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:21.252 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:21.252 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:21.353 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:21.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:21.383 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:21.383 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:21.484 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:21.527 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:21.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:21.528 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:21.553 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:47:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:47:21] "GET /metrics HTTP/1.1" 200 224358 "" "Prometheus/2.43.0" 2024-09-11T08:47:21.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:21.670 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:21.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:21.671 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:21.772 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:21.803 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:21.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:21.804 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:21.905 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:21.933 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:21.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:21.934 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:22.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:22.087 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:22.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:22.088 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:22.178 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:47:22.178 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:47:22.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:22.208 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:47:22.209 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:47:22.225 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:22.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:22.226 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:22.244 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:47:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:47:22.282 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:47:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:47:22.296 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:47:22.297 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:47:22.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.297 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.15126.0:143753 8.7 e185/168)", 2024-09-11T08:47:22.297 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:22.214480+0000", 2024-09-11T08:47:22.297 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00044735100000000002, 2024-09-11T08:47:22.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000471788, 2024-09-11T08:47:22.297 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:47:22.297 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-11T08:47:22.297 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:47:22.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:47:22.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214480+0000", 2024-09-11T08:47:22.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:22.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:47:22.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214480+0000", 2024-09-11T08:47:22.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:22.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:47:22.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214479+0000", 2024-09-11T08:47:22.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:47:22.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:47:22.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214486+0000", 2024-09-11T08:47:22.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8059999999999997e-06 2024-09-11T08:47:22.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:47:22.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214498+0000", 2024-09-11T08:47:22.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1518000000000001e-05 2024-09-11T08:47:22.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:47:22.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214503+0000", 2024-09-11T08:47:22.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6200000000000004e-06 2024-09-11T08:47:22.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:22.299 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:47:22.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:22.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.300 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.15126.0:143754 8.7 e185/168)", 2024-09-11T08:47:22.300 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:22.214510+0000", 2024-09-11T08:47:22.300 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00041730899999999999, 2024-09-11T08:47:22.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049603099999999997, 2024-09-11T08:47:22.300 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:47:22.300 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-11T08:47:22.300 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:47:22.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:47:22.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214510+0000", 2024-09-11T08:47:22.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:22.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:47:22.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214510+0000", 2024-09-11T08:47:22.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:22.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:47:22.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214510+0000", 2024-09-11T08:47:22.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:47:22.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:47:22.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214513+0000", 2024-09-11T08:47:22.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.627e-06 2024-09-11T08:47:22.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:47:22.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214518+0000", 2024-09-11T08:47:22.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2390000000000002e-06 2024-09-11T08:47:22.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:47:22.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214521+0000", 2024-09-11T08:47:22.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3890000000000001e-06 2024-09-11T08:47:22.302 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:22.303 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:47:22.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:22.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.303 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.15126.0:143760 8.7 e185/168)", 2024-09-11T08:47:22.303 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:22.214525+0000", 2024-09-11T08:47:22.303 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00040244500000000003, 2024-09-11T08:47:22.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052873899999999997, 2024-09-11T08:47:22.303 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:47:22.303 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-11T08:47:22.303 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:47:22.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:47:22.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214525+0000", 2024-09-11T08:47:22.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:22.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:47:22.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214525+0000", 2024-09-11T08:47:22.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:22.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:47:22.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214525+0000", 2024-09-11T08:47:22.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967296 2024-09-11T08:47:22.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:47:22.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214527+0000", 2024-09-11T08:47:22.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1179999999999999e-06 2024-09-11T08:47:22.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:47:22.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214532+0000", 2024-09-11T08:47:22.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2209999999999996e-06 2024-09-11T08:47:22.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:22.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:22.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:47:22.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:22.214536+0000", 2024-09-11T08:47:22.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.27e-06 2024-09-11T08:47:22.305 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:22.305 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:47:22.305 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:22.306 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:22.306 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:47:22.306 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 3 2024-09-11T08:47:22.306 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:47:22.320 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:47:22.321 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:47:22.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:22.357 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:47:22.358 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:47:22.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:22.373 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:22.394 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:47:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:47:22.426 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:47:22.426 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:47:22.426 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:22 smithi092 ceph-mon[29647]: 5.7 scrub starts 2024-09-11T08:47:22.426 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:22 smithi092 ceph-mon[29647]: pgmap v362: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s; 2927/303198 objects misplaced (0.965%); 4.7 MiB/s, 75 objects/s recovering 2024-09-11T08:47:22.426 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:22 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T08:47:22.426 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:22 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2581006844' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:47:22.427 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:22 smithi092 ceph-mon[41114]: 5.7 scrub starts 2024-09-11T08:47:22.427 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:22 smithi092 ceph-mon[41114]: pgmap v362: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s; 2927/303198 objects misplaced (0.965%); 4.7 MiB/s, 75 objects/s recovering 2024-09-11T08:47:22.427 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:22 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T08:47:22.427 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:22 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2581006844' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:47:22.474 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:22.507 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:22.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:22.508 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:22.609 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:22.647 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:22.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:22.648 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:22.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:22 smithi145 ceph-mon[33384]: 5.7 scrub starts 2024-09-11T08:47:22.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:22 smithi145 ceph-mon[33384]: pgmap v362: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s; 2927/303198 objects misplaced (0.965%); 4.7 MiB/s, 75 objects/s recovering 2024-09-11T08:47:22.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:22 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T08:47:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:22 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2581006844' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:47:22.749 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:22.784 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:22.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:22.786 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:22.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:22.923 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:22.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:22.924 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:23.025 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:23.062 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:23.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:23.063 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:23.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:47:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:47:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:47:23.163 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:23.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:23.198 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:23.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:23.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:23.335 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:23.336 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:23.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:23.489 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:23.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:23.490 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:23.591 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:23.623 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:23.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:23.624 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:23.725 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:23.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:23.763 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:23.763 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:23.864 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:23.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:23.893 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:23.894 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:23.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:24.015 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:24.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:24.016 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:24.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:24.152 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:24.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:24.153 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:24.254 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:24.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:24.285 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:24.285 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:24.386 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:24.418 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:24.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:24.419 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:24.520 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:24.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:24.553 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:24.553 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:24.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:24 smithi092 ceph-mon[29647]: pgmap v363: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.9 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.21k op/s; 2374/291618 objects misplaced (0.814%); 7.6 MiB/s, 121 objects/s recovering 2024-09-11T08:47:24.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:24 smithi092 ceph-mon[41114]: pgmap v363: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.9 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.21k op/s; 2374/291618 objects misplaced (0.814%); 7.6 MiB/s, 121 objects/s recovering 2024-09-11T08:47:24.654 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:24.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:24 smithi145 ceph-mon[33384]: pgmap v363: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.9 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.21k op/s; 2374/291618 objects misplaced (0.814%); 7.6 MiB/s, 121 objects/s recovering 2024-09-11T08:47:24.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:24.686 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:24.686 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:24.787 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:24.818 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:24.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:24.819 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:24.884 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-11T08:47:24.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:24.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:24.969 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:24.969 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:25.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:25.122 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:25.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:25.156 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:25.156 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:25.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:47:25.257 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:25.289 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:25.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:25.290 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:25.391 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:25.416 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:25.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:25.417 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:25.518 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:25.542 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:25.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:25.543 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:25.644 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:25.677 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:25.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:25.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:25.779 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:25.813 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:25.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:25.814 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:25.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:25.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:25.961 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:26.062 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:26.070 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:26.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:26.112 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:26.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:26.213 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:26.249 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:26.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:26.250 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:26.273 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:47:26.350 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:26.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:26.385 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:26.386 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:26.466 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:26 smithi092 ceph-mon[29647]: pgmap v364: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.9 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 906 op/s; 2374/291618 objects misplaced (0.814%); 5.3 MiB/s, 85 objects/s recovering 2024-09-11T08:47:26.466 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:26 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:47:26.467 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:26 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:47:26.467 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:26 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2147367924' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:47:26.467 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:26 smithi092 ceph-mon[41114]: pgmap v364: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.9 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 906 op/s; 2374/291618 objects misplaced (0.814%); 5.3 MiB/s, 85 objects/s recovering 2024-09-11T08:47:26.467 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:26 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:47:26.467 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:26 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:47:26.467 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:26 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2147367924' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:47:26.487 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:26.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:26.547 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:26.548 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:26.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:26.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:47:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:47:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:26 smithi145 ceph-mon[33384]: pgmap v364: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.9 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 906 op/s; 2374/291618 objects misplaced (0.814%); 5.3 MiB/s, 85 objects/s recovering 2024-09-11T08:47:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:26 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:47:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:26 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:47:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:26 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2147367924' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:47:26.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:26.678 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:26.679 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:26.779 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:26.811 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:26.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:26.813 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:26.913 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:26.942 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:26.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:26.943 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:27.044 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:27.082 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:27.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:27.084 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:27.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:27.225 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:27.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:27.226 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:27.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:27 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:47:27.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:27 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2147367924' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:47:27.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:27 smithi092 ceph-mon[29647]: osdmap e186: 8 total, 8 up, 8 in 2024-09-11T08:47:27.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:27 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:27.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:27 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:47:27.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:27 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2147367924' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:47:27.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:27 smithi092 ceph-mon[41114]: osdmap e186: 8 total, 8 up, 8 in 2024-09-11T08:47:27.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:27 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:27.327 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:27.355 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:27.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:27.357 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:27.457 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:27.488 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:27.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:27.489 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:27.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:27.645 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:27.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:27.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:27.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:27 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:47:27.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:27 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2147367924' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:47:27.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:27 smithi145 ceph-mon[33384]: osdmap e186: 8 total, 8 up, 8 in 2024-09-11T08:47:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:27 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:27.747 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:27.774 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:27.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:27.775 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:27.856 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:47:27.859 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:47:27.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:27.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:27.921 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:28.022 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:28.055 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:28.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:28.056 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:28.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:28.194 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:28.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:28.195 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:28.296 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:28.329 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:28.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:28.330 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:28.430 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:28.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:28.466 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:28.466 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:28.562 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:28 smithi145 ceph-mon[33384]: pgmap v366: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.6 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.11k op/s; 1799/275145 objects misplaced (0.654%); 7.0 MiB/s, 112 objects/s recovering 2024-09-11T08:47:28.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:28 smithi092 ceph-mon[29647]: pgmap v366: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.6 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.11k op/s; 1799/275145 objects misplaced (0.654%); 7.0 MiB/s, 112 objects/s recovering 2024-09-11T08:47:28.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:28 smithi092 ceph-mon[41114]: pgmap v366: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.6 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.11k op/s; 1799/275145 objects misplaced (0.654%); 7.0 MiB/s, 112 objects/s recovering 2024-09-11T08:47:28.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:28.602 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:28.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:28.603 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:28.704 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:28.740 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:28.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:28.741 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:28.842 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:28.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:28.879 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:28.879 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:28.980 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:29.010 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:29.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:29.011 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:29.036 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-11T08:47:27.183974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5998319860,"num_objects":91715,"num_object_clones":0,"num_object_copies":275145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1799,"num_objects_unfound":0,"num_objects_dirty":91715,"num_whiteouts":0,"num_read":1371,"num_read_kb":1228,"num_write":148942,"num_write_kb":7678984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6074,"num_bytes_recovered":391841936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85115,"ondisk_log_size":85115,"up":444,"acting":443,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":446,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":20612848,"kb_used_data":18183784,"kb_used_omap":13,"kb_used_meta":2428658,"kb_avail":729348368,"statfs":{"total":767960285184,"available":746852728832,"internally_reserved":0,"allocated":18620194816,"data_stored":18617699070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2486946496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,3,9,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-730529792,"num_objects":-11147,"num_object_clones":0,"num_object_copies":-33441,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1128,"num_objects_unfound":0,"num_objects_dirty":-11147,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11147,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1128,"num_bytes_recovered":73924608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003903"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":179,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313865+0000","last_change":"2024-09-11T08:45:06.574593+0000","last_active":"2024-09-11T08:47:07.313865+0000","last_peered":"2024-09-11T08:47:07.313865+0000","last_clean":"2024-09-11T08:47:07.313865+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:47:07.313865+0000","last_undegraded":"2024-09-11T08:47:07.313865+0000","last_fullsized":"2024-09-11T08:47:07.313865+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:06.574539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.574539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028477630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":226,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654433+0000","last_change":"2024-09-11T08:46:01.991434+0000","last_active":"2024-09-11T08:47:02.654433+0000","last_peered":"2024-09-11T08:47:02.654433+0000","last_clean":"2024-09-11T08:47:02.654433+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:47:02.654433+0000","last_undegraded":"2024-09-11T08:47:02.654433+0000","last_fullsized":"2024-09-11T08:47:02.654433+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:45:13.113315+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.113315+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018068470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":175,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313478+0000","last_change":"2024-09-11T08:44:25.796953+0000","last_active":"2024-09-11T08:47:07.313478+0000","last_peered":"2024-09-11T08:47:07.313478+0000","last_clean":"2024-09-11T08:47:07.313478+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:47:07.313478+0000","last_undegraded":"2024-09-11T08:47:07.313478+0000","last_fullsized":"2024-09-11T08:47:07.313478+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:25.796699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:25.796699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025405829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":152,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654874+0000","last_change":"2024-09-11T08:44:26.005322+0000","last_active":"2024-09-11T08:47:02.654874+0000","last_peered":"2024-09-11T08:47:02.654874+0000","last_clean":"2024-09-11T08:47:02.654874+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:47:02.654874+0000","last_undegraded":"2024-09-11T08:47:02.654874+0000","last_fullsized":"2024-09-11T08:47:02.654874+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.005270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.005270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038675070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":182,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.308483+0000","last_change":"2024-09-11T08:46:01.984446+0000","last_active":"2024-09-11T08:47:07.308483+0000","last_peered":"2024-09-11T08:47:07.308483+0000","last_clean":"2024-09-11T08:47:07.308483+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:47:07.308483+0000","last_undegraded":"2024-09-11T08:47:07.308483+0000","last_fullsized":"2024-09-11T08:47:07.308483+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:28.356172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.356172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029955870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":188,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.311991+0000","last_change":"2024-09-11T08:46:01.981971+0000","last_active":"2024-09-11T08:47:07.311991+0000","last_peered":"2024-09-11T08:47:07.311991+0000","last_clean":"2024-09-11T08:47:07.311991+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:47:07.311991+0000","last_undegraded":"2024-09-11T08:47:07.311991+0000","last_fullsized":"2024-09-11T08:47:07.311991+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.330642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.330642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0087830849999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":198,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.311591+0000","last_change":"2024-09-11T08:44:24.344328+0000","last_active":"2024-09-11T08:47:07.311591+0000","last_peered":"2024-09-11T08:47:07.311591+0000","last_clean":"2024-09-11T08:47:07.311591+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:47:07.311591+0000","last_undegraded":"2024-09-11T08:47:07.311591+0000","last_fullsized":"2024-09-11T08:47:07.311591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:44:24.344127+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.344127+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010996336000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":154,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654424+0000","last_change":"2024-09-11T08:43:28.988751+0000","last_active":"2024-09-11T08:47:02.654424+0000","last_peered":"2024-09-11T08:47:02.654424+0000","last_clean":"2024-09-11T08:47:02.654424+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:47:02.654424+0000","last_undegraded":"2024-09-11T08:47:02.654424+0000","last_fullsized":"2024-09-11T08:47:02.654424+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:43:28.988697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00375019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":191,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.650076+0000","last_change":"2024-09-11T08:46:17.669115+0000","last_active":"2024-09-11T08:47:02.650076+0000","last_peered":"2024-09-11T08:47:02.650076+0000","last_clean":"2024-09-11T08:47:02.650076+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:47:02.650076+0000","last_undegraded":"2024-09-11T08:47:02.650076+0000","last_fullsized":"2024-09-11T08:47:02.650076+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:12.878132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.878132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077623500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":186,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655020+0000","last_change":"2024-09-11T08:44:27.963772+0000","last_active":"2024-09-11T08:47:02.655020+0000","last_peered":"2024-09-11T08:47:02.655020+0000","last_clean":"2024-09-11T08:47:02.655020+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:47:02.655020+0000","last_undegraded":"2024-09-11T08:47:02.655020+0000","last_fullsized":"2024-09-11T08:47:02.655020+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:44:27.963689+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:44:27.963689+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011544877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":170,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:09.804752+0000","last_change":"2024-09-11T08:47:09.804752+0000","last_active":"2024-09-11T08:47:09.804752+0000","last_peered":"2024-09-11T08:47:09.804752+0000","last_clean":"2024-09-11T08:47:09.804752+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:47:09.804752+0000","last_undegraded":"2024-09-11T08:47:09.804752+0000","last_fullsized":"2024-09-11T08:47:09.804752+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:33.644869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.644869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020832210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":173,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.314377+0000","last_change":"2024-09-11T08:45:31.457775+0000","last_active":"2024-09-11T08:47:07.314377+0000","last_peered":"2024-09-11T08:47:07.314377+0000","last_clean":"2024-09-11T08:47:07.314377+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:47:07.314377+0000","last_undegraded":"2024-09-11T08:47:07.314377+0000","last_fullsized":"2024-09-11T08:47:07.314377+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.457652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.457652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006759045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":171,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.650405+0000","last_change":"2024-09-11T08:46:22.625087+0000","last_active":"2024-09-11T08:47:02.650405+0000","last_peered":"2024-09-11T08:47:02.650405+0000","last_clean":"2024-09-11T08:47:02.650405+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:47:02.650405+0000","last_undegraded":"2024-09-11T08:47:02.650405+0000","last_fullsized":"2024-09-11T08:47:02.650405+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:09.871637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:09.871637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029269349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":166,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655235+0000","last_change":"2024-09-11T08:43:26.983719+0000","last_active":"2024-09-11T08:47:02.655235+0000","last_peered":"2024-09-11T08:47:02.655235+0000","last_clean":"2024-09-11T08:47:02.655235+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:47:02.655235+0000","last_undegraded":"2024-09-11T08:47:02.655235+0000","last_fullsized":"2024-09-11T08:47:02.655235+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:43:26.983587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:26.983587+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082679390000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":173,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668371+0000","last_change":"2024-09-11T08:45:13.320968+0000","last_active":"2024-09-11T08:47:02.668371+0000","last_peered":"2024-09-11T08:47:02.668371+0000","last_clean":"2024-09-11T08:47:02.668371+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:47:02.668371+0000","last_undegraded":"2024-09-11T08:47:02.668371+0000","last_fullsized":"2024-09-11T08:47:02.668371+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:13.320898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.320898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013000240000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":165,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.650587+0000","last_change":"2024-09-11T08:46:31.564511+0000","last_active":"2024-09-11T08:47:02.650587+0000","last_peered":"2024-09-11T08:47:02.650587+0000","last_clean":"2024-09-11T08:47:02.650587+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:47:02.650587+0000","last_undegraded":"2024-09-11T08:47:02.650587+0000","last_fullsized":"2024-09-11T08:47:02.650587+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.850684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.850684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058755379999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":182,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654895+0000","last_change":"2024-09-11T08:46:01.978488+0000","last_active":"2024-09-11T08:47:02.654895+0000","last_peered":"2024-09-11T08:47:02.654895+0000","last_clean":"2024-09-11T08:47:02.654895+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:47:02.654895+0000","last_undegraded":"2024-09-11T08:47:02.654895+0000","last_fullsized":"2024-09-11T08:47:02.654895+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:24.197149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.197149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001510267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":169,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.314555+0000","last_change":"2024-09-11T08:43:33.346941+0000","last_active":"2024-09-11T08:47:07.314555+0000","last_peered":"2024-09-11T08:47:07.314555+0000","last_clean":"2024-09-11T08:47:07.314555+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:47:07.314555+0000","last_undegraded":"2024-09-11T08:47:07.314555+0000","last_fullsized":"2024-09-11T08:47:07.314555+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:33.346856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.346856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053605370000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":220,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:09.601761+0000","last_change":"2024-09-11T08:47:09.601761+0000","last_active":"2024-09-11T08:47:09.601761+0000","last_peered":"2024-09-11T08:47:09.601761+0000","last_clean":"2024-09-11T08:47:09.601761+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:47:09.601761+0000","last_undegraded":"2024-09-11T08:47:09.601761+0000","last_fullsized":"2024-09-11T08:47:09.601761+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:44:18.678748+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:44:18.678748+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016068020999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":175,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.314442+0000","last_change":"2024-09-11T08:44:12.518229+0000","last_active":"2024-09-11T08:47:07.314442+0000","last_peered":"2024-09-11T08:47:07.314442+0000","last_clean":"2024-09-11T08:47:07.314442+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:47:07.314442+0000","last_undegraded":"2024-09-11T08:47:07.314442+0000","last_fullsized":"2024-09-11T08:47:07.314442+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:12.518151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.518151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013208541000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":191,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.314029+0000","last_change":"2024-09-11T08:46:01.998163+0000","last_active":"2024-09-11T08:47:07.314029+0000","last_peered":"2024-09-11T08:47:07.314029+0000","last_clean":"2024-09-11T08:47:07.314029+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:47:07.314029+0000","last_undegraded":"2024-09-11T08:47:07.314029+0000","last_fullsized":"2024-09-11T08:47:07.314029+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:24.223237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.223237+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019917720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":168,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655123+0000","last_change":"2024-09-11T08:45:28.194198+0000","last_active":"2024-09-11T08:47:02.655123+0000","last_peered":"2024-09-11T08:47:02.655123+0000","last_clean":"2024-09-11T08:47:02.655123+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:47:02.655123+0000","last_undegraded":"2024-09-11T08:47:02.655123+0000","last_fullsized":"2024-09-11T08:47:02.655123+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.194095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.194095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001559936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":189,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.308874+0000","last_change":"2024-09-11T08:45:16.162671+0000","last_active":"2024-09-11T08:47:07.308874+0000","last_peered":"2024-09-11T08:47:07.308874+0000","last_clean":"2024-09-11T08:47:07.308874+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:47:07.308874+0000","last_undegraded":"2024-09-11T08:47:07.308874+0000","last_fullsized":"2024-09-11T08:47:07.308874+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:16.162616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:16.162616+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001890833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":164,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654730+0000","last_change":"2024-09-11T08:45:22.244097+0000","last_active":"2024-09-11T08:47:02.654730+0000","last_peered":"2024-09-11T08:47:02.654730+0000","last_clean":"2024-09-11T08:47:02.654730+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:47:02.654730+0000","last_undegraded":"2024-09-11T08:47:02.654730+0000","last_fullsized":"2024-09-11T08:47:02.654730+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.244038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.244038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015252620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":172,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668897+0000","last_change":"2024-09-11T08:45:19.335163+0000","last_active":"2024-09-11T08:47:02.668897+0000","last_peered":"2024-09-11T08:47:02.668897+0000","last_clean":"2024-09-11T08:47:02.668897+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:47:02.668897+0000","last_undegraded":"2024-09-11T08:47:02.668897+0000","last_fullsized":"2024-09-11T08:47:02.668897+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:19.335065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:19.335065+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026052929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":179,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313596+0000","last_change":"2024-09-11T08:44:33.758098+0000","last_active":"2024-09-11T08:47:07.313596+0000","last_peered":"2024-09-11T08:47:07.313596+0000","last_clean":"2024-09-11T08:47:07.313596+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:47:07.313596+0000","last_undegraded":"2024-09-11T08:47:07.313596+0000","last_fullsized":"2024-09-11T08:47:07.313596+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:44:33.758018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025150559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":182,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313475+0000","last_change":"2024-09-11T08:44:10.704528+0000","last_active":"2024-09-11T08:47:07.313475+0000","last_peered":"2024-09-11T08:47:07.313475+0000","last_clean":"2024-09-11T08:47:07.313475+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:47:07.313475+0000","last_undegraded":"2024-09-11T08:47:07.313475+0000","last_fullsized":"2024-09-11T08:47:07.313475+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:44:10.704471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:10.704471+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020557259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":289,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:17.777599+0000","last_change":"2024-09-11T08:45:08.310590+0000","last_active":"2024-09-11T08:47:17.777599+0000","last_peered":"2024-09-11T08:47:17.777599+0000","last_clean":"2024-09-11T08:47:17.777599+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:47:17.777599+0000","last_undegraded":"2024-09-11T08:47:17.777599+0000","last_fullsized":"2024-09-11T08:47:17.777599+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:08.310527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.310527+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021434650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"186'9136","reported_seq":9181,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-11T08:47:26.412729+0000","last_change":"2024-09-11T08:45:15.261892+0000","last_active":"2024-09-11T08:47:26.412729+0000","last_peered":"2024-09-11T08:47:26.412729+0000","last_clean":"2024-09-11T08:47:26.412729+0000","last_became_active":"2024-09-11T08:45:15.261608+0000","last_became_peered":"2024-09-11T08:45:15.261608+0000","last_unstale":"2024-09-11T08:47:26.412729+0000","last_undegraded":"2024-09-11T08:47:26.412729+0000","last_fullsized":"2024-09-11T08:47:26.412729+0000","mapping_epoch":149,"log_start":"170'4200","ondisk_log_start":"170'4200","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4936,"log_dups_size":0,"ondisk_log_size":4936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":374341632,"num_objects":5712,"num_object_clones":0,"num_object_copies":17136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9136,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"124'22","reported_seq":209,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.318399+0000","last_change":"2024-09-11T08:45:18.777528+0000","last_active":"2024-09-11T08:47:07.318399+0000","last_peered":"2024-09-11T08:47:07.318399+0000","last_clean":"2024-09-11T08:47:07.318399+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:47:07.318399+0000","last_undegraded":"2024-09-11T08:47:07.318399+0000","last_fullsized":"2024-09-11T08:47:07.318399+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:45:18.777466+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.777466+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045440580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":168,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.318002+0000","last_change":"2024-09-11T08:45:13.214007+0000","last_active":"2024-09-11T08:47:07.318002+0000","last_peered":"2024-09-11T08:47:07.318002+0000","last_clean":"2024-09-11T08:47:07.318002+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:47:07.318002+0000","last_undegraded":"2024-09-11T08:47:07.318002+0000","last_fullsized":"2024-09-11T08:47:07.318002+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:13.213922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.213922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016325280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"154'9","reported_seq":278,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:17.777498+0000","last_change":"2024-09-11T08:45:26.208887+0000","last_active":"2024-09-11T08:47:17.777498+0000","last_peered":"2024-09-11T08:47:17.777498+0000","last_clean":"2024-09-11T08:47:17.777498+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:47:17.777498+0000","last_undegraded":"2024-09-11T08:47:17.777498+0000","last_fullsized":"2024-09-11T08:47:17.777498+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'9","last_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.208818+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028933230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.649765+0000","last_change":"2024-09-11T08:44:24.813906+0000","last_active":"2024-09-11T08:47:02.649765+0000","last_peered":"2024-09-11T08:47:02.649765+0000","last_clean":"2024-09-11T08:47:02.649765+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:47:02.649765+0000","last_undegraded":"2024-09-11T08:47:02.649765+0000","last_fullsized":"2024-09-11T08:47:02.649765+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:24.813857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.813857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01118625,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"186'9311","reported_seq":9356,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-11T08:47:26.732448+0000","last_change":"2024-09-11T08:45:15.297710+0000","last_active":"2024-09-11T08:47:26.732448+0000","last_peered":"2024-09-11T08:47:26.732448+0000","last_clean":"2024-09-11T08:47:26.732448+0000","last_became_active":"2024-09-11T08:45:15.297378+0000","last_became_peered":"2024-09-11T08:45:15.297378+0000","last_unstale":"2024-09-11T08:47:26.732448+0000","last_undegraded":"2024-09-11T08:47:26.732448+0000","last_fullsized":"2024-09-11T08:47:26.732448+0000","mapping_epoch":149,"log_start":"165'3600","ondisk_log_start":"165'3600","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5711,"log_dups_size":0,"ondisk_log_size":5711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":379256832,"num_objects":5787,"num_object_clones":0,"num_object_copies":17361,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9311,"num_write_kb":483136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":198,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.317810+0000","last_change":"2024-09-11T08:44:33.387066+0000","last_active":"2024-09-11T08:47:07.317810+0000","last_peered":"2024-09-11T08:47:07.317810+0000","last_clean":"2024-09-11T08:47:07.317810+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:47:07.317810+0000","last_undegraded":"2024-09-11T08:47:07.317810+0000","last_fullsized":"2024-09-11T08:47:07.317810+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-11T08:44:33.386972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:33.386972+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022646007999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":168,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313253+0000","last_change":"2024-09-11T08:44:25.327692+0000","last_active":"2024-09-11T08:47:07.313253+0000","last_peered":"2024-09-11T08:47:07.313253+0000","last_clean":"2024-09-11T08:47:07.313253+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:47:07.313253+0000","last_undegraded":"2024-09-11T08:47:07.313253+0000","last_fullsized":"2024-09-11T08:47:07.313253+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:25.327553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:25.327553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016327754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":164,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654486+0000","last_change":"2024-09-11T08:45:17.150827+0000","last_active":"2024-09-11T08:47:02.654486+0000","last_peered":"2024-09-11T08:47:02.654486+0000","last_clean":"2024-09-11T08:47:02.654486+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:47:02.654486+0000","last_undegraded":"2024-09-11T08:47:02.654486+0000","last_fullsized":"2024-09-11T08:47:02.654486+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.150767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.150767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001327707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654978+0000","last_change":"2024-09-11T08:46:03.144149+0000","last_active":"2024-09-11T08:47:02.654978+0000","last_peered":"2024-09-11T08:47:02.654978+0000","last_clean":"2024-09-11T08:47:02.654978+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:47:02.654978+0000","last_undegraded":"2024-09-11T08:47:02.654978+0000","last_fullsized":"2024-09-11T08:47:02.654978+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:46:03.144096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:46:03.144096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:47:11.742114+0000","scrub_duration":0.001497151,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"185'9195","reported_seq":9240,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:26.156897+0000","last_change":"2024-09-11T08:45:15.627562+0000","last_active":"2024-09-11T08:47:26.156897+0000","last_peered":"2024-09-11T08:47:26.156897+0000","last_clean":"2024-09-11T08:47:26.156897+0000","last_became_active":"2024-09-11T08:45:15.627210+0000","last_became_peered":"2024-09-11T08:45:15.627210+0000","last_unstale":"2024-09-11T08:47:26.156897+0000","last_undegraded":"2024-09-11T08:47:26.156897+0000","last_fullsized":"2024-09-11T08:47:26.156897+0000","mapping_epoch":149,"log_start":"166'3800","ondisk_log_start":"166'3800","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5395,"log_dups_size":0,"ondisk_log_size":5395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369754136,"num_objects":5643,"num_object_clones":0,"num_object_copies":16929,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5643,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9195,"num_write_kb":474753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"86'21","reported_seq":184,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654628+0000","last_change":"2024-09-11T08:43:30.997804+0000","last_active":"2024-09-11T08:47:02.654628+0000","last_peered":"2024-09-11T08:47:02.654628+0000","last_clean":"2024-09-11T08:47:02.654628+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:47:02.654628+0000","last_undegraded":"2024-09-11T08:47:02.654628+0000","last_fullsized":"2024-09-11T08:47:02.654628+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'21","last_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:43:30.997710+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025287299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":179,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313571+0000","last_change":"2024-09-11T08:45:30.475446+0000","last_active":"2024-09-11T08:47:07.313571+0000","last_peered":"2024-09-11T08:47:07.313571+0000","last_clean":"2024-09-11T08:47:07.313571+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:47:07.313571+0000","last_undegraded":"2024-09-11T08:47:07.313571+0000","last_fullsized":"2024-09-11T08:47:07.313571+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:30.475365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:30.475365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010690405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":270,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:22.782092+0000","last_change":"2024-09-11T08:44:11.198993+0000","last_active":"2024-09-11T08:47:22.782092+0000","last_peered":"2024-09-11T08:47:22.782092+0000","last_clean":"2024-09-11T08:47:22.782092+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:47:22.782092+0000","last_undegraded":"2024-09-11T08:47:22.782092+0000","last_fullsized":"2024-09-11T08:47:22.782092+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:44:11.198936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:11.198936+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019660649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":191,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668525+0000","last_change":"2024-09-11T08:46:01.983654+0000","last_active":"2024-09-11T08:47:02.668525+0000","last_peered":"2024-09-11T08:47:02.668525+0000","last_clean":"2024-09-11T08:47:02.668525+0000","last_became_active":"2024-09-11T08:46:01.983335+0000","last_became_peered":"2024-09-11T08:46:01.983335+0000","last_unstale":"2024-09-11T08:47:02.668525+0000","last_undegraded":"2024-09-11T08:47:02.668525+0000","last_fullsized":"2024-09-11T08:47:02.668525+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.303194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.303194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001492431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"186'9293","reported_seq":9338,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-11T08:47:26.743988+0000","last_change":"2024-09-11T08:45:15.626487+0000","last_active":"2024-09-11T08:47:26.743988+0000","last_peered":"2024-09-11T08:47:26.743988+0000","last_clean":"2024-09-11T08:47:26.743988+0000","last_became_active":"2024-09-11T08:45:15.626210+0000","last_became_peered":"2024-09-11T08:45:15.626210+0000","last_unstale":"2024-09-11T08:47:26.743988+0000","last_undegraded":"2024-09-11T08:47:26.743988+0000","last_fullsized":"2024-09-11T08:47:26.743988+0000","mapping_epoch":149,"log_start":"165'3600","ondisk_log_start":"165'3600","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5693,"log_dups_size":0,"ondisk_log_size":5693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368246784,"num_objects":5619,"num_object_clones":0,"num_object_copies":16857,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9293,"num_write_kb":477184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"100'15","reported_seq":197,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313549+0000","last_change":"2024-09-11T08:45:12.769299+0000","last_active":"2024-09-11T08:47:07.313549+0000","last_peered":"2024-09-11T08:47:07.313549+0000","last_clean":"2024-09-11T08:47:07.313549+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:47:07.313549+0000","last_undegraded":"2024-09-11T08:47:07.313549+0000","last_fullsized":"2024-09-11T08:47:07.313549+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'15","last_scrub_stamp":"2024-09-11T08:45:12.769239+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:41:56.250393+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.769239+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026298300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":171,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.650117+0000","last_change":"2024-09-11T08:46:30.615598+0000","last_active":"2024-09-11T08:47:02.650117+0000","last_peered":"2024-09-11T08:47:02.650117+0000","last_clean":"2024-09-11T08:47:02.650117+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:47:02.650117+0000","last_undegraded":"2024-09-11T08:47:02.650117+0000","last_fullsized":"2024-09-11T08:47:02.650117+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:19.905459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:19.905459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015766237999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":278,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:22.781836+0000","last_change":"2024-09-11T08:44:34.630457+0000","last_active":"2024-09-11T08:47:22.781836+0000","last_peered":"2024-09-11T08:47:22.781836+0000","last_clean":"2024-09-11T08:47:22.781836+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:47:22.781836+0000","last_undegraded":"2024-09-11T08:47:22.781836+0000","last_fullsized":"2024-09-11T08:47:22.781836+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:44:34.630191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:34.630191+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089275420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":163,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.314505+0000","last_change":"2024-09-11T08:44:16.681124+0000","last_active":"2024-09-11T08:47:07.314505+0000","last_peered":"2024-09-11T08:47:07.314505+0000","last_clean":"2024-09-11T08:47:07.314505+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:47:07.314505+0000","last_undegraded":"2024-09-11T08:47:07.314505+0000","last_fullsized":"2024-09-11T08:47:07.314505+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:16.681036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:44:16.681036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047481509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"186'9214","reported_seq":9259,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-11T08:47:26.602944+0000","last_change":"2024-09-11T08:45:15.627960+0000","last_active":"2024-09-11T08:47:26.602944+0000","last_peered":"2024-09-11T08:47:26.602944+0000","last_clean":"2024-09-11T08:47:26.602944+0000","last_became_active":"2024-09-11T08:45:15.627639+0000","last_became_peered":"2024-09-11T08:45:15.627639+0000","last_unstale":"2024-09-11T08:47:26.602944+0000","last_undegraded":"2024-09-11T08:47:26.602944+0000","last_fullsized":"2024-09-11T08:47:26.602944+0000","mapping_epoch":149,"log_start":"171'4300","ondisk_log_start":"171'4300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4914,"log_dups_size":0,"ondisk_log_size":4914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":374472704,"num_objects":5714,"num_object_clones":0,"num_object_copies":17142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9214,"num_write_kb":477696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":203,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.314328+0000","last_change":"2024-09-11T08:45:11.576184+0000","last_active":"2024-09-11T08:47:07.314328+0000","last_peered":"2024-09-11T08:47:07.314328+0000","last_clean":"2024-09-11T08:47:07.314328+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:47:07.314328+0000","last_undegraded":"2024-09-11T08:47:07.314328+0000","last_fullsized":"2024-09-11T08:47:07.314328+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:45:11.576085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.576085+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002366754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":179,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313641+0000","last_change":"2024-09-11T08:44:30.802099+0000","last_active":"2024-09-11T08:47:07.313641+0000","last_peered":"2024-09-11T08:47:07.313641+0000","last_clean":"2024-09-11T08:47:07.313641+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:47:07.313641+0000","last_undegraded":"2024-09-11T08:47:07.313641+0000","last_fullsized":"2024-09-11T08:47:07.313641+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:30.800097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:30.800097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071432789999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":173,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653048+0000","last_change":"2024-09-11T08:45:07.264895+0000","last_active":"2024-09-11T08:47:02.653048+0000","last_peered":"2024-09-11T08:47:02.653048+0000","last_clean":"2024-09-11T08:47:02.653048+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:47:02.653048+0000","last_undegraded":"2024-09-11T08:47:02.653048+0000","last_fullsized":"2024-09-11T08:47:02.653048+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.264822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.264822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014425250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":193,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313415+0000","last_change":"2024-09-11T08:46:01.985315+0000","last_active":"2024-09-11T08:47:07.313415+0000","last_peered":"2024-09-11T08:47:07.313415+0000","last_clean":"2024-09-11T08:47:07.313415+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:47:07.313415+0000","last_undegraded":"2024-09-11T08:47:07.313415+0000","last_fullsized":"2024-09-11T08:47:07.313415+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.805220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.805220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012010009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"185'9275","reported_seq":9319,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:26.150193+0000","last_change":"2024-09-11T08:45:15.298396+0000","last_active":"2024-09-11T08:47:26.150193+0000","last_peered":"2024-09-11T08:47:26.150193+0000","last_clean":"2024-09-11T08:47:26.150193+0000","last_became_active":"2024-09-11T08:45:15.298053+0000","last_became_peered":"2024-09-11T08:45:15.298053+0000","last_unstale":"2024-09-11T08:47:26.150193+0000","last_undegraded":"2024-09-11T08:47:26.150193+0000","last_fullsized":"2024-09-11T08:47:26.150193+0000","mapping_epoch":149,"log_start":"166'3900","ondisk_log_start":"166'3900","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5375,"log_dups_size":0,"ondisk_log_size":5375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377683968,"num_objects":5763,"num_object_clones":0,"num_object_copies":17289,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9275,"num_write_kb":481216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":350,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:22.779358+0000","last_change":"2024-09-11T08:45:26.666261+0000","last_active":"2024-09-11T08:47:22.779358+0000","last_peered":"2024-09-11T08:47:22.779358+0000","last_clean":"2024-09-11T08:47:22.779358+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:47:22.779358+0000","last_undegraded":"2024-09-11T08:47:22.779358+0000","last_fullsized":"2024-09-11T08:47:22.779358+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:45:26.666199+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.666199+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020004279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":168,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655579+0000","last_change":"2024-09-11T08:45:15.130637+0000","last_active":"2024-09-11T08:47:02.655579+0000","last_peered":"2024-09-11T08:47:02.655579+0000","last_clean":"2024-09-11T08:47:02.655579+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:47:02.655579+0000","last_undegraded":"2024-09-11T08:47:02.655579+0000","last_fullsized":"2024-09-11T08:47:02.655579+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.130572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.130572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019779749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":209,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.718136+0000","last_change":"2024-09-11T08:47:02.718120+0000","last_active":"2024-09-11T08:47:02.718136+0000","last_peered":"2024-09-11T08:47:02.718136+0000","last_clean":"2024-09-11T08:47:02.718136+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:47:02.718136+0000","last_undegraded":"2024-09-11T08:47:02.718136+0000","last_fullsized":"2024-09-11T08:47:02.718136+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:43:38.669837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:38.669837+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001723715,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":165,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.316715+0000","last_change":"2024-09-11T08:44:14.679790+0000","last_active":"2024-09-11T08:47:07.316715+0000","last_peered":"2024-09-11T08:47:07.316715+0000","last_clean":"2024-09-11T08:47:07.316715+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:47:07.316715+0000","last_undegraded":"2024-09-11T08:47:07.316715+0000","last_fullsized":"2024-09-11T08:47:07.316715+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:14.679598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:14.679598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011950855999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":162,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.311733+0000","last_change":"2024-09-11T08:45:22.169436+0000","last_active":"2024-09-11T08:47:07.311733+0000","last_peered":"2024-09-11T08:47:07.311733+0000","last_clean":"2024-09-11T08:47:07.311733+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:47:07.311733+0000","last_undegraded":"2024-09-11T08:47:07.311733+0000","last_fullsized":"2024-09-11T08:47:07.311733+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.169367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.169367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005349693,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"186'9242","reported_seq":9287,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-11T08:47:26.607029+0000","last_change":"2024-09-11T08:45:15.261656+0000","last_active":"2024-09-11T08:47:26.607029+0000","last_peered":"2024-09-11T08:47:26.607029+0000","last_clean":"2024-09-11T08:47:26.607029+0000","last_became_active":"2024-09-11T08:45:15.261370+0000","last_became_peered":"2024-09-11T08:45:15.261370+0000","last_unstale":"2024-09-11T08:47:26.607029+0000","last_undegraded":"2024-09-11T08:47:26.607029+0000","last_fullsized":"2024-09-11T08:47:26.607029+0000","mapping_epoch":149,"log_start":"171'4400","ondisk_log_start":"171'4400","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4842,"log_dups_size":0,"ondisk_log_size":4842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":376832000,"num_objects":5750,"num_object_clones":0,"num_object_copies":17250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9242,"num_write_kb":479744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":214,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.318470+0000","last_change":"2024-09-11T08:45:27.666590+0000","last_active":"2024-09-11T08:47:07.318470+0000","last_peered":"2024-09-11T08:47:07.318470+0000","last_clean":"2024-09-11T08:47:07.318470+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:47:07.318470+0000","last_undegraded":"2024-09-11T08:47:07.318470+0000","last_fullsized":"2024-09-11T08:47:07.318470+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:45:27.666502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.666502+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023582730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":199,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.316031+0000","last_change":"2024-09-11T08:46:01.982031+0000","last_active":"2024-09-11T08:47:07.316031+0000","last_peered":"2024-09-11T08:47:07.316031+0000","last_clean":"2024-09-11T08:47:07.316031+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:47:07.316031+0000","last_undegraded":"2024-09-11T08:47:07.316031+0000","last_fullsized":"2024-09-11T08:47:07.316031+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.775804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.775804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011899289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":194,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.658005+0000","last_change":"2024-09-11T08:45:32.632342+0000","last_active":"2024-09-11T08:47:02.658005+0000","last_peered":"2024-09-11T08:47:02.658005+0000","last_clean":"2024-09-11T08:47:02.658005+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:47:02.658005+0000","last_undegraded":"2024-09-11T08:47:02.658005+0000","last_fullsized":"2024-09-11T08:47:02.658005+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:45:32.632242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:32.632242+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049221250000000003,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":183,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:15.501290+0000","last_change":"2024-09-11T08:47:15.501290+0000","last_active":"2024-09-11T08:47:15.501290+0000","last_peered":"2024-09-11T08:47:15.501290+0000","last_clean":"2024-09-11T08:47:15.501290+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:47:15.501290+0000","last_undegraded":"2024-09-11T08:47:15.501290+0000","last_fullsized":"2024-09-11T08:47:15.501290+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:09.638694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:02.444279+0000","last_clean_scrub_stamp":"2024-09-11T08:45:09.638694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00151615,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":191,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669163+0000","last_change":"2024-09-11T08:46:01.984350+0000","last_active":"2024-09-11T08:47:02.669163+0000","last_peered":"2024-09-11T08:47:02.669163+0000","last_clean":"2024-09-11T08:47:02.669163+0000","last_became_active":"2024-09-11T08:46:01.983095+0000","last_became_peered":"2024-09-11T08:46:01.983095+0000","last_unstale":"2024-09-11T08:47:02.669163+0000","last_undegraded":"2024-09-11T08:47:02.669163+0000","last_fullsized":"2024-09-11T08:47:02.669163+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:12.278185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.278185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015184790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"186'9450","reported_seq":9495,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-11T08:47:26.420639+0000","last_change":"2024-09-11T08:45:15.627633+0000","last_active":"2024-09-11T08:47:26.420639+0000","last_peered":"2024-09-11T08:47:26.420639+0000","last_clean":"2024-09-11T08:47:26.420639+0000","last_became_active":"2024-09-11T08:45:15.627360+0000","last_became_peered":"2024-09-11T08:45:15.627360+0000","last_unstale":"2024-09-11T08:47:26.420639+0000","last_undegraded":"2024-09-11T08:47:26.420639+0000","last_fullsized":"2024-09-11T08:47:26.420639+0000","mapping_epoch":149,"log_start":"171'4500","ondisk_log_start":"171'4500","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4950,"log_dups_size":0,"ondisk_log_size":4950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385351680,"num_objects":5880,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9450,"num_write_kb":490560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":185,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313498+0000","last_change":"2024-09-11T08:44:15.699227+0000","last_active":"2024-09-11T08:47:07.313498+0000","last_peered":"2024-09-11T08:47:07.313498+0000","last_clean":"2024-09-11T08:47:07.313498+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:47:07.313498+0000","last_undegraded":"2024-09-11T08:47:07.313498+0000","last_fullsized":"2024-09-11T08:47:07.313498+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:44:15.699067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:15.699067+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014313306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":179,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313352+0000","last_change":"2024-09-11T08:45:20.455183+0000","last_active":"2024-09-11T08:47:07.313352+0000","last_peered":"2024-09-11T08:47:07.313352+0000","last_clean":"2024-09-11T08:47:07.313352+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:47:07.313352+0000","last_undegraded":"2024-09-11T08:47:07.313352+0000","last_fullsized":"2024-09-11T08:47:07.313352+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:20.455081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.455081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016372450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":715,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:24.799296+0000","last_change":"2024-09-11T08:45:21.370742+0000","last_active":"2024-09-11T08:47:24.799296+0000","last_peered":"2024-09-11T08:47:24.799296+0000","last_clean":"2024-09-11T08:47:24.799296+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:47:24.799296+0000","last_undegraded":"2024-09-11T08:47:24.799296+0000","last_fullsized":"2024-09-11T08:47:24.799296+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:45:21.370673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.370673+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017307240000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":520,"num_read_kb":515,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":170,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:20.796422+0000","last_change":"2024-09-11T08:47:20.796422+0000","last_active":"2024-09-11T08:47:20.796422+0000","last_peered":"2024-09-11T08:47:20.796422+0000","last_clean":"2024-09-11T08:47:20.796422+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:47:20.796422+0000","last_undegraded":"2024-09-11T08:47:20.796422+0000","last_fullsized":"2024-09-11T08:47:20.796422+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:16.924404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:16.924404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024072690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":173,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313334+0000","last_change":"2024-09-11T08:45:17.819757+0000","last_active":"2024-09-11T08:47:07.313334+0000","last_peered":"2024-09-11T08:47:07.313334+0000","last_clean":"2024-09-11T08:47:07.313334+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:47:07.313334+0000","last_undegraded":"2024-09-11T08:47:07.313334+0000","last_fullsized":"2024-09-11T08:47:07.313334+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.819707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.819707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012611440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"185'9381","reported_seq":9424,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:26.106222+0000","last_change":"2024-09-11T08:45:15.262738+0000","last_active":"2024-09-11T08:47:26.106222+0000","last_peered":"2024-09-11T08:47:26.106222+0000","last_clean":"2024-09-11T08:47:26.106222+0000","last_became_active":"2024-09-11T08:45:15.262440+0000","last_became_peered":"2024-09-11T08:45:15.262440+0000","last_unstale":"2024-09-11T08:47:26.106222+0000","last_undegraded":"2024-09-11T08:47:26.106222+0000","last_fullsized":"2024-09-11T08:47:26.106222+0000","mapping_epoch":149,"log_start":"163'3100","ondisk_log_start":"163'3100","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":6281,"log_dups_size":0,"ondisk_log_size":6281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383320064,"num_objects":5849,"num_object_clones":0,"num_object_copies":17547,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9381,"num_write_kb":487360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":43,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.657854+0000","last_change":"2024-09-11T08:46:01.996394+0000","last_active":"2024-09-11T08:47:02.657854+0000","last_peered":"2024-09-11T08:47:02.657854+0000","last_clean":"2024-09-11T08:47:02.657854+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:47:02.657854+0000","last_undegraded":"2024-09-11T08:47:02.657854+0000","last_fullsized":"2024-09-11T08:47:02.657854+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:43:26.127412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:26.127412+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":177,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.308661+0000","last_change":"2024-09-11T08:45:18.127946+0000","last_active":"2024-09-11T08:47:07.308661+0000","last_peered":"2024-09-11T08:47:07.308661+0000","last_clean":"2024-09-11T08:47:07.308661+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:47:07.308661+0000","last_undegraded":"2024-09-11T08:47:07.308661+0000","last_fullsized":"2024-09-11T08:47:07.308661+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:18.127857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.127857+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018334099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":221,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669998+0000","last_change":"2024-09-11T08:45:29.524681+0000","last_active":"2024-09-11T08:47:02.669998+0000","last_peered":"2024-09-11T08:47:02.669998+0000","last_clean":"2024-09-11T08:47:02.669998+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:47:02.669998+0000","last_undegraded":"2024-09-11T08:47:02.669998+0000","last_fullsized":"2024-09-11T08:47:02.669998+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'79","last_scrub_stamp":"2024-09-11T08:45:29.524608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:45:29.524608+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002308867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":173,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668726+0000","last_change":"2024-09-11T08:44:35.441051+0000","last_active":"2024-09-11T08:47:02.668726+0000","last_peered":"2024-09-11T08:47:02.668726+0000","last_clean":"2024-09-11T08:47:02.668726+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:47:02.668726+0000","last_undegraded":"2024-09-11T08:47:02.668726+0000","last_fullsized":"2024-09-11T08:47:02.668726+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:35.440995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:35.440995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0080921540000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":168,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:12.832716+0000","last_change":"2024-09-11T08:47:12.832716+0000","last_active":"2024-09-11T08:47:12.832716+0000","last_peered":"2024-09-11T08:47:12.832716+0000","last_clean":"2024-09-11T08:47:12.832716+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:47:12.832716+0000","last_undegraded":"2024-09-11T08:47:12.832716+0000","last_fullsized":"2024-09-11T08:47:12.832716+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.570246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.570246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041809610000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"185'9424","reported_seq":9468,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:26.062981+0000","last_change":"2024-09-11T08:45:15.263087+0000","last_active":"2024-09-11T08:47:26.062981+0000","last_peered":"2024-09-11T08:47:26.062981+0000","last_clean":"2024-09-11T08:47:26.062981+0000","last_became_active":"2024-09-11T08:45:15.262698+0000","last_became_peered":"2024-09-11T08:45:15.262698+0000","last_unstale":"2024-09-11T08:47:26.062981+0000","last_undegraded":"2024-09-11T08:47:26.062981+0000","last_fullsized":"2024-09-11T08:47:26.062981+0000","mapping_epoch":149,"log_start":"172'4600","ondisk_log_start":"172'4600","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4824,"log_dups_size":0,"ondisk_log_size":4824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377487360,"num_objects":5760,"num_object_clones":0,"num_object_copies":17280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9424,"num_write_kb":485888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":241,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655060+0000","last_change":"2024-09-11T08:46:01.997857+0000","last_active":"2024-09-11T08:47:02.655060+0000","last_peered":"2024-09-11T08:47:02.655060+0000","last_clean":"2024-09-11T08:47:02.655060+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:47:02.655060+0000","last_undegraded":"2024-09-11T08:47:02.655060+0000","last_fullsized":"2024-09-11T08:47:02.655060+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:21.002140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:21.002140+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025090400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":158,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654692+0000","last_change":"2024-09-11T08:44:22.010602+0000","last_active":"2024-09-11T08:47:02.654692+0000","last_peered":"2024-09-11T08:47:02.654692+0000","last_clean":"2024-09-11T08:47:02.654692+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:47:02.654692+0000","last_undegraded":"2024-09-11T08:47:02.654692+0000","last_fullsized":"2024-09-11T08:47:02.654692+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.010528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.010528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047628439999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":174,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.649688+0000","last_change":"2024-09-11T08:44:26.726624+0000","last_active":"2024-09-11T08:47:02.649688+0000","last_peered":"2024-09-11T08:47:02.649688+0000","last_clean":"2024-09-11T08:47:02.649688+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:47:02.649688+0000","last_undegraded":"2024-09-11T08:47:02.649688+0000","last_fullsized":"2024-09-11T08:47:02.649688+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.726573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00117395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":182,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.309362+0000","last_change":"2024-09-11T08:46:01.988315+0000","last_active":"2024-09-11T08:47:07.309362+0000","last_peered":"2024-09-11T08:47:07.309362+0000","last_clean":"2024-09-11T08:47:07.309362+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:47:07.309362+0000","last_undegraded":"2024-09-11T08:47:07.309362+0000","last_fullsized":"2024-09-11T08:47:07.309362+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.167071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.167071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001796483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"186'9152","reported_seq":9197,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-11T08:47:26.423416+0000","last_change":"2024-09-11T08:45:15.626015+0000","last_active":"2024-09-11T08:47:26.423416+0000","last_peered":"2024-09-11T08:47:26.423416+0000","last_clean":"2024-09-11T08:47:26.423416+0000","last_became_active":"2024-09-11T08:45:15.625726+0000","last_became_peered":"2024-09-11T08:45:15.625726+0000","last_unstale":"2024-09-11T08:47:26.423416+0000","last_undegraded":"2024-09-11T08:47:26.423416+0000","last_fullsized":"2024-09-11T08:47:26.423416+0000","mapping_epoch":149,"log_start":"170'4200","ondisk_log_start":"170'4200","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4952,"log_dups_size":0,"ondisk_log_size":4952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368574464,"num_objects":5624,"num_object_clones":0,"num_object_copies":16872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9152,"num_write_kb":472832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":194,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655631+0000","last_change":"2024-09-11T08:44:28.940078+0000","last_active":"2024-09-11T08:47:02.655631+0000","last_peered":"2024-09-11T08:47:02.655631+0000","last_clean":"2024-09-11T08:47:02.655631+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:47:02.655631+0000","last_undegraded":"2024-09-11T08:47:02.655631+0000","last_fullsized":"2024-09-11T08:47:02.655631+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:44:28.940023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.940023+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012856796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":187,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653279+0000","last_change":"2024-09-11T08:45:25.273200+0000","last_active":"2024-09-11T08:47:02.653279+0000","last_peered":"2024-09-11T08:47:02.653279+0000","last_clean":"2024-09-11T08:47:02.653279+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:47:02.653279+0000","last_undegraded":"2024-09-11T08:47:02.653279+0000","last_fullsized":"2024-09-11T08:47:02.653279+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.273134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014307370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":175,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313404+0000","last_change":"2024-09-11T08:44:24.816974+0000","last_active":"2024-09-11T08:47:07.313404+0000","last_peered":"2024-09-11T08:47:07.313404+0000","last_clean":"2024-09-11T08:47:07.313404+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:47:07.313404+0000","last_undegraded":"2024-09-11T08:47:07.313404+0000","last_fullsized":"2024-09-11T08:47:07.313404+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:24.816890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.816890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086069140000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":184,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:04.543421+0000","last_change":"2024-09-11T08:47:04.543347+0000","last_active":"2024-09-11T08:47:04.543421+0000","last_peered":"2024-09-11T08:47:04.543421+0000","last_clean":"2024-09-11T08:47:04.543421+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:47:04.543421+0000","last_undegraded":"2024-09-11T08:47:04.543421+0000","last_fullsized":"2024-09-11T08:47:04.543421+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.756166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.756166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059059109999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"186'9326","reported_seq":9370,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-11T08:47:26.611217+0000","last_change":"2024-09-11T08:45:15.260658+0000","last_active":"2024-09-11T08:47:26.611217+0000","last_peered":"2024-09-11T08:47:26.611217+0000","last_clean":"2024-09-11T08:47:26.611217+0000","last_became_active":"2024-09-11T08:45:15.260383+0000","last_became_peered":"2024-09-11T08:45:15.260383+0000","last_unstale":"2024-09-11T08:47:26.611217+0000","last_undegraded":"2024-09-11T08:47:26.611217+0000","last_fullsized":"2024-09-11T08:47:26.611217+0000","mapping_epoch":149,"log_start":"171'4400","ondisk_log_start":"171'4400","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":372506624,"num_objects":5684,"num_object_clones":0,"num_object_copies":17052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9326,"num_write_kb":480320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":215,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655091+0000","last_change":"2024-09-11T08:46:02.002847+0000","last_active":"2024-09-11T08:47:02.655091+0000","last_peered":"2024-09-11T08:47:02.655091+0000","last_clean":"2024-09-11T08:47:02.655091+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:47:02.655091+0000","last_undegraded":"2024-09-11T08:47:02.655091+0000","last_fullsized":"2024-09-11T08:47:02.655091+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:43:24.983740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:24.983740+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069750209999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":168,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.308793+0000","last_change":"2024-09-11T08:45:32.219463+0000","last_active":"2024-09-11T08:47:07.308793+0000","last_peered":"2024-09-11T08:47:07.308793+0000","last_clean":"2024-09-11T08:47:07.308793+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:47:07.308793+0000","last_undegraded":"2024-09-11T08:47:07.308793+0000","last_fullsized":"2024-09-11T08:47:07.308793+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:32.219374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:32.219374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069622119999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":296,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:21.477169+0000","last_change":"2024-09-11T08:47:21.477136+0000","last_active":"2024-09-11T08:47:21.477169+0000","last_peered":"2024-09-11T08:47:21.477169+0000","last_clean":"2024-09-11T08:47:21.477169+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:47:21.477169+0000","last_undegraded":"2024-09-11T08:47:21.477169+0000","last_fullsized":"2024-09-11T08:47:21.477169+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:10.659079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:10.659079+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017208200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":162,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654814+0000","last_change":"2024-09-11T08:45:31.196687+0000","last_active":"2024-09-11T08:47:02.654814+0000","last_peered":"2024-09-11T08:47:02.654814+0000","last_clean":"2024-09-11T08:47:02.654814+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:47:02.654814+0000","last_undegraded":"2024-09-11T08:47:02.654814+0000","last_fullsized":"2024-09-11T08:47:02.654814+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.196599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.196599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025248190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"186'9431","reported_seq":14616,"reported_epoch":186,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T08:47:26.438248+0000","last_change":"2024-09-11T08:46:03.286008+0000","last_active":"2024-09-11T08:47:26.438248+0000","last_peered":"2024-09-11T08:47:26.438248+0000","last_clean":"2024-09-11T08:46:00.945841+0000","last_became_active":"2024-09-11T08:46:03.248256+0000","last_became_peered":"2024-09-11T08:46:03.248256+0000","last_unstale":"2024-09-11T08:47:26.438248+0000","last_undegraded":"2024-09-11T08:47:26.438248+0000","last_fullsized":"2024-09-11T08:46:02.976155+0000","mapping_epoch":168,"log_start":"172'4500","ondisk_log_start":"172'4500","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4931,"log_dups_size":0,"ondisk_log_size":4931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":379125760,"num_objects":5785,"num_object_clones":0,"num_object_copies":17355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1364,"num_objects_unfound":0,"num_objects_dirty":5785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9431,"num_write_kb":486912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2565,"num_bytes_recovered":168099840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3],"avail_no_missing":["1","2","3"],"object_location_counts":[{"shards":"1,2,3","objects":1364},{"shards":"1,3","objects":4421}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":183,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.314311+0000","last_change":"2024-09-11T08:44:31.662275+0000","last_active":"2024-09-11T08:47:07.314311+0000","last_peered":"2024-09-11T08:47:07.314311+0000","last_clean":"2024-09-11T08:47:07.314311+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:47:07.314311+0000","last_undegraded":"2024-09-11T08:47:07.314311+0000","last_fullsized":"2024-09-11T08:47:07.314311+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:31.662190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.662190+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040714619999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":178,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.650496+0000","last_change":"2024-09-11T08:45:10.607400+0000","last_active":"2024-09-11T08:47:02.650496+0000","last_peered":"2024-09-11T08:47:02.650496+0000","last_clean":"2024-09-11T08:47:02.650496+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:47:02.650496+0000","last_undegraded":"2024-09-11T08:47:02.650496+0000","last_fullsized":"2024-09-11T08:47:02.650496+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:10.607288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:45:10.607288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016158909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":174,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.658667+0000","last_change":"2024-09-11T08:45:12.646009+0000","last_active":"2024-09-11T08:47:02.658667+0000","last_peered":"2024-09-11T08:47:02.658667+0000","last_clean":"2024-09-11T08:47:02.658667+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:47:02.658667+0000","last_undegraded":"2024-09-11T08:47:02.658667+0000","last_fullsized":"2024-09-11T08:47:02.658667+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:12.645956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.645956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014834379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":163,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313280+0000","last_change":"2024-09-11T08:44:17.731558+0000","last_active":"2024-09-11T08:47:07.313280+0000","last_peered":"2024-09-11T08:47:07.313280+0000","last_clean":"2024-09-11T08:47:07.313280+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:47:07.313280+0000","last_undegraded":"2024-09-11T08:47:07.313280+0000","last_fullsized":"2024-09-11T08:47:07.313280+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:17.731489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:17.731489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002184517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"186'9109","reported_seq":9154,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-11T08:47:26.712920+0000","last_change":"2024-09-11T08:45:15.297737+0000","last_active":"2024-09-11T08:47:26.712920+0000","last_peered":"2024-09-11T08:47:26.712920+0000","last_clean":"2024-09-11T08:47:26.712920+0000","last_became_active":"2024-09-11T08:45:15.297420+0000","last_became_peered":"2024-09-11T08:45:15.297420+0000","last_unstale":"2024-09-11T08:47:26.712920+0000","last_undegraded":"2024-09-11T08:47:26.712920+0000","last_fullsized":"2024-09-11T08:47:26.712920+0000","mapping_epoch":149,"log_start":"165'3400","ondisk_log_start":"165'3400","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5709,"log_dups_size":0,"ondisk_log_size":5709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363266048,"num_objects":5543,"num_object_clones":0,"num_object_copies":16629,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9109,"num_write_kb":468864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":47,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653113+0000","last_change":"2024-09-11T08:46:01.997537+0000","last_active":"2024-09-11T08:47:02.653113+0000","last_peered":"2024-09-11T08:47:02.653113+0000","last_clean":"2024-09-11T08:47:02.653113+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:47:02.653113+0000","last_undegraded":"2024-09-11T08:47:02.653113+0000","last_fullsized":"2024-09-11T08:47:02.653113+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:44:27.810879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:27.810879+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001826121,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":177,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669416+0000","last_change":"2024-09-11T08:44:31.494470+0000","last_active":"2024-09-11T08:47:02.669416+0000","last_peered":"2024-09-11T08:47:02.669416+0000","last_clean":"2024-09-11T08:47:02.669416+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:47:02.669416+0000","last_undegraded":"2024-09-11T08:47:02.669416+0000","last_fullsized":"2024-09-11T08:47:02.669416+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.494397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.494397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021588290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":183,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:14.548620+0000","last_change":"2024-09-11T08:47:14.548620+0000","last_active":"2024-09-11T08:47:14.548620+0000","last_peered":"2024-09-11T08:47:14.548620+0000","last_clean":"2024-09-11T08:47:14.548620+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:47:14.548620+0000","last_undegraded":"2024-09-11T08:47:14.548620+0000","last_fullsized":"2024-09-11T08:47:14.548620+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:30.717364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:30.717364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044727940000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":161,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669976+0000","last_change":"2024-09-11T08:44:23.367862+0000","last_active":"2024-09-11T08:47:02.669976+0000","last_peered":"2024-09-11T08:47:02.669976+0000","last_clean":"2024-09-11T08:47:02.669976+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:47:02.669976+0000","last_undegraded":"2024-09-11T08:47:02.669976+0000","last_fullsized":"2024-09-11T08:47:02.669976+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:23.367786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:23.367786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030117270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"186'9177","reported_seq":16040,"reported_epoch":186,"state":"active+recovering+remapped","last_fresh":"2024-09-11T08:47:26.747471+0000","last_change":"2024-09-11T08:46:03.325028+0000","last_active":"2024-09-11T08:47:26.747471+0000","last_peered":"2024-09-11T08:47:26.747471+0000","last_clean":"2024-09-11T08:46:00.945574+0000","last_became_active":"2024-09-11T08:46:03.262389+0000","last_became_peered":"2024-09-11T08:46:03.262389+0000","last_unstale":"2024-09-11T08:47:26.747471+0000","last_undegraded":"2024-09-11T08:47:26.747471+0000","last_fullsized":"2024-09-11T08:47:26.747471+0000","mapping_epoch":168,"log_start":"165'3500","ondisk_log_start":"165'3500","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5677,"log_dups_size":0,"ondisk_log_size":5677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":365756416,"num_objects":5581,"num_object_clones":0,"num_object_copies":16743,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":435,"num_objects_unfound":0,"num_objects_dirty":5581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9177,"num_write_kb":472256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3405,"num_bytes_recovered":223150080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[0,4,7],"avail_no_missing":["0","4","7"],"object_location_counts":[{"shards":"0,4,7","objects":5146}],"blocked_by":[],"up_primary":6,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":189,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654574+0000","last_change":"2024-09-11T08:45:08.140910+0000","last_active":"2024-09-11T08:47:02.654574+0000","last_peered":"2024-09-11T08:47:02.654574+0000","last_clean":"2024-09-11T08:47:02.654574+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:47:02.654574+0000","last_undegraded":"2024-09-11T08:47:02.654574+0000","last_fullsized":"2024-09-11T08:47:02.654574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:08.140839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.140839+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003679547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":177,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669047+0000","last_change":"2024-09-11T08:45:14.322786+0000","last_active":"2024-09-11T08:47:02.669047+0000","last_peered":"2024-09-11T08:47:02.669047+0000","last_clean":"2024-09-11T08:47:02.669047+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:47:02.669047+0000","last_undegraded":"2024-09-11T08:47:02.669047+0000","last_fullsized":"2024-09-11T08:47:02.669047+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.322217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.322217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014821319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":183,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:12.554506+0000","last_change":"2024-09-11T08:47:12.554506+0000","last_active":"2024-09-11T08:47:12.554506+0000","last_peered":"2024-09-11T08:47:12.554506+0000","last_clean":"2024-09-11T08:47:12.554506+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:47:12.554506+0000","last_undegraded":"2024-09-11T08:47:12.554506+0000","last_fullsized":"2024-09-11T08:47:12.554506+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:29.692933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:29.692933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007324077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":162,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.308552+0000","last_change":"2024-09-11T08:45:33.228265+0000","last_active":"2024-09-11T08:47:07.308552+0000","last_peered":"2024-09-11T08:47:07.308552+0000","last_clean":"2024-09-11T08:47:07.308552+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:47:07.308552+0000","last_undegraded":"2024-09-11T08:47:07.308552+0000","last_fullsized":"2024-09-11T08:47:07.308552+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:45:33.228082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064426500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"185'9283","reported_seq":9327,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:26.113173+0000","last_change":"2024-09-11T08:45:15.626107+0000","last_active":"2024-09-11T08:47:26.113173+0000","last_peered":"2024-09-11T08:47:26.113173+0000","last_clean":"2024-09-11T08:47:26.113173+0000","last_became_active":"2024-09-11T08:45:15.625778+0000","last_became_peered":"2024-09-11T08:45:15.625778+0000","last_unstale":"2024-09-11T08:47:26.113173+0000","last_undegraded":"2024-09-11T08:47:26.113173+0000","last_fullsized":"2024-09-11T08:47:26.113173+0000","mapping_epoch":149,"log_start":"166'3900","ondisk_log_start":"166'3900","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5383,"log_dups_size":0,"ondisk_log_size":5383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":381747200,"num_objects":5825,"num_object_clones":0,"num_object_copies":17475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9283,"num_write_kb":483456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"124'22","reported_seq":256,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:13.872981+0000","last_change":"2024-09-11T08:47:13.872981+0000","last_active":"2024-09-11T08:47:13.872981+0000","last_peered":"2024-09-11T08:47:13.872981+0000","last_clean":"2024-09-11T08:47:13.872981+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:47:13.872981+0000","last_undegraded":"2024-09-11T08:47:13.872981+0000","last_fullsized":"2024-09-11T08:47:13.872981+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:43:32.625376+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073394039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":194,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668965+0000","last_change":"2024-09-11T08:45:20.372571+0000","last_active":"2024-09-11T08:47:02.668965+0000","last_peered":"2024-09-11T08:47:02.668965+0000","last_clean":"2024-09-11T08:47:02.668965+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:47:02.668965+0000","last_undegraded":"2024-09-11T08:47:02.668965+0000","last_fullsized":"2024-09-11T08:47:02.668965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:45:20.372454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.372454+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019547269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"168'10","reported_seq":314,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:17.777471+0000","last_change":"2024-09-11T08:46:01.988356+0000","last_active":"2024-09-11T08:47:17.777471+0000","last_peered":"2024-09-11T08:47:17.777471+0000","last_clean":"2024-09-11T08:47:17.777471+0000","last_became_active":"2024-09-11T08:46:01.983055+0000","last_became_peered":"2024-09-11T08:46:01.983055+0000","last_unstale":"2024-09-11T08:47:17.777471+0000","last_undegraded":"2024-09-11T08:47:17.777471+0000","last_fullsized":"2024-09-11T08:47:17.777471+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"135'9","last_scrub_stamp":"2024-09-11T08:45:11.298652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.298652+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001669032,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":165,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.649746+0000","last_change":"2024-09-11T08:46:27.611148+0000","last_active":"2024-09-11T08:47:02.649746+0000","last_peered":"2024-09-11T08:47:02.649746+0000","last_clean":"2024-09-11T08:47:02.649746+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:47:02.649746+0000","last_undegraded":"2024-09-11T08:47:02.649746+0000","last_fullsized":"2024-09-11T08:47:02.649746+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:15.888394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:15.888394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003201476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":187,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.652477+0000","last_change":"2024-09-11T08:45:28.247838+0000","last_active":"2024-09-11T08:47:02.652477+0000","last_peered":"2024-09-11T08:47:02.652477+0000","last_clean":"2024-09-11T08:47:02.652477+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:47:02.652477+0000","last_undegraded":"2024-09-11T08:47:02.652477+0000","last_fullsized":"2024-09-11T08:47:02.652477+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.247771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015413499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":193,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313046+0000","last_change":"2024-09-11T08:44:35.390285+0000","last_active":"2024-09-11T08:47:07.313046+0000","last_peered":"2024-09-11T08:47:07.313046+0000","last_clean":"2024-09-11T08:47:07.313046+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:47:07.313046+0000","last_undegraded":"2024-09-11T08:47:07.313046+0000","last_fullsized":"2024-09-11T08:47:07.313046+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-11T08:44:35.390134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:35.390134+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017427033000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":174,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.649474+0000","last_change":"2024-09-11T08:45:08.675251+0000","last_active":"2024-09-11T08:47:02.649474+0000","last_peered":"2024-09-11T08:47:02.649474+0000","last_clean":"2024-09-11T08:47:02.649474+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:47:02.649474+0000","last_undegraded":"2024-09-11T08:47:02.649474+0000","last_fullsized":"2024-09-11T08:47:02.649474+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:08.675179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.675179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003515392,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":191,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669102+0000","last_change":"2024-09-11T08:46:01.983864+0000","last_active":"2024-09-11T08:47:02.669102+0000","last_peered":"2024-09-11T08:47:02.669102+0000","last_clean":"2024-09-11T08:47:02.669102+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:47:02.669102+0000","last_undegraded":"2024-09-11T08:47:02.669102+0000","last_fullsized":"2024-09-11T08:47:02.669102+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:25.344348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.344348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013849730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":183,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313555+0000","last_change":"2024-09-11T08:44:11.687456+0000","last_active":"2024-09-11T08:47:07.313555+0000","last_peered":"2024-09-11T08:47:07.313555+0000","last_clean":"2024-09-11T08:47:07.313555+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:47:07.313555+0000","last_undegraded":"2024-09-11T08:47:07.313555+0000","last_fullsized":"2024-09-11T08:47:07.313555+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:11.687402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:11.687402+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018414239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":177,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668600+0000","last_change":"2024-09-11T08:44:32.483521+0000","last_active":"2024-09-11T08:47:02.668600+0000","last_peered":"2024-09-11T08:47:02.668600+0000","last_clean":"2024-09-11T08:47:02.668600+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:47:02.668600+0000","last_undegraded":"2024-09-11T08:47:02.668600+0000","last_fullsized":"2024-09-11T08:47:02.668600+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:44:32.483439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013511770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":164,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.312647+0000","last_change":"2024-09-11T08:45:23.213642+0000","last_active":"2024-09-11T08:47:07.312647+0000","last_peered":"2024-09-11T08:47:07.312647+0000","last_clean":"2024-09-11T08:47:07.312647+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:47:07.312647+0000","last_undegraded":"2024-09-11T08:47:07.312647+0000","last_fullsized":"2024-09-11T08:47:07.312647+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.213592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.213592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001600473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":172,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.657859+0000","last_change":"2024-09-11T08:45:07.618618+0000","last_active":"2024-09-11T08:47:02.657859+0000","last_peered":"2024-09-11T08:47:02.657859+0000","last_clean":"2024-09-11T08:47:02.657859+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:47:02.657859+0000","last_undegraded":"2024-09-11T08:47:02.657859+0000","last_fullsized":"2024-09-11T08:47:02.657859+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.618564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:52.493013+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.618564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016216729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":185,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.650043+0000","last_change":"2024-09-11T08:46:15.658863+0000","last_active":"2024-09-11T08:47:02.650043+0000","last_peered":"2024-09-11T08:47:02.650043+0000","last_clean":"2024-09-11T08:47:02.650043+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:47:02.650043+0000","last_undegraded":"2024-09-11T08:47:02.650043+0000","last_fullsized":"2024-09-11T08:47:02.650043+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:18.928481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:18.928481+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047130310000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":169,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.317443+0000","last_change":"2024-09-11T08:44:09.678084+0000","last_active":"2024-09-11T08:47:07.317443+0000","last_peered":"2024-09-11T08:47:07.317443+0000","last_clean":"2024-09-11T08:47:07.317443+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:47:07.317443+0000","last_undegraded":"2024-09-11T08:47:07.317443+0000","last_fullsized":"2024-09-11T08:47:07.317443+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:09.677991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:09.677991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066552809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":277,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:22.781338+0000","last_change":"2024-09-11T08:45:20.119727+0000","last_active":"2024-09-11T08:47:22.781338+0000","last_peered":"2024-09-11T08:47:22.781338+0000","last_clean":"2024-09-11T08:47:22.781338+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:47:22.781338+0000","last_undegraded":"2024-09-11T08:47:22.781338+0000","last_fullsized":"2024-09-11T08:47:22.781338+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:20.119626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.119626+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018087660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":152,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655729+0000","last_change":"2024-09-11T08:44:20.017341+0000","last_active":"2024-09-11T08:47:02.655729+0000","last_peered":"2024-09-11T08:47:02.655729+0000","last_clean":"2024-09-11T08:47:02.655729+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:47:02.655729+0000","last_undegraded":"2024-09-11T08:47:02.655729+0000","last_fullsized":"2024-09-11T08:47:02.655729+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:20.017261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:20.017261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001558369,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"152'15","reported_seq":186,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.311720+0000","last_change":"2024-09-11T08:45:21.132138+0000","last_active":"2024-09-11T08:47:07.311720+0000","last_peered":"2024-09-11T08:47:07.311720+0000","last_clean":"2024-09-11T08:47:07.311720+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:47:07.311720+0000","last_undegraded":"2024-09-11T08:47:07.311720+0000","last_fullsized":"2024-09-11T08:47:07.311720+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'15","last_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.132083+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003984156,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":177,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668724+0000","last_change":"2024-09-11T08:45:27.436673+0000","last_active":"2024-09-11T08:47:02.668724+0000","last_peered":"2024-09-11T08:47:02.668724+0000","last_clean":"2024-09-11T08:47:02.668724+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:47:02.668724+0000","last_undegraded":"2024-09-11T08:47:02.668724+0000","last_fullsized":"2024-09-11T08:47:02.668724+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.436552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001660176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":287,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:17.777656+0000","last_change":"2024-09-11T08:45:28.692173+0000","last_active":"2024-09-11T08:47:17.777656+0000","last_peered":"2024-09-11T08:47:17.777656+0000","last_clean":"2024-09-11T08:47:17.777656+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:47:17.777656+0000","last_undegraded":"2024-09-11T08:47:17.777656+0000","last_fullsized":"2024-09-11T08:47:17.777656+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:28.692100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.692100+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019579020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":173,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313418+0000","last_change":"2024-09-11T08:45:14.581287+0000","last_active":"2024-09-11T08:47:07.313418+0000","last_peered":"2024-09-11T08:47:07.313418+0000","last_clean":"2024-09-11T08:47:07.313418+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:47:07.313418+0000","last_undegraded":"2024-09-11T08:47:07.313418+0000","last_fullsized":"2024-09-11T08:47:07.313418+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.581231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.581231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001580932,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":175,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.655317+0000","last_change":"2024-09-11T08:43:37.137260+0000","last_active":"2024-09-11T08:47:02.655317+0000","last_peered":"2024-09-11T08:47:02.655317+0000","last_clean":"2024-09-11T08:47:02.655317+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:47:02.655317+0000","last_undegraded":"2024-09-11T08:47:02.655317+0000","last_fullsized":"2024-09-11T08:47:02.655317+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:37.137173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:37.137173+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085039490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":168,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.654928+0000","last_change":"2024-09-11T08:45:27.198021+0000","last_active":"2024-09-11T08:47:02.654928+0000","last_peered":"2024-09-11T08:47:02.654928+0000","last_clean":"2024-09-11T08:47:02.654928+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:47:02.654928+0000","last_undegraded":"2024-09-11T08:47:02.654928+0000","last_fullsized":"2024-09-11T08:47:02.654928+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.197944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.197944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015230210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":175,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313416+0000","last_change":"2024-09-11T08:45:21.736656+0000","last_active":"2024-09-11T08:47:07.313416+0000","last_peered":"2024-09-11T08:47:07.313416+0000","last_clean":"2024-09-11T08:47:07.313416+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:47:07.313416+0000","last_undegraded":"2024-09-11T08:47:07.313416+0000","last_fullsized":"2024-09-11T08:47:07.313416+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:21.736573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.736573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020360389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":162,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.308073+0000","last_change":"2024-09-11T08:45:28.259156+0000","last_active":"2024-09-11T08:47:07.308073+0000","last_peered":"2024-09-11T08:47:07.308073+0000","last_clean":"2024-09-11T08:47:07.308073+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:47:07.308073+0000","last_undegraded":"2024-09-11T08:47:07.308073+0000","last_fullsized":"2024-09-11T08:47:07.308073+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.259103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.259103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015518800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":205,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.649557+0000","last_change":"2024-09-11T08:46:01.993547+0000","last_active":"2024-09-11T08:47:02.649557+0000","last_peered":"2024-09-11T08:47:02.649557+0000","last_clean":"2024-09-11T08:47:02.649557+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:47:02.649557+0000","last_undegraded":"2024-09-11T08:47:02.649557+0000","last_fullsized":"2024-09-11T08:47:02.649557+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:45:06.657912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.657912+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023744109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":177,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669312+0000","last_change":"2024-09-11T08:45:15.298564+0000","last_active":"2024-09-11T08:47:02.669312+0000","last_peered":"2024-09-11T08:47:02.669312+0000","last_clean":"2024-09-11T08:47:02.669312+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:47:02.669312+0000","last_undegraded":"2024-09-11T08:47:02.669312+0000","last_fullsized":"2024-09-11T08:47:02.669312+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.298476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.298476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033059629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":164,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.308616+0000","last_change":"2024-09-11T08:45:14.165632+0000","last_active":"2024-09-11T08:47:07.308616+0000","last_peered":"2024-09-11T08:47:07.308616+0000","last_clean":"2024-09-11T08:47:07.308616+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:47:07.308616+0000","last_undegraded":"2024-09-11T08:47:07.308616+0000","last_fullsized":"2024-09-11T08:47:07.308616+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.165576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.165576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012147639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":163,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.314243+0000","last_change":"2024-09-11T08:43:29.349431+0000","last_active":"2024-09-11T08:47:07.314243+0000","last_peered":"2024-09-11T08:47:07.314243+0000","last_clean":"2024-09-11T08:47:07.314243+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:47:07.314243+0000","last_undegraded":"2024-09-11T08:47:07.314243+0000","last_fullsized":"2024-09-11T08:47:07.314243+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.349351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.349351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006276294,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":175,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.311344+0000","last_change":"2024-09-11T08:44:12.198399+0000","last_active":"2024-09-11T08:47:07.311344+0000","last_peered":"2024-09-11T08:47:07.311344+0000","last_clean":"2024-09-11T08:47:07.311344+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:47:07.311344+0000","last_undegraded":"2024-09-11T08:47:07.311344+0000","last_fullsized":"2024-09-11T08:47:07.311344+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:44:12.198335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.198335+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020517040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":178,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.649548+0000","last_change":"2024-09-11T08:45:14.574739+0000","last_active":"2024-09-11T08:47:02.649548+0000","last_peered":"2024-09-11T08:47:02.649548+0000","last_clean":"2024-09-11T08:47:02.649548+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:47:02.649548+0000","last_undegraded":"2024-09-11T08:47:02.649548+0000","last_fullsized":"2024-09-11T08:47:02.649548+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.574678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.574678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013819469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":173,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.669807+0000","last_change":"2024-09-11T08:44:28.484120+0000","last_active":"2024-09-11T08:47:02.669807+0000","last_peered":"2024-09-11T08:47:02.669807+0000","last_clean":"2024-09-11T08:47:02.669807+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:47:02.669807+0000","last_undegraded":"2024-09-11T08:47:02.669807+0000","last_fullsized":"2024-09-11T08:47:02.669807+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:28.484047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.484047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001479172,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":172,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.308733+0000","last_change":"2024-09-11T08:46:01.984620+0000","last_active":"2024-09-11T08:47:07.308733+0000","last_peered":"2024-09-11T08:47:07.308733+0000","last_clean":"2024-09-11T08:47:07.308733+0000","last_became_active":"2024-09-11T08:46:01.984167+0000","last_became_peered":"2024-09-11T08:46:01.984167+0000","last_unstale":"2024-09-11T08:47:07.308733+0000","last_undegraded":"2024-09-11T08:47:07.308733+0000","last_fullsized":"2024-09-11T08:47:07.308733+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:32.114990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:32.114990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014089513999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"136'6","reported_seq":206,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.317732+0000","last_change":"2024-09-11T08:46:01.996138+0000","last_active":"2024-09-11T08:47:07.317732+0000","last_peered":"2024-09-11T08:47:07.317732+0000","last_clean":"2024-09-11T08:47:07.317732+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:47:07.317732+0000","last_undegraded":"2024-09-11T08:47:07.317732+0000","last_fullsized":"2024-09-11T08:47:07.317732+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'6","last_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:44:21.302870+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012512683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":186,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313944+0000","last_change":"2024-09-11T08:45:21.481725+0000","last_active":"2024-09-11T08:47:07.313944+0000","last_peered":"2024-09-11T08:47:07.313944+0000","last_clean":"2024-09-11T08:47:07.313944+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:47:07.313944+0000","last_undegraded":"2024-09-11T08:47:07.313944+0000","last_fullsized":"2024-09-11T08:47:07.313944+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:21.481634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.481634+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025664590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":27,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.652651+0000","last_change":"2024-09-11T08:46:01.979915+0000","last_active":"2024-09-11T08:47:02.652651+0000","last_peered":"2024-09-11T08:47:02.652651+0000","last_clean":"2024-09-11T08:47:02.652651+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:47:02.652651+0000","last_undegraded":"2024-09-11T08:47:02.652651+0000","last_fullsized":"2024-09-11T08:47:02.652651+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.196277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.196277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":181,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.668351+0000","last_change":"2024-09-11T08:45:23.348327+0000","last_active":"2024-09-11T08:47:02.668351+0000","last_peered":"2024-09-11T08:47:02.668351+0000","last_clean":"2024-09-11T08:47:02.668351+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:47:02.668351+0000","last_undegraded":"2024-09-11T08:47:02.668351+0000","last_fullsized":"2024-09-11T08:47:02.668351+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.348275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.348275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035843659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":183,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313325+0000","last_change":"2024-09-11T08:45:23.674163+0000","last_active":"2024-09-11T08:47:07.313325+0000","last_peered":"2024-09-11T08:47:07.313325+0000","last_clean":"2024-09-11T08:47:07.313325+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:47:07.313325+0000","last_undegraded":"2024-09-11T08:47:07.313325+0000","last_fullsized":"2024-09-11T08:47:07.313325+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.674102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.674102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001440293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":168,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.317934+0000","last_change":"2024-09-11T08:44:23.320368+0000","last_active":"2024-09-11T08:47:07.317934+0000","last_peered":"2024-09-11T08:47:07.317934+0000","last_clean":"2024-09-11T08:47:07.317934+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:47:07.317934+0000","last_undegraded":"2024-09-11T08:47:07.317934+0000","last_fullsized":"2024-09-11T08:47:07.317934+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:23.320301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:23.320301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017133368999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":39,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-11T08:47:02.653769+0000","last_change":"2024-09-11T08:46:02.002471+0000","last_active":"2024-09-11T08:47:02.653769+0000","last_peered":"2024-09-11T08:47:02.653769+0000","last_clean":"2024-09-11T08:47:02.653769+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:47:02.653769+0000","last_undegraded":"2024-09-11T08:47:02.653769+0000","last_fullsized":"2024-09-11T08:47:02.653769+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:45:25.620635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.620635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":165,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-11T08:47:07.313397+0000","last_change":"2024-09-11T08:43:30.336272+0000","last_active":"2024-09-11T08:47:07.313397+0000","last_peered":"2024-09-11T08:47:07.313397+0000","last_clean":"2024-09-11T08:47:07.313397+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:47:07.313397+0000","last_undegraded":"2024-09-11T08:47:07.313397+0000","last_fullsized":"2024-09-11T08:47:07.313397+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:30.336188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:30.336188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044263979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":5997723672,"num_objects":91519,"num_object_clones":0,"num_object_copies":274557,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1799,"num_objects_unfound":0,"num_objects_dirty":91519,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":148399,"num_write_kb":7677313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5970,"num_bytes_recovered":391249920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18521075712,"data_stored":18521063496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84499,"ondisk_log_size":84499,"up":48,"acting":47,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":1584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":456,"ondisk_log_size":456,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":520,"num_read_kb":515,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299503,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2145028,"kb_used_data":1835196,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91600124,"statfs":{"total":95995035648,"available":93798526976,"internally_reserved":0,"allocated":1879240704,"data_stored":1878927985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":2.726,"5min":1.3240000000000001,"15min":0.78200000000000003},"min":{"1min":0.315,"5min":0.22,"15min":0.22},"max":{"1min":32.171999999999997,"5min":32.171999999999997,"15min":32.171999999999997},"last":0.997},{"interface":"front","average":{"1min":2.734,"5min":1.427,"15min":0.82799999999999996},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":32.149999999999999,"5min":32.149999999999999,"15min":32.149999999999999},"last":3.665}]},{"osd":1,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":2.891,"5min":1.5149999999999999,"15min":0.84899999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":33.692999999999998,"5min":33.692999999999998,"15min":33.692999999999998},"last":1.5509999999999999},{"interface":"front","average":{"1min":2.851,"5min":1.343,"15min":0.79800000000000004},"min":{"1min":0.32100000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":35.591000000000001,"5min":35.591000000000001,"15min":35.591000000000001},"last":3.6259999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.145,"15min":0.72299999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":4.7649999999999997,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.7609999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.0660000000000001,"15min":0.69499999999999995},"min":{"1min":0.33900000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.609,"5min":5.7990000000000004,"15min":5.7990000000000004},"last":0.36699999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.179,"15min":0.74299999999999999},"min":{"1min":0.33000000000000002,"5min":0.25,"15min":0.25},"max":{"1min":2.4849999999999999,"5min":7.8609999999999998,"15min":7.8609999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.2789999999999999,"15min":0.79400000000000004},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.8370000000000002,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":5.625}]},{"osd":4,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.499,"15min":0.44900000000000001},"min":{"1min":0.24099999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.6020000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48699999999999999,"15min":0.44400000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.804,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.34999999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.51600000000000001,"15min":0.46800000000000003},"min":{"1min":0.32000000000000001,"5min":0.255,"15min":0.23300000000000001},"max":{"1min":1.6399999999999999,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.625},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.622,"15min":0.51000000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.496,"5min":2.806,"15min":2.806},"last":0.32900000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64400000000000002,"15min":0.53300000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.315},"max":{"1min":2.0259999999999998,"5min":3.5979999999999999,"15min":3.5979999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.52300000000000002},"min":{"1min":0.30499999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.0990000000000002,"5min":2.633,"15min":2.633},"last":0.38}]}]},{"osd":6,"up_from":45,"seq":193273528436,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2024308,"kb_used_data":1722796,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91720844,"statfs":{"total":95995035648,"available":93922144256,"internally_reserved":0,"allocated":1764143104,"data_stored":1763846581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.3400000000000001,"15min":0.89500000000000002},"min":{"1min":0.23300000000000001,"5min":0.217,"15min":0.217},"max":{"1min":3.5619999999999998,"5min":28.302,"15min":28.302},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.4450000000000001,"15min":0.875},"min":{"1min":0.33700000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.9630000000000001,"5min":28.314,"15min":28.314},"last":0.374}]},{"osd":1,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":2.173,"5min":1.4370000000000001,"15min":0.85599999999999998},"min":{"1min":0.32300000000000001,"5min":0.29399999999999998,"15min":0.28299999999999997},"max":{"1min":7.8810000000000002,"5min":12.586,"15min":12.586},"last":1.5},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.1850000000000001,"15min":0.77300000000000002},"min":{"1min":0.30099999999999999,"5min":0.254,"15min":0.253},"max":{"1min":9.5370000000000008,"5min":10.834,"15min":10.834},"last":0.622}]},{"osd":2,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.97999999999999998,"15min":0.70199999999999996},"min":{"1min":0.29499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.8849999999999998,"5min":14.313000000000001,"15min":14.313000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.365,"5min":1.0569999999999999,"15min":0.71599999999999997},"min":{"1min":0.40500000000000003,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":3.593,"5min":6.5979999999999999,"15min":6.5979999999999999},"last":1.173}]},{"osd":3,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":1.889,"5min":1.401,"15min":0.85399999999999998},"min":{"1min":0.35799999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":4.2270000000000003,"5min":11.433,"15min":11.433},"last":4.2270000000000003},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.042,"15min":0.76200000000000001},"min":{"1min":0.33100000000000002,"5min":0.224,"15min":0.224},"max":{"1min":2.738,"5min":9.1069999999999993,"15min":9.1069999999999993},"last":0.50700000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.50900000000000001,"15min":0.44600000000000001},"min":{"1min":0.27300000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.502,"15min":0.443},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":0.29999999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.59399999999999997,"15min":0.49299999999999999},"min":{"1min":0.28699999999999998,"5min":0.245,"15min":0.245},"max":{"1min":3.097,"5min":3.097,"15min":3.097},"last":0.32300000000000001},{"interface":"front","average":{"1min":1.042,"5min":0.61599999999999999,"15min":0.501},"min":{"1min":0.312,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":5.0819999999999999,"5min":5.0819999999999999,"15min":5.0819999999999999},"last":0.34000000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:46:34 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.61499999999999999,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.25900000000000001,"15min":0.247},"max":{"1min":7.327,"5min":7.327,"15min":7.327},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53100000000000003,"15min":0.505},"min":{"1min":0.373,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.95899999999999996,"5min":1.3,"15min":1.3},"last":0.40799999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757368,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3274112,"kb_used_data":2972600,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":90471040,"statfs":{"total":95995035648,"available":92642344960,"internally_reserved":0,"allocated":3043942400,"data_stored":3043633866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.0509999999999999,"15min":0.72799999999999998},"min":{"1min":0.23100000000000001,"5min":0.20899999999999999,"15min":0.20599999999999999},"max":{"1min":3.1850000000000001,"5min":7.3899999999999997,"15min":7.3899999999999997},"last":0.23200000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.226,"15min":0.77300000000000002},"min":{"1min":0.245,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":3.3799999999999999,"5min":6.5789999999999997,"15min":6.5789999999999997},"last":0.59999999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.0549999999999999,"15min":0.755},"min":{"1min":0.23799999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":3.8029999999999999,"5min":3.8029999999999999,"15min":4.8419999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.0840000000000001,"15min":0.71599999999999997},"min":{"1min":0.35599999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.2120000000000002,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.76000000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.2010000000000001,"15min":0.76400000000000001},"min":{"1min":0.29499999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.0919999999999996,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.089,"5min":0.98399999999999999,"15min":0.70499999999999996},"min":{"1min":0.38500000000000001,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":3.2509999999999999,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.33400000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.0429999999999999,"15min":0.73499999999999999},"min":{"1min":0.35399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.6429999999999998,"5min":6.0460000000000003,"15min":6.0460000000000003},"last":0.27000000000000002},{"interface":"front","average":{"1min":1.429,"5min":1.2889999999999999,"15min":0.82799999999999996},"min":{"1min":0.49199999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.415,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":0.40000000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.54800000000000004,"15min":0.44},"min":{"1min":0.27900000000000003,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":3.645,"5min":3.645,"15min":3.645},"last":0.247},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.52200000000000002,"15min":0.44700000000000001},"min":{"1min":0.25800000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.3079999999999998,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.311}]},{"osd":6,"last update":"Wed Sep 11 08:47:24 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.58399999999999996,"15min":0.48499999999999999},"min":{"1min":0.29399999999999998,"5min":0.27700000000000002,"15min":0.23599999999999999},"max":{"1min":3.2509999999999999,"5min":3.4329999999999998,"15min":3.4329999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.58899999999999997,"15min":0.48299999999999998},"min":{"1min":0.29099999999999998,"5min":0.26100000000000001,"15min":0.251},"max":{"1min":3,"5min":3,"15min":3},"last":0.36799999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:46:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.51700000000000002,"15min":0.496},"min":{"1min":0.36299999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.307,"15min":0.307},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.41099999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986300,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2139068,"kb_used_data":1837556,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91606084,"statfs":{"total":95995035648,"available":93804630016,"internally_reserved":0,"allocated":1881657344,"data_stored":1881351711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":2.6720000000000002,"15min":1.2509999999999999},"min":{"1min":0.23999999999999999,"5min":0.218,"15min":0.20399999999999999},"max":{"1min":1.7090000000000001,"5min":89.510000000000005,"15min":89.510000000000005},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":2.7280000000000002,"15min":1.2889999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.9060000000000001,"5min":89.765000000000001,"15min":89.765000000000001},"last":0.54500000000000004}]},{"osd":1,"last update":"Wed Sep 11 08:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.228,"15min":0.78100000000000003},"min":{"1min":0.28799999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.0960000000000001,"5min":18.596,"15min":18.596},"last":6.1020000000000003},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.2070000000000001,"15min":0.77300000000000002},"min":{"1min":0.26100000000000001,"5min":0.24399999999999999,"15min":0.23599999999999999},"max":{"1min":6.3300000000000001,"5min":18.754000000000001,"15min":18.754000000000001},"last":0.26400000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.92700000000000005,"15min":0.68500000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":5.6660000000000004,"5min":5.6660000000000004,"15min":5.6660000000000004},"last":3.2999999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.91800000000000004,"15min":0.68200000000000005},"min":{"1min":0.27500000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":3.0030000000000001,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.28599999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":3.0590000000000002,"15min":1.4039999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":2.9820000000000002,"5min":89.881,"15min":89.881},"last":0.316},{"interface":"front","average":{"1min":1.3480000000000001,"5min":2.9049999999999998,"15min":1.343},"min":{"1min":0.32300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":7.8330000000000002,"5min":89.715000000000003,"15min":89.715000000000003},"last":6.0650000000000004}]},{"osd":5,"last update":"Wed Sep 11 08:47:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.53300000000000003,"15min":0.45000000000000001},"min":{"1min":0.29799999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":2.2389999999999999,"5min":2.516,"15min":2.516},"last":3.4020000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.54800000000000004,"15min":0.46100000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.23699999999999999},"max":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.41999999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:46:27 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.48899999999999999,"15min":0.45300000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":2.9390000000000001,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.57199999999999995,"15min":0.503},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":2.371,"5min":2.371,"15min":2.371},"last":3.4590000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:46:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.505,"15min":0.495},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":3.339},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.54400000000000004,"15min":0.502},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.28799999999999998},"max":{"1min":2.9609999999999999,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":5.1970000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149826,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2516004,"kb_used_data":2206168,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91229148,"statfs":{"total":95995035648,"available":93418647552,"internally_reserved":0,"allocated":2259116032,"data_stored":2258783321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,5,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:21 2024","interfaces":[{"interface":"back","average":{"1min":1.776,"5min":1.7529999999999999,"15min":0.89500000000000002},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":7.1399999999999997,"5min":10.901999999999999,"15min":10.901999999999999},"last":3.3130000000000002},{"interface":"front","average":{"1min":2.3639999999999999,"5min":2.1850000000000001,"15min":1.0820000000000001},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":5.4249999999999998,"5min":9.6820000000000004,"15min":9.6820000000000004},"last":1.343}]},{"osd":1,"last update":"Wed Sep 11 08:47:21 2024","interfaces":[{"interface":"back","average":{"1min":2.7909999999999999,"5min":3.278,"15min":1.446},"min":{"1min":0.26700000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":8.2439999999999998,"5min":115.601,"15min":115.601},"last":0.60499999999999998},{"interface":"front","average":{"1min":2.089,"5min":3.1259999999999999,"15min":1.373},"min":{"1min":0.29499999999999998,"5min":0.20300000000000001,"15min":0.17999999999999999},"max":{"1min":8.4420000000000002,"5min":115.474,"15min":115.474},"last":3.1709999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:47:21 2024","interfaces":[{"interface":"back","average":{"1min":1.6879999999999999,"5min":1.9199999999999999,"15min":0.99199999999999999},"min":{"1min":0.28599999999999998,"5min":0.251,"15min":0.23200000000000001},"max":{"1min":8.2699999999999996,"5min":8.9510000000000005,"15min":8.9510000000000005},"last":3.464},{"interface":"front","average":{"1min":1.48,"5min":1.746,"15min":0.94499999999999995},"min":{"1min":0.30299999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":5.5999999999999996,"5min":10.695,"15min":10.695},"last":3.339}]},{"osd":4,"last update":"Wed Sep 11 08:46:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.4279999999999999,"15min":0.85599999999999998},"min":{"1min":0.28599999999999998,"5min":0.27000000000000002,"15min":0.26700000000000002},"max":{"1min":5.5460000000000003,"5min":8.9969999999999999,"15min":8.9969999999999999},"last":3.4420000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.2450000000000001,"15min":0.78200000000000003},"min":{"1min":0.34599999999999997,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.7599999999999998,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":3.2450000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:46:49 2024","interfaces":[{"interface":"back","average":{"1min":1.843,"5min":1.6020000000000001,"15min":0.92300000000000004},"min":{"1min":0.29799999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":8.3599999999999994,"5min":11.526999999999999,"15min":11.526999999999999},"last":3.391},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.296,"15min":0.81999999999999995},"min":{"1min":0.40300000000000002,"5min":0.30599999999999999,"15min":0.30499999999999999},"max":{"1min":3.7949999999999999,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":3.266}]},{"osd":6,"last update":"Wed Sep 11 08:47:21 2024","interfaces":[{"interface":"back","average":{"1min":1.9359999999999999,"5min":1.7849999999999999,"15min":1.0109999999999999},"min":{"1min":0.34899999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.4160000000000004,"5min":11.494999999999999,"15min":11.494999999999999},"last":3.484},{"interface":"front","average":{"1min":1.6659999999999999,"5min":1.6559999999999999,"15min":0.98299999999999998},"min":{"1min":0.41399999999999998,"5min":0.311,"15min":0.311},"max":{"1min":3.5630000000000002,"5min":7.7249999999999996,"15min":7.7249999999999996},"last":0.64800000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:46:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.597,"15min":0.94899999999999995},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":8.3879999999999999,"5min":11.561,"15min":11.561},"last":3.4209999999999998},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.343,"15min":0.85299999999999998},"min":{"1min":0.40000000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":5.6280000000000001,"5min":6.3170000000000002,"15min":6.3170000000000002},"last":3.21}]}]},{"osd":2,"up_from":18,"seq":77309411462,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2774188,"kb_used_data":2472420,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":90970964,"statfs":{"total":95995035648,"available":93154267136,"internally_reserved":0,"allocated":2531758080,"data_stored":2531468566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:54 2024","interfaces":[{"interface":"back","average":{"1min":4.9740000000000002,"5min":2.7919999999999998,"15min":1.1970000000000001},"min":{"1min":0.16500000000000001,"5min":0.159,"15min":0.159},"max":{"1min":34.713999999999999,"5min":34.713999999999999,"15min":34.713999999999999},"last":2.3090000000000002},{"interface":"front","average":{"1min":4.2430000000000003,"5min":2.9350000000000001,"15min":1.282},"min":{"1min":0.26200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":34.399000000000001,"5min":34.399000000000001,"15min":34.399000000000001},"last":3.5739999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:46:54 2024","interfaces":[{"interface":"back","average":{"1min":4.6509999999999998,"5min":2.1560000000000001,"15min":1.0009999999999999},"min":{"1min":0.28299999999999997,"5min":0.215,"15min":0.19},"max":{"1min":34.771000000000001,"5min":34.771000000000001,"15min":34.771000000000001},"last":3.4420000000000002},{"interface":"front","average":{"1min":4.0880000000000001,"5min":2.2639999999999998,"15min":1.044},"min":{"1min":0.252,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":35.209000000000003,"5min":35.209000000000003,"15min":35.209000000000003},"last":3.371}]},{"osd":3,"last update":"Wed Sep 11 08:47:10 2024","interfaces":[{"interface":"back","average":{"1min":4.2190000000000003,"5min":2.3809999999999998,"15min":1.167},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":34.75,"5min":34.75,"15min":34.75},"last":2.8079999999999998},{"interface":"front","average":{"1min":3.948,"5min":2.3239999999999998,"15min":1.1479999999999999},"min":{"1min":0.40999999999999998,"5min":0.27200000000000002,"15min":0.26800000000000002},"max":{"1min":32.802,"5min":32.802,"15min":32.802},"last":3.6000000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:46:26 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.381,"15min":0.82699999999999996},"min":{"1min":0.29799999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":8.0820000000000007,"5min":8.0820000000000007,"15min":8.0820000000000007},"last":2.3849999999999998},{"interface":"front","average":{"1min":1.367,"5min":1.304,"15min":0.80200000000000005},"min":{"1min":0.35499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":4.851,"5min":6.9320000000000004,"15min":6.9320000000000004},"last":4.4450000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:46:54 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.296,"15min":0.81899999999999995},"min":{"1min":0.439,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.8039999999999998,"5min":6.9550000000000001,"15min":6.9550000000000001},"last":4.4809999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0900000000000001,"15min":0.74199999999999999},"min":{"1min":0.29199999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.9940000000000002,"5min":5.9530000000000003,"15min":5.9530000000000003},"last":0.28999999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:47:19 2024","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.4730000000000001,"15min":0.93400000000000005},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":5.1980000000000004,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":2.3650000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.2210000000000001,"15min":0.82699999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.931,"5min":6.9779999999999998,"15min":6.9779999999999998},"last":3.4060000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:46:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.137,"15min":0.76400000000000001},"min":{"1min":0.32500000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.9489999999999998,"5min":5.9909999999999997,"15min":5.9909999999999997},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.607,"5min":1.444,"15min":0.878},"min":{"1min":0.42099999999999999,"5min":0.371,"15min":0.36099999999999999},"max":{"1min":8.1400000000000006,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":2.3380000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2896600,"kb_used_data":2595088,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":90848552,"statfs":{"total":95995035648,"available":93028917248,"internally_reserved":0,"allocated":2657370112,"data_stored":2657036767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:31 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":2.8929999999999998,"15min":1.1879999999999999},"min":{"1min":0.20499999999999999,"5min":0.183,"15min":0.183},"max":{"1min":4.8319999999999999,"5min":117.01600000000001,"15min":117.01600000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":1.915,"5min":3.6200000000000001,"15min":1.4490000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":6.7370000000000001,"5min":117.62,"15min":117.62},"last":5.9409999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:46:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.694,"15min":0.90200000000000002},"min":{"1min":0.22800000000000001,"5min":0.191,"15min":0.191},"max":{"1min":4.6379999999999999,"5min":9.5719999999999992,"15min":20.628},"last":0.25700000000000001},{"interface":"front","average":{"1min":1.256,"5min":1.647,"15min":0.86899999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":7.4080000000000004,"5min":11.461,"15min":20.148},"last":1.1799999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:47:11 2024","interfaces":[{"interface":"back","average":{"1min":2.3799999999999999,"5min":1.752,"15min":1.248},"min":{"1min":0.39700000000000002,"5min":0.191,"15min":0.191},"max":{"1min":5.8490000000000002,"5min":8.3550000000000004,"15min":116.98},"last":0.26800000000000002},{"interface":"front","average":{"1min":1.242,"5min":1.7190000000000001,"15min":1.304},"min":{"1min":0.27100000000000002,"5min":0.22,"15min":0.22},"max":{"1min":4.5140000000000002,"5min":10.379,"15min":117.56699999999999},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:46:31 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.1899999999999999,"15min":0.75800000000000001},"min":{"1min":0.38400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.4780000000000002,"5min":11.19,"15min":11.19},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.262,"5min":1.0880000000000001,"15min":0.73699999999999999},"min":{"1min":0.32600000000000001,"5min":0.251,"15min":0.251},"max":{"1min":5.056,"5min":7.7169999999999996,"15min":7.7169999999999996},"last":0.30199999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":2.6240000000000001,"15min":1.2769999999999999},"min":{"1min":0.34200000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.6680000000000001,"5min":103.337,"15min":103.337},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.05,"5min":2.4980000000000002,"15min":1.2170000000000001},"min":{"1min":0.41599999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.6269999999999998,"5min":103.163,"15min":103.163},"last":1.546}]},{"osd":6,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.1819999999999999,"15min":0.78900000000000003},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.29399999999999998},"max":{"1min":3.907,"5min":8.9149999999999991,"15min":8.9149999999999991},"last":0.41499999999999998},{"interface":"front","average":{"1min":1.53,"5min":1.488,"15min":0.92300000000000004},"min":{"1min":0.45000000000000001,"5min":0.312,"15min":0.31},"max":{"1min":4.4480000000000004,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":0.51200000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:46:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.2450000000000001,"15min":0.81100000000000005},"min":{"1min":0.38400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":3.7519999999999998,"5min":11.315,"15min":11.315},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.1000000000000001,"15min":0.77000000000000002},"min":{"1min":0.376,"5min":0.315,"15min":0.315},"max":{"1min":4.8739999999999997,"5min":8.8800000000000008,"15min":8.8800000000000008},"last":0.81299999999999994}]}]},{"osd":0,"up_from":8,"seq":34359738511,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2843540,"kb_used_data":2541960,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":90901612,"statfs":{"total":95995035648,"available":93083250688,"internally_reserved":0,"allocated":2602967040,"data_stored":2602650273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:46:28 2024","interfaces":[{"interface":"back","average":{"1min":2,"5min":1.9610000000000001,"15min":0.86599999999999999},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":8.3900000000000006,"5min":8.3900000000000006,"15min":8.3900000000000006},"last":0.182},{"interface":"front","average":{"1min":2.431,"5min":1.712,"15min":0.79700000000000004},"min":{"1min":0.30599999999999999,"5min":0.186,"15min":0.186},"max":{"1min":12.532,"5min":12.532,"15min":12.532},"last":0.98799999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:46:53 2024","interfaces":[{"interface":"back","average":{"1min":2.4830000000000001,"5min":1.843,"15min":0.96299999999999997},"min":{"1min":0.32800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":12.481,"5min":12.481,"15min":19.940000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":2.1240000000000001,"5min":1.9410000000000001,"15min":1.0069999999999999},"min":{"1min":0.39800000000000002,"5min":0.23599999999999999,"15min":0.22700000000000001},"max":{"1min":6.6479999999999997,"5min":8.3049999999999997,"15min":20.757000000000001},"last":1.0049999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:47:09 2024","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":2.1749999999999998,"15min":1.0960000000000001},"min":{"1min":0.312,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.5709999999999997,"5min":11.951000000000001,"15min":11.951000000000001},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.919,"15min":1.0109999999999999},"min":{"1min":0.28100000000000003,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":7.6399999999999997,"5min":7.6399999999999997,"15min":7.6399999999999997},"last":1.6819999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:46:32 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.47,"15min":0.85199999999999998},"min":{"1min":0.30599999999999999,"5min":0.25,"15min":0.25},"max":{"1min":4.8490000000000002,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.3759999999999999,"15min":0.82299999999999995},"min":{"1min":0.27600000000000002,"5min":0.255,"15min":0.255},"max":{"1min":3.9860000000000002,"5min":6.9539999999999997,"15min":6.9539999999999997},"last":0.317}]},{"osd":5,"last update":"Wed Sep 11 08:46:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.5369999999999999,"15min":0.91800000000000004},"min":{"1min":0.374,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.887,"5min":7.4009999999999998,"15min":7.4009999999999998},"last":1.036},{"interface":"front","average":{"1min":1.754,"5min":1.577,"15min":0.91600000000000004},"min":{"1min":0.32700000000000001,"5min":0.30499999999999999,"15min":0.28299999999999997},"max":{"1min":4.5970000000000004,"5min":7.423,"15min":7.423},"last":0.48299999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:47:13 2024","interfaces":[{"interface":"back","average":{"1min":1.9139999999999999,"5min":1.6890000000000001,"15min":0.99199999999999999},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.1310000000000002,"5min":7.4240000000000004,"15min":7.4240000000000004},"last":1.0600000000000001},{"interface":"front","average":{"1min":2.149,"5min":1.738,"15min":1.028},"min":{"1min":0.434,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":6.1740000000000004,"5min":7.4489999999999998,"15min":7.4489999999999998},"last":0.53100000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:46:41 2024","interfaces":[{"interface":"back","average":{"1min":1.716,"5min":1.6000000000000001,"15min":0.93500000000000005},"min":{"1min":0.32700000000000001,"5min":0.313,"15min":0.313},"max":{"1min":7.4900000000000002,"5min":7.4900000000000002,"15min":7.4900000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.6180000000000001,"15min":0.92400000000000004},"min":{"1min":0.41499999999999998,"5min":0.307,"15min":0.307},"max":{"1min":3.8279999999999998,"5min":7.4450000000000003,"15min":7.4450000000000003},"last":1.0800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2597388288,"data_stored":2597388288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2633957376,"data_stored":2633957376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2509574144,"data_stored":2509570072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2256732160,"data_stored":2256732160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1878982656,"data_stored":1878982656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3026653184,"data_stored":3026649112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1753219072,"data_stored":1753219072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1864568832,"data_stored":1864564760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:47:29.044 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph status --format=json 2024-09-11T08:47:29.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:29.159 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:29.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:29.160 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:29.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:29.301 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:29.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:29.302 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:29.394 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:29 smithi145 ceph-mon[33384]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:47:29.402 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:29.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:29.445 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:29.445 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:29.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:29.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:29 smithi092 ceph-mon[29647]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:47:29.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:29 smithi092 ceph-mon[41114]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:47:29.608 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:29.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:29.609 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:29.710 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:29.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:29.748 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:29.748 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:29.852 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:29.892 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:29.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:29.893 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:29.994 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:30.039 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:30.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:30.041 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:30.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:30.176 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:30.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:30.177 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:30.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:30.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:30.316 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:30.317 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:30.417 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:30.457 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:30.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:30.458 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:30.559 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:30.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:30 smithi092 ceph-mon[41114]: pgmap v367: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.6 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.11k op/s; 1799/275145 objects misplaced (0.654%); 7.0 MiB/s, 112 objects/s recovering 2024-09-11T08:47:30.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:30 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:30.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:30.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:30 smithi092 ceph-mon[29647]: pgmap v367: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.6 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.11k op/s; 1799/275145 objects misplaced (0.654%); 7.0 MiB/s, 112 objects/s recovering 2024-09-11T08:47:30.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:30 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:30.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:30.590 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:30.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:30.592 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:30.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:30 smithi145 ceph-mon[33384]: pgmap v367: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.6 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.11k op/s; 1799/275145 objects misplaced (0.654%); 7.0 MiB/s, 112 objects/s recovering 2024-09-11T08:47:30.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:30 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:47:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:30.693 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:30.730 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:30.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:30.731 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:30.832 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:30.862 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:30.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:30.864 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:30.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:30.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:30.998 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:31.099 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:31.129 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:31.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:31.130 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:31.192 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:31.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:31.287 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:31.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:31.289 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:31.389 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:31.429 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:31.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:31.430 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:31.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:31 smithi092 ceph-mon[41114]: 4.15 deep-scrub starts 2024-09-11T08:47:31.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:31 smithi092 ceph-mon[41114]: 5.a scrub starts 2024-09-11T08:47:31.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:31 smithi092 ceph-mon[29647]: 4.15 deep-scrub starts 2024-09-11T08:47:31.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:31 smithi092 ceph-mon[29647]: 5.a scrub starts 2024-09-11T08:47:31.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:31.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:31.578 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:31.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:31.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:31 smithi145 ceph-mon[33384]: 4.15 deep-scrub starts 2024-09-11T08:47:31.918 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:31 smithi145 ceph-mon[33384]: 5.a scrub starts 2024-09-11T08:47:31.919 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:31.920 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:47:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:47:31] "GET /metrics HTTP/1.1" 200 224329 "" "Prometheus/2.43.0" 2024-09-11T08:47:31.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:31.956 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:31.956 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:32.057 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:32.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:32.102 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:32.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:32.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:32.242 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:32.243 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:32.343 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:32.374 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:32.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:32.376 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:32.476 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:32.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:32.507 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:32.507 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:32 smithi092 ceph-mon[29647]: pgmap v368: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s; 1799/269529 objects misplaced (0.667%); 7.0 MiB/s, 112 objects/s recovering 2024-09-11T08:47:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:32 smithi092 ceph-mon[29647]: 4.e scrub starts 2024-09-11T08:47:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:32 smithi092 ceph-mon[41114]: pgmap v368: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s; 1799/269529 objects misplaced (0.667%); 7.0 MiB/s, 112 objects/s recovering 2024-09-11T08:47:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:32 smithi092 ceph-mon[41114]: 4.e scrub starts 2024-09-11T08:47:32.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:32.634 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:32.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:32.635 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:32.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:32 smithi145 ceph-mon[33384]: pgmap v368: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s; 1799/269529 objects misplaced (0.667%); 7.0 MiB/s, 112 objects/s recovering 2024-09-11T08:47:32.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:32 smithi145 ceph-mon[33384]: 4.e scrub starts 2024-09-11T08:47:32.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:32.759 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:32.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:32.760 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:32.861 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:32.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:32.884 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:32.884 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:32.985 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:33.009 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:33.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:33.010 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:33.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:33.133 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:33.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:33.134 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:33.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:47:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:47:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:47:33.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:47:33.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:33.280 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:47:33.280 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:47:33.280 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:47:33.280 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:47:33.280 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:47:33.295 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:33.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:33.296 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:33.397 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:33.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:33.447 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:33.447 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:33.548 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:33.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:33 smithi092 ceph-mon[29647]: osdmap e187: 8 total, 8 up, 8 in 2024-09-11T08:47:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:33 smithi092 ceph-mon[29647]: 5.10 scrub starts 2024-09-11T08:47:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:33 smithi092 ceph-mon[29647]: 5.1e scrub starts 2024-09-11T08:47:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:33 smithi092 ceph-mon[29647]: 5.1e scrub ok 2024-09-11T08:47:33.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:33 smithi092 ceph-mon[41114]: osdmap e187: 8 total, 8 up, 8 in 2024-09-11T08:47:33.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:33 smithi092 ceph-mon[41114]: 5.10 scrub starts 2024-09-11T08:47:33.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:33 smithi092 ceph-mon[41114]: 5.1e scrub starts 2024-09-11T08:47:33.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:33 smithi092 ceph-mon[41114]: 5.1e scrub ok 2024-09-11T08:47:33.587 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:33.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:33.588 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:33.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:33 smithi145 ceph-mon[33384]: osdmap e187: 8 total, 8 up, 8 in 2024-09-11T08:47:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:33 smithi145 ceph-mon[33384]: 5.10 scrub starts 2024-09-11T08:47:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:33 smithi145 ceph-mon[33384]: 5.1e scrub starts 2024-09-11T08:47:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:33 smithi145 ceph-mon[33384]: 5.1e scrub ok 2024-09-11T08:47:33.688 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:33.719 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:33.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:33.720 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:33.811 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:47:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:47:33.821 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:33.848 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:47:33.848 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:47:33.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:33.870 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:33.870 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:33.883 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:47:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:47:33.915 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:47:33.915 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:47:33.942 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:47:33.943 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:47:33.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:33.971 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:47:33.972 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:47:34.002 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:47:34.002 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:47:34.013 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:34.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:34.014 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:34.031 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:47:34.031 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:47:34.115 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:34.144 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:34.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:34.145 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:34.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:34.282 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:34.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:34.283 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:34.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:34.417 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:34.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:34.418 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:34.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-11T08:47:34.519 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:34.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:34.551 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:34.552 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:34.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:34.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:34 smithi145 ceph-mon[33384]: 3.1d scrub starts 2024-09-11T08:47:34.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:34 smithi145 ceph-mon[33384]: pgmap v370: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.2 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.41k op/s; 1112/257817 objects misplaced (0.431%); 9.9 MiB/s, 157 objects/s recovering 2024-09-11T08:47:34.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:34 smithi145 ceph-mon[33384]: osdmap e188: 8 total, 8 up, 8 in 2024-09-11T08:47:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:34 smithi145 ceph-mon[33384]: 3.1c scrub starts 2024-09-11T08:47:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:34 smithi145 ceph-mon[33384]: 6.10 scrub starts 2024-09-11T08:47:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:34 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3500386146' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:47:34.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:34.691 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:34.691 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:34.792 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:34.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:34 smithi092 ceph-mon[41114]: 3.1d scrub starts 2024-09-11T08:47:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:34 smithi092 ceph-mon[41114]: pgmap v370: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.2 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.41k op/s; 1112/257817 objects misplaced (0.431%); 9.9 MiB/s, 157 objects/s recovering 2024-09-11T08:47:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:34 smithi092 ceph-mon[41114]: osdmap e188: 8 total, 8 up, 8 in 2024-09-11T08:47:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:34 smithi092 ceph-mon[41114]: 3.1c scrub starts 2024-09-11T08:47:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:34 smithi092 ceph-mon[41114]: 6.10 scrub starts 2024-09-11T08:47:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:34 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3500386146' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:47:34.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:34 smithi092 ceph-mon[29647]: 3.1d scrub starts 2024-09-11T08:47:34.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:34 smithi092 ceph-mon[29647]: pgmap v370: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.2 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.41k op/s; 1112/257817 objects misplaced (0.431%); 9.9 MiB/s, 157 objects/s recovering 2024-09-11T08:47:34.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:34 smithi092 ceph-mon[29647]: osdmap e188: 8 total, 8 up, 8 in 2024-09-11T08:47:34.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:34 smithi092 ceph-mon[29647]: 3.1c scrub starts 2024-09-11T08:47:34.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:34 smithi092 ceph-mon[29647]: 6.10 scrub starts 2024-09-11T08:47:34.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:34 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3500386146' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:47:34.819 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:34.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:34.820 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:34.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:34.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:34.962 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:34.963 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:35.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:35.094 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:35.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:35.095 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:35.196 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:35.225 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:35.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:35.226 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:35.270 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:35.312 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:35.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:35.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:35.380 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:35.380 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:35.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:35.522 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:35.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:35.523 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:35.526 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:35 smithi092 ceph-mon[29647]: 6.14 deep-scrub starts 2024-09-11T08:47:35.526 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:35 smithi092 ceph-mon[29647]: 6.14 deep-scrub ok 2024-09-11T08:47:35.526 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:35 smithi092 ceph-mon[29647]: 6.a scrub starts 2024-09-11T08:47:35.526 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:35 smithi092 ceph-mon[29647]: 3.1b scrub starts 2024-09-11T08:47:35.526 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:35 smithi092 ceph-mon[29647]: 3.1b scrub ok 2024-09-11T08:47:35.526 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:35 smithi092 ceph-mon[29647]: 4.c scrub starts 2024-09-11T08:47:35.527 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:35 smithi092 ceph-mon[41114]: 6.14 deep-scrub starts 2024-09-11T08:47:35.527 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:35 smithi092 ceph-mon[41114]: 6.14 deep-scrub ok 2024-09-11T08:47:35.527 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:35 smithi092 ceph-mon[41114]: 6.a scrub starts 2024-09-11T08:47:35.527 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:35 smithi092 ceph-mon[41114]: 3.1b scrub starts 2024-09-11T08:47:35.527 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:35 smithi092 ceph-mon[41114]: 3.1b scrub ok 2024-09-11T08:47:35.527 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:35 smithi092 ceph-mon[41114]: 4.c scrub starts 2024-09-11T08:47:35.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:35.654 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:35.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:35.655 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:35.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:35 smithi145 ceph-mon[33384]: 6.14 deep-scrub starts 2024-09-11T08:47:35.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:35 smithi145 ceph-mon[33384]: 6.14 deep-scrub ok 2024-09-11T08:47:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:35 smithi145 ceph-mon[33384]: 6.a scrub starts 2024-09-11T08:47:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:35 smithi145 ceph-mon[33384]: 3.1b scrub starts 2024-09-11T08:47:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:35 smithi145 ceph-mon[33384]: 3.1b scrub ok 2024-09-11T08:47:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:35 smithi145 ceph-mon[33384]: 4.c scrub starts 2024-09-11T08:47:35.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:35.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:35.798 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:35.798 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:35.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:35.935 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:35.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:35.936 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:36.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:36.073 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:36.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:36.074 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:36.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:36.209 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:36.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:36.210 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:36.311 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:36.334 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:36.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:36.335 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:36.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:36.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:36.471 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:36.471 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:36.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:36.613 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:36.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:36.614 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:36.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:36 smithi145 ceph-mon[33384]: pgmap v372: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.2 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 722 op/s; 1112/257817 objects misplaced (0.431%); 5.4 MiB/s, 85 objects/s recovering 2024-09-11T08:47:36.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:36 smithi145 ceph-mon[33384]: 5.4 scrub starts 2024-09-11T08:47:36.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:36 smithi145 ceph-mon[33384]: 5.4 scrub ok 2024-09-11T08:47:36.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:47:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:47:36.714 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:36.743 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:36.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:36.744 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:36 smithi092 ceph-mon[29647]: pgmap v372: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.2 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 722 op/s; 1112/257817 objects misplaced (0.431%); 5.4 MiB/s, 85 objects/s recovering 2024-09-11T08:47:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:36 smithi092 ceph-mon[29647]: 5.4 scrub starts 2024-09-11T08:47:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:36 smithi092 ceph-mon[29647]: 5.4 scrub ok 2024-09-11T08:47:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:36 smithi092 ceph-mon[41114]: pgmap v372: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+remapped, 146 active+clean; 5.2 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 722 op/s; 1112/257817 objects misplaced (0.431%); 5.4 MiB/s, 85 objects/s recovering 2024-09-11T08:47:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:36 smithi092 ceph-mon[41114]: 5.4 scrub starts 2024-09-11T08:47:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:36 smithi092 ceph-mon[41114]: 5.4 scrub ok 2024-09-11T08:47:36.845 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:36.875 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:36.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:36.876 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:36.977 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:37.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:37.014 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:37.014 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:37.115 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:37.151 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:37.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:37.152 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:37.234 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:47:37.253 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:37.281 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:37.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:37.283 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:37.383 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:37.426 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:37.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:37.427 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:37.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:37.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:37 smithi092 ceph-mon[41114]: 4.1e scrub starts 2024-09-11T08:47:37.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:37 smithi092 ceph-mon[41114]: 2.1 scrub starts 2024-09-11T08:47:37.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:37 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:37.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:37 smithi092 ceph-mon[29647]: 4.1e scrub starts 2024-09-11T08:47:37.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:37 smithi092 ceph-mon[29647]: 2.1 scrub starts 2024-09-11T08:47:37.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:37 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:37.575 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:37.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:37.576 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:37 smithi145 ceph-mon[33384]: 4.1e scrub starts 2024-09-11T08:47:37.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:37 smithi145 ceph-mon[33384]: 2.1 scrub starts 2024-09-11T08:47:37.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:37 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:37.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:37.697 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:37.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:37.698 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:37.799 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:37.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:37.826 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:37.826 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:37.877 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:37.927 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:37.955 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:37.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:37.956 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:38.057 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:38.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:38.095 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:38.096 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:38.197 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:38.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:38.239 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:38.240 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:38.340 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:38.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:38.378 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:38.379 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:38.479 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:38.512 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:38.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:38.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:38.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:38.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:38.653 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:38.653 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:38.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:38 smithi145 ceph-mon[33384]: pgmap v373: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.35k op/s; 639/242736 objects misplaced (0.263%); 9.1 MiB/s, 144 objects/s recovering 2024-09-11T08:47:38.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:38 smithi145 ceph-mon[33384]: 3.5 scrub starts 2024-09-11T08:47:38.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:38.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:38 smithi092 ceph-mon[29647]: pgmap v373: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.35k op/s; 639/242736 objects misplaced (0.263%); 9.1 MiB/s, 144 objects/s recovering 2024-09-11T08:47:38.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:38 smithi092 ceph-mon[29647]: 3.5 scrub starts 2024-09-11T08:47:38.759 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:38 smithi092 ceph-mon[41114]: pgmap v373: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.35k op/s; 639/242736 objects misplaced (0.263%); 9.1 MiB/s, 144 objects/s recovering 2024-09-11T08:47:38.759 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:38 smithi092 ceph-mon[41114]: 3.5 scrub starts 2024-09-11T08:47:38.790 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:38.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:38.791 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:38.892 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:38.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:38.924 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:38.924 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:39.025 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:39.050 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:39.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:39.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:39.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:39.190 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:39.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:39.191 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:39.292 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:39.324 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:39.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:39.326 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:39.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:39.466 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:39.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:39.467 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:39.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:39.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:39.609 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:39.609 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:39.643 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:39 smithi092 ceph-mon[29647]: 6.f deep-scrub starts 2024-09-11T08:47:39.644 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:39 smithi092 ceph-mon[29647]: 3.11 scrub starts 2024-09-11T08:47:39.644 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:39 smithi092 ceph-mon[29647]: 6.19 scrub starts 2024-09-11T08:47:39.644 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:39 smithi092 ceph-mon[41114]: 6.f deep-scrub starts 2024-09-11T08:47:39.644 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:39 smithi092 ceph-mon[41114]: 3.11 scrub starts 2024-09-11T08:47:39.644 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:39 smithi092 ceph-mon[41114]: 6.19 scrub starts 2024-09-11T08:47:39.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:39 smithi145 ceph-mon[33384]: 6.f deep-scrub starts 2024-09-11T08:47:39.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:39 smithi145 ceph-mon[33384]: 3.11 scrub starts 2024-09-11T08:47:39.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:39 smithi145 ceph-mon[33384]: 6.19 scrub starts 2024-09-11T08:47:39.710 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:39.732 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:39.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:39.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:39.834 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:39.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:39.861 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:39.861 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:39.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:39.990 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:39.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:39.991 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:40.092 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:40.127 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:40.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:40.128 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:40.229 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:40.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:40.261 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:40.261 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:40.362 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:40.388 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:40.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:40.389 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:40.489 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:40.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:40.521 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:40.522 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:40.623 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:40.659 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:40 smithi092 ceph-mon[29647]: 5.5 scrub starts 2024-09-11T08:47:40.659 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:40 smithi092 ceph-mon[29647]: pgmap v374: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.12k op/s; 639/242736 objects misplaced (0.263%); 9.1 MiB/s, 144 objects/s recovering 2024-09-11T08:47:40.659 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:40 smithi092 ceph-mon[29647]: 6.d scrub starts 2024-09-11T08:47:40.659 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:40 smithi092 ceph-mon[41114]: 5.5 scrub starts 2024-09-11T08:47:40.659 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:40 smithi092 ceph-mon[41114]: pgmap v374: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.12k op/s; 639/242736 objects misplaced (0.263%); 9.1 MiB/s, 144 objects/s recovering 2024-09-11T08:47:40.659 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:40 smithi092 ceph-mon[41114]: 6.d scrub starts 2024-09-11T08:47:40.660 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:40.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:40.661 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:40 smithi145 ceph-mon[33384]: 5.5 scrub starts 2024-09-11T08:47:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:40 smithi145 ceph-mon[33384]: pgmap v374: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.12k op/s; 639/242736 objects misplaced (0.263%); 9.1 MiB/s, 144 objects/s recovering 2024-09-11T08:47:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:40 smithi145 ceph-mon[33384]: 6.d scrub starts 2024-09-11T08:47:40.763 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:40.796 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:40.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:40.797 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:40.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:40.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:40.935 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:40.936 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:41.036 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:41.069 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:41.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:41.070 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:41.171 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:41.205 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:41.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:41.206 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:41.307 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:41.338 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:41.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:41.339 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:41.440 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:41.475 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:41.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:41.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:41.577 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:41.599 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:41.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:41.600 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:41 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T08:47:41.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:41.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:41.745 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:41.745 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:41.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:41 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T08:47:41.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:41 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T08:47:41.816 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:47:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:47:41] "GET /metrics HTTP/1.1" 200 224329 "" "Prometheus/2.43.0" 2024-09-11T08:47:41.846 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:41.884 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:41.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:41.886 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:41.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:42.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:42.042 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:42.142 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:42.165 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:42.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:42.166 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:42.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:42.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:42.310 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:42.310 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:42.411 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:42.436 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:42.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:42.438 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:42.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:42.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:42.568 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:42 smithi145 ceph-mon[33384]: pgmap v375: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 807 op/s; 639/236394 objects misplaced (0.270%); 3.3 MiB/s, 53 objects/s recovering 2024-09-11T08:47:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:42 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:47:42.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:42 smithi145 ceph-mon[33384]: 6.8 scrub starts 2024-09-11T08:47:42.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:42 smithi145 ceph-mon[33384]: 5.f scrub starts 2024-09-11T08:47:42.668 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:42.702 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:42.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:42.703 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:42.772 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:42 smithi092 ceph-mon[29647]: pgmap v375: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 807 op/s; 639/236394 objects misplaced (0.270%); 3.3 MiB/s, 53 objects/s recovering 2024-09-11T08:47:42.772 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:42 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:47:42.772 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:42 smithi092 ceph-mon[29647]: 6.8 scrub starts 2024-09-11T08:47:42.772 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:42 smithi092 ceph-mon[29647]: 5.f scrub starts 2024-09-11T08:47:42.772 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:42 smithi092 ceph-mon[41114]: pgmap v375: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 807 op/s; 639/236394 objects misplaced (0.270%); 3.3 MiB/s, 53 objects/s recovering 2024-09-11T08:47:42.773 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:42 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:47:42.773 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:42 smithi092 ceph-mon[41114]: 6.8 scrub starts 2024-09-11T08:47:42.773 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:42 smithi092 ceph-mon[41114]: 5.f scrub starts 2024-09-11T08:47:42.803 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:42.842 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:42.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:42.843 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:42.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:42.976 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:42.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:42.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:43.078 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:43.112 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:43.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:43.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:43.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:47:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:47:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:47:43.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:43.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:43.258 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:43.258 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:43.359 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:43.386 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:43.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:43.387 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:43.488 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:43.517 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:43.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:43.518 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:43.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:43.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:43.653 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:43.653 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:43.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:43.789 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:43.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:43.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:43.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:43.931 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:43.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:43.932 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:44.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:44.068 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:44.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:44.069 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:44.151 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:44.171 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:44.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:44.232 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:44.333 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:44.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:44.373 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:44.373 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:44.474 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:44.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:44.503 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:44.503 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:44.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:44 smithi092 ceph-mon[29647]: 6.0 scrub starts 2024-09-11T08:47:44.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:44 smithi092 ceph-mon[29647]: pgmap v376: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.10k op/s; 271/224952 objects misplaced (0.120%); 5.3 MiB/s, 84 objects/s recovering 2024-09-11T08:47:44.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:44 smithi092 ceph-mon[29647]: 5.b scrub starts 2024-09-11T08:47:44.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:44 smithi092 ceph-mon[29647]: 3.c scrub starts 2024-09-11T08:47:44.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:44 smithi092 ceph-mon[29647]: 5.b scrub ok 2024-09-11T08:47:44.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:44 smithi092 ceph-mon[41114]: 6.0 scrub starts 2024-09-11T08:47:44.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:44 smithi092 ceph-mon[41114]: pgmap v376: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.10k op/s; 271/224952 objects misplaced (0.120%); 5.3 MiB/s, 84 objects/s recovering 2024-09-11T08:47:44.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:44 smithi092 ceph-mon[41114]: 5.b scrub starts 2024-09-11T08:47:44.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:44 smithi092 ceph-mon[41114]: 3.c scrub starts 2024-09-11T08:47:44.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:44 smithi092 ceph-mon[41114]: 5.b scrub ok 2024-09-11T08:47:44.604 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:44.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:44.634 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:44.634 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:44 smithi145 ceph-mon[33384]: 6.0 scrub starts 2024-09-11T08:47:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:44 smithi145 ceph-mon[33384]: pgmap v376: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.10k op/s; 271/224952 objects misplaced (0.120%); 5.3 MiB/s, 84 objects/s recovering 2024-09-11T08:47:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:44 smithi145 ceph-mon[33384]: 5.b scrub starts 2024-09-11T08:47:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:44 smithi145 ceph-mon[33384]: 3.c scrub starts 2024-09-11T08:47:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:44 smithi145 ceph-mon[33384]: 5.b scrub ok 2024-09-11T08:47:44.735 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:44.761 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:44.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:44.762 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:44.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:44.886 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:44.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:44.887 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:44.988 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:45.010 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:45.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:45.011 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:45.112 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:45.133 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:45.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:45.134 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:45.235 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:45.256 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:45.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:45.257 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:45.358 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:45.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:45.382 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:45.383 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:45.427 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:47:45.428 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:47:45.483 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:45.516 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:45.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:45.517 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:45.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[29647]: 5.6 deep-scrub starts 2024-09-11T08:47:45.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[29647]: 8.4 scrub starts 2024-09-11T08:47:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[29647]: 5.6 deep-scrub ok 2024-09-11T08:47:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[29647]: 3.d scrub starts 2024-09-11T08:47:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[29647]: 3.d scrub ok 2024-09-11T08:47:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[29647]: osdmap e189: 8 total, 8 up, 8 in 2024-09-11T08:47:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[29647]: 4.1b scrub starts 2024-09-11T08:47:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[29647]: 4.1b scrub ok 2024-09-11T08:47:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[41114]: 5.6 deep-scrub starts 2024-09-11T08:47:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[41114]: 8.4 scrub starts 2024-09-11T08:47:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[41114]: 5.6 deep-scrub ok 2024-09-11T08:47:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[41114]: 3.d scrub starts 2024-09-11T08:47:45.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[41114]: 3.d scrub ok 2024-09-11T08:47:45.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[41114]: osdmap e189: 8 total, 8 up, 8 in 2024-09-11T08:47:45.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[41114]: 4.1b scrub starts 2024-09-11T08:47:45.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[41114]: 4.1b scrub ok 2024-09-11T08:47:45.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:45.617 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:45.647 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:45.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:45.648 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:45.749 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:45.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:45.783 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:45.784 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:45.884 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:45 smithi145 ceph-mon[33384]: 5.6 deep-scrub starts 2024-09-11T08:47:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:45 smithi145 ceph-mon[33384]: 8.4 scrub starts 2024-09-11T08:47:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:45 smithi145 ceph-mon[33384]: 5.6 deep-scrub ok 2024-09-11T08:47:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:45 smithi145 ceph-mon[33384]: 3.d scrub starts 2024-09-11T08:47:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:45 smithi145 ceph-mon[33384]: 3.d scrub ok 2024-09-11T08:47:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:45 smithi145 ceph-mon[33384]: osdmap e189: 8 total, 8 up, 8 in 2024-09-11T08:47:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:45 smithi145 ceph-mon[33384]: 4.1b scrub starts 2024-09-11T08:47:45.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:45 smithi145 ceph-mon[33384]: 4.1b scrub ok 2024-09-11T08:47:45.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:47:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:45.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:45.930 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:46.030 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:46.060 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:46.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:46.061 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:46.161 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:46.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:46.197 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:46.197 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:46.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T08:47:46.298 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:46.326 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:46.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:46.327 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:46.428 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:46.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:46.466 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:46.466 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:46.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[29647]: 4.16 scrub starts 2024-09-11T08:47:46.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[29647]: 4.16 scrub ok 2024-09-11T08:47:46.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[29647]: 6.13 scrub starts 2024-09-11T08:47:46.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[29647]: 6.13 scrub ok 2024-09-11T08:47:46.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[29647]: 5.19 scrub starts 2024-09-11T08:47:46.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[29647]: 5.19 scrub ok 2024-09-11T08:47:46.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[29647]: 8.4 scrub ok 2024-09-11T08:47:46.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[29647]: pgmap v378: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.10k op/s; 271/224952 objects misplaced (0.120%); 5.3 MiB/s, 84 objects/s recovering 2024-09-11T08:47:46.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:47:46.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:47:46.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[29647]: osdmap e190: 8 total, 8 up, 8 in 2024-09-11T08:47:46.556 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[41114]: 4.16 scrub starts 2024-09-11T08:47:46.557 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[41114]: 4.16 scrub ok 2024-09-11T08:47:46.557 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[41114]: 6.13 scrub starts 2024-09-11T08:47:46.557 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[41114]: 6.13 scrub ok 2024-09-11T08:47:46.557 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[41114]: 5.19 scrub starts 2024-09-11T08:47:46.557 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[41114]: 5.19 scrub ok 2024-09-11T08:47:46.557 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[41114]: 8.4 scrub ok 2024-09-11T08:47:46.557 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[41114]: pgmap v378: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.10k op/s; 271/224952 objects misplaced (0.120%); 5.3 MiB/s, 84 objects/s recovering 2024-09-11T08:47:46.557 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:47:46.557 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:47:46.557 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:46 smithi092 ceph-mon[41114]: osdmap e190: 8 total, 8 up, 8 in 2024-09-11T08:47:46.567 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:46.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:46.596 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:46.597 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:46.698 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:46.726 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:46.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:46.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:46.828 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:46.863 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:46.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:46.864 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:46.912 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:47:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:47:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:46 smithi145 ceph-mon[33384]: 4.16 scrub starts 2024-09-11T08:47:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:46 smithi145 ceph-mon[33384]: 4.16 scrub ok 2024-09-11T08:47:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:46 smithi145 ceph-mon[33384]: 6.13 scrub starts 2024-09-11T08:47:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:46 smithi145 ceph-mon[33384]: 6.13 scrub ok 2024-09-11T08:47:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:46 smithi145 ceph-mon[33384]: 5.19 scrub starts 2024-09-11T08:47:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:46 smithi145 ceph-mon[33384]: 5.19 scrub ok 2024-09-11T08:47:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:46 smithi145 ceph-mon[33384]: 8.4 scrub ok 2024-09-11T08:47:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:46 smithi145 ceph-mon[33384]: pgmap v378: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.10k op/s; 271/224952 objects misplaced (0.120%); 5.3 MiB/s, 84 objects/s recovering 2024-09-11T08:47:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:46 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:47:46.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:46 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:47:46.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:46 smithi145 ceph-mon[33384]: osdmap e190: 8 total, 8 up, 8 in 2024-09-11T08:47:46.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:47.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:47.001 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:47.101 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:47.134 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:47.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:47.135 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:47.226 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:47:47.226 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"211625d4-7018-11ef-bcea-c7b262605968","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1 pg undersized","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":761,"monmap":{"epoch":3,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":190,"num_osds":8,"num_up_osds":8,"osd_up_since":1726043903,"num_in_osds":8,"osd_in_since":1726043886,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":147},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":148,"num_pools":7,"num_objects":74984,"data_bytes":4901837044,"bytes_used":17935286272,"bytes_avail":750024998912,"bytes_total":767960285184,"misplaced_objects":271,"misplaced_total":224952,"misplaced_ratio":0.001204701447419894,"recovering_objects_per_sec":84,"recovering_bytes_per_sec":5509372,"recovering_keys_per_sec":0,"num_objects_recovered":841,"num_bytes_recovered":55115776,"num_keys_recovered":0,"read_bytes_sec":1023,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":1095},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.92:8443/","prometheus":"http://172.21.15.92:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-11T08:40:17.013941+0000","services":{"rgw":{"daemons":{"summary":"","24352":{"start_epoch":12,"start_stamp":"2024-09-11T08:38:46.107187+0000","gid":24352,"addr":"172.21.15.92:0/1175084397","metadata":{"arch":"x86_64","ceph_release":"quincy","ceph_version":"ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)","ceph_version_short":"17.2.7-1623-gd0e6828a","container_hostname":"smithi092","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40","cpu":"Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi092","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024","kernel_version":"5.14.0-505.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32487284","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"02abb3ce-34bc-4920-b6fd-8ad4b7819694","zone_name":"default","zonegroup_id":"8abc369a-51df-4e8f-b783-2e835134b342","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T08:47:47.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:47.274 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:47.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:47.275 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:47.376 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:47.406 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:47.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:47.407 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:47.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:47.554 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:47.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:47.555 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:47.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[29647]: 4.5 scrub starts 2024-09-11T08:47:47.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[29647]: 4.5 scrub ok 2024-09-11T08:47:47.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[29647]: 3.12 scrub starts 2024-09-11T08:47:47.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[29647]: 3.12 scrub ok 2024-09-11T08:47:47.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[29647]: 8.d deep-scrub starts 2024-09-11T08:47:47.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[29647]: 5.1a scrub starts 2024-09-11T08:47:47.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[29647]: 5.1a scrub ok 2024-09-11T08:47:47.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:47.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/122112233' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-11T08:47:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[41114]: 4.5 scrub starts 2024-09-11T08:47:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[41114]: 4.5 scrub ok 2024-09-11T08:47:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[41114]: 3.12 scrub starts 2024-09-11T08:47:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[41114]: 3.12 scrub ok 2024-09-11T08:47:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[41114]: 8.d deep-scrub starts 2024-09-11T08:47:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[41114]: 5.1a scrub starts 2024-09-11T08:47:47.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[41114]: 5.1a scrub ok 2024-09-11T08:47:47.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:47.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:47 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/122112233' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-11T08:47:47.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:47.695 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:47.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:47.696 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:47.797 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:47.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:47.835 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:47.835 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:47.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:47 smithi145 ceph-mon[33384]: 4.5 scrub starts 2024-09-11T08:47:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:47 smithi145 ceph-mon[33384]: 4.5 scrub ok 2024-09-11T08:47:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:47 smithi145 ceph-mon[33384]: 3.12 scrub starts 2024-09-11T08:47:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:47 smithi145 ceph-mon[33384]: 3.12 scrub ok 2024-09-11T08:47:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:47 smithi145 ceph-mon[33384]: 8.d deep-scrub starts 2024-09-11T08:47:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:47 smithi145 ceph-mon[33384]: 5.1a scrub starts 2024-09-11T08:47:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:47 smithi145 ceph-mon[33384]: 5.1a scrub ok 2024-09-11T08:47:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:47 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:47 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/122112233' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-11T08:47:47.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:47.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:47.972 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:48.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:48.118 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:48.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:48.120 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:48.221 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:48.250 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:48.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:48.251 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:48.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:48.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:48.385 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:48.385 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:47:48.486 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:48.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:48.522 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:48.522 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:48.544 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[29647]: 4.a scrub starts 2024-09-11T08:47:48.544 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[29647]: 4.a scrub ok 2024-09-11T08:47:48.544 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[29647]: pgmap v380: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s; 5.0 MiB/s, 79 objects/s recovering 2024-09-11T08:47:48.544 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T08:47:48.544 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[29647]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1 pg undersized) 2024-09-11T08:47:48.544 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:47:48.544 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[29647]: 5.d scrub ok 2024-09-11T08:47:48.544 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/870202140' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:47:48.544 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:47:48.545 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:47:48.545 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[41114]: 4.a scrub starts 2024-09-11T08:47:48.545 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[41114]: 4.a scrub ok 2024-09-11T08:47:48.545 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[41114]: pgmap v380: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s; 5.0 MiB/s, 79 objects/s recovering 2024-09-11T08:47:48.545 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T08:47:48.545 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[41114]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1 pg undersized) 2024-09-11T08:47:48.545 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:47:48.545 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[41114]: 5.d scrub ok 2024-09-11T08:47:48.545 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/870202140' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:47:48.545 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:47:48.545 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:48 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:47:48.623 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:48.654 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:48.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:48.655 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:48.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:48.789 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:48.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:48.790 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:48.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:48.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:48 smithi145 ceph-mon[33384]: 4.a scrub starts 2024-09-11T08:47:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:48 smithi145 ceph-mon[33384]: 4.a scrub ok 2024-09-11T08:47:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:48 smithi145 ceph-mon[33384]: pgmap v380: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s; 5.0 MiB/s, 79 objects/s recovering 2024-09-11T08:47:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:48 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T08:47:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:48 smithi145 ceph-mon[33384]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1 pg undersized) 2024-09-11T08:47:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:48 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:47:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:48 smithi145 ceph-mon[33384]: 5.d scrub ok 2024-09-11T08:47:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:48 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/870202140' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:47:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:48 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:47:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:48 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:47:48.922 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:48.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:48.923 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:48.998 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-11T08:47:48.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:47:49.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:49.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:49.096 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:49.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:49.237 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:49.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:49.238 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:49.339 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:49.371 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:49.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:49.372 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:49.473 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:49.507 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:49.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:49.507 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:49.608 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:49.637 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:49.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:49.638 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:49.739 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:49.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:49.776 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:49.776 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:49.801 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:47:49.801 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:47:49.801 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:47:49.802 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:47:49.802 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:47:49.802 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:47:49.802 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:47:49.802 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:47:49.802 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:47:49.802 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:47:49.802 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:47:49.802 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:47:49.803 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:47:49.803 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:47:49.803 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:47:49.803 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:47:49.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[29647]: 6.9 scrub starts 2024-09-11T08:47:49.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[29647]: 6.9 scrub ok 2024-09-11T08:47:49.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[29647]: 3.15 scrub starts 2024-09-11T08:47:49.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[29647]: 3.15 scrub ok 2024-09-11T08:47:49.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:47:49.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/870202140' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:47:49.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[29647]: osdmap e191: 8 total, 8 up, 8 in 2024-09-11T08:47:49.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4141783648' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:47:49.805 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[41114]: 6.9 scrub starts 2024-09-11T08:47:49.805 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[41114]: 6.9 scrub ok 2024-09-11T08:47:49.805 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[41114]: 3.15 scrub starts 2024-09-11T08:47:49.806 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[41114]: 3.15 scrub ok 2024-09-11T08:47:49.806 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:47:49.806 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/870202140' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:47:49.806 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[41114]: osdmap e191: 8 total, 8 up, 8 in 2024-09-11T08:47:49.806 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:49 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4141783648' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:47:49.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:49.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:49.906 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:49.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:49 smithi145 ceph-mon[33384]: 6.9 scrub starts 2024-09-11T08:47:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:49 smithi145 ceph-mon[33384]: 6.9 scrub ok 2024-09-11T08:47:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:49 smithi145 ceph-mon[33384]: 3.15 scrub starts 2024-09-11T08:47:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:49 smithi145 ceph-mon[33384]: 3.15 scrub ok 2024-09-11T08:47:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:49 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:47:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:49 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/870202140' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:47:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:49 smithi145 ceph-mon[33384]: osdmap e191: 8 total, 8 up, 8 in 2024-09-11T08:47:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:49 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4141783648' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:47:50.007 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:50.039 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:50.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:50.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:50.141 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:50.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:50.172 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:50.172 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:50.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:50.308 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:50.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:50.309 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:50.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:50.447 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:50.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:50.448 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:50.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:50.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:50 smithi092 ceph-mon[29647]: pgmap v382: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 805 op/s; 2.8 MiB/s, 45 objects/s recovering 2024-09-11T08:47:50.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:50 smithi092 ceph-mon[41114]: pgmap v382: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 805 op/s; 2.8 MiB/s, 45 objects/s recovering 2024-09-11T08:47:50.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:50.584 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:50.584 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:50.685 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:50.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:50.722 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:50.723 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:50.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:50.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:50.855 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:50.855 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:50.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:50 smithi145 ceph-mon[33384]: pgmap v382: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 805 op/s; 2.8 MiB/s, 45 objects/s recovering 2024-09-11T08:47:50.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:50.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:50.997 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:50.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:51.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:51.130 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:51.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:51.131 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:51.232 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:51.259 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:51.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:51.260 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:51.361 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:51.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:51.394 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:51.394 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:51.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:51.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:51.529 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:51.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:51.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:51.665 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:51.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:51.673 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:51.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:51.804 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:47:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:47:51] "GET /metrics HTTP/1.1" 200 224351 "" "Prometheus/2.43.0" 2024-09-11T08:47:51.808 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:51.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:51.810 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:51.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:51.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:51.947 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:51.947 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:52.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:52.081 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:52.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:52.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:52.183 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:52.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:52.226 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:52.226 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:52.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:47:52.327 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:52.356 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:52.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:52.357 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:52.458 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:52.494 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:52.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:52.495 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:52.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:52.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:52.628 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:52.628 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:52.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:52 smithi145 ceph-mon[33384]: pgmap v383: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 876 op/s; 2.5 MiB/s, 40 objects/s recovering 2024-09-11T08:47:52.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:52 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:47:52.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:52 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:47:52.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:52 smithi145 ceph-mon[33384]: osdmap e192: 8 total, 8 up, 8 in 2024-09-11T08:47:52.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:52 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:47:52.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:52 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:47:52.729 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:52.755 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:52.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:52.756 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:52.785 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:52 smithi092 ceph-mon[29647]: pgmap v383: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 876 op/s; 2.5 MiB/s, 40 objects/s recovering 2024-09-11T08:47:52.785 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:52 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:47:52.785 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:52 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:47:52.785 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:52 smithi092 ceph-mon[29647]: osdmap e192: 8 total, 8 up, 8 in 2024-09-11T08:47:52.785 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:52 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:47:52.785 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:52 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:47:52.786 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:52 smithi092 ceph-mon[41114]: pgmap v383: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 876 op/s; 2.5 MiB/s, 40 objects/s recovering 2024-09-11T08:47:52.786 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:52 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:47:52.786 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:52 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:47:52.786 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:52 smithi092 ceph-mon[41114]: osdmap e192: 8 total, 8 up, 8 in 2024-09-11T08:47:52.786 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:52 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:47:52.786 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:52 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:47:52.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:52.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:52.892 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:52.892 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:52.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:47:52.993 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:53.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:53.033 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:53.033 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:53.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:53.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:47:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:47:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:47:53.170 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:53.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:53.171 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:53.271 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:53.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:53.308 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:53.308 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:53.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:53.442 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:53.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:53.443 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:53.544 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:53.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:53.573 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:53.573 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:53.674 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:53.703 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:53.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:53.705 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:53.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:53 smithi092 ceph-mon[29647]: osdmap e193: 8 total, 8 up, 8 in 2024-09-11T08:47:53.760 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:53 smithi092 ceph-mon[41114]: osdmap e193: 8 total, 8 up, 8 in 2024-09-11T08:47:53.806 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:53.861 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:53.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:53.863 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:53.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:53 smithi145 ceph-mon[33384]: osdmap e193: 8 total, 8 up, 8 in 2024-09-11T08:47:53.964 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:53.998 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:53.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:53.999 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:54.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:54.131 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:54.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:54.132 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:54.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:54.267 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:54.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:54.268 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:54.296 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:54.318 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:47:54.318 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:47:54.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.318 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:53607 8.d e194/149 8:b5907f68:::benchmark_data_smithi092_117541_object75206:head v 194'12519, mlcod=194'12513)", 2024-09-11T08:47:54.319 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:54.294962+0000", 2024-09-11T08:47:54.319 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.020477919000000001, 2024-09-11T08:47:54.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020507221999999999, 2024-09-11T08:47:54.319 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:47:54.319 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:47:54.319 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:47:54.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:47:54.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.294962+0000", 2024-09-11T08:47:54.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:47:54.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.294962+0000", 2024-09-11T08:47:54.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:47:54.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.294962+0000", 2024-09-11T08:47:54.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:47:54.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:47:54.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.294967+0000", 2024-09-11T08:47:54.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4149999999999998e-06 2024-09-11T08:47:54.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:47:54.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.294974+0000", 2024-09-11T08:47:54.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4449999999999999e-06 2024-09-11T08:47:54.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:47:54.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.294979+0000", 2024-09-11T08:47:54.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1389999999999997e-06 2024-09-11T08:47:54.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:47:54.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.297080+0000", 2024-09-11T08:47:54.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021010730000000002 2024-09-11T08:47:54.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:47:54.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.297097+0000", 2024-09-11T08:47:54.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7499e-05 2024-09-11T08:47:54.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.324 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:47:54.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.324 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:53609 8.d e194/149 8:b5921cc1:::benchmark_data_smithi092_117541_object77836:head v 194'12520, mlcod=194'12513)", 2024-09-11T08:47:54.324 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:54.301848+0000", 2024-09-11T08:47:54.324 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.013591489999999999, 2024-09-11T08:47:54.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013691709, 2024-09-11T08:47:54.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:47:54.324 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:47:54.324 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:47:54.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:47:54.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301848+0000", 2024-09-11T08:47:54.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:47:54.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301848+0000", 2024-09-11T08:47:54.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:47:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301847+0000", 2024-09-11T08:47:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:47:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:47:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301858+0000", 2024-09-11T08:47:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0368e-05 2024-09-11T08:47:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:47:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301875+0000", 2024-09-11T08:47:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7193e-05 2024-09-11T08:47:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:47:54.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301882+0000", 2024-09-11T08:47:54.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0659999999999998e-06 2024-09-11T08:47:54.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:47:54.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.303164+0000", 2024-09-11T08:47:54.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001282135 2024-09-11T08:47:54.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:47:54.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.303183+0000", 2024-09-11T08:47:54.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9466e-05 2024-09-11T08:47:54.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.328 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:47:54.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.328 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:53610 8.d e194/149 8:b5926917:::benchmark_data_smithi092_117541_object112299:head v 194'12521, mlcod=194'12513)", 2024-09-11T08:47:54.328 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:54.301890+0000", 2024-09-11T08:47:54.328 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.013550002, 2024-09-11T08:47:54.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013737343000000001, 2024-09-11T08:47:54.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:47:54.328 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:47:54.328 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:47:54.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:47:54.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301890+0000", 2024-09-11T08:47:54.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:47:54.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301890+0000", 2024-09-11T08:47:54.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:47:54.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301890+0000", 2024-09-11T08:47:54.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:47:54.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:47:54.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301894+0000", 2024-09-11T08:47:54.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3980000000000001e-06 2024-09-11T08:47:54.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:47:54.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301902+0000", 2024-09-11T08:47:54.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3610000000000001e-06 2024-09-11T08:47:54.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:47:54.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301906+0000", 2024-09-11T08:47:54.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.692e-06 2024-09-11T08:47:54.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:47:54.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.307312+0000", 2024-09-11T08:47:54.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054056390000000003 2024-09-11T08:47:54.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:47:54.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.307461+0000", 2024-09-11T08:47:54.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000149109 2024-09-11T08:47:54.332 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.332 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:47:54.332 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.332 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:53611 8.d e194/149)", 2024-09-11T08:47:54.332 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:54.301910+0000", 2024-09-11T08:47:54.332 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.013529931, 2024-09-11T08:47:54.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013775314, 2024-09-11T08:47:54.333 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:47:54.333 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-11T08:47:54.333 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:47:54.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:47:54.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301910+0000", 2024-09-11T08:47:54.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:47:54.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301910+0000", 2024-09-11T08:47:54.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:47:54.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301910+0000", 2024-09-11T08:47:54.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967296 2024-09-11T08:47:54.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:47:54.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301918+0000", 2024-09-11T08:47:54.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1939999999999998e-06 2024-09-11T08:47:54.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:47:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301925+0000", 2024-09-11T08:47:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2919999999999998e-06 2024-09-11T08:47:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:47:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.301929+0000", 2024-09-11T08:47:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.58e-06 2024-09-11T08:47:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:47:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.336 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:53612 8.d e194/149)", 2024-09-11T08:47:54.336 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:54.303896+0000", 2024-09-11T08:47:54.336 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011543706000000001, 2024-09-11T08:47:54.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011834302, 2024-09-11T08:47:54.336 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:47:54.336 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-11T08:47:54.336 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:47:54.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:47:54.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.303896+0000", 2024-09-11T08:47:54.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:47:54.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.303896+0000", 2024-09-11T08:47:54.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:47:54.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.303895+0000", 2024-09-11T08:47:54.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:47:54.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:47:54.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.303903+0000", 2024-09-11T08:47:54.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0229999999999992e-06 2024-09-11T08:47:54.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:47:54.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.303915+0000", 2024-09-11T08:47:54.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1377e-05 2024-09-11T08:47:54.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:47:54.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.303920+0000", 2024-09-11T08:47:54.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6010000000000002e-06 2024-09-11T08:47:54.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.338 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:47:54.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.339 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:53614 8.d e194/149)", 2024-09-11T08:47:54.339 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:54.306066+0000", 2024-09-11T08:47:54.339 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0093738060000000002, 2024-09-11T08:47:54.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097197029999999997, 2024-09-11T08:47:54.339 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:47:54.339 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-11T08:47:54.339 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:47:54.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:47:54.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.306066+0000", 2024-09-11T08:47:54.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:47:54.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.306066+0000", 2024-09-11T08:47:54.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:47:54.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.306065+0000", 2024-09-11T08:47:54.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:47:54.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:47:54.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.306074+0000", 2024-09-11T08:47:54.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6289999999999999e-06 2024-09-11T08:47:54.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:47:54.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.306083+0000", 2024-09-11T08:47:54.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2790000000000004e-06 2024-09-11T08:47:54.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:47:54.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.306089+0000", 2024-09-11T08:47:54.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5339999999999999e-06 2024-09-11T08:47:54.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.342 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:47:54.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.342 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:53615 8.d e194/149)", 2024-09-11T08:47:54.342 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:54.308134+0000", 2024-09-11T08:47:54.342 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.007305654, 2024-09-11T08:47:54.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077118849999999999, 2024-09-11T08:47:54.342 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:47:54.342 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-11T08:47:54.342 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:47:54.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:47:54.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.308134+0000", 2024-09-11T08:47:54.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:47:54.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.308134+0000", 2024-09-11T08:47:54.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:47:54.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.308132+0000", 2024-09-11T08:47:54.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999976 2024-09-11T08:47:54.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:47:54.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.308140+0000", 2024-09-11T08:47:54.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4800000000000001e-06 2024-09-11T08:47:54.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:47:54.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.308151+0000", 2024-09-11T08:47:54.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0757000000000001e-05 2024-09-11T08:47:54.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:47:54.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.308156+0000", 2024-09-11T08:47:54.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.181e-06 2024-09-11T08:47:54.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.345 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:47:54.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.346 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:53617 8.d e194/149)", 2024-09-11T08:47:54.346 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:54.310685+0000", 2024-09-11T08:47:54.346 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0047549460000000003, 2024-09-11T08:47:54.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052046169999999999, 2024-09-11T08:47:54.346 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:47:54.346 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-11T08:47:54.346 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:47:54.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:47:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.310685+0000", 2024-09-11T08:47:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:47:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.310685+0000", 2024-09-11T08:47:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:47:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.310684+0000", 2024-09-11T08:47:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:47:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:47:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.310691+0000", 2024-09-11T08:47:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7410000000000003e-06 2024-09-11T08:47:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:47:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.310705+0000", 2024-09-11T08:47:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4372999999999999e-05 2024-09-11T08:47:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:47:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.310711+0000", 2024-09-11T08:47:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4e-06 2024-09-11T08:47:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:47:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15126.0:178762 8.a e194/149 8:5453dec1:::benchmark_data_smithi092_117541_object58801:head v 194'11468, mlcod=194'11466)", 2024-09-11T08:47:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:54.313858+0000", 2024-09-11T08:47:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0015816949999999999, 2024-09-11T08:47:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002080115, 2024-09-11T08:47:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:47:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:47:54.350 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:47:54.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:47:54.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.313858+0000", 2024-09-11T08:47:54.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:47:54.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.313858+0000", 2024-09-11T08:47:54.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:47:54.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.313857+0000", 2024-09-11T08:47:54.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:47:54.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:47:54.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.313865+0000", 2024-09-11T08:47:54.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6359999999999994e-06 2024-09-11T08:47:54.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:47:54.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.313881+0000", 2024-09-11T08:47:54.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6521999999999999e-05 2024-09-11T08:47:54.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:47:54.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.313887+0000", 2024-09-11T08:47:54.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.908e-06 2024-09-11T08:47:54.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:47:54.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.313986+0000", 2024-09-11T08:47:54.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8636999999999998e-05 2024-09-11T08:47:54.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:47:54.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.314004+0000", 2024-09-11T08:47:54.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8139e-05 2024-09-11T08:47:54.353 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.353 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:47:54.353 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.353 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:53619 8.d e194/149)", 2024-09-11T08:47:54.353 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:54.313884+0000", 2024-09-11T08:47:54.353 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0015561730000000001, 2024-09-11T08:47:54.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021118389999999999, 2024-09-11T08:47:54.353 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:47:54.354 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-11T08:47:54.354 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:47:54.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:47:54.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.313884+0000", 2024-09-11T08:47:54.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:47:54.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.313884+0000", 2024-09-11T08:47:54.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:47:54.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.313883+0000", 2024-09-11T08:47:54.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:47:54.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:47:54.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.313889+0000", 2024-09-11T08:47:54.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.472e-06 2024-09-11T08:47:54.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:47:54.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.313900+0000", 2024-09-11T08:47:54.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0711e-05 2024-09-11T08:47:54.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:47:54.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.313905+0000", 2024-09-11T08:47:54.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3970000000000002e-06 2024-09-11T08:47:54.357 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.357 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:47:54.358 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.358 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15126.0:178761 8.e e194/149)", 2024-09-11T08:47:54.358 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:54.314980+0000", 2024-09-11T08:47:54.358 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.000459562, 2024-09-11T08:47:54.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010600679999999999, 2024-09-11T08:47:54.358 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:47:54.358 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-11T08:47:54.358 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:47:54.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:47:54.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.314980+0000", 2024-09-11T08:47:54.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:47:54.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.314980+0000", 2024-09-11T08:47:54.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:47:54.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:47:54.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.314980+0000", 2024-09-11T08:47:54.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:47:54.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:47:54.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.314986+0000", 2024-09-11T08:47:54.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8549999999999996e-06 2024-09-11T08:47:54.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:47:54.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.314997+0000", 2024-09-11T08:47:54.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0515e-05 2024-09-11T08:47:54.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:47:54.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:47:54.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:47:54.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:54.315002+0000", 2024-09-11T08:47:54.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.977e-06 2024-09-11T08:47:54.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.361 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:47:54.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:47:54.361 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:47:54.361 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 11 2024-09-11T08:47:54.361 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:47:54.369 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:54.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:54.423 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:54.423 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:54.524 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:54.549 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:54.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:54.550 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:54.651 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:54.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:54 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:47:54.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:54 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:47:54.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:54 smithi092 ceph-mon[29647]: pgmap v386: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 940 op/s 2024-09-11T08:47:54.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:54 smithi092 ceph-mon[29647]: osdmap e194: 8 total, 8 up, 8 in 2024-09-11T08:47:54.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:54 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:47:54.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:54 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:47:54.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:54 smithi092 ceph-mon[41114]: pgmap v386: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 940 op/s 2024-09-11T08:47:54.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:54 smithi092 ceph-mon[41114]: osdmap e194: 8 total, 8 up, 8 in 2024-09-11T08:47:54.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:54.679 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:54.679 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:54.780 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:54.811 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:54.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:54.813 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:54.889 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:47:54.896 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:47:54.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:54 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:47:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:54 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:47:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:54 smithi145 ceph-mon[33384]: pgmap v386: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 940 op/s 2024-09-11T08:47:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:54 smithi145 ceph-mon[33384]: osdmap e194: 8 total, 8 up, 8 in 2024-09-11T08:47:54.914 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:54.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:54.941 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:54.942 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:55.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:55.076 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:55.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:55.077 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:55.178 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:55.206 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:55.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:55.207 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:55.308 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:55.340 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:55.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:55.341 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:55.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:55.473 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:55.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:55.474 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:55.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:55.607 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:55.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:55.608 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:55.709 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:55.741 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:55.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:55.742 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:55.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:55.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:55.879 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:55.879 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:55.980 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:56.010 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:56.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:56.011 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:56.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:56.143 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:56.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:56.144 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:56.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-11T08:47:56.245 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:56.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:56.275 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:56.275 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:56.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:56.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:56.444 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:56.544 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:56.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:56.581 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:56.582 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:56.683 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:56.719 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:56.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:56.720 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:56.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:56 smithi092 ceph-mon[29647]: from='client.25103 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:47:56.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:56 smithi092 ceph-mon[29647]: pgmap v388: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 940 op/s 2024-09-11T08:47:56.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:56 smithi092 ceph-mon[41114]: from='client.25103 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:47:56.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:56 smithi092 ceph-mon[41114]: pgmap v388: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 940 op/s 2024-09-11T08:47:56.820 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:56.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:56.850 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:56.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:56 smithi145 ceph-mon[33384]: from='client.25103 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:47:56.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:56 smithi145 ceph-mon[33384]: pgmap v388: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 940 op/s 2024-09-11T08:47:56.912 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:47:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:47:56.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:56.984 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:56.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:56.986 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:57.087 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:57.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:57.124 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:57.124 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:57.225 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:57.257 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:57.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:57.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:57.359 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:57.377 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:57.389 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:57.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:57.391 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:57.409 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-11T08:47:53.468242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4215609588,"num_objects":64513,"num_object_clones":0,"num_object_copies":193539,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64513,"num_whiteouts":0,"num_read":1396,"num_read_kb":1253,"num_write":176144,"num_write_kb":7678984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7873,"num_bytes_recovered":509741200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85977,"ondisk_log_size":85977,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":446,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":15494372,"kb_used_data":13057100,"kb_used_omap":13,"kb_used_meta":2436978,"kb_avail":734466844,"statfs":{"total":767960285184,"available":752094048256,"internally_reserved":0,"allocated":13370470400,"data_stored":13367828568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2495466176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,8,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-369623040,"num_objects":-5640,"num_object_clones":0,"num_object_copies":-16920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5640,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5640,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.024994"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":185,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.449463+0000","last_change":"2024-09-11T08:45:06.574593+0000","last_active":"2024-09-11T08:47:48.449463+0000","last_peered":"2024-09-11T08:47:48.449463+0000","last_clean":"2024-09-11T08:47:48.449463+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:47:48.449463+0000","last_undegraded":"2024-09-11T08:47:48.449463+0000","last_fullsized":"2024-09-11T08:47:48.449463+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:06.574539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.574539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028477630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":232,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.445447+0000","last_change":"2024-09-11T08:46:01.991434+0000","last_active":"2024-09-11T08:47:45.445447+0000","last_peered":"2024-09-11T08:47:45.445447+0000","last_clean":"2024-09-11T08:47:45.445447+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:47:45.445447+0000","last_undegraded":"2024-09-11T08:47:45.445447+0000","last_fullsized":"2024-09-11T08:47:45.445447+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:45:13.113315+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.113315+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018068470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":191,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.457340+0000","last_change":"2024-09-11T08:47:44.884292+0000","last_active":"2024-09-11T08:47:48.457340+0000","last_peered":"2024-09-11T08:47:48.457340+0000","last_clean":"2024-09-11T08:47:48.457340+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:47:48.457340+0000","last_undegraded":"2024-09-11T08:47:48.457340+0000","last_fullsized":"2024-09-11T08:47:48.457340+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.884217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.884217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:57.658768+0000","scrub_duration":0.011480829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":158,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.445735+0000","last_change":"2024-09-11T08:44:26.005322+0000","last_active":"2024-09-11T08:47:45.445735+0000","last_peered":"2024-09-11T08:47:45.445735+0000","last_clean":"2024-09-11T08:47:45.445735+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:47:45.445735+0000","last_undegraded":"2024-09-11T08:47:45.445735+0000","last_fullsized":"2024-09-11T08:47:45.445735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.005270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.005270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038675070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":188,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.450047+0000","last_change":"2024-09-11T08:46:01.984446+0000","last_active":"2024-09-11T08:47:48.450047+0000","last_peered":"2024-09-11T08:47:48.450047+0000","last_clean":"2024-09-11T08:47:48.450047+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:47:48.450047+0000","last_undegraded":"2024-09-11T08:47:48.450047+0000","last_fullsized":"2024-09-11T08:47:48.450047+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:28.356172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.356172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029955870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":194,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.450484+0000","last_change":"2024-09-11T08:46:01.981971+0000","last_active":"2024-09-11T08:47:48.450484+0000","last_peered":"2024-09-11T08:47:48.450484+0000","last_clean":"2024-09-11T08:47:48.450484+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:47:48.450484+0000","last_undegraded":"2024-09-11T08:47:48.450484+0000","last_fullsized":"2024-09-11T08:47:48.450484+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.330642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.330642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0087830849999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":204,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.447528+0000","last_change":"2024-09-11T08:44:24.344328+0000","last_active":"2024-09-11T08:47:48.447528+0000","last_peered":"2024-09-11T08:47:48.447528+0000","last_clean":"2024-09-11T08:47:48.447528+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:47:48.447528+0000","last_undegraded":"2024-09-11T08:47:48.447528+0000","last_fullsized":"2024-09-11T08:47:48.447528+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:44:24.344127+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.344127+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010996336000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":160,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.445417+0000","last_change":"2024-09-11T08:43:28.988751+0000","last_active":"2024-09-11T08:47:45.445417+0000","last_peered":"2024-09-11T08:47:45.445417+0000","last_clean":"2024-09-11T08:47:45.445417+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:47:45.445417+0000","last_undegraded":"2024-09-11T08:47:45.445417+0000","last_fullsized":"2024-09-11T08:47:45.445417+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:43:28.988697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00375019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":200,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.423047+0000","last_change":"2024-09-11T08:47:32.636647+0000","last_active":"2024-09-11T08:47:45.423047+0000","last_peered":"2024-09-11T08:47:45.423047+0000","last_clean":"2024-09-11T08:47:45.423047+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:47:45.423047+0000","last_undegraded":"2024-09-11T08:47:45.423047+0000","last_fullsized":"2024-09-11T08:47:45.423047+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:12.878132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.878132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077623500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":192,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.445886+0000","last_change":"2024-09-11T08:44:27.963772+0000","last_active":"2024-09-11T08:47:45.445886+0000","last_peered":"2024-09-11T08:47:45.445886+0000","last_clean":"2024-09-11T08:47:45.445886+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:47:45.445886+0000","last_undegraded":"2024-09-11T08:47:45.445886+0000","last_fullsized":"2024-09-11T08:47:45.445886+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:44:27.963689+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:44:27.963689+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011544877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":176,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.422089+0000","last_change":"2024-09-11T08:47:09.804752+0000","last_active":"2024-09-11T08:47:45.422089+0000","last_peered":"2024-09-11T08:47:45.422089+0000","last_clean":"2024-09-11T08:47:45.422089+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:47:45.422089+0000","last_undegraded":"2024-09-11T08:47:45.422089+0000","last_fullsized":"2024-09-11T08:47:45.422089+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:33.644869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.644869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020832210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":179,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.447378+0000","last_change":"2024-09-11T08:45:31.457775+0000","last_active":"2024-09-11T08:47:48.447378+0000","last_peered":"2024-09-11T08:47:48.447378+0000","last_clean":"2024-09-11T08:47:48.447378+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:47:48.447378+0000","last_undegraded":"2024-09-11T08:47:48.447378+0000","last_fullsized":"2024-09-11T08:47:48.447378+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.457652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.457652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006759045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":180,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.422977+0000","last_change":"2024-09-11T08:47:33.593768+0000","last_active":"2024-09-11T08:47:45.422977+0000","last_peered":"2024-09-11T08:47:45.422977+0000","last_clean":"2024-09-11T08:47:45.422977+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:47:45.422977+0000","last_undegraded":"2024-09-11T08:47:45.422977+0000","last_fullsized":"2024-09-11T08:47:45.422977+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:09.871637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:09.871637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029269349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":182,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.445898+0000","last_change":"2024-09-11T08:47:44.866144+0000","last_active":"2024-09-11T08:47:45.445898+0000","last_peered":"2024-09-11T08:47:45.445898+0000","last_clean":"2024-09-11T08:47:45.445898+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:47:45.445898+0000","last_undegraded":"2024-09-11T08:47:45.445898+0000","last_fullsized":"2024-09-11T08:47:45.445898+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:47:44.866057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.866057+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:03.406837+0000","scrub_duration":0.0020690169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":189,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:46.410534+0000","last_change":"2024-09-11T08:47:46.410478+0000","last_active":"2024-09-11T08:47:46.410534+0000","last_peered":"2024-09-11T08:47:46.410534+0000","last_clean":"2024-09-11T08:47:46.410534+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:47:46.410534+0000","last_undegraded":"2024-09-11T08:47:46.410534+0000","last_fullsized":"2024-09-11T08:47:46.410534+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:46.410402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:47:46.410402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:51.920397+0000","scrub_duration":0.002594884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":174,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.422240+0000","last_change":"2024-09-11T08:47:38.519964+0000","last_active":"2024-09-11T08:47:45.422240+0000","last_peered":"2024-09-11T08:47:45.422240+0000","last_clean":"2024-09-11T08:47:45.422240+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:47:45.422240+0000","last_undegraded":"2024-09-11T08:47:45.422240+0000","last_fullsized":"2024-09-11T08:47:45.422240+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.850684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.850684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058755379999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":188,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.447437+0000","last_change":"2024-09-11T08:46:01.978488+0000","last_active":"2024-09-11T08:47:45.447437+0000","last_peered":"2024-09-11T08:47:45.447437+0000","last_clean":"2024-09-11T08:47:45.447437+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:47:45.447437+0000","last_undegraded":"2024-09-11T08:47:45.447437+0000","last_fullsized":"2024-09-11T08:47:45.447437+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:24.197149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.197149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001510267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":185,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.449317+0000","last_change":"2024-09-11T08:47:34.610411+0000","last_active":"2024-09-11T08:47:48.449317+0000","last_peered":"2024-09-11T08:47:48.449317+0000","last_clean":"2024-09-11T08:47:48.449317+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:47:48.449317+0000","last_undegraded":"2024-09-11T08:47:48.449317+0000","last_fullsized":"2024-09-11T08:47:48.449317+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:34.610331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:34.610331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:51.178932+0000","scrub_duration":0.0021231090000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":229,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.448116+0000","last_change":"2024-09-11T08:47:40.327468+0000","last_active":"2024-09-11T08:47:45.448116+0000","last_peered":"2024-09-11T08:47:45.448116+0000","last_clean":"2024-09-11T08:47:45.448116+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:47:45.448116+0000","last_undegraded":"2024-09-11T08:47:45.448116+0000","last_fullsized":"2024-09-11T08:47:45.448116+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:44:18.678748+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:44:18.678748+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016068020999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":181,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.459304+0000","last_change":"2024-09-11T08:44:12.518229+0000","last_active":"2024-09-11T08:47:48.459304+0000","last_peered":"2024-09-11T08:47:48.459304+0000","last_clean":"2024-09-11T08:47:48.459304+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:47:48.459304+0000","last_undegraded":"2024-09-11T08:47:48.459304+0000","last_fullsized":"2024-09-11T08:47:48.459304+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:12.518151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.518151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013208541000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":197,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.451860+0000","last_change":"2024-09-11T08:46:01.998163+0000","last_active":"2024-09-11T08:47:48.451860+0000","last_peered":"2024-09-11T08:47:48.451860+0000","last_clean":"2024-09-11T08:47:48.451860+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:47:48.451860+0000","last_undegraded":"2024-09-11T08:47:48.451860+0000","last_fullsized":"2024-09-11T08:47:48.451860+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:24.223237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.223237+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019917720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":174,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.445953+0000","last_change":"2024-09-11T08:45:28.194198+0000","last_active":"2024-09-11T08:47:45.445953+0000","last_peered":"2024-09-11T08:47:45.445953+0000","last_clean":"2024-09-11T08:47:45.445953+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:47:45.445953+0000","last_undegraded":"2024-09-11T08:47:45.445953+0000","last_fullsized":"2024-09-11T08:47:45.445953+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.194095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.194095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001559936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":195,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.450311+0000","last_change":"2024-09-11T08:45:16.162671+0000","last_active":"2024-09-11T08:47:48.450311+0000","last_peered":"2024-09-11T08:47:48.450311+0000","last_clean":"2024-09-11T08:47:48.450311+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:47:48.450311+0000","last_undegraded":"2024-09-11T08:47:48.450311+0000","last_fullsized":"2024-09-11T08:47:48.450311+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:16.162616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:16.162616+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001890833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":170,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.446947+0000","last_change":"2024-09-11T08:45:22.244097+0000","last_active":"2024-09-11T08:47:45.446947+0000","last_peered":"2024-09-11T08:47:45.446947+0000","last_clean":"2024-09-11T08:47:45.446947+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:47:45.446947+0000","last_undegraded":"2024-09-11T08:47:45.446947+0000","last_fullsized":"2024-09-11T08:47:45.446947+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.244038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.244038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015252620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":178,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.453662+0000","last_change":"2024-09-11T08:45:19.335163+0000","last_active":"2024-09-11T08:47:45.453662+0000","last_peered":"2024-09-11T08:47:45.453662+0000","last_clean":"2024-09-11T08:47:45.453662+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:47:45.453662+0000","last_undegraded":"2024-09-11T08:47:45.453662+0000","last_fullsized":"2024-09-11T08:47:45.453662+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:19.335065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:19.335065+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026052929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":185,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.457408+0000","last_change":"2024-09-11T08:44:33.758098+0000","last_active":"2024-09-11T08:47:48.457408+0000","last_peered":"2024-09-11T08:47:48.457408+0000","last_clean":"2024-09-11T08:47:48.457408+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:47:48.457408+0000","last_undegraded":"2024-09-11T08:47:48.457408+0000","last_fullsized":"2024-09-11T08:47:48.457408+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:44:33.758018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025150559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":191,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.452993+0000","last_change":"2024-09-11T08:47:35.655231+0000","last_active":"2024-09-11T08:47:48.452993+0000","last_peered":"2024-09-11T08:47:48.452993+0000","last_clean":"2024-09-11T08:47:48.452993+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:47:48.452993+0000","last_undegraded":"2024-09-11T08:47:48.452993+0000","last_fullsized":"2024-09-11T08:47:48.452993+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:44:10.704471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:10.704471+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020557259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":300,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:47.784769+0000","last_change":"2024-09-11T08:45:08.310590+0000","last_active":"2024-09-11T08:47:47.784769+0000","last_peered":"2024-09-11T08:47:47.784769+0000","last_clean":"2024-09-11T08:47:47.784769+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:47:47.784769+0000","last_undegraded":"2024-09-11T08:47:47.784769+0000","last_fullsized":"2024-09-11T08:47:47.784769+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:08.310527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.310527+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021434650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"191'10855","reported_seq":10906,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:51.447044+0000","last_change":"2024-09-11T08:45:15.261892+0000","last_active":"2024-09-11T08:47:51.447044+0000","last_peered":"2024-09-11T08:47:51.447044+0000","last_clean":"2024-09-11T08:47:51.447044+0000","last_became_active":"2024-09-11T08:45:15.261608+0000","last_became_peered":"2024-09-11T08:45:15.261608+0000","last_unstale":"2024-09-11T08:47:51.447044+0000","last_undegraded":"2024-09-11T08:47:51.447044+0000","last_fullsized":"2024-09-11T08:47:51.447044+0000","mapping_epoch":149,"log_start":"179'5900","ondisk_log_start":"179'5900","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4955,"log_dups_size":0,"ondisk_log_size":4955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":261685248,"num_objects":3993,"num_object_clones":0,"num_object_copies":11979,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10855,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"124'22","reported_seq":215,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.461273+0000","last_change":"2024-09-11T08:45:18.777528+0000","last_active":"2024-09-11T08:47:48.461273+0000","last_peered":"2024-09-11T08:47:48.461273+0000","last_clean":"2024-09-11T08:47:48.461273+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:47:48.461273+0000","last_undegraded":"2024-09-11T08:47:48.461273+0000","last_fullsized":"2024-09-11T08:47:48.461273+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:45:18.777466+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.777466+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045440580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":174,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.455532+0000","last_change":"2024-09-11T08:45:13.214007+0000","last_active":"2024-09-11T08:47:48.455532+0000","last_peered":"2024-09-11T08:47:48.455532+0000","last_clean":"2024-09-11T08:47:48.455532+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:47:48.455532+0000","last_undegraded":"2024-09-11T08:47:48.455532+0000","last_fullsized":"2024-09-11T08:47:48.455532+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:13.213922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.213922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016325280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"154'9","reported_seq":289,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:47.784278+0000","last_change":"2024-09-11T08:45:26.208887+0000","last_active":"2024-09-11T08:47:47.784278+0000","last_peered":"2024-09-11T08:47:47.784278+0000","last_clean":"2024-09-11T08:47:47.784278+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:47:47.784278+0000","last_undegraded":"2024-09-11T08:47:47.784278+0000","last_fullsized":"2024-09-11T08:47:47.784278+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'9","last_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.208818+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028933230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":171,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.422575+0000","last_change":"2024-09-11T08:47:39.523803+0000","last_active":"2024-09-11T08:47:45.422575+0000","last_peered":"2024-09-11T08:47:45.422575+0000","last_clean":"2024-09-11T08:47:45.422575+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:47:45.422575+0000","last_undegraded":"2024-09-11T08:47:45.422575+0000","last_fullsized":"2024-09-11T08:47:45.422575+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:24.813857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.813857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01118625,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"192'11014","reported_seq":11065,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-11T08:47:51.743726+0000","last_change":"2024-09-11T08:45:15.297710+0000","last_active":"2024-09-11T08:47:51.743726+0000","last_peered":"2024-09-11T08:47:51.743726+0000","last_clean":"2024-09-11T08:47:51.743726+0000","last_became_active":"2024-09-11T08:45:15.297378+0000","last_became_peered":"2024-09-11T08:45:15.297378+0000","last_unstale":"2024-09-11T08:47:51.743726+0000","last_undegraded":"2024-09-11T08:47:51.743726+0000","last_fullsized":"2024-09-11T08:47:51.743726+0000","mapping_epoch":149,"log_start":"175'5300","ondisk_log_start":"175'5300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5714,"log_dups_size":0,"ondisk_log_size":5714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267649024,"num_objects":4084,"num_object_clones":0,"num_object_copies":12252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11014,"num_write_kb":483136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":204,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.455357+0000","last_change":"2024-09-11T08:44:33.387066+0000","last_active":"2024-09-11T08:47:48.455357+0000","last_peered":"2024-09-11T08:47:48.455357+0000","last_clean":"2024-09-11T08:47:48.455357+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:47:48.455357+0000","last_undegraded":"2024-09-11T08:47:48.455357+0000","last_fullsized":"2024-09-11T08:47:48.455357+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-11T08:44:33.386972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:33.386972+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022646007999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":174,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.451777+0000","last_change":"2024-09-11T08:44:25.327692+0000","last_active":"2024-09-11T08:47:48.451777+0000","last_peered":"2024-09-11T08:47:48.451777+0000","last_clean":"2024-09-11T08:47:48.451777+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:47:48.451777+0000","last_undegraded":"2024-09-11T08:47:48.451777+0000","last_fullsized":"2024-09-11T08:47:48.451777+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:25.327553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:25.327553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016327754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":170,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.445583+0000","last_change":"2024-09-11T08:45:17.150827+0000","last_active":"2024-09-11T08:47:45.445583+0000","last_peered":"2024-09-11T08:47:45.445583+0000","last_clean":"2024-09-11T08:47:45.445583+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:47:45.445583+0000","last_undegraded":"2024-09-11T08:47:45.445583+0000","last_fullsized":"2024-09-11T08:47:45.445583+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.150767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.150767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001327707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":168,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.445842+0000","last_change":"2024-09-11T08:46:03.144149+0000","last_active":"2024-09-11T08:47:45.445842+0000","last_peered":"2024-09-11T08:47:45.445842+0000","last_clean":"2024-09-11T08:47:45.445842+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:47:45.445842+0000","last_undegraded":"2024-09-11T08:47:45.445842+0000","last_fullsized":"2024-09-11T08:47:45.445842+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:46:03.144096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:46:03.144096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001497151,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"191'11750","reported_seq":11834,"reported_epoch":191,"state":"active+clean+scrubbing+deep","last_fresh":"2024-09-11T08:47:51.202431+0000","last_change":"2024-09-11T08:47:46.326890+0000","last_active":"2024-09-11T08:47:51.202431+0000","last_peered":"2024-09-11T08:47:51.202431+0000","last_clean":"2024-09-11T08:47:51.202431+0000","last_became_active":"2024-09-11T08:45:15.627210+0000","last_became_peered":"2024-09-11T08:45:15.627210+0000","last_unstale":"2024-09-11T08:47:51.202431+0000","last_undegraded":"2024-09-11T08:47:51.202431+0000","last_fullsized":"2024-09-11T08:47:51.202431+0000","mapping_epoch":149,"log_start":"180'6300","ondisk_log_start":"180'6300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":860,"log_size":5450,"log_dups_size":0,"ondisk_log_size":5450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258670616,"num_objects":3948,"num_object_clones":0,"num_object_copies":11844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3948,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10890,"num_write_kb":474753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"86'21","reported_seq":190,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.445815+0000","last_change":"2024-09-11T08:43:30.997804+0000","last_active":"2024-09-11T08:47:45.445815+0000","last_peered":"2024-09-11T08:47:45.445815+0000","last_clean":"2024-09-11T08:47:45.445815+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:47:45.445815+0000","last_undegraded":"2024-09-11T08:47:45.445815+0000","last_fullsized":"2024-09-11T08:47:45.445815+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'21","last_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:43:30.997710+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025287299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":185,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.453230+0000","last_change":"2024-09-11T08:45:30.475446+0000","last_active":"2024-09-11T08:47:48.453230+0000","last_peered":"2024-09-11T08:47:48.453230+0000","last_clean":"2024-09-11T08:47:48.453230+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:47:48.453230+0000","last_undegraded":"2024-09-11T08:47:48.453230+0000","last_fullsized":"2024-09-11T08:47:48.453230+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:30.475365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:30.475365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010690405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":281,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.455210+0000","last_change":"2024-09-11T08:44:11.198993+0000","last_active":"2024-09-11T08:47:48.455210+0000","last_peered":"2024-09-11T08:47:48.455210+0000","last_clean":"2024-09-11T08:47:48.455210+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:47:48.455210+0000","last_undegraded":"2024-09-11T08:47:48.455210+0000","last_fullsized":"2024-09-11T08:47:48.455210+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:44:11.198936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:11.198936+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019660649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":197,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.453581+0000","last_change":"2024-09-11T08:46:01.983654+0000","last_active":"2024-09-11T08:47:45.453581+0000","last_peered":"2024-09-11T08:47:45.453581+0000","last_clean":"2024-09-11T08:47:45.453581+0000","last_became_active":"2024-09-11T08:46:01.983335+0000","last_became_peered":"2024-09-11T08:46:01.983335+0000","last_unstale":"2024-09-11T08:47:45.453581+0000","last_undegraded":"2024-09-11T08:47:45.453581+0000","last_fullsized":"2024-09-11T08:47:45.453581+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.303194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.303194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001492431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"192'10903","reported_seq":10954,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-11T08:47:51.770737+0000","last_change":"2024-09-11T08:45:15.626487+0000","last_active":"2024-09-11T08:47:51.770737+0000","last_peered":"2024-09-11T08:47:51.770737+0000","last_clean":"2024-09-11T08:47:51.770737+0000","last_became_active":"2024-09-11T08:45:15.626210+0000","last_became_peered":"2024-09-11T08:45:15.626210+0000","last_unstale":"2024-09-11T08:47:51.770737+0000","last_undegraded":"2024-09-11T08:47:51.770737+0000","last_fullsized":"2024-09-11T08:47:51.770737+0000","mapping_epoch":149,"log_start":"175'5200","ondisk_log_start":"175'5200","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5703,"log_dups_size":0,"ondisk_log_size":5703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262733824,"num_objects":4009,"num_object_clones":0,"num_object_copies":12027,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10903,"num_write_kb":477184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"100'15","reported_seq":203,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.458583+0000","last_change":"2024-09-11T08:45:12.769299+0000","last_active":"2024-09-11T08:47:48.458583+0000","last_peered":"2024-09-11T08:47:48.458583+0000","last_clean":"2024-09-11T08:47:48.458583+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:47:48.458583+0000","last_undegraded":"2024-09-11T08:47:48.458583+0000","last_fullsized":"2024-09-11T08:47:48.458583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'15","last_scrub_stamp":"2024-09-11T08:45:12.769239+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:41:56.250393+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.769239+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026298300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":180,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.423115+0000","last_change":"2024-09-11T08:47:37.534624+0000","last_active":"2024-09-11T08:47:45.423115+0000","last_peered":"2024-09-11T08:47:45.423115+0000","last_clean":"2024-09-11T08:47:45.423115+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:47:45.423115+0000","last_undegraded":"2024-09-11T08:47:45.423115+0000","last_fullsized":"2024-09-11T08:47:45.423115+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:19.905459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:19.905459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015766237999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":289,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.448939+0000","last_change":"2024-09-11T08:44:34.630457+0000","last_active":"2024-09-11T08:47:48.448939+0000","last_peered":"2024-09-11T08:47:48.448939+0000","last_clean":"2024-09-11T08:47:48.448939+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:47:48.448939+0000","last_undegraded":"2024-09-11T08:47:48.448939+0000","last_fullsized":"2024-09-11T08:47:48.448939+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:44:34.630191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:34.630191+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089275420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":172,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.447505+0000","last_change":"2024-09-11T08:47:42.685960+0000","last_active":"2024-09-11T08:47:48.447505+0000","last_peered":"2024-09-11T08:47:48.447505+0000","last_clean":"2024-09-11T08:47:48.447505+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:47:48.447505+0000","last_undegraded":"2024-09-11T08:47:48.447505+0000","last_fullsized":"2024-09-11T08:47:48.447505+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:16.681036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:44:16.681036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047481509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"192'10936","reported_seq":10987,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-11T08:47:51.695941+0000","last_change":"2024-09-11T08:45:15.627960+0000","last_active":"2024-09-11T08:47:51.695941+0000","last_peered":"2024-09-11T08:47:51.695941+0000","last_clean":"2024-09-11T08:47:51.695941+0000","last_became_active":"2024-09-11T08:45:15.627639+0000","last_became_peered":"2024-09-11T08:45:15.627639+0000","last_unstale":"2024-09-11T08:47:51.695941+0000","last_undegraded":"2024-09-11T08:47:51.695941+0000","last_fullsized":"2024-09-11T08:47:51.695941+0000","mapping_epoch":149,"log_start":"179'6000","ondisk_log_start":"179'6000","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4936,"log_dups_size":0,"ondisk_log_size":4936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":261619712,"num_objects":3992,"num_object_clones":0,"num_object_copies":11976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10936,"num_write_kb":477696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":209,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.449120+0000","last_change":"2024-09-11T08:45:11.576184+0000","last_active":"2024-09-11T08:47:48.449120+0000","last_peered":"2024-09-11T08:47:48.449120+0000","last_clean":"2024-09-11T08:47:48.449120+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:47:48.449120+0000","last_undegraded":"2024-09-11T08:47:48.449120+0000","last_fullsized":"2024-09-11T08:47:48.449120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:45:11.576085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.576085+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002366754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":185,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.459931+0000","last_change":"2024-09-11T08:44:30.802099+0000","last_active":"2024-09-11T08:47:48.459931+0000","last_peered":"2024-09-11T08:47:48.459931+0000","last_clean":"2024-09-11T08:47:48.459931+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:47:48.459931+0000","last_undegraded":"2024-09-11T08:47:48.459931+0000","last_fullsized":"2024-09-11T08:47:48.459931+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:30.800097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:30.800097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071432789999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":179,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.421555+0000","last_change":"2024-09-11T08:45:07.264895+0000","last_active":"2024-09-11T08:47:45.421555+0000","last_peered":"2024-09-11T08:47:45.421555+0000","last_clean":"2024-09-11T08:47:45.421555+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:47:45.421555+0000","last_undegraded":"2024-09-11T08:47:45.421555+0000","last_fullsized":"2024-09-11T08:47:45.421555+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.264822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.264822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014425250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":199,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.457312+0000","last_change":"2024-09-11T08:46:01.985315+0000","last_active":"2024-09-11T08:47:48.457312+0000","last_peered":"2024-09-11T08:47:48.457312+0000","last_clean":"2024-09-11T08:47:48.457312+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:47:48.457312+0000","last_undegraded":"2024-09-11T08:47:48.457312+0000","last_fullsized":"2024-09-11T08:47:48.457312+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.805220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.805220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012010009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"191'10967","reported_seq":11017,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:51.209565+0000","last_change":"2024-09-11T08:45:15.298396+0000","last_active":"2024-09-11T08:47:51.209565+0000","last_peered":"2024-09-11T08:47:51.209565+0000","last_clean":"2024-09-11T08:47:51.209565+0000","last_became_active":"2024-09-11T08:45:15.298053+0000","last_became_peered":"2024-09-11T08:45:15.298053+0000","last_unstale":"2024-09-11T08:47:51.209565+0000","last_undegraded":"2024-09-11T08:47:51.209565+0000","last_fullsized":"2024-09-11T08:47:51.209565+0000","mapping_epoch":149,"log_start":"177'5600","ondisk_log_start":"177'5600","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5367,"log_dups_size":0,"ondisk_log_size":5367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266797056,"num_objects":4071,"num_object_clones":0,"num_object_copies":12213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10967,"num_write_kb":481216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":361,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.459779+0000","last_change":"2024-09-11T08:45:26.666261+0000","last_active":"2024-09-11T08:47:48.459779+0000","last_peered":"2024-09-11T08:47:48.459779+0000","last_clean":"2024-09-11T08:47:48.459779+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:47:48.459779+0000","last_undegraded":"2024-09-11T08:47:48.459779+0000","last_fullsized":"2024-09-11T08:47:48.459779+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:45:26.666199+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.666199+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020004279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":174,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.446318+0000","last_change":"2024-09-11T08:45:15.130637+0000","last_active":"2024-09-11T08:47:45.446318+0000","last_peered":"2024-09-11T08:47:45.446318+0000","last_clean":"2024-09-11T08:47:45.446318+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:47:45.446318+0000","last_undegraded":"2024-09-11T08:47:45.446318+0000","last_fullsized":"2024-09-11T08:47:45.446318+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.130572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.130572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019779749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":215,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.422696+0000","last_change":"2024-09-11T08:47:02.718120+0000","last_active":"2024-09-11T08:47:45.422696+0000","last_peered":"2024-09-11T08:47:45.422696+0000","last_clean":"2024-09-11T08:47:45.422696+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:47:45.422696+0000","last_undegraded":"2024-09-11T08:47:45.422696+0000","last_fullsized":"2024-09-11T08:47:45.422696+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:43:38.669837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:38.669837+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001723715,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":174,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.449629+0000","last_change":"2024-09-11T08:47:38.663094+0000","last_active":"2024-09-11T08:47:48.449629+0000","last_peered":"2024-09-11T08:47:48.449629+0000","last_clean":"2024-09-11T08:47:48.449629+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:47:48.449629+0000","last_undegraded":"2024-09-11T08:47:48.449629+0000","last_fullsized":"2024-09-11T08:47:48.449629+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:14.679598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:14.679598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011950855999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":168,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.450256+0000","last_change":"2024-09-11T08:45:22.169436+0000","last_active":"2024-09-11T08:47:48.450256+0000","last_peered":"2024-09-11T08:47:48.450256+0000","last_clean":"2024-09-11T08:47:48.450256+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:47:48.450256+0000","last_undegraded":"2024-09-11T08:47:48.450256+0000","last_fullsized":"2024-09-11T08:47:48.450256+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.169367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.169367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005349693,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"192'10980","reported_seq":11031,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-11T08:47:51.706695+0000","last_change":"2024-09-11T08:45:15.261656+0000","last_active":"2024-09-11T08:47:51.706695+0000","last_peered":"2024-09-11T08:47:51.706695+0000","last_clean":"2024-09-11T08:47:51.706695+0000","last_became_active":"2024-09-11T08:45:15.261370+0000","last_became_peered":"2024-09-11T08:45:15.261370+0000","last_unstale":"2024-09-11T08:47:51.706695+0000","last_undegraded":"2024-09-11T08:47:51.706695+0000","last_fullsized":"2024-09-11T08:47:51.706695+0000","mapping_epoch":149,"log_start":"180'6100","ondisk_log_start":"180'6100","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4880,"log_dups_size":0,"ondisk_log_size":4880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262930432,"num_objects":4012,"num_object_clones":0,"num_object_copies":12036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10980,"num_write_kb":479744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":220,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.461342+0000","last_change":"2024-09-11T08:45:27.666590+0000","last_active":"2024-09-11T08:47:48.461342+0000","last_peered":"2024-09-11T08:47:48.461342+0000","last_clean":"2024-09-11T08:47:48.461342+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:47:48.461342+0000","last_undegraded":"2024-09-11T08:47:48.461342+0000","last_fullsized":"2024-09-11T08:47:48.461342+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:45:27.666502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.666502+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023582730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":205,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.458164+0000","last_change":"2024-09-11T08:46:01.982031+0000","last_active":"2024-09-11T08:47:48.458164+0000","last_peered":"2024-09-11T08:47:48.458164+0000","last_clean":"2024-09-11T08:47:48.458164+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:47:48.458164+0000","last_undegraded":"2024-09-11T08:47:48.458164+0000","last_fullsized":"2024-09-11T08:47:48.458164+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.775804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.775804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011899289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":203,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.448456+0000","last_change":"2024-09-11T08:47:36.417328+0000","last_active":"2024-09-11T08:47:45.448456+0000","last_peered":"2024-09-11T08:47:45.448456+0000","last_clean":"2024-09-11T08:47:45.448456+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:47:45.448456+0000","last_undegraded":"2024-09-11T08:47:45.448456+0000","last_fullsized":"2024-09-11T08:47:45.448456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:45:32.632242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:32.632242+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049221250000000003,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":199,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.448993+0000","last_change":"2024-09-11T08:47:44.344421+0000","last_active":"2024-09-11T08:47:45.448993+0000","last_peered":"2024-09-11T08:47:45.448993+0000","last_clean":"2024-09-11T08:47:45.448993+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:47:45.448993+0000","last_undegraded":"2024-09-11T08:47:45.448993+0000","last_fullsized":"2024-09-11T08:47:45.448993+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.344206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:48.790110+0000","scrub_duration":0.0085971020000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":197,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.454030+0000","last_change":"2024-09-11T08:46:01.984350+0000","last_active":"2024-09-11T08:47:45.454030+0000","last_peered":"2024-09-11T08:47:45.454030+0000","last_clean":"2024-09-11T08:47:45.454030+0000","last_became_active":"2024-09-11T08:46:01.983095+0000","last_became_peered":"2024-09-11T08:46:01.983095+0000","last_unstale":"2024-09-11T08:47:45.454030+0000","last_undegraded":"2024-09-11T08:47:45.454030+0000","last_fullsized":"2024-09-11T08:47:45.454030+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:12.278185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.278185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015184790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"191'11251","reported_seq":11302,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:51.448079+0000","last_change":"2024-09-11T08:45:15.627633+0000","last_active":"2024-09-11T08:47:51.448079+0000","last_peered":"2024-09-11T08:47:51.448079+0000","last_clean":"2024-09-11T08:47:51.448079+0000","last_became_active":"2024-09-11T08:45:15.627360+0000","last_became_peered":"2024-09-11T08:45:15.627360+0000","last_unstale":"2024-09-11T08:47:51.448079+0000","last_undegraded":"2024-09-11T08:47:51.448079+0000","last_fullsized":"2024-09-11T08:47:51.448079+0000","mapping_epoch":149,"log_start":"180'6300","ondisk_log_start":"180'6300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4951,"log_dups_size":0,"ondisk_log_size":4951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267321344,"num_objects":4079,"num_object_clones":0,"num_object_copies":12237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11251,"num_write_kb":490560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":191,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.453076+0000","last_change":"2024-09-11T08:44:15.699227+0000","last_active":"2024-09-11T08:47:48.453076+0000","last_peered":"2024-09-11T08:47:48.453076+0000","last_clean":"2024-09-11T08:47:48.453076+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:47:48.453076+0000","last_undegraded":"2024-09-11T08:47:48.453076+0000","last_fullsized":"2024-09-11T08:47:48.453076+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:44:15.699067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:15.699067+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014313306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":185,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.448406+0000","last_change":"2024-09-11T08:45:20.455183+0000","last_active":"2024-09-11T08:47:48.448406+0000","last_peered":"2024-09-11T08:47:48.448406+0000","last_clean":"2024-09-11T08:47:48.448406+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:47:48.448406+0000","last_undegraded":"2024-09-11T08:47:48.448406+0000","last_fullsized":"2024-09-11T08:47:48.448406+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:20.455081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.455081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016372450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":746,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:49.839273+0000","last_change":"2024-09-11T08:45:21.370742+0000","last_active":"2024-09-11T08:47:49.839273+0000","last_peered":"2024-09-11T08:47:49.839273+0000","last_clean":"2024-09-11T08:47:49.839273+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:47:49.839273+0000","last_undegraded":"2024-09-11T08:47:49.839273+0000","last_fullsized":"2024-09-11T08:47:49.839273+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:45:21.370673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.370673+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017307240000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":545,"num_read_kb":540,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":176,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.423172+0000","last_change":"2024-09-11T08:47:20.796422+0000","last_active":"2024-09-11T08:47:45.423172+0000","last_peered":"2024-09-11T08:47:45.423172+0000","last_clean":"2024-09-11T08:47:45.423172+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:47:45.423172+0000","last_undegraded":"2024-09-11T08:47:45.423172+0000","last_fullsized":"2024-09-11T08:47:45.423172+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:16.924404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:16.924404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024072690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":179,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.458855+0000","last_change":"2024-09-11T08:45:17.819757+0000","last_active":"2024-09-11T08:47:48.458855+0000","last_peered":"2024-09-11T08:47:48.458855+0000","last_clean":"2024-09-11T08:47:48.458855+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:47:48.458855+0000","last_undegraded":"2024-09-11T08:47:48.458855+0000","last_fullsized":"2024-09-11T08:47:48.458855+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.819707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.819707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012611440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"191'11105","reported_seq":11155,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:51.095624+0000","last_change":"2024-09-11T08:45:15.262738+0000","last_active":"2024-09-11T08:47:51.095624+0000","last_peered":"2024-09-11T08:47:51.095624+0000","last_clean":"2024-09-11T08:47:51.095624+0000","last_became_active":"2024-09-11T08:45:15.262440+0000","last_became_peered":"2024-09-11T08:45:15.262440+0000","last_unstale":"2024-09-11T08:47:51.095624+0000","last_undegraded":"2024-09-11T08:47:51.095624+0000","last_fullsized":"2024-09-11T08:47:51.095624+0000","mapping_epoch":149,"log_start":"173'4800","ondisk_log_start":"173'4800","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":6305,"log_dups_size":0,"ondisk_log_size":6305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270336000,"num_objects":4125,"num_object_clones":0,"num_object_copies":12375,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11105,"num_write_kb":487360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":59,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.585275+0000","last_change":"2024-09-11T08:47:45.585212+0000","last_active":"2024-09-11T08:47:45.585275+0000","last_peered":"2024-09-11T08:47:45.585275+0000","last_clean":"2024-09-11T08:47:45.585275+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:47:45.585275+0000","last_undegraded":"2024-09-11T08:47:45.585275+0000","last_fullsized":"2024-09-11T08:47:45.585275+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:47:45.585135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:45.585135+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:47.607612+0000","scrub_duration":0.0080208199999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":183,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.447765+0000","last_change":"2024-09-11T08:45:18.127946+0000","last_active":"2024-09-11T08:47:48.447765+0000","last_peered":"2024-09-11T08:47:48.447765+0000","last_clean":"2024-09-11T08:47:48.447765+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:47:48.447765+0000","last_undegraded":"2024-09-11T08:47:48.447765+0000","last_fullsized":"2024-09-11T08:47:48.447765+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:18.127857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.127857+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018334099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":227,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.453517+0000","last_change":"2024-09-11T08:45:29.524681+0000","last_active":"2024-09-11T08:47:45.453517+0000","last_peered":"2024-09-11T08:47:45.453517+0000","last_clean":"2024-09-11T08:47:45.453517+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:47:45.453517+0000","last_undegraded":"2024-09-11T08:47:45.453517+0000","last_fullsized":"2024-09-11T08:47:45.453517+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'79","last_scrub_stamp":"2024-09-11T08:45:29.524608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:45:29.524608+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002308867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":189,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.453852+0000","last_change":"2024-09-11T08:47:35.433723+0000","last_active":"2024-09-11T08:47:45.453852+0000","last_peered":"2024-09-11T08:47:45.453852+0000","last_clean":"2024-09-11T08:47:45.453852+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:47:45.453852+0000","last_undegraded":"2024-09-11T08:47:45.453852+0000","last_fullsized":"2024-09-11T08:47:45.453852+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:35.433651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:35.433651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:37.450817+0000","scrub_duration":0.013907892999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":174,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.423028+0000","last_change":"2024-09-11T08:47:12.832716+0000","last_active":"2024-09-11T08:47:45.423028+0000","last_peered":"2024-09-11T08:47:45.423028+0000","last_clean":"2024-09-11T08:47:45.423028+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:47:45.423028+0000","last_undegraded":"2024-09-11T08:47:45.423028+0000","last_fullsized":"2024-09-11T08:47:45.423028+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.570246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.570246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041809610000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"191'11126","reported_seq":11176,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:51.091716+0000","last_change":"2024-09-11T08:45:15.263087+0000","last_active":"2024-09-11T08:47:51.091716+0000","last_peered":"2024-09-11T08:47:51.091716+0000","last_clean":"2024-09-11T08:47:51.091716+0000","last_became_active":"2024-09-11T08:45:15.262698+0000","last_became_peered":"2024-09-11T08:45:15.262698+0000","last_unstale":"2024-09-11T08:47:51.091716+0000","last_undegraded":"2024-09-11T08:47:51.091716+0000","last_fullsized":"2024-09-11T08:47:51.091716+0000","mapping_epoch":149,"log_start":"180'6300","ondisk_log_start":"180'6300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4826,"log_dups_size":0,"ondisk_log_size":4826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":265945088,"num_objects":4058,"num_object_clones":0,"num_object_copies":12174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11126,"num_write_kb":485888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":250,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.447486+0000","last_change":"2024-09-11T08:47:32.052566+0000","last_active":"2024-09-11T08:47:45.447486+0000","last_peered":"2024-09-11T08:47:45.447486+0000","last_clean":"2024-09-11T08:47:45.447486+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:47:45.447486+0000","last_undegraded":"2024-09-11T08:47:45.447486+0000","last_fullsized":"2024-09-11T08:47:45.447486+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:21.002140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:21.002140+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025090400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":164,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.445876+0000","last_change":"2024-09-11T08:44:22.010602+0000","last_active":"2024-09-11T08:47:45.445876+0000","last_peered":"2024-09-11T08:47:45.445876+0000","last_clean":"2024-09-11T08:47:45.445876+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:47:45.445876+0000","last_undegraded":"2024-09-11T08:47:45.445876+0000","last_fullsized":"2024-09-11T08:47:45.445876+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.010528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.010528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047628439999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":183,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.423206+0000","last_change":"2024-09-11T08:47:41.425451+0000","last_active":"2024-09-11T08:47:45.423206+0000","last_peered":"2024-09-11T08:47:45.423206+0000","last_clean":"2024-09-11T08:47:45.423206+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:47:45.423206+0000","last_undegraded":"2024-09-11T08:47:45.423206+0000","last_fullsized":"2024-09-11T08:47:45.423206+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.726573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00117395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":188,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.451691+0000","last_change":"2024-09-11T08:46:01.988315+0000","last_active":"2024-09-11T08:47:48.451691+0000","last_peered":"2024-09-11T08:47:48.451691+0000","last_clean":"2024-09-11T08:47:48.451691+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:47:48.451691+0000","last_undegraded":"2024-09-11T08:47:48.451691+0000","last_fullsized":"2024-09-11T08:47:48.451691+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.167071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.167071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001796483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"192'10862","reported_seq":10910,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-11T08:47:51.453401+0000","last_change":"2024-09-11T08:45:15.626015+0000","last_active":"2024-09-11T08:47:51.453401+0000","last_peered":"2024-09-11T08:47:51.453401+0000","last_clean":"2024-09-11T08:47:51.453401+0000","last_became_active":"2024-09-11T08:45:15.625726+0000","last_became_peered":"2024-09-11T08:45:15.625726+0000","last_unstale":"2024-09-11T08:47:51.453401+0000","last_undegraded":"2024-09-11T08:47:51.453401+0000","last_fullsized":"2024-09-11T08:47:51.453401+0000","mapping_epoch":149,"log_start":"179'5900","ondisk_log_start":"179'5900","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4962,"log_dups_size":0,"ondisk_log_size":4962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":256507904,"num_objects":3914,"num_object_clones":0,"num_object_copies":11742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10862,"num_write_kb":472832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":200,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.446746+0000","last_change":"2024-09-11T08:44:28.940078+0000","last_active":"2024-09-11T08:47:45.446746+0000","last_peered":"2024-09-11T08:47:45.446746+0000","last_clean":"2024-09-11T08:47:45.446746+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:47:45.446746+0000","last_undegraded":"2024-09-11T08:47:45.446746+0000","last_fullsized":"2024-09-11T08:47:45.446746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:44:28.940023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.940023+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012856796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":193,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.421802+0000","last_change":"2024-09-11T08:45:25.273200+0000","last_active":"2024-09-11T08:47:45.421802+0000","last_peered":"2024-09-11T08:47:45.421802+0000","last_clean":"2024-09-11T08:47:45.421802+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:47:45.421802+0000","last_undegraded":"2024-09-11T08:47:45.421802+0000","last_fullsized":"2024-09-11T08:47:45.421802+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.273134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014307370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":181,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.458934+0000","last_change":"2024-09-11T08:44:24.816974+0000","last_active":"2024-09-11T08:47:48.458934+0000","last_peered":"2024-09-11T08:47:48.458934+0000","last_clean":"2024-09-11T08:47:48.458934+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:47:48.458934+0000","last_undegraded":"2024-09-11T08:47:48.458934+0000","last_fullsized":"2024-09-11T08:47:48.458934+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:24.816890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.816890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086069140000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":193,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.448126+0000","last_change":"2024-09-11T08:47:38.366007+0000","last_active":"2024-09-11T08:47:45.448126+0000","last_peered":"2024-09-11T08:47:45.448126+0000","last_clean":"2024-09-11T08:47:45.448126+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:47:45.448126+0000","last_undegraded":"2024-09-11T08:47:45.448126+0000","last_fullsized":"2024-09-11T08:47:45.448126+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.756166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.756166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059059109999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"192'11029","reported_seq":11080,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-11T08:47:51.687201+0000","last_change":"2024-09-11T08:45:15.260658+0000","last_active":"2024-09-11T08:47:51.687201+0000","last_peered":"2024-09-11T08:47:51.687201+0000","last_clean":"2024-09-11T08:47:51.687201+0000","last_became_active":"2024-09-11T08:45:15.260383+0000","last_became_peered":"2024-09-11T08:45:15.260383+0000","last_unstale":"2024-09-11T08:47:51.687201+0000","last_undegraded":"2024-09-11T08:47:51.687201+0000","last_fullsized":"2024-09-11T08:47:51.687201+0000","mapping_epoch":149,"log_start":"180'6100","ondisk_log_start":"180'6100","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4929,"log_dups_size":0,"ondisk_log_size":4929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260898816,"num_objects":3981,"num_object_clones":0,"num_object_copies":11943,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11029,"num_write_kb":480320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":224,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.447022+0000","last_change":"2024-09-11T08:47:35.011515+0000","last_active":"2024-09-11T08:47:45.447022+0000","last_peered":"2024-09-11T08:47:45.447022+0000","last_clean":"2024-09-11T08:47:45.447022+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:47:45.447022+0000","last_undegraded":"2024-09-11T08:47:45.447022+0000","last_fullsized":"2024-09-11T08:47:45.447022+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:43:24.983740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:24.983740+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069750209999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":174,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.450364+0000","last_change":"2024-09-11T08:45:32.219463+0000","last_active":"2024-09-11T08:47:48.450364+0000","last_peered":"2024-09-11T08:47:48.450364+0000","last_clean":"2024-09-11T08:47:48.450364+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:47:48.450364+0000","last_undegraded":"2024-09-11T08:47:48.450364+0000","last_fullsized":"2024-09-11T08:47:48.450364+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:32.219374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:32.219374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069622119999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":317,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:47.811758+0000","last_change":"2024-09-11T08:47:47.542558+0000","last_active":"2024-09-11T08:47:47.811758+0000","last_peered":"2024-09-11T08:47:47.811758+0000","last_clean":"2024-09-11T08:47:47.811758+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:47:47.811758+0000","last_undegraded":"2024-09-11T08:47:47.811758+0000","last_fullsized":"2024-09-11T08:47:47.811758+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:47:47.542483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:47.542483+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:17.503546+0000","scrub_duration":0.15193762999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":168,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.447339+0000","last_change":"2024-09-11T08:45:31.196687+0000","last_active":"2024-09-11T08:47:45.447339+0000","last_peered":"2024-09-11T08:47:45.447339+0000","last_clean":"2024-09-11T08:47:45.447339+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:47:45.447339+0000","last_undegraded":"2024-09-11T08:47:45.447339+0000","last_fullsized":"2024-09-11T08:47:45.447339+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.196599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.196599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025248190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"191'11104","reported_seq":19035,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:51.445479+0000","last_change":"2024-09-11T08:47:45.439155+0000","last_active":"2024-09-11T08:47:51.445479+0000","last_peered":"2024-09-11T08:47:51.445479+0000","last_clean":"2024-09-11T08:47:51.445479+0000","last_became_active":"2024-09-11T08:47:45.422820+0000","last_became_peered":"2024-09-11T08:47:45.422820+0000","last_unstale":"2024-09-11T08:47:51.445479+0000","last_undegraded":"2024-09-11T08:47:51.445479+0000","last_fullsized":"2024-09-11T08:47:51.445479+0000","mapping_epoch":189,"log_start":"180'6100","ondisk_log_start":"180'6100","created":149,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5004,"log_dups_size":0,"ondisk_log_size":5004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":269484032,"num_objects":4112,"num_object_clones":0,"num_object_copies":12336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11104,"num_write_kb":486912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3929,"num_bytes_recovered":257490944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":189,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.449030+0000","last_change":"2024-09-11T08:44:31.662275+0000","last_active":"2024-09-11T08:47:48.449030+0000","last_peered":"2024-09-11T08:47:48.449030+0000","last_clean":"2024-09-11T08:47:48.449030+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:47:48.449030+0000","last_undegraded":"2024-09-11T08:47:48.449030+0000","last_fullsized":"2024-09-11T08:47:48.449030+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:31.662190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.662190+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040714619999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":187,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.423091+0000","last_change":"2024-09-11T08:47:43.360558+0000","last_active":"2024-09-11T08:47:45.423091+0000","last_peered":"2024-09-11T08:47:45.423091+0000","last_clean":"2024-09-11T08:47:45.423091+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:47:45.423091+0000","last_undegraded":"2024-09-11T08:47:45.423091+0000","last_fullsized":"2024-09-11T08:47:45.423091+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:10.607288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:45:10.607288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016158909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":183,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.447054+0000","last_change":"2024-09-11T08:47:30.449632+0000","last_active":"2024-09-11T08:47:45.447054+0000","last_peered":"2024-09-11T08:47:45.447054+0000","last_clean":"2024-09-11T08:47:45.447054+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:47:45.447054+0000","last_undegraded":"2024-09-11T08:47:45.447054+0000","last_fullsized":"2024-09-11T08:47:45.447054+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:12.645956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.645956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014834379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":179,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.448338+0000","last_change":"2024-09-11T08:47:47.699529+0000","last_active":"2024-09-11T08:47:48.448338+0000","last_peered":"2024-09-11T08:47:48.448338+0000","last_clean":"2024-09-11T08:47:48.448338+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:47:48.448338+0000","last_undegraded":"2024-09-11T08:47:48.448338+0000","last_fullsized":"2024-09-11T08:47:48.448338+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:47.699453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:47:47.699453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:55.018390+0000","scrub_duration":0.016178818000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"192'10781","reported_seq":10832,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-11T08:47:51.765484+0000","last_change":"2024-09-11T08:45:15.297737+0000","last_active":"2024-09-11T08:47:51.765484+0000","last_peered":"2024-09-11T08:47:51.765484+0000","last_clean":"2024-09-11T08:47:51.765484+0000","last_became_active":"2024-09-11T08:45:15.297420+0000","last_became_peered":"2024-09-11T08:45:15.297420+0000","last_unstale":"2024-09-11T08:47:51.765484+0000","last_undegraded":"2024-09-11T08:47:51.765484+0000","last_fullsized":"2024-09-11T08:47:51.765484+0000","mapping_epoch":149,"log_start":"175'5100","ondisk_log_start":"175'5100","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5681,"log_dups_size":0,"ondisk_log_size":5681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":253689856,"num_objects":3871,"num_object_clones":0,"num_object_copies":11613,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10781,"num_write_kb":468864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":63,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:46.582530+0000","last_change":"2024-09-11T08:47:46.582476+0000","last_active":"2024-09-11T08:47:46.582530+0000","last_peered":"2024-09-11T08:47:46.582530+0000","last_clean":"2024-09-11T08:47:46.582530+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:47:46.582530+0000","last_undegraded":"2024-09-11T08:47:46.582530+0000","last_fullsized":"2024-09-11T08:47:46.582530+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:47:46.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:46.582408+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:56.458790+0000","scrub_duration":0.006189015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":193,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.454316+0000","last_change":"2024-09-11T08:47:44.411099+0000","last_active":"2024-09-11T08:47:45.454316+0000","last_peered":"2024-09-11T08:47:45.454316+0000","last_clean":"2024-09-11T08:47:45.454316+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:47:45.454316+0000","last_undegraded":"2024-09-11T08:47:45.454316+0000","last_fullsized":"2024-09-11T08:47:45.454316+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.411022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.411022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:47.936745+0000","scrub_duration":0.0047762170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":199,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.447104+0000","last_change":"2024-09-11T08:47:43.361569+0000","last_active":"2024-09-11T08:47:45.447104+0000","last_peered":"2024-09-11T08:47:45.447104+0000","last_clean":"2024-09-11T08:47:45.447104+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:47:45.447104+0000","last_undegraded":"2024-09-11T08:47:45.447104+0000","last_fullsized":"2024-09-11T08:47:45.447104+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:43.361491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:43.361491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:11.065199+0000","scrub_duration":0.0055115299999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":170,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.453305+0000","last_change":"2024-09-11T08:47:41.382331+0000","last_active":"2024-09-11T08:47:45.453305+0000","last_peered":"2024-09-11T08:47:45.453305+0000","last_clean":"2024-09-11T08:47:45.453305+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:47:45.453305+0000","last_undegraded":"2024-09-11T08:47:45.453305+0000","last_fullsized":"2024-09-11T08:47:45.453305+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:23.367786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:23.367786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030117270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"191'10775","reported_seq":18525,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:51.117889+0000","last_change":"2024-09-11T08:47:33.390084+0000","last_active":"2024-09-11T08:47:51.117889+0000","last_peered":"2024-09-11T08:47:51.117889+0000","last_clean":"2024-09-11T08:47:51.117889+0000","last_became_active":"2024-09-11T08:47:33.341629+0000","last_became_peered":"2024-09-11T08:47:33.341629+0000","last_unstale":"2024-09-11T08:47:51.117889+0000","last_undegraded":"2024-09-11T08:47:51.117889+0000","last_fullsized":"2024-09-11T08:47:51.117889+0000","mapping_epoch":187,"log_start":"172'4500","ondisk_log_start":"172'4500","created":149,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":6275,"log_dups_size":0,"ondisk_log_size":6275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":261029888,"num_objects":3983,"num_object_clones":0,"num_object_copies":11949,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10775,"num_write_kb":472256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3840,"num_bytes_recovered":251658240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":195,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.445694+0000","last_change":"2024-09-11T08:45:08.140910+0000","last_active":"2024-09-11T08:47:45.445694+0000","last_peered":"2024-09-11T08:47:45.445694+0000","last_clean":"2024-09-11T08:47:45.445694+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:47:45.445694+0000","last_undegraded":"2024-09-11T08:47:45.445694+0000","last_fullsized":"2024-09-11T08:47:45.445694+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:08.140839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.140839+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003679547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":183,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.454870+0000","last_change":"2024-09-11T08:45:14.322786+0000","last_active":"2024-09-11T08:47:45.454870+0000","last_peered":"2024-09-11T08:47:45.454870+0000","last_clean":"2024-09-11T08:47:45.454870+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:47:45.454870+0000","last_undegraded":"2024-09-11T08:47:45.454870+0000","last_fullsized":"2024-09-11T08:47:45.454870+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.322217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.322217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014821319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":192,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.448056+0000","last_change":"2024-09-11T08:47:41.322355+0000","last_active":"2024-09-11T08:47:45.448056+0000","last_peered":"2024-09-11T08:47:45.448056+0000","last_clean":"2024-09-11T08:47:45.448056+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:47:45.448056+0000","last_undegraded":"2024-09-11T08:47:45.448056+0000","last_fullsized":"2024-09-11T08:47:45.448056+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:29.692933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:29.692933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007324077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":168,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.450113+0000","last_change":"2024-09-11T08:45:33.228265+0000","last_active":"2024-09-11T08:47:48.450113+0000","last_peered":"2024-09-11T08:47:48.450113+0000","last_clean":"2024-09-11T08:47:48.450113+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:47:48.450113+0000","last_undegraded":"2024-09-11T08:47:48.450113+0000","last_fullsized":"2024-09-11T08:47:48.450113+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:45:33.228082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064426500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"191'11023","reported_seq":11264,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:51.160753+0000","last_change":"2024-09-11T08:47:45.030515+0000","last_active":"2024-09-11T08:47:51.160753+0000","last_peered":"2024-09-11T08:47:51.160753+0000","last_clean":"2024-09-11T08:47:51.160753+0000","last_became_active":"2024-09-11T08:45:15.625778+0000","last_became_peered":"2024-09-11T08:45:15.625778+0000","last_unstale":"2024-09-11T08:47:51.160753+0000","last_undegraded":"2024-09-11T08:47:51.160753+0000","last_fullsized":"2024-09-11T08:47:51.160753+0000","mapping_epoch":149,"log_start":"176'5600","ondisk_log_start":"176'5600","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'10571","last_scrub_stamp":"2024-09-11T08:47:45.030446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:47:45.030446+0000","objects_scrubbed":4541,"log_size":5423,"log_dups_size":0,"ondisk_log_size":5423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:56.591715+0000","scrub_duration":0.68909561500000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267714560,"num_objects":4085,"num_object_clones":0,"num_object_copies":12255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11023,"num_write_kb":483456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"124'22","reported_seq":262,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.421913+0000","last_change":"2024-09-11T08:47:13.872981+0000","last_active":"2024-09-11T08:47:45.421913+0000","last_peered":"2024-09-11T08:47:45.421913+0000","last_clean":"2024-09-11T08:47:45.421913+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:47:45.421913+0000","last_undegraded":"2024-09-11T08:47:45.421913+0000","last_fullsized":"2024-09-11T08:47:45.421913+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:43:32.625376+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073394039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":200,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.453781+0000","last_change":"2024-09-11T08:45:20.372571+0000","last_active":"2024-09-11T08:47:45.453781+0000","last_peered":"2024-09-11T08:47:45.453781+0000","last_clean":"2024-09-11T08:47:45.453781+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:47:45.453781+0000","last_undegraded":"2024-09-11T08:47:45.453781+0000","last_fullsized":"2024-09-11T08:47:45.453781+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:45:20.372454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.372454+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019547269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"168'10","reported_seq":325,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:47.783904+0000","last_change":"2024-09-11T08:46:01.988356+0000","last_active":"2024-09-11T08:47:47.783904+0000","last_peered":"2024-09-11T08:47:47.783904+0000","last_clean":"2024-09-11T08:47:47.783904+0000","last_became_active":"2024-09-11T08:46:01.983055+0000","last_became_peered":"2024-09-11T08:46:01.983055+0000","last_unstale":"2024-09-11T08:47:47.783904+0000","last_undegraded":"2024-09-11T08:47:47.783904+0000","last_fullsized":"2024-09-11T08:47:47.783904+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"135'9","last_scrub_stamp":"2024-09-11T08:45:11.298652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.298652+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001669032,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":174,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.422267+0000","last_change":"2024-09-11T08:47:34.554720+0000","last_active":"2024-09-11T08:47:45.422267+0000","last_peered":"2024-09-11T08:47:45.422267+0000","last_clean":"2024-09-11T08:47:45.422267+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:47:45.422267+0000","last_undegraded":"2024-09-11T08:47:45.422267+0000","last_fullsized":"2024-09-11T08:47:45.422267+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:15.888394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:15.888394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003201476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":193,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.421596+0000","last_change":"2024-09-11T08:45:28.247838+0000","last_active":"2024-09-11T08:47:45.421596+0000","last_peered":"2024-09-11T08:47:45.421596+0000","last_clean":"2024-09-11T08:47:45.421596+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:47:45.421596+0000","last_undegraded":"2024-09-11T08:47:45.421596+0000","last_fullsized":"2024-09-11T08:47:45.421596+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.247771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015413499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":199,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.451566+0000","last_change":"2024-09-11T08:44:35.390285+0000","last_active":"2024-09-11T08:47:48.451566+0000","last_peered":"2024-09-11T08:47:48.451566+0000","last_clean":"2024-09-11T08:47:48.451566+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:47:48.451566+0000","last_undegraded":"2024-09-11T08:47:48.451566+0000","last_fullsized":"2024-09-11T08:47:48.451566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-11T08:44:35.390134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:35.390134+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017427033000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":180,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.422905+0000","last_change":"2024-09-11T08:45:08.675251+0000","last_active":"2024-09-11T08:47:45.422905+0000","last_peered":"2024-09-11T08:47:45.422905+0000","last_clean":"2024-09-11T08:47:45.422905+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:47:45.422905+0000","last_undegraded":"2024-09-11T08:47:45.422905+0000","last_fullsized":"2024-09-11T08:47:45.422905+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:08.675179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.675179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003515392,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":197,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.453963+0000","last_change":"2024-09-11T08:46:01.983864+0000","last_active":"2024-09-11T08:47:45.453963+0000","last_peered":"2024-09-11T08:47:45.453963+0000","last_clean":"2024-09-11T08:47:45.453963+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:47:45.453963+0000","last_undegraded":"2024-09-11T08:47:45.453963+0000","last_fullsized":"2024-09-11T08:47:45.453963+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:25.344348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.344348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013849730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":199,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.453157+0000","last_change":"2024-09-11T08:47:44.737776+0000","last_active":"2024-09-11T08:47:48.453157+0000","last_peered":"2024-09-11T08:47:48.453157+0000","last_clean":"2024-09-11T08:47:48.453157+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:47:48.453157+0000","last_undegraded":"2024-09-11T08:47:48.453157+0000","last_fullsized":"2024-09-11T08:47:48.453157+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:47:44.737721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.737721+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:04.218319+0000","scrub_duration":0.0032009640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":186,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.453435+0000","last_change":"2024-09-11T08:47:38.415255+0000","last_active":"2024-09-11T08:47:45.453435+0000","last_peered":"2024-09-11T08:47:45.453435+0000","last_clean":"2024-09-11T08:47:45.453435+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:47:45.453435+0000","last_undegraded":"2024-09-11T08:47:45.453435+0000","last_fullsized":"2024-09-11T08:47:45.453435+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:44:32.483439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013511770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":170,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.451381+0000","last_change":"2024-09-11T08:45:23.213642+0000","last_active":"2024-09-11T08:47:48.451381+0000","last_peered":"2024-09-11T08:47:48.451381+0000","last_clean":"2024-09-11T08:47:48.451381+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:47:48.451381+0000","last_undegraded":"2024-09-11T08:47:48.451381+0000","last_fullsized":"2024-09-11T08:47:48.451381+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.213592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.213592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001600473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":188,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.447749+0000","last_change":"2024-09-11T08:47:34.423379+0000","last_active":"2024-09-11T08:47:45.447749+0000","last_peered":"2024-09-11T08:47:45.447749+0000","last_clean":"2024-09-11T08:47:45.447749+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:47:45.447749+0000","last_undegraded":"2024-09-11T08:47:45.447749+0000","last_fullsized":"2024-09-11T08:47:45.447749+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_clean_scrub_stamp":"2024-09-11T08:47:34.420932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:45.412817+0000","scrub_duration":0.01242356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":194,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.422966+0000","last_change":"2024-09-11T08:47:29.687278+0000","last_active":"2024-09-11T08:47:45.422966+0000","last_peered":"2024-09-11T08:47:45.422966+0000","last_clean":"2024-09-11T08:47:45.422966+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:47:45.422966+0000","last_undegraded":"2024-09-11T08:47:45.422966+0000","last_fullsized":"2024-09-11T08:47:45.422966+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:18.928481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:18.928481+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047130310000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":185,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.448061+0000","last_change":"2024-09-11T08:47:45.751605+0000","last_active":"2024-09-11T08:47:48.448061+0000","last_peered":"2024-09-11T08:47:48.448061+0000","last_clean":"2024-09-11T08:47:48.448061+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:47:48.448061+0000","last_undegraded":"2024-09-11T08:47:48.448061+0000","last_fullsized":"2024-09-11T08:47:48.448061+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:45.751518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:45.751518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:47.231246+0000","scrub_duration":0.0061587329999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":288,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.451887+0000","last_change":"2024-09-11T08:45:20.119727+0000","last_active":"2024-09-11T08:47:48.451887+0000","last_peered":"2024-09-11T08:47:48.451887+0000","last_clean":"2024-09-11T08:47:48.451887+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:47:48.451887+0000","last_undegraded":"2024-09-11T08:47:48.451887+0000","last_fullsized":"2024-09-11T08:47:48.451887+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:20.119626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.119626+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018087660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":158,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.446374+0000","last_change":"2024-09-11T08:44:20.017341+0000","last_active":"2024-09-11T08:47:45.446374+0000","last_peered":"2024-09-11T08:47:45.446374+0000","last_clean":"2024-09-11T08:47:45.446374+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:47:45.446374+0000","last_undegraded":"2024-09-11T08:47:45.446374+0000","last_fullsized":"2024-09-11T08:47:45.446374+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:20.017261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:20.017261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001558369,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"152'15","reported_seq":192,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.452129+0000","last_change":"2024-09-11T08:45:21.132138+0000","last_active":"2024-09-11T08:47:48.452129+0000","last_peered":"2024-09-11T08:47:48.452129+0000","last_clean":"2024-09-11T08:47:48.452129+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:47:48.452129+0000","last_undegraded":"2024-09-11T08:47:48.452129+0000","last_fullsized":"2024-09-11T08:47:48.452129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'15","last_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.132083+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003984156,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":183,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.453853+0000","last_change":"2024-09-11T08:45:27.436673+0000","last_active":"2024-09-11T08:47:45.453853+0000","last_peered":"2024-09-11T08:47:45.453853+0000","last_clean":"2024-09-11T08:47:45.453853+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:47:45.453853+0000","last_undegraded":"2024-09-11T08:47:45.453853+0000","last_fullsized":"2024-09-11T08:47:45.453853+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.436552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001660176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":298,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:47.814753+0000","last_change":"2024-09-11T08:45:28.692173+0000","last_active":"2024-09-11T08:47:47.814753+0000","last_peered":"2024-09-11T08:47:47.814753+0000","last_clean":"2024-09-11T08:47:47.814753+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:47:47.814753+0000","last_undegraded":"2024-09-11T08:47:47.814753+0000","last_fullsized":"2024-09-11T08:47:47.814753+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:28.692100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.692100+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019579020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":179,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.449154+0000","last_change":"2024-09-11T08:45:14.581287+0000","last_active":"2024-09-11T08:47:48.449154+0000","last_peered":"2024-09-11T08:47:48.449154+0000","last_clean":"2024-09-11T08:47:48.449154+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:47:48.449154+0000","last_undegraded":"2024-09-11T08:47:48.449154+0000","last_fullsized":"2024-09-11T08:47:48.449154+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.581231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.581231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001580932,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":181,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.446332+0000","last_change":"2024-09-11T08:43:37.137260+0000","last_active":"2024-09-11T08:47:45.446332+0000","last_peered":"2024-09-11T08:47:45.446332+0000","last_clean":"2024-09-11T08:47:45.446332+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:47:45.446332+0000","last_undegraded":"2024-09-11T08:47:45.446332+0000","last_fullsized":"2024-09-11T08:47:45.446332+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:37.137173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:37.137173+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085039490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":174,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.447019+0000","last_change":"2024-09-11T08:45:27.198021+0000","last_active":"2024-09-11T08:47:45.447019+0000","last_peered":"2024-09-11T08:47:45.447019+0000","last_clean":"2024-09-11T08:47:45.447019+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:47:45.447019+0000","last_undegraded":"2024-09-11T08:47:45.447019+0000","last_fullsized":"2024-09-11T08:47:45.447019+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.197944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.197944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015230210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":181,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.458495+0000","last_change":"2024-09-11T08:45:21.736656+0000","last_active":"2024-09-11T08:47:48.458495+0000","last_peered":"2024-09-11T08:47:48.458495+0000","last_clean":"2024-09-11T08:47:48.458495+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:47:48.458495+0000","last_undegraded":"2024-09-11T08:47:48.458495+0000","last_fullsized":"2024-09-11T08:47:48.458495+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:21.736573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.736573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020360389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":168,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.447291+0000","last_change":"2024-09-11T08:45:28.259156+0000","last_active":"2024-09-11T08:47:48.447291+0000","last_peered":"2024-09-11T08:47:48.447291+0000","last_clean":"2024-09-11T08:47:48.447291+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:47:48.447291+0000","last_undegraded":"2024-09-11T08:47:48.447291+0000","last_fullsized":"2024-09-11T08:47:48.447291+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.259103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.259103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015518800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":211,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.421922+0000","last_change":"2024-09-11T08:46:01.993547+0000","last_active":"2024-09-11T08:47:45.421922+0000","last_peered":"2024-09-11T08:47:45.421922+0000","last_clean":"2024-09-11T08:47:45.421922+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:47:45.421922+0000","last_undegraded":"2024-09-11T08:47:45.421922+0000","last_fullsized":"2024-09-11T08:47:45.421922+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:45:06.657912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.657912+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023744109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":193,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.373466+0000","last_change":"2024-09-11T08:47:48.373407+0000","last_active":"2024-09-11T08:47:48.373466+0000","last_peered":"2024-09-11T08:47:48.373466+0000","last_clean":"2024-09-11T08:47:48.373466+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:47:48.373466+0000","last_undegraded":"2024-09-11T08:47:48.373466+0000","last_fullsized":"2024-09-11T08:47:48.373466+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:48.373336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:48.373336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:54.062621+0000","scrub_duration":0.001705364,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":170,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.450176+0000","last_change":"2024-09-11T08:45:14.165632+0000","last_active":"2024-09-11T08:47:48.450176+0000","last_peered":"2024-09-11T08:47:48.450176+0000","last_clean":"2024-09-11T08:47:48.450176+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:47:48.450176+0000","last_undegraded":"2024-09-11T08:47:48.450176+0000","last_fullsized":"2024-09-11T08:47:48.450176+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.165576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.165576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012147639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":172,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.447243+0000","last_change":"2024-09-11T08:47:33.592756+0000","last_active":"2024-09-11T08:47:48.447243+0000","last_peered":"2024-09-11T08:47:48.447243+0000","last_clean":"2024-09-11T08:47:48.447243+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:47:48.447243+0000","last_undegraded":"2024-09-11T08:47:48.447243+0000","last_fullsized":"2024-09-11T08:47:48.447243+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.349351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.349351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006276294,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":181,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.447433+0000","last_change":"2024-09-11T08:44:12.198399+0000","last_active":"2024-09-11T08:47:48.447433+0000","last_peered":"2024-09-11T08:47:48.447433+0000","last_clean":"2024-09-11T08:47:48.447433+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:47:48.447433+0000","last_undegraded":"2024-09-11T08:47:48.447433+0000","last_fullsized":"2024-09-11T08:47:48.447433+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:44:12.198335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.198335+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020517040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":184,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.422740+0000","last_change":"2024-09-11T08:45:14.574739+0000","last_active":"2024-09-11T08:47:45.422740+0000","last_peered":"2024-09-11T08:47:45.422740+0000","last_clean":"2024-09-11T08:47:45.422740+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:47:45.422740+0000","last_undegraded":"2024-09-11T08:47:45.422740+0000","last_fullsized":"2024-09-11T08:47:45.422740+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.574678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.574678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013819469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":182,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.453009+0000","last_change":"2024-09-11T08:47:32.404383+0000","last_active":"2024-09-11T08:47:45.453009+0000","last_peered":"2024-09-11T08:47:45.453009+0000","last_clean":"2024-09-11T08:47:45.453009+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:47:45.453009+0000","last_undegraded":"2024-09-11T08:47:45.453009+0000","last_fullsized":"2024-09-11T08:47:45.453009+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:28.484047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.484047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001479172,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":188,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.450298+0000","last_change":"2024-09-11T08:47:44.807380+0000","last_active":"2024-09-11T08:47:48.450298+0000","last_peered":"2024-09-11T08:47:48.450298+0000","last_clean":"2024-09-11T08:47:48.450298+0000","last_became_active":"2024-09-11T08:46:01.984167+0000","last_became_peered":"2024-09-11T08:46:01.984167+0000","last_unstale":"2024-09-11T08:47:48.450298+0000","last_undegraded":"2024-09-11T08:47:48.450298+0000","last_fullsized":"2024-09-11T08:47:48.450298+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.807331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.807331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:03.561680+0000","scrub_duration":0.0013867529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"136'6","reported_seq":212,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.455286+0000","last_change":"2024-09-11T08:46:01.996138+0000","last_active":"2024-09-11T08:47:48.455286+0000","last_peered":"2024-09-11T08:47:48.455286+0000","last_clean":"2024-09-11T08:47:48.455286+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:47:48.455286+0000","last_undegraded":"2024-09-11T08:47:48.455286+0000","last_fullsized":"2024-09-11T08:47:48.455286+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'6","last_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:44:21.302870+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012512683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":192,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.449398+0000","last_change":"2024-09-11T08:45:21.481725+0000","last_active":"2024-09-11T08:47:48.449398+0000","last_peered":"2024-09-11T08:47:48.449398+0000","last_clean":"2024-09-11T08:47:48.449398+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:47:48.449398+0000","last_undegraded":"2024-09-11T08:47:48.449398+0000","last_fullsized":"2024-09-11T08:47:48.449398+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:21.481634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.481634+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025664590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":33,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.421925+0000","last_change":"2024-09-11T08:46:01.979915+0000","last_active":"2024-09-11T08:47:45.421925+0000","last_peered":"2024-09-11T08:47:45.421925+0000","last_clean":"2024-09-11T08:47:45.421925+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:47:45.421925+0000","last_undegraded":"2024-09-11T08:47:45.421925+0000","last_fullsized":"2024-09-11T08:47:45.421925+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.196277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.196277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":187,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.453278+0000","last_change":"2024-09-11T08:45:23.348327+0000","last_active":"2024-09-11T08:47:45.453278+0000","last_peered":"2024-09-11T08:47:45.453278+0000","last_clean":"2024-09-11T08:47:45.453278+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:47:45.453278+0000","last_undegraded":"2024-09-11T08:47:45.453278+0000","last_fullsized":"2024-09-11T08:47:45.453278+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.348275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.348275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035843659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":189,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.459152+0000","last_change":"2024-09-11T08:45:23.674163+0000","last_active":"2024-09-11T08:47:48.459152+0000","last_peered":"2024-09-11T08:47:48.459152+0000","last_clean":"2024-09-11T08:47:48.459152+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:47:48.459152+0000","last_undegraded":"2024-09-11T08:47:48.459152+0000","last_fullsized":"2024-09-11T08:47:48.459152+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.674102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.674102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001440293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":174,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.455473+0000","last_change":"2024-09-11T08:44:23.320368+0000","last_active":"2024-09-11T08:47:48.455473+0000","last_peered":"2024-09-11T08:47:48.455473+0000","last_clean":"2024-09-11T08:47:48.455473+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:47:48.455473+0000","last_undegraded":"2024-09-11T08:47:48.455473+0000","last_fullsized":"2024-09-11T08:47:48.455473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:23.320301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:23.320301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017133368999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":45,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-11T08:47:45.422446+0000","last_change":"2024-09-11T08:46:02.002471+0000","last_active":"2024-09-11T08:47:45.422446+0000","last_peered":"2024-09-11T08:47:45.422446+0000","last_clean":"2024-09-11T08:47:45.422446+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:47:45.422446+0000","last_undegraded":"2024-09-11T08:47:45.422446+0000","last_fullsized":"2024-09-11T08:47:45.422446+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:45:25.620635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.620635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":181,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-11T08:47:48.452906+0000","last_change":"2024-09-11T08:47:32.555942+0000","last_active":"2024-09-11T08:47:48.452906+0000","last_peered":"2024-09-11T08:47:48.452906+0000","last_clean":"2024-09-11T08:47:48.452906+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:47:48.452906+0000","last_undegraded":"2024-09-11T08:47:48.452906+0000","last_fullsized":"2024-09-11T08:47:48.452906+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:32.555890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:32.555890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:41.110543+0000","scrub_duration":0.0014425740000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":4215013400,"num_objects":64317,"num_object_clones":0,"num_object_copies":192951,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64317,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":175601,"num_write_kb":7677313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7769,"num_bytes_recovered":509149184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13215215616,"data_stored":13215203400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85361,"ondisk_log_size":85361,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":1584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":456,"ondisk_log_size":456,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":545,"num_read_kb":540,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299508,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1601996,"kb_used_data":1304644,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92143156,"statfs":{"total":95995035648,"available":94354591744,"internally_reserved":0,"allocated":1335955456,"data_stored":1335617911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.2889999999999999,"15min":0.81799999999999995},"min":{"1min":0.372,"5min":0.22,"15min":0.22},"max":{"1min":2.585,"5min":32.171999999999997,"15min":32.171999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":1.571,"5min":1.4219999999999999,"15min":0.89900000000000002},"min":{"1min":0.29399999999999998,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":3.665,"5min":32.149999999999999,"15min":32.149999999999999},"last":1.9019999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.4830000000000001,"15min":0.90100000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.4980000000000002,"5min":33.692999999999998,"15min":33.692999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":1.228,"5min":1.3700000000000001,"15min":0.84499999999999997},"min":{"1min":0.32000000000000001,"5min":0.26600000000000001,"15min":0.23300000000000001},"max":{"1min":3.6259999999999999,"5min":35.591000000000001,"15min":35.591000000000001},"last":1.8660000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.98899999999999999,"15min":0.754},"min":{"1min":0.30299999999999999,"5min":0.249,"15min":0.249},"max":{"1min":2.8140000000000001,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.8280000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":1.0429999999999999,"15min":0.71599999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.782,"5min":5.7990000000000004,"15min":5.7990000000000004},"last":1.254}]},{"osd":3,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.083,"15min":0.77800000000000002},"min":{"1min":0.32700000000000001,"5min":0.25,"15min":0.25},"max":{"1min":3.2839999999999998,"5min":7.8609999999999998,"15min":7.8609999999999998},"last":1.97},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.29,"15min":0.85899999999999999},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":5.625,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.41399999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.53500000000000003,"15min":0.46700000000000003},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.23400000000000001},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.56299999999999994,"15min":0.47899999999999998},"min":{"1min":0.27000000000000002,"5min":0.216,"15min":0.216},"max":{"1min":4.8920000000000003,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":1.7789999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.60599999999999998,"15min":0.499},"min":{"1min":0.31,"5min":0.255,"15min":0.23300000000000001},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":1.6599999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.67800000000000005,"15min":0.54200000000000004},"min":{"1min":0.32900000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.093,"5min":3.093,"15min":3.093},"last":0.441}]},{"osd":6,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.68100000000000005,"15min":0.55800000000000005},"min":{"1min":0.35399999999999998,"5min":0.317,"15min":0.315},"max":{"1min":2.8820000000000001,"5min":3.5979999999999999,"15min":3.5979999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.63400000000000001,"15min":0.54600000000000004},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":1.8080000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528441,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1615164,"kb_used_data":1305332,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92129988,"statfs":{"total":95995035648,"available":94341107712,"internally_reserved":0,"allocated":1336659968,"data_stored":1336338619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.3400000000000001,"15min":0.89500000000000002},"min":{"1min":0.23300000000000001,"5min":0.217,"15min":0.217},"max":{"1min":3.5619999999999998,"5min":28.302,"15min":28.302},"last":1.264},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.4450000000000001,"15min":0.875},"min":{"1min":0.33700000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.9630000000000001,"5min":28.314,"15min":28.314},"last":2.0430000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":2.173,"5min":1.4370000000000001,"15min":0.85599999999999998},"min":{"1min":0.32300000000000001,"5min":0.29399999999999998,"15min":0.28299999999999997},"max":{"1min":7.8810000000000002,"5min":12.586,"15min":12.586},"last":0.26300000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.1850000000000001,"15min":0.77300000000000002},"min":{"1min":0.30099999999999999,"5min":0.254,"15min":0.253},"max":{"1min":9.5370000000000008,"5min":10.834,"15min":10.834},"last":0.44600000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.97999999999999998,"15min":0.70199999999999996},"min":{"1min":0.29499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.8849999999999998,"5min":14.313000000000001,"15min":14.313000000000001},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.0569999999999999,"15min":0.71599999999999997},"min":{"1min":0.40500000000000003,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":3.593,"5min":6.5979999999999999,"15min":6.5979999999999999},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":1.889,"5min":1.401,"15min":0.85399999999999998},"min":{"1min":0.35799999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":4.2270000000000003,"5min":11.433,"15min":11.433},"last":0.372},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.042,"15min":0.76200000000000001},"min":{"1min":0.33100000000000002,"5min":0.224,"15min":0.224},"max":{"1min":2.738,"5min":9.1069999999999993,"15min":9.1069999999999993},"last":1.236}]},{"osd":4,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.50900000000000001,"15min":0.44600000000000001},"min":{"1min":0.27300000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.502,"15min":0.443},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":0.309}]},{"osd":5,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.59399999999999997,"15min":0.49299999999999999},"min":{"1min":0.28699999999999998,"5min":0.245,"15min":0.245},"max":{"1min":3.097,"5min":3.097,"15min":3.097},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.042,"5min":0.61599999999999999,"15min":0.501},"min":{"1min":0.312,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":5.0819999999999999,"5min":5.0819999999999999,"15min":5.0819999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:47:37 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64200000000000002,"15min":0.51300000000000001},"min":{"1min":0.29899999999999999,"5min":0.25900000000000001,"15min":0.247},"max":{"1min":2.7810000000000001,"5min":7.327,"15min":7.327},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.55600000000000005,"15min":0.51600000000000001},"min":{"1min":0.38400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.41699999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757373,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2414988,"kb_used_data":2109248,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91330164,"statfs":{"total":95995035648,"available":93522087936,"internally_reserved":0,"allocated":2159869952,"data_stored":2159536592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.0509999999999999,"15min":0.72799999999999998},"min":{"1min":0.23100000000000001,"5min":0.20899999999999999,"15min":0.20599999999999999},"max":{"1min":3.1850000000000001,"5min":7.3899999999999997,"15min":7.3899999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.226,"15min":0.77300000000000002},"min":{"1min":0.245,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":3.3799999999999999,"5min":6.5789999999999997,"15min":6.5789999999999997},"last":2.6219999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.0549999999999999,"15min":0.755},"min":{"1min":0.23799999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":3.8029999999999999,"5min":3.8029999999999999,"15min":4.8419999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.0840000000000001,"15min":0.71599999999999997},"min":{"1min":0.35599999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.2120000000000002,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.89600000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.2010000000000001,"15min":0.76400000000000001},"min":{"1min":0.29499999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.0919999999999996,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":2.492},{"interface":"front","average":{"1min":1.089,"5min":0.98399999999999999,"15min":0.70499999999999996},"min":{"1min":0.38500000000000001,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":3.2509999999999999,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":1.0880000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.0429999999999999,"15min":0.73499999999999999},"min":{"1min":0.35399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.6429999999999998,"5min":6.0460000000000003,"15min":6.0460000000000003},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.429,"5min":1.2889999999999999,"15min":0.82799999999999996},"min":{"1min":0.49199999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.415,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":1.431}]},{"osd":4,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.54800000000000004,"15min":0.44},"min":{"1min":0.27900000000000003,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":3.645,"5min":3.645,"15min":3.645},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.52200000000000002,"15min":0.44700000000000001},"min":{"1min":0.25800000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.3079999999999998,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.69799999999999995}]},{"osd":6,"last update":"Wed Sep 11 08:47:24 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.58399999999999996,"15min":0.48499999999999999},"min":{"1min":0.29399999999999998,"5min":0.27700000000000002,"15min":0.23599999999999999},"max":{"1min":3.2509999999999999,"5min":3.4329999999999998,"15min":3.4329999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.58899999999999997,"15min":0.48299999999999998},"min":{"1min":0.29099999999999998,"5min":0.26100000000000001,"15min":0.251},"max":{"1min":3,"5min":3,"15min":3},"last":0.39100000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:47:34 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.64400000000000002,"15min":0.53400000000000003},"min":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.27800000000000002},"max":{"1min":3.7639999999999998,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.64400000000000002,"15min":0.57099999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.2730000000000001,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":0.71099999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986305,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1606276,"kb_used_data":1304764,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":92138876,"statfs":{"total":95995035648,"available":94350209024,"internally_reserved":0,"allocated":1336078336,"data_stored":1335747877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.6060000000000001,"15min":1.2949999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.20399999999999999},"max":{"1min":4.2830000000000004,"5min":48.451999999999998,"15min":89.510000000000005},"last":0.56999999999999995},{"interface":"front","average":{"1min":2.294,"5min":1.8700000000000001,"15min":1.4039999999999999},"min":{"1min":0.23999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":5.569,"5min":49.115000000000002,"15min":89.765000000000001},"last":0.68799999999999994}]},{"osd":1,"last update":"Wed Sep 11 08:47:39 2024","interfaces":[{"interface":"back","average":{"1min":2.1200000000000001,"5min":1.5549999999999999,"15min":0.88500000000000001},"min":{"1min":0.63900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":6.1020000000000003,"5min":18.596,"15min":18.596},"last":0.76800000000000002},{"interface":"front","average":{"1min":2.052,"5min":1.4570000000000001,"15min":0.872},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.23599999999999999},"max":{"1min":5.3810000000000002,"5min":18.754000000000001,"15min":18.754000000000001},"last":0.77700000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:47:39 2024","interfaces":[{"interface":"back","average":{"1min":2.3580000000000001,"5min":1.2310000000000001,"15min":0.80300000000000005},"min":{"1min":0.39300000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":17.303000000000001,"5min":17.303000000000001,"15min":17.303000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":2.1520000000000001,"5min":1.1859999999999999,"15min":0.78700000000000003},"min":{"1min":0.28599999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":20.702999999999999,"5min":20.702999999999999,"15min":20.702999999999999},"last":2.6389999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:47:39 2024","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.9550000000000001,"15min":1.4650000000000001},"min":{"1min":0.314,"5min":0.255,"15min":0.255},"max":{"1min":4.2229999999999999,"5min":48.194000000000003,"15min":89.881},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.8720000000000001,"5min":1.925,"15min":1.4299999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":6.0650000000000004,"5min":48.840000000000003,"15min":89.715000000000003},"last":2.9620000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:47:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.53300000000000003,"15min":0.45000000000000001},"min":{"1min":0.29799999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":2.2389999999999999,"5min":2.516,"15min":2.516},"last":0.879},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.54800000000000004,"15min":0.46100000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.23699999999999999},"max":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.74399999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.495,"15min":0.45500000000000002},"min":{"1min":0.28100000000000003,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.0640000000000001,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.309},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.61199999999999999,"15min":0.52200000000000002},"min":{"1min":0.22800000000000001,"5min":0.221,"15min":0.221},"max":{"1min":3.4590000000000001,"5min":3.4590000000000001,"15min":3.4590000000000001},"last":0.44500000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:47:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.621,"15min":0.53000000000000003},"min":{"1min":0.29199999999999998,"5min":0.252,"15min":0.252},"max":{"1min":3.339,"5min":3.339,"15min":3.339},"last":0.375},{"interface":"front","average":{"1min":1.194,"5min":0.68700000000000006,"15min":0.55000000000000004},"min":{"1min":0.32000000000000001,"5min":0.29599999999999999,"15min":0.28799999999999998},"max":{"1min":5.1970000000000001,"5min":5.1970000000000001,"15min":5.1970000000000001},"last":0.40699999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149831,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1870044,"kb_used_data":1572692,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91875108,"statfs":{"total":95995035648,"available":94080110592,"internally_reserved":0,"allocated":1610436608,"data_stored":1610083154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:21 2024","interfaces":[{"interface":"back","average":{"1min":1.776,"5min":1.7529999999999999,"15min":0.89500000000000002},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":7.1399999999999997,"5min":10.901999999999999,"15min":10.901999999999999},"last":2.363},{"interface":"front","average":{"1min":2.3639999999999999,"5min":2.1850000000000001,"15min":1.0820000000000001},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":5.4249999999999998,"5min":9.6820000000000004,"15min":9.6820000000000004},"last":0.69699999999999995}]},{"osd":1,"last update":"Wed Sep 11 08:47:21 2024","interfaces":[{"interface":"back","average":{"1min":2.7909999999999999,"5min":3.278,"15min":1.446},"min":{"1min":0.26700000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":8.2439999999999998,"5min":115.601,"15min":115.601},"last":0.72199999999999998},{"interface":"front","average":{"1min":2.089,"5min":3.1259999999999999,"15min":1.373},"min":{"1min":0.29499999999999998,"5min":0.20300000000000001,"15min":0.17999999999999999},"max":{"1min":8.4420000000000002,"5min":115.474,"15min":115.474},"last":0.82599999999999996}]},{"osd":2,"last update":"Wed Sep 11 08:47:21 2024","interfaces":[{"interface":"back","average":{"1min":1.6879999999999999,"5min":1.9199999999999999,"15min":0.99199999999999999},"min":{"1min":0.28599999999999998,"5min":0.251,"15min":0.23200000000000001},"max":{"1min":8.2699999999999996,"5min":8.9510000000000005,"15min":8.9510000000000005},"last":2.0699999999999998},{"interface":"front","average":{"1min":1.48,"5min":1.746,"15min":0.94499999999999995},"min":{"1min":0.30299999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":5.5999999999999996,"5min":10.695,"15min":10.695},"last":2.3839999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:47:33 2024","interfaces":[{"interface":"back","average":{"1min":2.6869999999999998,"5min":1.73,"15min":0.997},"min":{"1min":0.35799999999999998,"5min":0.27500000000000002,"15min":0.26700000000000002},"max":{"1min":8.3279999999999994,"5min":8.9969999999999999,"15min":8.9969999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":2.0779999999999998,"5min":1.4359999999999999,"15min":0.88300000000000001},"min":{"1min":0.39500000000000002,"5min":0.28000000000000003,"15min":0.27700000000000002},"max":{"1min":7.234,"5min":7.234,"15min":7.234},"last":2.3220000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:46:49 2024","interfaces":[{"interface":"back","average":{"1min":1.843,"5min":1.6020000000000001,"15min":0.92300000000000004},"min":{"1min":0.29799999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":8.3599999999999994,"5min":11.526999999999999,"15min":11.526999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.296,"15min":0.81999999999999995},"min":{"1min":0.40300000000000002,"5min":0.30599999999999999,"15min":0.30499999999999999},"max":{"1min":3.7949999999999999,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":2.2909999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:47:21 2024","interfaces":[{"interface":"back","average":{"1min":1.9359999999999999,"5min":1.7849999999999999,"15min":1.0109999999999999},"min":{"1min":0.34899999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.4160000000000004,"5min":11.494999999999999,"15min":11.494999999999999},"last":0.75},{"interface":"front","average":{"1min":1.6659999999999999,"5min":1.6559999999999999,"15min":0.98299999999999998},"min":{"1min":0.41399999999999998,"5min":0.311,"15min":0.311},"max":{"1min":3.5630000000000002,"5min":7.7249999999999996,"15min":7.7249999999999996},"last":0.36299999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":2.4969999999999999,"5min":1.831,"15min":1.0740000000000001},"min":{"1min":0.39600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":9.1539999999999999,"5min":11.561,"15min":11.561},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.8779999999999999,"5min":1.4890000000000001,"15min":0.93799999999999994},"min":{"1min":0.44500000000000001,"5min":0.371,"15min":0.34699999999999998},"max":{"1min":6.6200000000000001,"5min":6.6200000000000001,"15min":6.6200000000000001},"last":2.343}]}]},{"osd":2,"up_from":18,"seq":77309411467,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2120116,"kb_used_data":1810284,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91625036,"statfs":{"total":95995035648,"available":93824036864,"internally_reserved":0,"allocated":1853730816,"data_stored":1853416476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:46:54 2024","interfaces":[{"interface":"back","average":{"1min":4.9740000000000002,"5min":2.7919999999999998,"15min":1.1970000000000001},"min":{"1min":0.16500000000000001,"5min":0.159,"15min":0.159},"max":{"1min":34.713999999999999,"5min":34.713999999999999,"15min":34.713999999999999},"last":1.6890000000000001},{"interface":"front","average":{"1min":4.2430000000000003,"5min":2.9350000000000001,"15min":1.282},"min":{"1min":0.26200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":34.399000000000001,"5min":34.399000000000001,"15min":34.399000000000001},"last":1.292}]},{"osd":1,"last update":"Wed Sep 11 08:46:54 2024","interfaces":[{"interface":"back","average":{"1min":4.6509999999999998,"5min":2.1560000000000001,"15min":1.0009999999999999},"min":{"1min":0.28299999999999997,"5min":0.215,"15min":0.19},"max":{"1min":34.771000000000001,"5min":34.771000000000001,"15min":34.771000000000001},"last":1.5820000000000001},{"interface":"front","average":{"1min":4.0880000000000001,"5min":2.2639999999999998,"15min":1.044},"min":{"1min":0.252,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":35.209000000000003,"5min":35.209000000000003,"15min":35.209000000000003},"last":1.331}]},{"osd":3,"last update":"Wed Sep 11 08:47:10 2024","interfaces":[{"interface":"back","average":{"1min":4.2190000000000003,"5min":2.3809999999999998,"15min":1.167},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":34.75,"5min":34.75,"15min":34.75},"last":1.669},{"interface":"front","average":{"1min":3.948,"5min":2.3239999999999998,"15min":1.1479999999999999},"min":{"1min":0.40999999999999998,"5min":0.27200000000000002,"15min":0.26800000000000002},"max":{"1min":32.802,"5min":32.802,"15min":32.802},"last":1.395}]},{"osd":4,"last update":"Wed Sep 11 08:47:28 2024","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.351,"15min":0.88200000000000001},"min":{"1min":0.72999999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":3.9449999999999998,"5min":8.0820000000000007,"15min":8.0820000000000007},"last":1.627},{"interface":"front","average":{"1min":1.319,"5min":1.256,"15min":0.85299999999999998},"min":{"1min":0.29999999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":4.4450000000000003,"5min":6.9320000000000004,"15min":6.9320000000000004},"last":1.3520000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:46:54 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.296,"15min":0.81899999999999995},"min":{"1min":0.439,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.8039999999999998,"5min":6.9550000000000001,"15min":6.9550000000000001},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0900000000000001,"15min":0.74199999999999999},"min":{"1min":0.29199999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.9940000000000002,"5min":5.9530000000000003,"15min":5.9530000000000003},"last":1.7609999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:47:19 2024","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.4730000000000001,"15min":0.93400000000000005},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":5.1980000000000004,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":1.9019999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.2210000000000001,"15min":0.82699999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.931,"5min":6.9779999999999998,"15min":6.9779999999999998},"last":1.375}]},{"osd":7,"last update":"Wed Sep 11 08:47:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.0780000000000001,"15min":0.80600000000000005},"min":{"1min":0.35599999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.1400000000000001,"5min":5.1820000000000004,"15min":5.9909999999999997},"last":1.73},{"interface":"front","average":{"1min":1.673,"5min":1.351,"15min":0.94899999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36099999999999999},"max":{"1min":6.0919999999999996,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":1.6499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574992,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2126880,"kb_used_data":1817048,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91618272,"statfs":{"total":95995035648,"available":93817110528,"internally_reserved":0,"allocated":1860657152,"data_stored":1860298981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.6399999999999999,"15min":1.2609999999999999},"min":{"1min":0.33600000000000002,"5min":0.192,"15min":0.183},"max":{"1min":4.6319999999999997,"5min":31.466999999999999,"15min":117.01600000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":1.669,"5min":2.3519999999999999,"15min":1.5389999999999999},"min":{"1min":0.318,"5min":0.193,"15min":0.193},"max":{"1min":5.9409999999999998,"5min":32.204000000000001,"15min":117.62},"last":0.55300000000000005}]},{"osd":2,"last update":"Wed Sep 11 08:46:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.694,"15min":0.90200000000000002},"min":{"1min":0.22800000000000001,"5min":0.191,"15min":0.191},"max":{"1min":4.6379999999999999,"5min":9.5719999999999992,"15min":20.628},"last":0.57599999999999996},{"interface":"front","average":{"1min":1.256,"5min":1.647,"15min":0.86899999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":7.4080000000000004,"5min":11.461,"15min":20.148},"last":1.881}]},{"osd":3,"last update":"Wed Sep 11 08:47:11 2024","interfaces":[{"interface":"back","average":{"1min":2.3799999999999999,"5min":1.752,"15min":1.248},"min":{"1min":0.39700000000000002,"5min":0.191,"15min":0.191},"max":{"1min":5.8490000000000002,"5min":8.3550000000000004,"15min":116.98},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.242,"5min":1.7190000000000001,"15min":1.304},"min":{"1min":0.27100000000000002,"5min":0.22,"15min":0.22},"max":{"1min":4.5140000000000002,"5min":10.379,"15min":117.56699999999999},"last":2.8969999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:47:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.2629999999999999,"15min":0.81200000000000006},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.26200000000000001},"max":{"1min":3.681,"5min":11.19,"15min":11.19},"last":2.0760000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.05,"15min":0.76600000000000001},"min":{"1min":0.30199999999999999,"5min":0.28499999999999998,"15min":0.251},"max":{"1min":3.2120000000000002,"5min":5.056,"15min":7.7169999999999996},"last":0.41399999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:46:56 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":2.6240000000000001,"15min":1.2769999999999999},"min":{"1min":0.34200000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.6680000000000001,"5min":103.337,"15min":103.337},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.05,"5min":2.4980000000000002,"15min":1.2170000000000001},"min":{"1min":0.41599999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.6269999999999998,"5min":103.163,"15min":103.163},"last":2.129}]},{"osd":6,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.1819999999999999,"15min":0.78900000000000003},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.29399999999999998},"max":{"1min":3.907,"5min":8.9149999999999991,"15min":8.9149999999999991},"last":0.44700000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.488,"15min":0.92300000000000004},"min":{"1min":0.45000000000000001,"5min":0.312,"15min":0.31},"max":{"1min":4.4480000000000004,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:47:31 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.3380000000000001,"15min":0.876},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":4.593,"5min":11.315,"15min":11.315},"last":2.1850000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.127,"15min":0.82299999999999995},"min":{"1min":0.39800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":3.8759999999999999,"5min":4.9100000000000001,"15min":8.8800000000000008},"last":0.499}]}]},{"osd":0,"up_from":8,"seq":34359738516,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2138908,"kb_used_data":1833088,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91606244,"statfs":{"total":95995035648,"available":93804793856,"internally_reserved":0,"allocated":1877082112,"data_stored":1876788958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:47:28 2024","interfaces":[{"interface":"back","average":{"1min":1.8460000000000001,"5min":1.954,"15min":0.96799999999999997},"min":{"1min":0.17699999999999999,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":6.2510000000000003,"5min":8.3900000000000006,"15min":8.3900000000000006},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.992,"5min":1.8220000000000001,"15min":0.91000000000000003},"min":{"1min":0.217,"5min":0.187,"15min":0.186},"max":{"1min":7.4169999999999998,"5min":12.532,"15min":12.532},"last":0.41599999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:46:53 2024","interfaces":[{"interface":"back","average":{"1min":2.4830000000000001,"5min":1.843,"15min":0.96299999999999997},"min":{"1min":0.32800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":12.481,"5min":12.481,"15min":19.940000000000001},"last":0.373},{"interface":"front","average":{"1min":2.1240000000000001,"5min":1.9410000000000001,"15min":1.0069999999999999},"min":{"1min":0.39800000000000002,"5min":0.23599999999999999,"15min":0.22700000000000001},"max":{"1min":6.6479999999999997,"5min":8.3049999999999997,"15min":20.757000000000001},"last":0.33300000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:47:09 2024","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":2.1749999999999998,"15min":1.0960000000000001},"min":{"1min":0.312,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.5709999999999997,"5min":11.951000000000001,"15min":11.951000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.919,"15min":1.0109999999999999},"min":{"1min":0.28100000000000003,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":7.6399999999999997,"5min":7.6399999999999997,"15min":7.6399999999999997},"last":1.2430000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:47:33 2024","interfaces":[{"interface":"back","average":{"1min":1.7470000000000001,"5min":1.5429999999999999,"15min":0.93300000000000005},"min":{"1min":0.32100000000000001,"5min":0.26600000000000001,"15min":0.25},"max":{"1min":6.2009999999999996,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.7949999999999999,"5min":1.46,"15min":0.90700000000000003},"min":{"1min":0.30599999999999999,"5min":0.255,"15min":0.255},"max":{"1min":7.4630000000000001,"5min":7.4630000000000001,"15min":7.4630000000000001},"last":1.1100000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:46:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.5369999999999999,"15min":0.91800000000000004},"min":{"1min":0.374,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.887,"5min":7.4009999999999998,"15min":7.4009999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":1.754,"5min":1.577,"15min":0.91600000000000004},"min":{"1min":0.32700000000000001,"5min":0.30499999999999999,"15min":0.28299999999999997},"max":{"1min":4.5970000000000004,"5min":7.423,"15min":7.423},"last":0.63300000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:47:13 2024","interfaces":[{"interface":"back","average":{"1min":1.9139999999999999,"5min":1.6890000000000001,"15min":0.99199999999999999},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.1310000000000002,"5min":7.4240000000000004,"15min":7.4240000000000004},"last":1.042},{"interface":"front","average":{"1min":2.149,"5min":1.738,"15min":1.028},"min":{"1min":0.434,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":6.1740000000000004,"5min":7.4489999999999998,"15min":7.4489999999999998},"last":1.2170000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:47:45 2024","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.6519999999999999,"15min":0.999},"min":{"1min":0.32800000000000001,"5min":0.313,"15min":0.313},"max":{"1min":5.8179999999999996,"5min":7.4900000000000002,"15min":7.4900000000000002},"last":0.438},{"interface":"front","average":{"1min":2.0150000000000001,"5min":1.629,"15min":1.0209999999999999},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.307},"max":{"1min":6.0800000000000001,"5min":7.4450000000000003,"15min":7.4450000000000003},"last":1.0149999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1873084416,"data_stored":1873084416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1843855360,"data_stored":1843855360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1825837056,"data_stored":1825832984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1586954240,"data_stored":1586954240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1326841856,"data_stored":1326841856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2122321920,"data_stored":2122317848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1321598976,"data_stored":1321598976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1314721792,"data_stored":1314717720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:47:57.491 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:57.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:57.532 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:57.532 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:57.633 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:57.665 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:57.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:57.666 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:57.696 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:57 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:57.696 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:57 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:57.767 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:57.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:57.804 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:57.804 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:57.905 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:57.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:57 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:47:57.929 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:57.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:57.930 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:47:58.031 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:58.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:58.064 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:58.065 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:58.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:58.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:58.198 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:58.198 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:58.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:47:58.343 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:58.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:58.344 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:47:58.445 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:58.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:58.478 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:58.479 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:58.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:47:58.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:58.625 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:58.625 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:47:58.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:58.739 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:58 smithi092 ceph-mon[29647]: pgmap v389: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.68k op/s 2024-09-11T08:47:58.740 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:58 smithi092 ceph-mon[41114]: pgmap v389: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.68k op/s 2024-09-11T08:47:58.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:58.760 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:58.761 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:58.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:58 smithi145 ceph-mon[33384]: pgmap v389: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.68k op/s 2024-09-11T08:47:58.861 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:47:58.896 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:58.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:58.897 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:47:58.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:59.036 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:59.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:59.037 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:59.138 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:47:59.168 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:59.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:59.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:47:59.217 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:47:59.270 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:47:59.303 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:59.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:59.304 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:47:59.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:59.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:59.439 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:59.498 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:47:59.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:59.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:59.575 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:59.575 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:59.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:47:59.712 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:59.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:59.713 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:47:59.730 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:47:59 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1848797438' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:47:59.731 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:47:59 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1848797438' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:47:59.814 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:47:59.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:59.863 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:47:59.863 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:47:59.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:47:59 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1848797438' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:47:59.964 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:47:59.990 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:47:59.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:59.991 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:00.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:00.130 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:00.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:00.131 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:00.232 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:00.259 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:00.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:00.260 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:00.361 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:00.391 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:00.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:00.392 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:00.418 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:48:00.493 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:00.527 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:00.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:00.528 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:00.629 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:00.666 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:00.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:00.667 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:00.713 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:00 smithi092 ceph-mon[29647]: pgmap v390: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 660 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-11T08:48:00.713 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:00 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:48:00.714 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:00 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1848797438' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:48:00.714 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:00 smithi092 ceph-mon[29647]: osdmap e195: 8 total, 8 up, 8 in 2024-09-11T08:48:00.714 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:00 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:48:00.714 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:00.714 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:00 smithi092 ceph-mon[41114]: pgmap v390: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 660 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-11T08:48:00.714 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:00 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:48:00.714 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:00 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1848797438' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:48:00.714 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:00 smithi092 ceph-mon[41114]: osdmap e195: 8 total, 8 up, 8 in 2024-09-11T08:48:00.714 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:00 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:48:00.714 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:00.768 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:00.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:00.806 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:00.806 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:00.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:00 smithi145 ceph-mon[33384]: pgmap v390: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 660 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-11T08:48:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:00 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:48:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:00 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1848797438' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:48:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:00 smithi145 ceph-mon[33384]: osdmap e195: 8 total, 8 up, 8 in 2024-09-11T08:48:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:00 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:48:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:00.968 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:00.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:00.969 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:01.070 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:01.112 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:01.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:01.114 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:01.214 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:01.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:01.258 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:01.259 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:01.359 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:01.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:01.389 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:01.389 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:01.490 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:01.522 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:01.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:01.523 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:01.524 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:48:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:48:01] "GET /metrics HTTP/1.1" 200 224381 "" "Prometheus/2.43.0" 2024-09-11T08:48:01.625 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:01.654 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:01.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:01.655 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:01.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:01.789 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:01.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:01.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:01.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:01 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:48:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:01 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:48:01.890 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:01 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:48:01.926 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:01.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:01.927 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:02.027 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:02.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:02.070 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:02.070 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:02.171 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:02.201 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:02.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:02.202 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:02.207 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:02.272 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:48:02.272 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:48:02.301 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:48:02.301 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:48:02.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:02.336 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:48:02.336 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:48:02.350 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:02.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:02.351 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:02.366 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:48:02.366 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:48:02.397 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:48:02.398 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:48:02.429 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:48:02.430 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:48:02.452 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:02.464 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:48:02.465 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:48:02.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:02.486 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:02.486 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:02.494 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:48:02.495 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:48:02.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:02.619 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:02.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:02.620 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:02.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:48:02.700 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:02 smithi092 ceph-mon[29647]: pgmap v392: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 821 op/s 2024-09-11T08:48:02.895 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:02 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:48:02.895 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:02 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:48:02.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:02 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:48:02.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:02 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3919150819' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:48:02.896 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:02 smithi092 ceph-mon[41114]: pgmap v392: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 821 op/s 2024-09-11T08:48:02.896 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:02 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:48:02.896 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:02 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:48:02.896 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:02 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:48:02.896 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:02 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3919150819' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:48:02.897 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:02 smithi145 ceph-mon[33384]: pgmap v392: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 821 op/s 2024-09-11T08:48:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:02 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:48:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:02 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:48:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:02 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:48:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:02 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3919150819' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:48:02.912 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:48:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:48:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:48:02.925 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:02.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:02.926 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:03.026 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:03.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:03.062 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:03.062 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:03.163 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:03.198 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:03.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:03.199 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:03.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:03.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:03.346 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:03.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:03.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:03.486 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:03.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:03.487 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:03.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:03.621 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:03.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:03.622 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:03.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:03.759 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:03.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:03.760 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:03.861 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:03.887 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:03.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:03.889 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:03.989 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:04.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:04.027 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:04.027 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:04.128 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:04.163 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:04.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:04.164 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:04.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:04.304 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:04.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:04.305 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:04.406 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:04.445 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:04.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:04.446 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:04.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:04.588 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:04.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:04.589 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:04.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:04.716 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:04.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:04.717 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:04.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:04 smithi092 ceph-mon[29647]: pgmap v393: 148 pgs: 148 active+clean; 3.2 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T08:48:04.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:04 smithi092 ceph-mon[41114]: pgmap v393: 148 pgs: 148 active+clean; 3.2 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T08:48:04.818 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:04.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:04.846 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:04.847 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:04 smithi145 ceph-mon[33384]: pgmap v393: 148 pgs: 148 active+clean; 3.2 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T08:48:04.947 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:04.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:04.981 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:04.981 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:05.082 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:05.120 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:05.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:05.121 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:05.222 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:05.257 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:05.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:05.259 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:05.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:48:05.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:05.396 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:05.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:05.397 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:05.498 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:05.548 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:05.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:05.549 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:05.649 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:05.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:05.690 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:05.690 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:05.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:05.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:05.848 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:05.848 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:05.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:05.984 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:05.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:05.985 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:06.086 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:06.111 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:06.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:06.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:06.213 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:06.245 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:06.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:06.246 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:06.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:06.381 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:06.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:06.382 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:06.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:06.515 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:06.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:06.516 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:06.617 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:06.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:06.652 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:06.653 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:06.754 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:06.785 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:06.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:06.786 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:06.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:06 smithi092 ceph-mon[29647]: pgmap v394: 148 pgs: 148 active+clean; 3.2 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-11T08:48:06.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:06 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:48:06.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:06 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:48:06.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:06 smithi092 ceph-mon[41114]: pgmap v394: 148 pgs: 148 active+clean; 3.2 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-11T08:48:06.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:06 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:48:06.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:06 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:48:06.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:06.912 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:48:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:48:06.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:06 smithi145 ceph-mon[33384]: pgmap v394: 148 pgs: 148 active+clean; 3.2 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-11T08:48:06.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:06 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:48:06.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:06 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:48:06.918 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:06.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:06.919 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:07.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:07.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:07.057 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:07.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:07.191 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:07.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:07.193 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:07.293 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:07.333 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:07.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:07.334 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:07.435 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:07.466 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:07.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:07.467 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:07.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:07.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:07.600 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:07.600 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:07.701 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:07.727 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:07.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:07.728 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:07.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:07 smithi092 ceph-mon[29647]: osdmap e196: 8 total, 8 up, 8 in 2024-09-11T08:48:07.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:07 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:07.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:07 smithi092 ceph-mon[41114]: osdmap e196: 8 total, 8 up, 8 in 2024-09-11T08:48:07.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:07 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:07.829 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:07.868 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:07.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:07.869 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:07.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:07 smithi145 ceph-mon[33384]: osdmap e196: 8 total, 8 up, 8 in 2024-09-11T08:48:07.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:07 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:07.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:07.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:07.997 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:07.998 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:08.099 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:08.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:08.134 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:08.134 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:08.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:08.278 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:08.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:08.288 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:08.389 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:08.412 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:08.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:08.413 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:08.514 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:08.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:08.551 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:08.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:08.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:08.694 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:08.694 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:08.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:08.844 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:08.862 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:08.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:08.882 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:08.882 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:08 smithi145 ceph-mon[33384]: pgmap v396: 148 pgs: 148 active+clean; 2.9 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.48k op/s 2024-09-11T08:48:08.937 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:08 smithi092 ceph-mon[29647]: pgmap v396: 148 pgs: 148 active+clean; 2.9 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.48k op/s 2024-09-11T08:48:08.938 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:08 smithi092 ceph-mon[41114]: pgmap v396: 148 pgs: 148 active+clean; 2.9 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.48k op/s 2024-09-11T08:48:08.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:09.028 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:09.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:09.029 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:09.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:09.175 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:09.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:09.176 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:09.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:09.312 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:09.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:09.313 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:09.414 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:09.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:09.445 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:09.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:09.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:09.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:09.592 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:09.692 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:09.737 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:09.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:09.738 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:09.839 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:09.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:09.876 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:09.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:09.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:10.012 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:10.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:10.013 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:10.114 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:10.141 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:09 smithi092 ceph-mon[29647]: pgmap v397: 148 pgs: 148 active+clean; 2.9 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T08:48:10.141 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:09 smithi092 ceph-mon[41114]: pgmap v397: 148 pgs: 148 active+clean; 2.9 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T08:48:10.147 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:10.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:10.148 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:10.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:09 smithi145 ceph-mon[33384]: pgmap v397: 148 pgs: 148 active+clean; 2.9 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T08:48:10.249 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:10.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:10.282 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:10.283 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:10.383 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:10.416 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:10.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:10.417 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:10.518 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:10.553 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:10.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:10.554 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:10.655 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:10.689 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:10.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:10.690 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:10.790 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:10.822 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:10.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:10.823 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:10.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:10.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:10.963 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:10.964 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:10.993 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:48:10.993 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:48:11.065 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:11.106 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:11.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:11.107 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:11.208 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:11.232 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:11.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:11.233 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:11.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:11.370 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:11.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:11.371 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:11.472 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:11.500 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:11.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:11.501 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:11.538 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:48:11.538 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:48:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:48:11] "GET /metrics HTTP/1.1" 200 224381 "" "Prometheus/2.43.0" 2024-09-11T08:48:11.545 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:48:11.601 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:11.634 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:11.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:11.634 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:11.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:11.770 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:11.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:11.771 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:11.872 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:11.900 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:11.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:11.901 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:12.002 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:12.030 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:12.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:12.031 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:12.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:12.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:12.169 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:12.172 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:12.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:12.305 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:12.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:12.309 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:12.409 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:12.440 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:12 smithi092 ceph-mon[29647]: pgmap v398: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.19k op/s 2024-09-11T08:48:12.440 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:12 smithi092 ceph-mon[29647]: from='client.25115 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:48:12.441 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:12 smithi092 ceph-mon[41114]: pgmap v398: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.19k op/s 2024-09-11T08:48:12.441 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:12 smithi092 ceph-mon[41114]: from='client.25115 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:48:12.441 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:12.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:12.442 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:12.543 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:12.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:12.572 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:12.573 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:12.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:12 smithi145 ceph-mon[33384]: pgmap v398: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.19k op/s 2024-09-11T08:48:12.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:12 smithi145 ceph-mon[33384]: from='client.25115 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:48:12.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:12.712 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:12.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:12.713 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:12.814 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:12.844 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:12.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:12.846 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:12.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:12.988 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:12.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:12.990 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:13.091 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:13.118 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:13.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:13.119 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:13.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:48:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:48:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:48:13.220 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:13.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:13.265 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:13.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:13.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:13.409 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:13.409 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:13.510 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:13.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:13.540 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:13.540 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:13.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:13.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:13.681 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:13.681 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:13.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:13.821 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:13.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:13.822 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:13.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-11T08:48:13.923 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:13.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:13.953 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:13.953 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:14.054 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:14.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:14.111 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:14.112 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:14.212 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:14.249 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:14.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:14.251 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:14.351 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:14.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:14.381 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:14.381 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:14.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:14.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:14.517 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:14.517 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:14.517 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:14 smithi092 ceph-mon[29647]: pgmap v399: 148 pgs: 148 active+clean; 2.5 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-11T08:48:14.518 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:14 smithi092 ceph-mon[41114]: pgmap v399: 148 pgs: 148 active+clean; 2.5 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-11T08:48:14.618 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:14.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:14.646 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:14.647 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:14.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:14 smithi145 ceph-mon[33384]: pgmap v399: 148 pgs: 148 active+clean; 2.5 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-11T08:48:14.747 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:14.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:14.790 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:14.790 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:14.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:14.927 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:14.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:14.928 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:15.007 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-11T08:48:11.200430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3005487348,"num_objects":46048,"num_object_clones":0,"num_object_copies":138144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46048,"num_whiteouts":0,"num_read":1416,"num_read_kb":1273,"num_write":194609,"num_write_kb":7678984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7873,"num_bytes_recovered":509741200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86190,"ondisk_log_size":86190,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":446,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":11732100,"kb_used_data":9282240,"kb_used_omap":13,"kb_used_meta":2449458,"kb_avail":738229116,"statfs":{"total":767960285184,"available":755946614784,"internally_reserved":0,"allocated":9505013760,"data_stored":9502565960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2508245696},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-780009472,"num_objects":-11902,"num_object_clones":0,"num_object_copies":-35706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-11902,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11902,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003368"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":189,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.509109+0000","last_change":"2024-09-11T08:45:06.574593+0000","last_active":"2024-09-11T08:47:59.509109+0000","last_peered":"2024-09-11T08:47:59.509109+0000","last_clean":"2024-09-11T08:47:59.509109+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:47:59.509109+0000","last_undegraded":"2024-09-11T08:47:59.509109+0000","last_fullsized":"2024-09-11T08:47:59.509109+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:06.574539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.574539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028477630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":237,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.502840+0000","last_change":"2024-09-11T08:46:01.991434+0000","last_active":"2024-09-11T08:47:59.502840+0000","last_peered":"2024-09-11T08:47:59.502840+0000","last_clean":"2024-09-11T08:47:59.502840+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:47:59.502840+0000","last_undegraded":"2024-09-11T08:47:59.502840+0000","last_fullsized":"2024-09-11T08:47:59.502840+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:45:13.113315+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.113315+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018068470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":194,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.480201+0000","last_change":"2024-09-11T08:47:44.884292+0000","last_active":"2024-09-11T08:47:53.480201+0000","last_peered":"2024-09-11T08:47:53.480201+0000","last_clean":"2024-09-11T08:47:53.480201+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:47:53.480201+0000","last_undegraded":"2024-09-11T08:47:53.480201+0000","last_fullsized":"2024-09-11T08:47:53.480201+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.884217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.884217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:57.658768+0000","scrub_duration":0.011480829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":163,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.503732+0000","last_change":"2024-09-11T08:44:26.005322+0000","last_active":"2024-09-11T08:47:59.503732+0000","last_peered":"2024-09-11T08:47:59.503732+0000","last_clean":"2024-09-11T08:47:59.503732+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:47:59.503732+0000","last_undegraded":"2024-09-11T08:47:59.503732+0000","last_fullsized":"2024-09-11T08:47:59.503732+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.005270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.005270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038675070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":192,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.501653+0000","last_change":"2024-09-11T08:46:01.984446+0000","last_active":"2024-09-11T08:47:59.501653+0000","last_peered":"2024-09-11T08:47:59.501653+0000","last_clean":"2024-09-11T08:47:59.501653+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:47:59.501653+0000","last_undegraded":"2024-09-11T08:47:59.501653+0000","last_fullsized":"2024-09-11T08:47:59.501653+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:28.356172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.356172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029955870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":198,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504551+0000","last_change":"2024-09-11T08:46:01.981971+0000","last_active":"2024-09-11T08:47:59.504551+0000","last_peered":"2024-09-11T08:47:59.504551+0000","last_clean":"2024-09-11T08:47:59.504551+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:47:59.504551+0000","last_undegraded":"2024-09-11T08:47:59.504551+0000","last_fullsized":"2024-09-11T08:47:59.504551+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.330642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.330642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0087830849999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":208,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.501543+0000","last_change":"2024-09-11T08:44:24.344328+0000","last_active":"2024-09-11T08:47:59.501543+0000","last_peered":"2024-09-11T08:47:59.501543+0000","last_clean":"2024-09-11T08:47:59.501543+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:47:59.501543+0000","last_undegraded":"2024-09-11T08:47:59.501543+0000","last_fullsized":"2024-09-11T08:47:59.501543+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:44:24.344127+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.344127+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010996336000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":165,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.502771+0000","last_change":"2024-09-11T08:43:28.988751+0000","last_active":"2024-09-11T08:47:59.502771+0000","last_peered":"2024-09-11T08:47:59.502771+0000","last_clean":"2024-09-11T08:47:59.502771+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:47:59.502771+0000","last_undegraded":"2024-09-11T08:47:59.502771+0000","last_fullsized":"2024-09-11T08:47:59.502771+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:43:28.988697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00375019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":204,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.476435+0000","last_change":"2024-09-11T08:47:32.636647+0000","last_active":"2024-09-11T08:47:53.476435+0000","last_peered":"2024-09-11T08:47:53.476435+0000","last_clean":"2024-09-11T08:47:53.476435+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:47:53.476435+0000","last_undegraded":"2024-09-11T08:47:53.476435+0000","last_fullsized":"2024-09-11T08:47:53.476435+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:12.878132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.878132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077623500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":197,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.503995+0000","last_change":"2024-09-11T08:44:27.963772+0000","last_active":"2024-09-11T08:47:59.503995+0000","last_peered":"2024-09-11T08:47:59.503995+0000","last_clean":"2024-09-11T08:47:59.503995+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:47:59.503995+0000","last_undegraded":"2024-09-11T08:47:59.503995+0000","last_fullsized":"2024-09-11T08:47:59.503995+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:44:27.963689+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:44:27.963689+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011544877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":180,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.473386+0000","last_change":"2024-09-11T08:47:09.804752+0000","last_active":"2024-09-11T08:47:53.473386+0000","last_peered":"2024-09-11T08:47:53.473386+0000","last_clean":"2024-09-11T08:47:53.473386+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:47:53.473386+0000","last_undegraded":"2024-09-11T08:47:53.473386+0000","last_fullsized":"2024-09-11T08:47:53.473386+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:33.644869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:43:33.644869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020832210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":183,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.509349+0000","last_change":"2024-09-11T08:45:31.457775+0000","last_active":"2024-09-11T08:47:59.509349+0000","last_peered":"2024-09-11T08:47:59.509349+0000","last_clean":"2024-09-11T08:47:59.509349+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:47:59.509349+0000","last_undegraded":"2024-09-11T08:47:59.509349+0000","last_fullsized":"2024-09-11T08:47:59.509349+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.457652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.457652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006759045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":184,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.474820+0000","last_change":"2024-09-11T08:47:33.593768+0000","last_active":"2024-09-11T08:47:53.474820+0000","last_peered":"2024-09-11T08:47:53.474820+0000","last_clean":"2024-09-11T08:47:53.474820+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:47:53.474820+0000","last_undegraded":"2024-09-11T08:47:53.474820+0000","last_fullsized":"2024-09-11T08:47:53.474820+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:09.871637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:09.871637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029269349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":187,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504041+0000","last_change":"2024-09-11T08:47:44.866144+0000","last_active":"2024-09-11T08:47:59.504041+0000","last_peered":"2024-09-11T08:47:59.504041+0000","last_clean":"2024-09-11T08:47:59.504041+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:47:59.504041+0000","last_undegraded":"2024-09-11T08:47:59.504041+0000","last_fullsized":"2024-09-11T08:47:59.504041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:47:44.866057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.866057+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:03.406837+0000","scrub_duration":0.0020690169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":194,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505920+0000","last_change":"2024-09-11T08:47:46.410478+0000","last_active":"2024-09-11T08:47:59.505920+0000","last_peered":"2024-09-11T08:47:59.505920+0000","last_clean":"2024-09-11T08:47:59.505920+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:47:59.505920+0000","last_undegraded":"2024-09-11T08:47:59.505920+0000","last_fullsized":"2024-09-11T08:47:59.505920+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:46.410402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:47:46.410402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:51.920397+0000","scrub_duration":0.002594884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":178,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.474183+0000","last_change":"2024-09-11T08:47:38.519964+0000","last_active":"2024-09-11T08:47:53.474183+0000","last_peered":"2024-09-11T08:47:53.474183+0000","last_clean":"2024-09-11T08:47:53.474183+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:47:53.474183+0000","last_undegraded":"2024-09-11T08:47:53.474183+0000","last_fullsized":"2024-09-11T08:47:53.474183+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.850684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.850684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058755379999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":193,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504259+0000","last_change":"2024-09-11T08:46:01.978488+0000","last_active":"2024-09-11T08:47:59.504259+0000","last_peered":"2024-09-11T08:47:59.504259+0000","last_clean":"2024-09-11T08:47:59.504259+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:47:59.504259+0000","last_undegraded":"2024-09-11T08:47:59.504259+0000","last_fullsized":"2024-09-11T08:47:59.504259+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:24.197149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.197149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001510267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":189,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.507966+0000","last_change":"2024-09-11T08:47:34.610411+0000","last_active":"2024-09-11T08:47:59.507966+0000","last_peered":"2024-09-11T08:47:59.507966+0000","last_clean":"2024-09-11T08:47:59.507966+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:47:59.507966+0000","last_undegraded":"2024-09-11T08:47:59.507966+0000","last_fullsized":"2024-09-11T08:47:59.507966+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:34.610331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:34.610331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:51.178932+0000","scrub_duration":0.0021231090000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":233,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.479632+0000","last_change":"2024-09-11T08:47:40.327468+0000","last_active":"2024-09-11T08:47:53.479632+0000","last_peered":"2024-09-11T08:47:53.479632+0000","last_clean":"2024-09-11T08:47:53.479632+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:47:53.479632+0000","last_undegraded":"2024-09-11T08:47:53.479632+0000","last_fullsized":"2024-09-11T08:47:53.479632+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:44:18.678748+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:44:18.678748+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016068020999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":184,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.481212+0000","last_change":"2024-09-11T08:44:12.518229+0000","last_active":"2024-09-11T08:47:53.481212+0000","last_peered":"2024-09-11T08:47:53.481212+0000","last_clean":"2024-09-11T08:47:53.481212+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:47:53.481212+0000","last_undegraded":"2024-09-11T08:47:53.481212+0000","last_fullsized":"2024-09-11T08:47:53.481212+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:12.518151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.518151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013208541000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":201,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505886+0000","last_change":"2024-09-11T08:46:01.998163+0000","last_active":"2024-09-11T08:47:59.505886+0000","last_peered":"2024-09-11T08:47:59.505886+0000","last_clean":"2024-09-11T08:47:59.505886+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:47:59.505886+0000","last_undegraded":"2024-09-11T08:47:59.505886+0000","last_fullsized":"2024-09-11T08:47:59.505886+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:24.223237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.223237+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019917720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":179,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504150+0000","last_change":"2024-09-11T08:45:28.194198+0000","last_active":"2024-09-11T08:47:59.504150+0000","last_peered":"2024-09-11T08:47:59.504150+0000","last_clean":"2024-09-11T08:47:59.504150+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:47:59.504150+0000","last_undegraded":"2024-09-11T08:47:59.504150+0000","last_fullsized":"2024-09-11T08:47:59.504150+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.194095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.194095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001559936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":199,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504294+0000","last_change":"2024-09-11T08:45:16.162671+0000","last_active":"2024-09-11T08:47:59.504294+0000","last_peered":"2024-09-11T08:47:59.504294+0000","last_clean":"2024-09-11T08:47:59.504294+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:47:59.504294+0000","last_undegraded":"2024-09-11T08:47:59.504294+0000","last_fullsized":"2024-09-11T08:47:59.504294+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:16.162616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:16.162616+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001890833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":175,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.503726+0000","last_change":"2024-09-11T08:45:22.244097+0000","last_active":"2024-09-11T08:47:59.503726+0000","last_peered":"2024-09-11T08:47:59.503726+0000","last_clean":"2024-09-11T08:47:59.503726+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:47:59.503726+0000","last_undegraded":"2024-09-11T08:47:59.503726+0000","last_fullsized":"2024-09-11T08:47:59.503726+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.244038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.244038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015252620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":183,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.506122+0000","last_change":"2024-09-11T08:45:19.335163+0000","last_active":"2024-09-11T08:47:59.506122+0000","last_peered":"2024-09-11T08:47:59.506122+0000","last_clean":"2024-09-11T08:47:59.506122+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:47:59.506122+0000","last_undegraded":"2024-09-11T08:47:59.506122+0000","last_fullsized":"2024-09-11T08:47:59.506122+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:19.335065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:19.335065+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026052929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":188,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.480275+0000","last_change":"2024-09-11T08:44:33.758098+0000","last_active":"2024-09-11T08:47:53.480275+0000","last_peered":"2024-09-11T08:47:53.480275+0000","last_clean":"2024-09-11T08:47:53.480275+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:47:53.480275+0000","last_undegraded":"2024-09-11T08:47:53.480275+0000","last_fullsized":"2024-09-11T08:47:53.480275+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:44:33.758018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025150559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":195,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505329+0000","last_change":"2024-09-11T08:47:35.655231+0000","last_active":"2024-09-11T08:47:59.505329+0000","last_peered":"2024-09-11T08:47:59.505329+0000","last_clean":"2024-09-11T08:47:59.505329+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:47:59.505329+0000","last_undegraded":"2024-09-11T08:47:59.505329+0000","last_fullsized":"2024-09-11T08:47:59.505329+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:44:10.704471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:10.704471+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020557259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":309,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.577601+0000","last_change":"2024-09-11T08:45:08.310590+0000","last_active":"2024-09-11T08:48:06.577601+0000","last_peered":"2024-09-11T08:48:06.577601+0000","last_clean":"2024-09-11T08:48:06.577601+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:48:06.577601+0000","last_undegraded":"2024-09-11T08:48:06.577601+0000","last_fullsized":"2024-09-11T08:48:06.577601+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:08.310527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.310527+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021434650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"195'11905","reported_seq":11959,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.446984+0000","last_change":"2024-09-11T08:45:15.261892+0000","last_active":"2024-09-11T08:48:06.446984+0000","last_peered":"2024-09-11T08:48:06.446984+0000","last_clean":"2024-09-11T08:48:06.446984+0000","last_became_active":"2024-09-11T08:45:15.261608+0000","last_became_peered":"2024-09-11T08:45:15.261608+0000","last_unstale":"2024-09-11T08:48:06.446984+0000","last_undegraded":"2024-09-11T08:48:06.446984+0000","last_fullsized":"2024-09-11T08:48:06.446984+0000","mapping_epoch":149,"log_start":"183'6900","ondisk_log_start":"183'6900","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5005,"log_dups_size":0,"ondisk_log_size":5005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192872448,"num_objects":2943,"num_object_clones":0,"num_object_copies":8829,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11905,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"124'22","reported_seq":218,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.481098+0000","last_change":"2024-09-11T08:45:18.777528+0000","last_active":"2024-09-11T08:47:53.481098+0000","last_peered":"2024-09-11T08:47:53.481098+0000","last_clean":"2024-09-11T08:47:53.481098+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:47:53.481098+0000","last_undegraded":"2024-09-11T08:47:53.481098+0000","last_fullsized":"2024-09-11T08:47:53.481098+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:45:18.777466+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.777466+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045440580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":178,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.502474+0000","last_change":"2024-09-11T08:45:13.214007+0000","last_active":"2024-09-11T08:47:59.502474+0000","last_peered":"2024-09-11T08:47:59.502474+0000","last_clean":"2024-09-11T08:47:59.502474+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:47:59.502474+0000","last_undegraded":"2024-09-11T08:47:59.502474+0000","last_fullsized":"2024-09-11T08:47:59.502474+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:13.213922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.213922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016325280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"154'9","reported_seq":298,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.582210+0000","last_change":"2024-09-11T08:45:26.208887+0000","last_active":"2024-09-11T08:48:06.582210+0000","last_peered":"2024-09-11T08:48:06.582210+0000","last_clean":"2024-09-11T08:48:06.582210+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:48:06.582210+0000","last_undegraded":"2024-09-11T08:48:06.582210+0000","last_fullsized":"2024-09-11T08:48:06.582210+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'9","last_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.208818+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028933230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":175,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.473583+0000","last_change":"2024-09-11T08:47:39.523803+0000","last_active":"2024-09-11T08:47:53.473583+0000","last_peered":"2024-09-11T08:47:53.473583+0000","last_clean":"2024-09-11T08:47:53.473583+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:47:53.473583+0000","last_undegraded":"2024-09-11T08:47:53.473583+0000","last_fullsized":"2024-09-11T08:47:53.473583+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:24.813857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.813857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01118625,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"196'12134","reported_seq":12189,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.748325+0000","last_change":"2024-09-11T08:45:15.297710+0000","last_active":"2024-09-11T08:48:06.748325+0000","last_peered":"2024-09-11T08:48:06.748325+0000","last_clean":"2024-09-11T08:48:06.748325+0000","last_became_active":"2024-09-11T08:45:15.297378+0000","last_became_peered":"2024-09-11T08:45:15.297378+0000","last_unstale":"2024-09-11T08:48:06.748325+0000","last_undegraded":"2024-09-11T08:48:06.748325+0000","last_fullsized":"2024-09-11T08:48:06.748325+0000","mapping_epoch":149,"log_start":"180'6400","ondisk_log_start":"180'6400","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5734,"log_dups_size":0,"ondisk_log_size":5734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":194248704,"num_objects":2964,"num_object_clones":0,"num_object_copies":8892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12134,"num_write_kb":483136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":208,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.501890+0000","last_change":"2024-09-11T08:44:33.387066+0000","last_active":"2024-09-11T08:47:59.501890+0000","last_peered":"2024-09-11T08:47:59.501890+0000","last_clean":"2024-09-11T08:47:59.501890+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:47:59.501890+0000","last_undegraded":"2024-09-11T08:47:59.501890+0000","last_fullsized":"2024-09-11T08:47:59.501890+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-11T08:44:33.386972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:33.386972+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022646007999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":178,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.507452+0000","last_change":"2024-09-11T08:44:25.327692+0000","last_active":"2024-09-11T08:47:59.507452+0000","last_peered":"2024-09-11T08:47:59.507452+0000","last_clean":"2024-09-11T08:47:59.507452+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:47:59.507452+0000","last_undegraded":"2024-09-11T08:47:59.507452+0000","last_fullsized":"2024-09-11T08:47:59.507452+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:25.327553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:25.327553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016327754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":175,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.503201+0000","last_change":"2024-09-11T08:45:17.150827+0000","last_active":"2024-09-11T08:47:59.503201+0000","last_peered":"2024-09-11T08:47:59.503201+0000","last_clean":"2024-09-11T08:47:59.503201+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:47:59.503201+0000","last_undegraded":"2024-09-11T08:47:59.503201+0000","last_fullsized":"2024-09-11T08:47:59.503201+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.150767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.150767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001327707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":173,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.503910+0000","last_change":"2024-09-11T08:46:03.144149+0000","last_active":"2024-09-11T08:47:59.503910+0000","last_peered":"2024-09-11T08:47:59.503910+0000","last_clean":"2024-09-11T08:47:59.503910+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:47:59.503910+0000","last_undegraded":"2024-09-11T08:47:59.503910+0000","last_fullsized":"2024-09-11T08:47:59.503910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:46:03.144096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:46:03.144096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001497151,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"195'14347","reported_seq":14507,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.236015+0000","last_change":"2024-09-11T08:47:59.605543+0000","last_active":"2024-09-11T08:48:06.236015+0000","last_peered":"2024-09-11T08:48:06.236015+0000","last_clean":"2024-09-11T08:48:06.236015+0000","last_became_active":"2024-09-11T08:45:15.627210+0000","last_became_peered":"2024-09-11T08:45:15.627210+0000","last_unstale":"2024-09-11T08:48:06.236015+0000","last_undegraded":"2024-09-11T08:48:06.236015+0000","last_fullsized":"2024-09-11T08:48:06.236015+0000","mapping_epoch":149,"log_start":"185'8901","ondisk_log_start":"185'8901","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":2409,"log_size":5446,"log_dups_size":0,"ondisk_log_size":5446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189988888,"num_objects":2900,"num_object_clones":0,"num_object_copies":8700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2900,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11938,"num_write_kb":474753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"86'21","reported_seq":195,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.503424+0000","last_change":"2024-09-11T08:43:30.997804+0000","last_active":"2024-09-11T08:47:59.503424+0000","last_peered":"2024-09-11T08:47:59.503424+0000","last_clean":"2024-09-11T08:47:59.503424+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:47:59.503424+0000","last_undegraded":"2024-09-11T08:47:59.503424+0000","last_fullsized":"2024-09-11T08:47:59.503424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'21","last_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:43:30.997710+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025287299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":189,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505548+0000","last_change":"2024-09-11T08:45:30.475446+0000","last_active":"2024-09-11T08:47:59.505548+0000","last_peered":"2024-09-11T08:47:59.505548+0000","last_clean":"2024-09-11T08:47:59.505548+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:47:59.505548+0000","last_undegraded":"2024-09-11T08:47:59.505548+0000","last_fullsized":"2024-09-11T08:47:59.505548+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:30.475365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:30.475365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010690405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":288,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:48:02.791437+0000","last_change":"2024-09-11T08:44:11.198993+0000","last_active":"2024-09-11T08:48:02.791437+0000","last_peered":"2024-09-11T08:48:02.791437+0000","last_clean":"2024-09-11T08:48:02.791437+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:48:02.791437+0000","last_undegraded":"2024-09-11T08:48:02.791437+0000","last_fullsized":"2024-09-11T08:48:02.791437+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:44:11.198936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:11.198936+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019660649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":202,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.506020+0000","last_change":"2024-09-11T08:46:01.983654+0000","last_active":"2024-09-11T08:47:59.506020+0000","last_peered":"2024-09-11T08:47:59.506020+0000","last_clean":"2024-09-11T08:47:59.506020+0000","last_became_active":"2024-09-11T08:46:01.983335+0000","last_became_peered":"2024-09-11T08:46:01.983335+0000","last_unstale":"2024-09-11T08:47:59.506020+0000","last_undegraded":"2024-09-11T08:47:59.506020+0000","last_fullsized":"2024-09-11T08:47:59.506020+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.303194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.303194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001492431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"196'12002","reported_seq":12057,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.793051+0000","last_change":"2024-09-11T08:45:15.626487+0000","last_active":"2024-09-11T08:48:06.793051+0000","last_peered":"2024-09-11T08:48:06.793051+0000","last_clean":"2024-09-11T08:48:06.793051+0000","last_became_active":"2024-09-11T08:45:15.626210+0000","last_became_peered":"2024-09-11T08:45:15.626210+0000","last_unstale":"2024-09-11T08:48:06.793051+0000","last_undegraded":"2024-09-11T08:48:06.793051+0000","last_fullsized":"2024-09-11T08:48:06.793051+0000","mapping_epoch":149,"log_start":"180'6300","ondisk_log_start":"180'6300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":190709760,"num_objects":2910,"num_object_clones":0,"num_object_copies":8730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12002,"num_write_kb":477184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"100'15","reported_seq":206,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.480676+0000","last_change":"2024-09-11T08:45:12.769299+0000","last_active":"2024-09-11T08:47:53.480676+0000","last_peered":"2024-09-11T08:47:53.480676+0000","last_clean":"2024-09-11T08:47:53.480676+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:47:53.480676+0000","last_undegraded":"2024-09-11T08:47:53.480676+0000","last_fullsized":"2024-09-11T08:47:53.480676+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'15","last_scrub_stamp":"2024-09-11T08:45:12.769239+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:41:56.250393+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.769239+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026298300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":184,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.474350+0000","last_change":"2024-09-11T08:47:37.534624+0000","last_active":"2024-09-11T08:47:53.474350+0000","last_peered":"2024-09-11T08:47:53.474350+0000","last_clean":"2024-09-11T08:47:53.474350+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:47:53.474350+0000","last_undegraded":"2024-09-11T08:47:53.474350+0000","last_fullsized":"2024-09-11T08:47:53.474350+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:19.905459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:19.905459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015766237999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":296,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:48:02.786301+0000","last_change":"2024-09-11T08:44:34.630457+0000","last_active":"2024-09-11T08:48:02.786301+0000","last_peered":"2024-09-11T08:48:02.786301+0000","last_clean":"2024-09-11T08:48:02.786301+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:48:02.786301+0000","last_undegraded":"2024-09-11T08:48:02.786301+0000","last_fullsized":"2024-09-11T08:48:02.786301+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:44:34.630191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:34.630191+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089275420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":176,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.509400+0000","last_change":"2024-09-11T08:47:42.685960+0000","last_active":"2024-09-11T08:47:59.509400+0000","last_peered":"2024-09-11T08:47:59.509400+0000","last_clean":"2024-09-11T08:47:59.509400+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:47:59.509400+0000","last_undegraded":"2024-09-11T08:47:59.509400+0000","last_fullsized":"2024-09-11T08:47:59.509400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:16.681036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:44:16.681036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047481509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"196'12015","reported_seq":12070,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.726968+0000","last_change":"2024-09-11T08:45:15.627960+0000","last_active":"2024-09-11T08:48:06.726968+0000","last_peered":"2024-09-11T08:48:06.726968+0000","last_clean":"2024-09-11T08:48:06.726968+0000","last_became_active":"2024-09-11T08:45:15.627639+0000","last_became_peered":"2024-09-11T08:45:15.627639+0000","last_unstale":"2024-09-11T08:48:06.726968+0000","last_undegraded":"2024-09-11T08:48:06.726968+0000","last_fullsized":"2024-09-11T08:48:06.726968+0000","mapping_epoch":149,"log_start":"183'7100","ondisk_log_start":"183'7100","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4915,"log_dups_size":0,"ondisk_log_size":4915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":190906368,"num_objects":2913,"num_object_clones":0,"num_object_copies":8739,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12015,"num_write_kb":477696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":213,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.507763+0000","last_change":"2024-09-11T08:45:11.576184+0000","last_active":"2024-09-11T08:47:59.507763+0000","last_peered":"2024-09-11T08:47:59.507763+0000","last_clean":"2024-09-11T08:47:59.507763+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:47:59.507763+0000","last_undegraded":"2024-09-11T08:47:59.507763+0000","last_fullsized":"2024-09-11T08:47:59.507763+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:45:11.576085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.576085+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002366754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":188,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.480862+0000","last_change":"2024-09-11T08:44:30.802099+0000","last_active":"2024-09-11T08:47:53.480862+0000","last_peered":"2024-09-11T08:47:53.480862+0000","last_clean":"2024-09-11T08:47:53.480862+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:47:53.480862+0000","last_undegraded":"2024-09-11T08:47:53.480862+0000","last_fullsized":"2024-09-11T08:47:53.480862+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:30.800097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:30.800097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071432789999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":184,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504865+0000","last_change":"2024-09-11T08:45:07.264895+0000","last_active":"2024-09-11T08:47:59.504865+0000","last_peered":"2024-09-11T08:47:59.504865+0000","last_clean":"2024-09-11T08:47:59.504865+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:47:59.504865+0000","last_undegraded":"2024-09-11T08:47:59.504865+0000","last_fullsized":"2024-09-11T08:47:59.504865+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.264822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.264822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014425250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":202,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.480141+0000","last_change":"2024-09-11T08:46:01.985315+0000","last_active":"2024-09-11T08:47:53.480141+0000","last_peered":"2024-09-11T08:47:53.480141+0000","last_clean":"2024-09-11T08:47:53.480141+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:47:53.480141+0000","last_undegraded":"2024-09-11T08:47:53.480141+0000","last_fullsized":"2024-09-11T08:47:53.480141+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:07.805220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:45:07.805220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012010009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"195'12072","reported_seq":12126,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.200261+0000","last_change":"2024-09-11T08:45:15.298396+0000","last_active":"2024-09-11T08:48:06.200261+0000","last_peered":"2024-09-11T08:48:06.200261+0000","last_clean":"2024-09-11T08:48:06.200261+0000","last_became_active":"2024-09-11T08:45:15.298053+0000","last_became_peered":"2024-09-11T08:45:15.298053+0000","last_unstale":"2024-09-11T08:48:06.200261+0000","last_undegraded":"2024-09-11T08:48:06.200261+0000","last_fullsized":"2024-09-11T08:48:06.200261+0000","mapping_epoch":149,"log_start":"183'6700","ondisk_log_start":"183'6700","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5372,"log_dups_size":0,"ondisk_log_size":5372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":194379776,"num_objects":2966,"num_object_clones":0,"num_object_copies":8898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12072,"num_write_kb":481216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":367,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.498572+0000","last_change":"2024-09-11T08:45:26.666261+0000","last_active":"2024-09-11T08:47:59.498572+0000","last_peered":"2024-09-11T08:47:59.498572+0000","last_clean":"2024-09-11T08:47:59.498572+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:47:59.498572+0000","last_undegraded":"2024-09-11T08:47:59.498572+0000","last_fullsized":"2024-09-11T08:47:59.498572+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:45:26.666199+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.666199+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020004279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":179,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504398+0000","last_change":"2024-09-11T08:45:15.130637+0000","last_active":"2024-09-11T08:47:59.504398+0000","last_peered":"2024-09-11T08:47:59.504398+0000","last_clean":"2024-09-11T08:47:59.504398+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:47:59.504398+0000","last_undegraded":"2024-09-11T08:47:59.504398+0000","last_fullsized":"2024-09-11T08:47:59.504398+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.130572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.130572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019779749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":219,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.474706+0000","last_change":"2024-09-11T08:47:02.718120+0000","last_active":"2024-09-11T08:47:53.474706+0000","last_peered":"2024-09-11T08:47:53.474706+0000","last_clean":"2024-09-11T08:47:53.474706+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:47:53.474706+0000","last_undegraded":"2024-09-11T08:47:53.474706+0000","last_fullsized":"2024-09-11T08:47:53.474706+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:43:38.669837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:43:38.669837+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001723715,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":178,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.508567+0000","last_change":"2024-09-11T08:47:38.663094+0000","last_active":"2024-09-11T08:47:59.508567+0000","last_peered":"2024-09-11T08:47:59.508567+0000","last_clean":"2024-09-11T08:47:59.508567+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:47:59.508567+0000","last_undegraded":"2024-09-11T08:47:59.508567+0000","last_fullsized":"2024-09-11T08:47:59.508567+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:14.679598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:14.679598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011950855999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":172,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504241+0000","last_change":"2024-09-11T08:45:22.169436+0000","last_active":"2024-09-11T08:47:59.504241+0000","last_peered":"2024-09-11T08:47:59.504241+0000","last_clean":"2024-09-11T08:47:59.504241+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:47:59.504241+0000","last_undegraded":"2024-09-11T08:47:59.504241+0000","last_fullsized":"2024-09-11T08:47:59.504241+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.169367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.169367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005349693,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"196'12073","reported_seq":12127,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.711879+0000","last_change":"2024-09-11T08:45:15.261656+0000","last_active":"2024-09-11T08:48:06.711879+0000","last_peered":"2024-09-11T08:48:06.711879+0000","last_clean":"2024-09-11T08:48:06.711879+0000","last_became_active":"2024-09-11T08:45:15.261370+0000","last_became_peered":"2024-09-11T08:45:15.261370+0000","last_unstale":"2024-09-11T08:48:06.711879+0000","last_undegraded":"2024-09-11T08:48:06.711879+0000","last_fullsized":"2024-09-11T08:48:06.711879+0000","mapping_epoch":149,"log_start":"183'7200","ondisk_log_start":"183'7200","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4873,"log_dups_size":0,"ondisk_log_size":4873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":191299584,"num_objects":2919,"num_object_clones":0,"num_object_copies":8757,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12073,"num_write_kb":479744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":223,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.481262+0000","last_change":"2024-09-11T08:45:27.666590+0000","last_active":"2024-09-11T08:47:53.481262+0000","last_peered":"2024-09-11T08:47:53.481262+0000","last_clean":"2024-09-11T08:47:53.481262+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:47:53.481262+0000","last_undegraded":"2024-09-11T08:47:53.481262+0000","last_fullsized":"2024-09-11T08:47:53.481262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:45:27.666502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.666502+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023582730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":208,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.480295+0000","last_change":"2024-09-11T08:46:01.982031+0000","last_active":"2024-09-11T08:47:53.480295+0000","last_peered":"2024-09-11T08:47:53.480295+0000","last_clean":"2024-09-11T08:47:53.480295+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:47:53.480295+0000","last_undegraded":"2024-09-11T08:47:53.480295+0000","last_fullsized":"2024-09-11T08:47:53.480295+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.775804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.775804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011899289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":207,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.478865+0000","last_change":"2024-09-11T08:47:36.417328+0000","last_active":"2024-09-11T08:47:53.478865+0000","last_peered":"2024-09-11T08:47:53.478865+0000","last_clean":"2024-09-11T08:47:53.478865+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:47:53.478865+0000","last_undegraded":"2024-09-11T08:47:53.478865+0000","last_fullsized":"2024-09-11T08:47:53.478865+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:45:32.632242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:32.632242+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049221250000000003,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":203,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.479086+0000","last_change":"2024-09-11T08:47:44.344421+0000","last_active":"2024-09-11T08:47:53.479086+0000","last_peered":"2024-09-11T08:47:53.479086+0000","last_clean":"2024-09-11T08:47:53.479086+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:47:53.479086+0000","last_undegraded":"2024-09-11T08:47:53.479086+0000","last_fullsized":"2024-09-11T08:47:53.479086+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.344206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:48.790110+0000","scrub_duration":0.0085971020000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":202,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.508393+0000","last_change":"2024-09-11T08:46:01.984350+0000","last_active":"2024-09-11T08:47:59.508393+0000","last_peered":"2024-09-11T08:47:59.508393+0000","last_clean":"2024-09-11T08:47:59.508393+0000","last_became_active":"2024-09-11T08:46:01.983095+0000","last_became_peered":"2024-09-11T08:46:01.983095+0000","last_unstale":"2024-09-11T08:47:59.508393+0000","last_undegraded":"2024-09-11T08:47:59.508393+0000","last_fullsized":"2024-09-11T08:47:59.508393+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:12.278185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.278185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015184790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"195'12414","reported_seq":12468,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.468161+0000","last_change":"2024-09-11T08:45:15.627633+0000","last_active":"2024-09-11T08:48:06.468161+0000","last_peered":"2024-09-11T08:48:06.468161+0000","last_clean":"2024-09-11T08:48:06.468161+0000","last_became_active":"2024-09-11T08:45:15.627360+0000","last_became_peered":"2024-09-11T08:45:15.627360+0000","last_unstale":"2024-09-11T08:48:06.468161+0000","last_undegraded":"2024-09-11T08:48:06.468161+0000","last_fullsized":"2024-09-11T08:48:06.468161+0000","mapping_epoch":149,"log_start":"183'7400","ondisk_log_start":"183'7400","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5014,"log_dups_size":0,"ondisk_log_size":5014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":191102976,"num_objects":2916,"num_object_clones":0,"num_object_copies":8748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12414,"num_write_kb":490560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":195,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505404+0000","last_change":"2024-09-11T08:44:15.699227+0000","last_active":"2024-09-11T08:47:59.505404+0000","last_peered":"2024-09-11T08:47:59.505404+0000","last_clean":"2024-09-11T08:47:59.505404+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:47:59.505404+0000","last_undegraded":"2024-09-11T08:47:59.505404+0000","last_fullsized":"2024-09-11T08:47:59.505404+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:44:15.699067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:15.699067+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014313306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":189,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505600+0000","last_change":"2024-09-11T08:45:20.455183+0000","last_active":"2024-09-11T08:47:59.505600+0000","last_peered":"2024-09-11T08:47:59.505600+0000","last_clean":"2024-09-11T08:47:59.505600+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:47:59.505600+0000","last_undegraded":"2024-09-11T08:47:59.505600+0000","last_fullsized":"2024-09-11T08:47:59.505600+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:20.455081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.455081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016372450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":771,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-11T08:48:09.867341+0000","last_change":"2024-09-11T08:45:21.370742+0000","last_active":"2024-09-11T08:48:09.867341+0000","last_peered":"2024-09-11T08:48:09.867341+0000","last_clean":"2024-09-11T08:48:09.867341+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:48:09.867341+0000","last_undegraded":"2024-09-11T08:48:09.867341+0000","last_fullsized":"2024-09-11T08:48:09.867341+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:45:21.370673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.370673+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017307240000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":565,"num_read_kb":560,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":180,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.473702+0000","last_change":"2024-09-11T08:47:20.796422+0000","last_active":"2024-09-11T08:47:53.473702+0000","last_peered":"2024-09-11T08:47:53.473702+0000","last_clean":"2024-09-11T08:47:53.473702+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:47:53.473702+0000","last_undegraded":"2024-09-11T08:47:53.473702+0000","last_fullsized":"2024-09-11T08:47:53.473702+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:16.924404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:16.924404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024072690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":182,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.480660+0000","last_change":"2024-09-11T08:45:17.819757+0000","last_active":"2024-09-11T08:47:53.480660+0000","last_peered":"2024-09-11T08:47:53.480660+0000","last_clean":"2024-09-11T08:47:53.480660+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:47:53.480660+0000","last_undegraded":"2024-09-11T08:47:53.480660+0000","last_fullsized":"2024-09-11T08:47:53.480660+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.819707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.819707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012611440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"196'12591","reported_seq":12644,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-11T08:48:11.115702+0000","last_change":"2024-09-11T08:45:15.262738+0000","last_active":"2024-09-11T08:48:11.115702+0000","last_peered":"2024-09-11T08:48:11.115702+0000","last_clean":"2024-09-11T08:48:11.115702+0000","last_became_active":"2024-09-11T08:45:15.262440+0000","last_became_peered":"2024-09-11T08:45:15.262440+0000","last_unstale":"2024-09-11T08:48:11.115702+0000","last_undegraded":"2024-09-11T08:48:11.115702+0000","last_fullsized":"2024-09-11T08:48:11.115702+0000","mapping_epoch":149,"log_start":"180'6300","ondisk_log_start":"180'6300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":6291,"log_dups_size":0,"ondisk_log_size":6291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172949504,"num_objects":2639,"num_object_clones":0,"num_object_copies":7917,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12591,"num_write_kb":487360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":64,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.506105+0000","last_change":"2024-09-11T08:47:45.585212+0000","last_active":"2024-09-11T08:47:59.506105+0000","last_peered":"2024-09-11T08:47:59.506105+0000","last_clean":"2024-09-11T08:47:59.506105+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:47:59.506105+0000","last_undegraded":"2024-09-11T08:47:59.506105+0000","last_fullsized":"2024-09-11T08:47:59.506105+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:47:45.585135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:45.585135+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:47.607612+0000","scrub_duration":0.0080208199999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":187,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.501926+0000","last_change":"2024-09-11T08:45:18.127946+0000","last_active":"2024-09-11T08:47:59.501926+0000","last_peered":"2024-09-11T08:47:59.501926+0000","last_clean":"2024-09-11T08:47:59.501926+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:47:59.501926+0000","last_undegraded":"2024-09-11T08:47:59.501926+0000","last_fullsized":"2024-09-11T08:47:59.501926+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:18.127857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.127857+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018334099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":232,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505644+0000","last_change":"2024-09-11T08:45:29.524681+0000","last_active":"2024-09-11T08:47:59.505644+0000","last_peered":"2024-09-11T08:47:59.505644+0000","last_clean":"2024-09-11T08:47:59.505644+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:47:59.505644+0000","last_undegraded":"2024-09-11T08:47:59.505644+0000","last_fullsized":"2024-09-11T08:47:59.505644+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'79","last_scrub_stamp":"2024-09-11T08:45:29.524608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:45:29.524608+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002308867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":194,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.506323+0000","last_change":"2024-09-11T08:47:35.433723+0000","last_active":"2024-09-11T08:47:59.506323+0000","last_peered":"2024-09-11T08:47:59.506323+0000","last_clean":"2024-09-11T08:47:59.506323+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:47:59.506323+0000","last_undegraded":"2024-09-11T08:47:59.506323+0000","last_fullsized":"2024-09-11T08:47:59.506323+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:35.433651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:35.433651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:37.450817+0000","scrub_duration":0.013907892999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":178,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.473577+0000","last_change":"2024-09-11T08:47:12.832716+0000","last_active":"2024-09-11T08:47:53.473577+0000","last_peered":"2024-09-11T08:47:53.473577+0000","last_clean":"2024-09-11T08:47:53.473577+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:47:53.473577+0000","last_undegraded":"2024-09-11T08:47:53.473577+0000","last_fullsized":"2024-09-11T08:47:53.473577+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.570246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.570246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041809610000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"196'12554","reported_seq":12609,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-11T08:48:11.050097+0000","last_change":"2024-09-11T08:45:15.263087+0000","last_active":"2024-09-11T08:48:11.050097+0000","last_peered":"2024-09-11T08:48:11.050097+0000","last_clean":"2024-09-11T08:48:11.050097+0000","last_became_active":"2024-09-11T08:45:15.262698+0000","last_became_peered":"2024-09-11T08:45:15.262698+0000","last_unstale":"2024-09-11T08:48:11.050097+0000","last_undegraded":"2024-09-11T08:48:11.050097+0000","last_fullsized":"2024-09-11T08:48:11.050097+0000","mapping_epoch":149,"log_start":"183'7700","ondisk_log_start":"183'7700","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172359680,"num_objects":2630,"num_object_clones":0,"num_object_copies":7890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12554,"num_write_kb":485888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":255,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504465+0000","last_change":"2024-09-11T08:47:32.052566+0000","last_active":"2024-09-11T08:47:59.504465+0000","last_peered":"2024-09-11T08:47:59.504465+0000","last_clean":"2024-09-11T08:47:59.504465+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:47:59.504465+0000","last_undegraded":"2024-09-11T08:47:59.504465+0000","last_fullsized":"2024-09-11T08:47:59.504465+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:21.002140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:21.002140+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025090400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":169,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.503527+0000","last_change":"2024-09-11T08:44:22.010602+0000","last_active":"2024-09-11T08:47:59.503527+0000","last_peered":"2024-09-11T08:47:59.503527+0000","last_clean":"2024-09-11T08:47:59.503527+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:47:59.503527+0000","last_undegraded":"2024-09-11T08:47:59.503527+0000","last_fullsized":"2024-09-11T08:47:59.503527+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.010528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.010528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047628439999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":187,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.473455+0000","last_change":"2024-09-11T08:47:41.425451+0000","last_active":"2024-09-11T08:47:53.473455+0000","last_peered":"2024-09-11T08:47:53.473455+0000","last_clean":"2024-09-11T08:47:53.473455+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:47:53.473455+0000","last_undegraded":"2024-09-11T08:47:53.473455+0000","last_fullsized":"2024-09-11T08:47:53.473455+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.726573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00117395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":192,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504174+0000","last_change":"2024-09-11T08:46:01.988315+0000","last_active":"2024-09-11T08:47:59.504174+0000","last_peered":"2024-09-11T08:47:59.504174+0000","last_clean":"2024-09-11T08:47:59.504174+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:47:59.504174+0000","last_undegraded":"2024-09-11T08:47:59.504174+0000","last_fullsized":"2024-09-11T08:47:59.504174+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.167071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.167071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001796483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"195'11903","reported_seq":11957,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.477226+0000","last_change":"2024-09-11T08:45:15.626015+0000","last_active":"2024-09-11T08:48:06.477226+0000","last_peered":"2024-09-11T08:48:06.477226+0000","last_clean":"2024-09-11T08:48:06.477226+0000","last_became_active":"2024-09-11T08:45:15.625726+0000","last_became_peered":"2024-09-11T08:45:15.625726+0000","last_unstale":"2024-09-11T08:48:06.477226+0000","last_undegraded":"2024-09-11T08:48:06.477226+0000","last_fullsized":"2024-09-11T08:48:06.477226+0000","mapping_epoch":149,"log_start":"183'6900","ondisk_log_start":"183'6900","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188284928,"num_objects":2873,"num_object_clones":0,"num_object_copies":8619,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11903,"num_write_kb":472832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":205,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504879+0000","last_change":"2024-09-11T08:44:28.940078+0000","last_active":"2024-09-11T08:47:59.504879+0000","last_peered":"2024-09-11T08:47:59.504879+0000","last_clean":"2024-09-11T08:47:59.504879+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:47:59.504879+0000","last_undegraded":"2024-09-11T08:47:59.504879+0000","last_fullsized":"2024-09-11T08:47:59.504879+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:44:28.940023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.940023+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012856796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":198,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505123+0000","last_change":"2024-09-11T08:45:25.273200+0000","last_active":"2024-09-11T08:47:59.505123+0000","last_peered":"2024-09-11T08:47:59.505123+0000","last_clean":"2024-09-11T08:47:59.505123+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:47:59.505123+0000","last_undegraded":"2024-09-11T08:47:59.505123+0000","last_fullsized":"2024-09-11T08:47:59.505123+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.273134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014307370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":184,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.480692+0000","last_change":"2024-09-11T08:44:24.816974+0000","last_active":"2024-09-11T08:47:53.480692+0000","last_peered":"2024-09-11T08:47:53.480692+0000","last_clean":"2024-09-11T08:47:53.480692+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:47:53.480692+0000","last_undegraded":"2024-09-11T08:47:53.480692+0000","last_fullsized":"2024-09-11T08:47:53.480692+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:24.816890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.816890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086069140000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":197,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.479977+0000","last_change":"2024-09-11T08:47:38.366007+0000","last_active":"2024-09-11T08:47:53.479977+0000","last_peered":"2024-09-11T08:47:53.479977+0000","last_clean":"2024-09-11T08:47:53.479977+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:47:53.479977+0000","last_undegraded":"2024-09-11T08:47:53.479977+0000","last_fullsized":"2024-09-11T08:47:53.479977+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:31.756166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.756166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059059109999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"196'12123","reported_seq":12178,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.706402+0000","last_change":"2024-09-11T08:45:15.260658+0000","last_active":"2024-09-11T08:48:06.706402+0000","last_peered":"2024-09-11T08:48:06.706402+0000","last_clean":"2024-09-11T08:48:06.706402+0000","last_became_active":"2024-09-11T08:45:15.260383+0000","last_became_peered":"2024-09-11T08:45:15.260383+0000","last_unstale":"2024-09-11T08:48:06.706402+0000","last_undegraded":"2024-09-11T08:48:06.706402+0000","last_fullsized":"2024-09-11T08:48:06.706402+0000","mapping_epoch":149,"log_start":"183'7200","ondisk_log_start":"183'7200","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4923,"log_dups_size":0,"ondisk_log_size":4923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189202432,"num_objects":2887,"num_object_clones":0,"num_object_copies":8661,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12123,"num_write_kb":480320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":229,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.503800+0000","last_change":"2024-09-11T08:47:35.011515+0000","last_active":"2024-09-11T08:47:59.503800+0000","last_peered":"2024-09-11T08:47:59.503800+0000","last_clean":"2024-09-11T08:47:59.503800+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:47:59.503800+0000","last_undegraded":"2024-09-11T08:47:59.503800+0000","last_fullsized":"2024-09-11T08:47:59.503800+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:43:24.983740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:24.983740+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069750209999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":178,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504338+0000","last_change":"2024-09-11T08:45:32.219463+0000","last_active":"2024-09-11T08:47:59.504338+0000","last_peered":"2024-09-11T08:47:59.504338+0000","last_clean":"2024-09-11T08:47:59.504338+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:47:59.504338+0000","last_undegraded":"2024-09-11T08:47:59.504338+0000","last_fullsized":"2024-09-11T08:47:59.504338+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:32.219374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:32.219374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069622119999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":324,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.528666+0000","last_change":"2024-09-11T08:47:47.542558+0000","last_active":"2024-09-11T08:47:59.528666+0000","last_peered":"2024-09-11T08:47:59.528666+0000","last_clean":"2024-09-11T08:47:59.528666+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:47:59.528666+0000","last_undegraded":"2024-09-11T08:47:59.528666+0000","last_fullsized":"2024-09-11T08:47:59.528666+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:47:47.542483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:47.542483+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:17.503546+0000","scrub_duration":0.15193762999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":173,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504180+0000","last_change":"2024-09-11T08:45:31.196687+0000","last_active":"2024-09-11T08:47:59.504180+0000","last_peered":"2024-09-11T08:47:59.504180+0000","last_clean":"2024-09-11T08:47:59.504180+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:47:59.504180+0000","last_undegraded":"2024-09-11T08:47:59.504180+0000","last_fullsized":"2024-09-11T08:47:59.504180+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.196599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.196599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025248190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"195'12206","reported_seq":20138,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.472710+0000","last_change":"2024-09-11T08:47:45.439155+0000","last_active":"2024-09-11T08:48:06.472710+0000","last_peered":"2024-09-11T08:48:06.472710+0000","last_clean":"2024-09-11T08:48:06.472710+0000","last_became_active":"2024-09-11T08:47:45.422820+0000","last_became_peered":"2024-09-11T08:47:45.422820+0000","last_unstale":"2024-09-11T08:48:06.472710+0000","last_undegraded":"2024-09-11T08:48:06.472710+0000","last_fullsized":"2024-09-11T08:48:06.472710+0000","mapping_epoch":189,"log_start":"183'7200","ondisk_log_start":"183'7200","created":149,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5006,"log_dups_size":0,"ondisk_log_size":5006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197263360,"num_objects":3010,"num_object_clones":0,"num_object_copies":9030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12206,"num_write_kb":486912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3929,"num_bytes_recovered":257490944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":193,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.507692+0000","last_change":"2024-09-11T08:44:31.662275+0000","last_active":"2024-09-11T08:47:59.507692+0000","last_peered":"2024-09-11T08:47:59.507692+0000","last_clean":"2024-09-11T08:47:59.507692+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:47:59.507692+0000","last_undegraded":"2024-09-11T08:47:59.507692+0000","last_fullsized":"2024-09-11T08:47:59.507692+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:31.662190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.662190+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040714619999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":191,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.475019+0000","last_change":"2024-09-11T08:47:43.360558+0000","last_active":"2024-09-11T08:47:53.475019+0000","last_peered":"2024-09-11T08:47:53.475019+0000","last_clean":"2024-09-11T08:47:53.475019+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:47:53.475019+0000","last_undegraded":"2024-09-11T08:47:53.475019+0000","last_fullsized":"2024-09-11T08:47:53.475019+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:10.607288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:45:10.607288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016158909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":187,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.480696+0000","last_change":"2024-09-11T08:47:30.449632+0000","last_active":"2024-09-11T08:47:53.480696+0000","last_peered":"2024-09-11T08:47:53.480696+0000","last_clean":"2024-09-11T08:47:53.480696+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:47:53.480696+0000","last_undegraded":"2024-09-11T08:47:53.480696+0000","last_fullsized":"2024-09-11T08:47:53.480696+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:12.645956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.645956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014834379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":183,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505536+0000","last_change":"2024-09-11T08:47:47.699529+0000","last_active":"2024-09-11T08:47:59.505536+0000","last_peered":"2024-09-11T08:47:59.505536+0000","last_clean":"2024-09-11T08:47:59.505536+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:47:59.505536+0000","last_undegraded":"2024-09-11T08:47:59.505536+0000","last_fullsized":"2024-09-11T08:47:59.505536+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:47.699453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:47:47.699453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:55.018390+0000","scrub_duration":0.016178818000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"196'11814","reported_seq":11869,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.786165+0000","last_change":"2024-09-11T08:45:15.297737+0000","last_active":"2024-09-11T08:48:06.786165+0000","last_peered":"2024-09-11T08:48:06.786165+0000","last_clean":"2024-09-11T08:48:06.786165+0000","last_became_active":"2024-09-11T08:45:15.297420+0000","last_became_peered":"2024-09-11T08:45:15.297420+0000","last_unstale":"2024-09-11T08:48:06.786165+0000","last_undegraded":"2024-09-11T08:48:06.786165+0000","last_fullsized":"2024-09-11T08:48:06.786165+0000","mapping_epoch":149,"log_start":"180'6100","ondisk_log_start":"180'6100","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5714,"log_dups_size":0,"ondisk_log_size":5714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185991168,"num_objects":2838,"num_object_clones":0,"num_object_copies":8514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11814,"num_write_kb":468864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":68,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504939+0000","last_change":"2024-09-11T08:47:46.582476+0000","last_active":"2024-09-11T08:47:59.504939+0000","last_peered":"2024-09-11T08:47:59.504939+0000","last_clean":"2024-09-11T08:47:59.504939+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:47:59.504939+0000","last_undegraded":"2024-09-11T08:47:59.504939+0000","last_fullsized":"2024-09-11T08:47:59.504939+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:47:46.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:46.582408+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:56.458790+0000","scrub_duration":0.006189015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":198,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.508521+0000","last_change":"2024-09-11T08:47:44.411099+0000","last_active":"2024-09-11T08:47:59.508521+0000","last_peered":"2024-09-11T08:47:59.508521+0000","last_clean":"2024-09-11T08:47:59.508521+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:47:59.508521+0000","last_undegraded":"2024-09-11T08:47:59.508521+0000","last_fullsized":"2024-09-11T08:47:59.508521+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.411022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.411022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:47.936745+0000","scrub_duration":0.0047762170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":203,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.479024+0000","last_change":"2024-09-11T08:47:43.361569+0000","last_active":"2024-09-11T08:47:53.479024+0000","last_peered":"2024-09-11T08:47:53.479024+0000","last_clean":"2024-09-11T08:47:53.479024+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:47:53.479024+0000","last_undegraded":"2024-09-11T08:47:53.479024+0000","last_fullsized":"2024-09-11T08:47:53.479024+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:43.361491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:43.361491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:11.065199+0000","scrub_duration":0.0055115299999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":175,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505565+0000","last_change":"2024-09-11T08:47:41.382331+0000","last_active":"2024-09-11T08:47:59.505565+0000","last_peered":"2024-09-11T08:47:59.505565+0000","last_clean":"2024-09-11T08:47:59.505565+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:47:59.505565+0000","last_undegraded":"2024-09-11T08:47:59.505565+0000","last_fullsized":"2024-09-11T08:47:59.505565+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:23.367786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:23.367786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030117270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"196'12191","reported_seq":19946,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-11T08:48:11.137514+0000","last_change":"2024-09-11T08:47:33.390084+0000","last_active":"2024-09-11T08:48:11.137514+0000","last_peered":"2024-09-11T08:48:11.137514+0000","last_clean":"2024-09-11T08:48:11.137514+0000","last_became_active":"2024-09-11T08:47:33.341629+0000","last_became_peered":"2024-09-11T08:47:33.341629+0000","last_unstale":"2024-09-11T08:48:11.137514+0000","last_undegraded":"2024-09-11T08:48:11.137514+0000","last_fullsized":"2024-09-11T08:48:11.137514+0000","mapping_epoch":187,"log_start":"179'5900","ondisk_log_start":"179'5900","created":149,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":6291,"log_dups_size":0,"ondisk_log_size":6291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168230912,"num_objects":2567,"num_object_clones":0,"num_object_copies":7701,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12191,"num_write_kb":472256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3840,"num_bytes_recovered":251658240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":200,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.503313+0000","last_change":"2024-09-11T08:45:08.140910+0000","last_active":"2024-09-11T08:47:59.503313+0000","last_peered":"2024-09-11T08:47:59.503313+0000","last_clean":"2024-09-11T08:47:59.503313+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:47:59.503313+0000","last_undegraded":"2024-09-11T08:47:59.503313+0000","last_fullsized":"2024-09-11T08:47:59.503313+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:08.140839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.140839+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003679547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":188,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.506395+0000","last_change":"2024-09-11T08:45:14.322786+0000","last_active":"2024-09-11T08:47:59.506395+0000","last_peered":"2024-09-11T08:47:59.506395+0000","last_clean":"2024-09-11T08:47:59.506395+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:47:59.506395+0000","last_undegraded":"2024-09-11T08:47:59.506395+0000","last_fullsized":"2024-09-11T08:47:59.506395+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.322217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.322217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014821319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":196,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.478848+0000","last_change":"2024-09-11T08:47:41.322355+0000","last_active":"2024-09-11T08:47:53.478848+0000","last_peered":"2024-09-11T08:47:53.478848+0000","last_clean":"2024-09-11T08:47:53.478848+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:47:53.478848+0000","last_undegraded":"2024-09-11T08:47:53.478848+0000","last_fullsized":"2024-09-11T08:47:53.478848+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:29.692933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:29.692933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007324077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":172,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504086+0000","last_change":"2024-09-11T08:45:33.228265+0000","last_active":"2024-09-11T08:47:59.504086+0000","last_peered":"2024-09-11T08:47:59.504086+0000","last_clean":"2024-09-11T08:47:59.504086+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:47:59.504086+0000","last_undegraded":"2024-09-11T08:47:59.504086+0000","last_fullsized":"2024-09-11T08:47:59.504086+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:45:33.228082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064426500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"195'12131","reported_seq":12376,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.215738+0000","last_change":"2024-09-11T08:47:45.030515+0000","last_active":"2024-09-11T08:48:06.215738+0000","last_peered":"2024-09-11T08:48:06.215738+0000","last_clean":"2024-09-11T08:48:06.215738+0000","last_became_active":"2024-09-11T08:45:15.625778+0000","last_became_peered":"2024-09-11T08:45:15.625778+0000","last_unstale":"2024-09-11T08:48:06.215738+0000","last_undegraded":"2024-09-11T08:48:06.215738+0000","last_fullsized":"2024-09-11T08:48:06.215738+0000","mapping_epoch":149,"log_start":"183'6700","ondisk_log_start":"183'6700","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'10571","last_scrub_stamp":"2024-09-11T08:47:45.030446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:47:45.030446+0000","objects_scrubbed":4541,"log_size":5431,"log_dups_size":0,"ondisk_log_size":5431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:56.591715+0000","scrub_duration":0.68909561500000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":195100672,"num_objects":2977,"num_object_clones":0,"num_object_copies":8931,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12131,"num_write_kb":483456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"124'22","reported_seq":266,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.472992+0000","last_change":"2024-09-11T08:47:13.872981+0000","last_active":"2024-09-11T08:47:53.472992+0000","last_peered":"2024-09-11T08:47:53.472992+0000","last_clean":"2024-09-11T08:47:53.472992+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:47:53.472992+0000","last_undegraded":"2024-09-11T08:47:53.472992+0000","last_fullsized":"2024-09-11T08:47:53.472992+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:43:32.625376+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073394039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":205,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.506526+0000","last_change":"2024-09-11T08:45:20.372571+0000","last_active":"2024-09-11T08:47:59.506526+0000","last_peered":"2024-09-11T08:47:59.506526+0000","last_clean":"2024-09-11T08:47:59.506526+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:47:59.506526+0000","last_undegraded":"2024-09-11T08:47:59.506526+0000","last_fullsized":"2024-09-11T08:47:59.506526+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:45:20.372454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.372454+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019547269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"168'10","reported_seq":334,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-11T08:48:06.584099+0000","last_change":"2024-09-11T08:46:01.988356+0000","last_active":"2024-09-11T08:48:06.584099+0000","last_peered":"2024-09-11T08:48:06.584099+0000","last_clean":"2024-09-11T08:48:06.584099+0000","last_became_active":"2024-09-11T08:46:01.983055+0000","last_became_peered":"2024-09-11T08:46:01.983055+0000","last_unstale":"2024-09-11T08:48:06.584099+0000","last_undegraded":"2024-09-11T08:48:06.584099+0000","last_fullsized":"2024-09-11T08:48:06.584099+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"135'9","last_scrub_stamp":"2024-09-11T08:45:11.298652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.298652+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001669032,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":178,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.473633+0000","last_change":"2024-09-11T08:47:34.554720+0000","last_active":"2024-09-11T08:47:53.473633+0000","last_peered":"2024-09-11T08:47:53.473633+0000","last_clean":"2024-09-11T08:47:53.473633+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:47:53.473633+0000","last_undegraded":"2024-09-11T08:47:53.473633+0000","last_fullsized":"2024-09-11T08:47:53.473633+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:15.888394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:15.888394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003201476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":198,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504726+0000","last_change":"2024-09-11T08:45:28.247838+0000","last_active":"2024-09-11T08:47:59.504726+0000","last_peered":"2024-09-11T08:47:59.504726+0000","last_clean":"2024-09-11T08:47:59.504726+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:47:59.504726+0000","last_undegraded":"2024-09-11T08:47:59.504726+0000","last_fullsized":"2024-09-11T08:47:59.504726+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.247771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015413499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":203,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.506110+0000","last_change":"2024-09-11T08:44:35.390285+0000","last_active":"2024-09-11T08:47:59.506110+0000","last_peered":"2024-09-11T08:47:59.506110+0000","last_clean":"2024-09-11T08:47:59.506110+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:47:59.506110+0000","last_undegraded":"2024-09-11T08:47:59.506110+0000","last_fullsized":"2024-09-11T08:47:59.506110+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-11T08:44:35.390134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:35.390134+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017427033000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":184,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.473168+0000","last_change":"2024-09-11T08:45:08.675251+0000","last_active":"2024-09-11T08:47:53.473168+0000","last_peered":"2024-09-11T08:47:53.473168+0000","last_clean":"2024-09-11T08:47:53.473168+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:47:53.473168+0000","last_undegraded":"2024-09-11T08:47:53.473168+0000","last_fullsized":"2024-09-11T08:47:53.473168+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:08.675179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.675179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003515392,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":202,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.508300+0000","last_change":"2024-09-11T08:46:01.983864+0000","last_active":"2024-09-11T08:47:59.508300+0000","last_peered":"2024-09-11T08:47:59.508300+0000","last_clean":"2024-09-11T08:47:59.508300+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:47:59.508300+0000","last_undegraded":"2024-09-11T08:47:59.508300+0000","last_fullsized":"2024-09-11T08:47:59.508300+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:25.344348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.344348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013849730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":203,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505480+0000","last_change":"2024-09-11T08:47:44.737776+0000","last_active":"2024-09-11T08:47:59.505480+0000","last_peered":"2024-09-11T08:47:59.505480+0000","last_clean":"2024-09-11T08:47:59.505480+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:47:59.505480+0000","last_undegraded":"2024-09-11T08:47:59.505480+0000","last_fullsized":"2024-09-11T08:47:59.505480+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:47:44.737721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.737721+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:04.218319+0000","scrub_duration":0.0032009640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":191,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505840+0000","last_change":"2024-09-11T08:47:38.415255+0000","last_active":"2024-09-11T08:47:59.505840+0000","last_peered":"2024-09-11T08:47:59.505840+0000","last_clean":"2024-09-11T08:47:59.505840+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:47:59.505840+0000","last_undegraded":"2024-09-11T08:47:59.505840+0000","last_fullsized":"2024-09-11T08:47:59.505840+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:44:32.483439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013511770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":174,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505960+0000","last_change":"2024-09-11T08:45:23.213642+0000","last_active":"2024-09-11T08:47:59.505960+0000","last_peered":"2024-09-11T08:47:59.505960+0000","last_clean":"2024-09-11T08:47:59.505960+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:47:59.505960+0000","last_undegraded":"2024-09-11T08:47:59.505960+0000","last_fullsized":"2024-09-11T08:47:59.505960+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.213592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.213592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001600473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":192,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.479220+0000","last_change":"2024-09-11T08:47:34.423379+0000","last_active":"2024-09-11T08:47:53.479220+0000","last_peered":"2024-09-11T08:47:53.479220+0000","last_clean":"2024-09-11T08:47:53.479220+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:47:53.479220+0000","last_undegraded":"2024-09-11T08:47:53.479220+0000","last_fullsized":"2024-09-11T08:47:53.479220+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_clean_scrub_stamp":"2024-09-11T08:47:34.420932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:45.412817+0000","scrub_duration":0.01242356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":198,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.474281+0000","last_change":"2024-09-11T08:47:29.687278+0000","last_active":"2024-09-11T08:47:53.474281+0000","last_peered":"2024-09-11T08:47:53.474281+0000","last_clean":"2024-09-11T08:47:53.474281+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:47:53.474281+0000","last_undegraded":"2024-09-11T08:47:53.474281+0000","last_fullsized":"2024-09-11T08:47:53.474281+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:18.928481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:18.928481+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047130310000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":189,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505571+0000","last_change":"2024-09-11T08:47:45.751605+0000","last_active":"2024-09-11T08:47:59.505571+0000","last_peered":"2024-09-11T08:47:59.505571+0000","last_clean":"2024-09-11T08:47:59.505571+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:47:59.505571+0000","last_undegraded":"2024-09-11T08:47:59.505571+0000","last_fullsized":"2024-09-11T08:47:59.505571+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:45.751518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:45.751518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:47.231246+0000","scrub_duration":0.0061587329999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":295,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:48:02.790974+0000","last_change":"2024-09-11T08:45:20.119727+0000","last_active":"2024-09-11T08:48:02.790974+0000","last_peered":"2024-09-11T08:48:02.790974+0000","last_clean":"2024-09-11T08:48:02.790974+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:48:02.790974+0000","last_undegraded":"2024-09-11T08:48:02.790974+0000","last_fullsized":"2024-09-11T08:48:02.790974+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:20.119626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.119626+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018087660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":163,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.503579+0000","last_change":"2024-09-11T08:44:20.017341+0000","last_active":"2024-09-11T08:47:59.503579+0000","last_peered":"2024-09-11T08:47:59.503579+0000","last_clean":"2024-09-11T08:47:59.503579+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:47:59.503579+0000","last_undegraded":"2024-09-11T08:47:59.503579+0000","last_fullsized":"2024-09-11T08:47:59.503579+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:20.017261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:20.017261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001558369,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"152'15","reported_seq":196,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.506106+0000","last_change":"2024-09-11T08:45:21.132138+0000","last_active":"2024-09-11T08:47:59.506106+0000","last_peered":"2024-09-11T08:47:59.506106+0000","last_clean":"2024-09-11T08:47:59.506106+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:47:59.506106+0000","last_undegraded":"2024-09-11T08:47:59.506106+0000","last_fullsized":"2024-09-11T08:47:59.506106+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'15","last_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.132083+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003984156,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":188,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.506280+0000","last_change":"2024-09-11T08:45:27.436673+0000","last_active":"2024-09-11T08:47:59.506280+0000","last_peered":"2024-09-11T08:47:59.506280+0000","last_clean":"2024-09-11T08:47:59.506280+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:47:59.506280+0000","last_undegraded":"2024-09-11T08:47:59.506280+0000","last_fullsized":"2024-09-11T08:47:59.506280+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.436552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001660176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":305,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.573437+0000","last_change":"2024-09-11T08:45:28.692173+0000","last_active":"2024-09-11T08:47:59.573437+0000","last_peered":"2024-09-11T08:47:59.573437+0000","last_clean":"2024-09-11T08:47:59.573437+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:47:59.573437+0000","last_undegraded":"2024-09-11T08:47:59.573437+0000","last_fullsized":"2024-09-11T08:47:59.573437+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:28.692100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.692100+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019579020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":183,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505008+0000","last_change":"2024-09-11T08:45:14.581287+0000","last_active":"2024-09-11T08:47:59.505008+0000","last_peered":"2024-09-11T08:47:59.505008+0000","last_clean":"2024-09-11T08:47:59.505008+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:47:59.505008+0000","last_undegraded":"2024-09-11T08:47:59.505008+0000","last_fullsized":"2024-09-11T08:47:59.505008+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.581231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.581231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001580932,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":186,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504116+0000","last_change":"2024-09-11T08:43:37.137260+0000","last_active":"2024-09-11T08:47:59.504116+0000","last_peered":"2024-09-11T08:47:59.504116+0000","last_clean":"2024-09-11T08:47:59.504116+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:47:59.504116+0000","last_undegraded":"2024-09-11T08:47:59.504116+0000","last_fullsized":"2024-09-11T08:47:59.504116+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:43:37.137173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:43:37.137173+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085039490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":179,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.503868+0000","last_change":"2024-09-11T08:45:27.198021+0000","last_active":"2024-09-11T08:47:59.503868+0000","last_peered":"2024-09-11T08:47:59.503868+0000","last_clean":"2024-09-11T08:47:59.503868+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:47:59.503868+0000","last_undegraded":"2024-09-11T08:47:59.503868+0000","last_fullsized":"2024-09-11T08:47:59.503868+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.197944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.197944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015230210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":184,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.480408+0000","last_change":"2024-09-11T08:45:21.736656+0000","last_active":"2024-09-11T08:47:53.480408+0000","last_peered":"2024-09-11T08:47:53.480408+0000","last_clean":"2024-09-11T08:47:53.480408+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:47:53.480408+0000","last_undegraded":"2024-09-11T08:47:53.480408+0000","last_fullsized":"2024-09-11T08:47:53.480408+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:21.736573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.736573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020360389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":172,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.501192+0000","last_change":"2024-09-11T08:45:28.259156+0000","last_active":"2024-09-11T08:47:59.501192+0000","last_peered":"2024-09-11T08:47:59.501192+0000","last_clean":"2024-09-11T08:47:59.501192+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:47:59.501192+0000","last_undegraded":"2024-09-11T08:47:59.501192+0000","last_fullsized":"2024-09-11T08:47:59.501192+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.259103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.259103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015518800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":215,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.473303+0000","last_change":"2024-09-11T08:46:01.993547+0000","last_active":"2024-09-11T08:47:53.473303+0000","last_peered":"2024-09-11T08:47:53.473303+0000","last_clean":"2024-09-11T08:47:53.473303+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:47:53.473303+0000","last_undegraded":"2024-09-11T08:47:53.473303+0000","last_fullsized":"2024-09-11T08:47:53.473303+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:45:06.657912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.657912+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023744109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":198,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.508492+0000","last_change":"2024-09-11T08:47:48.373407+0000","last_active":"2024-09-11T08:47:59.508492+0000","last_peered":"2024-09-11T08:47:59.508492+0000","last_clean":"2024-09-11T08:47:59.508492+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:47:59.508492+0000","last_undegraded":"2024-09-11T08:47:59.508492+0000","last_fullsized":"2024-09-11T08:47:59.508492+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:48.373336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:48.373336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:54.062621+0000","scrub_duration":0.001705364,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":174,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504164+0000","last_change":"2024-09-11T08:45:14.165632+0000","last_active":"2024-09-11T08:47:59.504164+0000","last_peered":"2024-09-11T08:47:59.504164+0000","last_clean":"2024-09-11T08:47:59.504164+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:47:59.504164+0000","last_undegraded":"2024-09-11T08:47:59.504164+0000","last_fullsized":"2024-09-11T08:47:59.504164+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.165576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.165576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012147639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":176,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.509249+0000","last_change":"2024-09-11T08:47:33.592756+0000","last_active":"2024-09-11T08:47:59.509249+0000","last_peered":"2024-09-11T08:47:59.509249+0000","last_clean":"2024-09-11T08:47:59.509249+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:47:59.509249+0000","last_undegraded":"2024-09-11T08:47:59.509249+0000","last_fullsized":"2024-09-11T08:47:59.509249+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:43:29.349351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:43:29.349351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006276294,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":185,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.501280+0000","last_change":"2024-09-11T08:44:12.198399+0000","last_active":"2024-09-11T08:47:59.501280+0000","last_peered":"2024-09-11T08:47:59.501280+0000","last_clean":"2024-09-11T08:47:59.501280+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:47:59.501280+0000","last_undegraded":"2024-09-11T08:47:59.501280+0000","last_fullsized":"2024-09-11T08:47:59.501280+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:44:12.198335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.198335+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020517040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":188,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.473938+0000","last_change":"2024-09-11T08:45:14.574739+0000","last_active":"2024-09-11T08:47:53.473938+0000","last_peered":"2024-09-11T08:47:53.473938+0000","last_clean":"2024-09-11T08:47:53.473938+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:47:53.473938+0000","last_undegraded":"2024-09-11T08:47:53.473938+0000","last_fullsized":"2024-09-11T08:47:53.473938+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.574678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.574678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013819469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":187,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505224+0000","last_change":"2024-09-11T08:47:32.404383+0000","last_active":"2024-09-11T08:47:59.505224+0000","last_peered":"2024-09-11T08:47:59.505224+0000","last_clean":"2024-09-11T08:47:59.505224+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:47:59.505224+0000","last_undegraded":"2024-09-11T08:47:59.505224+0000","last_fullsized":"2024-09-11T08:47:59.505224+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:28.484047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.484047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001479172,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":192,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504266+0000","last_change":"2024-09-11T08:47:44.807380+0000","last_active":"2024-09-11T08:47:59.504266+0000","last_peered":"2024-09-11T08:47:59.504266+0000","last_clean":"2024-09-11T08:47:59.504266+0000","last_became_active":"2024-09-11T08:46:01.984167+0000","last_became_peered":"2024-09-11T08:46:01.984167+0000","last_unstale":"2024-09-11T08:47:59.504266+0000","last_undegraded":"2024-09-11T08:47:59.504266+0000","last_fullsized":"2024-09-11T08:47:59.504266+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.807331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.807331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:03.561680+0000","scrub_duration":0.0013867529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"136'6","reported_seq":216,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.501801+0000","last_change":"2024-09-11T08:46:01.996138+0000","last_active":"2024-09-11T08:47:59.501801+0000","last_peered":"2024-09-11T08:47:59.501801+0000","last_clean":"2024-09-11T08:47:59.501801+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:47:59.501801+0000","last_undegraded":"2024-09-11T08:47:59.501801+0000","last_fullsized":"2024-09-11T08:47:59.501801+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'6","last_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:44:21.302870+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012512683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":196,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.509184+0000","last_change":"2024-09-11T08:45:21.481725+0000","last_active":"2024-09-11T08:47:59.509184+0000","last_peered":"2024-09-11T08:47:59.509184+0000","last_clean":"2024-09-11T08:47:59.509184+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:47:59.509184+0000","last_undegraded":"2024-09-11T08:47:59.509184+0000","last_fullsized":"2024-09-11T08:47:59.509184+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:21.481634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.481634+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025664590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.504529+0000","last_change":"2024-09-11T08:46:01.979915+0000","last_active":"2024-09-11T08:47:59.504529+0000","last_peered":"2024-09-11T08:47:59.504529+0000","last_clean":"2024-09-11T08:47:59.504529+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:47:59.504529+0000","last_undegraded":"2024-09-11T08:47:59.504529+0000","last_fullsized":"2024-09-11T08:47:59.504529+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.196277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.196277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":192,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505886+0000","last_change":"2024-09-11T08:45:23.348327+0000","last_active":"2024-09-11T08:47:59.505886+0000","last_peered":"2024-09-11T08:47:59.505886+0000","last_clean":"2024-09-11T08:47:59.505886+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:47:59.505886+0000","last_undegraded":"2024-09-11T08:47:59.505886+0000","last_fullsized":"2024-09-11T08:47:59.505886+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.348275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.348275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035843659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":192,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-11T08:47:53.480990+0000","last_change":"2024-09-11T08:45:23.674163+0000","last_active":"2024-09-11T08:47:53.480990+0000","last_peered":"2024-09-11T08:47:53.480990+0000","last_clean":"2024-09-11T08:47:53.480990+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:47:53.480990+0000","last_undegraded":"2024-09-11T08:47:53.480990+0000","last_fullsized":"2024-09-11T08:47:53.480990+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.674102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.674102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001440293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":178,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.502430+0000","last_change":"2024-09-11T08:44:23.320368+0000","last_active":"2024-09-11T08:47:59.502430+0000","last_peered":"2024-09-11T08:47:59.502430+0000","last_clean":"2024-09-11T08:47:59.502430+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:47:59.502430+0000","last_undegraded":"2024-09-11T08:47:59.502430+0000","last_fullsized":"2024-09-11T08:47:59.502430+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:23.320301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:23.320301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017133368999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":50,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505688+0000","last_change":"2024-09-11T08:46:02.002471+0000","last_active":"2024-09-11T08:47:59.505688+0000","last_peered":"2024-09-11T08:47:59.505688+0000","last_clean":"2024-09-11T08:47:59.505688+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:47:59.505688+0000","last_undegraded":"2024-09-11T08:47:59.505688+0000","last_fullsized":"2024-09-11T08:47:59.505688+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:45:25.620635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.620635+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":185,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-11T08:47:59.505256+0000","last_change":"2024-09-11T08:47:32.555942+0000","last_active":"2024-09-11T08:47:59.505256+0000","last_peered":"2024-09-11T08:47:59.505256+0000","last_clean":"2024-09-11T08:47:59.505256+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:47:59.505256+0000","last_undegraded":"2024-09-11T08:47:59.505256+0000","last_fullsized":"2024-09-11T08:47:59.505256+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:32.555890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:32.555890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:41.110543+0000","scrub_duration":0.0014425740000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":3004891160,"num_objects":45852,"num_object_clones":0,"num_object_copies":137556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45852,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":194066,"num_write_kb":7677313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7769,"num_bytes_recovered":509149184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9370873856,"data_stored":9370861640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85574,"ondisk_log_size":85574,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":1584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":456,"ondisk_log_size":456,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":565,"num_read_kb":560,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299512,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1119032,"kb_used_data":817520,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":92626120,"statfs":{"total":95995035648,"available":94849146880,"internally_reserved":0,"allocated":837140480,"data_stored":836830661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.2889999999999999,"15min":0.81799999999999995},"min":{"1min":0.372,"5min":0.22,"15min":0.22},"max":{"1min":2.585,"5min":32.171999999999997,"15min":32.171999999999997},"last":3.7919999999999998},{"interface":"front","average":{"1min":1.571,"5min":1.4219999999999999,"15min":0.89900000000000002},"min":{"1min":0.29399999999999998,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":3.665,"5min":32.149999999999999,"15min":32.149999999999999},"last":0.50700000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.4830000000000001,"15min":0.90100000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.4980000000000002,"5min":33.692999999999998,"15min":33.692999999999998},"last":3.8149999999999999},{"interface":"front","average":{"1min":1.228,"5min":1.3700000000000001,"15min":0.84499999999999997},"min":{"1min":0.32000000000000001,"5min":0.26600000000000001,"15min":0.23300000000000001},"max":{"1min":3.6259999999999999,"5min":35.591000000000001,"15min":35.591000000000001},"last":0.88700000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.98899999999999999,"15min":0.754},"min":{"1min":0.30299999999999999,"5min":0.249,"15min":0.249},"max":{"1min":2.8140000000000001,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.806},{"interface":"front","average":{"1min":0.79500000000000004,"5min":1.0429999999999999,"15min":0.71599999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.782,"5min":5.7990000000000004,"15min":5.7990000000000004},"last":3.8380000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.083,"15min":0.77800000000000002},"min":{"1min":0.32700000000000001,"5min":0.25,"15min":0.25},"max":{"1min":3.2839999999999998,"5min":7.8609999999999998,"15min":7.8609999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.29,"15min":0.85899999999999999},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":5.625,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":3.7669999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.53500000000000003,"15min":0.46700000000000003},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.23400000000000001},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.56299999999999994,"15min":0.47899999999999998},"min":{"1min":0.27000000000000002,"5min":0.216,"15min":0.216},"max":{"1min":4.8920000000000003,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.44700000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.60599999999999998,"15min":0.499},"min":{"1min":0.31,"5min":0.255,"15min":0.23300000000000001},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":3.7309999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.67800000000000005,"15min":0.54200000000000004},"min":{"1min":0.32900000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.093,"5min":3.093,"15min":3.093},"last":1.177}]},{"osd":6,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.68100000000000005,"15min":0.55800000000000005},"min":{"1min":0.35399999999999998,"5min":0.317,"15min":0.315},"max":{"1min":2.8820000000000001,"5min":3.5979999999999999,"15min":3.5979999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.63400000000000001,"15min":0.54600000000000004},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":0.376}]}]},{"osd":6,"up_from":45,"seq":193273528445,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1141480,"kb_used_data":835680,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92603672,"statfs":{"total":95995035648,"available":94826160128,"internally_reserved":0,"allocated":855736320,"data_stored":855442697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.3400000000000001,"15min":0.89500000000000002},"min":{"1min":0.23300000000000001,"5min":0.217,"15min":0.217},"max":{"1min":3.5619999999999998,"5min":28.302,"15min":28.302},"last":3.6989999999999998},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.4450000000000001,"15min":0.875},"min":{"1min":0.33700000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.9630000000000001,"5min":28.314,"15min":28.314},"last":1.492}]},{"osd":1,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":2.173,"5min":1.4370000000000001,"15min":0.85599999999999998},"min":{"1min":0.32300000000000001,"5min":0.29399999999999998,"15min":0.28299999999999997},"max":{"1min":7.8810000000000002,"5min":12.586,"15min":12.586},"last":2.2360000000000002},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.1850000000000001,"15min":0.77300000000000002},"min":{"1min":0.30099999999999999,"5min":0.254,"15min":0.253},"max":{"1min":9.5370000000000008,"5min":10.834,"15min":10.834},"last":0.34300000000000003}]},{"osd":2,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.97999999999999998,"15min":0.70199999999999996},"min":{"1min":0.29499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.8849999999999998,"5min":14.313000000000001,"15min":14.313000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.365,"5min":1.0569999999999999,"15min":0.71599999999999997},"min":{"1min":0.40500000000000003,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":3.593,"5min":6.5979999999999999,"15min":6.5979999999999999},"last":0.61299999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":1.889,"5min":1.401,"15min":0.85399999999999998},"min":{"1min":0.35799999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":4.2270000000000003,"5min":11.433,"15min":11.433},"last":0.29199999999999998},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.042,"15min":0.76200000000000001},"min":{"1min":0.33100000000000002,"5min":0.224,"15min":0.224},"max":{"1min":2.738,"5min":9.1069999999999993,"15min":9.1069999999999993},"last":7.0250000000000004}]},{"osd":4,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.50900000000000001,"15min":0.44600000000000001},"min":{"1min":0.27300000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.502,"15min":0.443},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":0.32500000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.59399999999999997,"15min":0.49299999999999999},"min":{"1min":0.28699999999999998,"5min":0.245,"15min":0.245},"max":{"1min":3.097,"5min":3.097,"15min":3.097},"last":0.23899999999999999},{"interface":"front","average":{"1min":1.042,"5min":0.61599999999999999,"15min":0.501},"min":{"1min":0.312,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":5.0819999999999999,"5min":5.0819999999999999,"15min":5.0819999999999999},"last":0.35499999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:47:37 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64200000000000002,"15min":0.51300000000000001},"min":{"1min":0.29899999999999999,"5min":0.25900000000000001,"15min":0.247},"max":{"1min":2.7810000000000001,"5min":7.327,"15min":7.327},"last":0.313},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.55600000000000005,"15min":0.51600000000000001},"min":{"1min":0.38400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.36799999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757376,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1862824,"kb_used_data":1552928,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91882328,"statfs":{"total":95995035648,"available":94087503872,"internally_reserved":0,"allocated":1590198272,"data_stored":1589896783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.04,"15min":0.751},"min":{"1min":0.23200000000000001,"5min":0.20899999999999999,"15min":0.20599999999999999},"max":{"1min":3.266,"5min":7.3899999999999997,"15min":7.3899999999999997},"last":0.41099999999999998},{"interface":"front","average":{"1min":1.7629999999999999,"5min":1.2230000000000001,"15min":0.85499999999999998},"min":{"1min":0.23499999999999999,"5min":0.23400000000000001,"15min":0.23100000000000001},"max":{"1min":8.0410000000000004,"5min":8.0410000000000004,"15min":8.0410000000000004},"last":0.309}]},{"osd":1,"last update":"Wed Sep 11 08:48:01 2024","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":1.2490000000000001,"15min":0.84799999999999998},"min":{"1min":0.35999999999999999,"5min":0.23799999999999999,"15min":0.22800000000000001},"max":{"1min":8,"5min":8,"15min":8},"last":0.35999999999999999},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.21,"15min":0.78700000000000003},"min":{"1min":0.27400000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.649,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":1.8580000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:48:01 2024","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.1819999999999999,"15min":0.81299999999999994},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.23799999999999999},"max":{"1min":3.903,"5min":4.0919999999999996,"15min":5.0309999999999997},"last":5.5339999999999998},{"interface":"front","average":{"1min":1.927,"5min":1.222,"15min":0.79600000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.26400000000000001},"max":{"1min":8.0640000000000001,"5min":8.0640000000000001,"15min":8.0640000000000001},"last":3.5990000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:48:01 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1020000000000001,"15min":0.78100000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":4.3940000000000001,"5min":6.0460000000000003,"15min":6.0460000000000003},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.641,"5min":1.3440000000000001,"15min":0.90000000000000002},"min":{"1min":0.33000000000000002,"5min":0.29199999999999998,"15min":0.27400000000000002},"max":{"1min":8.0860000000000003,"5min":8.0860000000000003,"15min":8.0860000000000003},"last":0.378}]},{"osd":4,"last update":"Wed Sep 11 08:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.65500000000000003,"15min":0.47799999999999998},"min":{"1min":0.247,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":4.7110000000000003,"5min":4.7110000000000003,"15min":4.7110000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.60399999999999998,"15min":0.47199999999999998},"min":{"1min":0.26200000000000001,"5min":0.23699999999999999,"15min":0.20499999999999999},"max":{"1min":3.7879999999999998,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.81599999999999995}]},{"osd":6,"last update":"Wed Sep 11 08:47:24 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.58399999999999996,"15min":0.48499999999999999},"min":{"1min":0.29399999999999998,"5min":0.27700000000000002,"15min":0.23599999999999999},"max":{"1min":3.2509999999999999,"5min":3.4329999999999998,"15min":3.4329999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.58899999999999997,"15min":0.48299999999999998},"min":{"1min":0.29099999999999998,"5min":0.26100000000000001,"15min":0.251},"max":{"1min":3,"5min":3,"15min":3},"last":0.33700000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:47:34 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.64400000000000002,"15min":0.53400000000000003},"min":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.27800000000000002},"max":{"1min":3.7639999999999998,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.64400000000000002,"15min":0.57099999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.2730000000000001,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":0.85699999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986309,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1149680,"kb_used_data":835624,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":92595472,"statfs":{"total":95995035648,"available":94817763328,"internally_reserved":0,"allocated":855678976,"data_stored":855376243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.6060000000000001,"15min":1.2949999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.20399999999999999},"max":{"1min":4.2830000000000004,"5min":48.451999999999998,"15min":89.510000000000005},"last":3.1179999999999999},{"interface":"front","average":{"1min":2.294,"5min":1.8700000000000001,"15min":1.4039999999999999},"min":{"1min":0.23999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":5.569,"5min":49.115000000000002,"15min":89.765000000000001},"last":3.4239999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:47:39 2024","interfaces":[{"interface":"back","average":{"1min":2.1200000000000001,"5min":1.5549999999999999,"15min":0.88500000000000001},"min":{"1min":0.63900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":6.1020000000000003,"5min":18.596,"15min":18.596},"last":3.0179999999999998},{"interface":"front","average":{"1min":2.052,"5min":1.4570000000000001,"15min":0.872},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.23599999999999999},"max":{"1min":5.3810000000000002,"5min":18.754000000000001,"15min":18.754000000000001},"last":0.36599999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:47:39 2024","interfaces":[{"interface":"back","average":{"1min":2.3580000000000001,"5min":1.2310000000000001,"15min":0.80300000000000005},"min":{"1min":0.39300000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":17.303000000000001,"5min":17.303000000000001,"15min":17.303000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":2.1520000000000001,"5min":1.1859999999999999,"15min":0.78700000000000003},"min":{"1min":0.28599999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":20.702999999999999,"5min":20.702999999999999,"15min":20.702999999999999},"last":0.65400000000000003}]},{"osd":3,"last update":"Wed Sep 11 08:47:39 2024","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.9550000000000001,"15min":1.4650000000000001},"min":{"1min":0.314,"5min":0.255,"15min":0.255},"max":{"1min":4.2229999999999999,"5min":48.194000000000003,"15min":89.881},"last":0.60099999999999998},{"interface":"front","average":{"1min":1.8720000000000001,"5min":1.925,"15min":1.4299999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":6.0650000000000004,"5min":48.840000000000003,"15min":89.715000000000003},"last":0.35399999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:48:04 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.64000000000000001,"15min":0.49099999999999999},"min":{"1min":0.27000000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":3.4020000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.65100000000000002,"15min":0.497},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.23699999999999999},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.45100000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:47:28 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.495,"15min":0.45500000000000002},"min":{"1min":0.28100000000000003,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.0640000000000001,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.61199999999999999,"15min":0.52200000000000002},"min":{"1min":0.22800000000000001,"5min":0.221,"15min":0.221},"max":{"1min":3.4590000000000001,"5min":3.4590000000000001,"15min":3.4590000000000001},"last":0.32400000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:47:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.621,"15min":0.53000000000000003},"min":{"1min":0.29199999999999998,"5min":0.252,"15min":0.252},"max":{"1min":3.339,"5min":3.339,"15min":3.339},"last":0.39300000000000002},{"interface":"front","average":{"1min":1.194,"5min":0.68700000000000006,"15min":0.55000000000000004},"min":{"1min":0.32000000000000001,"5min":0.29599999999999999,"15min":0.28799999999999998},"max":{"1min":5.1970000000000001,"5min":5.1970000000000001,"15min":5.1970000000000001},"last":0.67300000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149834,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1457276,"kb_used_data":1151540,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92287876,"statfs":{"total":95995035648,"available":94502785024,"internally_reserved":0,"allocated":1179176960,"data_stored":1178855377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:21 2024","interfaces":[{"interface":"back","average":{"1min":1.776,"5min":1.7529999999999999,"15min":0.89500000000000002},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":7.1399999999999997,"5min":10.901999999999999,"15min":10.901999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":2.3639999999999999,"5min":2.1850000000000001,"15min":1.0820000000000001},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":5.4249999999999998,"5min":9.6820000000000004,"15min":9.6820000000000004},"last":0.60799999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:47:21 2024","interfaces":[{"interface":"back","average":{"1min":2.7909999999999999,"5min":3.278,"15min":1.446},"min":{"1min":0.26700000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":8.2439999999999998,"5min":115.601,"15min":115.601},"last":0.71399999999999997},{"interface":"front","average":{"1min":2.089,"5min":3.1259999999999999,"15min":1.373},"min":{"1min":0.29499999999999998,"5min":0.20300000000000001,"15min":0.17999999999999999},"max":{"1min":8.4420000000000002,"5min":115.474,"15min":115.474},"last":0.92500000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:47:21 2024","interfaces":[{"interface":"back","average":{"1min":1.6879999999999999,"5min":1.9199999999999999,"15min":0.99199999999999999},"min":{"1min":0.28599999999999998,"5min":0.251,"15min":0.23200000000000001},"max":{"1min":8.2699999999999996,"5min":8.9510000000000005,"15min":8.9510000000000005},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.48,"5min":1.746,"15min":0.94499999999999995},"min":{"1min":0.30299999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":5.5999999999999996,"5min":10.695,"15min":10.695},"last":0.63100000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:47:33 2024","interfaces":[{"interface":"back","average":{"1min":2.6869999999999998,"5min":1.73,"15min":0.997},"min":{"1min":0.35799999999999998,"5min":0.27500000000000002,"15min":0.26700000000000002},"max":{"1min":8.3279999999999994,"5min":8.9969999999999999,"15min":8.9969999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":2.0779999999999998,"5min":1.4359999999999999,"15min":0.88300000000000001},"min":{"1min":0.39500000000000002,"5min":0.28000000000000003,"15min":0.27700000000000002},"max":{"1min":7.234,"5min":7.234,"15min":7.234},"last":2.524}]},{"osd":5,"last update":"Wed Sep 11 08:47:53 2024","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.653,"15min":1.0009999999999999},"min":{"1min":0.438,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.2380000000000004,"5min":11.526999999999999,"15min":11.526999999999999},"last":0.87},{"interface":"front","average":{"1min":1.8169999999999999,"5min":1.427,"15min":0.90500000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.1689999999999996,"5min":7.1689999999999996,"15min":7.1689999999999996},"last":2.492}]},{"osd":6,"last update":"Wed Sep 11 08:47:21 2024","interfaces":[{"interface":"back","average":{"1min":1.9359999999999999,"5min":1.7849999999999999,"15min":1.0109999999999999},"min":{"1min":0.34899999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.4160000000000004,"5min":11.494999999999999,"15min":11.494999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.6659999999999999,"5min":1.6559999999999999,"15min":0.98299999999999998},"min":{"1min":0.41399999999999998,"5min":0.311,"15min":0.311},"max":{"1min":3.5630000000000002,"5min":7.7249999999999996,"15min":7.7249999999999996},"last":0.68500000000000005}]},{"osd":7,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":2.4969999999999999,"5min":1.831,"15min":1.0740000000000001},"min":{"1min":0.39600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":9.1539999999999999,"5min":11.561,"15min":11.561},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.8779999999999999,"5min":1.4890000000000001,"15min":0.93799999999999994},"min":{"1min":0.44500000000000001,"5min":0.371,"15min":0.34699999999999998},"max":{"1min":6.6200000000000001,"5min":6.6200000000000001,"15min":6.6200000000000001},"last":2.5499999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411470,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1676372,"kb_used_data":1366540,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92068780,"statfs":{"total":95995035648,"available":94278430720,"internally_reserved":0,"allocated":1399336960,"data_stored":1399054491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:59 2024","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":2.6749999999999998,"15min":1.272},"min":{"1min":0.22700000000000001,"5min":0.159,"15min":0.159},"max":{"1min":4.1689999999999996,"5min":34.713999999999999,"15min":34.713999999999999},"last":16.606999999999999},{"interface":"front","average":{"1min":2.008,"5min":2.8239999999999998,"15min":1.391},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":5.1619999999999999,"5min":34.399000000000001,"15min":34.399000000000001},"last":16.855}]},{"osd":1,"last update":"Wed Sep 11 08:47:59 2024","interfaces":[{"interface":"back","average":{"1min":1.9179999999999999,"5min":2.1190000000000002,"15min":1.103},"min":{"1min":0.433,"5min":0.215,"15min":0.19},"max":{"1min":6.8700000000000001,"5min":34.771000000000001,"15min":34.771000000000001},"last":72.867000000000004},{"interface":"front","average":{"1min":2.355,"5min":2.3170000000000002,"15min":1.1759999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":6.5800000000000001,"5min":35.209000000000003,"15min":35.209000000000003},"last":67.799000000000007}]},{"osd":3,"last update":"Wed Sep 11 08:47:10 2024","interfaces":[{"interface":"back","average":{"1min":4.2190000000000003,"5min":2.3809999999999998,"15min":1.167},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":34.75,"5min":34.75,"15min":34.75},"last":0.28499999999999998},{"interface":"front","average":{"1min":3.948,"5min":2.3239999999999998,"15min":1.1479999999999999},"min":{"1min":0.40999999999999998,"5min":0.27200000000000002,"15min":0.26800000000000002},"max":{"1min":32.802,"5min":32.802,"15min":32.802},"last":0.25700000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:47:28 2024","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.351,"15min":0.88200000000000001},"min":{"1min":0.72999999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":3.9449999999999998,"5min":8.0820000000000007,"15min":8.0820000000000007},"last":0.375},{"interface":"front","average":{"1min":1.319,"5min":1.256,"15min":0.85299999999999998},"min":{"1min":0.29999999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":4.4450000000000003,"5min":6.9320000000000004,"15min":6.9320000000000004},"last":0.33900000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:47:59 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.24,"15min":0.86899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":4.4809999999999999,"5min":6.9550000000000001,"15min":6.9550000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.0449999999999999,"15min":0.78900000000000003},"min":{"1min":0.28999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.226,"5min":4.2000000000000002,"15min":5.9530000000000003},"last":0.84899999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:47:19 2024","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.4730000000000001,"15min":0.93400000000000005},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":5.1980000000000004,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.2210000000000001,"15min":0.82699999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.931,"5min":6.9779999999999998,"15min":6.9779999999999998},"last":0.91200000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:47:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.0780000000000001,"15min":0.80600000000000005},"min":{"1min":0.35599999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.1400000000000001,"5min":5.1820000000000004,"15min":5.9909999999999997},"last":0.33200000000000002},{"interface":"front","average":{"1min":1.673,"5min":1.351,"15min":0.94899999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36099999999999999},"max":{"1min":6.0919999999999996,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":1.079}]}]},{"osd":1,"up_from":13,"seq":55834574995,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1636544,"kb_used_data":1330872,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92108608,"statfs":{"total":95995035648,"available":94319214592,"internally_reserved":0,"allocated":1362812928,"data_stored":1362486628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.6399999999999999,"15min":1.2609999999999999},"min":{"1min":0.33600000000000002,"5min":0.192,"15min":0.183},"max":{"1min":4.6319999999999997,"5min":31.466999999999999,"15min":117.01600000000001},"last":1.472},{"interface":"front","average":{"1min":1.669,"5min":2.3519999999999999,"15min":1.5389999999999999},"min":{"1min":0.318,"5min":0.193,"15min":0.193},"max":{"1min":5.9409999999999998,"5min":32.204000000000001,"15min":117.62},"last":1.01}]},{"osd":2,"last update":"Wed Sep 11 08:47:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.5409999999999999,"15min":0.97299999999999998},"min":{"1min":0.25700000000000001,"5min":0.191,"15min":0.191},"max":{"1min":4.9829999999999997,"5min":9.5719999999999992,"15min":20.628},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.472,"15min":0.94199999999999995},"min":{"1min":0.23200000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.5529999999999999,"5min":7.4080000000000004,"15min":20.148},"last":2.0979999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:47:11 2024","interfaces":[{"interface":"back","average":{"1min":2.3799999999999999,"5min":1.752,"15min":1.248},"min":{"1min":0.39700000000000002,"5min":0.191,"15min":0.191},"max":{"1min":5.8490000000000002,"5min":8.3550000000000004,"15min":116.98},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.7190000000000001,"15min":1.304},"min":{"1min":0.27100000000000002,"5min":0.22,"15min":0.22},"max":{"1min":4.5140000000000002,"5min":10.379,"15min":117.56699999999999},"last":2.0590000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:47:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.2629999999999999,"15min":0.81200000000000006},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.26200000000000001},"max":{"1min":3.681,"5min":11.19,"15min":11.19},"last":2.0390000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.05,"15min":0.76600000000000001},"min":{"1min":0.30199999999999999,"5min":0.28499999999999998,"15min":0.251},"max":{"1min":3.2120000000000002,"5min":5.056,"15min":7.7169999999999996},"last":0.44800000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:47:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":2.5070000000000001,"15min":1.3240000000000001},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.125,"5min":103.337,"15min":103.337},"last":0.497},{"interface":"front","average":{"1min":1.6619999999999999,"5min":2.48,"15min":1.29},"min":{"1min":0.39000000000000001,"5min":0.32300000000000001,"15min":0.27400000000000002},"max":{"1min":3.9279999999999999,"5min":103.163,"15min":103.163},"last":2.0750000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:47:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.1819999999999999,"15min":0.78900000000000003},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.29399999999999998},"max":{"1min":3.907,"5min":8.9149999999999991,"15min":8.9149999999999991},"last":0.46800000000000003},{"interface":"front","average":{"1min":1.53,"5min":1.488,"15min":0.92300000000000004},"min":{"1min":0.45000000000000001,"5min":0.312,"15min":0.31},"max":{"1min":4.4480000000000004,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":0.56200000000000006}]},{"osd":7,"last update":"Wed Sep 11 08:47:31 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.3380000000000001,"15min":0.876},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":4.593,"5min":11.315,"15min":11.315},"last":2.1179999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.127,"15min":0.82299999999999995},"min":{"1min":0.39800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":3.8759999999999999,"5min":4.9100000000000001,"15min":8.8800000000000008},"last":0.40899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738519,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1688892,"kb_used_data":1391536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92056260,"statfs":{"total":95995035648,"available":94265610240,"internally_reserved":0,"allocated":1424932864,"data_stored":1424623080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:47:28 2024","interfaces":[{"interface":"back","average":{"1min":1.8460000000000001,"5min":1.954,"15min":0.96799999999999997},"min":{"1min":0.17699999999999999,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":6.2510000000000003,"5min":8.3900000000000006,"15min":8.3900000000000006},"last":1.028},{"interface":"front","average":{"1min":1.992,"5min":1.8220000000000001,"15min":0.91000000000000003},"min":{"1min":0.217,"5min":0.187,"15min":0.186},"max":{"1min":7.4169999999999998,"5min":12.532,"15min":12.532},"last":1.591}]},{"osd":2,"last update":"Wed Sep 11 08:47:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.6379999999999999,"15min":1.0089999999999999},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":5.7489999999999997,"5min":12.481,"15min":19.940000000000001},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.9179999999999999,"15min":1.077},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":4.2530000000000001,"5min":8.3049999999999997,"15min":20.757000000000001},"last":0.28599999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:47:09 2024","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":2.1749999999999998,"15min":1.0960000000000001},"min":{"1min":0.312,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.5709999999999997,"5min":11.951000000000001,"15min":11.951000000000001},"last":0.33000000000000002},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.919,"15min":1.0109999999999999},"min":{"1min":0.28100000000000003,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":7.6399999999999997,"5min":7.6399999999999997,"15min":7.6399999999999997},"last":0.81999999999999995}]},{"osd":4,"last update":"Wed Sep 11 08:47:33 2024","interfaces":[{"interface":"back","average":{"1min":1.7470000000000001,"5min":1.5429999999999999,"15min":0.93300000000000005},"min":{"1min":0.32100000000000001,"5min":0.26600000000000001,"15min":0.25},"max":{"1min":6.2009999999999996,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.7949999999999999,"5min":1.46,"15min":0.90700000000000003},"min":{"1min":0.30599999999999999,"5min":0.255,"15min":0.255},"max":{"1min":7.4630000000000001,"5min":7.4630000000000001,"15min":7.4630000000000001},"last":1.639}]},{"osd":5,"last update":"Wed Sep 11 08:47:56 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5629999999999999,"15min":0.98699999999999999},"min":{"1min":0.33700000000000002,"5min":0.30299999999999999,"15min":0.27800000000000002},"max":{"1min":4.0209999999999999,"5min":7.4009999999999998,"15min":7.4009999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.4830000000000001,"15min":0.98499999999999999},"min":{"1min":0.45800000000000002,"5min":0.32700000000000001,"15min":0.28299999999999997},"max":{"1min":6.2240000000000002,"5min":6.2240000000000002,"15min":7.423},"last":1.0049999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:47:13 2024","interfaces":[{"interface":"back","average":{"1min":1.9139999999999999,"5min":1.6890000000000001,"15min":0.99199999999999999},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.1310000000000002,"5min":7.4240000000000004,"15min":7.4240000000000004},"last":0.374},{"interface":"front","average":{"1min":2.149,"5min":1.738,"15min":1.028},"min":{"1min":0.434,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":6.1740000000000004,"5min":7.4489999999999998,"15min":7.4489999999999998},"last":0.95799999999999996}]},{"osd":7,"last update":"Wed Sep 11 08:47:45 2024","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.6519999999999999,"15min":0.999},"min":{"1min":0.32800000000000001,"5min":0.313,"15min":0.313},"max":{"1min":5.8179999999999996,"5min":7.4900000000000002,"15min":7.4900000000000002},"last":1.659},{"interface":"front","average":{"1min":2.0150000000000001,"5min":1.629,"15min":1.0209999999999999},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.307},"max":{"1min":6.0800000000000001,"5min":7.4450000000000003,"15min":7.4450000000000003},"last":0.439}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1419444224,"data_stored":1419444224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1344536576,"data_stored":1344536576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1369247744,"data_stored":1369243672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1154154496,"data_stored":1154154496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":849412096,"data_stored":849412096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1551175680,"data_stored":1551171608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":847314944,"data_stored":847314944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":835588096,"data_stored":835584024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:48:15.014 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-11T08:48:15.014 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:48:15.014 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-11T08:48:15.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-11T08:48:15.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:15.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:15.100 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:15.100 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:15.201 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:15.230 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:15.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:15.231 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:15.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:15 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1784983842' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:48:15.319 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:15.320 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:15 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1784983842' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:48:15.320 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:15.331 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:15.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:15.359 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:15.359 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:15.460 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:15.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:15.486 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:15.486 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:15.586 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:15.615 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:15.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:15.616 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:15.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:15 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1784983842' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:48:15.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:15.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:15.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:15.759 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:15.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:15.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:15.895 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:15.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:16.030 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:16.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:16.031 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:16.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:16.166 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:16.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:16.167 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:16.173 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:48:16.174 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:48:16.174 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:48:16.174 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:48:16.174 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:48:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:48:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:48:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:48:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:48:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:48:16.175 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:48:16.175 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:48:16.175 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:48:16.175 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:48:16.175 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:48:16.175 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:48:16.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:16.294 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:48:16.313 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:16.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:16.314 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:16.414 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:16.456 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:16.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:16.458 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:16.558 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:16.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:16 smithi092 ceph-mon[29647]: pgmap v400: 148 pgs: 148 active+clean; 2.5 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-11T08:48:16.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:16 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1394784527' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:48:16.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:16 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:48:16.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:16 smithi092 ceph-mon[41114]: pgmap v400: 148 pgs: 148 active+clean; 2.5 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-11T08:48:16.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:16 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1394784527' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:48:16.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:16 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:48:16.579 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:16 smithi145 ceph-mon[33384]: pgmap v400: 148 pgs: 148 active+clean; 2.5 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-11T08:48:16.579 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:16 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1394784527' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:48:16.579 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:16 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:48:16.594 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:16.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:16.596 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:16.697 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:16.729 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:16.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:16.730 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:16.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:16.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:16.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:16.911 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:48:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:48:16.969 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:17.001 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:17.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:17.002 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:17.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:17.152 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:17.159 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:17.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:17.160 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:17.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:17.294 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:17.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:17.295 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:17.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:17.411 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:17 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:48:17.411 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:17 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:48:17.411 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:17 smithi092 ceph-mon[29647]: osdmap e197: 8 total, 8 up, 8 in 2024-09-11T08:48:17.411 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:17 smithi092 ceph-mon[29647]: 2.2 scrub starts 2024-09-11T08:48:17.411 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:17 smithi092 ceph-mon[29647]: 2.2 scrub ok 2024-09-11T08:48:17.411 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:17 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:17.411 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:17 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:48:17.411 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:17 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:48:17.411 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:17 smithi092 ceph-mon[41114]: osdmap e197: 8 total, 8 up, 8 in 2024-09-11T08:48:17.411 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:17 smithi092 ceph-mon[41114]: 2.2 scrub starts 2024-09-11T08:48:17.412 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:17 smithi092 ceph-mon[41114]: 2.2 scrub ok 2024-09-11T08:48:17.412 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:17 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:17.433 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:17.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:17.434 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:17.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:17.576 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:17.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:17.577 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:17.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:17 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:48:17.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:17 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:48:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:17 smithi145 ceph-mon[33384]: osdmap e197: 8 total, 8 up, 8 in 2024-09-11T08:48:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:17 smithi145 ceph-mon[33384]: 2.2 scrub starts 2024-09-11T08:48:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:17 smithi145 ceph-mon[33384]: 2.2 scrub ok 2024-09-11T08:48:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:17 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:17.677 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:17.708 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:17.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:17.709 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:17.810 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:17.848 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:17.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:17.849 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:17.950 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:17.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:17.980 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:17.980 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:18.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:18.121 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:18.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:18.122 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:18.223 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:18.251 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:18.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:18.252 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:18.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:18.387 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:18.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:18.389 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:18.489 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:18.527 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:18.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:18.528 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:18.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:18 smithi092 ceph-mon[29647]: pgmap v402: 148 pgs: 148 active+clean; 2.1 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T08:48:18.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:18 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T08:48:18.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:18 smithi092 ceph-mon[29647]: 6.f scrub ok 2024-09-11T08:48:18.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:18 smithi092 ceph-mon[41114]: pgmap v402: 148 pgs: 148 active+clean; 2.1 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T08:48:18.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:18 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T08:48:18.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:18 smithi092 ceph-mon[41114]: 6.f scrub ok 2024-09-11T08:48:18.629 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:18.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:18.659 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:18.659 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:18.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:18 smithi145 ceph-mon[33384]: pgmap v402: 148 pgs: 148 active+clean; 2.1 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T08:48:18.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:18 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T08:48:18.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:18 smithi145 ceph-mon[33384]: 6.f scrub ok 2024-09-11T08:48:18.761 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:18.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:18.795 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:18.795 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:18.895 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:18.933 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:18.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:18.934 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:19.002 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:19.034 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:19.065 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:19.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:19.066 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:19.137 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:48:19.167 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:19.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:19.202 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:19.202 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:19.303 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:19.334 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:19.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:19.335 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:19.436 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:19.465 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:19.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:19.466 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:19.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:19.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[29647]: 5.1f scrub starts 2024-09-11T08:48:19.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[29647]: 5.1f scrub ok 2024-09-11T08:48:19.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[29647]: 5.1b scrub starts 2024-09-11T08:48:19.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[29647]: 5.1b scrub ok 2024-09-11T08:48:19.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[29647]: 6.10 scrub starts 2024-09-11T08:48:19.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[29647]: 6.10 scrub ok 2024-09-11T08:48:19.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[29647]: 5.10 scrub starts 2024-09-11T08:48:19.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[29647]: 5.10 scrub ok 2024-09-11T08:48:19.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[41114]: 5.1f scrub starts 2024-09-11T08:48:19.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[41114]: 5.1f scrub ok 2024-09-11T08:48:19.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[41114]: 5.1b scrub starts 2024-09-11T08:48:19.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[41114]: 5.1b scrub ok 2024-09-11T08:48:19.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[41114]: 6.10 scrub starts 2024-09-11T08:48:19.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[41114]: 6.10 scrub ok 2024-09-11T08:48:19.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[41114]: 5.10 scrub starts 2024-09-11T08:48:19.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:19 smithi092 ceph-mon[41114]: 5.10 scrub ok 2024-09-11T08:48:19.617 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:19.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:19.618 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:19.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:19 smithi145 ceph-mon[33384]: 5.1f scrub starts 2024-09-11T08:48:19.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:19 smithi145 ceph-mon[33384]: 5.1f scrub ok 2024-09-11T08:48:19.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:19 smithi145 ceph-mon[33384]: 5.1b scrub starts 2024-09-11T08:48:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:19 smithi145 ceph-mon[33384]: 5.1b scrub ok 2024-09-11T08:48:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:19 smithi145 ceph-mon[33384]: 6.10 scrub starts 2024-09-11T08:48:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:19 smithi145 ceph-mon[33384]: 6.10 scrub ok 2024-09-11T08:48:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:19 smithi145 ceph-mon[33384]: 5.10 scrub starts 2024-09-11T08:48:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:19 smithi145 ceph-mon[33384]: 5.10 scrub ok 2024-09-11T08:48:19.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:19.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:19.757 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:19.757 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:19.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:48:19.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:19.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:19.905 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:19.905 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:20.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:20.044 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:20.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:20.045 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:20.146 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:20.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:20.201 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:20.201 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:20.301 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:20.331 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:20.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:20.332 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:20.414 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:48:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-11T08:48:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-11T08:48:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:48:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:65018 7.e 7:7bf36b07:::benchmark_data_smithi092_84565_object65017:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:48:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:45.059616+0000", 2024-09-11T08:48:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 395.27409506399999, 2024-09-11T08:48:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.112304603, 2024-09-11T08:48:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 65018 2024-09-11T08:48:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059616+0000", 2024-09-11T08:48:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059616+0000", 2024-09-11T08:48:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059615+0000", 2024-09-11T08:48:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059639+0000", 2024-09-11T08:48:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4264999999999999e-05 2024-09-11T08:48:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059653+0000", 2024-09-11T08:48:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3691e-05 2024-09-11T08:48:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.059659+0000", 2024-09-11T08:48:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.263e-06 2024-09-11T08:48:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062216+0000", 2024-09-11T08:48:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025571280000000001 2024-09-11T08:48:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062351+0000", 2024-09-11T08:48:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013464400000000001 2024-09-11T08:48:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062504+0000", 2024-09-11T08:48:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015295099999999999 2024-09-11T08:48:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.064861+0000", 2024-09-11T08:48:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023569789999999999 2024-09-11T08:48:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.069304+0000", 2024-09-11T08:48:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044433959999999996 2024-09-11T08:48:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.171603+0000", 2024-09-11T08:48:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10229874 2024-09-11T08:48:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.171882+0000", 2024-09-11T08:48:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027927099999999999 2024-09-11T08:48:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.171921+0000", 2024-09-11T08:48:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8711000000000003e-05 2024-09-11T08:48:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:65021 7.e 7:7eb86f3b:::benchmark_data_smithi092_84565_object65020:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:48:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:45.062298+0000", 2024-09-11T08:48:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 395.27141343599999, 2024-09-11T08:48:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11573973999999999, 2024-09-11T08:48:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 65021 2024-09-11T08:48:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062298+0000", 2024-09-11T08:48:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062298+0000", 2024-09-11T08:48:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062296+0000", 2024-09-11T08:48:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:48:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062323+0000", 2024-09-11T08:48:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6262999999999999e-05 2024-09-11T08:48:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062337+0000", 2024-09-11T08:48:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3883000000000001e-05 2024-09-11T08:48:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.062343+0000", 2024-09-11T08:48:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.438e-06 2024-09-11T08:48:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.064881+0000", 2024-09-11T08:48:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025380279999999999 2024-09-11T08:48:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.065038+0000", 2024-09-11T08:48:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015697000000000001 2024-09-11T08:48:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.065171+0000", 2024-09-11T08:48:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013319499999999999 2024-09-11T08:48:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.069338+0000", 2024-09-11T08:48:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041663999999999998 2024-09-11T08:48:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.069603+0000", 2024-09-11T08:48:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026567600000000001 2024-09-11T08:48:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.177874+0000", 2024-09-11T08:48:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.108271144 2024-09-11T08:48:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.178003+0000", 2024-09-11T08:48:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012872900000000001 2024-09-11T08:48:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:45.178038+0000", 2024-09-11T08:48:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4688999999999999e-05 2024-09-11T08:48:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:122188 7.e 7:78531627:::benchmark_data_smithi092_84565_object122187:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:48:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:38.166756+0000", 2024-09-11T08:48:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 342.16695517199997, 2024-09-11T08:48:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.122095332, 2024-09-11T08:48:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122188 2024-09-11T08:48:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.166756+0000", 2024-09-11T08:48:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.166756+0000", 2024-09-11T08:48:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.166755+0000", 2024-09-11T08:48:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.166776+0000", 2024-09-11T08:48:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1301000000000002e-05 2024-09-11T08:48:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.166787+0000", 2024-09-11T08:48:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0686999999999999e-05 2024-09-11T08:48:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.166792+0000", 2024-09-11T08:48:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4e-06 2024-09-11T08:48:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.166850+0000", 2024-09-11T08:48:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7219000000000002e-05 2024-09-11T08:48:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.167337+0000", 2024-09-11T08:48:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048781699999999998 2024-09-11T08:48:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.167481+0000", 2024-09-11T08:48:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014400099999999999 2024-09-11T08:48:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.172215+0000", 2024-09-11T08:48:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047336779999999998 2024-09-11T08:48:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.173225+0000", 2024-09-11T08:48:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001009519 2024-09-11T08:48:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.288575+0000", 2024-09-11T08:48:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11535047900000001 2024-09-11T08:48:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.288718+0000", 2024-09-11T08:48:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014307 2024-09-11T08:48:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.288852+0000", 2024-09-11T08:48:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133609 2024-09-11T08:48:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:122203 7.e 7:778cd3fb:::benchmark_data_smithi092_84565_object122202:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:48:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:38.180983+0000", 2024-09-11T08:48:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 342.15272864500002, 2024-09-11T08:48:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.108224366, 2024-09-11T08:48:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122203 2024-09-11T08:48:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.180983+0000", 2024-09-11T08:48:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.180983+0000", 2024-09-11T08:48:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.180982+0000", 2024-09-11T08:48:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181008+0000", 2024-09-11T08:48:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6332000000000001e-05 2024-09-11T08:48:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181074+0000", 2024-09-11T08:48:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6341999999999996e-05 2024-09-11T08:48:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181080+0000", 2024-09-11T08:48:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2179999999999996e-06 2024-09-11T08:48:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181512+0000", 2024-09-11T08:48:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00043139699999999998 2024-09-11T08:48:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181772+0000", 2024-09-11T08:48:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026024700000000002 2024-09-11T08:48:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181931+0000", 2024-09-11T08:48:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015916199999999999 2024-09-11T08:48:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.189271+0000", 2024-09-11T08:48:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007340254 2024-09-11T08:48:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.189314+0000", 2024-09-11T08:48:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2324000000000003e-05 2024-09-11T08:48:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.288992+0000", 2024-09-11T08:48:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.099678418000000005 2024-09-11T08:48:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.289106+0000", 2024-09-11T08:48:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113891 2024-09-11T08:48:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.289207+0000", 2024-09-11T08:48:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101184 2024-09-11T08:48:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:122204 7.e 7:7bdd875c:::benchmark_data_smithi092_84565_object122203:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:48:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:38.181101+0000", 2024-09-11T08:48:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 342.15261054899997, 2024-09-11T08:48:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.109219345, 2024-09-11T08:48:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122204 2024-09-11T08:48:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181101+0000", 2024-09-11T08:48:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181101+0000", 2024-09-11T08:48:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181100+0000", 2024-09-11T08:48:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:48:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181121+0000", 2024-09-11T08:48:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0336e-05 2024-09-11T08:48:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181129+0000", 2024-09-11T08:48:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9880000000000004e-06 2024-09-11T08:48:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.181133+0000", 2024-09-11T08:48:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4800000000000003e-06 2024-09-11T08:48:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.185885+0000", 2024-09-11T08:48:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047516800000000003 2024-09-11T08:48:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.185987+0000", 2024-09-11T08:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010206599999999999 2024-09-11T08:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.186113+0000", 2024-09-11T08:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012586499999999999 2024-09-11T08:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.189355+0000", 2024-09-11T08:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032418759999999999 2024-09-11T08:48:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.191836+0000", 2024-09-11T08:48:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002481254 2024-09-11T08:48:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.289325+0000", 2024-09-11T08:48:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.097488915999999995 2024-09-11T08:48:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.290282+0000", 2024-09-11T08:48:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00095695399999999999 2024-09-11T08:48:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.290320+0000", 2024-09-11T08:48:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8665999999999999e-05 2024-09-11T08:48:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:122211 7.e 7:7644088b:::benchmark_data_smithi092_84565_object122210:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:48:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:38.187420+0000", 2024-09-11T08:48:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 342.14629157100001, 2024-09-11T08:48:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103023819, 2024-09-11T08:48:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122211 2024-09-11T08:48:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.187420+0000", 2024-09-11T08:48:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.187420+0000", 2024-09-11T08:48:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.187419+0000", 2024-09-11T08:48:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:48:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.187437+0000", 2024-09-11T08:48:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8065999999999999e-05 2024-09-11T08:48:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.187447+0000", 2024-09-11T08:48:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7000000000000003e-06 2024-09-11T08:48:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.187451+0000", 2024-09-11T08:48:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4220000000000002e-06 2024-09-11T08:48:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.189375+0000", 2024-09-11T08:48:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019236629999999999 2024-09-11T08:48:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.189444+0000", 2024-09-11T08:48:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9036999999999996e-05 2024-09-11T08:48:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.189522+0000", 2024-09-11T08:48:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8615000000000005e-05 2024-09-11T08:48:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.191958+0000", 2024-09-11T08:48:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024359690000000001 2024-09-11T08:48:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.192499+0000", 2024-09-11T08:48:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054097100000000001 2024-09-11T08:48:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.290364+0000", 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.097864958000000002 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.290406+0000", 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1990999999999998e-05 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.290444+0000", 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.748e-05 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:122815 7.e 7:7bbdcb6b:::benchmark_data_smithi092_84565_object122814:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:48:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:38.925138+0000", 2024-09-11T08:48:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 341.408573332, 2024-09-11T08:48:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.110102964, 2024-09-11T08:48:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122815 2024-09-11T08:48:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.925138+0000", 2024-09-11T08:48:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.925138+0000", 2024-09-11T08:48:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.925138+0000", 2024-09-11T08:48:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:48:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.925160+0000", 2024-09-11T08:48:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2076e-05 2024-09-11T08:48:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.925167+0000", 2024-09-11T08:48:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2259999999999999e-06 2024-09-11T08:48:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.925171+0000", 2024-09-11T08:48:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9160000000000003e-06 2024-09-11T08:48:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.926590+0000", 2024-09-11T08:48:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014194100000000001 2024-09-11T08:48:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.926701+0000", 2024-09-11T08:48:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110402 2024-09-11T08:48:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.927767+0000", 2024-09-11T08:48:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010665869999999999 2024-09-11T08:48:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.930973+0000", 2024-09-11T08:48:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00320587 2024-09-11T08:48:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.931384+0000", 2024-09-11T08:48:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00041067099999999999 2024-09-11T08:48:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:39.035160+0000", 2024-09-11T08:48:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10377653000000001 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:39.035207+0000", 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6576000000000003e-05 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:39.035241+0000", 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4161e-05 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:149662 7.e 7:7558e195:::benchmark_data_smithi092_84565_object24134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:48:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:06.838684+0000", 2024-09-11T08:48:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 313.49502774000001, 2024-09-11T08:48:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11249593500000001, 2024-09-11T08:48:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 149662 2024-09-11T08:48:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.838684+0000", 2024-09-11T08:48:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.838684+0000", 2024-09-11T08:48:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.838683+0000", 2024-09-11T08:48:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:48:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.838689+0000", 2024-09-11T08:48:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5949999999999998e-06 2024-09-11T08:48:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.838698+0000", 2024-09-11T08:48:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6949999999999995e-06 2024-09-11T08:48:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.838703+0000", 2024-09-11T08:48:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1950000000000002e-06 2024-09-11T08:48:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.839996+0000", 2024-09-11T08:48:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012930350000000001 2024-09-11T08:48:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.840087+0000", 2024-09-11T08:48:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0987000000000001e-05 2024-09-11T08:48:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.840184+0000", 2024-09-11T08:48:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6106000000000004e-05 2024-09-11T08:48:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850972+0000", 2024-09-11T08:48:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010788904 2024-09-11T08:48:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.851257+0000", 2024-09-11T08:48:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028416200000000002 2024-09-11T08:48:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.951095+0000", 2024-09-11T08:48:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.099838475999999995 2024-09-11T08:48:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.951140+0000", 2024-09-11T08:48:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4660000000000003e-05 2024-09-11T08:48:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.951180+0000", 2024-09-11T08:48:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0005000000000001e-05 2024-09-11T08:48:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:149668 7.e 7:72833b7f:::benchmark_data_smithi092_84565_object24140:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:48:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:06.842304+0000", 2024-09-11T08:48:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 313.491407095, 2024-09-11T08:48:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.115108186, 2024-09-11T08:48:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 149668 2024-09-11T08:48:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.842304+0000", 2024-09-11T08:48:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.842304+0000", 2024-09-11T08:48:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.842303+0000", 2024-09-11T08:48:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.842308+0000", 2024-09-11T08:48:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1900000000000003e-06 2024-09-11T08:48:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.842321+0000", 2024-09-11T08:48:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2679e-05 2024-09-11T08:48:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.842327+0000", 2024-09-11T08:48:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8799999999999996e-06 2024-09-11T08:48:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.847003+0000", 2024-09-11T08:48:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046758160000000002 2024-09-11T08:48:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.847105+0000", 2024-09-11T08:48:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010242999999999999 2024-09-11T08:48:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.847211+0000", 2024-09-11T08:48:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105655 2024-09-11T08:48:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.851306+0000", 2024-09-11T08:48:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040949929999999999 2024-09-11T08:48:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.854970+0000", 2024-09-11T08:48:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003663805 2024-09-11T08:48:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.957107+0000", 2024-09-11T08:48:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102137193 2024-09-11T08:48:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.957267+0000", 2024-09-11T08:48:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160106 2024-09-11T08:48:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.957413+0000", 2024-09-11T08:48:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014575100000000001 2024-09-11T08:48:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:149675 7.e 7:78c0dd9b:::benchmark_data_smithi092_84565_object24147:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:48:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:06.849986+0000", 2024-09-11T08:48:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 313.48372558400001, 2024-09-11T08:48:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11731897400000001, 2024-09-11T08:48:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 149675 2024-09-11T08:48:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.849986+0000", 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.849986+0000", 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.849985+0000", 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:48:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.849989+0000", 2024-09-11T08:48:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5419999999999999e-06 2024-09-11T08:48:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.849997+0000", 2024-09-11T08:48:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0150000000000003e-06 2024-09-11T08:48:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850001+0000", 2024-09-11T08:48:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9509999999999999e-06 2024-09-11T08:48:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.851332+0000", 2024-09-11T08:48:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013308560000000001 2024-09-11T08:48:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.851428+0000", 2024-09-11T08:48:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6181999999999995e-05 2024-09-11T08:48:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.851640+0000", 2024-09-11T08:48:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021141000000000001 2024-09-11T08:48:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.951068+0000", 2024-09-11T08:48:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.099427984999999997 2024-09-11T08:48:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.951212+0000", 2024-09-11T08:48:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143979 2024-09-11T08:48:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.966788+0000", 2024-09-11T08:48:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01557618 2024-09-11T08:48:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.966972+0000", 2024-09-11T08:48:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018452400000000001 2024-09-11T08:48:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.967305+0000", 2024-09-11T08:48:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033248799999999998 2024-09-11T08:48:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:149676 7.e 7:74140a03:::benchmark_data_smithi092_84565_object24148:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:48:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:06.850577+0000", 2024-09-11T08:48:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 313.48313422500001, 2024-09-11T08:48:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11728794100000001, 2024-09-11T08:48:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 149676 2024-09-11T08:48:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850577+0000", 2024-09-11T08:48:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850577+0000", 2024-09-11T08:48:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850576+0000", 2024-09-11T08:48:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:48:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850582+0000", 2024-09-11T08:48:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3079999999999998e-06 2024-09-11T08:48:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850593+0000", 2024-09-11T08:48:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1253000000000001e-05 2024-09-11T08:48:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.850599+0000", 2024-09-11T08:48:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.609e-06 2024-09-11T08:48:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.855054+0000", 2024-09-11T08:48:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044557570000000003 2024-09-11T08:48:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.855255+0000", 2024-09-11T08:48:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020054099999999999 2024-09-11T08:48:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.855366+0000", 2024-09-11T08:48:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111146 2024-09-11T08:48:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.951218+0000", 2024-09-11T08:48:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.095852437999999998 2024-09-11T08:48:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.951392+0000", 2024-09-11T08:48:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017315700000000001 2024-09-11T08:48:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.967493+0000", 2024-09-11T08:48:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016101358999999999 2024-09-11T08:48:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.967702+0000", 2024-09-11T08:48:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208851 2024-09-11T08:48:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.967865+0000", 2024-09-11T08:48:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016345199999999999 2024-09-11T08:48:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:149685 7.8 7:17ce0a5b:::benchmark_data_smithi092_84565_object24157:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:48:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:06.857814+0000", 2024-09-11T08:48:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 313.47589783400002, 2024-09-11T08:48:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10512039099999999, 2024-09-11T08:48:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 149685 2024-09-11T08:48:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.857814+0000", 2024-09-11T08:48:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.857814+0000", 2024-09-11T08:48:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.857813+0000", 2024-09-11T08:48:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.857818+0000", 2024-09-11T08:48:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8289999999999996e-06 2024-09-11T08:48:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.950984+0000", 2024-09-11T08:48:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.093165534999999994 2024-09-11T08:48:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.950997+0000", 2024-09-11T08:48:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3461e-05 2024-09-11T08:48:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.954783+0000", 2024-09-11T08:48:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037859780000000002 2024-09-11T08:48:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.954893+0000", 2024-09-11T08:48:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109236 2024-09-11T08:48:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,7", 2024-09-11T08:48:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.954996+0000", 2024-09-11T08:48:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010373 2024-09-11T08:48:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.958806+0000", 2024-09-11T08:48:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003809729 2024-09-11T08:48:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.962452+0000", 2024-09-11T08:48:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003645676 2024-09-11T08:48:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.962839+0000", 2024-09-11T08:48:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038698200000000002 2024-09-11T08:48:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.962893+0000", 2024-09-11T08:48:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4413999999999998e-05 2024-09-11T08:48:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:06.962934+0000", 2024-09-11T08:48:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1031000000000001e-05 2024-09-11T08:48:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:150318 7.e 7:75745c33:::benchmark_data_smithi092_84565_object24790:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:48:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:07.591034+0000", 2024-09-11T08:48:20.489 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 312.74267770699998, 2024-09-11T08:48:20.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20382811000000001, 2024-09-11T08:48:20.489 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.489 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.489 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.489 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.489 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.489 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 150318 2024-09-11T08:48:20.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.489 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591034+0000", 2024-09-11T08:48:20.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591034+0000", 2024-09-11T08:48:20.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591032+0000", 2024-09-11T08:48:20.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:20.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591039+0000", 2024-09-11T08:48:20.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4189999999999999e-06 2024-09-11T08:48:20.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591053+0000", 2024-09-11T08:48:20.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4749000000000001e-05 2024-09-11T08:48:20.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591059+0000", 2024-09-11T08:48:20.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.604e-06 2024-09-11T08:48:20.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591198+0000", 2024-09-11T08:48:20.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013907700000000001 2024-09-11T08:48:20.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.591470+0000", 2024-09-11T08:48:20.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027224200000000001 2024-09-11T08:48:20.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.592889+0000", 2024-09-11T08:48:20.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00141906 2024-09-11T08:48:20.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.598327+0000", 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054377890000000002 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.600216+0000", 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018887369999999999 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.794706+0000", 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.194490462 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.794805+0000", 2024-09-11T08:48:20.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8669e-05 2024-09-11T08:48:20.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.794862+0000", 2024-09-11T08:48:20.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6833000000000003e-05 2024-09-11T08:48:20.494 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.494 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.494 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.494 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:150321 7.e 7:7128a19a:::benchmark_data_smithi092_84565_object24793:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:48:20.494 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:07.594313+0000", 2024-09-11T08:48:20.494 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 312.73939842300001, 2024-09-11T08:48:20.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20085424599999999, 2024-09-11T08:48:20.495 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.495 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.495 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.495 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.495 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.495 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 150321 2024-09-11T08:48:20.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.495 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.594313+0000", 2024-09-11T08:48:20.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.594313+0000", 2024-09-11T08:48:20.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.594311+0000", 2024-09-11T08:48:20.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:48:20.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.594318+0000", 2024-09-11T08:48:20.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-11T08:48:20.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.594329+0000", 2024-09-11T08:48:20.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1574e-05 2024-09-11T08:48:20.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.594335+0000", 2024-09-11T08:48:20.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6690000000000003e-06 2024-09-11T08:48:20.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.595742+0000", 2024-09-11T08:48:20.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001407373 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.595820+0000", 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7473999999999997e-05 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.595905+0000", 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5161999999999995e-05 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.598524+0000", 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026189059999999998 2024-09-11T08:48:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.600433+0000", 2024-09-11T08:48:20.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019096569999999999 2024-09-11T08:48:20.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.795062+0000", 2024-09-11T08:48:20.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19462868899999999 2024-09-11T08:48:20.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.795133+0000", 2024-09-11T08:48:20.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1321000000000004e-05 2024-09-11T08:48:20.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:07.795167+0000", 2024-09-11T08:48:20.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4019000000000001e-05 2024-09-11T08:48:20.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.500 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.500 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:165644 7.e 7:72bc5608:::benchmark_data_smithi092_84565_object40116:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T08:48:20.500 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:21.990537+0000", 2024-09-11T08:48:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 298.34317470600001, 2024-09-11T08:48:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.120823112, 2024-09-11T08:48:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 165644 2024-09-11T08:48:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990537+0000", 2024-09-11T08:48:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990537+0000", 2024-09-11T08:48:20.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990536+0000", 2024-09-11T08:48:20.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:20.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990542+0000", 2024-09-11T08:48:20.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.925e-06 2024-09-11T08:48:20.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990558+0000", 2024-09-11T08:48:20.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6025e-05 2024-09-11T08:48:20.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990564+0000", 2024-09-11T08:48:20.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1199999999999999e-06 2024-09-11T08:48:20.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.001643+0000", 2024-09-11T08:48:20.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011079435 2024-09-11T08:48:20.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.002405+0000", 2024-09-11T08:48:20.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00076212999999999997 2024-09-11T08:48:20.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.002507+0000", 2024-09-11T08:48:20.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010155800000000001 2024-09-11T08:48:20.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.012490+0000", 2024-09-11T08:48:20.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099833030000000007 2024-09-11T08:48:20.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.017752+0000", 2024-09-11T08:48:20.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052617610000000002 2024-09-11T08:48:20.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.111298+0000", 2024-09-11T08:48:20.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.093546291000000004 2024-09-11T08:48:20.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.111332+0000", 2024-09-11T08:48:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4039999999999999e-05 2024-09-11T08:48:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.111360+0000", 2024-09-11T08:48:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7724000000000001e-05 2024-09-11T08:48:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:165648 7.e 7:7cd3e667:::benchmark_data_smithi092_84565_object40120:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T08:48:20.506 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:21.990836+0000", 2024-09-11T08:48:20.507 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 298.34287530400002, 2024-09-11T08:48:20.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.13651323500000001, 2024-09-11T08:48:20.507 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.507 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.507 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 165648 2024-09-11T08:48:20.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.507 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990836+0000", 2024-09-11T08:48:20.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990836+0000", 2024-09-11T08:48:20.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990823+0000", 2024-09-11T08:48:20.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999866 2024-09-11T08:48:20.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990842+0000", 2024-09-11T08:48:20.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8722000000000001e-05 2024-09-11T08:48:20.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990857+0000", 2024-09-11T08:48:20.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5017999999999999e-05 2024-09-11T08:48:20.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.990871+0000", 2024-09-11T08:48:20.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4874e-05 2024-09-11T08:48:20.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.005959+0000", 2024-09-11T08:48:20.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015087233 2024-09-11T08:48:20.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.008192+0000", 2024-09-11T08:48:20.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022335369999999998 2024-09-11T08:48:20.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.008315+0000", 2024-09-11T08:48:20.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012303499999999999 2024-09-11T08:48:20.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.012524+0000", 2024-09-11T08:48:20.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042090540000000003 2024-09-11T08:48:20.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.017757+0000", 2024-09-11T08:48:20.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052328549999999998 2024-09-11T08:48:20.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127227+0000", 2024-09-11T08:48:20.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.109469724 2024-09-11T08:48:20.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127288+0000", 2024-09-11T08:48:20.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0661000000000001e-05 2024-09-11T08:48:20.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127349+0000", 2024-09-11T08:48:20.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1885999999999997e-05 2024-09-11T08:48:20.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.512 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.512 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:165654 7.e 7:79193999:::benchmark_data_smithi092_84565_object40126:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T08:48:20.512 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:21.993884+0000", 2024-09-11T08:48:20.512 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 298.33982795899999, 2024-09-11T08:48:20.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.13364512100000001, 2024-09-11T08:48:20.513 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.513 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.513 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.513 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.513 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.513 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 165654 2024-09-11T08:48:20.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.513 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.993884+0000", 2024-09-11T08:48:20.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.993884+0000", 2024-09-11T08:48:20.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.993882+0000", 2024-09-11T08:48:20.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:20.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.993888+0000", 2024-09-11T08:48:20.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7699999999999998e-06 2024-09-11T08:48:20.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.993902+0000", 2024-09-11T08:48:20.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4158999999999999e-05 2024-09-11T08:48:20.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:21.993908+0000", 2024-09-11T08:48:20.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3940000000000004e-06 2024-09-11T08:48:20.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.012627+0000", 2024-09-11T08:48:20.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018719448 2024-09-11T08:48:20.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.014391+0000", 2024-09-11T08:48:20.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001763861 2024-09-11T08:48:20.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.014513+0000", 2024-09-11T08:48:20.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012184999999999999 2024-09-11T08:48:20.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.020541+0000", 2024-09-11T08:48:20.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060278110000000001 2024-09-11T08:48:20.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.022550+0000", 2024-09-11T08:48:20.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020094650000000002 2024-09-11T08:48:20.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127411+0000", 2024-09-11T08:48:20.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10486058299999999 2024-09-11T08:48:20.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127488+0000", 2024-09-11T08:48:20.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7458000000000003e-05 2024-09-11T08:48:20.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127529+0000", 2024-09-11T08:48:20.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0655999999999998e-05 2024-09-11T08:48:20.518 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.518 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.518 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.519 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:165664 7.e 7:7c58b4a4:::benchmark_data_smithi092_84565_object40136:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T08:48:20.519 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:22.006095+0000", 2024-09-11T08:48:20.519 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 298.327616358, 2024-09-11T08:48:20.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12157319699999999, 2024-09-11T08:48:20.519 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.519 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.519 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.519 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.519 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.519 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 165664 2024-09-11T08:48:20.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.519 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006095+0000", 2024-09-11T08:48:20.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006095+0000", 2024-09-11T08:48:20.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006095+0000", 2024-09-11T08:48:20.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:48:20.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006099+0000", 2024-09-11T08:48:20.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3479999999999997e-06 2024-09-11T08:48:20.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006119+0000", 2024-09-11T08:48:20.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0120999999999999e-05 2024-09-11T08:48:20.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006125+0000", 2024-09-11T08:48:20.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8950000000000003e-06 2024-09-11T08:48:20.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.017785+0000", 2024-09-11T08:48:20.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011659653000000001 2024-09-11T08:48:20.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.522 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:20.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.017884+0000", 2024-09-11T08:48:20.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9389000000000007e-05 2024-09-11T08:48:20.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.017993+0000", 2024-09-11T08:48:20.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109109 2024-09-11T08:48:20.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.022566+0000", 2024-09-11T08:48:20.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045726350000000002 2024-09-11T08:48:20.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.023501+0000", 2024-09-11T08:48:20.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093503 2024-09-11T08:48:20.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127566+0000", 2024-09-11T08:48:20.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104065012 2024-09-11T08:48:20.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127635+0000", 2024-09-11T08:48:20.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9608000000000002e-05 2024-09-11T08:48:20.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127668+0000", 2024-09-11T08:48:20.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3065000000000003e-05 2024-09-11T08:48:20.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.526 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.526 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:165665 7.e 7:72a055f1:::benchmark_data_smithi092_84565_object40137:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T08:48:20.526 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:22.006131+0000", 2024-09-11T08:48:20.526 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 298.32758007299998, 2024-09-11T08:48:20.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.121650015, 2024-09-11T08:48:20.526 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.527 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.527 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.527 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.527 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.527 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 165665 2024-09-11T08:48:20.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.527 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006131+0000", 2024-09-11T08:48:20.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006131+0000", 2024-09-11T08:48:20.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006131+0000", 2024-09-11T08:48:20.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:48:20.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006134+0000", 2024-09-11T08:48:20.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0259999999999999e-06 2024-09-11T08:48:20.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006147+0000", 2024-09-11T08:48:20.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.294e-05 2024-09-11T08:48:20.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.006158+0000", 2024-09-11T08:48:20.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1144999999999999e-05 2024-09-11T08:48:20.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.020564+0000", 2024-09-11T08:48:20.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014405545000000001 2024-09-11T08:48:20.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.020641+0000", 2024-09-11T08:48:20.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7546999999999995e-05 2024-09-11T08:48:20.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.020718+0000", 2024-09-11T08:48:20.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6656000000000006e-05 2024-09-11T08:48:20.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.022668+0000", 2024-09-11T08:48:20.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001950476 2024-09-11T08:48:20.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.023605+0000", 2024-09-11T08:48:20.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093641799999999995 2024-09-11T08:48:20.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127714+0000", 2024-09-11T08:48:20.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104109722 2024-09-11T08:48:20.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127751+0000", 2024-09-11T08:48:20.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6189999999999997e-05 2024-09-11T08:48:20.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:22.127781+0000", 2024-09-11T08:48:20.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0880000000000002e-05 2024-09-11T08:48:20.532 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.532 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.532 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.532 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:242111 7.e 7:70cafa55:::benchmark_data_smithi092_84565_object116583:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-11T08:48:20.532 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:36.320509+0000", 2024-09-11T08:48:20.532 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 224.013202261, 2024-09-11T08:48:20.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.110695035, 2024-09-11T08:48:20.532 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:20.532 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:20.532 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:20.533 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:20.533 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:20.533 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 242111 2024-09-11T08:48:20.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.533 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:20.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:20.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320509+0000", 2024-09-11T08:48:20.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:20.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320509+0000", 2024-09-11T08:48:20.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:20.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:20.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320508+0000", 2024-09-11T08:48:20.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:20.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:20.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320514+0000", 2024-09-11T08:48:20.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1959999999999996e-06 2024-09-11T08:48:20.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:20.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320526+0000", 2024-09-11T08:48:20.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2131e-05 2024-09-11T08:48:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320533+0000", 2024-09-11T08:48:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0700000000000001e-06 2024-09-11T08:48:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320708+0000", 2024-09-11T08:48:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017503500000000001 2024-09-11T08:48:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320810+0000", 2024-09-11T08:48:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101578 2024-09-11T08:48:20.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:48:20.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320920+0000", 2024-09-11T08:48:20.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110511 2024-09-11T08:48:20.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:20.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.323603+0000", 2024-09-11T08:48:20.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002682612 2024-09-11T08:48:20.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.323679+0000", 2024-09-11T08:48:20.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6241000000000004e-05 2024-09-11T08:48:20.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:20.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430763+0000", 2024-09-11T08:48:20.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10708406400000001 2024-09-11T08:48:20.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:20.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430956+0000", 2024-09-11T08:48:20.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000192497 2024-09-11T08:48:20.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:20.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:20.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:20.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.431204+0000", 2024-09-11T08:48:20.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024852899999999999 2024-09-11T08:48:20.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.538 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:20.538 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:20.538 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:48:20.538 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[29647]: 8.5 scrub starts 2024-09-11T08:48:20.539 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[29647]: 4.11 scrub starts 2024-09-11T08:48:20.539 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[29647]: 4.11 scrub ok 2024-09-11T08:48:20.539 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[29647]: pgmap v403: 148 pgs: 148 active+clean; 2.1 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T08:48:20.539 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[29647]: 6.7 scrub starts 2024-09-11T08:48:20.539 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[29647]: 6.7 scrub ok 2024-09-11T08:48:20.539 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[29647]: 8.5 scrub ok 2024-09-11T08:48:20.539 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[29647]: 4.1e scrub starts 2024-09-11T08:48:20.539 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[29647]: 4.1e scrub ok 2024-09-11T08:48:20.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[41114]: 8.5 scrub starts 2024-09-11T08:48:20.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[41114]: 4.11 scrub starts 2024-09-11T08:48:20.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[41114]: 4.11 scrub ok 2024-09-11T08:48:20.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[41114]: pgmap v403: 148 pgs: 148 active+clean; 2.1 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T08:48:20.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[41114]: 6.7 scrub starts 2024-09-11T08:48:20.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[41114]: 6.7 scrub ok 2024-09-11T08:48:20.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[41114]: 8.5 scrub ok 2024-09-11T08:48:20.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[41114]: 4.1e scrub starts 2024-09-11T08:48:20.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:20 smithi092 ceph-mon[41114]: 4.1e scrub ok 2024-09-11T08:48:20.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:20.553 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:20.554 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:20.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:20.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:20 smithi145 ceph-mon[33384]: 8.5 scrub starts 2024-09-11T08:48:20.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:20 smithi145 ceph-mon[33384]: 4.11 scrub starts 2024-09-11T08:48:20.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:20 smithi145 ceph-mon[33384]: 4.11 scrub ok 2024-09-11T08:48:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:20 smithi145 ceph-mon[33384]: pgmap v403: 148 pgs: 148 active+clean; 2.1 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T08:48:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:20 smithi145 ceph-mon[33384]: 6.7 scrub starts 2024-09-11T08:48:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:20 smithi145 ceph-mon[33384]: 6.7 scrub ok 2024-09-11T08:48:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:20 smithi145 ceph-mon[33384]: 8.5 scrub ok 2024-09-11T08:48:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:20 smithi145 ceph-mon[33384]: 4.1e scrub starts 2024-09-11T08:48:20.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:20 smithi145 ceph-mon[33384]: 4.1e scrub ok 2024-09-11T08:48:20.691 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:20.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:20.692 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:20.793 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:20.818 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:20.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:20.819 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:20.919 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:20.944 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:20.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:20.945 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:21.045 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:21.079 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:21.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:21.080 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:21.181 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:21.222 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:21.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:21.223 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:21.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:21.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:21.363 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:21.363 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:21.464 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:21.494 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:21.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:21.495 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:21.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[29647]: 5.1d scrub starts 2024-09-11T08:48:21.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[29647]: 5.1d scrub ok 2024-09-11T08:48:21.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T08:48:21.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[29647]: 4.1c scrub ok 2024-09-11T08:48:21.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[29647]: 4.8 scrub starts 2024-09-11T08:48:21.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[29647]: 4.8 scrub ok 2024-09-11T08:48:21.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[29647]: 4.e scrub starts 2024-09-11T08:48:21.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[29647]: 4.e scrub ok 2024-09-11T08:48:21.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[29647]: 3.11 scrub starts 2024-09-11T08:48:21.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[29647]: 3.11 scrub ok 2024-09-11T08:48:21.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:48:21] "GET /metrics HTTP/1.1" 200 224351 "" "Prometheus/2.43.0" 2024-09-11T08:48:21.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[41114]: 5.1d scrub starts 2024-09-11T08:48:21.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[41114]: 5.1d scrub ok 2024-09-11T08:48:21.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T08:48:21.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[41114]: 4.1c scrub ok 2024-09-11T08:48:21.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[41114]: 4.8 scrub starts 2024-09-11T08:48:21.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[41114]: 4.8 scrub ok 2024-09-11T08:48:21.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[41114]: 4.e scrub starts 2024-09-11T08:48:21.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[41114]: 4.e scrub ok 2024-09-11T08:48:21.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[41114]: 3.11 scrub starts 2024-09-11T08:48:21.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:21 smithi092 ceph-mon[41114]: 3.11 scrub ok 2024-09-11T08:48:21.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:21.635 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:21.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:21.636 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:21.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:21 smithi145 ceph-mon[33384]: 5.1d scrub starts 2024-09-11T08:48:21.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:21 smithi145 ceph-mon[33384]: 5.1d scrub ok 2024-09-11T08:48:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:21 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T08:48:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:21 smithi145 ceph-mon[33384]: 4.1c scrub ok 2024-09-11T08:48:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:21 smithi145 ceph-mon[33384]: 4.8 scrub starts 2024-09-11T08:48:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:21 smithi145 ceph-mon[33384]: 4.8 scrub ok 2024-09-11T08:48:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:21 smithi145 ceph-mon[33384]: 4.e scrub starts 2024-09-11T08:48:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:21 smithi145 ceph-mon[33384]: 4.e scrub ok 2024-09-11T08:48:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:21 smithi145 ceph-mon[33384]: 3.11 scrub starts 2024-09-11T08:48:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:21 smithi145 ceph-mon[33384]: 3.11 scrub ok 2024-09-11T08:48:21.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:21.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:21.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:21.868 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:21.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-11T08:48:21.923 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:21.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:21.925 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:22.025 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:22.061 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:22.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:22.062 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:22.163 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:22.192 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:22.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:22.193 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:22.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:22.334 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:22.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:22.335 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:22.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:22.453 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:22.509 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:22.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:22.510 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:22.594 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:22 smithi092 ceph-mon[29647]: 4.1f scrub starts 2024-09-11T08:48:22.595 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:22 smithi092 ceph-mon[29647]: 4.1f scrub ok 2024-09-11T08:48:22.595 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:22 smithi092 ceph-mon[29647]: pgmap v404: 148 pgs: 148 active+clean; 2.0 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T08:48:22.595 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:22 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:48:22.595 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:22 smithi092 ceph-mon[29647]: 5.8 scrub ok 2024-09-11T08:48:22.595 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:22 smithi092 ceph-mon[29647]: 4.c scrub starts 2024-09-11T08:48:22.595 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:22 smithi092 ceph-mon[29647]: 4.c scrub ok 2024-09-11T08:48:22.595 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:22 smithi092 ceph-mon[41114]: 4.1f scrub starts 2024-09-11T08:48:22.595 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:22 smithi092 ceph-mon[41114]: 4.1f scrub ok 2024-09-11T08:48:22.595 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:22 smithi092 ceph-mon[41114]: pgmap v404: 148 pgs: 148 active+clean; 2.0 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T08:48:22.595 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:22 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:48:22.595 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:22 smithi092 ceph-mon[41114]: 5.8 scrub ok 2024-09-11T08:48:22.596 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:22 smithi092 ceph-mon[41114]: 4.c scrub starts 2024-09-11T08:48:22.596 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:22 smithi092 ceph-mon[41114]: 4.c scrub ok 2024-09-11T08:48:22.610 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:22.639 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:22.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:22.640 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:22.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:22 smithi145 ceph-mon[33384]: 4.1f scrub starts 2024-09-11T08:48:22.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:22 smithi145 ceph-mon[33384]: 4.1f scrub ok 2024-09-11T08:48:22.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:22 smithi145 ceph-mon[33384]: pgmap v404: 148 pgs: 148 active+clean; 2.0 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T08:48:22.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:22 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:48:22.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:22 smithi145 ceph-mon[33384]: 5.8 scrub ok 2024-09-11T08:48:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:22 smithi145 ceph-mon[33384]: 4.c scrub starts 2024-09-11T08:48:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:22 smithi145 ceph-mon[33384]: 4.c scrub ok 2024-09-11T08:48:22.740 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:22.783 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:22.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:22.784 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:22.885 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:22.919 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:22.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:22.921 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:23.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:23.057 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:23.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:23.058 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:23.159 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:23.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:48:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:48:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:48:23.194 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:23.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:23.195 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:23.296 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:23.299 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:23.299 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:23.324 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:23.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:23.325 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:23.426 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:23.450 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:23.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:23.451 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:23.551 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:23.576 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:23.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:23.577 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:23.611 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:48:23.613 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:48:23.613 INFO:teuthology.orchestra.run.smithi092.stdout: "epoch": 197, 2024-09-11T08:48:23.613 INFO:teuthology.orchestra.run.smithi092.stdout: "fsid": "211625d4-7018-11ef-bcea-c7b262605968", 2024-09-11T08:48:23.613 INFO:teuthology.orchestra.run.smithi092.stdout: "created": "2024-09-11T08:33:32.174444+0000", 2024-09-11T08:48:23.613 INFO:teuthology.orchestra.run.smithi092.stdout: "modified": "2024-09-11T08:48:16.274474+0000", 2024-09-11T08:48:23.613 INFO:teuthology.orchestra.run.smithi092.stdout: "last_up_change": "2024-09-11T08:38:23.097480+0000", 2024-09-11T08:48:23.613 INFO:teuthology.orchestra.run.smithi092.stdout: "last_in_change": "2024-09-11T08:38:06.259586+0000", 2024-09-11T08:48:23.613 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T08:48:23.613 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_num": 5804032, 2024-09-11T08:48:23.613 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_set": [ 2024-09-11T08:48:23.613 INFO:teuthology.orchestra.run.smithi092.stdout: "nodeep-scrub", 2024-09-11T08:48:23.613 INFO:teuthology.orchestra.run.smithi092.stdout: "pglog_hardlimit", 2024-09-11T08:48:23.614 INFO:teuthology.orchestra.run.smithi092.stdout: "purged_snapdirs", 2024-09-11T08:48:23.614 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_deletes", 2024-09-11T08:48:23.614 INFO:teuthology.orchestra.run.smithi092.stdout: "sortbitwise" 2024-09-11T08:48:23.614 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:48:23.614 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_version": 18, 2024-09-11T08:48:23.614 INFO:teuthology.orchestra.run.smithi092.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T08:48:23.614 INFO:teuthology.orchestra.run.smithi092.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T08:48:23.614 INFO:teuthology.orchestra.run.smithi092.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T08:48:23.614 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_snapshot": "", 2024-09-11T08:48:23.614 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_max": 8, 2024-09-11T08:48:23.614 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd": 8, 2024-09-11T08:48:23.614 INFO:teuthology.orchestra.run.smithi092.stdout: "require_min_compat_client": "luminous", 2024-09-11T08:48:23.614 INFO:teuthology.orchestra.run.smithi092.stdout: "min_compat_client": "luminous", 2024-09-11T08:48:23.615 INFO:teuthology.orchestra.run.smithi092.stdout: "require_osd_release": "quincy", 2024-09-11T08:48:23.615 INFO:teuthology.orchestra.run.smithi092.stdout: "pools": [ 2024-09-11T08:48:23.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.615 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 1, 2024-09-11T08:48:23.615 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": ".mgr", 2024-09-11T08:48:23.615 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:36:29.606460+0000", 2024-09-11T08:48:23.615 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:48:23.615 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:48:23.615 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:48:23.615 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:48:23.615 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:48:23.615 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:48:23.615 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:48:23.615 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:48:23.616 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:48:23.616 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:48:23.616 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:48:23.616 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:48:23.616 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 1, 2024-09-11T08:48:23.616 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 1, 2024-09-11T08:48:23.616 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 1, 2024-09-11T08:48:23.616 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 1, 2024-09-11T08:48:23.616 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 1, 2024-09-11T08:48:23.616 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:48:23.616 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:48:23.616 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:48:23.616 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:48:23.616 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:48:23.617 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:48:23.617 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:48:23.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.617 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "97", 2024-09-11T08:48:23.617 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:48:23.617 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:48:23.617 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:48:23.617 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:48:23.617 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:48:23.617 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:48:23.617 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:48:23.617 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:48:23.618 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:48:23.618 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:48:23.618 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:48:23.618 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:48:23.618 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:48:23.618 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:48:23.618 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:48:23.618 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:48:23.618 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:48:23.618 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:48:23.618 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:48:23.618 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:48:23.618 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:48:23.619 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:48:23.619 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:48:23.619 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:48:23.619 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:48:23.619 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:48:23.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.619 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:48:23.619 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:48:23.619 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:48:23.619 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:48:23.619 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:48:23.619 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:48:23.619 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:48:23.620 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:48:23.620 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:48:23.620 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:48:23.620 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:48:23.620 INFO:teuthology.orchestra.run.smithi092.stdout: "options": { 2024-09-11T08:48:23.620 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_max": 32, 2024-09-11T08:48:23.620 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_min": 1 2024-09-11T08:48:23.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.620 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:48:23.620 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": {} 2024-09-11T08:48:23.620 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.621 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 2, 2024-09-11T08:48:23.624 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "datapool", 2024-09-11T08:48:23.624 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:37.674272+0000", 2024-09-11T08:48:23.624 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 8193, 2024-09-11T08:48:23.625 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T08:48:23.625 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:48:23.625 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:48:23.625 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:48:23.625 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:48:23.625 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:48:23.625 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:48:23.625 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:48:23.625 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:48:23.625 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:48:23.625 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:48:23.625 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 3, 2024-09-11T08:48:23.625 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 3, 2024-09-11T08:48:23.625 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 3, 2024-09-11T08:48:23.626 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 3, 2024-09-11T08:48:23.626 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 3, 2024-09-11T08:48:23.626 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:48:23.626 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:48:23.626 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:48:23.626 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:48:23.626 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:48:23.626 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:48:23.626 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:48:23.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.626 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "60", 2024-09-11T08:48:23.626 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:48:23.626 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:48:23.627 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:48:23.627 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:48:23.627 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:48:23.627 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 3, 2024-09-11T08:48:23.627 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 60, 2024-09-11T08:48:23.627 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:48:23.627 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:48:23.627 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:48:23.627 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:48:23.627 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:48:23.627 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:48:23.627 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:48:23.627 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:48:23.627 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:48:23.628 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:48:23.628 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:48:23.628 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:48:23.628 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:48:23.628 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:48:23.628 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:48:23.628 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:48:23.628 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:48:23.629 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:48:23.629 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:48:23.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.629 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:48:23.629 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:48:23.629 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:48:23.629 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:48:23.629 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:48:23.629 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:48:23.629 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:48:23.630 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:48:23.630 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:48:23.630 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:48:23.630 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:48:23.630 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:48:23.630 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:48:23.630 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd": {} 2024-09-11T08:48:23.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.630 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 3, 2024-09-11T08:48:23.631 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": ".rgw.root", 2024-09-11T08:48:23.631 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:37.912906+0000", 2024-09-11T08:48:23.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:48:23.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:48:23.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:48:23.631 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:48:23.631 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:48:23.631 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:48:23.631 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:48:23.631 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:48:23.631 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:48:23.632 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:48:23.632 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:48:23.632 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:48:23.632 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:48:23.632 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:48:23.632 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:48:23.632 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:48:23.632 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:48:23.632 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:48:23.632 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:48:23.632 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:48:23.632 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:48:23.633 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:48:23.633 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:48:23.633 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:48:23.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.633 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "57", 2024-09-11T08:48:23.633 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:48:23.633 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:48:23.633 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:48:23.633 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:48:23.633 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:48:23.633 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:48:23.634 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:48:23.634 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:48:23.634 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:48:23.634 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:48:23.634 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:48:23.634 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:48:23.634 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:48:23.634 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:48:23.634 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:48:23.634 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:48:23.634 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:48:23.634 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:48:23.635 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:48:23.635 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:48:23.635 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:48:23.635 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:48:23.635 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:48:23.635 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:48:23.635 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:48:23.635 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:48:23.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.636 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:48:23.636 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:48:23.636 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:48:23.636 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:48:23.636 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:48:23.636 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:48:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:48:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:48:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:48:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:48:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:48:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:48:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:48:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:48:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 4, 2024-09-11T08:48:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "default.rgw.log", 2024-09-11T08:48:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:39.234173+0000", 2024-09-11T08:48:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:48:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:48:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:48:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:48:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:48:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:48:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:48:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:48:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:48:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:48:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:48:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:48:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:48:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:48:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:48:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:48:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:48:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:48:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:48:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:48:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:48:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:48:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:48:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:48:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "59", 2024-09-11T08:48:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:48:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:48:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:48:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:48:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:48:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:48:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:48:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:48:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:48:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:48:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:48:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:48:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:48:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:48:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:48:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:48:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:48:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:48:23.642 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:48:23.642 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:48:23.642 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:48:23.642 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:48:23.642 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:48:23.642 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:48:23.642 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:48:23.642 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:48:23.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.642 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:48:23.642 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:48:23.643 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:48:23.643 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:48:23.643 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:48:23.643 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:48:23.643 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:48:23.643 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:48:23.643 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:48:23.643 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:48:23.643 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:48:23.643 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:48:23.643 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:48:23.643 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:48:23.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.644 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 5, 2024-09-11T08:48:23.644 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "default.rgw.control", 2024-09-11T08:48:23.644 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:41.216386+0000", 2024-09-11T08:48:23.644 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:48:23.644 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:48:23.644 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:48:23.644 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:48:23.644 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:48:23.644 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:48:23.644 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:48:23.645 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:48:23.645 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:48:23.645 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:48:23.645 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:48:23.645 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:48:23.645 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:48:23.645 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:48:23.645 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:48:23.645 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:48:23.645 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:48:23.645 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:48:23.646 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:48:23.646 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:48:23.646 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:48:23.646 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:48:23.646 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:48:23.646 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:48:23.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.646 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "61", 2024-09-11T08:48:23.646 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:48:23.646 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:48:23.646 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:48:23.646 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:48:23.646 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:48:23.647 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:48:23.647 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:48:23.647 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:48:23.647 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:48:23.647 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:48:23.647 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:48:23.647 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:48:23.647 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:48:23.647 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:48:23.647 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:48:23.647 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:48:23.647 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:48:23.648 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:48:23.648 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:48:23.648 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:48:23.648 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:48:23.648 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:48:23.648 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:48:23.648 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:48:23.648 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:48:23.648 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:48:23.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.648 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:48:23.649 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:48:23.649 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:48:23.649 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:48:23.649 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:48:23.649 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:48:23.649 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:48:23.649 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:48:23.649 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:48:23.649 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:48:23.649 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:48:23.649 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:48:23.650 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:48:23.650 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:48:23.650 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.650 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 6, 2024-09-11T08:48:23.650 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "default.rgw.meta", 2024-09-11T08:48:23.650 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:43.285497+0000", 2024-09-11T08:48:23.650 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:48:23.650 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:48:23.650 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:48:23.651 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:48:23.651 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:48:23.651 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:48:23.651 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:48:23.651 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:48:23.651 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:48:23.651 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:48:23.651 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:48:23.651 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:48:23.651 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:48:23.652 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:48:23.652 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:48:23.652 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:48:23.652 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:48:23.652 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:48:23.652 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:48:23.652 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:48:23.652 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:48:23.652 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:48:23.652 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:48:23.653 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:48:23.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.653 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "63", 2024-09-11T08:48:23.653 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:48:23.653 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:48:23.653 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:48:23.653 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:48:23.653 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:48:23.653 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:48:23.653 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:48:23.654 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:48:23.654 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:48:23.654 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:48:23.654 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:48:23.654 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:48:23.654 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:48:23.654 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:48:23.654 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:48:23.654 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:48:23.654 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:48:23.654 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:48:23.655 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:48:23.655 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:48:23.655 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:48:23.655 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:48:23.655 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:48:23.655 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:48:23.655 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:48:23.655 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:48:23.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.655 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:48:23.655 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:48:23.656 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:48:23.656 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:48:23.656 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:48:23.656 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:48:23.656 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:48:23.656 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:48:23.656 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:48:23.656 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:48:23.656 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:48:23.656 INFO:teuthology.orchestra.run.smithi092.stdout: "options": { 2024-09-11T08:48:23.656 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_bias": 4 2024-09-11T08:48:23.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.657 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:48:23.657 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:48:23.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.657 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 8, 2024-09-11T08:48:23.657 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "unique_pool_1", 2024-09-11T08:48:23.657 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:45:13.608438+0000", 2024-09-11T08:48:23.657 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:48:23.657 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:48:23.658 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:48:23.658 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:48:23.658 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:48:23.658 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:48:23.658 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:48:23.658 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:48:23.658 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:48:23.658 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:48:23.658 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:48:23.658 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:48:23.659 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 16, 2024-09-11T08:48:23.659 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 16, 2024-09-11T08:48:23.659 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 16, 2024-09-11T08:48:23.659 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 16, 2024-09-11T08:48:23.659 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 16, 2024-09-11T08:48:23.659 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:48:23.659 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:48:23.659 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:48:23.659 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:48:23.659 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:48:23.659 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:48:23.660 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:48:23.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.660 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "156", 2024-09-11T08:48:23.660 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:48:23.660 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:48:23.660 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:48:23.660 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:48:23.660 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:48:23.660 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:48:23.661 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:48:23.661 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:48:23.661 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:48:23.661 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:48:23.661 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:48:23.661 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:48:23.661 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:48:23.661 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:48:23.661 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:48:23.661 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:48:23.662 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:48:23.662 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:48:23.662 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:48:23.662 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:48:23.662 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:48:23.662 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:48:23.662 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:48:23.662 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:48:23.662 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:48:23.662 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:48:23.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.663 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:48:23.663 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:48:23.663 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:48:23.663 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:48:23.663 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:48:23.663 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:48:23.663 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:48:23.663 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:48:23.663 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:48:23.663 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:48:23.664 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:48:23.664 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:48:23.664 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:48:23.664 INFO:teuthology.orchestra.run.smithi092.stdout: "rados": {} 2024-09-11T08:48:23.664 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.664 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.664 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:48:23.664 INFO:teuthology.orchestra.run.smithi092.stdout: "osds": [ 2024-09-11T08:48:23.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.664 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-09-11T08:48:23.664 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "9d8a3811-e801-4b6a-9a63-83936283cb7d", 2024-09-11T08:48:23.664 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:48:23.665 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:48:23.665 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:48:23.665 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:48:23.665 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:48:23.665 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:48:23.665 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 8, 2024-09-11T08:48:23.665 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 186, 2024-09-11T08:48:23.665 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:48:23.665 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:48:23.665 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:48:23.665 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.666 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.666 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6802", 2024-09-11T08:48:23.666 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:48:23.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.666 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.666 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6803", 2024-09-11T08:48:23.666 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:48:23.666 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.666 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.667 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:48:23.667 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.667 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.667 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6804", 2024-09-11T08:48:23.667 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:48:23.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.667 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.667 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6805", 2024-09-11T08:48:23.668 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:48:23.668 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.668 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.668 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:48:23.668 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.668 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.668 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6808", 2024-09-11T08:48:23.668 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:48:23.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.669 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.669 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6809", 2024-09-11T08:48:23.669 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:48:23.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.669 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.669 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:48:23.669 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.669 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.670 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6806", 2024-09-11T08:48:23.670 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:48:23.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.670 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.670 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6807", 2024-09-11T08:48:23.670 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:48:23.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.670 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.670 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6803/2040613580", 2024-09-11T08:48:23.670 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6805/2040613580", 2024-09-11T08:48:23.671 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6809/2040613580", 2024-09-11T08:48:23.671 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6807/2040613580", 2024-09-11T08:48:23.671 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:48:23.671 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:48:23.671 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:48:23.671 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.671 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-09-11T08:48:23.671 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "e7a18e45-0137-45ef-8b85-e458cf3737ac", 2024-09-11T08:48:23.672 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:48:23.672 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:48:23.672 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:48:23.672 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:48:23.672 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:48:23.672 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:48:23.672 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 13, 2024-09-11T08:48:23.672 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 189, 2024-09-11T08:48:23.672 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:48:23.672 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:48:23.672 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:48:23.673 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.673 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.673 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6810", 2024-09-11T08:48:23.673 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:48:23.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.674 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.674 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6811", 2024-09-11T08:48:23.674 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:48:23.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.674 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.674 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:48:23.674 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.674 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.674 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6812", 2024-09-11T08:48:23.675 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:48:23.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.675 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.675 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6813", 2024-09-11T08:48:23.675 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:48:23.675 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.675 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.675 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:48:23.675 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.676 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.676 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6816", 2024-09-11T08:48:23.676 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:48:23.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.676 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.676 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6817", 2024-09-11T08:48:23.676 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:48:23.676 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.676 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.677 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:48:23.677 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.677 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.677 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6814", 2024-09-11T08:48:23.677 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:48:23.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.677 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.677 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6815", 2024-09-11T08:48:23.677 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:48:23.678 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.678 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.678 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6811/544779328", 2024-09-11T08:48:23.678 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6813/544779328", 2024-09-11T08:48:23.678 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6817/544779328", 2024-09-11T08:48:23.678 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6815/544779328", 2024-09-11T08:48:23.678 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:48:23.678 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:48:23.678 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:48:23.679 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.679 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-09-11T08:48:23.679 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "017282e2-8053-4fa7-affc-a05408fa481e", 2024-09-11T08:48:23.679 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:48:23.679 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:48:23.679 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:48:23.679 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:48:23.679 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:48:23.679 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:48:23.679 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 18, 2024-09-11T08:48:23.680 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 88, 2024-09-11T08:48:23.680 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:48:23.680 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:48:23.680 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:48:23.680 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.680 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:23.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.682 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.682 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6818", 2024-09-11T08:48:23.682 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:48:23.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.682 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.682 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6819", 2024-09-11T08:48:23.682 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:48:23.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.683 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.683 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:48:23.683 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.683 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.683 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6820", 2024-09-11T08:48:23.683 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:48:23.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.683 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.684 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6821", 2024-09-11T08:48:23.684 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:48:23.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.684 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.684 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:48:23.684 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.684 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.684 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6824", 2024-09-11T08:48:23.685 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:48:23.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.685 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.685 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6825", 2024-09-11T08:48:23.685 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:48:23.685 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.686 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.686 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:48:23.686 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.686 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.686 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6822", 2024-09-11T08:48:23.686 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:48:23.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.686 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.687 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6823", 2024-09-11T08:48:23.687 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:48:23.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.687 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.687 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6819/933736777", 2024-09-11T08:48:23.687 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6821/933736777", 2024-09-11T08:48:23.687 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6825/933736777", 2024-09-11T08:48:23.687 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6823/933736777", 2024-09-11T08:48:23.687 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:48:23.687 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:48:23.687 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:48:23.687 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.688 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-09-11T08:48:23.688 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "58e5a8a4-9e58-40ad-9b40-90cb5ff5414b", 2024-09-11T08:48:23.688 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:48:23.688 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:48:23.688 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:48:23.688 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:48:23.688 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:48:23.688 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:48:23.689 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 26, 2024-09-11T08:48:23.689 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 167, 2024-09-11T08:48:23.689 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:48:23.689 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:48:23.689 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:48:23.689 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.689 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.689 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6826", 2024-09-11T08:48:23.689 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:48:23.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.690 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.690 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6827", 2024-09-11T08:48:23.690 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:48:23.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.690 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.690 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:48:23.690 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.690 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.691 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6828", 2024-09-11T08:48:23.691 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:48:23.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.691 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.691 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6829", 2024-09-11T08:48:23.691 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:48:23.691 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.691 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.691 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:48:23.692 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.692 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.692 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6832", 2024-09-11T08:48:23.692 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:48:23.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.692 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.692 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6833", 2024-09-11T08:48:23.692 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:48:23.692 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.693 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.693 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:48:23.693 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.693 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.693 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6830", 2024-09-11T08:48:23.693 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:48:23.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.693 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.694 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6831", 2024-09-11T08:48:23.694 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:48:23.694 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.694 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.694 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6827/878892459", 2024-09-11T08:48:23.694 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6829/878892459", 2024-09-11T08:48:23.694 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6833/878892459", 2024-09-11T08:48:23.694 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6831/878892459", 2024-09-11T08:48:23.694 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:48:23.694 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:48:23.695 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:48:23.695 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.695 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-09-11T08:48:23.695 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "e144e102-963b-48b7-92a9-9391706164bf", 2024-09-11T08:48:23.695 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:48:23.695 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:48:23.695 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:48:23.695 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:48:23.695 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:48:23.696 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:48:23.696 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 31, 2024-09-11T08:48:23.696 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 167, 2024-09-11T08:48:23.696 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:48:23.696 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:48:23.696 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:48:23.696 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.696 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6800", 2024-09-11T08:48:23.696 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:48:23.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.697 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.697 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6801", 2024-09-11T08:48:23.697 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:48:23.697 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.697 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.697 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:48:23.697 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.697 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.697 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6802", 2024-09-11T08:48:23.697 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:48:23.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.698 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.698 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6803", 2024-09-11T08:48:23.698 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:48:23.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.698 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.698 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:48:23.698 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.698 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.698 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6806", 2024-09-11T08:48:23.699 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:48:23.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.699 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.699 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6807", 2024-09-11T08:48:23.699 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:48:23.699 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.699 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.699 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:48:23.699 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.700 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.700 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6804", 2024-09-11T08:48:23.700 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:48:23.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.700 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.700 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6805", 2024-09-11T08:48:23.700 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:48:23.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.700 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.701 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6801/2792751669", 2024-09-11T08:48:23.701 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6803/2792751669", 2024-09-11T08:48:23.701 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6807/2792751669", 2024-09-11T08:48:23.701 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6805/2792751669", 2024-09-11T08:48:23.701 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:48:23.701 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:48:23.701 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:48:23.701 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.701 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-09-11T08:48:23.701 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "811b0dc9-b7a7-439a-86c1-5dc9d0d54e16", 2024-09-11T08:48:23.702 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:48:23.702 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:48:23.702 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:48:23.702 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:48:23.702 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:48:23.702 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:48:23.702 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 38, 2024-09-11T08:48:23.702 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 167, 2024-09-11T08:48:23.702 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:48:23.702 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:48:23.703 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:48:23.703 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.703 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.703 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6808", 2024-09-11T08:48:23.703 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:48:23.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.703 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.703 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6809", 2024-09-11T08:48:23.703 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:48:23.703 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.703 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.704 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:48:23.704 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.704 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.704 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6810", 2024-09-11T08:48:23.704 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:48:23.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.704 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.704 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6811", 2024-09-11T08:48:23.704 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:48:23.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.705 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.705 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:48:23.705 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.705 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.705 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6814", 2024-09-11T08:48:23.705 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:48:23.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.706 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.706 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6815", 2024-09-11T08:48:23.706 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:48:23.706 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.706 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.706 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:48:23.706 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6812", 2024-09-11T08:48:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:48:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6813", 2024-09-11T08:48:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:48:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6809/3569637188", 2024-09-11T08:48:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6811/3569637188", 2024-09-11T08:48:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6815/3569637188", 2024-09-11T08:48:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6813/3569637188", 2024-09-11T08:48:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:48:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:48:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:48:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-09-11T08:48:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "bd162f52-097c-4c5d-9d37-1289779a196b", 2024-09-11T08:48:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:48:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:48:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:48:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:48:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:48:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:48:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 45, 2024-09-11T08:48:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 187, 2024-09-11T08:48:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:48:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:48:23.710 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:48:23.710 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.710 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.710 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6816", 2024-09-11T08:48:23.710 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:48:23.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.710 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.710 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6817", 2024-09-11T08:48:23.710 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:48:23.710 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.711 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.711 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:48:23.711 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.711 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6818", 2024-09-11T08:48:23.711 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:48:23.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.711 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6819", 2024-09-11T08:48:23.712 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:48:23.712 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.712 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.712 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:48:23.712 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.712 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.712 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6822", 2024-09-11T08:48:23.712 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:48:23.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.713 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.713 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6823", 2024-09-11T08:48:23.713 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:48:23.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.713 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.713 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:48:23.713 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.713 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.713 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6820", 2024-09-11T08:48:23.714 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:48:23.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.714 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.714 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6821", 2024-09-11T08:48:23.714 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:48:23.714 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.714 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.714 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6817/2615047110", 2024-09-11T08:48:23.714 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6819/2615047110", 2024-09-11T08:48:23.714 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6823/2615047110", 2024-09-11T08:48:23.715 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6821/2615047110", 2024-09-11T08:48:23.715 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:48:23.715 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:48:23.715 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:48:23.715 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.715 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-09-11T08:48:23.715 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "4e4021bf-78c7-4bc6-b222-3494f724ddec", 2024-09-11T08:48:23.715 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:48:23.715 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:48:23.715 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:48:23.716 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:48:23.716 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:48:23.716 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:48:23.716 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 52, 2024-09-11T08:48:23.716 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 167, 2024-09-11T08:48:23.716 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:48:23.716 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:48:23.716 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:48:23.716 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.716 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.716 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6824", 2024-09-11T08:48:23.717 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:48:23.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.717 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.717 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6825", 2024-09-11T08:48:23.717 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:48:23.717 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.717 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.717 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:48:23.717 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.718 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.718 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6826", 2024-09-11T08:48:23.718 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:48:23.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.718 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.718 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6827", 2024-09-11T08:48:23.718 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:48:23.718 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.718 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6830", 2024-09-11T08:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6831", 2024-09-11T08:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6828", 2024-09-11T08:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6829", 2024-09-11T08:48:23.721 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:48:23.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.721 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.721 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6825/3195898530", 2024-09-11T08:48:23.721 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6827/3195898530", 2024-09-11T08:48:23.721 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6831/3195898530", 2024-09-11T08:48:23.721 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6829/3195898530", 2024-09-11T08:48:23.721 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:48:23.721 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:48:23.721 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_xinfo": [ 2024-09-11T08:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-09-11T08:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:47:51.307443+0000", 2024-09-11T08:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-09-11T08:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:47:52.928564+0000", 2024-09-11T08:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.724 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-09-11T08:48:23.724 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:48:23.724 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:48:23.724 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:48:23.724 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:48:23.724 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:48:23.724 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:47:47.922993+0000", 2024-09-11T08:48:23.724 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:48:23.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.724 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-09-11T08:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:47:06.378485+0000", 2024-09-11T08:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-09-11T08:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:48:23.726 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:48:23.726 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:48:23.726 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:48:23.726 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:48:05.942170+0000", 2024-09-11T08:48:23.726 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:48:23.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.726 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-09-11T08:48:23.726 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:48:23.726 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:48:23.726 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:48:23.727 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:48:23.727 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:48:23.727 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:47:45.323294+0000", 2024-09-11T08:48:23.727 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:48:23.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.727 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-09-11T08:48:23.727 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:48:23.727 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:48:23.727 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:48:23.727 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:47:25.301068+0000", 2024-09-11T08:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-09-11T08:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:47:51.701087+0000", 2024-09-11T08:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap": [], 2024-09-11T08:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap_items": [ 2024-09-11T08:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.e", 2024-09-11T08:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T08:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.12", 2024-09-11T08:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 5 2024-09-11T08:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.17", 2024-09-11T08:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T08:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:48:23.732 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_temp": [], 2024-09-11T08:48:23.732 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_temp": [], 2024-09-11T08:48:23.732 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklist": { 2024-09-11T08:48:23.732 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6800/1861854063": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:48:23.732 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/4113910808": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:48:23.732 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/3501902574": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:48:23.732 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/1583695466": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:48:23.732 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/736380628": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:48:23.732 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6800/4060453592": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:48:23.732 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/2234574262": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:48:23.732 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6800/2480574848": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:48:23.733 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/1540079149": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:48:23.733 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/2680744738": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:48:23.733 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6801/4060453592": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:48:23.733 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6801/1861854063": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:48:23.733 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6801/2480574848": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:48:23.733 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/12697418": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:48:23.733 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/4280958069": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:48:23.733 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/2686720854": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:48:23.733 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/969221731": "2024-09-12T08:39:14.984442+0000" 2024-09-11T08:48:23.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.733 INFO:teuthology.orchestra.run.smithi092.stdout: "range_blocklist": {}, 2024-09-11T08:48:23.733 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profiles": { 2024-09-11T08:48:23.734 INFO:teuthology.orchestra.run.smithi092.stdout: "default": { 2024-09-11T08:48:23.734 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-failure-domain": "osd", 2024-09-11T08:48:23.734 INFO:teuthology.orchestra.run.smithi092.stdout: "k": "2", 2024-09-11T08:48:23.734 INFO:teuthology.orchestra.run.smithi092.stdout: "m": "1", 2024-09-11T08:48:23.734 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin": "jerasure", 2024-09-11T08:48:23.734 INFO:teuthology.orchestra.run.smithi092.stdout: "technique": "reed_sol_van" 2024-09-11T08:48:23.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:23.734 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps_queue": [], 2024-09-11T08:48:23.734 INFO:teuthology.orchestra.run.smithi092.stdout: "new_removed_snaps": [], 2024-09-11T08:48:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: "new_purged_snaps": [], 2024-09-11T08:48:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_node_flags": {}, 2024-09-11T08:48:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: "device_class_flags": {}, 2024-09-11T08:48:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode": { 2024-09-11T08:48:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_enabled": false, 2024-09-11T08:48:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_bucket_count": 0, 2024-09-11T08:48:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: "degraded_stretch_mode": 0, 2024-09-11T08:48:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: "recovering_stretch_mode": 0, 2024-09-11T08:48:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_bucket": 0 2024-09-11T08:48:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:23.735 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:48:23.771 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:23.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:23.772 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:23.873 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:23.906 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:23.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:23.908 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:24.008 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:24.037 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:24.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:24.038 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:24.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:24.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:24.174 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:24.174 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:24.275 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:24.303 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:24.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:24.304 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:24.405 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:24.433 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:24.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:24.434 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:24.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:24.581 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:24.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:24.582 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:24 smithi145 ceph-mon[33384]: 5.c scrub starts 2024-09-11T08:48:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:24 smithi145 ceph-mon[33384]: 5.c scrub ok 2024-09-11T08:48:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:24 smithi145 ceph-mon[33384]: pgmap v405: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 1.7 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T08:48:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:24 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:48:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:24 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:48:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:24 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1978583988' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T08:48:24.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:24 smithi145 ceph-mon[33384]: 4.1 scrub starts 2024-09-11T08:48:24.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:24 smithi145 ceph-mon[33384]: 4.1 scrub ok 2024-09-11T08:48:24.665 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[29647]: 5.c scrub starts 2024-09-11T08:48:24.665 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[29647]: 5.c scrub ok 2024-09-11T08:48:24.665 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[29647]: pgmap v405: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 1.7 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T08:48:24.665 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:48:24.665 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:48:24.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1978583988' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T08:48:24.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[29647]: 4.1 scrub starts 2024-09-11T08:48:24.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[29647]: 4.1 scrub ok 2024-09-11T08:48:24.666 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[41114]: 5.c scrub starts 2024-09-11T08:48:24.666 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[41114]: 5.c scrub ok 2024-09-11T08:48:24.666 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[41114]: pgmap v405: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 1.7 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T08:48:24.666 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:48:24.666 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:48:24.666 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1978583988' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T08:48:24.666 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[41114]: 4.1 scrub starts 2024-09-11T08:48:24.666 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:24 smithi092 ceph-mon[41114]: 4.1 scrub ok 2024-09-11T08:48:24.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:24.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:24.716 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:24.716 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:24.817 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:24.845 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:24.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:24.846 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:24.947 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:24.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:24.974 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:24.974 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:25.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:25.105 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:25.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:25.106 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:25.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:25.238 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:25.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:25.239 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:25.339 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:25.382 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:25.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:25.383 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:25.484 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:25.518 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:25.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:25.519 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:25.620 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:25.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:25.656 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:25.656 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:25 smithi092 ceph-mon[29647]: 3.10 scrub starts 2024-09-11T08:48:25.656 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:25 smithi092 ceph-mon[29647]: 3.10 scrub ok 2024-09-11T08:48:25.656 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:25 smithi092 ceph-mon[29647]: 3.4 scrub starts 2024-09-11T08:48:25.656 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:25 smithi092 ceph-mon[29647]: 3.4 scrub ok 2024-09-11T08:48:25.656 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:25 smithi092 ceph-mon[29647]: 5.a scrub starts 2024-09-11T08:48:25.656 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:25 smithi092 ceph-mon[29647]: osdmap e198: 8 total, 8 up, 8 in 2024-09-11T08:48:25.656 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:25 smithi092 ceph-mon[29647]: 5.a scrub ok 2024-09-11T08:48:25.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:25 smithi092 ceph-mon[41114]: 3.10 scrub starts 2024-09-11T08:48:25.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:25 smithi092 ceph-mon[41114]: 3.10 scrub ok 2024-09-11T08:48:25.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:25 smithi092 ceph-mon[41114]: 3.4 scrub starts 2024-09-11T08:48:25.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:25 smithi092 ceph-mon[41114]: 3.4 scrub ok 2024-09-11T08:48:25.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:25 smithi092 ceph-mon[41114]: 5.a scrub starts 2024-09-11T08:48:25.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:25 smithi092 ceph-mon[41114]: osdmap e198: 8 total, 8 up, 8 in 2024-09-11T08:48:25.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:25 smithi092 ceph-mon[41114]: 5.a scrub ok 2024-09-11T08:48:25.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:25 smithi145 ceph-mon[33384]: 3.10 scrub starts 2024-09-11T08:48:25.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:25 smithi145 ceph-mon[33384]: 3.10 scrub ok 2024-09-11T08:48:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:25 smithi145 ceph-mon[33384]: 3.4 scrub starts 2024-09-11T08:48:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:25 smithi145 ceph-mon[33384]: 3.4 scrub ok 2024-09-11T08:48:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:25 smithi145 ceph-mon[33384]: 5.a scrub starts 2024-09-11T08:48:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:25 smithi145 ceph-mon[33384]: osdmap e198: 8 total, 8 up, 8 in 2024-09-11T08:48:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:25 smithi145 ceph-mon[33384]: 5.a scrub ok 2024-09-11T08:48:25.757 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:25.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:25.785 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:25.786 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:25.886 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:25.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:25.916 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:25.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:26.017 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:26.047 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:26.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:26.048 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:26.149 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:26.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:26.183 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:26.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:26.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:26.321 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:26.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:26.322 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:26.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:26.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:26.457 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:26.457 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:26.558 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:26 smithi092 ceph-mon[29647]: pgmap v407: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 1.7 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 573 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-11T08:48:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:26 smithi092 ceph-mon[29647]: 5.5 scrub starts 2024-09-11T08:48:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:26 smithi092 ceph-mon[29647]: 5.5 scrub ok 2024-09-11T08:48:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:26 smithi092 ceph-mon[29647]: 4.13 scrub starts 2024-09-11T08:48:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:26 smithi092 ceph-mon[29647]: 4.13 scrub ok 2024-09-11T08:48:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:26 smithi092 ceph-mon[41114]: pgmap v407: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 1.7 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 573 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-11T08:48:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:26 smithi092 ceph-mon[41114]: 5.5 scrub starts 2024-09-11T08:48:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:26 smithi092 ceph-mon[41114]: 5.5 scrub ok 2024-09-11T08:48:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:26 smithi092 ceph-mon[41114]: 4.13 scrub starts 2024-09-11T08:48:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:26 smithi092 ceph-mon[41114]: 4.13 scrub ok 2024-09-11T08:48:26.594 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:26.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:26.595 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:26.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:48:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:48:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:26 smithi145 ceph-mon[33384]: pgmap v407: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 1.7 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 573 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-11T08:48:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:26 smithi145 ceph-mon[33384]: 5.5 scrub starts 2024-09-11T08:48:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:26 smithi145 ceph-mon[33384]: 5.5 scrub ok 2024-09-11T08:48:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:26 smithi145 ceph-mon[33384]: 4.13 scrub starts 2024-09-11T08:48:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:26 smithi145 ceph-mon[33384]: 4.13 scrub ok 2024-09-11T08:48:26.697 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:26.729 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:26.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:26.730 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:26.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:26.869 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:26.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:26.870 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:26.971 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:27.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:27.005 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:27.005 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:27.108 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:27.142 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:27.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:27.143 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:27.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:27.285 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:27.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:27.286 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:27.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:27.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:27.408 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:27.509 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:27.539 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:27.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:27.540 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:27.641 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:27.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:27 smithi145 ceph-mon[33384]: 5.2 scrub starts 2024-09-11T08:48:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:27 smithi145 ceph-mon[33384]: 5.2 scrub ok 2024-09-11T08:48:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:27 smithi145 ceph-mon[33384]: 3.19 scrub starts 2024-09-11T08:48:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:27 smithi145 ceph-mon[33384]: 3.19 scrub ok 2024-09-11T08:48:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:27 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:27.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:27.672 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:27.672 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:27.773 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:27.795 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:27.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:27.797 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:27.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:27 smithi092 ceph-mon[41114]: 5.2 scrub starts 2024-09-11T08:48:27.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:27 smithi092 ceph-mon[41114]: 5.2 scrub ok 2024-09-11T08:48:27.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:27 smithi092 ceph-mon[41114]: 3.19 scrub starts 2024-09-11T08:48:27.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:27 smithi092 ceph-mon[41114]: 3.19 scrub ok 2024-09-11T08:48:27.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:27 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:27.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:27 smithi092 ceph-mon[29647]: 5.2 scrub starts 2024-09-11T08:48:27.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:27 smithi092 ceph-mon[29647]: 5.2 scrub ok 2024-09-11T08:48:27.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:27 smithi092 ceph-mon[29647]: 3.19 scrub starts 2024-09-11T08:48:27.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:27 smithi092 ceph-mon[29647]: 3.19 scrub ok 2024-09-11T08:48:27.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:27 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:27.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:27.933 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:27.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:27.934 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:28.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:28.066 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:28.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:28.067 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:28.167 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:28.201 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:28.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:28.202 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:28.303 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:28.333 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:28.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:28.334 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:28.435 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:28.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:28.475 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:28.475 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:28.576 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:28.586 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:28 smithi092 ceph-mon[29647]: pgmap v408: 148 pgs: 148 active+clean; 1.4 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T08:48:28.587 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:28 smithi092 ceph-mon[41114]: pgmap v408: 148 pgs: 148 active+clean; 1.4 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T08:48:28.613 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:28.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:28.614 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:28.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:28 smithi145 ceph-mon[33384]: pgmap v408: 148 pgs: 148 active+clean; 1.4 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T08:48:28.715 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:28.752 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:28.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:28.753 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:28.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:28.897 INFO:tasks.thrashosds.thrasher:j is {'epoch': 197, 'fsid': '211625d4-7018-11ef-bcea-c7b262605968', 'created': '2024-09-11T08:33:32.174444+0000', 'modified': '2024-09-11T08:48:16.274474+0000', 'last_up_change': '2024-09-11T08:38:23.097480+0000', 'last_in_change': '2024-09-11T08:38:06.259586+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 8, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T08:36:29.606460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-11T08:38:37.674272+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-11T08:38:37.912906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-11T08:38:39.234173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-11T08:38:41.216386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-11T08:38:43.285497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 8, 'pool_name': 'unique_pool_1', 'create_time': '2024-09-11T08:45:13.608438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '9d8a3811-e801-4b6a-9a63-83936283cb7d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 186, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 2040613580}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 2040613580}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 2040613580}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 2040613580}]}, 'public_addr': '172.21.15.92:6803/2040613580', 'cluster_addr': '172.21.15.92:6805/2040613580', 'heartbeat_back_addr': '172.21.15.92:6809/2040613580', 'heartbeat_front_addr': '172.21.15.92:6807/2040613580', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e7a18e45-0137-45ef-8b85-e458cf3737ac', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 189, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 544779328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 544779328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 544779328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 544779328}]}, 'public_addr': '172.21.15.92:6811/544779328', 'cluster_addr': '172.21.15.92:6813/544779328', 'heartbeat_back_addr': '172.21.15.92:6817/544779328', 'heartbeat_front_addr': '172.21.15.92:6815/544779328', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '017282e2-8053-4fa7-affc-a05408fa481e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 933736777}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 933736777}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 933736777}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 933736777}]}, 'public_addr': '172.21.15.92:6819/933736777', 'cluster_addr': '172.21.15.92:6821/933736777', 'heartbeat_back_addr': '172.21.15.92:6825/933736777', 'heartbeat_front_addr': '172.21.15.92:6823/933736777', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '58e5a8a4-9e58-40ad-9b40-90cb5ff5414b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 167, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 878892459}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 878892459}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 878892459}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 878892459}]}, 'public_addr': '172.21.15.92:6827/878892459', 'cluster_addr': '172.21.15.92:6829/878892459', 'heartbeat_back_addr': '172.21.15.92:6833/878892459', 'heartbeat_front_addr': '172.21.15.92:6831/878892459', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e144e102-963b-48b7-92a9-9391706164bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 167, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6800', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6801', 'nonce': 2792751669}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6802', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6803', 'nonce': 2792751669}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6806', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6807', 'nonce': 2792751669}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6804', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6805', 'nonce': 2792751669}]}, 'public_addr': '172.21.15.145:6801/2792751669', 'cluster_addr': '172.21.15.145:6803/2792751669', 'heartbeat_back_addr': '172.21.15.145:6807/2792751669', 'heartbeat_front_addr': '172.21.15.145:6805/2792751669', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '811b0dc9-b7a7-439a-86c1-5dc9d0d54e16', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 38, 'up_thru': 167, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6808', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6809', 'nonce': 3569637188}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6810', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6811', 'nonce': 3569637188}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6814', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6815', 'nonce': 3569637188}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6812', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6813', 'nonce': 3569637188}]}, 'public_addr': '172.21.15.145:6809/3569637188', 'cluster_addr': '172.21.15.145:6811/3569637188', 'heartbeat_back_addr': '172.21.15.145:6815/3569637188', 'heartbeat_front_addr': '172.21.15.145:6813/3569637188', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'bd162f52-097c-4c5d-9d37-1289779a196b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 187, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6816', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6817', 'nonce': 2615047110}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6818', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6819', 'nonce': 2615047110}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6822', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6823', 'nonce': 2615047110}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6820', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6821', 'nonce': 2615047110}]}, 'public_addr': '172.21.15.145:6817/2615047110', 'cluster_addr': '172.21.15.145:6819/2615047110', 'heartbeat_back_addr': '172.21.15.145:6823/2615047110', 'heartbeat_front_addr': '172.21.15.145:6821/2615047110', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '4e4021bf-78c7-4bc6-b222-3494f724ddec', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 52, 'up_thru': 167, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6824', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6825', 'nonce': 3195898530}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6826', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6827', 'nonce': 3195898530}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6830', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6831', 'nonce': 3195898530}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6828', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6829', 'nonce': 3195898530}]}, 'public_addr': '172.21.15.145:6825/3195898530', 'cluster_addr': '172.21.15.145:6827/3195898530', 'heartbeat_back_addr': '172.21.15.145:6831/3195898530', 'heartbeat_front_addr': '172.21.15.145:6829/3195898530', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:47:51.307443+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:47:52.928564+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:47:47.922993+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:47:06.378485+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:48:05.942170+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:47:45.323294+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:47:25.301068+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:47:51.701087+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '4.e', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.12', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.92:6800/1861854063': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/4113910808': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:0/3501902574': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/1583695466': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:0/736380628': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:6800/4060453592': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:0/2234574262': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:6800/2480574848': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/1540079149': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/2680744738': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:6801/4060453592': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:6801/1861854063': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:6801/2480574848': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/12697418': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/4280958069': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/2686720854': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/969221731': '2024-09-12T08:39:14.984442+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T08:48:28.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:48:28.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:28.903 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:28.904 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:29.005 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:29.034 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:29.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:29.035 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:29.136 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:29.174 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:29.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:29.175 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:29.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:29.329 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:29.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:29.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:29.431 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:29.458 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:29.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:29.459 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:29.550 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:29 smithi092 ceph-mon[29647]: 6.1 scrub starts 2024-09-11T08:48:29.550 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:29 smithi092 ceph-mon[29647]: 6.1 scrub ok 2024-09-11T08:48:29.551 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:29 smithi092 ceph-mon[29647]: 2.1 scrub starts 2024-09-11T08:48:29.551 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:29 smithi092 ceph-mon[29647]: 2.1 scrub ok 2024-09-11T08:48:29.552 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:29 smithi092 ceph-mon[41114]: 6.1 scrub starts 2024-09-11T08:48:29.552 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:29 smithi092 ceph-mon[41114]: 6.1 scrub ok 2024-09-11T08:48:29.552 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:29 smithi092 ceph-mon[41114]: 2.1 scrub starts 2024-09-11T08:48:29.552 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:29 smithi092 ceph-mon[41114]: 2.1 scrub ok 2024-09-11T08:48:29.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:29.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:29.619 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:29.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:29.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:29 smithi145 ceph-mon[33384]: 6.1 scrub starts 2024-09-11T08:48:29.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:29 smithi145 ceph-mon[33384]: 6.1 scrub ok 2024-09-11T08:48:29.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:29 smithi145 ceph-mon[33384]: 2.1 scrub starts 2024-09-11T08:48:29.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:29 smithi145 ceph-mon[33384]: 2.1 scrub ok 2024-09-11T08:48:29.720 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:29.757 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:29.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:29.761 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:29.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:29.885 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:29.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:29.886 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:29.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:30.024 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:30.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:30.025 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:30.126 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:30.154 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:30.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:30.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:30.256 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:30.277 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:30.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:30.278 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:30.378 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:30.407 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:30.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:30.408 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:30.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:30.546 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:30.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:30.547 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:30.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:30.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:30 smithi145 ceph-mon[33384]: 3.a scrub starts 2024-09-11T08:48:30.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:30 smithi145 ceph-mon[33384]: 3.a scrub ok 2024-09-11T08:48:30.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:30 smithi145 ceph-mon[33384]: 8.a scrub starts 2024-09-11T08:48:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:30 smithi145 ceph-mon[33384]: pgmap v409: 148 pgs: 148 active+clean; 1.4 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T08:48:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:30 smithi145 ceph-mon[33384]: 8.a scrub ok 2024-09-11T08:48:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:30.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:30.687 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:30.687 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:30.788 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:30 smithi092 ceph-mon[29647]: 3.a scrub starts 2024-09-11T08:48:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:30 smithi092 ceph-mon[29647]: 3.a scrub ok 2024-09-11T08:48:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:30 smithi092 ceph-mon[29647]: 8.a scrub starts 2024-09-11T08:48:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:30 smithi092 ceph-mon[29647]: pgmap v409: 148 pgs: 148 active+clean; 1.4 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T08:48:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:30 smithi092 ceph-mon[29647]: 8.a scrub ok 2024-09-11T08:48:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:30 smithi092 ceph-mon[41114]: 3.a scrub starts 2024-09-11T08:48:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:30 smithi092 ceph-mon[41114]: 3.a scrub ok 2024-09-11T08:48:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:30 smithi092 ceph-mon[41114]: 8.a scrub starts 2024-09-11T08:48:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:30 smithi092 ceph-mon[41114]: pgmap v409: 148 pgs: 148 active+clean; 1.4 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T08:48:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:30 smithi092 ceph-mon[41114]: 8.a scrub ok 2024-09-11T08:48:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:30.828 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:30.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:30.829 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:30.929 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:30.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:30.963 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:30.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:31.064 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:31.096 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:31.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:31.098 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:31.198 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:31.237 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:31.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:31.238 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:31.338 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:31.366 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:31.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:31.367 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:31.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:31.505 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:31.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:31.506 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:31.602 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:48:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:48:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15126.0:224118 8.8 e198/149 8:12b28508:::benchmark_data_smithi092_117541_object104157:head v 198'14000, mlcod=198'13998)", 2024-09-11T08:48:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:48:31.591708+0000", 2024-09-11T08:48:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0099979130000000006, 2024-09-11T08:48:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010026986, 2024-09-11T08:48:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:48:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.591708+0000", 2024-09-11T08:48:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.591708+0000", 2024-09-11T08:48:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.591707+0000", 2024-09-11T08:48:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:48:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.591714+0000", 2024-09-11T08:48:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8310000000000003e-06 2024-09-11T08:48:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.591723+0000", 2024-09-11T08:48:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8500000000000006e-06 2024-09-11T08:48:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.591729+0000", 2024-09-11T08:48:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1370000000000001e-06 2024-09-11T08:48:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.591790+0000", 2024-09-11T08:48:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1679000000000004e-05 2024-09-11T08:48:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.591809+0000", 2024-09-11T08:48:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8933e-05 2024-09-11T08:48:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:224124 8.f 8:f9224ab1:::benchmark_data_smithi092_117541_object104163:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T08:48:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:48:31.592013+0000", 2024-09-11T08:48:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0096920410000000002, 2024-09-11T08:48:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097854169999999994, 2024-09-11T08:48:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:48:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:48:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:48:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 224124 2024-09-11T08:48:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.592013+0000", 2024-09-11T08:48:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.592013+0000", 2024-09-11T08:48:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.592012+0000", 2024-09-11T08:48:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.592018+0000", 2024-09-11T08:48:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6640000000000003e-06 2024-09-11T08:48:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.592028+0000", 2024-09-11T08:48:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0672999999999999e-05 2024-09-11T08:48:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.592034+0000", 2024-09-11T08:48:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.356e-06 2024-09-11T08:48:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.592302+0000", 2024-09-11T08:48:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026784600000000002 2024-09-11T08:48:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.592403+0000", 2024-09-11T08:48:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010152 2024-09-11T08:48:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:48:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.592502+0000", 2024-09-11T08:48:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8639000000000002e-05 2024-09-11T08:48:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.595848+0000", 2024-09-11T08:48:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033466630000000002 2024-09-11T08:48:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15126.0:224120 8.8 e198/149 8:196cb6cb:::benchmark_data_smithi092_117541_object104159:head v 198'14001, mlcod=198'13998)", 2024-09-11T08:48:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:48:31.593019+0000", 2024-09-11T08:48:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0086865680000000004, 2024-09-11T08:48:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088594940000000007, 2024-09-11T08:48:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:48:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.593019+0000", 2024-09-11T08:48:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.593019+0000", 2024-09-11T08:48:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.593018+0000", 2024-09-11T08:48:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:48:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.593024+0000", 2024-09-11T08:48:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3079999999999998e-06 2024-09-11T08:48:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.593035+0000", 2024-09-11T08:48:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1634000000000001e-05 2024-09-11T08:48:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.593040+0000", 2024-09-11T08:48:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9409999999999998e-06 2024-09-11T08:48:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.597405+0000", 2024-09-11T08:48:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043647909999999998 2024-09-11T08:48:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.597423+0000", 2024-09-11T08:48:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8131e-05 2024-09-11T08:48:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:224127 8.0 8:0e915bc0:::benchmark_data_smithi092_117541_object104166:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T08:48:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:48:31.593409+0000", 2024-09-11T08:48:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0082966989999999994, 2024-09-11T08:48:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085239180000000001, 2024-09-11T08:48:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:48:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:48:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:48:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 224127 2024-09-11T08:48:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.593409+0000", 2024-09-11T08:48:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.593409+0000", 2024-09-11T08:48:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.618 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.593408+0000", 2024-09-11T08:48:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:48:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.593413+0000", 2024-09-11T08:48:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2780000000000001e-06 2024-09-11T08:48:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.593427+0000", 2024-09-11T08:48:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.411e-05 2024-09-11T08:48:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.593433+0000", 2024-09-11T08:48:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8939999999999998e-06 2024-09-11T08:48:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.595880+0000", 2024-09-11T08:48:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024471010000000001 2024-09-11T08:48:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.595983+0000", 2024-09-11T08:48:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102816 2024-09-11T08:48:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,5", 2024-09-11T08:48:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.596081+0000", 2024-09-11T08:48:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8117999999999998e-05 2024-09-11T08:48:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:224133 8.f 8:f3cca44f:::benchmark_data_smithi092_117541_object104172:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T08:48:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:48:31.598128+0000", 2024-09-11T08:48:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00357728, 2024-09-11T08:48:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038711990000000001, 2024-09-11T08:48:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:48:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:48:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:48:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 224133 2024-09-11T08:48:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.598128+0000", 2024-09-11T08:48:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.598128+0000", 2024-09-11T08:48:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.598126+0000", 2024-09-11T08:48:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:48:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.598133+0000", 2024-09-11T08:48:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2359999999999998e-06 2024-09-11T08:48:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.598161+0000", 2024-09-11T08:48:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7640999999999998e-05 2024-09-11T08:48:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.598168+0000", 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.328e-06 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.598790+0000", 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062182299999999997 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.598896+0000", 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010628600000000001 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:48:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.598998+0000", 2024-09-11T08:48:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101498 2024-09-11T08:48:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15126.0:224130 8.8 e198/149 8:1fdbaaa0:::benchmark_data_smithi092_117541_object104169:head v 198'14002, mlcod=198'13998)", 2024-09-11T08:48:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:48:31.599385+0000", 2024-09-11T08:48:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0023199779999999999, 2024-09-11T08:48:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026767739999999998, 2024-09-11T08:48:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:48:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.599385+0000", 2024-09-11T08:48:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.599385+0000", 2024-09-11T08:48:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.599385+0000", 2024-09-11T08:48:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:48:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.599391+0000", 2024-09-11T08:48:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8580000000000002e-06 2024-09-11T08:48:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.599403+0000", 2024-09-11T08:48:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-05 2024-09-11T08:48:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.599409+0000", 2024-09-11T08:48:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6439999999999997e-06 2024-09-11T08:48:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.600813+0000", 2024-09-11T08:48:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014045290000000001 2024-09-11T08:48:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.601072+0000", 2024-09-11T08:48:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025857199999999999 2024-09-11T08:48:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15126.0:224129 8.7 e198/189)", 2024-09-11T08:48:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:48:31.601030+0000", 2024-09-11T08:48:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00067553199999999996, 2024-09-11T08:48:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010807150000000001, 2024-09-11T08:48:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-11T08:48:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.601030+0000", 2024-09-11T08:48:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.601030+0000", 2024-09-11T08:48:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.601029+0000", 2024-09-11T08:48:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:48:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.601036+0000", 2024-09-11T08:48:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9800000000000001e-06 2024-09-11T08:48:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.601054+0000", 2024-09-11T08:48:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7456e-05 2024-09-11T08:48:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.601060+0000", 2024-09-11T08:48:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0440000000000001e-06 2024-09-11T08:48:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15126.0:224125 8.c e198/149 8:35abb08d:::benchmark_data_smithi092_117541_object104164:head v 198'14077, mlcod=198'14075)", 2024-09-11T08:48:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:48:31.601086+0000", 2024-09-11T08:48:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00061948999999999999, 2024-09-11T08:48:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001070353, 2024-09-11T08:48:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:48:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.601086+0000", 2024-09-11T08:48:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.601086+0000", 2024-09-11T08:48:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.601085+0000", 2024-09-11T08:48:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:48:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.601090+0000", 2024-09-11T08:48:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.476e-06 2024-09-11T08:48:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.601102+0000", 2024-09-11T08:48:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1644000000000001e-05 2024-09-11T08:48:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.601106+0000", 2024-09-11T08:48:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4989999999999996e-06 2024-09-11T08:48:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.601234+0000", 2024-09-11T08:48:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012767 2024-09-11T08:48:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:48:31.601253+0000", 2024-09-11T08:48:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9094e-05 2024-09-11T08:48:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:48:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 8 2024-09-11T08:48:31.636 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:48:31.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:31 smithi092 ceph-mon[29647]: 5.11 scrub starts 2024-09-11T08:48:31.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:31 smithi092 ceph-mon[29647]: 5.11 scrub ok 2024-09-11T08:48:31.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:31 smithi092 ceph-mon[29647]: 4.19 scrub starts 2024-09-11T08:48:31.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:31 smithi092 ceph-mon[29647]: 4.19 scrub ok 2024-09-11T08:48:31.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:31 smithi092 ceph-mon[29647]: 5.7 scrub starts 2024-09-11T08:48:31.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:31 smithi092 ceph-mon[29647]: 5.7 scrub ok 2024-09-11T08:48:31.637 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:48:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:48:31] "GET /metrics HTTP/1.1" 200 224340 "" "Prometheus/2.43.0" 2024-09-11T08:48:31.638 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:31 smithi092 ceph-mon[41114]: 5.11 scrub starts 2024-09-11T08:48:31.638 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:31 smithi092 ceph-mon[41114]: 5.11 scrub ok 2024-09-11T08:48:31.638 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:31 smithi092 ceph-mon[41114]: 4.19 scrub starts 2024-09-11T08:48:31.638 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:31 smithi092 ceph-mon[41114]: 4.19 scrub ok 2024-09-11T08:48:31.638 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:31 smithi092 ceph-mon[41114]: 5.7 scrub starts 2024-09-11T08:48:31.638 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:31 smithi092 ceph-mon[41114]: 5.7 scrub ok 2024-09-11T08:48:31.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:31 smithi145 ceph-mon[33384]: 5.11 scrub starts 2024-09-11T08:48:31.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:31 smithi145 ceph-mon[33384]: 5.11 scrub ok 2024-09-11T08:48:31.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:31 smithi145 ceph-mon[33384]: 4.19 scrub starts 2024-09-11T08:48:31.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:31 smithi145 ceph-mon[33384]: 4.19 scrub ok 2024-09-11T08:48:31.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:31 smithi145 ceph-mon[33384]: 5.7 scrub starts 2024-09-11T08:48:31.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:31 smithi145 ceph-mon[33384]: 5.7 scrub ok 2024-09-11T08:48:31.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:31.663 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:31.663 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:31.764 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:31.792 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:31.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:31.793 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:31.894 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:31.926 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:31.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:31.927 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:32.028 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:32.061 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:32.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:32.062 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:32.163 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:32.196 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:32.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:32.196 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:32.297 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:32.326 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:32.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:32.327 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:32.428 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:32.454 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:32.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:32.455 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:32.556 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:32.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:32.614 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:32.614 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:32.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:32 smithi145 ceph-mon[33384]: 5.12 scrub starts 2024-09-11T08:48:32.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:32 smithi145 ceph-mon[33384]: 5.12 scrub ok 2024-09-11T08:48:32.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:32 smithi145 ceph-mon[33384]: pgmap v410: 148 pgs: 148 active+clean; 1.3 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T08:48:32.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:32 smithi145 ceph-mon[33384]: 6.0 scrub starts 2024-09-11T08:48:32.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:32 smithi145 ceph-mon[33384]: 6.0 scrub ok 2024-09-11T08:48:32.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:32 smithi092 ceph-mon[29647]: 5.12 scrub starts 2024-09-11T08:48:32.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:32 smithi092 ceph-mon[29647]: 5.12 scrub ok 2024-09-11T08:48:32.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:32 smithi092 ceph-mon[29647]: pgmap v410: 148 pgs: 148 active+clean; 1.3 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T08:48:32.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:32 smithi092 ceph-mon[29647]: 6.0 scrub starts 2024-09-11T08:48:32.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:32 smithi092 ceph-mon[29647]: 6.0 scrub ok 2024-09-11T08:48:32.678 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:32 smithi092 ceph-mon[41114]: 5.12 scrub starts 2024-09-11T08:48:32.678 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:32 smithi092 ceph-mon[41114]: 5.12 scrub ok 2024-09-11T08:48:32.678 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:32 smithi092 ceph-mon[41114]: pgmap v410: 148 pgs: 148 active+clean; 1.3 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T08:48:32.678 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:32 smithi092 ceph-mon[41114]: 6.0 scrub starts 2024-09-11T08:48:32.678 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:32 smithi092 ceph-mon[41114]: 6.0 scrub ok 2024-09-11T08:48:32.715 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:32.744 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:32.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:32.745 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:32.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:32.881 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:32.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:32.882 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:32.982 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:33.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:33.017 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:33.017 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:33.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-11T08:48:33.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:33.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:48:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:48:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:48:33.180 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:33.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:33.181 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:33.282 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:33.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:33.313 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:33.314 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:33.415 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:33.431 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:48:33.450 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:33.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:33.451 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:33.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:33.593 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:33.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:33.594 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:33.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[29647]: 4.10 scrub starts 2024-09-11T08:48:33.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[29647]: 4.10 scrub ok 2024-09-11T08:48:33.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/421777281' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:48:33.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:48:33.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[29647]: 5.18 scrub starts 2024-09-11T08:48:33.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[29647]: 5.18 scrub ok 2024-09-11T08:48:33.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[29647]: 6.8 scrub starts 2024-09-11T08:48:33.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[29647]: 6.8 scrub ok 2024-09-11T08:48:33.683 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[41114]: 4.10 scrub starts 2024-09-11T08:48:33.683 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[41114]: 4.10 scrub ok 2024-09-11T08:48:33.683 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/421777281' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:48:33.683 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:48:33.684 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[41114]: 5.18 scrub starts 2024-09-11T08:48:33.684 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[41114]: 5.18 scrub ok 2024-09-11T08:48:33.684 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[41114]: 6.8 scrub starts 2024-09-11T08:48:33.684 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:33 smithi092 ceph-mon[41114]: 6.8 scrub ok 2024-09-11T08:48:33.695 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:33.729 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:33.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:33.730 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:33.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:33.874 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:33.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:33.875 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:33.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:33 smithi145 ceph-mon[33384]: 4.10 scrub starts 2024-09-11T08:48:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:33 smithi145 ceph-mon[33384]: 4.10 scrub ok 2024-09-11T08:48:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:33 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/421777281' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:48:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:33 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:48:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:33 smithi145 ceph-mon[33384]: 5.18 scrub starts 2024-09-11T08:48:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:33 smithi145 ceph-mon[33384]: 5.18 scrub ok 2024-09-11T08:48:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:33 smithi145 ceph-mon[33384]: 6.8 scrub starts 2024-09-11T08:48:33.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:33 smithi145 ceph-mon[33384]: 6.8 scrub ok 2024-09-11T08:48:33.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:34.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:34.018 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:34.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:34.075 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:34.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:34.135 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:48:34.135 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:48:34.172 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:48:34.172 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:48:34.174 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:34.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:34.175 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:34.201 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:48:34.201 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:48:34.236 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:48:34.237 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:48:34.268 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:48:34.268 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:48:34.275 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:34.295 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:48:34.295 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:48:34.310 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:34.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:34.311 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:34.323 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:48:34.323 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:48:34.357 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:48:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:48:34.412 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:34.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:34.445 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:34.445 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:34.546 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:34.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[29647]: 8.9 scrub starts 2024-09-11T08:48:34.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[29647]: 6.4 scrub starts 2024-09-11T08:48:34.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[29647]: 6.4 scrub ok 2024-09-11T08:48:34.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[29647]: 8.9 scrub ok 2024-09-11T08:48:34.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[29647]: pgmap v411: 148 pgs: 148 active+clean; 995 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:48:34.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:48:34.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:48:34.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:48:34.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[29647]: osdmap e199: 8 total, 8 up, 8 in 2024-09-11T08:48:34.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/284961735' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:48:34.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[41114]: 8.9 scrub starts 2024-09-11T08:48:34.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[41114]: 6.4 scrub starts 2024-09-11T08:48:34.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[41114]: 6.4 scrub ok 2024-09-11T08:48:34.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[41114]: 8.9 scrub ok 2024-09-11T08:48:34.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[41114]: pgmap v411: 148 pgs: 148 active+clean; 995 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:48:34.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:48:34.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:48:34.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:48:34.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[41114]: osdmap e199: 8 total, 8 up, 8 in 2024-09-11T08:48:34.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:34 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/284961735' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:48:34.576 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:34.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:34.577 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:34.678 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:34.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:34.715 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:34.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:34.816 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:34.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:34.843 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:34.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:34.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:34 smithi145 ceph-mon[33384]: 8.9 scrub starts 2024-09-11T08:48:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:34 smithi145 ceph-mon[33384]: 6.4 scrub starts 2024-09-11T08:48:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:34 smithi145 ceph-mon[33384]: 6.4 scrub ok 2024-09-11T08:48:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:34 smithi145 ceph-mon[33384]: 8.9 scrub ok 2024-09-11T08:48:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:34 smithi145 ceph-mon[33384]: pgmap v411: 148 pgs: 148 active+clean; 995 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:48:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:34 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:48:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:34 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:48:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:34 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:48:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:34 smithi145 ceph-mon[33384]: osdmap e199: 8 total, 8 up, 8 in 2024-09-11T08:48:34.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:34 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/284961735' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:48:34.944 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:34.977 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:34.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:34.978 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:35.079 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:35.106 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:35.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:35.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:35.208 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:35.238 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:35.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:35.239 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:35.340 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:35.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:35.377 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:35.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:35.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:35.515 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:35.516 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:35.616 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:35.647 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:35.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:35.648 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:35.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:35.786 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:35.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:35.787 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:35.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:35 smithi092 ceph-mon[29647]: 6.1b scrub starts 2024-09-11T08:48:35.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:35 smithi092 ceph-mon[29647]: 6.1b scrub ok 2024-09-11T08:48:35.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:35 smithi092 ceph-mon[41114]: 6.1b scrub starts 2024-09-11T08:48:35.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:35 smithi092 ceph-mon[41114]: 6.1b scrub ok 2024-09-11T08:48:35.888 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:35.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:35 smithi145 ceph-mon[33384]: 6.1b scrub starts 2024-09-11T08:48:35.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:35 smithi145 ceph-mon[33384]: 6.1b scrub ok 2024-09-11T08:48:35.917 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:35.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:35.918 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:36.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:36.056 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:36.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:36.057 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:36.158 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:36.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:36.195 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:36.195 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:36.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:36.332 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:36.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:36.334 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:36.334 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:36.435 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:36.480 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:36.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:36.481 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:36 smithi092 ceph-mon[29647]: 3.1e scrub starts 2024-09-11T08:48:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:36 smithi092 ceph-mon[29647]: 3.1e scrub ok 2024-09-11T08:48:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:36 smithi092 ceph-mon[29647]: pgmap v413: 148 pgs: 148 active+clean; 995 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:48:36.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:36 smithi092 ceph-mon[41114]: 3.1e scrub starts 2024-09-11T08:48:36.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:36 smithi092 ceph-mon[41114]: 3.1e scrub ok 2024-09-11T08:48:36.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:36 smithi092 ceph-mon[41114]: pgmap v413: 148 pgs: 148 active+clean; 995 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:48:36.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:36.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:36.615 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:36.615 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:36.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:36.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:36.752 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:36.752 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:36.755 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:48:36.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:36.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:36.889 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:36.889 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:36.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:36 smithi145 ceph-mon[33384]: 3.1e scrub starts 2024-09-11T08:48:36.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:36 smithi145 ceph-mon[33384]: 3.1e scrub ok 2024-09-11T08:48:36.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:36 smithi145 ceph-mon[33384]: pgmap v413: 148 pgs: 148 active+clean; 995 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:48:36.912 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:48:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:48:36.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:37.037 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:37.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:37.038 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:37.139 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:37.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:37.167 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:37.167 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:37.268 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:37.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:37.300 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:37.300 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:37.401 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:37.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:37.431 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:37.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:37.532 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:37.562 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:37.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:37.563 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:37.664 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:37.693 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:37.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:37.694 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:37.795 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:37 smithi092 ceph-mon[29647]: 5.0 scrub starts 2024-09-11T08:48:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:37 smithi092 ceph-mon[29647]: 5.0 scrub ok 2024-09-11T08:48:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:37 smithi092 ceph-mon[29647]: 4.15 scrub starts 2024-09-11T08:48:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:37 smithi092 ceph-mon[29647]: 4.15 scrub ok 2024-09-11T08:48:37.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:37 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:37 smithi092 ceph-mon[41114]: 5.0 scrub starts 2024-09-11T08:48:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:37 smithi092 ceph-mon[41114]: 5.0 scrub ok 2024-09-11T08:48:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:37 smithi092 ceph-mon[41114]: 4.15 scrub starts 2024-09-11T08:48:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:37 smithi092 ceph-mon[41114]: 4.15 scrub ok 2024-09-11T08:48:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:37 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:37.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:37.825 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:37.825 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:37.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:37 smithi145 ceph-mon[33384]: 5.0 scrub starts 2024-09-11T08:48:37.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:37 smithi145 ceph-mon[33384]: 5.0 scrub ok 2024-09-11T08:48:37.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:37 smithi145 ceph-mon[33384]: 4.15 scrub starts 2024-09-11T08:48:37.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:37 smithi145 ceph-mon[33384]: 4.15 scrub ok 2024-09-11T08:48:37.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:37 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:37.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:37.962 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:37.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:37.963 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:38.064 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:38.097 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:38.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:38.098 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:38.198 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:38.231 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:38.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:38.232 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:38.317 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:48:38.333 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:38.359 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:38.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:38.360 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:38.461 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:38.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:38.509 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:38.509 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:38.609 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:38.635 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:38.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:38.637 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:38.737 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:38.766 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:38.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:38.767 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:38.781 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:38 smithi092 ceph-mon[29647]: pgmap v414: 148 pgs: 148 active+clean; 602 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:48:38.781 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:38 smithi092 ceph-mon[29647]: 4.6 scrub starts 2024-09-11T08:48:38.781 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:38 smithi092 ceph-mon[29647]: 4.6 scrub ok 2024-09-11T08:48:38.782 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:38 smithi092 ceph-mon[41114]: pgmap v414: 148 pgs: 148 active+clean; 602 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:48:38.782 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:38 smithi092 ceph-mon[41114]: 4.6 scrub starts 2024-09-11T08:48:38.782 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:38 smithi092 ceph-mon[41114]: 4.6 scrub ok 2024-09-11T08:48:38.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:38.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:38.908 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:38.908 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:38.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:38 smithi145 ceph-mon[33384]: pgmap v414: 148 pgs: 148 active+clean; 602 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:48:38.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:38 smithi145 ceph-mon[33384]: 4.6 scrub starts 2024-09-11T08:48:38.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:38 smithi145 ceph-mon[33384]: 4.6 scrub ok 2024-09-11T08:48:39.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:39.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:39.050 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:39.050 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:39.151 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:39.185 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:39.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:39.186 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:39.287 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:39.323 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:39.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:39.324 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:39.425 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:39.454 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:39.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:39.455 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:39.557 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:39.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:39.589 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:39.589 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:39.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:39.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:39.727 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:39.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:39 smithi092 ceph-mon[29647]: 4.2 deep-scrub starts 2024-09-11T08:48:39.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:39 smithi092 ceph-mon[29647]: 4.2 deep-scrub ok 2024-09-11T08:48:39.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:39 smithi092 ceph-mon[29647]: 3.7 scrub starts 2024-09-11T08:48:39.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:39 smithi092 ceph-mon[29647]: 3.7 scrub ok 2024-09-11T08:48:39.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:39 smithi092 ceph-mon[29647]: 3.13 scrub starts 2024-09-11T08:48:39.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:39 smithi092 ceph-mon[29647]: 3.13 scrub ok 2024-09-11T08:48:39.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:39 smithi092 ceph-mon[41114]: 4.2 deep-scrub starts 2024-09-11T08:48:39.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:39 smithi092 ceph-mon[41114]: 4.2 deep-scrub ok 2024-09-11T08:48:39.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:39 smithi092 ceph-mon[41114]: 3.7 scrub starts 2024-09-11T08:48:39.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:39 smithi092 ceph-mon[41114]: 3.7 scrub ok 2024-09-11T08:48:39.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:39 smithi092 ceph-mon[41114]: 3.13 scrub starts 2024-09-11T08:48:39.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:39 smithi092 ceph-mon[41114]: 3.13 scrub ok 2024-09-11T08:48:39.828 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:39.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:39.869 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:39.869 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:39.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:39 smithi145 ceph-mon[33384]: 4.2 deep-scrub starts 2024-09-11T08:48:39.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:39 smithi145 ceph-mon[33384]: 4.2 deep-scrub ok 2024-09-11T08:48:39.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:39 smithi145 ceph-mon[33384]: 3.7 scrub starts 2024-09-11T08:48:39.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:39 smithi145 ceph-mon[33384]: 3.7 scrub ok 2024-09-11T08:48:39.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:39 smithi145 ceph-mon[33384]: 3.13 scrub starts 2024-09-11T08:48:39.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:39 smithi145 ceph-mon[33384]: 3.13 scrub ok 2024-09-11T08:48:39.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:40.007 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:40.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:40.008 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:40.109 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:40.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:40.137 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:40.138 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:40.238 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:40.268 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:40.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:40.269 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:40.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:40.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:40.410 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:40.410 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:40.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:40.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:40.547 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:40.547 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:40.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:40.683 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:40.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:40.685 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:40.719 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:40 smithi092 ceph-mon[29647]: pgmap v415: 148 pgs: 148 active+clean; 602 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:48:40.719 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:40 smithi092 ceph-mon[41114]: pgmap v415: 148 pgs: 148 active+clean; 602 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:48:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:48:40.732 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:48:40.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:40.834 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:40.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:40.835 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:40.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:40 smithi145 ceph-mon[33384]: pgmap v415: 148 pgs: 148 active+clean; 602 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:48:40.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:40.971 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:40.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:40.972 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:41.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:41.116 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:41.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:41.118 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:41.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:41.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:41.265 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:41.265 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:41.366 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:41.397 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:41.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:41.399 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:41.500 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:41.543 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:41.544 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:41.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:41.544 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:41.548 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:41.593 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:48:41.594 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:48:41.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:41.681 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:41.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:41.682 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:41.685 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:41 smithi092 ceph-mon[29647]: 3.18 deep-scrub starts 2024-09-11T08:48:41.686 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:41 smithi092 ceph-mon[29647]: 3.18 deep-scrub ok 2024-09-11T08:48:41.686 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:41 smithi092 ceph-mon[29647]: from='client.25039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:48:41.686 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:41 smithi092 ceph-mon[29647]: 5.9 scrub starts 2024-09-11T08:48:41.686 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:41 smithi092 ceph-mon[29647]: 5.9 scrub ok 2024-09-11T08:48:41.686 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:48:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:48:41] "GET /metrics HTTP/1.1" 200 224340 "" "Prometheus/2.43.0" 2024-09-11T08:48:41.687 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:41 smithi092 ceph-mon[41114]: 3.18 deep-scrub starts 2024-09-11T08:48:41.687 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:41 smithi092 ceph-mon[41114]: 3.18 deep-scrub ok 2024-09-11T08:48:41.687 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:41 smithi092 ceph-mon[41114]: from='client.25039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:48:41.687 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:41 smithi092 ceph-mon[41114]: 5.9 scrub starts 2024-09-11T08:48:41.687 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:41 smithi092 ceph-mon[41114]: 5.9 scrub ok 2024-09-11T08:48:41.784 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:41.815 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:41.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:41.816 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:41.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:41 smithi145 ceph-mon[33384]: 3.18 deep-scrub starts 2024-09-11T08:48:41.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:41 smithi145 ceph-mon[33384]: 3.18 deep-scrub ok 2024-09-11T08:48:41.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:41 smithi145 ceph-mon[33384]: from='client.25039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:48:41.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:41 smithi145 ceph-mon[33384]: 5.9 scrub starts 2024-09-11T08:48:41.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:41 smithi145 ceph-mon[33384]: 5.9 scrub ok 2024-09-11T08:48:41.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:41.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:41.954 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:41.954 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:42.055 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:42.083 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:42.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:42.084 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:42.185 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:42.211 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:42.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:42.212 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:42.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:42.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:42.348 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:42.348 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:42.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:42.490 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:42.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:42.491 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:42.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[29647]: 8.1 scrub starts 2024-09-11T08:48:42.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[29647]: 8.1 scrub ok 2024-09-11T08:48:42.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[29647]: pgmap v416: 148 pgs: 148 active+clean; 531 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-11T08:48:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[29647]: 3.1d scrub starts 2024-09-11T08:48:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[29647]: 3.1d scrub ok 2024-09-11T08:48:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[29647]: 6.17 scrub starts 2024-09-11T08:48:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[29647]: 6.17 scrub ok 2024-09-11T08:48:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[29647]: 4.b scrub starts 2024-09-11T08:48:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[29647]: 4.b scrub ok 2024-09-11T08:48:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[41114]: 8.1 scrub starts 2024-09-11T08:48:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[41114]: 8.1 scrub ok 2024-09-11T08:48:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[41114]: pgmap v416: 148 pgs: 148 active+clean; 531 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-11T08:48:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[41114]: 3.1d scrub starts 2024-09-11T08:48:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[41114]: 3.1d scrub ok 2024-09-11T08:48:42.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[41114]: 6.17 scrub starts 2024-09-11T08:48:42.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[41114]: 6.17 scrub ok 2024-09-11T08:48:42.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[41114]: 4.b scrub starts 2024-09-11T08:48:42.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:42 smithi092 ceph-mon[41114]: 4.b scrub ok 2024-09-11T08:48:42.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:42.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:42.623 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:42.624 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:42.721 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:42 smithi145 ceph-mon[33384]: 8.1 scrub starts 2024-09-11T08:48:42.721 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:42 smithi145 ceph-mon[33384]: 8.1 scrub ok 2024-09-11T08:48:42.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:42 smithi145 ceph-mon[33384]: pgmap v416: 148 pgs: 148 active+clean; 531 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-11T08:48:42.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:42 smithi145 ceph-mon[33384]: 3.1d scrub starts 2024-09-11T08:48:42.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:42 smithi145 ceph-mon[33384]: 3.1d scrub ok 2024-09-11T08:48:42.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:42 smithi145 ceph-mon[33384]: 6.17 scrub starts 2024-09-11T08:48:42.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:42 smithi145 ceph-mon[33384]: 6.17 scrub ok 2024-09-11T08:48:42.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:42 smithi145 ceph-mon[33384]: 4.b scrub starts 2024-09-11T08:48:42.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:42 smithi145 ceph-mon[33384]: 4.b scrub ok 2024-09-11T08:48:42.724 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:42.757 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:42.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:42.757 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:42.858 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:42.887 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:42.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:42.888 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:42.989 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:43.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:43.016 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:43.016 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:43.117 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:43.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:43.152 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:43.152 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:43.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:48:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:48:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:48:43.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:43.283 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-09-11T08:48:39.210766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":630790388,"num_objects":9813,"num_object_clones":0,"num_object_copies":29439,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9813,"num_whiteouts":0,"num_read":1441,"num_read_kb":1298,"num_write":230844,"num_write_kb":7678984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7873,"num_bytes_recovered":509741200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85925,"ondisk_log_size":85925,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":446,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":4818768,"kb_used_data":2377296,"kb_used_omap":13,"kb_used_meta":2441138,"kb_avail":745142448,"statfs":{"total":767960285184,"available":763025866752,"internally_reserved":0,"allocated":2434351104,"data_stored":2431786444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2499726016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,4,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-820576256,"num_objects":-12521,"num_object_clones":0,"num_object_copies":-37563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-12521,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":12521,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003607"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":192,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.383327+0000","last_change":"2024-09-11T08:45:06.574593+0000","last_active":"2024-09-11T08:48:24.383327+0000","last_peered":"2024-09-11T08:48:24.383327+0000","last_clean":"2024-09-11T08:48:24.383327+0000","last_became_active":"2024-09-11T08:38:38.953197+0000","last_became_peered":"2024-09-11T08:38:38.953197+0000","last_unstale":"2024-09-11T08:48:24.383327+0000","last_undegraded":"2024-09-11T08:48:24.383327+0000","last_fullsized":"2024-09-11T08:48:24.383327+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:06.574539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.574539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028477630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"78'13","reported_seq":240,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.374945+0000","last_change":"2024-09-11T08:46:01.991434+0000","last_active":"2024-09-11T08:48:24.374945+0000","last_peered":"2024-09-11T08:48:24.374945+0000","last_clean":"2024-09-11T08:48:24.374945+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:48:24.374945+0000","last_undegraded":"2024-09-11T08:48:24.374945+0000","last_fullsized":"2024-09-11T08:48:24.374945+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:45:13.113315+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.113315+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018068470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":198,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.373613+0000","last_change":"2024-09-11T08:47:44.884292+0000","last_active":"2024-09-11T08:48:24.373613+0000","last_peered":"2024-09-11T08:48:24.373613+0000","last_clean":"2024-09-11T08:48:24.373613+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:48:24.373613+0000","last_undegraded":"2024-09-11T08:48:24.373613+0000","last_fullsized":"2024-09-11T08:48:24.373613+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.884217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.884217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:57.658768+0000","scrub_duration":0.011480829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":166,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375269+0000","last_change":"2024-09-11T08:44:26.005322+0000","last_active":"2024-09-11T08:48:24.375269+0000","last_peered":"2024-09-11T08:48:24.375269+0000","last_clean":"2024-09-11T08:48:24.375269+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:48:24.375269+0000","last_undegraded":"2024-09-11T08:48:24.375269+0000","last_fullsized":"2024-09-11T08:48:24.375269+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.005270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.005270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038675070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":205,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:34.059667+0000","last_change":"2024-09-11T08:48:34.059380+0000","last_active":"2024-09-11T08:48:34.059667+0000","last_peered":"2024-09-11T08:48:34.059667+0000","last_clean":"2024-09-11T08:48:34.059667+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:48:34.059667+0000","last_undegraded":"2024-09-11T08:48:34.059667+0000","last_fullsized":"2024-09-11T08:48:34.059667+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:34.059309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:34.059309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:59.700862+0000","scrub_duration":0.0067175359999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":211,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:35.046208+0000","last_change":"2024-09-11T08:48:35.046165+0000","last_active":"2024-09-11T08:48:35.046208+0000","last_peered":"2024-09-11T08:48:35.046208+0000","last_clean":"2024-09-11T08:48:35.046208+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:48:35.046208+0000","last_undegraded":"2024-09-11T08:48:35.046208+0000","last_fullsized":"2024-09-11T08:48:35.046208+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:35.046075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:35.046075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:46.499902+0000","scrub_duration":0.010511866,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"78'22","reported_seq":221,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:30.029974+0000","last_change":"2024-09-11T08:48:30.029907+0000","last_active":"2024-09-11T08:48:30.029974+0000","last_peered":"2024-09-11T08:48:30.029974+0000","last_clean":"2024-09-11T08:48:30.029974+0000","last_became_active":"2024-09-11T08:38:40.972787+0000","last_became_peered":"2024-09-11T08:38:40.972787+0000","last_unstale":"2024-09-11T08:48:30.029974+0000","last_undegraded":"2024-09-11T08:48:30.029974+0000","last_fullsized":"2024-09-11T08:48:30.029974+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:48:30.029817+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.029817+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:59.033520+0000","scrub_duration":0.0055582840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":178,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:32.827275+0000","last_change":"2024-09-11T08:48:32.827213+0000","last_active":"2024-09-11T08:48:32.827275+0000","last_peered":"2024-09-11T08:48:32.827275+0000","last_clean":"2024-09-11T08:48:32.827275+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:48:32.827275+0000","last_undegraded":"2024-09-11T08:48:32.827275+0000","last_fullsized":"2024-09-11T08:48:32.827275+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:32.827110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:48:32.827110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:37.036882+0000","scrub_duration":0.0029807079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":208,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.366205+0000","last_change":"2024-09-11T08:47:32.636647+0000","last_active":"2024-09-11T08:48:24.366205+0000","last_peered":"2024-09-11T08:48:24.366205+0000","last_clean":"2024-09-11T08:48:24.366205+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:48:24.366205+0000","last_undegraded":"2024-09-11T08:48:24.366205+0000","last_fullsized":"2024-09-11T08:48:24.366205+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:12.878132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:44:12.878132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077623500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"78'13","reported_seq":200,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375388+0000","last_change":"2024-09-11T08:44:27.963772+0000","last_active":"2024-09-11T08:48:24.375388+0000","last_peered":"2024-09-11T08:48:24.375388+0000","last_clean":"2024-09-11T08:48:24.375388+0000","last_became_active":"2024-09-11T08:38:40.973911+0000","last_became_peered":"2024-09-11T08:38:40.973911+0000","last_unstale":"2024-09-11T08:48:24.375388+0000","last_undegraded":"2024-09-11T08:48:24.375388+0000","last_fullsized":"2024-09-11T08:48:24.375388+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:44:27.963689+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:44:27.963689+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011544877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":194,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.365195+0000","last_change":"2024-09-11T08:48:18.341763+0000","last_active":"2024-09-11T08:48:24.365195+0000","last_peered":"2024-09-11T08:48:24.365195+0000","last_clean":"2024-09-11T08:48:24.365195+0000","last_became_active":"2024-09-11T08:38:42.981368+0000","last_became_peered":"2024-09-11T08:38:42.981368+0000","last_unstale":"2024-09-11T08:48:24.365195+0000","last_undegraded":"2024-09-11T08:48:24.365195+0000","last_fullsized":"2024-09-11T08:48:24.365195+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:18.341709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.341709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:22.690931+0000","scrub_duration":0.0038165500000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":186,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.382008+0000","last_change":"2024-09-11T08:45:31.457775+0000","last_active":"2024-09-11T08:48:24.382008+0000","last_peered":"2024-09-11T08:48:24.382008+0000","last_clean":"2024-09-11T08:48:24.382008+0000","last_became_active":"2024-09-11T08:38:44.997396+0000","last_became_peered":"2024-09-11T08:38:44.997396+0000","last_unstale":"2024-09-11T08:48:24.382008+0000","last_undegraded":"2024-09-11T08:48:24.382008+0000","last_fullsized":"2024-09-11T08:48:24.382008+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.457652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.457652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006759045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":188,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.366157+0000","last_change":"2024-09-11T08:47:33.593768+0000","last_active":"2024-09-11T08:48:24.366157+0000","last_peered":"2024-09-11T08:48:24.366157+0000","last_clean":"2024-09-11T08:48:24.366157+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:48:24.366157+0000","last_undegraded":"2024-09-11T08:48:24.366157+0000","last_fullsized":"2024-09-11T08:48:24.366157+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:09.871637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:09.871637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029269349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":190,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375211+0000","last_change":"2024-09-11T08:47:44.866144+0000","last_active":"2024-09-11T08:48:24.375211+0000","last_peered":"2024-09-11T08:48:24.375211+0000","last_clean":"2024-09-11T08:48:24.375211+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:48:24.375211+0000","last_undegraded":"2024-09-11T08:48:24.375211+0000","last_fullsized":"2024-09-11T08:48:24.375211+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:47:44.866057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.866057+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:03.406837+0000","scrub_duration":0.0020690169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":197,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.367327+0000","last_change":"2024-09-11T08:47:46.410478+0000","last_active":"2024-09-11T08:48:24.367327+0000","last_peered":"2024-09-11T08:48:24.367327+0000","last_clean":"2024-09-11T08:48:24.367327+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:48:24.367327+0000","last_undegraded":"2024-09-11T08:48:24.367327+0000","last_fullsized":"2024-09-11T08:48:24.367327+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:46.410402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:47:46.410402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:51.920397+0000","scrub_duration":0.002594884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":182,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.365025+0000","last_change":"2024-09-11T08:47:38.519964+0000","last_active":"2024-09-11T08:48:24.365025+0000","last_peered":"2024-09-11T08:48:24.365025+0000","last_clean":"2024-09-11T08:48:24.365025+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:48:24.365025+0000","last_undegraded":"2024-09-11T08:48:24.365025+0000","last_fullsized":"2024-09-11T08:48:24.365025+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.850684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.850684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058755379999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":196,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375496+0000","last_change":"2024-09-11T08:46:01.978488+0000","last_active":"2024-09-11T08:48:24.375496+0000","last_peered":"2024-09-11T08:48:24.375496+0000","last_clean":"2024-09-11T08:48:24.375496+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:48:24.375496+0000","last_undegraded":"2024-09-11T08:48:24.375496+0000","last_fullsized":"2024-09-11T08:48:24.375496+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:24.197149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.197149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001510267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":192,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.383078+0000","last_change":"2024-09-11T08:47:34.610411+0000","last_active":"2024-09-11T08:48:24.383078+0000","last_peered":"2024-09-11T08:48:24.383078+0000","last_clean":"2024-09-11T08:48:24.383078+0000","last_became_active":"2024-09-11T08:38:38.953763+0000","last_became_peered":"2024-09-11T08:38:38.953763+0000","last_unstale":"2024-09-11T08:48:24.383078+0000","last_undegraded":"2024-09-11T08:48:24.383078+0000","last_fullsized":"2024-09-11T08:48:24.383078+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:34.610331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:34.610331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:51.178932+0000","scrub_duration":0.0021231090000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"78'22","reported_seq":247,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.378512+0000","last_change":"2024-09-11T08:48:20.311201+0000","last_active":"2024-09-11T08:48:24.378512+0000","last_peered":"2024-09-11T08:48:24.378512+0000","last_clean":"2024-09-11T08:48:24.378512+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:48:24.378512+0000","last_undegraded":"2024-09-11T08:48:24.378512+0000","last_fullsized":"2024-09-11T08:48:24.378512+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:48:20.311122+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.311122+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:35.297396+0000","scrub_duration":0.011484708999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":198,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375169+0000","last_change":"2024-09-11T08:48:19.961535+0000","last_active":"2024-09-11T08:48:24.375169+0000","last_peered":"2024-09-11T08:48:24.375169+0000","last_clean":"2024-09-11T08:48:24.375169+0000","last_became_active":"2024-09-11T08:38:42.975540+0000","last_became_peered":"2024-09-11T08:38:42.975540+0000","last_unstale":"2024-09-11T08:48:24.375169+0000","last_undegraded":"2024-09-11T08:48:24.375169+0000","last_fullsized":"2024-09-11T08:48:24.375169+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:19.961460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.961460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:30.369869+0000","scrub_duration":0.0091369829999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":204,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.374761+0000","last_change":"2024-09-11T08:46:01.998163+0000","last_active":"2024-09-11T08:48:24.374761+0000","last_peered":"2024-09-11T08:48:24.374761+0000","last_clean":"2024-09-11T08:48:24.374761+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:48:24.374761+0000","last_undegraded":"2024-09-11T08:48:24.374761+0000","last_fullsized":"2024-09-11T08:48:24.374761+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:24.223237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.223237+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019917720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":182,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375456+0000","last_change":"2024-09-11T08:45:28.194198+0000","last_active":"2024-09-11T08:48:24.375456+0000","last_peered":"2024-09-11T08:48:24.375456+0000","last_clean":"2024-09-11T08:48:24.375456+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:48:24.375456+0000","last_undegraded":"2024-09-11T08:48:24.375456+0000","last_fullsized":"2024-09-11T08:48:24.375456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.194095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.194095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001559936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":202,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.372083+0000","last_change":"2024-09-11T08:45:16.162671+0000","last_active":"2024-09-11T08:48:24.372083+0000","last_peered":"2024-09-11T08:48:24.372083+0000","last_clean":"2024-09-11T08:48:24.372083+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:48:24.372083+0000","last_undegraded":"2024-09-11T08:48:24.372083+0000","last_fullsized":"2024-09-11T08:48:24.372083+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:16.162616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:16.162616+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001890833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":178,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375251+0000","last_change":"2024-09-11T08:45:22.244097+0000","last_active":"2024-09-11T08:48:24.375251+0000","last_peered":"2024-09-11T08:48:24.375251+0000","last_clean":"2024-09-11T08:48:24.375251+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:48:24.375251+0000","last_undegraded":"2024-09-11T08:48:24.375251+0000","last_fullsized":"2024-09-11T08:48:24.375251+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.244038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.244038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015252620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":186,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.367760+0000","last_change":"2024-09-11T08:45:19.335163+0000","last_active":"2024-09-11T08:48:24.367760+0000","last_peered":"2024-09-11T08:48:24.367760+0000","last_clean":"2024-09-11T08:48:24.367760+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:48:24.367760+0000","last_undegraded":"2024-09-11T08:48:24.367760+0000","last_fullsized":"2024-09-11T08:48:24.367760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:19.335065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:19.335065+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026052929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":202,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:25.971451+0000","last_change":"2024-09-11T08:48:25.971280+0000","last_active":"2024-09-11T08:48:25.971451+0000","last_peered":"2024-09-11T08:48:25.971451+0000","last_clean":"2024-09-11T08:48:25.971451+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:48:25.971451+0000","last_undegraded":"2024-09-11T08:48:25.971451+0000","last_fullsized":"2024-09-11T08:48:25.971451+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:25.971151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.971151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:47.552719+0000","scrub_duration":0.0095705709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":208,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.384574+0000","last_change":"2024-09-11T08:48:19.705275+0000","last_active":"2024-09-11T08:48:24.384574+0000","last_peered":"2024-09-11T08:48:24.384574+0000","last_clean":"2024-09-11T08:48:24.384574+0000","last_became_active":"2024-09-11T08:38:41.040636+0000","last_became_peered":"2024-09-11T08:38:41.040636+0000","last_unstale":"2024-09-11T08:48:24.384574+0000","last_undegraded":"2024-09-11T08:48:24.384574+0000","last_fullsized":"2024-09-11T08:48:24.384574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:48:19.705185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.705185+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:42.863183+0000","scrub_duration":0.0050623580000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":327,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:32.791379+0000","last_change":"2024-09-11T08:48:17.812378+0000","last_active":"2024-09-11T08:48:32.791379+0000","last_peered":"2024-09-11T08:48:32.791379+0000","last_clean":"2024-09-11T08:48:32.791379+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:48:32.791379+0000","last_undegraded":"2024-09-11T08:48:32.791379+0000","last_fullsized":"2024-09-11T08:48:32.791379+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:48:17.812256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:17.812256+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:40.308022+0000","scrub_duration":0.010983438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"199'14249","reported_seq":14306,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.494545+0000","last_change":"2024-09-11T08:45:15.261892+0000","last_active":"2024-09-11T08:48:36.494545+0000","last_peered":"2024-09-11T08:48:36.494545+0000","last_clean":"2024-09-11T08:48:36.494545+0000","last_became_active":"2024-09-11T08:45:15.261608+0000","last_became_peered":"2024-09-11T08:45:15.261608+0000","last_unstale":"2024-09-11T08:48:36.494545+0000","last_undegraded":"2024-09-11T08:48:36.494545+0000","last_fullsized":"2024-09-11T08:48:36.494545+0000","mapping_epoch":149,"log_start":"186'9300","ondisk_log_start":"186'9300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39256064,"num_objects":599,"num_object_clones":0,"num_object_copies":1797,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14249,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"124'22","reported_seq":222,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.376698+0000","last_change":"2024-09-11T08:45:18.777528+0000","last_active":"2024-09-11T08:48:24.376698+0000","last_peered":"2024-09-11T08:48:24.376698+0000","last_clean":"2024-09-11T08:48:24.376698+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:48:24.376698+0000","last_undegraded":"2024-09-11T08:48:24.376698+0000","last_fullsized":"2024-09-11T08:48:24.376698+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:45:18.777466+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.777466+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045440580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":181,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.376112+0000","last_change":"2024-09-11T08:45:13.214007+0000","last_active":"2024-09-11T08:48:24.376112+0000","last_peered":"2024-09-11T08:48:24.376112+0000","last_clean":"2024-09-11T08:48:24.376112+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:48:24.376112+0000","last_undegraded":"2024-09-11T08:48:24.376112+0000","last_fullsized":"2024-09-11T08:48:24.376112+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:13.213922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.213922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016325280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"154'9","reported_seq":306,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:32.791383+0000","last_change":"2024-09-11T08:45:26.208887+0000","last_active":"2024-09-11T08:48:32.791383+0000","last_peered":"2024-09-11T08:48:32.791383+0000","last_clean":"2024-09-11T08:48:32.791383+0000","last_became_active":"2024-09-11T08:38:42.985537+0000","last_became_peered":"2024-09-11T08:38:42.985537+0000","last_unstale":"2024-09-11T08:48:32.791383+0000","last_undegraded":"2024-09-11T08:48:32.791383+0000","last_fullsized":"2024-09-11T08:48:32.791383+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'9","last_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.208818+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028933230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":179,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.365696+0000","last_change":"2024-09-11T08:47:39.523803+0000","last_active":"2024-09-11T08:48:24.365696+0000","last_peered":"2024-09-11T08:48:24.365696+0000","last_clean":"2024-09-11T08:48:24.365696+0000","last_became_active":"2024-09-11T08:38:44.991303+0000","last_became_peered":"2024-09-11T08:38:44.991303+0000","last_unstale":"2024-09-11T08:48:24.365696+0000","last_undegraded":"2024-09-11T08:48:24.365696+0000","last_fullsized":"2024-09-11T08:48:24.365696+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:24.813857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.813857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01118625,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"199'14465","reported_seq":14523,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.824935+0000","last_change":"2024-09-11T08:45:15.297710+0000","last_active":"2024-09-11T08:48:36.824935+0000","last_peered":"2024-09-11T08:48:36.824935+0000","last_clean":"2024-09-11T08:48:36.824935+0000","last_became_active":"2024-09-11T08:45:15.297378+0000","last_became_peered":"2024-09-11T08:45:15.297378+0000","last_unstale":"2024-09-11T08:48:36.824935+0000","last_undegraded":"2024-09-11T08:48:36.824935+0000","last_fullsized":"2024-09-11T08:48:36.824935+0000","mapping_epoch":149,"log_start":"185'8800","ondisk_log_start":"185'8800","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5665,"log_dups_size":0,"ondisk_log_size":5665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41484288,"num_objects":633,"num_object_clones":0,"num_object_copies":1899,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14465,"num_write_kb":483136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":211,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375979+0000","last_change":"2024-09-11T08:44:33.387066+0000","last_active":"2024-09-11T08:48:24.375979+0000","last_peered":"2024-09-11T08:48:24.375979+0000","last_clean":"2024-09-11T08:48:24.375979+0000","last_became_active":"2024-09-11T08:38:41.040221+0000","last_became_peered":"2024-09-11T08:38:41.040221+0000","last_unstale":"2024-09-11T08:48:24.375979+0000","last_undegraded":"2024-09-11T08:48:24.375979+0000","last_fullsized":"2024-09-11T08:48:24.375979+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-11T08:44:33.386972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:33.386972+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022646007999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":181,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375266+0000","last_change":"2024-09-11T08:44:25.327692+0000","last_active":"2024-09-11T08:48:24.375266+0000","last_peered":"2024-09-11T08:48:24.375266+0000","last_clean":"2024-09-11T08:48:24.375266+0000","last_became_active":"2024-09-11T08:38:38.955305+0000","last_became_peered":"2024-09-11T08:38:38.955305+0000","last_unstale":"2024-09-11T08:48:24.375266+0000","last_undegraded":"2024-09-11T08:48:24.375266+0000","last_fullsized":"2024-09-11T08:48:24.375266+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:25.327553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:25.327553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016327754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":178,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.374943+0000","last_change":"2024-09-11T08:45:17.150827+0000","last_active":"2024-09-11T08:48:24.374943+0000","last_peered":"2024-09-11T08:48:24.374943+0000","last_clean":"2024-09-11T08:48:24.374943+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:48:24.374943+0000","last_undegraded":"2024-09-11T08:48:24.374943+0000","last_fullsized":"2024-09-11T08:48:24.374943+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.150767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.150767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001327707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":176,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375354+0000","last_change":"2024-09-11T08:46:03.144149+0000","last_active":"2024-09-11T08:48:24.375354+0000","last_peered":"2024-09-11T08:48:24.375354+0000","last_clean":"2024-09-11T08:48:24.375354+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:48:24.375354+0000","last_undegraded":"2024-09-11T08:48:24.375354+0000","last_fullsized":"2024-09-11T08:48:24.375354+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:46:03.144096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:46:03.144096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001497151,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"199'16649","reported_seq":16810,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.286900+0000","last_change":"2024-09-11T08:47:59.605543+0000","last_active":"2024-09-11T08:48:36.286900+0000","last_peered":"2024-09-11T08:48:36.286900+0000","last_clean":"2024-09-11T08:48:36.286900+0000","last_became_active":"2024-09-11T08:45:15.627210+0000","last_became_peered":"2024-09-11T08:45:15.627210+0000","last_unstale":"2024-09-11T08:48:36.286900+0000","last_undegraded":"2024-09-11T08:48:36.286900+0000","last_fullsized":"2024-09-11T08:48:36.286900+0000","mapping_epoch":149,"log_start":"191'11201","ondisk_log_start":"191'11201","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":2409,"log_size":5448,"log_dups_size":0,"ondisk_log_size":5448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39125016,"num_objects":598,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":598,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14240,"num_write_kb":474753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"86'21","reported_seq":208,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375244+0000","last_change":"2024-09-11T08:48:23.862534+0000","last_active":"2024-09-11T08:48:24.375244+0000","last_peered":"2024-09-11T08:48:24.375244+0000","last_clean":"2024-09-11T08:48:24.375244+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:48:24.375244+0000","last_undegraded":"2024-09-11T08:48:24.375244+0000","last_fullsized":"2024-09-11T08:48:24.375244+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'21","last_scrub_stamp":"2024-09-11T08:48:23.862482+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:48:23.862482+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:27.496799+0000","scrub_duration":0.001795776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":192,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.384791+0000","last_change":"2024-09-11T08:45:30.475446+0000","last_active":"2024-09-11T08:48:24.384791+0000","last_peered":"2024-09-11T08:48:24.384791+0000","last_clean":"2024-09-11T08:48:24.384791+0000","last_became_active":"2024-09-11T08:38:38.956977+0000","last_became_peered":"2024-09-11T08:38:38.956977+0000","last_unstale":"2024-09-11T08:48:24.384791+0000","last_undegraded":"2024-09-11T08:48:24.384791+0000","last_fullsized":"2024-09-11T08:48:24.384791+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:30.475365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:30.475365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010690405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":307,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.052816+0000","last_change":"2024-09-11T08:48:36.052650+0000","last_active":"2024-09-11T08:48:36.052816+0000","last_peered":"2024-09-11T08:48:36.052816+0000","last_clean":"2024-09-11T08:48:36.052816+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:48:36.052816+0000","last_undegraded":"2024-09-11T08:48:36.052816+0000","last_fullsized":"2024-09-11T08:48:36.052816+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:48:36.052514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:36.052514+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:47.227630+0000","scrub_duration":0.015857353000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":205,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.367564+0000","last_change":"2024-09-11T08:46:01.983654+0000","last_active":"2024-09-11T08:48:24.367564+0000","last_peered":"2024-09-11T08:48:24.367564+0000","last_clean":"2024-09-11T08:48:24.367564+0000","last_became_active":"2024-09-11T08:46:01.983335+0000","last_became_peered":"2024-09-11T08:46:01.983335+0000","last_unstale":"2024-09-11T08:48:24.367564+0000","last_undegraded":"2024-09-11T08:48:24.367564+0000","last_fullsized":"2024-09-11T08:48:24.367564+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.303194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.303194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001492431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"199'14316","reported_seq":14374,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.805734+0000","last_change":"2024-09-11T08:45:15.626487+0000","last_active":"2024-09-11T08:48:36.805734+0000","last_peered":"2024-09-11T08:48:36.805734+0000","last_clean":"2024-09-11T08:48:36.805734+0000","last_became_active":"2024-09-11T08:45:15.626210+0000","last_became_peered":"2024-09-11T08:45:15.626210+0000","last_unstale":"2024-09-11T08:48:36.805734+0000","last_undegraded":"2024-09-11T08:48:36.805734+0000","last_fullsized":"2024-09-11T08:48:36.805734+0000","mapping_epoch":149,"log_start":"185'8600","ondisk_log_start":"185'8600","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5716,"log_dups_size":0,"ondisk_log_size":5716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39059456,"num_objects":596,"num_object_clones":0,"num_object_copies":1788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14316,"num_write_kb":477184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"100'15","reported_seq":210,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.373905+0000","last_change":"2024-09-11T08:45:12.769299+0000","last_active":"2024-09-11T08:48:24.373905+0000","last_peered":"2024-09-11T08:48:24.373905+0000","last_clean":"2024-09-11T08:48:24.373905+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:48:24.373905+0000","last_undegraded":"2024-09-11T08:48:24.373905+0000","last_fullsized":"2024-09-11T08:48:24.373905+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'15","last_scrub_stamp":"2024-09-11T08:45:12.769239+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:41:56.250393+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.769239+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026298300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":188,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.366269+0000","last_change":"2024-09-11T08:47:37.534624+0000","last_active":"2024-09-11T08:48:24.366269+0000","last_peered":"2024-09-11T08:48:24.366269+0000","last_clean":"2024-09-11T08:48:24.366269+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:48:24.366269+0000","last_undegraded":"2024-09-11T08:48:24.366269+0000","last_fullsized":"2024-09-11T08:48:24.366269+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:19.905459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:19.905459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015766237999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":305,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:32.792202+0000","last_change":"2024-09-11T08:44:34.630457+0000","last_active":"2024-09-11T08:48:32.792202+0000","last_peered":"2024-09-11T08:48:32.792202+0000","last_clean":"2024-09-11T08:48:32.792202+0000","last_became_active":"2024-09-11T08:38:42.981476+0000","last_became_peered":"2024-09-11T08:38:42.981476+0000","last_unstale":"2024-09-11T08:48:32.792202+0000","last_undegraded":"2024-09-11T08:48:32.792202+0000","last_fullsized":"2024-09-11T08:48:32.792202+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:44:34.630191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:44:34.630191+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089275420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":189,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:31.710957+0000","last_change":"2024-09-11T08:48:31.710895+0000","last_active":"2024-09-11T08:48:31.710957+0000","last_peered":"2024-09-11T08:48:31.710957+0000","last_clean":"2024-09-11T08:48:31.710957+0000","last_became_active":"2024-09-11T08:38:44.997784+0000","last_became_peered":"2024-09-11T08:38:44.997784+0000","last_unstale":"2024-09-11T08:48:31.710957+0000","last_undegraded":"2024-09-11T08:48:31.710957+0000","last_fullsized":"2024-09-11T08:48:31.710957+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:31.710797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:48:31.710797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:01.612068+0000","scrub_duration":0.010736144,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"199'14368","reported_seq":14426,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.759024+0000","last_change":"2024-09-11T08:45:15.627960+0000","last_active":"2024-09-11T08:48:36.759024+0000","last_peered":"2024-09-11T08:48:36.759024+0000","last_clean":"2024-09-11T08:48:36.759024+0000","last_became_active":"2024-09-11T08:45:15.627639+0000","last_became_peered":"2024-09-11T08:45:15.627639+0000","last_unstale":"2024-09-11T08:48:36.759024+0000","last_undegraded":"2024-09-11T08:48:36.759024+0000","last_fullsized":"2024-09-11T08:48:36.759024+0000","mapping_epoch":149,"log_start":"186'9500","ondisk_log_start":"186'9500","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4868,"log_dups_size":0,"ondisk_log_size":4868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36700160,"num_objects":560,"num_object_clones":0,"num_object_copies":1680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14368,"num_write_kb":477696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":216,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.382948+0000","last_change":"2024-09-11T08:45:11.576184+0000","last_active":"2024-09-11T08:48:24.382948+0000","last_peered":"2024-09-11T08:48:24.382948+0000","last_clean":"2024-09-11T08:48:24.382948+0000","last_became_active":"2024-09-11T08:38:41.039722+0000","last_became_peered":"2024-09-11T08:38:41.039722+0000","last_unstale":"2024-09-11T08:48:24.382948+0000","last_undegraded":"2024-09-11T08:48:24.382948+0000","last_fullsized":"2024-09-11T08:48:24.382948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:45:11.576085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.576085+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002366754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":202,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.374818+0000","last_change":"2024-09-11T08:48:24.062209+0000","last_active":"2024-09-11T08:48:24.374818+0000","last_peered":"2024-09-11T08:48:24.374818+0000","last_clean":"2024-09-11T08:48:24.374818+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:48:24.374818+0000","last_undegraded":"2024-09-11T08:48:24.374818+0000","last_fullsized":"2024-09-11T08:48:24.374818+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:24.062110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:24.062110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:38.001551+0000","scrub_duration":0.037841910999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":197,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:25.855445+0000","last_change":"2024-09-11T08:48:25.855383+0000","last_active":"2024-09-11T08:48:25.855445+0000","last_peered":"2024-09-11T08:48:25.855445+0000","last_clean":"2024-09-11T08:48:25.855445+0000","last_became_active":"2024-09-11T08:38:42.984172+0000","last_became_peered":"2024-09-11T08:38:42.984172+0000","last_unstale":"2024-09-11T08:48:25.855445+0000","last_undegraded":"2024-09-11T08:48:25.855445+0000","last_fullsized":"2024-09-11T08:48:25.855445+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:25.855301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.855301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:36.628834+0000","scrub_duration":0.0039132810000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":216,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:27.981221+0000","last_change":"2024-09-11T08:48:27.981040+0000","last_active":"2024-09-11T08:48:27.981221+0000","last_peered":"2024-09-11T08:48:27.981221+0000","last_clean":"2024-09-11T08:48:27.981221+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:48:27.981221+0000","last_undegraded":"2024-09-11T08:48:27.981221+0000","last_fullsized":"2024-09-11T08:48:27.981221+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:27.980860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:48:27.980860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:30.102782+0000","scrub_duration":0.021574010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"199'14393","reported_seq":14451,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.283794+0000","last_change":"2024-09-11T08:45:15.298396+0000","last_active":"2024-09-11T08:48:36.283794+0000","last_peered":"2024-09-11T08:48:36.283794+0000","last_clean":"2024-09-11T08:48:36.283794+0000","last_became_active":"2024-09-11T08:45:15.298053+0000","last_became_peered":"2024-09-11T08:45:15.298053+0000","last_unstale":"2024-09-11T08:48:36.283794+0000","last_undegraded":"2024-09-11T08:48:36.283794+0000","last_fullsized":"2024-09-11T08:48:36.283794+0000","mapping_epoch":149,"log_start":"185'9000","ondisk_log_start":"185'9000","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5393,"log_dups_size":0,"ondisk_log_size":5393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42270720,"num_objects":645,"num_object_clones":0,"num_object_copies":1935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14393,"num_write_kb":481216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"78'38","reported_seq":377,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:32.792115+0000","last_change":"2024-09-11T08:45:26.666261+0000","last_active":"2024-09-11T08:48:32.792115+0000","last_peered":"2024-09-11T08:48:32.792115+0000","last_clean":"2024-09-11T08:48:32.792115+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:48:32.792115+0000","last_undegraded":"2024-09-11T08:48:32.792115+0000","last_fullsized":"2024-09-11T08:48:32.792115+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:45:26.666199+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.666199+0000","objects_scrubbed":10,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020004279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":182,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.376126+0000","last_change":"2024-09-11T08:45:15.130637+0000","last_active":"2024-09-11T08:48:24.376126+0000","last_peered":"2024-09-11T08:48:24.376126+0000","last_clean":"2024-09-11T08:48:24.376126+0000","last_became_active":"2024-09-11T08:38:38.954675+0000","last_became_peered":"2024-09-11T08:38:38.954675+0000","last_unstale":"2024-09-11T08:48:24.376126+0000","last_undegraded":"2024-09-11T08:48:24.376126+0000","last_fullsized":"2024-09-11T08:48:24.376126+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.130572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.130572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019779749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":233,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.365544+0000","last_change":"2024-09-11T08:48:16.376429+0000","last_active":"2024-09-11T08:48:24.365544+0000","last_peered":"2024-09-11T08:48:24.365544+0000","last_clean":"2024-09-11T08:48:24.365544+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:48:24.365544+0000","last_undegraded":"2024-09-11T08:48:24.365544+0000","last_fullsized":"2024-09-11T08:48:24.365544+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:48:16.376340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:16.376340+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:46.168511+0000","scrub_duration":0.00247796,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":191,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:25.682769+0000","last_change":"2024-09-11T08:48:25.682714+0000","last_active":"2024-09-11T08:48:25.682769+0000","last_peered":"2024-09-11T08:48:25.682769+0000","last_clean":"2024-09-11T08:48:25.682769+0000","last_became_active":"2024-09-11T08:38:42.975470+0000","last_became_peered":"2024-09-11T08:38:42.975470+0000","last_unstale":"2024-09-11T08:48:25.682769+0000","last_undegraded":"2024-09-11T08:48:25.682769+0000","last_fullsized":"2024-09-11T08:48:25.682769+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:25.682632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.682632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:55.181001+0000","scrub_duration":0.002088215,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":175,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.372806+0000","last_change":"2024-09-11T08:45:22.169436+0000","last_active":"2024-09-11T08:48:24.372806+0000","last_peered":"2024-09-11T08:48:24.372806+0000","last_clean":"2024-09-11T08:48:24.372806+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:48:24.372806+0000","last_undegraded":"2024-09-11T08:48:24.372806+0000","last_fullsized":"2024-09-11T08:48:24.372806+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.169367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.169367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005349693,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"199'14397","reported_seq":14455,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.746845+0000","last_change":"2024-09-11T08:45:15.261656+0000","last_active":"2024-09-11T08:48:36.746845+0000","last_peered":"2024-09-11T08:48:36.746845+0000","last_clean":"2024-09-11T08:48:36.746845+0000","last_became_active":"2024-09-11T08:45:15.261370+0000","last_became_peered":"2024-09-11T08:45:15.261370+0000","last_unstale":"2024-09-11T08:48:36.746845+0000","last_undegraded":"2024-09-11T08:48:36.746845+0000","last_fullsized":"2024-09-11T08:48:36.746845+0000","mapping_epoch":149,"log_start":"186'9500","ondisk_log_start":"186'9500","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4897,"log_dups_size":0,"ondisk_log_size":4897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38993920,"num_objects":595,"num_object_clones":0,"num_object_copies":1785,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14397,"num_write_kb":479744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":227,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.376781+0000","last_change":"2024-09-11T08:45:27.666590+0000","last_active":"2024-09-11T08:48:24.376781+0000","last_peered":"2024-09-11T08:48:24.376781+0000","last_clean":"2024-09-11T08:48:24.376781+0000","last_became_active":"2024-09-11T08:38:40.978467+0000","last_became_peered":"2024-09-11T08:38:40.978467+0000","last_unstale":"2024-09-11T08:48:24.376781+0000","last_undegraded":"2024-09-11T08:48:24.376781+0000","last_fullsized":"2024-09-11T08:48:24.376781+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:45:27.666502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.666502+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023582730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":212,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.374286+0000","last_change":"2024-09-11T08:46:01.982031+0000","last_active":"2024-09-11T08:48:24.374286+0000","last_peered":"2024-09-11T08:48:24.374286+0000","last_clean":"2024-09-11T08:48:24.374286+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:48:24.374286+0000","last_undegraded":"2024-09-11T08:48:24.374286+0000","last_fullsized":"2024-09-11T08:48:24.374286+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.775804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.775804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011899289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":221,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:28.359267+0000","last_change":"2024-09-11T08:48:28.359187+0000","last_active":"2024-09-11T08:48:28.359267+0000","last_peered":"2024-09-11T08:48:28.359267+0000","last_clean":"2024-09-11T08:48:28.359267+0000","last_became_active":"2024-09-11T08:38:38.954126+0000","last_became_peered":"2024-09-11T08:38:38.954126+0000","last_unstale":"2024-09-11T08:48:28.359267+0000","last_undegraded":"2024-09-11T08:48:28.359267+0000","last_fullsized":"2024-09-11T08:48:28.359267+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:48:28.359095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:28.359095+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:39.523205+0000","scrub_duration":0.0084904620000000007,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":207,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.379117+0000","last_change":"2024-09-11T08:47:44.344421+0000","last_active":"2024-09-11T08:48:24.379117+0000","last_peered":"2024-09-11T08:48:24.379117+0000","last_clean":"2024-09-11T08:48:24.379117+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:48:24.379117+0000","last_undegraded":"2024-09-11T08:48:24.379117+0000","last_fullsized":"2024-09-11T08:48:24.379117+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.344206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:48.790110+0000","scrub_duration":0.0085971020000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":205,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.368136+0000","last_change":"2024-09-11T08:46:01.984350+0000","last_active":"2024-09-11T08:48:24.368136+0000","last_peered":"2024-09-11T08:48:24.368136+0000","last_clean":"2024-09-11T08:48:24.368136+0000","last_became_active":"2024-09-11T08:46:01.983095+0000","last_became_peered":"2024-09-11T08:46:01.983095+0000","last_unstale":"2024-09-11T08:48:24.368136+0000","last_undegraded":"2024-09-11T08:48:24.368136+0000","last_fullsized":"2024-09-11T08:48:24.368136+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:12.278185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:12.278185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015184790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"199'14797","reported_seq":14910,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.465725+0000","last_change":"2024-09-11T08:48:29.580936+0000","last_active":"2024-09-11T08:48:36.465725+0000","last_peered":"2024-09-11T08:48:36.465725+0000","last_clean":"2024-09-11T08:48:36.465725+0000","last_became_active":"2024-09-11T08:45:15.627360+0000","last_became_peered":"2024-09-11T08:45:15.627360+0000","last_unstale":"2024-09-11T08:48:36.465725+0000","last_undegraded":"2024-09-11T08:48:36.465725+0000","last_fullsized":"2024-09-11T08:48:36.465725+0000","mapping_epoch":149,"log_start":"186'9800","ondisk_log_start":"186'9800","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'14266","last_scrub_stamp":"2024-09-11T08:48:29.580850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:48:29.580850+0000","objects_scrubbed":1086,"log_size":4997,"log_dups_size":0,"ondisk_log_size":4997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:58.547035+0000","scrub_duration":0.57800255499999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34930688,"num_objects":533,"num_object_clones":0,"num_object_copies":1599,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14797,"num_write_kb":490560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":198,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.384648+0000","last_change":"2024-09-11T08:44:15.699227+0000","last_active":"2024-09-11T08:48:24.384648+0000","last_peered":"2024-09-11T08:48:24.384648+0000","last_clean":"2024-09-11T08:48:24.384648+0000","last_became_active":"2024-09-11T08:38:40.975671+0000","last_became_peered":"2024-09-11T08:38:40.975671+0000","last_unstale":"2024-09-11T08:48:24.384648+0000","last_undegraded":"2024-09-11T08:48:24.384648+0000","last_fullsized":"2024-09-11T08:48:24.384648+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:44:15.699067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:15.699067+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014313306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":192,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.383892+0000","last_change":"2024-09-11T08:45:20.455183+0000","last_active":"2024-09-11T08:48:24.383892+0000","last_peered":"2024-09-11T08:48:24.383892+0000","last_clean":"2024-09-11T08:48:24.383892+0000","last_became_active":"2024-09-11T08:38:38.957027+0000","last_became_peered":"2024-09-11T08:38:38.957027+0000","last_unstale":"2024-09-11T08:48:24.383892+0000","last_undegraded":"2024-09-11T08:48:24.383892+0000","last_fullsized":"2024-09-11T08:48:24.383892+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:20.455081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.455081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016372450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":799,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:34.909229+0000","last_change":"2024-09-11T08:45:21.370742+0000","last_active":"2024-09-11T08:48:34.909229+0000","last_peered":"2024-09-11T08:48:34.909229+0000","last_clean":"2024-09-11T08:48:34.909229+0000","last_became_active":"2024-09-11T08:38:38.954692+0000","last_became_peered":"2024-09-11T08:38:38.954692+0000","last_unstale":"2024-09-11T08:48:34.909229+0000","last_undegraded":"2024-09-11T08:48:34.909229+0000","last_fullsized":"2024-09-11T08:48:34.909229+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:45:21.370673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.370673+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017307240000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":590,"num_read_kb":585,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":194,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:30.532833+0000","last_change":"2024-09-11T08:48:30.532776+0000","last_active":"2024-09-11T08:48:30.532833+0000","last_peered":"2024-09-11T08:48:30.532833+0000","last_clean":"2024-09-11T08:48:30.532833+0000","last_became_active":"2024-09-11T08:38:42.976214+0000","last_became_peered":"2024-09-11T08:38:42.976214+0000","last_unstale":"2024-09-11T08:48:30.532833+0000","last_undegraded":"2024-09-11T08:48:30.532833+0000","last_fullsized":"2024-09-11T08:48:30.532833+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:30.532701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.532701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:51.212642+0000","scrub_duration":0.012853409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":196,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:33.026017+0000","last_change":"2024-09-11T08:48:33.025967+0000","last_active":"2024-09-11T08:48:33.026017+0000","last_peered":"2024-09-11T08:48:33.026017+0000","last_clean":"2024-09-11T08:48:33.026017+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:48:33.026017+0000","last_undegraded":"2024-09-11T08:48:33.026017+0000","last_fullsized":"2024-09-11T08:48:33.026017+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:33.025897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:48:33.025897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:50.371310+0000","scrub_duration":0.007471208,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"199'14613","reported_seq":14716,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.140469+0000","last_change":"2024-09-11T08:48:33.132209+0000","last_active":"2024-09-11T08:48:36.140469+0000","last_peered":"2024-09-11T08:48:36.140469+0000","last_clean":"2024-09-11T08:48:36.140469+0000","last_became_active":"2024-09-11T08:45:15.262440+0000","last_became_peered":"2024-09-11T08:45:15.262440+0000","last_unstale":"2024-09-11T08:48:36.140469+0000","last_undegraded":"2024-09-11T08:48:36.140469+0000","last_fullsized":"2024-09-11T08:48:36.140469+0000","mapping_epoch":149,"log_start":"185'8300","ondisk_log_start":"185'8300","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'14375","last_scrub_stamp":"2024-09-11T08:48:33.132064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:48:33.132064+0000","objects_scrubbed":862,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:02.404529+0000","scrub_duration":0.27961396199999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40435712,"num_objects":617,"num_object_clones":0,"num_object_copies":1851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14613,"num_write_kb":487360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":67,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.370579+0000","last_change":"2024-09-11T08:47:45.585212+0000","last_active":"2024-09-11T08:48:24.370579+0000","last_peered":"2024-09-11T08:48:24.370579+0000","last_clean":"2024-09-11T08:48:24.370579+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:48:24.370579+0000","last_undegraded":"2024-09-11T08:48:24.370579+0000","last_fullsized":"2024-09-11T08:48:24.370579+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-11T08:47:45.585135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:45.585135+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:47.607612+0000","scrub_duration":0.0080208199999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":190,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.373448+0000","last_change":"2024-09-11T08:45:18.127946+0000","last_active":"2024-09-11T08:48:24.373448+0000","last_peered":"2024-09-11T08:48:24.373448+0000","last_clean":"2024-09-11T08:48:24.373448+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:48:24.373448+0000","last_undegraded":"2024-09-11T08:48:24.373448+0000","last_fullsized":"2024-09-11T08:48:24.373448+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:18.127857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.127857+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018334099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":235,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.368877+0000","last_change":"2024-09-11T08:45:29.524681+0000","last_active":"2024-09-11T08:48:24.368877+0000","last_peered":"2024-09-11T08:48:24.368877+0000","last_clean":"2024-09-11T08:48:24.368877+0000","last_became_active":"2024-09-11T08:38:24.663448+0000","last_became_peered":"2024-09-11T08:38:24.663448+0000","last_unstale":"2024-09-11T08:48:24.368877+0000","last_undegraded":"2024-09-11T08:48:24.368877+0000","last_fullsized":"2024-09-11T08:48:24.368877+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'79","last_scrub_stamp":"2024-09-11T08:45:29.524608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:36:29.974954+0000","last_clean_scrub_stamp":"2024-09-11T08:45:29.524608+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002308867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":197,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.367963+0000","last_change":"2024-09-11T08:47:35.433723+0000","last_active":"2024-09-11T08:48:24.367963+0000","last_peered":"2024-09-11T08:48:24.367963+0000","last_clean":"2024-09-11T08:48:24.367963+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:48:24.367963+0000","last_undegraded":"2024-09-11T08:48:24.367963+0000","last_fullsized":"2024-09-11T08:48:24.367963+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:35.433651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:35.433651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:37.450817+0000","scrub_duration":0.013907892999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":192,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.365955+0000","last_change":"2024-09-11T08:48:19.308968+0000","last_active":"2024-09-11T08:48:24.365955+0000","last_peered":"2024-09-11T08:48:24.365955+0000","last_clean":"2024-09-11T08:48:24.365955+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:48:24.365955+0000","last_undegraded":"2024-09-11T08:48:24.365955+0000","last_fullsized":"2024-09-11T08:48:24.365955+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:19.308869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.308869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:27.449816+0000","scrub_duration":0.011385526999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"199'14525","reported_seq":14583,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.118829+0000","last_change":"2024-09-11T08:45:15.263087+0000","last_active":"2024-09-11T08:48:36.118829+0000","last_peered":"2024-09-11T08:48:36.118829+0000","last_clean":"2024-09-11T08:48:36.118829+0000","last_became_active":"2024-09-11T08:45:15.262698+0000","last_became_peered":"2024-09-11T08:45:15.262698+0000","last_unstale":"2024-09-11T08:48:36.118829+0000","last_undegraded":"2024-09-11T08:48:36.118829+0000","last_fullsized":"2024-09-11T08:48:36.118829+0000","mapping_epoch":149,"log_start":"186'9700","ondisk_log_start":"186'9700","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4825,"log_dups_size":0,"ondisk_log_size":4825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43188224,"num_objects":659,"num_object_clones":0,"num_object_copies":1977,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14525,"num_write_kb":485888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":268,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375670+0000","last_change":"2024-09-11T08:48:20.860266+0000","last_active":"2024-09-11T08:48:24.375670+0000","last_peered":"2024-09-11T08:48:24.375670+0000","last_clean":"2024-09-11T08:48:24.375670+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:48:24.375670+0000","last_undegraded":"2024-09-11T08:48:24.375670+0000","last_fullsized":"2024-09-11T08:48:24.375670+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:20.860186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.860186+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:43.681747+0000","scrub_duration":0.0045417560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":172,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375169+0000","last_change":"2024-09-11T08:44:22.010602+0000","last_active":"2024-09-11T08:48:24.375169+0000","last_peered":"2024-09-11T08:48:24.375169+0000","last_clean":"2024-09-11T08:48:24.375169+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:48:24.375169+0000","last_undegraded":"2024-09-11T08:48:24.375169+0000","last_fullsized":"2024-09-11T08:48:24.375169+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.010528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.010528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047628439999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":191,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.365836+0000","last_change":"2024-09-11T08:47:41.425451+0000","last_active":"2024-09-11T08:48:24.365836+0000","last_peered":"2024-09-11T08:48:24.365836+0000","last_clean":"2024-09-11T08:48:24.365836+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:48:24.365836+0000","last_undegraded":"2024-09-11T08:48:24.365836+0000","last_fullsized":"2024-09-11T08:48:24.365836+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.726573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00117395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":195,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.372364+0000","last_change":"2024-09-11T08:46:01.988315+0000","last_active":"2024-09-11T08:48:24.372364+0000","last_peered":"2024-09-11T08:48:24.372364+0000","last_clean":"2024-09-11T08:48:24.372364+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:48:24.372364+0000","last_undegraded":"2024-09-11T08:48:24.372364+0000","last_fullsized":"2024-09-11T08:48:24.372364+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.167071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.167071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001796483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"199'14195","reported_seq":14253,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.467119+0000","last_change":"2024-09-11T08:45:15.626015+0000","last_active":"2024-09-11T08:48:36.467119+0000","last_peered":"2024-09-11T08:48:36.467119+0000","last_clean":"2024-09-11T08:48:36.467119+0000","last_became_active":"2024-09-11T08:45:15.625726+0000","last_became_peered":"2024-09-11T08:45:15.625726+0000","last_unstale":"2024-09-11T08:48:36.467119+0000","last_undegraded":"2024-09-11T08:48:36.467119+0000","last_fullsized":"2024-09-11T08:48:36.467119+0000","mapping_epoch":149,"log_start":"186'9200","ondisk_log_start":"186'9200","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4995,"log_dups_size":0,"ondisk_log_size":4995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38076416,"num_objects":581,"num_object_clones":0,"num_object_copies":1743,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14195,"num_write_kb":472832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":208,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375649+0000","last_change":"2024-09-11T08:44:28.940078+0000","last_active":"2024-09-11T08:48:24.375649+0000","last_peered":"2024-09-11T08:48:24.375649+0000","last_clean":"2024-09-11T08:48:24.375649+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:48:24.375649+0000","last_undegraded":"2024-09-11T08:48:24.375649+0000","last_fullsized":"2024-09-11T08:48:24.375649+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:44:28.940023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:28.940023+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012856796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":211,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:28.828750+0000","last_change":"2024-09-11T08:48:28.828694+0000","last_active":"2024-09-11T08:48:28.828750+0000","last_peered":"2024-09-11T08:48:28.828750+0000","last_clean":"2024-09-11T08:48:28.828750+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:48:28.828750+0000","last_undegraded":"2024-09-11T08:48:28.828750+0000","last_fullsized":"2024-09-11T08:48:28.828750+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:28.828618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_clean_scrub_stamp":"2024-09-11T08:48:28.828618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:32.802521+0000","scrub_duration":0.0029516289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":198,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.374556+0000","last_change":"2024-09-11T08:48:22.989252+0000","last_active":"2024-09-11T08:48:24.374556+0000","last_peered":"2024-09-11T08:48:24.374556+0000","last_clean":"2024-09-11T08:48:24.374556+0000","last_became_active":"2024-09-11T08:38:42.974803+0000","last_became_peered":"2024-09-11T08:38:42.974803+0000","last_unstale":"2024-09-11T08:48:24.374556+0000","last_undegraded":"2024-09-11T08:48:24.374556+0000","last_fullsized":"2024-09-11T08:48:24.374556+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:22.989179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:22.989179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:47.279934+0000","scrub_duration":0.0033708700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":211,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.379164+0000","last_change":"2024-09-11T08:48:17.333093+0000","last_active":"2024-09-11T08:48:24.379164+0000","last_peered":"2024-09-11T08:48:24.379164+0000","last_clean":"2024-09-11T08:48:24.379164+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:48:24.379164+0000","last_undegraded":"2024-09-11T08:48:24.379164+0000","last_fullsized":"2024-09-11T08:48:24.379164+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:17.333005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:17.333005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:21.947510+0000","scrub_duration":0.0086773309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"199'14470","reported_seq":14528,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.739057+0000","last_change":"2024-09-11T08:45:15.260658+0000","last_active":"2024-09-11T08:48:36.739057+0000","last_peered":"2024-09-11T08:48:36.739057+0000","last_clean":"2024-09-11T08:48:36.739057+0000","last_became_active":"2024-09-11T08:45:15.260383+0000","last_became_peered":"2024-09-11T08:45:15.260383+0000","last_unstale":"2024-09-11T08:48:36.739057+0000","last_undegraded":"2024-09-11T08:48:36.739057+0000","last_fullsized":"2024-09-11T08:48:36.739057+0000","mapping_epoch":149,"log_start":"186'9600","ondisk_log_start":"186'9600","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4870,"log_dups_size":0,"ondisk_log_size":4870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35389440,"num_objects":540,"num_object_clones":0,"num_object_copies":1620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14470,"num_write_kb":480320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":242,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375159+0000","last_change":"2024-09-11T08:48:21.885766+0000","last_active":"2024-09-11T08:48:24.375159+0000","last_peered":"2024-09-11T08:48:24.375159+0000","last_clean":"2024-09-11T08:48:24.375159+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:48:24.375159+0000","last_undegraded":"2024-09-11T08:48:24.375159+0000","last_fullsized":"2024-09-11T08:48:24.375159+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:48:21.885681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:21.885681+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:24.781430+0000","scrub_duration":0.0051165580000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":181,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.372494+0000","last_change":"2024-09-11T08:45:32.219463+0000","last_active":"2024-09-11T08:48:24.372494+0000","last_peered":"2024-09-11T08:48:24.372494+0000","last_clean":"2024-09-11T08:48:24.372494+0000","last_became_active":"2024-09-11T08:38:38.956935+0000","last_became_peered":"2024-09-11T08:38:38.956935+0000","last_unstale":"2024-09-11T08:48:24.372494+0000","last_undegraded":"2024-09-11T08:48:24.372494+0000","last_fullsized":"2024-09-11T08:48:24.372494+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:32.219374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:32.219374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069622119999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":334,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:32.791745+0000","last_change":"2024-09-11T08:47:47.542558+0000","last_active":"2024-09-11T08:48:32.791745+0000","last_peered":"2024-09-11T08:48:32.791745+0000","last_clean":"2024-09-11T08:48:32.791745+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:48:32.791745+0000","last_undegraded":"2024-09-11T08:48:32.791745+0000","last_fullsized":"2024-09-11T08:48:32.791745+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:47:47.542483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:47.542483+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:17.503546+0000","scrub_duration":0.15193762999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":176,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375279+0000","last_change":"2024-09-11T08:45:31.196687+0000","last_active":"2024-09-11T08:48:24.375279+0000","last_peered":"2024-09-11T08:48:24.375279+0000","last_clean":"2024-09-11T08:48:24.375279+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:48:24.375279+0000","last_undegraded":"2024-09-11T08:48:24.375279+0000","last_fullsized":"2024-09-11T08:48:24.375279+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.196599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.196599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025248190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"199'14568","reported_seq":22506,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.492199+0000","last_change":"2024-09-11T08:47:45.439155+0000","last_active":"2024-09-11T08:48:36.492199+0000","last_peered":"2024-09-11T08:48:36.492199+0000","last_clean":"2024-09-11T08:48:36.492199+0000","last_became_active":"2024-09-11T08:47:45.422820+0000","last_became_peered":"2024-09-11T08:47:45.422820+0000","last_unstale":"2024-09-11T08:48:36.492199+0000","last_undegraded":"2024-09-11T08:48:36.492199+0000","last_fullsized":"2024-09-11T08:48:36.492199+0000","mapping_epoch":189,"log_start":"186'9600","ondisk_log_start":"186'9600","created":149,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":4968,"log_dups_size":0,"ondisk_log_size":4968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42467328,"num_objects":648,"num_object_clones":0,"num_object_copies":1944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14568,"num_write_kb":486912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3929,"num_bytes_recovered":257490944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":196,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.382933+0000","last_change":"2024-09-11T08:44:31.662275+0000","last_active":"2024-09-11T08:48:24.382933+0000","last_peered":"2024-09-11T08:48:24.382933+0000","last_clean":"2024-09-11T08:48:24.382933+0000","last_became_active":"2024-09-11T08:38:40.976861+0000","last_became_peered":"2024-09-11T08:38:40.976861+0000","last_unstale":"2024-09-11T08:48:24.382933+0000","last_undegraded":"2024-09-11T08:48:24.382933+0000","last_fullsized":"2024-09-11T08:48:24.382933+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:31.662190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:31.662190+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040714619999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":195,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.366267+0000","last_change":"2024-09-11T08:47:43.360558+0000","last_active":"2024-09-11T08:48:24.366267+0000","last_peered":"2024-09-11T08:48:24.366267+0000","last_clean":"2024-09-11T08:48:24.366267+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:48:24.366267+0000","last_undegraded":"2024-09-11T08:48:24.366267+0000","last_fullsized":"2024-09-11T08:48:24.366267+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:10.607288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:45:10.607288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016158909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":201,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.380728+0000","last_change":"2024-09-11T08:48:24.380508+0000","last_active":"2024-09-11T08:48:24.380728+0000","last_peered":"2024-09-11T08:48:24.380728+0000","last_clean":"2024-09-11T08:48:24.380728+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:48:24.380728+0000","last_undegraded":"2024-09-11T08:48:24.380728+0000","last_fullsized":"2024-09-11T08:48:24.380728+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:24.380424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:24.380424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:39.122714+0000","scrub_duration":0.018256864000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":186,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.384039+0000","last_change":"2024-09-11T08:47:47.699529+0000","last_active":"2024-09-11T08:48:24.384039+0000","last_peered":"2024-09-11T08:48:24.384039+0000","last_clean":"2024-09-11T08:48:24.384039+0000","last_became_active":"2024-09-11T08:38:44.991139+0000","last_became_peered":"2024-09-11T08:38:44.991139+0000","last_unstale":"2024-09-11T08:48:24.384039+0000","last_undegraded":"2024-09-11T08:48:24.384039+0000","last_fullsized":"2024-09-11T08:48:24.384039+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:47.699453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:47:47.699453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:55.018390+0000","scrub_duration":0.016178818000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"199'14093","reported_seq":14151,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.785113+0000","last_change":"2024-09-11T08:45:15.297737+0000","last_active":"2024-09-11T08:48:36.785113+0000","last_peered":"2024-09-11T08:48:36.785113+0000","last_clean":"2024-09-11T08:48:36.785113+0000","last_became_active":"2024-09-11T08:45:15.297420+0000","last_became_peered":"2024-09-11T08:45:15.297420+0000","last_unstale":"2024-09-11T08:48:36.785113+0000","last_undegraded":"2024-09-11T08:48:36.785113+0000","last_fullsized":"2024-09-11T08:48:36.785113+0000","mapping_epoch":149,"log_start":"185'8400","ondisk_log_start":"185'8400","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.240526+0000","objects_scrubbed":0,"log_size":5693,"log_dups_size":0,"ondisk_log_size":5693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36634624,"num_objects":559,"num_object_clones":0,"num_object_copies":1677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14093,"num_write_kb":468864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":71,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.369829+0000","last_change":"2024-09-11T08:47:46.582476+0000","last_active":"2024-09-11T08:48:24.369829+0000","last_peered":"2024-09-11T08:48:24.369829+0000","last_clean":"2024-09-11T08:48:24.369829+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:48:24.369829+0000","last_undegraded":"2024-09-11T08:48:24.369829+0000","last_fullsized":"2024-09-11T08:48:24.369829+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:47:46.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:46.582408+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:56.458790+0000","scrub_duration":0.006189015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":201,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.368417+0000","last_change":"2024-09-11T08:47:44.411099+0000","last_active":"2024-09-11T08:48:24.368417+0000","last_peered":"2024-09-11T08:48:24.368417+0000","last_clean":"2024-09-11T08:48:24.368417+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:48:24.368417+0000","last_undegraded":"2024-09-11T08:48:24.368417+0000","last_fullsized":"2024-09-11T08:48:24.368417+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.411022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.411022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:47.936745+0000","scrub_duration":0.0047762170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":207,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.378150+0000","last_change":"2024-09-11T08:47:43.361569+0000","last_active":"2024-09-11T08:48:24.378150+0000","last_peered":"2024-09-11T08:48:24.378150+0000","last_clean":"2024-09-11T08:48:24.378150+0000","last_became_active":"2024-09-11T08:38:42.985642+0000","last_became_peered":"2024-09-11T08:38:42.985642+0000","last_unstale":"2024-09-11T08:48:24.378150+0000","last_undegraded":"2024-09-11T08:48:24.378150+0000","last_fullsized":"2024-09-11T08:48:24.378150+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:43.361491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:43.361491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:11.065199+0000","scrub_duration":0.0055115299999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":188,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:33.065214+0000","last_change":"2024-09-11T08:48:33.065142+0000","last_active":"2024-09-11T08:48:33.065214+0000","last_peered":"2024-09-11T08:48:33.065214+0000","last_clean":"2024-09-11T08:48:33.065214+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:48:33.065214+0000","last_undegraded":"2024-09-11T08:48:33.065214+0000","last_fullsized":"2024-09-11T08:48:33.065214+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:33.065053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:33.065053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:54.909406+0000","scrub_duration":0.013998086999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"199'14136","reported_seq":21980,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.101257+0000","last_change":"2024-09-11T08:48:19.326000+0000","last_active":"2024-09-11T08:48:36.101257+0000","last_peered":"2024-09-11T08:48:36.101257+0000","last_clean":"2024-09-11T08:48:36.101257+0000","last_became_active":"2024-09-11T08:47:33.341629+0000","last_became_peered":"2024-09-11T08:47:33.341629+0000","last_unstale":"2024-09-11T08:48:36.101257+0000","last_undegraded":"2024-09-11T08:48:36.101257+0000","last_fullsized":"2024-09-11T08:48:36.101257+0000","mapping_epoch":187,"log_start":"184'7800","ondisk_log_start":"184'7800","created":149,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"197'12871","last_scrub_stamp":"2024-09-11T08:48:19.325895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.325895+0000","objects_scrubbed":1903,"log_size":6336,"log_dups_size":0,"ondisk_log_size":6336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:30.506000+0000","scrub_duration":0.50748997100000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40763392,"num_objects":622,"num_object_clones":0,"num_object_copies":1866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14136,"num_write_kb":472256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3840,"num_bytes_recovered":251658240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":203,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375046+0000","last_change":"2024-09-11T08:45:08.140910+0000","last_active":"2024-09-11T08:48:24.375046+0000","last_peered":"2024-09-11T08:48:24.375046+0000","last_clean":"2024-09-11T08:48:24.375046+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:48:24.375046+0000","last_undegraded":"2024-09-11T08:48:24.375046+0000","last_fullsized":"2024-09-11T08:48:24.375046+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:08.140839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.140839+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003679547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":191,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.368206+0000","last_change":"2024-09-11T08:45:14.322786+0000","last_active":"2024-09-11T08:48:24.368206+0000","last_peered":"2024-09-11T08:48:24.368206+0000","last_clean":"2024-09-11T08:48:24.368206+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:48:24.368206+0000","last_undegraded":"2024-09-11T08:48:24.368206+0000","last_fullsized":"2024-09-11T08:48:24.368206+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.322217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.322217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014821319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":210,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.377577+0000","last_change":"2024-09-11T08:48:21.336871+0000","last_active":"2024-09-11T08:48:24.377577+0000","last_peered":"2024-09-11T08:48:24.377577+0000","last_clean":"2024-09-11T08:48:24.377577+0000","last_became_active":"2024-09-11T08:38:42.981054+0000","last_became_peered":"2024-09-11T08:38:42.981054+0000","last_unstale":"2024-09-11T08:48:24.377577+0000","last_undegraded":"2024-09-11T08:48:24.377577+0000","last_fullsized":"2024-09-11T08:48:24.377577+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:21.336766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:21.336766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:36.105077+0000","scrub_duration":0.0077730009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":175,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.372090+0000","last_change":"2024-09-11T08:45:33.228265+0000","last_active":"2024-09-11T08:48:24.372090+0000","last_peered":"2024-09-11T08:48:24.372090+0000","last_clean":"2024-09-11T08:48:24.372090+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:48:24.372090+0000","last_undegraded":"2024-09-11T08:48:24.372090+0000","last_fullsized":"2024-09-11T08:48:24.372090+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:45:33.228082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064426500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"199'14476","reported_seq":14725,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T08:48:36.261288+0000","last_change":"2024-09-11T08:47:45.030515+0000","last_active":"2024-09-11T08:48:36.261288+0000","last_peered":"2024-09-11T08:48:36.261288+0000","last_clean":"2024-09-11T08:48:36.261288+0000","last_became_active":"2024-09-11T08:45:15.625778+0000","last_became_peered":"2024-09-11T08:45:15.625778+0000","last_unstale":"2024-09-11T08:48:36.261288+0000","last_undegraded":"2024-09-11T08:48:36.261288+0000","last_fullsized":"2024-09-11T08:48:36.261288+0000","mapping_epoch":149,"log_start":"185'9100","ondisk_log_start":"185'9100","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'10571","last_scrub_stamp":"2024-09-11T08:47:45.030446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:14.240526+0000","last_clean_scrub_stamp":"2024-09-11T08:47:45.030446+0000","objects_scrubbed":4541,"log_size":5376,"log_dups_size":0,"ondisk_log_size":5376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:56.591715+0000","scrub_duration":0.68909561500000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41418752,"num_objects":632,"num_object_clones":0,"num_object_copies":1896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14476,"num_write_kb":483456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"124'22","reported_seq":280,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.369238+0000","last_change":"2024-09-11T08:48:20.352021+0000","last_active":"2024-09-11T08:48:24.369238+0000","last_peered":"2024-09-11T08:48:24.369238+0000","last_clean":"2024-09-11T08:48:24.369238+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:48:24.369238+0000","last_undegraded":"2024-09-11T08:48:24.369238+0000","last_fullsized":"2024-09-11T08:48:24.369238+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:48:20.351963+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.351963+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:26.736295+0000","scrub_duration":0.0058139180000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":208,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.368020+0000","last_change":"2024-09-11T08:45:20.372571+0000","last_active":"2024-09-11T08:48:24.368020+0000","last_peered":"2024-09-11T08:48:24.368020+0000","last_clean":"2024-09-11T08:48:24.368020+0000","last_became_active":"2024-09-11T08:38:38.955778+0000","last_became_peered":"2024-09-11T08:38:38.955778+0000","last_unstale":"2024-09-11T08:48:24.368020+0000","last_undegraded":"2024-09-11T08:48:24.368020+0000","last_fullsized":"2024-09-11T08:48:24.368020+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:45:20.372454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.372454+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019547269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"168'10","reported_seq":342,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:32.791372+0000","last_change":"2024-09-11T08:46:01.988356+0000","last_active":"2024-09-11T08:48:32.791372+0000","last_peered":"2024-09-11T08:48:32.791372+0000","last_clean":"2024-09-11T08:48:32.791372+0000","last_became_active":"2024-09-11T08:46:01.983055+0000","last_became_peered":"2024-09-11T08:46:01.983055+0000","last_unstale":"2024-09-11T08:48:32.791372+0000","last_undegraded":"2024-09-11T08:48:32.791372+0000","last_fullsized":"2024-09-11T08:48:32.791372+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"135'9","last_scrub_stamp":"2024-09-11T08:45:11.298652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.298652+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001669032,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":182,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.365359+0000","last_change":"2024-09-11T08:47:34.554720+0000","last_active":"2024-09-11T08:48:24.365359+0000","last_peered":"2024-09-11T08:48:24.365359+0000","last_clean":"2024-09-11T08:48:24.365359+0000","last_became_active":"2024-09-11T08:38:45.182711+0000","last_became_peered":"2024-09-11T08:38:45.182711+0000","last_unstale":"2024-09-11T08:48:24.365359+0000","last_undegraded":"2024-09-11T08:48:24.365359+0000","last_fullsized":"2024-09-11T08:48:24.365359+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:15.888394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:15.888394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003201476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":211,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.369707+0000","last_change":"2024-09-11T08:48:23.796971+0000","last_active":"2024-09-11T08:48:24.369707+0000","last_peered":"2024-09-11T08:48:24.369707+0000","last_clean":"2024-09-11T08:48:24.369707+0000","last_became_active":"2024-09-11T08:38:38.956653+0000","last_became_peered":"2024-09-11T08:38:38.956653+0000","last_unstale":"2024-09-11T08:48:24.369707+0000","last_undegraded":"2024-09-11T08:48:24.369707+0000","last_fullsized":"2024-09-11T08:48:24.369707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:23.796894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:48:23.796894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:36.905113+0000","scrub_duration":0.0042577149999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":206,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.374964+0000","last_change":"2024-09-11T08:44:35.390285+0000","last_active":"2024-09-11T08:48:24.374964+0000","last_peered":"2024-09-11T08:48:24.374964+0000","last_clean":"2024-09-11T08:48:24.374964+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:48:24.374964+0000","last_undegraded":"2024-09-11T08:48:24.374964+0000","last_fullsized":"2024-09-11T08:48:24.374964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-11T08:44:35.390134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:35.390134+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017427033000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":188,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.365522+0000","last_change":"2024-09-11T08:45:08.675251+0000","last_active":"2024-09-11T08:48:24.365522+0000","last_peered":"2024-09-11T08:48:24.365522+0000","last_clean":"2024-09-11T08:48:24.365522+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:48:24.365522+0000","last_undegraded":"2024-09-11T08:48:24.365522+0000","last_fullsized":"2024-09-11T08:48:24.365522+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:08.675179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.675179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003515392,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":205,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.368098+0000","last_change":"2024-09-11T08:46:01.983864+0000","last_active":"2024-09-11T08:48:24.368098+0000","last_peered":"2024-09-11T08:48:24.368098+0000","last_clean":"2024-09-11T08:48:24.368098+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:48:24.368098+0000","last_undegraded":"2024-09-11T08:48:24.368098+0000","last_fullsized":"2024-09-11T08:48:24.368098+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:25.344348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.344348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013849730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"63'9","reported_seq":206,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.384723+0000","last_change":"2024-09-11T08:47:44.737776+0000","last_active":"2024-09-11T08:48:24.384723+0000","last_peered":"2024-09-11T08:48:24.384723+0000","last_clean":"2024-09-11T08:48:24.384723+0000","last_became_active":"2024-09-11T08:38:41.041191+0000","last_became_peered":"2024-09-11T08:38:41.041191+0000","last_unstale":"2024-09-11T08:48:24.384723+0000","last_undegraded":"2024-09-11T08:48:24.384723+0000","last_fullsized":"2024-09-11T08:48:24.384723+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:47:44.737721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.737721+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:04.218319+0000","scrub_duration":0.0032009640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":204,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.367541+0000","last_change":"2024-09-11T08:48:20.977749+0000","last_active":"2024-09-11T08:48:24.367541+0000","last_peered":"2024-09-11T08:48:24.367541+0000","last_clean":"2024-09-11T08:48:24.367541+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:48:24.367541+0000","last_undegraded":"2024-09-11T08:48:24.367541+0000","last_fullsized":"2024-09-11T08:48:24.367541+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:20.977665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.977665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:45.220147+0000","scrub_duration":0.0075643139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":177,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.374839+0000","last_change":"2024-09-11T08:45:23.213642+0000","last_active":"2024-09-11T08:48:24.374839+0000","last_peered":"2024-09-11T08:48:24.374839+0000","last_clean":"2024-09-11T08:48:24.374839+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:48:24.374839+0000","last_undegraded":"2024-09-11T08:48:24.374839+0000","last_fullsized":"2024-09-11T08:48:24.374839+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.213592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.213592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001600473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":196,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.378029+0000","last_change":"2024-09-11T08:47:34.423379+0000","last_active":"2024-09-11T08:48:24.378029+0000","last_peered":"2024-09-11T08:48:24.378029+0000","last_clean":"2024-09-11T08:48:24.378029+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:48:24.378029+0000","last_undegraded":"2024-09-11T08:48:24.378029+0000","last_fullsized":"2024-09-11T08:48:24.378029+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_clean_scrub_stamp":"2024-09-11T08:47:34.420932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:45.412817+0000","scrub_duration":0.01242356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":202,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.366123+0000","last_change":"2024-09-11T08:47:29.687278+0000","last_active":"2024-09-11T08:48:24.366123+0000","last_peered":"2024-09-11T08:48:24.366123+0000","last_clean":"2024-09-11T08:48:24.366123+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:48:24.366123+0000","last_undegraded":"2024-09-11T08:48:24.366123+0000","last_fullsized":"2024-09-11T08:48:24.366123+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:44:18.928481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:44:18.928481+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047130310000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":192,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.385065+0000","last_change":"2024-09-11T08:47:45.751605+0000","last_active":"2024-09-11T08:48:24.385065+0000","last_peered":"2024-09-11T08:48:24.385065+0000","last_clean":"2024-09-11T08:48:24.385065+0000","last_became_active":"2024-09-11T08:38:38.957761+0000","last_became_peered":"2024-09-11T08:38:38.957761+0000","last_unstale":"2024-09-11T08:48:24.385065+0000","last_undegraded":"2024-09-11T08:48:24.385065+0000","last_fullsized":"2024-09-11T08:48:24.385065+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:45.751518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:45.751518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:47.231246+0000","scrub_duration":0.0061587329999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":304,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:32.792710+0000","last_change":"2024-09-11T08:45:20.119727+0000","last_active":"2024-09-11T08:48:32.792710+0000","last_peered":"2024-09-11T08:48:32.792710+0000","last_clean":"2024-09-11T08:48:32.792710+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:48:32.792710+0000","last_undegraded":"2024-09-11T08:48:32.792710+0000","last_fullsized":"2024-09-11T08:48:32.792710+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:20.119626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.119626+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018087660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":166,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375282+0000","last_change":"2024-09-11T08:44:20.017341+0000","last_active":"2024-09-11T08:48:24.375282+0000","last_peered":"2024-09-11T08:48:24.375282+0000","last_clean":"2024-09-11T08:48:24.375282+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:48:24.375282+0000","last_undegraded":"2024-09-11T08:48:24.375282+0000","last_fullsized":"2024-09-11T08:48:24.375282+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:20.017261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:20.017261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001558369,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"152'15","reported_seq":199,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.373130+0000","last_change":"2024-09-11T08:45:21.132138+0000","last_active":"2024-09-11T08:48:24.373130+0000","last_peered":"2024-09-11T08:48:24.373130+0000","last_clean":"2024-09-11T08:48:24.373130+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:48:24.373130+0000","last_undegraded":"2024-09-11T08:48:24.373130+0000","last_fullsized":"2024-09-11T08:48:24.373130+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'15","last_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.132083+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003984156,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":191,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.367836+0000","last_change":"2024-09-11T08:45:27.436673+0000","last_active":"2024-09-11T08:48:24.367836+0000","last_peered":"2024-09-11T08:48:24.367836+0000","last_clean":"2024-09-11T08:48:24.367836+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:48:24.367836+0000","last_undegraded":"2024-09-11T08:48:24.367836+0000","last_fullsized":"2024-09-11T08:48:24.367836+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.436552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001660176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":315,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:32.791578+0000","last_change":"2024-09-11T08:45:28.692173+0000","last_active":"2024-09-11T08:48:32.791578+0000","last_peered":"2024-09-11T08:48:32.791578+0000","last_clean":"2024-09-11T08:48:32.791578+0000","last_became_active":"2024-09-11T08:38:42.975613+0000","last_became_peered":"2024-09-11T08:38:42.975613+0000","last_unstale":"2024-09-11T08:48:32.791578+0000","last_undegraded":"2024-09-11T08:48:32.791578+0000","last_fullsized":"2024-09-11T08:48:32.791578+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:28.692100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.692100+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019579020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":186,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.384305+0000","last_change":"2024-09-11T08:45:14.581287+0000","last_active":"2024-09-11T08:48:24.384305+0000","last_peered":"2024-09-11T08:48:24.384305+0000","last_clean":"2024-09-11T08:48:24.384305+0000","last_became_active":"2024-09-11T08:38:44.997402+0000","last_became_peered":"2024-09-11T08:38:44.997402+0000","last_unstale":"2024-09-11T08:48:24.384305+0000","last_undegraded":"2024-09-11T08:48:24.384305+0000","last_fullsized":"2024-09-11T08:48:24.384305+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.581231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.581231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001580932,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":199,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:25.793069+0000","last_change":"2024-09-11T08:48:25.792916+0000","last_active":"2024-09-11T08:48:25.793069+0000","last_peered":"2024-09-11T08:48:25.793069+0000","last_clean":"2024-09-11T08:48:25.793069+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:48:25.793069+0000","last_undegraded":"2024-09-11T08:48:25.793069+0000","last_fullsized":"2024-09-11T08:48:25.793069+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:25.792777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.792777+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:26.832474+0000","scrub_duration":0.0045475469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":182,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.375248+0000","last_change":"2024-09-11T08:45:27.198021+0000","last_active":"2024-09-11T08:48:24.375248+0000","last_peered":"2024-09-11T08:48:24.375248+0000","last_clean":"2024-09-11T08:48:24.375248+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:48:24.375248+0000","last_undegraded":"2024-09-11T08:48:24.375248+0000","last_fullsized":"2024-09-11T08:48:24.375248+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.197944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.197944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015230210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":198,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:30.985772+0000","last_change":"2024-09-11T08:48:30.985518+0000","last_active":"2024-09-11T08:48:30.985772+0000","last_peered":"2024-09-11T08:48:30.985772+0000","last_clean":"2024-09-11T08:48:30.985772+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:48:30.985772+0000","last_undegraded":"2024-09-11T08:48:30.985772+0000","last_fullsized":"2024-09-11T08:48:30.985772+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:30.985271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.985271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:50.920428+0000","scrub_duration":0.0099058919999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":175,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.372049+0000","last_change":"2024-09-11T08:45:28.259156+0000","last_active":"2024-09-11T08:48:24.372049+0000","last_peered":"2024-09-11T08:48:24.372049+0000","last_clean":"2024-09-11T08:48:24.372049+0000","last_became_active":"2024-09-11T08:38:44.993938+0000","last_became_peered":"2024-09-11T08:38:44.993938+0000","last_unstale":"2024-09-11T08:48:24.372049+0000","last_undegraded":"2024-09-11T08:48:24.372049+0000","last_fullsized":"2024-09-11T08:48:24.372049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.259103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.259103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015518800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":219,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.365212+0000","last_change":"2024-09-11T08:46:01.993547+0000","last_active":"2024-09-11T08:48:24.365212+0000","last_peered":"2024-09-11T08:48:24.365212+0000","last_clean":"2024-09-11T08:48:24.365212+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:48:24.365212+0000","last_undegraded":"2024-09-11T08:48:24.365212+0000","last_fullsized":"2024-09-11T08:48:24.365212+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:45:06.657912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.657912+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023744109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":201,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.368344+0000","last_change":"2024-09-11T08:47:48.373407+0000","last_active":"2024-09-11T08:48:24.368344+0000","last_peered":"2024-09-11T08:48:24.368344+0000","last_clean":"2024-09-11T08:48:24.368344+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:48:24.368344+0000","last_undegraded":"2024-09-11T08:48:24.368344+0000","last_fullsized":"2024-09-11T08:48:24.368344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:48.373336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:48.373336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:54.062621+0000","scrub_duration":0.001705364,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":177,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.372275+0000","last_change":"2024-09-11T08:45:14.165632+0000","last_active":"2024-09-11T08:48:24.372275+0000","last_peered":"2024-09-11T08:48:24.372275+0000","last_clean":"2024-09-11T08:48:24.372275+0000","last_became_active":"2024-09-11T08:38:42.983638+0000","last_became_peered":"2024-09-11T08:38:42.983638+0000","last_unstale":"2024-09-11T08:48:24.372275+0000","last_undegraded":"2024-09-11T08:48:24.372275+0000","last_fullsized":"2024-09-11T08:48:24.372275+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.165576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.165576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012147639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":189,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.381889+0000","last_change":"2024-09-11T08:48:18.685268+0000","last_active":"2024-09-11T08:48:24.381889+0000","last_peered":"2024-09-11T08:48:24.381889+0000","last_clean":"2024-09-11T08:48:24.381889+0000","last_became_active":"2024-09-11T08:38:44.999329+0000","last_became_peered":"2024-09-11T08:38:44.999329+0000","last_unstale":"2024-09-11T08:48:24.381889+0000","last_undegraded":"2024-09-11T08:48:24.381889+0000","last_fullsized":"2024-09-11T08:48:24.381889+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:18.685188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.685188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:21.015279+0000","scrub_duration":0.0043815649999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":198,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.372174+0000","last_change":"2024-09-11T08:48:18.890324+0000","last_active":"2024-09-11T08:48:24.372174+0000","last_peered":"2024-09-11T08:48:24.372174+0000","last_clean":"2024-09-11T08:48:24.372174+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:48:24.372174+0000","last_undegraded":"2024-09-11T08:48:24.372174+0000","last_fullsized":"2024-09-11T08:48:24.372174+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:18.890233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.890233+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:33.893054+0000","scrub_duration":0.0070086059999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":192,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.366317+0000","last_change":"2024-09-11T08:45:14.574739+0000","last_active":"2024-09-11T08:48:24.366317+0000","last_peered":"2024-09-11T08:48:24.366317+0000","last_clean":"2024-09-11T08:48:24.366317+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:48:24.366317+0000","last_undegraded":"2024-09-11T08:48:24.366317+0000","last_fullsized":"2024-09-11T08:48:24.366317+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.574678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.574678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013819469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":200,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.368632+0000","last_change":"2024-09-11T08:48:18.958742+0000","last_active":"2024-09-11T08:48:24.368632+0000","last_peered":"2024-09-11T08:48:24.368632+0000","last_clean":"2024-09-11T08:48:24.368632+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:48:24.368632+0000","last_undegraded":"2024-09-11T08:48:24.368632+0000","last_fullsized":"2024-09-11T08:48:24.368632+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:18.958609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.958609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:43.228394+0000","scrub_duration":0.001704777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":195,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.372417+0000","last_change":"2024-09-11T08:47:44.807380+0000","last_active":"2024-09-11T08:48:24.372417+0000","last_peered":"2024-09-11T08:48:24.372417+0000","last_clean":"2024-09-11T08:48:24.372417+0000","last_became_active":"2024-09-11T08:46:01.984167+0000","last_became_peered":"2024-09-11T08:46:01.984167+0000","last_unstale":"2024-09-11T08:48:24.372417+0000","last_undegraded":"2024-09-11T08:48:24.372417+0000","last_fullsized":"2024-09-11T08:48:24.372417+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.807331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.807331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:03.561680+0000","scrub_duration":0.0013867529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"136'6","reported_seq":229,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:32.020329+0000","last_change":"2024-09-11T08:48:32.020078+0000","last_active":"2024-09-11T08:48:32.020329+0000","last_peered":"2024-09-11T08:48:32.020329+0000","last_clean":"2024-09-11T08:48:32.020329+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:48:32.020329+0000","last_undegraded":"2024-09-11T08:48:32.020329+0000","last_fullsized":"2024-09-11T08:48:32.020329+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'6","last_scrub_stamp":"2024-09-11T08:48:32.019991+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:48:32.019991+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:37.913420+0000","scrub_duration":0.010394439,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":199,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.383392+0000","last_change":"2024-09-11T08:45:21.481725+0000","last_active":"2024-09-11T08:48:24.383392+0000","last_peered":"2024-09-11T08:48:24.383392+0000","last_clean":"2024-09-11T08:48:24.383392+0000","last_became_active":"2024-09-11T08:38:38.956230+0000","last_became_peered":"2024-09-11T08:38:38.956230+0000","last_unstale":"2024-09-11T08:48:24.383392+0000","last_undegraded":"2024-09-11T08:48:24.383392+0000","last_fullsized":"2024-09-11T08:48:24.383392+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:21.481634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.481634+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025664590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":51,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:29.801855+0000","last_change":"2024-09-11T08:48:29.801804+0000","last_active":"2024-09-11T08:48:29.801855+0000","last_peered":"2024-09-11T08:48:29.801855+0000","last_clean":"2024-09-11T08:48:29.801855+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:48:29.801855+0000","last_undegraded":"2024-09-11T08:48:29.801855+0000","last_fullsized":"2024-09-11T08:48:29.801855+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:29.801722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:29.801722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:39.014794+0000","scrub_duration":0.0013425760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":195,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.367343+0000","last_change":"2024-09-11T08:45:23.348327+0000","last_active":"2024-09-11T08:48:24.367343+0000","last_peered":"2024-09-11T08:48:24.367343+0000","last_clean":"2024-09-11T08:48:24.367343+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:48:24.367343+0000","last_undegraded":"2024-09-11T08:48:24.367343+0000","last_fullsized":"2024-09-11T08:48:24.367343+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.348275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.348275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035843659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":196,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.374959+0000","last_change":"2024-09-11T08:45:23.674163+0000","last_active":"2024-09-11T08:48:24.374959+0000","last_peered":"2024-09-11T08:48:24.374959+0000","last_clean":"2024-09-11T08:48:24.374959+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:48:24.374959+0000","last_undegraded":"2024-09-11T08:48:24.374959+0000","last_fullsized":"2024-09-11T08:48:24.374959+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.674102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.674102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001440293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":181,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.376048+0000","last_change":"2024-09-11T08:44:23.320368+0000","last_active":"2024-09-11T08:48:24.376048+0000","last_peered":"2024-09-11T08:48:24.376048+0000","last_clean":"2024-09-11T08:48:24.376048+0000","last_became_active":"2024-09-11T08:38:38.956108+0000","last_became_peered":"2024-09-11T08:38:38.956108+0000","last_unstale":"2024-09-11T08:48:24.376048+0000","last_undegraded":"2024-09-11T08:48:24.376048+0000","last_fullsized":"2024-09-11T08:48:24.376048+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:23.320301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:23.320301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017133368999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":63,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.370727+0000","last_change":"2024-09-11T08:48:20.841324+0000","last_active":"2024-09-11T08:48:24.370727+0000","last_peered":"2024-09-11T08:48:24.370727+0000","last_clean":"2024-09-11T08:48:24.370727+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:48:24.370727+0000","last_undegraded":"2024-09-11T08:48:24.370727+0000","last_fullsized":"2024-09-11T08:48:24.370727+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:20.841243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.841243+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:44.045269+0000","scrub_duration":0.0096371630000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":188,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T08:48:24.384502+0000","last_change":"2024-09-11T08:47:32.555942+0000","last_active":"2024-09-11T08:48:24.384502+0000","last_peered":"2024-09-11T08:48:24.384502+0000","last_clean":"2024-09-11T08:48:24.384502+0000","last_became_active":"2024-09-11T08:38:42.983722+0000","last_became_peered":"2024-09-11T08:38:42.983722+0000","last_unstale":"2024-09-11T08:48:24.384502+0000","last_undegraded":"2024-09-11T08:48:24.384502+0000","last_fullsized":"2024-09-11T08:48:24.384502+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:32.555890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:32.555890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:48:41.110543+0000","scrub_duration":0.0014425740000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":630194200,"num_objects":9617,"num_object_clones":0,"num_object_copies":28851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9617,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230301,"num_write_kb":7677313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7769,"num_bytes_recovered":509149184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2285187072,"data_stored":2285174856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85309,"ondisk_log_size":85309,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":1584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":456,"ondisk_log_size":456,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":590,"num_read_kb":585,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299517,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":512284,"kb_used_data":206612,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93232868,"statfs":{"total":95995035648,"available":95470456832,"internally_reserved":0,"allocated":211570688,"data_stored":211248334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.2889999999999999,"15min":0.81799999999999995},"min":{"1min":0.372,"5min":0.22,"15min":0.22},"max":{"1min":2.585,"5min":32.171999999999997,"15min":32.171999999999997},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.571,"5min":1.4219999999999999,"15min":0.89900000000000002},"min":{"1min":0.29399999999999998,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":3.665,"5min":32.149999999999999,"15min":32.149999999999999},"last":1.637}]},{"osd":1,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.4830000000000001,"15min":0.90100000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.4980000000000002,"5min":33.692999999999998,"15min":33.692999999999998},"last":1.175},{"interface":"front","average":{"1min":1.228,"5min":1.3700000000000001,"15min":0.84499999999999997},"min":{"1min":0.32000000000000001,"5min":0.26600000000000001,"15min":0.23300000000000001},"max":{"1min":3.6259999999999999,"5min":35.591000000000001,"15min":35.591000000000001},"last":0.82099999999999995}]},{"osd":2,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.98899999999999999,"15min":0.754},"min":{"1min":0.30299999999999999,"5min":0.249,"15min":0.249},"max":{"1min":2.8140000000000001,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.5349999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":1.0429999999999999,"15min":0.71599999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.782,"5min":5.7990000000000004,"15min":5.7990000000000004},"last":1.129}]},{"osd":3,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.083,"15min":0.77800000000000002},"min":{"1min":0.32700000000000001,"5min":0.25,"15min":0.25},"max":{"1min":3.2839999999999998,"5min":7.8609999999999998,"15min":7.8609999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.29,"15min":0.85899999999999999},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":5.625,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":2.827}]},{"osd":4,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.53500000000000003,"15min":0.46700000000000003},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.23400000000000001},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.56299999999999994,"15min":0.47899999999999998},"min":{"1min":0.27000000000000002,"5min":0.216,"15min":0.216},"max":{"1min":4.8920000000000003,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.51200000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.60599999999999998,"15min":0.499},"min":{"1min":0.31,"5min":0.255,"15min":0.23300000000000001},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":0.746},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.67800000000000005,"15min":0.54200000000000004},"min":{"1min":0.32900000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.093,"5min":3.093,"15min":3.093},"last":0.80600000000000005}]},{"osd":6,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.68100000000000005,"15min":0.55800000000000005},"min":{"1min":0.35399999999999998,"5min":0.317,"15min":0.315},"max":{"1min":2.8820000000000001,"5min":3.5979999999999999,"15min":3.5979999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.63400000000000001,"15min":0.54600000000000004},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":0.48099999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528450,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":521292,"kb_used_data":211460,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93223860,"statfs":{"total":95995035648,"available":95461232640,"internally_reserved":0,"allocated":216535040,"data_stored":216228882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:48:26 2024","interfaces":[{"interface":"back","average":{"1min":1.9259999999999999,"5min":1.2470000000000001,"15min":0.98799999999999999},"min":{"1min":0.36699999999999999,"5min":0.217,"15min":0.217},"max":{"1min":3.6989999999999998,"5min":4.3890000000000002,"15min":28.302},"last":2.4420000000000002},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.149,"15min":0.92700000000000005},"min":{"1min":0.24299999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.6040000000000001,"5min":4.577,"15min":28.314},"last":3.024}]},{"osd":1,"last update":"Wed Sep 11 08:48:26 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.498,"15min":0.91000000000000003},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.0090000000000003,"5min":12.586,"15min":12.586},"last":2.0670000000000002},{"interface":"front","average":{"1min":1.863,"5min":1.369,"15min":0.86199999999999999},"min":{"1min":0.34300000000000003,"5min":0.25600000000000001,"15min":0.253},"max":{"1min":5.2619999999999996,"5min":10.834,"15min":10.834},"last":0.84599999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:48:26 2024","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.1040000000000001,"15min":0.77100000000000002},"min":{"1min":0.38200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":3.4580000000000002,"5min":14.313000000000001,"15min":14.313000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.1160000000000001,"15min":0.77000000000000002},"min":{"1min":0.29199999999999998,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":3.7240000000000002,"5min":6.5979999999999999,"15min":6.5979999999999999},"last":0.88100000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:48:26 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.401,"15min":0.92700000000000005},"min":{"1min":0.29199999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":5.1550000000000002,"5min":11.433,"15min":11.433},"last":1.7649999999999999},{"interface":"front","average":{"1min":2.7229999999999999,"5min":1.262,"15min":0.90700000000000003},"min":{"1min":0.32100000000000001,"5min":0.224,"15min":0.224},"max":{"1min":7.0250000000000004,"5min":7.0250000000000004,"15min":9.1069999999999993},"last":0.86399999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.59099999999999997,"15min":0.47499999999999998},"min":{"1min":0.28899999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.60799999999999998,"15min":0.47699999999999998},"min":{"1min":0.29799999999999999,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":4.0209999999999999,"5min":4.0209999999999999,"15min":4.0209999999999999},"last":0.33700000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:48:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":0.755,"15min":0.55300000000000005},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":8.6829999999999998,"5min":8.6829999999999998,"15min":8.6829999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.72699999999999998,"15min":0.54200000000000004},"min":{"1min":0.34499999999999997,"5min":0.29899999999999999,"15min":0.28100000000000003},"max":{"1min":4.0629999999999997,"5min":5.0819999999999999,"15min":5.0819999999999999},"last":0.36499999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:47:37 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64200000000000002,"15min":0.51300000000000001},"min":{"1min":0.29899999999999999,"5min":0.25900000000000001,"15min":0.247},"max":{"1min":2.7810000000000001,"5min":7.327,"15min":7.327},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.55600000000000005,"15min":0.51600000000000001},"min":{"1min":0.38400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.97799999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757382,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":649560,"kb_used_data":352208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93095592,"statfs":{"total":95995035648,"available":95329886208,"internally_reserved":0,"allocated":360660992,"data_stored":360342823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.04,"15min":0.751},"min":{"1min":0.23200000000000001,"5min":0.20899999999999999,"15min":0.20599999999999999},"max":{"1min":3.266,"5min":7.3899999999999997,"15min":7.3899999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.7629999999999999,"5min":1.2230000000000001,"15min":0.85499999999999998},"min":{"1min":0.23499999999999999,"5min":0.23400000000000001,"15min":0.23100000000000001},"max":{"1min":8.0410000000000004,"5min":8.0410000000000004,"15min":8.0410000000000004},"last":0.89800000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:48:01 2024","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":1.2490000000000001,"15min":0.84799999999999998},"min":{"1min":0.35999999999999999,"5min":0.23799999999999999,"15min":0.22800000000000001},"max":{"1min":8,"5min":8,"15min":8},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.21,"15min":0.78700000000000003},"min":{"1min":0.27400000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.649,"5min":6.6719999999999997,"15min":6.6719999999999997},"last":0.69799999999999995}]},{"osd":2,"last update":"Wed Sep 11 08:48:01 2024","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.1819999999999999,"15min":0.81299999999999994},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.23799999999999999},"max":{"1min":3.903,"5min":4.0919999999999996,"15min":5.0309999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.927,"5min":1.222,"15min":0.79600000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.26400000000000001},"max":{"1min":8.0640000000000001,"5min":8.0640000000000001,"15min":8.0640000000000001},"last":0.51200000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:48:01 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1020000000000001,"15min":0.78100000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":4.3940000000000001,"5min":6.0460000000000003,"15min":6.0460000000000003},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.641,"5min":1.3440000000000001,"15min":0.90000000000000002},"min":{"1min":0.33000000000000002,"5min":0.29199999999999998,"15min":0.27400000000000002},"max":{"1min":8.0860000000000003,"5min":8.0860000000000003,"15min":8.0860000000000003},"last":1.024}]},{"osd":4,"last update":"Wed Sep 11 08:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.65500000000000003,"15min":0.47799999999999998},"min":{"1min":0.247,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":4.7110000000000003,"5min":4.7110000000000003,"15min":4.7110000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.60399999999999998,"15min":0.47199999999999998},"min":{"1min":0.26200000000000001,"5min":0.23699999999999999,"15min":0.20499999999999999},"max":{"1min":3.7879999999999998,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.57499999999999996}]},{"osd":6,"last update":"Wed Sep 11 08:48:27 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":0.71099999999999997,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.27700000000000002,"15min":0.23599999999999999},"max":{"1min":6.4530000000000003,"5min":6.4530000000000003,"15min":6.4530000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.69099999999999995,"15min":0.51800000000000002},"min":{"1min":0.28100000000000003,"5min":0.26100000000000001,"15min":0.251},"max":{"1min":8.1920000000000002,"5min":8.1920000000000002,"15min":8.1920000000000002},"last":0.46800000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:47:34 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.64400000000000002,"15min":0.53400000000000003},"min":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.27800000000000002},"max":{"1min":3.7639999999999998,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.64400000000000002,"15min":0.57099999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.2730000000000001,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":0.78300000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986314,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":519380,"kb_used_data":217740,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93225772,"statfs":{"total":95995035648,"available":95463190528,"internally_reserved":0,"allocated":222965760,"data_stored":222650492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.6060000000000001,"15min":1.2949999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.20399999999999999},"max":{"1min":4.2830000000000004,"5min":48.451999999999998,"15min":89.510000000000005},"last":0.79300000000000004},{"interface":"front","average":{"1min":2.294,"5min":1.8700000000000001,"15min":1.4039999999999999},"min":{"1min":0.23999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":5.569,"5min":49.115000000000002,"15min":89.765000000000001},"last":2.0819999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:47:39 2024","interfaces":[{"interface":"back","average":{"1min":2.1200000000000001,"5min":1.5549999999999999,"15min":0.88500000000000001},"min":{"1min":0.63900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":6.1020000000000003,"5min":18.596,"15min":18.596},"last":1.998},{"interface":"front","average":{"1min":2.052,"5min":1.4570000000000001,"15min":0.872},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.23599999999999999},"max":{"1min":5.3810000000000002,"5min":18.754000000000001,"15min":18.754000000000001},"last":2.0510000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:47:39 2024","interfaces":[{"interface":"back","average":{"1min":2.3580000000000001,"5min":1.2310000000000001,"15min":0.80300000000000005},"min":{"1min":0.39300000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":17.303000000000001,"5min":17.303000000000001,"15min":17.303000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":2.1520000000000001,"5min":1.1859999999999999,"15min":0.78700000000000003},"min":{"1min":0.28599999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":20.702999999999999,"5min":20.702999999999999,"15min":20.702999999999999},"last":2.6459999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:47:39 2024","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.9550000000000001,"15min":1.4650000000000001},"min":{"1min":0.314,"5min":0.255,"15min":0.255},"max":{"1min":4.2229999999999999,"5min":48.194000000000003,"15min":89.881},"last":5.5590000000000002},{"interface":"front","average":{"1min":1.8720000000000001,"5min":1.925,"15min":1.4299999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":6.0650000000000004,"5min":48.840000000000003,"15min":89.715000000000003},"last":0.60699999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:48:04 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.64000000000000001,"15min":0.49099999999999999},"min":{"1min":0.27000000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":3.4020000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.65100000000000002,"15min":0.497},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.23699999999999999},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.65500000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.54800000000000004,"15min":0.47599999999999998},"min":{"1min":0.28599999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.8839999999999999,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":2.1640000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.65300000000000002,"15min":0.54700000000000004},"min":{"1min":0.30399999999999999,"5min":0.221,"15min":0.221},"max":{"1min":3.8109999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":3.1019999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:47:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.621,"15min":0.53000000000000003},"min":{"1min":0.29199999999999998,"5min":0.252,"15min":0.252},"max":{"1min":3.339,"5min":3.339,"15min":3.339},"last":0.628},{"interface":"front","average":{"1min":1.194,"5min":0.68700000000000006,"15min":0.55000000000000004},"min":{"1min":0.32000000000000001,"5min":0.29599999999999999,"15min":0.28799999999999998},"max":{"1min":5.1970000000000001,"5min":5.1970000000000001,"15min":5.1970000000000001},"last":2.1080000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149840,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":557100,"kb_used_data":251300,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93188052,"statfs":{"total":95995035648,"available":95424565248,"internally_reserved":0,"allocated":257331200,"data_stored":256992937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:48:24 2024","interfaces":[{"interface":"back","average":{"1min":2.1840000000000002,"5min":1.6859999999999999,"15min":1.0149999999999999},"min":{"1min":0.16500000000000001,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":24.280000000000001,"5min":24.280000000000001,"15min":24.280000000000001},"last":1.9399999999999999},{"interface":"front","average":{"1min":2.2509999999999999,"5min":2.1099999999999999,"15min":1.2050000000000001},"min":{"1min":0.184,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":22.225000000000001,"5min":22.225000000000001,"15min":22.225000000000001},"last":1.087}]},{"osd":1,"last update":"Wed Sep 11 08:48:24 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":3,"15min":1.5089999999999999},"min":{"1min":0.32700000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":5.0739999999999998,"5min":115.601,"15min":115.601},"last":3.4780000000000002},{"interface":"front","average":{"1min":1.2350000000000001,"5min":2.9220000000000002,"15min":1.427},"min":{"1min":0.314,"5min":0.23300000000000001,"15min":0.17999999999999999},"max":{"1min":3.6629999999999998,"5min":115.474,"15min":115.474},"last":1.5840000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:48:24 2024","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.8069999999999999,"15min":1.0669999999999999},"min":{"1min":0.30499999999999999,"5min":0.26400000000000001,"15min":0.23200000000000001},"max":{"1min":5.1429999999999998,"5min":8.9510000000000005,"15min":8.9510000000000005},"last":1.9139999999999999},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5860000000000001,"15min":1.0169999999999999},"min":{"1min":0.35799999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":4.3250000000000002,"5min":10.170999999999999,"15min":10.695},"last":1.8240000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:48:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6140000000000001,"5min":1.6499999999999999,"15min":1.0669999999999999},"min":{"1min":0.41699999999999998,"5min":0.28599999999999998,"15min":0.26700000000000002},"max":{"1min":6.0519999999999996,"5min":8.3279999999999994,"15min":8.9969999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.292,"15min":0.90800000000000003},"min":{"1min":0.318,"5min":0.30099999999999999,"15min":0.27700000000000002},"max":{"1min":2.524,"5min":7.234,"15min":7.234},"last":1.899}]},{"osd":5,"last update":"Wed Sep 11 08:47:53 2024","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.653,"15min":1.0009999999999999},"min":{"1min":0.438,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.2380000000000004,"5min":11.526999999999999,"15min":11.526999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":1.8169999999999999,"5min":1.427,"15min":0.90500000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.1689999999999996,"5min":7.1689999999999996,"15min":7.1689999999999996},"last":1.8520000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:48:24 2024","interfaces":[{"interface":"back","average":{"1min":1.633,"5min":1.736,"15min":1.0800000000000001},"min":{"1min":0.34799999999999998,"5min":0.311,"15min":0.29999999999999999},"max":{"1min":5.1929999999999996,"5min":11.494999999999999,"15min":11.494999999999999},"last":0.435},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.625,"15min":1.0549999999999999},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.311},"max":{"1min":5.0949999999999998,"5min":7.7249999999999996,"15min":7.7249999999999996},"last":1.1259999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:47:41 2024","interfaces":[{"interface":"back","average":{"1min":2.4969999999999999,"5min":1.831,"15min":1.0740000000000001},"min":{"1min":0.39600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":9.1539999999999999,"5min":11.561,"15min":11.561},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.8779999999999999,"5min":1.4890000000000001,"15min":0.93799999999999994},"min":{"1min":0.44500000000000001,"5min":0.371,"15min":0.34699999999999998},"max":{"1min":6.6200000000000001,"5min":6.6200000000000001,"15min":6.6200000000000001},"last":1.8779999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411476,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":726724,"kb_used_data":416892,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93018428,"statfs":{"total":95995035648,"available":95250870272,"internally_reserved":0,"allocated":426897408,"data_stored":426598259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:47:59 2024","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":2.6749999999999998,"15min":1.272},"min":{"1min":0.22700000000000001,"5min":0.159,"15min":0.159},"max":{"1min":4.1689999999999996,"5min":34.713999999999999,"15min":34.713999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":2.008,"5min":2.8239999999999998,"15min":1.391},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":5.1619999999999999,"5min":34.399000000000001,"15min":34.399000000000001},"last":0.51500000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:47:59 2024","interfaces":[{"interface":"back","average":{"1min":1.9179999999999999,"5min":2.1190000000000002,"15min":1.103},"min":{"1min":0.433,"5min":0.215,"15min":0.19},"max":{"1min":6.8700000000000001,"5min":34.771000000000001,"15min":34.771000000000001},"last":0.373},{"interface":"front","average":{"1min":2.355,"5min":2.3170000000000002,"15min":1.1759999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":6.5800000000000001,"5min":35.209000000000003,"15min":35.209000000000003},"last":3.1829999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:48:10 2024","interfaces":[{"interface":"back","average":{"1min":1.917,"5min":2.3540000000000001,"15min":1.266},"min":{"1min":0.28499999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":5.0149999999999997,"5min":34.75,"15min":34.75},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.766,"5min":2.169,"15min":1.236},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":6.9169999999999998,"5min":32.802,"15min":32.802},"last":0.47399999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:48:31 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.278,"15min":0.92200000000000004},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":3.9689999999999999,"5min":8.0820000000000007,"15min":8.0820000000000007},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.2210000000000001,"15min":0.88700000000000001},"min":{"1min":0.33000000000000002,"5min":0.29999999999999999,"15min":0.28399999999999997},"max":{"1min":3.427,"5min":6.9320000000000004,"15min":6.9320000000000004},"last":0.56399999999999995}]},{"osd":5,"last update":"Wed Sep 11 08:47:59 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.24,"15min":0.86899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":4.4809999999999999,"5min":6.9550000000000001,"15min":6.9550000000000001},"last":3.1549999999999998},{"interface":"front","average":{"1min":1.274,"5min":1.0449999999999999,"15min":0.78900000000000003},"min":{"1min":0.28999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.226,"5min":4.2000000000000002,"15min":5.9530000000000003},"last":0.84199999999999997}]},{"osd":6,"last update":"Wed Sep 11 08:48:20 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.409,"15min":0.98299999999999998},"min":{"1min":0.36099999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.9910000000000001,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.1779999999999999,"15min":0.874},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.452,"5min":6.9779999999999998,"15min":6.9779999999999998},"last":3.0790000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:47:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.0780000000000001,"15min":0.80600000000000005},"min":{"1min":0.35599999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.1400000000000001,"5min":5.1820000000000004,"15min":5.9909999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":1.673,"5min":1.351,"15min":0.94899999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36099999999999999},"max":{"1min":6.0919999999999996,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.54000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575001,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":596656,"kb_used_data":282664,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":93148496,"statfs":{"total":95995035648,"available":95384059904,"internally_reserved":0,"allocated":289447936,"data_stored":289104956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:48:32 2024","interfaces":[{"interface":"back","average":{"1min":1.7569999999999999,"5min":1.637,"15min":1.357},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":3.9470000000000001,"5min":31.466999999999999,"15min":117.01600000000001},"last":3.052},{"interface":"front","average":{"1min":2.0089999999999999,"5min":2.1150000000000002,"15min":1.651},"min":{"1min":0.17499999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":8.1549999999999994,"5min":32.204000000000001,"15min":117.62},"last":3.4980000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:47:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.5409999999999999,"15min":0.97299999999999998},"min":{"1min":0.25700000000000001,"5min":0.191,"15min":0.191},"max":{"1min":4.9829999999999997,"5min":9.5719999999999992,"15min":20.628},"last":3.383},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.472,"15min":0.94199999999999995},"min":{"1min":0.23200000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.5529999999999999,"5min":7.4080000000000004,"15min":20.148},"last":8.9120000000000008}]},{"osd":3,"last update":"Wed Sep 11 08:48:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":1.4930000000000001,"15min":1.2829999999999999},"min":{"1min":0.22,"5min":0.191,"15min":0.191},"max":{"1min":2.9510000000000001,"5min":8.3550000000000004,"15min":116.98},"last":3.0329999999999999},{"interface":"front","average":{"1min":1.671,"5min":1.5169999999999999,"15min":1.3859999999999999},"min":{"1min":0.31,"5min":0.24099999999999999,"15min":0.22},"max":{"1min":4.6719999999999997,"5min":7.415,"15min":117.56699999999999},"last":3.472}]},{"osd":4,"last update":"Wed Sep 11 08:48:32 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.1879999999999999,"15min":0.871},"min":{"1min":0.32700000000000001,"5min":0.27500000000000002,"15min":0.26200000000000001},"max":{"1min":4.6580000000000004,"5min":6.3490000000000002,"15min":11.19},"last":2.4420000000000002},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.1539999999999999,"15min":0.83099999999999996},"min":{"1min":0.34799999999999998,"5min":0.30199999999999999,"15min":0.251},"max":{"1min":3.8460000000000001,"5min":5.056,"15min":7.7169999999999996},"last":2.9430000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:47:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":2.5070000000000001,"15min":1.3240000000000001},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.125,"5min":103.337,"15min":103.337},"last":3.4460000000000002},{"interface":"front","average":{"1min":1.6619999999999999,"5min":2.48,"15min":1.29},"min":{"1min":0.39000000000000001,"5min":0.32300000000000001,"15min":0.27400000000000002},"max":{"1min":3.9279999999999999,"5min":103.163,"15min":103.163},"last":2.5030000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:48:27 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.135,"15min":0.83799999999999997},"min":{"1min":0.39300000000000002,"5min":0.34100000000000003,"15min":0.29399999999999998},"max":{"1min":3.8980000000000001,"5min":3.907,"15min":8.9149999999999991},"last":3.0129999999999999},{"interface":"front","average":{"1min":1.6399999999999999,"5min":1.3480000000000001,"15min":0.99099999999999999},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.31},"max":{"1min":4.7080000000000002,"5min":6.3250000000000002,"15min":9.5199999999999996},"last":3.415}]},{"osd":7,"last update":"Wed Sep 11 08:48:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.2450000000000001,"15min":0.92900000000000005},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":4.3799999999999999,"5min":6.4059999999999997,"15min":11.315},"last":4.3799999999999999},{"interface":"front","average":{"1min":1.75,"5min":1.26,"15min":0.90000000000000002},"min":{"1min":0.40000000000000002,"5min":0.36299999999999999,"15min":0.315},"max":{"1min":10.266,"5min":10.266,"15min":10.266},"last":2.9780000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738525,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":735772,"kb_used_data":438420,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93009380,"statfs":{"total":95995035648,"available":95241605120,"internally_reserved":0,"allocated":448942080,"data_stored":448619761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:48:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.714,"15min":1.052},"min":{"1min":0.19,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":4.484,"5min":8.3900000000000006,"15min":8.3900000000000006},"last":3.8959999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.754,"15min":0.98399999999999999},"min":{"1min":0.375,"5min":0.187,"15min":0.186},"max":{"1min":3.9169999999999998,"5min":12.532,"15min":12.532},"last":1.0529999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:47:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.6379999999999999,"15min":1.0089999999999999},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":5.7489999999999997,"5min":12.481,"15min":19.940000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.9179999999999999,"15min":1.077},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":4.2530000000000001,"5min":8.3049999999999997,"15min":20.757000000000001},"last":1.1200000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:48:14 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.962,"15min":1.165},"min":{"1min":0.27200000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.0469999999999997,"5min":11.951000000000001,"15min":11.951000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.825,"5min":1.7629999999999999,"15min":1.1060000000000001},"min":{"1min":0.38600000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":6.8029999999999999,"5min":7.6399999999999997,"15min":7.6399999999999997},"last":0.71699999999999997}]},{"osd":4,"last update":"Wed Sep 11 08:48:35 2024","interfaces":[{"interface":"back","average":{"1min":1.669,"5min":1.3700000000000001,"15min":1.0089999999999999},"min":{"1min":0.27200000000000002,"5min":0.26600000000000001,"15min":0.25},"max":{"1min":4.6959999999999997,"5min":6.2009999999999996,"15min":7.4690000000000003},"last":0.748},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.3300000000000001,"15min":0.96999999999999997},"min":{"1min":0.32500000000000001,"5min":0.27500000000000002,"15min":0.255},"max":{"1min":3.9670000000000001,"5min":7.4630000000000001,"15min":7.4630000000000001},"last":2.7770000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:47:56 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5629999999999999,"15min":0.98699999999999999},"min":{"1min":0.33700000000000002,"5min":0.30299999999999999,"15min":0.27800000000000002},"max":{"1min":4.0209999999999999,"5min":7.4009999999999998,"15min":7.4009999999999998},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.4830000000000001,"15min":0.98499999999999999},"min":{"1min":0.45800000000000002,"5min":0.32700000000000001,"15min":0.28299999999999997},"max":{"1min":6.2240000000000002,"5min":6.2240000000000002,"15min":7.423},"last":3.9329999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:48:14 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.647,"15min":1.0549999999999999},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.32600000000000001},"max":{"1min":4.2270000000000003,"5min":7.4240000000000004,"15min":7.4240000000000004},"last":1.093},{"interface":"front","average":{"1min":1.613,"5min":1.619,"15min":1.0960000000000001},"min":{"1min":0.53100000000000003,"5min":0.35699999999999998,"15min":0.311},"max":{"1min":4.0659999999999998,"5min":6.1740000000000004,"15min":7.4489999999999998},"last":0.76900000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:47:45 2024","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.6519999999999999,"15min":0.999},"min":{"1min":0.32800000000000001,"5min":0.313,"15min":0.313},"max":{"1min":5.8179999999999996,"5min":7.4900000000000002,"15min":7.4900000000000002},"last":2.8130000000000002},{"interface":"front","average":{"1min":2.0150000000000001,"5min":1.629,"15min":1.0209999999999999},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.307},"max":{"1min":6.0800000000000001,"5min":7.4450000000000003,"15min":7.4450000000000003},"last":1.141}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":444137472,"data_stored":444137472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":269877248,"data_stored":269877248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":392695808,"data_stored":392691736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":232980480,"data_stored":232980480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":211222528,"data_stored":211222528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":318771200,"data_stored":318767128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":206962688,"data_stored":206962688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":208539648,"data_stored":208535576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:48:43.291 INFO:tasks.thrashosds.thrasher:Setting 6.19 to [6, 5, 4, 3, 7, 1] 2024-09-11T08:48:43.291 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '6.19', '6', '5', '4', '3', '7', '1'] 2024-09-11T08:48:43.291 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pg-upmap-items 6.19 6 5 4 3 7 1 2024-09-11T08:48:43.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:43.303 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:43.303 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:43.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:43.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:43.470 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:43.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:43.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:43.610 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:43.610 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:43.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:43.754 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:43.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:43.755 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:43.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-11T08:48:43.855 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:43.880 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:43.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:43.881 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:43.981 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:44.011 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:44.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:44.012 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:44.112 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:44.144 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:44.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:44.145 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:44.246 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:44.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:44.282 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:44.282 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:44.383 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:44.423 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:44.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:44.424 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:44.524 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:44.554 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Removed 119958 objects 2024-09-11T08:48:44.555 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Clean up completed and total clean up time :104.374 2024-09-11T08:48:44.558 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:44.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:44.559 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:44.660 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:44.681 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:44.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:44.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:44.755 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:44 smithi092 ceph-mon[29647]: pgmap v417: 148 pgs: 148 active+clean; 221 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-11T08:48:44.755 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:44 smithi092 ceph-mon[29647]: 4.d scrub starts 2024-09-11T08:48:44.755 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:44 smithi092 ceph-mon[29647]: 4.d scrub ok 2024-09-11T08:48:44.755 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:44 smithi092 ceph-mon[29647]: 6.5 scrub starts 2024-09-11T08:48:44.755 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:44 smithi092 ceph-mon[29647]: 6.5 scrub ok 2024-09-11T08:48:44.755 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:44 smithi092 ceph-mon[41114]: pgmap v417: 148 pgs: 148 active+clean; 221 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-11T08:48:44.755 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:44 smithi092 ceph-mon[41114]: 4.d scrub starts 2024-09-11T08:48:44.755 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:44 smithi092 ceph-mon[41114]: 4.d scrub ok 2024-09-11T08:48:44.756 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:44 smithi092 ceph-mon[41114]: 6.5 scrub starts 2024-09-11T08:48:44.756 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:44 smithi092 ceph-mon[41114]: 6.5 scrub ok 2024-09-11T08:48:44.783 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:44.808 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:44.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:44.809 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:44.910 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:44 smithi145 ceph-mon[33384]: pgmap v417: 148 pgs: 148 active+clean; 221 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-11T08:48:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:44 smithi145 ceph-mon[33384]: 4.d scrub starts 2024-09-11T08:48:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:44 smithi145 ceph-mon[33384]: 4.d scrub ok 2024-09-11T08:48:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:44 smithi145 ceph-mon[33384]: 6.5 scrub starts 2024-09-11T08:48:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:44 smithi145 ceph-mon[33384]: 6.5 scrub ok 2024-09-11T08:48:44.932 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:44.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:44.934 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:45.035 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:45.056 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:45.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:45.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:45.158 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:45.180 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:45.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:45.181 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:45.261 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:45.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:45.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:45.304 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:45.304 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:45.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:45.426 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:45.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:45.427 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:45.493 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[29647]: 4.0 deep-scrub starts 2024-09-11T08:48:45.493 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[29647]: 4.0 deep-scrub ok 2024-09-11T08:48:45.493 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[29647]: 3.1c scrub starts 2024-09-11T08:48:45.493 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[29647]: 3.1c scrub ok 2024-09-11T08:48:45.493 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[29647]: 5.3 scrub starts 2024-09-11T08:48:45.494 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[29647]: 5.3 scrub ok 2024-09-11T08:48:45.494 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:48:45.494 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:45.494 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[41114]: 4.0 deep-scrub starts 2024-09-11T08:48:45.494 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[41114]: 4.0 deep-scrub ok 2024-09-11T08:48:45.494 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[41114]: 3.1c scrub starts 2024-09-11T08:48:45.494 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[41114]: 3.1c scrub ok 2024-09-11T08:48:45.494 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[41114]: 5.3 scrub starts 2024-09-11T08:48:45.494 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[41114]: 5.3 scrub ok 2024-09-11T08:48:45.494 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:48:45.494 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:45.504 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_1 2024-09-11T08:48:45.504 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2024-09-11T08:48:45.527 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:45.568 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:45.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:45.570 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:45.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:45.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:45.697 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:45.697 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:45.705 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:45.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:45.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:45.820 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:45.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:45 smithi145 ceph-mon[33384]: 4.0 deep-scrub starts 2024-09-11T08:48:45.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:45 smithi145 ceph-mon[33384]: 4.0 deep-scrub ok 2024-09-11T08:48:45.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:45 smithi145 ceph-mon[33384]: 3.1c scrub starts 2024-09-11T08:48:45.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:45 smithi145 ceph-mon[33384]: 3.1c scrub ok 2024-09-11T08:48:45.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:45 smithi145 ceph-mon[33384]: 5.3 scrub starts 2024-09-11T08:48:45.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:45 smithi145 ceph-mon[33384]: 5.3 scrub ok 2024-09-11T08:48:45.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:45 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:48:45.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:48:45.921 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:45.941 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:48:45.942 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:48:45.945 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:45.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:45.946 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:45.958 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:48:45.958 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:48:45.973 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:48:45.973 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:48:45.988 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:48:45.988 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:48:46.003 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:48:46.004 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:48:46.019 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:48:46.019 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:48:46.036 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:48:46.036 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:48:46.047 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:46.051 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:48:46.051 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:48:46.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:46.071 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:46.071 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:46.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:46.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:46.194 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:46.195 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:46.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:46.316 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:46.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:46.320 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:46.421 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:46.446 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:46.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:46.447 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:46.548 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:46.570 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:46.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:46.571 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:46.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:46.693 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:46.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:46.694 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:46.795 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:46.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:48:46.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:48:46.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[29647]: pgmap v418: 148 pgs: 148 active+clean; 221 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 868 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-11T08:48:46.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[29647]: 8.e deep-scrub starts 2024-09-11T08:48:46.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/859745482' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:48:46.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[29647]: 1.0 deep-scrub starts 2024-09-11T08:48:46.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[29647]: 1.0 deep-scrub ok 2024-09-11T08:48:46.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[29647]: osdmap e200: 8 total, 8 up, 8 in 2024-09-11T08:48:46.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4170843546' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:48:46.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:48:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:48:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:48:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[41114]: pgmap v418: 148 pgs: 148 active+clean; 221 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 868 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-11T08:48:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[41114]: 8.e deep-scrub starts 2024-09-11T08:48:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/859745482' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:48:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[41114]: 1.0 deep-scrub starts 2024-09-11T08:48:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[41114]: 1.0 deep-scrub ok 2024-09-11T08:48:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[41114]: osdmap e200: 8 total, 8 up, 8 in 2024-09-11T08:48:46.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4170843546' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:48:46.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:46 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:48:46.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:46.817 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:46.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:46.912 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:48:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:48:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:46 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:48:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:46 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:48:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:46 smithi145 ceph-mon[33384]: pgmap v418: 148 pgs: 148 active+clean; 221 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 868 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-11T08:48:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:46 smithi145 ceph-mon[33384]: 8.e deep-scrub starts 2024-09-11T08:48:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:46 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/859745482' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:48:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:46 smithi145 ceph-mon[33384]: 1.0 deep-scrub starts 2024-09-11T08:48:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:46 smithi145 ceph-mon[33384]: 1.0 deep-scrub ok 2024-09-11T08:48:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:46 smithi145 ceph-mon[33384]: osdmap e200: 8 total, 8 up, 8 in 2024-09-11T08:48:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:46 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4170843546' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:48:46.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:46 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:48:46.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:46.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:46.940 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:47.041 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:47.062 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:47.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:47.063 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:47.089 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:48:47.165 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:47.186 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:47.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:47.187 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:47.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:47.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:47.310 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:47.310 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:47.411 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:47.432 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:47.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:47.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:47.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:47.555 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:47.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:47.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:47.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:47.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:47.678 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:47.730 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:47 smithi092 ceph-mon[29647]: 8.e deep-scrub ok 2024-09-11T08:48:47.730 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:47 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:47.730 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:47 smithi092 ceph-mon[29647]: 3.e scrub starts 2024-09-11T08:48:47.731 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:47 smithi092 ceph-mon[29647]: 3.e scrub ok 2024-09-11T08:48:47.731 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:47 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:48:47.731 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:47 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:48:47.731 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:47 smithi092 ceph-mon[29647]: osdmap e201: 8 total, 8 up, 8 in 2024-09-11T08:48:47.731 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:47 smithi092 ceph-mon[41114]: 8.e deep-scrub ok 2024-09-11T08:48:47.731 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:47 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:47.731 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:47 smithi092 ceph-mon[41114]: 3.e scrub starts 2024-09-11T08:48:47.731 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:47 smithi092 ceph-mon[41114]: 3.e scrub ok 2024-09-11T08:48:47.731 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:47 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:48:47.731 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:47 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:48:47.731 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:47 smithi092 ceph-mon[41114]: osdmap e201: 8 total, 8 up, 8 in 2024-09-11T08:48:47.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:47.800 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:47.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:47.801 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:47.902 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:47.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:47 smithi145 ceph-mon[33384]: 8.e deep-scrub ok 2024-09-11T08:48:47.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:47 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:47.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:47 smithi145 ceph-mon[33384]: 3.e scrub starts 2024-09-11T08:48:47.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:47 smithi145 ceph-mon[33384]: 3.e scrub ok 2024-09-11T08:48:47.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:47 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:48:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:47 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:48:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:47 smithi145 ceph-mon[33384]: osdmap e201: 8 total, 8 up, 8 in 2024-09-11T08:48:47.928 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:47.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:47.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:48.030 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:48.051 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:48.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:48.052 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:48.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:48.174 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:48.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:48.175 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:48.276 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:48.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:48.303 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:48.303 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:48.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:48.426 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:48.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:48.428 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:48.529 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:48.555 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:48.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:48.557 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:48.658 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:48.684 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:48.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:48.685 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:48.760 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[29647]: 8.6 scrub starts 2024-09-11T08:48:48.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[29647]: 8.6 scrub ok 2024-09-11T08:48:48.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[29647]: pgmap v421: 148 pgs: 148 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:48:48.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:48:48.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:48:48.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[29647]: 6.14 scrub starts 2024-09-11T08:48:48.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[29647]: 6.14 scrub ok 2024-09-11T08:48:48.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[29647]: osdmap e202: 8 total, 8 up, 8 in 2024-09-11T08:48:48.761 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[41114]: 8.6 scrub starts 2024-09-11T08:48:48.761 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[41114]: 8.6 scrub ok 2024-09-11T08:48:48.761 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[41114]: pgmap v421: 148 pgs: 148 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:48:48.762 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:48:48.762 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:48:48.762 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[41114]: 6.14 scrub starts 2024-09-11T08:48:48.762 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[41114]: 6.14 scrub ok 2024-09-11T08:48:48.762 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:48 smithi092 ceph-mon[41114]: osdmap e202: 8 total, 8 up, 8 in 2024-09-11T08:48:48.785 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:48.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:48.811 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:48.811 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:48.912 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:48.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:48 smithi145 ceph-mon[33384]: 8.6 scrub starts 2024-09-11T08:48:48.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:48 smithi145 ceph-mon[33384]: 8.6 scrub ok 2024-09-11T08:48:48.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:48 smithi145 ceph-mon[33384]: pgmap v421: 148 pgs: 148 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:48:48.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:48 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:48:48.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:48 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:48:48.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:48 smithi145 ceph-mon[33384]: 6.14 scrub starts 2024-09-11T08:48:48.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:48 smithi145 ceph-mon[33384]: 6.14 scrub ok 2024-09-11T08:48:48.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:48 smithi145 ceph-mon[33384]: osdmap e202: 8 total, 8 up, 8 in 2024-09-11T08:48:48.936 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:48.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:48.937 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:48.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:48:49.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:49.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:49.060 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:49.092 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:49.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:49.183 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:49.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:49.184 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:49.285 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:49.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:49.307 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:49.307 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:49.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:49.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:49.429 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:49.429 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:49.530 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:49.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:49.552 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:49.552 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:49.653 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:49.678 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:49.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:49.679 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:49.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:49.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:49.803 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:49.803 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:49.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:49 smithi092 ceph-mon[29647]: 4.5 deep-scrub starts 2024-09-11T08:48:49.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:49 smithi092 ceph-mon[29647]: 4.5 deep-scrub ok 2024-09-11T08:48:49.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:49 smithi092 ceph-mon[29647]: 3.1f scrub starts 2024-09-11T08:48:49.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:49 smithi092 ceph-mon[29647]: 3.1f scrub ok 2024-09-11T08:48:49.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:49 smithi092 ceph-mon[29647]: 3.0 scrub starts 2024-09-11T08:48:49.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:49 smithi092 ceph-mon[41114]: 4.5 deep-scrub starts 2024-09-11T08:48:49.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:49 smithi092 ceph-mon[41114]: 4.5 deep-scrub ok 2024-09-11T08:48:49.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:49 smithi092 ceph-mon[41114]: 3.1f scrub starts 2024-09-11T08:48:49.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:49 smithi092 ceph-mon[41114]: 3.1f scrub ok 2024-09-11T08:48:49.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:49 smithi092 ceph-mon[41114]: 3.0 scrub starts 2024-09-11T08:48:49.905 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:49.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:49 smithi145 ceph-mon[33384]: 4.5 deep-scrub starts 2024-09-11T08:48:49.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:49 smithi145 ceph-mon[33384]: 4.5 deep-scrub ok 2024-09-11T08:48:49.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:49 smithi145 ceph-mon[33384]: 3.1f scrub starts 2024-09-11T08:48:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:49 smithi145 ceph-mon[33384]: 3.1f scrub ok 2024-09-11T08:48:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:49 smithi145 ceph-mon[33384]: 3.0 scrub starts 2024-09-11T08:48:49.930 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:49.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:49.931 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:50.032 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:50.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:50.055 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:50.055 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:50.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:50.178 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:50.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:50.179 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:50.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:48:50.280 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:50.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:50.308 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:50.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:50.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:50.432 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:50.432 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:50.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:50.554 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:50.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:50.555 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:50.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:50 smithi092 ceph-mon[29647]: pgmap v423: 148 pgs: 148 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 645 op/s 2024-09-11T08:48:50.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:50 smithi092 ceph-mon[41114]: pgmap v423: 148 pgs: 148 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 645 op/s 2024-09-11T08:48:50.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:50.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:50.679 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:50.679 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:50.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:50.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:50.802 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:50.903 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:50.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:50 smithi145 ceph-mon[33384]: pgmap v423: 148 pgs: 148 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 645 op/s 2024-09-11T08:48:50.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:50.925 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:50.925 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:51.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:51.047 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:51.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:51.048 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:51.149 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:51.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:51.176 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:51.177 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:51.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:51.299 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:51.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:51.300 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:51.401 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:51.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:51.428 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:51.428 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:51.529 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:51.555 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:51.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:51.556 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:51.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:48:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:48:51] "GET /metrics HTTP/1.1" 200 224309 "" "Prometheus/2.43.0" 2024-09-11T08:48:51.657 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:51.681 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:51.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:51.682 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:51.783 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:51.807 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:51.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:51.808 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:51.909 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:51.935 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:51.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:51.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:52.037 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:52.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:52.063 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:52.064 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:52.165 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:52.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:52.191 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:52.191 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:52.292 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:52.317 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:52.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:52.318 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:52.419 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:52.444 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:52.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:52.445 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:52.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:52.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:52.573 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:52.573 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:52.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:52.697 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:52.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:52.698 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:52.700 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:52 smithi092 ceph-mon[29647]: pgmap v424: 148 pgs: 148 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 61 KiB/s rd, 0 B/s wr, 688 op/s 2024-09-11T08:48:52.700 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:52 smithi092 ceph-mon[41114]: pgmap v424: 148 pgs: 148 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 61 KiB/s rd, 0 B/s wr, 688 op/s 2024-09-11T08:48:52.799 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:52.825 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:52.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:52.826 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:52 smithi145 ceph-mon[33384]: pgmap v424: 148 pgs: 148 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 61 KiB/s rd, 0 B/s wr, 688 op/s 2024-09-11T08:48:52.912 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:48:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:48:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:48:52.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:52.949 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:52.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:52.950 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:53.050 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:53.076 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:53.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:53.077 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:53.178 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:53.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:53.205 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:53.205 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:53.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:53.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:53.328 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:53.329 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:53.356 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:53.356 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:48:53.357 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:48:53.357 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-11T08:48:53.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-11T08:48:53.357 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:48:53.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.357 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:66214 7.2 7:43d51050:::benchmark_data_smithi092_84565_object66213:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:48:53.357 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.169283+0000", 2024-09-11T08:48:53.357 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 427.18662952900002, 2024-09-11T08:48:53.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.111582084, 2024-09-11T08:48:53.357 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.358 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.358 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.358 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:53.358 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:53.358 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 66214 2024-09-11T08:48:53.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.358 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.169283+0000", 2024-09-11T08:48:53.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.169283+0000", 2024-09-11T08:48:53.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.169282+0000", 2024-09-11T08:48:53.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:53.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.169305+0000", 2024-09-11T08:48:53.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2558e-05 2024-09-11T08:48:53.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.169318+0000", 2024-09-11T08:48:53.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3168e-05 2024-09-11T08:48:53.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.169323+0000", 2024-09-11T08:48:53.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1819999999999997e-06 2024-09-11T08:48:53.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.171852+0000", 2024-09-11T08:48:53.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002529183 2024-09-11T08:48:53.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.171957+0000", 2024-09-11T08:48:53.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010511800000000001 2024-09-11T08:48:53.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:48:53.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.172078+0000", 2024-09-11T08:48:53.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012122500000000001 2024-09-11T08:48:53.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.178227+0000", 2024-09-11T08:48:53.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061484770000000003 2024-09-11T08:48:53.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.178241+0000", 2024-09-11T08:48:53.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3675999999999999e-05 2024-09-11T08:48:53.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280731+0000", 2024-09-11T08:48:53.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102490704 2024-09-11T08:48:53.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280803+0000", 2024-09-11T08:48:53.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1419000000000002e-05 2024-09-11T08:48:53.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280865+0000", 2024-09-11T08:48:53.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2639999999999997e-05 2024-09-11T08:48:53.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.363 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.364 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:66217 7.2 7:49541d7a:::benchmark_data_smithi092_84565_object66216:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:48:53.364 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:41:46.171978+0000", 2024-09-11T08:48:53.364 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 427.18393464000002, 2024-09-11T08:48:53.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10905237, 2024-09-11T08:48:53.364 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.364 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.364 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.364 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:53.364 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:53.364 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 66217 2024-09-11T08:48:53.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.365 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.171978+0000", 2024-09-11T08:48:53.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.171978+0000", 2024-09-11T08:48:53.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.171977+0000", 2024-09-11T08:48:53.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:53.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.171999+0000", 2024-09-11T08:48:53.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2807999999999999e-05 2024-09-11T08:48:53.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.172011+0000", 2024-09-11T08:48:53.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1895e-05 2024-09-11T08:48:53.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.172017+0000", 2024-09-11T08:48:53.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5879999999999997e-06 2024-09-11T08:48:53.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174638+0000", 2024-09-11T08:48:53.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026211419999999999 2024-09-11T08:48:53.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174745+0000", 2024-09-11T08:48:53.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107122 2024-09-11T08:48:53.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:48:53.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.174871+0000", 2024-09-11T08:48:53.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012604700000000001 2024-09-11T08:48:53.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.179145+0000", 2024-09-11T08:48:53.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004273326 2024-09-11T08:48:53.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.179616+0000", 2024-09-11T08:48:53.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047181799999999999 2024-09-11T08:48:53.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280942+0000", 2024-09-11T08:48:53.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101325657 2024-09-11T08:48:53.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.280993+0000", 2024-09-11T08:48:53.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0571000000000001e-05 2024-09-11T08:48:53.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:41:46.281030+0000", 2024-09-11T08:48:53.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7926999999999998e-05 2024-09-11T08:48:53.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.370 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.370 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:122659 7.c e109/88 7:37de432f:::benchmark_data_smithi092_84565_object122658:head v 109'7559, mlcod=109'7554)", 2024-09-11T08:48:53.370 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:38.706495+0000", 2024-09-11T08:48:53.370 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 374.64941735799999, 2024-09-11T08:48:53.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103840132, 2024-09-11T08:48:53.370 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.370 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.370 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.706495+0000", 2024-09-11T08:48:53.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.706495+0000", 2024-09-11T08:48:53.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.706495+0000", 2024-09-11T08:48:53.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:48:53.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.706529+0000", 2024-09-11T08:48:53.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4011999999999999e-05 2024-09-11T08:48:53.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.706539+0000", 2024-09-11T08:48:53.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0664e-05 2024-09-11T08:48:53.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.706545+0000", 2024-09-11T08:48:53.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6849999999999998e-06 2024-09-11T08:48:53.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.706606+0000", 2024-09-11T08:48:53.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0773999999999998e-05 2024-09-11T08:48:53.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.706624+0000", 2024-09-11T08:48:53.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8295999999999999e-05 2024-09-11T08:48:53.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.809251+0000", 2024-09-11T08:48:53.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102627326 2024-09-11T08:48:53.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.810336+0000", 2024-09-11T08:48:53.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010840730000000001 2024-09-11T08:48:53.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.375 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.375 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:122670 7.7 e109/88 7:e7a085d3:::benchmark_data_smithi092_84565_object122669:head v 109'7633, mlcod=109'7631)", 2024-09-11T08:48:53.375 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:38.707680+0000", 2024-09-11T08:48:53.375 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 374.64823244500002, 2024-09-11T08:48:53.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.113649625, 2024-09-11T08:48:53.376 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.376 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.376 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.707680+0000", 2024-09-11T08:48:53.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.707680+0000", 2024-09-11T08:48:53.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.707680+0000", 2024-09-11T08:48:53.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:48:53.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.707728+0000", 2024-09-11T08:48:53.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8554000000000003e-05 2024-09-11T08:48:53.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.707739+0000", 2024-09-11T08:48:53.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1263999999999999e-05 2024-09-11T08:48:53.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.707745+0000", 2024-09-11T08:48:53.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4970000000000001e-06 2024-09-11T08:48:53.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.707904+0000", 2024-09-11T08:48:53.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015863800000000001 2024-09-11T08:48:53.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.801999+0000", 2024-09-11T08:48:53.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.094095511000000007 2024-09-11T08:48:53.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.821061+0000", 2024-09-11T08:48:53.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019062055000000001 2024-09-11T08:48:53.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.821330+0000", 2024-09-11T08:48:53.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026882400000000003 2024-09-11T08:48:53.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.380 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.380 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:122672 7.c e109/88 7:30125efa:::benchmark_data_smithi092_84565_object122671:head v 109'7560, mlcod=109'7557)", 2024-09-11T08:48:53.380 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:42:38.707885+0000", 2024-09-11T08:48:53.380 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 374.64802817499998, 2024-09-11T08:48:53.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.110353068, 2024-09-11T08:48:53.380 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.380 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.380 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.707885+0000", 2024-09-11T08:48:53.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.707885+0000", 2024-09-11T08:48:53.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.707884+0000", 2024-09-11T08:48:53.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:48:53.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.707948+0000", 2024-09-11T08:48:53.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3607000000000003e-05 2024-09-11T08:48:53.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.707955+0000", 2024-09-11T08:48:53.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5889999999999997e-06 2024-09-11T08:48:53.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.801992+0000", 2024-09-11T08:48:53.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.094036731999999998 2024-09-11T08:48:53.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.805318+0000", 2024-09-11T08:48:53.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033255170000000001 2024-09-11T08:48:53.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.805337+0000", 2024-09-11T08:48:53.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9063e-05 2024-09-11T08:48:53.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.818199+0000", 2024-09-11T08:48:53.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012862837 2024-09-11T08:48:53.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:42:38.818238+0000", 2024-09-11T08:48:53.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8179000000000002e-05 2024-09-11T08:48:53.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.384 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.385 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:196054 7.a 7:5d930975:::benchmark_data_smithi092_84565_object70526:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T08:48:53.385 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:50.514799+0000", 2024-09-11T08:48:53.385 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 302.84111409100001, 2024-09-11T08:48:53.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11866589, 2024-09-11T08:48:53.385 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.385 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:53.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:53.386 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 196054 2024-09-11T08:48:53.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.386 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.514799+0000", 2024-09-11T08:48:53.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.514799+0000", 2024-09-11T08:48:53.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.514797+0000", 2024-09-11T08:48:53.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:53.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.514803+0000", 2024-09-11T08:48:53.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8289999999999996e-06 2024-09-11T08:48:53.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.514815+0000", 2024-09-11T08:48:53.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1768e-05 2024-09-11T08:48:53.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.514821+0000", 2024-09-11T08:48:53.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6130000000000003e-06 2024-09-11T08:48:53.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.520890+0000", 2024-09-11T08:48:53.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006069825 2024-09-11T08:48:53.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.521022+0000", 2024-09-11T08:48:53.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013121399999999999 2024-09-11T08:48:53.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,5", 2024-09-11T08:48:53.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.521120+0000", 2024-09-11T08:48:53.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8622999999999994e-05 2024-09-11T08:48:53.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.524379+0000", 2024-09-11T08:48:53.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032583429999999999 2024-09-11T08:48:53.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.525225+0000", 2024-09-11T08:48:53.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00084646500000000002 2024-09-11T08:48:53.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.633400+0000", 2024-09-11T08:48:53.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10817438 2024-09-11T08:48:53.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.633432+0000", 2024-09-11T08:48:53.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2614999999999999e-05 2024-09-11T08:48:53.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.633465+0000", 2024-09-11T08:48:53.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.243e-05 2024-09-11T08:48:53.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.391 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.391 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:196061 7.2 7:4b968b56:::benchmark_data_smithi092_84565_object70533:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T08:48:53.392 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:50.524636+0000", 2024-09-11T08:48:53.392 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 302.83127679, 2024-09-11T08:48:53.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.108727723, 2024-09-11T08:48:53.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:53.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:53.392 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 196061 2024-09-11T08:48:53.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.392 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.524636+0000", 2024-09-11T08:48:53.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.524636+0000", 2024-09-11T08:48:53.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.524635+0000", 2024-09-11T08:48:53.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:48:53.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.524639+0000", 2024-09-11T08:48:53.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9299999999999996e-06 2024-09-11T08:48:53.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.524646+0000", 2024-09-11T08:48:53.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3010000000000001e-06 2024-09-11T08:48:53.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.524650+0000", 2024-09-11T08:48:53.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8670000000000004e-06 2024-09-11T08:48:53.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.525249+0000", 2024-09-11T08:48:53.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059896899999999998 2024-09-11T08:48:53.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.525343+0000", 2024-09-11T08:48:53.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4055999999999995e-05 2024-09-11T08:48:53.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:48:53.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.525441+0000", 2024-09-11T08:48:53.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8171999999999995e-05 2024-09-11T08:48:53.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.529531+0000", 2024-09-11T08:48:53.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040898230000000002 2024-09-11T08:48:53.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.532940+0000", 2024-09-11T08:48:53.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034092390000000001 2024-09-11T08:48:53.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.633280+0000", 2024-09-11T08:48:53.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100339367 2024-09-11T08:48:53.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.633325+0000", 2024-09-11T08:48:53.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5751000000000003e-05 2024-09-11T08:48:53.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.633364+0000", 2024-09-11T08:48:53.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8347999999999999e-05 2024-09-11T08:48:53.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.398 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.398 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:196063 7.a 7:5b8820d7:::benchmark_data_smithi092_84565_object70535:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T08:48:53.398 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:43:50.525372+0000", 2024-09-11T08:48:53.398 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 302.830540558, 2024-09-11T08:48:53.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.108212899, 2024-09-11T08:48:53.398 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.398 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:53.399 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:53.399 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 196063 2024-09-11T08:48:53.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.399 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.525372+0000", 2024-09-11T08:48:53.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.525372+0000", 2024-09-11T08:48:53.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.525371+0000", 2024-09-11T08:48:53.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:48:53.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.525376+0000", 2024-09-11T08:48:53.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7960000000000003e-06 2024-09-11T08:48:53.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.525387+0000", 2024-09-11T08:48:53.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1164e-05 2024-09-11T08:48:53.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.525393+0000", 2024-09-11T08:48:53.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5169999999999999e-06 2024-09-11T08:48:53.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.525428+0000", 2024-09-11T08:48:53.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5302999999999997e-05 2024-09-11T08:48:53.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.525524+0000", 2024-09-11T08:48:53.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5866000000000007e-05 2024-09-11T08:48:53.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,5", 2024-09-11T08:48:53.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.525626+0000", 2024-09-11T08:48:53.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102409 2024-09-11T08:48:53.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.529525+0000", 2024-09-11T08:48:53.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038989150000000002 2024-09-11T08:48:53.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.532969+0000", 2024-09-11T08:48:53.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034434750000000001 2024-09-11T08:48:53.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.633521+0000", 2024-09-11T08:48:53.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100551895 2024-09-11T08:48:53.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.633557+0000", 2024-09-11T08:48:53.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6035000000000003e-05 2024-09-11T08:48:53.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:43:50.633585+0000", 2024-09-11T08:48:53.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8325000000000001e-05 2024-09-11T08:48:53.404 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.404 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.404 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.404 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:242089 7.a 7:562e1108:::benchmark_data_smithi092_84565_object116561:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-11T08:48:53.404 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:36.307092+0000", 2024-09-11T08:48:53.405 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 257.04882038900001, 2024-09-11T08:48:53.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.123527491, 2024-09-11T08:48:53.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.405 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:53.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:53.405 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 242089 2024-09-11T08:48:53.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.405 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.307092+0000", 2024-09-11T08:48:53.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.307092+0000", 2024-09-11T08:48:53.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.307091+0000", 2024-09-11T08:48:53.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:53.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.307096+0000", 2024-09-11T08:48:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2024-09-11T08:48:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.307108+0000", 2024-09-11T08:48:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1867e-05 2024-09-11T08:48:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.307113+0000", 2024-09-11T08:48:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.575e-06 2024-09-11T08:48:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.308471+0000", 2024-09-11T08:48:53.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001357673 2024-09-11T08:48:53.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.308577+0000", 2024-09-11T08:48:53.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106085 2024-09-11T08:48:53.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,5", 2024-09-11T08:48:53.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.308681+0000", 2024-09-11T08:48:53.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010356099999999999 2024-09-11T08:48:53.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.318220+0000", 2024-09-11T08:48:53.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095396560000000005 2024-09-11T08:48:53.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.318330+0000", 2024-09-11T08:48:53.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010933399999999999 2024-09-11T08:48:53.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430532+0000", 2024-09-11T08:48:53.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.112202185 2024-09-11T08:48:53.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430580+0000", 2024-09-11T08:48:53.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8353999999999998e-05 2024-09-11T08:48:53.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430620+0000", 2024-09-11T08:48:53.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9521000000000003e-05 2024-09-11T08:48:53.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.411 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.411 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:242102 7.2 7:41e8ab28:::benchmark_data_smithi092_84565_object116574:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-11T08:48:53.412 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:36.316413+0000", 2024-09-11T08:48:53.412 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 257.03949983299998, 2024-09-11T08:48:53.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11449888900000001, 2024-09-11T08:48:53.412 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.412 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:53.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:53.412 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 242102 2024-09-11T08:48:53.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.412 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.316413+0000", 2024-09-11T08:48:53.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.316413+0000", 2024-09-11T08:48:53.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.316411+0000", 2024-09-11T08:48:53.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:53.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.316418+0000", 2024-09-11T08:48:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5339999999999996e-06 2024-09-11T08:48:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.316430+0000", 2024-09-11T08:48:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2404e-05 2024-09-11T08:48:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.316436+0000", 2024-09-11T08:48:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5670000000000002e-06 2024-09-11T08:48:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.318239+0000", 2024-09-11T08:48:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018029249999999999 2024-09-11T08:48:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.318441+0000", 2024-09-11T08:48:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020176100000000001 2024-09-11T08:48:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:48:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.318536+0000", 2024-09-11T08:48:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5029000000000001e-05 2024-09-11T08:48:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.422464+0000", 2024-09-11T08:48:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103928796 2024-09-11T08:48:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430628+0000", 2024-09-11T08:48:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0081634259999999997 2024-09-11T08:48:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430835+0000", 2024-09-11T08:48:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020704 2024-09-11T08:48:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430878+0000", 2024-09-11T08:48:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2635000000000001e-05 2024-09-11T08:48:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430912+0000", 2024-09-11T08:48:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4292999999999998e-05 2024-09-11T08:48:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.14814.0:242104 7.6 e138/88 7:68d8e567:::benchmark_data_smithi092_84565_object116576:head v 138'14792, mlcod=138'14790)", 2024-09-11T08:48:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:36.319758+0000", 2024-09-11T08:48:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 257.03615479500002, 2024-09-11T08:48:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10554783399999999, 2024-09-11T08:48:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.319758+0000", 2024-09-11T08:48:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.319758+0000", 2024-09-11T08:48:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.319757+0000", 2024-09-11T08:48:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:48:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.319764+0000", 2024-09-11T08:48:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5690000000000001e-06 2024-09-11T08:48:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.319774+0000", 2024-09-11T08:48:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0151e-05 2024-09-11T08:48:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.319779+0000", 2024-09-11T08:48:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1710000000000001e-06 2024-09-11T08:48:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.420236+0000", 2024-09-11T08:48:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100456743 2024-09-11T08:48:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.420255+0000", 2024-09-11T08:48:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8771e-05 2024-09-11T08:48:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.423819+0000", 2024-09-11T08:48:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035640870000000001 2024-09-11T08:48:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.425306+0000", 2024-09-11T08:48:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001487143 2024-09-11T08:48:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:242110 7.a 7:5cca1602:::benchmark_data_smithi092_84565_object116582:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-11T08:48:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:36.320281+0000", 2024-09-11T08:48:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 257.03563179399998, 2024-09-11T08:48:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11067882599999999, 2024-09-11T08:48:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:48:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:48:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 242110 2024-09-11T08:48:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320281+0000", 2024-09-11T08:48:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320281+0000", 2024-09-11T08:48:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320280+0000", 2024-09-11T08:48:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.320286+0000", 2024-09-11T08:48:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.708e-06 2024-09-11T08:48:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.416710+0000", 2024-09-11T08:48:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.096423297000000005 2024-09-11T08:48:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.416723+0000", 2024-09-11T08:48:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3475999999999999e-05 2024-09-11T08:48:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.416874+0000", 2024-09-11T08:48:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015111899999999999 2024-09-11T08:48:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.416990+0000", 2024-09-11T08:48:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115858 2024-09-11T08:48:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,5", 2024-09-11T08:48:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.417103+0000", 2024-09-11T08:48:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113225 2024-09-11T08:48:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.422634+0000", 2024-09-11T08:48:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005530503 2024-09-11T08:48:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430633+0000", 2024-09-11T08:48:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079996649999999996 2024-09-11T08:48:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430680+0000", 2024-09-11T08:48:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6150999999999999e-05 2024-09-11T08:48:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430727+0000", 2024-09-11T08:48:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7664999999999998e-05 2024-09-11T08:48:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430960+0000", 2024-09-11T08:48:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023255500000000001 2024-09-11T08:48:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:32214 8.b 8:d802682b:::benchmark_data_smithi092_117541_object32213:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-11T08:48:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:45:42.063050+0000", 2024-09-11T08:48:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 191.29286248400001, 2024-09-11T08:48:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10608921, 2024-09-11T08:48:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:48:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:48:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32214 2024-09-11T08:48:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.063050+0000", 2024-09-11T08:48:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.063050+0000", 2024-09-11T08:48:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.063049+0000", 2024-09-11T08:48:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:48:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.063084+0000", 2024-09-11T08:48:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4705000000000001e-05 2024-09-11T08:48:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.136195+0000", 2024-09-11T08:48:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.073111192000000005 2024-09-11T08:48:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.136205+0000", 2024-09-11T08:48:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0196e-05 2024-09-11T08:48:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.137256+0000", 2024-09-11T08:48:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010505740000000001 2024-09-11T08:48:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.137379+0000", 2024-09-11T08:48:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012367300000000001 2024-09-11T08:48:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,4", 2024-09-11T08:48:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.137502+0000", 2024-09-11T08:48:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122378 2024-09-11T08:48:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.139579+0000", 2024-09-11T08:48:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002077017 2024-09-11T08:48:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.139602+0000", 2024-09-11T08:48:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3649e-05 2024-09-11T08:48:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.169066+0000", 2024-09-11T08:48:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029463742000000001 2024-09-11T08:48:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.169107+0000", 2024-09-11T08:48:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0364000000000002e-05 2024-09-11T08:48:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.169139+0000", 2024-09-11T08:48:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2903e-05 2024-09-11T08:48:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:33055 8.0 8:09190f81:::benchmark_data_smithi092_117541_object33054:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-11T08:48:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:45:42.600975+0000", 2024-09-11T08:48:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 190.754937402, 2024-09-11T08:48:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.107976037, 2024-09-11T08:48:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:48:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:48:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33055 2024-09-11T08:48:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.600975+0000", 2024-09-11T08:48:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.600975+0000", 2024-09-11T08:48:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.600974+0000", 2024-09-11T08:48:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:48:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.600999+0000", 2024-09-11T08:48:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5174999999999999e-05 2024-09-11T08:48:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.601012+0000", 2024-09-11T08:48:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3006e-05 2024-09-11T08:48:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.601017+0000", 2024-09-11T08:48:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6940000000000003e-06 2024-09-11T08:48:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.601050+0000", 2024-09-11T08:48:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3339e-05 2024-09-11T08:48:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.601124+0000", 2024-09-11T08:48:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4344000000000005e-05 2024-09-11T08:48:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,5", 2024-09-11T08:48:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.601205+0000", 2024-09-11T08:48:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0487000000000004e-05 2024-09-11T08:48:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.602259+0000", 2024-09-11T08:48:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010544319999999999 2024-09-11T08:48:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.602659+0000", 2024-09-11T08:48:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00039940700000000001 2024-09-11T08:48:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.708818+0000", 2024-09-11T08:48:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.106159745 2024-09-11T08:48:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.708913+0000", 2024-09-11T08:48:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4265999999999995e-05 2024-09-11T08:48:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:45:42.708951+0000", 2024-09-11T08:48:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8819999999999998e-05 2024-09-11T08:48:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:64797 8.f 8:f34e078c:::benchmark_data_smithi092_117541_object64796:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-11T08:48:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:46:04.809475+0000", 2024-09-11T08:48:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 168.54643799600001, 2024-09-11T08:48:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10852308500000001, 2024-09-11T08:48:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:48:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:48:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 64797 2024-09-11T08:48:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.809475+0000", 2024-09-11T08:48:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.809475+0000", 2024-09-11T08:48:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.809474+0000", 2024-09-11T08:48:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:48:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.809497+0000", 2024-09-11T08:48:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2663e-05 2024-09-11T08:48:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.809506+0000", 2024-09-11T08:48:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2890000000000003e-06 2024-09-11T08:48:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.809511+0000", 2024-09-11T08:48:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3319999999999999e-06 2024-09-11T08:48:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.812053+0000", 2024-09-11T08:48:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025415260000000001 2024-09-11T08:48:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.812169+0000", 2024-09-11T08:48:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115795 2024-09-11T08:48:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:48:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.812313+0000", 2024-09-11T08:48:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014435399999999999 2024-09-11T08:48:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.816314+0000", 2024-09-11T08:48:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040008719999999999 2024-09-11T08:48:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.821222+0000", 2024-09-11T08:48:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049085420000000001 2024-09-11T08:48:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.917904+0000", 2024-09-11T08:48:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.096681794000000001 2024-09-11T08:48:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.917953+0000", 2024-09-11T08:48:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8915999999999997e-05 2024-09-11T08:48:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.917998+0000", 2024-09-11T08:48:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4832e-05 2024-09-11T08:48:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:64817 8.f 8:ffe75452:::benchmark_data_smithi092_117541_object64816:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-11T08:48:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:46:04.823173+0000", 2024-09-11T08:48:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 168.53273964900001, 2024-09-11T08:48:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104584722, 2024-09-11T08:48:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:48:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:48:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 64817 2024-09-11T08:48:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.823173+0000", 2024-09-11T08:48:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.823173+0000", 2024-09-11T08:48:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.823172+0000", 2024-09-11T08:48:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.823197+0000", 2024-09-11T08:48:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5678e-05 2024-09-11T08:48:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.823207+0000", 2024-09-11T08:48:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3319999999999998e-06 2024-09-11T08:48:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.823211+0000", 2024-09-11T08:48:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1269999999999996e-06 2024-09-11T08:48:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.823244+0000", 2024-09-11T08:48:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3065999999999998e-05 2024-09-11T08:48:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.823377+0000", 2024-09-11T08:48:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013257300000000001 2024-09-11T08:48:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:48:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.823494+0000", 2024-09-11T08:48:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011754 2024-09-11T08:48:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.828243+0000", 2024-09-11T08:48:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047484490000000001 2024-09-11T08:48:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.828496+0000", 2024-09-11T08:48:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025384700000000002 2024-09-11T08:48:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.927329+0000", 2024-09-11T08:48:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.098832429999999999 2024-09-11T08:48:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.927718+0000", 2024-09-11T08:48:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000389381 2024-09-11T08:48:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:04.927758+0000", 2024-09-11T08:48:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9595000000000003e-05 2024-09-11T08:48:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:125090 8.f 8:fb475083:::benchmark_data_smithi092_117541_object5129:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e184)", 2024-09-11T08:48:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:04.928024+0000", 2024-09-11T08:48:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 108.427888861, 2024-09-11T08:48:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.130102088, 2024-09-11T08:48:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:48:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:48:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 125090 2024-09-11T08:48:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928024+0000", 2024-09-11T08:48:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928024+0000", 2024-09-11T08:48:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928023+0000", 2024-09-11T08:48:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928028+0000", 2024-09-11T08:48:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5539999999999997e-06 2024-09-11T08:48:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928042+0000", 2024-09-11T08:48:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.359e-05 2024-09-11T08:48:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928047+0000", 2024-09-11T08:48:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5820000000000001e-06 2024-09-11T08:48:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928163+0000", 2024-09-11T08:48:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116062 2024-09-11T08:48:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928549+0000", 2024-09-11T08:48:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000385604 2024-09-11T08:48:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:48:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928912+0000", 2024-09-11T08:48:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036309800000000001 2024-09-11T08:48:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.936819+0000", 2024-09-11T08:48:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079072350000000003 2024-09-11T08:48:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.938487+0000", 2024-09-11T08:48:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016671349999999999 2024-09-11T08:48:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:05.058044+0000", 2024-09-11T08:48:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.119557371 2024-09-11T08:48:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:05.058090+0000", 2024-09-11T08:48:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6236000000000003e-05 2024-09-11T08:48:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:05.058126+0000", 2024-09-11T08:48:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5846000000000002e-05 2024-09-11T08:48:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:125110 8.f 8:f74db299:::benchmark_data_smithi092_117541_object5149:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e184)", 2024-09-11T08:48:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:04.948134+0000", 2024-09-11T08:48:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 108.40777901600001, 2024-09-11T08:48:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.110087273, 2024-09-11T08:48:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:48:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:48:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 125110 2024-09-11T08:48:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.948134+0000", 2024-09-11T08:48:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.948134+0000", 2024-09-11T08:48:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.948133+0000", 2024-09-11T08:48:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:48:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.948138+0000", 2024-09-11T08:48:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2059999999999998e-06 2024-09-11T08:48:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.948149+0000", 2024-09-11T08:48:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0744000000000001e-05 2024-09-11T08:48:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.948155+0000", 2024-09-11T08:48:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9549999999999997e-06 2024-09-11T08:48:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.948194+0000", 2024-09-11T08:48:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9422999999999998e-05 2024-09-11T08:48:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.948298+0000", 2024-09-11T08:48:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010388 2024-09-11T08:48:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:48:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.948402+0000", 2024-09-11T08:48:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010451 2024-09-11T08:48:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.956621+0000", 2024-09-11T08:48:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082187430000000006 2024-09-11T08:48:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.957090+0000", 2024-09-11T08:48:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046856099999999999 2024-09-11T08:48:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:05.058156+0000", 2024-09-11T08:48:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101066143 2024-09-11T08:48:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:05.058190+0000", 2024-09-11T08:48:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4301999999999998e-05 2024-09-11T08:48:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:05.058221+0000", 2024-09-11T08:48:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0781999999999998e-05 2024-09-11T08:48:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:125115 8.f 8:f1aad719:::benchmark_data_smithi092_117541_object5154:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e184)", 2024-09-11T08:48:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:04.950964+0000", 2024-09-11T08:48:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 108.404949132, 2024-09-11T08:48:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.107349642, 2024-09-11T08:48:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 125115 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.950964+0000", 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.950964+0000", 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.950962+0000", 2024-09-11T08:48:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.950969+0000", 2024-09-11T08:48:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3459999999999999e-06 2024-09-11T08:48:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.950980+0000", 2024-09-11T08:48:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1691e-05 2024-09-11T08:48:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.950987+0000", 2024-09-11T08:48:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1890000000000004e-06 2024-09-11T08:48:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.953367+0000", 2024-09-11T08:48:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023799089999999999 2024-09-11T08:48:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.953523+0000", 2024-09-11T08:48:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015653600000000001 2024-09-11T08:48:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:48:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.953627+0000", 2024-09-11T08:48:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104139 2024-09-11T08:48:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.956626+0000", 2024-09-11T08:48:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029984550000000001 2024-09-11T08:48:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.960401+0000", 2024-09-11T08:48:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037751719999999998 2024-09-11T08:48:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:05.058251+0000", 2024-09-11T08:48:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.097849729999999996 2024-09-11T08:48:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:05.058283+0000", 2024-09-11T08:48:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2678000000000001e-05 2024-09-11T08:48:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:05.058313+0000", 2024-09-11T08:48:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0003000000000001e-05 2024-09-11T08:48:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:183312 8.b 8:d751352d:::benchmark_data_smithi092_117541_object63351:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-11T08:48:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:58.457317+0000", 2024-09-11T08:48:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.898595381, 2024-09-11T08:48:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10307928500000001, 2024-09-11T08:48:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:48:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:48:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:48:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:48:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:48:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 183312 2024-09-11T08:48:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:48:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:48:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:58.457317+0000", 2024-09-11T08:48:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:48:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:58.457317+0000", 2024-09-11T08:48:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:48:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:48:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:58.457316+0000", 2024-09-11T08:48:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:48:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:48:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:58.457323+0000", 2024-09-11T08:48:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.562e-06 2024-09-11T08:48:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:48:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:58.457335+0000", 2024-09-11T08:48:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1865000000000001e-05 2024-09-11T08:48:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:48:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:58.457340+0000", 2024-09-11T08:48:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2399999999999998e-06 2024-09-11T08:48:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:48:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:58.457404+0000", 2024-09-11T08:48:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4117999999999998e-05 2024-09-11T08:48:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:48:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:58.457527+0000", 2024-09-11T08:48:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012285900000000001 2024-09-11T08:48:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,4", 2024-09-11T08:48:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:58.457626+0000", 2024-09-11T08:48:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8954000000000002e-05 2024-09-11T08:48:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:58.463144+0000", 2024-09-11T08:48:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055185479999999999 2024-09-11T08:48:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:48:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:58.465835+0000", 2024-09-11T08:48:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026904939999999999 2024-09-11T08:48:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:48:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:58.560097+0000", 2024-09-11T08:48:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.094262703000000003 2024-09-11T08:48:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:48:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:58.560353+0000", 2024-09-11T08:48:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025560900000000001 2024-09-11T08:48:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:48:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:48:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:48:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:58.560397+0000", 2024-09-11T08:48:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3608e-05 2024-09-11T08:48:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:48:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:48:53.481 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:48:53.495 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'unique_pool_1' removed 2024-09-11T08:48:53.501 INFO:teuthology.orchestra.run.smithi092.stderr:set 6.19 pg_upmap_items mapping to [6->5,4->3,7->1] 2024-09-11T08:48:53.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:53.507 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:53.507 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:53.608 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:53.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:53.632 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:53.632 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:53.729 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:53 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1560648284' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:48:53.729 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:53 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4225896075' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.19", "id": [6, 5, 4, 3, 7, 1]}]: dispatch 2024-09-11T08:48:53.729 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:53 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.19", "id": [6, 5, 4, 3, 7, 1]}]: dispatch 2024-09-11T08:48:53.730 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:53 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1560648284' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:48:53.730 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:53 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4225896075' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.19", "id": [6, 5, 4, 3, 7, 1]}]: dispatch 2024-09-11T08:48:53.730 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:53 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.19", "id": [6, 5, 4, 3, 7, 1]}]: dispatch 2024-09-11T08:48:53.733 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:53.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:53.755 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:53.756 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:53.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:53.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:53.879 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:53.879 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:53 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1560648284' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:48:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:53 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4225896075' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.19", "id": [6, 5, 4, 3, 7, 1]}]: dispatch 2024-09-11T08:48:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:53 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.19", "id": [6, 5, 4, 3, 7, 1]}]: dispatch 2024-09-11T08:48:53.980 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:54.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:54.002 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:54.002 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:54.102 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:54.128 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:54.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:54.129 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:54.230 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:54.255 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:54.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:54.256 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:54.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:54.378 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:54.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:54.379 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:54.480 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:54.503 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:54.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:54.504 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:54.605 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:54.628 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:54.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:54.629 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:54.730 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:54.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:54.753 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:54.753 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:54.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:54 smithi092 ceph-mon[41114]: pgmap v425: 148 pgs: 148 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 62 KiB/s rd, 0 B/s wr, 596 op/s 2024-09-11T08:48:54.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:54 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1560648284' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:48:54.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:54 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "6.19", "id": [6, 5, 4, 3, 7, 1]}]': finished 2024-09-11T08:48:54.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:54 smithi092 ceph-mon[41114]: osdmap e203: 8 total, 8 up, 8 in 2024-09-11T08:48:54.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:54 smithi092 ceph-mon[29647]: pgmap v425: 148 pgs: 148 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 62 KiB/s rd, 0 B/s wr, 596 op/s 2024-09-11T08:48:54.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:54 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1560648284' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:48:54.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:54 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "6.19", "id": [6, 5, 4, 3, 7, 1]}]': finished 2024-09-11T08:48:54.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:54 smithi092 ceph-mon[29647]: osdmap e203: 8 total, 8 up, 8 in 2024-09-11T08:48:54.854 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:54.877 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:54.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:54.878 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:54 smithi145 ceph-mon[33384]: pgmap v425: 148 pgs: 148 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 62 KiB/s rd, 0 B/s wr, 596 op/s 2024-09-11T08:48:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:54 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1560648284' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:48:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:54 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "6.19", "id": [6, 5, 4, 3, 7, 1]}]': finished 2024-09-11T08:48:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:54 smithi145 ceph-mon[33384]: osdmap e203: 8 total, 8 up, 8 in 2024-09-11T08:48:54.979 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:55.000 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:55.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:55.001 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:55.102 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:55.123 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:55.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:55.124 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:55.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:55.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:55.248 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:55.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:55.373 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:55.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:55.374 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:55.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:55.496 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:55.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:55.497 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:55.598 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:55.620 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:55.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:55.621 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:55.722 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:55.743 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:55.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:55.744 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:55.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:55.863 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-11T08:48:55.867 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:55.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:55.868 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:55.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:55.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:55.991 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:56.092 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:56.104 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:48:56.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:56.114 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:56.114 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:56.215 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:56.236 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:56.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:56.237 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:56.338 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:56.360 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:56.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:56.361 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:56.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:56.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:56.485 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:56.586 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:56.607 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:56.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:56.608 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:56.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:56.731 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:56.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:56.732 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:56.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:56 smithi092 ceph-mon[29647]: pgmap v427: 132 pgs: 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 29 KiB/s rd, 28 op/s 2024-09-11T08:48:56.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:56 smithi092 ceph-mon[41114]: pgmap v427: 132 pgs: 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 29 KiB/s rd, 28 op/s 2024-09-11T08:48:56.834 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:48:56.855 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:56.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:56.856 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:48:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:56 smithi145 ceph-mon[33384]: pgmap v427: 132 pgs: 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 29 KiB/s rd, 28 op/s 2024-09-11T08:48:56.912 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:48:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:48:56.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:56.978 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:56.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:56.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:57.080 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:57.102 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:57.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:57.103 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:57.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:57.225 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:57.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:57.226 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:57.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:57.348 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:57.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:57.349 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:57.450 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:57.472 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:57.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:57.473 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:57.574 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:57.595 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:57.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:57.596 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:57.697 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:57.723 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:57.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:57.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:57.735 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:57 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:57.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:57 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:57 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:48:57.825 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:57.847 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:57.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:57.849 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:57.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:48:57.975 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:57.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:57.976 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:48:57.980 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:48:57.980 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T08:48:57.980 INFO:teuthology.orchestra.run.smithi145.stdout: "num_ops": 0 2024-09-11T08:48:57.980 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:48:58.077 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:58.099 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:58.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:58.100 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:58.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:58.226 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:58.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:58.227 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:58.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:58.352 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:58.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:58.353 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:58.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:58.478 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:58.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:58.479 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:58.580 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:48:58.601 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:58.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:58.603 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:48:58.607 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:48:58 smithi145 ceph-mon[33384]: pgmap v428: 132 pgs: 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 25 op/s 2024-09-11T08:48:58.631 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-11T08:48:58.703 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:58.727 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:58.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:58.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:48:58 smithi092 ceph-mon[29647]: pgmap v428: 132 pgs: 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 25 op/s 2024-09-11T08:48:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:48:58 smithi092 ceph-mon[41114]: pgmap v428: 132 pgs: 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 25 op/s 2024-09-11T08:48:58.829 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:58.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:58.852 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:58.852 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:58.913 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:48:58.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:58.978 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:58.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:58.980 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:59.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:59.103 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:59.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:59.104 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:59.205 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:59.227 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:59.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:59.228 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:59.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:48:59.353 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:59.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:59.354 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:48:59.455 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:48:59.521 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-11T08:48:59.522 INFO:tasks.radosbench:Beginning radosbench... 2024-09-11T08:48:59.522 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_2 2024-09-11T08:48:59.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool create unique_pool_2 16 2024-09-11T08:48:59.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:59.524 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:59.524 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:48:59.624 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:48:59.646 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:48:59.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:59.647 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:48:59.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:48:59.772 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:59.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:59.773 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:48:59.874 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:48:59.897 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:48:59.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:48:59.898 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:48:59.998 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:00.020 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:00.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:00.021 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:00.122 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:00.144 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:00.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:00.145 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:00.245 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:00.266 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:00.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:00.267 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:00.367 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:00.391 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:00.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:00.392 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:00.492 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:00.507 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:49:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:00.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:00.520 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:00.524 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:00 smithi092 ceph-mon[29647]: pgmap v429: 132 pgs: 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 23 op/s 2024-09-11T08:49:00.524 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:00 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/290872774' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:49:00.524 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:00 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:00.524 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:00.524 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:00 smithi092 ceph-mon[41114]: pgmap v429: 132 pgs: 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 23 op/s 2024-09-11T08:49:00.525 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:00 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/290872774' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:49:00.525 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:00 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:00.525 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:00.617 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:49:00.618 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T08:49:00.618 INFO:teuthology.orchestra.run.smithi145.stdout: "complaint_time": 30, 2024-09-11T08:49:00.618 INFO:teuthology.orchestra.run.smithi145.stdout: "num_blocked_ops": 0 2024-09-11T08:49:00.618 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:49:00.621 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:00.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:00.644 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:00.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:00.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:00 smithi145 ceph-mon[33384]: pgmap v429: 132 pgs: 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 23 op/s 2024-09-11T08:49:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:00 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/290872774' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:49:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:00 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:00.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:00.767 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:00.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:00.768 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:00.869 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:00.890 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:00.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:00.891 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:00.942 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:49:00.942 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:49:00.942 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:49:00.942 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:49:00.942 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:49:00.943 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:49:00.943 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:49:00.943 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:49:00.943 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:49:00.943 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:49:00.943 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:49:00.943 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:49:00.943 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:49:00.943 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:49:00.944 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:49:00.944 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:49:00.992 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:01.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:01.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:01.115 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:01.137 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:01.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:01.138 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:01.239 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:01.260 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:01.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:01.263 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:01.287 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-11T08:49:01.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:01.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:01.386 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:01.386 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:01.487 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:01.510 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:01.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:01.511 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:01.514 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:49:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:49:01] "GET /metrics HTTP/1.1" 200 221872 "" "Prometheus/2.43.0" 2024-09-11T08:49:01.534 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:49:01.612 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:01.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:01.635 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:01.635 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:01.726 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:01.736 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:01.757 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:01.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:01.758 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:01.790 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:01 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:49:01.790 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:01 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:49:01.790 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:01 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:49:01.790 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:01 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/290872774' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:49:01.790 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:01 smithi145 ceph-mon[33384]: osdmap e204: 8 total, 8 up, 8 in 2024-09-11T08:49:01.790 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:01 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4059238975' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:01.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:01 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:49:01.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:01 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:49:01.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:01 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:49:01.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:01 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/290872774' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:49:01.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:01 smithi092 ceph-mon[29647]: osdmap e204: 8 total, 8 up, 8 in 2024-09-11T08:49:01.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:01 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4059238975' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:01 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:49:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:01 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:49:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:01 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:49:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:01 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/290872774' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:49:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:01 smithi092 ceph-mon[41114]: osdmap e204: 8 total, 8 up, 8 in 2024-09-11T08:49:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:01 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4059238975' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:01.859 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:01.880 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:01.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:01.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:01.981 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:02.004 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:02.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:02.005 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:02.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:02.127 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:02.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:02.128 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:02.229 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:02.251 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:02.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:02.252 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:02.353 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:02.375 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:02.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:02.376 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:02.477 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:02.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:02.500 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:02.501 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:02.602 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:02.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:02.625 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:02.626 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:02 smithi145 ceph-mon[33384]: pgmap v431: 132 pgs: 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:49:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:02 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:49:02.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:02.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:02.751 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:02.751 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:02.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:02 smithi092 ceph-mon[29647]: pgmap v431: 132 pgs: 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:49:02.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:02 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:49:02.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:02 smithi092 ceph-mon[41114]: pgmap v431: 132 pgs: 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:49:02.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:02 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:49:02.852 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:02.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:02.875 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:02.875 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:02.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:02.977 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:49:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:49:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:49:02.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:02.999 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:02.999 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:03.100 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:03.122 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:03.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:03.123 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:03.216 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:49:03.216 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:49:03.224 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:03.245 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:03.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:03.246 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:03.347 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:03.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:03.370 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:03.370 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:03.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:03.494 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:03.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:03.495 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:03.595 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:03.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:03.618 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:03.619 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:03.664 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:49:03.706 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:49:03.706 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:49:03.707 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:49:03.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd out 0 2024-09-11T08:49:03.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:03.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:03.775 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:03.775 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:03.794 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:03 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:49:03.794 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:03 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:49:03.794 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:03 smithi092 ceph-mon[29647]: osdmap e205: 8 total, 8 up, 8 in 2024-09-11T08:49:03.794 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:03 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:49:03.794 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:03 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:49:03.794 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:03 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:03.794 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:03 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:49:03.795 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:03 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:49:03.795 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:03 smithi092 ceph-mon[41114]: osdmap e205: 8 total, 8 up, 8 in 2024-09-11T08:49:03.795 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:03 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:49:03.795 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:03 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:49:03.795 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:03 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:03.828 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:03 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:49:03.829 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:03 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:49:03.829 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:03 smithi145 ceph-mon[33384]: osdmap e205: 8 total, 8 up, 8 in 2024-09-11T08:49:03.829 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:03 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:49:03.829 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:03 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:49:03.829 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:03 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:03.853 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-11T08:49:03.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:03.897 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:03.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:03.898 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:03.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:04.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:04.021 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:04.021 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:04.094 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:49:04.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:04.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:04.145 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:04.246 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:04.260 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:04.262 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:04.268 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:04.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:04.269 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:04.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:49:04.369 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:04.391 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:04.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:04.392 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:04.493 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:04.514 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:04.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:04.515 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:04.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:04.639 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:04.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:04.640 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:04.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:04.762 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:04.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:04.763 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:04.864 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:04.865 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'unique_pool_2' created 2024-09-11T08:49:04.886 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:04.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:04.887 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:04 smithi145 ceph-mon[33384]: pgmap v433: 132 pgs: 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:04.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:05.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:05.009 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:05.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:05.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:04 smithi092 ceph-mon[41114]: pgmap v433: 132 pgs: 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:05.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:04 smithi092 ceph-mon[29647]: pgmap v433: 132 pgs: 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:05.110 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:05.132 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:05.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:05.133 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:05.234 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:05.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:05.255 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:05.255 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:05.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:05.379 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:05.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:05.380 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:05.481 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:05.503 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:05.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:05.504 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:05.604 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:05.626 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:05.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:05.627 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:05.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:05.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:05.751 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:05.752 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:05.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:05 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/314716197' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-11T08:49:05.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:05 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-11T08:49:05.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:05 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-11T08:49:05.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:05 smithi092 ceph-mon[29647]: osdmap e206: 8 total, 8 up, 8 in 2024-09-11T08:49:05.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:05 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/314716197' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-11T08:49:05.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:05 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-11T08:49:05.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:05 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-11T08:49:05.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:05 smithi092 ceph-mon[41114]: osdmap e206: 8 total, 8 up, 8 in 2024-09-11T08:49:05.853 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:05.875 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:05 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/314716197' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-11T08:49:05.875 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:05 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-11T08:49:05.875 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:05 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-11T08:49:05.876 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:05 smithi145 ceph-mon[33384]: osdmap e206: 8 total, 8 up, 8 in 2024-09-11T08:49:05.879 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:05.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:05.881 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:05.881 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:49:05.881 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:49:05.982 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:06.004 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:06.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:06.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:06.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:06.128 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:06.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:06.129 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:06.229 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:06.250 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:06.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:06.252 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:06.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:06.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:06.376 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:06.377 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:06.477 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:06.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:06.502 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:06.503 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:06.604 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:06.628 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:06.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:06.629 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:06.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2024-09-11T08:49:06.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:06.737 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-11T08:49:06.751 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:06.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:06.753 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:06.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:06.877 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:06.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:06.878 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:06.911 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:49:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:49:06.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:06 smithi145 ceph-mon[33384]: pgmap v435: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:06.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:06 smithi145 ceph-mon[33384]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:49:06.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:06 smithi145 ceph-mon[33384]: osdmap e207: 8 total, 8 up, 8 in 2024-09-11T08:49:06.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:06.980 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:49:07.000 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:07.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:07.001 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:07.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:06 smithi092 ceph-mon[29647]: pgmap v435: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:07.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:06 smithi092 ceph-mon[29647]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:49:07.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:06 smithi092 ceph-mon[29647]: osdmap e207: 8 total, 8 up, 8 in 2024-09-11T08:49:07.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:06 smithi092 ceph-mon[41114]: pgmap v435: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:49:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:06 smithi092 ceph-mon[41114]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:49:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:06 smithi092 ceph-mon[41114]: osdmap e207: 8 total, 8 up, 8 in 2024-09-11T08:49:07.101 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:07.122 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:07.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:07.123 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:07.224 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:07.247 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:07.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:07.248 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:07.333 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:07.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:07.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:07.372 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:07.372 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:07.473 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:07.495 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:07.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:07.496 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:07.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:07.618 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:07.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:07.619 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:07.720 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:07.742 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:07.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:07.743 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:07.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:07.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:07.866 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:07.866 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:07.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:07 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:07.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:07 smithi145 ceph-mon[33384]: osdmap e208: 8 total, 8 up, 8 in 2024-09-11T08:49:07.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:07.990 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:07.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:07.992 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:07.992 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:07 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:07.992 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:07 smithi092 ceph-mon[41114]: osdmap e208: 8 total, 8 up, 8 in 2024-09-11T08:49:07.992 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:07 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:07.992 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:07 smithi092 ceph-mon[29647]: osdmap e208: 8 total, 8 up, 8 in 2024-09-11T08:49:08.092 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:08.114 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:08.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:08.116 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:08.217 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:08.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:08.241 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:08.241 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:08.342 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:08.363 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:08.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:08.364 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:08.465 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:08.486 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:08.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:08.487 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:08.588 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:08.610 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:08.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:08.611 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:08.639 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:08 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a[29624]: 2024-09-11T08:49:08.414+0000 7f9ae2e34640 -1 mon.a@0(leader).osd e209 definitely_dead 0 2024-09-11T08:49:08.696 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:49:08.696 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:49:08.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:08.715 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:08.734 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:08.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:08.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:08.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:08.858 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:08.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:08.859 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:08.883 INFO:teuthology.orchestra.run.smithi092.stderr:marked out osd.0. 2024-09-11T08:49:08.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:08 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:49:08.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:08 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:49:08.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:08 smithi092 ceph-mon[29647]: pgmap v438: 148 pgs: 2 creating+activating, 146 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:49:08.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:08 smithi092 ceph-mon[29647]: osdmap e209: 8 total, 8 up, 8 in 2024-09-11T08:49:08.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:08 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/488364570' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-09-11T08:49:08.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:08 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-09-11T08:49:08.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:08 smithi092 ceph-mon[29647]: Client client.admin marked osd.0 out, while it was still marked up 2024-09-11T08:49:08.906 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:08 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:49:08.906 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:08 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:49:08.906 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:08 smithi092 ceph-mon[41114]: pgmap v438: 148 pgs: 2 creating+activating, 146 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:49:08.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:08 smithi092 ceph-mon[41114]: osdmap e209: 8 total, 8 up, 8 in 2024-09-11T08:49:08.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:08 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/488364570' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-09-11T08:49:08.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:08 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-09-11T08:49:08.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:08 smithi092 ceph-mon[41114]: Client client.admin marked osd.0 out, while it was still marked up 2024-09-11T08:49:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:08 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:49:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:08 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:49:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:08 smithi145 ceph-mon[33384]: pgmap v438: 148 pgs: 2 creating+activating, 146 active+clean; 582 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:49:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:08 smithi145 ceph-mon[33384]: osdmap e209: 8 total, 8 up, 8 in 2024-09-11T08:49:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:08 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/488364570' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-09-11T08:49:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:08 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-09-11T08:49:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:08 smithi145 ceph-mon[33384]: Client client.admin marked osd.0 out, while it was still marked up 2024-09-11T08:49:08.959 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:08.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:08.982 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:08.982 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:09.083 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:09.104 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:09.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:09.106 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:09.207 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:09.228 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:09.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:09.229 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:09.315 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:49:09.315 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:49:09.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:09.331 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:49:09.331 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:49:09.347 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:49:09.347 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:49:09.354 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:09.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:09.355 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:09.358 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-11T08:49:09.365 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:49:09.365 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:49:09.380 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:49:09.381 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:49:09.396 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:49:09.397 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:49:09.412 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:49:09.412 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:49:09.427 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:49:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:49:09.456 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:09.478 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:09.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:09.479 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:09.579 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:09.602 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:09.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:09.603 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:09.603 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:49:09.704 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:09.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:09.727 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:09.727 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:09.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:09.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:09.850 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:09.851 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:09.951 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:09.972 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:09.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:09.973 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:10.054 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:09 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["0"]}]': finished 2024-09-11T08:49:10.054 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:09 smithi145 ceph-mon[33384]: osdmap e210: 8 total, 8 up, 7 in 2024-09-11T08:49:10.054 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:09 smithi145 ceph-mon[33384]: pgmap v441: 148 pgs: 2 creating+activating, 146 active+clean; 582 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:10.054 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:09 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1454084258' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:10.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:09 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["0"]}]': finished 2024-09-11T08:49:10.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:09 smithi092 ceph-mon[29647]: osdmap e210: 8 total, 8 up, 7 in 2024-09-11T08:49:10.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:09 smithi092 ceph-mon[29647]: pgmap v441: 148 pgs: 2 creating+activating, 146 active+clean; 582 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:10.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:09 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1454084258' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:10.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:09 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["0"]}]': finished 2024-09-11T08:49:10.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:09 smithi092 ceph-mon[41114]: osdmap e210: 8 total, 8 up, 7 in 2024-09-11T08:49:10.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:09 smithi092 ceph-mon[41114]: pgmap v441: 148 pgs: 2 creating+activating, 146 active+clean; 582 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:49:10.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:09 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1454084258' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:10.074 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:10.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:10.096 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:10.096 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:10.197 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:10.219 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:10.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:10.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:10.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:10.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:10.343 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:10.343 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:10.444 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:10.466 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:10.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:10.467 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:10.568 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:10.589 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:10.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:10.590 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:10.691 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:10.713 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:10.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:10.714 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:10.816 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:10.837 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:10.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:10.838 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:10.939 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:10.961 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:10.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:10.962 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:11.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:11.085 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:11.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:11.089 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:11.154 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:10 smithi145 ceph-mon[33384]: osdmap e211: 8 total, 8 up, 7 in 2024-09-11T08:49:11.189 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:11.211 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:11.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:11.212 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:11.221 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:10 smithi092 ceph-mon[29647]: osdmap e211: 8 total, 8 up, 7 in 2024-09-11T08:49:11.222 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:10 smithi092 ceph-mon[41114]: osdmap e211: 8 total, 8 up, 7 in 2024-09-11T08:49:11.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:11.336 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:11.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:11.337 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:11.403 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:49:11.403 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T08:49:11.403 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T08:49:11.403 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:49:11.404 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.404 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:143095 7.0 7:00694a13:::benchmark_data_smithi092_84565_object17567:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-11T08:49:11.404 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:00.351549+0000", 2024-09-11T08:49:11.404 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 371.039798529, 2024-09-11T08:49:11.404 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.125918113, 2024-09-11T08:49:11.404 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.404 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.404 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.404 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:11.404 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:11.404 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 143095 2024-09-11T08:49:11.404 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.405 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.405 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.405 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.405 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.351549+0000", 2024-09-11T08:49:11.405 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.405 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.405 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.405 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.405 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.351549+0000", 2024-09-11T08:49:11.405 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.405 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.405 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.405 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.405 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.351547+0000", 2024-09-11T08:49:11.406 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:49:11.406 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.406 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.406 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.406 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.351554+0000", 2024-09-11T08:49:11.406 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.7399999999999998e-06 2024-09-11T08:49:11.406 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.406 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.406 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.406 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.351568+0000", 2024-09-11T08:49:11.406 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3596e-05 2024-09-11T08:49:11.406 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.406 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.407 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.407 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.351573+0000", 2024-09-11T08:49:11.407 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5840000000000002e-06 2024-09-11T08:49:11.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.407 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.407 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.407 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.387857+0000", 2024-09-11T08:49:11.407 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.036283637000000001 2024-09-11T08:49:11.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.407 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.407 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.407 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.387976+0000", 2024-09-11T08:49:11.407 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000118647 2024-09-11T08:49:11.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:49:11.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.388117+0000", 2024-09-11T08:49:11.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014104699999999999 2024-09-11T08:49:11.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.390432+0000", 2024-09-11T08:49:11.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.002314956 2024-09-11T08:49:11.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.459292+0000", 2024-09-11T08:49:11.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.068860307999999995 2024-09-11T08:49:11.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.409 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.477379+0000", 2024-09-11T08:49:11.409 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.018086787999999999 2024-09-11T08:49:11.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.409 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.477430+0000", 2024-09-11T08:49:11.409 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1319000000000001e-05 2024-09-11T08:49:11.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.410 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.477467+0000", 2024-09-11T08:49:11.410 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7224000000000001e-05 2024-09-11T08:49:11.410 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.410 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.410 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.410 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.410 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.410 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:143096 7.0 7:057334a6:::benchmark_data_smithi092_84565_object17568:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-11T08:49:11.410 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:00.352663+0000", 2024-09-11T08:49:11.410 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 371.03868457800002, 2024-09-11T08:49:11.410 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.13865506699999999, 2024-09-11T08:49:11.410 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.410 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.410 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.411 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:11.411 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:11.411 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 143096 2024-09-11T08:49:11.411 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.411 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.411 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.411 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.411 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.352663+0000", 2024-09-11T08:49:11.411 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.411 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.411 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.411 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.411 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.352663+0000", 2024-09-11T08:49:11.411 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.411 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.412 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.412 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.412 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.352662+0000", 2024-09-11T08:49:11.412 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:11.412 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.412 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.412 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.412 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.352668+0000", 2024-09-11T08:49:11.412 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7010000000000001e-06 2024-09-11T08:49:11.412 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.412 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.412 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.412 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.352682+0000", 2024-09-11T08:49:11.413 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4144999999999999e-05 2024-09-11T08:49:11.413 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.413 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.413 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.413 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.352688+0000", 2024-09-11T08:49:11.413 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2500000000000003e-06 2024-09-11T08:49:11.413 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.413 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.413 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.413 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.390728+0000", 2024-09-11T08:49:11.413 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.038040167999999999 2024-09-11T08:49:11.413 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.413 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.414 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.414 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.390813+0000", 2024-09-11T08:49:11.414 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.4892e-05 2024-09-11T08:49:11.414 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.414 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.414 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:49:11.414 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.390895+0000", 2024-09-11T08:49:11.414 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.1382000000000001e-05 2024-09-11T08:49:11.414 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.414 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.414 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.414 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.396111+0000", 2024-09-11T08:49:11.414 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0052167530000000002 2024-09-11T08:49:11.414 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.415 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.415 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.415 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.459328+0000", 2024-09-11T08:49:11.415 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.063217086000000006 2024-09-11T08:49:11.415 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.415 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.415 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.415 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.491246+0000", 2024-09-11T08:49:11.415 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.031917883000000001 2024-09-11T08:49:11.415 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.415 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.415 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.415 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.491279+0000", 2024-09-11T08:49:11.415 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3124000000000003e-05 2024-09-11T08:49:11.415 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.416 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.416 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.416 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.491318+0000", 2024-09-11T08:49:11.416 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8785999999999998e-05 2024-09-11T08:49:11.416 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.416 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.416 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.416 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.416 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.416 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:143102 7.0 7:0af89bd2:::benchmark_data_smithi092_84565_object17574:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-11T08:49:11.416 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:00.360464+0000", 2024-09-11T08:49:11.416 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 371.03088409499998, 2024-09-11T08:49:11.416 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.130978655, 2024-09-11T08:49:11.416 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.417 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.417 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.417 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:11.417 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:11.417 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 143102 2024-09-11T08:49:11.417 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.417 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.417 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.417 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.417 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.360464+0000", 2024-09-11T08:49:11.417 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.417 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.417 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.418 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.418 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.360464+0000", 2024-09-11T08:49:11.418 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.418 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.418 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.418 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.418 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.360462+0000", 2024-09-11T08:49:11.418 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:49:11.418 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.418 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.418 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.418 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.360468+0000", 2024-09-11T08:49:11.418 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.066e-06 2024-09-11T08:49:11.418 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.360479+0000", 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0706999999999999e-05 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.360484+0000", 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1630000000000004e-06 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.393231+0000", 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.032746738999999997 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.419 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.420 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.420 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.393344+0000", 2024-09-11T08:49:11.420 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011286 2024-09-11T08:49:11.420 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.420 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.420 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:49:11.420 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.393451+0000", 2024-09-11T08:49:11.420 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000106708 2024-09-11T08:49:11.420 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.421 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.421 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.421 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.396120+0000", 2024-09-11T08:49:11.421 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0026692650000000001 2024-09-11T08:49:11.421 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.421 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.421 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.421 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.459367+0000", 2024-09-11T08:49:11.421 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.063247252000000004 2024-09-11T08:49:11.421 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.421 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.421 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.421 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.491364+0000", 2024-09-11T08:49:11.422 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.031996614 2024-09-11T08:49:11.422 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.422 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.422 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.422 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.491404+0000", 2024-09-11T08:49:11.422 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.049e-05 2024-09-11T08:49:11.422 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.422 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.422 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.422 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.491442+0000", 2024-09-11T08:49:11.422 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8012999999999997e-05 2024-09-11T08:49:11.422 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.422 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.422 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.423 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.423 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.423 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:143161 7.0 7:0ef15c43:::benchmark_data_smithi092_84565_object17633:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-11T08:49:11.423 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:00.436458+0000", 2024-09-11T08:49:11.423 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 370.95488945300002, 2024-09-11T08:49:11.423 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.12616966900000001, 2024-09-11T08:49:11.423 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.423 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.423 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.423 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:11.423 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:11.423 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 143161 2024-09-11T08:49:11.423 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.424 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.424 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.424 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.424 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.436458+0000", 2024-09-11T08:49:11.424 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.424 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.424 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.424 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.424 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.436458+0000", 2024-09-11T08:49:11.424 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.424 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.424 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.424 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.425 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.436456+0000", 2024-09-11T08:49:11.425 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:49:11.425 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.425 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.425 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.425 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.436464+0000", 2024-09-11T08:49:11.425 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.8150000000000006e-06 2024-09-11T08:49:11.425 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.425 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.425 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.425 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.436482+0000", 2024-09-11T08:49:11.425 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.8114999999999999e-05 2024-09-11T08:49:11.425 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.425 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.426 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.426 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.436489+0000", 2024-09-11T08:49:11.426 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9129999999999997e-06 2024-09-11T08:49:11.426 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.426 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.426 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.426 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.436537+0000", 2024-09-11T08:49:11.426 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7919e-05 2024-09-11T08:49:11.426 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.426 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.426 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.426 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.436659+0000", 2024-09-11T08:49:11.427 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000121784 2024-09-11T08:49:11.427 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.427 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.427 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:49:11.427 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.436803+0000", 2024-09-11T08:49:11.427 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014448099999999999 2024-09-11T08:49:11.427 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.427 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.427 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.427 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.440005+0000", 2024-09-11T08:49:11.427 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003201864 2024-09-11T08:49:11.427 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.428 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.428 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.459549+0000", 2024-09-11T08:49:11.428 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.019544008000000002 2024-09-11T08:49:11.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.428 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.428 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.562534+0000", 2024-09-11T08:49:11.428 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.102984963 2024-09-11T08:49:11.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.428 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.428 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.562586+0000", 2024-09-11T08:49:11.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1606e-05 2024-09-11T08:49:11.429 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.429 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.429 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.429 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:00.562628+0000", 2024-09-11T08:49:11.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2262000000000003e-05 2024-09-11T08:49:11.429 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.429 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.429 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.429 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.429 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.429 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:144045 7.0 7:0bc4e42c:::benchmark_data_smithi092_84565_object18517:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-11T08:49:11.429 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:01.370623+0000", 2024-09-11T08:49:11.430 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 370.02072428899999, 2024-09-11T08:49:11.430 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.12646512500000001, 2024-09-11T08:49:11.430 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.430 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.430 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.430 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:11.430 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:11.430 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 144045 2024-09-11T08:49:11.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.430 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.430 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.430 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.430 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.370623+0000", 2024-09-11T08:49:11.430 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.431 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.431 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.370623+0000", 2024-09-11T08:49:11.431 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.431 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.431 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.370622+0000", 2024-09-11T08:49:11.431 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:49:11.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.432 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.432 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.432 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.370631+0000", 2024-09-11T08:49:11.432 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.9150000000000002e-06 2024-09-11T08:49:11.432 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.432 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.432 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.432 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.370646+0000", 2024-09-11T08:49:11.432 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.482e-05 2024-09-11T08:49:11.432 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.432 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.432 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.370651+0000", 2024-09-11T08:49:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.814e-06 2024-09-11T08:49:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.417660+0000", 2024-09-11T08:49:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.047008825999999997 2024-09-11T08:49:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.417769+0000", 2024-09-11T08:49:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000108961 2024-09-11T08:49:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.433 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.434 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:49:11.434 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.417877+0000", 2024-09-11T08:49:11.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000108842 2024-09-11T08:49:11.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.434 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.434 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.434 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.421732+0000", 2024-09-11T08:49:11.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0038548580000000001 2024-09-11T08:49:11.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.434 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.434 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.434 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.481377+0000", 2024-09-11T08:49:11.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.059644557000000001 2024-09-11T08:49:11.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.435 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.435 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.497001+0000", 2024-09-11T08:49:11.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.015624644 2024-09-11T08:49:11.435 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.435 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.435 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.497052+0000", 2024-09-11T08:49:11.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1002999999999999e-05 2024-09-11T08:49:11.435 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.435 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.435 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.497089+0000", 2024-09-11T08:49:11.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6084000000000002e-05 2024-09-11T08:49:11.436 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.436 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.436 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.436 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.436 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:144672 7.0 7:0efb4f93:::benchmark_data_smithi092_84565_object19144:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-11T08:49:11.436 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:01.984560+0000", 2024-09-11T08:49:11.436 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 369.40678782800001, 2024-09-11T08:49:11.436 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.121865743, 2024-09-11T08:49:11.436 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.436 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.436 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.436 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:11.436 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:11.437 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 144672 2024-09-11T08:49:11.437 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.437 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.437 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.437 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.437 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.984560+0000", 2024-09-11T08:49:11.437 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.437 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.437 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.437 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.437 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.984560+0000", 2024-09-11T08:49:11.437 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.437 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.437 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.438 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.438 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.984558+0000", 2024-09-11T08:49:11.438 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:49:11.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.438 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.438 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.438 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.984564+0000", 2024-09-11T08:49:11.438 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4260000000000002e-06 2024-09-11T08:49:11.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.438 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.438 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.438 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.984582+0000", 2024-09-11T08:49:11.438 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7776999999999999e-05 2024-09-11T08:49:11.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.438 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.439 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:01.984587+0000", 2024-09-11T08:49:11.439 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4750000000000001e-06 2024-09-11T08:49:11.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.439 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.439 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.004305+0000", 2024-09-11T08:49:11.439 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.019718372000000001 2024-09-11T08:49:11.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.439 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.439 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.004371+0000", 2024-09-11T08:49:11.439 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6062000000000005e-05 2024-09-11T08:49:11.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.440 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:49:11.440 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.004438+0000", 2024-09-11T08:49:11.440 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6622e-05 2024-09-11T08:49:11.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.440 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.440 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.008112+0000", 2024-09-11T08:49:11.440 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003673492 2024-09-11T08:49:11.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.440 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.440 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.034315+0000", 2024-09-11T08:49:11.440 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.026203290000000001 2024-09-11T08:49:11.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.441 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.441 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.441 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.106375+0000", 2024-09-11T08:49:11.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.072060025 2024-09-11T08:49:11.441 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.441 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.441 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.441 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.106405+0000", 2024-09-11T08:49:11.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9828999999999999e-05 2024-09-11T08:49:11.441 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.441 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.441 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.441 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.106426+0000", 2024-09-11T08:49:11.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0871000000000001e-05 2024-09-11T08:49:11.442 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.442 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.442 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.442 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.442 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.442 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:145228 7.0 7:0d7964f3:::benchmark_data_smithi092_84565_object19700:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-11T08:49:11.442 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:02.560006+0000", 2024-09-11T08:49:11.442 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 368.83134217700001, 2024-09-11T08:49:11.442 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.12579411900000001, 2024-09-11T08:49:11.442 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.442 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.442 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.442 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:11.442 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:11.442 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 145228 2024-09-11T08:49:11.443 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.443 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.443 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.443 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.443 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.560006+0000", 2024-09-11T08:49:11.443 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.443 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.443 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.443 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.443 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.560006+0000", 2024-09-11T08:49:11.443 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.443 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.443 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.443 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.444 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.560004+0000", 2024-09-11T08:49:11.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:11.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.444 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.560009+0000", 2024-09-11T08:49:11.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.741e-06 2024-09-11T08:49:11.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.444 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.560017+0000", 2024-09-11T08:49:11.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4270000000000002e-06 2024-09-11T08:49:11.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.560020+0000", 2024-09-11T08:49:11.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6720000000000002e-06 2024-09-11T08:49:11.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.445 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.445 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.601802+0000", 2024-09-11T08:49:11.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.041781873999999997 2024-09-11T08:49:11.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.445 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.445 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.601878+0000", 2024-09-11T08:49:11.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.5768999999999999e-05 2024-09-11T08:49:11.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.445 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:11.448 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:49:11.448 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.601950+0000", 2024-09-11T08:49:11.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2082999999999994e-05 2024-09-11T08:49:11.448 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.448 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.448 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.448 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.603644+0000", 2024-09-11T08:49:11.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001694096 2024-09-11T08:49:11.448 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.449 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.449 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.449 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.668803+0000", 2024-09-11T08:49:11.449 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.065159318999999993 2024-09-11T08:49:11.449 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.449 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.449 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.449 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.685745+0000", 2024-09-11T08:49:11.449 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.016941225000000001 2024-09-11T08:49:11.449 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.449 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.449 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.449 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.685778+0000", 2024-09-11T08:49:11.449 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3062999999999998e-05 2024-09-11T08:49:11.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:02.685800+0000", 2024-09-11T08:49:11.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.1977e-05 2024-09-11T08:49:11.450 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.450 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.450 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.450 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:172809 8.d 8:bc53f48c:::benchmark_data_smithi092_117541_object52848:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-11T08:49:11.450 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:47:49.033228+0000", 2024-09-11T08:49:11.450 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 82.358119672000001, 2024-09-11T08:49:11.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.15128206999999999, 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 172809 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:49.033228+0000", 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:49.033228+0000", 2024-09-11T08:49:11.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:49.033227+0000", 2024-09-11T08:49:11.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:11.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:49.033232+0000", 2024-09-11T08:49:11.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8169999999999998e-06 2024-09-11T08:49:11.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:49.033241+0000", 2024-09-11T08:49:11.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.9579999999999996e-06 2024-09-11T08:49:11.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:49.033244+0000", 2024-09-11T08:49:11.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.489e-06 2024-09-11T08:49:11.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:49.061650+0000", 2024-09-11T08:49:11.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.028406176000000002 2024-09-11T08:49:11.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:49.061771+0000", 2024-09-11T08:49:11.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000120494 2024-09-11T08:49:11.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,7", 2024-09-11T08:49:11.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:49.061897+0000", 2024-09-11T08:49:11.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012649199999999999 2024-09-11T08:49:11.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:49.063399+0000", 2024-09-11T08:49:11.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0015015129999999999 2024-09-11T08:49:11.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.455 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.455 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:49.063556+0000", 2024-09-11T08:49:11.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015716999999999999 2024-09-11T08:49:11.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.455 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.455 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:49.184384+0000", 2024-09-11T08:49:11.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.120828036 2024-09-11T08:49:11.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.455 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.455 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:49.184460+0000", 2024-09-11T08:49:11.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.5432000000000006e-05 2024-09-11T08:49:11.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.456 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.456 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.456 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:49.184510+0000", 2024-09-11T08:49:11.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0586e-05 2024-09-11T08:49:11.456 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.456 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.456 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.456 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.456 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:174135 8.d 8:b537f65f:::benchmark_data_smithi092_117541_object54174:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-11T08:49:11.456 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:47:50.199643+0000", 2024-09-11T08:49:11.456 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 81.191704376999994, 2024-09-11T08:49:11.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.13175445, 2024-09-11T08:49:11.456 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.457 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.457 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.457 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:49:11.457 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:49:11.457 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 174135 2024-09-11T08:49:11.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.457 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.199643+0000", 2024-09-11T08:49:11.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.199643+0000", 2024-09-11T08:49:11.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.199642+0000", 2024-09-11T08:49:11.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:49:11.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.199649+0000", 2024-09-11T08:49:11.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5230000000000001e-06 2024-09-11T08:49:11.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.199662+0000", 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3108000000000001e-05 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.199667+0000", 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4670000000000003e-06 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.212500+0000", 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.012832988 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.212617+0000", 2024-09-11T08:49:11.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000117373 2024-09-11T08:49:11.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,7", 2024-09-11T08:49:11.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.212728+0000", 2024-09-11T08:49:11.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000110596 2024-09-11T08:49:11.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.215368+0000", 2024-09-11T08:49:11.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0026394959999999999 2024-09-11T08:49:11.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.221659+0000", 2024-09-11T08:49:11.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0062915169999999999 2024-09-11T08:49:11.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.331317+0000", 2024-09-11T08:49:11.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.109657741 2024-09-11T08:49:11.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.331362+0000", 2024-09-11T08:49:11.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.528e-05 2024-09-11T08:49:11.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.331398+0000", 2024-09-11T08:49:11.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5689999999999999e-05 2024-09-11T08:49:11.462 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.462 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.462 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.462 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:174144 8.d 8:bc17d3d2:::benchmark_data_smithi092_117541_object54183:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-11T08:49:11.462 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:47:50.206906+0000", 2024-09-11T08:49:11.462 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 81.184441844000006, 2024-09-11T08:49:11.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.124796434, 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 174144 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.206906+0000", 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.206906+0000", 2024-09-11T08:49:11.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.206905+0000", 2024-09-11T08:49:11.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:11.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.206910+0000", 2024-09-11T08:49:11.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7769999999999999e-06 2024-09-11T08:49:11.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.206921+0000", 2024-09-11T08:49:11.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0275000000000001e-05 2024-09-11T08:49:11.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.206926+0000", 2024-09-11T08:49:11.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0390000000000004e-06 2024-09-11T08:49:11.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.215566+0000", 2024-09-11T08:49:11.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0086406539999999993 2024-09-11T08:49:11.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.466 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.215627+0000", 2024-09-11T08:49:11.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0662999999999998e-05 2024-09-11T08:49:11.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,7", 2024-09-11T08:49:11.466 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.215689+0000", 2024-09-11T08:49:11.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1803000000000004e-05 2024-09-11T08:49:11.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.466 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.221690+0000", 2024-09-11T08:49:11.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0060008850000000001 2024-09-11T08:49:11.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.467 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.229646+0000", 2024-09-11T08:49:11.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0079561749999999994 2024-09-11T08:49:11.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.467 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.331626+0000", 2024-09-11T08:49:11.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.101980215 2024-09-11T08:49:11.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.467 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.331668+0000", 2024-09-11T08:49:11.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2029e-05 2024-09-11T08:49:11.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.331702+0000", 2024-09-11T08:49:11.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4082999999999999e-05 2024-09-11T08:49:11.468 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.468 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.468 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.468 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:174149 8.d 8:b56b3b7e:::benchmark_data_smithi092_117541_object54188:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-11T08:49:11.468 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:47:50.207268+0000", 2024-09-11T08:49:11.468 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 81.184080097999995, 2024-09-11T08:49:11.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.124519243, 2024-09-11T08:49:11.469 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.469 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.469 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.469 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:49:11.469 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:49:11.469 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 174149 2024-09-11T08:49:11.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.469 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.469 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.469 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.207268+0000", 2024-09-11T08:49:11.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.469 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.207268+0000", 2024-09-11T08:49:11.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.207267+0000", 2024-09-11T08:49:11.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:49:11.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.207271+0000", 2024-09-11T08:49:11.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8929999999999998e-06 2024-09-11T08:49:11.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.207279+0000", 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.0690000000000002e-06 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.207283+0000", 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5719999999999996e-06 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.218408+0000", 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.01112461 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.218524+0000", 2024-09-11T08:49:11.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000116382 2024-09-11T08:49:11.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,7", 2024-09-11T08:49:11.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.218647+0000", 2024-09-11T08:49:11.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012281999999999999 2024-09-11T08:49:11.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.229597+0000", 2024-09-11T08:49:11.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.010949652000000001 2024-09-11T08:49:11.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.229652+0000", 2024-09-11T08:49:11.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5050000000000003e-05 2024-09-11T08:49:11.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.331733+0000", 2024-09-11T08:49:11.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.102081537 2024-09-11T08:49:11.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.331764+0000", 2024-09-11T08:49:11.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0874000000000003e-05 2024-09-11T08:49:11.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.331787+0000", 2024-09-11T08:49:11.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.2470999999999999e-05 2024-09-11T08:49:11.474 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.474 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.474 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.474 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:174150 8.d 8:b8132919:::benchmark_data_smithi092_117541_object54189:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-11T08:49:11.474 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:47:50.208820+0000", 2024-09-11T08:49:11.474 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 81.182527680999996, 2024-09-11T08:49:11.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.123075535, 2024-09-11T08:49:11.474 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.474 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.475 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.475 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:49:11.475 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:49:11.475 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 174150 2024-09-11T08:49:11.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.475 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.208820+0000", 2024-09-11T08:49:11.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.208820+0000", 2024-09-11T08:49:11.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.208819+0000", 2024-09-11T08:49:11.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:11.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.208824+0000", 2024-09-11T08:49:11.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8470000000000003e-06 2024-09-11T08:49:11.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.208836+0000", 2024-09-11T08:49:11.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1838000000000001e-05 2024-09-11T08:49:11.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.208842+0000", 2024-09-11T08:49:11.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1800000000000001e-06 2024-09-11T08:49:11.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.221716+0000", 2024-09-11T08:49:11.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.012874396999999999 2024-09-11T08:49:11.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.221818+0000", 2024-09-11T08:49:11.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010149699999999999 2024-09-11T08:49:11.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,7", 2024-09-11T08:49:11.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.221924+0000", 2024-09-11T08:49:11.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010600299999999999 2024-09-11T08:49:11.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.479 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.229640+0000", 2024-09-11T08:49:11.479 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.007716362 2024-09-11T08:49:11.479 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.479 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.479 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.479 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.229656+0000", 2024-09-11T08:49:11.479 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6047999999999999e-05 2024-09-11T08:49:11.479 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.479 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.480 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.480 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.331855+0000", 2024-09-11T08:49:11.480 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10219898500000001 2024-09-11T08:49:11.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.480 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.480 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.480 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.331878+0000", 2024-09-11T08:49:11.480 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.2858e-05 2024-09-11T08:49:11.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.480 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.480 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.480 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:50.331896+0000", 2024-09-11T08:49:11.480 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7479e-05 2024-09-11T08:49:11.480 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.481 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.481 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.481 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:177002 8.d 8:bac606be:::benchmark_data_smithi092_117541_object57041:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-11T08:49:11.481 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:47:52.715284+0000", 2024-09-11T08:49:11.481 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.676063459999995, 2024-09-11T08:49:11.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.135943117, 2024-09-11T08:49:11.481 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.481 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.481 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.481 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:49:11.481 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:49:11.481 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 177002 2024-09-11T08:49:11.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.482 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:52.715284+0000", 2024-09-11T08:49:11.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:52.715284+0000", 2024-09-11T08:49:11.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:52.715283+0000", 2024-09-11T08:49:11.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:49:11.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:52.715290+0000", 2024-09-11T08:49:11.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6109999999999999e-06 2024-09-11T08:49:11.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:52.715302+0000", 2024-09-11T08:49:11.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2517999999999999e-05 2024-09-11T08:49:11.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:52.715308+0000", 2024-09-11T08:49:11.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0399999999999998e-06 2024-09-11T08:49:11.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.484 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.484 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:52.715350+0000", 2024-09-11T08:49:11.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2347e-05 2024-09-11T08:49:11.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.484 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.484 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:52.715479+0000", 2024-09-11T08:49:11.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012846799999999999 2024-09-11T08:49:11.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.484 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.485 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,7", 2024-09-11T08:49:11.485 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:52.715611+0000", 2024-09-11T08:49:11.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013195599999999999 2024-09-11T08:49:11.485 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.485 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.485 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.485 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:52.718470+0000", 2024-09-11T08:49:11.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0028593659999999999 2024-09-11T08:49:11.485 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.485 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.485 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.485 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:52.718616+0000", 2024-09-11T08:49:11.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000145329 2024-09-11T08:49:11.485 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:52.851170+0000", 2024-09-11T08:49:11.486 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.13255399400000001 2024-09-11T08:49:11.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:52.851202+0000", 2024-09-11T08:49:11.486 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2218000000000002e-05 2024-09-11T08:49:11.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:52.851227+0000", 2024-09-11T08:49:11.486 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5605999999999998e-05 2024-09-11T08:49:11.486 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.487 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.487 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.487 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.487 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.487 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:177509 8.d 8:b7f92758:::benchmark_data_smithi092_117541_object57548:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-11T08:49:11.487 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:47:53.179040+0000", 2024-09-11T08:49:11.487 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 78.212307367999998, 2024-09-11T08:49:11.487 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.12409329199999999, 2024-09-11T08:49:11.487 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.487 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.487 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.487 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:49:11.487 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:49:11.487 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 177509 2024-09-11T08:49:11.487 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.488 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.179040+0000", 2024-09-11T08:49:11.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.179040+0000", 2024-09-11T08:49:11.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.179039+0000", 2024-09-11T08:49:11.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:11.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.179045+0000", 2024-09-11T08:49:11.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6289999999999998e-06 2024-09-11T08:49:11.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.179057+0000", 2024-09-11T08:49:11.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1766999999999999e-05 2024-09-11T08:49:11.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.179064+0000", 2024-09-11T08:49:11.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.7320000000000001e-06 2024-09-11T08:49:11.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.179185+0000", 2024-09-11T08:49:11.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000121378 2024-09-11T08:49:11.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.179304+0000", 2024-09-11T08:49:11.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000119369 2024-09-11T08:49:11.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,7", 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.179410+0000", 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000105486 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.193894+0000", 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.014484716999999999 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.202873+0000", 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0089781589999999994 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.491 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.303034+0000", 2024-09-11T08:49:11.492 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.100161351 2024-09-11T08:49:11.492 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.492 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.492 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.303096+0000", 2024-09-11T08:49:11.492 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1774999999999997e-05 2024-09-11T08:49:11.492 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.492 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.492 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.303134+0000", 2024-09-11T08:49:11.492 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7829e-05 2024-09-11T08:49:11.492 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.492 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.493 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.493 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.493 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:178366 8.d 8:b6eaa26f:::benchmark_data_smithi092_117541_object58405:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-11T08:49:11.493 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:47:53.951248+0000", 2024-09-11T08:49:11.493 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 77.440099930000002, 2024-09-11T08:49:11.493 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.13203811300000001, 2024-09-11T08:49:11.493 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.493 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.493 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.493 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:49:11.493 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:49:11.493 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 178366 2024-09-11T08:49:11.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.494 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.494 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.494 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.951248+0000", 2024-09-11T08:49:11.494 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.951248+0000", 2024-09-11T08:49:11.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.951246+0000", 2024-09-11T08:49:11.497 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:49:11.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.497 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.497 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.497 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.951253+0000", 2024-09-11T08:49:11.497 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.528e-06 2024-09-11T08:49:11.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.497 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.497 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.497 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.951268+0000", 2024-09-11T08:49:11.497 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4608e-05 2024-09-11T08:49:11.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.497 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.498 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.498 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.951273+0000", 2024-09-11T08:49:11.498 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7629999999999997e-06 2024-09-11T08:49:11.498 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.498 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.498 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.498 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.986563+0000", 2024-09-11T08:49:11.498 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.035289922000000001 2024-09-11T08:49:11.498 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.498 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.498 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.498 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.986672+0000", 2024-09-11T08:49:11.498 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000109064 2024-09-11T08:49:11.498 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.499 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.499 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,7", 2024-09-11T08:49:11.499 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.986781+0000", 2024-09-11T08:49:11.499 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000108683 2024-09-11T08:49:11.499 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.499 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.499 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.499 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.013378+0000", 2024-09-11T08:49:11.499 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.026597201000000001 2024-09-11T08:49:11.499 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.499 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.499 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.499 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.013432+0000", 2024-09-11T08:49:11.499 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3304000000000003e-05 2024-09-11T08:49:11.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.500 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.500 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.500 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.083218+0000", 2024-09-11T08:49:11.500 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.069786263000000001 2024-09-11T08:49:11.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.500 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.500 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.500 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.083249+0000", 2024-09-11T08:49:11.500 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1328000000000002e-05 2024-09-11T08:49:11.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.500 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.500 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.500 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.083286+0000", 2024-09-11T08:49:11.501 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6730999999999998e-05 2024-09-11T08:49:11.501 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.501 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.501 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.501 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.501 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.501 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:178378 8.d 8:b8cf3fe6:::benchmark_data_smithi092_117541_object58417:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-11T08:49:11.501 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:47:53.960387+0000", 2024-09-11T08:49:11.501 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 77.430960718999998, 2024-09-11T08:49:11.501 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.12313131500000001, 2024-09-11T08:49:11.501 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.501 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.501 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.501 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:49:11.502 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:49:11.502 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 178378 2024-09-11T08:49:11.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.502 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.502 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.502 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.502 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.960387+0000", 2024-09-11T08:49:11.502 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.502 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.502 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.502 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.960387+0000", 2024-09-11T08:49:11.502 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.503 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.503 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.503 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.960386+0000", 2024-09-11T08:49:11.503 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:49:11.503 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.503 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.503 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.503 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.960392+0000", 2024-09-11T08:49:11.503 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5100000000000004e-06 2024-09-11T08:49:11.503 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.503 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.503 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.503 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.960408+0000", 2024-09-11T08:49:11.503 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6324000000000002e-05 2024-09-11T08:49:11.504 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.504 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.504 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.504 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.960414+0000", 2024-09-11T08:49:11.504 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0310000000000004e-06 2024-09-11T08:49:11.504 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.504 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.504 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.504 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.991321+0000", 2024-09-11T08:49:11.504 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.030906241000000001 2024-09-11T08:49:11.504 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.504 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.504 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.505 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.991420+0000", 2024-09-11T08:49:11.505 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.8827999999999996e-05 2024-09-11T08:49:11.505 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.505 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.505 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,7", 2024-09-11T08:49:11.505 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:53.991521+0000", 2024-09-11T08:49:11.505 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010145 2024-09-11T08:49:11.505 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.505 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.505 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.505 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.013387+0000", 2024-09-11T08:49:11.505 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.021865800000000001 2024-09-11T08:49:11.505 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.506 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.506 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.506 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.013471+0000", 2024-09-11T08:49:11.506 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.4513999999999998e-05 2024-09-11T08:49:11.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.506 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.506 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.506 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.083472+0000", 2024-09-11T08:49:11.506 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.070000356 2024-09-11T08:49:11.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.506 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.506 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.506 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.083498+0000", 2024-09-11T08:49:11.506 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.6594e-05 2024-09-11T08:49:11.507 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.507 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.507 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.507 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.083518+0000", 2024-09-11T08:49:11.507 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0047e-05 2024-09-11T08:49:11.507 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.507 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.507 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.507 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.507 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.507 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:178677 8.d 8:bc5e23b5:::benchmark_data_smithi092_117541_object58716:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-11T08:49:11.507 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:47:54.224953+0000", 2024-09-11T08:49:11.507 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 77.166394803000003, 2024-09-11T08:49:11.507 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.121996787, 2024-09-11T08:49:11.507 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.508 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.508 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.508 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:49:11.508 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:49:11.508 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 178677 2024-09-11T08:49:11.508 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.508 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.508 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.508 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.508 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.224953+0000", 2024-09-11T08:49:11.508 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.508 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.508 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.508 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.509 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.224953+0000", 2024-09-11T08:49:11.509 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.509 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.509 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.509 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.509 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.224952+0000", 2024-09-11T08:49:11.509 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:11.509 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.509 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.509 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.509 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.224965+0000", 2024-09-11T08:49:11.509 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2756e-05 2024-09-11T08:49:11.509 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.509 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.510 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.510 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.224978+0000", 2024-09-11T08:49:11.510 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3517e-05 2024-09-11T08:49:11.510 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.510 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.510 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.510 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.224984+0000", 2024-09-11T08:49:11.510 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.49e-06 2024-09-11T08:49:11.510 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.510 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.510 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.510 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.248336+0000", 2024-09-11T08:49:11.510 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.023351984999999999 2024-09-11T08:49:11.510 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.511 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.511 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.511 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.248441+0000", 2024-09-11T08:49:11.511 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010529799999999999 2024-09-11T08:49:11.511 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.511 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.511 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,7", 2024-09-11T08:49:11.511 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.248565+0000", 2024-09-11T08:49:11.511 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000124179 2024-09-11T08:49:11.511 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.511 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.511 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.511 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.251372+0000", 2024-09-11T08:49:11.511 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0028069950000000001 2024-09-11T08:49:11.512 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.512 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.512 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.512 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.251382+0000", 2024-09-11T08:49:11.512 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.6479999999999998e-06 2024-09-11T08:49:11.512 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.512 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.512 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.512 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.346832+0000", 2024-09-11T08:49:11.512 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.095450185000000007 2024-09-11T08:49:11.512 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.512 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.512 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.512 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.346921+0000", 2024-09-11T08:49:11.513 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.9338000000000004e-05 2024-09-11T08:49:11.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.513 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.513 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.513 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:54.346950+0000", 2024-09-11T08:49:11.513 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.8489000000000001e-05 2024-09-11T08:49:11.513 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.513 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.513 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.513 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.513 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:180452 8.d 8:b2ec4e59:::benchmark_data_smithi092_117541_object60491:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-11T08:49:11.513 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:47:55.858842+0000", 2024-09-11T08:49:11.513 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 75.532506076999994, 2024-09-11T08:49:11.514 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.13617252199999999, 2024-09-11T08:49:11.514 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.514 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.514 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.514 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:49:11.514 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:49:11.514 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 180452 2024-09-11T08:49:11.514 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.514 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.514 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.514 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.514 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:55.858842+0000", 2024-09-11T08:49:11.514 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.514 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.515 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.515 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.515 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:55.858842+0000", 2024-09-11T08:49:11.515 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.515 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.515 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.515 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.515 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:55.858841+0000", 2024-09-11T08:49:11.515 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:11.515 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.515 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.515 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.515 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:55.858845+0000", 2024-09-11T08:49:11.515 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2440000000000004e-06 2024-09-11T08:49:11.516 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.516 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.516 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.516 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:55.858853+0000", 2024-09-11T08:49:11.516 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6960000000000005e-06 2024-09-11T08:49:11.516 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.516 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.516 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.516 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:55.858857+0000", 2024-09-11T08:49:11.516 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4290000000000003e-06 2024-09-11T08:49:11.516 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.516 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.516 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.516 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:55.880262+0000", 2024-09-11T08:49:11.516 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.021404554999999999 2024-09-11T08:49:11.517 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.517 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.517 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.517 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:55.880367+0000", 2024-09-11T08:49:11.517 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010482199999999999 2024-09-11T08:49:11.517 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.517 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.517 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,7", 2024-09-11T08:49:11.517 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:55.880493+0000", 2024-09-11T08:49:11.517 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012616099999999999 2024-09-11T08:49:11.517 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.517 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.517 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.518 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:55.885630+0000", 2024-09-11T08:49:11.518 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0051377510000000003 2024-09-11T08:49:11.518 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.518 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.518 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.518 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:55.886712+0000", 2024-09-11T08:49:11.518 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0010814780000000001 2024-09-11T08:49:11.518 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.518 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.518 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.518 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:55.994922+0000", 2024-09-11T08:49:11.518 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10820959300000001 2024-09-11T08:49:11.518 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.518 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.519 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.519 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:55.994973+0000", 2024-09-11T08:49:11.519 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1263e-05 2024-09-11T08:49:11.519 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.519 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.519 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.519 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:55.995014+0000", 2024-09-11T08:49:11.519 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1369000000000003e-05 2024-09-11T08:49:11.519 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.519 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.519 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.519 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.519 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.519 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:181269 8.d 8:b04d5f07:::benchmark_data_smithi092_117541_object61308:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-11T08:49:11.520 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:47:56.601598+0000", 2024-09-11T08:49:11.520 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 74.789749583000003, 2024-09-11T08:49:11.520 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.12200818500000001, 2024-09-11T08:49:11.520 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.520 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.520 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.520 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:49:11.520 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:49:11.520 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 181269 2024-09-11T08:49:11.520 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.520 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.520 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.520 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.520 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:56.601598+0000", 2024-09-11T08:49:11.520 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.521 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.521 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.521 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.521 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:56.601598+0000", 2024-09-11T08:49:11.521 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.521 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.521 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.521 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.521 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:56.601597+0000", 2024-09-11T08:49:11.521 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:11.521 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.521 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.521 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.522 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:56.601603+0000", 2024-09-11T08:49:11.522 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0090000000000004e-06 2024-09-11T08:49:11.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.522 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.522 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.522 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:56.601613+0000", 2024-09-11T08:49:11.522 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.8409999999999995e-06 2024-09-11T08:49:11.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.522 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.522 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.522 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:56.601618+0000", 2024-09-11T08:49:11.522 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0869999999999997e-06 2024-09-11T08:49:11.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.522 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.523 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.523 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:56.613156+0000", 2024-09-11T08:49:11.523 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011537726999999999 2024-09-11T08:49:11.523 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.523 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.523 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.523 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:56.613279+0000", 2024-09-11T08:49:11.523 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000123464 2024-09-11T08:49:11.523 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.523 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.523 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,7", 2024-09-11T08:49:11.523 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:56.613425+0000", 2024-09-11T08:49:11.523 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014609599999999999 2024-09-11T08:49:11.524 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.524 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.524 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.524 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:56.636686+0000", 2024-09-11T08:49:11.524 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.023260485000000001 2024-09-11T08:49:11.524 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.524 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.524 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.524 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:56.640440+0000", 2024-09-11T08:49:11.524 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0037546929999999999 2024-09-11T08:49:11.524 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.524 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.524 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.525 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:56.723489+0000", 2024-09-11T08:49:11.525 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.083048979999999994 2024-09-11T08:49:11.525 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.525 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.525 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.525 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:56.723566+0000", 2024-09-11T08:49:11.525 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6101999999999997e-05 2024-09-11T08:49:11.525 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.525 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.525 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.525 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:56.723606+0000", 2024-09-11T08:49:11.525 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0722000000000001e-05 2024-09-11T08:49:11.525 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.525 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.526 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.526 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.526 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.526 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15126.0:184338 8.d 8:b5d08c3d:::benchmark_data_smithi092_117541_object64377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-11T08:49:11.526 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:47:59.441282+0000", 2024-09-11T08:49:11.526 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 71.950066066999995, 2024-09-11T08:49:11.526 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.131872185, 2024-09-11T08:49:11.526 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:11.526 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:11.526 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:11.526 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15126", 2024-09-11T08:49:11.526 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:49:11.526 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 184338 2024-09-11T08:49:11.526 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:59.441282+0000", 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:59.441282+0000", 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:59.441281+0000", 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:49:11.527 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.528 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.528 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:11.528 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:59.441285+0000", 2024-09-11T08:49:11.528 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8419999999999998e-06 2024-09-11T08:49:11.528 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.528 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.528 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:11.528 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:59.441294+0000", 2024-09-11T08:49:11.528 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8070000000000006e-06 2024-09-11T08:49:11.528 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.528 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.528 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:11.528 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:59.441298+0000", 2024-09-11T08:49:11.528 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3229999999999999e-06 2024-09-11T08:49:11.529 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.529 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.529 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:11.529 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:59.465988+0000", 2024-09-11T08:49:11.529 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.024690126999999999 2024-09-11T08:49:11.529 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.529 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.529 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:11.529 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:59.466089+0000", 2024-09-11T08:49:11.529 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000101028 2024-09-11T08:49:11.529 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.529 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.529 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,7", 2024-09-11T08:49:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:59.466215+0000", 2024-09-11T08:49:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012595600000000001 2024-09-11T08:49:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:59.491056+0000", 2024-09-11T08:49:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.024841205000000002 2024-09-11T08:49:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:59.494080+0000", 2024-09-11T08:49:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0030236920000000001 2024-09-11T08:49:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:11.531 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:59.573055+0000", 2024-09-11T08:49:11.531 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.078975064999999997 2024-09-11T08:49:11.531 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.531 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.531 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:11.531 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:59.573109+0000", 2024-09-11T08:49:11.531 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3454e-05 2024-09-11T08:49:11.531 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:11.531 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:11.531 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:11.531 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:47:59.573154+0000", 2024-09-11T08:49:11.531 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5223999999999999e-05 2024-09-11T08:49:11.531 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.531 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.532 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.532 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:11.532 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:11.532 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:49:11.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:11.533 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:11.534 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:11.534 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:49:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:49:11] "GET /metrics HTTP/1.1" 200 221872 "" "Prometheus/2.43.0" 2024-09-11T08:49:11.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:11.656 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:11.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:11.657 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:11.758 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:11.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:11.780 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:11.780 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:11.881 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:11.906 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:11.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:11.907 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:12.008 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:12.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:12.030 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:12.030 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:12.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:12.145 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:11 smithi092 ceph-mon[29647]: pgmap v443: 148 pgs: 2 creating+activating, 146 active+clean; 582 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 470 B/s rd, 0 op/s; 133 KiB/s, 3 objects/s recovering 2024-09-11T08:49:12.146 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:11 smithi092 ceph-mon[41114]: pgmap v443: 148 pgs: 2 creating+activating, 146 active+clean; 582 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 470 B/s rd, 0 op/s; 133 KiB/s, 3 objects/s recovering 2024-09-11T08:49:12.153 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:12.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:12.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:12.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:11 smithi145 ceph-mon[33384]: pgmap v443: 148 pgs: 2 creating+activating, 146 active+clean; 582 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 470 B/s rd, 0 op/s; 133 KiB/s, 3 objects/s recovering 2024-09-11T08:49:12.255 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:12.276 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:12.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:12.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:12.340 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-11T08:49:12.379 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:12.400 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:12.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:12.401 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:12.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:12.523 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:12.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:12.524 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:12.584 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:49:12.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:49:12.626 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:12.647 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:12.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:12.648 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:12.749 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:12.771 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:12.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:12.772 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:12.873 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:12.894 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:12.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:12.895 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:12.901 INFO:teuthology.orchestra.run.smithi092.stderr:enabled application 'rados' on pool 'unique_pool_2' 2024-09-11T08:49:12.996 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:13.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:13.018 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:13.018 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:13.037 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:12 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4122380306' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:49:13.037 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:12 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:49:13.038 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:49:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:49:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:49:13.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:13.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:13.143 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:13.243 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:13.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:13.266 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:13.266 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:13.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:12 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4122380306' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:49:13.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:12 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:49:13.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:12 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4122380306' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:49:13.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:12 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:49:13.367 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:13.388 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:13.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:13.389 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:13.490 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:13.511 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:13.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:13.512 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:13.613 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:13.634 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:13.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:13.635 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:13.735 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:13.757 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:13.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:13.758 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:13.859 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:13.883 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:13.884 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:13.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:13.885 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:13.985 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:14.008 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:14.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:14.009 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:14.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:14.119 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:13 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:49:14.119 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:13 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:49:14.119 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:13 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:49:14.119 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:13 smithi145 ceph-mon[33384]: osdmap e212: 8 total, 8 up, 7 in 2024-09-11T08:49:14.119 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:13 smithi145 ceph-mon[33384]: pgmap v445: 148 pgs: 148 active+clean; 582 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 108 KiB/s, 11 objects/s recovering 2024-09-11T08:49:14.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:14.135 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:14.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:14.258 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:14.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:14.260 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:14.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:13 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:49:14.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:13 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:49:14.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:13 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:49:14.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:13 smithi092 ceph-mon[29647]: osdmap e212: 8 total, 8 up, 7 in 2024-09-11T08:49:14.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:13 smithi092 ceph-mon[29647]: pgmap v445: 148 pgs: 148 active+clean; 582 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 108 KiB/s, 11 objects/s recovering 2024-09-11T08:49:14.312 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:13 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:49:14.313 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:13 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:49:14.313 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:13 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:49:14.313 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:13 smithi092 ceph-mon[41114]: osdmap e212: 8 total, 8 up, 7 in 2024-09-11T08:49:14.313 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:13 smithi092 ceph-mon[41114]: pgmap v445: 148 pgs: 148 active+clean; 582 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 108 KiB/s, 11 objects/s recovering 2024-09-11T08:49:14.361 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:14.371 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:49:14.371 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T08:49:14.371 INFO:teuthology.orchestra.run.smithi145.stdout: "num_ops": 0 2024-09-11T08:49:14.372 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:49:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:14.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:14.383 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:14.484 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:14.506 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:14.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:14.507 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:14.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:14.630 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:14.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:14.631 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:14.638 DEBUG:teuthology.orchestra.run.smithi092:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_2 bench 90 write' 2024-09-11T08:49:14.674 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-11T08:49:14.675 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-11T08:49:14.719 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:hints = 1 2024-09-11T08:49:14.719 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-11T08:49:14.719 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Object prefix: benchmark_data_smithi092_145083 2024-09-11T08:49:14.720 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:49:14.720 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 0 0 0 0 0 0 - 0 2024-09-11T08:49:14.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:14.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:14.769 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:14.769 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:14.831 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:49:14.855 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:49:14.855 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_2 2024-09-11T08:49:14.855 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:49:14.870 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:14.905 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:14.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:14.907 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:14.924 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:49:15.008 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:15.039 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:14 smithi145 ceph-mon[33384]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:49:15.040 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:14 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2342923817' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:49:15.040 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:14 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:49:15.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:15.041 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:15.041 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:15.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:14 smithi092 ceph-mon[29647]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:49:15.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:14 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2342923817' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:49:15.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:14 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:49:15.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:14 smithi092 ceph-mon[41114]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:49:15.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:14 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2342923817' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:49:15.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:14 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:49:15.084 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-11T08:49:15.142 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:15.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:15.185 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:15.185 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:15.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:15.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:15.320 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:15.421 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:15.462 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:15.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:15.463 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:15.524 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:49:15.564 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:15.602 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:15.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:15.603 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:15.704 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:15.719 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 1 16 3603 3587 224.17 224.188 0.00097685 0.00420323 2024-09-11T08:49:15.726 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:15.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:15.727 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:15.828 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:15.869 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:15.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:15.870 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:15 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:49:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:15 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:49:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:15 smithi145 ceph-mon[33384]: osdmap e213: 8 total, 8 up, 7 in 2024-09-11T08:49:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2024-09-11T08:49:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-09-11T08:49:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:15 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2024-09-11T08:49:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:15 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-09-11T08:49:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:15 smithi145 ceph-mon[33384]: pgmap v447: 148 pgs: 148 active+clean; 582 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 96 KiB/s, 10 objects/s recovering 2024-09-11T08:49:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:15 smithi145 ceph-mon[33384]: 4.17 scrub starts 2024-09-11T08:49:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:15 smithi145 ceph-mon[33384]: 4.17 scrub ok 2024-09-11T08:49:15.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:16.005 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:16.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:16.006 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:16.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:16.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:16.143 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:16.240 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:49:16.240 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:49:16.240 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[29647]: osdmap e213: 8 total, 8 up, 7 in 2024-09-11T08:49:16.240 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:16.240 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2024-09-11T08:49:16.241 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-09-11T08:49:16.241 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2024-09-11T08:49:16.241 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-09-11T08:49:16.241 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[29647]: pgmap v447: 148 pgs: 148 active+clean; 582 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 96 KiB/s, 10 objects/s recovering 2024-09-11T08:49:16.241 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[29647]: 4.17 scrub starts 2024-09-11T08:49:16.241 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[29647]: 4.17 scrub ok 2024-09-11T08:49:16.241 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:49:16.241 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:49:16.241 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[41114]: osdmap e213: 8 total, 8 up, 7 in 2024-09-11T08:49:16.241 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:16.242 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2024-09-11T08:49:16.242 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-09-11T08:49:16.242 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2024-09-11T08:49:16.242 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-09-11T08:49:16.242 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[41114]: pgmap v447: 148 pgs: 148 active+clean; 582 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 96 KiB/s, 10 objects/s recovering 2024-09-11T08:49:16.242 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[41114]: 4.17 scrub starts 2024-09-11T08:49:16.242 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:15 smithi092 ceph-mon[41114]: 4.17 scrub ok 2024-09-11T08:49:16.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:16.290 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:16.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:16.291 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:16.392 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:16.427 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:16.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:16.428 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:16.529 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:16.560 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:16.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:16.561 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:16.662 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:16.696 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:16.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:16.697 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:16.719 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 2 16 7141 7125 222.639 221.125 0.00316742 0.00448253 2024-09-11T08:49:16.798 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:16.829 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:16.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:16.830 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:16.854 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:49:16.855 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:49:16.861 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:49:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:49:16.888 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:49:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:49:16.918 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:49:16.918 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:49:16.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:16.952 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:49:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:49:16.970 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:16.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:16.971 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:16.982 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:49:16.982 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:49:17.013 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:49:17.014 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:49:17.045 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:49:17.045 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:49:17.046 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:16 smithi092 ceph-mon[29647]: 5.6 scrub starts 2024-09-11T08:49:17.046 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:16 smithi092 ceph-mon[29647]: 5.6 scrub ok 2024-09-11T08:49:17.046 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:16 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]': finished 2024-09-11T08:49:17.046 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:16 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-09-11T08:49:17.046 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:16 smithi092 ceph-mon[29647]: osdmap e214: 8 total, 8 up, 7 in 2024-09-11T08:49:17.046 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:16 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:17.046 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:16 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4105209279' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:17.046 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:16 smithi092 ceph-mon[41114]: 5.6 scrub starts 2024-09-11T08:49:17.047 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:16 smithi092 ceph-mon[41114]: 5.6 scrub ok 2024-09-11T08:49:17.047 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:16 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]': finished 2024-09-11T08:49:17.047 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:16 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-09-11T08:49:17.047 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:16 smithi092 ceph-mon[41114]: osdmap e214: 8 total, 8 up, 7 in 2024-09-11T08:49:17.047 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:16 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:17.047 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:16 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4105209279' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:17.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:17.076 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:49:17.076 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:49:17.110 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:17.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:17.111 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:16 smithi145 ceph-mon[33384]: 5.6 scrub starts 2024-09-11T08:49:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:16 smithi145 ceph-mon[33384]: 5.6 scrub ok 2024-09-11T08:49:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:16 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]': finished 2024-09-11T08:49:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:16 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-09-11T08:49:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:16 smithi145 ceph-mon[33384]: osdmap e214: 8 total, 8 up, 7 in 2024-09-11T08:49:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:16 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:16 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4105209279' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:17.212 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:17.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:17.249 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:17.350 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:17.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:17.387 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:17.388 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:17.488 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:17.524 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:17.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:17.525 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:17.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:17.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:17.658 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:17.658 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:17.719 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 3 16 10352 10336 215.316 200.688 0.00202245 0.0046358 2024-09-11T08:49:17.759 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:17.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:17.782 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:17.782 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:17.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:17.916 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:17.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:17.918 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:18.019 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:18.055 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:18.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:18.056 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:18.084 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:17 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:49:18.084 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:17 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:49:18.084 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:17 smithi145 ceph-mon[33384]: osdmap e215: 8 total, 8 up, 7 in 2024-09-11T08:49:18.085 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:17 smithi145 ceph-mon[33384]: pgmap v450: 148 pgs: 2 peering, 146 active+clean; 375 MiB data, 2.8 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-11T08:49:18.085 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:49:18.085 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:49:18.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:18.190 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:18.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:18.192 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:18.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:49:18.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:18.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:17 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:49:18.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:17 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:49:18.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:17 smithi092 ceph-mon[29647]: osdmap e215: 8 total, 8 up, 7 in 2024-09-11T08:49:18.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:17 smithi092 ceph-mon[29647]: pgmap v450: 148 pgs: 2 peering, 146 active+clean; 375 MiB data, 2.8 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-11T08:49:18.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:17 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:49:18.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:17 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:49:18.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:17 smithi092 ceph-mon[41114]: osdmap e215: 8 total, 8 up, 7 in 2024-09-11T08:49:18.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:17 smithi092 ceph-mon[41114]: pgmap v450: 148 pgs: 2 peering, 146 active+clean; 375 MiB data, 2.8 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-11T08:49:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:18.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:18.355 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:18.456 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:18.493 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:18.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:18.494 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:18.595 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:18.631 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:18.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:18.632 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:18.720 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 4 16 13527 13511 211.093 198.438 0.00218084 0.0047292 2024-09-11T08:49:18.732 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:18.767 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:18.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:18.768 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:18.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:18.900 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:18.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:18.901 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:19.002 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:19.036 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:19.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:19.037 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:19.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:19.171 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:19.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:19.172 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:19.201 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:18 smithi145 ceph-mon[33384]: 3.f scrub starts 2024-09-11T08:49:19.202 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:18 smithi145 ceph-mon[33384]: 3.f scrub ok 2024-09-11T08:49:19.202 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:18 smithi145 ceph-mon[33384]: 6.a scrub starts 2024-09-11T08:49:19.202 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:18 smithi145 ceph-mon[33384]: 6.a scrub ok 2024-09-11T08:49:19.203 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-11T08:49:19.272 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:19.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:19.312 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:19.312 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:19.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:18 smithi092 ceph-mon[29647]: 3.f scrub starts 2024-09-11T08:49:19.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:18 smithi092 ceph-mon[29647]: 3.f scrub ok 2024-09-11T08:49:19.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:18 smithi092 ceph-mon[29647]: 6.a scrub starts 2024-09-11T08:49:19.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:18 smithi092 ceph-mon[29647]: 6.a scrub ok 2024-09-11T08:49:19.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:18 smithi092 ceph-mon[41114]: 3.f scrub starts 2024-09-11T08:49:19.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:18 smithi092 ceph-mon[41114]: 3.f scrub ok 2024-09-11T08:49:19.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:18 smithi092 ceph-mon[41114]: 6.a scrub starts 2024-09-11T08:49:19.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:18 smithi092 ceph-mon[41114]: 6.a scrub ok 2024-09-11T08:49:19.413 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:19.449 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:19.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:19.450 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:19.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:19.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:19.589 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:19.589 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:19.638 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:49:19.690 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:19.720 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 5 16 16358 16342 204.257 176.938 0.00467548 0.00488907 2024-09-11T08:49:19.725 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:19.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:19.726 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:19.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:19.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:19.864 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:19.864 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:19.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:19.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:49:19.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:19 smithi092 ceph-mon[29647]: 3.9 scrub starts 2024-09-11T08:49:19.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:19 smithi092 ceph-mon[29647]: 3.9 scrub ok 2024-09-11T08:49:19.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:19 smithi092 ceph-mon[29647]: pgmap v451: 148 pgs: 2 peering, 146 active+clean; 375 MiB data, 2.8 GiB used, 623 GiB / 626 GiB avail; 853 B/s rd, 62 MiB/s wr, 1k op/s 2024-09-11T08:49:19.985 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:19 smithi092 ceph-mon[41114]: 3.9 scrub starts 2024-09-11T08:49:19.985 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:19 smithi092 ceph-mon[41114]: 3.9 scrub ok 2024-09-11T08:49:19.985 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:19 smithi092 ceph-mon[41114]: pgmap v451: 148 pgs: 2 peering, 146 active+clean; 375 MiB data, 2.8 GiB used, 623 GiB / 626 GiB avail; 853 B/s rd, 62 MiB/s wr, 1k op/s 2024-09-11T08:49:19.996 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:20.005 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:20.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:20.006 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:20.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:20.129 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:19 smithi145 ceph-mon[33384]: 3.9 scrub starts 2024-09-11T08:49:20.129 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:19 smithi145 ceph-mon[33384]: 3.9 scrub ok 2024-09-11T08:49:20.129 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:19 smithi145 ceph-mon[33384]: pgmap v451: 148 pgs: 2 peering, 146 active+clean; 375 MiB data, 2.8 GiB used, 623 GiB / 626 GiB avail; 853 B/s rd, 62 MiB/s wr, 1k op/s 2024-09-11T08:49:20.148 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:20.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:20.149 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:20.250 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:20.283 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:20.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:20.284 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:20.365 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:49:20.368 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:49:20.385 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:20.419 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:20.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:20.420 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:20.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:20.556 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:20.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:20.558 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:20.658 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:20.694 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:20.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:20.695 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:20.720 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 6 15 19094 19079 198.722 171.062 0.00546655 0.00502694 2024-09-11T08:49:20.796 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:20.831 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:20.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:20.832 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:20.933 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:20.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:20.969 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:20.970 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:21.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:21.104 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:21.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:21.105 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:21.186 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:20 smithi145 ceph-mon[33384]: 4.1a scrub starts 2024-09-11T08:49:21.187 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:20 smithi145 ceph-mon[33384]: 4.1a scrub ok 2024-09-11T08:49:21.187 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:20 smithi145 ceph-mon[33384]: 6.d scrub starts 2024-09-11T08:49:21.187 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:20 smithi145 ceph-mon[33384]: 6.d scrub ok 2024-09-11T08:49:21.187 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:20 smithi145 ceph-mon[33384]: from='client.25099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:49:21.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:21.238 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:21.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:21.239 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:21.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:20 smithi092 ceph-mon[29647]: 4.1a scrub starts 2024-09-11T08:49:21.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:20 smithi092 ceph-mon[29647]: 4.1a scrub ok 2024-09-11T08:49:21.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:20 smithi092 ceph-mon[29647]: 6.d scrub starts 2024-09-11T08:49:21.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:20 smithi092 ceph-mon[29647]: 6.d scrub ok 2024-09-11T08:49:21.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:20 smithi092 ceph-mon[29647]: from='client.25099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:49:21.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:20 smithi092 ceph-mon[41114]: 4.1a scrub starts 2024-09-11T08:49:21.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:20 smithi092 ceph-mon[41114]: 4.1a scrub ok 2024-09-11T08:49:21.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:20 smithi092 ceph-mon[41114]: 6.d scrub starts 2024-09-11T08:49:21.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:20 smithi092 ceph-mon[41114]: 6.d scrub ok 2024-09-11T08:49:21.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:20 smithi092 ceph-mon[41114]: from='client.25099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:49:21.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:21.378 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:21.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:21.379 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:21.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:21.513 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:21.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:21.514 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:21.592 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:49:21.592 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:49:21.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:21.645 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:21.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:21.646 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:21.720 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 7 16 21771 21755 194.224 167.25 0.00598813 0.00514259 2024-09-11T08:49:21.747 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:21.781 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:21.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:21.782 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:21.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:49:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:49:21] "GET /metrics HTTP/1.1" 200 224194 "" "Prometheus/2.43.0" 2024-09-11T08:49:21.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:21.917 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:21.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:21.918 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:22.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:22.056 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:22.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:22.057 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:22.158 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:22.194 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:22.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:22.195 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:22.296 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:22.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:21 smithi092 ceph-mon[29647]: pgmap v452: 148 pgs: 1 peering, 147 active+clean; 675 MiB data, 3.7 GiB used, 622 GiB / 626 GiB avail; 1.6 KiB/s rd, 107 MiB/s wr, 1.71k op/s; 17 B/s, 0 objects/s recovering 2024-09-11T08:49:22.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:21 smithi092 ceph-mon[41114]: pgmap v452: 148 pgs: 1 peering, 147 active+clean; 675 MiB data, 3.7 GiB used, 622 GiB / 626 GiB avail; 1.6 KiB/s rd, 107 MiB/s wr, 1.71k op/s; 17 B/s, 0 objects/s recovering 2024-09-11T08:49:22.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:22.330 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:22.330 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:22.353 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:21 smithi145 ceph-mon[33384]: pgmap v452: 148 pgs: 1 peering, 147 active+clean; 675 MiB data, 3.7 GiB used, 622 GiB / 626 GiB avail; 1.6 KiB/s rd, 107 MiB/s wr, 1.71k op/s; 17 B/s, 0 objects/s recovering 2024-09-11T08:49:22.378 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T08:49:22.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:22.464 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:22.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:22.465 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:22.566 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:22.604 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:22.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:22.605 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:22.706 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:22.720 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 8 16 24282 24266 189.562 156.938 0.00563927 0.00526898 2024-09-11T08:49:22.749 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:22.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:22.750 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:22.842 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:49:22.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:22.884 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:22.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:22.885 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:22.979 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:49:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:49:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:49:22.980 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:22 smithi145 ceph-mon[33384]: 6.16 scrub starts 2024-09-11T08:49:22.980 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:22 smithi145 ceph-mon[33384]: 6.16 scrub ok 2024-09-11T08:49:22.980 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:22 smithi145 ceph-mon[33384]: 4.a scrub starts 2024-09-11T08:49:22.980 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:22 smithi145 ceph-mon[33384]: 4.a scrub ok 2024-09-11T08:49:22.985 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:23.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:23.020 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:23.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:23.154 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:23.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:23.155 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:23.256 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:23.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:23.290 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:23.290 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:23.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:22 smithi092 ceph-mon[41114]: 6.16 scrub starts 2024-09-11T08:49:23.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:22 smithi092 ceph-mon[41114]: 6.16 scrub ok 2024-09-11T08:49:23.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:22 smithi092 ceph-mon[41114]: 4.a scrub starts 2024-09-11T08:49:23.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:22 smithi092 ceph-mon[41114]: 4.a scrub ok 2024-09-11T08:49:23.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:22 smithi092 ceph-mon[29647]: 6.16 scrub starts 2024-09-11T08:49:23.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:22 smithi092 ceph-mon[29647]: 6.16 scrub ok 2024-09-11T08:49:23.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:22 smithi092 ceph-mon[29647]: 4.a scrub starts 2024-09-11T08:49:23.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:22 smithi092 ceph-mon[29647]: 4.a scrub ok 2024-09-11T08:49:23.391 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:23.425 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:23.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:23.426 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:23.526 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:23.565 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:23.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:23.566 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:23.667 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:23.700 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:23.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:23.701 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:23.720 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 9 16 26484 26468 183.79 137.625 0.00650956 0.00543616 2024-09-11T08:49:23.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:23.841 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-11T08:49:19.222043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":393615580,"num_objects":6193,"num_object_clones":0,"num_object_copies":18579,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6193,"num_whiteouts":0,"num_read":1896,"num_read_kb":1753,"num_write":6826,"num_write_kb":385479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":1184547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6913,"ondisk_log_size":6913,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":445,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2960664,"kb_used_data":833392,"kb_used_omap":11,"kb_used_meta":2127156,"kb_avail":653255400,"statfs":{"total":671965249536,"available":668933529600,"internally_reserved":0,"allocated":853393408,"data_stored":850980486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2178208104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":393019392,"num_objects":5997,"num_object_clones":0,"num_object_copies":17991,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5997,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5997,"num_write_kb":383808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001395"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.922563+0000","last_change":"2024-09-11T08:49:09.906451+0000","last_active":"2024-09-11T08:49:14.922563+0000","last_peered":"2024-09-11T08:49:14.922563+0000","last_clean":"2024-09-11T08:49:14.922563+0000","last_became_active":"2024-09-11T08:49:09.905661+0000","last_became_peered":"2024-09-11T08:49:09.905661+0000","last_unstale":"2024-09-11T08:49:14.922563+0000","last_undegraded":"2024-09-11T08:49:14.922563+0000","last_fullsized":"2024-09-11T08:49:14.922563+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:47.838700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:47.838700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:53.194173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"200'19","reported_seq":264,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920398+0000","last_change":"2024-09-11T08:46:01.991434+0000","last_active":"2024-09-11T08:49:14.920398+0000","last_peered":"2024-09-11T08:49:14.920398+0000","last_clean":"2024-09-11T08:49:14.920398+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:49:14.920398+0000","last_undegraded":"2024-09-11T08:49:14.920398+0000","last_fullsized":"2024-09-11T08:49:14.920398+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:45:13.113315+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.113315+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018068470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":212,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.925658+0000","last_change":"2024-09-11T08:47:44.884292+0000","last_active":"2024-09-11T08:49:14.925658+0000","last_peered":"2024-09-11T08:49:14.925658+0000","last_clean":"2024-09-11T08:49:14.925658+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:49:14.925658+0000","last_undegraded":"2024-09-11T08:49:14.925658+0000","last_fullsized":"2024-09-11T08:49:14.925658+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.884217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.884217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011480829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":181,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920940+0000","last_change":"2024-09-11T08:44:26.005322+0000","last_active":"2024-09-11T08:49:14.920940+0000","last_peered":"2024-09-11T08:49:14.920940+0000","last_clean":"2024-09-11T08:49:14.920940+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:49:14.920940+0000","last_undegraded":"2024-09-11T08:49:14.920940+0000","last_fullsized":"2024-09-11T08:49:14.920940+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.005270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.005270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038675070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":220,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923551+0000","last_change":"2024-09-11T08:48:34.059380+0000","last_active":"2024-09-11T08:49:14.923551+0000","last_peered":"2024-09-11T08:49:14.923551+0000","last_clean":"2024-09-11T08:49:14.923551+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:49:14.923551+0000","last_undegraded":"2024-09-11T08:49:14.923551+0000","last_fullsized":"2024-09-11T08:49:14.923551+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:34.059309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:34.059309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:59.700862+0000","scrub_duration":0.0067175359999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":226,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923206+0000","last_change":"2024-09-11T08:48:35.046165+0000","last_active":"2024-09-11T08:49:14.923206+0000","last_peered":"2024-09-11T08:49:14.923206+0000","last_clean":"2024-09-11T08:49:14.923206+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:49:14.923206+0000","last_undegraded":"2024-09-11T08:49:14.923206+0000","last_fullsized":"2024-09-11T08:49:14.923206+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:35.046075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:35.046075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:46.499902+0000","scrub_duration":0.010511866,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"200'34","reported_seq":280,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.925472+0000","last_change":"2024-09-11T08:49:09.927417+0000","last_active":"2024-09-11T08:49:14.925472+0000","last_peered":"2024-09-11T08:49:14.925472+0000","last_clean":"2024-09-11T08:49:14.925472+0000","last_became_active":"2024-09-11T08:49:09.909107+0000","last_became_peered":"2024-09-11T08:49:09.909107+0000","last_unstale":"2024-09-11T08:49:14.925472+0000","last_undegraded":"2024-09-11T08:49:14.925472+0000","last_fullsized":"2024-09-11T08:49:14.925472+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:48:30.029817+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.029817+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:55.427664+0000","scrub_duration":0.0055582840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":193,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920321+0000","last_change":"2024-09-11T08:48:32.827213+0000","last_active":"2024-09-11T08:49:14.920321+0000","last_peered":"2024-09-11T08:49:14.920321+0000","last_clean":"2024-09-11T08:49:14.920321+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:49:14.920321+0000","last_undegraded":"2024-09-11T08:49:14.920321+0000","last_fullsized":"2024-09-11T08:49:14.920321+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:32.827110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:48:32.827110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:37.036882+0000","scrub_duration":0.0029807079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":232,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.922160+0000","last_change":"2024-09-11T08:48:41.415476+0000","last_active":"2024-09-11T08:49:14.922160+0000","last_peered":"2024-09-11T08:49:14.922160+0000","last_clean":"2024-09-11T08:49:14.922160+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:49:14.922160+0000","last_undegraded":"2024-09-11T08:49:14.922160+0000","last_fullsized":"2024-09-11T08:49:14.922160+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:41.415403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.415403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:46.015177+0000","scrub_duration":0.0017214509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"200'19","reported_seq":244,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921141+0000","last_change":"2024-09-11T08:49:09.922025+0000","last_active":"2024-09-11T08:49:14.921141+0000","last_peered":"2024-09-11T08:49:14.921141+0000","last_clean":"2024-09-11T08:49:14.921141+0000","last_became_active":"2024-09-11T08:49:09.910441+0000","last_became_peered":"2024-09-11T08:49:09.910441+0000","last_unstale":"2024-09-11T08:49:14.921141+0000","last_undegraded":"2024-09-11T08:49:14.921141+0000","last_fullsized":"2024-09-11T08:49:14.921141+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'13","last_scrub_stamp":"2024-09-11T08:44:27.963689+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:44:27.963689+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011544877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":218,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920268+0000","last_change":"2024-09-11T08:49:09.900683+0000","last_active":"2024-09-11T08:49:14.920268+0000","last_peered":"2024-09-11T08:49:14.920268+0000","last_clean":"2024-09-11T08:49:14.920268+0000","last_became_active":"2024-09-11T08:49:09.899188+0000","last_became_peered":"2024-09-11T08:49:09.899188+0000","last_unstale":"2024-09-11T08:49:14.920268+0000","last_undegraded":"2024-09-11T08:49:14.920268+0000","last_fullsized":"2024-09-11T08:49:14.920268+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:18.341709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.341709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:33.530991+0000","scrub_duration":0.0038165500000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":16,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.928257+0000","last_change":"2024-09-11T08:49:09.908252+0000","last_active":"2024-09-11T08:49:14.928257+0000","last_peered":"2024-09-11T08:49:14.928257+0000","last_clean":"2024-09-11T08:49:14.928257+0000","last_became_active":"2024-09-11T08:49:09.907856+0000","last_became_peered":"2024-09-11T08:49:09.907856+0000","last_unstale":"2024-09-11T08:49:14.928257+0000","last_undegraded":"2024-09-11T08:49:14.928257+0000","last_fullsized":"2024-09-11T08:49:14.928257+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.457652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.457652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":212,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920332+0000","last_change":"2024-09-11T08:48:44.398447+0000","last_active":"2024-09-11T08:49:14.920332+0000","last_peered":"2024-09-11T08:49:14.920332+0000","last_clean":"2024-09-11T08:49:14.920332+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:49:14.920332+0000","last_undegraded":"2024-09-11T08:49:14.920332+0000","last_fullsized":"2024-09-11T08:49:14.920332+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:44.398373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.398373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:53.243691+0000","scrub_duration":0.0030847359999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"200'7","reported_seq":208,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921275+0000","last_change":"2024-09-11T08:47:44.866144+0000","last_active":"2024-09-11T08:49:14.921275+0000","last_peered":"2024-09-11T08:49:14.921275+0000","last_clean":"2024-09-11T08:49:14.921275+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:49:14.921275+0000","last_undegraded":"2024-09-11T08:49:14.921275+0000","last_fullsized":"2024-09-11T08:49:14.921275+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:47:44.866057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.866057+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020690169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":212,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923846+0000","last_change":"2024-09-11T08:47:46.410478+0000","last_active":"2024-09-11T08:49:14.923846+0000","last_peered":"2024-09-11T08:49:14.923846+0000","last_clean":"2024-09-11T08:49:14.923846+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:49:14.923846+0000","last_undegraded":"2024-09-11T08:49:14.923846+0000","last_fullsized":"2024-09-11T08:49:14.923846+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:46.410402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:47:46.410402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002594884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":196,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920110+0000","last_change":"2024-09-11T08:47:38.519964+0000","last_active":"2024-09-11T08:49:14.920110+0000","last_peered":"2024-09-11T08:49:14.920110+0000","last_clean":"2024-09-11T08:49:14.920110+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:49:14.920110+0000","last_undegraded":"2024-09-11T08:49:14.920110+0000","last_fullsized":"2024-09-11T08:49:14.920110+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.850684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.850684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058755379999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":211,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921849+0000","last_change":"2024-09-11T08:46:01.978488+0000","last_active":"2024-09-11T08:49:14.921849+0000","last_peered":"2024-09-11T08:49:14.921849+0000","last_clean":"2024-09-11T08:49:14.921849+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:49:14.921849+0000","last_undegraded":"2024-09-11T08:49:14.921849+0000","last_fullsized":"2024-09-11T08:49:14.921849+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:24.197149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.197149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001510267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921423+0000","last_change":"2024-09-11T08:49:09.908672+0000","last_active":"2024-09-11T08:49:14.921423+0000","last_peered":"2024-09-11T08:49:14.921423+0000","last_clean":"2024-09-11T08:49:14.921423+0000","last_became_active":"2024-09-11T08:49:09.908237+0000","last_became_peered":"2024-09-11T08:49:09.908237+0000","last_unstale":"2024-09-11T08:49:14.921423+0000","last_undegraded":"2024-09-11T08:49:14.921423+0000","last_fullsized":"2024-09-11T08:49:14.921423+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:34.610331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:34.610331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"200'34","reported_seq":280,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.925133+0000","last_change":"2024-09-11T08:48:20.311201+0000","last_active":"2024-09-11T08:49:14.925133+0000","last_peered":"2024-09-11T08:49:14.925133+0000","last_clean":"2024-09-11T08:49:14.925133+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:49:14.925133+0000","last_undegraded":"2024-09-11T08:49:14.925133+0000","last_fullsized":"2024-09-11T08:49:14.925133+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:48:20.311122+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.311122+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:35.297396+0000","scrub_duration":0.011484708999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":222,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.927814+0000","last_change":"2024-09-11T08:49:09.903399+0000","last_active":"2024-09-11T08:49:14.927814+0000","last_peered":"2024-09-11T08:49:14.927814+0000","last_clean":"2024-09-11T08:49:14.927814+0000","last_became_active":"2024-09-11T08:49:09.903078+0000","last_became_peered":"2024-09-11T08:49:09.903078+0000","last_unstale":"2024-09-11T08:49:14.927814+0000","last_undegraded":"2024-09-11T08:49:14.927814+0000","last_fullsized":"2024-09-11T08:49:14.927814+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:19.961460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.961460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:47.571235+0000","scrub_duration":0.0091369829999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":219,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923862+0000","last_change":"2024-09-11T08:46:01.998163+0000","last_active":"2024-09-11T08:49:14.923862+0000","last_peered":"2024-09-11T08:49:14.923862+0000","last_clean":"2024-09-11T08:49:14.923862+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:49:14.923862+0000","last_undegraded":"2024-09-11T08:49:14.923862+0000","last_fullsized":"2024-09-11T08:49:14.923862+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:24.223237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.223237+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019917720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":197,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921265+0000","last_change":"2024-09-11T08:45:28.194198+0000","last_active":"2024-09-11T08:49:14.921265+0000","last_peered":"2024-09-11T08:49:14.921265+0000","last_clean":"2024-09-11T08:49:14.921265+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:49:14.921265+0000","last_undegraded":"2024-09-11T08:49:14.921265+0000","last_fullsized":"2024-09-11T08:49:14.921265+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.194095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.194095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001559936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"200'18","reported_seq":228,"reported_epoch":214,"state":"peering","last_fresh":"2024-09-11T08:49:15.931119+0000","last_change":"2024-09-11T08:49:15.931362+0000","last_active":"2024-09-11T08:49:15.931089+0000","last_peered":"2024-09-11T08:49:14.923374+0000","last_clean":"2024-09-11T08:49:14.923374+0000","last_became_active":"2024-09-11T08:38:40.974139+0000","last_became_peered":"2024-09-11T08:38:40.974139+0000","last_unstale":"2024-09-11T08:49:15.931119+0000","last_undegraded":"2024-09-11T08:49:15.931119+0000","last_fullsized":"2024-09-11T08:49:15.931119+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:16.162616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:16.162616+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001890833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":193,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921459+0000","last_change":"2024-09-11T08:45:22.244097+0000","last_active":"2024-09-11T08:49:14.921459+0000","last_peered":"2024-09-11T08:49:14.921459+0000","last_clean":"2024-09-11T08:49:14.921459+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:49:14.921459+0000","last_undegraded":"2024-09-11T08:49:14.921459+0000","last_fullsized":"2024-09-11T08:49:14.921459+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.244038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.244038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015252620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":201,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923973+0000","last_change":"2024-09-11T08:45:19.335163+0000","last_active":"2024-09-11T08:49:14.923973+0000","last_peered":"2024-09-11T08:49:14.923973+0000","last_clean":"2024-09-11T08:49:14.923973+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:49:14.923973+0000","last_undegraded":"2024-09-11T08:49:14.923973+0000","last_fullsized":"2024-09-11T08:49:14.923973+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:19.335065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:19.335065+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026052929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":216,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.925790+0000","last_change":"2024-09-11T08:48:25.971280+0000","last_active":"2024-09-11T08:49:14.925790+0000","last_peered":"2024-09-11T08:49:14.925790+0000","last_clean":"2024-09-11T08:49:14.925790+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:49:14.925790+0000","last_undegraded":"2024-09-11T08:49:14.925790+0000","last_fullsized":"2024-09-11T08:49:14.925790+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:25.971151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.971151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:47.552719+0000","scrub_duration":0.0095705709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"200'20","reported_seq":250,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924674+0000","last_change":"2024-09-11T08:49:09.936134+0000","last_active":"2024-09-11T08:49:14.924674+0000","last_peered":"2024-09-11T08:49:14.924674+0000","last_clean":"2024-09-11T08:49:14.924674+0000","last_became_active":"2024-09-11T08:49:09.913968+0000","last_became_peered":"2024-09-11T08:49:09.913968+0000","last_unstale":"2024-09-11T08:49:14.924674+0000","last_undegraded":"2024-09-11T08:49:14.924674+0000","last_fullsized":"2024-09-11T08:49:14.924674+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:48:19.705185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.705185+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:27.125934+0000","scrub_duration":0.0050623580000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":350,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926739+0000","last_change":"2024-09-11T08:48:17.812378+0000","last_active":"2024-09-11T08:49:14.926739+0000","last_peered":"2024-09-11T08:49:14.926739+0000","last_clean":"2024-09-11T08:49:14.926739+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:49:14.926739+0000","last_undegraded":"2024-09-11T08:49:14.926739+0000","last_fullsized":"2024-09-11T08:49:14.926739+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:48:17.812256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:17.812256+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:40.308022+0000","scrub_duration":0.010983438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"214'434","reported_seq":450,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.788003+0000","last_change":"2024-09-11T08:49:06.519462+0000","last_active":"2024-09-11T08:49:16.788003+0000","last_peered":"2024-09-11T08:49:16.788003+0000","last_clean":"2024-09-11T08:49:16.788003+0000","last_became_active":"2024-09-11T08:49:06.519147+0000","last_became_peered":"2024-09-11T08:49:06.519147+0000","last_unstale":"2024-09-11T08:49:16.788003+0000","last_undegraded":"2024-09-11T08:49:16.788003+0000","last_fullsized":"2024-09-11T08:49:16.788003+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:14.340457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28442624,"num_objects":434,"num_object_clones":0,"num_object_copies":1302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":434,"num_write_kb":27776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"200'34","reported_seq":254,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926752+0000","last_change":"2024-09-11T08:45:18.777528+0000","last_active":"2024-09-11T08:49:14.926752+0000","last_peered":"2024-09-11T08:49:14.926752+0000","last_clean":"2024-09-11T08:49:14.926752+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:49:14.926752+0000","last_undegraded":"2024-09-11T08:49:14.926752+0000","last_fullsized":"2024-09-11T08:49:14.926752+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:45:18.777466+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.777466+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045440580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":196,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926282+0000","last_change":"2024-09-11T08:45:13.214007+0000","last_active":"2024-09-11T08:49:14.926282+0000","last_peered":"2024-09-11T08:49:14.926282+0000","last_clean":"2024-09-11T08:49:14.926282+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:49:14.926282+0000","last_undegraded":"2024-09-11T08:49:14.926282+0000","last_fullsized":"2024-09-11T08:49:14.926282+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:13.213922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:13.213922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016325280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"211'10","reported_seq":343,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921929+0000","last_change":"2024-09-11T08:49:09.915699+0000","last_active":"2024-09-11T08:49:14.921929+0000","last_peered":"2024-09-11T08:49:14.921929+0000","last_clean":"2024-09-11T08:49:14.921929+0000","last_became_active":"2024-09-11T08:49:09.905262+0000","last_became_peered":"2024-09-11T08:49:09.905262+0000","last_unstale":"2024-09-11T08:49:14.921929+0000","last_undegraded":"2024-09-11T08:49:14.921929+0000","last_fullsized":"2024-09-11T08:49:14.921929+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'9","last_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.208818+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028933230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":203,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921721+0000","last_change":"2024-09-11T08:49:09.908339+0000","last_active":"2024-09-11T08:49:14.921721+0000","last_peered":"2024-09-11T08:49:14.921721+0000","last_clean":"2024-09-11T08:49:14.921721+0000","last_became_active":"2024-09-11T08:49:09.906872+0000","last_became_peered":"2024-09-11T08:49:09.906872+0000","last_unstale":"2024-09-11T08:49:14.921721+0000","last_undegraded":"2024-09-11T08:49:14.921721+0000","last_fullsized":"2024-09-11T08:49:14.921721+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:24.813857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:24.813857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01118625,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"214'320","reported_seq":336,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.157479+0000","last_change":"2024-09-11T08:49:05.875564+0000","last_active":"2024-09-11T08:49:16.157479+0000","last_peered":"2024-09-11T08:49:16.157479+0000","last_clean":"2024-09-11T08:49:16.157479+0000","last_became_active":"2024-09-11T08:49:05.875264+0000","last_became_peered":"2024-09-11T08:49:05.875264+0000","last_unstale":"2024-09-11T08:49:16.157479+0000","last_undegraded":"2024-09-11T08:49:16.157479+0000","last_fullsized":"2024-09-11T08:49:16.157479+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:10.708155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":320,"num_object_clones":0,"num_object_copies":960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"200'38","reported_seq":290,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926013+0000","last_change":"2024-09-11T08:49:09.936915+0000","last_active":"2024-09-11T08:49:14.926013+0000","last_peered":"2024-09-11T08:49:14.926013+0000","last_clean":"2024-09-11T08:49:14.926013+0000","last_became_active":"2024-09-11T08:49:09.917354+0000","last_became_peered":"2024-09-11T08:49:09.917354+0000","last_unstale":"2024-09-11T08:49:14.926013+0000","last_undegraded":"2024-09-11T08:49:14.926013+0000","last_fullsized":"2024-09-11T08:49:14.926013+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'26","last_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_deep_scrub":"199'26","last_deep_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.093994+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:10.328025+0000","scrub_duration":0.026938573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":216,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924316+0000","last_change":"2024-09-11T08:49:09.915472+0000","last_active":"2024-09-11T08:49:14.924316+0000","last_peered":"2024-09-11T08:49:14.924316+0000","last_clean":"2024-09-11T08:49:14.924316+0000","last_became_active":"2024-09-11T08:49:09.915191+0000","last_became_peered":"2024-09-11T08:49:09.915191+0000","last_unstale":"2024-09-11T08:49:14.924316+0000","last_undegraded":"2024-09-11T08:49:14.924316+0000","last_fullsized":"2024-09-11T08:49:14.924316+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:38.041303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:38.041303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:41.529572+0000","scrub_duration":0.0073485030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":193,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920491+0000","last_change":"2024-09-11T08:45:17.150827+0000","last_active":"2024-09-11T08:49:14.920491+0000","last_peered":"2024-09-11T08:49:14.920491+0000","last_clean":"2024-09-11T08:49:14.920491+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:49:14.920491+0000","last_undegraded":"2024-09-11T08:49:14.920491+0000","last_fullsized":"2024-09-11T08:49:14.920491+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.150767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.150767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001327707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":191,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921055+0000","last_change":"2024-09-11T08:46:03.144149+0000","last_active":"2024-09-11T08:49:14.921055+0000","last_peered":"2024-09-11T08:49:14.921055+0000","last_clean":"2024-09-11T08:49:14.921055+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:49:14.921055+0000","last_undegraded":"2024-09-11T08:49:14.921055+0000","last_fullsized":"2024-09-11T08:49:14.921055+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:46:03.144096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:46:03.144096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001497151,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"214'465","reported_seq":480,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.775578+0000","last_change":"2024-09-11T08:49:05.874640+0000","last_active":"2024-09-11T08:49:16.775578+0000","last_peered":"2024-09-11T08:49:16.775578+0000","last_clean":"2024-09-11T08:49:16.775578+0000","last_became_active":"2024-09-11T08:49:05.874365+0000","last_became_peered":"2024-09-11T08:49:05.874365+0000","last_unstale":"2024-09-11T08:49:16.775578+0000","last_undegraded":"2024-09-11T08:49:16.775578+0000","last_fullsized":"2024-09-11T08:49:16.775578+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:32.415086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30474240,"num_objects":465,"num_object_clones":0,"num_object_copies":1395,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":465,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"200'35","reported_seq":244,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920761+0000","last_change":"2024-09-11T08:48:23.862534+0000","last_active":"2024-09-11T08:49:14.920761+0000","last_peered":"2024-09-11T08:49:14.920761+0000","last_clean":"2024-09-11T08:49:14.920761+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:49:14.920761+0000","last_undegraded":"2024-09-11T08:49:14.920761+0000","last_fullsized":"2024-09-11T08:49:14.920761+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'21","last_scrub_stamp":"2024-09-11T08:48:23.862482+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:48:23.862482+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:27.496799+0000","scrub_duration":0.001795776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926330+0000","last_change":"2024-09-11T08:49:09.908334+0000","last_active":"2024-09-11T08:49:14.926330+0000","last_peered":"2024-09-11T08:49:14.926330+0000","last_clean":"2024-09-11T08:49:14.926330+0000","last_became_active":"2024-09-11T08:49:09.908041+0000","last_became_peered":"2024-09-11T08:49:09.908041+0000","last_unstale":"2024-09-11T08:49:14.926330+0000","last_undegraded":"2024-09-11T08:49:14.926330+0000","last_fullsized":"2024-09-11T08:49:14.926330+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:30.475365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:30.475365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":330,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.922627+0000","last_change":"2024-09-11T08:48:36.052650+0000","last_active":"2024-09-11T08:49:14.922627+0000","last_peered":"2024-09-11T08:49:14.922627+0000","last_clean":"2024-09-11T08:49:14.922627+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:49:14.922627+0000","last_undegraded":"2024-09-11T08:49:14.922627+0000","last_fullsized":"2024-09-11T08:49:14.922627+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:48:36.052514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:36.052514+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:47.227630+0000","scrub_duration":0.015857353000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":220,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923807+0000","last_change":"2024-09-11T08:46:01.983654+0000","last_active":"2024-09-11T08:49:14.923807+0000","last_peered":"2024-09-11T08:49:14.923807+0000","last_clean":"2024-09-11T08:49:14.923807+0000","last_became_active":"2024-09-11T08:46:01.983335+0000","last_became_peered":"2024-09-11T08:46:01.983335+0000","last_unstale":"2024-09-11T08:49:14.923807+0000","last_undegraded":"2024-09-11T08:49:14.923807+0000","last_fullsized":"2024-09-11T08:49:14.923807+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:17.303194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:17.303194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001492431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"214'371","reported_seq":386,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.303787+0000","last_change":"2024-09-11T08:49:05.871535+0000","last_active":"2024-09-11T08:49:16.303787+0000","last_peered":"2024-09-11T08:49:16.303787+0000","last_clean":"2024-09-11T08:49:16.303787+0000","last_became_active":"2024-09-11T08:49:05.871255+0000","last_became_peered":"2024-09-11T08:49:05.871255+0000","last_unstale":"2024-09-11T08:49:16.303787+0000","last_undegraded":"2024-09-11T08:49:16.303787+0000","last_fullsized":"2024-09-11T08:49:16.303787+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:09.000693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24313856,"num_objects":371,"num_object_clones":0,"num_object_copies":1113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":371,"num_write_kb":23744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"200'25","reported_seq":249,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926196+0000","last_change":"2024-09-11T08:48:38.032124+0000","last_active":"2024-09-11T08:49:14.926196+0000","last_peered":"2024-09-11T08:49:14.926196+0000","last_clean":"2024-09-11T08:49:14.926196+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:49:14.926196+0000","last_undegraded":"2024-09-11T08:49:14.926196+0000","last_fullsized":"2024-09-11T08:49:14.926196+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'15","last_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_clean_scrub_stamp":"2024-09-11T08:48:38.032039+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:48.925034+0000","scrub_duration":0.0049584310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":202,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.922227+0000","last_change":"2024-09-11T08:47:37.534624+0000","last_active":"2024-09-11T08:49:14.922227+0000","last_peered":"2024-09-11T08:49:14.922227+0000","last_clean":"2024-09-11T08:49:14.922227+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:49:14.922227+0000","last_undegraded":"2024-09-11T08:49:14.922227+0000","last_fullsized":"2024-09-11T08:49:14.922227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:19.905459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:44:19.905459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015766237999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"211'9","reported_seq":40,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926366+0000","last_change":"2024-09-11T08:49:09.920108+0000","last_active":"2024-09-11T08:49:14.926366+0000","last_peered":"2024-09-11T08:49:14.926366+0000","last_clean":"2024-09-11T08:49:14.926366+0000","last_became_active":"2024-09-11T08:49:09.910279+0000","last_became_peered":"2024-09-11T08:49:09.910279+0000","last_unstale":"2024-09-11T08:49:14.926366+0000","last_undegraded":"2024-09-11T08:49:14.926366+0000","last_fullsized":"2024-09-11T08:49:14.926366+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:48:44.796704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.796704+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:45.012801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":16,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.922548+0000","last_change":"2024-09-11T08:49:09.908727+0000","last_active":"2024-09-11T08:49:14.922548+0000","last_peered":"2024-09-11T08:49:14.922548+0000","last_clean":"2024-09-11T08:49:14.922548+0000","last_became_active":"2024-09-11T08:49:09.907944+0000","last_became_peered":"2024-09-11T08:49:09.907944+0000","last_unstale":"2024-09-11T08:49:14.922548+0000","last_undegraded":"2024-09-11T08:49:14.922548+0000","last_fullsized":"2024-09-11T08:49:14.922548+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:31.710797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:48:31.710797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:33.291331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"214'323","reported_seq":339,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.157262+0000","last_change":"2024-09-11T08:49:05.875096+0000","last_active":"2024-09-11T08:49:16.157262+0000","last_peered":"2024-09-11T08:49:16.157262+0000","last_clean":"2024-09-11T08:49:16.157262+0000","last_became_active":"2024-09-11T08:49:05.874803+0000","last_became_peered":"2024-09-11T08:49:05.874803+0000","last_unstale":"2024-09-11T08:49:16.157262+0000","last_undegraded":"2024-09-11T08:49:16.157262+0000","last_fullsized":"2024-09-11T08:49:16.157262+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:12.899212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21168128,"num_objects":323,"num_object_clones":0,"num_object_copies":969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":323,"num_write_kb":20672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"200'29","reported_seq":264,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920410+0000","last_change":"2024-09-11T08:49:09.928210+0000","last_active":"2024-09-11T08:49:14.920410+0000","last_peered":"2024-09-11T08:49:14.920410+0000","last_clean":"2024-09-11T08:49:14.920410+0000","last_became_active":"2024-09-11T08:49:09.901065+0000","last_became_peered":"2024-09-11T08:49:09.901065+0000","last_unstale":"2024-09-11T08:49:14.920410+0000","last_undegraded":"2024-09-11T08:49:14.920410+0000","last_fullsized":"2024-09-11T08:49:14.920410+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:45:11.576085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.576085+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002366754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":216,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.927869+0000","last_change":"2024-09-11T08:48:24.062209+0000","last_active":"2024-09-11T08:49:14.927869+0000","last_peered":"2024-09-11T08:49:14.927869+0000","last_clean":"2024-09-11T08:49:14.927869+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:49:14.927869+0000","last_undegraded":"2024-09-11T08:49:14.927869+0000","last_fullsized":"2024-09-11T08:49:14.927869+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:24.062110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:24.062110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:38.001551+0000","scrub_duration":0.037841910999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":222,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926125+0000","last_change":"2024-09-11T08:49:09.909596+0000","last_active":"2024-09-11T08:49:14.926125+0000","last_peered":"2024-09-11T08:49:14.926125+0000","last_clean":"2024-09-11T08:49:14.926125+0000","last_became_active":"2024-09-11T08:49:09.909306+0000","last_became_peered":"2024-09-11T08:49:09.909306+0000","last_unstale":"2024-09-11T08:49:14.926125+0000","last_undegraded":"2024-09-11T08:49:14.926125+0000","last_fullsized":"2024-09-11T08:49:14.926125+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:25.855301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:57.596889+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.855301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:25.949497+0000","scrub_duration":0.0039132810000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":230,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.925585+0000","last_change":"2024-09-11T08:48:27.981040+0000","last_active":"2024-09-11T08:49:14.925585+0000","last_peered":"2024-09-11T08:49:14.925585+0000","last_clean":"2024-09-11T08:49:14.925585+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:49:14.925585+0000","last_undegraded":"2024-09-11T08:49:14.925585+0000","last_fullsized":"2024-09-11T08:49:14.925585+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:27.980860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:48:27.980860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:30.102782+0000","scrub_duration":0.021574010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"214'334","reported_seq":347,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.150913+0000","last_change":"2024-09-11T08:49:09.913293+0000","last_active":"2024-09-11T08:49:16.150913+0000","last_peered":"2024-09-11T08:49:16.150913+0000","last_clean":"2024-09-11T08:49:16.150913+0000","last_became_active":"2024-09-11T08:49:09.912997+0000","last_became_peered":"2024-09-11T08:49:09.912997+0000","last_unstale":"2024-09-11T08:49:16.150913+0000","last_undegraded":"2024-09-11T08:49:16.150913+0000","last_fullsized":"2024-09-11T08:49:16.150913+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:12.200890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21889024,"num_objects":334,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":334,"num_write_kb":21376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"200'50","reported_seq":417,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.927585+0000","last_change":"2024-09-11T08:45:26.666261+0000","last_active":"2024-09-11T08:49:14.927585+0000","last_peered":"2024-09-11T08:49:14.927585+0000","last_clean":"2024-09-11T08:49:14.927585+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:49:14.927585+0000","last_undegraded":"2024-09-11T08:49:14.927585+0000","last_fullsized":"2024-09-11T08:49:14.927585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:45:26.666199+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.666199+0000","objects_scrubbed":10,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020004279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":207,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921586+0000","last_change":"2024-09-11T08:49:09.908511+0000","last_active":"2024-09-11T08:49:14.921586+0000","last_peered":"2024-09-11T08:49:14.921586+0000","last_clean":"2024-09-11T08:49:14.921586+0000","last_became_active":"2024-09-11T08:49:09.907850+0000","last_became_peered":"2024-09-11T08:49:09.907850+0000","last_unstale":"2024-09-11T08:49:14.921586+0000","last_undegraded":"2024-09-11T08:49:14.921586+0000","last_fullsized":"2024-09-11T08:49:14.921586+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.130572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.130572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019779749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":247,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921050+0000","last_change":"2024-09-11T08:48:16.376429+0000","last_active":"2024-09-11T08:49:14.921050+0000","last_peered":"2024-09-11T08:49:14.921050+0000","last_clean":"2024-09-11T08:49:14.921050+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:49:14.921050+0000","last_undegraded":"2024-09-11T08:49:14.921050+0000","last_fullsized":"2024-09-11T08:49:14.921050+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:48:16.376340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:16.376340+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:46.168511+0000","scrub_duration":0.00247796,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":16,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926915+0000","last_change":"2024-09-11T08:49:09.903912+0000","last_active":"2024-09-11T08:49:14.926915+0000","last_peered":"2024-09-11T08:49:14.926915+0000","last_clean":"2024-09-11T08:49:14.926915+0000","last_became_active":"2024-09-11T08:49:09.903614+0000","last_became_peered":"2024-09-11T08:49:09.903614+0000","last_unstale":"2024-09-11T08:49:14.926915+0000","last_undegraded":"2024-09-11T08:49:14.926915+0000","last_fullsized":"2024-09-11T08:49:14.926915+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:25.682632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.682632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:38.248660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":190,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924365+0000","last_change":"2024-09-11T08:45:22.169436+0000","last_active":"2024-09-11T08:49:14.924365+0000","last_peered":"2024-09-11T08:49:14.924365+0000","last_clean":"2024-09-11T08:49:14.924365+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:49:14.924365+0000","last_undegraded":"2024-09-11T08:49:14.924365+0000","last_fullsized":"2024-09-11T08:49:14.924365+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.169367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.169367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005349693,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"214'364","reported_seq":379,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.306127+0000","last_change":"2024-09-11T08:49:05.872817+0000","last_active":"2024-09-11T08:49:16.306127+0000","last_peered":"2024-09-11T08:49:16.306127+0000","last_clean":"2024-09-11T08:49:16.306127+0000","last_became_active":"2024-09-11T08:49:05.872392+0000","last_became_peered":"2024-09-11T08:49:05.872392+0000","last_unstale":"2024-09-11T08:49:16.306127+0000","last_undegraded":"2024-09-11T08:49:16.306127+0000","last_fullsized":"2024-09-11T08:49:16.306127+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:05.342707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23855104,"num_objects":364,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":364,"num_write_kb":23296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"200'17","reported_seq":269,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926831+0000","last_change":"2024-09-11T08:49:09.928583+0000","last_active":"2024-09-11T08:49:14.926831+0000","last_peered":"2024-09-11T08:49:14.926831+0000","last_clean":"2024-09-11T08:49:14.926831+0000","last_became_active":"2024-09-11T08:49:09.902884+0000","last_became_peered":"2024-09-11T08:49:09.902884+0000","last_unstale":"2024-09-11T08:49:14.926831+0000","last_undegraded":"2024-09-11T08:49:14.926831+0000","last_fullsized":"2024-09-11T08:49:14.926831+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:45:27.666502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.666502+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023582730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":232,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.927555+0000","last_change":"2024-09-11T08:48:47.958737+0000","last_active":"2024-09-11T08:49:14.927555+0000","last_peered":"2024-09-11T08:49:14.927555+0000","last_clean":"2024-09-11T08:49:14.927555+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:49:14.927555+0000","last_undegraded":"2024-09-11T08:49:14.927555+0000","last_fullsized":"2024-09-11T08:49:14.927555+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.775804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.775804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011899289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":250,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924531+0000","last_change":"2024-09-11T08:49:09.923415+0000","last_active":"2024-09-11T08:49:14.924531+0000","last_peered":"2024-09-11T08:49:14.924531+0000","last_clean":"2024-09-11T08:49:14.924531+0000","last_became_active":"2024-09-11T08:49:09.909804+0000","last_became_peered":"2024-09-11T08:49:09.909804+0000","last_unstale":"2024-09-11T08:49:14.924531+0000","last_undegraded":"2024-09-11T08:49:14.924531+0000","last_fullsized":"2024-09-11T08:49:14.924531+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:48:28.359095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:28.359095+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:29.087007+0000","scrub_duration":0.0084904620000000007,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":232,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:15.477607+0000","last_change":"2024-09-11T08:49:15.477556+0000","last_active":"2024-09-11T08:49:15.477607+0000","last_peered":"2024-09-11T08:49:15.477607+0000","last_clean":"2024-09-11T08:49:15.477607+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:49:15.477607+0000","last_undegraded":"2024-09-11T08:49:15.477607+0000","last_fullsized":"2024-09-11T08:49:15.477607+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:15.477483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_clean_scrub_stamp":"2024-09-11T08:49:15.477483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:35.478111+0000","scrub_duration":0.0049012409999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":230,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924214+0000","last_change":"2024-09-11T08:48:44.031924+0000","last_active":"2024-09-11T08:49:14.924214+0000","last_peered":"2024-09-11T08:49:14.924214+0000","last_clean":"2024-09-11T08:49:14.924214+0000","last_became_active":"2024-09-11T08:46:01.983095+0000","last_became_peered":"2024-09-11T08:46:01.983095+0000","last_unstale":"2024-09-11T08:49:14.924214+0000","last_undegraded":"2024-09-11T08:49:14.924214+0000","last_fullsized":"2024-09-11T08:49:14.924214+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:44.031842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.031842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:55.851846+0000","scrub_duration":0.0021799599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"214'324","reported_seq":340,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.154688+0000","last_change":"2024-09-11T08:49:09.909742+0000","last_active":"2024-09-11T08:49:16.154688+0000","last_peered":"2024-09-11T08:49:16.154688+0000","last_clean":"2024-09-11T08:49:16.154688+0000","last_became_active":"2024-09-11T08:49:09.909429+0000","last_became_peered":"2024-09-11T08:49:09.909429+0000","last_unstale":"2024-09-11T08:49:16.154688+0000","last_undegraded":"2024-09-11T08:49:16.154688+0000","last_fullsized":"2024-09-11T08:49:16.154688+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:19.361954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":324,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":324,"num_write_kb":20736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"200'24","reported_seq":255,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926262+0000","last_change":"2024-09-11T08:49:09.930720+0000","last_active":"2024-09-11T08:49:14.926262+0000","last_peered":"2024-09-11T08:49:14.926262+0000","last_clean":"2024-09-11T08:49:14.926262+0000","last_became_active":"2024-09-11T08:49:09.911956+0000","last_became_peered":"2024-09-11T08:49:09.911956+0000","last_unstale":"2024-09-11T08:49:14.926262+0000","last_undegraded":"2024-09-11T08:49:14.926262+0000","last_fullsized":"2024-09-11T08:49:14.926262+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:48:37.680688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:37.680688+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:50.208675+0000","scrub_duration":0.0037660860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920583+0000","last_change":"2024-09-11T08:49:09.910130+0000","last_active":"2024-09-11T08:49:14.920583+0000","last_peered":"2024-09-11T08:49:14.920583+0000","last_clean":"2024-09-11T08:49:14.920583+0000","last_became_active":"2024-09-11T08:49:09.909803+0000","last_became_peered":"2024-09-11T08:49:09.909803+0000","last_unstale":"2024-09-11T08:49:14.920583+0000","last_undegraded":"2024-09-11T08:49:14.920583+0000","last_fullsized":"2024-09-11T08:49:14.920583+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:20.455081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.455081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":867,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:15.923278+0000","last_change":"2024-09-11T08:49:09.929102+0000","last_active":"2024-09-11T08:49:15.923278+0000","last_peered":"2024-09-11T08:49:15.923278+0000","last_clean":"2024-09-11T08:49:15.923278+0000","last_became_active":"2024-09-11T08:49:09.912659+0000","last_became_peered":"2024-09-11T08:49:09.912659+0000","last_unstale":"2024-09-11T08:49:15.923278+0000","last_undegraded":"2024-09-11T08:49:15.923278+0000","last_fullsized":"2024-09-11T08:49:15.923278+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:45:21.370673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.370673+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017307240000000001,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":628,"num_read_kb":623,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":218,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.922485+0000","last_change":"2024-09-11T08:49:09.903588+0000","last_active":"2024-09-11T08:49:14.922485+0000","last_peered":"2024-09-11T08:49:14.922485+0000","last_clean":"2024-09-11T08:49:14.922485+0000","last_became_active":"2024-09-11T08:49:09.903283+0000","last_became_peered":"2024-09-11T08:49:09.903283+0000","last_unstale":"2024-09-11T08:49:14.922485+0000","last_undegraded":"2024-09-11T08:49:14.922485+0000","last_fullsized":"2024-09-11T08:49:14.922485+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:30.532701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.532701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:58.509201+0000","scrub_duration":0.012853409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":210,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.927340+0000","last_change":"2024-09-11T08:48:33.025967+0000","last_active":"2024-09-11T08:49:14.927340+0000","last_peered":"2024-09-11T08:49:14.927340+0000","last_clean":"2024-09-11T08:49:14.927340+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:49:14.927340+0000","last_undegraded":"2024-09-11T08:49:14.927340+0000","last_fullsized":"2024-09-11T08:49:14.927340+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:33.025897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:48:33.025897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:50.371310+0000","scrub_duration":0.007471208,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"214'424","reported_seq":450,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.780571+0000","last_change":"2024-09-11T08:49:09.915717+0000","last_active":"2024-09-11T08:49:16.780571+0000","last_peered":"2024-09-11T08:49:16.780571+0000","last_clean":"2024-09-11T08:49:16.780571+0000","last_became_active":"2024-09-11T08:49:09.914863+0000","last_became_peered":"2024-09-11T08:49:09.914863+0000","last_unstale":"2024-09-11T08:49:16.780571+0000","last_undegraded":"2024-09-11T08:49:16.780571+0000","last_fullsized":"2024-09-11T08:49:16.780571+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:26.375120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27787264,"num_objects":424,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":27136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"201'27","reported_seq":111,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926506+0000","last_change":"2024-09-11T08:48:47.827201+0000","last_active":"2024-09-11T08:49:14.926506+0000","last_peered":"2024-09-11T08:49:14.926506+0000","last_clean":"2024-09-11T08:49:14.926506+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:49:14.926506+0000","last_undegraded":"2024-09-11T08:49:14.926506+0000","last_fullsized":"2024-09-11T08:49:14.926506+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'27","last_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_deep_scrub":"201'27","last_deep_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_clean_scrub_stamp":"2024-09-11T08:48:47.827134+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:12.693583+0000","scrub_duration":0.0038402419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":205,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923665+0000","last_change":"2024-09-11T08:45:18.127946+0000","last_active":"2024-09-11T08:49:14.923665+0000","last_peered":"2024-09-11T08:49:14.923665+0000","last_clean":"2024-09-11T08:49:14.923665+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:49:14.923665+0000","last_undegraded":"2024-09-11T08:49:14.923665+0000","last_fullsized":"2024-09-11T08:49:14.923665+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:18.127857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.127857+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018334099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"199'81","reported_seq":278,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923646+0000","last_change":"2024-09-11T08:49:09.926313+0000","last_active":"2024-09-11T08:49:14.923646+0000","last_peered":"2024-09-11T08:49:14.923646+0000","last_clean":"2024-09-11T08:49:14.923646+0000","last_became_active":"2024-09-11T08:49:09.904681+0000","last_became_peered":"2024-09-11T08:49:09.904681+0000","last_unstale":"2024-09-11T08:49:14.923646+0000","last_undegraded":"2024-09-11T08:49:14.923646+0000","last_fullsized":"2024-09-11T08:49:14.923646+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:05.407039+0000","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":212,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924094+0000","last_change":"2024-09-11T08:47:35.433723+0000","last_active":"2024-09-11T08:49:14.924094+0000","last_peered":"2024-09-11T08:49:14.924094+0000","last_clean":"2024-09-11T08:49:14.924094+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:49:14.924094+0000","last_undegraded":"2024-09-11T08:49:14.924094+0000","last_fullsized":"2024-09-11T08:49:14.924094+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:35.433651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:35.433651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013907892999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":206,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.922532+0000","last_change":"2024-09-11T08:48:19.308968+0000","last_active":"2024-09-11T08:49:14.922532+0000","last_peered":"2024-09-11T08:49:14.922532+0000","last_clean":"2024-09-11T08:49:14.922532+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:49:14.922532+0000","last_undegraded":"2024-09-11T08:49:14.922532+0000","last_fullsized":"2024-09-11T08:49:14.922532+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:19.308869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.308869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:27.449816+0000","scrub_duration":0.011385526999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"214'323","reported_seq":339,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.151902+0000","last_change":"2024-09-11T08:49:05.874474+0000","last_active":"2024-09-11T08:49:16.151902+0000","last_peered":"2024-09-11T08:49:16.151902+0000","last_clean":"2024-09-11T08:49:16.151902+0000","last_became_active":"2024-09-11T08:49:05.874151+0000","last_became_peered":"2024-09-11T08:49:05.874151+0000","last_unstale":"2024-09-11T08:49:16.151902+0000","last_undegraded":"2024-09-11T08:49:16.151902+0000","last_fullsized":"2024-09-11T08:49:16.151902+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:33.548168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21168128,"num_objects":323,"num_object_clones":0,"num_object_copies":969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":323,"num_write_kb":20672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"200'19","reported_seq":295,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921941+0000","last_change":"2024-09-11T08:48:20.860266+0000","last_active":"2024-09-11T08:49:14.921941+0000","last_peered":"2024-09-11T08:49:14.921941+0000","last_clean":"2024-09-11T08:49:14.921941+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:49:14.921941+0000","last_undegraded":"2024-09-11T08:49:14.921941+0000","last_fullsized":"2024-09-11T08:49:14.921941+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:20.860186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.860186+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:43.681747+0000","scrub_duration":0.0045417560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":187,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920804+0000","last_change":"2024-09-11T08:44:22.010602+0000","last_active":"2024-09-11T08:49:14.920804+0000","last_peered":"2024-09-11T08:49:14.920804+0000","last_clean":"2024-09-11T08:49:14.920804+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:49:14.920804+0000","last_undegraded":"2024-09-11T08:49:14.920804+0000","last_fullsized":"2024-09-11T08:49:14.920804+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:22.010528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:44:22.010528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047628439999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":205,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921355+0000","last_change":"2024-09-11T08:47:41.425451+0000","last_active":"2024-09-11T08:49:14.921355+0000","last_peered":"2024-09-11T08:49:14.921355+0000","last_clean":"2024-09-11T08:49:14.921355+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:49:14.921355+0000","last_undegraded":"2024-09-11T08:49:14.921355+0000","last_fullsized":"2024-09-11T08:49:14.921355+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:44:26.726573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00117395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":210,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.922984+0000","last_change":"2024-09-11T08:46:01.988315+0000","last_active":"2024-09-11T08:49:14.922984+0000","last_peered":"2024-09-11T08:49:14.922984+0000","last_clean":"2024-09-11T08:49:14.922984+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:49:14.922984+0000","last_undegraded":"2024-09-11T08:49:14.922984+0000","last_fullsized":"2024-09-11T08:49:14.922984+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.167071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.167071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001796483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"214'386","reported_seq":401,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.522579+0000","last_change":"2024-09-11T08:49:06.519555+0000","last_active":"2024-09-11T08:49:16.522579+0000","last_peered":"2024-09-11T08:49:16.522579+0000","last_clean":"2024-09-11T08:49:16.522579+0000","last_became_active":"2024-09-11T08:49:06.519263+0000","last_became_peered":"2024-09-11T08:49:06.519263+0000","last_unstale":"2024-09-11T08:49:16.522579+0000","last_undegraded":"2024-09-11T08:49:16.522579+0000","last_fullsized":"2024-09-11T08:49:16.522579+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:05.582121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25296896,"num_objects":386,"num_object_clones":0,"num_object_copies":1158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":386,"num_write_kb":24704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"200'31","reported_seq":254,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.922103+0000","last_change":"2024-09-11T08:48:43.728009+0000","last_active":"2024-09-11T08:49:14.922103+0000","last_peered":"2024-09-11T08:49:14.922103+0000","last_clean":"2024-09-11T08:49:14.922103+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:49:14.922103+0000","last_undegraded":"2024-09-11T08:49:14.922103+0000","last_fullsized":"2024-09-11T08:49:14.922103+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:48:43.727938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:43.727938+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:49.803069+0000","scrub_duration":0.016024792999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":226,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926413+0000","last_change":"2024-09-11T08:48:28.828694+0000","last_active":"2024-09-11T08:49:14.926413+0000","last_peered":"2024-09-11T08:49:14.926413+0000","last_clean":"2024-09-11T08:49:14.926413+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:49:14.926413+0000","last_undegraded":"2024-09-11T08:49:14.926413+0000","last_fullsized":"2024-09-11T08:49:14.926413+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:28.828618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_clean_scrub_stamp":"2024-09-11T08:48:28.828618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:32.802521+0000","scrub_duration":0.0029516289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":222,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.927779+0000","last_change":"2024-09-11T08:49:09.903898+0000","last_active":"2024-09-11T08:49:14.927779+0000","last_peered":"2024-09-11T08:49:14.927779+0000","last_clean":"2024-09-11T08:49:14.927779+0000","last_became_active":"2024-09-11T08:49:09.903589+0000","last_became_peered":"2024-09-11T08:49:09.903589+0000","last_unstale":"2024-09-11T08:49:14.927779+0000","last_undegraded":"2024-09-11T08:49:14.927779+0000","last_fullsized":"2024-09-11T08:49:14.927779+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:22.989179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:22.989179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:39.554788+0000","scrub_duration":0.0033708700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":226,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.925707+0000","last_change":"2024-09-11T08:48:17.333093+0000","last_active":"2024-09-11T08:49:14.925707+0000","last_peered":"2024-09-11T08:49:14.925707+0000","last_clean":"2024-09-11T08:49:14.925707+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:49:14.925707+0000","last_undegraded":"2024-09-11T08:49:14.925707+0000","last_fullsized":"2024-09-11T08:49:14.925707+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:17.333005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:17.333005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:21.947510+0000","scrub_duration":0.0086773309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"214'468","reported_seq":484,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.776479+0000","last_change":"2024-09-11T08:49:06.077463+0000","last_active":"2024-09-11T08:49:16.776479+0000","last_peered":"2024-09-11T08:49:16.776479+0000","last_clean":"2024-09-11T08:49:16.776479+0000","last_became_active":"2024-09-11T08:49:06.077144+0000","last_became_peered":"2024-09-11T08:49:06.077144+0000","last_unstale":"2024-09-11T08:49:16.776479+0000","last_undegraded":"2024-09-11T08:49:16.776479+0000","last_fullsized":"2024-09-11T08:49:16.776479+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:14.241327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30670848,"num_objects":468,"num_object_clones":0,"num_object_copies":1404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":468,"num_write_kb":29952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"200'20","reported_seq":272,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921504+0000","last_change":"2024-09-11T08:48:21.885766+0000","last_active":"2024-09-11T08:49:14.921504+0000","last_peered":"2024-09-11T08:49:14.921504+0000","last_clean":"2024-09-11T08:49:14.921504+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:49:14.921504+0000","last_undegraded":"2024-09-11T08:49:14.921504+0000","last_fullsized":"2024-09-11T08:49:14.921504+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:48:21.885681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:21.885681+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:24.781430+0000","scrub_duration":0.0051165580000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":206,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924059+0000","last_change":"2024-09-11T08:49:09.911420+0000","last_active":"2024-09-11T08:49:14.924059+0000","last_peered":"2024-09-11T08:49:14.924059+0000","last_clean":"2024-09-11T08:49:14.924059+0000","last_became_active":"2024-09-11T08:49:09.911101+0000","last_became_peered":"2024-09-11T08:49:09.911101+0000","last_unstale":"2024-09-11T08:49:14.924059+0000","last_undegraded":"2024-09-11T08:49:14.924059+0000","last_fullsized":"2024-09-11T08:49:14.924059+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:32.219374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:32.219374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069622119999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":357,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924740+0000","last_change":"2024-09-11T08:47:47.542558+0000","last_active":"2024-09-11T08:49:14.924740+0000","last_peered":"2024-09-11T08:49:14.924740+0000","last_clean":"2024-09-11T08:49:14.924740+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:49:14.924740+0000","last_undegraded":"2024-09-11T08:49:14.924740+0000","last_fullsized":"2024-09-11T08:49:14.924740+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:47:47.542483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:47.542483+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:17.503546+0000","scrub_duration":0.15193762999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":191,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921838+0000","last_change":"2024-09-11T08:45:31.196687+0000","last_active":"2024-09-11T08:49:14.921838+0000","last_peered":"2024-09-11T08:49:14.921838+0000","last_clean":"2024-09-11T08:49:14.921838+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:49:14.921838+0000","last_undegraded":"2024-09-11T08:49:14.921838+0000","last_fullsized":"2024-09-11T08:49:14.921838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.196599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.196599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025248190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"214'348","reported_seq":362,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.303147+0000","last_change":"2024-09-11T08:49:06.519232+0000","last_active":"2024-09-11T08:49:16.303147+0000","last_peered":"2024-09-11T08:49:16.303147+0000","last_clean":"2024-09-11T08:49:16.303147+0000","last_became_active":"2024-09-11T08:49:06.518991+0000","last_became_peered":"2024-09-11T08:49:06.518991+0000","last_unstale":"2024-09-11T08:49:16.303147+0000","last_undegraded":"2024-09-11T08:49:16.303147+0000","last_fullsized":"2024-09-11T08:49:16.303147+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:34.806916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22806528,"num_objects":348,"num_object_clones":0,"num_object_copies":1044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":348,"num_write_kb":22272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"200'15","reported_seq":240,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.927420+0000","last_change":"2024-09-11T08:49:09.925808+0000","last_active":"2024-09-11T08:49:14.927420+0000","last_peered":"2024-09-11T08:49:14.927420+0000","last_clean":"2024-09-11T08:49:14.927420+0000","last_became_active":"2024-09-11T08:49:09.915666+0000","last_became_peered":"2024-09-11T08:49:09.915666+0000","last_unstale":"2024-09-11T08:49:14.927420+0000","last_undegraded":"2024-09-11T08:49:14.927420+0000","last_fullsized":"2024-09-11T08:49:14.927420+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:48:41.787219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.787219+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:08.422014+0000","scrub_duration":0.0060192070000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":209,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920856+0000","last_change":"2024-09-11T08:47:43.360558+0000","last_active":"2024-09-11T08:49:14.920856+0000","last_peered":"2024-09-11T08:49:14.920856+0000","last_clean":"2024-09-11T08:49:14.920856+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:49:14.920856+0000","last_undegraded":"2024-09-11T08:49:14.920856+0000","last_fullsized":"2024-09-11T08:49:14.920856+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:10.607288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:45:10.607288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016158909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":216,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.925565+0000","last_change":"2024-09-11T08:48:24.380508+0000","last_active":"2024-09-11T08:49:14.925565+0000","last_peered":"2024-09-11T08:49:14.925565+0000","last_clean":"2024-09-11T08:49:14.925565+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:49:14.925565+0000","last_undegraded":"2024-09-11T08:49:14.925565+0000","last_fullsized":"2024-09-11T08:49:14.925565+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:24.380424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:24.380424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:39.122714+0000","scrub_duration":0.018256864000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":16,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923711+0000","last_change":"2024-09-11T08:49:09.912893+0000","last_active":"2024-09-11T08:49:14.923711+0000","last_peered":"2024-09-11T08:49:14.923711+0000","last_clean":"2024-09-11T08:49:14.923711+0000","last_became_active":"2024-09-11T08:49:09.912556+0000","last_became_peered":"2024-09-11T08:49:09.912556+0000","last_unstale":"2024-09-11T08:49:14.923711+0000","last_undegraded":"2024-09-11T08:49:14.923711+0000","last_fullsized":"2024-09-11T08:49:14.923711+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:47.699453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:47:47.699453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"214'352","reported_seq":367,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.307783+0000","last_change":"2024-09-11T08:49:05.871158+0000","last_active":"2024-09-11T08:49:16.307783+0000","last_peered":"2024-09-11T08:49:16.307783+0000","last_clean":"2024-09-11T08:49:16.307783+0000","last_became_active":"2024-09-11T08:49:05.870797+0000","last_became_peered":"2024-09-11T08:49:05.870797+0000","last_unstale":"2024-09-11T08:49:16.307783+0000","last_undegraded":"2024-09-11T08:49:16.307783+0000","last_fullsized":"2024-09-11T08:49:16.307783+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:33.662137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23068672,"num_objects":352,"num_object_clones":0,"num_object_copies":1056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":22528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"200'35","reported_seq":110,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926182+0000","last_change":"2024-09-11T08:47:46.582476+0000","last_active":"2024-09-11T08:49:14.926182+0000","last_peered":"2024-09-11T08:49:14.926182+0000","last_clean":"2024-09-11T08:49:14.926182+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:49:14.926182+0000","last_undegraded":"2024-09-11T08:49:14.926182+0000","last_fullsized":"2024-09-11T08:49:14.926182+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:47:46.582408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:46.582408+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006189015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":216,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924761+0000","last_change":"2024-09-11T08:47:44.411099+0000","last_active":"2024-09-11T08:49:14.924761+0000","last_peered":"2024-09-11T08:49:14.924761+0000","last_clean":"2024-09-11T08:49:14.924761+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:49:14.924761+0000","last_undegraded":"2024-09-11T08:49:14.924761+0000","last_fullsized":"2024-09-11T08:49:14.924761+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.411022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.411022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047762170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":232,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924651+0000","last_change":"2024-09-11T08:49:09.907847+0000","last_active":"2024-09-11T08:49:14.924651+0000","last_peered":"2024-09-11T08:49:14.924651+0000","last_clean":"2024-09-11T08:49:14.924651+0000","last_became_active":"2024-09-11T08:49:09.907542+0000","last_became_peered":"2024-09-11T08:49:09.907542+0000","last_unstale":"2024-09-11T08:49:14.924651+0000","last_undegraded":"2024-09-11T08:49:14.924651+0000","last_fullsized":"2024-09-11T08:49:14.924651+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:43.361491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:43.361491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055115299999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":203,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923694+0000","last_change":"2024-09-11T08:48:33.065142+0000","last_active":"2024-09-11T08:49:14.923694+0000","last_peered":"2024-09-11T08:49:14.923694+0000","last_clean":"2024-09-11T08:49:14.923694+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:49:14.923694+0000","last_undegraded":"2024-09-11T08:49:14.923694+0000","last_fullsized":"2024-09-11T08:49:14.923694+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:33.065053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:33.065053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:54.909406+0000","scrub_duration":0.013998086999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"214'415","reported_seq":430,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.523534+0000","last_change":"2024-09-11T08:49:05.871175+0000","last_active":"2024-09-11T08:49:16.523534+0000","last_peered":"2024-09-11T08:49:16.523534+0000","last_clean":"2024-09-11T08:49:16.523534+0000","last_became_active":"2024-09-11T08:49:05.870890+0000","last_became_peered":"2024-09-11T08:49:05.870890+0000","last_unstale":"2024-09-11T08:49:16.523534+0000","last_undegraded":"2024-09-11T08:49:16.523534+0000","last_fullsized":"2024-09-11T08:49:16.523534+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:21.232303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27197440,"num_objects":415,"num_object_clones":0,"num_object_copies":1245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":415,"num_write_kb":26560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"200'18","reported_seq":227,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920672+0000","last_change":"2024-09-11T08:45:08.140910+0000","last_active":"2024-09-11T08:49:14.920672+0000","last_peered":"2024-09-11T08:49:14.920672+0000","last_clean":"2024-09-11T08:49:14.920672+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:49:14.920672+0000","last_undegraded":"2024-09-11T08:49:14.920672+0000","last_fullsized":"2024-09-11T08:49:14.920672+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:08.140839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.140839+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003679547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":216,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924762+0000","last_change":"2024-09-11T08:48:47.033544+0000","last_active":"2024-09-11T08:49:14.924762+0000","last_peered":"2024-09-11T08:49:14.924762+0000","last_clean":"2024-09-11T08:49:14.924762+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:49:14.924762+0000","last_undegraded":"2024-09-11T08:49:14.924762+0000","last_fullsized":"2024-09-11T08:49:14.924762+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:47.033487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:47.033487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:51.790809+0000","scrub_duration":0.0014934150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":235,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924489+0000","last_change":"2024-09-11T08:49:09.907865+0000","last_active":"2024-09-11T08:49:14.924489+0000","last_peered":"2024-09-11T08:49:14.924489+0000","last_clean":"2024-09-11T08:49:14.924489+0000","last_became_active":"2024-09-11T08:49:09.907535+0000","last_became_peered":"2024-09-11T08:49:09.907535+0000","last_unstale":"2024-09-11T08:49:14.924489+0000","last_undegraded":"2024-09-11T08:49:14.924489+0000","last_fullsized":"2024-09-11T08:49:14.924489+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:21.336766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:21.336766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:38.274762+0000","scrub_duration":0.0077730009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":190,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923771+0000","last_change":"2024-09-11T08:45:33.228265+0000","last_active":"2024-09-11T08:49:14.923771+0000","last_peered":"2024-09-11T08:49:14.923771+0000","last_clean":"2024-09-11T08:49:14.923771+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:49:14.923771+0000","last_undegraded":"2024-09-11T08:49:14.923771+0000","last_fullsized":"2024-09-11T08:49:14.923771+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:45:33.228082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064426500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"214'346","reported_seq":362,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-11T08:49:16.295852+0000","last_change":"2024-09-11T08:49:09.900869+0000","last_active":"2024-09-11T08:49:16.295852+0000","last_peered":"2024-09-11T08:49:16.295852+0000","last_clean":"2024-09-11T08:49:16.295852+0000","last_became_active":"2024-09-11T08:49:09.900548+0000","last_became_peered":"2024-09-11T08:49:09.900548+0000","last_unstale":"2024-09-11T08:49:16.295852+0000","last_undegraded":"2024-09-11T08:49:16.295852+0000","last_fullsized":"2024-09-11T08:49:16.295852+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:08.175978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22675456,"num_objects":346,"num_object_clones":0,"num_object_copies":1038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":346,"num_write_kb":22144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"200'34","reported_seq":312,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.919908+0000","last_change":"2024-09-11T08:48:20.352021+0000","last_active":"2024-09-11T08:49:14.919908+0000","last_peered":"2024-09-11T08:49:14.919908+0000","last_clean":"2024-09-11T08:49:14.919908+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:49:14.919908+0000","last_undegraded":"2024-09-11T08:49:14.919908+0000","last_fullsized":"2024-09-11T08:49:14.919908+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:48:20.351963+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.351963+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:26.736295+0000","scrub_duration":0.0058139180000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":239,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924781+0000","last_change":"2024-09-11T08:49:09.921462+0000","last_active":"2024-09-11T08:49:14.924781+0000","last_peered":"2024-09-11T08:49:14.924781+0000","last_clean":"2024-09-11T08:49:14.924781+0000","last_became_active":"2024-09-11T08:49:09.904656+0000","last_became_peered":"2024-09-11T08:49:09.904656+0000","last_unstale":"2024-09-11T08:49:14.924781+0000","last_undegraded":"2024-09-11T08:49:14.924781+0000","last_fullsized":"2024-09-11T08:49:14.924781+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:45:20.372454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.372454+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019547269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"168'10","reported_seq":375,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923605+0000","last_change":"2024-09-11T08:48:41.038840+0000","last_active":"2024-09-11T08:49:14.923605+0000","last_peered":"2024-09-11T08:49:14.923605+0000","last_clean":"2024-09-11T08:49:14.923605+0000","last_became_active":"2024-09-11T08:46:01.983055+0000","last_became_peered":"2024-09-11T08:46:01.983055+0000","last_unstale":"2024-09-11T08:49:14.923605+0000","last_undegraded":"2024-09-11T08:49:14.923605+0000","last_fullsized":"2024-09-11T08:49:14.923605+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"168'10","last_scrub_stamp":"2024-09-11T08:48:41.038786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.038786+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:04.138712+0000","scrub_duration":0.00164314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":206,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920800+0000","last_change":"2024-09-11T08:49:09.906488+0000","last_active":"2024-09-11T08:49:14.920800+0000","last_peered":"2024-09-11T08:49:14.920800+0000","last_clean":"2024-09-11T08:49:14.920800+0000","last_became_active":"2024-09-11T08:49:09.906201+0000","last_became_peered":"2024-09-11T08:49:09.906201+0000","last_unstale":"2024-09-11T08:49:14.920800+0000","last_undegraded":"2024-09-11T08:49:14.920800+0000","last_fullsized":"2024-09-11T08:49:14.920800+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:44:15.888394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:44:15.888394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003201476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":236,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926290+0000","last_change":"2024-09-11T08:49:09.910593+0000","last_active":"2024-09-11T08:49:14.926290+0000","last_peered":"2024-09-11T08:49:14.926290+0000","last_clean":"2024-09-11T08:49:14.926290+0000","last_became_active":"2024-09-11T08:49:09.910329+0000","last_became_peered":"2024-09-11T08:49:09.910329+0000","last_unstale":"2024-09-11T08:49:14.926290+0000","last_undegraded":"2024-09-11T08:49:14.926290+0000","last_fullsized":"2024-09-11T08:49:14.926290+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:23.796894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:48:23.796894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:29.589922+0000","scrub_duration":0.0042577149999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"200'12","reported_seq":240,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:15.347746+0000","last_change":"2024-09-11T08:49:15.347679+0000","last_active":"2024-09-11T08:49:15.347746+0000","last_peered":"2024-09-11T08:49:15.347746+0000","last_clean":"2024-09-11T08:49:15.347746+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:49:15.347746+0000","last_undegraded":"2024-09-11T08:49:15.347746+0000","last_fullsized":"2024-09-11T08:49:15.347746+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'12","last_scrub_stamp":"2024-09-11T08:49:15.347591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:49:15.347591+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:31.103948+0000","scrub_duration":0.0048379369999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":202,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920622+0000","last_change":"2024-09-11T08:45:08.675251+0000","last_active":"2024-09-11T08:49:14.920622+0000","last_peered":"2024-09-11T08:49:14.920622+0000","last_clean":"2024-09-11T08:49:14.920622+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:49:14.920622+0000","last_undegraded":"2024-09-11T08:49:14.920622+0000","last_fullsized":"2024-09-11T08:49:14.920622+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:08.675179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.675179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003515392,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":220,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924255+0000","last_change":"2024-09-11T08:46:01.983864+0000","last_active":"2024-09-11T08:49:14.924255+0000","last_peered":"2024-09-11T08:49:14.924255+0000","last_clean":"2024-09-11T08:49:14.924255+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:49:14.924255+0000","last_undegraded":"2024-09-11T08:49:14.924255+0000","last_fullsized":"2024-09-11T08:49:14.924255+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:25.344348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.344348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013849730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"200'15","reported_seq":240,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923064+0000","last_change":"2024-09-11T08:49:09.933608+0000","last_active":"2024-09-11T08:49:14.923064+0000","last_peered":"2024-09-11T08:49:14.923064+0000","last_clean":"2024-09-11T08:49:14.923064+0000","last_became_active":"2024-09-11T08:49:09.916927+0000","last_became_peered":"2024-09-11T08:49:09.916927+0000","last_unstale":"2024-09-11T08:49:14.923064+0000","last_undegraded":"2024-09-11T08:49:14.923064+0000","last_fullsized":"2024-09-11T08:49:14.923064+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:47:44.737721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.737721+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032009640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":219,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924040+0000","last_change":"2024-09-11T08:48:20.977749+0000","last_active":"2024-09-11T08:49:14.924040+0000","last_peered":"2024-09-11T08:49:14.924040+0000","last_clean":"2024-09-11T08:49:14.924040+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:49:14.924040+0000","last_undegraded":"2024-09-11T08:49:14.924040+0000","last_fullsized":"2024-09-11T08:49:14.924040+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:20.977665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.977665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:45.220147+0000","scrub_duration":0.0075643139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":192,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923921+0000","last_change":"2024-09-11T08:45:23.213642+0000","last_active":"2024-09-11T08:49:14.923921+0000","last_peered":"2024-09-11T08:49:14.923921+0000","last_clean":"2024-09-11T08:49:14.923921+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:49:14.923921+0000","last_undegraded":"2024-09-11T08:49:14.923921+0000","last_fullsized":"2024-09-11T08:49:14.923921+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.213592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.213592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001600473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":221,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924788+0000","last_change":"2024-09-11T08:48:47.485315+0000","last_active":"2024-09-11T08:49:14.924788+0000","last_peered":"2024-09-11T08:49:14.924788+0000","last_clean":"2024-09-11T08:49:14.924788+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:49:14.924788+0000","last_undegraded":"2024-09-11T08:49:14.924788+0000","last_fullsized":"2024-09-11T08:49:14.924788+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:47.485264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_clean_scrub_stamp":"2024-09-11T08:48:47.485264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:17.166643+0000","scrub_duration":0.0019043490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"200'15","reported_seq":235,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.922089+0000","last_change":"2024-09-11T08:48:36.403872+0000","last_active":"2024-09-11T08:49:14.922089+0000","last_peered":"2024-09-11T08:49:14.922089+0000","last_clean":"2024-09-11T08:49:14.922089+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:49:14.922089+0000","last_undegraded":"2024-09-11T08:49:14.922089+0000","last_fullsized":"2024-09-11T08:49:14.922089+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:48:36.403646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:36.403646+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:38.622276+0000","scrub_duration":0.0057924789999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923838+0000","last_change":"2024-09-11T08:49:09.912712+0000","last_active":"2024-09-11T08:49:14.923838+0000","last_peered":"2024-09-11T08:49:14.923838+0000","last_clean":"2024-09-11T08:49:14.923838+0000","last_became_active":"2024-09-11T08:49:09.912417+0000","last_became_peered":"2024-09-11T08:49:09.912417+0000","last_unstale":"2024-09-11T08:49:14.923838+0000","last_undegraded":"2024-09-11T08:49:14.923838+0000","last_fullsized":"2024-09-11T08:49:14.923838+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:45.751518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:45.751518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":327,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923103+0000","last_change":"2024-09-11T08:45:20.119727+0000","last_active":"2024-09-11T08:49:14.923103+0000","last_peered":"2024-09-11T08:49:14.923103+0000","last_clean":"2024-09-11T08:49:14.923103+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:49:14.923103+0000","last_undegraded":"2024-09-11T08:49:14.923103+0000","last_fullsized":"2024-09-11T08:49:14.923103+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:20.119626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.119626+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018087660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":191,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.922375+0000","last_change":"2024-09-11T08:48:41.648829+0000","last_active":"2024-09-11T08:49:14.922375+0000","last_peered":"2024-09-11T08:49:14.922375+0000","last_clean":"2024-09-11T08:49:14.922375+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:49:14.922375+0000","last_undegraded":"2024-09-11T08:49:14.922375+0000","last_fullsized":"2024-09-11T08:49:14.922375+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:41.648756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.648756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:51.312266+0000","scrub_duration":0.0019862030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"200'25","reported_seq":229,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.928327+0000","last_change":"2024-09-11T08:45:21.132138+0000","last_active":"2024-09-11T08:49:14.928327+0000","last_peered":"2024-09-11T08:49:14.928327+0000","last_clean":"2024-09-11T08:49:14.928327+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:49:14.928327+0000","last_undegraded":"2024-09-11T08:49:14.928327+0000","last_fullsized":"2024-09-11T08:49:14.928327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'15","last_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.132083+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003984156,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":216,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924014+0000","last_change":"2024-09-11T08:48:39.064220+0000","last_active":"2024-09-11T08:49:14.924014+0000","last_peered":"2024-09-11T08:49:14.924014+0000","last_clean":"2024-09-11T08:49:14.924014+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:49:14.924014+0000","last_undegraded":"2024-09-11T08:49:14.924014+0000","last_fullsized":"2024-09-11T08:49:14.924014+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:39.064163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_clean_scrub_stamp":"2024-09-11T08:48:39.064163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:03.250634+0000","scrub_duration":0.0014228540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"211'9","reported_seq":351,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921848+0000","last_change":"2024-09-11T08:49:09.926226+0000","last_active":"2024-09-11T08:49:14.921848+0000","last_peered":"2024-09-11T08:49:14.921848+0000","last_clean":"2024-09-11T08:49:14.921848+0000","last_became_active":"2024-09-11T08:49:09.908380+0000","last_became_peered":"2024-09-11T08:49:09.908380+0000","last_unstale":"2024-09-11T08:49:14.921848+0000","last_undegraded":"2024-09-11T08:49:14.921848+0000","last_fullsized":"2024-09-11T08:49:14.921848+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:28.692100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.692100+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019579020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":16,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924402+0000","last_change":"2024-09-11T08:49:09.914832+0000","last_active":"2024-09-11T08:49:14.924402+0000","last_peered":"2024-09-11T08:49:14.924402+0000","last_clean":"2024-09-11T08:49:14.924402+0000","last_became_active":"2024-09-11T08:49:09.914396+0000","last_became_peered":"2024-09-11T08:49:09.914396+0000","last_unstale":"2024-09-11T08:49:14.924402+0000","last_undegraded":"2024-09-11T08:49:14.924402+0000","last_fullsized":"2024-09-11T08:49:14.924402+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.581231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.581231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"200'19","reported_seq":228,"reported_epoch":214,"state":"peering","last_fresh":"2024-09-11T08:49:15.928240+0000","last_change":"2024-09-11T08:49:15.928481+0000","last_active":"2024-09-11T08:49:15.927921+0000","last_peered":"2024-09-11T08:49:14.921352+0000","last_clean":"2024-09-11T08:49:14.921352+0000","last_became_active":"2024-09-11T08:38:40.973440+0000","last_became_peered":"2024-09-11T08:38:40.973440+0000","last_unstale":"2024-09-11T08:49:15.928240+0000","last_undegraded":"2024-09-11T08:49:15.928240+0000","last_fullsized":"2024-09-11T08:49:15.928240+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:25.792777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.792777+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:47.727170+0000","scrub_duration":0.0045475469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":197,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.921533+0000","last_change":"2024-09-11T08:45:27.198021+0000","last_active":"2024-09-11T08:49:14.921533+0000","last_peered":"2024-09-11T08:49:14.921533+0000","last_clean":"2024-09-11T08:49:14.921533+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:49:14.921533+0000","last_undegraded":"2024-09-11T08:49:14.921533+0000","last_fullsized":"2024-09-11T08:49:14.921533+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.197944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.197944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015230210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":212,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.925973+0000","last_change":"2024-09-11T08:48:30.985518+0000","last_active":"2024-09-11T08:49:14.925973+0000","last_peered":"2024-09-11T08:49:14.925973+0000","last_clean":"2024-09-11T08:49:14.925973+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:49:14.925973+0000","last_undegraded":"2024-09-11T08:49:14.925973+0000","last_fullsized":"2024-09-11T08:49:14.925973+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:30.985271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.985271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:50.920428+0000","scrub_duration":0.0099058919999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":200,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924897+0000","last_change":"2024-09-11T08:49:09.906202+0000","last_active":"2024-09-11T08:49:14.924897+0000","last_peered":"2024-09-11T08:49:14.924897+0000","last_clean":"2024-09-11T08:49:14.924897+0000","last_became_active":"2024-09-11T08:49:09.905289+0000","last_became_peered":"2024-09-11T08:49:09.905289+0000","last_unstale":"2024-09-11T08:49:14.924897+0000","last_undegraded":"2024-09-11T08:49:14.924897+0000","last_fullsized":"2024-09-11T08:49:14.924897+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.259103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.259103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015518800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"200'15","reported_seq":242,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920859+0000","last_change":"2024-09-11T08:46:01.993547+0000","last_active":"2024-09-11T08:49:14.920859+0000","last_peered":"2024-09-11T08:49:14.920859+0000","last_clean":"2024-09-11T08:49:14.920859+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:49:14.920859+0000","last_undegraded":"2024-09-11T08:49:14.920859+0000","last_fullsized":"2024-09-11T08:49:14.920859+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:45:06.657912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.657912+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023744109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":216,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924630+0000","last_change":"2024-09-11T08:47:48.373407+0000","last_active":"2024-09-11T08:49:14.924630+0000","last_peered":"2024-09-11T08:49:14.924630+0000","last_clean":"2024-09-11T08:49:14.924630+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:49:14.924630+0000","last_undegraded":"2024-09-11T08:49:14.924630+0000","last_fullsized":"2024-09-11T08:49:14.924630+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:48.373336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:48.373336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001705364,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":202,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923853+0000","last_change":"2024-09-11T08:49:09.910884+0000","last_active":"2024-09-11T08:49:14.923853+0000","last_peered":"2024-09-11T08:49:14.923853+0000","last_clean":"2024-09-11T08:49:14.923853+0000","last_became_active":"2024-09-11T08:49:09.910548+0000","last_became_peered":"2024-09-11T08:49:09.910548+0000","last_unstale":"2024-09-11T08:49:14.923853+0000","last_undegraded":"2024-09-11T08:49:14.923853+0000","last_fullsized":"2024-09-11T08:49:14.923853+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.165576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.165576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012147639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":16,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.919824+0000","last_change":"2024-09-11T08:49:09.904856+0000","last_active":"2024-09-11T08:49:14.919824+0000","last_peered":"2024-09-11T08:49:14.919824+0000","last_clean":"2024-09-11T08:49:14.919824+0000","last_became_active":"2024-09-11T08:49:09.904622+0000","last_became_peered":"2024-09-11T08:49:09.904622+0000","last_unstale":"2024-09-11T08:49:14.919824+0000","last_undegraded":"2024-09-11T08:49:14.919824+0000","last_fullsized":"2024-09-11T08:49:14.919824+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:18.685188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.685188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:32.423044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"200'19","reported_seq":225,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.925119+0000","last_change":"2024-09-11T08:48:18.890324+0000","last_active":"2024-09-11T08:49:14.925119+0000","last_peered":"2024-09-11T08:49:14.925119+0000","last_clean":"2024-09-11T08:49:14.925119+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:49:14.925119+0000","last_undegraded":"2024-09-11T08:49:14.925119+0000","last_fullsized":"2024-09-11T08:49:14.925119+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:18.890233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.890233+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:33.893054+0000","scrub_duration":0.0070086059999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":206,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.920663+0000","last_change":"2024-09-11T08:45:14.574739+0000","last_active":"2024-09-11T08:49:14.920663+0000","last_peered":"2024-09-11T08:49:14.920663+0000","last_clean":"2024-09-11T08:49:14.920663+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:49:14.920663+0000","last_undegraded":"2024-09-11T08:49:14.920663+0000","last_fullsized":"2024-09-11T08:49:14.920663+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:14.574678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:14.574678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013819469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":215,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923347+0000","last_change":"2024-09-11T08:48:18.958742+0000","last_active":"2024-09-11T08:49:14.923347+0000","last_peered":"2024-09-11T08:49:14.923347+0000","last_clean":"2024-09-11T08:49:14.923347+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:49:14.923347+0000","last_undegraded":"2024-09-11T08:49:14.923347+0000","last_fullsized":"2024-09-11T08:49:14.923347+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:18.958609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.958609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:43.228394+0000","scrub_duration":0.001704777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":220,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923984+0000","last_change":"2024-09-11T08:49:09.905933+0000","last_active":"2024-09-11T08:49:14.923984+0000","last_peered":"2024-09-11T08:49:14.923984+0000","last_clean":"2024-09-11T08:49:14.923984+0000","last_became_active":"2024-09-11T08:49:09.905634+0000","last_became_peered":"2024-09-11T08:49:09.905634+0000","last_unstale":"2024-09-11T08:49:14.923984+0000","last_undegraded":"2024-09-11T08:49:14.923984+0000","last_fullsized":"2024-09-11T08:49:14.923984+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.807331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.807331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013867529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"200'10","reported_seq":250,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.925924+0000","last_change":"2024-09-11T08:48:32.020078+0000","last_active":"2024-09-11T08:49:14.925924+0000","last_peered":"2024-09-11T08:49:14.925924+0000","last_clean":"2024-09-11T08:49:14.925924+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:49:14.925924+0000","last_undegraded":"2024-09-11T08:49:14.925924+0000","last_fullsized":"2024-09-11T08:49:14.925924+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'6","last_scrub_stamp":"2024-09-11T08:48:32.019991+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:48:32.019991+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:37.913420+0000","scrub_duration":0.010394439,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":28,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.922619+0000","last_change":"2024-09-11T08:49:09.919241+0000","last_active":"2024-09-11T08:49:14.922619+0000","last_peered":"2024-09-11T08:49:14.922619+0000","last_clean":"2024-09-11T08:49:14.922619+0000","last_became_active":"2024-09-11T08:49:09.907483+0000","last_became_peered":"2024-09-11T08:49:09.907483+0000","last_unstale":"2024-09-11T08:49:14.922619+0000","last_undegraded":"2024-09-11T08:49:14.922619+0000","last_fullsized":"2024-09-11T08:49:14.922619+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:21.481634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.481634+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":66,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926109+0000","last_change":"2024-09-11T08:48:29.801804+0000","last_active":"2024-09-11T08:49:14.926109+0000","last_peered":"2024-09-11T08:49:14.926109+0000","last_clean":"2024-09-11T08:49:14.926109+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:49:14.926109+0000","last_undegraded":"2024-09-11T08:49:14.926109+0000","last_fullsized":"2024-09-11T08:49:14.926109+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:29.801722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:29.801722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:39.014794+0000","scrub_duration":0.0013425760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":210,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.923690+0000","last_change":"2024-09-11T08:45:23.348327+0000","last_active":"2024-09-11T08:49:14.923690+0000","last_peered":"2024-09-11T08:49:14.923690+0000","last_clean":"2024-09-11T08:49:14.923690+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:49:14.923690+0000","last_undegraded":"2024-09-11T08:49:14.923690+0000","last_fullsized":"2024-09-11T08:49:14.923690+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.348275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.348275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035843659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":210,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926786+0000","last_change":"2024-09-11T08:45:23.674163+0000","last_active":"2024-09-11T08:49:14.926786+0000","last_peered":"2024-09-11T08:49:14.926786+0000","last_clean":"2024-09-11T08:49:14.926786+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:49:14.926786+0000","last_undegraded":"2024-09-11T08:49:14.926786+0000","last_fullsized":"2024-09-11T08:49:14.926786+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.674102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.674102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001440293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":216,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926221+0000","last_change":"2024-09-11T08:49:09.919371+0000","last_active":"2024-09-11T08:49:14.926221+0000","last_peered":"2024-09-11T08:49:14.926221+0000","last_clean":"2024-09-11T08:49:14.926221+0000","last_became_active":"2024-09-11T08:49:09.918992+0000","last_became_peered":"2024-09-11T08:49:09.918992+0000","last_unstale":"2024-09-11T08:49:14.926221+0000","last_undegraded":"2024-09-11T08:49:14.926221+0000","last_fullsized":"2024-09-11T08:49:14.926221+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:40.033594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:48:40.033594+0000","last_clean_scrub_stamp":"2024-09-11T08:48:40.033594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:56.836029+0000","scrub_duration":0.0022129379999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"200'19","reported_seq":90,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.926698+0000","last_change":"2024-09-11T08:48:20.841324+0000","last_active":"2024-09-11T08:49:14.926698+0000","last_peered":"2024-09-11T08:49:14.926698+0000","last_clean":"2024-09-11T08:49:14.926698+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:49:14.926698+0000","last_undegraded":"2024-09-11T08:49:14.926698+0000","last_fullsized":"2024-09-11T08:49:14.926698+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:20.841243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.841243+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:44.045269+0000","scrub_duration":0.0096371630000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":16,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-11T08:49:14.924490+0000","last_change":"2024-09-11T08:49:09.905224+0000","last_active":"2024-09-11T08:49:14.924490+0000","last_peered":"2024-09-11T08:49:14.924490+0000","last_clean":"2024-09-11T08:49:14.924490+0000","last_became_active":"2024-09-11T08:49:09.904721+0000","last_became_peered":"2024-09-11T08:49:09.904721+0000","last_unstale":"2024-09-11T08:49:14.924490+0000","last_undegraded":"2024-09-11T08:49:14.924490+0000","last_fullsized":"2024-09-11T08:49:14.924490+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:32.555890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:32.555890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":393019392,"num_objects":5997,"num_object_clones":0,"num_object_copies":17991,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5997,"num_write_kb":383808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1172045824,"data_stored":1172045824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5997,"ondisk_log_size":5997,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1119,"num_read_kb":876,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":2904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":12426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":751,"ondisk_log_size":751,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":628,"num_read_kb":623,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299525,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":483172,"kb_used_data":173328,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93261980,"statfs":{"total":95995035648,"available":95500267520,"internally_reserved":0,"allocated":177487872,"data_stored":177138748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":1.2470000000000001,"15min":0.84499999999999997},"min":{"1min":0.23200000000000001,"5min":0.22,"15min":0.22},"max":{"1min":3.7919999999999998,"5min":32.171999999999997,"15min":32.171999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.411,"15min":0.93200000000000005},"min":{"1min":0.26700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":3.1829999999999998,"5min":32.149999999999999,"15min":32.149999999999999},"last":0.47399999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:48:44 2024","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.5009999999999999,"15min":0.98199999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":4.069,"5min":33.692999999999998,"15min":33.692999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":1.431,"5min":1.4099999999999999,"15min":0.90600000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23300000000000001},"max":{"1min":5.2030000000000003,"5min":35.591000000000001,"15min":35.591000000000001},"last":0.45900000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:48:44 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0680000000000001,"15min":0.78800000000000003},"min":{"1min":0.29799999999999999,"5min":0.249,"15min":0.249},"max":{"1min":2.5259999999999998,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0840000000000001,"15min":0.755},"min":{"1min":0.36699999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":3.8380000000000001,"5min":5.7990000000000004,"15min":5.7990000000000004},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:48:44 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.0369999999999999,"15min":0.82199999999999995},"min":{"1min":0.27200000000000002,"5min":0.25,"15min":0.25},"max":{"1min":3.9060000000000001,"5min":6.0259999999999998,"15min":7.8609999999999998},"last":0.376},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.119,"15min":0.90200000000000002},"min":{"1min":0.41399999999999998,"5min":0.27700000000000002,"15min":0.27400000000000002},"max":{"1min":3.7669999999999999,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.44800000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.54900000000000004,"15min":0.48199999999999998},"min":{"1min":0.27100000000000002,"5min":0.24099999999999999,"15min":0.23400000000000001},"max":{"1min":1.7669999999999999,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":0.439},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.68000000000000005,"15min":0.52100000000000002},"min":{"1min":0.312,"5min":0.216,"15min":0.216},"max":{"1min":2.9359999999999999,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.29199999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:48:44 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":0.72099999999999997,"15min":0.55100000000000005},"min":{"1min":0.30299999999999999,"5min":0.255,"15min":0.23300000000000001},"max":{"1min":5.1059999999999999,"5min":5.1059999999999999,"15min":5.1059999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.68899999999999995,"15min":0.56599999999999995},"min":{"1min":0.308,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.0299999999999998,"5min":3.093,"15min":3.093},"last":0.33400000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.67600000000000005,"15min":0.57999999999999996},"min":{"1min":0.32000000000000001,"5min":0.317,"15min":0.315},"max":{"1min":2.7749999999999999,"5min":2.8820000000000001,"15min":3.5979999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":1.175,"5min":0.73699999999999999,"15min":0.59299999999999997},"min":{"1min":0.376,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.9700000000000002,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.39900000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528458,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":369420,"kb_used_data":59580,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93375732,"statfs":{"total":95995035648,"available":95616749568,"internally_reserved":0,"allocated":61009920,"data_stored":60681144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:48:26 2024","interfaces":[{"interface":"back","average":{"1min":1.9259999999999999,"5min":1.2470000000000001,"15min":0.98799999999999999},"min":{"1min":0.36699999999999999,"5min":0.217,"15min":0.217},"max":{"1min":3.6989999999999998,"5min":4.3890000000000002,"15min":28.302},"last":0.50900000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.149,"15min":0.92700000000000005},"min":{"1min":0.24299999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.6040000000000001,"5min":4.577,"15min":28.314},"last":0.501}]},{"osd":1,"last update":"Wed Sep 11 08:48:26 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.498,"15min":0.91000000000000003},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.0090000000000003,"5min":12.586,"15min":12.586},"last":0.49199999999999999},{"interface":"front","average":{"1min":1.863,"5min":1.369,"15min":0.86199999999999999},"min":{"1min":0.34300000000000003,"5min":0.25600000000000001,"15min":0.253},"max":{"1min":5.2619999999999996,"5min":10.834,"15min":10.834},"last":0.56000000000000005}]},{"osd":2,"last update":"Wed Sep 11 08:48:26 2024","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.1040000000000001,"15min":0.77100000000000002},"min":{"1min":0.38200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":3.4580000000000002,"5min":14.313000000000001,"15min":14.313000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.1160000000000001,"15min":0.77000000000000002},"min":{"1min":0.29199999999999998,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":3.7240000000000002,"5min":6.5979999999999999,"15min":6.5979999999999999},"last":0.42199999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:48:26 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.401,"15min":0.92700000000000005},"min":{"1min":0.29199999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":5.1550000000000002,"5min":11.433,"15min":11.433},"last":0.52100000000000002},{"interface":"front","average":{"1min":2.7229999999999999,"5min":1.262,"15min":0.90700000000000003},"min":{"1min":0.32100000000000001,"5min":0.224,"15min":0.224},"max":{"1min":7.0250000000000004,"5min":7.0250000000000004,"15min":9.1069999999999993},"last":0.53600000000000003}]},{"osd":4,"last update":"Wed Sep 11 08:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.59099999999999997,"15min":0.47499999999999998},"min":{"1min":0.28899999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.60799999999999998,"15min":0.47699999999999998},"min":{"1min":0.29799999999999999,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":4.0209999999999999,"5min":4.0209999999999999,"15min":4.0209999999999999},"last":0.40799999999999997}]},{"osd":5,"last update":"Wed Sep 11 08:48:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":0.755,"15min":0.55300000000000005},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":8.6829999999999998,"5min":8.6829999999999998,"15min":8.6829999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.72699999999999998,"15min":0.54200000000000004},"min":{"1min":0.34499999999999997,"5min":0.29899999999999999,"15min":0.28100000000000003},"max":{"1min":4.0629999999999997,"5min":5.0819999999999999,"15min":5.0819999999999999},"last":0.45600000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:48:39 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.74099999999999999,"15min":0.55300000000000005},"min":{"1min":0.313,"5min":0.25900000000000001,"15min":0.247},"max":{"1min":3.355,"5min":7.327,"15min":7.327},"last":0.38900000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":0.68200000000000005,"15min":0.56200000000000006},"min":{"1min":0.36699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.4500000000000002,"5min":5.4500000000000002,"15min":5.4500000000000002},"last":0.436}]}]},{"osd":5,"up_from":38,"seq":163208757390,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":393416,"kb_used_data":83576,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93351736,"statfs":{"total":95995035648,"available":95592177664,"internally_reserved":0,"allocated":85581824,"data_stored":85248936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:49:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.93000000000000005,"15min":0.77000000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.20599999999999999},"max":{"1min":2.7599999999999998,"5min":3.266,"15min":7.3899999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.3,"5min":1.2150000000000001,"15min":0.90700000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23100000000000001},"max":{"1min":5.569,"5min":8.0410000000000004,"15min":8.0410000000000004},"last":0.59799999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:49:04 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.23,"15min":0.88300000000000001},"min":{"1min":0.28699999999999998,"5min":0.23799999999999999,"15min":0.22800000000000001},"max":{"1min":2.7749999999999999,"5min":8,"15min":8},"last":0.496},{"interface":"front","average":{"1min":1.002,"5min":1.119,"15min":0.81899999999999995},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.9780000000000002,"5min":3.9780000000000002,"15min":6.6719999999999997},"last":0.53200000000000003}]},{"osd":2,"last update":"Wed Sep 11 08:49:04 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.171,"15min":0.85099999999999998},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.23799999999999999},"max":{"1min":5.5339999999999998,"5min":5.5339999999999998,"15min":5.5339999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":1.1539999999999999,"15min":0.81399999999999995},"min":{"1min":0.34100000000000003,"5min":0.28699999999999998,"15min":0.26400000000000001},"max":{"1min":3.5990000000000002,"5min":8.0640000000000001,"15min":8.0640000000000001},"last":0.54500000000000004}]},{"osd":3,"last update":"Wed Sep 11 08:49:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.99199999999999999,"15min":0.80700000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.7309999999999999,"5min":4.3940000000000001,"15min":6.0460000000000003},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.319,"5min":1.2909999999999999,"15min":0.95099999999999996},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.27400000000000002},"max":{"1min":8.0899999999999999,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.57499999999999996}]},{"osd":4,"last update":"Wed Sep 11 08:49:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70599999999999996,"15min":0.501},"min":{"1min":0.26400000000000001,"5min":0.246,"15min":0.20699999999999999},"max":{"1min":3.5129999999999999,"5min":4.7110000000000003,"15min":4.7110000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.621,"15min":0.48899999999999999},"min":{"1min":0.29099999999999998,"5min":0.23699999999999999,"15min":0.20499999999999999},"max":{"1min":1.7749999999999999,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.47199999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:48:27 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":0.71099999999999997,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.27700000000000002,"15min":0.23599999999999999},"max":{"1min":6.4530000000000003,"5min":6.4530000000000003,"15min":6.4530000000000003},"last":0.434},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.69099999999999995,"15min":0.51800000000000002},"min":{"1min":0.28100000000000003,"5min":0.26100000000000001,"15min":0.251},"max":{"1min":8.1920000000000002,"5min":8.1920000000000002,"15min":8.1920000000000002},"last":0.41799999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:48:38 2024","interfaces":[{"interface":"back","average":{"1min":4.7910000000000004,"5min":1.5089999999999999,"15min":0.82099999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.27800000000000002},"max":{"1min":88.513000000000005,"5min":88.513000000000005,"15min":88.513000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":4.7969999999999997,"5min":1.514,"15min":0.85399999999999998},"min":{"1min":0.317,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":86.963999999999999,"5min":86.963999999999999,"15min":86.963999999999999},"last":0.66600000000000004}]}]},{"osd":4,"up_from":31,"seq":133143986322,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":346816,"kb_used_data":45300,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93398336,"statfs":{"total":95995035648,"available":95639896064,"internally_reserved":0,"allocated":46387200,"data_stored":46057127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:48:39 2024","interfaces":[{"interface":"back","average":{"1min":4.6319999999999997,"5min":2.1469999999999998,"15min":1.5669999999999999},"min":{"1min":0.23100000000000001,"5min":0.21199999999999999,"15min":0.20399999999999999},"max":{"1min":44.140999999999998,"5min":48.451999999999998,"15min":89.510000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":4.5,"5min":2.4540000000000002,"15min":1.665},"min":{"1min":0.246,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":44.450000000000003,"5min":49.115000000000002,"15min":89.765000000000001},"last":0.46700000000000003}]},{"osd":1,"last update":"Wed Sep 11 08:48:39 2024","interfaces":[{"interface":"back","average":{"1min":3.54,"5min":1.98,"15min":1.083},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":44.152999999999999,"5min":44.152999999999999,"15min":44.152999999999999},"last":0.443},{"interface":"front","average":{"1min":3.46,"5min":1.8999999999999999,"15min":1.0649999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":45.048000000000002,"5min":45.048000000000002,"15min":45.048000000000002},"last":0.42399999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":1.1060000000000001,"15min":0.80700000000000005},"min":{"1min":0.254,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.9810000000000001,"5min":17.303000000000001,"15min":17.303000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0600000000000001,"15min":0.81999999999999995},"min":{"1min":0.29899999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.7469999999999999,"5min":20.702999999999999,"15min":20.702999999999999},"last":0.40999999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:48:39 2024","interfaces":[{"interface":"back","average":{"1min":3.657,"5min":2.2669999999999999,"15min":1.6699999999999999},"min":{"1min":0.27400000000000002,"5min":0.255,"15min":0.255},"max":{"1min":45.268000000000001,"5min":48.194000000000003,"15min":89.881},"last":0.48499999999999999},{"interface":"front","average":{"1min":3.532,"5min":2.2589999999999999,"15min":1.6279999999999999},"min":{"1min":0.29299999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":44.234000000000002,"5min":48.840000000000003,"15min":89.715000000000003},"last":0.38900000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:49:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.70499999999999996,"15min":0.51800000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.5449999999999999,"5min":3.5449999999999999,"15min":3.5449999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.65700000000000003,"15min":0.504},"min":{"1min":0.30599999999999999,"5min":0.26100000000000001,"15min":0.23699999999999999},"max":{"1min":0.82799999999999996,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.31900000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.54800000000000004,"15min":0.47599999999999998},"min":{"1min":0.28599999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.8839999999999999,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.65300000000000002,"15min":0.54700000000000004},"min":{"1min":0.30399999999999999,"5min":0.221,"15min":0.221},"max":{"1min":3.8109999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":0.503}]},{"osd":7,"last update":"Wed Sep 11 08:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.622,"15min":0.53500000000000003},"min":{"1min":0.308,"5min":0.252,"15min":0.252},"max":{"1min":1.7290000000000001,"5min":3.339,"15min":3.339},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.753,"15min":0.57399999999999995},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.28799999999999998},"max":{"1min":2.6299999999999999,"5min":5.1970000000000001,"15min":5.1970000000000001},"last":0.51300000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149848,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":505648,"kb_used_data":208288,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93239504,"statfs":{"total":95995035648,"available":95477252096,"internally_reserved":0,"allocated":213286912,"data_stored":212922072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:48:24 2024","interfaces":[{"interface":"back","average":{"1min":2.1840000000000002,"5min":1.6859999999999999,"15min":1.0149999999999999},"min":{"1min":0.16500000000000001,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":24.280000000000001,"5min":24.280000000000001,"15min":24.280000000000001},"last":0.248},{"interface":"front","average":{"1min":2.2509999999999999,"5min":2.1099999999999999,"15min":1.2050000000000001},"min":{"1min":0.184,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":22.225000000000001,"5min":22.225000000000001,"15min":22.225000000000001},"last":0.48199999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:48:24 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":3,"15min":1.5089999999999999},"min":{"1min":0.32700000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":5.0739999999999998,"5min":115.601,"15min":115.601},"last":0.55700000000000005},{"interface":"front","average":{"1min":1.2350000000000001,"5min":2.9220000000000002,"15min":1.427},"min":{"1min":0.314,"5min":0.23300000000000001,"15min":0.17999999999999999},"max":{"1min":3.6629999999999998,"5min":115.474,"15min":115.474},"last":0.61299999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:48:24 2024","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.8069999999999999,"15min":1.0669999999999999},"min":{"1min":0.30499999999999999,"5min":0.26400000000000001,"15min":0.23200000000000001},"max":{"1min":5.1429999999999998,"5min":8.9510000000000005,"15min":8.9510000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5860000000000001,"15min":1.0169999999999999},"min":{"1min":0.35799999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":4.3250000000000002,"5min":10.170999999999999,"15min":10.695},"last":0.29399999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:48:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6140000000000001,"5min":1.6499999999999999,"15min":1.0669999999999999},"min":{"1min":0.41699999999999998,"5min":0.28599999999999998,"15min":0.26700000000000002},"max":{"1min":6.0519999999999996,"5min":8.3279999999999994,"15min":8.9969999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.292,"15min":0.90800000000000003},"min":{"1min":0.318,"5min":0.30099999999999999,"15min":0.27700000000000002},"max":{"1min":2.524,"5min":7.234,"15min":7.234},"last":0.41399999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:48:56 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.49,"15min":1.0449999999999999},"min":{"1min":0.434,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":6.0119999999999996,"5min":8.3599999999999994,"15min":11.526999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.3360000000000001,"15min":0.93000000000000005},"min":{"1min":0.33200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.492,"5min":7.1689999999999996,"15min":7.1689999999999996},"last":0.38600000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:48:24 2024","interfaces":[{"interface":"back","average":{"1min":1.633,"5min":1.736,"15min":1.0800000000000001},"min":{"1min":0.34799999999999998,"5min":0.311,"15min":0.29999999999999999},"max":{"1min":5.1929999999999996,"5min":11.494999999999999,"15min":11.494999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.625,"15min":1.0549999999999999},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.311},"max":{"1min":5.0949999999999998,"5min":7.7249999999999996,"15min":7.7249999999999996},"last":0.60099999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:48:45 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.6679999999999999,"15min":1.121},"min":{"1min":0.49299999999999999,"5min":0.36399999999999999,"15min":0.35299999999999998},"max":{"1min":6.133,"5min":9.1539999999999999,"15min":11.561},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.399,"15min":0.97799999999999998},"min":{"1min":0.377,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":4.2729999999999997,"5min":6.6200000000000001,"15min":6.6200000000000001},"last":0.441}]}]},{"osd":2,"up_from":18,"seq":77309411484,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":420852,"kb_used_data":123496,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93324300,"statfs":{"total":95995035648,"available":95564083200,"internally_reserved":0,"allocated":126459904,"data_stored":126142050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:48:59 2024","interfaces":[{"interface":"back","average":{"1min":3.6659999999999999,"5min":2.7599999999999998,"15min":1.4930000000000001},"min":{"1min":0.214,"5min":0.16500000000000001,"15min":0.159},"max":{"1min":16.606999999999999,"5min":34.713999999999999,"15min":34.713999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":3.4390000000000001,"5min":2.8130000000000002,"15min":1.595},"min":{"1min":0.246,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":16.855,"5min":34.399000000000001,"15min":34.399000000000001},"last":0.45800000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:48:59 2024","interfaces":[{"interface":"back","average":{"1min":6.7919999999999998,"5min":3.2040000000000002,"15min":1.5309999999999999},"min":{"1min":0.29099999999999998,"5min":0.224,"15min":0.19},"max":{"1min":72.867000000000004,"5min":72.867000000000004,"15min":72.867000000000004},"last":0.26600000000000001},{"interface":"front","average":{"1min":7.1219999999999999,"5min":3.4319999999999999,"15min":1.6259999999999999},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":67.799000000000007,"5min":67.799000000000007,"15min":67.799000000000007},"last":1.147}]},{"osd":3,"last update":"Wed Sep 11 08:49:12 2024","interfaces":[{"interface":"back","average":{"1min":2.706,"5min":2.379,"15min":1.4179999999999999},"min":{"1min":0.26100000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":16.353999999999999,"5min":34.75,"15min":34.75},"last":0.60999999999999999},{"interface":"front","average":{"1min":2.282,"5min":2.2160000000000002,"15min":1.359},"min":{"1min":0.29499999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":16.138999999999999,"5min":32.802,"15min":32.802},"last":0.79300000000000004}]},{"osd":4,"last update":"Wed Sep 11 08:48:31 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.278,"15min":0.92200000000000004},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":3.9689999999999999,"5min":8.0820000000000007,"15min":8.0820000000000007},"last":0.443},{"interface":"front","average":{"1min":1.044,"5min":1.2210000000000001,"15min":0.88700000000000001},"min":{"1min":0.33000000000000002,"5min":0.29999999999999999,"15min":0.28399999999999997},"max":{"1min":3.427,"5min":6.9320000000000004,"15min":6.9320000000000004},"last":0.75800000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:48:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.2370000000000001,"15min":0.91000000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.7080000000000002,"5min":6.9550000000000001,"15min":6.9550000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.89700000000000002,"5min":1.0049999999999999,"15min":0.81200000000000006},"min":{"1min":0.31900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.859,"5min":4.2000000000000002,"15min":5.9530000000000003},"last":1.0569999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:48:20 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.409,"15min":0.98299999999999998},"min":{"1min":0.36099999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.9910000000000001,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.1779999999999999,"15min":0.874},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.452,"5min":6.9779999999999998,"15min":6.9779999999999998},"last":0.82099999999999995}]},{"osd":7,"last update":"Wed Sep 11 08:48:36 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.073,"15min":0.84699999999999998},"min":{"1min":0.32200000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.2709999999999999,"5min":4.2380000000000004,"15min":5.9909999999999997},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.375,"15min":0.99299999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.0899999999999999,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.85099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575009,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":441340,"kb_used_data":139824,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93303812,"statfs":{"total":95995035648,"available":95543103488,"internally_reserved":0,"allocated":143179776,"data_stored":142790409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:48:32 2024","interfaces":[{"interface":"back","average":{"1min":1.7569999999999999,"5min":1.637,"15min":1.357},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":3.9470000000000001,"5min":31.466999999999999,"15min":117.01600000000001},"last":0.20699999999999999},{"interface":"front","average":{"1min":2.0089999999999999,"5min":2.1150000000000002,"15min":1.651},"min":{"1min":0.17499999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":8.1549999999999994,"5min":32.204000000000001,"15min":117.62},"last":0.28499999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:48:56 2024","interfaces":[{"interface":"back","average":{"1min":1.968,"5min":1.4419999999999999,"15min":1.0780000000000001},"min":{"1min":0.248,"5min":0.191,"15min":0.191},"max":{"1min":8.1329999999999991,"5min":8.1329999999999991,"15min":20.628},"last":0.221},{"interface":"front","average":{"1min":2.3250000000000002,"5min":1.5449999999999999,"15min":1.0720000000000001},"min":{"1min":0.24299999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":9.7200000000000006,"5min":9.7200000000000006,"15min":20.148},"last":0.25800000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:49:12 2024","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.4830000000000001,"15min":1.3640000000000001},"min":{"1min":0.27200000000000002,"5min":0.191,"15min":0.191},"max":{"1min":4.2859999999999996,"5min":8.3550000000000004,"15min":116.98},"last":0.27200000000000002},{"interface":"front","average":{"1min":1.502,"5min":1.4890000000000001,"15min":1.458},"min":{"1min":0.246,"5min":0.24099999999999999,"15min":0.22},"max":{"1min":3.8690000000000002,"5min":7.415,"15min":117.56699999999999},"last":0.246}]},{"osd":4,"last update":"Wed Sep 11 08:48:32 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.1879999999999999,"15min":0.871},"min":{"1min":0.32700000000000001,"5min":0.27500000000000002,"15min":0.26200000000000001},"max":{"1min":4.6580000000000004,"5min":6.3490000000000002,"15min":11.19},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.1539999999999999,"15min":0.83099999999999996},"min":{"1min":0.34799999999999998,"5min":0.30199999999999999,"15min":0.251},"max":{"1min":3.8460000000000001,"5min":5.056,"15min":7.7169999999999996},"last":1.1399999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:49:00 2024","interfaces":[{"interface":"back","average":{"1min":1.637,"5min":1.361,"15min":1.395},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":8.0999999999999996,"5min":8.0999999999999996,"15min":103.337},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.407,"5min":1.3080000000000001,"15min":1.345},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.27400000000000002},"max":{"1min":5.9020000000000001,"5min":6.3799999999999999,"15min":103.163},"last":0.42599999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:48:27 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.135,"15min":0.83799999999999997},"min":{"1min":0.39300000000000002,"5min":0.34100000000000003,"15min":0.29399999999999998},"max":{"1min":3.8980000000000001,"5min":3.907,"15min":8.9149999999999991},"last":1.121},{"interface":"front","average":{"1min":1.6399999999999999,"5min":1.3480000000000001,"15min":0.99099999999999999},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.31},"max":{"1min":4.7080000000000002,"5min":6.3250000000000002,"15min":9.5199999999999996},"last":1.1479999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:48:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.2450000000000001,"15min":0.92900000000000005},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":4.3799999999999999,"5min":6.4059999999999997,"15min":11.315},"last":0.502},{"interface":"front","average":{"1min":1.75,"5min":1.26,"15min":0.90000000000000002},"min":{"1min":0.40000000000000002,"5min":0.36299999999999999,"15min":0.315},"max":{"1min":10.266,"5min":10.266,"15min":10.266},"last":1.157}]}]},{"osd":0,"up_from":0,"seq":34359738533,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":2668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":263847936,"data_stored":263847936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92471296,"data_stored":92471296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":274137088,"data_stored":274137088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":62324736,"data_stored":62324736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":208142336,"data_stored":208142336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61603840,"data_stored":61603840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":209518592,"data_stored":209518592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:49:23.850 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool set unique_pool_2 pgp_num 16 2024-09-11T08:49:23.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:23.851 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:23.851 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:23.941 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:23.953 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:23.981 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:23.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:23.982 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:24.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:23 smithi092 ceph-mon[41114]: pgmap v453: 148 pgs: 148 active+clean; 1.3 GiB data, 5.7 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 164 MiB/s wr, 2.63k op/s; 41 B/s, 1 objects/s recovering 2024-09-11T08:49:24.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:23 smithi092 ceph-mon[41114]: 3.0 scrub starts 2024-09-11T08:49:24.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:23 smithi092 ceph-mon[41114]: 3.0 scrub ok 2024-09-11T08:49:24.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:23 smithi092 ceph-mon[29647]: pgmap v453: 148 pgs: 148 active+clean; 1.3 GiB data, 5.7 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 164 MiB/s wr, 2.63k op/s; 41 B/s, 1 objects/s recovering 2024-09-11T08:49:24.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:23 smithi092 ceph-mon[29647]: 3.0 scrub starts 2024-09-11T08:49:24.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:23 smithi092 ceph-mon[29647]: 3.0 scrub ok 2024-09-11T08:49:24.082 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:24.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:24.137 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:24.138 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:24.238 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:24.276 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:24.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:24.277 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:24.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:24.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:23 smithi145 ceph-mon[33384]: pgmap v453: 148 pgs: 148 active+clean; 1.3 GiB data, 5.7 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 164 MiB/s wr, 2.63k op/s; 41 B/s, 1 objects/s recovering 2024-09-11T08:49:24.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:23 smithi145 ceph-mon[33384]: 3.0 scrub starts 2024-09-11T08:49:24.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:23 smithi145 ceph-mon[33384]: 3.0 scrub ok 2024-09-11T08:49:24.415 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:24.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:24.416 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:24.517 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:24.538 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:24.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:24.539 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:24.640 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:24.676 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:24.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:24.677 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:24.720 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 10 16 28510 28494 178.072 126.625 0.00491385 0.00561045 2024-09-11T08:49:24.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:24.807 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:24.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:24.808 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:24.909 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:24.942 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:24.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:24.943 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:25.044 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:25.082 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:25.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:25.084 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:25.184 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:25.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:25.218 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:25.218 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:25.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:24 smithi092 ceph-mon[29647]: 6.1a scrub starts 2024-09-11T08:49:25.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:24 smithi092 ceph-mon[29647]: 6.1a scrub ok 2024-09-11T08:49:25.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:24 smithi092 ceph-mon[29647]: 3.6 scrub starts 2024-09-11T08:49:25.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:24 smithi092 ceph-mon[29647]: 3.6 scrub ok 2024-09-11T08:49:25.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:24 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3179188449' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:49:25.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:24 smithi092 ceph-mon[41114]: 6.1a scrub starts 2024-09-11T08:49:25.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:24 smithi092 ceph-mon[41114]: 6.1a scrub ok 2024-09-11T08:49:25.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:24 smithi092 ceph-mon[41114]: 3.6 scrub starts 2024-09-11T08:49:25.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:24 smithi092 ceph-mon[41114]: 3.6 scrub ok 2024-09-11T08:49:25.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:24 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3179188449' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:49:25.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:25.351 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:25.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:25.352 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:25.353 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:24 smithi145 ceph-mon[33384]: 6.1a scrub starts 2024-09-11T08:49:25.353 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:24 smithi145 ceph-mon[33384]: 6.1a scrub ok 2024-09-11T08:49:25.353 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:24 smithi145 ceph-mon[33384]: 3.6 scrub starts 2024-09-11T08:49:25.353 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:24 smithi145 ceph-mon[33384]: 3.6 scrub ok 2024-09-11T08:49:25.353 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:24 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3179188449' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:49:25.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:25.475 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:25.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:25.476 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:25.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:25.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:25.599 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:25.600 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:25.700 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:25.720 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 11 16 30357 30341 172.377 115.438 0.00791699 0.00579544 2024-09-11T08:49:25.732 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:25.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:25.734 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:25.834 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:25.862 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:25.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:25.863 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:25.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:25.999 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:25.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:26.000 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:26.101 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:26.122 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:26.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:26.123 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:26.203 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:25 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:49:26.203 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:25 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:49:26.203 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:25 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3179188449' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:49:26.203 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:25 smithi092 ceph-mon[29647]: osdmap e216: 8 total, 8 up, 7 in 2024-09-11T08:49:26.204 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:25 smithi092 ceph-mon[29647]: pgmap v455: 148 pgs: 148 active+clean; 1.3 GiB data, 5.7 GiB used, 620 GiB / 626 GiB avail; 617 B/s rd, 113 MiB/s wr, 1.81k op/s; 39 B/s, 1 objects/s recovering 2024-09-11T08:49:26.204 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:25 smithi092 ceph-mon[29647]: 3.8 scrub starts 2024-09-11T08:49:26.204 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:25 smithi092 ceph-mon[29647]: 3.8 scrub ok 2024-09-11T08:49:26.204 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:25 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:49:26.204 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:25 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:49:26.204 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:25 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3179188449' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:49:26.204 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:25 smithi092 ceph-mon[41114]: osdmap e216: 8 total, 8 up, 7 in 2024-09-11T08:49:26.204 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:25 smithi092 ceph-mon[41114]: pgmap v455: 148 pgs: 148 active+clean; 1.3 GiB data, 5.7 GiB used, 620 GiB / 626 GiB avail; 617 B/s rd, 113 MiB/s wr, 1.81k op/s; 39 B/s, 1 objects/s recovering 2024-09-11T08:49:26.204 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:25 smithi092 ceph-mon[41114]: 3.8 scrub starts 2024-09-11T08:49:26.204 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:25 smithi092 ceph-mon[41114]: 3.8 scrub ok 2024-09-11T08:49:26.224 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:26.259 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:26.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:26.260 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:26.361 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:26.389 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:26.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:26.390 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:26.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:25 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:49:26.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:25 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:49:26.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:25 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3179188449' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:49:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:25 smithi145 ceph-mon[33384]: osdmap e216: 8 total, 8 up, 7 in 2024-09-11T08:49:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:25 smithi145 ceph-mon[33384]: pgmap v455: 148 pgs: 148 active+clean; 1.3 GiB data, 5.7 GiB used, 620 GiB / 626 GiB avail; 617 B/s rd, 113 MiB/s wr, 1.81k op/s; 39 B/s, 1 objects/s recovering 2024-09-11T08:49:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:25 smithi145 ceph-mon[33384]: 3.8 scrub starts 2024-09-11T08:49:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:25 smithi145 ceph-mon[33384]: 3.8 scrub ok 2024-09-11T08:49:26.491 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:26.526 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:26.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:26.527 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:26.628 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:26.648 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:26.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:26.653 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:26.721 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 12 16 32247 32231 167.855 118.125 0.00708288 0.00595261 2024-09-11T08:49:26.753 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:26.786 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:26.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:26.788 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:26.888 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:26.911 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:49:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:49:26.924 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:26.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:26.925 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:27.026 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:27.051 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:27.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:27.052 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:27.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:27.195 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:27.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:27.197 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:27.259 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:49:27.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[29647]: 5.b scrub starts 2024-09-11T08:49:27.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[29647]: 5.b scrub ok 2024-09-11T08:49:27.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[29647]: 3.3 deep-scrub starts 2024-09-11T08:49:27.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[29647]: 3.3 deep-scrub ok 2024-09-11T08:49:27.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[29647]: 3.5 scrub starts 2024-09-11T08:49:27.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[29647]: 3.5 scrub ok 2024-09-11T08:49:27.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[29647]: 4.1d scrub starts 2024-09-11T08:49:27.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[29647]: 4.1d scrub ok 2024-09-11T08:49:27.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:27.260 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[41114]: 5.b scrub starts 2024-09-11T08:49:27.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[41114]: 5.b scrub ok 2024-09-11T08:49:27.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[41114]: 3.3 deep-scrub starts 2024-09-11T08:49:27.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[41114]: 3.3 deep-scrub ok 2024-09-11T08:49:27.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[41114]: 3.5 scrub starts 2024-09-11T08:49:27.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[41114]: 3.5 scrub ok 2024-09-11T08:49:27.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[41114]: 4.1d scrub starts 2024-09-11T08:49:27.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[41114]: 4.1d scrub ok 2024-09-11T08:49:27.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:26 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:27.298 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:27.319 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:27.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:27.320 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:27.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:26 smithi145 ceph-mon[33384]: 5.b scrub starts 2024-09-11T08:49:27.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:26 smithi145 ceph-mon[33384]: 5.b scrub ok 2024-09-11T08:49:27.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:26 smithi145 ceph-mon[33384]: 3.3 deep-scrub starts 2024-09-11T08:49:27.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:26 smithi145 ceph-mon[33384]: 3.3 deep-scrub ok 2024-09-11T08:49:27.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:26 smithi145 ceph-mon[33384]: 3.5 scrub starts 2024-09-11T08:49:27.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:26 smithi145 ceph-mon[33384]: 3.5 scrub ok 2024-09-11T08:49:27.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:26 smithi145 ceph-mon[33384]: 4.1d scrub starts 2024-09-11T08:49:27.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:26 smithi145 ceph-mon[33384]: 4.1d scrub ok 2024-09-11T08:49:27.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:26 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:27.421 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:27.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:27.455 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:27.456 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:27.556 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:27.591 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:27.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:27.592 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:27.693 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:27.720 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 13 16 34335 34319 164.981 130.5 0.00413319 0.00605553 2024-09-11T08:49:27.730 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:27.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:27.732 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:27.832 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:27.868 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:27.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:27.869 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:27.969 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:28.006 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:28.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:28.007 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:28.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:28.136 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:28.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:28.137 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:28.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:28 smithi145 ceph-mon[33384]: pgmap v456: 148 pgs: 148 active+clean; 1.9 GiB data, 7.8 GiB used, 618 GiB / 626 GiB avail; 1023 B/s rd, 160 MiB/s wr, 2.56k op/s; 32 B/s, 0 objects/s recovering 2024-09-11T08:49:28.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:28 smithi145 ceph-mon[33384]: 5.13 scrub starts 2024-09-11T08:49:28.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:28 smithi145 ceph-mon[33384]: 5.13 scrub ok 2024-09-11T08:49:28.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:28.268 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:28.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:28.269 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:28.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:28 smithi092 ceph-mon[29647]: pgmap v456: 148 pgs: 148 active+clean; 1.9 GiB data, 7.8 GiB used, 618 GiB / 626 GiB avail; 1023 B/s rd, 160 MiB/s wr, 2.56k op/s; 32 B/s, 0 objects/s recovering 2024-09-11T08:49:28.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:28 smithi092 ceph-mon[29647]: 5.13 scrub starts 2024-09-11T08:49:28.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:28 smithi092 ceph-mon[29647]: 5.13 scrub ok 2024-09-11T08:49:28.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:28 smithi092 ceph-mon[41114]: pgmap v456: 148 pgs: 148 active+clean; 1.9 GiB data, 7.8 GiB used, 618 GiB / 626 GiB avail; 1023 B/s rd, 160 MiB/s wr, 2.56k op/s; 32 B/s, 0 objects/s recovering 2024-09-11T08:49:28.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:28 smithi092 ceph-mon[41114]: 5.13 scrub starts 2024-09-11T08:49:28.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:28 smithi092 ceph-mon[41114]: 5.13 scrub ok 2024-09-11T08:49:28.370 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:28.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:28.406 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:28.406 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:28.507 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:28.539 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:28.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:28.540 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:28.641 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:28.663 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:28.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:28.664 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:28.721 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 14 15 36184 36169 161.455 115.625 0.00598122 0.00618887 2024-09-11T08:49:28.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:28.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:28.799 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:28.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:28.929 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:28.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:28.930 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:29.031 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:29.060 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:29.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:29.061 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:29.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T08:49:29.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[29647]: 5.d scrub ok 2024-09-11T08:49:29.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[29647]: 2.0 scrub starts 2024-09-11T08:49:29.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[29647]: 2.0 scrub ok 2024-09-11T08:49:29.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[29647]: 5.2 deep-scrub starts 2024-09-11T08:49:29.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[29647]: 5.2 deep-scrub ok 2024-09-11T08:49:29.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[29647]: 6.19 scrub starts 2024-09-11T08:49:29.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[29647]: 6.19 scrub ok 2024-09-11T08:49:29.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[29647]: 4.14 scrub starts 2024-09-11T08:49:29.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[29647]: 4.14 scrub ok 2024-09-11T08:49:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T08:49:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[41114]: 5.d scrub ok 2024-09-11T08:49:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[41114]: 2.0 scrub starts 2024-09-11T08:49:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[41114]: 2.0 scrub ok 2024-09-11T08:49:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[41114]: 5.2 deep-scrub starts 2024-09-11T08:49:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[41114]: 5.2 deep-scrub ok 2024-09-11T08:49:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[41114]: 6.19 scrub starts 2024-09-11T08:49:29.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[41114]: 6.19 scrub ok 2024-09-11T08:49:29.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[41114]: 4.14 scrub starts 2024-09-11T08:49:29.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:29 smithi092 ceph-mon[41114]: 4.14 scrub ok 2024-09-11T08:49:29.162 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:29.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:29 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T08:49:29.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:29 smithi145 ceph-mon[33384]: 5.d scrub ok 2024-09-11T08:49:29.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:29 smithi145 ceph-mon[33384]: 2.0 scrub starts 2024-09-11T08:49:29.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:29 smithi145 ceph-mon[33384]: 2.0 scrub ok 2024-09-11T08:49:29.164 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:29 smithi145 ceph-mon[33384]: 5.2 deep-scrub starts 2024-09-11T08:49:29.164 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:29 smithi145 ceph-mon[33384]: 5.2 deep-scrub ok 2024-09-11T08:49:29.164 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:29 smithi145 ceph-mon[33384]: 6.19 scrub starts 2024-09-11T08:49:29.164 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:29 smithi145 ceph-mon[33384]: 6.19 scrub ok 2024-09-11T08:49:29.164 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:29 smithi145 ceph-mon[33384]: 4.14 scrub starts 2024-09-11T08:49:29.164 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:29 smithi145 ceph-mon[33384]: 4.14 scrub ok 2024-09-11T08:49:29.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:29.202 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:29.202 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:29.273 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:29.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:29.316 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:49:29.316 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:49:29.346 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:49:29.346 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:49:29.348 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:29.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:29.349 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:29.370 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:49:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:49:29.394 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:49:29.394 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:49:29.403 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:49:29.403 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:49:29.421 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:49:29.422 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:49:29.450 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:49:29.450 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:49:29.450 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:29.483 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:49:29.483 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:49:29.487 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:29.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:29.488 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:29.519 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:49:29.520 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:49:29.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:29.611 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:29.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:29.612 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:29.713 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:29.720 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 15 16 37883 37867 157.766 106.125 0.00624127 0.00633226 2024-09-11T08:49:29.747 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:29.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:29.748 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:29.849 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:29.884 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:29.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:29.885 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:29.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:49:29.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:30.023 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:30.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:30.024 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:30.125 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:30.162 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:30.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:30.163 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:30.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:30.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:30.305 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:30.305 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:30.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:30 smithi092 ceph-mon[41114]: pgmap v457: 148 pgs: 148 active+clean; 1.9 GiB data, 7.8 GiB used, 618 GiB / 626 GiB avail; 1023 B/s rd, 160 MiB/s wr, 2.56k op/s; 32 B/s, 0 objects/s recovering 2024-09-11T08:49:30.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:30 smithi092 ceph-mon[41114]: 5.f scrub starts 2024-09-11T08:49:30.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:30 smithi092 ceph-mon[41114]: 5.f scrub ok 2024-09-11T08:49:30.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:30 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4290124433' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:30.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:30 smithi092 ceph-mon[41114]: 6.1d scrub starts 2024-09-11T08:49:30.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:30 smithi092 ceph-mon[41114]: 6.1d scrub ok 2024-09-11T08:49:30.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:30 smithi092 ceph-mon[29647]: pgmap v457: 148 pgs: 148 active+clean; 1.9 GiB data, 7.8 GiB used, 618 GiB / 626 GiB avail; 1023 B/s rd, 160 MiB/s wr, 2.56k op/s; 32 B/s, 0 objects/s recovering 2024-09-11T08:49:30.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:30 smithi092 ceph-mon[29647]: 5.f scrub starts 2024-09-11T08:49:30.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:30 smithi092 ceph-mon[29647]: 5.f scrub ok 2024-09-11T08:49:30.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:30 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4290124433' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:30.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:30 smithi092 ceph-mon[29647]: 6.1d scrub starts 2024-09-11T08:49:30.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:30 smithi092 ceph-mon[29647]: 6.1d scrub ok 2024-09-11T08:49:30.406 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:30 smithi145 ceph-mon[33384]: pgmap v457: 148 pgs: 148 active+clean; 1.9 GiB data, 7.8 GiB used, 618 GiB / 626 GiB avail; 1023 B/s rd, 160 MiB/s wr, 2.56k op/s; 32 B/s, 0 objects/s recovering 2024-09-11T08:49:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:30 smithi145 ceph-mon[33384]: 5.f scrub starts 2024-09-11T08:49:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:30 smithi145 ceph-mon[33384]: 5.f scrub ok 2024-09-11T08:49:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:30 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4290124433' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:30 smithi145 ceph-mon[33384]: 6.1d scrub starts 2024-09-11T08:49:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:30 smithi145 ceph-mon[33384]: 6.1d scrub ok 2024-09-11T08:49:30.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:30.445 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:30.445 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:30.476 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-11T08:49:30.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:30.583 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:30.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:30.584 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:30.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:30.719 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:30.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:30.720 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:30.721 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 16 15 39857 39842 155.619 123.438 0.00782814 0.00642118 2024-09-11T08:49:30.821 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:30.853 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:30.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:30.854 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:30.954 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:49:30.955 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:31.004 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:31.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:31.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:31.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:31.144 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:31.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:31.145 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:31.246 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:31.282 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:31 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T08:49:31.282 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:31 smithi145 ceph-mon[33384]: 6.f scrub ok 2024-09-11T08:49:31.282 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:31 smithi145 ceph-mon[33384]: 5.1 deep-scrub starts 2024-09-11T08:49:31.282 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:31 smithi145 ceph-mon[33384]: 5.1 deep-scrub ok 2024-09-11T08:49:31.283 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:31 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:31.283 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:31 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:31.283 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:31 smithi145 ceph-mon[33384]: 3.c scrub starts 2024-09-11T08:49:31.283 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:31 smithi145 ceph-mon[33384]: 3.c scrub ok 2024-09-11T08:49:31.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:31.285 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:31.285 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:31.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T08:49:31.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[29647]: 6.f scrub ok 2024-09-11T08:49:31.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[29647]: 5.1 deep-scrub starts 2024-09-11T08:49:31.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[29647]: 5.1 deep-scrub ok 2024-09-11T08:49:31.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:31.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:31.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[29647]: 3.c scrub starts 2024-09-11T08:49:31.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[29647]: 3.c scrub ok 2024-09-11T08:49:31.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T08:49:31.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[41114]: 6.f scrub ok 2024-09-11T08:49:31.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[41114]: 5.1 deep-scrub starts 2024-09-11T08:49:31.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[41114]: 5.1 deep-scrub ok 2024-09-11T08:49:31.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:31.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:31.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[41114]: 3.c scrub starts 2024-09-11T08:49:31.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-mon[41114]: 3.c scrub ok 2024-09-11T08:49:31.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:31.413 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:31.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:31.414 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:31.515 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:31.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:31.554 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:31.554 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:31.655 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:31.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:31.691 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:31.691 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:31.721 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 17 16 41760 41744 153.457 118.875 0.00708773 0.00651227 2024-09-11T08:49:31.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:31.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:49:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:49:31] "GET /metrics HTTP/1.1" 200 224367 "" "Prometheus/2.43.0" 2024-09-11T08:49:31.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:31.826 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:31.827 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:31.888 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:31.927 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:31.961 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:31.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:31.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:32.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:32.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:32.097 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:32.097 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:32.180 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:32 smithi092 ceph-mon[41114]: 6.12 deep-scrub starts 2024-09-11T08:49:32.180 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:32 smithi092 ceph-mon[41114]: 6.12 deep-scrub ok 2024-09-11T08:49:32.180 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:32 smithi092 ceph-mon[41114]: pgmap v458: 148 pgs: 148 active+clean; 2.1 GiB data, 8.4 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 149 MiB/s wr, 2.39k op/s; 21 B/s, 0 objects/s recovering 2024-09-11T08:49:32.180 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:32 smithi092 ceph-mon[29647]: 6.12 deep-scrub starts 2024-09-11T08:49:32.181 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:32 smithi092 ceph-mon[29647]: 6.12 deep-scrub ok 2024-09-11T08:49:32.181 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:32 smithi092 ceph-mon[29647]: pgmap v458: 148 pgs: 148 active+clean; 2.1 GiB data, 8.4 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 149 MiB/s wr, 2.39k op/s; 21 B/s, 0 objects/s recovering 2024-09-11T08:49:32.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:32.237 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:32.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:32.238 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:32.330 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:32 smithi145 ceph-mon[33384]: 6.12 deep-scrub starts 2024-09-11T08:49:32.331 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:32 smithi145 ceph-mon[33384]: 6.12 deep-scrub ok 2024-09-11T08:49:32.331 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:32 smithi145 ceph-mon[33384]: pgmap v458: 148 pgs: 148 active+clean; 2.1 GiB data, 8.4 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 149 MiB/s wr, 2.39k op/s; 21 B/s, 0 objects/s recovering 2024-09-11T08:49:32.338 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:32.371 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:32.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:32.372 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:32.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:32.510 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:32.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:32.512 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:32.613 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:32.647 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:32.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:32.648 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:32.721 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 18 16 43748 43732 151.834 124.25 0.0101396 0.00658072 2024-09-11T08:49:32.748 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:32.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:32.788 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:32.788 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:32.860 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:49:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:49:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:49:32.890 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:32.925 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:32.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:32.926 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:33.021 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:49:33.021 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T08:49:33.021 INFO:teuthology.orchestra.run.smithi145.stdout: "complaint_time": 30, 2024-09-11T08:49:33.021 INFO:teuthology.orchestra.run.smithi145.stdout: "num_blocked_ops": 0 2024-09-11T08:49:33.021 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:49:33.027 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:33.060 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:33.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:33.061 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:33.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:49:33.161 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:33.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:33 smithi145 ceph-mon[33384]: 6.3 scrub starts 2024-09-11T08:49:33.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:33 smithi145 ceph-mon[33384]: 6.3 scrub ok 2024-09-11T08:49:33.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:33 smithi145 ceph-mon[33384]: 2.1 scrub starts 2024-09-11T08:49:33.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:33 smithi145 ceph-mon[33384]: 2.1 scrub ok 2024-09-11T08:49:33.197 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:33.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:33.198 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:33.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:33.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:33 smithi092 ceph-mon[29647]: 6.3 scrub starts 2024-09-11T08:49:33.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:33 smithi092 ceph-mon[29647]: 6.3 scrub ok 2024-09-11T08:49:33.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:33 smithi092 ceph-mon[29647]: 2.1 scrub starts 2024-09-11T08:49:33.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:33 smithi092 ceph-mon[29647]: 2.1 scrub ok 2024-09-11T08:49:33.300 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:33 smithi092 ceph-mon[41114]: 6.3 scrub starts 2024-09-11T08:49:33.300 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:33 smithi092 ceph-mon[41114]: 6.3 scrub ok 2024-09-11T08:49:33.300 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:33 smithi092 ceph-mon[41114]: 2.1 scrub starts 2024-09-11T08:49:33.301 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:33 smithi092 ceph-mon[41114]: 2.1 scrub ok 2024-09-11T08:49:33.339 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:33.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:33.341 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:33.442 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:33.477 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:33.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:33.478 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:33.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:33.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:33.620 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:33.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:33.722 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 19 15 45576 45561 149.859 114.312 0.00746718 0.00666805 2024-09-11T08:49:33.744 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-11T08:49:33.751 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:33.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:33.752 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:33.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:33.888 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:33.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:33.889 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:33.990 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:34.043 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:34.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:34.047 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:34.057 INFO:teuthology.orchestra.run.smithi092.stderr:set pool 9 pgp_num to 16 2024-09-11T08:49:34.148 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:34.165 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:49:34.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:34.176 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:34.277 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:34.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:34 smithi092 ceph-mon[29647]: 3.16 scrub starts 2024-09-11T08:49:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:34 smithi092 ceph-mon[29647]: 3.16 scrub ok 2024-09-11T08:49:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:34 smithi092 ceph-mon[29647]: pgmap v459: 148 pgs: 148 active+clean; 2.5 GiB data, 9.5 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 126 MiB/s wr, 2.02k op/s 2024-09-11T08:49:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:34 smithi092 ceph-mon[29647]: 6.18 deep-scrub starts 2024-09-11T08:49:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:34 smithi092 ceph-mon[29647]: 6.18 deep-scrub ok 2024-09-11T08:49:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:34 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4216416330' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-11T08:49:34.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:34 smithi092 ceph-mon[41114]: 3.16 scrub starts 2024-09-11T08:49:34.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:34 smithi092 ceph-mon[41114]: 3.16 scrub ok 2024-09-11T08:49:34.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:34 smithi092 ceph-mon[41114]: pgmap v459: 148 pgs: 148 active+clean; 2.5 GiB data, 9.5 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 126 MiB/s wr, 2.02k op/s 2024-09-11T08:49:34.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:34 smithi092 ceph-mon[41114]: 6.18 deep-scrub starts 2024-09-11T08:49:34.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:34 smithi092 ceph-mon[41114]: 6.18 deep-scrub ok 2024-09-11T08:49:34.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:34 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4216416330' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-11T08:49:34.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:34.318 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:34.318 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:34.318 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:34 smithi145 ceph-mon[33384]: 3.16 scrub starts 2024-09-11T08:49:34.318 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:34 smithi145 ceph-mon[33384]: 3.16 scrub ok 2024-09-11T08:49:34.319 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:34 smithi145 ceph-mon[33384]: pgmap v459: 148 pgs: 148 active+clean; 2.5 GiB data, 9.5 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 126 MiB/s wr, 2.02k op/s 2024-09-11T08:49:34.319 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:34 smithi145 ceph-mon[33384]: 6.18 deep-scrub starts 2024-09-11T08:49:34.319 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:34 smithi145 ceph-mon[33384]: 6.18 deep-scrub ok 2024-09-11T08:49:34.319 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:34 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4216416330' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-11T08:49:34.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:34.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:34.454 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:34.454 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:34.555 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:34.592 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:34.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:34.593 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:34.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:34.721 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:49:34.722618+0000 min lat: 0.00097685 max lat: 0.147323 avg lat: 0.00677808 2024-09-11T08:49:34.721 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:49:34.721 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 20 16 47201 47185 147.44 101.5 0.0102988 0.00677808 2024-09-11T08:49:34.729 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:34.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:34.730 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:34.831 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:34.859 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:34.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:34.860 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:34.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:34.997 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:34.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:34.998 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:35.098 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:35.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:35.134 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:35.134 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:35.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:35.268 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:35.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:35.269 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:35.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:35 smithi092 ceph-mon[29647]: 3.10 scrub starts 2024-09-11T08:49:35.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:35 smithi092 ceph-mon[29647]: 3.10 scrub ok 2024-09-11T08:49:35.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:35 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4216416330' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-11T08:49:35.309 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:35 smithi092 ceph-mon[29647]: osdmap e217: 8 total, 8 up, 7 in 2024-09-11T08:49:35.310 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:35 smithi092 ceph-mon[41114]: 3.10 scrub starts 2024-09-11T08:49:35.310 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:35 smithi092 ceph-mon[41114]: 3.10 scrub ok 2024-09-11T08:49:35.310 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:35 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4216416330' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-11T08:49:35.310 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:35 smithi092 ceph-mon[41114]: osdmap e217: 8 total, 8 up, 7 in 2024-09-11T08:49:35.369 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:35.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:35.404 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:35.404 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:35.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:35 smithi145 ceph-mon[33384]: 3.10 scrub starts 2024-09-11T08:49:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:35 smithi145 ceph-mon[33384]: 3.10 scrub ok 2024-09-11T08:49:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:35 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4216416330' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-11T08:49:35.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:35 smithi145 ceph-mon[33384]: osdmap e217: 8 total, 8 up, 7 in 2024-09-11T08:49:35.504 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:35.541 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:35.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:35.542 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:35.643 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:35.674 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:35.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:35.676 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:35.721 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 21 16 49082 49066 146.017 117.562 0.0062693 0.00684348 2024-09-11T08:49:35.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:35.808 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:35.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:35.810 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:35.911 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:35.950 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:35.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:35.951 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:36.052 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:36.084 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:36.085 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:36 smithi092 ceph-mon[29647]: 4.18 scrub starts 2024-09-11T08:49:36.085 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:36 smithi092 ceph-mon[29647]: 4.18 scrub ok 2024-09-11T08:49:36.085 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:36 smithi092 ceph-mon[29647]: pgmap v461: 148 pgs: 148 active+clean; 2.5 GiB data, 9.5 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 126 MiB/s wr, 2.02k op/s 2024-09-11T08:49:36.086 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:36 smithi092 ceph-mon[41114]: 4.18 scrub starts 2024-09-11T08:49:36.086 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:36 smithi092 ceph-mon[41114]: 4.18 scrub ok 2024-09-11T08:49:36.086 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:36 smithi092 ceph-mon[41114]: pgmap v461: 148 pgs: 148 active+clean; 2.5 GiB data, 9.5 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 126 MiB/s wr, 2.02k op/s 2024-09-11T08:49:36.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:36.087 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:36.161 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:36.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:36.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:36.236 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:36.237 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:36.312 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:36 smithi145 ceph-mon[33384]: 4.18 scrub starts 2024-09-11T08:49:36.312 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:36 smithi145 ceph-mon[33384]: 4.18 scrub ok 2024-09-11T08:49:36.312 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:36 smithi145 ceph-mon[33384]: pgmap v461: 148 pgs: 148 active+clean; 2.5 GiB data, 9.5 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 126 MiB/s wr, 2.02k op/s 2024-09-11T08:49:36.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:36.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:36.379 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:36.379 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:36.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:36.518 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:36.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:36.519 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:36.620 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:36.633 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:49:36.634 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T08:49:36.634 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T08:49:36.634 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:49:36.634 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.634 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:65023 7.5 7:a37fd951:::benchmark_data_smithi092_84565_object65022:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:49:36.634 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:45.062497+0000", 2024-09-11T08:49:36.634 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 471.56750198700001, 2024-09-11T08:49:36.634 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.110606182, 2024-09-11T08:49:36.634 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.634 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.634 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.635 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:36.635 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:36.635 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 65023 2024-09-11T08:49:36.635 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.635 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.635 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.635 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.635 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.062497+0000", 2024-09-11T08:49:36.635 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.635 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.635 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.635 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.636 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.062497+0000", 2024-09-11T08:49:36.636 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.636 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.636 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.636 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.636 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.062496+0000", 2024-09-11T08:49:36.636 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:36.636 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.636 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.636 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.636 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.062551+0000", 2024-09-11T08:49:36.636 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4549999999999998e-05 2024-09-11T08:49:36.637 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.637 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.637 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.637 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.062561+0000", 2024-09-11T08:49:36.637 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0617e-05 2024-09-11T08:49:36.637 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.637 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.637 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.637 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.062567+0000", 2024-09-11T08:49:36.637 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5639999999999996e-06 2024-09-11T08:49:36.638 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.638 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.638 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.638 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.064577+0000", 2024-09-11T08:49:36.638 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0020101759999999998 2024-09-11T08:49:36.638 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.638 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.638 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.638 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.064713+0000", 2024-09-11T08:49:36.638 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013615900000000001 2024-09-11T08:49:36.638 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.638 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.638 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:49:36.638 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.064861+0000", 2024-09-11T08:49:36.639 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014763999999999999 2024-09-11T08:49:36.639 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.639 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.639 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.639 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.073641+0000", 2024-09-11T08:49:36.639 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0087798459999999991 2024-09-11T08:49:36.639 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.639 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.639 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.639 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.073664+0000", 2024-09-11T08:49:36.639 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3561999999999999e-05 2024-09-11T08:49:36.639 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.640 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.640 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.640 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.173016+0000", 2024-09-11T08:49:36.640 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.099352316999999996 2024-09-11T08:49:36.640 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.640 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.640 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.640 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.173063+0000", 2024-09-11T08:49:36.640 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6925000000000002e-05 2024-09-11T08:49:36.640 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.640 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.640 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.640 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.173103+0000", 2024-09-11T08:49:36.641 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9626999999999999e-05 2024-09-11T08:49:36.641 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.641 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.641 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.641 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.641 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.641 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:65027 7.5 7:aa341fe1:::benchmark_data_smithi092_84565_object65026:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:49:36.641 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:45.065213+0000", 2024-09-11T08:49:36.641 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 471.56478628899998, 2024-09-11T08:49:36.641 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10812904700000001, 2024-09-11T08:49:36.641 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.641 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.641 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.642 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:36.642 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:36.642 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 65027 2024-09-11T08:49:36.642 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.642 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.642 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.642 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.643 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.065213+0000", 2024-09-11T08:49:36.643 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.643 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.643 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.643 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.643 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.065213+0000", 2024-09-11T08:49:36.643 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.643 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.644 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.644 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.644 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.065211+0000", 2024-09-11T08:49:36.644 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:36.644 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.644 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.644 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.644 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.065272+0000", 2024-09-11T08:49:36.645 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1016e-05 2024-09-11T08:49:36.645 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.645 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.645 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.645 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.065286+0000", 2024-09-11T08:49:36.645 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3913999999999999e-05 2024-09-11T08:49:36.645 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.646 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.646 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.646 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.065293+0000", 2024-09-11T08:49:36.646 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3949999999999998e-06 2024-09-11T08:49:36.646 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.646 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.646 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.646 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.067452+0000", 2024-09-11T08:49:36.647 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0021589980000000001 2024-09-11T08:49:36.647 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.647 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.647 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.647 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.067562+0000", 2024-09-11T08:49:36.647 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000109898 2024-09-11T08:49:36.647 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.647 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.648 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:49:36.648 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.067704+0000", 2024-09-11T08:49:36.648 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000142001 2024-09-11T08:49:36.648 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.648 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.648 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.648 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.073668+0000", 2024-09-11T08:49:36.648 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0059640730000000003 2024-09-11T08:49:36.648 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.649 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.649 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.649 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.073714+0000", 2024-09-11T08:49:36.649 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6826999999999997e-05 2024-09-11T08:49:36.649 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.649 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.649 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.650 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.173274+0000", 2024-09-11T08:49:36.650 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.09955957 2024-09-11T08:49:36.650 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.650 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.650 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.650 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.173312+0000", 2024-09-11T08:49:36.650 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8235000000000002e-05 2024-09-11T08:49:36.650 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.650 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.651 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.651 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.173342+0000", 2024-09-11T08:49:36.651 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9249999999999999e-05 2024-09-11T08:49:36.651 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.651 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.651 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.651 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.651 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.652 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:65028 7.5 7:a8f242f4:::benchmark_data_smithi092_84565_object65027:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-11T08:49:36.652 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:41:45.066916+0000", 2024-09-11T08:49:36.652 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 471.563082377, 2024-09-11T08:49:36.652 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.109535827, 2024-09-11T08:49:36.652 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.652 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.652 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.652 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:36.653 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:36.653 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 65028 2024-09-11T08:49:36.653 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.653 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.653 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.653 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.653 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.066916+0000", 2024-09-11T08:49:36.654 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.654 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.654 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.654 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.654 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.066916+0000", 2024-09-11T08:49:36.654 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.654 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.655 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.655 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.655 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.066915+0000", 2024-09-11T08:49:36.655 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:49:36.655 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.655 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.655 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.655 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.066965+0000", 2024-09-11T08:49:36.656 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0275000000000002e-05 2024-09-11T08:49:36.656 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.656 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.656 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.656 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.066978+0000", 2024-09-11T08:49:36.656 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2699e-05 2024-09-11T08:49:36.656 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.656 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.656 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.656 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.066985+0000", 2024-09-11T08:49:36.656 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8940000000000004e-06 2024-09-11T08:49:36.656 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.656 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.657 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.657 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.070840+0000", 2024-09-11T08:49:36.657 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0038552539999999998 2024-09-11T08:49:36.657 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.657 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.657 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.657 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.070951+0000", 2024-09-11T08:49:36.657 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000110853 2024-09-11T08:49:36.657 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.657 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.657 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:49:36.657 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.071781+0000", 2024-09-11T08:49:36.657 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00083066599999999998 2024-09-11T08:49:36.658 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.658 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.658 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.658 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.073852+0000", 2024-09-11T08:49:36.658 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0020706499999999998 2024-09-11T08:49:36.658 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.658 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.658 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.658 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.074529+0000", 2024-09-11T08:49:36.658 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00067736599999999999 2024-09-11T08:49:36.658 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.658 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.658 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.658 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.176399+0000", 2024-09-11T08:49:36.659 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.101869521 2024-09-11T08:49:36.659 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.659 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.659 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.659 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.176432+0000", 2024-09-11T08:49:36.659 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2635000000000002e-05 2024-09-11T08:49:36.659 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.659 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.659 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.659 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:41:45.176452+0000", 2024-09-11T08:49:36.659 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0608999999999999e-05 2024-09-11T08:49:36.659 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.659 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.660 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.660 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.660 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.660 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:115423 7.5 7:af8edbfa:::benchmark_data_smithi092_84565_object115422:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:49:36.660 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:31.172384+0000", 2024-09-11T08:49:36.660 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 425.45761470999997, 2024-09-11T08:49:36.660 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.100952319, 2024-09-11T08:49:36.660 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.660 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.660 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.660 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:36.660 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:36.660 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 115423 2024-09-11T08:49:36.660 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.660 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.661 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.661 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.661 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.172384+0000", 2024-09-11T08:49:36.661 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.661 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.661 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.661 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.661 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.172384+0000", 2024-09-11T08:49:36.661 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.661 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.661 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.661 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.662 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.172383+0000", 2024-09-11T08:49:36.662 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:36.662 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.662 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.662 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.662 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.172427+0000", 2024-09-11T08:49:36.662 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3574e-05 2024-09-11T08:49:36.662 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.662 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.662 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.662 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.172439+0000", 2024-09-11T08:49:36.663 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2447e-05 2024-09-11T08:49:36.663 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.663 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.663 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.663 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.172445+0000", 2024-09-11T08:49:36.663 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9340000000000003e-06 2024-09-11T08:49:36.663 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.663 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.663 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.663 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.174023+0000", 2024-09-11T08:49:36.663 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001578168 2024-09-11T08:49:36.664 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.664 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.664 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.664 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.174123+0000", 2024-09-11T08:49:36.664 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.9572000000000001e-05 2024-09-11T08:49:36.664 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.664 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.664 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:49:36.664 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.174263+0000", 2024-09-11T08:49:36.664 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000140512 2024-09-11T08:49:36.664 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.664 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.665 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.665 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.176061+0000", 2024-09-11T08:49:36.665 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0017974930000000001 2024-09-11T08:49:36.665 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.665 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.665 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.665 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.179485+0000", 2024-09-11T08:49:36.665 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0034245920000000002 2024-09-11T08:49:36.665 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.665 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.665 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.665 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.273239+0000", 2024-09-11T08:49:36.665 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.093753603000000005 2024-09-11T08:49:36.666 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.666 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.666 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.666 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.273290+0000", 2024-09-11T08:49:36.666 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1557999999999997e-05 2024-09-11T08:49:36.666 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.666 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.666 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.666 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:31.273336+0000", 2024-09-11T08:49:36.666 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5932e-05 2024-09-11T08:49:36.666 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.666 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.667 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.667 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.667 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.667 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122199 7.5 7:af733e33:::benchmark_data_smithi092_84565_object122198:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-11T08:49:36.667 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.178485+0000", 2024-09-11T08:49:36.667 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 418.45151341100001, 2024-09-11T08:49:36.667 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.098087341999999994, 2024-09-11T08:49:36.667 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.667 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.667 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.667 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:36.667 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:36.667 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122199 2024-09-11T08:49:36.668 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.668 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.668 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.668 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.668 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.178485+0000", 2024-09-11T08:49:36.668 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.668 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.668 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.668 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.669 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.178485+0000", 2024-09-11T08:49:36.669 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.669 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.669 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.669 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.669 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.178484+0000", 2024-09-11T08:49:36.669 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:36.669 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.670 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.670 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.670 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.178607+0000", 2024-09-11T08:49:36.670 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000122714 2024-09-11T08:49:36.670 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.671 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.671 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.671 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.178629+0000", 2024-09-11T08:49:36.671 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.1466000000000001e-05 2024-09-11T08:49:36.671 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.671 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.671 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.671 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.178641+0000", 2024-09-11T08:49:36.671 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.236e-05 2024-09-11T08:49:36.671 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.671 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.671 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.671 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.178684+0000", 2024-09-11T08:49:36.672 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3149000000000003e-05 2024-09-11T08:49:36.672 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.672 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.672 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.672 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.178845+0000", 2024-09-11T08:49:36.672 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000160743 2024-09-11T08:49:36.672 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.672 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.672 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:49:36.672 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.179008+0000", 2024-09-11T08:49:36.672 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000162786 2024-09-11T08:49:36.672 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.672 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.672 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.673 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.182819+0000", 2024-09-11T08:49:36.673 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0038115480000000001 2024-09-11T08:49:36.673 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.673 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.673 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.673 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.186443+0000", 2024-09-11T08:49:36.673 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003623753 2024-09-11T08:49:36.673 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.673 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.673 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.673 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.276511+0000", 2024-09-11T08:49:36.673 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.090068100999999998 2024-09-11T08:49:36.673 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.673 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.674 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.674 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.276549+0000", 2024-09-11T08:49:36.674 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7616e-05 2024-09-11T08:49:36.674 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.674 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.674 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.674 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.276573+0000", 2024-09-11T08:49:36.674 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4082000000000001e-05 2024-09-11T08:49:36.674 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.674 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.674 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.674 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.675 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.675 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122653 7.5 7:a89a7b3a:::benchmark_data_smithi092_84565_object122652:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:49:36.675 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.693490+0000", 2024-09-11T08:49:36.675 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 417.936508602, 2024-09-11T08:49:36.675 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10533427099999999, 2024-09-11T08:49:36.675 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.675 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.675 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.675 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:36.675 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:36.675 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122653 2024-09-11T08:49:36.675 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.675 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.675 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.676 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.676 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.693490+0000", 2024-09-11T08:49:36.676 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.676 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.676 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.676 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.676 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.693490+0000", 2024-09-11T08:49:36.676 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.676 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.676 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.676 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.676 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.693489+0000", 2024-09-11T08:49:36.676 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:49:36.676 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.677 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.677 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.677 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.693537+0000", 2024-09-11T08:49:36.677 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8069999999999999e-05 2024-09-11T08:49:36.677 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.677 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.677 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.677 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.693552+0000", 2024-09-11T08:49:36.677 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5412000000000001e-05 2024-09-11T08:49:36.677 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.677 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.677 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.677 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.693559+0000", 2024-09-11T08:49:36.678 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.0369999999999997e-06 2024-09-11T08:49:36.678 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.678 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.678 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.678 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.693598+0000", 2024-09-11T08:49:36.678 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8392000000000001e-05 2024-09-11T08:49:36.678 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.678 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.678 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.678 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.693713+0000", 2024-09-11T08:49:36.678 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000115116 2024-09-11T08:49:36.678 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.678 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.678 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:49:36.679 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.693856+0000", 2024-09-11T08:49:36.679 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014262000000000001 2024-09-11T08:49:36.679 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.679 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.679 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.679 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.696160+0000", 2024-09-11T08:49:36.679 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0023044110000000001 2024-09-11T08:49:36.679 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.679 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.679 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.679 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.700185+0000", 2024-09-11T08:49:36.679 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0040252789999999997 2024-09-11T08:49:36.679 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.679 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.680 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.680 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.798720+0000", 2024-09-11T08:49:36.680 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.098534439000000001 2024-09-11T08:49:36.680 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.680 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.680 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.680 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.798783+0000", 2024-09-11T08:49:36.680 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3607000000000003e-05 2024-09-11T08:49:36.680 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.680 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.680 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.680 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.798824+0000", 2024-09-11T08:49:36.680 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1187999999999999e-05 2024-09-11T08:49:36.680 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.681 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.681 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.681 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.681 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.681 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122665 7.5 7:a70b47f7:::benchmark_data_smithi092_84565_object122664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:49:36.681 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.702681+0000", 2024-09-11T08:49:36.681 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 417.92731764500002, 2024-09-11T08:49:36.681 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.103187668, 2024-09-11T08:49:36.681 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.681 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.681 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.681 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:36.681 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:36.682 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122665 2024-09-11T08:49:36.682 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.682 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.682 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.682 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.682 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702681+0000", 2024-09-11T08:49:36.682 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.682 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.682 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.682 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.682 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702681+0000", 2024-09-11T08:49:36.683 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.683 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.683 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.683 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.683 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702680+0000", 2024-09-11T08:49:36.683 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:49:36.683 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.683 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.683 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.683 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702739+0000", 2024-09-11T08:49:36.683 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9080999999999999e-05 2024-09-11T08:49:36.683 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.683 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.683 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.683 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702751+0000", 2024-09-11T08:49:36.684 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2344000000000001e-05 2024-09-11T08:49:36.684 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.684 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.684 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.684 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702755+0000", 2024-09-11T08:49:36.684 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1400000000000002e-06 2024-09-11T08:49:36.684 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.684 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.684 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.684 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702790+0000", 2024-09-11T08:49:36.684 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4499999999999998e-05 2024-09-11T08:49:36.684 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.684 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.684 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.685 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.702893+0000", 2024-09-11T08:49:36.685 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010382000000000001 2024-09-11T08:49:36.685 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.685 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.685 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:49:36.685 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.703052+0000", 2024-09-11T08:49:36.685 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015861299999999999 2024-09-11T08:49:36.685 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.685 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.685 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.685 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.705868+0000", 2024-09-11T08:49:36.685 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0028158699999999998 2024-09-11T08:49:36.685 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.686 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.686 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.686 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.706900+0000", 2024-09-11T08:49:36.686 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001032537 2024-09-11T08:49:36.686 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.686 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.686 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.686 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.805737+0000", 2024-09-11T08:49:36.686 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.098836842999999994 2024-09-11T08:49:36.686 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.686 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.686 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.686 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.805825+0000", 2024-09-11T08:49:36.686 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.7247e-05 2024-09-11T08:49:36.687 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.687 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.687 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.687 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.805869+0000", 2024-09-11T08:49:36.687 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4289000000000002e-05 2024-09-11T08:49:36.687 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.687 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.687 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.687 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.687 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.687 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122668 7.5 7:a699dc0b:::benchmark_data_smithi092_84565_object122667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:49:36.687 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.706873+0000", 2024-09-11T08:49:36.687 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 417.923125825, 2024-09-11T08:49:36.687 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.099919888999999998, 2024-09-11T08:49:36.688 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.688 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.688 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.688 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:36.688 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:36.688 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122668 2024-09-11T08:49:36.688 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.688 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.688 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.688 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.688 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.706873+0000", 2024-09-11T08:49:36.688 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.688 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.689 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.689 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.689 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.706873+0000", 2024-09-11T08:49:36.689 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.689 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.689 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.689 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.689 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.706872+0000", 2024-09-11T08:49:36.689 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:49:36.689 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.689 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.690 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.690 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.706905+0000", 2024-09-11T08:49:36.690 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3192999999999999e-05 2024-09-11T08:49:36.690 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.690 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.690 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.690 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.706914+0000", 2024-09-11T08:49:36.690 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.5820000000000008e-06 2024-09-11T08:49:36.690 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.690 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.690 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.690 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.706919+0000", 2024-09-11T08:49:36.690 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4739999999999999e-06 2024-09-11T08:49:36.690 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.690 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.691 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.691 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.706967+0000", 2024-09-11T08:49:36.691 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7765000000000001e-05 2024-09-11T08:49:36.691 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.691 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.691 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.691 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707066+0000", 2024-09-11T08:49:36.691 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.9535000000000002e-05 2024-09-11T08:49:36.691 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.691 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.691 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:49:36.691 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.707153+0000", 2024-09-11T08:49:36.691 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.6666e-05 2024-09-11T08:49:36.691 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.692 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.692 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.692 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.709545+0000", 2024-09-11T08:49:36.692 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0023925079999999998 2024-09-11T08:49:36.692 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.692 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.692 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.692 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.710236+0000", 2024-09-11T08:49:36.692 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00069045 2024-09-11T08:49:36.692 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.692 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.692 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.693 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.806691+0000", 2024-09-11T08:49:36.693 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.096454839000000001 2024-09-11T08:49:36.693 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.693 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.693 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.693 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.806740+0000", 2024-09-11T08:49:36.693 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9823000000000001e-05 2024-09-11T08:49:36.693 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.693 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.693 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.693 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.806793+0000", 2024-09-11T08:49:36.693 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.2367999999999997e-05 2024-09-11T08:49:36.693 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.694 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.694 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.694 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.694 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.694 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122816 7.5 7:abd59fc9:::benchmark_data_smithi092_84565_object122815:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:49:36.694 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.925200+0000", 2024-09-11T08:49:36.694 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 417.70479895, 2024-09-11T08:49:36.694 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.108443875, 2024-09-11T08:49:36.694 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.694 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.694 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.694 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:36.695 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:36.695 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122816 2024-09-11T08:49:36.695 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.695 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.695 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.695 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.695 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.925200+0000", 2024-09-11T08:49:36.695 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.695 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.695 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.695 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.695 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.925200+0000", 2024-09-11T08:49:36.695 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.696 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.696 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.696 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.696 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.925196+0000", 2024-09-11T08:49:36.696 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999962 2024-09-11T08:49:36.696 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.696 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.696 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.696 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.925269+0000", 2024-09-11T08:49:36.696 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.3100000000000001e-05 2024-09-11T08:49:36.696 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.697 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.697 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.697 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.925282+0000", 2024-09-11T08:49:36.697 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2901999999999999e-05 2024-09-11T08:49:36.697 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.697 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.697 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.697 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.925288+0000", 2024-09-11T08:49:36.697 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1349999999999997e-06 2024-09-11T08:49:36.697 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.697 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.697 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.697 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.925338+0000", 2024-09-11T08:49:36.698 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0074000000000002e-05 2024-09-11T08:49:36.698 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.698 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.698 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.698 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.925469+0000", 2024-09-11T08:49:36.698 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013060599999999999 2024-09-11T08:49:36.698 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.698 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.698 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:49:36.698 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.925597+0000", 2024-09-11T08:49:36.698 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000127806 2024-09-11T08:49:36.698 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.698 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.698 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.699 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.930679+0000", 2024-09-11T08:49:36.699 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0050824980000000004 2024-09-11T08:49:36.699 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.699 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.699 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.699 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.933018+0000", 2024-09-11T08:49:36.699 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0023391079999999999 2024-09-11T08:49:36.699 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.699 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.699 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.699 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:39.033564+0000", 2024-09-11T08:49:36.699 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10054529099999999 2024-09-11T08:49:36.699 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.699 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.700 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.700 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:39.033599+0000", 2024-09-11T08:49:36.700 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5689999999999999e-05 2024-09-11T08:49:36.700 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.700 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.700 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.700 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:39.033644+0000", 2024-09-11T08:49:36.700 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4323999999999998e-05 2024-09-11T08:49:36.700 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.700 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.700 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.701 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.701 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.701 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122820 7.5 7:ab85771e:::benchmark_data_smithi092_84565_object122819:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:49:36.701 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.926358+0000", 2024-09-11T08:49:36.701 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 417.70364055700003, 2024-09-11T08:49:36.701 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.107411546, 2024-09-11T08:49:36.701 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.701 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.701 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.701 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:36.701 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:36.701 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122820 2024-09-11T08:49:36.702 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.702 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.702 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.702 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.702 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.926358+0000", 2024-09-11T08:49:36.702 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.702 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.702 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.702 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.702 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.926358+0000", 2024-09-11T08:49:36.703 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.703 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.703 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.703 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.926357+0000", 2024-09-11T08:49:36.703 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:36.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.703 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.703 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.703 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.926425+0000", 2024-09-11T08:49:36.703 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.7377000000000002e-05 2024-09-11T08:49:36.704 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.704 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.704 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.704 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.926438+0000", 2024-09-11T08:49:36.704 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3078000000000001e-05 2024-09-11T08:49:36.704 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.704 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.704 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.704 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.926443+0000", 2024-09-11T08:49:36.704 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4859999999999997e-06 2024-09-11T08:49:36.704 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.704 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.704 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.705 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.927945+0000", 2024-09-11T08:49:36.705 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0015016879999999999 2024-09-11T08:49:36.705 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.705 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.705 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.705 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.928072+0000", 2024-09-11T08:49:36.705 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000126697 2024-09-11T08:49:36.705 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.705 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.705 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:49:36.705 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.928218+0000", 2024-09-11T08:49:36.705 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014666500000000001 2024-09-11T08:49:36.705 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.706 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.706 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.706 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.932968+0000", 2024-09-11T08:49:36.706 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0047498469999999997 2024-09-11T08:49:36.706 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.706 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.706 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.706 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.933039+0000", 2024-09-11T08:49:36.706 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.0976000000000006e-05 2024-09-11T08:49:36.706 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.706 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.706 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.706 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:39.033678+0000", 2024-09-11T08:49:36.706 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10063868500000001 2024-09-11T08:49:36.706 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.707 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.707 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.707 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:39.033730+0000", 2024-09-11T08:49:36.707 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.2091e-05 2024-09-11T08:49:36.707 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.707 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.707 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.707 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:39.033770+0000", 2024-09-11T08:49:36.707 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9898000000000003e-05 2024-09-11T08:49:36.707 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.707 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.707 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.707 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.708 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.708 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:122822 7.5 7:ab29c9fc:::benchmark_data_smithi092_84565_object122821:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T08:49:36.708 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:42:38.927105+0000", 2024-09-11T08:49:36.708 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 417.70289346499999, 2024-09-11T08:49:36.708 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10677537099999999, 2024-09-11T08:49:36.708 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.708 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.708 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.708 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:36.709 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:36.709 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122822 2024-09-11T08:49:36.709 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.709 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.709 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.709 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.709 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.927105+0000", 2024-09-11T08:49:36.710 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.710 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.710 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.710 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.710 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.927105+0000", 2024-09-11T08:49:36.710 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.710 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.710 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.710 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.710 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.927105+0000", 2024-09-11T08:49:36.710 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:36.711 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.711 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.711 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.711 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.927169+0000", 2024-09-11T08:49:36.711 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4404000000000002e-05 2024-09-11T08:49:36.711 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.711 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.711 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.711 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.927181+0000", 2024-09-11T08:49:36.711 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.163e-05 2024-09-11T08:49:36.712 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.712 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.712 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.712 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.927185+0000", 2024-09-11T08:49:36.712 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8570000000000003e-06 2024-09-11T08:49:36.712 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.712 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.712 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.712 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.930718+0000", 2024-09-11T08:49:36.712 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0035328159999999998 2024-09-11T08:49:36.712 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.712 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.713 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.713 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.930866+0000", 2024-09-11T08:49:36.713 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000147854 2024-09-11T08:49:36.713 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.713 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.713 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:49:36.713 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.931031+0000", 2024-09-11T08:49:36.713 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000165098 2024-09-11T08:49:36.713 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.713 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.713 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.713 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.933086+0000", 2024-09-11T08:49:36.713 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0020551459999999999 2024-09-11T08:49:36.713 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.714 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.714 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.714 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:38.933096+0000", 2024-09-11T08:49:36.714 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.5810000000000003e-06 2024-09-11T08:49:36.714 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.714 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.714 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.714 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:39.033803+0000", 2024-09-11T08:49:36.714 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10070678199999999 2024-09-11T08:49:36.714 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.714 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:36.716 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:36.717 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:36.717 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.717 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:39.033848+0000", 2024-09-11T08:49:36.717 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5008e-05 2024-09-11T08:49:36.717 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.718 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.718 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.718 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:42:39.033881+0000", 2024-09-11T08:49:36.718 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2950999999999997e-05 2024-09-11T08:49:36.718 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.718 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.718 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.718 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.718 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.718 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14814.0:150323 7.5 7:a4951b48:::benchmark_data_smithi092_84565_object24795:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-11T08:49:36.718 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:43:07.597494+0000", 2024-09-11T08:49:36.719 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 389.032504964, 2024-09-11T08:49:36.719 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.20765920199999999, 2024-09-11T08:49:36.719 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.719 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.719 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.719 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14814", 2024-09-11T08:49:36.719 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:49:36.719 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 150323 2024-09-11T08:49:36.719 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.719 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.719 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.719 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.720 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.597494+0000", 2024-09-11T08:49:36.720 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.720 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.720 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.720 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.720 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.597494+0000", 2024-09-11T08:49:36.720 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.720 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.720 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.720 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.720 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.597492+0000", 2024-09-11T08:49:36.721 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:49:36.721 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.721 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.721 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.721 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.597499+0000", 2024-09-11T08:49:36.721 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9319999999999999e-06 2024-09-11T08:49:36.721 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.721 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.721 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.721 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.597514+0000", 2024-09-11T08:49:36.721 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4586999999999999e-05 2024-09-11T08:49:36.721 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.722 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.722 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.722 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.597519+0000", 2024-09-11T08:49:36.722 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6529999999999999e-06 2024-09-11T08:49:36.722 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.722 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.722 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.722 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.597577+0000", 2024-09-11T08:49:36.722 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8116999999999999e-05 2024-09-11T08:49:36.722 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.723 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.723 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.723 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.597718+0000", 2024-09-11T08:49:36.723 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014044 2024-09-11T08:49:36.723 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.723 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.723 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:49:36.723 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.597855+0000", 2024-09-11T08:49:36.723 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000137541 2024-09-11T08:49:36.723 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.723 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.723 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.723 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.600954+0000", 2024-09-11T08:49:36.723 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003098923 2024-09-11T08:49:36.724 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.724 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.724 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.724 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.601027+0000", 2024-09-11T08:49:36.724 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2137000000000004e-05 2024-09-11T08:49:36.724 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.724 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.724 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.724 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.805065+0000", 2024-09-11T08:49:36.725 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.20403835000000001 2024-09-11T08:49:36.725 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.725 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.725 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.725 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.805113+0000", 2024-09-11T08:49:36.725 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7883999999999998e-05 2024-09-11T08:49:36.725 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.725 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.725 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.726 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:43:07.805153+0000", 2024-09-11T08:49:36.726 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0259000000000002e-05 2024-09-11T08:49:36.726 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.726 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.726 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.727 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.727 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.727 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:3590 9.e 9:7cd12c9d:::benchmark_data_smithi092_145083_object3589:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-11T08:49:36.727 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:49:15.662270+0000", 2024-09-11T08:49:36.727 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 20.967728416, 2024-09-11T08:49:36.727 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.110210946, 2024-09-11T08:49:36.727 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.727 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.728 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.728 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:49:36.728 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:49:36.728 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 3590 2024-09-11T08:49:36.728 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.728 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.728 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.728 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.728 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:15.662270+0000", 2024-09-11T08:49:36.728 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.728 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.729 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.729 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.729 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:15.662270+0000", 2024-09-11T08:49:36.729 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.729 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.729 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.729 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.729 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:15.662269+0000", 2024-09-11T08:49:36.729 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:36.729 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.729 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.729 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.729 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:15.662369+0000", 2024-09-11T08:49:36.729 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000100092 2024-09-11T08:49:36.730 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.730 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.730 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.730 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:15.662379+0000", 2024-09-11T08:49:36.730 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.5689999999999994e-06 2024-09-11T08:49:36.730 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.730 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.730 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.730 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:15.662425+0000", 2024-09-11T08:49:36.730 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6341999999999998e-05 2024-09-11T08:49:36.730 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.730 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.730 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.731 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:15.662479+0000", 2024-09-11T08:49:36.731 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3418999999999998e-05 2024-09-11T08:49:36.731 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.731 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.731 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.731 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:15.662590+0000", 2024-09-11T08:49:36.731 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011089199999999999 2024-09-11T08:49:36.731 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.731 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.731 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,4", 2024-09-11T08:49:36.731 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:15.662721+0000", 2024-09-11T08:49:36.731 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000131567 2024-09-11T08:49:36.731 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.732 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.732 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.732 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:15.663356+0000", 2024-09-11T08:49:36.732 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00063447299999999998 2024-09-11T08:49:36.732 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.732 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.732 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.732 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:15.663565+0000", 2024-09-11T08:49:36.732 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00020931899999999999 2024-09-11T08:49:36.732 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.732 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.732 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.732 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:15.772411+0000", 2024-09-11T08:49:36.732 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.108845566 2024-09-11T08:49:36.732 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.733 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.733 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.733 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:15.772445+0000", 2024-09-11T08:49:36.733 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4238e-05 2024-09-11T08:49:36.733 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.733 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.733 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.733 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:15.772481+0000", 2024-09-11T08:49:36.733 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6551999999999998e-05 2024-09-11T08:49:36.733 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.733 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.733 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.734 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.734 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.734 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:37658 9.3 9:c5315ace:::benchmark_data_smithi092_145083_object37657:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e216)", 2024-09-11T08:49:36.734 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:49:29.532484+0000", 2024-09-11T08:49:36.734 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 7.0975143950000001, 2024-09-11T08:49:36.734 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.094454870999999996, 2024-09-11T08:49:36.734 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.734 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.734 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.734 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:49:36.734 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:49:36.734 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 37658 2024-09-11T08:49:36.734 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.735 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.735 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.735 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.735 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.532484+0000", 2024-09-11T08:49:36.735 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.735 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.735 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.735 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.735 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.532484+0000", 2024-09-11T08:49:36.735 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.735 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.735 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.736 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.736 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.532483+0000", 2024-09-11T08:49:36.736 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:36.736 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.736 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.736 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.736 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.532518+0000", 2024-09-11T08:49:36.736 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4492000000000001e-05 2024-09-11T08:49:36.736 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.736 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.736 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.736 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.532528+0000", 2024-09-11T08:49:36.736 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.058e-05 2024-09-11T08:49:36.737 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.737 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.737 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.737 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.532534+0000", 2024-09-11T08:49:36.737 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6110000000000001e-06 2024-09-11T08:49:36.737 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.737 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.737 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.737 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.532607+0000", 2024-09-11T08:49:36.737 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.3510000000000006e-05 2024-09-11T08:49:36.737 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.737 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.738 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.738 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.532718+0000", 2024-09-11T08:49:36.738 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011058299999999999 2024-09-11T08:49:36.738 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.738 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.738 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:49:36.738 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.532856+0000", 2024-09-11T08:49:36.738 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000137751 2024-09-11T08:49:36.738 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.738 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.738 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.738 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.534731+0000", 2024-09-11T08:49:36.738 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001875566 2024-09-11T08:49:36.738 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.738 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.739 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.739 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537230+0000", 2024-09-11T08:49:36.739 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0024986679999999999 2024-09-11T08:49:36.739 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.739 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.739 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.739 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.626864+0000", 2024-09-11T08:49:36.739 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.089633472000000006 2024-09-11T08:49:36.739 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.739 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.739 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.739 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.626908+0000", 2024-09-11T08:49:36.739 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4159999999999997e-05 2024-09-11T08:49:36.740 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.740 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.740 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.740 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.626939+0000", 2024-09-11T08:49:36.740 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1600000000000002e-05 2024-09-11T08:49:36.740 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.740 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.740 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.740 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.740 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.740 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:37661 9.d 9:b28d841c:::benchmark_data_smithi092_145083_object37660:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e216)", 2024-09-11T08:49:36.740 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:49:29.532819+0000", 2024-09-11T08:49:36.740 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 7.0971794389999996, 2024-09-11T08:49:36.741 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.091945451999999997, 2024-09-11T08:49:36.741 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.741 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.741 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.741 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:49:36.741 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:49:36.741 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 37661 2024-09-11T08:49:36.741 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.741 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.741 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.741 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.741 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.532819+0000", 2024-09-11T08:49:36.741 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.741 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.742 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.742 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.742 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.532819+0000", 2024-09-11T08:49:36.742 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.742 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.742 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.742 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.742 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.532819+0000", 2024-09-11T08:49:36.742 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:36.742 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.742 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.742 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.742 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.532850+0000", 2024-09-11T08:49:36.743 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0939000000000003e-05 2024-09-11T08:49:36.743 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.743 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.743 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.743 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.532860+0000", 2024-09-11T08:49:36.743 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0477e-05 2024-09-11T08:49:36.743 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.743 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.743 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.743 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.532867+0000", 2024-09-11T08:49:36.743 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6100000000000002e-06 2024-09-11T08:49:36.743 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.743 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.743 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.744 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.533322+0000", 2024-09-11T08:49:36.744 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00045502900000000002 2024-09-11T08:49:36.744 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.744 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.744 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.744 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.533432+0000", 2024-09-11T08:49:36.744 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000110369 2024-09-11T08:49:36.744 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.744 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.744 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,5", 2024-09-11T08:49:36.744 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.533546+0000", 2024-09-11T08:49:36.745 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000114267 2024-09-11T08:49:36.745 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.745 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.745 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.745 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.536864+0000", 2024-09-11T08:49:36.745 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0033180900000000001 2024-09-11T08:49:36.745 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.745 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.745 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.745 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.623471+0000", 2024-09-11T08:49:36.745 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.086606403999999998 2024-09-11T08:49:36.745 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.745 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.746 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.746 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.624684+0000", 2024-09-11T08:49:36.746 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0012128810000000001 2024-09-11T08:49:36.746 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.746 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.746 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.746 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.624730+0000", 2024-09-11T08:49:36.746 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5951000000000001e-05 2024-09-11T08:49:36.746 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.746 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.746 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.746 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.624765+0000", 2024-09-11T08:49:36.747 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5213000000000003e-05 2024-09-11T08:49:36.747 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.747 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.747 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.747 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.747 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.747 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15366.0:37660 9.a e216/206 9:54ac65c7:::benchmark_data_smithi092_145083_object37659:head v 216'2335, mlcod=216'2332)", 2024-09-11T08:49:36.747 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:49:29.533544+0000", 2024-09-11T08:49:36.747 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 7.0964546430000004, 2024-09-11T08:49:36.747 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.089502683, 2024-09-11T08:49:36.747 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.747 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.747 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.747 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.747 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.748 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.533544+0000", 2024-09-11T08:49:36.748 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.748 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.748 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.748 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.748 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.533544+0000", 2024-09-11T08:49:36.748 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.748 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.748 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.748 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.749 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.533544+0000", 2024-09-11T08:49:36.749 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:49:36.749 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.749 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.749 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.749 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.533563+0000", 2024-09-11T08:49:36.749 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.9358e-05 2024-09-11T08:49:36.749 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.749 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.749 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.749 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.533573+0000", 2024-09-11T08:49:36.749 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0023999999999999e-05 2024-09-11T08:49:36.749 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.750 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.750 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.750 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.533578+0000", 2024-09-11T08:49:36.750 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7709999999999997e-06 2024-09-11T08:49:36.750 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.750 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.750 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.750 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.533661+0000", 2024-09-11T08:49:36.750 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2953000000000004e-05 2024-09-11T08:49:36.750 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.750 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.750 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.750 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.533677+0000", 2024-09-11T08:49:36.750 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6520000000000001e-05 2024-09-11T08:49:36.751 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.751 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.751 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.751 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.623004+0000", 2024-09-11T08:49:36.751 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.089327232000000006 2024-09-11T08:49:36.751 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.751 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.751 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.751 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.623047+0000", 2024-09-11T08:49:36.751 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2395999999999999e-05 2024-09-11T08:49:36.751 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.751 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.751 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.751 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.751 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.752 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15366.0:37657 9.7 e216/206 9:e1e6fb05:::benchmark_data_smithi092_145083_object37656:head v 216'2369, mlcod=216'2366)", 2024-09-11T08:49:36.752 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:49:29.533590+0000", 2024-09-11T08:49:36.752 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 7.096408372, 2024-09-11T08:49:36.752 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.090102623000000007, 2024-09-11T08:49:36.752 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.752 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.752 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.752 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.752 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.752 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.533590+0000", 2024-09-11T08:49:36.752 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.752 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.752 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.753 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.753 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.533590+0000", 2024-09-11T08:49:36.753 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.753 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.753 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.753 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.753 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.533590+0000", 2024-09-11T08:49:36.753 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:49:36.753 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.753 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.753 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.753 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.535389+0000", 2024-09-11T08:49:36.753 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0017984559999999999 2024-09-11T08:49:36.754 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.754 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.754 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.754 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.535401+0000", 2024-09-11T08:49:36.754 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2761e-05 2024-09-11T08:49:36.754 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.754 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.754 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.754 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.535407+0000", 2024-09-11T08:49:36.754 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.541e-06 2024-09-11T08:49:36.754 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.754 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.754 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.754 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.535445+0000", 2024-09-11T08:49:36.755 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8219000000000001e-05 2024-09-11T08:49:36.755 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.755 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.755 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.755 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.535461+0000", 2024-09-11T08:49:36.755 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6175999999999999e-05 2024-09-11T08:49:36.755 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.755 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.755 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.755 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.623664+0000", 2024-09-11T08:49:36.755 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.088202736000000004 2024-09-11T08:49:36.755 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.755 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.755 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.755 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.623693+0000", 2024-09-11T08:49:36.756 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9057e-05 2024-09-11T08:49:36.756 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.756 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.756 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.756 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.756 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.756 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15366.0:37669 9.7 e216/206 9:e7d030b2:::benchmark_data_smithi092_145083_object37668:head v 216'2370, mlcod=216'2367)", 2024-09-11T08:49:36.756 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:49:29.537144+0000", 2024-09-11T08:49:36.756 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 7.0928551339999997, 2024-09-11T08:49:36.756 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.088109614000000003, 2024-09-11T08:49:36.756 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.756 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.756 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.756 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.757 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.757 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537144+0000", 2024-09-11T08:49:36.757 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.757 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.757 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.757 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.757 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537144+0000", 2024-09-11T08:49:36.757 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.757 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.757 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.757 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.757 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537143+0000", 2024-09-11T08:49:36.757 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:49:36.757 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.758 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.758 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.758 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537166+0000", 2024-09-11T08:49:36.758 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.2892999999999999e-05 2024-09-11T08:49:36.758 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.758 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.758 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.758 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537178+0000", 2024-09-11T08:49:36.758 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1718e-05 2024-09-11T08:49:36.758 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.758 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.758 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.758 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537184+0000", 2024-09-11T08:49:36.758 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0279999999999997e-06 2024-09-11T08:49:36.759 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.759 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.759 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.759 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.622455+0000", 2024-09-11T08:49:36.759 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.085271277000000006 2024-09-11T08:49:36.759 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.759 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.759 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.759 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.622474+0000", 2024-09-11T08:49:36.759 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.9364e-05 2024-09-11T08:49:36.759 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.759 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.759 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.759 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.625227+0000", 2024-09-11T08:49:36.759 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0027529239999999999 2024-09-11T08:49:36.760 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.760 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.760 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.760 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.625253+0000", 2024-09-11T08:49:36.760 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.6069000000000001e-05 2024-09-11T08:49:36.760 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.760 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.760 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.760 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.760 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.760 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:37671 9.3 9:c392ba02:::benchmark_data_smithi092_145083_object37670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e216)", 2024-09-11T08:49:36.760 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:49:29.537592+0000", 2024-09-11T08:49:36.760 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 7.0924072819999999, 2024-09-11T08:49:36.760 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.091336582999999999, 2024-09-11T08:49:36.761 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.761 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.761 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.761 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:49:36.761 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:49:36.761 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 37671 2024-09-11T08:49:36.761 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.761 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.761 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.761 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.761 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537592+0000", 2024-09-11T08:49:36.761 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.762 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.762 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.762 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.762 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537592+0000", 2024-09-11T08:49:36.762 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.762 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.762 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.762 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.762 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537590+0000", 2024-09-11T08:49:36.762 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:49:36.762 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.762 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.762 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.762 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537625+0000", 2024-09-11T08:49:36.763 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5490000000000001e-05 2024-09-11T08:49:36.763 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.763 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.763 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.763 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537638+0000", 2024-09-11T08:49:36.763 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2228999999999999e-05 2024-09-11T08:49:36.763 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.763 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.763 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.763 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537643+0000", 2024-09-11T08:49:36.763 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.519e-06 2024-09-11T08:49:36.763 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.763 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.763 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.764 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537687+0000", 2024-09-11T08:49:36.764 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3516999999999997e-05 2024-09-11T08:49:36.764 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.764 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.764 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.764 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.624668+0000", 2024-09-11T08:49:36.764 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.086981679000000006 2024-09-11T08:49:36.764 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.764 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.764 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:49:36.764 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.624799+0000", 2024-09-11T08:49:36.764 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013028700000000001 2024-09-11T08:49:36.764 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.764 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.764 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.765 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.626980+0000", 2024-09-11T08:49:36.765 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0021814579999999998 2024-09-11T08:49:36.765 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.765 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.765 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.765 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.628846+0000", 2024-09-11T08:49:36.765 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001865914 2024-09-11T08:49:36.765 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.765 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.765 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.765 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.628852+0000", 2024-09-11T08:49:36.765 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7289999999999997e-06 2024-09-11T08:49:36.765 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.766 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.766 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.766 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.628893+0000", 2024-09-11T08:49:36.766 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.083e-05 2024-09-11T08:49:36.766 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.766 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.766 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.766 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.628928+0000", 2024-09-11T08:49:36.766 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5519000000000003e-05 2024-09-11T08:49:36.766 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.766 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.766 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.766 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.766 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.767 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:37672 9.e 9:711cac60:::benchmark_data_smithi092_145083_object37671:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e216)", 2024-09-11T08:49:36.767 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:49:29.537706+0000", 2024-09-11T08:49:36.767 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 7.0922931909999996, 2024-09-11T08:49:36.767 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.092047967999999994, 2024-09-11T08:49:36.767 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:49:36.767 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:49:36.767 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:49:36.767 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:49:36.767 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:49:36.767 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 37672 2024-09-11T08:49:36.767 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.767 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:49:36.767 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.768 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:49:36.768 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537706+0000", 2024-09-11T08:49:36.768 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.768 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.768 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.768 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:49:36.768 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537706+0000", 2024-09-11T08:49:36.768 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:49:36.768 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.768 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.769 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:49:36.769 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537705+0000", 2024-09-11T08:49:36.769 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:49:36.769 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.769 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.769 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:49:36.769 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.537742+0000", 2024-09-11T08:49:36.769 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7710000000000003e-05 2024-09-11T08:49:36.769 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.769 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.769 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:49:36.769 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.622003+0000", 2024-09-11T08:49:36.770 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.084260827999999996 2024-09-11T08:49:36.770 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.770 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.770 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:49:36.770 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.622018+0000", 2024-09-11T08:49:36.770 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4399999999999999e-05 2024-09-11T08:49:36.770 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.770 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.770 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:49:36.770 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.622921+0000", 2024-09-11T08:49:36.770 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00090386100000000005 2024-09-11T08:49:36.770 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.771 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.771 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:49:36.771 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.623068+0000", 2024-09-11T08:49:36.771 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014677000000000001 2024-09-11T08:49:36.771 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.771 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.771 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,4", 2024-09-11T08:49:36.771 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.623213+0000", 2024-09-11T08:49:36.771 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000145208 2024-09-11T08:49:36.771 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.771 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.771 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:49:36.771 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.629515+0000", 2024-09-11T08:49:36.771 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0063016979999999997 2024-09-11T08:49:36.772 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.772 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.772 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.772 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.629603+0000", 2024-09-11T08:49:36.772 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8279e-05 2024-09-11T08:49:36.772 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.772 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.772 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:49:36.772 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.629672+0000", 2024-09-11T08:49:36.772 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8238000000000006e-05 2024-09-11T08:49:36.772 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.772 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.772 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:49:36.773 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.629722+0000", 2024-09-11T08:49:36.773 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0081999999999999e-05 2024-09-11T08:49:36.773 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:49:36.773 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:49:36.773 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:49:36.773 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:49:29.629754+0000", 2024-09-11T08:49:36.773 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1909000000000002e-05 2024-09-11T08:49:36.773 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.773 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.773 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.773 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:49:36.773 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:49:36.773 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:49:36.774 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 22 16 50799 50783 144.257 107.312 0.00944319 0.00692556 2024-09-11T08:49:36.817 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:36.853 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:36.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:36.854 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:36.912 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:49:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:49:36.954 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:36.989 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:36.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:36.990 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:37.087 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:49:37.091 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:37.124 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:37.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:37.126 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:37.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:37.265 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:37.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:37.266 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:37.366 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:37.403 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:37.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:37.404 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:37.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:37 smithi145 ceph-mon[33384]: 5.e scrub starts 2024-09-11T08:49:37.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:37 smithi145 ceph-mon[33384]: 5.e scrub ok 2024-09-11T08:49:37.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:37 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1808046866' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:49:37.505 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:37.539 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:37.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:37.540 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:37.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:37 smithi092 ceph-mon[29647]: 5.e scrub starts 2024-09-11T08:49:37.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:37 smithi092 ceph-mon[29647]: 5.e scrub ok 2024-09-11T08:49:37.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:37 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1808046866' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:49:37.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:37 smithi092 ceph-mon[41114]: 5.e scrub starts 2024-09-11T08:49:37.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:37 smithi092 ceph-mon[41114]: 5.e scrub ok 2024-09-11T08:49:37.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:37 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1808046866' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:49:37.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:37.646 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-11T08:49:37.675 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:37.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:37.676 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:37.722 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 23 16 52593 52577 142.854 112.125 0.0116466 0.00699555 2024-09-11T08:49:37.777 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:37.812 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:37.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:37.813 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:37.914 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:37.947 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:37.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:37.948 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:38.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:38.091 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:38.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:38.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:38.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:38.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:38.238 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:38.239 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:38.340 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:38.379 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:38.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:38.380 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:38.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:38 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:38.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:38 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:49:38.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:38 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1808046866' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:49:38.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:38 smithi145 ceph-mon[33384]: osdmap e218: 8 total, 8 up, 7 in 2024-09-11T08:49:38.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:38 smithi145 ceph-mon[33384]: pgmap v463: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 146 MiB/s wr, 2.34k op/s 2024-09-11T08:49:38.481 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:38.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:38.521 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:38.521 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:38 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:38 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:49:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:38 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1808046866' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:49:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:38 smithi092 ceph-mon[29647]: osdmap e218: 8 total, 8 up, 7 in 2024-09-11T08:49:38.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:38 smithi092 ceph-mon[29647]: pgmap v463: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 146 MiB/s wr, 2.34k op/s 2024-09-11T08:49:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:38 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:38 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:49:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:38 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1808046866' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:49:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:38 smithi092 ceph-mon[41114]: osdmap e218: 8 total, 8 up, 7 in 2024-09-11T08:49:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:38 smithi092 ceph-mon[41114]: pgmap v463: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 146 MiB/s wr, 2.34k op/s 2024-09-11T08:49:38.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:38.656 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:38.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:38.657 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:38.724 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 24 16 54348 54332 141.472 109.688 0.0114352 0.0070643 2024-09-11T08:49:38.738 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:38.759 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:38.802 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:38.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:38.803 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:38.904 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:38.940 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:38.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:38.940 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:39.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:39.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:39.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:39.183 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:39.221 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:39.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:39.222 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:39.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:39.352 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:39.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:39.353 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:39.454 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:39.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:39.487 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:39.488 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:39.576 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:49:39.577 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:49:39.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:39.609 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:49:39.609 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:49:39.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:39.623 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:39.623 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:39.638 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:49:39.638 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:49:39.672 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:49:39.672 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:49:39.705 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:49:39.705 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:49:39.722 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 25 16 56016 56000 139.983 104.25 0.0104117 0.00713893 2024-09-11T08:49:39.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:39.737 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:49:39.737 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:49:39.760 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:39.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:39.761 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:39.768 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:49:39.768 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:49:39.798 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:49:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:49:39.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:39.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:39.896 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:39.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:40.031 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:40.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:40.032 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:40.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:40.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:39 smithi145 ceph-mon[33384]: osdmap e219: 8 total, 8 up, 7 in 2024-09-11T08:49:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:39 smithi145 ceph-mon[33384]: pgmap v465: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 853 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-11T08:49:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:39 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1603617215' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:40.165 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:40.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:40.166 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:40.267 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:40.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:40.292 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:40.293 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:39 smithi092 ceph-mon[29647]: osdmap e219: 8 total, 8 up, 7 in 2024-09-11T08:49:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:39 smithi092 ceph-mon[29647]: pgmap v465: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 853 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-11T08:49:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:39 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1603617215' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:40.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:39 smithi092 ceph-mon[41114]: osdmap e219: 8 total, 8 up, 7 in 2024-09-11T08:49:40.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:39 smithi092 ceph-mon[41114]: pgmap v465: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 853 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-11T08:49:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:39 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1603617215' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:40.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:40.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:40.426 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:40.427 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:40.528 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:40.565 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:40.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:40.566 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:40.667 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:40.702 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:40.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:40.703 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:40.722 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 26 16 57658 57642 138.545 102.625 0.00827956 0.00721333 2024-09-11T08:49:40.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:49:40.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:40.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:40.842 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:40.842 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:40.864 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:49:40.864 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:49:40.864 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_2 2024-09-11T08:49:40.864 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:49:40.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:40.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:40.979 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:40.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:41.080 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:41.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:41.117 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:41.117 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:41.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:41.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:41.254 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:41.255 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:41.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:41.398 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:41.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:41.399 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:41.500 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:41.529 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:41.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:41.530 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:41.530 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:41.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:41.668 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:41.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:41.669 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:41.723 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 27 16 59169 59153 136.912 94.4375 0.00404917 0.00729892 2024-09-11T08:49:41.770 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:41.774 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:49:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:49:41] "GET /metrics HTTP/1.1" 200 224367 "" "Prometheus/2.43.0" 2024-09-11T08:49:41.813 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:41.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:41.814 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:41.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:41.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:41.947 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:41.947 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:42.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:42.087 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:42.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:42.088 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:42.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:42.223 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:42.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:42.224 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:42.325 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:42.356 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:42.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:42.357 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:42.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:42.494 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:42.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:42.495 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:42.535 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:42 smithi092 ceph-mon[29647]: pgmap v466: 148 pgs: 148 active+clean; 3.2 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.4 KiB/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-11T08:49:42.536 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:42 smithi092 ceph-mon[41114]: pgmap v466: 148 pgs: 148 active+clean; 3.2 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.4 KiB/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-11T08:49:42.596 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:42.631 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:42.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:42.632 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:42 smithi145 ceph-mon[33384]: pgmap v466: 148 pgs: 148 active+clean; 3.2 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.4 KiB/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-11T08:49:42.724 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 28 16 60620 60604 135.258 90.6875 0.00412775 0.0073883 2024-09-11T08:49:42.733 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:42.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:42.773 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:42.773 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:42.874 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:42.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:42.910 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:42.910 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:43.011 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:43.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:43.045 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:43.046 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:43.146 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:43.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:49:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:49:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:49:43.190 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:43.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:43.192 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:43.293 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:43.329 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:43.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:43.330 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:43.431 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:43.466 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:43.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:43.467 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:43.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:49:43.568 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:43.600 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:43.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:43.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:43.664 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:43.702 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:43.709 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:49:43.709 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:49:43.724 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 29 16 62156 62140 133.904 96 0.00943165 0.00746199 2024-09-11T08:49:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:43.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:43.753 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:43.854 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:43.890 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:43.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:43.891 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:43.992 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:44.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:44.028 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:44.028 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:44.129 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:44.164 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:44.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:44.165 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:44.266 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:44.303 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:44.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:44.304 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:44.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:44.440 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:44.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:44.441 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:44.542 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:44.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:44 smithi092 ceph-mon[29647]: pgmap v467: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 137 MiB/s wr, 2.19k op/s 2024-09-11T08:49:44.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:44 smithi092 ceph-mon[29647]: osdmap e220: 8 total, 8 up, 7 in 2024-09-11T08:49:44.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:44 smithi092 ceph-mon[41114]: pgmap v467: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 137 MiB/s wr, 2.19k op/s 2024-09-11T08:49:44.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:44 smithi092 ceph-mon[41114]: osdmap e220: 8 total, 8 up, 7 in 2024-09-11T08:49:44.582 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:44.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:44.583 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:44 smithi145 ceph-mon[33384]: pgmap v467: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 137 MiB/s wr, 2.19k op/s 2024-09-11T08:49:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:44 smithi145 ceph-mon[33384]: osdmap e220: 8 total, 8 up, 7 in 2024-09-11T08:49:44.683 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:44.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:44.716 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:44.716 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:44.723 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 30 16 63764 63748 132.79 100.5 0.00489794 0.00752603 2024-09-11T08:49:44.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:44.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:44.857 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:44.857 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:44.958 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:44.994 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:44.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:44.995 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:45.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:45.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:45.141 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:45.141 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:45.242 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:45.274 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:45.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:45.275 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:45.376 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:45.416 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:45.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:45.417 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:45.518 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:45.557 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:45.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:45.572 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:45.673 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:45.707 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:45.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:45.708 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:45.723 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 31 16 65357 65341 131.718 99.5625 0.00767702 0.0075872 2024-09-11T08:49:45.809 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:45.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:45.842 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:45.843 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:45.944 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:45.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-11T08:49:45.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:45.979 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:45.979 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:46.080 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:46.112 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:46.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:46.113 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:46.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:46.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:46.254 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:46.254 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:46.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:46 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:46 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:46 smithi092 ceph-mon[29647]: pgmap v469: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-11T08:49:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:46 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:46 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:46 smithi092 ceph-mon[41114]: pgmap v469: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-11T08:49:46.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:46.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:46.398 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:46.399 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:46.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:46 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:49:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:46 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:49:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:46 smithi145 ceph-mon[33384]: pgmap v469: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-11T08:49:46.499 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:46.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:46.536 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:46.536 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:46.571 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:49:46.575 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:49:46.636 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:46.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:46.675 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:46.675 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:46.724 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 32 16 66822 66806 130.463 91.5625 0.00558368 0.00766003 2024-09-11T08:49:46.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:46.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:46.824 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:46.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:46.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:46.965 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:46.965 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:47.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:47.096 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:49:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:49:47.100 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:47.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:47.101 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:47.202 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:47.233 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:47.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:47.234 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:47.335 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:47.365 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:47.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:47.366 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:47.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:47 smithi145 ceph-mon[33384]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:49:47.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:47.499 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:47.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:47.500 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:47.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:47 smithi092 ceph-mon[29647]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:49:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:47 smithi092 ceph-mon[41114]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:49:47.601 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:47.634 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:47.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:47.635 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:47.724 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 33 16 68339 68323 129.382 94.8125 0.00386778 0.0077241 2024-09-11T08:49:47.735 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:47.769 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:47.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:47.770 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:47.870 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:47.891 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:47.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:47.892 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:47.993 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:48.027 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:48.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:48.028 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:48.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:48.163 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:48.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:48.164 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:48.265 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:48.301 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:48.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:48.302 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:48.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:48.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:48 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:48.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:48 smithi145 ceph-mon[33384]: pgmap v470: 148 pgs: 148 active+clean; 4.1 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 120 MiB/s wr, 1.92k op/s 2024-09-11T08:49:48.435 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:48.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:48.436 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:48.460 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:48 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:48.460 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:48 smithi092 ceph-mon[29647]: pgmap v470: 148 pgs: 148 active+clean; 4.1 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 120 MiB/s wr, 1.92k op/s 2024-09-11T08:49:48.461 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:48 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:48.461 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:48 smithi092 ceph-mon[41114]: pgmap v470: 148 pgs: 148 active+clean; 4.1 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 120 MiB/s wr, 1.92k op/s 2024-09-11T08:49:48.498 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-11T08:49:45.232180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3848018140,"num_objects":58903,"num_object_clones":0,"num_object_copies":176709,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58903,"num_whiteouts":0,"num_read":1921,"num_read_kb":1778,"num_write":59536,"num_write_kb":3758919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":1184877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59623,"ondisk_log_size":59623,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":445,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":13349516,"kb_used_data":11226436,"kb_used_omap":11,"kb_used_meta":2122996,"kb_avail":642866548,"statfs":{"total":671965249536,"available":658295345152,"internally_reserved":0,"allocated":11495870464,"data_stored":11493299273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2173948264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,4,12,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":549912576,"num_objects":8391,"num_object_clones":0,"num_object_copies":25173,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8391,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":8391,"num_write_kb":537024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003426"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.088229+0000","last_change":"2024-09-11T08:49:09.906451+0000","last_active":"2024-09-11T08:49:37.088229+0000","last_peered":"2024-09-11T08:49:37.088229+0000","last_clean":"2024-09-11T08:49:37.088229+0000","last_became_active":"2024-09-11T08:49:09.905661+0000","last_became_peered":"2024-09-11T08:49:09.905661+0000","last_unstale":"2024-09-11T08:49:37.088229+0000","last_undegraded":"2024-09-11T08:49:37.088229+0000","last_fullsized":"2024-09-11T08:49:37.088229+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:47.838700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:47.838700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:53.194173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"200'19","reported_seq":279,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094054+0000","last_change":"2024-09-11T08:49:34.732293+0000","last_active":"2024-09-11T08:49:37.094054+0000","last_peered":"2024-09-11T08:49:37.094054+0000","last_clean":"2024-09-11T08:49:37.094054+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:49:37.094054+0000","last_undegraded":"2024-09-11T08:49:37.094054+0000","last_fullsized":"2024-09-11T08:49:37.094054+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:49:34.732198+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:49:34.732198+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:49.013371+0000","scrub_duration":0.0076844679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":217,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087528+0000","last_change":"2024-09-11T08:47:44.884292+0000","last_active":"2024-09-11T08:49:37.087528+0000","last_peered":"2024-09-11T08:49:37.087528+0000","last_clean":"2024-09-11T08:49:37.087528+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:49:37.087528+0000","last_undegraded":"2024-09-11T08:49:37.087528+0000","last_fullsized":"2024-09-11T08:49:37.087528+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.884217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.884217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011480829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":196,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094328+0000","last_change":"2024-09-11T08:49:23.674071+0000","last_active":"2024-09-11T08:49:37.094328+0000","last_peered":"2024-09-11T08:49:37.094328+0000","last_clean":"2024-09-11T08:49:37.094328+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:49:37.094328+0000","last_undegraded":"2024-09-11T08:49:37.094328+0000","last_fullsized":"2024-09-11T08:49:37.094328+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:23.673975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:23.673975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:34.389310+0000","scrub_duration":0.081616116000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":225,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090781+0000","last_change":"2024-09-11T08:48:34.059380+0000","last_active":"2024-09-11T08:49:37.090781+0000","last_peered":"2024-09-11T08:49:37.090781+0000","last_clean":"2024-09-11T08:49:37.090781+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:49:37.090781+0000","last_undegraded":"2024-09-11T08:49:37.090781+0000","last_fullsized":"2024-09-11T08:49:37.090781+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:34.059309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:34.059309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:59.700862+0000","scrub_duration":0.0067175359999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":231,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.093558+0000","last_change":"2024-09-11T08:48:35.046165+0000","last_active":"2024-09-11T08:49:37.093558+0000","last_peered":"2024-09-11T08:49:37.093558+0000","last_clean":"2024-09-11T08:49:37.093558+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:49:37.093558+0000","last_undegraded":"2024-09-11T08:49:37.093558+0000","last_fullsized":"2024-09-11T08:49:37.093558+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:35.046075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:35.046075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:46.499902+0000","scrub_duration":0.010511866,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"200'34","reported_seq":285,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091295+0000","last_change":"2024-09-11T08:49:09.927417+0000","last_active":"2024-09-11T08:49:37.091295+0000","last_peered":"2024-09-11T08:49:37.091295+0000","last_clean":"2024-09-11T08:49:37.091295+0000","last_became_active":"2024-09-11T08:49:09.909107+0000","last_became_peered":"2024-09-11T08:49:09.909107+0000","last_unstale":"2024-09-11T08:49:37.091295+0000","last_undegraded":"2024-09-11T08:49:37.091295+0000","last_fullsized":"2024-09-11T08:49:37.091295+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:48:30.029817+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.029817+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:55.427664+0000","scrub_duration":0.0055582840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":198,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.093879+0000","last_change":"2024-09-11T08:48:32.827213+0000","last_active":"2024-09-11T08:49:37.093879+0000","last_peered":"2024-09-11T08:49:37.093879+0000","last_clean":"2024-09-11T08:49:37.093879+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:49:37.093879+0000","last_undegraded":"2024-09-11T08:49:37.093879+0000","last_fullsized":"2024-09-11T08:49:37.093879+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:32.827110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:48:32.827110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029807079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":237,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.088253+0000","last_change":"2024-09-11T08:48:41.415476+0000","last_active":"2024-09-11T08:49:37.088253+0000","last_peered":"2024-09-11T08:49:37.088253+0000","last_clean":"2024-09-11T08:49:37.088253+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:49:37.088253+0000","last_undegraded":"2024-09-11T08:49:37.088253+0000","last_fullsized":"2024-09-11T08:49:37.088253+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:41.415403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.415403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:46.015177+0000","scrub_duration":0.0017214509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"200'19","reported_seq":259,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094581+0000","last_change":"2024-09-11T08:49:19.682800+0000","last_active":"2024-09-11T08:49:37.094581+0000","last_peered":"2024-09-11T08:49:37.094581+0000","last_clean":"2024-09-11T08:49:37.094581+0000","last_became_active":"2024-09-11T08:49:09.910441+0000","last_became_peered":"2024-09-11T08:49:09.910441+0000","last_unstale":"2024-09-11T08:49:37.094581+0000","last_undegraded":"2024-09-11T08:49:37.094581+0000","last_fullsized":"2024-09-11T08:49:37.094581+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:49:19.682694+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:49:19.682694+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:29.537157+0000","scrub_duration":0.0024238570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":223,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087222+0000","last_change":"2024-09-11T08:49:09.900683+0000","last_active":"2024-09-11T08:49:37.087222+0000","last_peered":"2024-09-11T08:49:37.087222+0000","last_clean":"2024-09-11T08:49:37.087222+0000","last_became_active":"2024-09-11T08:49:09.899188+0000","last_became_peered":"2024-09-11T08:49:09.899188+0000","last_unstale":"2024-09-11T08:49:37.087222+0000","last_undegraded":"2024-09-11T08:49:37.087222+0000","last_fullsized":"2024-09-11T08:49:37.087222+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:18.341709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.341709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038165500000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":31,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087714+0000","last_change":"2024-09-11T08:49:33.520537+0000","last_active":"2024-09-11T08:49:37.087714+0000","last_peered":"2024-09-11T08:49:37.087714+0000","last_clean":"2024-09-11T08:49:37.087714+0000","last_became_active":"2024-09-11T08:49:09.907856+0000","last_became_peered":"2024-09-11T08:49:09.907856+0000","last_unstale":"2024-09-11T08:49:37.087714+0000","last_undegraded":"2024-09-11T08:49:37.087714+0000","last_fullsized":"2024-09-11T08:49:37.087714+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:33.520462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:33.520462+0000","last_clean_scrub_stamp":"2024-09-11T08:49:33.520462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:52.380831+0000","scrub_duration":0.0043744099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":217,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.086334+0000","last_change":"2024-09-11T08:48:44.398447+0000","last_active":"2024-09-11T08:49:37.086334+0000","last_peered":"2024-09-11T08:49:37.086334+0000","last_clean":"2024-09-11T08:49:37.086334+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:49:37.086334+0000","last_undegraded":"2024-09-11T08:49:37.086334+0000","last_fullsized":"2024-09-11T08:49:37.086334+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:44.398373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.398373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:53.243691+0000","scrub_duration":0.0030847359999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"200'7","reported_seq":213,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094248+0000","last_change":"2024-09-11T08:47:44.866144+0000","last_active":"2024-09-11T08:49:37.094248+0000","last_peered":"2024-09-11T08:49:37.094248+0000","last_clean":"2024-09-11T08:49:37.094248+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:49:37.094248+0000","last_undegraded":"2024-09-11T08:49:37.094248+0000","last_fullsized":"2024-09-11T08:49:37.094248+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:47:44.866057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.866057+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020690169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":217,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091212+0000","last_change":"2024-09-11T08:47:46.410478+0000","last_active":"2024-09-11T08:49:37.091212+0000","last_peered":"2024-09-11T08:49:37.091212+0000","last_clean":"2024-09-11T08:49:37.091212+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:49:37.091212+0000","last_undegraded":"2024-09-11T08:49:37.091212+0000","last_fullsized":"2024-09-11T08:49:37.091212+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:46.410402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:47:46.410402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002594884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":211,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.086264+0000","last_change":"2024-09-11T08:49:28.283272+0000","last_active":"2024-09-11T08:49:37.086264+0000","last_peered":"2024-09-11T08:49:37.086264+0000","last_clean":"2024-09-11T08:49:37.086264+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:49:37.086264+0000","last_undegraded":"2024-09-11T08:49:37.086264+0000","last_fullsized":"2024-09-11T08:49:37.086264+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:28.283167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:49:28.283167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:28.913788+0000","scrub_duration":0.0040125320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":216,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.096338+0000","last_change":"2024-09-11T08:46:01.978488+0000","last_active":"2024-09-11T08:49:37.096338+0000","last_peered":"2024-09-11T08:49:37.096338+0000","last_clean":"2024-09-11T08:49:37.096338+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:49:37.096338+0000","last_undegraded":"2024-09-11T08:49:37.096338+0000","last_fullsized":"2024-09-11T08:49:37.096338+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:24.197149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.197149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001510267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094483+0000","last_change":"2024-09-11T08:49:09.908672+0000","last_active":"2024-09-11T08:49:37.094483+0000","last_peered":"2024-09-11T08:49:37.094483+0000","last_clean":"2024-09-11T08:49:37.094483+0000","last_became_active":"2024-09-11T08:49:09.908237+0000","last_became_peered":"2024-09-11T08:49:09.908237+0000","last_unstale":"2024-09-11T08:49:37.094483+0000","last_undegraded":"2024-09-11T08:49:37.094483+0000","last_fullsized":"2024-09-11T08:49:37.094483+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:34.610331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:34.610331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"200'34","reported_seq":285,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087963+0000","last_change":"2024-09-11T08:48:20.311201+0000","last_active":"2024-09-11T08:49:37.087963+0000","last_peered":"2024-09-11T08:49:37.087963+0000","last_clean":"2024-09-11T08:49:37.087963+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:49:37.087963+0000","last_undegraded":"2024-09-11T08:49:37.087963+0000","last_fullsized":"2024-09-11T08:49:37.087963+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:48:20.311122+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.311122+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011484708999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":227,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091713+0000","last_change":"2024-09-11T08:49:09.903399+0000","last_active":"2024-09-11T08:49:37.091713+0000","last_peered":"2024-09-11T08:49:37.091713+0000","last_clean":"2024-09-11T08:49:37.091713+0000","last_became_active":"2024-09-11T08:49:09.903078+0000","last_became_peered":"2024-09-11T08:49:09.903078+0000","last_unstale":"2024-09-11T08:49:37.091713+0000","last_undegraded":"2024-09-11T08:49:37.091713+0000","last_fullsized":"2024-09-11T08:49:37.091713+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:19.961460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.961460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:47.571235+0000","scrub_duration":0.0091369829999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":224,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.093499+0000","last_change":"2024-09-11T08:46:01.998163+0000","last_active":"2024-09-11T08:49:37.093499+0000","last_peered":"2024-09-11T08:49:37.093499+0000","last_clean":"2024-09-11T08:49:37.093499+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:49:37.093499+0000","last_undegraded":"2024-09-11T08:49:37.093499+0000","last_fullsized":"2024-09-11T08:49:37.093499+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:24.223237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.223237+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019917720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":202,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094698+0000","last_change":"2024-09-11T08:45:28.194198+0000","last_active":"2024-09-11T08:49:37.094698+0000","last_peered":"2024-09-11T08:49:37.094698+0000","last_clean":"2024-09-11T08:49:37.094698+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:49:37.094698+0000","last_undegraded":"2024-09-11T08:49:37.094698+0000","last_fullsized":"2024-09-11T08:49:37.094698+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.194095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.194095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001559936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"200'18","reported_seq":263,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.093722+0000","last_change":"2024-09-11T08:49:26.388924+0000","last_active":"2024-09-11T08:49:37.093722+0000","last_peered":"2024-09-11T08:49:37.093722+0000","last_clean":"2024-09-11T08:49:37.093722+0000","last_became_active":"2024-09-11T08:49:16.943886+0000","last_became_peered":"2024-09-11T08:49:16.943886+0000","last_unstale":"2024-09-11T08:49:37.093722+0000","last_undegraded":"2024-09-11T08:49:37.093722+0000","last_fullsized":"2024-09-11T08:49:37.093722+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'18","last_scrub_stamp":"2024-09-11T08:49:26.388870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:49:26.388870+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:53.627087+0000","scrub_duration":0.001482014,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":198,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094992+0000","last_change":"2024-09-11T08:45:22.244097+0000","last_active":"2024-09-11T08:49:37.094992+0000","last_peered":"2024-09-11T08:49:37.094992+0000","last_clean":"2024-09-11T08:49:37.094992+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:49:37.094992+0000","last_undegraded":"2024-09-11T08:49:37.094992+0000","last_fullsized":"2024-09-11T08:49:37.094992+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.244038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.244038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015252620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":206,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091342+0000","last_change":"2024-09-11T08:45:19.335163+0000","last_active":"2024-09-11T08:49:37.091342+0000","last_peered":"2024-09-11T08:49:37.091342+0000","last_clean":"2024-09-11T08:49:37.091342+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:49:37.091342+0000","last_undegraded":"2024-09-11T08:49:37.091342+0000","last_fullsized":"2024-09-11T08:49:37.091342+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:19.335065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:19.335065+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026052929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":221,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087656+0000","last_change":"2024-09-11T08:48:25.971280+0000","last_active":"2024-09-11T08:49:37.087656+0000","last_peered":"2024-09-11T08:49:37.087656+0000","last_clean":"2024-09-11T08:49:37.087656+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:49:37.087656+0000","last_undegraded":"2024-09-11T08:49:37.087656+0000","last_fullsized":"2024-09-11T08:49:37.087656+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:25.971151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.971151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:47.552719+0000","scrub_duration":0.0095705709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"200'20","reported_seq":255,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090494+0000","last_change":"2024-09-11T08:49:09.936134+0000","last_active":"2024-09-11T08:49:37.090494+0000","last_peered":"2024-09-11T08:49:37.090494+0000","last_clean":"2024-09-11T08:49:37.090494+0000","last_became_active":"2024-09-11T08:49:09.913968+0000","last_became_peered":"2024-09-11T08:49:09.913968+0000","last_unstale":"2024-09-11T08:49:37.090494+0000","last_undegraded":"2024-09-11T08:49:37.090494+0000","last_fullsized":"2024-09-11T08:49:37.090494+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:48:19.705185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.705185+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050623580000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":360,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.802407+0000","last_change":"2024-09-11T08:48:17.812378+0000","last_active":"2024-09-11T08:49:37.802407+0000","last_peered":"2024-09-11T08:49:37.802407+0000","last_clean":"2024-09-11T08:49:37.802407+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:49:37.802407+0000","last_undegraded":"2024-09-11T08:49:37.802407+0000","last_fullsized":"2024-09-11T08:49:37.802407+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:48:17.812256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:17.812256+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:40.308022+0000","scrub_duration":0.010983438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"219'3656","reported_seq":3675,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.779055+0000","last_change":"2024-09-11T08:49:06.519462+0000","last_active":"2024-09-11T08:49:41.779055+0000","last_peered":"2024-09-11T08:49:41.779055+0000","last_clean":"2024-09-11T08:49:41.779055+0000","last_became_active":"2024-09-11T08:49:06.519147+0000","last_became_peered":"2024-09-11T08:49:06.519147+0000","last_unstale":"2024-09-11T08:49:41.779055+0000","last_undegraded":"2024-09-11T08:49:41.779055+0000","last_fullsized":"2024-09-11T08:49:41.779055+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3656,"log_dups_size":0,"ondisk_log_size":3656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:14.340457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239599616,"num_objects":3656,"num_object_clones":0,"num_object_copies":10968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3656,"num_write_kb":233984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"200'34","reported_seq":259,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090909+0000","last_change":"2024-09-11T08:45:18.777528+0000","last_active":"2024-09-11T08:49:37.090909+0000","last_peered":"2024-09-11T08:49:37.090909+0000","last_clean":"2024-09-11T08:49:37.090909+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:49:37.090909+0000","last_undegraded":"2024-09-11T08:49:37.090909+0000","last_fullsized":"2024-09-11T08:49:37.090909+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:45:18.777466+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.777466+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045440580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":211,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091623+0000","last_change":"2024-09-11T08:49:25.383955+0000","last_active":"2024-09-11T08:49:37.091623+0000","last_peered":"2024-09-11T08:49:37.091623+0000","last_clean":"2024-09-11T08:49:37.091623+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:49:37.091623+0000","last_undegraded":"2024-09-11T08:49:37.091623+0000","last_fullsized":"2024-09-11T08:49:37.091623+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:25.383554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:25.383554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:46.160564+0000","scrub_duration":0.0084378879999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"211'10","reported_seq":363,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.803152+0000","last_change":"2024-09-11T08:49:35.694375+0000","last_active":"2024-09-11T08:49:37.803152+0000","last_peered":"2024-09-11T08:49:37.803152+0000","last_clean":"2024-09-11T08:49:37.803152+0000","last_became_active":"2024-09-11T08:49:09.905262+0000","last_became_peered":"2024-09-11T08:49:09.905262+0000","last_unstale":"2024-09-11T08:49:37.803152+0000","last_undegraded":"2024-09-11T08:49:37.803152+0000","last_fullsized":"2024-09-11T08:49:37.803152+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'10","last_scrub_stamp":"2024-09-11T08:49:35.694282+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:49:35.694282+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:45.457363+0000","scrub_duration":0.0051461679999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":218,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087856+0000","last_change":"2024-09-11T08:49:20.335925+0000","last_active":"2024-09-11T08:49:37.087856+0000","last_peered":"2024-09-11T08:49:37.087856+0000","last_clean":"2024-09-11T08:49:37.087856+0000","last_became_active":"2024-09-11T08:49:09.906872+0000","last_became_peered":"2024-09-11T08:49:09.906872+0000","last_unstale":"2024-09-11T08:49:37.087856+0000","last_undegraded":"2024-09-11T08:49:37.087856+0000","last_fullsized":"2024-09-11T08:49:37.087856+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:20.335841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:20.335841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:20.443355+0000","scrub_duration":0.0041148479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"219'3713","reported_seq":3734,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.173409+0000","last_change":"2024-09-11T08:49:05.875564+0000","last_active":"2024-09-11T08:49:41.173409+0000","last_peered":"2024-09-11T08:49:41.173409+0000","last_clean":"2024-09-11T08:49:41.173409+0000","last_became_active":"2024-09-11T08:49:05.875264+0000","last_became_peered":"2024-09-11T08:49:05.875264+0000","last_unstale":"2024-09-11T08:49:41.173409+0000","last_undegraded":"2024-09-11T08:49:41.173409+0000","last_fullsized":"2024-09-11T08:49:41.173409+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3713,"log_dups_size":0,"ondisk_log_size":3713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:10.708155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243335168,"num_objects":3713,"num_object_clones":0,"num_object_copies":11139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3713,"num_write_kb":237632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"200'38","reported_seq":295,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091481+0000","last_change":"2024-09-11T08:49:09.936915+0000","last_active":"2024-09-11T08:49:37.091481+0000","last_peered":"2024-09-11T08:49:37.091481+0000","last_clean":"2024-09-11T08:49:37.091481+0000","last_became_active":"2024-09-11T08:49:09.917354+0000","last_became_peered":"2024-09-11T08:49:09.917354+0000","last_unstale":"2024-09-11T08:49:37.091481+0000","last_undegraded":"2024-09-11T08:49:37.091481+0000","last_fullsized":"2024-09-11T08:49:37.091481+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'26","last_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_deep_scrub":"199'26","last_deep_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.093994+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:10.328025+0000","scrub_duration":0.026938573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":221,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.095191+0000","last_change":"2024-09-11T08:49:09.915472+0000","last_active":"2024-09-11T08:49:37.095191+0000","last_peered":"2024-09-11T08:49:37.095191+0000","last_clean":"2024-09-11T08:49:37.095191+0000","last_became_active":"2024-09-11T08:49:09.915191+0000","last_became_peered":"2024-09-11T08:49:09.915191+0000","last_unstale":"2024-09-11T08:49:37.095191+0000","last_undegraded":"2024-09-11T08:49:37.095191+0000","last_fullsized":"2024-09-11T08:49:37.095191+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:38.041303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:38.041303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:41.529572+0000","scrub_duration":0.0073485030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":208,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094900+0000","last_change":"2024-09-11T08:49:29.687666+0000","last_active":"2024-09-11T08:49:37.094900+0000","last_peered":"2024-09-11T08:49:37.094900+0000","last_clean":"2024-09-11T08:49:37.094900+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:49:37.094900+0000","last_undegraded":"2024-09-11T08:49:37.094900+0000","last_fullsized":"2024-09-11T08:49:37.094900+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:29.687577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:29.687577+0000","last_clean_scrub_stamp":"2024-09-11T08:49:29.687577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:55.182660+0000","scrub_duration":0.0082090140000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":196,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094506+0000","last_change":"2024-09-11T08:46:03.144149+0000","last_active":"2024-09-11T08:49:37.094506+0000","last_peered":"2024-09-11T08:49:37.094506+0000","last_clean":"2024-09-11T08:49:37.094506+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:49:37.094506+0000","last_undegraded":"2024-09-11T08:49:37.094506+0000","last_fullsized":"2024-09-11T08:49:37.094506+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:46:03.144096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:46:03.144096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001497151,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"219'3727","reported_seq":3748,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.788899+0000","last_change":"2024-09-11T08:49:05.874640+0000","last_active":"2024-09-11T08:49:41.788899+0000","last_peered":"2024-09-11T08:49:41.788899+0000","last_clean":"2024-09-11T08:49:41.788899+0000","last_became_active":"2024-09-11T08:49:05.874365+0000","last_became_peered":"2024-09-11T08:49:05.874365+0000","last_unstale":"2024-09-11T08:49:41.788899+0000","last_undegraded":"2024-09-11T08:49:41.788899+0000","last_fullsized":"2024-09-11T08:49:41.788899+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3727,"log_dups_size":0,"ondisk_log_size":3727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:32.415086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":244252672,"num_objects":3727,"num_object_clones":0,"num_object_copies":11181,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3727,"num_write_kb":238528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"200'35","reported_seq":249,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.095240+0000","last_change":"2024-09-11T08:48:23.862534+0000","last_active":"2024-09-11T08:49:37.095240+0000","last_peered":"2024-09-11T08:49:37.095240+0000","last_clean":"2024-09-11T08:49:37.095240+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:49:37.095240+0000","last_undegraded":"2024-09-11T08:49:37.095240+0000","last_fullsized":"2024-09-11T08:49:37.095240+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'21","last_scrub_stamp":"2024-09-11T08:48:23.862482+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:48:23.862482+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001795776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.088297+0000","last_change":"2024-09-11T08:49:24.594111+0000","last_active":"2024-09-11T08:49:37.088297+0000","last_peered":"2024-09-11T08:49:37.088297+0000","last_clean":"2024-09-11T08:49:37.088297+0000","last_became_active":"2024-09-11T08:49:09.908041+0000","last_became_peered":"2024-09-11T08:49:09.908041+0000","last_unstale":"2024-09-11T08:49:37.088297+0000","last_undegraded":"2024-09-11T08:49:37.088297+0000","last_fullsized":"2024-09-11T08:49:37.088297+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:24.593902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:24.593902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:39.993524+0000","scrub_duration":0.023705506000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":340,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.802640+0000","last_change":"2024-09-11T08:48:36.052650+0000","last_active":"2024-09-11T08:49:37.802640+0000","last_peered":"2024-09-11T08:49:37.802640+0000","last_clean":"2024-09-11T08:49:37.802640+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:49:37.802640+0000","last_undegraded":"2024-09-11T08:49:37.802640+0000","last_fullsized":"2024-09-11T08:49:37.802640+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:48:36.052514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:36.052514+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:47.227630+0000","scrub_duration":0.015857353000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":235,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.089779+0000","last_change":"2024-09-11T08:49:31.758729+0000","last_active":"2024-09-11T08:49:37.089779+0000","last_peered":"2024-09-11T08:49:37.089779+0000","last_clean":"2024-09-11T08:49:37.089779+0000","last_became_active":"2024-09-11T08:46:01.983335+0000","last_became_peered":"2024-09-11T08:46:01.983335+0000","last_unstale":"2024-09-11T08:49:37.089779+0000","last_undegraded":"2024-09-11T08:49:37.089779+0000","last_fullsized":"2024-09-11T08:49:37.089779+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:31.758573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:31.758573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:36.177031+0000","scrub_duration":0.0084881980000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"219'3620","reported_seq":3640,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.310022+0000","last_change":"2024-09-11T08:49:05.871535+0000","last_active":"2024-09-11T08:49:41.310022+0000","last_peered":"2024-09-11T08:49:41.310022+0000","last_clean":"2024-09-11T08:49:41.310022+0000","last_became_active":"2024-09-11T08:49:05.871255+0000","last_became_peered":"2024-09-11T08:49:05.871255+0000","last_unstale":"2024-09-11T08:49:41.310022+0000","last_undegraded":"2024-09-11T08:49:41.310022+0000","last_fullsized":"2024-09-11T08:49:41.310022+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3620,"log_dups_size":0,"ondisk_log_size":3620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:09.000693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237240320,"num_objects":3620,"num_object_clones":0,"num_object_copies":10860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3620,"num_write_kb":231680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"200'25","reported_seq":254,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.089698+0000","last_change":"2024-09-11T08:48:38.032124+0000","last_active":"2024-09-11T08:49:37.089698+0000","last_peered":"2024-09-11T08:49:37.089698+0000","last_clean":"2024-09-11T08:49:37.089698+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:49:37.089698+0000","last_undegraded":"2024-09-11T08:49:37.089698+0000","last_fullsized":"2024-09-11T08:49:37.089698+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'15","last_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_clean_scrub_stamp":"2024-09-11T08:48:38.032039+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:48.925034+0000","scrub_duration":0.0049584310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":217,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.088315+0000","last_change":"2024-09-11T08:49:26.248106+0000","last_active":"2024-09-11T08:49:37.088315+0000","last_peered":"2024-09-11T08:49:37.088315+0000","last_clean":"2024-09-11T08:49:37.088315+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:49:37.088315+0000","last_undegraded":"2024-09-11T08:49:37.088315+0000","last_fullsized":"2024-09-11T08:49:37.088315+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:26.247212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:26.247212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:30.023699+0000","scrub_duration":0.0056073520000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"211'9","reported_seq":50,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.802419+0000","last_change":"2024-09-11T08:49:09.920108+0000","last_active":"2024-09-11T08:49:37.802419+0000","last_peered":"2024-09-11T08:49:37.802419+0000","last_clean":"2024-09-11T08:49:37.802419+0000","last_became_active":"2024-09-11T08:49:09.910279+0000","last_became_peered":"2024-09-11T08:49:09.910279+0000","last_unstale":"2024-09-11T08:49:37.802419+0000","last_undegraded":"2024-09-11T08:49:37.802419+0000","last_fullsized":"2024-09-11T08:49:37.802419+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:48:44.796704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.796704+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:45.012801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091235+0000","last_change":"2024-09-11T08:49:09.908727+0000","last_active":"2024-09-11T08:49:37.091235+0000","last_peered":"2024-09-11T08:49:37.091235+0000","last_clean":"2024-09-11T08:49:37.091235+0000","last_became_active":"2024-09-11T08:49:09.907944+0000","last_became_peered":"2024-09-11T08:49:09.907944+0000","last_unstale":"2024-09-11T08:49:37.091235+0000","last_undegraded":"2024-09-11T08:49:37.091235+0000","last_fullsized":"2024-09-11T08:49:37.091235+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:31.710797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:48:31.710797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"219'3691","reported_seq":3712,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.171338+0000","last_change":"2024-09-11T08:49:05.875096+0000","last_active":"2024-09-11T08:49:41.171338+0000","last_peered":"2024-09-11T08:49:41.171338+0000","last_clean":"2024-09-11T08:49:41.171338+0000","last_became_active":"2024-09-11T08:49:05.874803+0000","last_became_peered":"2024-09-11T08:49:05.874803+0000","last_unstale":"2024-09-11T08:49:41.171338+0000","last_undegraded":"2024-09-11T08:49:41.171338+0000","last_fullsized":"2024-09-11T08:49:41.171338+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3691,"log_dups_size":0,"ondisk_log_size":3691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:12.899212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241893376,"num_objects":3691,"num_object_clones":0,"num_object_copies":11073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3691,"num_write_kb":236224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"200'29","reported_seq":269,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087474+0000","last_change":"2024-09-11T08:49:09.928210+0000","last_active":"2024-09-11T08:49:37.087474+0000","last_peered":"2024-09-11T08:49:37.087474+0000","last_clean":"2024-09-11T08:49:37.087474+0000","last_became_active":"2024-09-11T08:49:09.901065+0000","last_became_peered":"2024-09-11T08:49:09.901065+0000","last_unstale":"2024-09-11T08:49:37.087474+0000","last_undegraded":"2024-09-11T08:49:37.087474+0000","last_fullsized":"2024-09-11T08:49:37.087474+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:45:11.576085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.576085+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002366754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":221,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091335+0000","last_change":"2024-09-11T08:48:24.062209+0000","last_active":"2024-09-11T08:49:37.091335+0000","last_peered":"2024-09-11T08:49:37.091335+0000","last_clean":"2024-09-11T08:49:37.091335+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:49:37.091335+0000","last_undegraded":"2024-09-11T08:49:37.091335+0000","last_fullsized":"2024-09-11T08:49:37.091335+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:24.062110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:24.062110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.037841910999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":237,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090160+0000","last_change":"2024-09-11T08:49:28.069561+0000","last_active":"2024-09-11T08:49:37.090160+0000","last_peered":"2024-09-11T08:49:37.090160+0000","last_clean":"2024-09-11T08:49:37.090160+0000","last_became_active":"2024-09-11T08:49:09.909306+0000","last_became_peered":"2024-09-11T08:49:09.909306+0000","last_unstale":"2024-09-11T08:49:37.090160+0000","last_undegraded":"2024-09-11T08:49:37.090160+0000","last_fullsized":"2024-09-11T08:49:37.090160+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_clean_scrub_stamp":"2024-09-11T08:49:28.069461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:54.916266+0000","scrub_duration":0.0022283350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":235,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087457+0000","last_change":"2024-09-11T08:48:27.981040+0000","last_active":"2024-09-11T08:49:37.087457+0000","last_peered":"2024-09-11T08:49:37.087457+0000","last_clean":"2024-09-11T08:49:37.087457+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:49:37.087457+0000","last_undegraded":"2024-09-11T08:49:37.087457+0000","last_fullsized":"2024-09-11T08:49:37.087457+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:27.980860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:48:27.980860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021574010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"219'3739","reported_seq":3760,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.092324+0000","last_change":"2024-09-11T08:49:09.913293+0000","last_active":"2024-09-11T08:49:41.092324+0000","last_peered":"2024-09-11T08:49:41.092324+0000","last_clean":"2024-09-11T08:49:41.092324+0000","last_became_active":"2024-09-11T08:49:09.912997+0000","last_became_peered":"2024-09-11T08:49:09.912997+0000","last_unstale":"2024-09-11T08:49:41.092324+0000","last_undegraded":"2024-09-11T08:49:41.092324+0000","last_fullsized":"2024-09-11T08:49:41.092324+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3739,"log_dups_size":0,"ondisk_log_size":3739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:12.200890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":245039104,"num_objects":3739,"num_object_clones":0,"num_object_copies":11217,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3739,"num_write_kb":239296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"200'50","reported_seq":427,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.802616+0000","last_change":"2024-09-11T08:45:26.666261+0000","last_active":"2024-09-11T08:49:37.802616+0000","last_peered":"2024-09-11T08:49:37.802616+0000","last_clean":"2024-09-11T08:49:37.802616+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:49:37.802616+0000","last_undegraded":"2024-09-11T08:49:37.802616+0000","last_fullsized":"2024-09-11T08:49:37.802616+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:45:26.666199+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.666199+0000","objects_scrubbed":10,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020004279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":222,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094642+0000","last_change":"2024-09-11T08:49:25.626134+0000","last_active":"2024-09-11T08:49:37.094642+0000","last_peered":"2024-09-11T08:49:37.094642+0000","last_clean":"2024-09-11T08:49:37.094642+0000","last_became_active":"2024-09-11T08:49:09.907850+0000","last_became_peered":"2024-09-11T08:49:09.907850+0000","last_unstale":"2024-09-11T08:49:37.094642+0000","last_undegraded":"2024-09-11T08:49:37.094642+0000","last_fullsized":"2024-09-11T08:49:37.094642+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_clean_scrub_stamp":"2024-09-11T08:49:25.626031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:26.494358+0000","scrub_duration":0.0028789509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":252,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.088174+0000","last_change":"2024-09-11T08:48:16.376429+0000","last_active":"2024-09-11T08:49:37.088174+0000","last_peered":"2024-09-11T08:49:37.088174+0000","last_clean":"2024-09-11T08:49:37.088174+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:49:37.088174+0000","last_undegraded":"2024-09-11T08:49:37.088174+0000","last_fullsized":"2024-09-11T08:49:37.088174+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:48:16.376340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:16.376340+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:46.168511+0000","scrub_duration":0.00247796,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":21,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091644+0000","last_change":"2024-09-11T08:49:09.903912+0000","last_active":"2024-09-11T08:49:37.091644+0000","last_peered":"2024-09-11T08:49:37.091644+0000","last_clean":"2024-09-11T08:49:37.091644+0000","last_became_active":"2024-09-11T08:49:09.903614+0000","last_became_peered":"2024-09-11T08:49:09.903614+0000","last_unstale":"2024-09-11T08:49:37.091644+0000","last_undegraded":"2024-09-11T08:49:37.091644+0000","last_fullsized":"2024-09-11T08:49:37.091644+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:25.682632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.682632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":195,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.093309+0000","last_change":"2024-09-11T08:45:22.169436+0000","last_active":"2024-09-11T08:49:37.093309+0000","last_peered":"2024-09-11T08:49:37.093309+0000","last_clean":"2024-09-11T08:49:37.093309+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:49:37.093309+0000","last_undegraded":"2024-09-11T08:49:37.093309+0000","last_fullsized":"2024-09-11T08:49:37.093309+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:22.169367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:22.169367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005349693,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"219'3640","reported_seq":3660,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.288538+0000","last_change":"2024-09-11T08:49:05.872817+0000","last_active":"2024-09-11T08:49:41.288538+0000","last_peered":"2024-09-11T08:49:41.288538+0000","last_clean":"2024-09-11T08:49:41.288538+0000","last_became_active":"2024-09-11T08:49:05.872392+0000","last_became_peered":"2024-09-11T08:49:05.872392+0000","last_unstale":"2024-09-11T08:49:41.288538+0000","last_undegraded":"2024-09-11T08:49:41.288538+0000","last_fullsized":"2024-09-11T08:49:41.288538+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3640,"log_dups_size":0,"ondisk_log_size":3640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:05.342707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238551040,"num_objects":3640,"num_object_clones":0,"num_object_copies":10920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3640,"num_write_kb":232960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"200'17","reported_seq":274,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090992+0000","last_change":"2024-09-11T08:49:09.928583+0000","last_active":"2024-09-11T08:49:37.090992+0000","last_peered":"2024-09-11T08:49:37.090992+0000","last_clean":"2024-09-11T08:49:37.090992+0000","last_became_active":"2024-09-11T08:49:09.902884+0000","last_became_peered":"2024-09-11T08:49:09.902884+0000","last_unstale":"2024-09-11T08:49:37.090992+0000","last_undegraded":"2024-09-11T08:49:37.090992+0000","last_fullsized":"2024-09-11T08:49:37.090992+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:45:27.666502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.666502+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023582730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":247,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087189+0000","last_change":"2024-09-11T08:49:23.588233+0000","last_active":"2024-09-11T08:49:37.087189+0000","last_peered":"2024-09-11T08:49:37.087189+0000","last_clean":"2024-09-11T08:49:37.087189+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:49:37.087189+0000","last_undegraded":"2024-09-11T08:49:37.087189+0000","last_fullsized":"2024-09-11T08:49:37.087189+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:23.588182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:23.588182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:42.336498+0000","scrub_duration":0.0010197089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":265,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087519+0000","last_change":"2024-09-11T08:49:32.558340+0000","last_active":"2024-09-11T08:49:37.087519+0000","last_peered":"2024-09-11T08:49:37.087519+0000","last_clean":"2024-09-11T08:49:37.087519+0000","last_became_active":"2024-09-11T08:49:09.909804+0000","last_became_peered":"2024-09-11T08:49:09.909804+0000","last_unstale":"2024-09-11T08:49:37.087519+0000","last_undegraded":"2024-09-11T08:49:37.087519+0000","last_fullsized":"2024-09-11T08:49:37.087519+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:49:32.558236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:32.558236+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:01.291601+0000","scrub_duration":0.0035759400000000001,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":237,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090513+0000","last_change":"2024-09-11T08:49:15.477556+0000","last_active":"2024-09-11T08:49:37.090513+0000","last_peered":"2024-09-11T08:49:37.090513+0000","last_clean":"2024-09-11T08:49:37.090513+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:49:37.090513+0000","last_undegraded":"2024-09-11T08:49:37.090513+0000","last_fullsized":"2024-09-11T08:49:37.090513+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:15.477483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_clean_scrub_stamp":"2024-09-11T08:49:15.477483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:35.478111+0000","scrub_duration":0.0049012409999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":235,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091237+0000","last_change":"2024-09-11T08:48:44.031924+0000","last_active":"2024-09-11T08:49:37.091237+0000","last_peered":"2024-09-11T08:49:37.091237+0000","last_clean":"2024-09-11T08:49:37.091237+0000","last_became_active":"2024-09-11T08:46:01.983095+0000","last_became_peered":"2024-09-11T08:46:01.983095+0000","last_unstale":"2024-09-11T08:49:37.091237+0000","last_undegraded":"2024-09-11T08:49:37.091237+0000","last_fullsized":"2024-09-11T08:49:37.091237+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:44.031842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.031842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:55.851846+0000","scrub_duration":0.0021799599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"219'3619","reported_seq":3640,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.164584+0000","last_change":"2024-09-11T08:49:09.909742+0000","last_active":"2024-09-11T08:49:41.164584+0000","last_peered":"2024-09-11T08:49:41.164584+0000","last_clean":"2024-09-11T08:49:41.164584+0000","last_became_active":"2024-09-11T08:49:09.909429+0000","last_became_peered":"2024-09-11T08:49:09.909429+0000","last_unstale":"2024-09-11T08:49:41.164584+0000","last_undegraded":"2024-09-11T08:49:41.164584+0000","last_fullsized":"2024-09-11T08:49:41.164584+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3619,"log_dups_size":0,"ondisk_log_size":3619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:19.361954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237174784,"num_objects":3619,"num_object_clones":0,"num_object_copies":10857,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3619,"num_write_kb":231616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"200'24","reported_seq":260,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.088232+0000","last_change":"2024-09-11T08:49:09.930720+0000","last_active":"2024-09-11T08:49:37.088232+0000","last_peered":"2024-09-11T08:49:37.088232+0000","last_clean":"2024-09-11T08:49:37.088232+0000","last_became_active":"2024-09-11T08:49:09.911956+0000","last_became_peered":"2024-09-11T08:49:09.911956+0000","last_unstale":"2024-09-11T08:49:37.088232+0000","last_undegraded":"2024-09-11T08:49:37.088232+0000","last_fullsized":"2024-09-11T08:49:37.088232+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:48:37.680688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:37.680688+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:50.208675+0000","scrub_duration":0.0037660860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094972+0000","last_change":"2024-09-11T08:49:09.910130+0000","last_active":"2024-09-11T08:49:37.094972+0000","last_peered":"2024-09-11T08:49:37.094972+0000","last_clean":"2024-09-11T08:49:37.094972+0000","last_became_active":"2024-09-11T08:49:09.909803+0000","last_became_peered":"2024-09-11T08:49:09.909803+0000","last_unstale":"2024-09-11T08:49:37.094972+0000","last_undegraded":"2024-09-11T08:49:37.094972+0000","last_fullsized":"2024-09-11T08:49:37.094972+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:20.455081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.455081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":907,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:39.965736+0000","last_change":"2024-09-11T08:49:27.805277+0000","last_active":"2024-09-11T08:49:39.965736+0000","last_peered":"2024-09-11T08:49:39.965736+0000","last_clean":"2024-09-11T08:49:39.965736+0000","last_became_active":"2024-09-11T08:49:09.912659+0000","last_became_peered":"2024-09-11T08:49:09.912659+0000","last_unstale":"2024-09-11T08:49:39.965736+0000","last_undegraded":"2024-09-11T08:49:39.965736+0000","last_fullsized":"2024-09-11T08:49:39.965736+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:49:27.805050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:27.805050+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:57.498333+0000","scrub_duration":0.0046228930000000003,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":653,"num_read_kb":648,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":223,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.088140+0000","last_change":"2024-09-11T08:49:09.903588+0000","last_active":"2024-09-11T08:49:37.088140+0000","last_peered":"2024-09-11T08:49:37.088140+0000","last_clean":"2024-09-11T08:49:37.088140+0000","last_became_active":"2024-09-11T08:49:09.903283+0000","last_became_peered":"2024-09-11T08:49:09.903283+0000","last_unstale":"2024-09-11T08:49:37.088140+0000","last_undegraded":"2024-09-11T08:49:37.088140+0000","last_fullsized":"2024-09-11T08:49:37.088140+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:30.532701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.532701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:58.509201+0000","scrub_duration":0.012853409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":215,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.088370+0000","last_change":"2024-09-11T08:48:33.025967+0000","last_active":"2024-09-11T08:49:37.088370+0000","last_peered":"2024-09-11T08:49:37.088370+0000","last_clean":"2024-09-11T08:49:37.088370+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:49:37.088370+0000","last_undegraded":"2024-09-11T08:49:37.088370+0000","last_fullsized":"2024-09-11T08:49:37.088370+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:33.025897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:48:33.025897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:50.371310+0000","scrub_duration":0.007471208,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"219'3634","reported_seq":3665,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.756387+0000","last_change":"2024-09-11T08:49:09.915717+0000","last_active":"2024-09-11T08:49:41.756387+0000","last_peered":"2024-09-11T08:49:41.756387+0000","last_clean":"2024-09-11T08:49:41.756387+0000","last_became_active":"2024-09-11T08:49:09.914863+0000","last_became_peered":"2024-09-11T08:49:09.914863+0000","last_unstale":"2024-09-11T08:49:41.756387+0000","last_undegraded":"2024-09-11T08:49:41.756387+0000","last_fullsized":"2024-09-11T08:49:41.756387+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3634,"log_dups_size":0,"ondisk_log_size":3634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:26.375120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238157824,"num_objects":3634,"num_object_clones":0,"num_object_copies":10902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3634,"num_write_kb":232576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"201'27","reported_seq":116,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090715+0000","last_change":"2024-09-11T08:48:47.827201+0000","last_active":"2024-09-11T08:49:37.090715+0000","last_peered":"2024-09-11T08:49:37.090715+0000","last_clean":"2024-09-11T08:49:37.090715+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:49:37.090715+0000","last_undegraded":"2024-09-11T08:49:37.090715+0000","last_fullsized":"2024-09-11T08:49:37.090715+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'27","last_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_deep_scrub":"201'27","last_deep_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_clean_scrub_stamp":"2024-09-11T08:48:47.827134+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:12.693583+0000","scrub_duration":0.0038402419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":210,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094096+0000","last_change":"2024-09-11T08:45:18.127946+0000","last_active":"2024-09-11T08:49:37.094096+0000","last_peered":"2024-09-11T08:49:37.094096+0000","last_clean":"2024-09-11T08:49:37.094096+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:49:37.094096+0000","last_undegraded":"2024-09-11T08:49:37.094096+0000","last_fullsized":"2024-09-11T08:49:37.094096+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:18.127857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.127857+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018334099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"199'81","reported_seq":283,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.089915+0000","last_change":"2024-09-11T08:49:09.926313+0000","last_active":"2024-09-11T08:49:37.089915+0000","last_peered":"2024-09-11T08:49:37.089915+0000","last_clean":"2024-09-11T08:49:37.089915+0000","last_became_active":"2024-09-11T08:49:09.904681+0000","last_became_peered":"2024-09-11T08:49:09.904681+0000","last_unstale":"2024-09-11T08:49:37.089915+0000","last_undegraded":"2024-09-11T08:49:37.089915+0000","last_fullsized":"2024-09-11T08:49:37.089915+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:05.407039+0000","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":217,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091077+0000","last_change":"2024-09-11T08:47:35.433723+0000","last_active":"2024-09-11T08:49:37.091077+0000","last_peered":"2024-09-11T08:49:37.091077+0000","last_clean":"2024-09-11T08:49:37.091077+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:49:37.091077+0000","last_undegraded":"2024-09-11T08:49:37.091077+0000","last_fullsized":"2024-09-11T08:49:37.091077+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:35.433651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:35.433651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013907892999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":211,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.088008+0000","last_change":"2024-09-11T08:48:19.308968+0000","last_active":"2024-09-11T08:49:37.088008+0000","last_peered":"2024-09-11T08:49:37.088008+0000","last_clean":"2024-09-11T08:49:37.088008+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:49:37.088008+0000","last_undegraded":"2024-09-11T08:49:37.088008+0000","last_fullsized":"2024-09-11T08:49:37.088008+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:19.308869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.308869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011385526999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"219'3655","reported_seq":3676,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.152834+0000","last_change":"2024-09-11T08:49:05.874474+0000","last_active":"2024-09-11T08:49:41.152834+0000","last_peered":"2024-09-11T08:49:41.152834+0000","last_clean":"2024-09-11T08:49:41.152834+0000","last_became_active":"2024-09-11T08:49:05.874151+0000","last_became_peered":"2024-09-11T08:49:05.874151+0000","last_unstale":"2024-09-11T08:49:41.152834+0000","last_undegraded":"2024-09-11T08:49:41.152834+0000","last_fullsized":"2024-09-11T08:49:41.152834+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3655,"log_dups_size":0,"ondisk_log_size":3655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:33.548168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239534080,"num_objects":3655,"num_object_clones":0,"num_object_copies":10965,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3655,"num_write_kb":233920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"200'19","reported_seq":300,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.096422+0000","last_change":"2024-09-11T08:48:20.860266+0000","last_active":"2024-09-11T08:49:37.096422+0000","last_peered":"2024-09-11T08:49:37.096422+0000","last_clean":"2024-09-11T08:49:37.096422+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:49:37.096422+0000","last_undegraded":"2024-09-11T08:49:37.096422+0000","last_fullsized":"2024-09-11T08:49:37.096422+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:20.860186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.860186+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:43.681747+0000","scrub_duration":0.0045417560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":202,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.095264+0000","last_change":"2024-09-11T08:49:18.665872+0000","last_active":"2024-09-11T08:49:37.095264+0000","last_peered":"2024-09-11T08:49:37.095264+0000","last_clean":"2024-09-11T08:49:37.095264+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:49:37.095264+0000","last_undegraded":"2024-09-11T08:49:37.095264+0000","last_fullsized":"2024-09-11T08:49:37.095264+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:18.665790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:49:18.665790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:40.249031+0000","scrub_duration":0.0041745330000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":220,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087801+0000","last_change":"2024-09-11T08:49:29.269384+0000","last_active":"2024-09-11T08:49:37.087801+0000","last_peered":"2024-09-11T08:49:37.087801+0000","last_clean":"2024-09-11T08:49:37.087801+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:49:37.087801+0000","last_undegraded":"2024-09-11T08:49:37.087801+0000","last_fullsized":"2024-09-11T08:49:37.087801+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:29.269302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:49:29.269302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:40.650236+0000","scrub_duration":0.0083588919999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":215,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094142+0000","last_change":"2024-09-11T08:46:01.988315+0000","last_active":"2024-09-11T08:49:37.094142+0000","last_peered":"2024-09-11T08:49:37.094142+0000","last_clean":"2024-09-11T08:49:37.094142+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:49:37.094142+0000","last_undegraded":"2024-09-11T08:49:37.094142+0000","last_fullsized":"2024-09-11T08:49:37.094142+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:15.167071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:15.167071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001796483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"219'3631","reported_seq":3651,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.540091+0000","last_change":"2024-09-11T08:49:06.519555+0000","last_active":"2024-09-11T08:49:41.540091+0000","last_peered":"2024-09-11T08:49:41.540091+0000","last_clean":"2024-09-11T08:49:41.540091+0000","last_became_active":"2024-09-11T08:49:06.519263+0000","last_became_peered":"2024-09-11T08:49:06.519263+0000","last_unstale":"2024-09-11T08:49:41.540091+0000","last_undegraded":"2024-09-11T08:49:41.540091+0000","last_fullsized":"2024-09-11T08:49:41.540091+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3631,"log_dups_size":0,"ondisk_log_size":3631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:05.582121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237961216,"num_objects":3631,"num_object_clones":0,"num_object_copies":10893,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3631,"num_write_kb":232384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"200'31","reported_seq":259,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094732+0000","last_change":"2024-09-11T08:48:43.728009+0000","last_active":"2024-09-11T08:49:37.094732+0000","last_peered":"2024-09-11T08:49:37.094732+0000","last_clean":"2024-09-11T08:49:37.094732+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:49:37.094732+0000","last_undegraded":"2024-09-11T08:49:37.094732+0000","last_fullsized":"2024-09-11T08:49:37.094732+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:48:43.727938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:43.727938+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:49.803069+0000","scrub_duration":0.016024792999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":231,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090435+0000","last_change":"2024-09-11T08:48:28.828694+0000","last_active":"2024-09-11T08:49:37.090435+0000","last_peered":"2024-09-11T08:49:37.090435+0000","last_clean":"2024-09-11T08:49:37.090435+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:49:37.090435+0000","last_undegraded":"2024-09-11T08:49:37.090435+0000","last_fullsized":"2024-09-11T08:49:37.090435+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:28.828618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_clean_scrub_stamp":"2024-09-11T08:48:28.828618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029516289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":227,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.088415+0000","last_change":"2024-09-11T08:49:09.903898+0000","last_active":"2024-09-11T08:49:37.088415+0000","last_peered":"2024-09-11T08:49:37.088415+0000","last_clean":"2024-09-11T08:49:37.088415+0000","last_became_active":"2024-09-11T08:49:09.903589+0000","last_became_peered":"2024-09-11T08:49:09.903589+0000","last_unstale":"2024-09-11T08:49:37.088415+0000","last_undegraded":"2024-09-11T08:49:37.088415+0000","last_fullsized":"2024-09-11T08:49:37.088415+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:22.989179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:22.989179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:39.554788+0000","scrub_duration":0.0033708700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":241,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090058+0000","last_change":"2024-09-11T08:49:29.528952+0000","last_active":"2024-09-11T08:49:37.090058+0000","last_peered":"2024-09-11T08:49:37.090058+0000","last_clean":"2024-09-11T08:49:37.090058+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:49:37.090058+0000","last_undegraded":"2024-09-11T08:49:37.090058+0000","last_fullsized":"2024-09-11T08:49:37.090058+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:29.528873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:29.528873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:46.400819+0000","scrub_duration":0.0024493430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"219'3745","reported_seq":3766,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.775445+0000","last_change":"2024-09-11T08:49:06.077463+0000","last_active":"2024-09-11T08:49:41.775445+0000","last_peered":"2024-09-11T08:49:41.775445+0000","last_clean":"2024-09-11T08:49:41.775445+0000","last_became_active":"2024-09-11T08:49:06.077144+0000","last_became_peered":"2024-09-11T08:49:06.077144+0000","last_unstale":"2024-09-11T08:49:41.775445+0000","last_undegraded":"2024-09-11T08:49:41.775445+0000","last_fullsized":"2024-09-11T08:49:41.775445+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3745,"log_dups_size":0,"ondisk_log_size":3745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:14.241327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":245432320,"num_objects":3745,"num_object_clones":0,"num_object_copies":11235,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3745,"num_write_kb":239680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"200'20","reported_seq":277,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.095071+0000","last_change":"2024-09-11T08:48:21.885766+0000","last_active":"2024-09-11T08:49:37.095071+0000","last_peered":"2024-09-11T08:49:37.095071+0000","last_clean":"2024-09-11T08:49:37.095071+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:49:37.095071+0000","last_undegraded":"2024-09-11T08:49:37.095071+0000","last_fullsized":"2024-09-11T08:49:37.095071+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:48:21.885681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:21.885681+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051165580000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":211,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091105+0000","last_change":"2024-09-11T08:49:09.911420+0000","last_active":"2024-09-11T08:49:37.091105+0000","last_peered":"2024-09-11T08:49:37.091105+0000","last_clean":"2024-09-11T08:49:37.091105+0000","last_became_active":"2024-09-11T08:49:09.911101+0000","last_became_peered":"2024-09-11T08:49:09.911101+0000","last_unstale":"2024-09-11T08:49:37.091105+0000","last_undegraded":"2024-09-11T08:49:37.091105+0000","last_fullsized":"2024-09-11T08:49:37.091105+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:32.219374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:32.219374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069622119999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":377,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.802329+0000","last_change":"2024-09-11T08:49:27.577662+0000","last_active":"2024-09-11T08:49:37.802329+0000","last_peered":"2024-09-11T08:49:37.802329+0000","last_clean":"2024-09-11T08:49:37.802329+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:49:37.802329+0000","last_undegraded":"2024-09-11T08:49:37.802329+0000","last_fullsized":"2024-09-11T08:49:37.802329+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:49:27.577575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:49:27.577575+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:34.177695+0000","scrub_duration":0.0084200460000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":196,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.096261+0000","last_change":"2024-09-11T08:45:31.196687+0000","last_active":"2024-09-11T08:49:37.096261+0000","last_peered":"2024-09-11T08:49:37.096261+0000","last_clean":"2024-09-11T08:49:37.096261+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:49:37.096261+0000","last_undegraded":"2024-09-11T08:49:37.096261+0000","last_fullsized":"2024-09-11T08:49:37.096261+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:31.196599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:45:31.196599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025248190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"219'3614","reported_seq":3634,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.311177+0000","last_change":"2024-09-11T08:49:06.519232+0000","last_active":"2024-09-11T08:49:41.311177+0000","last_peered":"2024-09-11T08:49:41.311177+0000","last_clean":"2024-09-11T08:49:41.311177+0000","last_became_active":"2024-09-11T08:49:06.518991+0000","last_became_peered":"2024-09-11T08:49:06.518991+0000","last_unstale":"2024-09-11T08:49:41.311177+0000","last_undegraded":"2024-09-11T08:49:41.311177+0000","last_fullsized":"2024-09-11T08:49:41.311177+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3614,"log_dups_size":0,"ondisk_log_size":3614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:34.806916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":236847104,"num_objects":3614,"num_object_clones":0,"num_object_copies":10842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3614,"num_write_kb":231296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"200'15","reported_seq":245,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.089203+0000","last_change":"2024-09-11T08:49:09.925808+0000","last_active":"2024-09-11T08:49:37.089203+0000","last_peered":"2024-09-11T08:49:37.089203+0000","last_clean":"2024-09-11T08:49:37.089203+0000","last_became_active":"2024-09-11T08:49:09.915666+0000","last_became_peered":"2024-09-11T08:49:09.915666+0000","last_unstale":"2024-09-11T08:49:37.089203+0000","last_undegraded":"2024-09-11T08:49:37.089203+0000","last_fullsized":"2024-09-11T08:49:37.089203+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:48:41.787219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.787219+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:08.422014+0000","scrub_duration":0.0060192070000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":224,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.086460+0000","last_change":"2024-09-11T08:49:30.234576+0000","last_active":"2024-09-11T08:49:37.086460+0000","last_peered":"2024-09-11T08:49:37.086460+0000","last_clean":"2024-09-11T08:49:37.086460+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:49:37.086460+0000","last_undegraded":"2024-09-11T08:49:37.086460+0000","last_fullsized":"2024-09-11T08:49:37.086460+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:30.234078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:49:30.234078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:55.702848+0000","scrub_duration":0.0025717829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":221,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.088472+0000","last_change":"2024-09-11T08:48:24.380508+0000","last_active":"2024-09-11T08:49:37.088472+0000","last_peered":"2024-09-11T08:49:37.088472+0000","last_clean":"2024-09-11T08:49:37.088472+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:49:37.088472+0000","last_undegraded":"2024-09-11T08:49:37.088472+0000","last_fullsized":"2024-09-11T08:49:37.088472+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:24.380424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:24.380424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:39.122714+0000","scrub_duration":0.018256864000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091752+0000","last_change":"2024-09-11T08:49:09.912893+0000","last_active":"2024-09-11T08:49:37.091752+0000","last_peered":"2024-09-11T08:49:37.091752+0000","last_clean":"2024-09-11T08:49:37.091752+0000","last_became_active":"2024-09-11T08:49:09.912556+0000","last_became_peered":"2024-09-11T08:49:09.912556+0000","last_unstale":"2024-09-11T08:49:37.091752+0000","last_undegraded":"2024-09-11T08:49:37.091752+0000","last_fullsized":"2024-09-11T08:49:37.091752+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:47.699453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:47:47.699453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"219'3673","reported_seq":3693,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.292831+0000","last_change":"2024-09-11T08:49:05.871158+0000","last_active":"2024-09-11T08:49:41.292831+0000","last_peered":"2024-09-11T08:49:41.292831+0000","last_clean":"2024-09-11T08:49:41.292831+0000","last_became_active":"2024-09-11T08:49:05.870797+0000","last_became_peered":"2024-09-11T08:49:05.870797+0000","last_unstale":"2024-09-11T08:49:41.292831+0000","last_undegraded":"2024-09-11T08:49:41.292831+0000","last_fullsized":"2024-09-11T08:49:41.292831+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3673,"log_dups_size":0,"ondisk_log_size":3673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:33.662137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240713728,"num_objects":3673,"num_object_clones":0,"num_object_copies":11019,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3673,"num_write_kb":235072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"200'35","reported_seq":125,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090264+0000","last_change":"2024-09-11T08:49:22.127201+0000","last_active":"2024-09-11T08:49:37.090264+0000","last_peered":"2024-09-11T08:49:37.090264+0000","last_clean":"2024-09-11T08:49:37.090264+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:49:37.090264+0000","last_undegraded":"2024-09-11T08:49:37.090264+0000","last_fullsized":"2024-09-11T08:49:37.090264+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'35","last_scrub_stamp":"2024-09-11T08:49:22.127119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:49:22.127119+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:33.163032+0000","scrub_duration":0.0038731960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":221,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091538+0000","last_change":"2024-09-11T08:47:44.411099+0000","last_active":"2024-09-11T08:49:37.091538+0000","last_peered":"2024-09-11T08:49:37.091538+0000","last_clean":"2024-09-11T08:49:37.091538+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:49:37.091538+0000","last_undegraded":"2024-09-11T08:49:37.091538+0000","last_fullsized":"2024-09-11T08:49:37.091538+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.411022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.411022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047762170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":247,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087753+0000","last_change":"2024-09-11T08:49:25.610133+0000","last_active":"2024-09-11T08:49:37.087753+0000","last_peered":"2024-09-11T08:49:37.087753+0000","last_clean":"2024-09-11T08:49:37.087753+0000","last_became_active":"2024-09-11T08:49:09.907542+0000","last_became_peered":"2024-09-11T08:49:09.907542+0000","last_unstale":"2024-09-11T08:49:37.087753+0000","last_undegraded":"2024-09-11T08:49:37.087753+0000","last_fullsized":"2024-09-11T08:49:37.087753+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:25.609666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:49:25.609666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:46.984958+0000","scrub_duration":0.003585071,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":208,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.089834+0000","last_change":"2024-09-11T08:48:33.065142+0000","last_active":"2024-09-11T08:49:37.089834+0000","last_peered":"2024-09-11T08:49:37.089834+0000","last_clean":"2024-09-11T08:49:37.089834+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:49:37.089834+0000","last_undegraded":"2024-09-11T08:49:37.089834+0000","last_fullsized":"2024-09-11T08:49:37.089834+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:33.065053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:33.065053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:54.909406+0000","scrub_duration":0.013998086999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"219'3720","reported_seq":3740,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.562991+0000","last_change":"2024-09-11T08:49:05.871175+0000","last_active":"2024-09-11T08:49:41.562991+0000","last_peered":"2024-09-11T08:49:41.562991+0000","last_clean":"2024-09-11T08:49:41.562991+0000","last_became_active":"2024-09-11T08:49:05.870890+0000","last_became_peered":"2024-09-11T08:49:05.870890+0000","last_unstale":"2024-09-11T08:49:41.562991+0000","last_undegraded":"2024-09-11T08:49:41.562991+0000","last_fullsized":"2024-09-11T08:49:41.562991+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3720,"log_dups_size":0,"ondisk_log_size":3720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:21.232303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243793920,"num_objects":3720,"num_object_clones":0,"num_object_copies":11160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3720,"num_write_kb":238080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"200'18","reported_seq":232,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.095047+0000","last_change":"2024-09-11T08:45:08.140910+0000","last_active":"2024-09-11T08:49:37.095047+0000","last_peered":"2024-09-11T08:49:37.095047+0000","last_clean":"2024-09-11T08:49:37.095047+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:49:37.095047+0000","last_undegraded":"2024-09-11T08:49:37.095047+0000","last_fullsized":"2024-09-11T08:49:37.095047+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:45:08.140839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.140839+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003679547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":221,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090493+0000","last_change":"2024-09-11T08:48:47.033544+0000","last_active":"2024-09-11T08:49:37.090493+0000","last_peered":"2024-09-11T08:49:37.090493+0000","last_clean":"2024-09-11T08:49:37.090493+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:49:37.090493+0000","last_undegraded":"2024-09-11T08:49:37.090493+0000","last_fullsized":"2024-09-11T08:49:37.090493+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:47.033487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:47.033487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:51.790809+0000","scrub_duration":0.0014934150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":240,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087799+0000","last_change":"2024-09-11T08:49:09.907865+0000","last_active":"2024-09-11T08:49:37.087799+0000","last_peered":"2024-09-11T08:49:37.087799+0000","last_clean":"2024-09-11T08:49:37.087799+0000","last_became_active":"2024-09-11T08:49:09.907535+0000","last_became_peered":"2024-09-11T08:49:09.907535+0000","last_unstale":"2024-09-11T08:49:37.087799+0000","last_undegraded":"2024-09-11T08:49:37.087799+0000","last_fullsized":"2024-09-11T08:49:37.087799+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:21.336766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:21.336766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077730009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":195,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090867+0000","last_change":"2024-09-11T08:45:33.228265+0000","last_active":"2024-09-11T08:49:37.090867+0000","last_peered":"2024-09-11T08:49:37.090867+0000","last_clean":"2024-09-11T08:49:37.090867+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:49:37.090867+0000","last_undegraded":"2024-09-11T08:49:37.090867+0000","last_fullsized":"2024-09-11T08:49:37.090867+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:45:33.228082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064426500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"219'3630","reported_seq":3651,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-11T08:49:41.266928+0000","last_change":"2024-09-11T08:49:09.900869+0000","last_active":"2024-09-11T08:49:41.266928+0000","last_peered":"2024-09-11T08:49:41.266928+0000","last_clean":"2024-09-11T08:49:41.266928+0000","last_became_active":"2024-09-11T08:49:09.900548+0000","last_became_peered":"2024-09-11T08:49:09.900548+0000","last_unstale":"2024-09-11T08:49:41.266928+0000","last_undegraded":"2024-09-11T08:49:41.266928+0000","last_fullsized":"2024-09-11T08:49:41.266928+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3630,"log_dups_size":0,"ondisk_log_size":3630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:08.175978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237895680,"num_objects":3630,"num_object_clones":0,"num_object_copies":10890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3630,"num_write_kb":232320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"200'34","reported_seq":317,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087397+0000","last_change":"2024-09-11T08:48:20.352021+0000","last_active":"2024-09-11T08:49:37.087397+0000","last_peered":"2024-09-11T08:49:37.087397+0000","last_clean":"2024-09-11T08:49:37.087397+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:49:37.087397+0000","last_undegraded":"2024-09-11T08:49:37.087397+0000","last_fullsized":"2024-09-11T08:49:37.087397+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:48:20.351963+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.351963+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058139180000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":254,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091808+0000","last_change":"2024-09-11T08:49:17.884190+0000","last_active":"2024-09-11T08:49:37.091808+0000","last_peered":"2024-09-11T08:49:37.091808+0000","last_clean":"2024-09-11T08:49:37.091808+0000","last_became_active":"2024-09-11T08:49:09.904656+0000","last_became_peered":"2024-09-11T08:49:09.904656+0000","last_unstale":"2024-09-11T08:49:37.091808+0000","last_undegraded":"2024-09-11T08:49:37.091808+0000","last_fullsized":"2024-09-11T08:49:37.091808+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:49:17.884108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:17.884108+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:33.993275+0000","scrub_duration":0.0031968909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"168'10","reported_seq":385,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.802420+0000","last_change":"2024-09-11T08:48:41.038840+0000","last_active":"2024-09-11T08:49:37.802420+0000","last_peered":"2024-09-11T08:49:37.802420+0000","last_clean":"2024-09-11T08:49:37.802420+0000","last_became_active":"2024-09-11T08:46:01.983055+0000","last_became_peered":"2024-09-11T08:46:01.983055+0000","last_unstale":"2024-09-11T08:49:37.802420+0000","last_undegraded":"2024-09-11T08:49:37.802420+0000","last_fullsized":"2024-09-11T08:49:37.802420+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"168'10","last_scrub_stamp":"2024-09-11T08:48:41.038786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.038786+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:04.138712+0000","scrub_duration":0.00164314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":221,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087876+0000","last_change":"2024-09-11T08:49:18.301286+0000","last_active":"2024-09-11T08:49:37.087876+0000","last_peered":"2024-09-11T08:49:37.087876+0000","last_clean":"2024-09-11T08:49:37.087876+0000","last_became_active":"2024-09-11T08:49:09.906201+0000","last_became_peered":"2024-09-11T08:49:09.906201+0000","last_unstale":"2024-09-11T08:49:37.087876+0000","last_undegraded":"2024-09-11T08:49:37.087876+0000","last_fullsized":"2024-09-11T08:49:37.087876+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:18.301203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:18.301203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:37.257368+0000","scrub_duration":0.0032551070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":251,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091076+0000","last_change":"2024-09-11T08:49:34.002358+0000","last_active":"2024-09-11T08:49:37.091076+0000","last_peered":"2024-09-11T08:49:37.091076+0000","last_clean":"2024-09-11T08:49:37.091076+0000","last_became_active":"2024-09-11T08:49:09.910329+0000","last_became_peered":"2024-09-11T08:49:09.910329+0000","last_unstale":"2024-09-11T08:49:37.091076+0000","last_undegraded":"2024-09-11T08:49:37.091076+0000","last_fullsized":"2024-09-11T08:49:37.091076+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:34.002280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:49:34.002280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:39.769687+0000","scrub_duration":0.0089946609999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"200'12","reported_seq":245,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094741+0000","last_change":"2024-09-11T08:49:15.347679+0000","last_active":"2024-09-11T08:49:37.094741+0000","last_peered":"2024-09-11T08:49:37.094741+0000","last_clean":"2024-09-11T08:49:37.094741+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:49:37.094741+0000","last_undegraded":"2024-09-11T08:49:37.094741+0000","last_fullsized":"2024-09-11T08:49:37.094741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'12","last_scrub_stamp":"2024-09-11T08:49:15.347591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:49:15.347591+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:31.103948+0000","scrub_duration":0.0048379369999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":207,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.086912+0000","last_change":"2024-09-11T08:45:08.675251+0000","last_active":"2024-09-11T08:49:37.086912+0000","last_peered":"2024-09-11T08:49:37.086912+0000","last_clean":"2024-09-11T08:49:37.086912+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:49:37.086912+0000","last_undegraded":"2024-09-11T08:49:37.086912+0000","last_fullsized":"2024-09-11T08:49:37.086912+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:08.675179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.675179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003515392,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":225,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090626+0000","last_change":"2024-09-11T08:46:01.983864+0000","last_active":"2024-09-11T08:49:37.090626+0000","last_peered":"2024-09-11T08:49:37.090626+0000","last_clean":"2024-09-11T08:49:37.090626+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:49:37.090626+0000","last_undegraded":"2024-09-11T08:49:37.090626+0000","last_fullsized":"2024-09-11T08:49:37.090626+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:25.344348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:25.344348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013849730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"200'15","reported_seq":245,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.093442+0000","last_change":"2024-09-11T08:49:09.933608+0000","last_active":"2024-09-11T08:49:37.093442+0000","last_peered":"2024-09-11T08:49:37.093442+0000","last_clean":"2024-09-11T08:49:37.093442+0000","last_became_active":"2024-09-11T08:49:09.916927+0000","last_became_peered":"2024-09-11T08:49:09.916927+0000","last_unstale":"2024-09-11T08:49:37.093442+0000","last_undegraded":"2024-09-11T08:49:37.093442+0000","last_fullsized":"2024-09-11T08:49:37.093442+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:47:44.737721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.737721+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032009640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":224,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091872+0000","last_change":"2024-09-11T08:48:20.977749+0000","last_active":"2024-09-11T08:49:37.091872+0000","last_peered":"2024-09-11T08:49:37.091872+0000","last_clean":"2024-09-11T08:49:37.091872+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:49:37.091872+0000","last_undegraded":"2024-09-11T08:49:37.091872+0000","last_fullsized":"2024-09-11T08:49:37.091872+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:20.977665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.977665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:45.220147+0000","scrub_duration":0.0075643139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":197,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.093573+0000","last_change":"2024-09-11T08:45:23.213642+0000","last_active":"2024-09-11T08:49:37.093573+0000","last_peered":"2024-09-11T08:49:37.093573+0000","last_clean":"2024-09-11T08:49:37.093573+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:49:37.093573+0000","last_undegraded":"2024-09-11T08:49:37.093573+0000","last_fullsized":"2024-09-11T08:49:37.093573+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.213592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.213592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001600473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":226,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087620+0000","last_change":"2024-09-11T08:48:47.485315+0000","last_active":"2024-09-11T08:49:37.087620+0000","last_peered":"2024-09-11T08:49:37.087620+0000","last_clean":"2024-09-11T08:49:37.087620+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:49:37.087620+0000","last_undegraded":"2024-09-11T08:49:37.087620+0000","last_fullsized":"2024-09-11T08:49:37.087620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:47.485264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_clean_scrub_stamp":"2024-09-11T08:48:47.485264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:17.166643+0000","scrub_duration":0.0019043490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"200'15","reported_seq":240,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.088190+0000","last_change":"2024-09-11T08:48:36.403872+0000","last_active":"2024-09-11T08:49:37.088190+0000","last_peered":"2024-09-11T08:49:37.088190+0000","last_clean":"2024-09-11T08:49:37.088190+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:49:37.088190+0000","last_undegraded":"2024-09-11T08:49:37.088190+0000","last_fullsized":"2024-09-11T08:49:37.088190+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:48:36.403646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:36.403646+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057924789999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091344+0000","last_change":"2024-09-11T08:49:09.912712+0000","last_active":"2024-09-11T08:49:37.091344+0000","last_peered":"2024-09-11T08:49:37.091344+0000","last_clean":"2024-09-11T08:49:37.091344+0000","last_became_active":"2024-09-11T08:49:09.912417+0000","last_became_peered":"2024-09-11T08:49:09.912417+0000","last_unstale":"2024-09-11T08:49:37.091344+0000","last_undegraded":"2024-09-11T08:49:37.091344+0000","last_fullsized":"2024-09-11T08:49:37.091344+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:45.751518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:45.751518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":337,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.802737+0000","last_change":"2024-09-11T08:45:20.119727+0000","last_active":"2024-09-11T08:49:37.802737+0000","last_peered":"2024-09-11T08:49:37.802737+0000","last_clean":"2024-09-11T08:49:37.802737+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:49:37.802737+0000","last_undegraded":"2024-09-11T08:49:37.802737+0000","last_fullsized":"2024-09-11T08:49:37.802737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:20.119626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.119626+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018087660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":196,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094883+0000","last_change":"2024-09-11T08:48:41.648829+0000","last_active":"2024-09-11T08:49:37.094883+0000","last_peered":"2024-09-11T08:49:37.094883+0000","last_clean":"2024-09-11T08:49:37.094883+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:49:37.094883+0000","last_undegraded":"2024-09-11T08:49:37.094883+0000","last_fullsized":"2024-09-11T08:49:37.094883+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:41.648756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.648756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:51.312266+0000","scrub_duration":0.0019862030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"200'25","reported_seq":244,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094799+0000","last_change":"2024-09-11T08:49:28.382609+0000","last_active":"2024-09-11T08:49:37.094799+0000","last_peered":"2024-09-11T08:49:37.094799+0000","last_clean":"2024-09-11T08:49:37.094799+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:49:37.094799+0000","last_undegraded":"2024-09-11T08:49:37.094799+0000","last_fullsized":"2024-09-11T08:49:37.094799+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:49:28.382409+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:49:28.382409+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:28.482336+0000","scrub_duration":0.0046362419999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":221,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090019+0000","last_change":"2024-09-11T08:48:39.064220+0000","last_active":"2024-09-11T08:49:37.090019+0000","last_peered":"2024-09-11T08:49:37.090019+0000","last_clean":"2024-09-11T08:49:37.090019+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:49:37.090019+0000","last_undegraded":"2024-09-11T08:49:37.090019+0000","last_fullsized":"2024-09-11T08:49:37.090019+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:39.064163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_clean_scrub_stamp":"2024-09-11T08:48:39.064163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:03.250634+0000","scrub_duration":0.0014228540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"211'9","reported_seq":361,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.802935+0000","last_change":"2024-09-11T08:49:09.926226+0000","last_active":"2024-09-11T08:49:37.802935+0000","last_peered":"2024-09-11T08:49:37.802935+0000","last_clean":"2024-09-11T08:49:37.802935+0000","last_became_active":"2024-09-11T08:49:09.908380+0000","last_became_peered":"2024-09-11T08:49:09.908380+0000","last_unstale":"2024-09-11T08:49:37.802935+0000","last_undegraded":"2024-09-11T08:49:37.802935+0000","last_fullsized":"2024-09-11T08:49:37.802935+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:28.692100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.692100+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019579020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":31,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090262+0000","last_change":"2024-09-11T08:49:21.855032+0000","last_active":"2024-09-11T08:49:37.090262+0000","last_peered":"2024-09-11T08:49:37.090262+0000","last_clean":"2024-09-11T08:49:37.090262+0000","last_became_active":"2024-09-11T08:49:09.914396+0000","last_became_peered":"2024-09-11T08:49:09.914396+0000","last_unstale":"2024-09-11T08:49:37.090262+0000","last_undegraded":"2024-09-11T08:49:37.090262+0000","last_fullsized":"2024-09-11T08:49:37.090262+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:21.854973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:21.854973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:48.440684+0000","scrub_duration":0.0064444309999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"200'19","reported_seq":253,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.094327+0000","last_change":"2024-09-11T08:49:16.951920+0000","last_active":"2024-09-11T08:49:37.094327+0000","last_peered":"2024-09-11T08:49:37.094327+0000","last_clean":"2024-09-11T08:49:37.094327+0000","last_became_active":"2024-09-11T08:49:16.937185+0000","last_became_peered":"2024-09-11T08:49:16.937185+0000","last_unstale":"2024-09-11T08:49:37.094327+0000","last_undegraded":"2024-09-11T08:49:37.094327+0000","last_fullsized":"2024-09-11T08:49:37.094327+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:25.792777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.792777+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:53.751222+0000","scrub_duration":0.0045475469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":202,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.095144+0000","last_change":"2024-09-11T08:45:27.198021+0000","last_active":"2024-09-11T08:49:37.095144+0000","last_peered":"2024-09-11T08:49:37.095144+0000","last_clean":"2024-09-11T08:49:37.095144+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:49:37.095144+0000","last_undegraded":"2024-09-11T08:49:37.095144+0000","last_fullsized":"2024-09-11T08:49:37.095144+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:27.197944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.197944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015230210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":217,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.089607+0000","last_change":"2024-09-11T08:48:30.985518+0000","last_active":"2024-09-11T08:49:37.089607+0000","last_peered":"2024-09-11T08:49:37.089607+0000","last_clean":"2024-09-11T08:49:37.089607+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:49:37.089607+0000","last_undegraded":"2024-09-11T08:49:37.089607+0000","last_fullsized":"2024-09-11T08:49:37.089607+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:30.985271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.985271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:50.920428+0000","scrub_duration":0.0099058919999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":205,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090785+0000","last_change":"2024-09-11T08:49:09.906202+0000","last_active":"2024-09-11T08:49:37.090785+0000","last_peered":"2024-09-11T08:49:37.090785+0000","last_clean":"2024-09-11T08:49:37.090785+0000","last_became_active":"2024-09-11T08:49:09.905289+0000","last_became_peered":"2024-09-11T08:49:09.905289+0000","last_unstale":"2024-09-11T08:49:37.090785+0000","last_undegraded":"2024-09-11T08:49:37.090785+0000","last_fullsized":"2024-09-11T08:49:37.090785+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.259103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.259103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015518800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"200'15","reported_seq":247,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087740+0000","last_change":"2024-09-11T08:46:01.993547+0000","last_active":"2024-09-11T08:49:37.087740+0000","last_peered":"2024-09-11T08:49:37.087740+0000","last_clean":"2024-09-11T08:49:37.087740+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:49:37.087740+0000","last_undegraded":"2024-09-11T08:49:37.087740+0000","last_fullsized":"2024-09-11T08:49:37.087740+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:45:06.657912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:06.657912+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023744109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":221,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091424+0000","last_change":"2024-09-11T08:47:48.373407+0000","last_active":"2024-09-11T08:49:37.091424+0000","last_peered":"2024-09-11T08:49:37.091424+0000","last_clean":"2024-09-11T08:49:37.091424+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:49:37.091424+0000","last_undegraded":"2024-09-11T08:49:37.091424+0000","last_fullsized":"2024-09-11T08:49:37.091424+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:48.373336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:48.373336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001705364,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":217,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090930+0000","last_change":"2024-09-11T08:49:27.342962+0000","last_active":"2024-09-11T08:49:37.090930+0000","last_peered":"2024-09-11T08:49:37.090930+0000","last_clean":"2024-09-11T08:49:37.090930+0000","last_became_active":"2024-09-11T08:49:09.910548+0000","last_became_peered":"2024-09-11T08:49:09.910548+0000","last_unstale":"2024-09-11T08:49:37.090930+0000","last_undegraded":"2024-09-11T08:49:37.090930+0000","last_fullsized":"2024-09-11T08:49:37.090930+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:27.342908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:49:27.342908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:27.754446+0000","scrub_duration":0.0020004860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.087317+0000","last_change":"2024-09-11T08:49:09.904856+0000","last_active":"2024-09-11T08:49:37.087317+0000","last_peered":"2024-09-11T08:49:37.087317+0000","last_clean":"2024-09-11T08:49:37.087317+0000","last_became_active":"2024-09-11T08:49:09.904622+0000","last_became_peered":"2024-09-11T08:49:09.904622+0000","last_unstale":"2024-09-11T08:49:37.087317+0000","last_undegraded":"2024-09-11T08:49:37.087317+0000","last_fullsized":"2024-09-11T08:49:37.087317+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:18.685188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.685188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"200'19","reported_seq":230,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090987+0000","last_change":"2024-09-11T08:48:18.890324+0000","last_active":"2024-09-11T08:49:37.090987+0000","last_peered":"2024-09-11T08:49:37.090987+0000","last_clean":"2024-09-11T08:49:37.090987+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:49:37.090987+0000","last_undegraded":"2024-09-11T08:49:37.090987+0000","last_fullsized":"2024-09-11T08:49:37.090987+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:18.890233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.890233+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070086059999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":221,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.086977+0000","last_change":"2024-09-11T08:49:33.209392+0000","last_active":"2024-09-11T08:49:37.086977+0000","last_peered":"2024-09-11T08:49:37.086977+0000","last_clean":"2024-09-11T08:49:37.086977+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:49:37.086977+0000","last_undegraded":"2024-09-11T08:49:37.086977+0000","last_fullsized":"2024-09-11T08:49:37.086977+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:33.209345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:33.209345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:45.794127+0000","scrub_duration":0.0052610230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":220,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.089421+0000","last_change":"2024-09-11T08:48:18.958742+0000","last_active":"2024-09-11T08:49:37.089421+0000","last_peered":"2024-09-11T08:49:37.089421+0000","last_clean":"2024-09-11T08:49:37.089421+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:49:37.089421+0000","last_undegraded":"2024-09-11T08:49:37.089421+0000","last_fullsized":"2024-09-11T08:49:37.089421+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:18.958609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.958609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:43.228394+0000","scrub_duration":0.001704777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":225,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091042+0000","last_change":"2024-09-11T08:49:09.905933+0000","last_active":"2024-09-11T08:49:37.091042+0000","last_peered":"2024-09-11T08:49:37.091042+0000","last_clean":"2024-09-11T08:49:37.091042+0000","last_became_active":"2024-09-11T08:49:09.905634+0000","last_became_peered":"2024-09-11T08:49:09.905634+0000","last_unstale":"2024-09-11T08:49:37.091042+0000","last_undegraded":"2024-09-11T08:49:37.091042+0000","last_fullsized":"2024-09-11T08:49:37.091042+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.807331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.807331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013867529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"200'10","reported_seq":255,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091392+0000","last_change":"2024-09-11T08:48:32.020078+0000","last_active":"2024-09-11T08:49:37.091392+0000","last_peered":"2024-09-11T08:49:37.091392+0000","last_clean":"2024-09-11T08:49:37.091392+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:49:37.091392+0000","last_undegraded":"2024-09-11T08:49:37.091392+0000","last_fullsized":"2024-09-11T08:49:37.091392+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'6","last_scrub_stamp":"2024-09-11T08:48:32.019991+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:48:32.019991+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010394439,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":33,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.088311+0000","last_change":"2024-09-11T08:49:09.919241+0000","last_active":"2024-09-11T08:49:37.088311+0000","last_peered":"2024-09-11T08:49:37.088311+0000","last_clean":"2024-09-11T08:49:37.088311+0000","last_became_active":"2024-09-11T08:49:09.907483+0000","last_became_peered":"2024-09-11T08:49:09.907483+0000","last_unstale":"2024-09-11T08:49:37.088311+0000","last_undegraded":"2024-09-11T08:49:37.088311+0000","last_fullsized":"2024-09-11T08:49:37.088311+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:21.481634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.481634+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":71,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090754+0000","last_change":"2024-09-11T08:48:29.801804+0000","last_active":"2024-09-11T08:49:37.090754+0000","last_peered":"2024-09-11T08:49:37.090754+0000","last_clean":"2024-09-11T08:49:37.090754+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:49:37.090754+0000","last_undegraded":"2024-09-11T08:49:37.090754+0000","last_fullsized":"2024-09-11T08:49:37.090754+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:29.801722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:29.801722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:39.014794+0000","scrub_duration":0.0013425760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":225,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091142+0000","last_change":"2024-09-11T08:49:30.788379+0000","last_active":"2024-09-11T08:49:37.091142+0000","last_peered":"2024-09-11T08:49:37.091142+0000","last_clean":"2024-09-11T08:49:37.091142+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:49:37.091142+0000","last_undegraded":"2024-09-11T08:49:37.091142+0000","last_fullsized":"2024-09-11T08:49:37.091142+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:30.788303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:30.788303+0000","last_clean_scrub_stamp":"2024-09-11T08:49:30.788303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:37.198202+0000","scrub_duration":0.0038913020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":225,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091512+0000","last_change":"2024-09-11T08:49:29.487522+0000","last_active":"2024-09-11T08:49:37.091512+0000","last_peered":"2024-09-11T08:49:37.091512+0000","last_clean":"2024-09-11T08:49:37.091512+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:49:37.091512+0000","last_undegraded":"2024-09-11T08:49:37.091512+0000","last_fullsized":"2024-09-11T08:49:37.091512+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:29.487391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:29.487391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:53.969768+0000","scrub_duration":0.0091739200000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":221,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.091563+0000","last_change":"2024-09-11T08:49:09.919371+0000","last_active":"2024-09-11T08:49:37.091563+0000","last_peered":"2024-09-11T08:49:37.091563+0000","last_clean":"2024-09-11T08:49:37.091563+0000","last_became_active":"2024-09-11T08:49:09.918992+0000","last_became_peered":"2024-09-11T08:49:09.918992+0000","last_unstale":"2024-09-11T08:49:37.091563+0000","last_undegraded":"2024-09-11T08:49:37.091563+0000","last_fullsized":"2024-09-11T08:49:37.091563+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:40.033594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:48:40.033594+0000","last_clean_scrub_stamp":"2024-09-11T08:48:40.033594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:56.836029+0000","scrub_duration":0.0022129379999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"200'19","reported_seq":95,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090743+0000","last_change":"2024-09-11T08:48:20.841324+0000","last_active":"2024-09-11T08:49:37.090743+0000","last_peered":"2024-09-11T08:49:37.090743+0000","last_clean":"2024-09-11T08:49:37.090743+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:49:37.090743+0000","last_undegraded":"2024-09-11T08:49:37.090743+0000","last_fullsized":"2024-09-11T08:49:37.090743+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:20.841243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.841243+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:44.045269+0000","scrub_duration":0.0096371630000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":21,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T08:49:37.090326+0000","last_change":"2024-09-11T08:49:09.905224+0000","last_active":"2024-09-11T08:49:37.090326+0000","last_peered":"2024-09-11T08:49:37.090326+0000","last_clean":"2024-09-11T08:49:37.090326+0000","last_became_active":"2024-09-11T08:49:09.904721+0000","last_became_peered":"2024-09-11T08:49:09.904721+0000","last_unstale":"2024-09-11T08:49:37.090326+0000","last_undegraded":"2024-09-11T08:49:37.090326+0000","last_fullsized":"2024-09-11T08:49:37.090326+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:32.555890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:32.555890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":3847421952,"num_objects":58707,"num_object_clones":0,"num_object_copies":176121,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58707,"num_write_kb":3757248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11542659072,"data_stored":11542659072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58707,"ondisk_log_size":58707,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1119,"num_read_kb":876,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":3234,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":12426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":751,"ondisk_log_size":751,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":653,"num_read_kb":648,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299530,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2626004,"kb_used_data":2328652,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91119148,"statfs":{"total":95995035648,"available":93306007552,"internally_reserved":0,"allocated":2384539648,"data_stored":2384170245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":1.2470000000000001,"15min":0.84499999999999997},"min":{"1min":0.23200000000000001,"5min":0.22,"15min":0.22},"max":{"1min":3.7919999999999998,"5min":32.171999999999997,"15min":32.171999999999997},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.411,"15min":0.93200000000000005},"min":{"1min":0.26700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":3.1829999999999998,"5min":32.149999999999999,"15min":32.149999999999999},"last":0.36499999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:48:44 2024","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.5009999999999999,"15min":0.98199999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":4.069,"5min":33.692999999999998,"15min":33.692999999999998},"last":1.238},{"interface":"front","average":{"1min":1.431,"5min":1.4099999999999999,"15min":0.90600000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23300000000000001},"max":{"1min":5.2030000000000003,"5min":35.591000000000001,"15min":35.591000000000001},"last":0.42299999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:48:44 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0680000000000001,"15min":0.78800000000000003},"min":{"1min":0.29799999999999999,"5min":0.249,"15min":0.249},"max":{"1min":2.5259999999999998,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0840000000000001,"15min":0.755},"min":{"1min":0.36699999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":3.8380000000000001,"5min":5.7990000000000004,"15min":5.7990000000000004},"last":1.2889999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:48:44 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.0369999999999999,"15min":0.82199999999999995},"min":{"1min":0.27200000000000002,"5min":0.25,"15min":0.25},"max":{"1min":3.9060000000000001,"5min":6.0259999999999998,"15min":7.8609999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.119,"15min":0.90200000000000002},"min":{"1min":0.41399999999999998,"5min":0.27700000000000002,"15min":0.27400000000000002},"max":{"1min":3.7669999999999999,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":1.702}]},{"osd":4,"last update":"Wed Sep 11 08:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.54900000000000004,"15min":0.48199999999999998},"min":{"1min":0.27100000000000002,"5min":0.24099999999999999,"15min":0.23400000000000001},"max":{"1min":1.7669999999999999,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.68000000000000005,"15min":0.52100000000000002},"min":{"1min":0.312,"5min":0.216,"15min":0.216},"max":{"1min":2.9359999999999999,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.76900000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:48:44 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":0.72099999999999997,"15min":0.55100000000000005},"min":{"1min":0.30299999999999999,"5min":0.255,"15min":0.23300000000000001},"max":{"1min":5.1059999999999999,"5min":5.1059999999999999,"15min":5.1059999999999999},"last":1.4850000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.68899999999999995,"15min":0.56599999999999995},"min":{"1min":0.308,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.0299999999999998,"5min":3.093,"15min":3.093},"last":0.46200000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.67600000000000005,"15min":0.57999999999999996},"min":{"1min":0.32000000000000001,"5min":0.317,"15min":0.315},"max":{"1min":2.7749999999999999,"5min":2.8820000000000001,"15min":3.5979999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":1.175,"5min":0.73699999999999999,"15min":0.59299999999999997},"min":{"1min":0.376,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.9700000000000002,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.82199999999999995}]}]},{"osd":6,"up_from":45,"seq":193273528463,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":999300,"kb_used_data":697784,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":92745852,"statfs":{"total":95995035648,"available":94971752448,"internally_reserved":0,"allocated":714530816,"data_stored":714181761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:49:26 2024","interfaces":[{"interface":"back","average":{"1min":2.3690000000000002,"5min":1.4710000000000001,"15min":1.111},"min":{"1min":0.252,"5min":0.217,"15min":0.217},"max":{"1min":27.908000000000001,"5min":27.908000000000001,"15min":28.302},"last":1.131},{"interface":"front","average":{"1min":2.738,"5min":1.349,"15min":1.073},"min":{"1min":0.378,"5min":0.24299999999999999,"15min":0.23300000000000001},"max":{"1min":27.771000000000001,"5min":27.771000000000001,"15min":28.314},"last":0.435}]},{"osd":1,"last update":"Wed Sep 11 08:49:26 2024","interfaces":[{"interface":"back","average":{"1min":2.29,"5min":1.7749999999999999,"15min":1.0269999999999999},"min":{"1min":0.307,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":27.984000000000002,"5min":27.984000000000002,"15min":27.984000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":2.1709999999999998,"5min":1.6060000000000001,"15min":0.97199999999999998},"min":{"1min":0.28000000000000003,"5min":0.25700000000000001,"15min":0.253},"max":{"1min":28.106000000000002,"5min":28.106000000000002,"15min":28.106000000000002},"last":0.67800000000000005}]},{"osd":2,"last update":"Wed Sep 11 08:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":1.0269999999999999,"15min":0.77000000000000002},"min":{"1min":0.28999999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1399999999999999,"5min":14.313000000000001,"15min":14.313000000000001},"last":0.747},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.998,"15min":0.77900000000000003},"min":{"1min":0.27700000000000002,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":2.847,"5min":3.7240000000000002,"15min":6.5979999999999999},"last":1.2450000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:49:26 2024","interfaces":[{"interface":"back","average":{"1min":2.1930000000000001,"5min":1.5669999999999999,"15min":1.038},"min":{"1min":0.41299999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":27.795000000000002,"5min":27.795000000000002,"15min":27.795000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":2.3929999999999998,"5min":1.54,"15min":1.0309999999999999},"min":{"1min":0.38900000000000001,"5min":0.28599999999999998,"15min":0.224},"max":{"1min":27.841999999999999,"5min":27.841999999999999,"15min":27.841999999999999},"last":4.1379999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.58599999999999997,"15min":0.48099999999999998},"min":{"1min":0.26200000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.3160000000000001,"5min":3.544,"15min":3.544},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.60099999999999998,"15min":0.47699999999999998},"min":{"1min":0.29499999999999998,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":0.65800000000000003,"5min":4.0209999999999999,"15min":4.0209999999999999},"last":0.33500000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.76700000000000002,"15min":0.56999999999999995},"min":{"1min":0.318,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":2.117,"5min":8.6829999999999998,"15min":8.6829999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.72399999999999998,"15min":0.54600000000000004},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.28100000000000003},"max":{"1min":1.012,"5min":5.0819999999999999,"15min":5.0819999999999999},"last":0.29999999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:48:39 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.74099999999999999,"15min":0.55300000000000005},"min":{"1min":0.313,"5min":0.25900000000000001,"15min":0.247},"max":{"1min":3.355,"5min":7.327,"15min":7.327},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.1659999999999999,"5min":0.68200000000000005,"15min":0.56200000000000006},"min":{"1min":0.36699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.4500000000000002,"5min":5.4500000000000002,"15min":5.4500000000000002},"last":0.77600000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757395,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2404876,"kb_used_data":2095044,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91340276,"statfs":{"total":95995035648,"available":93532442624,"internally_reserved":0,"allocated":2145325056,"data_stored":2144963824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,9,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:49:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.93000000000000005,"15min":0.77000000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.20599999999999999},"max":{"1min":2.7599999999999998,"5min":3.266,"15min":7.3899999999999997},"last":0.35899999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.2150000000000001,"15min":0.90700000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23100000000000001},"max":{"1min":5.569,"5min":8.0410000000000004,"15min":8.0410000000000004},"last":0.92200000000000004}]},{"osd":1,"last update":"Wed Sep 11 08:49:04 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.23,"15min":0.88300000000000001},"min":{"1min":0.28699999999999998,"5min":0.23799999999999999,"15min":0.22800000000000001},"max":{"1min":2.7749999999999999,"5min":8,"15min":8},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.119,"15min":0.81899999999999995},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.9780000000000002,"5min":3.9780000000000002,"15min":6.6719999999999997},"last":0.40200000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:49:04 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.171,"15min":0.85099999999999998},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.23799999999999999},"max":{"1min":5.5339999999999998,"5min":5.5339999999999998,"15min":5.5339999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":1.1539999999999999,"15min":0.81399999999999995},"min":{"1min":0.34100000000000003,"5min":0.28699999999999998,"15min":0.26400000000000001},"max":{"1min":3.5990000000000002,"5min":8.0640000000000001,"15min":8.0640000000000001},"last":1.024}]},{"osd":3,"last update":"Wed Sep 11 08:49:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.99199999999999999,"15min":0.80700000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.7309999999999999,"5min":4.3940000000000001,"15min":6.0460000000000003},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.319,"5min":1.2909999999999999,"15min":0.95099999999999996},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.27400000000000002},"max":{"1min":8.0899999999999999,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.94899999999999995}]},{"osd":4,"last update":"Wed Sep 11 08:49:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70599999999999996,"15min":0.501},"min":{"1min":0.26400000000000001,"5min":0.246,"15min":0.20699999999999999},"max":{"1min":3.5129999999999999,"5min":4.7110000000000003,"15min":4.7110000000000003},"last":3.2109999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.621,"15min":0.48899999999999999},"min":{"1min":0.29099999999999998,"5min":0.23699999999999999,"15min":0.20499999999999999},"max":{"1min":1.7749999999999999,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":3.5419999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.78700000000000003,"15min":0.55600000000000005},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.23599999999999999},"max":{"1min":2.802,"5min":6.4530000000000003,"15min":6.4530000000000003},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.76500000000000001,"15min":0.55100000000000005},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.251},"max":{"1min":3.1850000000000001,"5min":8.1920000000000002,"15min":8.1920000000000002},"last":0.98399999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:49:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.6839999999999999,"15min":0.88400000000000001},"min":{"1min":0.33000000000000002,"5min":0.29999999999999999,"15min":0.27800000000000002},"max":{"1min":9.4009999999999998,"5min":88.513000000000005,"15min":88.513000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.667,"5min":1.7450000000000001,"15min":0.92900000000000005},"min":{"1min":0.40200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":9.7089999999999996,"5min":86.963999999999999,"15min":86.963999999999999},"last":3.5800000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986327,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1010824,"kb_used_data":700988,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92734328,"statfs":{"total":95995035648,"available":94959951872,"internally_reserved":0,"allocated":717811712,"data_stored":717457281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:48:39 2024","interfaces":[{"interface":"back","average":{"1min":4.6319999999999997,"5min":2.1469999999999998,"15min":1.5669999999999999},"min":{"1min":0.23100000000000001,"5min":0.21199999999999999,"15min":0.20399999999999999},"max":{"1min":44.140999999999998,"5min":48.451999999999998,"15min":89.510000000000005},"last":0.72199999999999998},{"interface":"front","average":{"1min":4.5,"5min":2.4540000000000002,"15min":1.665},"min":{"1min":0.246,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":44.450000000000003,"5min":49.115000000000002,"15min":89.765000000000001},"last":0.26600000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:48:39 2024","interfaces":[{"interface":"back","average":{"1min":3.54,"5min":1.98,"15min":1.083},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":44.152999999999999,"5min":44.152999999999999,"15min":44.152999999999999},"last":0.28199999999999997},{"interface":"front","average":{"1min":3.46,"5min":1.8999999999999999,"15min":1.0649999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":45.048000000000002,"5min":45.048000000000002,"15min":45.048000000000002},"last":1.738}]},{"osd":2,"last update":"Wed Sep 11 08:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":1.1060000000000001,"15min":0.80700000000000005},"min":{"1min":0.254,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.9810000000000001,"5min":17.303000000000001,"15min":17.303000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0600000000000001,"15min":0.81999999999999995},"min":{"1min":0.29899999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.7469999999999999,"5min":20.702999999999999,"15min":20.702999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:48:39 2024","interfaces":[{"interface":"back","average":{"1min":3.657,"5min":2.2669999999999999,"15min":1.6699999999999999},"min":{"1min":0.27400000000000002,"5min":0.255,"15min":0.255},"max":{"1min":45.268000000000001,"5min":48.194000000000003,"15min":89.881},"last":0.36199999999999999},{"interface":"front","average":{"1min":3.532,"5min":2.2589999999999999,"15min":1.6279999999999999},"min":{"1min":0.29299999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":44.234000000000002,"5min":48.840000000000003,"15min":89.715000000000003},"last":0.77900000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:49:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.70499999999999996,"15min":0.51800000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.5449999999999999,"5min":3.5449999999999999,"15min":3.5449999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.65700000000000003,"15min":0.504},"min":{"1min":0.30599999999999999,"5min":0.26100000000000001,"15min":0.23699999999999999},"max":{"1min":0.82799999999999996,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.59799999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.61899999999999999,"15min":0.504},"min":{"1min":0.28100000000000003,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":2.9359999999999999,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69699999999999995,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.221,"15min":0.221},"max":{"1min":3.1019999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":0.38500000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.622,"15min":0.53500000000000003},"min":{"1min":0.308,"5min":0.252,"15min":0.252},"max":{"1min":1.7290000000000001,"5min":3.339,"15min":3.339},"last":0.749},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.753,"15min":0.57399999999999995},"min":{"1min":0.33400000000000002,"5min":0.29599999999999999,"15min":0.28799999999999998},"max":{"1min":2.6299999999999999,"5min":5.1970000000000001,"15min":5.1970000000000001},"last":0.41799999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149853,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2414948,"kb_used_data":2113436,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91330204,"statfs":{"total":95995035648,"available":93522128896,"internally_reserved":0,"allocated":2164158464,"data_stored":2163773345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.4810000000000001,"15min":1.0289999999999999},"min":{"1min":0.221,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":1.9399999999999999,"5min":24.280000000000001,"15min":24.280000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":1.7130000000000001,"15min":1.2210000000000001},"min":{"1min":0.20100000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":2.7309999999999999,"5min":22.225000000000001,"15min":22.225000000000001},"last":0.53900000000000003}]},{"osd":1,"last update":"Wed Sep 11 08:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":2.7170000000000001,"15min":1.5369999999999999},"min":{"1min":0.247,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":3.4780000000000002,"5min":115.601,"15min":115.601},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":2.5910000000000002,"15min":1.466},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.17999999999999999},"max":{"1min":5.968,"5min":115.474,"15min":115.474},"last":0.59899999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":1.452,"15min":1.0940000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23200000000000001},"max":{"1min":6.077,"5min":8.2699999999999996,"15min":8.9510000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":1.236,"15min":1.0269999999999999},"min":{"1min":0.22700000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.8240000000000001,"5min":10.170999999999999,"15min":10.695},"last":0.497}]},{"osd":4,"last update":"Wed Sep 11 08:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":1.4379999999999999,"15min":1.0740000000000001},"min":{"1min":0.33700000000000002,"5min":0.28599999999999998,"15min":0.26700000000000002},"max":{"1min":2.3580000000000001,"5min":8.3279999999999994,"15min":8.9969999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":1.1379999999999999,"15min":0.91700000000000004},"min":{"1min":0.35799999999999998,"5min":0.318,"15min":0.27700000000000002},"max":{"1min":1.6080000000000001,"5min":7.234,"15min":7.234},"last":0.61599999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:48:56 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.49,"15min":1.0449999999999999},"min":{"1min":0.434,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":6.0119999999999996,"5min":8.3599999999999994,"15min":11.526999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.3360000000000001,"15min":0.93000000000000005},"min":{"1min":0.33200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.492,"5min":7.1689999999999996,"15min":7.1689999999999996},"last":1.2330000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:49:28 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.4299999999999999,"15min":1.1080000000000001},"min":{"1min":0.35899999999999999,"5min":0.311,"15min":0.29999999999999999},"max":{"1min":6.1100000000000003,"5min":8.4160000000000004,"15min":11.494999999999999},"last":2.6440000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":1.3819999999999999,"15min":1.071},"min":{"1min":0.36499999999999999,"5min":0.314,"15min":0.311},"max":{"1min":2.6400000000000001,"5min":7.7249999999999996,"15min":7.7249999999999996},"last":2.871}]},{"osd":7,"last update":"Wed Sep 11 08:48:45 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.6679999999999999,"15min":1.121},"min":{"1min":0.49299999999999999,"5min":0.36399999999999999,"15min":0.35299999999999998},"max":{"1min":6.133,"5min":9.1539999999999999,"15min":11.561},"last":3.73},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.399,"15min":0.97799999999999998},"min":{"1min":0.377,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":4.2729999999999997,"5min":6.6200000000000001,"15min":6.6200000000000001},"last":3.6150000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411489,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1245572,"kb_used_data":944056,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":92499580,"statfs":{"total":95995035648,"available":94719569920,"internally_reserved":0,"allocated":966713344,"data_stored":966363160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:48:59 2024","interfaces":[{"interface":"back","average":{"1min":3.6659999999999999,"5min":2.7599999999999998,"15min":1.4930000000000001},"min":{"1min":0.214,"5min":0.16500000000000001,"15min":0.159},"max":{"1min":16.606999999999999,"5min":34.713999999999999,"15min":34.713999999999999},"last":0.435},{"interface":"front","average":{"1min":3.4390000000000001,"5min":2.8130000000000002,"15min":1.595},"min":{"1min":0.246,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":16.855,"5min":34.399000000000001,"15min":34.399000000000001},"last":0.22600000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:48:59 2024","interfaces":[{"interface":"back","average":{"1min":6.7919999999999998,"5min":3.2040000000000002,"15min":1.5309999999999999},"min":{"1min":0.29099999999999998,"5min":0.224,"15min":0.19},"max":{"1min":72.867000000000004,"5min":72.867000000000004,"15min":72.867000000000004},"last":0.97699999999999998},{"interface":"front","average":{"1min":7.1219999999999999,"5min":3.4319999999999999,"15min":1.6259999999999999},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":67.799000000000007,"5min":67.799000000000007,"15min":67.799000000000007},"last":1.881}]},{"osd":3,"last update":"Wed Sep 11 08:49:12 2024","interfaces":[{"interface":"back","average":{"1min":2.706,"5min":2.379,"15min":1.4179999999999999},"min":{"1min":0.26100000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":16.353999999999999,"5min":34.75,"15min":34.75},"last":0.57699999999999996},{"interface":"front","average":{"1min":2.282,"5min":2.2160000000000002,"15min":1.359},"min":{"1min":0.29499999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":16.138999999999999,"5min":32.802,"15min":32.802},"last":1.843}]},{"osd":4,"last update":"Wed Sep 11 08:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":1.107,"15min":0.93500000000000005},"min":{"1min":0.33900000000000002,"5min":0.255,"15min":0.255},"max":{"1min":3.0139999999999998,"5min":8.0820000000000007,"15min":8.0820000000000007},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":1.0780000000000001,"15min":0.88900000000000001},"min":{"1min":0.33100000000000002,"5min":0.29999999999999999,"15min":0.28399999999999997},"max":{"1min":1.3759999999999999,"5min":6.9320000000000004,"15min":6.9320000000000004},"last":0.32400000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:48:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.2370000000000001,"15min":0.91000000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.7080000000000002,"5min":6.9550000000000001,"15min":6.9550000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.89700000000000002,"5min":1.0049999999999999,"15min":0.81200000000000006},"min":{"1min":0.31900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.859,"5min":4.2000000000000002,"15min":5.9530000000000003},"last":0.81599999999999995}]},{"osd":6,"last update":"Wed Sep 11 08:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":1.25,"15min":0.99199999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.9950000000000001,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.628},{"interface":"front","average":{"1min":0.877,"5min":1.1020000000000001,"15min":0.89500000000000002},"min":{"1min":0.35299999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.0790000000000002,"5min":6.9779999999999998,"15min":6.9779999999999998},"last":0.47399999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.96099999999999997,"15min":0.85099999999999998},"min":{"1min":0.35899999999999999,"5min":0.32200000000000001,"15min":0.30399999999999999},"max":{"1min":1.361,"5min":3.2709999999999999,"15min":5.9909999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.3009999999999999,"15min":1.032},"min":{"1min":0.39500000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.3730000000000002,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.94999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575014,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2647992,"kb_used_data":2346476,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91097160,"statfs":{"total":95995035648,"available":93283491840,"internally_reserved":0,"allocated":2402791424,"data_stored":2402389657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":1.1599999999999999,"15min":1.3819999999999999},"min":{"1min":0.17699999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":3.9020000000000001,"5min":4.8319999999999999,"15min":117.01600000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":1.391,"15min":1.6779999999999999},"min":{"1min":0.26500000000000001,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":1.8129999999999999,"5min":8.1549999999999994,"15min":117.62},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed Sep 11 08:48:56 2024","interfaces":[{"interface":"back","average":{"1min":1.968,"5min":1.4419999999999999,"15min":1.0780000000000001},"min":{"1min":0.248,"5min":0.191,"15min":0.191},"max":{"1min":8.1329999999999991,"5min":8.1329999999999991,"15min":20.628},"last":0.58999999999999997},{"interface":"front","average":{"1min":2.3250000000000002,"5min":1.5449999999999999,"15min":1.0720000000000001},"min":{"1min":0.24299999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":9.7200000000000006,"5min":9.7200000000000006,"15min":20.148},"last":0.45800000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:49:12 2024","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.4830000000000001,"15min":1.3640000000000001},"min":{"1min":0.27200000000000002,"5min":0.191,"15min":0.191},"max":{"1min":4.2859999999999996,"5min":8.3550000000000004,"15min":116.98},"last":0.36499999999999999},{"interface":"front","average":{"1min":1.502,"5min":1.4890000000000001,"15min":1.458},"min":{"1min":0.246,"5min":0.24099999999999999,"15min":0.22},"max":{"1min":3.8690000000000002,"5min":7.415,"15min":117.56699999999999},"last":0.69699999999999995}]},{"osd":4,"last update":"Wed Sep 11 08:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":0.89800000000000002},"min":{"1min":0.34999999999999998,"5min":0.27500000000000002,"15min":0.26200000000000001},"max":{"1min":1.845,"5min":4.6580000000000004,"15min":11.19},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":1.0649999999999999,"15min":0.83999999999999997},"min":{"1min":0.35699999999999998,"5min":0.30199999999999999,"15min":0.251},"max":{"1min":1.466,"5min":5.056,"15min":7.7169999999999996},"last":0.441}]},{"osd":5,"last update":"Wed Sep 11 08:49:00 2024","interfaces":[{"interface":"back","average":{"1min":1.637,"5min":1.361,"15min":1.395},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":8.0999999999999996,"5min":8.0999999999999996,"15min":103.337},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.407,"5min":1.3080000000000001,"15min":1.345},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.27400000000000002},"max":{"1min":5.9020000000000001,"5min":6.3799999999999999,"15min":103.163},"last":0.54100000000000004}]},{"osd":6,"last update":"Wed Sep 11 08:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":1.1040000000000001,"15min":0.84999999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.29399999999999998},"max":{"1min":3.0129999999999999,"5min":3.907,"15min":8.9149999999999991},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.871,"5min":1.27,"15min":1.0089999999999999},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.31},"max":{"1min":3.415,"5min":4.7080000000000002,"15min":9.5199999999999996},"last":0.71899999999999997}]},{"osd":7,"last update":"Wed Sep 11 08:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.1319999999999999,"15min":0.95099999999999996},"min":{"1min":0.41799999999999998,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":1.956,"5min":4.593,"15min":11.315},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":1.1919999999999999,"15min":0.90800000000000003},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.315},"max":{"1min":1.1739999999999999,"5min":10.266,"15min":10.266},"last":0.66300000000000003}]}]},{"osd":0,"up_from":0,"seq":34359738538,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2408710144,"data_stored":2408710144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":949551104,"data_stored":949551104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2190278656,"data_stored":2190278656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":722862080,"data_stored":722862080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2154758144,"data_stored":2154758144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":715063296,"data_stored":715063296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2401435648,"data_stored":2401435648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:49:48.507 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool set unique_pool_2 pgp_num 16 2024-09-11T08:49:48.537 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:48.566 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:48.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:48.567 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:48.617 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:48.667 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:48.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:48.711 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:48.712 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:48.724 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 34 16 69750 69734 128.17 88.1875 0.00838111 0.00779685 2024-09-11T08:49:48.813 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:48.852 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:48.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:48.853 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:48.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:48.991 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:48.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:48.992 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:49.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:49.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:49.133 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:49.133 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:49.234 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:49.268 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:49.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:49.269 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:49.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:49 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:49:49.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:49 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:49:49.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:49 smithi092 ceph-mon[29647]: osdmap e221: 8 total, 8 up, 7 in 2024-09-11T08:49:49.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:49 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:49:49.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:49 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:49:49.283 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:49 smithi092 ceph-mon[29647]: osdmap e222: 8 total, 8 up, 7 in 2024-09-11T08:49:49.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:49 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:49:49.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:49 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:49:49.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:49 smithi092 ceph-mon[41114]: osdmap e221: 8 total, 8 up, 7 in 2024-09-11T08:49:49.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:49 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:49:49.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:49 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:49:49.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:49 smithi092 ceph-mon[41114]: osdmap e222: 8 total, 8 up, 7 in 2024-09-11T08:49:49.370 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:49.406 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:49.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:49.407 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:49.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:49 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:49:49.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:49 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:49:49.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:49 smithi145 ceph-mon[33384]: osdmap e221: 8 total, 8 up, 7 in 2024-09-11T08:49:49.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:49 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:49:49.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:49 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:49:49.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:49 smithi145 ceph-mon[33384]: osdmap e222: 8 total, 8 up, 7 in 2024-09-11T08:49:49.508 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:49.532 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:49.541 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:49.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:49.542 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:49.643 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:49.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:49.681 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:49.681 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:49.724 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 35 16 71107 71091 126.931 84.8125 0.00957155 0.00787359 2024-09-11T08:49:49.781 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:49.820 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:49.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:49.821 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:49.922 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:49.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:49.962 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:49.963 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:50.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:50.101 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:50.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:50.102 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:50.112 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:49:50.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:50.233 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:50.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:50.234 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:50.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:50 smithi092 ceph-mon[41114]: pgmap v473: 148 pgs: 148 active+clean; 4.1 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 852 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-11T08:49:50.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:50 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/21108314' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:49:50.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:50 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:49:50.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:50 smithi092 ceph-mon[29647]: pgmap v473: 148 pgs: 148 active+clean; 4.1 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 852 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-11T08:49:50.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:50 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/21108314' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:49:50.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:50 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:49:50.335 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:50.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:50.369 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:50.369 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:50.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:50 smithi145 ceph-mon[33384]: pgmap v473: 148 pgs: 148 active+clean; 4.1 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 852 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-11T08:49:50.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:50 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/21108314' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:49:50.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:50 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:49:50.470 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:50.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:50.502 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:50.502 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:50.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:50.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:50.641 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:50.684 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:50.724 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 36 16 72594 72578 125.986 92.9375 0.0116624 0.00793262 2024-09-11T08:49:50.742 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:50.779 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:50.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:50.781 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:50.882 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:50.924 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:50.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:50.925 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:51.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:51.066 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:51.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:51.067 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:51.168 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:51.204 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:51.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:51.205 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:51.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:51.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:51.345 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:51.345 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:51.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:51 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:49:51.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:51 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:49:51.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:51 smithi145 ceph-mon[33384]: osdmap e223: 8 total, 8 up, 7 in 2024-09-11T08:49:51.432 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:51 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:49:51.432 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:51 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:49:51.432 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:51 smithi092 ceph-mon[41114]: osdmap e223: 8 total, 8 up, 7 in 2024-09-11T08:49:51.433 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:51 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:49:51.433 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:51 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:49:51.433 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:51 smithi092 ceph-mon[29647]: osdmap e223: 8 total, 8 up, 7 in 2024-09-11T08:49:51.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:51.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:51.503 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:51.604 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:51.635 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:51.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:51.636 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:51.707 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:49:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:49:51] "GET /metrics HTTP/1.1" 200 224399 "" "Prometheus/2.43.0" 2024-09-11T08:49:51.724 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 37 16 74067 74051 125.069 92.0625 0.0115676 0.0079913 2024-09-11T08:49:51.738 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:51.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:51.781 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:51.781 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:51.882 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:51.914 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:51.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:51.915 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:52.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:52.045 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:52.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:52.046 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:52.147 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:52.186 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:52.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:52.187 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:52.288 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:52.297 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:52 smithi092 ceph-mon[29647]: pgmap v475: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.7 KiB/s rd, 103 MiB/s wr, 1.66k op/s 2024-09-11T08:49:52.298 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:52 smithi092 ceph-mon[41114]: pgmap v475: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.7 KiB/s rd, 103 MiB/s wr, 1.66k op/s 2024-09-11T08:49:52.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:52.328 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:52.328 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:52.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:52 smithi145 ceph-mon[33384]: pgmap v475: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.7 KiB/s rd, 103 MiB/s wr, 1.66k op/s 2024-09-11T08:49:52.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:52.469 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:52.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:52.470 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:52.571 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:52.607 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:52.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:52.608 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:52.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:52.724 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 38 16 75431 75415 124.021 85.25 0.0167623 0.0080586 2024-09-11T08:49:52.746 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:52.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:52.748 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:52.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:52.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:52.885 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:52.885 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:52.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:49:53.018 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:49:53.019 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:49:53.019 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:49:53.019 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:49:53.019 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:49:53.019 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:49:53.020 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:49:53.020 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:49:53.020 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:49:53.020 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:49:53.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:53.021 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:53.021 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:53.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:53.129 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:49:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:49:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:49:53.159 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:53.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:53.160 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:53.176 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:49:53.245 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:53 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1597128916' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:53.245 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:53 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1597128916' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:53.262 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:53.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:53.333 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:53.333 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:53.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:53 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1597128916' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:49:53.434 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:53.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:53.468 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:53.468 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:53.569 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:53.606 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:53.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:53.608 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:53.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:53.725 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 39 16 76832 76816 123.084 87.5625 0.012663 0.00811944 2024-09-11T08:49:53.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:53.744 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:53.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:53.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:53.881 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:53.982 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:54.018 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:54.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:54.019 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:54.119 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:54.155 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:54.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:54.156 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:54.256 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:54.298 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:54.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:54.299 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:54.400 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:54.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:54 smithi145 ceph-mon[33384]: pgmap v476: 148 pgs: 148 active+clean; 4.5 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 852 B/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-11T08:49:54.434 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:54.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:54.435 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:54.465 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:54 smithi092 ceph-mon[29647]: pgmap v476: 148 pgs: 148 active+clean; 4.5 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 852 B/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-11T08:49:54.466 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:54 smithi092 ceph-mon[41114]: pgmap v476: 148 pgs: 148 active+clean; 4.5 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 852 B/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-11T08:49:54.536 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:54.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:54.572 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:54.572 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:54.673 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:54.709 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:54.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:54.710 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:54.725 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:49:54.726910+0000 min lat: 0.00097685 max lat: 0.147323 avg lat: 0.00817185 2024-09-11T08:49:54.725 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:49:54.725 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 40 16 78303 78287 122.305 91.9375 0.00718553 0.00817185 2024-09-11T08:49:54.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:54.847 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:54.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:54.848 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:54.949 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:54.985 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:54.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:54.986 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:55.087 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:55.122 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:55.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:55.123 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:55.224 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:55.262 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:55.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:55.263 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:55.364 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:55.403 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:55.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:55.404 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:55.505 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:55.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:55.540 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:55.540 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:55.641 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:55.674 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:55.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:55.675 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:55.726 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 41 16 79771 79755 121.559 91.75 0.00634947 0.00822202 2024-09-11T08:49:55.775 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:55.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:55.811 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:55.811 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:55.857 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:49:55.892 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:49:55.892 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:49:55.892 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:49:55.892 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:49:55.892 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:49:55.912 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:55.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:55.944 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:55.944 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:56.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:56.080 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:56.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:56.081 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:56.182 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:56.214 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:56.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:56.215 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:56.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:56.348 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:56.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:56.349 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:56.450 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:56.486 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:56.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:56.487 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:56.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:56 smithi092 ceph-mon[29647]: pgmap v477: 148 pgs: 148 active+clean; 4.5 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 717 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-11T08:49:56.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:56 smithi092 ceph-mon[41114]: pgmap v477: 148 pgs: 148 active+clean; 4.5 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 717 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-11T08:49:56.588 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:56.624 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:56.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:56.625 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:56.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:56 smithi145 ceph-mon[33384]: pgmap v477: 148 pgs: 148 active+clean; 4.5 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 717 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-11T08:49:56.726 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:56.726 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 42 16 81238 81222 120.848 91.6875 0.0135721 0.00827082 2024-09-11T08:49:56.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:56.767 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:56.767 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:56.868 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:56.904 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:56.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:56.907 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:56.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:49:57.007 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:57.037 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:57.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:57.038 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:57.139 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:57.155 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-11T08:49:57.162 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:49:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:49:57.170 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:57.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:57.171 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:57.272 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:57.302 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:57.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:57.303 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:57.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:57.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:57.452 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:57.453 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:57.553 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:57.586 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:57.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:57.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:57.688 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:57.726 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 43 16 82614 82598 120.036 86 0.00529732 0.0083263 2024-09-11T08:49:57.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:57.730 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:57.730 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:57.831 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:57.868 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:57.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:57.869 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:57.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:58.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:58.019 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:58.019 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:58.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:49:58.160 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:58.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:58.161 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:49:58.262 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:58.295 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:58.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:58.297 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:58.321 INFO:teuthology.orchestra.run.smithi092.stderr:set pool 9 pgp_num to 16 2024-09-11T08:49:58.398 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:58.434 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:58.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:58.435 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:49:58.536 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:58.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:58 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:58.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:58 smithi092 ceph-mon[29647]: pgmap v478: 148 pgs: 148 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 108 MiB/s wr, 1.73k op/s 2024-09-11T08:49:58.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:58 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1150040892' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-11T08:49:58.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:58 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:58.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:58 smithi092 ceph-mon[41114]: pgmap v478: 148 pgs: 148 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 108 MiB/s wr, 1.73k op/s 2024-09-11T08:49:58.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:58 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1150040892' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-11T08:49:58.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:58.568 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:58.568 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:58.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:58 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:49:58.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:58 smithi145 ceph-mon[33384]: pgmap v478: 148 pgs: 148 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 108 MiB/s wr, 1.73k op/s 2024-09-11T08:49:58.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:58 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1150040892' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-11T08:49:58.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:58.700 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:58.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:58.701 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:58.726 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 44 16 84074 84058 119.381 91.25 0.0126973 0.00837182 2024-09-11T08:49:58.802 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:49:58.843 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:58.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:58.844 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:49:58.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:58.978 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:58.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:58.980 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:59.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:59.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:59.123 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:59.124 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:59.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:49:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:59.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:59.258 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:49:59.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:49:59.388 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:59.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:59.389 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:49:59.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:59 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1150040892' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-11T08:49:59.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:59 smithi145 ceph-mon[33384]: osdmap e224: 8 total, 8 up, 7 in 2024-09-11T08:49:59.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:49:59 smithi145 ceph-mon[33384]: osdmap e225: 8 total, 8 up, 7 in 2024-09-11T08:49:59.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:59 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1150040892' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-11T08:49:59.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:59 smithi092 ceph-mon[29647]: osdmap e224: 8 total, 8 up, 7 in 2024-09-11T08:49:59.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:49:59 smithi092 ceph-mon[29647]: osdmap e225: 8 total, 8 up, 7 in 2024-09-11T08:49:59.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:59 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1150040892' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-11T08:49:59.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:59 smithi092 ceph-mon[41114]: osdmap e224: 8 total, 8 up, 7 in 2024-09-11T08:49:59.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:49:59 smithi092 ceph-mon[41114]: osdmap e225: 8 total, 8 up, 7 in 2024-09-11T08:49:59.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:49:59.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:59.522 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:49:59.523 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:49:59.623 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:49:59.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:59.666 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:59.666 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:49:59.726 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 45 16 85400 85384 118.57 82.875 0.00727156 0.00842914 2024-09-11T08:49:59.767 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:49:59.807 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:59.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:59.808 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:49:59.909 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:49:59.947 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:49:59.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:49:59.950 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:00.051 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:00.078 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:00.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:00.078 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:00.179 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:00.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:00.212 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:00.212 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:00.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:00.346 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:00.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:00.347 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:00.448 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:00.484 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:00.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:00.485 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:00.529 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:00.534 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:00.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:00 smithi092 ceph-mon[29647]: pgmap v481: 148 pgs: 148 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-11T08:50:00.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:00 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3340831155' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:50:00.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:00 smithi092 ceph-mon[29647]: Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set 2024-09-11T08:50:00.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:00 smithi092 ceph-mon[29647]: [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-09-11T08:50:00.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:00 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:50:00.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:50:00.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:00 smithi092 ceph-mon[41114]: pgmap v481: 148 pgs: 148 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-11T08:50:00.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:00 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3340831155' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:50:00.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:00 smithi092 ceph-mon[41114]: Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set 2024-09-11T08:50:00.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:00 smithi092 ceph-mon[41114]: [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-09-11T08:50:00.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:00 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:50:00.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:50:00.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:00.625 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:00.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:00.626 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:00.644 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:00 smithi145 ceph-mon[33384]: pgmap v481: 148 pgs: 148 active+clean; 4.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-11T08:50:00.644 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:00 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3340831155' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:50:00.645 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:00 smithi145 ceph-mon[33384]: Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set 2024-09-11T08:50:00.645 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:00 smithi145 ceph-mon[33384]: [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-09-11T08:50:00.645 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:00 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:50:00.645 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:50:00.727 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 46 16 86722 86706 117.789 82.625 0.00818425 0.00848503 2024-09-11T08:50:00.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:00.761 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:00.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:00.762 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:00.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:50:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:00.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:00.903 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:01.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:01.037 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:01.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:01.038 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:01.139 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:01.176 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:01.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:01.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:01.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:01.319 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:01.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:01.320 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:01.421 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:01.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:01.458 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:01.458 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:01.558 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:01.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:01 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:50:01.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:01 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:50:01.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:01 smithi092 ceph-mon[29647]: osdmap e226: 8 total, 8 up, 7 in 2024-09-11T08:50:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:01 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:50:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:01 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:50:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:01 smithi092 ceph-mon[41114]: osdmap e226: 8 total, 8 up, 7 in 2024-09-11T08:50:01.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:50:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:50:01] "GET /metrics HTTP/1.1" 200 224389 "" "Prometheus/2.43.0" 2024-09-11T08:50:01.593 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:01.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:01.594 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:01.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:01 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:50:01.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:01 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:50:01.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:01 smithi145 ceph-mon[33384]: osdmap e226: 8 total, 8 up, 7 in 2024-09-11T08:50:01.695 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:01.726 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 47 16 88179 88163 117.22 91.0625 0.0143138 0.00852662 2024-09-11T08:50:01.729 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:01.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:01.730 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:01.831 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:01.862 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:01.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:01.864 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:01.964 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:01.999 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:02.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:02.000 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:02.101 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:02.131 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:02.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:02.133 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:02.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:02.270 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:02.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:02.271 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:02.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:02.394 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:02.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:02.395 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:02.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:02.525 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:02.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:02.526 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:02.626 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:02 smithi145 ceph-mon[33384]: pgmap v483: 148 pgs: 148 active+clean; 5.1 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.7 KiB/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-11T08:50:02.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:02 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T08:50:02.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:02 smithi145 ceph-mon[33384]: 4.1c scrub ok 2024-09-11T08:50:02.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:02.664 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:02.664 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:02.727 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 48 15 89595 89580 116.623 88.5625 0.0118373 0.00857043 2024-09-11T08:50:02.765 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:02.800 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:02.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:02.801 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:02.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:02 smithi092 ceph-mon[41114]: pgmap v483: 148 pgs: 148 active+clean; 5.1 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.7 KiB/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-11T08:50:02.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:02 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T08:50:02.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:02 smithi092 ceph-mon[41114]: 4.1c scrub ok 2024-09-11T08:50:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:02 smithi092 ceph-mon[29647]: pgmap v483: 148 pgs: 148 active+clean; 5.1 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.7 KiB/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-11T08:50:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:02 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T08:50:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:02 smithi092 ceph-mon[29647]: 4.1c scrub ok 2024-09-11T08:50:02.902 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:02.937 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:02.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:02.939 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:03.039 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:03.080 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:03.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:03.081 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:03.163 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:50:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:50:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:50:03.181 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:03.225 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:03.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:03.226 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:03.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:03.364 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:03.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:03.365 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:03.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:03.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:03.499 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:03.499 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:03.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:03 smithi092 ceph-mon[29647]: 3.a scrub starts 2024-09-11T08:50:03.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:03 smithi092 ceph-mon[29647]: 3.a scrub ok 2024-09-11T08:50:03.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:03 smithi092 ceph-mon[29647]: 6.6 scrub starts 2024-09-11T08:50:03.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:03 smithi092 ceph-mon[29647]: 6.6 scrub ok 2024-09-11T08:50:03.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:03 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:50:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:03 smithi092 ceph-mon[41114]: 3.a scrub starts 2024-09-11T08:50:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:03 smithi092 ceph-mon[41114]: 3.a scrub ok 2024-09-11T08:50:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:03 smithi092 ceph-mon[41114]: 6.6 scrub starts 2024-09-11T08:50:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:03 smithi092 ceph-mon[41114]: 6.6 scrub ok 2024-09-11T08:50:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:03 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:50:03.601 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:03.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:03.635 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:03.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:03 smithi145 ceph-mon[33384]: 3.a scrub starts 2024-09-11T08:50:03.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:03 smithi145 ceph-mon[33384]: 3.a scrub ok 2024-09-11T08:50:03.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:03 smithi145 ceph-mon[33384]: 6.6 scrub starts 2024-09-11T08:50:03.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:03 smithi145 ceph-mon[33384]: 6.6 scrub ok 2024-09-11T08:50:03.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:03 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:50:03.727 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 49 16 90939 90923 115.956 83.9375 0.00542548 0.00861857 2024-09-11T08:50:03.735 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:03.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:03.757 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:03.758 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:03.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:50:03.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:03.902 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:03.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:03.903 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:04.004 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:04.040 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:04.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:04.041 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:04.141 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:04.180 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:04.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:04.182 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:04.283 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:04.319 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:04.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:04.320 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:04.421 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:04.442 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:04.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:04.443 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:04.544 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:04.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: pgmap v484: 148 pgs: 148 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 72 MiB/s wr, 1.16k op/s 2024-09-11T08:50:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: 4.12 scrub starts 2024-09-11T08:50:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: 4.12 scrub ok 2024-09-11T08:50:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: Adjusting osd_memory_target on smithi145 to 3248M 2024-09-11T08:50:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:50:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: 4.f scrub starts 2024-09-11T08:50:04.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: 4.f scrub ok 2024-09-11T08:50:04.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: 4.9 scrub starts 2024-09-11T08:50:04.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: 4.9 scrub ok 2024-09-11T08:50:04.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[29647]: osdmap e227: 8 total, 8 up, 7 in 2024-09-11T08:50:04.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: pgmap v484: 148 pgs: 148 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 72 MiB/s wr, 1.16k op/s 2024-09-11T08:50:04.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: 4.12 scrub starts 2024-09-11T08:50:04.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: 4.12 scrub ok 2024-09-11T08:50:04.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: Adjusting osd_memory_target on smithi145 to 3248M 2024-09-11T08:50:04.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:50:04.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: 4.f scrub starts 2024-09-11T08:50:04.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: 4.f scrub ok 2024-09-11T08:50:04.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: 4.9 scrub starts 2024-09-11T08:50:04.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: 4.9 scrub ok 2024-09-11T08:50:04.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:04 smithi092 ceph-mon[41114]: osdmap e227: 8 total, 8 up, 7 in 2024-09-11T08:50:04.577 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:04.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:04.578 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:04.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: pgmap v484: 148 pgs: 148 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 72 MiB/s wr, 1.16k op/s 2024-09-11T08:50:04.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: 4.12 scrub starts 2024-09-11T08:50:04.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: 4.12 scrub ok 2024-09-11T08:50:04.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.658 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.658 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.658 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.658 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:04.658 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: Adjusting osd_memory_target on smithi145 to 3248M 2024-09-11T08:50:04.658 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:50:04.658 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: 4.f scrub starts 2024-09-11T08:50:04.658 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: 4.f scrub ok 2024-09-11T08:50:04.658 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: 4.9 scrub starts 2024-09-11T08:50:04.658 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: 4.9 scrub ok 2024-09-11T08:50:04.658 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-mon[33384]: osdmap e227: 8 total, 8 up, 7 in 2024-09-11T08:50:04.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:04.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:04.717 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:04.718 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:04.727 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 50 16 91953 91937 114.904 63.375 0.00710572 0.00869831 2024-09-11T08:50:04.819 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:04.850 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:04.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:04.851 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:04.912 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=sqlstore.transactions t=2024-09-11T08:50:04.656785962Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:50:04.912 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:50:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=cleanup t=2024-09-11T08:50:04.659233486Z level=info msg="Completed cleanup jobs" duration=12.791736ms 2024-09-11T08:50:04.952 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:04.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:04.993 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:04.993 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:05.094 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:05.136 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:05.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:05.137 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:05.238 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:05.278 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:05.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:05.279 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:05.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:05.413 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:05.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:05.414 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:05.473 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:50:05.473 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:50:05.474 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_2 2024-09-11T08:50:05.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:50:05.515 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:05.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:05.563 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:05.564 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:05.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:05 smithi145 ceph-mon[33384]: 3.b scrub starts 2024-09-11T08:50:05.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:05 smithi145 ceph-mon[33384]: 3.b scrub ok 2024-09-11T08:50:05.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:05 smithi145 ceph-mon[33384]: 3.14 scrub starts 2024-09-11T08:50:05.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:05 smithi145 ceph-mon[33384]: 3.14 scrub ok 2024-09-11T08:50:05.664 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:05.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:05.705 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:05.705 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:05.727 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 51 16 93356 93340 114.37 87.6875 0.00589808 0.00873869 2024-09-11T08:50:05.778 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:05 smithi092 ceph-mon[29647]: 3.b scrub starts 2024-09-11T08:50:05.778 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:05 smithi092 ceph-mon[29647]: 3.b scrub ok 2024-09-11T08:50:05.778 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:05 smithi092 ceph-mon[29647]: 3.14 scrub starts 2024-09-11T08:50:05.778 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:05 smithi092 ceph-mon[29647]: 3.14 scrub ok 2024-09-11T08:50:05.778 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:05 smithi092 ceph-mon[41114]: 3.b scrub starts 2024-09-11T08:50:05.779 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:05 smithi092 ceph-mon[41114]: 3.b scrub ok 2024-09-11T08:50:05.779 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:05 smithi092 ceph-mon[41114]: 3.14 scrub starts 2024-09-11T08:50:05.779 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:05 smithi092 ceph-mon[41114]: 3.14 scrub ok 2024-09-11T08:50:05.806 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:05.842 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:05.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:05.843 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:05.885 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:05.930 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:05.944 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:05.981 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:05.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:05.982 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:06.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:06.117 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:06.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:06.118 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:06.219 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:06.257 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:06.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:06.259 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:06.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:06.399 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:06.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:06.400 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:06.501 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:06.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:06.537 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:06.537 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: 5.11 scrub starts 2024-09-11T08:50:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: 5.11 scrub ok 2024-09-11T08:50:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: pgmap v486: 148 pgs: 148 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 804 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-11T08:50:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: Adjusting osd_memory_target on smithi092 to 3248M 2024-09-11T08:50:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:50:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:50:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:50:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:50:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: 6.e scrub starts 2024-09-11T08:50:06.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[29647]: 6.e scrub ok 2024-09-11T08:50:06.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: 5.11 scrub starts 2024-09-11T08:50:06.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: 5.11 scrub ok 2024-09-11T08:50:06.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: pgmap v486: 148 pgs: 148 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 804 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-11T08:50:06.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: Adjusting osd_memory_target on smithi092 to 3248M 2024-09-11T08:50:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:50:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:50:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:50:06.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:50:06.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: 6.e scrub starts 2024-09-11T08:50:06.569 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:06 smithi092 ceph-mon[41114]: 6.e scrub ok 2024-09-11T08:50:06.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: 5.11 scrub starts 2024-09-11T08:50:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: 5.11 scrub ok 2024-09-11T08:50:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: pgmap v486: 148 pgs: 148 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 804 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-11T08:50:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:50:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: Adjusting osd_memory_target on smithi092 to 3248M 2024-09-11T08:50:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:50:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:50:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:50:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:50:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: 6.e scrub starts 2024-09-11T08:50:06.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-mon[33384]: 6.e scrub ok 2024-09-11T08:50:06.674 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:06.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:06.675 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:06.727 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 52 16 94740 94724 113.834 86.5 0.00575276 0.0087806 2024-09-11T08:50:06.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:06.821 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:06.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:06.822 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:06.923 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:06.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:06.965 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:06.965 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:07.066 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:07.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:07.099 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:07.099 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:07.161 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:50:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:50:07.200 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:07.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:07.241 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:07.241 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:07.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:07.376 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:07.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:07.377 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:07.478 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:07.510 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:07.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:07.511 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:07.612 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:07.652 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:07.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:07.654 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:07 smithi145 ceph-mon[33384]: 5.1f scrub starts 2024-09-11T08:50:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:07 smithi145 ceph-mon[33384]: 5.1f scrub ok 2024-09-11T08:50:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:07 smithi145 ceph-mon[33384]: 6.c scrub starts 2024-09-11T08:50:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:07 smithi145 ceph-mon[33384]: 6.c scrub ok 2024-09-11T08:50:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:07 smithi145 ceph-mon[33384]: 4.3 scrub starts 2024-09-11T08:50:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:07 smithi145 ceph-mon[33384]: 4.3 scrub ok 2024-09-11T08:50:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:07 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:50:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:07 smithi145 ceph-mon[33384]: 5.8 scrub ok 2024-09-11T08:50:07.727 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 53 16 96200 96184 113.408 91.25 0.0166895 0.00881364 2024-09-11T08:50:07.755 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:07.764 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[41114]: 5.1f scrub starts 2024-09-11T08:50:07.764 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[41114]: 5.1f scrub ok 2024-09-11T08:50:07.764 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[41114]: 6.c scrub starts 2024-09-11T08:50:07.765 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[41114]: 6.c scrub ok 2024-09-11T08:50:07.765 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[41114]: 4.3 scrub starts 2024-09-11T08:50:07.765 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[41114]: 4.3 scrub ok 2024-09-11T08:50:07.765 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:50:07.765 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[41114]: 5.8 scrub ok 2024-09-11T08:50:07.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[29647]: 5.1f scrub starts 2024-09-11T08:50:07.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[29647]: 5.1f scrub ok 2024-09-11T08:50:07.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[29647]: 6.c scrub starts 2024-09-11T08:50:07.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[29647]: 6.c scrub ok 2024-09-11T08:50:07.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[29647]: 4.3 scrub starts 2024-09-11T08:50:07.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[29647]: 4.3 scrub ok 2024-09-11T08:50:07.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:50:07.766 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:07 smithi092 ceph-mon[29647]: 5.8 scrub ok 2024-09-11T08:50:07.796 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:07.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:07.797 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:07.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:07.933 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:07.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:07.935 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:07.943 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:50:07.944 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:50:07.978 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:50:07.978 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:50:08.009 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:50:08.009 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:50:08.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:08.044 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:50:08.045 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:50:08.075 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:50:08.076 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:50:08.076 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:08.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:08.077 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:08.108 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:50:08.109 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:50:08.137 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:50:08.137 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:50:08.170 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:50:08.171 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:50:08.178 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:08.214 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:08.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:08.215 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:08.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:08.351 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:08.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:08.352 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:08.452 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:08.488 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:08.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:08.489 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:08.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:08.624 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:08.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:08.625 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:08 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:08 smithi145 ceph-mon[33384]: pgmap v487: 148 pgs: 148 active+clean; 5.8 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 106 MiB/s wr, 1.69k op/s 2024-09-11T08:50:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:08 smithi145 ceph-mon[33384]: 6.1f scrub starts 2024-09-11T08:50:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:08 smithi145 ceph-mon[33384]: 6.1f scrub ok 2024-09-11T08:50:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:08 smithi145 ceph-mon[33384]: 5.16 scrub starts 2024-09-11T08:50:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:08 smithi145 ceph-mon[33384]: 5.16 scrub ok 2024-09-11T08:50:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:08 smithi145 ceph-mon[33384]: 5.a scrub starts 2024-09-11T08:50:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:08 smithi145 ceph-mon[33384]: 5.a scrub ok 2024-09-11T08:50:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:08 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1889147807' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:50:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:08 smithi145 ceph-mon[33384]: 6.15 scrub starts 2024-09-11T08:50:08.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:08 smithi145 ceph-mon[33384]: 6.15 scrub ok 2024-09-11T08:50:08.727 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:08.727 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 54 16 97613 97597 112.943 88.3125 0.0061851 0.00884984 2024-09-11T08:50:08.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:08.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[29647]: pgmap v487: 148 pgs: 148 active+clean; 5.8 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 106 MiB/s wr, 1.69k op/s 2024-09-11T08:50:08.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[29647]: 6.1f scrub starts 2024-09-11T08:50:08.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[29647]: 6.1f scrub ok 2024-09-11T08:50:08.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[29647]: 5.16 scrub starts 2024-09-11T08:50:08.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[29647]: 5.16 scrub ok 2024-09-11T08:50:08.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[29647]: 5.a scrub starts 2024-09-11T08:50:08.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[29647]: 5.a scrub ok 2024-09-11T08:50:08.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1889147807' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:50:08.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[29647]: 6.15 scrub starts 2024-09-11T08:50:08.761 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[29647]: 6.15 scrub ok 2024-09-11T08:50:08.762 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:08.762 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[41114]: pgmap v487: 148 pgs: 148 active+clean; 5.8 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 106 MiB/s wr, 1.69k op/s 2024-09-11T08:50:08.762 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[41114]: 6.1f scrub starts 2024-09-11T08:50:08.762 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[41114]: 6.1f scrub ok 2024-09-11T08:50:08.762 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[41114]: 5.16 scrub starts 2024-09-11T08:50:08.762 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[41114]: 5.16 scrub ok 2024-09-11T08:50:08.762 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[41114]: 5.a scrub starts 2024-09-11T08:50:08.762 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[41114]: 5.a scrub ok 2024-09-11T08:50:08.763 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1889147807' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:50:08.763 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[41114]: 6.15 scrub starts 2024-09-11T08:50:08.763 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:08 smithi092 ceph-mon[41114]: 6.15 scrub ok 2024-09-11T08:50:08.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:08.764 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:08.764 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:08.865 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:08.904 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:08.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:08.905 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:09.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:09.041 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:09.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:09.043 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:09.143 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:09.176 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:09.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:09.177 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:09.278 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:09.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:09.315 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:09.315 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:09.382 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:09 smithi092 ceph-mon[29647]: 3.2 scrub starts 2024-09-11T08:50:09.382 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:09 smithi092 ceph-mon[29647]: 3.2 scrub ok 2024-09-11T08:50:09.382 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:09 smithi092 ceph-mon[29647]: 5.15 scrub starts 2024-09-11T08:50:09.382 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:09 smithi092 ceph-mon[29647]: 5.15 scrub ok 2024-09-11T08:50:09.383 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:09 smithi092 ceph-mon[29647]: 6.1c scrub starts 2024-09-11T08:50:09.383 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:09 smithi092 ceph-mon[29647]: 6.1c scrub ok 2024-09-11T08:50:09.383 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:09 smithi092 ceph-mon[41114]: 3.2 scrub starts 2024-09-11T08:50:09.383 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:09 smithi092 ceph-mon[41114]: 3.2 scrub ok 2024-09-11T08:50:09.383 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:09 smithi092 ceph-mon[41114]: 5.15 scrub starts 2024-09-11T08:50:09.383 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:09 smithi092 ceph-mon[41114]: 5.15 scrub ok 2024-09-11T08:50:09.383 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:09 smithi092 ceph-mon[41114]: 6.1c scrub starts 2024-09-11T08:50:09.383 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:09 smithi092 ceph-mon[41114]: 6.1c scrub ok 2024-09-11T08:50:09.417 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:09.456 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:09.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:09.457 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:09.558 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:09.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:09.593 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:09 smithi145 ceph-mon[33384]: 3.2 scrub starts 2024-09-11T08:50:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:09 smithi145 ceph-mon[33384]: 3.2 scrub ok 2024-09-11T08:50:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:09 smithi145 ceph-mon[33384]: 5.15 scrub starts 2024-09-11T08:50:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:09 smithi145 ceph-mon[33384]: 5.15 scrub ok 2024-09-11T08:50:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:09 smithi145 ceph-mon[33384]: 6.1c scrub starts 2024-09-11T08:50:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:09 smithi145 ceph-mon[33384]: 6.1c scrub ok 2024-09-11T08:50:09.693 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:09.726 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:09.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:09.727 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:09.727 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 55 16 99039 99023 112.51 89.125 0.0165791 0.00888385 2024-09-11T08:50:09.828 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:09.856 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:09.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:09.857 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:09.958 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:09.994 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:09.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:09.995 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:10.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:10.129 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:10.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:10.130 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:10.231 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:10.265 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:10.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:10.266 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:10.367 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:10.402 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:10.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:10.403 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:10.504 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:10.537 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:10.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:10.538 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:10.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:10 smithi145 ceph-mon[33384]: pgmap v488: 148 pgs: 148 active+clean; 5.8 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-11T08:50:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:10 smithi145 ceph-mon[33384]: 3.17 scrub starts 2024-09-11T08:50:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:10 smithi145 ceph-mon[33384]: 3.17 scrub ok 2024-09-11T08:50:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:10 smithi145 ceph-mon[33384]: 5.1c scrub starts 2024-09-11T08:50:10.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:10 smithi145 ceph-mon[33384]: 5.1c scrub ok 2024-09-11T08:50:10.673 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:10.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:10.674 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:10.727 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 56 16 100410 100394 112.031 85.6875 0.00497393 0.00892184 2024-09-11T08:50:10.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:10.807 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:10.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:10.808 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:10 smithi092 ceph-mon[29647]: pgmap v488: 148 pgs: 148 active+clean; 5.8 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-11T08:50:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:10 smithi092 ceph-mon[29647]: 3.17 scrub starts 2024-09-11T08:50:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:10 smithi092 ceph-mon[29647]: 3.17 scrub ok 2024-09-11T08:50:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:10 smithi092 ceph-mon[29647]: 5.1c scrub starts 2024-09-11T08:50:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:10 smithi092 ceph-mon[29647]: 5.1c scrub ok 2024-09-11T08:50:10.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:10 smithi092 ceph-mon[41114]: pgmap v488: 148 pgs: 148 active+clean; 5.8 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-11T08:50:10.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:10 smithi092 ceph-mon[41114]: 3.17 scrub starts 2024-09-11T08:50:10.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:10 smithi092 ceph-mon[41114]: 3.17 scrub ok 2024-09-11T08:50:10.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:10 smithi092 ceph-mon[41114]: 5.1c scrub starts 2024-09-11T08:50:10.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:10 smithi092 ceph-mon[41114]: 5.1c scrub ok 2024-09-11T08:50:10.909 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:10.943 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:10.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:10.944 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:11.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:11.084 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:11.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:11.085 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:11.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:50:11.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:11.229 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:11.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:11.230 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:11.331 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:11.366 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:11.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:11.367 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:11.467 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:11.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:11.501 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:11.502 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:11.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:11 smithi092 ceph-mon[29647]: 4.1f scrub starts 2024-09-11T08:50:11.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:11 smithi092 ceph-mon[29647]: 4.1f scrub ok 2024-09-11T08:50:11.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:50:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:50:11] "GET /metrics HTTP/1.1" 200 224389 "" "Prometheus/2.43.0" 2024-09-11T08:50:11.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:11 smithi092 ceph-mon[41114]: 4.1f scrub starts 2024-09-11T08:50:11.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:11 smithi092 ceph-mon[41114]: 4.1f scrub ok 2024-09-11T08:50:11.573 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:50:11.573 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:50:11.602 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:11.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:11.637 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:11.637 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:11.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:11 smithi145 ceph-mon[33384]: 4.1f scrub starts 2024-09-11T08:50:11.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:11 smithi145 ceph-mon[33384]: 4.1f scrub ok 2024-09-11T08:50:11.727 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 57 16 101807 101791 111.597 87.3125 0.00723386 0.00895701 2024-09-11T08:50:11.738 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:11.772 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:11.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:11.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:11.874 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:11.908 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:11.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:11.909 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:12.010 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:12.045 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:12.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:12.046 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:12.147 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:12.182 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:12.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:12.183 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:12.284 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:12.319 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:12.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:12.320 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:12.421 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:12.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:12.469 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:12.469 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:12.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:12 smithi092 ceph-mon[41114]: pgmap v489: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-11T08:50:12.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:12 smithi092 ceph-mon[41114]: 4.8 scrub starts 2024-09-11T08:50:12.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:12 smithi092 ceph-mon[41114]: 4.8 scrub ok 2024-09-11T08:50:12.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:12 smithi092 ceph-mon[29647]: pgmap v489: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-11T08:50:12.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:12 smithi092 ceph-mon[29647]: 4.8 scrub starts 2024-09-11T08:50:12.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:12 smithi092 ceph-mon[29647]: 4.8 scrub ok 2024-09-11T08:50:12.570 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:12.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:12.611 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:12.611 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:12.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:12 smithi145 ceph-mon[33384]: pgmap v489: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-11T08:50:12.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:12 smithi145 ceph-mon[33384]: 4.8 scrub starts 2024-09-11T08:50:12.665 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:12 smithi145 ceph-mon[33384]: 4.8 scrub ok 2024-09-11T08:50:12.712 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:12.727 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 58 16 103222 103206 111.198 88.4375 0.0151077 0.00898873 2024-09-11T08:50:12.749 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:12.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:12.750 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:12.779 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:50:12.781 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:50:12.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:12.886 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:12.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:12.888 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:12.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:13.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:13.022 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:13.022 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:13.123 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:13.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:13.156 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:13.156 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:13.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:50:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:50:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:50:13.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:13.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:13.301 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:13.301 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:13.402 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:13.435 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:13.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:13.436 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:13.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:13.574 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:13.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:13.575 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:13.649 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:13 smithi092 ceph-mon[29647]: 5.3 scrub starts 2024-09-11T08:50:13.650 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:13 smithi092 ceph-mon[29647]: 5.3 scrub ok 2024-09-11T08:50:13.650 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:13 smithi092 ceph-mon[29647]: 3.1 scrub starts 2024-09-11T08:50:13.650 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:13 smithi092 ceph-mon[29647]: 3.1 scrub ok 2024-09-11T08:50:13.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:13 smithi092 ceph-mon[41114]: 5.3 scrub starts 2024-09-11T08:50:13.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:13 smithi092 ceph-mon[41114]: 5.3 scrub ok 2024-09-11T08:50:13.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:13 smithi092 ceph-mon[41114]: 3.1 scrub starts 2024-09-11T08:50:13.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:13 smithi092 ceph-mon[41114]: 3.1 scrub ok 2024-09-11T08:50:13.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:13 smithi145 ceph-mon[33384]: 5.3 scrub starts 2024-09-11T08:50:13.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:13 smithi145 ceph-mon[33384]: 5.3 scrub ok 2024-09-11T08:50:13.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:13 smithi145 ceph-mon[33384]: 3.1 scrub starts 2024-09-11T08:50:13.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:13 smithi145 ceph-mon[33384]: 3.1 scrub ok 2024-09-11T08:50:13.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:13.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-11T08:50:13.721 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:13.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:13.722 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:13.728 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 59 16 104571 104555 110.741 84.3125 0.0203157 0.00902526 2024-09-11T08:50:13.823 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:13.854 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:13.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:13.855 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:13.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:14.011 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:14.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:14.012 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:14.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:14.149 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:14.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:14.150 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:14.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:14.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:14.285 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:14.285 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:14.386 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:14.421 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:14.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:14.422 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:14.497 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-11T08:50:11.244735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6327834844,"num_objects":96742,"num_object_clones":0,"num_object_copies":290226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96742,"num_whiteouts":0,"num_read":1951,"num_read_kb":1808,"num_write":97375,"num_write_kb":6180615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":1184877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73762,"ondisk_log_size":73762,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":20595612,"kb_used_data":18460068,"kb_used_omap":11,"kb_used_meta":2135476,"kb_avail":635620452,"statfs":{"total":671965249536,"available":650875342848,"internally_reserved":0,"allocated":18903109632,"data_stored":18900309849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2186727784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,2,8,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":885850112,"num_objects":13517,"num_object_clones":0,"num_object_copies":40551,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13517,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":13517,"num_write_kb":865088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004343"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.887765+0000","last_change":"2024-09-11T08:49:09.906451+0000","last_active":"2024-09-11T08:50:00.887765+0000","last_peered":"2024-09-11T08:50:00.887765+0000","last_clean":"2024-09-11T08:50:00.887765+0000","last_became_active":"2024-09-11T08:49:09.905661+0000","last_became_peered":"2024-09-11T08:49:09.905661+0000","last_unstale":"2024-09-11T08:50:00.887765+0000","last_undegraded":"2024-09-11T08:50:00.887765+0000","last_fullsized":"2024-09-11T08:50:00.887765+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:47.838700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:47.838700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"200'19","reported_seq":287,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.894490+0000","last_change":"2024-09-11T08:49:34.732293+0000","last_active":"2024-09-11T08:50:00.894490+0000","last_peered":"2024-09-11T08:50:00.894490+0000","last_clean":"2024-09-11T08:50:00.894490+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:50:00.894490+0000","last_undegraded":"2024-09-11T08:50:00.894490+0000","last_fullsized":"2024-09-11T08:50:00.894490+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:49:34.732198+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:49:34.732198+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:49.013371+0000","scrub_duration":0.0076844679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":225,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.888626+0000","last_change":"2024-09-11T08:47:44.884292+0000","last_active":"2024-09-11T08:50:00.888626+0000","last_peered":"2024-09-11T08:50:00.888626+0000","last_clean":"2024-09-11T08:50:00.888626+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:50:00.888626+0000","last_undegraded":"2024-09-11T08:50:00.888626+0000","last_fullsized":"2024-09-11T08:50:00.888626+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.884217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.884217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011480829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":204,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.895178+0000","last_change":"2024-09-11T08:49:23.674071+0000","last_active":"2024-09-11T08:50:00.895178+0000","last_peered":"2024-09-11T08:50:00.895178+0000","last_clean":"2024-09-11T08:50:00.895178+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:50:00.895178+0000","last_undegraded":"2024-09-11T08:50:00.895178+0000","last_fullsized":"2024-09-11T08:50:00.895178+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:23.673975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:23.673975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:34.389310+0000","scrub_duration":0.081616116000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":233,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.898547+0000","last_change":"2024-09-11T08:48:34.059380+0000","last_active":"2024-09-11T08:50:00.898547+0000","last_peered":"2024-09-11T08:50:00.898547+0000","last_clean":"2024-09-11T08:50:00.898547+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:50:00.898547+0000","last_undegraded":"2024-09-11T08:50:00.898547+0000","last_fullsized":"2024-09-11T08:50:00.898547+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:34.059309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:34.059309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0067175359999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":239,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.897169+0000","last_change":"2024-09-11T08:48:35.046165+0000","last_active":"2024-09-11T08:50:00.897169+0000","last_peered":"2024-09-11T08:50:00.897169+0000","last_clean":"2024-09-11T08:50:00.897169+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:50:00.897169+0000","last_undegraded":"2024-09-11T08:50:00.897169+0000","last_fullsized":"2024-09-11T08:50:00.897169+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:35.046075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:35.046075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010511866,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"200'34","reported_seq":293,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.897274+0000","last_change":"2024-09-11T08:49:09.927417+0000","last_active":"2024-09-11T08:50:00.897274+0000","last_peered":"2024-09-11T08:50:00.897274+0000","last_clean":"2024-09-11T08:50:00.897274+0000","last_became_active":"2024-09-11T08:49:09.909107+0000","last_became_peered":"2024-09-11T08:49:09.909107+0000","last_unstale":"2024-09-11T08:50:00.897274+0000","last_undegraded":"2024-09-11T08:50:00.897274+0000","last_fullsized":"2024-09-11T08:50:00.897274+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:48:30.029817+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.029817+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055582840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":206,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.894392+0000","last_change":"2024-09-11T08:48:32.827213+0000","last_active":"2024-09-11T08:50:00.894392+0000","last_peered":"2024-09-11T08:50:00.894392+0000","last_clean":"2024-09-11T08:50:00.894392+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:50:00.894392+0000","last_undegraded":"2024-09-11T08:50:00.894392+0000","last_fullsized":"2024-09-11T08:50:00.894392+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:32.827110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:48:32.827110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029807079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":245,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.886507+0000","last_change":"2024-09-11T08:48:41.415476+0000","last_active":"2024-09-11T08:50:00.886507+0000","last_peered":"2024-09-11T08:50:00.886507+0000","last_clean":"2024-09-11T08:50:00.886507+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:50:00.886507+0000","last_undegraded":"2024-09-11T08:50:00.886507+0000","last_fullsized":"2024-09-11T08:50:00.886507+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:41.415403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.415403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017214509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"200'19","reported_seq":267,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.895408+0000","last_change":"2024-09-11T08:49:19.682800+0000","last_active":"2024-09-11T08:50:00.895408+0000","last_peered":"2024-09-11T08:50:00.895408+0000","last_clean":"2024-09-11T08:50:00.895408+0000","last_became_active":"2024-09-11T08:49:09.910441+0000","last_became_peered":"2024-09-11T08:49:09.910441+0000","last_unstale":"2024-09-11T08:50:00.895408+0000","last_undegraded":"2024-09-11T08:50:00.895408+0000","last_fullsized":"2024-09-11T08:50:00.895408+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:49:19.682694+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:49:19.682694+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:29.537157+0000","scrub_duration":0.0024238570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":231,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.888140+0000","last_change":"2024-09-11T08:49:09.900683+0000","last_active":"2024-09-11T08:50:00.888140+0000","last_peered":"2024-09-11T08:50:00.888140+0000","last_clean":"2024-09-11T08:50:00.888140+0000","last_became_active":"2024-09-11T08:49:09.899188+0000","last_became_peered":"2024-09-11T08:49:09.899188+0000","last_unstale":"2024-09-11T08:50:00.888140+0000","last_undegraded":"2024-09-11T08:50:00.888140+0000","last_fullsized":"2024-09-11T08:50:00.888140+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:18.341709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.341709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038165500000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":39,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.888739+0000","last_change":"2024-09-11T08:49:33.520537+0000","last_active":"2024-09-11T08:50:00.888739+0000","last_peered":"2024-09-11T08:50:00.888739+0000","last_clean":"2024-09-11T08:50:00.888739+0000","last_became_active":"2024-09-11T08:49:09.907856+0000","last_became_peered":"2024-09-11T08:49:09.907856+0000","last_unstale":"2024-09-11T08:50:00.888739+0000","last_undegraded":"2024-09-11T08:50:00.888739+0000","last_fullsized":"2024-09-11T08:50:00.888739+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:33.520462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:33.520462+0000","last_clean_scrub_stamp":"2024-09-11T08:49:33.520462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:52.380831+0000","scrub_duration":0.0043744099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":225,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.885470+0000","last_change":"2024-09-11T08:48:44.398447+0000","last_active":"2024-09-11T08:50:00.885470+0000","last_peered":"2024-09-11T08:50:00.885470+0000","last_clean":"2024-09-11T08:50:00.885470+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:50:00.885470+0000","last_undegraded":"2024-09-11T08:50:00.885470+0000","last_fullsized":"2024-09-11T08:50:00.885470+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:44.398373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.398373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030847359999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"200'7","reported_seq":221,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.895734+0000","last_change":"2024-09-11T08:47:44.866144+0000","last_active":"2024-09-11T08:50:00.895734+0000","last_peered":"2024-09-11T08:50:00.895734+0000","last_clean":"2024-09-11T08:50:00.895734+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:50:00.895734+0000","last_undegraded":"2024-09-11T08:50:00.895734+0000","last_fullsized":"2024-09-11T08:50:00.895734+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:47:44.866057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.866057+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020690169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":225,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.889333+0000","last_change":"2024-09-11T08:47:46.410478+0000","last_active":"2024-09-11T08:50:00.889333+0000","last_peered":"2024-09-11T08:50:00.889333+0000","last_clean":"2024-09-11T08:50:00.889333+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:50:00.889333+0000","last_undegraded":"2024-09-11T08:50:00.889333+0000","last_fullsized":"2024-09-11T08:50:00.889333+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:46.410402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:51.166051+0000","last_clean_scrub_stamp":"2024-09-11T08:47:46.410402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002594884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":219,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.886462+0000","last_change":"2024-09-11T08:49:28.283272+0000","last_active":"2024-09-11T08:50:00.886462+0000","last_peered":"2024-09-11T08:50:00.886462+0000","last_clean":"2024-09-11T08:50:00.886462+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:50:00.886462+0000","last_undegraded":"2024-09-11T08:50:00.886462+0000","last_fullsized":"2024-09-11T08:50:00.886462+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:28.283167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:49:28.283167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:28.913788+0000","scrub_duration":0.0040125320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":224,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.896611+0000","last_change":"2024-09-11T08:46:01.978488+0000","last_active":"2024-09-11T08:50:00.896611+0000","last_peered":"2024-09-11T08:50:00.896611+0000","last_clean":"2024-09-11T08:50:00.896611+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:50:00.896611+0000","last_undegraded":"2024-09-11T08:50:00.896611+0000","last_fullsized":"2024-09-11T08:50:00.896611+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:24.197149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.197149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001510267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.895829+0000","last_change":"2024-09-11T08:49:09.908672+0000","last_active":"2024-09-11T08:50:00.895829+0000","last_peered":"2024-09-11T08:50:00.895829+0000","last_clean":"2024-09-11T08:50:00.895829+0000","last_became_active":"2024-09-11T08:49:09.908237+0000","last_became_peered":"2024-09-11T08:49:09.908237+0000","last_unstale":"2024-09-11T08:50:00.895829+0000","last_undegraded":"2024-09-11T08:50:00.895829+0000","last_fullsized":"2024-09-11T08:50:00.895829+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:34.610331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:34.610331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"200'34","reported_seq":303,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:01.824316+0000","last_change":"2024-09-11T08:50:01.824223+0000","last_active":"2024-09-11T08:50:01.824316+0000","last_peered":"2024-09-11T08:50:01.824316+0000","last_clean":"2024-09-11T08:50:01.824316+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:50:01.824316+0000","last_undegraded":"2024-09-11T08:50:01.824316+0000","last_fullsized":"2024-09-11T08:50:01.824316+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:50:01.824114+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:50:01.824114+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:20.462751+0000","scrub_duration":0.008843989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":235,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.889882+0000","last_change":"2024-09-11T08:49:09.903399+0000","last_active":"2024-09-11T08:50:00.889882+0000","last_peered":"2024-09-11T08:50:00.889882+0000","last_clean":"2024-09-11T08:50:00.889882+0000","last_became_active":"2024-09-11T08:49:09.903078+0000","last_became_peered":"2024-09-11T08:49:09.903078+0000","last_unstale":"2024-09-11T08:50:00.889882+0000","last_undegraded":"2024-09-11T08:50:00.889882+0000","last_fullsized":"2024-09-11T08:50:00.889882+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:19.961460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.961460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0091369829999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":232,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.899486+0000","last_change":"2024-09-11T08:46:01.998163+0000","last_active":"2024-09-11T08:50:00.899486+0000","last_peered":"2024-09-11T08:50:00.899486+0000","last_clean":"2024-09-11T08:50:00.899486+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:50:00.899486+0000","last_undegraded":"2024-09-11T08:50:00.899486+0000","last_fullsized":"2024-09-11T08:50:00.899486+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:45:24.223237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:24.223237+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019917720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":210,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.895511+0000","last_change":"2024-09-11T08:45:28.194198+0000","last_active":"2024-09-11T08:50:00.895511+0000","last_peered":"2024-09-11T08:50:00.895511+0000","last_clean":"2024-09-11T08:50:00.895511+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:50:00.895511+0000","last_undegraded":"2024-09-11T08:50:00.895511+0000","last_fullsized":"2024-09-11T08:50:00.895511+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.194095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.194095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001559936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"200'18","reported_seq":271,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.896349+0000","last_change":"2024-09-11T08:49:26.388924+0000","last_active":"2024-09-11T08:50:00.896349+0000","last_peered":"2024-09-11T08:50:00.896349+0000","last_clean":"2024-09-11T08:50:00.896349+0000","last_became_active":"2024-09-11T08:49:16.943886+0000","last_became_peered":"2024-09-11T08:49:16.943886+0000","last_unstale":"2024-09-11T08:50:00.896349+0000","last_undegraded":"2024-09-11T08:50:00.896349+0000","last_fullsized":"2024-09-11T08:50:00.896349+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'18","last_scrub_stamp":"2024-09-11T08:49:26.388870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:49:26.388870+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:53.627087+0000","scrub_duration":0.001482014,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":216,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:09.746786+0000","last_change":"2024-09-11T08:50:09.746351+0000","last_active":"2024-09-11T08:50:09.746786+0000","last_peered":"2024-09-11T08:50:09.746786+0000","last_clean":"2024-09-11T08:50:09.746786+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:50:09.746786+0000","last_undegraded":"2024-09-11T08:50:09.746786+0000","last_fullsized":"2024-09-11T08:50:09.746786+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:09.745640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:09.745640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:10.823254+0000","scrub_duration":0.010457068999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":224,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:09.078043+0000","last_change":"2024-09-11T08:50:09.077952+0000","last_active":"2024-09-11T08:50:09.078043+0000","last_peered":"2024-09-11T08:50:09.078043+0000","last_clean":"2024-09-11T08:50:09.078043+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:50:09.078043+0000","last_undegraded":"2024-09-11T08:50:09.078043+0000","last_fullsized":"2024-09-11T08:50:09.078043+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:50:09.077871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:09.077871+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:12.452212+0000","scrub_duration":0.0059438809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":229,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.888757+0000","last_change":"2024-09-11T08:48:25.971280+0000","last_active":"2024-09-11T08:50:00.888757+0000","last_peered":"2024-09-11T08:50:00.888757+0000","last_clean":"2024-09-11T08:50:00.888757+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:50:00.888757+0000","last_undegraded":"2024-09-11T08:50:00.888757+0000","last_fullsized":"2024-09-11T08:50:00.888757+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:25.971151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.971151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095705709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"200'20","reported_seq":263,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.890857+0000","last_change":"2024-09-11T08:49:09.936134+0000","last_active":"2024-09-11T08:50:00.890857+0000","last_peered":"2024-09-11T08:50:00.890857+0000","last_clean":"2024-09-11T08:50:00.890857+0000","last_became_active":"2024-09-11T08:49:09.913968+0000","last_became_peered":"2024-09-11T08:49:09.913968+0000","last_unstale":"2024-09-11T08:50:00.890857+0000","last_undegraded":"2024-09-11T08:50:00.890857+0000","last_fullsized":"2024-09-11T08:50:00.890857+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:48:19.705185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.705185+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050623580000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":384,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:06.195500+0000","last_change":"2024-09-11T08:50:06.195464+0000","last_active":"2024-09-11T08:50:06.195500+0000","last_peered":"2024-09-11T08:50:06.195500+0000","last_clean":"2024-09-11T08:50:06.195500+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:50:06.195500+0000","last_undegraded":"2024-09-11T08:50:06.195500+0000","last_fullsized":"2024-09-11T08:50:06.195500+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:50:06.195410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:06.195410+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:35.557316+0000","scrub_duration":0.0013222749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"227'5807","reported_seq":5836,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:06.835389+0000","last_change":"2024-09-11T08:49:06.519462+0000","last_active":"2024-09-11T08:50:06.835389+0000","last_peered":"2024-09-11T08:50:06.835389+0000","last_clean":"2024-09-11T08:50:06.835389+0000","last_became_active":"2024-09-11T08:49:06.519147+0000","last_became_peered":"2024-09-11T08:49:06.519147+0000","last_unstale":"2024-09-11T08:50:06.835389+0000","last_undegraded":"2024-09-11T08:50:06.835389+0000","last_fullsized":"2024-09-11T08:50:06.835389+0000","mapping_epoch":206,"log_start":"215'1500","ondisk_log_start":"215'1500","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4307,"log_dups_size":0,"ondisk_log_size":4307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:14.340457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":380567552,"num_objects":5807,"num_object_clones":0,"num_object_copies":17421,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5807,"num_write_kb":371648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"200'34","reported_seq":277,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:03.458697+0000","last_change":"2024-09-11T08:50:03.458529+0000","last_active":"2024-09-11T08:50:03.458697+0000","last_peered":"2024-09-11T08:50:03.458697+0000","last_clean":"2024-09-11T08:50:03.458697+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:50:03.458697+0000","last_undegraded":"2024-09-11T08:50:03.458697+0000","last_fullsized":"2024-09-11T08:50:03.458697+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:50:03.458381+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:50:03.458381+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:28.396520+0000","scrub_duration":0.0023474210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":219,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.900217+0000","last_change":"2024-09-11T08:49:25.383955+0000","last_active":"2024-09-11T08:50:00.900217+0000","last_peered":"2024-09-11T08:50:00.900217+0000","last_clean":"2024-09-11T08:50:00.900217+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:50:00.900217+0000","last_undegraded":"2024-09-11T08:50:00.900217+0000","last_fullsized":"2024-09-11T08:50:00.900217+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:25.383554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:25.383554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:46.160564+0000","scrub_duration":0.0084378879999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"211'10","reported_seq":377,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:03.887946+0000","last_change":"2024-09-11T08:49:35.694375+0000","last_active":"2024-09-11T08:50:03.887946+0000","last_peered":"2024-09-11T08:50:03.887946+0000","last_clean":"2024-09-11T08:50:03.887946+0000","last_became_active":"2024-09-11T08:49:09.905262+0000","last_became_peered":"2024-09-11T08:49:09.905262+0000","last_unstale":"2024-09-11T08:50:03.887946+0000","last_undegraded":"2024-09-11T08:50:03.887946+0000","last_fullsized":"2024-09-11T08:50:03.887946+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'10","last_scrub_stamp":"2024-09-11T08:49:35.694282+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:49:35.694282+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:45.457363+0000","scrub_duration":0.0051461679999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":226,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.886112+0000","last_change":"2024-09-11T08:49:20.335925+0000","last_active":"2024-09-11T08:50:00.886112+0000","last_peered":"2024-09-11T08:50:00.886112+0000","last_clean":"2024-09-11T08:50:00.886112+0000","last_became_active":"2024-09-11T08:49:09.906872+0000","last_became_peered":"2024-09-11T08:49:09.906872+0000","last_unstale":"2024-09-11T08:50:00.886112+0000","last_undegraded":"2024-09-11T08:50:00.886112+0000","last_fullsized":"2024-09-11T08:50:00.886112+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:20.335841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:20.335841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:20.443355+0000","scrub_duration":0.0041148479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"227'6332","reported_seq":6361,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:11.181654+0000","last_change":"2024-09-11T08:49:05.875564+0000","last_active":"2024-09-11T08:50:11.181654+0000","last_peered":"2024-09-11T08:50:11.181654+0000","last_clean":"2024-09-11T08:50:11.181654+0000","last_became_active":"2024-09-11T08:49:05.875264+0000","last_became_peered":"2024-09-11T08:49:05.875264+0000","last_unstale":"2024-09-11T08:50:11.181654+0000","last_undegraded":"2024-09-11T08:50:11.181654+0000","last_fullsized":"2024-09-11T08:50:11.181654+0000","mapping_epoch":206,"log_start":"216'1800","ondisk_log_start":"216'1800","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4532,"log_dups_size":0,"ondisk_log_size":4532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":414973952,"num_objects":6332,"num_object_clones":0,"num_object_copies":18996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6332,"num_write_kb":405248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"200'38","reported_seq":303,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.899979+0000","last_change":"2024-09-11T08:49:09.936915+0000","last_active":"2024-09-11T08:50:00.899979+0000","last_peered":"2024-09-11T08:50:00.899979+0000","last_clean":"2024-09-11T08:50:00.899979+0000","last_became_active":"2024-09-11T08:49:09.917354+0000","last_became_peered":"2024-09-11T08:49:09.917354+0000","last_unstale":"2024-09-11T08:50:00.899979+0000","last_undegraded":"2024-09-11T08:50:00.899979+0000","last_fullsized":"2024-09-11T08:50:00.899979+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'26","last_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_deep_scrub":"199'26","last_deep_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.093994+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:10.328025+0000","scrub_duration":0.026938573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":229,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.899910+0000","last_change":"2024-09-11T08:49:09.915472+0000","last_active":"2024-09-11T08:50:00.899910+0000","last_peered":"2024-09-11T08:50:00.899910+0000","last_clean":"2024-09-11T08:50:00.899910+0000","last_became_active":"2024-09-11T08:49:09.915191+0000","last_became_peered":"2024-09-11T08:49:09.915191+0000","last_unstale":"2024-09-11T08:50:00.899910+0000","last_undegraded":"2024-09-11T08:50:00.899910+0000","last_fullsized":"2024-09-11T08:50:00.899910+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:38.041303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:38.041303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073485030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":216,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.894682+0000","last_change":"2024-09-11T08:49:29.687666+0000","last_active":"2024-09-11T08:50:00.894682+0000","last_peered":"2024-09-11T08:50:00.894682+0000","last_clean":"2024-09-11T08:50:00.894682+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:50:00.894682+0000","last_undegraded":"2024-09-11T08:50:00.894682+0000","last_fullsized":"2024-09-11T08:50:00.894682+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:29.687577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:29.687577+0000","last_clean_scrub_stamp":"2024-09-11T08:49:29.687577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:55.182660+0000","scrub_duration":0.0082090140000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":204,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.895339+0000","last_change":"2024-09-11T08:46:03.144149+0000","last_active":"2024-09-11T08:50:00.895339+0000","last_peered":"2024-09-11T08:50:00.895339+0000","last_clean":"2024-09-11T08:50:00.895339+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:50:00.895339+0000","last_undegraded":"2024-09-11T08:50:00.895339+0000","last_fullsized":"2024-09-11T08:50:00.895339+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:46:03.144096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:46:03.144096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001497151,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"227'5885","reported_seq":5914,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:06.793646+0000","last_change":"2024-09-11T08:49:05.874640+0000","last_active":"2024-09-11T08:50:06.793646+0000","last_peered":"2024-09-11T08:50:06.793646+0000","last_clean":"2024-09-11T08:50:06.793646+0000","last_became_active":"2024-09-11T08:49:05.874365+0000","last_became_peered":"2024-09-11T08:49:05.874365+0000","last_unstale":"2024-09-11T08:50:06.793646+0000","last_undegraded":"2024-09-11T08:50:06.793646+0000","last_fullsized":"2024-09-11T08:50:06.793646+0000","mapping_epoch":206,"log_start":"215'1600","ondisk_log_start":"215'1600","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4285,"log_dups_size":0,"ondisk_log_size":4285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:32.415086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385679360,"num_objects":5885,"num_object_clones":0,"num_object_copies":17655,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5885,"num_write_kb":376640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"200'35","reported_seq":257,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.895056+0000","last_change":"2024-09-11T08:48:23.862534+0000","last_active":"2024-09-11T08:50:00.895056+0000","last_peered":"2024-09-11T08:50:00.895056+0000","last_clean":"2024-09-11T08:50:00.895056+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:50:00.895056+0000","last_undegraded":"2024-09-11T08:50:00.895056+0000","last_fullsized":"2024-09-11T08:50:00.895056+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'21","last_scrub_stamp":"2024-09-11T08:48:23.862482+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:48:23.862482+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001795776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.890851+0000","last_change":"2024-09-11T08:49:24.594111+0000","last_active":"2024-09-11T08:50:00.890851+0000","last_peered":"2024-09-11T08:50:00.890851+0000","last_clean":"2024-09-11T08:50:00.890851+0000","last_became_active":"2024-09-11T08:49:09.908041+0000","last_became_peered":"2024-09-11T08:49:09.908041+0000","last_unstale":"2024-09-11T08:50:00.890851+0000","last_undegraded":"2024-09-11T08:50:00.890851+0000","last_fullsized":"2024-09-11T08:50:00.890851+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:24.593902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:24.593902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:39.993524+0000","scrub_duration":0.023705506000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":353,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:02.809018+0000","last_change":"2024-09-11T08:48:36.052650+0000","last_active":"2024-09-11T08:50:02.809018+0000","last_peered":"2024-09-11T08:50:02.809018+0000","last_clean":"2024-09-11T08:50:02.809018+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:50:02.809018+0000","last_undegraded":"2024-09-11T08:50:02.809018+0000","last_fullsized":"2024-09-11T08:50:02.809018+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:48:36.052514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:36.052514+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015857353000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":243,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.894559+0000","last_change":"2024-09-11T08:49:31.758729+0000","last_active":"2024-09-11T08:50:00.894559+0000","last_peered":"2024-09-11T08:50:00.894559+0000","last_clean":"2024-09-11T08:50:00.894559+0000","last_became_active":"2024-09-11T08:46:01.983335+0000","last_became_peered":"2024-09-11T08:46:01.983335+0000","last_unstale":"2024-09-11T08:50:00.894559+0000","last_undegraded":"2024-09-11T08:50:00.894559+0000","last_fullsized":"2024-09-11T08:50:00.894559+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:31.758573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:31.758573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:36.177031+0000","scrub_duration":0.0084881980000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"227'5855","reported_seq":5883,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:06.322030+0000","last_change":"2024-09-11T08:49:05.871535+0000","last_active":"2024-09-11T08:50:06.322030+0000","last_peered":"2024-09-11T08:50:06.322030+0000","last_clean":"2024-09-11T08:50:06.322030+0000","last_became_active":"2024-09-11T08:49:05.871255+0000","last_became_peered":"2024-09-11T08:49:05.871255+0000","last_unstale":"2024-09-11T08:50:06.322030+0000","last_undegraded":"2024-09-11T08:50:06.322030+0000","last_fullsized":"2024-09-11T08:50:06.322030+0000","mapping_epoch":206,"log_start":"215'1200","ondisk_log_start":"215'1200","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4655,"log_dups_size":0,"ondisk_log_size":4655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:09.000693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383713280,"num_objects":5855,"num_object_clones":0,"num_object_copies":17565,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5855,"num_write_kb":374720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"200'25","reported_seq":262,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.895146+0000","last_change":"2024-09-11T08:48:38.032124+0000","last_active":"2024-09-11T08:50:00.895146+0000","last_peered":"2024-09-11T08:50:00.895146+0000","last_clean":"2024-09-11T08:50:00.895146+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:50:00.895146+0000","last_undegraded":"2024-09-11T08:50:00.895146+0000","last_fullsized":"2024-09-11T08:50:00.895146+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'15","last_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_clean_scrub_stamp":"2024-09-11T08:48:38.032039+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049584310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":225,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.886569+0000","last_change":"2024-09-11T08:49:26.248106+0000","last_active":"2024-09-11T08:50:00.886569+0000","last_peered":"2024-09-11T08:50:00.886569+0000","last_clean":"2024-09-11T08:50:00.886569+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:50:00.886569+0000","last_undegraded":"2024-09-11T08:50:00.886569+0000","last_fullsized":"2024-09-11T08:50:00.886569+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:26.247212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:26.247212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:30.023699+0000","scrub_duration":0.0056073520000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"211'9","reported_seq":64,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:03.889268+0000","last_change":"2024-09-11T08:49:09.920108+0000","last_active":"2024-09-11T08:50:03.889268+0000","last_peered":"2024-09-11T08:50:03.889268+0000","last_clean":"2024-09-11T08:50:03.889268+0000","last_became_active":"2024-09-11T08:49:09.910279+0000","last_became_peered":"2024-09-11T08:49:09.910279+0000","last_unstale":"2024-09-11T08:50:03.889268+0000","last_undegraded":"2024-09-11T08:50:03.889268+0000","last_fullsized":"2024-09-11T08:50:03.889268+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:48:44.796704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.796704+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":29,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.899741+0000","last_change":"2024-09-11T08:49:09.908727+0000","last_active":"2024-09-11T08:50:00.899741+0000","last_peered":"2024-09-11T08:50:00.899741+0000","last_clean":"2024-09-11T08:50:00.899741+0000","last_became_active":"2024-09-11T08:49:09.907944+0000","last_became_peered":"2024-09-11T08:49:09.907944+0000","last_unstale":"2024-09-11T08:50:00.899741+0000","last_undegraded":"2024-09-11T08:50:00.899741+0000","last_fullsized":"2024-09-11T08:50:00.899741+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:31.710797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:48:31.710797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"227'6426","reported_seq":6455,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:11.161120+0000","last_change":"2024-09-11T08:49:05.875096+0000","last_active":"2024-09-11T08:50:11.161120+0000","last_peered":"2024-09-11T08:50:11.161120+0000","last_clean":"2024-09-11T08:50:11.161120+0000","last_became_active":"2024-09-11T08:49:05.874803+0000","last_became_peered":"2024-09-11T08:49:05.874803+0000","last_unstale":"2024-09-11T08:50:11.161120+0000","last_undegraded":"2024-09-11T08:50:11.161120+0000","last_fullsized":"2024-09-11T08:50:11.161120+0000","mapping_epoch":206,"log_start":"216'1900","ondisk_log_start":"216'1900","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4526,"log_dups_size":0,"ondisk_log_size":4526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:12.899212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421134336,"num_objects":6426,"num_object_clones":0,"num_object_copies":19278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6426,"num_write_kb":411264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"200'29","reported_seq":277,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.888213+0000","last_change":"2024-09-11T08:49:09.928210+0000","last_active":"2024-09-11T08:50:00.888213+0000","last_peered":"2024-09-11T08:50:00.888213+0000","last_clean":"2024-09-11T08:50:00.888213+0000","last_became_active":"2024-09-11T08:49:09.901065+0000","last_became_peered":"2024-09-11T08:49:09.901065+0000","last_unstale":"2024-09-11T08:50:00.888213+0000","last_undegraded":"2024-09-11T08:50:00.888213+0000","last_fullsized":"2024-09-11T08:50:00.888213+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-11T08:45:11.576085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:11.576085+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002366754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":229,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.894991+0000","last_change":"2024-09-11T08:48:24.062209+0000","last_active":"2024-09-11T08:50:00.894991+0000","last_peered":"2024-09-11T08:50:00.894991+0000","last_clean":"2024-09-11T08:50:00.894991+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:50:00.894991+0000","last_undegraded":"2024-09-11T08:50:00.894991+0000","last_fullsized":"2024-09-11T08:50:00.894991+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:24.062110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:24.062110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.037841910999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":245,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.890378+0000","last_change":"2024-09-11T08:49:28.069561+0000","last_active":"2024-09-11T08:50:00.890378+0000","last_peered":"2024-09-11T08:50:00.890378+0000","last_clean":"2024-09-11T08:50:00.890378+0000","last_became_active":"2024-09-11T08:49:09.909306+0000","last_became_peered":"2024-09-11T08:49:09.909306+0000","last_unstale":"2024-09-11T08:50:00.890378+0000","last_undegraded":"2024-09-11T08:50:00.890378+0000","last_fullsized":"2024-09-11T08:50:00.890378+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_clean_scrub_stamp":"2024-09-11T08:49:28.069461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:54.916266+0000","scrub_duration":0.0022283350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":243,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.888559+0000","last_change":"2024-09-11T08:48:27.981040+0000","last_active":"2024-09-11T08:50:00.888559+0000","last_peered":"2024-09-11T08:50:00.888559+0000","last_clean":"2024-09-11T08:50:00.888559+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:50:00.888559+0000","last_undegraded":"2024-09-11T08:50:00.888559+0000","last_fullsized":"2024-09-11T08:50:00.888559+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:27.980860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:48:27.980860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021574010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"227'6365","reported_seq":6394,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:11.163709+0000","last_change":"2024-09-11T08:49:09.913293+0000","last_active":"2024-09-11T08:50:11.163709+0000","last_peered":"2024-09-11T08:50:11.163709+0000","last_clean":"2024-09-11T08:50:11.163709+0000","last_became_active":"2024-09-11T08:49:09.912997+0000","last_became_peered":"2024-09-11T08:49:09.912997+0000","last_unstale":"2024-09-11T08:50:11.163709+0000","last_undegraded":"2024-09-11T08:50:11.163709+0000","last_fullsized":"2024-09-11T08:50:11.163709+0000","mapping_epoch":210,"log_start":"215'1800","ondisk_log_start":"215'1800","created":206,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4565,"log_dups_size":0,"ondisk_log_size":4565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:12.200890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":417136640,"num_objects":6365,"num_object_clones":0,"num_object_copies":19095,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6365,"num_write_kb":407360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"200'50","reported_seq":440,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:02.807286+0000","last_change":"2024-09-11T08:45:26.666261+0000","last_active":"2024-09-11T08:50:02.807286+0000","last_peered":"2024-09-11T08:50:02.807286+0000","last_clean":"2024-09-11T08:50:02.807286+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:50:02.807286+0000","last_undegraded":"2024-09-11T08:50:02.807286+0000","last_fullsized":"2024-09-11T08:50:02.807286+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'38","last_scrub_stamp":"2024-09-11T08:45:26.666199+0000","last_deep_scrub":"78'38","last_deep_scrub_stamp":"2024-09-11T08:39:53.492945+0000","last_clean_scrub_stamp":"2024-09-11T08:45:26.666199+0000","objects_scrubbed":10,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020004279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":230,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.895993+0000","last_change":"2024-09-11T08:49:25.626134+0000","last_active":"2024-09-11T08:50:00.895993+0000","last_peered":"2024-09-11T08:50:00.895993+0000","last_clean":"2024-09-11T08:50:00.895993+0000","last_became_active":"2024-09-11T08:49:09.907850+0000","last_became_peered":"2024-09-11T08:49:09.907850+0000","last_unstale":"2024-09-11T08:50:00.895993+0000","last_undegraded":"2024-09-11T08:50:00.895993+0000","last_fullsized":"2024-09-11T08:50:00.895993+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_clean_scrub_stamp":"2024-09-11T08:49:25.626031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:26.494358+0000","scrub_duration":0.0028789509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":260,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.885630+0000","last_change":"2024-09-11T08:48:16.376429+0000","last_active":"2024-09-11T08:50:00.885630+0000","last_peered":"2024-09-11T08:50:00.885630+0000","last_clean":"2024-09-11T08:50:00.885630+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:50:00.885630+0000","last_undegraded":"2024-09-11T08:50:00.885630+0000","last_fullsized":"2024-09-11T08:50:00.885630+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:48:16.376340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:16.376340+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00247796,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":29,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.889747+0000","last_change":"2024-09-11T08:49:09.903912+0000","last_active":"2024-09-11T08:50:00.889747+0000","last_peered":"2024-09-11T08:50:00.889747+0000","last_clean":"2024-09-11T08:50:00.889747+0000","last_became_active":"2024-09-11T08:49:09.903614+0000","last_became_peered":"2024-09-11T08:49:09.903614+0000","last_unstale":"2024-09-11T08:50:00.889747+0000","last_undegraded":"2024-09-11T08:50:00.889747+0000","last_fullsized":"2024-09-11T08:50:00.889747+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:25.682632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.682632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":213,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:02.248573+0000","last_change":"2024-09-11T08:50:02.248519+0000","last_active":"2024-09-11T08:50:02.248573+0000","last_peered":"2024-09-11T08:50:02.248573+0000","last_clean":"2024-09-11T08:50:02.248573+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:50:02.248573+0000","last_undegraded":"2024-09-11T08:50:02.248573+0000","last_fullsized":"2024-09-11T08:50:02.248573+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:02.248447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:02.248447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:06.671112+0000","scrub_duration":0.0029743439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"227'5975","reported_seq":6003,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:06.309393+0000","last_change":"2024-09-11T08:49:05.872817+0000","last_active":"2024-09-11T08:50:06.309393+0000","last_peered":"2024-09-11T08:50:06.309393+0000","last_clean":"2024-09-11T08:50:06.309393+0000","last_became_active":"2024-09-11T08:49:05.872392+0000","last_became_peered":"2024-09-11T08:49:05.872392+0000","last_unstale":"2024-09-11T08:50:06.309393+0000","last_undegraded":"2024-09-11T08:50:06.309393+0000","last_fullsized":"2024-09-11T08:50:06.309393+0000","mapping_epoch":206,"log_start":"215'1300","ondisk_log_start":"215'1300","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4675,"log_dups_size":0,"ondisk_log_size":4675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":391577600,"num_objects":5975,"num_object_clones":0,"num_object_copies":17925,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5975,"num_write_kb":382400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"200'17","reported_seq":282,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.889489+0000","last_change":"2024-09-11T08:49:09.928583+0000","last_active":"2024-09-11T08:50:00.889489+0000","last_peered":"2024-09-11T08:50:00.889489+0000","last_clean":"2024-09-11T08:50:00.889489+0000","last_became_active":"2024-09-11T08:49:09.902884+0000","last_became_peered":"2024-09-11T08:49:09.902884+0000","last_unstale":"2024-09-11T08:50:00.889489+0000","last_undegraded":"2024-09-11T08:50:00.889489+0000","last_fullsized":"2024-09-11T08:50:00.889489+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'13","last_scrub_stamp":"2024-09-11T08:45:27.666502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:45:27.666502+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023582730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":255,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.888217+0000","last_change":"2024-09-11T08:49:23.588233+0000","last_active":"2024-09-11T08:50:00.888217+0000","last_peered":"2024-09-11T08:50:00.888217+0000","last_clean":"2024-09-11T08:50:00.888217+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:50:00.888217+0000","last_undegraded":"2024-09-11T08:50:00.888217+0000","last_fullsized":"2024-09-11T08:50:00.888217+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:23.588182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:23.588182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:42.336498+0000","scrub_duration":0.0010197089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":273,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.902227+0000","last_change":"2024-09-11T08:49:32.558340+0000","last_active":"2024-09-11T08:50:00.902227+0000","last_peered":"2024-09-11T08:50:00.902227+0000","last_clean":"2024-09-11T08:50:00.902227+0000","last_became_active":"2024-09-11T08:49:09.909804+0000","last_became_peered":"2024-09-11T08:49:09.909804+0000","last_unstale":"2024-09-11T08:50:00.902227+0000","last_undegraded":"2024-09-11T08:50:00.902227+0000","last_fullsized":"2024-09-11T08:50:00.902227+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:49:32.558236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:32.558236+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:01.291601+0000","scrub_duration":0.0035759400000000001,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":245,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.903110+0000","last_change":"2024-09-11T08:49:15.477556+0000","last_active":"2024-09-11T08:50:00.903110+0000","last_peered":"2024-09-11T08:50:00.903110+0000","last_clean":"2024-09-11T08:50:00.903110+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:50:00.903110+0000","last_undegraded":"2024-09-11T08:50:00.903110+0000","last_fullsized":"2024-09-11T08:50:00.903110+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:15.477483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_clean_scrub_stamp":"2024-09-11T08:49:15.477483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:35.478111+0000","scrub_duration":0.0049012409999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":243,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.890274+0000","last_change":"2024-09-11T08:48:44.031924+0000","last_active":"2024-09-11T08:50:00.890274+0000","last_peered":"2024-09-11T08:50:00.890274+0000","last_clean":"2024-09-11T08:50:00.890274+0000","last_became_active":"2024-09-11T08:46:01.983095+0000","last_became_peered":"2024-09-11T08:46:01.983095+0000","last_unstale":"2024-09-11T08:50:00.890274+0000","last_undegraded":"2024-09-11T08:50:00.890274+0000","last_fullsized":"2024-09-11T08:50:00.890274+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:44.031842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.031842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021799599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"227'6307","reported_seq":6336,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:11.174851+0000","last_change":"2024-09-11T08:49:09.909742+0000","last_active":"2024-09-11T08:50:11.174851+0000","last_peered":"2024-09-11T08:50:11.174851+0000","last_clean":"2024-09-11T08:50:11.174851+0000","last_became_active":"2024-09-11T08:49:09.909429+0000","last_became_peered":"2024-09-11T08:49:09.909429+0000","last_unstale":"2024-09-11T08:50:11.174851+0000","last_undegraded":"2024-09-11T08:50:11.174851+0000","last_fullsized":"2024-09-11T08:50:11.174851+0000","mapping_epoch":210,"log_start":"214'400","ondisk_log_start":"214'400","created":206,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":5907,"log_dups_size":0,"ondisk_log_size":5907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:19.361954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413335552,"num_objects":6307,"num_object_clones":0,"num_object_copies":18921,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6307,"num_write_kb":403648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"200'24","reported_seq":268,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.890769+0000","last_change":"2024-09-11T08:49:09.930720+0000","last_active":"2024-09-11T08:50:00.890769+0000","last_peered":"2024-09-11T08:50:00.890769+0000","last_clean":"2024-09-11T08:50:00.890769+0000","last_became_active":"2024-09-11T08:49:09.911956+0000","last_became_peered":"2024-09-11T08:49:09.911956+0000","last_unstale":"2024-09-11T08:50:00.890769+0000","last_undegraded":"2024-09-11T08:50:00.890769+0000","last_fullsized":"2024-09-11T08:50:00.890769+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:48:37.680688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:37.680688+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037660860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.894753+0000","last_change":"2024-09-11T08:49:09.910130+0000","last_active":"2024-09-11T08:50:00.894753+0000","last_peered":"2024-09-11T08:50:00.894753+0000","last_clean":"2024-09-11T08:50:00.894753+0000","last_became_active":"2024-09-11T08:49:09.909803+0000","last_became_peered":"2024-09-11T08:49:09.909803+0000","last_unstale":"2024-09-11T08:50:00.894753+0000","last_undegraded":"2024-09-11T08:50:00.894753+0000","last_fullsized":"2024-09-11T08:50:00.894753+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:20.455081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.455081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":945,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:10.011855+0000","last_change":"2024-09-11T08:49:27.805277+0000","last_active":"2024-09-11T08:50:10.011855+0000","last_peered":"2024-09-11T08:50:10.011855+0000","last_clean":"2024-09-11T08:50:10.011855+0000","last_became_active":"2024-09-11T08:49:09.912659+0000","last_became_peered":"2024-09-11T08:49:09.912659+0000","last_unstale":"2024-09-11T08:50:10.011855+0000","last_undegraded":"2024-09-11T08:50:10.011855+0000","last_fullsized":"2024-09-11T08:50:10.011855+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:49:27.805050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:27.805050+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:57.498333+0000","scrub_duration":0.0046228930000000003,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":683,"num_read_kb":678,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":231,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.887698+0000","last_change":"2024-09-11T08:49:09.903588+0000","last_active":"2024-09-11T08:50:00.887698+0000","last_peered":"2024-09-11T08:50:00.887698+0000","last_clean":"2024-09-11T08:50:00.887698+0000","last_became_active":"2024-09-11T08:49:09.903283+0000","last_became_peered":"2024-09-11T08:49:09.903283+0000","last_unstale":"2024-09-11T08:50:00.887698+0000","last_undegraded":"2024-09-11T08:50:00.887698+0000","last_fullsized":"2024-09-11T08:50:00.887698+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:30.532701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.532701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012853409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":223,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.889021+0000","last_change":"2024-09-11T08:48:33.025967+0000","last_active":"2024-09-11T08:50:00.889021+0000","last_peered":"2024-09-11T08:50:00.889021+0000","last_clean":"2024-09-11T08:50:00.889021+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:50:00.889021+0000","last_undegraded":"2024-09-11T08:50:00.889021+0000","last_fullsized":"2024-09-11T08:50:00.889021+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:33.025897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:48:33.025897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007471208,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"227'5864","reported_seq":5903,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:06.801607+0000","last_change":"2024-09-11T08:49:09.915717+0000","last_active":"2024-09-11T08:50:06.801607+0000","last_peered":"2024-09-11T08:50:06.801607+0000","last_clean":"2024-09-11T08:50:06.801607+0000","last_became_active":"2024-09-11T08:49:09.914863+0000","last_became_peered":"2024-09-11T08:49:09.914863+0000","last_unstale":"2024-09-11T08:50:06.801607+0000","last_undegraded":"2024-09-11T08:50:06.801607+0000","last_fullsized":"2024-09-11T08:50:06.801607+0000","mapping_epoch":210,"log_start":"215'1600","ondisk_log_start":"215'1600","created":206,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4264,"log_dups_size":0,"ondisk_log_size":4264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:26.375120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":384303104,"num_objects":5864,"num_object_clones":0,"num_object_copies":17592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5864,"num_write_kb":375296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"201'27","reported_seq":124,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.890937+0000","last_change":"2024-09-11T08:48:47.827201+0000","last_active":"2024-09-11T08:50:00.890937+0000","last_peered":"2024-09-11T08:50:00.890937+0000","last_clean":"2024-09-11T08:50:00.890937+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:50:00.890937+0000","last_undegraded":"2024-09-11T08:50:00.890937+0000","last_fullsized":"2024-09-11T08:50:00.890937+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'27","last_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_deep_scrub":"201'27","last_deep_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_clean_scrub_stamp":"2024-09-11T08:48:47.827134+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:12.693583+0000","scrub_duration":0.0038402419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":218,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.896242+0000","last_change":"2024-09-11T08:45:18.127946+0000","last_active":"2024-09-11T08:50:00.896242+0000","last_peered":"2024-09-11T08:50:00.896242+0000","last_clean":"2024-09-11T08:50:00.896242+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:50:00.896242+0000","last_undegraded":"2024-09-11T08:50:00.896242+0000","last_fullsized":"2024-09-11T08:50:00.896242+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:18.127857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:18.127857+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018334099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"199'81","reported_seq":291,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.894990+0000","last_change":"2024-09-11T08:49:09.926313+0000","last_active":"2024-09-11T08:50:00.894990+0000","last_peered":"2024-09-11T08:50:00.894990+0000","last_clean":"2024-09-11T08:50:00.894990+0000","last_became_active":"2024-09-11T08:49:09.904681+0000","last_became_peered":"2024-09-11T08:49:09.904681+0000","last_unstale":"2024-09-11T08:50:00.894990+0000","last_undegraded":"2024-09-11T08:50:00.894990+0000","last_fullsized":"2024-09-11T08:50:00.894990+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:05.407039+0000","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":225,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.889553+0000","last_change":"2024-09-11T08:47:35.433723+0000","last_active":"2024-09-11T08:50:00.889553+0000","last_peered":"2024-09-11T08:50:00.889553+0000","last_clean":"2024-09-11T08:50:00.889553+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:50:00.889553+0000","last_undegraded":"2024-09-11T08:50:00.889553+0000","last_fullsized":"2024-09-11T08:50:00.889553+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:35.433651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:35.433651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013907892999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":219,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.887585+0000","last_change":"2024-09-11T08:48:19.308968+0000","last_active":"2024-09-11T08:50:00.887585+0000","last_peered":"2024-09-11T08:50:00.887585+0000","last_clean":"2024-09-11T08:50:00.887585+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:50:00.887585+0000","last_undegraded":"2024-09-11T08:50:00.887585+0000","last_fullsized":"2024-09-11T08:50:00.887585+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:19.308869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.308869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011385526999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"227'6315","reported_seq":6343,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:11.172114+0000","last_change":"2024-09-11T08:49:05.874474+0000","last_active":"2024-09-11T08:50:11.172114+0000","last_peered":"2024-09-11T08:50:11.172114+0000","last_clean":"2024-09-11T08:50:11.172114+0000","last_became_active":"2024-09-11T08:49:05.874151+0000","last_became_peered":"2024-09-11T08:49:05.874151+0000","last_unstale":"2024-09-11T08:50:11.172114+0000","last_undegraded":"2024-09-11T08:50:11.172114+0000","last_fullsized":"2024-09-11T08:50:11.172114+0000","mapping_epoch":206,"log_start":"215'1800","ondisk_log_start":"215'1800","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4515,"log_dups_size":0,"ondisk_log_size":4515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:33.548168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413859840,"num_objects":6315,"num_object_clones":0,"num_object_copies":18945,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6315,"num_write_kb":404160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"200'19","reported_seq":308,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.896694+0000","last_change":"2024-09-11T08:48:20.860266+0000","last_active":"2024-09-11T08:50:00.896694+0000","last_peered":"2024-09-11T08:50:00.896694+0000","last_clean":"2024-09-11T08:50:00.896694+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:50:00.896694+0000","last_undegraded":"2024-09-11T08:50:00.896694+0000","last_fullsized":"2024-09-11T08:50:00.896694+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:20.860186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.860186+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045417560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":210,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.894999+0000","last_change":"2024-09-11T08:49:18.665872+0000","last_active":"2024-09-11T08:50:00.894999+0000","last_peered":"2024-09-11T08:50:00.894999+0000","last_clean":"2024-09-11T08:50:00.894999+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:50:00.894999+0000","last_undegraded":"2024-09-11T08:50:00.894999+0000","last_fullsized":"2024-09-11T08:50:00.894999+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:18.665790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:49:18.665790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:40.249031+0000","scrub_duration":0.0041745330000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":228,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.887402+0000","last_change":"2024-09-11T08:49:29.269384+0000","last_active":"2024-09-11T08:50:00.887402+0000","last_peered":"2024-09-11T08:50:00.887402+0000","last_clean":"2024-09-11T08:50:00.887402+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:50:00.887402+0000","last_undegraded":"2024-09-11T08:50:00.887402+0000","last_fullsized":"2024-09-11T08:50:00.887402+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:29.269302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:49:29.269302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:40.650236+0000","scrub_duration":0.0083588919999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":233,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:06.234541+0000","last_change":"2024-09-11T08:50:06.234445+0000","last_active":"2024-09-11T08:50:06.234541+0000","last_peered":"2024-09-11T08:50:06.234541+0000","last_clean":"2024-09-11T08:50:06.234541+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:50:06.234541+0000","last_undegraded":"2024-09-11T08:50:06.234541+0000","last_fullsized":"2024-09-11T08:50:06.234541+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:06.234119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:06.234119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:19.471101+0000","scrub_duration":0.0034529980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"227'5916","reported_seq":5941,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:06.595981+0000","last_change":"2024-09-11T08:49:06.519555+0000","last_active":"2024-09-11T08:50:06.595981+0000","last_peered":"2024-09-11T08:50:06.595981+0000","last_clean":"2024-09-11T08:50:06.595981+0000","last_became_active":"2024-09-11T08:49:06.519263+0000","last_became_peered":"2024-09-11T08:49:06.519263+0000","last_unstale":"2024-09-11T08:50:06.595981+0000","last_undegraded":"2024-09-11T08:50:06.595981+0000","last_fullsized":"2024-09-11T08:50:06.595981+0000","mapping_epoch":206,"log_start":"216'1800","ondisk_log_start":"216'1800","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4116,"log_dups_size":0,"ondisk_log_size":4116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387710976,"num_objects":5916,"num_object_clones":0,"num_object_copies":17748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5916,"num_write_kb":378624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"200'31","reported_seq":267,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.896608+0000","last_change":"2024-09-11T08:48:43.728009+0000","last_active":"2024-09-11T08:50:00.896608+0000","last_peered":"2024-09-11T08:50:00.896608+0000","last_clean":"2024-09-11T08:50:00.896608+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:50:00.896608+0000","last_undegraded":"2024-09-11T08:50:00.896608+0000","last_fullsized":"2024-09-11T08:50:00.896608+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:48:43.727938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:43.727938+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016024792999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":249,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:02.223944+0000","last_change":"2024-09-11T08:50:02.223875+0000","last_active":"2024-09-11T08:50:02.223944+0000","last_peered":"2024-09-11T08:50:02.223944+0000","last_clean":"2024-09-11T08:50:02.223944+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:50:02.223944+0000","last_undegraded":"2024-09-11T08:50:02.223944+0000","last_fullsized":"2024-09-11T08:50:02.223944+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:02.223777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_clean_scrub_stamp":"2024-09-11T08:50:02.223777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:26.269731+0000","scrub_duration":0.0026962679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":235,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.889051+0000","last_change":"2024-09-11T08:49:09.903898+0000","last_active":"2024-09-11T08:50:00.889051+0000","last_peered":"2024-09-11T08:50:00.889051+0000","last_clean":"2024-09-11T08:50:00.889051+0000","last_became_active":"2024-09-11T08:49:09.903589+0000","last_became_peered":"2024-09-11T08:49:09.903589+0000","last_unstale":"2024-09-11T08:50:00.889051+0000","last_undegraded":"2024-09-11T08:50:00.889051+0000","last_fullsized":"2024-09-11T08:50:00.889051+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:22.989179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:22.989179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033708700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":249,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.902965+0000","last_change":"2024-09-11T08:49:29.528952+0000","last_active":"2024-09-11T08:50:00.902965+0000","last_peered":"2024-09-11T08:50:00.902965+0000","last_clean":"2024-09-11T08:50:00.902965+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:50:00.902965+0000","last_undegraded":"2024-09-11T08:50:00.902965+0000","last_fullsized":"2024-09-11T08:50:00.902965+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:29.528873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:29.528873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:46.400819+0000","scrub_duration":0.0024493430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"227'5983","reported_seq":6012,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:06.828006+0000","last_change":"2024-09-11T08:49:06.077463+0000","last_active":"2024-09-11T08:50:06.828006+0000","last_peered":"2024-09-11T08:50:06.828006+0000","last_clean":"2024-09-11T08:50:06.828006+0000","last_became_active":"2024-09-11T08:49:06.077144+0000","last_became_peered":"2024-09-11T08:49:06.077144+0000","last_unstale":"2024-09-11T08:50:06.828006+0000","last_undegraded":"2024-09-11T08:50:06.828006+0000","last_fullsized":"2024-09-11T08:50:06.828006+0000","mapping_epoch":206,"log_start":"215'1700","ondisk_log_start":"215'1700","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4283,"log_dups_size":0,"ondisk_log_size":4283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:14.241327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":392101888,"num_objects":5983,"num_object_clones":0,"num_object_copies":17949,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5983,"num_write_kb":382912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"200'20","reported_seq":285,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.896095+0000","last_change":"2024-09-11T08:48:21.885766+0000","last_active":"2024-09-11T08:50:00.896095+0000","last_peered":"2024-09-11T08:50:00.896095+0000","last_clean":"2024-09-11T08:50:00.896095+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:50:00.896095+0000","last_undegraded":"2024-09-11T08:50:00.896095+0000","last_fullsized":"2024-09-11T08:50:00.896095+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:48:21.885681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:21.885681+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051165580000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":229,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:04.191932+0000","last_change":"2024-09-11T08:50:04.191785+0000","last_active":"2024-09-11T08:50:04.191932+0000","last_peered":"2024-09-11T08:50:04.191932+0000","last_clean":"2024-09-11T08:50:04.191932+0000","last_became_active":"2024-09-11T08:49:09.911101+0000","last_became_peered":"2024-09-11T08:49:09.911101+0000","last_unstale":"2024-09-11T08:50:04.191932+0000","last_undegraded":"2024-09-11T08:50:04.191932+0000","last_fullsized":"2024-09-11T08:50:04.191932+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:04.191702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:04.191702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:17.639050+0000","scrub_duration":0.0034932320000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":390,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:02.807432+0000","last_change":"2024-09-11T08:49:27.577662+0000","last_active":"2024-09-11T08:50:02.807432+0000","last_peered":"2024-09-11T08:50:02.807432+0000","last_clean":"2024-09-11T08:50:02.807432+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:50:02.807432+0000","last_undegraded":"2024-09-11T08:50:02.807432+0000","last_fullsized":"2024-09-11T08:50:02.807432+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:49:27.577575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:49:27.577575+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:34.177695+0000","scrub_duration":0.0084200460000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":214,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:05.815340+0000","last_change":"2024-09-11T08:50:05.815267+0000","last_active":"2024-09-11T08:50:05.815340+0000","last_peered":"2024-09-11T08:50:05.815340+0000","last_clean":"2024-09-11T08:50:05.815340+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:50:05.815340+0000","last_undegraded":"2024-09-11T08:50:05.815340+0000","last_fullsized":"2024-09-11T08:50:05.815340+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:05.815178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:50:05.815178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:10.392227+0000","scrub_duration":0.0078777670000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"227'5805","reported_seq":5833,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:06.326411+0000","last_change":"2024-09-11T08:49:06.519232+0000","last_active":"2024-09-11T08:50:06.326411+0000","last_peered":"2024-09-11T08:50:06.326411+0000","last_clean":"2024-09-11T08:50:06.326411+0000","last_became_active":"2024-09-11T08:49:06.518991+0000","last_became_peered":"2024-09-11T08:49:06.518991+0000","last_unstale":"2024-09-11T08:50:06.326411+0000","last_undegraded":"2024-09-11T08:50:06.326411+0000","last_fullsized":"2024-09-11T08:50:06.326411+0000","mapping_epoch":206,"log_start":"215'1100","ondisk_log_start":"215'1100","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4705,"log_dups_size":0,"ondisk_log_size":4705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:34.806916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":380436480,"num_objects":5805,"num_object_clones":0,"num_object_copies":17415,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5805,"num_write_kb":371520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"200'15","reported_seq":253,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.889479+0000","last_change":"2024-09-11T08:49:09.925808+0000","last_active":"2024-09-11T08:50:00.889479+0000","last_peered":"2024-09-11T08:50:00.889479+0000","last_clean":"2024-09-11T08:50:00.889479+0000","last_became_active":"2024-09-11T08:49:09.915666+0000","last_became_peered":"2024-09-11T08:49:09.915666+0000","last_unstale":"2024-09-11T08:50:00.889479+0000","last_undegraded":"2024-09-11T08:50:00.889479+0000","last_fullsized":"2024-09-11T08:50:00.889479+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:48:41.787219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.787219+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:08.422014+0000","scrub_duration":0.0060192070000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":232,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.885583+0000","last_change":"2024-09-11T08:49:30.234576+0000","last_active":"2024-09-11T08:50:00.885583+0000","last_peered":"2024-09-11T08:50:00.885583+0000","last_clean":"2024-09-11T08:50:00.885583+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:50:00.885583+0000","last_undegraded":"2024-09-11T08:50:00.885583+0000","last_fullsized":"2024-09-11T08:50:00.885583+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:30.234078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:49:30.234078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:55.702848+0000","scrub_duration":0.0025717829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":229,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.904162+0000","last_change":"2024-09-11T08:48:24.380508+0000","last_active":"2024-09-11T08:50:00.904162+0000","last_peered":"2024-09-11T08:50:00.904162+0000","last_clean":"2024-09-11T08:50:00.904162+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:50:00.904162+0000","last_undegraded":"2024-09-11T08:50:00.904162+0000","last_fullsized":"2024-09-11T08:50:00.904162+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:24.380424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:24.380424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018256864000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":29,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.895663+0000","last_change":"2024-09-11T08:49:09.912893+0000","last_active":"2024-09-11T08:50:00.895663+0000","last_peered":"2024-09-11T08:50:00.895663+0000","last_clean":"2024-09-11T08:50:00.895663+0000","last_became_active":"2024-09-11T08:49:09.912556+0000","last_became_peered":"2024-09-11T08:49:09.912556+0000","last_unstale":"2024-09-11T08:50:00.895663+0000","last_undegraded":"2024-09-11T08:50:00.895663+0000","last_fullsized":"2024-09-11T08:50:00.895663+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:47.699453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:47:47.699453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"227'5865","reported_seq":5893,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:06.300117+0000","last_change":"2024-09-11T08:49:05.871158+0000","last_active":"2024-09-11T08:50:06.300117+0000","last_peered":"2024-09-11T08:50:06.300117+0000","last_clean":"2024-09-11T08:50:06.300117+0000","last_became_active":"2024-09-11T08:49:05.870797+0000","last_became_peered":"2024-09-11T08:49:05.870797+0000","last_unstale":"2024-09-11T08:50:06.300117+0000","last_undegraded":"2024-09-11T08:50:06.300117+0000","last_fullsized":"2024-09-11T08:50:06.300117+0000","mapping_epoch":206,"log_start":"215'1200","ondisk_log_start":"215'1200","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4665,"log_dups_size":0,"ondisk_log_size":4665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:33.662137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":384368640,"num_objects":5865,"num_object_clones":0,"num_object_copies":17595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5865,"num_write_kb":375360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"200'35","reported_seq":133,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.890469+0000","last_change":"2024-09-11T08:49:22.127201+0000","last_active":"2024-09-11T08:50:00.890469+0000","last_peered":"2024-09-11T08:50:00.890469+0000","last_clean":"2024-09-11T08:50:00.890469+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:50:00.890469+0000","last_undegraded":"2024-09-11T08:50:00.890469+0000","last_fullsized":"2024-09-11T08:50:00.890469+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'35","last_scrub_stamp":"2024-09-11T08:49:22.127119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:49:22.127119+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:33.163032+0000","scrub_duration":0.0038731960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":229,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.894676+0000","last_change":"2024-09-11T08:47:44.411099+0000","last_active":"2024-09-11T08:50:00.894676+0000","last_peered":"2024-09-11T08:50:00.894676+0000","last_clean":"2024-09-11T08:50:00.894676+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:50:00.894676+0000","last_undegraded":"2024-09-11T08:50:00.894676+0000","last_fullsized":"2024-09-11T08:50:00.894676+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.411022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.411022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047762170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":255,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.905321+0000","last_change":"2024-09-11T08:49:25.610133+0000","last_active":"2024-09-11T08:50:00.905321+0000","last_peered":"2024-09-11T08:50:00.905321+0000","last_clean":"2024-09-11T08:50:00.905321+0000","last_became_active":"2024-09-11T08:49:09.907542+0000","last_became_peered":"2024-09-11T08:49:09.907542+0000","last_unstale":"2024-09-11T08:50:00.905321+0000","last_undegraded":"2024-09-11T08:50:00.905321+0000","last_fullsized":"2024-09-11T08:50:00.905321+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:25.609666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:49:25.609666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:46.984958+0000","scrub_duration":0.003585071,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":216,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.894822+0000","last_change":"2024-09-11T08:48:33.065142+0000","last_active":"2024-09-11T08:50:00.894822+0000","last_peered":"2024-09-11T08:50:00.894822+0000","last_clean":"2024-09-11T08:50:00.894822+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:50:00.894822+0000","last_undegraded":"2024-09-11T08:50:00.894822+0000","last_fullsized":"2024-09-11T08:50:00.894822+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:33.065053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:33.065053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013998086999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"227'5958","reported_seq":5986,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:06.593248+0000","last_change":"2024-09-11T08:49:05.871175+0000","last_active":"2024-09-11T08:50:06.593248+0000","last_peered":"2024-09-11T08:50:06.593248+0000","last_clean":"2024-09-11T08:50:06.593248+0000","last_became_active":"2024-09-11T08:49:05.870890+0000","last_became_peered":"2024-09-11T08:49:05.870890+0000","last_unstale":"2024-09-11T08:50:06.593248+0000","last_undegraded":"2024-09-11T08:50:06.593248+0000","last_fullsized":"2024-09-11T08:50:06.593248+0000","mapping_epoch":206,"log_start":"215'1800","ondisk_log_start":"215'1800","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4158,"log_dups_size":0,"ondisk_log_size":4158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:21.232303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":390463488,"num_objects":5958,"num_object_clones":0,"num_object_copies":17874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5958,"num_write_kb":381312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"200'18","reported_seq":250,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:03.790710+0000","last_change":"2024-09-11T08:50:03.790643+0000","last_active":"2024-09-11T08:50:03.790710+0000","last_peered":"2024-09-11T08:50:03.790710+0000","last_clean":"2024-09-11T08:50:03.790710+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:50:03.790710+0000","last_undegraded":"2024-09-11T08:50:03.790710+0000","last_fullsized":"2024-09-11T08:50:03.790710+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'18","last_scrub_stamp":"2024-09-11T08:50:03.790561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:03.790561+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:12.360934+0000","scrub_duration":0.050104567000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":229,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.890926+0000","last_change":"2024-09-11T08:48:47.033544+0000","last_active":"2024-09-11T08:50:00.890926+0000","last_peered":"2024-09-11T08:50:00.890926+0000","last_clean":"2024-09-11T08:50:00.890926+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:50:00.890926+0000","last_undegraded":"2024-09-11T08:50:00.890926+0000","last_fullsized":"2024-09-11T08:50:00.890926+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:47.033487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:47.033487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014934150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":248,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.907399+0000","last_change":"2024-09-11T08:49:09.907865+0000","last_active":"2024-09-11T08:50:00.907399+0000","last_peered":"2024-09-11T08:50:00.907399+0000","last_clean":"2024-09-11T08:50:00.907399+0000","last_became_active":"2024-09-11T08:49:09.907535+0000","last_became_peered":"2024-09-11T08:49:09.907535+0000","last_unstale":"2024-09-11T08:50:00.907399+0000","last_undegraded":"2024-09-11T08:50:00.907399+0000","last_fullsized":"2024-09-11T08:50:00.907399+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:21.336766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:21.336766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077730009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":203,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.898624+0000","last_change":"2024-09-11T08:45:33.228265+0000","last_active":"2024-09-11T08:50:00.898624+0000","last_peered":"2024-09-11T08:50:00.898624+0000","last_clean":"2024-09-11T08:50:00.898624+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:50:00.898624+0000","last_undegraded":"2024-09-11T08:50:00.898624+0000","last_fullsized":"2024-09-11T08:50:00.898624+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:45:33.228082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064426500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"227'5888","reported_seq":5917,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:06.325839+0000","last_change":"2024-09-11T08:49:09.900869+0000","last_active":"2024-09-11T08:50:06.325839+0000","last_peered":"2024-09-11T08:50:06.325839+0000","last_clean":"2024-09-11T08:50:06.325839+0000","last_became_active":"2024-09-11T08:49:09.900548+0000","last_became_peered":"2024-09-11T08:49:09.900548+0000","last_unstale":"2024-09-11T08:50:06.325839+0000","last_undegraded":"2024-09-11T08:50:06.325839+0000","last_fullsized":"2024-09-11T08:50:06.325839+0000","mapping_epoch":210,"log_start":"215'1200","ondisk_log_start":"215'1200","created":206,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4688,"log_dups_size":0,"ondisk_log_size":4688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:08.175978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385875968,"num_objects":5888,"num_object_clones":0,"num_object_copies":17664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5888,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"200'34","reported_seq":325,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.885827+0000","last_change":"2024-09-11T08:48:20.352021+0000","last_active":"2024-09-11T08:50:00.885827+0000","last_peered":"2024-09-11T08:50:00.885827+0000","last_clean":"2024-09-11T08:50:00.885827+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:50:00.885827+0000","last_undegraded":"2024-09-11T08:50:00.885827+0000","last_fullsized":"2024-09-11T08:50:00.885827+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'22","last_scrub_stamp":"2024-09-11T08:48:20.351963+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.351963+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058139180000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":262,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.890053+0000","last_change":"2024-09-11T08:49:17.884190+0000","last_active":"2024-09-11T08:50:00.890053+0000","last_peered":"2024-09-11T08:50:00.890053+0000","last_clean":"2024-09-11T08:50:00.890053+0000","last_became_active":"2024-09-11T08:49:09.904656+0000","last_became_peered":"2024-09-11T08:49:09.904656+0000","last_unstale":"2024-09-11T08:50:00.890053+0000","last_undegraded":"2024-09-11T08:50:00.890053+0000","last_fullsized":"2024-09-11T08:50:00.890053+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:49:17.884108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:17.884108+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:33.993275+0000","scrub_duration":0.0031968909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"168'10","reported_seq":399,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:03.886015+0000","last_change":"2024-09-11T08:48:41.038840+0000","last_active":"2024-09-11T08:50:03.886015+0000","last_peered":"2024-09-11T08:50:03.886015+0000","last_clean":"2024-09-11T08:50:03.886015+0000","last_became_active":"2024-09-11T08:46:01.983055+0000","last_became_peered":"2024-09-11T08:46:01.983055+0000","last_unstale":"2024-09-11T08:50:03.886015+0000","last_undegraded":"2024-09-11T08:50:03.886015+0000","last_fullsized":"2024-09-11T08:50:03.886015+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"168'10","last_scrub_stamp":"2024-09-11T08:48:41.038786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.038786+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00164314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":229,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.885334+0000","last_change":"2024-09-11T08:49:18.301286+0000","last_active":"2024-09-11T08:50:00.885334+0000","last_peered":"2024-09-11T08:50:00.885334+0000","last_clean":"2024-09-11T08:50:00.885334+0000","last_became_active":"2024-09-11T08:49:09.906201+0000","last_became_peered":"2024-09-11T08:49:09.906201+0000","last_unstale":"2024-09-11T08:50:00.885334+0000","last_undegraded":"2024-09-11T08:50:00.885334+0000","last_fullsized":"2024-09-11T08:50:00.885334+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:18.301203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:18.301203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:37.257368+0000","scrub_duration":0.0032551070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":259,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.891397+0000","last_change":"2024-09-11T08:49:34.002358+0000","last_active":"2024-09-11T08:50:00.891397+0000","last_peered":"2024-09-11T08:50:00.891397+0000","last_clean":"2024-09-11T08:50:00.891397+0000","last_became_active":"2024-09-11T08:49:09.910329+0000","last_became_peered":"2024-09-11T08:49:09.910329+0000","last_unstale":"2024-09-11T08:50:00.891397+0000","last_undegraded":"2024-09-11T08:50:00.891397+0000","last_fullsized":"2024-09-11T08:50:00.891397+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:34.002280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:49:34.002280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:39.769687+0000","scrub_duration":0.0089946609999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"200'12","reported_seq":253,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.899753+0000","last_change":"2024-09-11T08:49:15.347679+0000","last_active":"2024-09-11T08:50:00.899753+0000","last_peered":"2024-09-11T08:50:00.899753+0000","last_clean":"2024-09-11T08:50:00.899753+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:50:00.899753+0000","last_undegraded":"2024-09-11T08:50:00.899753+0000","last_fullsized":"2024-09-11T08:50:00.899753+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'12","last_scrub_stamp":"2024-09-11T08:49:15.347591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:49:15.347591+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:31.103948+0000","scrub_duration":0.0048379369999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":215,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.887753+0000","last_change":"2024-09-11T08:45:08.675251+0000","last_active":"2024-09-11T08:50:00.887753+0000","last_peered":"2024-09-11T08:50:00.887753+0000","last_clean":"2024-09-11T08:50:00.887753+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:50:00.887753+0000","last_undegraded":"2024-09-11T08:50:00.887753+0000","last_fullsized":"2024-09-11T08:50:00.887753+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:08.675179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:08.675179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003515392,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":243,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:08.089758+0000","last_change":"2024-09-11T08:50:08.089714+0000","last_active":"2024-09-11T08:50:08.089758+0000","last_peered":"2024-09-11T08:50:08.089758+0000","last_clean":"2024-09-11T08:50:08.089758+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:50:08.089758+0000","last_undegraded":"2024-09-11T08:50:08.089758+0000","last_fullsized":"2024-09-11T08:50:08.089758+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:08.089015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:08.089015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:36.555156+0000","scrub_duration":0.0075245379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"200'15","reported_seq":253,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.897031+0000","last_change":"2024-09-11T08:49:09.933608+0000","last_active":"2024-09-11T08:50:00.897031+0000","last_peered":"2024-09-11T08:50:00.897031+0000","last_clean":"2024-09-11T08:50:00.897031+0000","last_became_active":"2024-09-11T08:49:09.916927+0000","last_became_peered":"2024-09-11T08:49:09.916927+0000","last_unstale":"2024-09-11T08:50:00.897031+0000","last_undegraded":"2024-09-11T08:50:00.897031+0000","last_fullsized":"2024-09-11T08:50:00.897031+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:47:44.737721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.737721+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032009640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":232,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.895927+0000","last_change":"2024-09-11T08:48:20.977749+0000","last_active":"2024-09-11T08:50:00.895927+0000","last_peered":"2024-09-11T08:50:00.895927+0000","last_clean":"2024-09-11T08:50:00.895927+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:50:00.895927+0000","last_undegraded":"2024-09-11T08:50:00.895927+0000","last_fullsized":"2024-09-11T08:50:00.895927+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:20.977665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.977665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075643139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":205,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.899562+0000","last_change":"2024-09-11T08:45:23.213642+0000","last_active":"2024-09-11T08:50:00.899562+0000","last_peered":"2024-09-11T08:50:00.899562+0000","last_clean":"2024-09-11T08:50:00.899562+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:50:00.899562+0000","last_undegraded":"2024-09-11T08:50:00.899562+0000","last_fullsized":"2024-09-11T08:50:00.899562+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:23.213592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:23.213592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001600473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":234,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.902435+0000","last_change":"2024-09-11T08:48:47.485315+0000","last_active":"2024-09-11T08:50:00.902435+0000","last_peered":"2024-09-11T08:50:00.902435+0000","last_clean":"2024-09-11T08:50:00.902435+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:50:00.902435+0000","last_undegraded":"2024-09-11T08:50:00.902435+0000","last_fullsized":"2024-09-11T08:50:00.902435+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:47.485264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_clean_scrub_stamp":"2024-09-11T08:48:47.485264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:17.166643+0000","scrub_duration":0.0019043490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"200'15","reported_seq":248,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.886442+0000","last_change":"2024-09-11T08:48:36.403872+0000","last_active":"2024-09-11T08:50:00.886442+0000","last_peered":"2024-09-11T08:50:00.886442+0000","last_clean":"2024-09-11T08:50:00.886442+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:50:00.886442+0000","last_undegraded":"2024-09-11T08:50:00.886442+0000","last_fullsized":"2024-09-11T08:50:00.886442+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:48:36.403646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:36.403646+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057924789999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.889453+0000","last_change":"2024-09-11T08:49:09.912712+0000","last_active":"2024-09-11T08:50:00.889453+0000","last_peered":"2024-09-11T08:50:00.889453+0000","last_clean":"2024-09-11T08:50:00.889453+0000","last_became_active":"2024-09-11T08:49:09.912417+0000","last_became_peered":"2024-09-11T08:49:09.912417+0000","last_unstale":"2024-09-11T08:50:00.889453+0000","last_undegraded":"2024-09-11T08:50:00.889453+0000","last_fullsized":"2024-09-11T08:50:00.889453+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:45.751518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:45.751518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":350,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:02.808503+0000","last_change":"2024-09-11T08:45:20.119727+0000","last_active":"2024-09-11T08:50:02.808503+0000","last_peered":"2024-09-11T08:50:02.808503+0000","last_clean":"2024-09-11T08:50:02.808503+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:50:02.808503+0000","last_undegraded":"2024-09-11T08:50:02.808503+0000","last_fullsized":"2024-09-11T08:50:02.808503+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:20.119626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:20.119626+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018087660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":204,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.896978+0000","last_change":"2024-09-11T08:48:41.648829+0000","last_active":"2024-09-11T08:50:00.896978+0000","last_peered":"2024-09-11T08:50:00.896978+0000","last_clean":"2024-09-11T08:50:00.896978+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:50:00.896978+0000","last_undegraded":"2024-09-11T08:50:00.896978+0000","last_fullsized":"2024-09-11T08:50:00.896978+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:41.648756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.648756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019862030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"200'25","reported_seq":252,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.899443+0000","last_change":"2024-09-11T08:49:28.382609+0000","last_active":"2024-09-11T08:50:00.899443+0000","last_peered":"2024-09-11T08:50:00.899443+0000","last_clean":"2024-09-11T08:50:00.899443+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:50:00.899443+0000","last_undegraded":"2024-09-11T08:50:00.899443+0000","last_fullsized":"2024-09-11T08:50:00.899443+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:49:28.382409+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:49:28.382409+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:28.482336+0000","scrub_duration":0.0046362419999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":229,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.894854+0000","last_change":"2024-09-11T08:48:39.064220+0000","last_active":"2024-09-11T08:50:00.894854+0000","last_peered":"2024-09-11T08:50:00.894854+0000","last_clean":"2024-09-11T08:50:00.894854+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:50:00.894854+0000","last_undegraded":"2024-09-11T08:50:00.894854+0000","last_fullsized":"2024-09-11T08:50:00.894854+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:39.064163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_clean_scrub_stamp":"2024-09-11T08:48:39.064163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014228540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"211'9","reported_seq":374,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:02.811708+0000","last_change":"2024-09-11T08:49:09.926226+0000","last_active":"2024-09-11T08:50:02.811708+0000","last_peered":"2024-09-11T08:50:02.811708+0000","last_clean":"2024-09-11T08:50:02.811708+0000","last_became_active":"2024-09-11T08:49:09.908380+0000","last_became_peered":"2024-09-11T08:49:09.908380+0000","last_unstale":"2024-09-11T08:50:02.811708+0000","last_undegraded":"2024-09-11T08:50:02.811708+0000","last_fullsized":"2024-09-11T08:50:02.811708+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:45:28.692100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.692100+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019579020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":39,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.889981+0000","last_change":"2024-09-11T08:49:21.855032+0000","last_active":"2024-09-11T08:50:00.889981+0000","last_peered":"2024-09-11T08:50:00.889981+0000","last_clean":"2024-09-11T08:50:00.889981+0000","last_became_active":"2024-09-11T08:49:09.914396+0000","last_became_peered":"2024-09-11T08:49:09.914396+0000","last_unstale":"2024-09-11T08:50:00.889981+0000","last_undegraded":"2024-09-11T08:50:00.889981+0000","last_fullsized":"2024-09-11T08:50:00.889981+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:21.854973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:21.854973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:48.440684+0000","scrub_duration":0.0064444309999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"200'19","reported_seq":261,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.895950+0000","last_change":"2024-09-11T08:49:16.951920+0000","last_active":"2024-09-11T08:50:00.895950+0000","last_peered":"2024-09-11T08:50:00.895950+0000","last_clean":"2024-09-11T08:50:00.895950+0000","last_became_active":"2024-09-11T08:49:16.937185+0000","last_became_peered":"2024-09-11T08:49:16.937185+0000","last_unstale":"2024-09-11T08:50:00.895950+0000","last_undegraded":"2024-09-11T08:50:00.895950+0000","last_fullsized":"2024-09-11T08:50:00.895950+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:25.792777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.792777+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045475469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":220,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:04.791697+0000","last_change":"2024-09-11T08:50:04.791525+0000","last_active":"2024-09-11T08:50:04.791697+0000","last_peered":"2024-09-11T08:50:04.791697+0000","last_clean":"2024-09-11T08:50:04.791697+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:50:04.791697+0000","last_undegraded":"2024-09-11T08:50:04.791697+0000","last_fullsized":"2024-09-11T08:50:04.791697+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:04.791327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:04.791327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:19.842232+0000","scrub_duration":0.0063425010000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":225,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.894898+0000","last_change":"2024-09-11T08:48:30.985518+0000","last_active":"2024-09-11T08:50:00.894898+0000","last_peered":"2024-09-11T08:50:00.894898+0000","last_clean":"2024-09-11T08:50:00.894898+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:50:00.894898+0000","last_undegraded":"2024-09-11T08:50:00.894898+0000","last_fullsized":"2024-09-11T08:50:00.894898+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:30.985271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.985271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099058919999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":213,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.896199+0000","last_change":"2024-09-11T08:49:09.906202+0000","last_active":"2024-09-11T08:50:00.896199+0000","last_peered":"2024-09-11T08:50:00.896199+0000","last_clean":"2024-09-11T08:50:00.896199+0000","last_became_active":"2024-09-11T08:49:09.905289+0000","last_became_peered":"2024-09-11T08:49:09.905289+0000","last_unstale":"2024-09-11T08:50:00.896199+0000","last_undegraded":"2024-09-11T08:50:00.896199+0000","last_fullsized":"2024-09-11T08:50:00.896199+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:45:28.259103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:45:28.259103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015518800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"200'15","reported_seq":265,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:03.319809+0000","last_change":"2024-09-11T08:50:03.319748+0000","last_active":"2024-09-11T08:50:03.319809+0000","last_peered":"2024-09-11T08:50:03.319809+0000","last_clean":"2024-09-11T08:50:03.319809+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:50:03.319809+0000","last_undegraded":"2024-09-11T08:50:03.319809+0000","last_fullsized":"2024-09-11T08:50:03.319809+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:50:03.319664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:03.319664+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:12.910342+0000","scrub_duration":0.0046538380000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":229,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.894348+0000","last_change":"2024-09-11T08:47:48.373407+0000","last_active":"2024-09-11T08:50:00.894348+0000","last_peered":"2024-09-11T08:50:00.894348+0000","last_clean":"2024-09-11T08:50:00.894348+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:50:00.894348+0000","last_undegraded":"2024-09-11T08:50:00.894348+0000","last_fullsized":"2024-09-11T08:50:00.894348+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:48.373336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:48.373336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001705364,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":225,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.898688+0000","last_change":"2024-09-11T08:49:27.342962+0000","last_active":"2024-09-11T08:50:00.898688+0000","last_peered":"2024-09-11T08:50:00.898688+0000","last_clean":"2024-09-11T08:50:00.898688+0000","last_became_active":"2024-09-11T08:49:09.910548+0000","last_became_peered":"2024-09-11T08:49:09.910548+0000","last_unstale":"2024-09-11T08:50:00.898688+0000","last_undegraded":"2024-09-11T08:50:00.898688+0000","last_fullsized":"2024-09-11T08:50:00.898688+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:27.342908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:49:27.342908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:27.754446+0000","scrub_duration":0.0020004860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":29,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.885758+0000","last_change":"2024-09-11T08:49:09.904856+0000","last_active":"2024-09-11T08:50:00.885758+0000","last_peered":"2024-09-11T08:50:00.885758+0000","last_clean":"2024-09-11T08:50:00.885758+0000","last_became_active":"2024-09-11T08:49:09.904622+0000","last_became_peered":"2024-09-11T08:49:09.904622+0000","last_unstale":"2024-09-11T08:50:00.885758+0000","last_undegraded":"2024-09-11T08:50:00.885758+0000","last_fullsized":"2024-09-11T08:50:00.885758+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:18.685188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.685188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"200'19","reported_seq":238,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.896388+0000","last_change":"2024-09-11T08:48:18.890324+0000","last_active":"2024-09-11T08:50:00.896388+0000","last_peered":"2024-09-11T08:50:00.896388+0000","last_clean":"2024-09-11T08:50:00.896388+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:50:00.896388+0000","last_undegraded":"2024-09-11T08:50:00.896388+0000","last_fullsized":"2024-09-11T08:50:00.896388+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:18.890233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.890233+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070086059999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":229,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.887821+0000","last_change":"2024-09-11T08:49:33.209392+0000","last_active":"2024-09-11T08:50:00.887821+0000","last_peered":"2024-09-11T08:50:00.887821+0000","last_clean":"2024-09-11T08:50:00.887821+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:50:00.887821+0000","last_undegraded":"2024-09-11T08:50:00.887821+0000","last_fullsized":"2024-09-11T08:50:00.887821+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:33.209345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:33.209345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:45.794127+0000","scrub_duration":0.0052610230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":228,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.889643+0000","last_change":"2024-09-11T08:48:18.958742+0000","last_active":"2024-09-11T08:50:00.889643+0000","last_peered":"2024-09-11T08:50:00.889643+0000","last_clean":"2024-09-11T08:50:00.889643+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:50:00.889643+0000","last_undegraded":"2024-09-11T08:50:00.889643+0000","last_fullsized":"2024-09-11T08:50:00.889643+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:18.958609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.958609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001704777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":233,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.898812+0000","last_change":"2024-09-11T08:49:09.905933+0000","last_active":"2024-09-11T08:50:00.898812+0000","last_peered":"2024-09-11T08:50:00.898812+0000","last_clean":"2024-09-11T08:50:00.898812+0000","last_became_active":"2024-09-11T08:49:09.905634+0000","last_became_peered":"2024-09-11T08:49:09.905634+0000","last_unstale":"2024-09-11T08:50:00.898812+0000","last_undegraded":"2024-09-11T08:50:00.898812+0000","last_fullsized":"2024-09-11T08:50:00.898812+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:44.807331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.807331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013867529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"200'10","reported_seq":263,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.899902+0000","last_change":"2024-09-11T08:48:32.020078+0000","last_active":"2024-09-11T08:50:00.899902+0000","last_peered":"2024-09-11T08:50:00.899902+0000","last_clean":"2024-09-11T08:50:00.899902+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:50:00.899902+0000","last_undegraded":"2024-09-11T08:50:00.899902+0000","last_fullsized":"2024-09-11T08:50:00.899902+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'6","last_scrub_stamp":"2024-09-11T08:48:32.019991+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:48:32.019991+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010394439,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":41,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.887841+0000","last_change":"2024-09-11T08:49:09.919241+0000","last_active":"2024-09-11T08:50:00.887841+0000","last_peered":"2024-09-11T08:50:00.887841+0000","last_clean":"2024-09-11T08:50:00.887841+0000","last_became_active":"2024-09-11T08:49:09.907483+0000","last_became_peered":"2024-09-11T08:49:09.907483+0000","last_unstale":"2024-09-11T08:50:00.887841+0000","last_undegraded":"2024-09-11T08:50:00.887841+0000","last_fullsized":"2024-09-11T08:50:00.887841+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:45:21.481634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:45:21.481634+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":89,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:05.193832+0000","last_change":"2024-09-11T08:50:05.193741+0000","last_active":"2024-09-11T08:50:05.193832+0000","last_peered":"2024-09-11T08:50:05.193832+0000","last_clean":"2024-09-11T08:50:05.193832+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:50:05.193832+0000","last_undegraded":"2024-09-11T08:50:05.193832+0000","last_fullsized":"2024-09-11T08:50:05.193832+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:05.193617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:05.193617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:31.887067+0000","scrub_duration":0.0042825290000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":233,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.889264+0000","last_change":"2024-09-11T08:49:30.788379+0000","last_active":"2024-09-11T08:50:00.889264+0000","last_peered":"2024-09-11T08:50:00.889264+0000","last_clean":"2024-09-11T08:50:00.889264+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:50:00.889264+0000","last_undegraded":"2024-09-11T08:50:00.889264+0000","last_fullsized":"2024-09-11T08:50:00.889264+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:30.788303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:30.788303+0000","last_clean_scrub_stamp":"2024-09-11T08:49:30.788303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:37.198202+0000","scrub_duration":0.0038913020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":233,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.889616+0000","last_change":"2024-09-11T08:49:29.487522+0000","last_active":"2024-09-11T08:50:00.889616+0000","last_peered":"2024-09-11T08:50:00.889616+0000","last_clean":"2024-09-11T08:50:00.889616+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:50:00.889616+0000","last_undegraded":"2024-09-11T08:50:00.889616+0000","last_fullsized":"2024-09-11T08:50:00.889616+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:29.487391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:29.487391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:53.969768+0000","scrub_duration":0.0091739200000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":229,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.900129+0000","last_change":"2024-09-11T08:49:09.919371+0000","last_active":"2024-09-11T08:50:00.900129+0000","last_peered":"2024-09-11T08:50:00.900129+0000","last_clean":"2024-09-11T08:50:00.900129+0000","last_became_active":"2024-09-11T08:49:09.918992+0000","last_became_peered":"2024-09-11T08:49:09.918992+0000","last_unstale":"2024-09-11T08:50:00.900129+0000","last_undegraded":"2024-09-11T08:50:00.900129+0000","last_fullsized":"2024-09-11T08:50:00.900129+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:40.033594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:48:40.033594+0000","last_clean_scrub_stamp":"2024-09-11T08:48:40.033594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022129379999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"200'19","reported_seq":113,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-09-11T08:50:10.181443+0000","last_change":"2024-09-11T08:50:10.181391+0000","last_active":"2024-09-11T08:50:10.181443+0000","last_peered":"2024-09-11T08:50:10.181443+0000","last_clean":"2024-09-11T08:50:10.181443+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:50:10.181443+0000","last_undegraded":"2024-09-11T08:50:10.181443+0000","last_fullsized":"2024-09-11T08:50:10.181443+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:50:10.181286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:10.181286+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:31.378775+0000","scrub_duration":0.0058024799999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":29,"reported_epoch":226,"state":"active+clean","last_fresh":"2024-09-11T08:50:00.890583+0000","last_change":"2024-09-11T08:49:09.905224+0000","last_active":"2024-09-11T08:50:00.890583+0000","last_peered":"2024-09-11T08:50:00.890583+0000","last_clean":"2024-09-11T08:50:00.890583+0000","last_became_active":"2024-09-11T08:49:09.904721+0000","last_became_peered":"2024-09-11T08:49:09.904721+0000","last_unstale":"2024-09-11T08:50:00.890583+0000","last_undegraded":"2024-09-11T08:50:00.890583+0000","last_fullsized":"2024-09-11T08:50:00.890583+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:32.555890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:47:32.555890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":6327238656,"num_objects":96546,"num_object_clones":0,"num_object_copies":289638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96546,"num_write_kb":6178944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19004981248,"data_stored":19004981248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72846,"ondisk_log_size":72846,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1119,"num_read_kb":876,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":3234,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":12426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":751,"ondisk_log_size":751,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":683,"num_read_kb":678,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299536,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4357236,"kb_used_data":4051564,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":89387916,"statfs":{"total":95995035648,"available":91533225984,"internally_reserved":0,"allocated":4148801536,"data_stored":4148399477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":1.2010000000000001,"15min":0.86099999999999999},"min":{"1min":0.27000000000000002,"5min":0.23200000000000001,"15min":0.22},"max":{"1min":2.0619999999999998,"5min":32.171999999999997,"15min":32.171999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.393,"15min":0.97399999999999998},"min":{"1min":0.36499999999999999,"5min":0.221,"15min":0.21199999999999999},"max":{"1min":3.1960000000000002,"5min":32.149999999999999,"15min":32.149999999999999},"last":0.75700000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":1.4490000000000001,"15min":1.0009999999999999},"min":{"1min":0.24099999999999999,"5min":0.224,"15min":0.224},"max":{"1min":2.101,"5min":33.692999999999998,"15min":33.692999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.167,"5min":1.4399999999999999,"15min":0.94799999999999995},"min":{"1min":0.40699999999999997,"5min":0.23799999999999999,"15min":0.23300000000000001},"max":{"1min":3.4049999999999998,"5min":35.591000000000001,"15min":35.591000000000001},"last":1.1870000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.97099999999999997,"15min":0.80900000000000005},"min":{"1min":0.35499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":1.9610000000000001,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":0.99299999999999999,"15min":0.80400000000000005},"min":{"1min":0.35399999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":7.5259999999999998,"5min":7.5259999999999998,"15min":7.5259999999999998},"last":0.81499999999999995}]},{"osd":3,"last update":"Wed Sep 11 08:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.98899999999999999,"15min":0.84899999999999998},"min":{"1min":0.36799999999999999,"5min":0.27200000000000002,"15min":0.25},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":7.8609999999999998},"last":1.2629999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":1.03,"15min":0.92300000000000004},"min":{"1min":0.38900000000000001,"5min":0.27700000000000002,"15min":0.27400000000000002},"max":{"1min":2.4849999999999999,"5min":5.625,"15min":6.9139999999999997},"last":1.3999999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:49:48 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":0.71099999999999997,"15min":0.53800000000000003},"min":{"1min":0.33400000000000002,"5min":0.24099999999999999,"15min":0.23400000000000001},"max":{"1min":7.1520000000000001,"5min":7.1520000000000001,"15min":7.1520000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.77800000000000002,"15min":0.55200000000000005},"min":{"1min":0.29199999999999998,"5min":0.216,"15min":0.216},"max":{"1min":3.875,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.52600000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.82499999999999996,"15min":0.57999999999999996},"min":{"1min":0.318,"5min":0.29399999999999998,"15min":0.23300000000000001},"max":{"1min":2.448,"5min":5.1059999999999999,"15min":5.1059999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":1.355,"5min":0.86599999999999999,"15min":0.627},"min":{"1min":0.33400000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":6.3650000000000002,"5min":6.3650000000000002,"15min":6.3650000000000002},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:49:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.77900000000000003,"15min":0.62},"min":{"1min":0.38,"5min":0.317,"15min":0.315},"max":{"1min":3.512,"5min":3.512,"15min":3.5979999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.097,"5min":0.86099999999999999,"15min":0.63400000000000001},"min":{"1min":0.38900000000000001,"5min":0.30499999999999999,"15min":0.27900000000000003},"max":{"1min":3.9940000000000002,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.55700000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528469,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1515680,"kb_used_data":1218328,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92229472,"statfs":{"total":95995035648,"available":94442979328,"internally_reserved":0,"allocated":1247567872,"data_stored":1247186161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:49:26 2024","interfaces":[{"interface":"back","average":{"1min":2.3690000000000002,"5min":1.4710000000000001,"15min":1.111},"min":{"1min":0.252,"5min":0.217,"15min":0.217},"max":{"1min":27.908000000000001,"5min":27.908000000000001,"15min":28.302},"last":0.58399999999999996},{"interface":"front","average":{"1min":2.738,"5min":1.349,"15min":1.073},"min":{"1min":0.378,"5min":0.24299999999999999,"15min":0.23300000000000001},"max":{"1min":27.771000000000001,"5min":27.771000000000001,"15min":28.314},"last":0.71299999999999997}]},{"osd":1,"last update":"Wed Sep 11 08:49:26 2024","interfaces":[{"interface":"back","average":{"1min":2.29,"5min":1.7749999999999999,"15min":1.0269999999999999},"min":{"1min":0.307,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":27.984000000000002,"5min":27.984000000000002,"15min":27.984000000000002},"last":1.014},{"interface":"front","average":{"1min":2.1709999999999998,"5min":1.6060000000000001,"15min":0.97199999999999998},"min":{"1min":0.28000000000000003,"5min":0.25700000000000001,"15min":0.253},"max":{"1min":28.106000000000002,"5min":28.106000000000002,"15min":28.106000000000002},"last":0.96199999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":1.0269999999999999,"15min":0.77000000000000002},"min":{"1min":0.28999999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1399999999999999,"5min":14.313000000000001,"15min":14.313000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.998,"15min":0.77900000000000003},"min":{"1min":0.27700000000000002,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":2.847,"5min":3.7240000000000002,"15min":6.5979999999999999},"last":0.68500000000000005}]},{"osd":3,"last update":"Wed Sep 11 08:49:26 2024","interfaces":[{"interface":"back","average":{"1min":2.1930000000000001,"5min":1.5669999999999999,"15min":1.038},"min":{"1min":0.41299999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":27.795000000000002,"5min":27.795000000000002,"15min":27.795000000000002},"last":0.94599999999999995},{"interface":"front","average":{"1min":2.3929999999999998,"5min":1.54,"15min":1.0309999999999999},"min":{"1min":0.38900000000000001,"5min":0.28599999999999998,"15min":0.224},"max":{"1min":27.841999999999999,"5min":27.841999999999999,"15min":27.841999999999999},"last":0.51400000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.58599999999999997,"15min":0.48099999999999998},"min":{"1min":0.26200000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.3160000000000001,"5min":3.544,"15min":3.544},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.60099999999999998,"15min":0.47699999999999998},"min":{"1min":0.29499999999999998,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":0.65800000000000003,"5min":4.0209999999999999,"15min":4.0209999999999999},"last":0.86799999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.76700000000000002,"15min":0.56999999999999995},"min":{"1min":0.318,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":2.117,"5min":8.6829999999999998,"15min":8.6829999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.72399999999999998,"15min":0.54600000000000004},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.28100000000000003},"max":{"1min":1.012,"5min":5.0819999999999999,"15min":5.0819999999999999},"last":0.91100000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78900000000000003,"15min":0.57499999999999996},"min":{"1min":0.32500000000000001,"5min":0.29899999999999999,"15min":0.247},"max":{"1min":1.867,"5min":7.327,"15min":7.327},"last":1.306},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.57799999999999996},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.27700000000000002},"max":{"1min":1.774,"5min":5.4500000000000002,"15min":5.4500000000000002},"last":0.47299999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757400,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3652332,"kb_used_data":3342500,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":90092820,"statfs":{"total":95995035648,"available":92255047680,"internally_reserved":0,"allocated":3422720000,"data_stored":3422326112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:49:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.93000000000000005,"15min":0.77000000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.20599999999999999},"max":{"1min":2.7599999999999998,"5min":3.266,"15min":7.3899999999999997},"last":4.6420000000000003},{"interface":"front","average":{"1min":1.3,"5min":1.2150000000000001,"15min":0.90700000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23100000000000001},"max":{"1min":5.569,"5min":8.0410000000000004,"15min":8.0410000000000004},"last":1.5449999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:49:04 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.23,"15min":0.88300000000000001},"min":{"1min":0.28699999999999998,"5min":0.23799999999999999,"15min":0.22800000000000001},"max":{"1min":2.7749999999999999,"5min":8,"15min":8},"last":3.2629999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.119,"15min":0.81899999999999995},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.9780000000000002,"5min":3.9780000000000002,"15min":6.6719999999999997},"last":1.4650000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:49:04 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.171,"15min":0.85099999999999998},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.23799999999999999},"max":{"1min":5.5339999999999998,"5min":5.5339999999999998,"15min":5.5339999999999998},"last":1.6220000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":1.1539999999999999,"15min":0.81399999999999995},"min":{"1min":0.34100000000000003,"5min":0.28699999999999998,"15min":0.26400000000000001},"max":{"1min":3.5990000000000002,"5min":8.0640000000000001,"15min":8.0640000000000001},"last":1.526}]},{"osd":3,"last update":"Wed Sep 11 08:49:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.99199999999999999,"15min":0.80700000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.7309999999999999,"5min":4.3940000000000001,"15min":6.0460000000000003},"last":2.4940000000000002},{"interface":"front","average":{"1min":1.319,"5min":1.2909999999999999,"15min":0.95099999999999996},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.27400000000000002},"max":{"1min":8.0899999999999999,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":3.0070000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:49:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70599999999999996,"15min":0.501},"min":{"1min":0.26400000000000001,"5min":0.246,"15min":0.20699999999999999},"max":{"1min":3.5129999999999999,"5min":4.7110000000000003,"15min":4.7110000000000003},"last":1.5109999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.621,"15min":0.48899999999999999},"min":{"1min":0.29099999999999998,"5min":0.23699999999999999,"15min":0.20499999999999999},"max":{"1min":1.7749999999999999,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.556}]},{"osd":6,"last update":"Wed Sep 11 08:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.78700000000000003,"15min":0.55600000000000005},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.23599999999999999},"max":{"1min":2.802,"5min":6.4530000000000003,"15min":6.4530000000000003},"last":1.595},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.76500000000000001,"15min":0.55100000000000005},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.251},"max":{"1min":3.1850000000000001,"5min":8.1920000000000002,"15min":8.1920000000000002},"last":1.496}]},{"osd":7,"last update":"Wed Sep 11 08:49:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.6839999999999999,"15min":0.88400000000000001},"min":{"1min":0.33000000000000002,"5min":0.29999999999999999,"15min":0.27800000000000002},"max":{"1min":9.4009999999999998,"5min":88.513000000000005,"15min":88.513000000000005},"last":1.7350000000000001},{"interface":"front","average":{"1min":1.667,"5min":1.7450000000000001,"15min":0.92900000000000005},"min":{"1min":0.40200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":9.7089999999999996,"5min":86.963999999999999,"15min":86.963999999999999},"last":1.71}]}]},{"osd":4,"up_from":31,"seq":133143986333,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1517092,"kb_used_data":1211420,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92228060,"statfs":{"total":95995035648,"available":94441533440,"internally_reserved":0,"allocated":1240494080,"data_stored":1240106993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":1.571,"15min":1.569},"min":{"1min":0.22600000000000001,"5min":0.21199999999999999,"15min":0.20399999999999999},"max":{"1min":1.2330000000000001,"5min":44.140999999999998,"15min":89.510000000000005},"last":1.3220000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":1.831,"15min":1.6659999999999999},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.4590000000000001,"5min":44.450000000000003,"15min":89.765000000000001},"last":0.28399999999999997}]},{"osd":1,"last update":"Wed Sep 11 08:49:42 2024","interfaces":[{"interface":"back","average":{"1min":3.8079999999999998,"5min":2.282,"15min":1.2989999999999999},"min":{"1min":0.28199999999999997,"5min":0.27300000000000002,"15min":0.24399999999999999},"max":{"1min":48.976999999999997,"5min":48.976999999999997,"15min":48.976999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":3.4140000000000001,"5min":2.2599999999999998,"15min":1.2549999999999999},"min":{"1min":0.308,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":45.787999999999997,"5min":45.787999999999997,"15min":45.787999999999997},"last":0.42799999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":1.097,"15min":0.81699999999999995},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.649,"5min":17.303000000000001,"15min":17.303000000000001},"last":1.351},{"interface":"front","average":{"1min":0.48999999999999999,"5min":1.026,"15min":0.81299999999999994},"min":{"1min":0.29199999999999998,"5min":0.27500000000000002,"15min":0.25800000000000001},"max":{"1min":0.88400000000000001,"5min":20.702999999999999,"15min":20.702999999999999},"last":1.147}]},{"osd":3,"last update":"Wed Sep 11 08:49:41 2024","interfaces":[{"interface":"back","average":{"1min":3.157,"5min":2.0649999999999999,"15min":1.841},"min":{"1min":0.28399999999999997,"5min":0.255,"15min":0.255},"max":{"1min":44.052,"5min":45.268000000000001,"15min":89.881},"last":0.252},{"interface":"front","average":{"1min":3.1539999999999999,"5min":2.1739999999999999,"15min":1.8},"min":{"1min":0.33900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":43.646000000000001,"5min":44.234000000000002,"15min":89.715000000000003},"last":0.49299999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:50:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.82499999999999996,"15min":0.56200000000000006},"min":{"1min":0.25900000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":4.2069999999999999,"5min":4.2069999999999999,"15min":4.2069999999999999},"last":2.2589999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.52600000000000002},"min":{"1min":0.31900000000000001,"5min":0.26100000000000001,"15min":0.23699999999999999},"max":{"1min":2.6850000000000001,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":1.3740000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.61899999999999999,"15min":0.504},"min":{"1min":0.28100000000000003,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":2.9359999999999999,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":2.4769999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69699999999999995,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.221,"15min":0.221},"max":{"1min":3.1019999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":0.46800000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64300000000000002,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.252,"15min":0.252},"max":{"1min":2.2010000000000001,"5min":3.339,"15min":3.339},"last":2.1280000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.79300000000000004,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.29599999999999999,"15min":0.28799999999999998},"max":{"1min":2.847,"5min":5.1970000000000001,"15min":5.1970000000000001},"last":1.1879999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149858,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3678276,"kb_used_data":3376764,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":90066876,"statfs":{"total":95995035648,"available":92228481024,"internally_reserved":0,"allocated":3457806336,"data_stored":3457388561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,4,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.4810000000000001,"15min":1.0289999999999999},"min":{"1min":0.221,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":1.9399999999999999,"5min":24.280000000000001,"15min":24.280000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":1.7130000000000001,"15min":1.2210000000000001},"min":{"1min":0.20100000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":2.7309999999999999,"5min":22.225000000000001,"15min":22.225000000000001},"last":0.71699999999999997}]},{"osd":1,"last update":"Wed Sep 11 08:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":2.7170000000000001,"15min":1.5369999999999999},"min":{"1min":0.247,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":3.4780000000000002,"5min":115.601,"15min":115.601},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.99399999999999999,"5min":2.5910000000000002,"15min":1.466},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.17999999999999999},"max":{"1min":5.968,"5min":115.474,"15min":115.474},"last":0.625}]},{"osd":2,"last update":"Wed Sep 11 08:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":1.452,"15min":1.0940000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23200000000000001},"max":{"1min":6.077,"5min":8.2699999999999996,"15min":8.9510000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":1.236,"15min":1.0269999999999999},"min":{"1min":0.22700000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.8240000000000001,"5min":10.170999999999999,"15min":10.695},"last":0.41199999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":1.4379999999999999,"15min":1.0740000000000001},"min":{"1min":0.33700000000000002,"5min":0.28599999999999998,"15min":0.26700000000000002},"max":{"1min":2.3580000000000001,"5min":8.3279999999999994,"15min":8.9969999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":1.1379999999999999,"15min":0.91700000000000004},"min":{"1min":0.35799999999999998,"5min":0.318,"15min":0.27700000000000002},"max":{"1min":1.6080000000000001,"5min":7.234,"15min":7.234},"last":0.68300000000000005}]},{"osd":5,"last update":"Wed Sep 11 08:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":1.2909999999999999,"15min":1.0700000000000001},"min":{"1min":0.33900000000000002,"5min":0.29799999999999999,"15min":0.28299999999999997},"max":{"1min":2.9100000000000001,"5min":8.3599999999999994,"15min":11.526999999999999},"last":2.8639999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.194,"15min":0.97499999999999998},"min":{"1min":0.314,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":4.2240000000000002,"5min":7.1689999999999996,"15min":7.1689999999999996},"last":0.86299999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:49:28 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.4299999999999999,"15min":1.1080000000000001},"min":{"1min":0.35899999999999999,"5min":0.311,"15min":0.29999999999999999},"max":{"1min":6.1100000000000003,"5min":8.4160000000000004,"15min":11.494999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":1.3819999999999999,"15min":1.071},"min":{"1min":0.36499999999999999,"5min":0.314,"15min":0.311},"max":{"1min":2.6400000000000001,"5min":7.7249999999999996,"15min":7.7249999999999996},"last":0.89200000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:49:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.369,"15min":1.135},"min":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.35299999999999998},"max":{"1min":3.73,"5min":9.1539999999999999,"15min":11.561},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2789999999999999,"15min":1.0249999999999999},"min":{"1min":0.441,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":5.96,"5min":6.6200000000000001,"15min":6.6200000000000001},"last":0.91900000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411494,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1783456,"kb_used_data":1481944,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91961696,"statfs":{"total":95995035648,"available":94168776704,"internally_reserved":0,"allocated":1517510656,"data_stored":1517127816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":2.323,"15min":1.51},"min":{"1min":0.218,"5min":0.16500000000000001,"15min":0.159},"max":{"1min":2.25,"5min":34.713999999999999,"15min":34.713999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":2.2109999999999999,"15min":1.5960000000000001},"min":{"1min":0.219,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.2490000000000001,"5min":34.399000000000001,"15min":34.399000000000001},"last":0.47699999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":3.0539999999999998,"15min":1.5680000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.19},"max":{"1min":3.754,"5min":72.867000000000004,"15min":72.867000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":3.04,"15min":1.6459999999999999},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.881,"5min":67.799000000000007,"15min":67.799000000000007},"last":0.58599999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:49:12 2024","interfaces":[{"interface":"back","average":{"1min":2.706,"5min":2.379,"15min":1.4179999999999999},"min":{"1min":0.26100000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":16.353999999999999,"5min":34.75,"15min":34.75},"last":0.49299999999999999},{"interface":"front","average":{"1min":2.282,"5min":2.2160000000000002,"15min":1.359},"min":{"1min":0.29499999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":16.138999999999999,"5min":32.802,"15min":32.802},"last":0.55600000000000005}]},{"osd":4,"last update":"Wed Sep 11 08:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":1.107,"15min":0.93500000000000005},"min":{"1min":0.33900000000000002,"5min":0.255,"15min":0.255},"max":{"1min":3.0139999999999998,"5min":8.0820000000000007,"15min":8.0820000000000007},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":1.0780000000000001,"15min":0.88900000000000001},"min":{"1min":0.33100000000000002,"5min":0.29999999999999999,"15min":0.28399999999999997},"max":{"1min":1.3759999999999999,"5min":6.9320000000000004,"15min":6.9320000000000004},"last":0.81200000000000006}]},{"osd":5,"last update":"Wed Sep 11 08:50:02 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.153,"15min":0.94599999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":3.4319999999999999,"5min":4.8949999999999996,"15min":6.9550000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.96599999999999997,"15min":0.83199999999999996},"min":{"1min":0.34999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.2400000000000002,"5min":3.226,"15min":5.9530000000000003},"last":1.1160000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":1.25,"15min":0.99199999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.9950000000000001,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.877,"5min":1.1020000000000001,"15min":0.89500000000000002},"min":{"1min":0.35299999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.0790000000000002,"5min":6.9779999999999998,"15min":6.9779999999999998},"last":1.1379999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.96099999999999997,"15min":0.85099999999999998},"min":{"1min":0.35899999999999999,"5min":0.32200000000000001,"15min":0.30399999999999999},"max":{"1min":1.361,"5min":3.2709999999999999,"15min":5.9909999999999997},"last":0.746},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.3009999999999999,"15min":1.032},"min":{"1min":0.39500000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.3730000000000002,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":1.05}]}]},{"osd":1,"up_from":13,"seq":55834575019,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4091540,"kb_used_data":3777548,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":89653612,"statfs":{"total":95995035648,"available":91805298688,"internally_reserved":0,"allocated":3868209152,"data_stored":3867774729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":1.1599999999999999,"15min":1.3819999999999999},"min":{"1min":0.17699999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":3.9020000000000001,"5min":4.8319999999999999,"15min":117.01600000000001},"last":1.659},{"interface":"front","average":{"1min":0.72799999999999998,"5min":1.391,"15min":1.6779999999999999},"min":{"1min":0.26500000000000001,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":1.8129999999999999,"5min":8.1549999999999994,"15min":117.62},"last":0.41799999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":1.179,"15min":1.099},"min":{"1min":0.221,"5min":0.21099999999999999,"15min":0.191},"max":{"1min":2.101,"5min":8.1329999999999991,"15min":20.628},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":1.3540000000000001,"15min":1.105},"min":{"1min":0.215,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.802,"5min":9.7200000000000006,"15min":20.148},"last":0.52200000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:49:12 2024","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.4830000000000001,"15min":1.3640000000000001},"min":{"1min":0.27200000000000002,"5min":0.191,"15min":0.191},"max":{"1min":4.2859999999999996,"5min":8.3550000000000004,"15min":116.98},"last":1.77},{"interface":"front","average":{"1min":1.502,"5min":1.4890000000000001,"15min":1.458},"min":{"1min":0.246,"5min":0.24099999999999999,"15min":0.22},"max":{"1min":3.8690000000000002,"5min":7.415,"15min":117.56699999999999},"last":1.202}]},{"osd":4,"last update":"Wed Sep 11 08:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":0.89800000000000002},"min":{"1min":0.34999999999999998,"5min":0.27500000000000002,"15min":0.26200000000000001},"max":{"1min":1.845,"5min":4.6580000000000004,"15min":11.19},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":1.0649999999999999,"15min":0.83999999999999997},"min":{"1min":0.35699999999999998,"5min":0.30199999999999999,"15min":0.251},"max":{"1min":1.466,"5min":5.056,"15min":7.7169999999999996},"last":1.744}]},{"osd":5,"last update":"Wed Sep 11 08:50:02 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.2689999999999999,"15min":1.4279999999999999},"min":{"1min":0.39100000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.5840000000000001,"5min":8.0999999999999996,"15min":103.337},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.2889999999999999,"15min":1.3939999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":3.891,"5min":5.9020000000000001,"15min":103.163},"last":0.498}]},{"osd":6,"last update":"Wed Sep 11 08:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":1.1040000000000001,"15min":0.84999999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.29399999999999998},"max":{"1min":3.0129999999999999,"5min":3.907,"15min":8.9149999999999991},"last":1.6919999999999999},{"interface":"front","average":{"1min":0.871,"5min":1.27,"15min":1.0089999999999999},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.31},"max":{"1min":3.415,"5min":4.7080000000000002,"15min":9.5199999999999996},"last":1.2330000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.1319999999999999,"15min":0.95099999999999996},"min":{"1min":0.41799999999999998,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":1.956,"5min":4.593,"15min":11.315},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":1.1919999999999999,"15min":0.90800000000000003},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.315},"max":{"1min":1.1739999999999999,"5min":10.266,"15min":10.266},"last":1.7190000000000001}]}]},{"osd":0,"up_from":0,"seq":34359738543,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3875143680,"data_stored":3875143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1528168448,"data_stored":1528168448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3490054144,"data_stored":3490054144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1246429184,"data_stored":1246429184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3467968512,"data_stored":3467968512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1247019008,"data_stored":1247019008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4150198272,"data_stored":4150198272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:50:14.504 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-11T08:50:14.505 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool set unique_pool_2 pg_num 26 2024-09-11T08:50:14.523 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:14.548 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:14.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:14.549 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:14.650 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:14.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:14 smithi145 ceph-mon[33384]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:50:14.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:14 smithi145 ceph-mon[33384]: pgmap v490: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1023 B/s rd, 85 MiB/s wr, 1.36k op/s 2024-09-11T08:50:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:14 smithi145 ceph-mon[33384]: 6.7 scrub starts 2024-09-11T08:50:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:14 smithi145 ceph-mon[33384]: 6.7 scrub ok 2024-09-11T08:50:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:14 smithi145 ceph-mon[33384]: 6.1e scrub starts 2024-09-11T08:50:14.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:14 smithi145 ceph-mon[33384]: 6.1e scrub ok 2024-09-11T08:50:14.686 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:14.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:14.687 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:14.729 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:50:14.730479+0000 min lat: 0.00097685 max lat: 0.224386 avg lat: 0.00906854 2024-09-11T08:50:14.729 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:50:14.729 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 60 16 105844 105828 110.22 79.5625 0.00630388 0.00906854 2024-09-11T08:50:14.780 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:14 smithi092 ceph-mon[29647]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:50:14.780 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:14 smithi092 ceph-mon[29647]: pgmap v490: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1023 B/s rd, 85 MiB/s wr, 1.36k op/s 2024-09-11T08:50:14.781 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:14 smithi092 ceph-mon[29647]: 6.7 scrub starts 2024-09-11T08:50:14.781 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:14 smithi092 ceph-mon[29647]: 6.7 scrub ok 2024-09-11T08:50:14.781 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:14 smithi092 ceph-mon[29647]: 6.1e scrub starts 2024-09-11T08:50:14.781 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:14 smithi092 ceph-mon[29647]: 6.1e scrub ok 2024-09-11T08:50:14.781 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:14 smithi092 ceph-mon[41114]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:50:14.781 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:14 smithi092 ceph-mon[41114]: pgmap v490: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1023 B/s rd, 85 MiB/s wr, 1.36k op/s 2024-09-11T08:50:14.781 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:14 smithi092 ceph-mon[41114]: 6.7 scrub starts 2024-09-11T08:50:14.781 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:14 smithi092 ceph-mon[41114]: 6.7 scrub ok 2024-09-11T08:50:14.781 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:14 smithi092 ceph-mon[41114]: 6.1e scrub starts 2024-09-11T08:50:14.782 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:14 smithi092 ceph-mon[41114]: 6.1e scrub ok 2024-09-11T08:50:14.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:14.854 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:14.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:14.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:14.956 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:14.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:14.986 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:14.987 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:15.087 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:15.087 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:15.088 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:15.110 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:15.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:15.111 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:15.186 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:15.212 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:15.253 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:15.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:15.254 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:15.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:15.393 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:15.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:15.394 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:15.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:15.529 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:15.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:15.531 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:50:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:15 smithi092 ceph-mon[29647]: 5.4 scrub starts 2024-09-11T08:50:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:15 smithi092 ceph-mon[29647]: 5.4 scrub ok 2024-09-11T08:50:15.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:50:15.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:15 smithi092 ceph-mon[41114]: 5.4 scrub starts 2024-09-11T08:50:15.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:15 smithi092 ceph-mon[41114]: 5.4 scrub ok 2024-09-11T08:50:15.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:15.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:50:15.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:15 smithi145 ceph-mon[33384]: 5.4 scrub starts 2024-09-11T08:50:15.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:15 smithi145 ceph-mon[33384]: 5.4 scrub ok 2024-09-11T08:50:15.668 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:15.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:15.670 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:15.729 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 61 15 107071 107056 109.671 76.75 0.00759561 0.00911387 2024-09-11T08:50:15.770 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:15.809 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:15.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:15.810 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:15.910 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:15.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:15.948 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:15.948 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:16.049 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:16.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:16.085 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:16.085 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:16.186 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:16.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:16.221 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:16.221 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:16.322 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:16.356 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:16.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:16.357 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:16.434 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:50:16.457 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:16.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:16.496 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:16.496 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:16.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:16.621 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:16.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:16.622 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:16.706 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:16 smithi145 ceph-mon[33384]: pgmap v491: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 901 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-11T08:50:16.706 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:16 smithi145 ceph-mon[33384]: 6.b scrub starts 2024-09-11T08:50:16.706 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:16 smithi145 ceph-mon[33384]: 6.b scrub ok 2024-09-11T08:50:16.706 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:16 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1212588250' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:50:16.706 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:16 smithi145 ceph-mon[33384]: 4.7 scrub starts 2024-09-11T08:50:16.706 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:16 smithi145 ceph-mon[33384]: 4.7 scrub ok 2024-09-11T08:50:16.706 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:16 smithi145 ceph-mon[33384]: 3.d scrub starts 2024-09-11T08:50:16.706 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:16 smithi145 ceph-mon[33384]: 3.d scrub ok 2024-09-11T08:50:16.723 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:16.729 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 62 16 108478 108462 109.319 87.875 0.0115477 0.00914291 2024-09-11T08:50:16.759 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:16.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:16.761 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:16.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[29647]: pgmap v491: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 901 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-11T08:50:16.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[29647]: 6.b scrub starts 2024-09-11T08:50:16.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[29647]: 6.b scrub ok 2024-09-11T08:50:16.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1212588250' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:50:16.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[29647]: 4.7 scrub starts 2024-09-11T08:50:16.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[29647]: 4.7 scrub ok 2024-09-11T08:50:16.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[29647]: 3.d scrub starts 2024-09-11T08:50:16.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[29647]: 3.d scrub ok 2024-09-11T08:50:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[41114]: pgmap v491: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 901 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-11T08:50:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[41114]: 6.b scrub starts 2024-09-11T08:50:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[41114]: 6.b scrub ok 2024-09-11T08:50:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1212588250' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:50:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[41114]: 4.7 scrub starts 2024-09-11T08:50:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[41114]: 4.7 scrub ok 2024-09-11T08:50:16.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[41114]: 3.d scrub starts 2024-09-11T08:50:16.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:16 smithi092 ceph-mon[41114]: 3.d scrub ok 2024-09-11T08:50:16.861 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:16.895 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:16.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:16.897 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:16.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:17.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:17.033 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:17.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:17.162 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:50:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:50:17.167 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:17.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:17.168 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:17.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:17.301 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:17.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:17.302 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:17.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:17.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:17.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:17.537 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:17.568 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:17.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:17.569 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:17.670 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:17.683 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[41114]: 4.5 scrub starts 2024-09-11T08:50:17.683 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[41114]: 4.5 scrub ok 2024-09-11T08:50:17.683 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[41114]: 5.14 scrub starts 2024-09-11T08:50:17.683 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[41114]: 5.14 scrub ok 2024-09-11T08:50:17.683 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:50:17.683 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:50:17.683 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1212588250' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:50:17.683 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[41114]: osdmap e228: 8 total, 8 up, 7 in 2024-09-11T08:50:17.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[29647]: 4.5 scrub starts 2024-09-11T08:50:17.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[29647]: 4.5 scrub ok 2024-09-11T08:50:17.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[29647]: 5.14 scrub starts 2024-09-11T08:50:17.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[29647]: 5.14 scrub ok 2024-09-11T08:50:17.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:50:17.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:50:17.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1212588250' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:50:17.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:17 smithi092 ceph-mon[29647]: osdmap e228: 8 total, 8 up, 7 in 2024-09-11T08:50:17.709 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:17.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:17.710 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:17.729 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 63 16 109905 109889 109 89.1875 0.00478921 0.00916996 2024-09-11T08:50:17.811 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:17.843 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:17.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:17.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:17 smithi145 ceph-mon[33384]: 4.5 scrub starts 2024-09-11T08:50:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:17 smithi145 ceph-mon[33384]: 4.5 scrub ok 2024-09-11T08:50:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:17 smithi145 ceph-mon[33384]: 5.14 scrub starts 2024-09-11T08:50:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:17 smithi145 ceph-mon[33384]: 5.14 scrub ok 2024-09-11T08:50:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:17 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:50:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:17 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:50:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:17 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1212588250' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:50:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:17 smithi145 ceph-mon[33384]: osdmap e228: 8 total, 8 up, 7 in 2024-09-11T08:50:17.945 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:17.980 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:17.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:17.981 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:18.082 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:18.117 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:18.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:18.118 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:18.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:18.251 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:18.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:18.252 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:18.353 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:18.391 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:18.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:18.392 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:18.493 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:18.527 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:18.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:18.528 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:18.629 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:18.663 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:18.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:18.664 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:18.729 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 64 16 111384 111368 108.741 92.4375 0.0199285 0.00919196 2024-09-11T08:50:18.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:18.797 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:18.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:18.798 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:18.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:18.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:50:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:50:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[41114]: pgmap v493: 148 pgs: 148 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-11T08:50:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[41114]: osdmap e229: 8 total, 8 up, 7 in 2024-09-11T08:50:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:50:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:50:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[41114]: 4.4 deep-scrub starts 2024-09-11T08:50:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[41114]: 4.4 deep-scrub ok 2024-09-11T08:50:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[41114]: 6.14 scrub starts 2024-09-11T08:50:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[41114]: 6.14 scrub ok 2024-09-11T08:50:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[41114]: 3.1a scrub starts 2024-09-11T08:50:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[41114]: 3.1a scrub ok 2024-09-11T08:50:18.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[41114]: 5.1a deep-scrub starts 2024-09-11T08:50:18.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[41114]: 5.1a deep-scrub ok 2024-09-11T08:50:18.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:18.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:50:18.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:50:18.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[29647]: pgmap v493: 148 pgs: 148 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-11T08:50:18.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[29647]: osdmap e229: 8 total, 8 up, 7 in 2024-09-11T08:50:18.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:50:18.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:50:18.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[29647]: 4.4 deep-scrub starts 2024-09-11T08:50:18.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[29647]: 4.4 deep-scrub ok 2024-09-11T08:50:18.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[29647]: 6.14 scrub starts 2024-09-11T08:50:18.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[29647]: 6.14 scrub ok 2024-09-11T08:50:18.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[29647]: 3.1a scrub starts 2024-09-11T08:50:18.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[29647]: 3.1a scrub ok 2024-09-11T08:50:18.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[29647]: 5.1a deep-scrub starts 2024-09-11T08:50:18.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:18 smithi092 ceph-mon[29647]: 5.1a deep-scrub ok 2024-09-11T08:50:18.899 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:18.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:18 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:18 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:50:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:18 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:50:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:18 smithi145 ceph-mon[33384]: pgmap v493: 148 pgs: 148 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-11T08:50:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:18 smithi145 ceph-mon[33384]: osdmap e229: 8 total, 8 up, 7 in 2024-09-11T08:50:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:18 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:50:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:18 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:50:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:18 smithi145 ceph-mon[33384]: 4.4 deep-scrub starts 2024-09-11T08:50:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:18 smithi145 ceph-mon[33384]: 4.4 deep-scrub ok 2024-09-11T08:50:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:18 smithi145 ceph-mon[33384]: 6.14 scrub starts 2024-09-11T08:50:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:18 smithi145 ceph-mon[33384]: 6.14 scrub ok 2024-09-11T08:50:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:18 smithi145 ceph-mon[33384]: 3.1a scrub starts 2024-09-11T08:50:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:18 smithi145 ceph-mon[33384]: 3.1a scrub ok 2024-09-11T08:50:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:18 smithi145 ceph-mon[33384]: 5.1a deep-scrub starts 2024-09-11T08:50:18.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:18 smithi145 ceph-mon[33384]: 5.1a deep-scrub ok 2024-09-11T08:50:18.927 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:18.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:18.928 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:19.029 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:19.068 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:19.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:19.070 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:19.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:19.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:19.212 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:19.212 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:19.313 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:19.349 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:19.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:19.350 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:19.451 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:19.487 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:19.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:19.488 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:19.589 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:19.599 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:19 smithi092 ceph-mon[41114]: osdmap e230: 8 total, 8 up, 7 in 2024-09-11T08:50:19.599 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:19 smithi092 ceph-mon[41114]: 5.19 scrub starts 2024-09-11T08:50:19.599 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:19 smithi092 ceph-mon[41114]: 5.19 scrub ok 2024-09-11T08:50:19.599 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:19 smithi092 ceph-mon[41114]: osdmap e231: 8 total, 8 up, 7 in 2024-09-11T08:50:19.599 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:19 smithi092 ceph-mon[29647]: osdmap e230: 8 total, 8 up, 7 in 2024-09-11T08:50:19.599 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:19 smithi092 ceph-mon[29647]: 5.19 scrub starts 2024-09-11T08:50:19.599 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:19 smithi092 ceph-mon[29647]: 5.19 scrub ok 2024-09-11T08:50:19.600 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:19 smithi092 ceph-mon[29647]: osdmap e231: 8 total, 8 up, 7 in 2024-09-11T08:50:19.618 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:19.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:19.619 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:19.720 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:19.729 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 65 16 112770 112754 108.401 86.625 0.015728 0.00922072 2024-09-11T08:50:19.759 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:19.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:19.760 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:19.861 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:19.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:19.897 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:19.897 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:19.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:19 smithi145 ceph-mon[33384]: osdmap e230: 8 total, 8 up, 7 in 2024-09-11T08:50:19.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:19 smithi145 ceph-mon[33384]: 5.19 scrub starts 2024-09-11T08:50:19.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:19 smithi145 ceph-mon[33384]: 5.19 scrub ok 2024-09-11T08:50:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:19 smithi145 ceph-mon[33384]: osdmap e231: 8 total, 8 up, 7 in 2024-09-11T08:50:19.998 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:20.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:20.033 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:20.033 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:20.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:20.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:20.170 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:20.170 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:20.271 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:20.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:20.305 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:20.405 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:20.435 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:20.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:20.436 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:20.505 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:50:20.505 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:50:20.505 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:20 smithi092 ceph-mon[29647]: pgmap v497: 148 pgs: 148 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.68k op/s 2024-09-11T08:50:20.506 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:20 smithi092 ceph-mon[29647]: 5.17 scrub starts 2024-09-11T08:50:20.506 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:20 smithi092 ceph-mon[29647]: 5.17 scrub ok 2024-09-11T08:50:20.506 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:20 smithi092 ceph-mon[29647]: 6.1 scrub starts 2024-09-11T08:50:20.506 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:20 smithi092 ceph-mon[29647]: 6.1 scrub ok 2024-09-11T08:50:20.506 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:20 smithi092 ceph-mon[29647]: 6.2 scrub starts 2024-09-11T08:50:20.506 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:20 smithi092 ceph-mon[29647]: 6.2 scrub ok 2024-09-11T08:50:20.506 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:20 smithi092 ceph-mon[41114]: pgmap v497: 148 pgs: 148 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.68k op/s 2024-09-11T08:50:20.506 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:20 smithi092 ceph-mon[41114]: 5.17 scrub starts 2024-09-11T08:50:20.506 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:20 smithi092 ceph-mon[41114]: 5.17 scrub ok 2024-09-11T08:50:20.506 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:20 smithi092 ceph-mon[41114]: 6.1 scrub starts 2024-09-11T08:50:20.507 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:20 smithi092 ceph-mon[41114]: 6.1 scrub ok 2024-09-11T08:50:20.507 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:20 smithi092 ceph-mon[41114]: 6.2 scrub starts 2024-09-11T08:50:20.507 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:20 smithi092 ceph-mon[41114]: 6.2 scrub ok 2024-09-11T08:50:20.537 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:20.579 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:20.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:20.580 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:20.655 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:50:20.681 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:20.721 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:20.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:20.722 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:20.729 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 66 16 114198 114182 108.11 89.25 0.00695029 0.00924532 2024-09-11T08:50:20.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:20.861 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:20.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:20.863 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:20 smithi145 ceph-mon[33384]: pgmap v497: 148 pgs: 148 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.68k op/s 2024-09-11T08:50:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:20 smithi145 ceph-mon[33384]: 5.17 scrub starts 2024-09-11T08:50:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:20 smithi145 ceph-mon[33384]: 5.17 scrub ok 2024-09-11T08:50:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:20 smithi145 ceph-mon[33384]: 6.1 scrub starts 2024-09-11T08:50:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:20 smithi145 ceph-mon[33384]: 6.1 scrub ok 2024-09-11T08:50:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:20 smithi145 ceph-mon[33384]: 6.2 scrub starts 2024-09-11T08:50:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:20 smithi145 ceph-mon[33384]: 6.2 scrub ok 2024-09-11T08:50:20.964 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:21.008 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:21.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:21.009 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:21.109 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:21.142 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:21.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:21.143 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:21.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:21.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:21.277 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:21.277 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:21.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:21.419 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:21.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:21.420 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:21.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:21.553 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:21.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:21.554 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:21.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:21 smithi092 ceph-mon[29647]: 9.b scrub starts 2024-09-11T08:50:21.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:21 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:50:21.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:21 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:50:21.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:21 smithi092 ceph-mon[29647]: 3.15 deep-scrub starts 2024-09-11T08:50:21.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:21 smithi092 ceph-mon[29647]: 3.15 deep-scrub ok 2024-09-11T08:50:21.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:50:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:50:21] "GET /metrics HTTP/1.1" 200 224368 "" "Prometheus/2.43.0" 2024-09-11T08:50:21.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:21 smithi092 ceph-mon[41114]: 9.b scrub starts 2024-09-11T08:50:21.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:21 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:50:21.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:21 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:50:21.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:21 smithi092 ceph-mon[41114]: 3.15 deep-scrub starts 2024-09-11T08:50:21.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:21 smithi092 ceph-mon[41114]: 3.15 deep-scrub ok 2024-09-11T08:50:21.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:21.690 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:21.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:21.691 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:21.729 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 67 16 115571 115555 107.777 85.8125 0.00975527 0.00927395 2024-09-11T08:50:21.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:21.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:21.828 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:21.828 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:21.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:21 smithi145 ceph-mon[33384]: 9.b scrub starts 2024-09-11T08:50:21.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:21 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:50:21.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:21 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:50:21.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:21 smithi145 ceph-mon[33384]: 3.15 deep-scrub starts 2024-09-11T08:50:21.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:21 smithi145 ceph-mon[33384]: 3.15 deep-scrub ok 2024-09-11T08:50:21.929 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:21.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:21.967 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:21.967 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:22.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:22.107 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:22.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:22.108 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:22.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:22.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:22.245 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:22.346 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:22.382 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:22.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:22.383 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:22.483 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:22.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:22.518 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:22.518 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:22 smithi092 ceph-mon[29647]: pgmap v498: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.7 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.0 KiB/s rd, 61 MiB/s wr, 983 op/s 2024-09-11T08:50:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:22 smithi092 ceph-mon[29647]: 6.11 deep-scrub starts 2024-09-11T08:50:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:22 smithi092 ceph-mon[29647]: 6.11 deep-scrub ok 2024-09-11T08:50:22.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:22 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:50:22.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:22 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:50:22.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:22 smithi092 ceph-mon[29647]: osdmap e232: 8 total, 8 up, 7 in 2024-09-11T08:50:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:22 smithi092 ceph-mon[41114]: pgmap v498: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.7 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.0 KiB/s rd, 61 MiB/s wr, 983 op/s 2024-09-11T08:50:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:22 smithi092 ceph-mon[41114]: 6.11 deep-scrub starts 2024-09-11T08:50:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:22 smithi092 ceph-mon[41114]: 6.11 deep-scrub ok 2024-09-11T08:50:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:22 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:50:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:22 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:50:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:22 smithi092 ceph-mon[41114]: osdmap e232: 8 total, 8 up, 7 in 2024-09-11T08:50:22.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:22.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:22.651 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:22.651 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:22.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:22 smithi145 ceph-mon[33384]: pgmap v498: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.7 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.0 KiB/s rd, 61 MiB/s wr, 983 op/s 2024-09-11T08:50:22.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:22 smithi145 ceph-mon[33384]: 6.11 deep-scrub starts 2024-09-11T08:50:22.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:22 smithi145 ceph-mon[33384]: 6.11 deep-scrub ok 2024-09-11T08:50:22.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:22 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:50:22.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:22 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:50:22.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:22 smithi145 ceph-mon[33384]: osdmap e232: 8 total, 8 up, 7 in 2024-09-11T08:50:22.729 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 68 16 116981 116965 107.488 88.125 0.00823855 0.00929929 2024-09-11T08:50:22.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:22.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:22.797 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:22.898 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:22.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:22.939 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:22.939 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:23.040 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:23.076 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:23.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:23.077 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:23.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:50:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:50:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:50:23.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:23.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:23.224 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:23.325 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:23.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:23.358 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:23.358 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:23.459 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:23.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:23.483 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:23.483 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:23.584 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:23.622 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:23.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:23.623 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:23.693 INFO:teuthology.orchestra.run.smithi092.stderr:set pool 9 pg_num to 26 2024-09-11T08:50:23.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:23.730 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 69 16 118273 118257 107.101 80.75 0.015537 0.00933267 2024-09-11T08:50:23.759 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:23.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:23.760 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:23.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:23 smithi092 ceph-mon[29647]: 9.b scrub ok 2024-09-11T08:50:23.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:23 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2967386436' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]: dispatch 2024-09-11T08:50:23.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:23 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]: dispatch 2024-09-11T08:50:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:23 smithi092 ceph-mon[41114]: 9.b scrub ok 2024-09-11T08:50:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:23 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2967386436' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]: dispatch 2024-09-11T08:50:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:23 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]: dispatch 2024-09-11T08:50:23.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:23.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:23.897 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:23.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:23 smithi145 ceph-mon[33384]: 9.b scrub ok 2024-09-11T08:50:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:23 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2967386436' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]: dispatch 2024-09-11T08:50:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:23 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]: dispatch 2024-09-11T08:50:23.998 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:24.031 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:24.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:24.032 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:24.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-11T08:50:24.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:24.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:24.179 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:24.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:24.280 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:24.311 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:24.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:24.312 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:24.343 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:50:24.344 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:50:24.364 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:50:24.364 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:50:24.394 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:50:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:50:24.413 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:24.423 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:50:24.423 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:50:24.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:24.448 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:24.448 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:24.451 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:50:24.451 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:50:24.482 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:50:24.482 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:50:24.517 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:50:24.518 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:50:24.549 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:50:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:50:24.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:24.585 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:24.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:24.586 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:24.687 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:24.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:24.722 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:24.722 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:24.730 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 70 16 119638 119622 106.789 85.3125 0.0178479 0.00936021 2024-09-11T08:50:24.779 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:24 smithi092 ceph-mon[29647]: pgmap v500: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.0 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 881 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-11T08:50:24.779 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:24 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]': finished 2024-09-11T08:50:24.779 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:24 smithi092 ceph-mon[29647]: osdmap e233: 8 total, 8 up, 7 in 2024-09-11T08:50:24.779 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:24 smithi092 ceph-mon[29647]: 3.4 scrub starts 2024-09-11T08:50:24.779 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:24 smithi092 ceph-mon[29647]: 3.4 scrub ok 2024-09-11T08:50:24.780 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:24 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1213812897' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:50:24.780 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:24 smithi092 ceph-mon[41114]: pgmap v500: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.0 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 881 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-11T08:50:24.780 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:24 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]': finished 2024-09-11T08:50:24.780 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:24 smithi092 ceph-mon[41114]: osdmap e233: 8 total, 8 up, 7 in 2024-09-11T08:50:24.780 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:24 smithi092 ceph-mon[41114]: 3.4 scrub starts 2024-09-11T08:50:24.780 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:24 smithi092 ceph-mon[41114]: 3.4 scrub ok 2024-09-11T08:50:24.780 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:24 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1213812897' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:50:24.824 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:24.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:24.865 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:24.865 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:24 smithi145 ceph-mon[33384]: pgmap v500: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.0 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 881 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-11T08:50:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:24 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]': finished 2024-09-11T08:50:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:24 smithi145 ceph-mon[33384]: osdmap e233: 8 total, 8 up, 7 in 2024-09-11T08:50:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:24 smithi145 ceph-mon[33384]: 3.4 scrub starts 2024-09-11T08:50:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:24 smithi145 ceph-mon[33384]: 3.4 scrub ok 2024-09-11T08:50:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:24 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1213812897' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:50:24.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:25.003 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:25.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:25.004 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:25.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:25.143 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:25.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:25.145 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:25.245 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:25.273 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:25.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:25.274 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:25.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:25.411 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:25.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:25.412 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:25.513 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:25.550 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:25.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:25.551 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:25.652 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:25.671 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:25.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:25.718 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:25.718 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:25.731 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 71 16 120777 120761 106.286 71.1875 0.181459 0.00937684 2024-09-11T08:50:25.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[29647]: 4.16 scrub starts 2024-09-11T08:50:25.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[29647]: 4.16 scrub ok 2024-09-11T08:50:25.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[29647]: 6.10 deep-scrub starts 2024-09-11T08:50:25.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[29647]: 6.10 deep-scrub ok 2024-09-11T08:50:25.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:50:25.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:50:25.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[29647]: osdmap e234: 8 total, 8 up, 7 in 2024-09-11T08:50:25.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[29647]: 3.1b scrub starts 2024-09-11T08:50:25.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[29647]: 3.1b scrub ok 2024-09-11T08:50:25.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[29647]: 5.5 scrub starts 2024-09-11T08:50:25.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[29647]: 5.5 scrub ok 2024-09-11T08:50:25.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-11T08:50:25.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-11T08:50:25.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[41114]: 4.16 scrub starts 2024-09-11T08:50:25.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[41114]: 4.16 scrub ok 2024-09-11T08:50:25.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[41114]: 6.10 deep-scrub starts 2024-09-11T08:50:25.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[41114]: 6.10 deep-scrub ok 2024-09-11T08:50:25.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:50:25.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:50:25.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[41114]: osdmap e234: 8 total, 8 up, 7 in 2024-09-11T08:50:25.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[41114]: 3.1b scrub starts 2024-09-11T08:50:25.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[41114]: 3.1b scrub ok 2024-09-11T08:50:25.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[41114]: 5.5 scrub starts 2024-09-11T08:50:25.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[41114]: 5.5 scrub ok 2024-09-11T08:50:25.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-11T08:50:25.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:25 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-11T08:50:25.819 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:25.840 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:25.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:25.841 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:25.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:25 smithi145 ceph-mon[33384]: 4.16 scrub starts 2024-09-11T08:50:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:25 smithi145 ceph-mon[33384]: 4.16 scrub ok 2024-09-11T08:50:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:25 smithi145 ceph-mon[33384]: 6.10 deep-scrub starts 2024-09-11T08:50:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:25 smithi145 ceph-mon[33384]: 6.10 deep-scrub ok 2024-09-11T08:50:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:25 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:50:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:25 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:50:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:25 smithi145 ceph-mon[33384]: osdmap e234: 8 total, 8 up, 7 in 2024-09-11T08:50:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:25 smithi145 ceph-mon[33384]: 3.1b scrub starts 2024-09-11T08:50:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:25 smithi145 ceph-mon[33384]: 3.1b scrub ok 2024-09-11T08:50:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:25 smithi145 ceph-mon[33384]: 5.5 scrub starts 2024-09-11T08:50:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:25 smithi145 ceph-mon[33384]: 5.5 scrub ok 2024-09-11T08:50:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:25 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-11T08:50:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:25 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-11T08:50:25.942 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:25.963 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:25.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:25.965 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:26.065 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:26.086 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:26.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:26.087 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:26.188 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:26.210 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:26.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:26.212 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:26.312 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:26.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:26.336 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:26.336 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:26.437 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:26.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:26.460 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:26.460 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:26.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:26.591 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:26.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:26.592 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:26.693 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:26.728 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:26.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:26.729 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:26.731 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 72 16 121128 121112 105.114 21.9375 0.0192998 0.0095093 2024-09-11T08:50:26.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[29647]: pgmap v503: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.0 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 853 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-11T08:50:26.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[29647]: 6.13 deep-scrub starts 2024-09-11T08:50:26.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[29647]: 6.13 deep-scrub ok 2024-09-11T08:50:26.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[29647]: 5.1b scrub starts 2024-09-11T08:50:26.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[29647]: 5.1b scrub ok 2024-09-11T08:50:26.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-11T08:50:26.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[29647]: osdmap e235: 8 total, 8 up, 7 in 2024-09-11T08:50:26.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[29647]: 5.1e scrub starts 2024-09-11T08:50:26.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[29647]: 5.1e scrub ok 2024-09-11T08:50:26.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[41114]: pgmap v503: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.0 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 853 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-11T08:50:26.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[41114]: 6.13 deep-scrub starts 2024-09-11T08:50:26.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[41114]: 6.13 deep-scrub ok 2024-09-11T08:50:26.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[41114]: 5.1b scrub starts 2024-09-11T08:50:26.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[41114]: 5.1b scrub ok 2024-09-11T08:50:26.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-11T08:50:26.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[41114]: osdmap e235: 8 total, 8 up, 7 in 2024-09-11T08:50:26.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[41114]: 5.1e scrub starts 2024-09-11T08:50:26.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:26 smithi092 ceph-mon[41114]: 5.1e scrub ok 2024-09-11T08:50:26.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:26.864 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:26.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:26.865 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:26.911 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:50:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:50:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:26 smithi145 ceph-mon[33384]: pgmap v503: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.0 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 853 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-11T08:50:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:26 smithi145 ceph-mon[33384]: 6.13 deep-scrub starts 2024-09-11T08:50:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:26 smithi145 ceph-mon[33384]: 6.13 deep-scrub ok 2024-09-11T08:50:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:26 smithi145 ceph-mon[33384]: 5.1b scrub starts 2024-09-11T08:50:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:26 smithi145 ceph-mon[33384]: 5.1b scrub ok 2024-09-11T08:50:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:26 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-11T08:50:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:26 smithi145 ceph-mon[33384]: osdmap e235: 8 total, 8 up, 7 in 2024-09-11T08:50:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:26 smithi145 ceph-mon[33384]: 5.1e scrub starts 2024-09-11T08:50:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:26 smithi145 ceph-mon[33384]: 5.1e scrub ok 2024-09-11T08:50:26.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:27.000 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:27.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:27.001 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:27.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:27.149 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:27.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:27.150 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:27.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:27.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:27.287 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:27.287 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:27.388 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:27.421 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:27.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:27.422 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:27.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:27.558 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:27.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:27.559 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:27.616 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:27 smithi092 ceph-mon[29647]: 4.15 scrub starts 2024-09-11T08:50:27.616 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:27 smithi092 ceph-mon[29647]: 4.15 scrub ok 2024-09-11T08:50:27.617 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:27 smithi092 ceph-mon[29647]: osdmap e236: 8 total, 8 up, 7 in 2024-09-11T08:50:27.617 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:27 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-11T08:50:27.617 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:27 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-11T08:50:27.617 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:27 smithi092 ceph-mon[41114]: 4.15 scrub starts 2024-09-11T08:50:27.618 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:27 smithi092 ceph-mon[41114]: 4.15 scrub ok 2024-09-11T08:50:27.618 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:27 smithi092 ceph-mon[41114]: osdmap e236: 8 total, 8 up, 7 in 2024-09-11T08:50:27.618 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:27 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-11T08:50:27.618 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:27 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-11T08:50:27.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:50:27.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:27.713 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:27.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:27.714 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:27.731 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 73 16 122678 122662 105.001 96.875 0.00351604 0.00949567 2024-09-11T08:50:27.815 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:27.836 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:27.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:27.837 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:27.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:27 smithi145 ceph-mon[33384]: 4.15 scrub starts 2024-09-11T08:50:27.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:27 smithi145 ceph-mon[33384]: 4.15 scrub ok 2024-09-11T08:50:27.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:27 smithi145 ceph-mon[33384]: osdmap e236: 8 total, 8 up, 7 in 2024-09-11T08:50:27.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:27 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-11T08:50:27.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:27 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-11T08:50:27.939 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:27.960 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:27.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:27.961 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:28.062 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:28.083 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:28.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:28.084 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:28.185 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:28.206 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:28.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:28.207 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:28.308 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:28.330 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:28.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:28.331 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:28.432 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:28.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:28.455 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:28.456 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:28.534 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:50:28.557 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:28.579 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:28.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:28.580 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:28.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:28.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:28.704 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:28.704 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:28.732 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 74 16 122678 122662 103.583 0 - 0.00949567 2024-09-11T08:50:28.805 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:28 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:28 smithi145 ceph-mon[33384]: pgmap v506: 158 pgs: 5 peering, 5 unknown, 148 active+clean; 6.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:50:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:28 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-11T08:50:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:28 smithi145 ceph-mon[33384]: osdmap e237: 8 total, 8 up, 7 in 2024-09-11T08:50:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:28 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/76631521' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:50:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:28 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:50:28.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:28 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:28.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:28 smithi092 ceph-mon[29647]: pgmap v506: 158 pgs: 5 peering, 5 unknown, 148 active+clean; 6.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:50:28.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:28 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-11T08:50:28.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:28 smithi092 ceph-mon[29647]: osdmap e237: 8 total, 8 up, 7 in 2024-09-11T08:50:28.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:28 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/76631521' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:50:28.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:28 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:50:28.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:28 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:28.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:28 smithi092 ceph-mon[41114]: pgmap v506: 158 pgs: 5 peering, 5 unknown, 148 active+clean; 6.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:50:28.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:28 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-11T08:50:28.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:28 smithi092 ceph-mon[41114]: osdmap e237: 8 total, 8 up, 7 in 2024-09-11T08:50:28.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:28 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/76631521' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:50:28.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:28 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:50:28.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:28.829 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:28.829 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:28.930 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:28.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:28.955 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:28.956 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:29.056 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:29.097 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:29.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:29.098 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:29.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:29.229 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:29.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:29.230 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:29.303 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:29.331 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:29.371 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:29.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:29.372 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:29.473 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:29.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:29.514 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:29.515 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:29.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:29.649 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:29.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:29.650 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:29.732 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 75 16 123681 123665 103.037 31.3438 0.00548237 0.00970073 2024-09-11T08:50:29.750 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:29.793 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:29.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:29.794 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:29.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:50:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:50:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[29647]: osdmap e238: 8 total, 8 up, 7 in 2024-09-11T08:50:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[29647]: 9.17 starting backfill to osd.1 from (0'0,0'0] MAX to 236'7583 2024-09-11T08:50:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[29647]: 9.15 starting backfill to osd.3 from (0'0,0'0] MAX to 236'7530 2024-09-11T08:50:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[29647]: 9.13 starting backfill to osd.5 from (0'0,0'0] MAX to 236'7638 2024-09-11T08:50:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[29647]: osdmap e239: 8 total, 8 up, 7 in 2024-09-11T08:50:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[29647]: 9.13 starting backfill to osd.6 from (0'0,0'0] MAX to 236'7638 2024-09-11T08:50:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[29647]: 9.17 starting backfill to osd.2 from (0'0,0'0] MAX to 236'7583 2024-09-11T08:50:29.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:50:29.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:50:29.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[41114]: osdmap e238: 8 total, 8 up, 7 in 2024-09-11T08:50:29.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[41114]: 9.17 starting backfill to osd.1 from (0'0,0'0] MAX to 236'7583 2024-09-11T08:50:29.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[41114]: 9.15 starting backfill to osd.3 from (0'0,0'0] MAX to 236'7530 2024-09-11T08:50:29.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[41114]: 9.13 starting backfill to osd.5 from (0'0,0'0] MAX to 236'7638 2024-09-11T08:50:29.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[41114]: osdmap e239: 8 total, 8 up, 7 in 2024-09-11T08:50:29.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[41114]: 9.13 starting backfill to osd.6 from (0'0,0'0] MAX to 236'7638 2024-09-11T08:50:29.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:29 smithi092 ceph-mon[41114]: 9.17 starting backfill to osd.2 from (0'0,0'0] MAX to 236'7583 2024-09-11T08:50:29.818 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:50:29.818 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:50:29.819 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-11T08:50:29.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:50:29.895 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:29.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:29 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:50:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:29 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:50:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:29 smithi145 ceph-mon[33384]: osdmap e238: 8 total, 8 up, 7 in 2024-09-11T08:50:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:29 smithi145 ceph-mon[33384]: 9.17 starting backfill to osd.1 from (0'0,0'0] MAX to 236'7583 2024-09-11T08:50:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:29 smithi145 ceph-mon[33384]: 9.15 starting backfill to osd.3 from (0'0,0'0] MAX to 236'7530 2024-09-11T08:50:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:29 smithi145 ceph-mon[33384]: 9.13 starting backfill to osd.5 from (0'0,0'0] MAX to 236'7638 2024-09-11T08:50:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:29 smithi145 ceph-mon[33384]: osdmap e239: 8 total, 8 up, 7 in 2024-09-11T08:50:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:29 smithi145 ceph-mon[33384]: 9.13 starting backfill to osd.6 from (0'0,0'0] MAX to 236'7638 2024-09-11T08:50:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:29 smithi145 ceph-mon[33384]: 9.17 starting backfill to osd.2 from (0'0,0'0] MAX to 236'7583 2024-09-11T08:50:29.934 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:29.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:29.935 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:30.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:30.072 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:30.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:30.073 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:30.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:30.206 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:30.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:30.207 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:30.308 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:30.347 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:30.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:30.348 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:30.449 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:30.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:30.486 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:30.486 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:30.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:30.626 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:30.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:30.627 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:30.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[29647]: 9.18 starting backfill to osd.6 from (0'0,0'0] MAX to 236'7570 2024-09-11T08:50:30.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[29647]: 9.19 starting backfill to osd.6 from (0'0,0'0] MAX to 236'7653 2024-09-11T08:50:30.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[29647]: 9.12 starting backfill to osd.5 from (0'0,0'0] MAX to 236'7477 2024-09-11T08:50:30.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[29647]: 9.16 starting backfill to osd.4 from (0'0,0'0] MAX to 236'7504 2024-09-11T08:50:30.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[29647]: 9.10 starting backfill to osd.4 from (0'0,0'0] MAX to 236'7583 2024-09-11T08:50:30.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[29647]: 9.10 starting backfill to osd.7 from (0'0,0'0] MAX to 236'7583 2024-09-11T08:50:30.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[29647]: pgmap v510: 158 pgs: 5 peering, 5 unknown, 148 active+clean; 6.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:50:30.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:50:30.682 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[41114]: 9.18 starting backfill to osd.6 from (0'0,0'0] MAX to 236'7570 2024-09-11T08:50:30.682 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[41114]: 9.19 starting backfill to osd.6 from (0'0,0'0] MAX to 236'7653 2024-09-11T08:50:30.682 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[41114]: 9.12 starting backfill to osd.5 from (0'0,0'0] MAX to 236'7477 2024-09-11T08:50:30.682 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[41114]: 9.16 starting backfill to osd.4 from (0'0,0'0] MAX to 236'7504 2024-09-11T08:50:30.682 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[41114]: 9.10 starting backfill to osd.4 from (0'0,0'0] MAX to 236'7583 2024-09-11T08:50:30.682 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[41114]: 9.10 starting backfill to osd.7 from (0'0,0'0] MAX to 236'7583 2024-09-11T08:50:30.682 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[41114]: pgmap v510: 158 pgs: 5 peering, 5 unknown, 148 active+clean; 6.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:50:30.682 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:50:30.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:30.731 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 76 16 124942 124926 102.718 78.8125 0.00848367 0.00973111 2024-09-11T08:50:30.761 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:30.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:30.762 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:30.763 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:50:30.763 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:50:30.763 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:50:30.763 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:50:30.763 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:50:30.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:30.886 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:30 smithi145 ceph-mon[33384]: 9.18 starting backfill to osd.6 from (0'0,0'0] MAX to 236'7570 2024-09-11T08:50:30.886 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:30 smithi145 ceph-mon[33384]: 9.19 starting backfill to osd.6 from (0'0,0'0] MAX to 236'7653 2024-09-11T08:50:30.886 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:30 smithi145 ceph-mon[33384]: 9.12 starting backfill to osd.5 from (0'0,0'0] MAX to 236'7477 2024-09-11T08:50:30.886 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:30 smithi145 ceph-mon[33384]: 9.16 starting backfill to osd.4 from (0'0,0'0] MAX to 236'7504 2024-09-11T08:50:30.886 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:30 smithi145 ceph-mon[33384]: 9.10 starting backfill to osd.4 from (0'0,0'0] MAX to 236'7583 2024-09-11T08:50:30.886 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:30 smithi145 ceph-mon[33384]: 9.10 starting backfill to osd.7 from (0'0,0'0] MAX to 236'7583 2024-09-11T08:50:30.886 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:30 smithi145 ceph-mon[33384]: pgmap v510: 158 pgs: 5 peering, 5 unknown, 148 active+clean; 6.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:50:30.886 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:50:30.893 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:30.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:30.894 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:30.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:31.033 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:31.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:31.034 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:31.135 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:31.174 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:31.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:31.175 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:31.276 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:31.321 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:31.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:31.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:31.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:31.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:31.462 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:31.463 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:31.484 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-11T08:50:31.564 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:31.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:31.608 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:31.608 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:31.709 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:31.732 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 77 16 126388 126372 102.558 90.375 0.00634516 0.0097463 2024-09-11T08:50:31.749 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:31.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:31.750 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:31.758 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:50:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:50:31] "GET /metrics HTTP/1.1" 200 224418 "" "Prometheus/2.43.0" 2024-09-11T08:50:31.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:31.884 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:31.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:31.886 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:31.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:50:31.987 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:31.998 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:32.021 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:32.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:32.022 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:32.123 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:32.161 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:32.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:32.162 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:32.263 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:32.295 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:32.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:32.296 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:32.397 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:32.433 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:32.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:32.434 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:32.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:32.570 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:32.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:32.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:32.671 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:32.714 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:32.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:32.715 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:32.732 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 78 16 127840 127824 102.407 90.75 0.00372388 0.00976078 2024-09-11T08:50:32.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:32 smithi092 ceph-mon[29647]: pgmap v511: 158 pgs: 3 active+remapped+backfilling, 3 peering, 2 unknown, 150 active+clean; 7.0 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.1 KiB/s rd, 201 MiB/s wr, 3.21k op/s; 18476/346446 objects misplaced (5.333%); 8.8 MiB/s, 141 objects/s recovering 2024-09-11T08:50:32.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:32 smithi092 ceph-mon[41114]: pgmap v511: 158 pgs: 3 active+remapped+backfilling, 3 peering, 2 unknown, 150 active+clean; 7.0 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.1 KiB/s rd, 201 MiB/s wr, 3.21k op/s; 18476/346446 objects misplaced (5.333%); 8.8 MiB/s, 141 objects/s recovering 2024-09-11T08:50:32.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:32.850 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:32.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:32.851 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:32.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:32 smithi145 ceph-mon[33384]: pgmap v511: 158 pgs: 3 active+remapped+backfilling, 3 peering, 2 unknown, 150 active+clean; 7.0 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.1 KiB/s rd, 201 MiB/s wr, 3.21k op/s; 18476/346446 objects misplaced (5.333%); 8.8 MiB/s, 141 objects/s recovering 2024-09-11T08:50:32.912 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:50:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:50:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:50:32.952 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:32.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:32.992 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:32.992 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:33.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:33.133 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:33.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:33.134 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:33.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:33.274 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:33.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:33.275 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:33.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:33.413 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:33.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:33.413 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:33.514 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:33.560 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:33.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:33.561 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:33.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:33.698 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:33.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:33.699 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:33.732 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 79 16 129317 129301 102.279 92.3125 0.0101609 0.00977329 2024-09-11T08:50:33.800 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:33.832 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:33.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:33.833 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:33.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:33 smithi145 ceph-mon[33384]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:50:33.934 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:33.975 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:33.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:33.977 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:34.012 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:34.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:33 smithi092 ceph-mon[29647]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:50:34.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:33 smithi092 ceph-mon[41114]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:50:34.078 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:34.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:34.100 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:34.100 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:34.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:34.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:34.241 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:34.242 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:34.251 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:34.298 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:50:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:50:34.328 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:50:34.329 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:50:34.342 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:34.359 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:50:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:50:34.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:34.382 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:34.382 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:34.388 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:50:34.388 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:50:34.417 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:50:34.418 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:50:34.450 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:50:34.450 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:50:34.479 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:50:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:50:34.484 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:34.513 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:50:34.513 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:50:34.521 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:34.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:34.522 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:34.623 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:34.660 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:34.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:34.661 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:34.732 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:50:34.733629+0000 min lat: 0.00097685 max lat: 1.44527 avg lat: 0.00979615 2024-09-11T08:50:34.732 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:50:34.732 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 80 16 130642 130626 102.035 82.8125 0.0069098 0.00979615 2024-09-11T08:50:34.762 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:34.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:34.799 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:34.799 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:34.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:34 smithi092 ceph-mon[29647]: pgmap v512: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 7.7 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 853 B/s rd, 248 MiB/s wr, 3.97k op/s; 40941/378429 objects misplaced (10.819%); 18 MiB/s, 283 objects/s recovering 2024-09-11T08:50:34.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:34 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3321166473' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:50:34.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:34 smithi092 ceph-mon[41114]: pgmap v512: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 7.7 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 853 B/s rd, 248 MiB/s wr, 3.97k op/s; 40941/378429 objects misplaced (10.819%); 18 MiB/s, 283 objects/s recovering 2024-09-11T08:50:34.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:34 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3321166473' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:50:34.900 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:34.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:34 smithi145 ceph-mon[33384]: pgmap v512: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 7.7 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 853 B/s rd, 248 MiB/s wr, 3.97k op/s; 40941/378429 objects misplaced (10.819%); 18 MiB/s, 283 objects/s recovering 2024-09-11T08:50:34.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:34 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3321166473' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:50:34.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:34.939 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:34.939 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:35.040 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:35.074 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:35.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:35.075 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:35.176 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:35.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:35.205 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:35.205 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:35.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:35.338 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:35.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:35.339 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:35.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:35.475 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:35.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:35.476 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:35.577 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:35.618 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:35.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:35.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:35.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:35.732 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 81 15 131962 131947 101.795 82.5625 0.021735 0.00981922 2024-09-11T08:50:35.752 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:35.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:35.753 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:35.854 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:35.891 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:35.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:35.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:35.993 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:36.030 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:36.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:36.031 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:36.132 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:36.169 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:36.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:36.170 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:36.270 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:36.315 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:36.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:36.316 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:36.417 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:36.454 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:36.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:36.455 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:36.556 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:36.579 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:36.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:36.580 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:36.681 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:36.717 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:36.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:36.718 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:36.727 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:36 smithi092 ceph-mon[29647]: pgmap v513: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 7.7 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 663 B/s rd, 193 MiB/s wr, 3.08k op/s; 40941/378429 objects misplaced (10.819%); 14 MiB/s, 220 objects/s recovering 2024-09-11T08:50:36.727 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:36 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:50:36.727 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:36 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:50:36.728 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:36 smithi092 ceph-mon[41114]: pgmap v513: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 7.7 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 663 B/s rd, 193 MiB/s wr, 3.08k op/s; 40941/378429 objects misplaced (10.819%); 14 MiB/s, 220 objects/s recovering 2024-09-11T08:50:36.728 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:36 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:50:36.728 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:36 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:50:36.732 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 82 16 133250 133234 101.534 80.4375 0.00843561 0.00984476 2024-09-11T08:50:36.819 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:36.898 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:36.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:36.900 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:36.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:36 smithi145 ceph-mon[33384]: pgmap v513: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 7.7 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 663 B/s rd, 193 MiB/s wr, 3.08k op/s; 40941/378429 objects misplaced (10.819%); 14 MiB/s, 220 objects/s recovering 2024-09-11T08:50:36.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:36 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:50:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:36 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:50:36.912 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:50:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:50:37.000 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:37.039 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:37.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:37.040 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:37.140 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:37.174 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:37.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:37.175 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:37.276 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:37.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:37.312 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:37.313 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:37.388 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:50:37.393 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:50:37.413 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:37.451 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:37.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:37.452 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:37.553 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:37.592 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:37.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:37.593 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:37.694 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:37.732 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 83 16 134695 134679 101.399 90.3125 0.00641991 0.00985757 2024-09-11T08:50:37.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:37.737 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:37.738 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:37.793 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:37 smithi092 ceph-mon[29647]: osdmap e240: 8 total, 8 up, 7 in 2024-09-11T08:50:37.793 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:37 smithi092 ceph-mon[41114]: osdmap e240: 8 total, 8 up, 7 in 2024-09-11T08:50:37.838 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:37.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:37.879 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:37.879 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:37.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:37 smithi145 ceph-mon[33384]: osdmap e240: 8 total, 8 up, 7 in 2024-09-11T08:50:37.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:50:37.980 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:38.014 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:38.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:38.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:38.116 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:38.155 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:38.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:38.156 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:38.257 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:38.296 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:38.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:38.298 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:38.398 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:38.420 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:38.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:38.421 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:38.522 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:38.565 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:38.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:38.566 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:38.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:38.701 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:38.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:38.702 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:38.732 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 84 16 135957 135941 101.131 78.875 0.0118604 0.00988417 2024-09-11T08:50:38.803 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:38.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:38.844 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:38.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:38.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:38 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:38.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:38 smithi145 ceph-mon[33384]: pgmap v515: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.1 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 230 MiB/s wr, 3.69k op/s; 38287/399441 objects misplaced (9.585%); 39 MiB/s, 626 objects/s recovering 2024-09-11T08:50:38.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:38 smithi145 ceph-mon[33384]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:50:38.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:38.966 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:38.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:38.967 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:39.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:38 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:39.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:38 smithi092 ceph-mon[29647]: pgmap v515: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.1 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 230 MiB/s wr, 3.69k op/s; 38287/399441 objects misplaced (9.585%); 39 MiB/s, 626 objects/s recovering 2024-09-11T08:50:39.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:38 smithi092 ceph-mon[29647]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:50:39.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:38 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:39.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:38 smithi092 ceph-mon[41114]: pgmap v515: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.1 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 230 MiB/s wr, 3.69k op/s; 38287/399441 objects misplaced (9.585%); 39 MiB/s, 626 objects/s recovering 2024-09-11T08:50:39.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:38 smithi092 ceph-mon[41114]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:50:39.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:39.099 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:39.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:39.101 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:39.201 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:39.240 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:39.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:39.241 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:39.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:39.372 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:39.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:39.373 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:39.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:39.512 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:39.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:39.513 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:39.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:39.648 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:39.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:39.649 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:39.733 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 85 16 137222 137206 100.871 79.0625 0.00994156 0.0099093 2024-09-11T08:50:39.750 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:39.790 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:39.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:39.791 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:39.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:39.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:39.927 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:39.928 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:40.028 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:40.058 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:40.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:40.059 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:40.160 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:40.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:39 smithi145 ceph-mon[33384]: osdmap e241: 8 total, 8 up, 7 in 2024-09-11T08:50:40.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:39 smithi145 ceph-mon[33384]: pgmap v517: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.1 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 639 B/s rd, 138 MiB/s wr, 2.21k op/s; 38287/399441 objects misplaced (9.585%); 36 MiB/s, 571 objects/s recovering 2024-09-11T08:50:40.199 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:40.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:40.200 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:40.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:39 smithi092 ceph-mon[29647]: osdmap e241: 8 total, 8 up, 7 in 2024-09-11T08:50:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:39 smithi092 ceph-mon[29647]: pgmap v517: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.1 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 639 B/s rd, 138 MiB/s wr, 2.21k op/s; 38287/399441 objects misplaced (9.585%); 36 MiB/s, 571 objects/s recovering 2024-09-11T08:50:40.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:39 smithi092 ceph-mon[41114]: osdmap e241: 8 total, 8 up, 7 in 2024-09-11T08:50:40.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:39 smithi092 ceph-mon[41114]: pgmap v517: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.1 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 639 B/s rd, 138 MiB/s wr, 2.21k op/s; 38287/399441 objects misplaced (9.585%); 36 MiB/s, 571 objects/s recovering 2024-09-11T08:50:40.323 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:40.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:40.325 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:40.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:40.457 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:40.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:40.458 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:40.559 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:40.583 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:40.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:40.584 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:40.684 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:40.722 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:40.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:40.723 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:40.733 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 86 16 138458 138442 100.596 77.25 0.0128754 0.00993635 2024-09-11T08:50:40.823 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:40.851 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:40.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:40.852 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:40.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:40.986 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:40.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:40.987 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:41.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:41.124 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:41.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:41.125 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:41.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:41.259 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:41.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:41.260 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:41.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:41.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:41.388 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:41.389 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:41.489 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:41.521 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:41.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:41.522 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:41.623 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:41.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:41.660 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:41.660 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:41.733 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 87 16 139733 139717 100.356 79.6875 0.0112336 0.00996043 2024-09-11T08:50:41.761 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:41.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:41.805 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:41.806 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:41.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:50:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:50:41] "GET /metrics HTTP/1.1" 200 224418 "" "Prometheus/2.43.0" 2024-09-11T08:50:41.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:41.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:41.943 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:41.943 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:42.044 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:42.073 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:42.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:42.074 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:42.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:42.201 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:42.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:42.202 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:42.303 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:42.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:42.343 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:42.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:42.434 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":515,"stamp":"2024-09-11T08:50:37.252900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8713672924,"num_objects":133147,"num_object_clones":0,"num_object_copies":399441,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":38287,"num_objects_unfound":0,"num_objects_dirty":133147,"num_whiteouts":0,"num_read":1976,"num_read_kb":1833,"num_write":133780,"num_write_kb":8510535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5418,"num_bytes_recovered":344790125,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122494,"ondisk_log_size":122494,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":485,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":28317708,"kb_used_data":25868820,"kb_used_omap":11,"kb_used_meta":2448820,"kb_avail":627898356,"statfs":{"total":671965249536,"available":642967916544,"internally_reserved":0,"allocated":26489671680,"data_stored":26486539551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2507592040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2019262464,"num_objects":30812,"num_object_clones":0,"num_object_copies":92436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":38287,"num_objects_unfound":0,"num_objects_dirty":30812,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":30812,"num_write_kb":1971936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5243,"num_bytes_recovered":343605248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.362154"},"pg_stats":[{"pgid":"9.15","version":"239'7849","reported_seq":9622,"reported_epoch":239,"state":"active+remapped+backfilling","last_fresh":"2024-09-11T08:50:36.216113+0000","last_change":"2024-09-11T08:50:28.944923+0000","last_active":"2024-09-11T08:50:36.216113+0000","last_peered":"2024-09-11T08:50:36.216113+0000","last_clean":"2024-09-11T08:50:27.436117+0000","last_became_active":"2024-09-11T08:50:28.930611+0000","last_became_peered":"2024-09-11T08:50:28.930611+0000","last_unstale":"2024-09-11T08:50:36.216113+0000","last_undegraded":"2024-09-11T08:50:36.216113+0000","last_fullsized":"2024-09-11T08:50:36.216113+0000","mapping_epoch":238,"log_start":"216'2800","ondisk_log_start":"216'2800","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":5049,"log_dups_size":0,"ondisk_log_size":5049,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":269549568,"num_objects":4113,"num_object_clones":0,"num_object_copies":12339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3221,"num_objects_unfound":0,"num_objects_dirty":4113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4113,"num_write_kb":263232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":855,"num_bytes_recovered":56033280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,5],"avail_no_missing":["7","1","5"],"object_location_counts":[{"shards":"1,5,7","objects":4113}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":41,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.533925+0000","last_change":"2024-09-11T08:49:09.906451+0000","last_active":"2024-09-11T08:50:28.533925+0000","last_peered":"2024-09-11T08:50:28.533925+0000","last_clean":"2024-09-11T08:50:28.533925+0000","last_became_active":"2024-09-11T08:49:09.905661+0000","last_became_peered":"2024-09-11T08:49:09.905661+0000","last_unstale":"2024-09-11T08:50:28.533925+0000","last_undegraded":"2024-09-11T08:50:28.533925+0000","last_fullsized":"2024-09-11T08:50:28.533925+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:47.838700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:47.838700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"200'19","reported_seq":299,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.530805+0000","last_change":"2024-09-11T08:49:34.732293+0000","last_active":"2024-09-11T08:50:28.530805+0000","last_peered":"2024-09-11T08:50:28.530805+0000","last_clean":"2024-09-11T08:50:28.530805+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:50:28.530805+0000","last_undegraded":"2024-09-11T08:50:28.530805+0000","last_fullsized":"2024-09-11T08:50:28.530805+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:49:34.732198+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:49:34.732198+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:49.013371+0000","scrub_duration":0.0076844679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":248,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.891397+0000","last_change":"2024-09-11T08:50:18.611806+0000","last_active":"2024-09-11T08:50:28.891397+0000","last_peered":"2024-09-11T08:50:28.891397+0000","last_clean":"2024-09-11T08:50:28.891397+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:50:28.891397+0000","last_undegraded":"2024-09-11T08:50:28.891397+0000","last_fullsized":"2024-09-11T08:50:28.891397+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:18.611725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:18.611725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:47.446057+0000","scrub_duration":0.0072923490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":216,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.530728+0000","last_change":"2024-09-11T08:49:23.674071+0000","last_active":"2024-09-11T08:50:28.530728+0000","last_peered":"2024-09-11T08:50:28.530728+0000","last_clean":"2024-09-11T08:50:28.530728+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:50:28.530728+0000","last_undegraded":"2024-09-11T08:50:28.530728+0000","last_fullsized":"2024-09-11T08:50:28.530728+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:23.673975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:23.673975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:34.389310+0000","scrub_duration":0.081616116000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"239'8022","reported_seq":8073,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.565883+0000","last_change":"2024-09-11T08:50:26.556742+0000","last_active":"2024-09-11T08:50:36.565883+0000","last_peered":"2024-09-11T08:50:36.565883+0000","last_clean":"2024-09-11T08:50:36.565883+0000","last_became_active":"2024-09-11T08:50:26.553605+0000","last_became_peered":"2024-09-11T08:50:26.553605+0000","last_unstale":"2024-09-11T08:50:36.565883+0000","last_undegraded":"2024-09-11T08:50:36.565883+0000","last_fullsized":"2024-09-11T08:50:36.565883+0000","mapping_epoch":235,"log_start":"219'3500","ondisk_log_start":"219'3500","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4522,"log_dups_size":0,"ondisk_log_size":4522,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276135936,"num_objects":4214,"num_object_clones":0,"num_object_copies":12642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4214,"num_write_kb":269664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":246,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.888645+0000","last_change":"2024-09-11T08:48:34.059380+0000","last_active":"2024-09-11T08:50:28.888645+0000","last_peered":"2024-09-11T08:50:28.888645+0000","last_clean":"2024-09-11T08:50:28.888645+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:50:28.888645+0000","last_undegraded":"2024-09-11T08:50:28.888645+0000","last_fullsized":"2024-09-11T08:50:28.888645+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:34.059309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:34.059309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0067175359999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":252,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.889272+0000","last_change":"2024-09-11T08:48:35.046165+0000","last_active":"2024-09-11T08:50:28.889272+0000","last_peered":"2024-09-11T08:50:28.889272+0000","last_clean":"2024-09-11T08:50:28.889272+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:50:28.889272+0000","last_undegraded":"2024-09-11T08:50:28.889272+0000","last_fullsized":"2024-09-11T08:50:28.889272+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:35.046075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:35.046075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010511866,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"200'34","reported_seq":306,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.895860+0000","last_change":"2024-09-11T08:49:09.927417+0000","last_active":"2024-09-11T08:50:28.895860+0000","last_peered":"2024-09-11T08:50:28.895860+0000","last_clean":"2024-09-11T08:50:28.895860+0000","last_became_active":"2024-09-11T08:49:09.909107+0000","last_became_peered":"2024-09-11T08:49:09.909107+0000","last_unstale":"2024-09-11T08:50:28.895860+0000","last_undegraded":"2024-09-11T08:50:28.895860+0000","last_fullsized":"2024-09-11T08:50:28.895860+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'22","last_scrub_stamp":"2024-09-11T08:48:30.029817+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:45.997803+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.029817+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055582840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":218,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.530691+0000","last_change":"2024-09-11T08:48:32.827213+0000","last_active":"2024-09-11T08:50:28.530691+0000","last_peered":"2024-09-11T08:50:28.530691+0000","last_clean":"2024-09-11T08:50:28.530691+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:50:28.530691+0000","last_undegraded":"2024-09-11T08:50:28.530691+0000","last_fullsized":"2024-09-11T08:50:28.530691+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:32.827110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:48:32.827110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029807079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.17","version":"239'7894","reported_seq":10438,"reported_epoch":239,"state":"active+remapped+backfilling","last_fresh":"2024-09-11T08:50:36.209939+0000","last_change":"2024-09-11T08:50:28.967168+0000","last_active":"2024-09-11T08:50:36.209939+0000","last_peered":"2024-09-11T08:50:36.209939+0000","last_clean":"2024-09-11T08:50:27.495589+0000","last_became_active":"2024-09-11T08:50:28.959196+0000","last_became_peered":"2024-09-11T08:50:28.959196+0000","last_unstale":"2024-09-11T08:50:36.209939+0000","last_undegraded":"2024-09-11T08:50:36.209939+0000","last_fullsized":"2024-09-11T08:50:36.209939+0000","mapping_epoch":238,"log_start":"217'2900","ondisk_log_start":"217'2900","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4994,"log_dups_size":0,"ondisk_log_size":4994,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271056896,"num_objects":4136,"num_object_clones":0,"num_object_copies":12408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6518,"num_objects_unfound":0,"num_objects_dirty":4136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4136,"num_write_kb":264704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":827,"num_bytes_recovered":54198272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,3,5],"avail_no_missing":["7","3","5"],"object_location_counts":[{"shards":"2","objects":1},{"shards":"3,5,7","objects":4135}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":257,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.533786+0000","last_change":"2024-09-11T08:48:41.415476+0000","last_active":"2024-09-11T08:50:28.533786+0000","last_peered":"2024-09-11T08:50:28.533786+0000","last_clean":"2024-09-11T08:50:28.533786+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:50:28.533786+0000","last_undegraded":"2024-09-11T08:50:28.533786+0000","last_fullsized":"2024-09-11T08:50:28.533786+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:41.415403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.415403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017214509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"200'19","reported_seq":279,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.531273+0000","last_change":"2024-09-11T08:49:19.682800+0000","last_active":"2024-09-11T08:50:28.531273+0000","last_peered":"2024-09-11T08:50:28.531273+0000","last_clean":"2024-09-11T08:50:28.531273+0000","last_became_active":"2024-09-11T08:49:09.910441+0000","last_became_peered":"2024-09-11T08:49:09.910441+0000","last_unstale":"2024-09-11T08:50:28.531273+0000","last_undegraded":"2024-09-11T08:50:28.531273+0000","last_fullsized":"2024-09-11T08:50:28.531273+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:49:19.682694+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:49:19.682694+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:29.537157+0000","scrub_duration":0.0024238570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":253,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.531921+0000","last_change":"2024-09-11T08:50:25.373627+0000","last_active":"2024-09-11T08:50:28.531921+0000","last_peered":"2024-09-11T08:50:28.531921+0000","last_clean":"2024-09-11T08:50:28.531921+0000","last_became_active":"2024-09-11T08:49:09.899188+0000","last_became_peered":"2024-09-11T08:49:09.899188+0000","last_unstale":"2024-09-11T08:50:28.531921+0000","last_undegraded":"2024-09-11T08:50:28.531921+0000","last_fullsized":"2024-09-11T08:50:28.531921+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:25.373549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:25.373549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:27.937005+0000","scrub_duration":0.0026496549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":52,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.896638+0000","last_change":"2024-09-11T08:49:33.520537+0000","last_active":"2024-09-11T08:50:28.896638+0000","last_peered":"2024-09-11T08:50:28.896638+0000","last_clean":"2024-09-11T08:50:28.896638+0000","last_became_active":"2024-09-11T08:49:09.907856+0000","last_became_peered":"2024-09-11T08:49:09.907856+0000","last_unstale":"2024-09-11T08:50:28.896638+0000","last_undegraded":"2024-09-11T08:50:28.896638+0000","last_fullsized":"2024-09-11T08:50:28.896638+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:33.520462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:33.520462+0000","last_clean_scrub_stamp":"2024-09-11T08:49:33.520462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:52.380831+0000","scrub_duration":0.0043744099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.16","version":"239'7823","reported_seq":10673,"reported_epoch":240,"state":"active+remapped+backfilling","last_fresh":"2024-09-11T08:50:36.675319+0000","last_change":"2024-09-11T08:50:28.971751+0000","last_active":"2024-09-11T08:50:36.675319+0000","last_peered":"2024-09-11T08:50:36.675319+0000","last_clean":"2024-09-11T08:50:27.507337+0000","last_became_active":"2024-09-11T08:50:28.964596+0000","last_became_peered":"2024-09-11T08:50:28.964596+0000","last_unstale":"2024-09-11T08:50:36.675319+0000","last_undegraded":"2024-09-11T08:50:36.675319+0000","last_fullsized":"2024-09-11T08:50:36.675319+0000","mapping_epoch":238,"log_start":"216'2800","ondisk_log_start":"216'2800","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":5023,"log_dups_size":0,"ondisk_log_size":5023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268533760,"num_objects":4098,"num_object_clones":0,"num_object_copies":12294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2651,"num_objects_unfound":0,"num_objects_dirty":4098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4098,"num_write_kb":262240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1398,"num_bytes_recovered":91619328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[1,2,5],"avail_no_missing":["1","2","5"],"object_location_counts":[{"shards":"1,2,5","objects":4098}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":237,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.532257+0000","last_change":"2024-09-11T08:48:44.398447+0000","last_active":"2024-09-11T08:50:28.532257+0000","last_peered":"2024-09-11T08:50:28.532257+0000","last_clean":"2024-09-11T08:50:28.532257+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:50:28.532257+0000","last_undegraded":"2024-09-11T08:50:28.532257+0000","last_fullsized":"2024-09-11T08:50:28.532257+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:44.398373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.398373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030847359999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"200'7","reported_seq":233,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.531187+0000","last_change":"2024-09-11T08:47:44.866144+0000","last_active":"2024-09-11T08:50:28.531187+0000","last_peered":"2024-09-11T08:50:28.531187+0000","last_clean":"2024-09-11T08:50:28.531187+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:50:28.531187+0000","last_undegraded":"2024-09-11T08:50:28.531187+0000","last_fullsized":"2024-09-11T08:50:28.531187+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:47:44.866057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.866057+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020690169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":247,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.536115+0000","last_change":"2024-09-11T08:50:18.199730+0000","last_active":"2024-09-11T08:50:28.536115+0000","last_peered":"2024-09-11T08:50:28.536115+0000","last_clean":"2024-09-11T08:50:28.536115+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:50:28.536115+0000","last_undegraded":"2024-09-11T08:50:28.536115+0000","last_fullsized":"2024-09-11T08:50:28.536115+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:18.199651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:18.199651+0000","last_clean_scrub_stamp":"2024-09-11T08:50:18.199651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:25.724164+0000","scrub_duration":0.0048782360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":231,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.532699+0000","last_change":"2024-09-11T08:49:28.283272+0000","last_active":"2024-09-11T08:50:28.532699+0000","last_peered":"2024-09-11T08:50:28.532699+0000","last_clean":"2024-09-11T08:50:28.532699+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:50:28.532699+0000","last_undegraded":"2024-09-11T08:50:28.532699+0000","last_fullsized":"2024-09-11T08:50:28.532699+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:28.283167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:49:28.283167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:28.913788+0000","scrub_duration":0.0040125320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"239'8006","reported_seq":8063,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.193009+0000","last_change":"2024-09-11T08:50:28.541687+0000","last_active":"2024-09-11T08:50:36.193009+0000","last_peered":"2024-09-11T08:50:36.193009+0000","last_clean":"2024-09-11T08:50:36.193009+0000","last_became_active":"2024-09-11T08:50:28.539715+0000","last_became_peered":"2024-09-11T08:50:28.539715+0000","last_unstale":"2024-09-11T08:50:36.193009+0000","last_undegraded":"2024-09-11T08:50:36.193009+0000","last_fullsized":"2024-09-11T08:50:36.193009+0000","mapping_epoch":237,"log_start":"218'3300","ondisk_log_start":"218'3300","created":235,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4706,"log_dups_size":0,"ondisk_log_size":4706,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":275382272,"num_objects":4202,"num_object_clones":0,"num_object_copies":12606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4202,"num_write_kb":268928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":246,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.531358+0000","last_change":"2024-09-11T08:50:13.762434+0000","last_active":"2024-09-11T08:50:28.531358+0000","last_peered":"2024-09-11T08:50:28.531358+0000","last_clean":"2024-09-11T08:50:28.531358+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:50:28.531358+0000","last_undegraded":"2024-09-11T08:50:28.531358+0000","last_fullsized":"2024-09-11T08:50:28.531358+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:13.762372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:13.762372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:41.024766+0000","scrub_duration":0.001701496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":51,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.531863+0000","last_change":"2024-09-11T08:50:24.573824+0000","last_active":"2024-09-11T08:50:28.531863+0000","last_peered":"2024-09-11T08:50:28.531863+0000","last_clean":"2024-09-11T08:50:28.531863+0000","last_became_active":"2024-09-11T08:49:09.908237+0000","last_became_peered":"2024-09-11T08:49:09.908237+0000","last_unstale":"2024-09-11T08:50:28.531863+0000","last_undegraded":"2024-09-11T08:50:28.531863+0000","last_fullsized":"2024-09-11T08:50:28.531863+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:24.573732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:24.573732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:48.045877+0000","scrub_duration":0.002943612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"200'34","reported_seq":315,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.542864+0000","last_change":"2024-09-11T08:50:01.824223+0000","last_active":"2024-09-11T08:50:28.542864+0000","last_peered":"2024-09-11T08:50:28.542864+0000","last_clean":"2024-09-11T08:50:28.542864+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:50:28.542864+0000","last_undegraded":"2024-09-11T08:50:28.542864+0000","last_fullsized":"2024-09-11T08:50:28.542864+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:50:01.824114+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:50:01.824114+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:20.462751+0000","scrub_duration":0.008843989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":248,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.892794+0000","last_change":"2024-09-11T08:49:09.903399+0000","last_active":"2024-09-11T08:50:28.892794+0000","last_peered":"2024-09-11T08:50:28.892794+0000","last_clean":"2024-09-11T08:50:28.892794+0000","last_became_active":"2024-09-11T08:49:09.903078+0000","last_became_peered":"2024-09-11T08:49:09.903078+0000","last_unstale":"2024-09-11T08:50:28.892794+0000","last_undegraded":"2024-09-11T08:50:28.892794+0000","last_fullsized":"2024-09-11T08:50:28.892794+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:19.961460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.961460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0091369829999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"239'7887","reported_seq":7949,"reported_epoch":239,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T08:50:36.522407+0000","last_change":"2024-09-11T08:50:28.971063+0000","last_active":"2024-09-11T08:50:36.522407+0000","last_peered":"2024-09-11T08:50:36.522407+0000","last_clean":"2024-09-11T08:50:27.522886+0000","last_became_active":"2024-09-11T08:50:28.966712+0000","last_became_peered":"2024-09-11T08:50:28.966712+0000","last_unstale":"2024-09-11T08:50:36.522407+0000","last_undegraded":"2024-09-11T08:50:36.522407+0000","last_fullsized":"2024-09-11T08:50:36.522407+0000","mapping_epoch":238,"log_start":"219'3400","ondisk_log_start":"219'3400","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4487,"log_dups_size":0,"ondisk_log_size":4487,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270401536,"num_objects":4126,"num_object_clones":0,"num_object_copies":12378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8252,"num_objects_unfound":0,"num_objects_dirty":4126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4126,"num_write_kb":264064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[1,2,6],"avail_no_missing":["1","2","6"],"object_location_counts":[{"shards":"1,2,6","objects":4126}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":255,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.889366+0000","last_change":"2024-09-11T08:50:07.275539+0000","last_active":"2024-09-11T08:50:28.889366+0000","last_peered":"2024-09-11T08:50:28.889366+0000","last_clean":"2024-09-11T08:50:28.889366+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:50:28.889366+0000","last_undegraded":"2024-09-11T08:50:28.889366+0000","last_fullsized":"2024-09-11T08:50:28.889366+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:50:07.275301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:07.275301+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:08.616101+0000","scrub_duration":0.019610208000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":232,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.532163+0000","last_change":"2024-09-11T08:50:17.743118+0000","last_active":"2024-09-11T08:50:28.532163+0000","last_peered":"2024-09-11T08:50:28.532163+0000","last_clean":"2024-09-11T08:50:28.532163+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:50:28.532163+0000","last_undegraded":"2024-09-11T08:50:28.532163+0000","last_fullsized":"2024-09-11T08:50:28.532163+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:17.743026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:17.743026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:40.993419+0000","scrub_duration":0.0061030829999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"200'18","reported_seq":284,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.888804+0000","last_change":"2024-09-11T08:49:26.388924+0000","last_active":"2024-09-11T08:50:28.888804+0000","last_peered":"2024-09-11T08:50:28.888804+0000","last_clean":"2024-09-11T08:50:28.888804+0000","last_became_active":"2024-09-11T08:49:16.943886+0000","last_became_peered":"2024-09-11T08:49:16.943886+0000","last_unstale":"2024-09-11T08:50:28.888804+0000","last_undegraded":"2024-09-11T08:50:28.888804+0000","last_fullsized":"2024-09-11T08:50:28.888804+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'18","last_scrub_stamp":"2024-09-11T08:49:26.388870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:49:26.388870+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:53.627087+0000","scrub_duration":0.001482014,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":228,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.532265+0000","last_change":"2024-09-11T08:50:09.746351+0000","last_active":"2024-09-11T08:50:28.532265+0000","last_peered":"2024-09-11T08:50:28.532265+0000","last_clean":"2024-09-11T08:50:28.532265+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:50:28.532265+0000","last_undegraded":"2024-09-11T08:50:28.532265+0000","last_fullsized":"2024-09-11T08:50:28.532265+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:09.745640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:09.745640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:10.823254+0000","scrub_duration":0.010457068999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"239'7948","reported_seq":10372,"reported_epoch":239,"state":"active+remapped+backfilling","last_fresh":"2024-09-11T08:50:36.210443+0000","last_change":"2024-09-11T08:50:28.964758+0000","last_active":"2024-09-11T08:50:36.210443+0000","last_peered":"2024-09-11T08:50:36.210443+0000","last_clean":"2024-09-11T08:50:27.506774+0000","last_became_active":"2024-09-11T08:50:28.953618+0000","last_became_peered":"2024-09-11T08:50:28.953618+0000","last_unstale":"2024-09-11T08:50:36.210443+0000","last_undegraded":"2024-09-11T08:50:36.210443+0000","last_fullsized":"2024-09-11T08:50:36.210443+0000","mapping_epoch":238,"log_start":"217'3000","ondisk_log_start":"217'3000","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4948,"log_dups_size":0,"ondisk_log_size":4948,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272629760,"num_objects":4160,"num_object_clones":0,"num_object_copies":12480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6692,"num_objects_unfound":0,"num_objects_dirty":4160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4160,"num_write_kb":266240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":787,"num_bytes_recovered":51576832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,1,3],"avail_no_missing":["7","1","3"],"object_location_counts":[{"shards":"1,3,7","objects":4160}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":236,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.536211+0000","last_change":"2024-09-11T08:50:09.077952+0000","last_active":"2024-09-11T08:50:28.536211+0000","last_peered":"2024-09-11T08:50:28.536211+0000","last_clean":"2024-09-11T08:50:28.536211+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:50:28.536211+0000","last_undegraded":"2024-09-11T08:50:28.536211+0000","last_fullsized":"2024-09-11T08:50:28.536211+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:50:09.077871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:09.077871+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:12.452212+0000","scrub_duration":0.0059438809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":242,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.891514+0000","last_change":"2024-09-11T08:48:25.971280+0000","last_active":"2024-09-11T08:50:28.891514+0000","last_peered":"2024-09-11T08:50:28.891514+0000","last_clean":"2024-09-11T08:50:28.891514+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:50:28.891514+0000","last_undegraded":"2024-09-11T08:50:28.891514+0000","last_fullsized":"2024-09-11T08:50:28.891514+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:25.971151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.971151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095705709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"200'20","reported_seq":275,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.536689+0000","last_change":"2024-09-11T08:49:09.936134+0000","last_active":"2024-09-11T08:50:28.536689+0000","last_peered":"2024-09-11T08:50:28.536689+0000","last_clean":"2024-09-11T08:50:28.536689+0000","last_became_active":"2024-09-11T08:49:09.913968+0000","last_became_peered":"2024-09-11T08:49:09.913968+0000","last_unstale":"2024-09-11T08:50:28.536689+0000","last_undegraded":"2024-09-11T08:50:28.536689+0000","last_fullsized":"2024-09-11T08:50:28.536689+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:48:19.705185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.705185+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050623580000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":401,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.909371+0000","last_change":"2024-09-11T08:50:06.195464+0000","last_active":"2024-09-11T08:50:28.909371+0000","last_peered":"2024-09-11T08:50:28.909371+0000","last_clean":"2024-09-11T08:50:28.909371+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:50:28.909371+0000","last_undegraded":"2024-09-11T08:50:28.909371+0000","last_fullsized":"2024-09-11T08:50:28.909371+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:50:06.195410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:06.195410+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:35.557316+0000","scrub_duration":0.0013222749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"240'7834","reported_seq":7887,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.726998+0000","last_change":"2024-09-11T08:50:26.529378+0000","last_active":"2024-09-11T08:50:36.726998+0000","last_peered":"2024-09-11T08:50:36.726998+0000","last_clean":"2024-09-11T08:50:36.726998+0000","last_became_active":"2024-09-11T08:50:26.526948+0000","last_became_peered":"2024-09-11T08:50:26.526948+0000","last_unstale":"2024-09-11T08:50:36.726998+0000","last_undegraded":"2024-09-11T08:50:36.726998+0000","last_fullsized":"2024-09-11T08:50:36.726998+0000","mapping_epoch":206,"log_start":"218'3200","ondisk_log_start":"218'3200","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4634,"log_dups_size":0,"ondisk_log_size":4634,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270073856,"num_objects":4121,"num_object_clones":0,"num_object_copies":12363,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4121,"num_write_kb":263744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"200'34","reported_seq":290,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.892951+0000","last_change":"2024-09-11T08:50:03.458529+0000","last_active":"2024-09-11T08:50:28.892951+0000","last_peered":"2024-09-11T08:50:28.892951+0000","last_clean":"2024-09-11T08:50:28.892951+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:50:28.892951+0000","last_undegraded":"2024-09-11T08:50:28.892951+0000","last_fullsized":"2024-09-11T08:50:28.892951+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:50:03.458381+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:50:03.458381+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:28.396520+0000","scrub_duration":0.0023474210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":232,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.897669+0000","last_change":"2024-09-11T08:49:25.383955+0000","last_active":"2024-09-11T08:50:28.897669+0000","last_peered":"2024-09-11T08:50:28.897669+0000","last_clean":"2024-09-11T08:50:28.897669+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:50:28.897669+0000","last_undegraded":"2024-09-11T08:50:28.897669+0000","last_fullsized":"2024-09-11T08:50:28.897669+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:25.383554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:25.383554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:46.160564+0000","scrub_duration":0.0084378879999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"211'10","reported_seq":394,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.892705+0000","last_change":"2024-09-11T08:49:35.694375+0000","last_active":"2024-09-11T08:50:28.892705+0000","last_peered":"2024-09-11T08:50:28.892705+0000","last_clean":"2024-09-11T08:50:28.892705+0000","last_became_active":"2024-09-11T08:49:09.905262+0000","last_became_peered":"2024-09-11T08:49:09.905262+0000","last_unstale":"2024-09-11T08:50:28.892705+0000","last_undegraded":"2024-09-11T08:50:28.892705+0000","last_fullsized":"2024-09-11T08:50:28.892705+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'10","last_scrub_stamp":"2024-09-11T08:49:35.694282+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:49:35.694282+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:45.457363+0000","scrub_duration":0.0051461679999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":238,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.533454+0000","last_change":"2024-09-11T08:49:20.335925+0000","last_active":"2024-09-11T08:50:28.533454+0000","last_peered":"2024-09-11T08:50:28.533454+0000","last_clean":"2024-09-11T08:50:28.533454+0000","last_became_active":"2024-09-11T08:49:09.906872+0000","last_became_peered":"2024-09-11T08:49:09.906872+0000","last_unstale":"2024-09-11T08:50:28.533454+0000","last_undegraded":"2024-09-11T08:50:28.533454+0000","last_fullsized":"2024-09-11T08:50:28.533454+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:20.335841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:20.335841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041148479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"239'8345","reported_seq":8386,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.191440+0000","last_change":"2024-09-11T08:49:05.875564+0000","last_active":"2024-09-11T08:50:36.191440+0000","last_peered":"2024-09-11T08:50:36.191440+0000","last_clean":"2024-09-11T08:50:36.191440+0000","last_became_active":"2024-09-11T08:49:05.875264+0000","last_became_peered":"2024-09-11T08:49:05.875264+0000","last_unstale":"2024-09-11T08:50:36.191440+0000","last_undegraded":"2024-09-11T08:50:36.191440+0000","last_fullsized":"2024-09-11T08:50:36.191440+0000","mapping_epoch":206,"log_start":"220'4266","ondisk_log_start":"220'4266","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4079,"log_dups_size":0,"ondisk_log_size":4079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":546897920,"num_objects":8345,"num_object_clones":0,"num_object_copies":25035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8345,"num_write_kb":534080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"200'38","reported_seq":316,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.897303+0000","last_change":"2024-09-11T08:49:09.936915+0000","last_active":"2024-09-11T08:50:28.897303+0000","last_peered":"2024-09-11T08:50:28.897303+0000","last_clean":"2024-09-11T08:50:28.897303+0000","last_became_active":"2024-09-11T08:49:09.917354+0000","last_became_peered":"2024-09-11T08:49:09.917354+0000","last_unstale":"2024-09-11T08:50:28.897303+0000","last_undegraded":"2024-09-11T08:50:28.897303+0000","last_fullsized":"2024-09-11T08:50:28.897303+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'26","last_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_deep_scrub":"199'26","last_deep_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.093994+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.026938573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":242,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.890053+0000","last_change":"2024-09-11T08:49:09.915472+0000","last_active":"2024-09-11T08:50:28.890053+0000","last_peered":"2024-09-11T08:50:28.890053+0000","last_clean":"2024-09-11T08:50:28.890053+0000","last_became_active":"2024-09-11T08:49:09.915191+0000","last_became_peered":"2024-09-11T08:49:09.915191+0000","last_unstale":"2024-09-11T08:50:28.890053+0000","last_undegraded":"2024-09-11T08:50:28.890053+0000","last_fullsized":"2024-09-11T08:50:28.890053+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:38.041303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:38.041303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073485030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":228,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.530664+0000","last_change":"2024-09-11T08:49:29.687666+0000","last_active":"2024-09-11T08:50:28.530664+0000","last_peered":"2024-09-11T08:50:28.530664+0000","last_clean":"2024-09-11T08:50:28.530664+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:50:28.530664+0000","last_undegraded":"2024-09-11T08:50:28.530664+0000","last_fullsized":"2024-09-11T08:50:28.530664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:29.687577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:29.687577+0000","last_clean_scrub_stamp":"2024-09-11T08:49:29.687577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:55.182660+0000","scrub_duration":0.0082090140000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":226,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.531180+0000","last_change":"2024-09-11T08:50:19.706569+0000","last_active":"2024-09-11T08:50:28.531180+0000","last_peered":"2024-09-11T08:50:28.531180+0000","last_clean":"2024-09-11T08:50:28.531180+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:50:28.531180+0000","last_undegraded":"2024-09-11T08:50:28.531180+0000","last_fullsized":"2024-09-11T08:50:28.531180+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:19.706509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:19.706509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:44.665049+0000","scrub_duration":0.045176716999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"240'8242","reported_seq":8284,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.835580+0000","last_change":"2024-09-11T08:49:05.874640+0000","last_active":"2024-09-11T08:50:36.835580+0000","last_peered":"2024-09-11T08:50:36.835580+0000","last_clean":"2024-09-11T08:50:36.835580+0000","last_became_active":"2024-09-11T08:49:05.874365+0000","last_became_peered":"2024-09-11T08:49:05.874365+0000","last_unstale":"2024-09-11T08:50:36.835580+0000","last_undegraded":"2024-09-11T08:50:36.835580+0000","last_fullsized":"2024-09-11T08:50:36.835580+0000","mapping_epoch":206,"log_start":"221'4383","ondisk_log_start":"221'4383","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3859,"log_dups_size":0,"ondisk_log_size":3859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540147712,"num_objects":8242,"num_object_clones":0,"num_object_copies":24726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8242,"num_write_kb":527488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"200'35","reported_seq":269,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.530935+0000","last_change":"2024-09-11T08:48:23.862534+0000","last_active":"2024-09-11T08:50:28.530935+0000","last_peered":"2024-09-11T08:50:28.530935+0000","last_clean":"2024-09-11T08:50:28.530935+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:50:28.530935+0000","last_undegraded":"2024-09-11T08:50:28.530935+0000","last_fullsized":"2024-09-11T08:50:28.530935+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'21","last_scrub_stamp":"2024-09-11T08:48:23.862482+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:48:23.862482+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001795776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":52,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.904905+0000","last_change":"2024-09-11T08:49:24.594111+0000","last_active":"2024-09-11T08:50:28.904905+0000","last_peered":"2024-09-11T08:50:28.904905+0000","last_clean":"2024-09-11T08:50:28.904905+0000","last_became_active":"2024-09-11T08:49:09.908041+0000","last_became_peered":"2024-09-11T08:49:09.908041+0000","last_unstale":"2024-09-11T08:50:28.904905+0000","last_undegraded":"2024-09-11T08:50:28.904905+0000","last_fullsized":"2024-09-11T08:50:28.904905+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:24.593902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:24.593902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:39.993524+0000","scrub_duration":0.023705506000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":372,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:32.812705+0000","last_change":"2024-09-11T08:48:36.052650+0000","last_active":"2024-09-11T08:50:32.812705+0000","last_peered":"2024-09-11T08:50:32.812705+0000","last_clean":"2024-09-11T08:50:32.812705+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:50:32.812705+0000","last_undegraded":"2024-09-11T08:50:32.812705+0000","last_fullsized":"2024-09-11T08:50:32.812705+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:48:36.052514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:36.052514+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015857353000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":255,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.537713+0000","last_change":"2024-09-11T08:49:31.758729+0000","last_active":"2024-09-11T08:50:28.537713+0000","last_peered":"2024-09-11T08:50:28.537713+0000","last_clean":"2024-09-11T08:50:28.537713+0000","last_became_active":"2024-09-11T08:46:01.983335+0000","last_became_peered":"2024-09-11T08:46:01.983335+0000","last_unstale":"2024-09-11T08:50:28.537713+0000","last_undegraded":"2024-09-11T08:50:28.537713+0000","last_fullsized":"2024-09-11T08:50:28.537713+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:31.758573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:31.758573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:36.177031+0000","scrub_duration":0.0084881980000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"239'8351","reported_seq":8391,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.327040+0000","last_change":"2024-09-11T08:49:05.871535+0000","last_active":"2024-09-11T08:50:36.327040+0000","last_peered":"2024-09-11T08:50:36.327040+0000","last_clean":"2024-09-11T08:50:36.327040+0000","last_became_active":"2024-09-11T08:49:05.871255+0000","last_became_peered":"2024-09-11T08:49:05.871255+0000","last_unstale":"2024-09-11T08:50:36.327040+0000","last_undegraded":"2024-09-11T08:50:36.327040+0000","last_fullsized":"2024-09-11T08:50:36.327040+0000","mapping_epoch":206,"log_start":"220'4166","ondisk_log_start":"220'4166","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4185,"log_dups_size":0,"ondisk_log_size":4185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":547291136,"num_objects":8351,"num_object_clones":0,"num_object_copies":25053,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8351,"num_write_kb":534464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"200'25","reported_seq":275,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.898144+0000","last_change":"2024-09-11T08:48:38.032124+0000","last_active":"2024-09-11T08:50:28.898144+0000","last_peered":"2024-09-11T08:50:28.898144+0000","last_clean":"2024-09-11T08:50:28.898144+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:50:28.898144+0000","last_undegraded":"2024-09-11T08:50:28.898144+0000","last_fullsized":"2024-09-11T08:50:28.898144+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'15","last_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_clean_scrub_stamp":"2024-09-11T08:48:38.032039+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049584310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":237,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.533857+0000","last_change":"2024-09-11T08:49:26.248106+0000","last_active":"2024-09-11T08:50:28.533857+0000","last_peered":"2024-09-11T08:50:28.533857+0000","last_clean":"2024-09-11T08:50:28.533857+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:50:28.533857+0000","last_undegraded":"2024-09-11T08:50:28.533857+0000","last_fullsized":"2024-09-11T08:50:28.533857+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:26.247212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:26.247212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:30.023699+0000","scrub_duration":0.0056073520000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"211'9","reported_seq":91,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.908575+0000","last_change":"2024-09-11T08:50:12.144286+0000","last_active":"2024-09-11T08:50:28.908575+0000","last_peered":"2024-09-11T08:50:28.908575+0000","last_clean":"2024-09-11T08:50:28.908575+0000","last_became_active":"2024-09-11T08:49:09.910279+0000","last_became_peered":"2024-09-11T08:49:09.910279+0000","last_unstale":"2024-09-11T08:50:28.908575+0000","last_undegraded":"2024-09-11T08:50:28.908575+0000","last_fullsized":"2024-09-11T08:50:28.908575+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'9","last_scrub_stamp":"2024-09-11T08:50:12.144196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:12.144196+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:38.877521+0000","scrub_duration":0.013090123,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":42,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.888191+0000","last_change":"2024-09-11T08:49:09.908727+0000","last_active":"2024-09-11T08:50:28.888191+0000","last_peered":"2024-09-11T08:50:28.888191+0000","last_clean":"2024-09-11T08:50:28.888191+0000","last_became_active":"2024-09-11T08:49:09.907944+0000","last_became_peered":"2024-09-11T08:49:09.907944+0000","last_unstale":"2024-09-11T08:50:28.888191+0000","last_undegraded":"2024-09-11T08:50:28.888191+0000","last_fullsized":"2024-09-11T08:50:28.888191+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:31.710797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:48:31.710797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"239'8468","reported_seq":8509,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.184136+0000","last_change":"2024-09-11T08:49:05.875096+0000","last_active":"2024-09-11T08:50:36.184136+0000","last_peered":"2024-09-11T08:50:36.184136+0000","last_clean":"2024-09-11T08:50:36.184136+0000","last_became_active":"2024-09-11T08:49:05.874803+0000","last_became_peered":"2024-09-11T08:49:05.874803+0000","last_unstale":"2024-09-11T08:50:36.184136+0000","last_undegraded":"2024-09-11T08:50:36.184136+0000","last_fullsized":"2024-09-11T08:50:36.184136+0000","mapping_epoch":206,"log_start":"221'4408","ondisk_log_start":"221'4408","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4060,"log_dups_size":0,"ondisk_log_size":4060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554958848,"num_objects":8468,"num_object_clones":0,"num_object_copies":25404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8468,"num_write_kb":541952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"200'29","reported_seq":299,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.531997+0000","last_change":"2024-09-11T08:50:06.372303+0000","last_active":"2024-09-11T08:50:28.531997+0000","last_peered":"2024-09-11T08:50:28.531997+0000","last_clean":"2024-09-11T08:50:28.531997+0000","last_became_active":"2024-09-11T08:49:09.901065+0000","last_became_peered":"2024-09-11T08:49:09.901065+0000","last_unstale":"2024-09-11T08:50:28.531997+0000","last_undegraded":"2024-09-11T08:50:28.531997+0000","last_fullsized":"2024-09-11T08:50:28.531997+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'29","last_scrub_stamp":"2024-09-11T08:50:06.372228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:06.372228+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:06.739718+0000","scrub_duration":0.0050905730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":252,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.894674+0000","last_change":"2024-09-11T08:50:23.612190+0000","last_active":"2024-09-11T08:50:28.894674+0000","last_peered":"2024-09-11T08:50:28.894674+0000","last_clean":"2024-09-11T08:50:28.894674+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:50:28.894674+0000","last_undegraded":"2024-09-11T08:50:28.894674+0000","last_fullsized":"2024-09-11T08:50:28.894674+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:23.612108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:23.612108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:33.850332+0000","scrub_duration":0.0044265900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":257,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.534406+0000","last_change":"2024-09-11T08:49:28.069561+0000","last_active":"2024-09-11T08:50:28.534406+0000","last_peered":"2024-09-11T08:50:28.534406+0000","last_clean":"2024-09-11T08:50:28.534406+0000","last_became_active":"2024-09-11T08:49:09.909306+0000","last_became_peered":"2024-09-11T08:49:09.909306+0000","last_unstale":"2024-09-11T08:50:28.534406+0000","last_undegraded":"2024-09-11T08:50:28.534406+0000","last_fullsized":"2024-09-11T08:50:28.534406+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_clean_scrub_stamp":"2024-09-11T08:49:28.069461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:54.916266+0000","scrub_duration":0.0022283350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":266,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.891330+0000","last_change":"2024-09-11T08:50:19.562077+0000","last_active":"2024-09-11T08:50:28.891330+0000","last_peered":"2024-09-11T08:50:28.891330+0000","last_clean":"2024-09-11T08:50:28.891330+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:50:28.891330+0000","last_undegraded":"2024-09-11T08:50:28.891330+0000","last_fullsized":"2024-09-11T08:50:28.891330+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:19.561995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:50:19.561995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:40.285903+0000","scrub_duration":0.0048084249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"239'7974","reported_seq":8026,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.193135+0000","last_change":"2024-09-11T08:50:26.529551+0000","last_active":"2024-09-11T08:50:36.193135+0000","last_peered":"2024-09-11T08:50:36.193135+0000","last_clean":"2024-09-11T08:50:36.193135+0000","last_became_active":"2024-09-11T08:50:26.526862+0000","last_became_peered":"2024-09-11T08:50:26.526862+0000","last_unstale":"2024-09-11T08:50:36.193135+0000","last_undegraded":"2024-09-11T08:50:36.193135+0000","last_fullsized":"2024-09-11T08:50:36.193135+0000","mapping_epoch":210,"log_start":"217'3100","ondisk_log_start":"217'3100","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4874,"log_dups_size":0,"ondisk_log_size":4874,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":273743872,"num_objects":4177,"num_object_clones":0,"num_object_copies":12531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4177,"num_write_kb":267328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"200'50","reported_seq":469,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:32.812591+0000","last_change":"2024-09-11T08:50:17.612220+0000","last_active":"2024-09-11T08:50:32.812591+0000","last_peered":"2024-09-11T08:50:32.812591+0000","last_clean":"2024-09-11T08:50:32.812591+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:50:32.812591+0000","last_undegraded":"2024-09-11T08:50:32.812591+0000","last_fullsized":"2024-09-11T08:50:32.812591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'50","last_scrub_stamp":"2024-09-11T08:50:17.611978+0000","last_deep_scrub":"200'50","last_deep_scrub_stamp":"2024-09-11T08:50:17.611978+0000","last_clean_scrub_stamp":"2024-09-11T08:50:17.611978+0000","objects_scrubbed":10,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:32.180116+0000","scrub_duration":0.0077394660000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":242,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.532040+0000","last_change":"2024-09-11T08:49:25.626134+0000","last_active":"2024-09-11T08:50:28.532040+0000","last_peered":"2024-09-11T08:50:28.532040+0000","last_clean":"2024-09-11T08:50:28.532040+0000","last_became_active":"2024-09-11T08:49:09.907850+0000","last_became_peered":"2024-09-11T08:49:09.907850+0000","last_unstale":"2024-09-11T08:50:28.532040+0000","last_undegraded":"2024-09-11T08:50:28.532040+0000","last_fullsized":"2024-09-11T08:50:28.532040+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_clean_scrub_stamp":"2024-09-11T08:49:25.626031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028789509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":272,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.532703+0000","last_change":"2024-09-11T08:48:16.376429+0000","last_active":"2024-09-11T08:50:28.532703+0000","last_peered":"2024-09-11T08:50:28.532703+0000","last_clean":"2024-09-11T08:50:28.532703+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:50:28.532703+0000","last_undegraded":"2024-09-11T08:50:28.532703+0000","last_fullsized":"2024-09-11T08:50:28.532703+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:48:16.376340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:16.376340+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00247796,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":52,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.892665+0000","last_change":"2024-09-11T08:50:24.593208+0000","last_active":"2024-09-11T08:50:28.892665+0000","last_peered":"2024-09-11T08:50:28.892665+0000","last_clean":"2024-09-11T08:50:28.892665+0000","last_became_active":"2024-09-11T08:49:09.903614+0000","last_became_peered":"2024-09-11T08:49:09.903614+0000","last_unstale":"2024-09-11T08:50:28.892665+0000","last_undegraded":"2024-09-11T08:50:28.892665+0000","last_fullsized":"2024-09-11T08:50:28.892665+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:24.593021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:24.593021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:30.665384+0000","scrub_duration":0.0022715140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":226,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.889057+0000","last_change":"2024-09-11T08:50:02.248519+0000","last_active":"2024-09-11T08:50:28.889057+0000","last_peered":"2024-09-11T08:50:28.889057+0000","last_clean":"2024-09-11T08:50:28.889057+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:50:28.889057+0000","last_undegraded":"2024-09-11T08:50:28.889057+0000","last_fullsized":"2024-09-11T08:50:28.889057+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:02.248447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:02.248447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:06.671112+0000","scrub_duration":0.0029743439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"239'8379","reported_seq":8419,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.357142+0000","last_change":"2024-09-11T08:49:05.872817+0000","last_active":"2024-09-11T08:50:36.357142+0000","last_peered":"2024-09-11T08:50:36.357142+0000","last_clean":"2024-09-11T08:50:36.357142+0000","last_became_active":"2024-09-11T08:49:05.872392+0000","last_became_peered":"2024-09-11T08:49:05.872392+0000","last_unstale":"2024-09-11T08:50:36.357142+0000","last_undegraded":"2024-09-11T08:50:36.357142+0000","last_fullsized":"2024-09-11T08:50:36.357142+0000","mapping_epoch":206,"log_start":"220'4164","ondisk_log_start":"220'4164","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4215,"log_dups_size":0,"ondisk_log_size":4215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":549126144,"num_objects":8379,"num_object_clones":0,"num_object_copies":25137,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8379,"num_write_kb":536256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"200'17","reported_seq":305,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.893032+0000","last_change":"2024-09-11T08:50:15.565377+0000","last_active":"2024-09-11T08:50:28.893032+0000","last_peered":"2024-09-11T08:50:28.893032+0000","last_clean":"2024-09-11T08:50:28.893032+0000","last_became_active":"2024-09-11T08:49:09.902884+0000","last_became_peered":"2024-09-11T08:49:09.902884+0000","last_unstale":"2024-09-11T08:50:28.893032+0000","last_undegraded":"2024-09-11T08:50:28.893032+0000","last_fullsized":"2024-09-11T08:50:28.893032+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'17","last_scrub_stamp":"2024-09-11T08:50:15.565290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:15.565290+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:29.627965+0000","scrub_duration":0.0060447950000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":268,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.894800+0000","last_change":"2024-09-11T08:49:23.588233+0000","last_active":"2024-09-11T08:50:28.894800+0000","last_peered":"2024-09-11T08:50:28.894800+0000","last_clean":"2024-09-11T08:50:28.894800+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:50:28.894800+0000","last_undegraded":"2024-09-11T08:50:28.894800+0000","last_fullsized":"2024-09-11T08:50:28.894800+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:23.588182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:23.588182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:42.336498+0000","scrub_duration":0.0010197089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"58'1","reported_seq":285,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.539332+0000","last_change":"2024-09-11T08:49:32.558340+0000","last_active":"2024-09-11T08:50:28.539332+0000","last_peered":"2024-09-11T08:50:28.539332+0000","last_clean":"2024-09-11T08:50:28.539332+0000","last_became_active":"2024-09-11T08:49:09.909804+0000","last_became_peered":"2024-09-11T08:49:09.909804+0000","last_unstale":"2024-09-11T08:50:28.539332+0000","last_undegraded":"2024-09-11T08:50:28.539332+0000","last_fullsized":"2024-09-11T08:50:28.539332+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-11T08:49:32.558236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:32.558236+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:01.291601+0000","scrub_duration":0.0035759400000000001,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":257,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.543722+0000","last_change":"2024-09-11T08:49:15.477556+0000","last_active":"2024-09-11T08:50:28.543722+0000","last_peered":"2024-09-11T08:50:28.543722+0000","last_clean":"2024-09-11T08:50:28.543722+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:50:28.543722+0000","last_undegraded":"2024-09-11T08:50:28.543722+0000","last_fullsized":"2024-09-11T08:50:28.543722+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:15.477483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_clean_scrub_stamp":"2024-09-11T08:49:15.477483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:35.478111+0000","scrub_duration":0.0049012409999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":255,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.537753+0000","last_change":"2024-09-11T08:48:44.031924+0000","last_active":"2024-09-11T08:50:28.537753+0000","last_peered":"2024-09-11T08:50:28.537753+0000","last_clean":"2024-09-11T08:50:28.537753+0000","last_became_active":"2024-09-11T08:46:01.983095+0000","last_became_peered":"2024-09-11T08:46:01.983095+0000","last_unstale":"2024-09-11T08:50:28.537753+0000","last_undegraded":"2024-09-11T08:50:28.537753+0000","last_fullsized":"2024-09-11T08:50:28.537753+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:44.031842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.031842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021799599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"239'8274","reported_seq":8611,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.166423+0000","last_change":"2024-09-11T08:50:22.510256+0000","last_active":"2024-09-11T08:50:36.166423+0000","last_peered":"2024-09-11T08:50:36.166423+0000","last_clean":"2024-09-11T08:50:36.166423+0000","last_became_active":"2024-09-11T08:49:09.909429+0000","last_became_peered":"2024-09-11T08:49:09.909429+0000","last_unstale":"2024-09-11T08:50:36.166423+0000","last_undegraded":"2024-09-11T08:50:36.166423+0000","last_fullsized":"2024-09-11T08:50:36.166423+0000","mapping_epoch":210,"log_start":"217'2885","ondisk_log_start":"217'2885","created":206,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'7271","last_scrub_stamp":"2024-09-11T08:50:22.510171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:50:22.510171+0000","objects_scrubbed":7171,"log_size":5389,"log_dups_size":0,"ondisk_log_size":5389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:45.661528+0000","scrub_duration":2.357543234,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":542244864,"num_objects":8274,"num_object_clones":0,"num_object_copies":24822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8274,"num_write_kb":529536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"200'24","reported_seq":281,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.892783+0000","last_change":"2024-09-11T08:49:09.930720+0000","last_active":"2024-09-11T08:50:28.892783+0000","last_peered":"2024-09-11T08:50:28.892783+0000","last_clean":"2024-09-11T08:50:28.892783+0000","last_became_active":"2024-09-11T08:49:09.911956+0000","last_became_peered":"2024-09-11T08:49:09.911956+0000","last_unstale":"2024-09-11T08:50:28.892783+0000","last_undegraded":"2024-09-11T08:50:28.892783+0000","last_fullsized":"2024-09-11T08:50:28.892783+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-11T08:48:37.680688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:37.680688+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037660860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":51,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.530737+0000","last_change":"2024-09-11T08:50:12.796879+0000","last_active":"2024-09-11T08:50:28.530737+0000","last_peered":"2024-09-11T08:50:28.530737+0000","last_clean":"2024-09-11T08:50:28.530737+0000","last_became_active":"2024-09-11T08:49:09.909803+0000","last_became_peered":"2024-09-11T08:49:09.909803+0000","last_unstale":"2024-09-11T08:50:28.530737+0000","last_undegraded":"2024-09-11T08:50:28.530737+0000","last_fullsized":"2024-09-11T08:50:28.530737+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:12.796804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:12.796804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:26.059189+0000","scrub_duration":0.0056357270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":982,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:35.124262+0000","last_change":"2024-09-11T08:49:27.805277+0000","last_active":"2024-09-11T08:50:35.124262+0000","last_peered":"2024-09-11T08:50:35.124262+0000","last_clean":"2024-09-11T08:50:35.124262+0000","last_became_active":"2024-09-11T08:49:09.912659+0000","last_became_peered":"2024-09-11T08:49:09.912659+0000","last_unstale":"2024-09-11T08:50:35.124262+0000","last_undegraded":"2024-09-11T08:50:35.124262+0000","last_fullsized":"2024-09-11T08:50:35.124262+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:49:27.805050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:27.805050+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:57.498333+0000","scrub_duration":0.0046228930000000003,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":708,"num_read_kb":703,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":243,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.533862+0000","last_change":"2024-09-11T08:49:09.903588+0000","last_active":"2024-09-11T08:50:28.533862+0000","last_peered":"2024-09-11T08:50:28.533862+0000","last_clean":"2024-09-11T08:50:28.533862+0000","last_became_active":"2024-09-11T08:49:09.903283+0000","last_became_peered":"2024-09-11T08:49:09.903283+0000","last_unstale":"2024-09-11T08:50:28.533862+0000","last_undegraded":"2024-09-11T08:50:28.533862+0000","last_fullsized":"2024-09-11T08:50:28.533862+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:30.532701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.532701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012853409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":236,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.891227+0000","last_change":"2024-09-11T08:48:33.025967+0000","last_active":"2024-09-11T08:50:28.891227+0000","last_peered":"2024-09-11T08:50:28.891227+0000","last_clean":"2024-09-11T08:50:28.891227+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:50:28.891227+0000","last_undegraded":"2024-09-11T08:50:28.891227+0000","last_fullsized":"2024-09-11T08:50:28.891227+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:33.025897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:48:33.025897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007471208,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"240'7873","reported_seq":7936,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.767382+0000","last_change":"2024-09-11T08:50:26.518231+0000","last_active":"2024-09-11T08:50:36.767382+0000","last_peered":"2024-09-11T08:50:36.767382+0000","last_clean":"2024-09-11T08:50:36.767382+0000","last_became_active":"2024-09-11T08:50:26.515366+0000","last_became_peered":"2024-09-11T08:50:26.515366+0000","last_unstale":"2024-09-11T08:50:36.767382+0000","last_undegraded":"2024-09-11T08:50:36.767382+0000","last_fullsized":"2024-09-11T08:50:36.767382+0000","mapping_epoch":210,"log_start":"218'3200","ondisk_log_start":"218'3200","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4673,"log_dups_size":0,"ondisk_log_size":4673,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":269647872,"num_objects":4114,"num_object_clones":0,"num_object_copies":12342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4114,"num_write_kb":263328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"201'27","reported_seq":146,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.535138+0000","last_change":"2024-09-11T08:50:16.136431+0000","last_active":"2024-09-11T08:50:28.535138+0000","last_peered":"2024-09-11T08:50:28.535138+0000","last_clean":"2024-09-11T08:50:28.535138+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:50:28.535138+0000","last_undegraded":"2024-09-11T08:50:28.535138+0000","last_fullsized":"2024-09-11T08:50:28.535138+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'27","last_scrub_stamp":"2024-09-11T08:50:16.136356+0000","last_deep_scrub":"201'27","last_deep_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_clean_scrub_stamp":"2024-09-11T08:50:16.136356+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:41.100937+0000","scrub_duration":0.0074356769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":241,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.889038+0000","last_change":"2024-09-11T08:50:08.276283+0000","last_active":"2024-09-11T08:50:28.889038+0000","last_peered":"2024-09-11T08:50:28.889038+0000","last_clean":"2024-09-11T08:50:28.889038+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:50:28.889038+0000","last_undegraded":"2024-09-11T08:50:28.889038+0000","last_fullsized":"2024-09-11T08:50:28.889038+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:50:08.276232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:08.276232+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:33.938275+0000","scrub_duration":0.018088246999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"199'81","reported_seq":303,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.538182+0000","last_change":"2024-09-11T08:49:09.926313+0000","last_active":"2024-09-11T08:50:28.538182+0000","last_peered":"2024-09-11T08:50:28.538182+0000","last_clean":"2024-09-11T08:50:28.538182+0000","last_became_active":"2024-09-11T08:49:09.904681+0000","last_became_peered":"2024-09-11T08:49:09.904681+0000","last_unstale":"2024-09-11T08:50:28.538182+0000","last_undegraded":"2024-09-11T08:50:28.538182+0000","last_fullsized":"2024-09-11T08:50:28.538182+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":247,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.536372+0000","last_change":"2024-09-11T08:50:15.173370+0000","last_active":"2024-09-11T08:50:28.536372+0000","last_peered":"2024-09-11T08:50:28.536372+0000","last_clean":"2024-09-11T08:50:28.536372+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:50:28.536372+0000","last_undegraded":"2024-09-11T08:50:28.536372+0000","last_fullsized":"2024-09-11T08:50:28.536372+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:15.173273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:15.173273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:32.728706+0000","scrub_duration":0.003216324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":241,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.533779+0000","last_change":"2024-09-11T08:50:13.347446+0000","last_active":"2024-09-11T08:50:28.533779+0000","last_peered":"2024-09-11T08:50:28.533779+0000","last_clean":"2024-09-11T08:50:28.533779+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:50:28.533779+0000","last_undegraded":"2024-09-11T08:50:28.533779+0000","last_fullsized":"2024-09-11T08:50:28.533779+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:13.347374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:13.347374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:17.893473+0000","scrub_duration":0.0062539859999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"239'7972","reported_seq":8024,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.201712+0000","last_change":"2024-09-11T08:50:26.527617+0000","last_active":"2024-09-11T08:50:36.201712+0000","last_peered":"2024-09-11T08:50:36.201712+0000","last_clean":"2024-09-11T08:50:36.201712+0000","last_became_active":"2024-09-11T08:50:26.523754+0000","last_became_peered":"2024-09-11T08:50:26.523754+0000","last_unstale":"2024-09-11T08:50:36.201712+0000","last_undegraded":"2024-09-11T08:50:36.201712+0000","last_fullsized":"2024-09-11T08:50:36.201712+0000","mapping_epoch":206,"log_start":"217'3000","ondisk_log_start":"217'3000","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4972,"log_dups_size":0,"ondisk_log_size":4972,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":274202624,"num_objects":4184,"num_object_clones":0,"num_object_copies":12552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4184,"num_write_kb":267776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"200'19","reported_seq":320,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.531462+0000","last_change":"2024-09-11T08:48:20.860266+0000","last_active":"2024-09-11T08:50:28.531462+0000","last_peered":"2024-09-11T08:50:28.531462+0000","last_clean":"2024-09-11T08:50:28.531462+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:50:28.531462+0000","last_undegraded":"2024-09-11T08:50:28.531462+0000","last_fullsized":"2024-09-11T08:50:28.531462+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:20.860186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.860186+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045417560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":222,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.531096+0000","last_change":"2024-09-11T08:49:18.665872+0000","last_active":"2024-09-11T08:50:28.531096+0000","last_peered":"2024-09-11T08:50:28.531096+0000","last_clean":"2024-09-11T08:50:28.531096+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:50:28.531096+0000","last_undegraded":"2024-09-11T08:50:28.531096+0000","last_fullsized":"2024-09-11T08:50:28.531096+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:18.665790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:49:18.665790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:40.249031+0000","scrub_duration":0.0041745330000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":240,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.533708+0000","last_change":"2024-09-11T08:49:29.269384+0000","last_active":"2024-09-11T08:50:28.533708+0000","last_peered":"2024-09-11T08:50:28.533708+0000","last_clean":"2024-09-11T08:50:28.533708+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:50:28.533708+0000","last_undegraded":"2024-09-11T08:50:28.533708+0000","last_fullsized":"2024-09-11T08:50:28.533708+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:29.269302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:49:29.269302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:40.650236+0000","scrub_duration":0.0083588919999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":246,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.888937+0000","last_change":"2024-09-11T08:50:06.234445+0000","last_active":"2024-09-11T08:50:28.888937+0000","last_peered":"2024-09-11T08:50:28.888937+0000","last_clean":"2024-09-11T08:50:28.888937+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:50:28.888937+0000","last_undegraded":"2024-09-11T08:50:28.888937+0000","last_fullsized":"2024-09-11T08:50:28.888937+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:06.234119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:06.234119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:19.471101+0000","scrub_duration":0.0034529980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"239'7930","reported_seq":7981,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.563655+0000","last_change":"2024-09-11T08:50:26.528932+0000","last_active":"2024-09-11T08:50:36.563655+0000","last_peered":"2024-09-11T08:50:36.563655+0000","last_clean":"2024-09-11T08:50:36.563655+0000","last_became_active":"2024-09-11T08:50:26.524700+0000","last_became_peered":"2024-09-11T08:50:26.524700+0000","last_unstale":"2024-09-11T08:50:36.563655+0000","last_undegraded":"2024-09-11T08:50:36.563655+0000","last_fullsized":"2024-09-11T08:50:36.563655+0000","mapping_epoch":206,"log_start":"219'3400","ondisk_log_start":"219'3400","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4530,"log_dups_size":0,"ondisk_log_size":4530,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":273219584,"num_objects":4169,"num_object_clones":0,"num_object_copies":12507,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4169,"num_write_kb":266816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"200'31","reported_seq":279,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.531483+0000","last_change":"2024-09-11T08:48:43.728009+0000","last_active":"2024-09-11T08:50:28.531483+0000","last_peered":"2024-09-11T08:50:28.531483+0000","last_clean":"2024-09-11T08:50:28.531483+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:50:28.531483+0000","last_undegraded":"2024-09-11T08:50:28.531483+0000","last_fullsized":"2024-09-11T08:50:28.531483+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'17","last_scrub_stamp":"2024-09-11T08:48:43.727938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:43.727938+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016024792999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":261,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.534766+0000","last_change":"2024-09-11T08:50:02.223875+0000","last_active":"2024-09-11T08:50:28.534766+0000","last_peered":"2024-09-11T08:50:28.534766+0000","last_clean":"2024-09-11T08:50:28.534766+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:50:28.534766+0000","last_undegraded":"2024-09-11T08:50:28.534766+0000","last_fullsized":"2024-09-11T08:50:28.534766+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:02.223777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_clean_scrub_stamp":"2024-09-11T08:50:02.223777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:26.269731+0000","scrub_duration":0.0026962679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":248,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.892129+0000","last_change":"2024-09-11T08:49:09.903898+0000","last_active":"2024-09-11T08:50:28.892129+0000","last_peered":"2024-09-11T08:50:28.892129+0000","last_clean":"2024-09-11T08:50:28.892129+0000","last_became_active":"2024-09-11T08:49:09.903589+0000","last_became_peered":"2024-09-11T08:49:09.903589+0000","last_unstale":"2024-09-11T08:50:28.892129+0000","last_undegraded":"2024-09-11T08:50:28.892129+0000","last_fullsized":"2024-09-11T08:50:28.892129+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:22.989179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:22.989179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033708700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":261,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.543562+0000","last_change":"2024-09-11T08:49:29.528952+0000","last_active":"2024-09-11T08:50:28.543562+0000","last_peered":"2024-09-11T08:50:28.543562+0000","last_clean":"2024-09-11T08:50:28.543562+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:50:28.543562+0000","last_undegraded":"2024-09-11T08:50:28.543562+0000","last_fullsized":"2024-09-11T08:50:28.543562+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:29.528873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:29.528873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:46.400819+0000","scrub_duration":0.0024493430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"240'7979","reported_seq":8030,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.834675+0000","last_change":"2024-09-11T08:50:26.528936+0000","last_active":"2024-09-11T08:50:36.834675+0000","last_peered":"2024-09-11T08:50:36.834675+0000","last_clean":"2024-09-11T08:50:36.834675+0000","last_became_active":"2024-09-11T08:50:26.523855+0000","last_became_peered":"2024-09-11T08:50:26.523855+0000","last_unstale":"2024-09-11T08:50:36.834675+0000","last_undegraded":"2024-09-11T08:50:36.834675+0000","last_fullsized":"2024-09-11T08:50:36.834675+0000","mapping_epoch":206,"log_start":"218'3300","ondisk_log_start":"218'3300","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4679,"log_dups_size":0,"ondisk_log_size":4679,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":273612800,"num_objects":4175,"num_object_clones":0,"num_object_copies":12525,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4175,"num_write_kb":267200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"200'20","reported_seq":297,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.533687+0000","last_change":"2024-09-11T08:48:21.885766+0000","last_active":"2024-09-11T08:50:28.533687+0000","last_peered":"2024-09-11T08:50:28.533687+0000","last_clean":"2024-09-11T08:50:28.533687+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:50:28.533687+0000","last_undegraded":"2024-09-11T08:50:28.533687+0000","last_fullsized":"2024-09-11T08:50:28.533687+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:48:21.885681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:21.885681+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051165580000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":242,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.888988+0000","last_change":"2024-09-11T08:50:04.191785+0000","last_active":"2024-09-11T08:50:28.888988+0000","last_peered":"2024-09-11T08:50:28.888988+0000","last_clean":"2024-09-11T08:50:28.888988+0000","last_became_active":"2024-09-11T08:49:09.911101+0000","last_became_peered":"2024-09-11T08:49:09.911101+0000","last_unstale":"2024-09-11T08:50:28.888988+0000","last_undegraded":"2024-09-11T08:50:28.888988+0000","last_fullsized":"2024-09-11T08:50:28.888988+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:04.191702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:04.191702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:17.639050+0000","scrub_duration":0.0034932320000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":408,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.899519+0000","last_change":"2024-09-11T08:49:27.577662+0000","last_active":"2024-09-11T08:50:28.899519+0000","last_peered":"2024-09-11T08:50:28.899519+0000","last_clean":"2024-09-11T08:50:28.899519+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:50:28.899519+0000","last_undegraded":"2024-09-11T08:50:28.899519+0000","last_fullsized":"2024-09-11T08:50:28.899519+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:49:27.577575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:49:27.577575+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:34.177695+0000","scrub_duration":0.0084200460000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":226,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.531278+0000","last_change":"2024-09-11T08:50:05.815267+0000","last_active":"2024-09-11T08:50:28.531278+0000","last_peered":"2024-09-11T08:50:28.531278+0000","last_clean":"2024-09-11T08:50:28.531278+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:50:28.531278+0000","last_undegraded":"2024-09-11T08:50:28.531278+0000","last_fullsized":"2024-09-11T08:50:28.531278+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:05.815178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:50:05.815178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:10.392227+0000","scrub_duration":0.0078777670000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"239'7829","reported_seq":7876,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.287348+0000","last_change":"2024-09-11T08:50:26.533353+0000","last_active":"2024-09-11T08:50:36.287348+0000","last_peered":"2024-09-11T08:50:36.287348+0000","last_clean":"2024-09-11T08:50:36.287348+0000","last_became_active":"2024-09-11T08:50:26.531198+0000","last_became_peered":"2024-09-11T08:50:26.531198+0000","last_unstale":"2024-09-11T08:50:36.287348+0000","last_undegraded":"2024-09-11T08:50:36.287348+0000","last_fullsized":"2024-09-11T08:50:36.287348+0000","mapping_epoch":206,"log_start":"216'2800","ondisk_log_start":"216'2800","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":5029,"log_dups_size":0,"ondisk_log_size":5029,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268926976,"num_objects":4103,"num_object_clones":0,"num_object_copies":12309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4103,"num_write_kb":262624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"200'15","reported_seq":266,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.892288+0000","last_change":"2024-09-11T08:49:09.925808+0000","last_active":"2024-09-11T08:50:28.892288+0000","last_peered":"2024-09-11T08:50:28.892288+0000","last_clean":"2024-09-11T08:50:28.892288+0000","last_became_active":"2024-09-11T08:49:09.915666+0000","last_became_peered":"2024-09-11T08:49:09.915666+0000","last_unstale":"2024-09-11T08:50:28.892288+0000","last_undegraded":"2024-09-11T08:50:28.892288+0000","last_fullsized":"2024-09-11T08:50:28.892288+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:48:41.787219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.787219+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060192070000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":244,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.532376+0000","last_change":"2024-09-11T08:49:30.234576+0000","last_active":"2024-09-11T08:50:28.532376+0000","last_peered":"2024-09-11T08:50:28.532376+0000","last_clean":"2024-09-11T08:50:28.532376+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:50:28.532376+0000","last_undegraded":"2024-09-11T08:50:28.532376+0000","last_fullsized":"2024-09-11T08:50:28.532376+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:30.234078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:49:30.234078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:55.702848+0000","scrub_duration":0.0025717829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":251,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.542864+0000","last_change":"2024-09-11T08:50:07.848009+0000","last_active":"2024-09-11T08:50:28.542864+0000","last_peered":"2024-09-11T08:50:28.542864+0000","last_clean":"2024-09-11T08:50:28.542864+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:50:28.542864+0000","last_undegraded":"2024-09-11T08:50:28.542864+0000","last_fullsized":"2024-09-11T08:50:28.542864+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:07.847927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:07.847927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:08.869403+0000","scrub_duration":0.0049449369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":41,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.538228+0000","last_change":"2024-09-11T08:49:09.912893+0000","last_active":"2024-09-11T08:50:28.538228+0000","last_peered":"2024-09-11T08:50:28.538228+0000","last_clean":"2024-09-11T08:50:28.538228+0000","last_became_active":"2024-09-11T08:49:09.912556+0000","last_became_peered":"2024-09-11T08:49:09.912556+0000","last_unstale":"2024-09-11T08:50:28.538228+0000","last_undegraded":"2024-09-11T08:50:28.538228+0000","last_fullsized":"2024-09-11T08:50:28.538228+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:47.699453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:47:47.699453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"239'7900","reported_seq":7951,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.253013+0000","last_change":"2024-09-11T08:50:26.521672+0000","last_active":"2024-09-11T08:50:36.253013+0000","last_peered":"2024-09-11T08:50:36.253013+0000","last_clean":"2024-09-11T08:50:36.253013+0000","last_became_active":"2024-09-11T08:50:26.519761+0000","last_became_peered":"2024-09-11T08:50:26.519761+0000","last_unstale":"2024-09-11T08:50:36.253013+0000","last_undegraded":"2024-09-11T08:50:36.253013+0000","last_fullsized":"2024-09-11T08:50:36.253013+0000","mapping_epoch":206,"log_start":"217'2900","ondisk_log_start":"217'2900","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":5000,"log_dups_size":0,"ondisk_log_size":5000,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271450112,"num_objects":4142,"num_object_clones":0,"num_object_copies":12426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4142,"num_write_kb":265088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"200'35","reported_seq":145,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.534808+0000","last_change":"2024-09-11T08:49:22.127201+0000","last_active":"2024-09-11T08:50:28.534808+0000","last_peered":"2024-09-11T08:50:28.534808+0000","last_clean":"2024-09-11T08:50:28.534808+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:50:28.534808+0000","last_undegraded":"2024-09-11T08:50:28.534808+0000","last_fullsized":"2024-09-11T08:50:28.534808+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'35","last_scrub_stamp":"2024-09-11T08:49:22.127119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:49:22.127119+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:33.163032+0000","scrub_duration":0.0038731960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":251,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.538343+0000","last_change":"2024-09-11T08:50:16.196911+0000","last_active":"2024-09-11T08:50:28.538343+0000","last_peered":"2024-09-11T08:50:28.538343+0000","last_clean":"2024-09-11T08:50:28.538343+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:50:28.538343+0000","last_undegraded":"2024-09-11T08:50:28.538343+0000","last_fullsized":"2024-09-11T08:50:28.538343+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:16.196521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:16.196521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:43.309271+0000","scrub_duration":0.0027210310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":267,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.539620+0000","last_change":"2024-09-11T08:49:25.610133+0000","last_active":"2024-09-11T08:50:28.539620+0000","last_peered":"2024-09-11T08:50:28.539620+0000","last_clean":"2024-09-11T08:50:28.539620+0000","last_became_active":"2024-09-11T08:49:09.907542+0000","last_became_peered":"2024-09-11T08:49:09.907542+0000","last_unstale":"2024-09-11T08:50:28.539620+0000","last_undegraded":"2024-09-11T08:50:28.539620+0000","last_fullsized":"2024-09-11T08:50:28.539620+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:25.609666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:49:25.609666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:46.984958+0000","scrub_duration":0.003585071,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":228,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.537998+0000","last_change":"2024-09-11T08:48:33.065142+0000","last_active":"2024-09-11T08:50:28.537998+0000","last_peered":"2024-09-11T08:50:28.537998+0000","last_clean":"2024-09-11T08:50:28.537998+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:50:28.537998+0000","last_undegraded":"2024-09-11T08:50:28.537998+0000","last_fullsized":"2024-09-11T08:50:28.537998+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:33.065053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:33.065053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013998086999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"239'7973","reported_seq":8025,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.551899+0000","last_change":"2024-09-11T08:50:26.523606+0000","last_active":"2024-09-11T08:50:36.551899+0000","last_peered":"2024-09-11T08:50:36.551899+0000","last_clean":"2024-09-11T08:50:36.551899+0000","last_became_active":"2024-09-11T08:50:26.521352+0000","last_became_peered":"2024-09-11T08:50:26.521352+0000","last_unstale":"2024-09-11T08:50:36.551899+0000","last_undegraded":"2024-09-11T08:50:36.551899+0000","last_fullsized":"2024-09-11T08:50:36.551899+0000","mapping_epoch":206,"log_start":"219'3500","ondisk_log_start":"219'3500","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4473,"log_dups_size":0,"ondisk_log_size":4473,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272924672,"num_objects":4164,"num_object_clones":0,"num_object_copies":12492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4164,"num_write_kb":266528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"200'18","reported_seq":262,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.530808+0000","last_change":"2024-09-11T08:50:03.790643+0000","last_active":"2024-09-11T08:50:28.530808+0000","last_peered":"2024-09-11T08:50:28.530808+0000","last_clean":"2024-09-11T08:50:28.530808+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:50:28.530808+0000","last_undegraded":"2024-09-11T08:50:28.530808+0000","last_fullsized":"2024-09-11T08:50:28.530808+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'18","last_scrub_stamp":"2024-09-11T08:50:03.790561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:03.790561+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:12.360934+0000","scrub_duration":0.050104567000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":241,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.536805+0000","last_change":"2024-09-11T08:48:47.033544+0000","last_active":"2024-09-11T08:50:28.536805+0000","last_peered":"2024-09-11T08:50:28.536805+0000","last_clean":"2024-09-11T08:50:28.536805+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:50:28.536805+0000","last_undegraded":"2024-09-11T08:50:28.536805+0000","last_fullsized":"2024-09-11T08:50:28.536805+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:47.033487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:47.033487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014934150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":270,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.543344+0000","last_change":"2024-09-11T08:50:06.808228+0000","last_active":"2024-09-11T08:50:28.543344+0000","last_peered":"2024-09-11T08:50:28.543344+0000","last_clean":"2024-09-11T08:50:28.543344+0000","last_became_active":"2024-09-11T08:49:09.907535+0000","last_became_peered":"2024-09-11T08:49:09.907535+0000","last_unstale":"2024-09-11T08:50:28.543344+0000","last_undegraded":"2024-09-11T08:50:28.543344+0000","last_fullsized":"2024-09-11T08:50:28.543344+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:06.808051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:06.808051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:30.861906+0000","scrub_duration":0.0080726240000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":226,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.888735+0000","last_change":"2024-09-11T08:50:15.277171+0000","last_active":"2024-09-11T08:50:28.888735+0000","last_peered":"2024-09-11T08:50:28.888735+0000","last_clean":"2024-09-11T08:50:28.888735+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:50:28.888735+0000","last_undegraded":"2024-09-11T08:50:28.888735+0000","last_fullsized":"2024-09-11T08:50:28.888735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:15.277094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:50:15.277094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:39.266295+0000","scrub_duration":0.004775751,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"239'7839","reported_seq":7891,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:36.321238+0000","last_change":"2024-09-11T08:50:26.517074+0000","last_active":"2024-09-11T08:50:36.321238+0000","last_peered":"2024-09-11T08:50:36.321238+0000","last_clean":"2024-09-11T08:50:36.321238+0000","last_became_active":"2024-09-11T08:50:26.511250+0000","last_became_peered":"2024-09-11T08:50:26.511250+0000","last_unstale":"2024-09-11T08:50:36.321238+0000","last_undegraded":"2024-09-11T08:50:36.321238+0000","last_fullsized":"2024-09-11T08:50:36.321238+0000","mapping_epoch":210,"log_start":"216'2800","ondisk_log_start":"216'2800","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":5039,"log_dups_size":0,"ondisk_log_size":5039,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268894208,"num_objects":4103,"num_object_clones":0,"num_object_copies":12309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4103,"num_write_kb":262592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"200'34","reported_seq":347,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.530995+0000","last_change":"2024-09-11T08:50:11.325671+0000","last_active":"2024-09-11T08:50:28.530995+0000","last_peered":"2024-09-11T08:50:28.530995+0000","last_clean":"2024-09-11T08:50:28.530995+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:50:28.530995+0000","last_undegraded":"2024-09-11T08:50:28.530995+0000","last_fullsized":"2024-09-11T08:50:28.530995+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:50:11.325616+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:50:11.325616+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:22.774524+0000","scrub_duration":0.0023307359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":274,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.537729+0000","last_change":"2024-09-11T08:49:17.884190+0000","last_active":"2024-09-11T08:50:28.537729+0000","last_peered":"2024-09-11T08:50:28.537729+0000","last_clean":"2024-09-11T08:50:28.537729+0000","last_became_active":"2024-09-11T08:49:09.904656+0000","last_became_peered":"2024-09-11T08:49:09.904656+0000","last_unstale":"2024-09-11T08:50:28.537729+0000","last_undegraded":"2024-09-11T08:50:28.537729+0000","last_fullsized":"2024-09-11T08:50:28.537729+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:49:17.884108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:17.884108+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:33.993275+0000","scrub_duration":0.0031968909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"168'10","reported_seq":416,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.889493+0000","last_change":"2024-09-11T08:48:41.038840+0000","last_active":"2024-09-11T08:50:28.889493+0000","last_peered":"2024-09-11T08:50:28.889493+0000","last_clean":"2024-09-11T08:50:28.889493+0000","last_became_active":"2024-09-11T08:46:01.983055+0000","last_became_peered":"2024-09-11T08:46:01.983055+0000","last_unstale":"2024-09-11T08:50:28.889493+0000","last_undegraded":"2024-09-11T08:50:28.889493+0000","last_fullsized":"2024-09-11T08:50:28.889493+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"168'10","last_scrub_stamp":"2024-09-11T08:48:41.038786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.038786+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00164314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":241,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.531281+0000","last_change":"2024-09-11T08:49:18.301286+0000","last_active":"2024-09-11T08:50:28.531281+0000","last_peered":"2024-09-11T08:50:28.531281+0000","last_clean":"2024-09-11T08:50:28.531281+0000","last_became_active":"2024-09-11T08:49:09.906201+0000","last_became_peered":"2024-09-11T08:49:09.906201+0000","last_unstale":"2024-09-11T08:50:28.531281+0000","last_undegraded":"2024-09-11T08:50:28.531281+0000","last_fullsized":"2024-09-11T08:50:28.531281+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:18.301203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:18.301203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:37.257368+0000","scrub_duration":0.0032551070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":271,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.535268+0000","last_change":"2024-09-11T08:49:34.002358+0000","last_active":"2024-09-11T08:50:28.535268+0000","last_peered":"2024-09-11T08:50:28.535268+0000","last_clean":"2024-09-11T08:50:28.535268+0000","last_became_active":"2024-09-11T08:49:09.910329+0000","last_became_peered":"2024-09-11T08:49:09.910329+0000","last_unstale":"2024-09-11T08:50:28.535268+0000","last_undegraded":"2024-09-11T08:50:28.535268+0000","last_fullsized":"2024-09-11T08:50:28.535268+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:34.002280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:49:34.002280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:39.769687+0000","scrub_duration":0.0089946609999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"200'12","reported_seq":266,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.889947+0000","last_change":"2024-09-11T08:49:15.347679+0000","last_active":"2024-09-11T08:50:28.889947+0000","last_peered":"2024-09-11T08:50:28.889947+0000","last_clean":"2024-09-11T08:50:28.889947+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:50:28.889947+0000","last_undegraded":"2024-09-11T08:50:28.889947+0000","last_fullsized":"2024-09-11T08:50:28.889947+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'12","last_scrub_stamp":"2024-09-11T08:49:15.347591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:49:15.347591+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048379369999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":237,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.532998+0000","last_change":"2024-09-11T08:50:07.390163+0000","last_active":"2024-09-11T08:50:28.532998+0000","last_peered":"2024-09-11T08:50:28.532998+0000","last_clean":"2024-09-11T08:50:28.532998+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:50:28.532998+0000","last_undegraded":"2024-09-11T08:50:28.532998+0000","last_fullsized":"2024-09-11T08:50:28.532998+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:07.389973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:07.389973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:13.015766+0000","scrub_duration":0.014729302999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":255,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.537672+0000","last_change":"2024-09-11T08:50:08.089714+0000","last_active":"2024-09-11T08:50:28.537672+0000","last_peered":"2024-09-11T08:50:28.537672+0000","last_clean":"2024-09-11T08:50:28.537672+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:50:28.537672+0000","last_undegraded":"2024-09-11T08:50:28.537672+0000","last_fullsized":"2024-09-11T08:50:28.537672+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:08.089015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:08.089015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:36.555156+0000","scrub_duration":0.0075245379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"200'15","reported_seq":276,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.889161+0000","last_change":"2024-09-11T08:50:24.263773+0000","last_active":"2024-09-11T08:50:28.889161+0000","last_peered":"2024-09-11T08:50:28.889161+0000","last_clean":"2024-09-11T08:50:28.889161+0000","last_became_active":"2024-09-11T08:49:09.916927+0000","last_became_peered":"2024-09-11T08:49:09.916927+0000","last_unstale":"2024-09-11T08:50:28.889161+0000","last_undegraded":"2024-09-11T08:50:28.889161+0000","last_fullsized":"2024-09-11T08:50:28.889161+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:50:24.263662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:24.263662+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:35.359306+0000","scrub_duration":0.0052890580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":244,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.539486+0000","last_change":"2024-09-11T08:48:20.977749+0000","last_active":"2024-09-11T08:50:28.539486+0000","last_peered":"2024-09-11T08:50:28.539486+0000","last_clean":"2024-09-11T08:50:28.539486+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:50:28.539486+0000","last_undegraded":"2024-09-11T08:50:28.539486+0000","last_fullsized":"2024-09-11T08:50:28.539486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:20.977665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:48:20.977665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075643139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":228,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.889390+0000","last_change":"2024-09-11T08:50:19.288275+0000","last_active":"2024-09-11T08:50:28.889390+0000","last_peered":"2024-09-11T08:50:28.889390+0000","last_clean":"2024-09-11T08:50:28.889390+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:50:28.889390+0000","last_undegraded":"2024-09-11T08:50:28.889390+0000","last_fullsized":"2024-09-11T08:50:28.889390+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:19.288216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:19.288216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:41.431437+0000","scrub_duration":0.0055736010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":256,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.539562+0000","last_change":"2024-09-11T08:50:17.717047+0000","last_active":"2024-09-11T08:50:28.539562+0000","last_peered":"2024-09-11T08:50:28.539562+0000","last_clean":"2024-09-11T08:50:28.539562+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:50:28.539562+0000","last_undegraded":"2024-09-11T08:50:28.539562+0000","last_fullsized":"2024-09-11T08:50:28.539562+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:17.716977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_clean_scrub_stamp":"2024-09-11T08:50:17.716977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:32.602486+0000","scrub_duration":0.0032296289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"240'7872","reported_seq":7945,"reported_epoch":240,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T08:50:36.861525+0000","last_change":"2024-09-11T08:50:28.971715+0000","last_active":"2024-09-11T08:50:36.861525+0000","last_peered":"2024-09-11T08:50:36.861525+0000","last_clean":"2024-09-11T08:50:27.504532+0000","last_became_active":"2024-09-11T08:50:28.966515+0000","last_became_peered":"2024-09-11T08:50:28.966515+0000","last_unstale":"2024-09-11T08:50:36.861525+0000","last_undegraded":"2024-09-11T08:50:36.861525+0000","last_fullsized":"2024-09-11T08:50:36.861525+0000","mapping_epoch":238,"log_start":"218'3200","ondisk_log_start":"218'3200","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4672,"log_dups_size":0,"ondisk_log_size":4672,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":269582336,"num_objects":4114,"num_object_clones":0,"num_object_copies":12342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4114,"num_objects_unfound":0,"num_objects_dirty":4114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4114,"num_write_kb":263264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[3,7,5],"avail_no_missing":["3","5","7"],"object_location_counts":[{"shards":"3,5,7","objects":4114}],"blocked_by":[],"up_primary":6,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"200'15","reported_seq":270,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.533711+0000","last_change":"2024-09-11T08:50:26.391719+0000","last_active":"2024-09-11T08:50:28.533711+0000","last_peered":"2024-09-11T08:50:28.533711+0000","last_clean":"2024-09-11T08:50:28.533711+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:50:28.533711+0000","last_undegraded":"2024-09-11T08:50:28.533711+0000","last_fullsized":"2024-09-11T08:50:28.533711+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:50:26.391622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:26.391622+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:38.458701+0000","scrub_duration":0.0021692130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":41,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.537085+0000","last_change":"2024-09-11T08:49:09.912712+0000","last_active":"2024-09-11T08:50:28.537085+0000","last_peered":"2024-09-11T08:50:28.537085+0000","last_clean":"2024-09-11T08:50:28.537085+0000","last_became_active":"2024-09-11T08:49:09.912417+0000","last_became_peered":"2024-09-11T08:49:09.912417+0000","last_unstale":"2024-09-11T08:50:28.537085+0000","last_undegraded":"2024-09-11T08:50:28.537085+0000","last_fullsized":"2024-09-11T08:50:28.537085+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:45.751518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:45.751518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":379,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:32.812771+0000","last_change":"2024-09-11T08:50:16.313863+0000","last_active":"2024-09-11T08:50:32.812771+0000","last_peered":"2024-09-11T08:50:32.812771+0000","last_clean":"2024-09-11T08:50:32.812771+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:50:32.812771+0000","last_undegraded":"2024-09-11T08:50:32.812771+0000","last_fullsized":"2024-09-11T08:50:32.812771+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:50:16.313779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:16.313779+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:39.660142+0000","scrub_duration":0.0049551839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":216,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.532273+0000","last_change":"2024-09-11T08:48:41.648829+0000","last_active":"2024-09-11T08:50:28.532273+0000","last_peered":"2024-09-11T08:50:28.532273+0000","last_clean":"2024-09-11T08:50:28.532273+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:50:28.532273+0000","last_undegraded":"2024-09-11T08:50:28.532273+0000","last_fullsized":"2024-09-11T08:50:28.532273+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:41.648756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.648756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019862030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.19","version":"240'8002","reported_seq":10817,"reported_epoch":240,"state":"active+remapped+backfilling","last_fresh":"2024-09-11T08:50:36.861889+0000","last_change":"2024-09-11T08:50:28.971326+0000","last_active":"2024-09-11T08:50:36.861889+0000","last_peered":"2024-09-11T08:50:36.861889+0000","last_clean":"2024-09-11T08:50:27.489720+0000","last_became_active":"2024-09-11T08:50:28.962144+0000","last_became_peered":"2024-09-11T08:50:28.962144+0000","last_unstale":"2024-09-11T08:50:36.861889+0000","last_undegraded":"2024-09-11T08:50:36.861889+0000","last_fullsized":"2024-09-11T08:50:36.861889+0000","mapping_epoch":238,"log_start":"217'3100","ondisk_log_start":"217'3100","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":275578880,"num_objects":4205,"num_object_clones":0,"num_object_copies":12615,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2767,"num_objects_unfound":0,"num_objects_dirty":4205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4205,"num_write_kb":269120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1376,"num_bytes_recovered":90177536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,5,7],"avail_no_missing":["3","5","7"],"object_location_counts":[{"shards":"3,5,7","objects":4205}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"200'25","reported_seq":265,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.889249+0000","last_change":"2024-09-11T08:49:28.382609+0000","last_active":"2024-09-11T08:50:28.889249+0000","last_peered":"2024-09-11T08:50:28.889249+0000","last_clean":"2024-09-11T08:50:28.889249+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:50:28.889249+0000","last_undegraded":"2024-09-11T08:50:28.889249+0000","last_fullsized":"2024-09-11T08:50:28.889249+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:49:28.382409+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:49:28.382409+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046362419999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":241,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.538056+0000","last_change":"2024-09-11T08:48:39.064220+0000","last_active":"2024-09-11T08:50:28.538056+0000","last_peered":"2024-09-11T08:50:28.538056+0000","last_clean":"2024-09-11T08:50:28.538056+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:50:28.538056+0000","last_undegraded":"2024-09-11T08:50:28.538056+0000","last_fullsized":"2024-09-11T08:50:28.538056+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:39.064163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_clean_scrub_stamp":"2024-09-11T08:48:39.064163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014228540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"211'9","reported_seq":402,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.887877+0000","last_change":"2024-09-11T08:50:08.389210+0000","last_active":"2024-09-11T08:50:28.887877+0000","last_peered":"2024-09-11T08:50:28.887877+0000","last_clean":"2024-09-11T08:50:28.887877+0000","last_became_active":"2024-09-11T08:49:09.908380+0000","last_became_peered":"2024-09-11T08:49:09.908380+0000","last_unstale":"2024-09-11T08:50:28.887877+0000","last_undegraded":"2024-09-11T08:50:28.887877+0000","last_fullsized":"2024-09-11T08:50:28.887877+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'9","last_scrub_stamp":"2024-09-11T08:50:08.388991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:08.388991+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:19.129916+0000","scrub_duration":0.019634517000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":51,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.536402+0000","last_change":"2024-09-11T08:49:21.855032+0000","last_active":"2024-09-11T08:50:28.536402+0000","last_peered":"2024-09-11T08:50:28.536402+0000","last_clean":"2024-09-11T08:50:28.536402+0000","last_became_active":"2024-09-11T08:49:09.914396+0000","last_became_peered":"2024-09-11T08:49:09.914396+0000","last_unstale":"2024-09-11T08:50:28.536402+0000","last_undegraded":"2024-09-11T08:50:28.536402+0000","last_fullsized":"2024-09-11T08:50:28.536402+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:21.854973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:21.854973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:48.440684+0000","scrub_duration":0.0064444309999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"200'19","reported_seq":273,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.531509+0000","last_change":"2024-09-11T08:49:16.951920+0000","last_active":"2024-09-11T08:50:28.531509+0000","last_peered":"2024-09-11T08:50:28.531509+0000","last_clean":"2024-09-11T08:50:28.531509+0000","last_became_active":"2024-09-11T08:49:16.937185+0000","last_became_peered":"2024-09-11T08:49:16.937185+0000","last_unstale":"2024-09-11T08:50:28.531509+0000","last_undegraded":"2024-09-11T08:50:28.531509+0000","last_fullsized":"2024-09-11T08:50:28.531509+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:25.792777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:25.792777+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045475469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":232,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.532586+0000","last_change":"2024-09-11T08:50:04.791525+0000","last_active":"2024-09-11T08:50:28.532586+0000","last_peered":"2024-09-11T08:50:28.532586+0000","last_clean":"2024-09-11T08:50:28.532586+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:50:28.532586+0000","last_undegraded":"2024-09-11T08:50:28.532586+0000","last_fullsized":"2024-09-11T08:50:28.532586+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:04.791327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:04.791327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:19.842232+0000","scrub_duration":0.0063425010000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":238,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.897965+0000","last_change":"2024-09-11T08:48:30.985518+0000","last_active":"2024-09-11T08:50:28.897965+0000","last_peered":"2024-09-11T08:50:28.897965+0000","last_clean":"2024-09-11T08:50:28.897965+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:50:28.897965+0000","last_undegraded":"2024-09-11T08:50:28.897965+0000","last_fullsized":"2024-09-11T08:50:28.897965+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:30.985271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.985271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099058919999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":236,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.896025+0000","last_change":"2024-09-11T08:50:21.261969+0000","last_active":"2024-09-11T08:50:28.896025+0000","last_peered":"2024-09-11T08:50:28.896025+0000","last_clean":"2024-09-11T08:50:28.896025+0000","last_became_active":"2024-09-11T08:49:09.905289+0000","last_became_peered":"2024-09-11T08:49:09.905289+0000","last_unstale":"2024-09-11T08:50:28.896025+0000","last_undegraded":"2024-09-11T08:50:28.896025+0000","last_fullsized":"2024-09-11T08:50:28.896025+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:21.261891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:21.261891+0000","last_clean_scrub_stamp":"2024-09-11T08:50:21.261891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:31.439294+0000","scrub_duration":0.0044878879999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"200'15","reported_seq":277,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.531013+0000","last_change":"2024-09-11T08:50:03.319748+0000","last_active":"2024-09-11T08:50:28.531013+0000","last_peered":"2024-09-11T08:50:28.531013+0000","last_clean":"2024-09-11T08:50:28.531013+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:50:28.531013+0000","last_undegraded":"2024-09-11T08:50:28.531013+0000","last_fullsized":"2024-09-11T08:50:28.531013+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:50:03.319664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:03.319664+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:12.910342+0000","scrub_duration":0.0046538380000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":251,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.538209+0000","last_change":"2024-09-11T08:50:21.208754+0000","last_active":"2024-09-11T08:50:28.538209+0000","last_peered":"2024-09-11T08:50:28.538209+0000","last_clean":"2024-09-11T08:50:28.538209+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:50:28.538209+0000","last_undegraded":"2024-09-11T08:50:28.538209+0000","last_fullsized":"2024-09-11T08:50:28.538209+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:21.208684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:21.208684+0000","last_clean_scrub_stamp":"2024-09-11T08:50:21.208684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:28.076257+0000","scrub_duration":0.0075563440000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":238,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.888828+0000","last_change":"2024-09-11T08:49:27.342962+0000","last_active":"2024-09-11T08:50:28.888828+0000","last_peered":"2024-09-11T08:50:28.888828+0000","last_clean":"2024-09-11T08:50:28.888828+0000","last_became_active":"2024-09-11T08:49:09.910548+0000","last_became_peered":"2024-09-11T08:49:09.910548+0000","last_unstale":"2024-09-11T08:50:28.888828+0000","last_undegraded":"2024-09-11T08:50:28.888828+0000","last_fullsized":"2024-09-11T08:50:28.888828+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:27.342908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:49:27.342908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020004860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":51,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.530900+0000","last_change":"2024-09-11T08:50:24.401692+0000","last_active":"2024-09-11T08:50:28.530900+0000","last_peered":"2024-09-11T08:50:28.530900+0000","last_clean":"2024-09-11T08:50:28.530900+0000","last_became_active":"2024-09-11T08:49:09.904622+0000","last_became_peered":"2024-09-11T08:49:09.904622+0000","last_unstale":"2024-09-11T08:50:28.530900+0000","last_undegraded":"2024-09-11T08:50:28.530900+0000","last_fullsized":"2024-09-11T08:50:28.530900+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:24.401480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:24.401480+0000","last_clean_scrub_stamp":"2024-09-11T08:50:24.401480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:43.701178+0000","scrub_duration":0.010533496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"200'19","reported_seq":251,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.895954+0000","last_change":"2024-09-11T08:48:18.890324+0000","last_active":"2024-09-11T08:50:28.895954+0000","last_peered":"2024-09-11T08:50:28.895954+0000","last_clean":"2024-09-11T08:50:28.895954+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:50:28.895954+0000","last_undegraded":"2024-09-11T08:50:28.895954+0000","last_fullsized":"2024-09-11T08:50:28.895954+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-11T08:48:18.890233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.890233+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070086059999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":241,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.533073+0000","last_change":"2024-09-11T08:49:33.209392+0000","last_active":"2024-09-11T08:50:28.533073+0000","last_peered":"2024-09-11T08:50:28.533073+0000","last_clean":"2024-09-11T08:50:28.533073+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:50:28.533073+0000","last_undegraded":"2024-09-11T08:50:28.533073+0000","last_fullsized":"2024-09-11T08:50:28.533073+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:33.209345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:33.209345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:45.794127+0000","scrub_duration":0.0052610230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":240,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.536036+0000","last_change":"2024-09-11T08:48:18.958742+0000","last_active":"2024-09-11T08:50:28.536036+0000","last_peered":"2024-09-11T08:50:28.536036+0000","last_clean":"2024-09-11T08:50:28.536036+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:50:28.536036+0000","last_undegraded":"2024-09-11T08:50:28.536036+0000","last_fullsized":"2024-09-11T08:50:28.536036+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:18.958609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:18.958609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001704777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":256,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.888912+0000","last_change":"2024-09-11T08:50:25.255389+0000","last_active":"2024-09-11T08:50:28.888912+0000","last_peered":"2024-09-11T08:50:28.888912+0000","last_clean":"2024-09-11T08:50:28.888912+0000","last_became_active":"2024-09-11T08:49:09.905634+0000","last_became_peered":"2024-09-11T08:49:09.905634+0000","last_unstale":"2024-09-11T08:50:28.888912+0000","last_undegraded":"2024-09-11T08:50:28.888912+0000","last_fullsized":"2024-09-11T08:50:28.888912+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:25.255332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:25.255332+0000","last_clean_scrub_stamp":"2024-09-11T08:50:25.255332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:39.868191+0000","scrub_duration":0.0046648280000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"200'10","reported_seq":276,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.888373+0000","last_change":"2024-09-11T08:48:32.020078+0000","last_active":"2024-09-11T08:50:28.888373+0000","last_peered":"2024-09-11T08:50:28.888373+0000","last_clean":"2024-09-11T08:50:28.888373+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:50:28.888373+0000","last_undegraded":"2024-09-11T08:50:28.888373+0000","last_fullsized":"2024-09-11T08:50:28.888373+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'6","last_scrub_stamp":"2024-09-11T08:48:32.019991+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:48:32.019991+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010394439,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":63,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.533968+0000","last_change":"2024-09-11T08:50:09.396223+0000","last_active":"2024-09-11T08:50:28.533968+0000","last_peered":"2024-09-11T08:50:28.533968+0000","last_clean":"2024-09-11T08:50:28.533968+0000","last_became_active":"2024-09-11T08:49:09.907483+0000","last_became_peered":"2024-09-11T08:49:09.907483+0000","last_unstale":"2024-09-11T08:50:28.533968+0000","last_undegraded":"2024-09-11T08:50:28.533968+0000","last_fullsized":"2024-09-11T08:50:28.533968+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:50:09.396148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:09.396148+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:10.843508+0000","scrub_duration":0.013176124000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":101,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.534285+0000","last_change":"2024-09-11T08:50:05.193741+0000","last_active":"2024-09-11T08:50:28.534285+0000","last_peered":"2024-09-11T08:50:28.534285+0000","last_clean":"2024-09-11T08:50:28.534285+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:50:28.534285+0000","last_undegraded":"2024-09-11T08:50:28.534285+0000","last_fullsized":"2024-09-11T08:50:28.534285+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:05.193617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:05.193617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:31.887067+0000","scrub_duration":0.0042825290000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":245,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.536321+0000","last_change":"2024-09-11T08:49:30.788379+0000","last_active":"2024-09-11T08:50:28.536321+0000","last_peered":"2024-09-11T08:50:28.536321+0000","last_clean":"2024-09-11T08:50:28.536321+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:50:28.536321+0000","last_undegraded":"2024-09-11T08:50:28.536321+0000","last_fullsized":"2024-09-11T08:50:28.536321+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:30.788303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:30.788303+0000","last_clean_scrub_stamp":"2024-09-11T08:49:30.788303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:37.198202+0000","scrub_duration":0.0038913020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.12","version":"239'7785","reported_seq":7846,"reported_epoch":239,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T08:50:36.520908+0000","last_change":"2024-09-11T08:50:28.953359+0000","last_active":"2024-09-11T08:50:36.520908+0000","last_peered":"2024-09-11T08:50:36.520908+0000","last_clean":"2024-09-11T08:50:27.499260+0000","last_became_active":"2024-09-11T08:50:28.949161+0000","last_became_peered":"2024-09-11T08:50:28.949161+0000","last_unstale":"2024-09-11T08:50:36.520908+0000","last_undegraded":"2024-09-11T08:50:36.520908+0000","last_fullsized":"2024-09-11T08:50:36.520908+0000","mapping_epoch":238,"log_start":"218'3200","ondisk_log_start":"218'3200","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4585,"log_dups_size":0,"ondisk_log_size":4585,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266862592,"num_objects":4072,"num_object_clones":0,"num_object_copies":12216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4072,"num_objects_unfound":0,"num_objects_dirty":4072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4072,"num_write_kb":260608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[1,2,3],"avail_no_missing":["1","2","3"],"object_location_counts":[{"shards":"1,2,3","objects":4072}],"blocked_by":[],"up_primary":5,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":246,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.892972+0000","last_change":"2024-09-11T08:49:29.487522+0000","last_active":"2024-09-11T08:50:28.892972+0000","last_peered":"2024-09-11T08:50:28.892972+0000","last_clean":"2024-09-11T08:50:28.892972+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:50:28.892972+0000","last_undegraded":"2024-09-11T08:50:28.892972+0000","last_fullsized":"2024-09-11T08:50:28.892972+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:29.487391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:29.487391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:50:53.969768+0000","scrub_duration":0.0091739200000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":242,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.897586+0000","last_change":"2024-09-11T08:49:09.919371+0000","last_active":"2024-09-11T08:50:28.897586+0000","last_peered":"2024-09-11T08:50:28.897586+0000","last_clean":"2024-09-11T08:50:28.897586+0000","last_became_active":"2024-09-11T08:49:09.918992+0000","last_became_peered":"2024-09-11T08:49:09.918992+0000","last_unstale":"2024-09-11T08:50:28.897586+0000","last_undegraded":"2024-09-11T08:50:28.897586+0000","last_fullsized":"2024-09-11T08:50:28.897586+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:40.033594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:48:40.033594+0000","last_clean_scrub_stamp":"2024-09-11T08:48:40.033594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022129379999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"200'19","reported_seq":125,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.535328+0000","last_change":"2024-09-11T08:50:10.181391+0000","last_active":"2024-09-11T08:50:28.535328+0000","last_peered":"2024-09-11T08:50:28.535328+0000","last_clean":"2024-09-11T08:50:28.535328+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:50:28.535328+0000","last_undegraded":"2024-09-11T08:50:28.535328+0000","last_fullsized":"2024-09-11T08:50:28.535328+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:50:10.181286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:10.181286+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:31.378775+0000","scrub_duration":0.0058024799999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":51,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-11T08:50:28.536490+0000","last_change":"2024-09-11T08:50:26.239880+0000","last_active":"2024-09-11T08:50:28.536490+0000","last_peered":"2024-09-11T08:50:28.536490+0000","last_clean":"2024-09-11T08:50:28.536490+0000","last_became_active":"2024-09-11T08:49:09.904721+0000","last_became_peered":"2024-09-11T08:49:09.904721+0000","last_unstale":"2024-09-11T08:50:28.536490+0000","last_undegraded":"2024-09-11T08:50:28.536490+0000","last_fullsized":"2024-09-11T08:50:28.536490+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:26.239829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:26.239829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:50.757374+0000","scrub_duration":0.001788675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":26,"stat_sum":{"num_bytes":8713076736,"num_objects":132951,"num_object_clones":0,"num_object_copies":398853,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":38287,"num_objects_unfound":0,"num_objects_dirty":132951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132951,"num_write_kb":8508864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5243,"num_bytes_recovered":343605248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":26606764032,"data_stored":26606764032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121578,"ondisk_log_size":121578,"up":78,"acting":78,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1119,"num_read_kb":876,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":3234,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":12426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":751,"ondisk_log_size":751,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":708,"num_read_kb":703,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299541,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5567556,"kb_used_data":5261884,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":88177596,"statfs":{"total":95995035648,"available":90293858304,"internally_reserved":0,"allocated":5388169216,"data_stored":5387719679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":1.2010000000000001,"15min":0.86099999999999999},"min":{"1min":0.27000000000000002,"5min":0.23200000000000001,"15min":0.22},"max":{"1min":2.0619999999999998,"5min":32.171999999999997,"15min":32.171999999999997},"last":2.927},{"interface":"front","average":{"1min":1.131,"5min":1.393,"15min":0.97399999999999998},"min":{"1min":0.36499999999999999,"5min":0.221,"15min":0.21199999999999999},"max":{"1min":3.1960000000000002,"5min":32.149999999999999,"15min":32.149999999999999},"last":0.75900000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":1.4490000000000001,"15min":1.0009999999999999},"min":{"1min":0.24099999999999999,"5min":0.224,"15min":0.224},"max":{"1min":2.101,"5min":33.692999999999998,"15min":33.692999999999998},"last":2.9780000000000002},{"interface":"front","average":{"1min":1.167,"5min":1.4399999999999999,"15min":0.94799999999999995},"min":{"1min":0.40699999999999997,"5min":0.23799999999999999,"15min":0.23300000000000001},"max":{"1min":3.4049999999999998,"5min":35.591000000000001,"15min":35.591000000000001},"last":0.70699999999999996}]},{"osd":2,"last update":"Wed Sep 11 08:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.97099999999999997,"15min":0.80900000000000005},"min":{"1min":0.35499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":1.9610000000000001,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":1.2170000000000001,"5min":0.99299999999999999,"15min":0.80400000000000005},"min":{"1min":0.35399999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":7.5259999999999998,"5min":7.5259999999999998,"15min":7.5259999999999998},"last":2.9529999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.98899999999999999,"15min":0.84899999999999998},"min":{"1min":0.36799999999999999,"5min":0.27200000000000002,"15min":0.25},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":7.8609999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":1.03,"15min":0.92300000000000004},"min":{"1min":0.38900000000000001,"5min":0.27700000000000002,"15min":0.27400000000000002},"max":{"1min":2.4849999999999999,"5min":5.625,"15min":6.9139999999999997},"last":2.9009999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:49:48 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":0.71099999999999997,"15min":0.53800000000000003},"min":{"1min":0.33400000000000002,"5min":0.24099999999999999,"15min":0.23400000000000001},"max":{"1min":7.1520000000000001,"5min":7.1520000000000001,"15min":7.1520000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.77800000000000002,"15min":0.55200000000000005},"min":{"1min":0.29199999999999998,"5min":0.216,"15min":0.216},"max":{"1min":3.875,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.57299999999999995}]},{"osd":5,"last update":"Wed Sep 11 08:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.82499999999999996,"15min":0.57999999999999996},"min":{"1min":0.318,"5min":0.29399999999999998,"15min":0.23300000000000001},"max":{"1min":2.448,"5min":5.1059999999999999,"15min":5.1059999999999999},"last":2.863},{"interface":"front","average":{"1min":1.355,"5min":0.86599999999999999,"15min":0.627},"min":{"1min":0.33400000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":6.3650000000000002,"5min":6.3650000000000002,"15min":6.3650000000000002},"last":0.93300000000000005}]},{"osd":6,"last update":"Wed Sep 11 08:49:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.77900000000000003,"15min":0.62},"min":{"1min":0.38,"5min":0.317,"15min":0.315},"max":{"1min":3.512,"5min":3.512,"15min":3.5979999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.097,"5min":0.86099999999999999,"15min":0.63400000000000001},"min":{"1min":0.38900000000000001,"5min":0.30499999999999999,"15min":0.27900000000000003},"max":{"1min":3.9940000000000002,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.34200000000000003}]}]},{"osd":6,"up_from":45,"seq":193273528474,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2034480,"kb_used_data":1728808,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91710672,"statfs":{"total":95995035648,"available":93911728128,"internally_reserved":0,"allocated":1770299392,"data_stored":1769870203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:50:30 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":1.4450000000000001,"15min":1.123},"min":{"1min":0.26800000000000002,"5min":0.222,"15min":0.217},"max":{"1min":2.1549999999999998,"5min":27.908000000000001,"15min":28.302},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.4099999999999999,"15min":1.1040000000000001},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.23300000000000001},"max":{"1min":4.8780000000000001,"5min":27.771000000000001,"15min":28.314},"last":0.26100000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:50:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.7010000000000001,"15min":1.089},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":15.919,"5min":27.984000000000002,"15min":27.984000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.581,"15min":1.0389999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.253},"max":{"1min":15.629,"5min":28.106000000000002,"15min":28.106000000000002},"last":0.74399999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:50:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.85299999999999998,"15min":0.77900000000000003},"min":{"1min":0.28399999999999997,"5min":0.27700000000000002,"15min":0.23200000000000001},"max":{"1min":1.7330000000000001,"5min":3.4580000000000002,"15min":14.313000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0229999999999999,"15min":0.81399999999999995},"min":{"1min":0.314,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":4.9020000000000001,"5min":4.9020000000000001,"15min":6.5979999999999999},"last":0.36799999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:50:30 2024","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":1.637,"15min":1.107},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":15.798999999999999,"5min":27.795000000000002,"15min":27.795000000000002},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.871,"5min":1.714,"15min":1.1200000000000001},"min":{"1min":0.34799999999999998,"5min":0.311,"15min":0.224},"max":{"1min":15.599,"5min":27.841999999999999,"15min":27.841999999999999},"last":0.83099999999999996}]},{"osd":4,"last update":"Wed Sep 11 08:50:30 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.67100000000000004,"15min":0.51300000000000001},"min":{"1min":0.29799999999999999,"5min":0.25600000000000001,"15min":0.20799999999999999},"max":{"1min":4.8360000000000003,"5min":4.8360000000000003,"15min":4.8360000000000003},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.879,"5min":0.67800000000000005,"15min":0.50900000000000001},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":5.5330000000000004,"5min":5.5330000000000004,"15min":5.5330000000000004},"last":0.70099999999999996}]},{"osd":5,"last update":"Wed Sep 11 08:50:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.875,"15min":0.60899999999999999},"min":{"1min":0.27600000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":4.9240000000000004,"5min":8.6829999999999998,"15min":8.6829999999999998},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.82099999999999995,"15min":0.58199999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.5680000000000001,"5min":5.0819999999999999,"15min":5.0819999999999999},"last":0.65300000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78900000000000003,"15min":0.57499999999999996},"min":{"1min":0.32500000000000001,"5min":0.29899999999999999,"15min":0.247},"max":{"1min":1.867,"5min":7.327,"15min":7.327},"last":1.9430000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.57799999999999996},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.27700000000000002},"max":{"1min":1.774,"5min":5.4500000000000002,"15min":5.4500000000000002},"last":1.778}]}]},{"osd":5,"up_from":38,"seq":163208757406,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5141628,"kb_used_data":4827636,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":88603524,"statfs":{"total":95995035648,"available":90730008576,"internally_reserved":0,"allocated":4943499264,"data_stored":4943057898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:50:06 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.044,"15min":0.81399999999999995},"min":{"1min":0.26200000000000001,"5min":0.21099999999999999,"15min":0.20599999999999999},"max":{"1min":4.6420000000000003,"5min":4.6420000000000003,"15min":7.3899999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":1.117,"5min":1.3009999999999999,"15min":0.94599999999999995},"min":{"1min":0.316,"5min":0.23400000000000001,"15min":0.23100000000000001},"max":{"1min":5.2560000000000002,"5min":8.0410000000000004,"15min":8.0410000000000004},"last":0.60799999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:50:06 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.2509999999999999,"15min":0.92500000000000004},"min":{"1min":0.26000000000000001,"5min":0.23799999999999999,"15min":0.22800000000000001},"max":{"1min":5.3280000000000003,"5min":8,"15min":8},"last":0.29299999999999998},{"interface":"front","average":{"1min":1.359,"5min":1.1779999999999999,"15min":0.876},"min":{"1min":0.40200000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":5.3949999999999996,"5min":5.3949999999999996,"15min":6.6719999999999997},"last":0.44900000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:50:06 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.206,"15min":0.89800000000000002},"min":{"1min":0.29199999999999998,"5min":0.27800000000000002,"15min":0.23799999999999999},"max":{"1min":6.4080000000000004,"5min":6.4080000000000004,"15min":6.4080000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.165,"15min":0.86099999999999999},"min":{"1min":0.35899999999999999,"5min":0.28699999999999998,"15min":0.26400000000000001},"max":{"1min":5.2329999999999997,"5min":8.0640000000000001,"15min":8.0640000000000001},"last":0.63300000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:50:06 2024","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.085,"15min":0.86299999999999999},"min":{"1min":0.42699999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":6.4299999999999997,"5min":6.4299999999999997,"15min":6.4299999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.3460000000000001,"15min":1.0129999999999999},"min":{"1min":0.39400000000000002,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":5.2889999999999997,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Sep 11 08:50:06 2024","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":0.90000000000000002,"15min":0.56899999999999995},"min":{"1min":0.27600000000000002,"5min":0.247,"15min":0.20699999999999999},"max":{"1min":5.4180000000000001,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":0.79800000000000004,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.25800000000000001,"15min":0.20499999999999999},"max":{"1min":6.3769999999999998,"5min":6.3769999999999998,"15min":6.3769999999999998},"last":0.308}]},{"osd":6,"last update":"Wed Sep 11 08:50:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":0.95799999999999996,"15min":0.61799999999999999},"min":{"1min":0.30399999999999999,"5min":0.29399999999999998,"15min":0.23599999999999999},"max":{"1min":6.4669999999999996,"5min":6.4669999999999996,"15min":6.4669999999999996},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.1850000000000001,"5min":0.90600000000000003,"15min":0.60199999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.251},"max":{"1min":5.3090000000000002,"5min":8.1920000000000002,"15min":8.1920000000000002},"last":0.379}]},{"osd":7,"last update":"Wed Sep 11 08:49:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.6839999999999999,"15min":0.88400000000000001},"min":{"1min":0.33000000000000002,"5min":0.29999999999999999,"15min":0.27800000000000002},"max":{"1min":9.4009999999999998,"5min":88.513000000000005,"15min":88.513000000000005},"last":0.39400000000000002},{"interface":"front","average":{"1min":1.667,"5min":1.7450000000000001,"15min":0.92900000000000005},"min":{"1min":0.40200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":9.7089999999999996,"5min":86.963999999999999,"15min":86.963999999999999},"last":0.73899999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986338,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1980788,"kb_used_data":1670956,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91764364,"statfs":{"total":95995035648,"available":93966708736,"internally_reserved":0,"allocated":1711058944,"data_stored":1710624379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":1.571,"15min":1.569},"min":{"1min":0.22600000000000001,"5min":0.21199999999999999,"15min":0.20399999999999999},"max":{"1min":1.2330000000000001,"5min":44.140999999999998,"15min":89.510000000000005},"last":0.442},{"interface":"front","average":{"1min":0.58899999999999997,"5min":1.831,"15min":1.6659999999999999},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.4590000000000001,"5min":44.450000000000003,"15min":89.765000000000001},"last":0.38800000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:49:42 2024","interfaces":[{"interface":"back","average":{"1min":3.8079999999999998,"5min":2.282,"15min":1.2989999999999999},"min":{"1min":0.28199999999999997,"5min":0.27300000000000002,"15min":0.24399999999999999},"max":{"1min":48.976999999999997,"5min":48.976999999999997,"15min":48.976999999999997},"last":0.626},{"interface":"front","average":{"1min":3.4140000000000001,"5min":2.2599999999999998,"15min":1.2549999999999999},"min":{"1min":0.308,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":45.787999999999997,"5min":45.787999999999997,"15min":45.787999999999997},"last":0.56399999999999995}]},{"osd":2,"last update":"Wed Sep 11 08:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":1.097,"15min":0.81699999999999995},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.649,"5min":17.303000000000001,"15min":17.303000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":1.026,"15min":0.81299999999999994},"min":{"1min":0.29199999999999998,"5min":0.27500000000000002,"15min":0.25800000000000001},"max":{"1min":0.88400000000000001,"5min":20.702999999999999,"15min":20.702999999999999},"last":0.64500000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:49:41 2024","interfaces":[{"interface":"back","average":{"1min":3.157,"5min":2.0649999999999999,"15min":1.841},"min":{"1min":0.28399999999999997,"5min":0.255,"15min":0.255},"max":{"1min":44.052,"5min":45.268000000000001,"15min":89.881},"last":0.42499999999999999},{"interface":"front","average":{"1min":3.1539999999999999,"5min":2.1739999999999999,"15min":1.8},"min":{"1min":0.33900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":43.646000000000001,"5min":44.234000000000002,"15min":89.715000000000003},"last":0.498}]},{"osd":5,"last update":"Wed Sep 11 08:50:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.82499999999999996,"15min":0.56200000000000006},"min":{"1min":0.25900000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":4.2069999999999999,"5min":4.2069999999999999,"15min":4.2069999999999999},"last":1.9870000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.52600000000000002},"min":{"1min":0.31900000000000001,"5min":0.26100000000000001,"15min":0.23699999999999999},"max":{"1min":2.6850000000000001,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.46700000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:50:35 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.72099999999999997,"15min":0.54000000000000004},"min":{"1min":0.27300000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":3.02,"5min":3.02,"15min":3.02},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.72999999999999998,"15min":0.57799999999999996},"min":{"1min":0.28299999999999997,"5min":0.221,"15min":0.221},"max":{"1min":3.7970000000000002,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":0.53600000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64300000000000002,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.252,"15min":0.252},"max":{"1min":2.2010000000000001,"5min":3.339,"15min":3.339},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.79300000000000004,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.29599999999999999,"15min":0.28799999999999998},"max":{"1min":2.847,"5min":5.1970000000000001,"15min":5.1970000000000001},"last":0.67600000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149864,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5427732,"kb_used_data":4833676,"kb_used_omap":1,"kb_used_meta":594046,"kb_avail":88317420,"statfs":{"total":95995035648,"available":90437038080,"internally_reserved":0,"allocated":4949684224,"data_stored":4949218971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":608303448},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":1.1859999999999999,"15min":1.0389999999999999},"min":{"1min":0.20100000000000001,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":1.5800000000000001,"5min":24.280000000000001,"15min":24.280000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.83699999999999997,"5min":1.4159999999999999,"15min":1.25},"min":{"1min":0.28999999999999998,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":2.3100000000000001,"5min":22.225000000000001,"15min":22.225000000000001},"last":0.46300000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:50:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":2.5819999999999999,"15min":1.5960000000000001},"min":{"1min":0.32700000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":7.4909999999999997,"5min":115.601,"15min":115.601},"last":2.282},{"interface":"front","average":{"1min":1.04,"5min":2.4390000000000001,"15min":1.5069999999999999},"min":{"1min":0.30099999999999999,"5min":0.22700000000000001,"15min":0.17999999999999999},"max":{"1min":6.6639999999999997,"5min":115.474,"15min":115.474},"last":0.71299999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":1.1299999999999999,"15min":1.0960000000000001},"min":{"1min":0.29499999999999998,"5min":0.24299999999999999,"15min":0.23200000000000001},"max":{"1min":0.89400000000000002,"5min":8.2699999999999996,"15min":8.9510000000000005},"last":0.318},{"interface":"front","average":{"1min":0.81100000000000005,"5min":1.0309999999999999,"15min":1.052},"min":{"1min":0.26200000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.3719999999999999,"5min":5.5999999999999996,"15min":10.695},"last":0.47699999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:50:35 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.4359999999999999,"15min":1.093},"min":{"1min":0.36399999999999999,"5min":0.28599999999999998,"15min":0.26700000000000002},"max":{"1min":2.8919999999999999,"5min":8.3279999999999994,"15min":8.9969999999999999},"last":0.502},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.099,"15min":0.93400000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.27700000000000002},"max":{"1min":2.4159999999999999,"5min":7.234,"15min":7.234},"last":0.625}]},{"osd":5,"last update":"Wed Sep 11 08:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":1.2909999999999999,"15min":1.0700000000000001},"min":{"1min":0.33900000000000002,"5min":0.29799999999999999,"15min":0.28299999999999997},"max":{"1min":2.9100000000000001,"5min":8.3599999999999994,"15min":11.526999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.194,"15min":0.97499999999999998},"min":{"1min":0.314,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":4.2240000000000002,"5min":7.1689999999999996,"15min":7.1689999999999996},"last":0.55700000000000005}]},{"osd":6,"last update":"Wed Sep 11 08:50:28 2024","interfaces":[{"interface":"back","average":{"1min":3.0150000000000001,"5min":1.7450000000000001,"15min":1.2689999999999999},"min":{"1min":0.38600000000000001,"5min":0.34799999999999998,"15min":0.29999999999999999},"max":{"1min":33.106999999999999,"5min":33.106999999999999,"15min":33.106999999999999},"last":1.7},{"interface":"front","average":{"1min":3.0649999999999999,"5min":1.669,"15min":1.238},"min":{"1min":0.39400000000000002,"5min":0.314,"15min":0.311},"max":{"1min":33.076000000000001,"5min":33.076000000000001,"15min":33.076000000000001},"last":2.25}]},{"osd":7,"last update":"Wed Sep 11 08:49:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.369,"15min":1.135},"min":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.35299999999999998},"max":{"1min":3.73,"5min":9.1539999999999999,"15min":11.561},"last":0.52800000000000002},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2789999999999999,"15min":1.0249999999999999},"min":{"1min":0.441,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":5.96,"5min":6.6200000000000001,"15min":6.6200000000000001},"last":0.59999999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411500,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2463920,"kb_used_data":2149928,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":91281232,"statfs":{"total":95995035648,"available":93471981568,"internally_reserved":0,"allocated":2201526272,"data_stored":2201095954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":2.323,"15min":1.51},"min":{"1min":0.218,"5min":0.16500000000000001,"15min":0.159},"max":{"1min":2.25,"5min":34.713999999999999,"15min":34.713999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":2.2109999999999999,"15min":1.5960000000000001},"min":{"1min":0.219,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.2490000000000001,"5min":34.399000000000001,"15min":34.399000000000001},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":3.0539999999999998,"15min":1.5680000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.19},"max":{"1min":3.754,"5min":72.867000000000004,"15min":72.867000000000004},"last":1.556},{"interface":"front","average":{"1min":0.67000000000000004,"5min":3.04,"15min":1.6459999999999999},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.881,"5min":67.799000000000007,"15min":67.799000000000007},"last":0.35999999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":2.1949999999999998,"15min":1.4430000000000001},"min":{"1min":0.27900000000000003,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.8740000000000001,"5min":34.75,"15min":34.75},"last":1.788},{"interface":"front","average":{"1min":0.70999999999999996,"5min":2.0630000000000002,"15min":1.377},"min":{"1min":0.32100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.843,"5min":32.802,"15min":32.802},"last":1.647}]},{"osd":4,"last update":"Wed Sep 11 08:50:34 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.1950000000000001,"15min":0.97299999999999998},"min":{"1min":0.32000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":3.7730000000000001,"5min":8.0820000000000007,"15min":8.0820000000000007},"last":0.55500000000000005},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1020000000000001,"15min":0.93200000000000005},"min":{"1min":0.32400000000000001,"5min":0.29999999999999999,"15min":0.28399999999999997},"max":{"1min":5.0359999999999996,"5min":5.0359999999999996,"15min":6.9320000000000004},"last":0.45600000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:50:02 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.153,"15min":0.94599999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":3.4319999999999999,"5min":4.8949999999999996,"15min":6.9550000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.96599999999999997,"15min":0.83199999999999996},"min":{"1min":0.34999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.2400000000000002,"5min":3.226,"15min":5.9530000000000003},"last":0.51000000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:50:21 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.27,"15min":1.0169999999999999},"min":{"1min":0.38700000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":3.0819999999999999,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":1.054,"5min":1.077,"15min":0.92800000000000005},"min":{"1min":0.44900000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.7799999999999998,"5min":3.8290000000000002,"15min":6.9779999999999998},"last":0.47699999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.96099999999999997,"15min":0.85099999999999998},"min":{"1min":0.35899999999999999,"5min":0.32200000000000001,"15min":0.30399999999999999},"max":{"1min":1.361,"5min":3.2709999999999999,"15min":5.9909999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.3009999999999999,"15min":1.032},"min":{"1min":0.39500000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.3730000000000002,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":1.4990000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575025,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5701604,"kb_used_data":5395932,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":88043548,"statfs":{"total":95995035648,"available":90156593152,"internally_reserved":0,"allocated":5525434368,"data_stored":5524952467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":1.161,"15min":1.4099999999999999},"min":{"1min":0.186,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":3.9329999999999998,"5min":4.8319999999999999,"15min":117.01600000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":1.4119999999999999,"15min":1.706},"min":{"1min":0.23899999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":2.2810000000000001,"5min":8.1549999999999994,"15min":117.62},"last":0.53800000000000003}]},{"osd":2,"last update":"Wed Sep 11 08:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":1.179,"15min":1.099},"min":{"1min":0.221,"5min":0.21099999999999999,"15min":0.191},"max":{"1min":2.101,"5min":8.1329999999999991,"15min":20.628},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":1.3540000000000001,"15min":1.105},"min":{"1min":0.215,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.802,"5min":9.7200000000000006,"15min":20.148},"last":0.51200000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:50:17 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.4199999999999999,"15min":1.4219999999999999},"min":{"1min":0.23300000000000001,"5min":0.191,"15min":0.191},"max":{"1min":3.927,"5min":5.8490000000000002,"15min":116.98},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.2909999999999999,"15min":1.498},"min":{"1min":0.35599999999999998,"5min":0.246,"15min":0.22},"max":{"1min":2.1779999999999999,"5min":4.6719999999999997,"15min":117.56699999999999},"last":1.073}]},{"osd":4,"last update":"Wed Sep 11 08:50:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.169,"15min":0.93600000000000005},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.26200000000000001},"max":{"1min":3.8370000000000002,"5min":4.6580000000000004,"15min":11.19},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":0.877},"min":{"1min":0.308,"5min":0.30199999999999999,"15min":0.251},"max":{"1min":4.0730000000000004,"5min":5.056,"15min":7.7169999999999996},"last":1.23}]},{"osd":5,"last update":"Wed Sep 11 08:50:02 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.2689999999999999,"15min":1.4279999999999999},"min":{"1min":0.39100000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.5840000000000001,"5min":8.0999999999999996,"15min":103.337},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.2889999999999999,"15min":1.3939999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":3.891,"5min":5.9020000000000001,"15min":103.163},"last":1.004}]},{"osd":6,"last update":"Wed Sep 11 08:50:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.1419999999999999,"15min":0.89400000000000002},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.29399999999999998},"max":{"1min":4.1349999999999998,"5min":4.1349999999999998,"15min":8.9149999999999991},"last":2.141},{"interface":"front","average":{"1min":1.26,"5min":1.2649999999999999,"15min":1.052},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.31},"max":{"1min":3.177,"5min":4.7080000000000002,"15min":9.5199999999999996},"last":1.1619999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:50:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.246,"15min":0.998},"min":{"1min":0.47299999999999998,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":3.8620000000000001,"5min":4.593,"15min":11.315},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.3049999999999999,"15min":0.96399999999999997},"min":{"1min":0.41699999999999998,"5min":0.376,"15min":0.315},"max":{"1min":4.444,"5min":10.266,"15min":10.266},"last":3.9569999999999999}]}]},{"osd":0,"up_from":0,"seq":34359738549,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5523963904,"data_stored":5523963904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2217672704,"data_stored":2217672704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4977393664,"data_stored":4977393664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1725104128,"data_stored":1725104128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4946657280,"data_stored":4946657280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1769996288,"data_stored":1769996288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5445976064,"data_stored":5445976064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:50:42.443 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-11T08:50:42.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool set .mgr pg_num 11 2024-09-11T08:50:42.444 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:42 smithi092 ceph-mon[29647]: pgmap v518: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.3 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.17k op/s; 35910/406572 objects misplaced (8.832%); 42 MiB/s, 667 objects/s recovering 2024-09-11T08:50:42.445 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:42 smithi092 ceph-mon[41114]: pgmap v518: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.3 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.17k op/s; 35910/406572 objects misplaced (8.832%); 42 MiB/s, 667 objects/s recovering 2024-09-11T08:50:42.445 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:42.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:42.529 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:42.630 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:42 smithi145 ceph-mon[33384]: pgmap v518: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.3 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.17k op/s; 35910/406572 objects misplaced (8.832%); 42 MiB/s, 667 objects/s recovering 2024-09-11T08:50:42.665 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:42.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:42.666 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:42.733 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 88 16 140825 140809 99.9911 68.25 0.00913808 0.00999669 2024-09-11T08:50:42.767 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:42.804 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:42.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:42.805 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:42.906 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:42.941 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:42.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:42.942 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:43.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:43.092 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:43.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:43.093 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:43.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:50:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:50:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:50:43.194 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:43.232 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:43.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:43.233 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:43.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:43.367 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:43.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:43.369 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:43.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:43.505 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:43.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:43.506 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:43.607 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:43.643 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:43.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:43.644 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:43.727 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:43.736 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 89 16 142111 142095 99.7673 80.375 0.0129621 0.0100189 2024-09-11T08:50:43.744 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:43.778 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:43.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:43.779 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:43.879 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:43.923 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:43.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:43.924 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:44.025 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:44.062 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:44.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:44.063 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:44.164 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:44.204 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:44.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:44.205 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:44.305 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:44.339 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:44.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:44.340 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:44.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:44.474 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:44.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:44.475 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:44.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:44.610 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:44.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:44.612 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:44 smithi145 ceph-mon[33384]: pgmap v519: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.5 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s; 35315/418863 objects misplaced (8.431%); 54 MiB/s, 869 objects/s recovering 2024-09-11T08:50:44.695 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:44 smithi092 ceph-mon[29647]: pgmap v519: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.5 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s; 35315/418863 objects misplaced (8.431%); 54 MiB/s, 869 objects/s recovering 2024-09-11T08:50:44.695 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:44 smithi092 ceph-mon[41114]: pgmap v519: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.5 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s; 35315/418863 objects misplaced (8.431%); 54 MiB/s, 869 objects/s recovering 2024-09-11T08:50:44.713 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:44.736 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Total time run: 90.007 2024-09-11T08:50:44.736 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Total writes made: 143395 2024-09-11T08:50:44.736 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Write size: 65536 2024-09-11T08:50:44.736 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Object size: 65536 2024-09-11T08:50:44.736 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Bandwidth (MB/sec): 99.5721 2024-09-11T08:50:44.736 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Stddev Bandwidth: 35.2519 2024-09-11T08:50:44.736 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Max bandwidth (MB/sec): 224.188 2024-09-11T08:50:44.736 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Min bandwidth (MB/sec): 0 2024-09-11T08:50:44.737 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Average IOPS: 1593 2024-09-11T08:50:44.737 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Stddev IOPS: 564.041 2024-09-11T08:50:44.737 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Max IOPS: 3587 2024-09-11T08:50:44.737 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Min IOPS: 0 2024-09-11T08:50:44.737 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Average Latency(s): 0.0100393 2024-09-11T08:50:44.737 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Stddev Latency(s): 0.0197058 2024-09-11T08:50:44.737 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Max latency(s): 1.44527 2024-09-11T08:50:44.737 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Min latency(s): 0.00097685 2024-09-11T08:50:44.742 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:44.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:44.743 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:44.743 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Cleaning up (deleting benchmark objects) 2024-09-11T08:50:44.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:44.876 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:44.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:44.877 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:44.978 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:45.000 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:45.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:45.001 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:45.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:45.139 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:45.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:45.140 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:45.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:45.272 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:45.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:45.273 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:45.374 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:45.409 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:45.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:45.410 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:45.511 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:45.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:45.550 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:45.551 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:45.651 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:45.674 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:45.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:45.675 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:45.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:45.813 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:45.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:45.814 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:45.915 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:45.956 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:45.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:45.957 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:45.957 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:46.058 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:46.100 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:46.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:46.101 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:46.202 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:46.244 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:46.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:46.245 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:46.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:46.377 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:46.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:46.378 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:46.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:46 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:50:46.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:46 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:50:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:46 smithi145 ceph-mon[33384]: pgmap v520: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.5 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 592 B/s rd, 67 MiB/s wr, 1.07k op/s; 35315/418863 objects misplaced (8.431%); 37 MiB/s, 597 objects/s recovering 2024-09-11T08:50:46.479 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:46.514 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:46.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:46.516 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:46 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:50:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:46 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:50:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:46 smithi092 ceph-mon[29647]: pgmap v520: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.5 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 592 B/s rd, 67 MiB/s wr, 1.07k op/s; 35315/418863 objects misplaced (8.431%); 37 MiB/s, 597 objects/s recovering 2024-09-11T08:50:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:46 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:50:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:46 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:50:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:46 smithi092 ceph-mon[41114]: pgmap v520: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.5 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 592 B/s rd, 67 MiB/s wr, 1.07k op/s; 35315/418863 objects misplaced (8.431%); 37 MiB/s, 597 objects/s recovering 2024-09-11T08:50:46.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:46.646 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:46.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:46.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:46.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:46.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:46.782 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:46.782 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:46.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:46.917 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:46.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:46.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:47.019 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:47.056 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:47.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:47.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:47.158 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:47.162 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:50:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:50:47.196 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:47.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:47.197 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:47.209 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:50:47.209 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:50:47.298 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:47.339 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:47.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:47.340 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:47.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:47.476 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:47.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:47.478 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:47.578 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:47.614 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:47.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:47.615 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:47.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:47.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:47.752 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:47.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:47.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:47.886 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:47.886 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:47.987 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:48.012 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:48.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:48.013 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:48.114 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:48.145 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:48.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:48.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:48.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:48.279 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:48.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:48.280 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:48.381 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:48.419 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:48.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:48.420 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:48.521 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:48.563 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:48.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:48.564 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:48.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:48 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:48.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:48 smithi092 ceph-mon[41114]: pgmap v521: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.6 GiB data, 30 GiB used, 596 GiB / 626 GiB avail; 1.1 KiB/s rd, 65 MiB/s wr, 1.27k op/s; 30609/424092 objects misplaced (7.218%); 44 MiB/s, 710 objects/s recovering 2024-09-11T08:50:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:48 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:48 smithi092 ceph-mon[29647]: pgmap v521: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.6 GiB data, 30 GiB used, 596 GiB / 626 GiB avail; 1.1 KiB/s rd, 65 MiB/s wr, 1.27k op/s; 30609/424092 objects misplaced (7.218%); 44 MiB/s, 710 objects/s recovering 2024-09-11T08:50:48.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:48 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:48 smithi145 ceph-mon[33384]: pgmap v521: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.6 GiB data, 30 GiB used, 596 GiB / 626 GiB avail; 1.1 KiB/s rd, 65 MiB/s wr, 1.27k op/s; 30609/424092 objects misplaced (7.218%); 44 MiB/s, 710 objects/s recovering 2024-09-11T08:50:48.664 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:48.700 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:48.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:48.701 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:48.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:48.837 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:48.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:48.838 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:48.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:48.976 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:48.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:48.977 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:49.080 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:49.120 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:49.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:49.121 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:49.222 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:49.254 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:49.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:49.255 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:49.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T08:50:49.357 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:49.394 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:49.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:49.395 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:49.496 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:49.528 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:49.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:49.529 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:49.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:49.673 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:49.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:49.675 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:49.776 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:49.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:49.812 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:49.812 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:49.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:49.954 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:49.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:49.955 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:50.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:50.090 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:50.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:50.091 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:50.109 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:49 smithi092 ceph-mon[29647]: pgmap v522: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.6 GiB data, 30 GiB used, 596 GiB / 626 GiB avail; 1.1 KiB/s rd, 63 MiB/s wr, 1.22k op/s; 30609/424092 objects misplaced (7.218%); 43 MiB/s, 685 objects/s recovering 2024-09-11T08:50:50.109 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:49 smithi092 ceph-mon[41114]: pgmap v522: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.6 GiB data, 30 GiB used, 596 GiB / 626 GiB avail; 1.1 KiB/s rd, 63 MiB/s wr, 1.22k op/s; 30609/424092 objects misplaced (7.218%); 43 MiB/s, 685 objects/s recovering 2024-09-11T08:50:50.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:49 smithi145 ceph-mon[33384]: pgmap v522: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.6 GiB data, 30 GiB used, 596 GiB / 626 GiB avail; 1.1 KiB/s rd, 63 MiB/s wr, 1.22k op/s; 30609/424092 objects misplaced (7.218%); 43 MiB/s, 685 objects/s recovering 2024-09-11T08:50:50.192 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:50.228 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:50.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:50.229 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:50.330 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:50.370 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:50.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:50.371 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:50.397 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:50.473 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:50.513 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:50.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:50.514 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:50.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:50.645 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:50.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:50.646 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:50.746 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:50.780 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:50.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:50.781 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:50.882 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:50.920 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:50.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:50.921 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:50.932 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:50:51.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:51.056 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:51.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:51.057 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:51.076 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:50 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2107077534' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:50:51.076 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:50 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2107077534' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:50:51.158 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:51.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:50 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2107077534' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:50:51.195 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:51.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:51.197 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:51.298 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:51.330 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:51.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:51.332 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:51.432 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:51.471 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:51.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:51.473 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:51.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:51.607 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:51.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:51.608 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:51.708 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:51.744 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:51.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:51.745 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:51.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:50:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:50:51] "GET /metrics HTTP/1.1" 200 224496 "" "Prometheus/2.43.0" 2024-09-11T08:50:51.846 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:51.883 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:51.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:51.884 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:51.985 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:52.023 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:52.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:52.025 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:52.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:52.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:52.159 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:52.160 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:52.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:51 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:50:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:51 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2107077534' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:50:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:51 smithi145 ceph-mon[33384]: osdmap e242: 8 total, 8 up, 7 in 2024-09-11T08:50:52.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:51 smithi145 ceph-mon[33384]: pgmap v524: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.6 GiB data, 30 GiB used, 596 GiB / 626 GiB avail; 614 B/s rd, 50 MiB/s wr, 1.07k op/s; 30609/422748 objects misplaced (7.240%); 33 MiB/s, 530 objects/s recovering 2024-09-11T08:50:52.164 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:51 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:50:52.164 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:51 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2107077534' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:50:52.164 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:51 smithi092 ceph-mon[29647]: osdmap e242: 8 total, 8 up, 7 in 2024-09-11T08:50:52.165 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:51 smithi092 ceph-mon[29647]: pgmap v524: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.6 GiB data, 30 GiB used, 596 GiB / 626 GiB avail; 614 B/s rd, 50 MiB/s wr, 1.07k op/s; 30609/422748 objects misplaced (7.240%); 33 MiB/s, 530 objects/s recovering 2024-09-11T08:50:52.165 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:51 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:50:52.165 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:51 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2107077534' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:50:52.165 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:51 smithi092 ceph-mon[41114]: osdmap e242: 8 total, 8 up, 7 in 2024-09-11T08:50:52.165 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:51 smithi092 ceph-mon[41114]: pgmap v524: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.6 GiB data, 30 GiB used, 596 GiB / 626 GiB avail; 614 B/s rd, 50 MiB/s wr, 1.07k op/s; 30609/422748 objects misplaced (7.240%); 33 MiB/s, 530 objects/s recovering 2024-09-11T08:50:52.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:52.294 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:52.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:52.295 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:52.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:52.430 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:52.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:52.431 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:52.531 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:52.566 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:52.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:52.567 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:52.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:52.700 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:52.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:52.701 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:52.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:52.834 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:52.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:52.836 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:52.936 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:52.979 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:52.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:52.980 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:53.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:53.112 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:53.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:53.113 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:53.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:50:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:50:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:50:53.214 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:53.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:53.247 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:53.248 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:53.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:53.374 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:53.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:53.375 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:53.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:53.501 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:53.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:53.502 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:53.603 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:53.629 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:53.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:53.630 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:53.731 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:53.758 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:53.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:53.759 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:53.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:53.883 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:53.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:53.885 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:53.985 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:54.018 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:54.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:54.019 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:54.094 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:53 smithi092 ceph-mon[29647]: osdmap e243: 8 total, 8 up, 7 in 2024-09-11T08:50:54.094 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:53 smithi092 ceph-mon[29647]: pgmap v526: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.2 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 1.4 KiB/s rd, 31 MiB/s wr, 1.65k op/s; 24388/403047 objects misplaced (6.051%); 57 MiB/s, 905 objects/s recovering 2024-09-11T08:50:54.094 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:53 smithi092 ceph-mon[29647]: osdmap e244: 8 total, 8 up, 7 in 2024-09-11T08:50:54.094 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:53 smithi092 ceph-mon[41114]: osdmap e243: 8 total, 8 up, 7 in 2024-09-11T08:50:54.095 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:53 smithi092 ceph-mon[41114]: pgmap v526: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.2 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 1.4 KiB/s rd, 31 MiB/s wr, 1.65k op/s; 24388/403047 objects misplaced (6.051%); 57 MiB/s, 905 objects/s recovering 2024-09-11T08:50:54.095 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:53 smithi092 ceph-mon[41114]: osdmap e244: 8 total, 8 up, 7 in 2024-09-11T08:50:54.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:54.139 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:50:54.139 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:50:54.153 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:54.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:54.154 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:54.255 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:54.286 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:54.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:54.287 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:54.388 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:54.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:53 smithi145 ceph-mon[33384]: osdmap e243: 8 total, 8 up, 7 in 2024-09-11T08:50:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:53 smithi145 ceph-mon[33384]: pgmap v526: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.2 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 1.4 KiB/s rd, 31 MiB/s wr, 1.65k op/s; 24388/403047 objects misplaced (6.051%); 57 MiB/s, 905 objects/s recovering 2024-09-11T08:50:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:53 smithi145 ceph-mon[33384]: osdmap e244: 8 total, 8 up, 7 in 2024-09-11T08:50:54.417 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:54.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:54.418 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:54.519 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:54.546 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:54.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:54.547 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:54.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:50:54.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:54.676 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:54.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:54.677 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:54.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:54.804 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:54.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:54.805 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:54.872 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-11T08:50:54.906 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:54.945 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:54.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:54.945 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:55.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:55.089 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:55.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:55.091 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:55.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:55.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:55.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:55.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:55.362 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:55.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:55.363 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:55.464 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:55.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:55.495 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:55.495 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:55.596 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:55.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:55.634 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:55.634 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:55.735 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:55.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:55.776 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:55.776 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:55.872 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:55.877 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:55.941 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:55.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:55.942 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:56.008 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:55 smithi092 ceph-mon[41114]: osdmap e245: 8 total, 8 up, 7 in 2024-09-11T08:50:56.008 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:55 smithi092 ceph-mon[41114]: pgmap v529: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.2 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.62k op/s; 24388/403047 objects misplaced (6.051%); 51 MiB/s, 821 objects/s recovering 2024-09-11T08:50:56.008 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:55 smithi092 ceph-mon[29647]: osdmap e245: 8 total, 8 up, 7 in 2024-09-11T08:50:56.008 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:55 smithi092 ceph-mon[29647]: pgmap v529: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.2 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.62k op/s; 24388/403047 objects misplaced (6.051%); 51 MiB/s, 821 objects/s recovering 2024-09-11T08:50:56.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:56.080 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:56.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:56.081 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:56.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:55 smithi145 ceph-mon[33384]: osdmap e245: 8 total, 8 up, 7 in 2024-09-11T08:50:56.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:55 smithi145 ceph-mon[33384]: pgmap v529: 158 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 8.2 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.62k op/s; 24388/403047 objects misplaced (6.051%); 51 MiB/s, 821 objects/s recovering 2024-09-11T08:50:56.182 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:56.217 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:56.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:56.219 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:56.319 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:56.358 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:56.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:56.359 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:50:56.403 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:50:56.404 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:50:56.404 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:50:56.404 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:50:56.404 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:50:56.404 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:50:56.404 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:50:56.411 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:50:56.412 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:50:56.412 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:50:56.449 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:50:56.449 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:50:56.460 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:56.497 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:56.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:56.498 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:56.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:56.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:56.635 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:56.635 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:56.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:56.772 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:56.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:56.773 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:56.874 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:56.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:56.911 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:56.911 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:57.012 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:57.050 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:57.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:57.051 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:57.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:57.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:56 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3491666167' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:50:57.162 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:50:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:50:57.185 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:57.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:57.186 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:57.286 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:57.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:56 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3491666167' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:50:57.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:56 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3491666167' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:50:57.327 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:57.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:57.328 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:57.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:57.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:57.462 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:57.462 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:57.563 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:57.599 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:57.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:57.600 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:57.701 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:57.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:57.738 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:57.738 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:57.839 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:57.875 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:57.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:57.877 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:57.946 INFO:teuthology.orchestra.run.smithi092.stderr:set pool 1 pg_num to 11 2024-09-11T08:50:57.977 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:58.016 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:58.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:58.017 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:58.117 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:58.153 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:58.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:58.154 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:58.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:50:58.283 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:58.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:58.284 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:50:58.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:57 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:58.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:57 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/343747083' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-11T08:50:58.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:57 smithi092 ceph-mon[29647]: pgmap v530: 158 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 152 active+clean; 7.8 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2.04k op/s; 17783/386025 objects misplaced (4.607%); 76 MiB/s, 1.22k objects/s recovering 2024-09-11T08:50:58.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:57 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:58.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:57 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/343747083' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-11T08:50:58.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:57 smithi092 ceph-mon[41114]: pgmap v530: 158 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 152 active+clean; 7.8 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2.04k op/s; 17783/386025 objects misplaced (4.607%); 76 MiB/s, 1.22k objects/s recovering 2024-09-11T08:50:58.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:58.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:57 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:50:58.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:57 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/343747083' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-11T08:50:58.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:57 smithi145 ceph-mon[33384]: pgmap v530: 158 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 152 active+clean; 7.8 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2.04k op/s; 17783/386025 objects misplaced (4.607%); 76 MiB/s, 1.22k objects/s recovering 2024-09-11T08:50:58.419 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:58.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:58.420 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:58.521 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:58.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:58.558 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:58.558 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:58.659 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:50:58.690 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:58.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:58.691 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:50:58.792 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:58.832 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:58.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:58.834 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:58.934 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:58.968 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:58.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:58.970 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:59.070 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:59.102 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:59.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:59.103 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:59.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:50:59.225 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:59.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:59.227 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:50:59.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:58 smithi092 ceph-mon[29647]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:50:59.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:58 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/343747083' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-11T08:50:59.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:58 smithi092 ceph-mon[29647]: osdmap e246: 8 total, 8 up, 7 in 2024-09-11T08:50:59.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:58 smithi092 ceph-mon[29647]: osdmap e247: 8 total, 8 up, 7 in 2024-09-11T08:50:59.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:58 smithi092 ceph-mon[41114]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:50:59.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:58 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/343747083' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-11T08:50:59.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:58 smithi092 ceph-mon[41114]: osdmap e246: 8 total, 8 up, 7 in 2024-09-11T08:50:59.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:58 smithi092 ceph-mon[41114]: osdmap e247: 8 total, 8 up, 7 in 2024-09-11T08:50:59.327 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:50:59.355 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:59.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:59.357 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:50:59.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:58 smithi145 ceph-mon[33384]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:50:59.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:58 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/343747083' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-11T08:50:59.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:58 smithi145 ceph-mon[33384]: osdmap e246: 8 total, 8 up, 7 in 2024-09-11T08:50:59.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:58 smithi145 ceph-mon[33384]: osdmap e247: 8 total, 8 up, 7 in 2024-09-11T08:50:59.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:50:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:50:59.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:59.485 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:50:59.585 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:50:59.610 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:59.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:59.611 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:50:59.712 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:50:59.737 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:59.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:59.738 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:50:59.839 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:59.862 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:59.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:59.863 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:50:59.964 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:50:59.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:50:59.988 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:50:59.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:50:59.989 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:00.090 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:00.124 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:00.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:00.125 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:00.226 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:00.256 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:00.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:00.257 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:00.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:59 smithi092 ceph-mon[29647]: pgmap v533: 158 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 152 active+clean; 7.8 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 954 B/s rd, 0 B/s wr, 1.06k op/s; 17783/386025 objects misplaced (4.607%); 44 MiB/s, 705 objects/s recovering 2024-09-11T08:51:00.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:59 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-11T08:51:00.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:59 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-11T08:51:00.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:59 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-11T08:51:00.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:50:59 smithi092 ceph-mon[29647]: osdmap e248: 8 total, 8 up, 7 in 2024-09-11T08:51:00.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:59 smithi092 ceph-mon[41114]: pgmap v533: 158 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 152 active+clean; 7.8 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 954 B/s rd, 0 B/s wr, 1.06k op/s; 17783/386025 objects misplaced (4.607%); 44 MiB/s, 705 objects/s recovering 2024-09-11T08:51:00.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:59 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-11T08:51:00.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:59 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-11T08:51:00.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:59 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-11T08:51:00.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:50:59 smithi092 ceph-mon[41114]: osdmap e248: 8 total, 8 up, 7 in 2024-09-11T08:51:00.358 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:00.389 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:00.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:00.390 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:00.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:59 smithi145 ceph-mon[33384]: pgmap v533: 158 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 152 active+clean; 7.8 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 954 B/s rd, 0 B/s wr, 1.06k op/s; 17783/386025 objects misplaced (4.607%); 44 MiB/s, 705 objects/s recovering 2024-09-11T08:51:00.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:59 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-11T08:51:00.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:59 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-11T08:51:00.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:59 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-11T08:51:00.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:50:59 smithi145 ceph-mon[33384]: osdmap e248: 8 total, 8 up, 7 in 2024-09-11T08:51:00.491 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:00.518 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:00.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:00.520 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:00.620 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:00.644 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:00.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:00.646 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:00.746 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:00.771 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:00.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:00.772 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:00.873 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:00.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:00.902 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:01.003 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:01.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:01.037 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:01.137 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:01.148 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:51:01.148 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:00 smithi092 ceph-mon[41114]: osdmap e249: 8 total, 8 up, 7 in 2024-09-11T08:51:01.148 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:51:01.149 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:00 smithi092 ceph-mon[29647]: osdmap e249: 8 total, 8 up, 7 in 2024-09-11T08:51:01.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:01.180 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:01.181 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:01.281 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:01.318 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:01.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:01.319 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:01.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:51:01.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:00 smithi145 ceph-mon[33384]: osdmap e249: 8 total, 8 up, 7 in 2024-09-11T08:51:01.420 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:01.457 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:01.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:01.458 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:01.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:01.595 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:01.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:01.597 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:01.697 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:01.738 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:01.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:01.739 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:01.787 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:51:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:51:01] "GET /metrics HTTP/1.1" 200 224509 "" "Prometheus/2.43.0" 2024-09-11T08:51:01.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:01.871 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:01.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:01.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:01.973 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:02.012 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:02.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:02.013 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:02.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:01 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:51:02.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:01 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:51:02.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:01 smithi092 ceph-mon[29647]: pgmap v536: 168 pgs: 10 unknown, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 152 active+clean; 7.8 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 0 B/s wr, 126 op/s; 17783/384504 objects misplaced (4.625%) 2024-09-11T08:51:02.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:01 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-11T08:51:02.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:01 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-11T08:51:02.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:01 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:51:02.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:01 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:51:02.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:01 smithi092 ceph-mon[41114]: pgmap v536: 168 pgs: 10 unknown, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 152 active+clean; 7.8 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 0 B/s wr, 126 op/s; 17783/384504 objects misplaced (4.625%) 2024-09-11T08:51:02.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:01 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-11T08:51:02.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:01 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-11T08:51:02.111 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:51:02.113 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:02.151 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:02.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:02.153 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:02.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:02.275 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:02.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:02.276 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:02.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:02.537 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:01 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:51:02.537 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:01 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:51:02.537 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:01 smithi145 ceph-mon[33384]: pgmap v536: 168 pgs: 10 unknown, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 152 active+clean; 7.8 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 0 B/s wr, 126 op/s; 17783/384504 objects misplaced (4.625%) 2024-09-11T08:51:02.537 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:01 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-11T08:51:02.538 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:01 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-11T08:51:02.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:02.569 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:02.570 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:02.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:02.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:02.702 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:02.803 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:02.833 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:02.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:02.834 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:02.934 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:02.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:02.956 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:02.957 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:03.057 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:03.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:03.093 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:03.093 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:03.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:51:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:51:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:51:03.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:02 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-11T08:51:03.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:02 smithi145 ceph-mon[33384]: osdmap e250: 8 total, 8 up, 7 in 2024-09-11T08:51:03.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:02 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:51:03.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:02 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:51:03.193 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:03.231 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:03.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:03.232 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:03.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:02 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-11T08:51:03.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:02 smithi092 ceph-mon[29647]: osdmap e250: 8 total, 8 up, 7 in 2024-09-11T08:51:03.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:02 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:51:03.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:02 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:51:03.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:02 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-11T08:51:03.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:02 smithi092 ceph-mon[41114]: osdmap e250: 8 total, 8 up, 7 in 2024-09-11T08:51:03.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:02 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:51:03.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:02 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:51:03.332 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:03.366 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:03.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:03.367 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:03.468 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:03.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:03.505 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:03.505 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:03.606 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:03.627 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:03.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:03.628 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:03.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:03.761 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:03.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:03.762 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:03.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:03.885 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:03.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:03.886 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:03.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:04.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:04.022 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:04.022 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:04.123 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:04.159 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:04.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:04.160 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:04.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:04.294 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:04.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:04.295 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:04.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:03 smithi092 ceph-mon[29647]: osdmap e251: 8 total, 8 up, 7 in 2024-09-11T08:51:04.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:03 smithi092 ceph-mon[29647]: pgmap v539: 168 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 164 active+clean; 7.5 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.36k op/s; 12381/369654 objects misplaced (3.349%); 56 MiB/s, 890 objects/s recovering 2024-09-11T08:51:04.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:03 smithi092 ceph-mon[41114]: osdmap e251: 8 total, 8 up, 7 in 2024-09-11T08:51:04.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:03 smithi092 ceph-mon[41114]: pgmap v539: 168 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 164 active+clean; 7.5 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.36k op/s; 12381/369654 objects misplaced (3.349%); 56 MiB/s, 890 objects/s recovering 2024-09-11T08:51:04.396 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:04.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:03 smithi145 ceph-mon[33384]: osdmap e251: 8 total, 8 up, 7 in 2024-09-11T08:51:04.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:03 smithi145 ceph-mon[33384]: pgmap v539: 168 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 164 active+clean; 7.5 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.36k op/s; 12381/369654 objects misplaced (3.349%); 56 MiB/s, 890 objects/s recovering 2024-09-11T08:51:04.431 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:04.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:04.432 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:04.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:04.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:04.565 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:04.565 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:04.666 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:04.708 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:04.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:04.709 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:04.810 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:04.841 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:04.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:04.842 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:04.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:04.987 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:04.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:04.988 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:05.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:05.120 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:05.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:05.121 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:05.221 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:05.256 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:05.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:05.258 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:05.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:04 smithi092 ceph-mon[29647]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:51:05.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:04 smithi092 ceph-mon[41114]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:51:05.358 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:05.396 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:05.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:05.397 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:05.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:04 smithi145 ceph-mon[33384]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:51:05.498 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:05.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:05.535 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:05.535 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:05.621 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:51:05.621 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:51:05.621 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-11T08:51:05.621 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T08:51:05.621 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T08:51:05.622 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 config set filestore_inject_stall 3 2024-09-11T08:51:05.636 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:05.712 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:05.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:05.713 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:05.814 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:05.839 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:05.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:05.840 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:05.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:05.969 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:05.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:05.970 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:06.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:06.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:06.105 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:06.105 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:06.108 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:51:06.206 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:06.246 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:05 smithi145 ceph-mon[33384]: pgmap v540: 168 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 164 active+clean; 7.5 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 957 B/s rd, 0 B/s wr, 1.02k op/s; 12381/369654 objects misplaced (3.349%); 42 MiB/s, 666 objects/s recovering 2024-09-11T08:51:06.247 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:05 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:51:06.249 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:06.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:06.250 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:06.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:05 smithi092 ceph-mon[29647]: pgmap v540: 168 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 164 active+clean; 7.5 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 957 B/s rd, 0 B/s wr, 1.02k op/s; 12381/369654 objects misplaced (3.349%); 42 MiB/s, 666 objects/s recovering 2024-09-11T08:51:06.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:05 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:51:06.310 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:05 smithi092 ceph-mon[41114]: pgmap v540: 168 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 164 active+clean; 7.5 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 957 B/s rd, 0 B/s wr, 1.02k op/s; 12381/369654 objects misplaced (3.349%); 42 MiB/s, 666 objects/s recovering 2024-09-11T08:51:06.310 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:05 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:51:06.351 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:06.384 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:06.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:06.385 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:06.485 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:06.535 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:06.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:06.536 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:06.637 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:06.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:06.680 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:06.681 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:06.781 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:06.818 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:06.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:06.819 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:06.912 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:51:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:51:06.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:06.940 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:06.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:06.941 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:07.042 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:07.077 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:07.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:07.078 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:07.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:07.209 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:07.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:07.210 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:07.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:07.346 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:07.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:07.347 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:07.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:07.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:07.484 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:07.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:07.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:07.630 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:07.630 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:07.731 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:07.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:07.767 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:07.767 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:07.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:07.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:07.907 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:08.008 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:08.042 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:08.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:08.044 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:08.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:08.180 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:08.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:08.181 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:08.282 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:08.313 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:08.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:08.314 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:08.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:08.449 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:08.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:08.451 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:08.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:08.581 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:08.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:08.582 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:08.619 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:08 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:08.619 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:08 smithi145 ceph-mon[33384]: pgmap v541: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 7.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.94k op/s; 10110/349158 objects misplaced (2.896%); 50 MiB/s, 795 objects/s recovering 2024-09-11T08:51:08.619 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:08 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/543352537' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:51:08.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:08.715 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:08.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:08.716 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:08.752 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:51:08.753 INFO:teuthology.orchestra.run.smithi145.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T08:51:08.753 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:51:08.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:08 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:08.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:08 smithi092 ceph-mon[29647]: pgmap v541: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 7.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.94k op/s; 10110/349158 objects misplaced (2.896%); 50 MiB/s, 795 objects/s recovering 2024-09-11T08:51:08.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:08 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/543352537' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:51:08.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:08 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:08.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:08 smithi092 ceph-mon[41114]: pgmap v541: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 7.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.94k op/s; 10110/349158 objects misplaced (2.896%); 50 MiB/s, 795 objects/s recovering 2024-09-11T08:51:08.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:08 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/543352537' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:51:08.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:08.849 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:08.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:08.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:08.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:08.984 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:08.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:08.985 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:09.086 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:09.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:09.119 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:09.119 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:09.220 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:09.254 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:09.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:09.255 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:09.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:09.391 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:09.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:09.392 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:09.493 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:09.535 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:09.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:09.536 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:09.637 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:09.642 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:09 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:51:09.642 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:09 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/543352537' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:51:09.643 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:09 smithi092 ceph-mon[29647]: osdmap e252: 8 total, 8 up, 7 in 2024-09-11T08:51:09.643 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:09 smithi092 ceph-mon[29647]: osdmap e253: 8 total, 8 up, 7 in 2024-09-11T08:51:09.643 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:09 smithi092 ceph-mon[29647]: 4.1b scrub starts 2024-09-11T08:51:09.643 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:09 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:51:09.643 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:09 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/543352537' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:51:09.643 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:09 smithi092 ceph-mon[41114]: osdmap e252: 8 total, 8 up, 7 in 2024-09-11T08:51:09.643 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:09 smithi092 ceph-mon[41114]: osdmap e253: 8 total, 8 up, 7 in 2024-09-11T08:51:09.643 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:09 smithi092 ceph-mon[41114]: 4.1b scrub starts 2024-09-11T08:51:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:09 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:51:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:09 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/543352537' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:51:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:09 smithi145 ceph-mon[33384]: osdmap e252: 8 total, 8 up, 7 in 2024-09-11T08:51:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:09 smithi145 ceph-mon[33384]: osdmap e253: 8 total, 8 up, 7 in 2024-09-11T08:51:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:09 smithi145 ceph-mon[33384]: 4.1b scrub starts 2024-09-11T08:51:09.672 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:09.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:09.673 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:09.774 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:09.810 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:09.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:09.811 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:09.912 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:09.935 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:09.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:09.936 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:10.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:10.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:10.066 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:10.066 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:10.167 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:10.203 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:10.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:10.204 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:10.305 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:10.340 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:10.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:10.341 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:10.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:10.475 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:10.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:10.477 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:10.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:10 smithi092 ceph-mon[29647]: pgmap v544: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 7.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 812 B/s rd, 0 B/s wr, 1.08k op/s; 10110/349158 objects misplaced (2.896%); 15 MiB/s, 235 objects/s recovering 2024-09-11T08:51:10.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:10 smithi092 ceph-mon[41114]: pgmap v544: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 7.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 812 B/s rd, 0 B/s wr, 1.08k op/s; 10110/349158 objects misplaced (2.896%); 15 MiB/s, 235 objects/s recovering 2024-09-11T08:51:10.578 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:10.605 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:10.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:10.606 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:10.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:10 smithi145 ceph-mon[33384]: pgmap v544: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 7.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 812 B/s rd, 0 B/s wr, 1.08k op/s; 10110/349158 objects misplaced (2.896%); 15 MiB/s, 235 objects/s recovering 2024-09-11T08:51:10.696 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:51:10.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:10.740 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:10.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:10.741 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:10.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:10.877 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:10.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:10.878 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:10.979 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:11.011 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:11.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:11.012 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:11.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:11.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:11.136 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:11.136 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:11.180 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:51:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:51:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:51:11.181 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:51:11.181 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:51:11.237 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:11.268 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:11.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:11.270 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:11.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:11.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:11.408 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:11.409 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:11.509 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:11.542 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:11.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:11.543 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:11.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:11.652 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:11 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T08:51:11.675 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:11.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:11.676 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:11.777 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:11.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:11 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T08:51:11.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:51:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:51:11] "GET /metrics HTTP/1.1" 200 224509 "" "Prometheus/2.43.0" 2024-09-11T08:51:11.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:11 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T08:51:11.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:11.821 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:11.821 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:11.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:11.960 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:11.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:11.961 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:12.062 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:12.108 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:12.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:12.109 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:12.209 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:12.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:12.252 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:12.252 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:12.353 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:12.389 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:12.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:12.390 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:12.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:12 smithi145 ceph-mon[33384]: 5.6 scrub starts 2024-09-11T08:51:12.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:12 smithi145 ceph-mon[33384]: pgmap v545: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 7.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 928 op/s; 10110/347379 objects misplaced (2.910%); 12 MiB/s, 185 objects/s recovering 2024-09-11T08:51:12.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:12 smithi145 ceph-mon[33384]: 4.c scrub starts 2024-09-11T08:51:12.490 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:12.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:12.528 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:12.528 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:12.629 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:12.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:12.665 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:12.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:12.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:12.802 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:12.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:12.803 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:12 smithi092 ceph-mon[29647]: 5.6 scrub starts 2024-09-11T08:51:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:12 smithi092 ceph-mon[29647]: pgmap v545: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 7.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 928 op/s; 10110/347379 objects misplaced (2.910%); 12 MiB/s, 185 objects/s recovering 2024-09-11T08:51:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:12 smithi092 ceph-mon[29647]: 4.c scrub starts 2024-09-11T08:51:12.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:12 smithi092 ceph-mon[41114]: 5.6 scrub starts 2024-09-11T08:51:12.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:12 smithi092 ceph-mon[41114]: pgmap v545: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 7.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 928 op/s; 10110/347379 objects misplaced (2.910%); 12 MiB/s, 185 objects/s recovering 2024-09-11T08:51:12.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:12 smithi092 ceph-mon[41114]: 4.c scrub starts 2024-09-11T08:51:12.903 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:12.943 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:12.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:12.944 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:13.045 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:51:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:51:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:51:13.045 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:13.084 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:13.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:13.085 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:13.185 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:13.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:13.212 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:13.212 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:13.313 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:13.349 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:13.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:13.350 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:13.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:13.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:13.488 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:13.488 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:13.589 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:13 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T08:51:13.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:13.590 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:13 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T08:51:13.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:13.621 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:13.621 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:13.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:13 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T08:51:13.721 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:13.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:13.770 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:13.770 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:13.871 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:13.911 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:13.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:13.912 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:14.013 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:14.050 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:14.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:14.051 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:14.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:14.184 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:14.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:14.185 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:14.285 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:14.325 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:14.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:14.326 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:14.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:14.460 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:14.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:14.461 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:14.562 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:14.594 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:14.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:14.595 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:14.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:14 smithi145 ceph-mon[33384]: 5.b scrub starts 2024-09-11T08:51:14.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:14 smithi145 ceph-mon[33384]: pgmap v546: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 6.7 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.65k op/s; 8167/330015 objects misplaced (2.475%); 23 MiB/s, 365 objects/s recovering 2024-09-11T08:51:14.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:14 smithi145 ceph-mon[33384]: 6.9 scrub starts 2024-09-11T08:51:14.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:14 smithi145 ceph-mon[33384]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:51:14.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:14 smithi145 ceph-mon[33384]: 4.1 scrub starts 2024-09-11T08:51:14.696 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:14.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:14.742 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:14.742 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:14.770 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:14 smithi092 ceph-mon[29647]: 5.b scrub starts 2024-09-11T08:51:14.770 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:14 smithi092 ceph-mon[29647]: pgmap v546: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 6.7 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.65k op/s; 8167/330015 objects misplaced (2.475%); 23 MiB/s, 365 objects/s recovering 2024-09-11T08:51:14.770 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:14 smithi092 ceph-mon[29647]: 6.9 scrub starts 2024-09-11T08:51:14.770 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:14 smithi092 ceph-mon[29647]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:51:14.770 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:14 smithi092 ceph-mon[29647]: 4.1 scrub starts 2024-09-11T08:51:14.771 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:14 smithi092 ceph-mon[41114]: 5.b scrub starts 2024-09-11T08:51:14.771 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:14 smithi092 ceph-mon[41114]: pgmap v546: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 6.7 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.65k op/s; 8167/330015 objects misplaced (2.475%); 23 MiB/s, 365 objects/s recovering 2024-09-11T08:51:14.771 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:14 smithi092 ceph-mon[41114]: 6.9 scrub starts 2024-09-11T08:51:14.771 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:14 smithi092 ceph-mon[41114]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:51:14.771 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:14 smithi092 ceph-mon[41114]: 4.1 scrub starts 2024-09-11T08:51:14.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-11T08:51:14.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:14.898 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:14.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:14.899 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:15.000 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:15.037 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:15.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:15.038 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:15.113 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:51:15.113 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:51:15.139 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:15.163 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:51:15.163 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:51:15.172 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:15.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:15.173 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:15.190 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:51:15.190 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:51:15.225 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:51:15.225 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:51:15.255 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:51:15.255 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:51:15.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:15.288 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:51:15.288 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:51:15.325 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:51:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:51:15.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:15.330 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:15.330 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:15.363 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:51:15.364 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:51:15.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:15.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:15.467 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:15.468 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:15.568 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:15.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:15.603 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:15.603 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:15.651 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:15 smithi092 ceph-mon[29647]: 2.1 scrub starts 2024-09-11T08:51:15.651 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:15 smithi092 ceph-mon[29647]: 3.12 scrub starts 2024-09-11T08:51:15.651 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:15 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2006429793' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:51:15.652 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:51:15.652 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:15 smithi092 ceph-mon[41114]: 2.1 scrub starts 2024-09-11T08:51:15.652 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:15 smithi092 ceph-mon[41114]: 3.12 scrub starts 2024-09-11T08:51:15.652 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:15 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2006429793' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:51:15.652 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:51:15.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:15 smithi145 ceph-mon[33384]: 2.1 scrub starts 2024-09-11T08:51:15.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:15 smithi145 ceph-mon[33384]: 3.12 scrub starts 2024-09-11T08:51:15.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:15 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2006429793' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:51:15.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:51:15.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:15.742 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:15.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:15.743 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:15.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:15.875 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:15.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:15.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:15.977 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:16.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:16.026 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:16.026 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:16.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:51:16.127 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:16.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:16.171 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:16.171 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:16.271 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:16.308 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:16.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:16.309 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:16.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:16.449 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:16.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:16.450 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:16.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:16.582 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:16.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:16.583 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:16.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:16 smithi145 ceph-mon[33384]: pgmap v547: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 6.7 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 797 op/s; 8167/330015 objects misplaced (2.475%); 11 MiB/s, 179 objects/s recovering 2024-09-11T08:51:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:16 smithi145 ceph-mon[33384]: 4.1e scrub starts 2024-09-11T08:51:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:16 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:16 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:16 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.17", "id": [7, 4]}]: dispatch 2024-09-11T08:51:16.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:16 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.17", "id": [7, 4]}]: dispatch 2024-09-11T08:51:16.683 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:16.694 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:51:16.694 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:51:16.694 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-11T08:51:16.695 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-11T08:51:16.716 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:16.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:16.717 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:16.725 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:16 smithi092 ceph-mon[29647]: pgmap v547: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 6.7 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 797 op/s; 8167/330015 objects misplaced (2.475%); 11 MiB/s, 179 objects/s recovering 2024-09-11T08:51:16.725 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:16 smithi092 ceph-mon[29647]: 4.1e scrub starts 2024-09-11T08:51:16.725 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:16 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:16.725 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:16 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:16.725 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:16 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.17", "id": [7, 4]}]: dispatch 2024-09-11T08:51:16.725 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:16 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.17", "id": [7, 4]}]: dispatch 2024-09-11T08:51:16.725 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:16 smithi092 ceph-mon[41114]: pgmap v547: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 6.7 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 797 op/s; 8167/330015 objects misplaced (2.475%); 11 MiB/s, 179 objects/s recovering 2024-09-11T08:51:16.726 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:16 smithi092 ceph-mon[41114]: 4.1e scrub starts 2024-09-11T08:51:16.726 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:16 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:16.726 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:16 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:16.726 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:16 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.17", "id": [7, 4]}]: dispatch 2024-09-11T08:51:16.726 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:16 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.17", "id": [7, 4]}]: dispatch 2024-09-11T08:51:16.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:16.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:16.853 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:16.954 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:16.983 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:16.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:16.984 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:17.084 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:17.110 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:17.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:17.111 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:17.161 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:51:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:51:17.213 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:17.239 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:17.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:17.240 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:17.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:17.362 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:17.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:17.363 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:17.464 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:17.486 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:17.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:17.487 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:17.587 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:17.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:17.611 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:17.611 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:17.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:17.734 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:17.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:17.735 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:17.836 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:17.858 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:17.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:17.859 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:17.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:17 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.17", "id": [7, 4]}]': finished 2024-09-11T08:51:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:17 smithi145 ceph-mon[33384]: osdmap e254: 8 total, 8 up, 7 in 2024-09-11T08:51:17.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:17.980 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:17.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:17.981 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:18.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:17 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.17", "id": [7, 4]}]': finished 2024-09-11T08:51:18.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:17 smithi092 ceph-mon[29647]: osdmap e254: 8 total, 8 up, 7 in 2024-09-11T08:51:18.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:17 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.17", "id": [7, 4]}]': finished 2024-09-11T08:51:18.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:17 smithi092 ceph-mon[41114]: osdmap e254: 8 total, 8 up, 7 in 2024-09-11T08:51:18.082 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:18.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:18.104 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:18.105 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:18.205 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:51:18.205 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:18.219 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:51:18.225 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:18.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:18.226 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:18.327 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:18.348 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:18.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:18.350 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:18.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:18.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:18.474 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:18.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:18.599 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:18.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:18.600 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:18.701 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:18.733 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:18.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:18.734 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:18.834 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:18.863 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:18.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:18.864 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:18.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:18 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:18 smithi145 ceph-mon[33384]: 4.a scrub starts 2024-09-11T08:51:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:18 smithi145 ceph-mon[33384]: pgmap v549: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 6.3 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s; 6315/309180 objects misplaced (2.042%); 21 MiB/s, 336 objects/s recovering 2024-09-11T08:51:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:18 smithi145 ceph-mon[33384]: 5.10 scrub starts 2024-09-11T08:51:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:18 smithi145 ceph-mon[33384]: 5.10 scrub ok 2024-09-11T08:51:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:18 smithi145 ceph-mon[33384]: osdmap e255: 8 total, 8 up, 7 in 2024-09-11T08:51:18.915 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:18 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:18.915 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:18 smithi092 ceph-mon[41114]: 4.a scrub starts 2024-09-11T08:51:18.915 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:18 smithi092 ceph-mon[41114]: pgmap v549: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 6.3 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s; 6315/309180 objects misplaced (2.042%); 21 MiB/s, 336 objects/s recovering 2024-09-11T08:51:18.915 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:18 smithi092 ceph-mon[41114]: 5.10 scrub starts 2024-09-11T08:51:18.915 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:18 smithi092 ceph-mon[41114]: 5.10 scrub ok 2024-09-11T08:51:18.915 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:18 smithi092 ceph-mon[41114]: osdmap e255: 8 total, 8 up, 7 in 2024-09-11T08:51:18.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:18 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:18.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:18 smithi092 ceph-mon[29647]: 4.a scrub starts 2024-09-11T08:51:18.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:18 smithi092 ceph-mon[29647]: pgmap v549: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 6.3 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s; 6315/309180 objects misplaced (2.042%); 21 MiB/s, 336 objects/s recovering 2024-09-11T08:51:18.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:18 smithi092 ceph-mon[29647]: 5.10 scrub starts 2024-09-11T08:51:18.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:18 smithi092 ceph-mon[29647]: 5.10 scrub ok 2024-09-11T08:51:18.916 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:18 smithi092 ceph-mon[29647]: osdmap e255: 8 total, 8 up, 7 in 2024-09-11T08:51:18.965 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:19.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:19.001 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:19.001 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:19.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:19.136 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:19.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:19.137 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:19.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:19.239 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:51:19.272 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:19.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:19.273 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:19.374 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:19.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:19.408 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:19.409 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:19.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:19.543 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:19.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:19.545 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:19.646 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:19.678 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:19.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:19.679 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:19.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:19.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:19.813 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:19.813 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:19.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:19 smithi145 ceph-mon[33384]: 3.1d scrub starts 2024-09-11T08:51:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:19 smithi145 ceph-mon[33384]: 3.1d scrub ok 2024-09-11T08:51:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:19 smithi145 ceph-mon[33384]: osdmap e256: 8 total, 8 up, 7 in 2024-09-11T08:51:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:19 smithi145 ceph-mon[33384]: 9.17 starting backfill to osd.4 from (0'0,0'0] MAX to 253'9458 2024-09-11T08:51:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:19 smithi145 ceph-mon[33384]: osdmap e257: 8 total, 8 up, 7 in 2024-09-11T08:51:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:19 smithi145 ceph-mon[33384]: 5.18 scrub starts 2024-09-11T08:51:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:19 smithi145 ceph-mon[33384]: 5.18 scrub ok 2024-09-11T08:51:19.914 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:19.935 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:19.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:19.936 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:20.037 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:20.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:19 smithi092 ceph-mon[41114]: 3.1d scrub starts 2024-09-11T08:51:20.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:19 smithi092 ceph-mon[41114]: 3.1d scrub ok 2024-09-11T08:51:20.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:19 smithi092 ceph-mon[41114]: osdmap e256: 8 total, 8 up, 7 in 2024-09-11T08:51:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:19 smithi092 ceph-mon[41114]: 9.17 starting backfill to osd.4 from (0'0,0'0] MAX to 253'9458 2024-09-11T08:51:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:19 smithi092 ceph-mon[41114]: osdmap e257: 8 total, 8 up, 7 in 2024-09-11T08:51:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:19 smithi092 ceph-mon[41114]: 5.18 scrub starts 2024-09-11T08:51:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:19 smithi092 ceph-mon[41114]: 5.18 scrub ok 2024-09-11T08:51:20.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:19 smithi092 ceph-mon[29647]: 3.1d scrub starts 2024-09-11T08:51:20.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:19 smithi092 ceph-mon[29647]: 3.1d scrub ok 2024-09-11T08:51:20.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:19 smithi092 ceph-mon[29647]: osdmap e256: 8 total, 8 up, 7 in 2024-09-11T08:51:20.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:19 smithi092 ceph-mon[29647]: 9.17 starting backfill to osd.4 from (0'0,0'0] MAX to 253'9458 2024-09-11T08:51:20.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:19 smithi092 ceph-mon[29647]: osdmap e257: 8 total, 8 up, 7 in 2024-09-11T08:51:20.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:19 smithi092 ceph-mon[29647]: 5.18 scrub starts 2024-09-11T08:51:20.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:19 smithi092 ceph-mon[29647]: 5.18 scrub ok 2024-09-11T08:51:20.074 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:20.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:20.075 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:20.176 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:20.216 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:20.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:20.217 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:20.318 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:20.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:20.360 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:20.360 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:20.461 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:20.493 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:20.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:20.494 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:20.595 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:20.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:20.627 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:20.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:20.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:20.769 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:20.769 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:20.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:20 smithi092 ceph-mon[29647]: pgmap v553: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 6.3 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.74k op/s; 6315/309180 objects misplaced (2.042%); 21 MiB/s, 343 objects/s recovering 2024-09-11T08:51:20.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:20 smithi092 ceph-mon[29647]: 2.2 scrub starts 2024-09-11T08:51:20.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:20 smithi092 ceph-mon[29647]: 2.2 scrub ok 2024-09-11T08:51:20.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:20 smithi092 ceph-mon[29647]: 3.11 scrub starts 2024-09-11T08:51:20.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:20 smithi092 ceph-mon[29647]: 3.11 scrub ok 2024-09-11T08:51:20.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:20 smithi092 ceph-mon[41114]: pgmap v553: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 6.3 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.74k op/s; 6315/309180 objects misplaced (2.042%); 21 MiB/s, 343 objects/s recovering 2024-09-11T08:51:20.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:20 smithi092 ceph-mon[41114]: 2.2 scrub starts 2024-09-11T08:51:20.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:20 smithi092 ceph-mon[41114]: 2.2 scrub ok 2024-09-11T08:51:20.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:20 smithi092 ceph-mon[41114]: 3.11 scrub starts 2024-09-11T08:51:20.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:20 smithi092 ceph-mon[41114]: 3.11 scrub ok 2024-09-11T08:51:20.871 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:20.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:20.906 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:20.906 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:20.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:20 smithi145 ceph-mon[33384]: pgmap v553: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 6.3 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.74k op/s; 6315/309180 objects misplaced (2.042%); 21 MiB/s, 343 objects/s recovering 2024-09-11T08:51:20.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:20 smithi145 ceph-mon[33384]: 2.2 scrub starts 2024-09-11T08:51:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:20 smithi145 ceph-mon[33384]: 2.2 scrub ok 2024-09-11T08:51:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:20 smithi145 ceph-mon[33384]: 3.11 scrub starts 2024-09-11T08:51:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:20 smithi145 ceph-mon[33384]: 3.11 scrub ok 2024-09-11T08:51:21.007 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:21.042 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:21.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:21.043 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:21.144 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:21.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:21.185 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:21.185 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:21.286 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:21.323 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:21.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:21.325 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:21.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:21.462 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:51:21.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:21.491 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:21.491 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:21.592 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:21.624 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:21.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:21.625 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:21.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:21.757 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:21.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:21.758 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:21.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:21 smithi092 ceph-mon[29647]: 3.10 scrub starts 2024-09-11T08:51:21.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:21 smithi092 ceph-mon[29647]: 3.10 scrub ok 2024-09-11T08:51:21.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:21 smithi092 ceph-mon[29647]: 6.0 scrub starts 2024-09-11T08:51:21.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:21 smithi092 ceph-mon[29647]: 6.0 scrub ok 2024-09-11T08:51:21.816 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:51:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:51:21] "GET /metrics HTTP/1.1" 200 224539 "" "Prometheus/2.43.0" 2024-09-11T08:51:21.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:21 smithi092 ceph-mon[41114]: 3.10 scrub starts 2024-09-11T08:51:21.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:21 smithi092 ceph-mon[41114]: 3.10 scrub ok 2024-09-11T08:51:21.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:21 smithi092 ceph-mon[41114]: 6.0 scrub starts 2024-09-11T08:51:21.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:21 smithi092 ceph-mon[41114]: 6.0 scrub ok 2024-09-11T08:51:21.859 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:21.892 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:21.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:21.893 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:21.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:21 smithi145 ceph-mon[33384]: 3.10 scrub starts 2024-09-11T08:51:21.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:21 smithi145 ceph-mon[33384]: 3.10 scrub ok 2024-09-11T08:51:21.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:21 smithi145 ceph-mon[33384]: 6.0 scrub starts 2024-09-11T08:51:21.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:21 smithi145 ceph-mon[33384]: 6.0 scrub ok 2024-09-11T08:51:21.961 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:51:21.994 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:22.034 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:22.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:22.035 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:22.136 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:22.172 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:22.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:22.173 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:22.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:22.306 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:22.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:22.307 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:22.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:22.447 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:22.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:22.448 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:22.549 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:22.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:22.584 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:22.584 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:22.685 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:22.714 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:22.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:22.715 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:22.815 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:22.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:22 smithi092 ceph-mon[29647]: pgmap v554: 168 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 165 active+clean; 6.2 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 0 B/s wr, 679 op/s; 6166/307359 objects misplaced (2.006%); 21 MiB/s, 331 objects/s recovering 2024-09-11T08:51:22.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:22 smithi092 ceph-mon[29647]: 5.2 scrub starts 2024-09-11T08:51:22.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:22 smithi092 ceph-mon[29647]: 5.2 scrub ok 2024-09-11T08:51:22.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:22 smithi092 ceph-mon[29647]: 4.11 scrub starts 2024-09-11T08:51:22.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:22 smithi092 ceph-mon[29647]: 4.11 scrub ok 2024-09-11T08:51:22.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:22 smithi092 ceph-mon[29647]: 4.e scrub starts 2024-09-11T08:51:22.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:22 smithi092 ceph-mon[29647]: 4.e scrub ok 2024-09-11T08:51:22.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:22 smithi092 ceph-mon[41114]: pgmap v554: 168 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 165 active+clean; 6.2 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 0 B/s wr, 679 op/s; 6166/307359 objects misplaced (2.006%); 21 MiB/s, 331 objects/s recovering 2024-09-11T08:51:22.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:22 smithi092 ceph-mon[41114]: 5.2 scrub starts 2024-09-11T08:51:22.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:22 smithi092 ceph-mon[41114]: 5.2 scrub ok 2024-09-11T08:51:22.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:22 smithi092 ceph-mon[41114]: 4.11 scrub starts 2024-09-11T08:51:22.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:22 smithi092 ceph-mon[41114]: 4.11 scrub ok 2024-09-11T08:51:22.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:22 smithi092 ceph-mon[41114]: 4.e scrub starts 2024-09-11T08:51:22.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:22 smithi092 ceph-mon[41114]: 4.e scrub ok 2024-09-11T08:51:22.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:22.852 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:22.853 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:22.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:51:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:51:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:51:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:22 smithi145 ceph-mon[33384]: pgmap v554: 168 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 165 active+clean; 6.2 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 0 B/s wr, 679 op/s; 6166/307359 objects misplaced (2.006%); 21 MiB/s, 331 objects/s recovering 2024-09-11T08:51:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:22 smithi145 ceph-mon[33384]: 5.2 scrub starts 2024-09-11T08:51:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:22 smithi145 ceph-mon[33384]: 5.2 scrub ok 2024-09-11T08:51:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:22 smithi145 ceph-mon[33384]: 4.11 scrub starts 2024-09-11T08:51:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:22 smithi145 ceph-mon[33384]: 4.11 scrub ok 2024-09-11T08:51:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:22 smithi145 ceph-mon[33384]: 4.e scrub starts 2024-09-11T08:51:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:22 smithi145 ceph-mon[33384]: 4.e scrub ok 2024-09-11T08:51:22.954 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:22.994 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:22.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:22.995 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:23.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:23.117 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:23.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:23.118 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:23.219 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:23.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:23.262 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:23.262 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:23.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:23.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:23.396 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:23.497 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:23.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:23.529 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:23.529 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:23.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:51:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:23.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:23.672 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:23.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:23.805 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:23.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:23.806 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:23.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[29647]: 5.a scrub starts 2024-09-11T08:51:23.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[29647]: 5.a scrub ok 2024-09-11T08:51:23.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:51:23.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:51:23.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[29647]: 1.7 scrub starts 2024-09-11T08:51:23.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[29647]: 1.7 scrub ok 2024-09-11T08:51:23.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[29647]: osdmap e258: 8 total, 8 up, 7 in 2024-09-11T08:51:23.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[29647]: 5.c scrub starts 2024-09-11T08:51:23.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[29647]: 5.c scrub ok 2024-09-11T08:51:23.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3215502645' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:51:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[41114]: 5.a scrub starts 2024-09-11T08:51:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[41114]: 5.a scrub ok 2024-09-11T08:51:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:51:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:51:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[41114]: 1.7 scrub starts 2024-09-11T08:51:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[41114]: 1.7 scrub ok 2024-09-11T08:51:23.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[41114]: osdmap e258: 8 total, 8 up, 7 in 2024-09-11T08:51:23.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[41114]: 5.c scrub starts 2024-09-11T08:51:23.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[41114]: 5.c scrub ok 2024-09-11T08:51:23.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:23 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3215502645' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:51:23.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:23.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:23 smithi145 ceph-mon[33384]: 5.a scrub starts 2024-09-11T08:51:23.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:23 smithi145 ceph-mon[33384]: 5.a scrub ok 2024-09-11T08:51:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:23 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:51:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:23 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:51:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:23 smithi145 ceph-mon[33384]: 1.7 scrub starts 2024-09-11T08:51:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:23 smithi145 ceph-mon[33384]: 1.7 scrub ok 2024-09-11T08:51:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:23 smithi145 ceph-mon[33384]: osdmap e258: 8 total, 8 up, 7 in 2024-09-11T08:51:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:23 smithi145 ceph-mon[33384]: 5.c scrub starts 2024-09-11T08:51:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:23 smithi145 ceph-mon[33384]: 5.c scrub ok 2024-09-11T08:51:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:23 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3215502645' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:51:23.942 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:23.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:23.943 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:24.044 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:24.082 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:24.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:24.083 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:24.184 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:24.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:24.223 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:24.223 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:24.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:24.366 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:24.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:24.368 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:24.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:24.501 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:24.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:24.502 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:24.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:24.638 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:24.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:24.639 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:24.649 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T08:51:24.649 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[29647]: 4.1c scrub ok 2024-09-11T08:51:24.649 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[29647]: pgmap v556: 168 pgs: 3 active+remapped+backfill_toofull, 165 active+clean; 6.0 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 902 B/s rd, 0 B/s wr, 913 op/s; 8208/293643 objects misplaced (2.795%); 8.7 MiB/s, 138 objects/s recovering 2024-09-11T08:51:24.649 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[29647]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:51:24.649 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:51:24.649 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3215502645' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:51:24.649 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[29647]: osdmap e259: 8 total, 8 up, 7 in 2024-09-11T08:51:24.649 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[29647]: 3.e deep-scrub starts 2024-09-11T08:51:24.650 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[29647]: 3.e deep-scrub ok 2024-09-11T08:51:24.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T08:51:24.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[41114]: 4.1c scrub ok 2024-09-11T08:51:24.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[41114]: pgmap v556: 168 pgs: 3 active+remapped+backfill_toofull, 165 active+clean; 6.0 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 902 B/s rd, 0 B/s wr, 913 op/s; 8208/293643 objects misplaced (2.795%); 8.7 MiB/s, 138 objects/s recovering 2024-09-11T08:51:24.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[41114]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:51:24.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:51:24.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3215502645' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:51:24.650 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[41114]: osdmap e259: 8 total, 8 up, 7 in 2024-09-11T08:51:24.651 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[41114]: 3.e deep-scrub starts 2024-09-11T08:51:24.651 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:24 smithi092 ceph-mon[41114]: 3.e deep-scrub ok 2024-09-11T08:51:24.740 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:24.791 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:24.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:24.792 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:24.867 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:51:24.867 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:51:24.892 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:24.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:24 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T08:51:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:24 smithi145 ceph-mon[33384]: 4.1c scrub ok 2024-09-11T08:51:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:24 smithi145 ceph-mon[33384]: pgmap v556: 168 pgs: 3 active+remapped+backfill_toofull, 165 active+clean; 6.0 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 902 B/s rd, 0 B/s wr, 913 op/s; 8208/293643 objects misplaced (2.795%); 8.7 MiB/s, 138 objects/s recovering 2024-09-11T08:51:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:24 smithi145 ceph-mon[33384]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:51:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:24 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:51:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:24 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3215502645' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:51:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:24 smithi145 ceph-mon[33384]: osdmap e259: 8 total, 8 up, 7 in 2024-09-11T08:51:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:24 smithi145 ceph-mon[33384]: 3.e deep-scrub starts 2024-09-11T08:51:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:24 smithi145 ceph-mon[33384]: 3.e deep-scrub ok 2024-09-11T08:51:24.932 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:24.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:24.933 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:25.034 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:25.071 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:25.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:25.073 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:25.173 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:25.211 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:25.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:25.212 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:25.312 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:25.344 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:25.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:25.345 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:25.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:25.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:25.477 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:25.477 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:25.578 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:25.610 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:25.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:25.611 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:25.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:25.743 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:25.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:25.744 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:25.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:25.884 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:25.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:25.885 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:25.986 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:26.028 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:26.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:26.029 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:26.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T08:51:26.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[29647]: 5.d scrub ok 2024-09-11T08:51:26.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:51:26.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:51:26.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[29647]: osdmap e260: 8 total, 8 up, 7 in 2024-09-11T08:51:26.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[29647]: 4.d scrub starts 2024-09-11T08:51:26.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[29647]: 4.d scrub ok 2024-09-11T08:51:26.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[29647]: pgmap v559: 168 pgs: 3 active+remapped+backfill_toofull, 165 active+clean; 6.0 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 863 op/s; 8208/293643 objects misplaced (2.795%); 8.2 MiB/s, 131 objects/s recovering 2024-09-11T08:51:26.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[29647]: 4.10 deep-scrub starts 2024-09-11T08:51:26.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T08:51:26.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[41114]: 5.d scrub ok 2024-09-11T08:51:26.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:51:26.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:51:26.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[41114]: osdmap e260: 8 total, 8 up, 7 in 2024-09-11T08:51:26.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[41114]: 4.d scrub starts 2024-09-11T08:51:26.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[41114]: 4.d scrub ok 2024-09-11T08:51:26.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[41114]: pgmap v559: 168 pgs: 3 active+remapped+backfill_toofull, 165 active+clean; 6.0 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 863 op/s; 8208/293643 objects misplaced (2.795%); 8.2 MiB/s, 131 objects/s recovering 2024-09-11T08:51:26.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:25 smithi092 ceph-mon[41114]: 4.10 deep-scrub starts 2024-09-11T08:51:26.129 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:26.164 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:26.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:26.165 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:26.266 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:26.299 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:26.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:26.300 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:26.401 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:26.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:25 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T08:51:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:25 smithi145 ceph-mon[33384]: 5.d scrub ok 2024-09-11T08:51:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:25 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:51:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:25 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:51:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:25 smithi145 ceph-mon[33384]: osdmap e260: 8 total, 8 up, 7 in 2024-09-11T08:51:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:25 smithi145 ceph-mon[33384]: 4.d scrub starts 2024-09-11T08:51:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:25 smithi145 ceph-mon[33384]: 4.d scrub ok 2024-09-11T08:51:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:25 smithi145 ceph-mon[33384]: pgmap v559: 168 pgs: 3 active+remapped+backfill_toofull, 165 active+clean; 6.0 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 863 op/s; 8208/293643 objects misplaced (2.795%); 8.2 MiB/s, 131 objects/s recovering 2024-09-11T08:51:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:25 smithi145 ceph-mon[33384]: 4.10 deep-scrub starts 2024-09-11T08:51:26.437 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:26.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:26.438 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:26.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:26.572 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:26.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:26.573 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:26.673 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:26.709 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:26.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:26.710 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:26.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:26.843 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:26.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:26.844 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:26.945 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:26.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:26.985 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:26.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:27.086 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:27.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:51:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:51:27.123 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:27.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:27.124 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:27.217 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-11T08:51:27.225 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:27.260 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:27.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:27.262 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:27.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:27.399 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:27.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:27.400 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:27.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:27 smithi145 ceph-mon[33384]: 5.6 scrub starts 2024-09-11T08:51:27.444 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:27 smithi092 ceph-mon[29647]: 5.6 scrub starts 2024-09-11T08:51:27.444 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:27 smithi092 ceph-mon[41114]: 5.6 scrub starts 2024-09-11T08:51:27.501 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:27.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:27.542 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:27.542 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:27.643 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:27.673 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:27.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:27.674 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:27.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:27.807 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:27.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:27.808 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:27.909 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:27.942 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:27.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:27.943 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:28.044 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:28.075 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:28.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:28.076 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:28.177 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:28.213 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:28.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:28.215 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:28.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:28 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:28.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:28 smithi092 ceph-mon[29647]: 6.17 scrub starts 2024-09-11T08:51:28.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:28 smithi092 ceph-mon[29647]: pgmap v560: 168 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 165 active+clean; 5.5 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 0 B/s wr, 2.03k op/s; 7357/270783 objects misplaced (2.717%); 7.7 MiB/s, 123 objects/s recovering 2024-09-11T08:51:28.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:28 smithi092 ceph-mon[29647]: 3.7 scrub starts 2024-09-11T08:51:28.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:28 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:28.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:28 smithi092 ceph-mon[41114]: 6.17 scrub starts 2024-09-11T08:51:28.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:28 smithi092 ceph-mon[41114]: pgmap v560: 168 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 165 active+clean; 5.5 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 0 B/s wr, 2.03k op/s; 7357/270783 objects misplaced (2.717%); 7.7 MiB/s, 123 objects/s recovering 2024-09-11T08:51:28.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:28 smithi092 ceph-mon[41114]: 3.7 scrub starts 2024-09-11T08:51:28.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:28.345 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:28.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:28.346 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:28.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:28 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:28.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:28 smithi145 ceph-mon[33384]: 6.17 scrub starts 2024-09-11T08:51:28.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:28 smithi145 ceph-mon[33384]: pgmap v560: 168 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 165 active+clean; 5.5 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 0 B/s wr, 2.03k op/s; 7357/270783 objects misplaced (2.717%); 7.7 MiB/s, 123 objects/s recovering 2024-09-11T08:51:28.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:28 smithi145 ceph-mon[33384]: 3.7 scrub starts 2024-09-11T08:51:28.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:28.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:28.474 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:28.474 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:28.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:28.597 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:28.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:28.598 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:28.612 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:51:28.612 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:51:28.612 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:51:28.612 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:51:28.612 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:51:28.613 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:51:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:51:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:51:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:51:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:51:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:51:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:51:28.614 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:51:28.614 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:51:28.614 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:51:28.614 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:51:28.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:51:28.699 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:28.721 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:28.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:28.722 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:28.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:28.844 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:28.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:28.845 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:28.946 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:28.968 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:28.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:28.969 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:29.069 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:29.090 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:29.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:29.092 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:29.192 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:29.228 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:29.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:29.229 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:29.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:29.361 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:29.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:29.362 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:29 smithi145 ceph-mon[33384]: osdmap e261: 8 total, 8 up, 7 in 2024-09-11T08:51:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:29 smithi145 ceph-mon[33384]: 1.6 scrub starts 2024-09-11T08:51:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:29 smithi145 ceph-mon[33384]: 1.6 scrub ok 2024-09-11T08:51:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:29 smithi145 ceph-mon[33384]: 3.1e scrub starts 2024-09-11T08:51:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:29 smithi145 ceph-mon[33384]: 3.1e scrub ok 2024-09-11T08:51:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:29 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4294061170' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:51:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:29 smithi145 ceph-mon[33384]: 3.19 scrub starts 2024-09-11T08:51:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:29 smithi145 ceph-mon[33384]: 3.19 scrub ok 2024-09-11T08:51:29.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:29 smithi145 ceph-mon[33384]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:51:29.463 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:29.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:29.506 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:29.506 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:29.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[29647]: osdmap e261: 8 total, 8 up, 7 in 2024-09-11T08:51:29.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[29647]: 1.6 scrub starts 2024-09-11T08:51:29.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[29647]: 1.6 scrub ok 2024-09-11T08:51:29.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[29647]: 3.1e scrub starts 2024-09-11T08:51:29.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[29647]: 3.1e scrub ok 2024-09-11T08:51:29.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4294061170' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:51:29.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[29647]: 3.19 scrub starts 2024-09-11T08:51:29.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[29647]: 3.19 scrub ok 2024-09-11T08:51:29.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[29647]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:51:29.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[41114]: osdmap e261: 8 total, 8 up, 7 in 2024-09-11T08:51:29.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[41114]: 1.6 scrub starts 2024-09-11T08:51:29.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[41114]: 1.6 scrub ok 2024-09-11T08:51:29.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[41114]: 3.1e scrub starts 2024-09-11T08:51:29.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[41114]: 3.1e scrub ok 2024-09-11T08:51:29.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4294061170' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:51:29.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[41114]: 3.19 scrub starts 2024-09-11T08:51:29.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[41114]: 3.19 scrub ok 2024-09-11T08:51:29.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:29 smithi092 ceph-mon[41114]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:51:29.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:29.633 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:29.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:29.634 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:29.735 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:29.767 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:29.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:29.768 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:29.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:29.899 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:29.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:29.900 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:30.001 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:30.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:30.031 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:30.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:30.168 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:30.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:30.169 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:30.270 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:30.299 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:30.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:30.303 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:30.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[29647]: 6.8 scrub starts 2024-09-11T08:51:30.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[29647]: 6.8 scrub ok 2024-09-11T08:51:30.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[29647]: osdmap e262: 8 total, 8 up, 7 in 2024-09-11T08:51:30.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[29647]: 9.18 deep-scrub starts 2024-09-11T08:51:30.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[29647]: pgmap v563: 168 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 165 active+clean; 5.5 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 0 B/s wr, 1.35k op/s; 7357/270783 objects misplaced (2.717%); 1.5 MiB/s, 23 objects/s recovering 2024-09-11T08:51:30.366 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[29647]: 5.0 scrub starts 2024-09-11T08:51:30.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[29647]: 5.0 scrub ok 2024-09-11T08:51:30.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[29647]: 3.1f deep-scrub starts 2024-09-11T08:51:30.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[29647]: 3.1f deep-scrub ok 2024-09-11T08:51:30.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[29647]: 5.1d scrub starts 2024-09-11T08:51:30.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[29647]: 5.1d scrub ok 2024-09-11T08:51:30.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:30.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:51:30.367 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[41114]: 6.8 scrub starts 2024-09-11T08:51:30.367 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[41114]: 6.8 scrub ok 2024-09-11T08:51:30.367 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[41114]: osdmap e262: 8 total, 8 up, 7 in 2024-09-11T08:51:30.367 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[41114]: 9.18 deep-scrub starts 2024-09-11T08:51:30.367 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[41114]: pgmap v563: 168 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 165 active+clean; 5.5 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 0 B/s wr, 1.35k op/s; 7357/270783 objects misplaced (2.717%); 1.5 MiB/s, 23 objects/s recovering 2024-09-11T08:51:30.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[41114]: 5.0 scrub starts 2024-09-11T08:51:30.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[41114]: 5.0 scrub ok 2024-09-11T08:51:30.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[41114]: 3.1f deep-scrub starts 2024-09-11T08:51:30.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[41114]: 3.1f deep-scrub ok 2024-09-11T08:51:30.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[41114]: 5.1d scrub starts 2024-09-11T08:51:30.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[41114]: 5.1d scrub ok 2024-09-11T08:51:30.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:30.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:51:30.404 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:30.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:30 smithi145 ceph-mon[33384]: 6.8 scrub starts 2024-09-11T08:51:30.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:30 smithi145 ceph-mon[33384]: 6.8 scrub ok 2024-09-11T08:51:30.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:30 smithi145 ceph-mon[33384]: osdmap e262: 8 total, 8 up, 7 in 2024-09-11T08:51:30.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:30 smithi145 ceph-mon[33384]: 9.18 deep-scrub starts 2024-09-11T08:51:30.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:30 smithi145 ceph-mon[33384]: pgmap v563: 168 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 165 active+clean; 5.5 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 0 B/s wr, 1.35k op/s; 7357/270783 objects misplaced (2.717%); 1.5 MiB/s, 23 objects/s recovering 2024-09-11T08:51:30.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:30 smithi145 ceph-mon[33384]: 5.0 scrub starts 2024-09-11T08:51:30.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:30 smithi145 ceph-mon[33384]: 5.0 scrub ok 2024-09-11T08:51:30.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:30 smithi145 ceph-mon[33384]: 3.1f deep-scrub starts 2024-09-11T08:51:30.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:30 smithi145 ceph-mon[33384]: 3.1f deep-scrub ok 2024-09-11T08:51:30.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:30 smithi145 ceph-mon[33384]: 5.1d scrub starts 2024-09-11T08:51:30.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:30 smithi145 ceph-mon[33384]: 5.1d scrub ok 2024-09-11T08:51:30.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:30 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:30.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:51:30.443 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:30.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:30.444 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:30.545 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:30.582 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:30.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:30.583 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:30.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:30.708 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:30.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:30.709 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:30.810 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:30.829 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:51:30.830 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:51:30.830 INFO:teuthology.orchestra.run.smithi092.stdout: "epoch": 262, 2024-09-11T08:51:30.830 INFO:teuthology.orchestra.run.smithi092.stdout: "fsid": "211625d4-7018-11ef-bcea-c7b262605968", 2024-09-11T08:51:30.830 INFO:teuthology.orchestra.run.smithi092.stdout: "created": "2024-09-11T08:33:32.174444+0000", 2024-09-11T08:51:30.830 INFO:teuthology.orchestra.run.smithi092.stdout: "modified": "2024-09-11T08:51:29.114563+0000", 2024-09-11T08:51:30.830 INFO:teuthology.orchestra.run.smithi092.stdout: "last_up_change": "2024-09-11T08:38:23.097480+0000", 2024-09-11T08:51:30.830 INFO:teuthology.orchestra.run.smithi092.stdout: "last_in_change": "2024-09-11T08:49:08.874272+0000", 2024-09-11T08:51:30.830 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T08:51:30.830 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_num": 5799936, 2024-09-11T08:51:30.830 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_set": [ 2024-09-11T08:51:30.830 INFO:teuthology.orchestra.run.smithi092.stdout: "pglog_hardlimit", 2024-09-11T08:51:30.830 INFO:teuthology.orchestra.run.smithi092.stdout: "purged_snapdirs", 2024-09-11T08:51:30.830 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_deletes", 2024-09-11T08:51:30.831 INFO:teuthology.orchestra.run.smithi092.stdout: "sortbitwise" 2024-09-11T08:51:30.831 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:51:30.831 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_version": 18, 2024-09-11T08:51:30.831 INFO:teuthology.orchestra.run.smithi092.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T08:51:30.831 INFO:teuthology.orchestra.run.smithi092.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T08:51:30.831 INFO:teuthology.orchestra.run.smithi092.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T08:51:30.831 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_snapshot": "", 2024-09-11T08:51:30.831 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_max": 9, 2024-09-11T08:51:30.831 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd": 8, 2024-09-11T08:51:30.831 INFO:teuthology.orchestra.run.smithi092.stdout: "require_min_compat_client": "luminous", 2024-09-11T08:51:30.831 INFO:teuthology.orchestra.run.smithi092.stdout: "min_compat_client": "luminous", 2024-09-11T08:51:30.831 INFO:teuthology.orchestra.run.smithi092.stdout: "require_osd_release": "quincy", 2024-09-11T08:51:30.831 INFO:teuthology.orchestra.run.smithi092.stdout: "pools": [ 2024-09-11T08:51:30.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.832 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 1, 2024-09-11T08:51:30.832 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": ".mgr", 2024-09-11T08:51:30.832 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:36:29.606460+0000", 2024-09-11T08:51:30.832 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:51:30.832 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:51:30.832 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:51:30.832 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:51:30.832 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:51:30.832 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:51:30.832 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:51:30.832 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:51:30.832 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:51:30.833 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:51:30.833 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:51:30.833 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:51:30.833 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 11, 2024-09-11T08:51:30.833 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 11, 2024-09-11T08:51:30.833 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 11, 2024-09-11T08:51:30.833 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 11, 2024-09-11T08:51:30.833 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 11, 2024-09-11T08:51:30.833 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:51:30.833 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:51:30.833 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:51:30.833 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:51:30.833 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:51:30.834 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:51:30.834 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:51:30.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.834 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "250", 2024-09-11T08:51:30.834 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:51:30.834 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:51:30.834 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "248", 2024-09-11T08:51:30.834 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:51:30.834 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:51:30.834 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:51:30.834 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:51:30.834 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:51:30.834 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:51:30.835 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:51:30.835 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:51:30.835 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:51:30.835 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:51:30.835 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:51:30.835 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:51:30.835 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:51:30.835 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:51:30.835 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:51:30.835 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:51:30.835 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:51:30.835 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:51:30.836 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:51:30.836 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:51:30.836 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:51:30.836 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:51:30.836 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:51:30.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.836 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:51:30.836 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:51:30.836 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:51:30.836 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:51:30.836 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:51:30.836 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:51:30.836 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:51:30.837 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:51:30.837 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:51:30.837 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:51:30.837 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:51:30.837 INFO:teuthology.orchestra.run.smithi092.stdout: "options": { 2024-09-11T08:51:30.837 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_max": 32, 2024-09-11T08:51:30.837 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_min": 1 2024-09-11T08:51:30.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.837 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:51:30.837 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": {} 2024-09-11T08:51:30.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.838 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 2, 2024-09-11T08:51:30.838 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "datapool", 2024-09-11T08:51:30.838 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:37.674272+0000", 2024-09-11T08:51:30.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 8193, 2024-09-11T08:51:30.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T08:51:30.838 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:51:30.838 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:51:30.838 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:51:30.838 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:51:30.838 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:51:30.838 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:51:30.838 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:51:30.839 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:51:30.839 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:51:30.839 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:51:30.839 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 3, 2024-09-11T08:51:30.839 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 3, 2024-09-11T08:51:30.839 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 3, 2024-09-11T08:51:30.839 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 3, 2024-09-11T08:51:30.839 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 3, 2024-09-11T08:51:30.839 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:51:30.839 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:51:30.839 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:51:30.839 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:51:30.840 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:51:30.840 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:51:30.840 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:51:30.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.840 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "60", 2024-09-11T08:51:30.840 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:51:30.840 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:51:30.840 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:51:30.840 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:51:30.840 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:51:30.840 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 3, 2024-09-11T08:51:30.840 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 60, 2024-09-11T08:51:30.840 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:51:30.841 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:51:30.841 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:51:30.841 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:51:30.841 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:51:30.841 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:51:30.841 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:51:30.841 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:51:30.841 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:51:30.841 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:51:30.841 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:51:30.841 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:51:30.841 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:51:30.841 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:51:30.842 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:51:30.842 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:51:30.842 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:51:30.842 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:51:30.842 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:51:30.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.842 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:51:30.842 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:51:30.842 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:51:30.842 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:51:30.842 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:51:30.842 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:51:30.842 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:51:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:51:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:51:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:51:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:51:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:51:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:51:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd": {} 2024-09-11T08:51:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 3, 2024-09-11T08:51:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": ".rgw.root", 2024-09-11T08:51:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:37.912906+0000", 2024-09-11T08:51:30.844 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:51:30.844 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:51:30.844 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:51:30.844 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:51:30.844 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:51:30.844 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:51:30.844 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:51:30.844 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:51:30.844 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:51:30.844 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:51:30.844 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:51:30.844 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:51:30.844 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:51:30.845 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:51:30.845 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:51:30.845 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:51:30.845 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:51:30.845 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:51:30.845 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:51:30.845 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:51:30.845 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:51:30.845 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:51:30.845 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:51:30.845 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:51:30.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.845 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "57", 2024-09-11T08:51:30.846 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:51:30.846 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:51:30.846 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:51:30.846 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:51:30.846 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:51:30.846 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:51:30.846 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:51:30.846 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:51:30.846 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:51:30.846 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:51:30.846 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:51:30.846 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:51:30.846 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:51:30.846 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:51:30.847 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:51:30.847 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:51:30.847 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:51:30.847 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:51:30.847 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:51:30.847 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:51:30.847 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:51:30.847 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:51:30.847 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:51:30.847 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:51:30.847 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:51:30.847 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:51:30.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.847 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:51:30.848 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:51:30.848 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:51:30.848 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:51:30.848 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:51:30.848 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:51:30.848 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:51:30.848 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:51:30.848 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:51:30.848 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:51:30.848 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:51:30.848 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:51:30.848 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:51:30.849 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:51:30.849 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.849 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 4, 2024-09-11T08:51:30.849 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "default.rgw.log", 2024-09-11T08:51:30.849 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:39.234173+0000", 2024-09-11T08:51:30.849 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:51:30.849 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:51:30.849 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:51:30.849 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:51:30.849 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:51:30.849 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:51:30.850 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:51:30.850 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:51:30.850 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:51:30.850 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:51:30.850 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:51:30.850 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:51:30.850 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:51:30.850 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:51:30.850 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:51:30.850 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:51:30.850 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:51:30.850 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:51:30.850 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:51:30.850 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:51:30.851 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:51:30.851 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:51:30.851 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:51:30.851 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:51:30.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.851 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "59", 2024-09-11T08:51:30.851 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:51:30.851 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:51:30.851 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:51:30.851 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:51:30.851 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:51:30.851 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:51:30.851 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:51:30.852 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:51:30.852 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:51:30.852 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:51:30.852 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:51:30.852 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:51:30.852 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:51:30.852 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:51:30.852 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:51:30.852 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:51:30.852 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:51:30.852 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:51:30.853 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:51:30.853 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:51:30.853 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:51:30.853 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:51:30.853 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:51:30.853 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:51:30.853 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:51:30.853 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:51:30.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.853 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:51:30.853 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:51:30.853 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:51:30.853 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:51:30.854 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:51:30.854 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:51:30.854 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:51:30.854 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:51:30.854 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:51:30.854 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:51:30.854 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:51:30.854 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:51:30.854 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:51:30.854 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:51:30.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.855 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 5, 2024-09-11T08:51:30.855 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "default.rgw.control", 2024-09-11T08:51:30.855 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:41.216386+0000", 2024-09-11T08:51:30.855 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:51:30.855 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:51:30.855 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:51:30.855 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:51:30.855 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:51:30.855 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:51:30.855 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:51:30.855 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:51:30.855 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:51:30.855 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:51:30.855 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:51:30.856 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:51:30.856 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:51:30.856 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:51:30.856 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:51:30.856 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:51:30.856 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:51:30.856 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:51:30.856 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:51:30.856 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:51:30.856 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:51:30.856 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:51:30.856 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:51:30.856 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:51:30.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.857 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "61", 2024-09-11T08:51:30.857 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:51:30.857 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:51:30.857 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:51:30.857 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:51:30.857 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:51:30.857 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:51:30.857 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:51:30.857 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:51:30.857 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:51:30.857 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:51:30.858 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:51:30.858 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:51:30.858 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:51:30.858 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:51:30.858 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:51:30.858 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:51:30.858 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:51:30.858 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:51:30.858 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:51:30.858 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:51:30.858 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:51:30.858 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:51:30.858 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:51:30.858 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:51:30.858 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:51:30.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:51:30.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.859 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:51:30.859 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:51:30.859 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:51:30.859 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:51:30.859 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:51:30.859 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:51:30.859 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:51:30.859 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:51:30.859 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:51:30.859 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:51:30.859 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:51:30.859 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:51:30.860 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:51:30.860 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:51:30.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.860 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 6, 2024-09-11T08:51:30.860 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "default.rgw.meta", 2024-09-11T08:51:30.860 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:43.285497+0000", 2024-09-11T08:51:30.860 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:51:30.860 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:51:30.860 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:51:30.860 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:51:30.860 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:51:30.861 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:51:30.861 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:51:30.861 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:51:30.861 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:51:30.861 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:51:30.861 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:51:30.861 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:51:30.861 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:51:30.861 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:51:30.861 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:51:30.861 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:51:30.861 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:51:30.861 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:51:30.861 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:51:30.862 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:51:30.862 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:51:30.862 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:51:30.862 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:51:30.862 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:51:30.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.862 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "63", 2024-09-11T08:51:30.862 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:51:30.862 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:51:30.862 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:51:30.862 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:51:30.862 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:51:30.863 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:51:30.863 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:51:30.863 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:51:30.863 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:51:30.863 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:51:30.863 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:51:30.863 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:51:30.863 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:51:30.863 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:51:30.863 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:51:30.863 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:51:30.863 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:51:30.863 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:51:30.863 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:51:30.863 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:51:30.864 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:51:30.864 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:51:30.864 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:51:30.864 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:51:30.864 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:51:30.864 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:51:30.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.864 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:51:30.864 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:51:30.864 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:51:30.864 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:51:30.864 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:51:30.864 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:51:30.864 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:51:30.865 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:51:30.865 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:51:30.865 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:51:30.865 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:51:30.865 INFO:teuthology.orchestra.run.smithi092.stdout: "options": { 2024-09-11T08:51:30.865 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_bias": 4 2024-09-11T08:51:30.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.865 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:51:30.866 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:51:30.866 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.866 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 9, 2024-09-11T08:51:30.866 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "unique_pool_2", 2024-09-11T08:51:30.866 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:49:04.633515+0000", 2024-09-11T08:51:30.866 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:51:30.866 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:51:30.866 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:51:30.866 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:51:30.866 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:51:30.867 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:51:30.867 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:51:30.867 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:51:30.867 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:51:30.867 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:51:30.867 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:51:30.867 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:51:30.867 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 26, 2024-09-11T08:51:30.867 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 26, 2024-09-11T08:51:30.867 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 26, 2024-09-11T08:51:30.867 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 26, 2024-09-11T08:51:30.867 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 26, 2024-09-11T08:51:30.867 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:51:30.868 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:51:30.868 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:51:30.868 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:51:30.868 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:51:30.868 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:51:30.868 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:51:30.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.868 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "237", 2024-09-11T08:51:30.868 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:51:30.868 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:51:30.868 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "235", 2024-09-11T08:51:30.868 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:51:30.868 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:51:30.868 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:51:30.869 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:51:30.869 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:51:30.869 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:51:30.869 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:51:30.869 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:51:30.869 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:51:30.869 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:51:30.869 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:51:30.869 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:51:30.869 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:51:30.869 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:51:30.869 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:51:30.869 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:51:30.870 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:51:30.870 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:51:30.870 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:51:30.870 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:51:30.870 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:51:30.870 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:51:30.870 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:51:30.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.870 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:51:30.870 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:51:30.870 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:51:30.870 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:51:30.871 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:51:30.871 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:51:30.871 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:51:30.871 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:51:30.871 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:51:30.871 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:51:30.871 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:51:30.871 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:51:30.871 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:51:30.871 INFO:teuthology.orchestra.run.smithi092.stdout: "rados": {} 2024-09-11T08:51:30.871 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.871 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.871 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:51:30.872 INFO:teuthology.orchestra.run.smithi092.stdout: "osds": [ 2024-09-11T08:51:30.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.872 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-09-11T08:51:30.872 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "9d8a3811-e801-4b6a-9a63-83936283cb7d", 2024-09-11T08:51:30.872 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:51:30.872 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 0, 2024-09-11T08:51:30.872 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 0, 2024-09-11T08:51:30.872 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:51:30.872 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:51:30.872 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:51:30.872 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 8, 2024-09-11T08:51:30.872 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 206, 2024-09-11T08:51:30.873 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:51:30.873 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:51:30.873 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:51:30.873 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.873 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6802", 2024-09-11T08:51:30.873 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:51:30.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.873 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6803", 2024-09-11T08:51:30.873 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:51:30.873 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.874 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.874 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:51:30.874 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.874 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.874 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6804", 2024-09-11T08:51:30.874 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:51:30.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.874 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.874 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6805", 2024-09-11T08:51:30.874 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:51:30.874 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.875 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.875 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:51:30.875 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.875 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.875 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6808", 2024-09-11T08:51:30.875 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:51:30.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.875 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.875 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6809", 2024-09-11T08:51:30.875 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:51:30.876 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.876 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.876 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:51:30.876 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.876 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.876 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6806", 2024-09-11T08:51:30.876 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:51:30.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.876 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.876 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6807", 2024-09-11T08:51:30.877 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:51:30.877 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.877 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.877 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6803/2040613580", 2024-09-11T08:51:30.877 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6805/2040613580", 2024-09-11T08:51:30.877 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6809/2040613580", 2024-09-11T08:51:30.877 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6807/2040613580", 2024-09-11T08:51:30.877 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:51:30.877 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:51:30.877 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:51:30.877 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.878 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-09-11T08:51:30.878 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "e7a18e45-0137-45ef-8b85-e458cf3737ac", 2024-09-11T08:51:30.878 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:51:30.878 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:51:30.878 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:51:30.878 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:51:30.878 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:51:30.878 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:51:30.878 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 13, 2024-09-11T08:51:30.878 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 255, 2024-09-11T08:51:30.878 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:51:30.878 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:51:30.878 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:51:30.878 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.879 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.879 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6810", 2024-09-11T08:51:30.879 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:51:30.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.879 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.879 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6811", 2024-09-11T08:51:30.879 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:51:30.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.879 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.879 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:51:30.879 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6812", 2024-09-11T08:51:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:51:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6813", 2024-09-11T08:51:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:51:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:51:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6816", 2024-09-11T08:51:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:51:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6817", 2024-09-11T08:51:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:51:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:51:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6814", 2024-09-11T08:51:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:51:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6815", 2024-09-11T08:51:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:51:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6811/544779328", 2024-09-11T08:51:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6813/544779328", 2024-09-11T08:51:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6817/544779328", 2024-09-11T08:51:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6815/544779328", 2024-09-11T08:51:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:51:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:51:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:51:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-09-11T08:51:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "017282e2-8053-4fa7-affc-a05408fa481e", 2024-09-11T08:51:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:51:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:51:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:51:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:51:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:51:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:51:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 18, 2024-09-11T08:51:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 210, 2024-09-11T08:51:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:51:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:51:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:51:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:30.885 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:30.886 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6818", 2024-09-11T08:51:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:51:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6819", 2024-09-11T08:51:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:51:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:51:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6820", 2024-09-11T08:51:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:51:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6821", 2024-09-11T08:51:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:51:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:51:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6824", 2024-09-11T08:51:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:51:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6825", 2024-09-11T08:51:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:51:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:51:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6822", 2024-09-11T08:51:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:51:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6823", 2024-09-11T08:51:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:51:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6819/933736777", 2024-09-11T08:51:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6821/933736777", 2024-09-11T08:51:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6825/933736777", 2024-09-11T08:51:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6823/933736777", 2024-09-11T08:51:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:51:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:51:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:51:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-09-11T08:51:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "58e5a8a4-9e58-40ad-9b40-90cb5ff5414b", 2024-09-11T08:51:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:51:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:51:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:51:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:51:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:51:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:51:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 26, 2024-09-11T08:51:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 254, 2024-09-11T08:51:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:51:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:51:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:51:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6826", 2024-09-11T08:51:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:51:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6827", 2024-09-11T08:51:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:51:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:51:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6828", 2024-09-11T08:51:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:51:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6829", 2024-09-11T08:51:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:51:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:51:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6832", 2024-09-11T08:51:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:51:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6833", 2024-09-11T08:51:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:51:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:51:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6830", 2024-09-11T08:51:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:51:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6831", 2024-09-11T08:51:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:51:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6827/878892459", 2024-09-11T08:51:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6829/878892459", 2024-09-11T08:51:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6833/878892459", 2024-09-11T08:51:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6831/878892459", 2024-09-11T08:51:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:51:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:51:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:51:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-09-11T08:51:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "e144e102-963b-48b7-92a9-9391706164bf", 2024-09-11T08:51:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:51:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:51:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:51:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:51:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:51:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:51:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 31, 2024-09-11T08:51:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 254, 2024-09-11T08:51:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:51:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:51:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:51:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6800", 2024-09-11T08:51:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:51:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6801", 2024-09-11T08:51:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:51:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:51:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6802", 2024-09-11T08:51:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:51:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6803", 2024-09-11T08:51:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:51:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:51:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6806", 2024-09-11T08:51:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:51:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6807", 2024-09-11T08:51:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:51:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:51:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6804", 2024-09-11T08:51:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:51:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6805", 2024-09-11T08:51:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:51:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6801/2792751669", 2024-09-11T08:51:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6803/2792751669", 2024-09-11T08:51:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6807/2792751669", 2024-09-11T08:51:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6805/2792751669", 2024-09-11T08:51:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:51:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:51:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:51:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-09-11T08:51:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "811b0dc9-b7a7-439a-86c1-5dc9d0d54e16", 2024-09-11T08:51:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:51:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:51:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:51:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:51:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:51:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:51:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 38, 2024-09-11T08:51:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 256, 2024-09-11T08:51:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:51:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:51:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:51:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6808", 2024-09-11T08:51:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:51:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6809", 2024-09-11T08:51:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:51:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:51:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6810", 2024-09-11T08:51:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:51:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6811", 2024-09-11T08:51:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:51:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:51:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6814", 2024-09-11T08:51:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:51:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6815", 2024-09-11T08:51:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:51:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:51:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6812", 2024-09-11T08:51:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:51:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6813", 2024-09-11T08:51:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:51:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6809/3569637188", 2024-09-11T08:51:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6811/3569637188", 2024-09-11T08:51:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6815/3569637188", 2024-09-11T08:51:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6813/3569637188", 2024-09-11T08:51:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:51:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:51:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:51:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-09-11T08:51:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "bd162f52-097c-4c5d-9d37-1289779a196b", 2024-09-11T08:51:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:51:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:51:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:51:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:51:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:51:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:51:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 45, 2024-09-11T08:51:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 255, 2024-09-11T08:51:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:51:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:51:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:51:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6816", 2024-09-11T08:51:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:51:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6817", 2024-09-11T08:51:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:51:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:51:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6818", 2024-09-11T08:51:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:51:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6819", 2024-09-11T08:51:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:51:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:51:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6822", 2024-09-11T08:51:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:51:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6823", 2024-09-11T08:51:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:51:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:51:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6820", 2024-09-11T08:51:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:51:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6821", 2024-09-11T08:51:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:51:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6817/2615047110", 2024-09-11T08:51:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6819/2615047110", 2024-09-11T08:51:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6823/2615047110", 2024-09-11T08:51:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6821/2615047110", 2024-09-11T08:51:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:51:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:51:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:51:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-09-11T08:51:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "4e4021bf-78c7-4bc6-b222-3494f724ddec", 2024-09-11T08:51:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:51:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:51:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:51:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:51:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:51:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:51:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 52, 2024-09-11T08:51:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 261, 2024-09-11T08:51:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:51:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:51:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:51:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6824", 2024-09-11T08:51:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:51:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6825", 2024-09-11T08:51:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:51:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:51:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6826", 2024-09-11T08:51:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:51:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6827", 2024-09-11T08:51:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:51:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:51:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6830", 2024-09-11T08:51:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:51:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6831", 2024-09-11T08:51:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:51:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:51:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:51:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:51:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6828", 2024-09-11T08:51:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:51:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:51:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6829", 2024-09-11T08:51:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:51:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6825/3195898530", 2024-09-11T08:51:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6827/3195898530", 2024-09-11T08:51:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6831/3195898530", 2024-09-11T08:51:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6829/3195898530", 2024-09-11T08:51:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:51:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:51:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:51:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:51:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_xinfo": [ 2024-09-11T08:51:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-09-11T08:51:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:51:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:51:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:51:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:51:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 65536, 2024-09-11T08:51:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:50:20.581943+0000", 2024-09-11T08:51:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:51:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-09-11T08:51:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:51:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:51:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:51:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:51:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:51:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:50:35.783550+0000", 2024-09-11T08:51:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:51:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-09-11T08:51:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:51:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:51:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:51:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:51:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:51:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:50:24.412901+0000", 2024-09-11T08:51:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:51:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-09-11T08:51:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:51:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:51:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:51:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:51:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:51:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:51:00.803958+0000", 2024-09-11T08:51:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:51:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-09-11T08:51:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:51:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:51:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:51:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:51:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:51:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:51:24.319896+0000", 2024-09-11T08:51:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:51:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-09-11T08:51:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:51:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:51:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:51:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:51:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:51:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:51:22.152188+0000", 2024-09-11T08:51:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:51:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-09-11T08:51:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:51:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:51:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:51:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:51:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:51:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:51:02.195241+0000", 2024-09-11T08:51:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:51:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-09-11T08:51:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:51:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:51:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:51:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:51:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:51:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:50:21.411986+0000", 2024-09-11T08:51:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:51:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:51:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap": [], 2024-09-11T08:51:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap_items": [ 2024-09-11T08:51:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.e", 2024-09-11T08:51:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:51:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:51:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T08:51:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.12", 2024-09-11T08:51:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:51:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:51:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 5 2024-09-11T08:51:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.13", 2024-09-11T08:51:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:51:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:51:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 5 2024-09-11T08:51:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.17", 2024-09-11T08:51:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:51:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:51:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T08:51:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.1d", 2024-09-11T08:51:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:51:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:51:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T08:51:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "9.17", 2024-09-11T08:51:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:51:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 7, 2024-09-11T08:51:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 4 2024-09-11T08:51:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:51:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_temp": [ 2024-09-11T08:51:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "9.10", 2024-09-11T08:51:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "osds": [ 2024-09-11T08:51:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: 1, 2024-09-11T08:51:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: 2, 2024-09-11T08:51:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: 6 2024-09-11T08:51:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:51:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "9.17", 2024-09-11T08:51:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "osds": [ 2024-09-11T08:51:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: 1, 2024-09-11T08:51:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: 2, 2024-09-11T08:51:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: 7 2024-09-11T08:51:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:51:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:51:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_temp": [], 2024-09-11T08:51:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklist": { 2024-09-11T08:51:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6800/1861854063": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:51:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/4113910808": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:51:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/3501902574": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:51:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/1583695466": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:51:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/736380628": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:51:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6800/4060453592": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:51:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/2234574262": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:51:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6800/2480574848": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:51:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/1540079149": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:51:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/2680744738": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:51:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6801/4060453592": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:51:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6801/1861854063": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:51:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6801/2480574848": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:51:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/12697418": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:51:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/4280958069": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:51:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/2686720854": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:51:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/969221731": "2024-09-12T08:39:14.984442+0000" 2024-09-11T08:51:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "range_blocklist": {}, 2024-09-11T08:51:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profiles": { 2024-09-11T08:51:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "default": { 2024-09-11T08:51:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-failure-domain": "osd", 2024-09-11T08:51:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "k": "2", 2024-09-11T08:51:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "m": "1", 2024-09-11T08:51:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin": "jerasure", 2024-09-11T08:51:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "technique": "reed_sol_van" 2024-09-11T08:51:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:51:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps_queue": [], 2024-09-11T08:51:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "new_removed_snaps": [], 2024-09-11T08:51:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "new_purged_snaps": [], 2024-09-11T08:51:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_node_flags": {}, 2024-09-11T08:51:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "device_class_flags": {}, 2024-09-11T08:51:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode": { 2024-09-11T08:51:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_enabled": false, 2024-09-11T08:51:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_bucket_count": 0, 2024-09-11T08:51:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "degraded_stretch_mode": 0, 2024-09-11T08:51:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "recovering_stretch_mode": 0, 2024-09-11T08:51:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_bucket": 0 2024-09-11T08:51:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:51:30.937 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:51:30.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:31.017 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:31.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:31.019 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:31.119 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:31.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:31.158 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:31.159 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:31.260 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:31.296 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:31.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:31.297 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:31.398 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:31.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:31 smithi145 ceph-mon[33384]: 3.a scrub starts 2024-09-11T08:51:31.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:31 smithi145 ceph-mon[33384]: 3.a scrub ok 2024-09-11T08:51:31.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:31 smithi145 ceph-mon[33384]: 3.1c deep-scrub starts 2024-09-11T08:51:31.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:31 smithi145 ceph-mon[33384]: 3.1c deep-scrub ok 2024-09-11T08:51:31.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:31 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1402939454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T08:51:31.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:31.425 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:31.425 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:31.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:31 smithi092 ceph-mon[41114]: 3.a scrub starts 2024-09-11T08:51:31.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:31 smithi092 ceph-mon[41114]: 3.a scrub ok 2024-09-11T08:51:31.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:31 smithi092 ceph-mon[41114]: 3.1c deep-scrub starts 2024-09-11T08:51:31.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:31 smithi092 ceph-mon[41114]: 3.1c deep-scrub ok 2024-09-11T08:51:31.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:31 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1402939454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T08:51:31.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:31 smithi092 ceph-mon[29647]: 3.a scrub starts 2024-09-11T08:51:31.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:31 smithi092 ceph-mon[29647]: 3.a scrub ok 2024-09-11T08:51:31.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:31 smithi092 ceph-mon[29647]: 3.1c deep-scrub starts 2024-09-11T08:51:31.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:31 smithi092 ceph-mon[29647]: 3.1c deep-scrub ok 2024-09-11T08:51:31.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:31 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1402939454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T08:51:31.526 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:31.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:31.562 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:31.562 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:31.563 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:51:31.663 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:31.699 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:31.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:31.700 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:31.702 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:51:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:51:31] "GET /metrics HTTP/1.1" 200 224529 "" "Prometheus/2.43.0" 2024-09-11T08:51:31.800 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:31.832 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:31.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:31.833 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:31.934 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:31.966 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:31.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:31.967 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:32.038 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:51:32.042 INFO:tasks.thrashosds.thrasher:j is {'epoch': 262, 'fsid': '211625d4-7018-11ef-bcea-c7b262605968', 'created': '2024-09-11T08:33:32.174444+0000', 'modified': '2024-09-11T08:51:29.114563+0000', 'last_up_change': '2024-09-11T08:38:23.097480+0000', 'last_in_change': '2024-09-11T08:49:08.874272+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 9, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T08:36:29.606460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '250', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '248', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-11T08:38:37.674272+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-11T08:38:37.912906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-11T08:38:39.234173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-11T08:38:41.216386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-11T08:38:43.285497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 9, 'pool_name': 'unique_pool_2', 'create_time': '2024-09-11T08:49:04.633515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '237', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '235', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '9d8a3811-e801-4b6a-9a63-83936283cb7d', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 206, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 2040613580}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 2040613580}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 2040613580}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 2040613580}]}, 'public_addr': '172.21.15.92:6803/2040613580', 'cluster_addr': '172.21.15.92:6805/2040613580', 'heartbeat_back_addr': '172.21.15.92:6809/2040613580', 'heartbeat_front_addr': '172.21.15.92:6807/2040613580', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e7a18e45-0137-45ef-8b85-e458cf3737ac', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 255, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 544779328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 544779328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 544779328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 544779328}]}, 'public_addr': '172.21.15.92:6811/544779328', 'cluster_addr': '172.21.15.92:6813/544779328', 'heartbeat_back_addr': '172.21.15.92:6817/544779328', 'heartbeat_front_addr': '172.21.15.92:6815/544779328', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '017282e2-8053-4fa7-affc-a05408fa481e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 210, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 933736777}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 933736777}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 933736777}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 933736777}]}, 'public_addr': '172.21.15.92:6819/933736777', 'cluster_addr': '172.21.15.92:6821/933736777', 'heartbeat_back_addr': '172.21.15.92:6825/933736777', 'heartbeat_front_addr': '172.21.15.92:6823/933736777', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '58e5a8a4-9e58-40ad-9b40-90cb5ff5414b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 254, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 878892459}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 878892459}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 878892459}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 878892459}]}, 'public_addr': '172.21.15.92:6827/878892459', 'cluster_addr': '172.21.15.92:6829/878892459', 'heartbeat_back_addr': '172.21.15.92:6833/878892459', 'heartbeat_front_addr': '172.21.15.92:6831/878892459', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e144e102-963b-48b7-92a9-9391706164bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 254, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6800', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6801', 'nonce': 2792751669}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6802', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6803', 'nonce': 2792751669}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6806', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6807', 'nonce': 2792751669}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6804', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6805', 'nonce': 2792751669}]}, 'public_addr': '172.21.15.145:6801/2792751669', 'cluster_addr': '172.21.15.145:6803/2792751669', 'heartbeat_back_addr': '172.21.15.145:6807/2792751669', 'heartbeat_front_addr': '172.21.15.145:6805/2792751669', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '811b0dc9-b7a7-439a-86c1-5dc9d0d54e16', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 38, 'up_thru': 256, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6808', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6809', 'nonce': 3569637188}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6810', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6811', 'nonce': 3569637188}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6814', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6815', 'nonce': 3569637188}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6812', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6813', 'nonce': 3569637188}]}, 'public_addr': '172.21.15.145:6809/3569637188', 'cluster_addr': '172.21.15.145:6811/3569637188', 'heartbeat_back_addr': '172.21.15.145:6815/3569637188', 'heartbeat_front_addr': '172.21.15.145:6813/3569637188', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'bd162f52-097c-4c5d-9d37-1289779a196b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 255, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6816', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6817', 'nonce': 2615047110}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6818', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6819', 'nonce': 2615047110}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6822', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6823', 'nonce': 2615047110}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6820', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6821', 'nonce': 2615047110}]}, 'public_addr': '172.21.15.145:6817/2615047110', 'cluster_addr': '172.21.15.145:6819/2615047110', 'heartbeat_back_addr': '172.21.15.145:6823/2615047110', 'heartbeat_front_addr': '172.21.15.145:6821/2615047110', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '4e4021bf-78c7-4bc6-b222-3494f724ddec', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 52, 'up_thru': 261, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6824', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6825', 'nonce': 3195898530}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6826', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6827', 'nonce': 3195898530}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6830', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6831', 'nonce': 3195898530}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6828', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6829', 'nonce': 3195898530}]}, 'public_addr': '172.21.15.145:6825/3195898530', 'cluster_addr': '172.21.15.145:6827/3195898530', 'heartbeat_back_addr': '172.21.15.145:6831/3195898530', 'heartbeat_front_addr': '172.21.15.145:6829/3195898530', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-11T08:50:20.581943+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:50:35.783550+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:50:24.412901+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:51:00.803958+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:51:24.319896+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:51:22.152188+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:51:02.195241+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:50:21.411986+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '4.e', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.12', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.1d', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '9.17', 'mappings': [{'from': 7, 'to': 4}]}], 'pg_temp': [{'pgid': '9.10', 'osds': [1, 2, 6]}, {'pgid': '9.17', 'osds': [1, 2, 7]}], 'primary_temp': [], 'blocklist': {'172.21.15.92:6800/1861854063': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/4113910808': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:0/3501902574': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/1583695466': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:0/736380628': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:6800/4060453592': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:0/2234574262': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:6800/2480574848': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/1540079149': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/2680744738': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:6801/4060453592': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:6801/1861854063': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:6801/2480574848': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/12697418': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/4280958069': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/2686720854': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/969221731': '2024-09-12T08:39:14.984442+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T08:51:32.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:51:32.068 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:32.102 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:32.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:32.104 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:32.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:32.226 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:32.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:32.227 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:32.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[29647]: 4.13 scrub starts 2024-09-11T08:51:32.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[29647]: 4.13 scrub ok 2024-09-11T08:51:32.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T08:51:32.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[29647]: 6.f scrub ok 2024-09-11T08:51:32.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[29647]: pgmap v564: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.5 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 0 B/s wr, 1.31k op/s; 7357/268500 objects misplaced (2.740%); 1.3 MiB/s, 20 objects/s recovering 2024-09-11T08:51:32.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[29647]: 4.19 deep-scrub starts 2024-09-11T08:51:32.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[29647]: 4.19 deep-scrub ok 2024-09-11T08:51:32.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[29647]: 4.2 scrub starts 2024-09-11T08:51:32.292 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[29647]: 4.2 scrub ok 2024-09-11T08:51:32.293 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[41114]: 4.13 scrub starts 2024-09-11T08:51:32.293 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[41114]: 4.13 scrub ok 2024-09-11T08:51:32.293 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T08:51:32.293 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[41114]: 6.f scrub ok 2024-09-11T08:51:32.293 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[41114]: pgmap v564: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.5 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 0 B/s wr, 1.31k op/s; 7357/268500 objects misplaced (2.740%); 1.3 MiB/s, 20 objects/s recovering 2024-09-11T08:51:32.293 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[41114]: 4.19 deep-scrub starts 2024-09-11T08:51:32.293 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[41114]: 4.19 deep-scrub ok 2024-09-11T08:51:32.293 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[41114]: 4.2 scrub starts 2024-09-11T08:51:32.294 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:32 smithi092 ceph-mon[41114]: 4.2 scrub ok 2024-09-11T08:51:32.328 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:32.360 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:32.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:32.362 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:32.367 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:51:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:32 smithi145 ceph-mon[33384]: 4.13 scrub starts 2024-09-11T08:51:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:32 smithi145 ceph-mon[33384]: 4.13 scrub ok 2024-09-11T08:51:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:32 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T08:51:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:32 smithi145 ceph-mon[33384]: 6.f scrub ok 2024-09-11T08:51:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:32 smithi145 ceph-mon[33384]: pgmap v564: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.5 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 0 B/s wr, 1.31k op/s; 7357/268500 objects misplaced (2.740%); 1.3 MiB/s, 20 objects/s recovering 2024-09-11T08:51:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:32 smithi145 ceph-mon[33384]: 4.19 deep-scrub starts 2024-09-11T08:51:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:32 smithi145 ceph-mon[33384]: 4.19 deep-scrub ok 2024-09-11T08:51:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:32 smithi145 ceph-mon[33384]: 4.2 scrub starts 2024-09-11T08:51:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:32 smithi145 ceph-mon[33384]: 4.2 scrub ok 2024-09-11T08:51:32.463 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:32.496 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:32.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:32.497 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:32.598 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:32.630 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:32.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:32.631 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:32.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:32.765 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:32.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:32.766 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:32.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:32.892 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:32.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:32.893 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:32.994 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:33.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:33.030 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:33.030 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:33.131 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:33.434 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:51:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:51:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:51:33.434 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:33 smithi145 ceph-mon[33384]: 6.5 scrub starts 2024-09-11T08:51:33.435 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:33 smithi145 ceph-mon[33384]: 6.5 scrub ok 2024-09-11T08:51:33.435 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:33 smithi145 ceph-mon[33384]: 4.1f scrub starts 2024-09-11T08:51:33.435 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:33 smithi145 ceph-mon[33384]: 4.1f scrub ok 2024-09-11T08:51:33.435 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:33 smithi145 ceph-mon[33384]: 3.18 deep-scrub starts 2024-09-11T08:51:33.435 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:33 smithi145 ceph-mon[33384]: 3.18 deep-scrub ok 2024-09-11T08:51:33.435 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:33 smithi145 ceph-mon[33384]: 5.7 scrub starts 2024-09-11T08:51:33.435 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:33 smithi145 ceph-mon[33384]: 5.7 scrub ok 2024-09-11T08:51:33.435 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:33 smithi145 ceph-mon[33384]: 4.6 scrub starts 2024-09-11T08:51:33.435 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:33 smithi145 ceph-mon[33384]: 4.6 scrub ok 2024-09-11T08:51:33.509 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:33.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:33.510 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:33.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[29647]: 6.5 scrub starts 2024-09-11T08:51:33.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[29647]: 6.5 scrub ok 2024-09-11T08:51:33.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[29647]: 4.1f scrub starts 2024-09-11T08:51:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[29647]: 4.1f scrub ok 2024-09-11T08:51:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[29647]: 3.18 deep-scrub starts 2024-09-11T08:51:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[29647]: 3.18 deep-scrub ok 2024-09-11T08:51:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[29647]: 5.7 scrub starts 2024-09-11T08:51:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[29647]: 5.7 scrub ok 2024-09-11T08:51:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[29647]: 4.6 scrub starts 2024-09-11T08:51:33.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[29647]: 4.6 scrub ok 2024-09-11T08:51:33.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[41114]: 6.5 scrub starts 2024-09-11T08:51:33.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[41114]: 6.5 scrub ok 2024-09-11T08:51:33.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[41114]: 4.1f scrub starts 2024-09-11T08:51:33.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[41114]: 4.1f scrub ok 2024-09-11T08:51:33.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[41114]: 3.18 deep-scrub starts 2024-09-11T08:51:33.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[41114]: 3.18 deep-scrub ok 2024-09-11T08:51:33.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[41114]: 5.7 scrub starts 2024-09-11T08:51:33.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[41114]: 5.7 scrub ok 2024-09-11T08:51:33.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[41114]: 4.6 scrub starts 2024-09-11T08:51:33.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:33 smithi092 ceph-mon[41114]: 4.6 scrub ok 2024-09-11T08:51:33.611 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:33.643 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:33.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:33.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:33.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:33.773 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:33.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:33.774 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:33.876 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:33.911 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:33.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:33.912 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:34.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:34.039 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:34.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:34.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:34.141 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:34.175 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:34.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:34.176 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:34.234 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:51:34.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:34.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:34.307 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:34.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:34 smithi145 ceph-mon[33384]: 3.3 scrub starts 2024-09-11T08:51:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:34 smithi145 ceph-mon[33384]: 3.3 scrub ok 2024-09-11T08:51:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:34 smithi145 ceph-mon[33384]: 5.11 scrub starts 2024-09-11T08:51:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:34 smithi145 ceph-mon[33384]: 5.11 scrub ok 2024-09-11T08:51:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:34 smithi145 ceph-mon[33384]: pgmap v565: 168 pgs: 1 active+clean+scrubbing+deep, 2 active+remapped+backfill_toofull, 165 active+clean; 5.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 0 B/s wr, 1.69k op/s; 6641/253098 objects misplaced (2.624%); 2.1 MiB/s, 33 objects/s recovering 2024-09-11T08:51:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:34 smithi145 ceph-mon[33384]: osdmap e263: 8 total, 8 up, 7 in 2024-09-11T08:51:34.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:34.436 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:34.436 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:34.474 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:34 smithi092 ceph-mon[29647]: 3.3 scrub starts 2024-09-11T08:51:34.474 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:34 smithi092 ceph-mon[29647]: 3.3 scrub ok 2024-09-11T08:51:34.474 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:34 smithi092 ceph-mon[29647]: 5.11 scrub starts 2024-09-11T08:51:34.474 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:34 smithi092 ceph-mon[29647]: 5.11 scrub ok 2024-09-11T08:51:34.474 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:34 smithi092 ceph-mon[29647]: pgmap v565: 168 pgs: 1 active+clean+scrubbing+deep, 2 active+remapped+backfill_toofull, 165 active+clean; 5.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 0 B/s wr, 1.69k op/s; 6641/253098 objects misplaced (2.624%); 2.1 MiB/s, 33 objects/s recovering 2024-09-11T08:51:34.474 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:34 smithi092 ceph-mon[29647]: osdmap e263: 8 total, 8 up, 7 in 2024-09-11T08:51:34.475 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:34 smithi092 ceph-mon[41114]: 3.3 scrub starts 2024-09-11T08:51:34.475 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:34 smithi092 ceph-mon[41114]: 3.3 scrub ok 2024-09-11T08:51:34.475 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:34 smithi092 ceph-mon[41114]: 5.11 scrub starts 2024-09-11T08:51:34.475 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:34 smithi092 ceph-mon[41114]: 5.11 scrub ok 2024-09-11T08:51:34.475 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:34 smithi092 ceph-mon[41114]: pgmap v565: 168 pgs: 1 active+clean+scrubbing+deep, 2 active+remapped+backfill_toofull, 165 active+clean; 5.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 0 B/s wr, 1.69k op/s; 6641/253098 objects misplaced (2.624%); 2.1 MiB/s, 33 objects/s recovering 2024-09-11T08:51:34.475 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:34 smithi092 ceph-mon[41114]: osdmap e263: 8 total, 8 up, 7 in 2024-09-11T08:51:34.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:34.568 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:34.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:34.570 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:34.670 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:34.709 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:34.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:34.710 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:34.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:34.839 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:34.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:34.840 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:34.941 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:34.974 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:34.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:34.976 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:35.077 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:35.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:35.112 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:35.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:35.213 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:35.245 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:35.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:35.246 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:35.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:35.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:35.378 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:35.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:35.506 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:35.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:35.507 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:35.509 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:35 smithi092 ceph-mon[29647]: 4.1a scrub starts 2024-09-11T08:51:35.509 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:35 smithi092 ceph-mon[29647]: 4.1a scrub ok 2024-09-11T08:51:35.509 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:35 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:35.509 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:35 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:35.509 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:35 smithi092 ceph-mon[29647]: 5.b deep-scrub starts 2024-09-11T08:51:35.509 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:35 smithi092 ceph-mon[29647]: 5.b deep-scrub ok 2024-09-11T08:51:35.509 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:35 smithi092 ceph-mon[41114]: 4.1a scrub starts 2024-09-11T08:51:35.510 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:35 smithi092 ceph-mon[41114]: 4.1a scrub ok 2024-09-11T08:51:35.510 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:35 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:35.510 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:35 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:35.510 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:35 smithi092 ceph-mon[41114]: 5.b deep-scrub starts 2024-09-11T08:51:35.510 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:35 smithi092 ceph-mon[41114]: 5.b deep-scrub ok 2024-09-11T08:51:35.609 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:35.636 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:35.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:35.637 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:35.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:35 smithi145 ceph-mon[33384]: 4.1a scrub starts 2024-09-11T08:51:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:35 smithi145 ceph-mon[33384]: 4.1a scrub ok 2024-09-11T08:51:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:35 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:35 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:35 smithi145 ceph-mon[33384]: 5.b deep-scrub starts 2024-09-11T08:51:35.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:35 smithi145 ceph-mon[33384]: 5.b deep-scrub ok 2024-09-11T08:51:35.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:35.770 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:35.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:35.771 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:35.872 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:35.895 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:35.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:35.896 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:35.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:36.023 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:36.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:36.024 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:36.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:36.151 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:36.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:36.152 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:36.253 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:36.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:36.290 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:36.290 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:36.391 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:36.433 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:36.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:36.434 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:36.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:36.561 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:36.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:36.562 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[29647]: 3.13 scrub starts 2024-09-11T08:51:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[29647]: 3.13 scrub ok 2024-09-11T08:51:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[29647]: 4.a scrub starts 2024-09-11T08:51:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[29647]: 4.a scrub ok 2024-09-11T08:51:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[29647]: pgmap v567: 168 pgs: 1 active+clean+scrubbing+deep, 2 active+remapped+backfill_toofull, 165 active+clean; 5.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 0 B/s wr, 822 op/s; 6641/253098 objects misplaced (2.624%); 1.2 MiB/s, 19 objects/s recovering 2024-09-11T08:51:36.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[29647]: 6.1b scrub starts 2024-09-11T08:51:36.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[29647]: 6.1b scrub ok 2024-09-11T08:51:36.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[29647]: 9.a scrub starts 2024-09-11T08:51:36.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:51:36.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:51:36.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:36.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[29647]: 6.4 scrub starts 2024-09-11T08:51:36.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[29647]: 6.4 scrub ok 2024-09-11T08:51:36.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[41114]: 3.13 scrub starts 2024-09-11T08:51:36.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[41114]: 3.13 scrub ok 2024-09-11T08:51:36.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[41114]: 4.a scrub starts 2024-09-11T08:51:36.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[41114]: 4.a scrub ok 2024-09-11T08:51:36.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[41114]: pgmap v567: 168 pgs: 1 active+clean+scrubbing+deep, 2 active+remapped+backfill_toofull, 165 active+clean; 5.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 0 B/s wr, 822 op/s; 6641/253098 objects misplaced (2.624%); 1.2 MiB/s, 19 objects/s recovering 2024-09-11T08:51:36.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[41114]: 6.1b scrub starts 2024-09-11T08:51:36.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[41114]: 6.1b scrub ok 2024-09-11T08:51:36.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[41114]: 9.a scrub starts 2024-09-11T08:51:36.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:51:36.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:51:36.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:36.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[41114]: 6.4 scrub starts 2024-09-11T08:51:36.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:36 smithi092 ceph-mon[41114]: 6.4 scrub ok 2024-09-11T08:51:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:36 smithi145 ceph-mon[33384]: 3.13 scrub starts 2024-09-11T08:51:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:36 smithi145 ceph-mon[33384]: 3.13 scrub ok 2024-09-11T08:51:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:36 smithi145 ceph-mon[33384]: 4.a scrub starts 2024-09-11T08:51:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:36 smithi145 ceph-mon[33384]: 4.a scrub ok 2024-09-11T08:51:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:36 smithi145 ceph-mon[33384]: pgmap v567: 168 pgs: 1 active+clean+scrubbing+deep, 2 active+remapped+backfill_toofull, 165 active+clean; 5.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 0 B/s wr, 822 op/s; 6641/253098 objects misplaced (2.624%); 1.2 MiB/s, 19 objects/s recovering 2024-09-11T08:51:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:36 smithi145 ceph-mon[33384]: 6.1b scrub starts 2024-09-11T08:51:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:36 smithi145 ceph-mon[33384]: 6.1b scrub ok 2024-09-11T08:51:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:36 smithi145 ceph-mon[33384]: 9.a scrub starts 2024-09-11T08:51:36.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:36 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:51:36.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:36 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:51:36.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:36 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:36.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:36 smithi145 ceph-mon[33384]: 6.4 scrub starts 2024-09-11T08:51:36.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:36 smithi145 ceph-mon[33384]: 6.4 scrub ok 2024-09-11T08:51:36.663 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:36.690 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:36.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:36.691 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:36.792 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:36.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:36.831 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:36.832 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:36.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:36.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:36.961 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:37.062 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:37.098 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:37.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:37.099 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:37.161 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:51:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:51:37.200 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:37.234 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:37.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:37.235 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:37.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:37 smithi092 ceph-mon[29647]: 5.9 scrub starts 2024-09-11T08:51:37.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:37 smithi092 ceph-mon[29647]: 5.9 scrub ok 2024-09-11T08:51:37.295 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:37 smithi092 ceph-mon[41114]: 5.9 scrub starts 2024-09-11T08:51:37.295 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:37 smithi092 ceph-mon[41114]: 5.9 scrub ok 2024-09-11T08:51:37.336 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:37.372 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:37.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:37.373 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:37.474 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:37.509 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:37.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:37.511 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:37.612 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:37.642 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:37.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:37.643 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:37 smithi145 ceph-mon[33384]: 5.9 scrub starts 2024-09-11T08:51:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:37 smithi145 ceph-mon[33384]: 5.9 scrub ok 2024-09-11T08:51:37.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:37.772 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:37.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:37.773 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:37.874 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:37.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:37.913 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:37.913 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:38.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:38.045 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:38.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:38.046 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:38.147 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:38.180 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:38.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:38.181 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:38.282 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:38.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:38.320 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:38.320 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:38.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:38 smithi092 ceph-mon[29647]: 9.d scrub starts 2024-09-11T08:51:38.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:38 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:38.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:38 smithi092 ceph-mon[29647]: 9.a scrub ok 2024-09-11T08:51:38.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:38 smithi092 ceph-mon[29647]: 6.1a scrub starts 2024-09-11T08:51:38.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:38 smithi092 ceph-mon[29647]: 6.1a scrub ok 2024-09-11T08:51:38.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:38 smithi092 ceph-mon[29647]: pgmap v568: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 4.8 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 0 B/s wr, 1.45k op/s; 6176/235320 objects misplaced (2.625%); 1.0 MiB/s, 16 objects/s recovering 2024-09-11T08:51:38.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:38 smithi092 ceph-mon[41114]: 9.d scrub starts 2024-09-11T08:51:38.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:38 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:38.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:38 smithi092 ceph-mon[41114]: 9.a scrub ok 2024-09-11T08:51:38.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:38 smithi092 ceph-mon[41114]: 6.1a scrub starts 2024-09-11T08:51:38.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:38 smithi092 ceph-mon[41114]: 6.1a scrub ok 2024-09-11T08:51:38.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:38 smithi092 ceph-mon[41114]: pgmap v568: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 4.8 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 0 B/s wr, 1.45k op/s; 6176/235320 objects misplaced (2.625%); 1.0 MiB/s, 16 objects/s recovering 2024-09-11T08:51:38.409 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:51:38.409 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:51:38.410 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:51:38.410 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:51:38.421 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:38.421 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:51:38.424 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:51:38.424 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:51:38.455 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:51:38.455 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:51:38.457 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:38.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:38.458 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:38.480 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:51:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:51:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:51:38.499 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:51:38.514 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:51:38.515 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:51:38.530 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:51:38.530 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:51:38.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:38.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:38.595 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:38.595 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:38 smithi145 ceph-mon[33384]: 9.d scrub starts 2024-09-11T08:51:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:38 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:38 smithi145 ceph-mon[33384]: 9.a scrub ok 2024-09-11T08:51:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:38 smithi145 ceph-mon[33384]: 6.1a scrub starts 2024-09-11T08:51:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:38 smithi145 ceph-mon[33384]: 6.1a scrub ok 2024-09-11T08:51:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:38 smithi145 ceph-mon[33384]: pgmap v568: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 4.8 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 0 B/s wr, 1.45k op/s; 6176/235320 objects misplaced (2.625%); 1.0 MiB/s, 16 objects/s recovering 2024-09-11T08:51:38.695 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:38.726 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:38.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:38.728 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:38.828 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:38.854 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:38.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:38.855 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:38.956 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:38.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:38.986 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:38.986 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:39.087 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:39.113 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:39.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:39.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:39.215 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:39.251 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:51:39.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:39.252 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:39.252 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:39.353 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:39.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:39.389 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:39.390 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:39.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:39 smithi145 ceph-mon[33384]: 3.9 scrub starts 2024-09-11T08:51:39.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:39 smithi145 ceph-mon[33384]: 3.9 scrub ok 2024-09-11T08:51:39.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:39 smithi145 ceph-mon[33384]: 9.d scrub ok 2024-09-11T08:51:39.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:39 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1642239276' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:51:39.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:39 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3713888852' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:51:39.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:39 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:51:39.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:39 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:51:39.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:39 smithi145 ceph-mon[33384]: 2.1 deep-scrub starts 2024-09-11T08:51:39.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:39 smithi145 ceph-mon[33384]: 2.1 deep-scrub ok 2024-09-11T08:51:39.490 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:39.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:39.521 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:39.524 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[29647]: 3.9 scrub starts 2024-09-11T08:51:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[29647]: 3.9 scrub ok 2024-09-11T08:51:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[29647]: 9.d scrub ok 2024-09-11T08:51:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1642239276' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:51:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3713888852' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:51:39.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:51:39.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:51:39.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[29647]: 2.1 deep-scrub starts 2024-09-11T08:51:39.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[29647]: 2.1 deep-scrub ok 2024-09-11T08:51:39.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[41114]: 3.9 scrub starts 2024-09-11T08:51:39.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[41114]: 3.9 scrub ok 2024-09-11T08:51:39.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[41114]: 9.d scrub ok 2024-09-11T08:51:39.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1642239276' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:51:39.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3713888852' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:51:39.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:51:39.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:51:39.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[41114]: 2.1 deep-scrub starts 2024-09-11T08:51:39.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:39 smithi092 ceph-mon[41114]: 2.1 deep-scrub ok 2024-09-11T08:51:39.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:39.654 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:39.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:39.655 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:39.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:39.780 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:39.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:39.781 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:39.882 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:39.914 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:39.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:39.915 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:40.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:40.039 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:40.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:40.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:40.141 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:40.172 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:40.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:40.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:40.274 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:40.311 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:40.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:40.312 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:40.412 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:40.441 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:40.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:40.442 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:40.543 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:40.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:40 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:51:40.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:40 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:51:40.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:40 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:51:40.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:40 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3713888852' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:51:40.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:40 smithi092 ceph-mon[29647]: osdmap e264: 8 total, 8 up, 7 in 2024-09-11T08:51:40.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:40 smithi092 ceph-mon[29647]: pgmap v570: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 4.8 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s; 6176/235320 objects misplaced (2.625%); 1.1 MiB/s, 16 objects/s recovering 2024-09-11T08:51:40.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:40 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:51:40.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:40 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:51:40.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:40 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:51:40.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:40 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3713888852' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:51:40.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:40 smithi092 ceph-mon[41114]: osdmap e264: 8 total, 8 up, 7 in 2024-09-11T08:51:40.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:40 smithi092 ceph-mon[41114]: pgmap v570: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 4.8 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s; 6176/235320 objects misplaced (2.625%); 1.1 MiB/s, 16 objects/s recovering 2024-09-11T08:51:40.570 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:40.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:40.571 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:40.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:40 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:51:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:40 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:51:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:40 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:51:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:40 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3713888852' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:51:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:40 smithi145 ceph-mon[33384]: osdmap e264: 8 total, 8 up, 7 in 2024-09-11T08:51:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:40 smithi145 ceph-mon[33384]: pgmap v570: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 4.8 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s; 6176/235320 objects misplaced (2.625%); 1.1 MiB/s, 16 objects/s recovering 2024-09-11T08:51:40.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:40.702 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:40.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:40.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:40.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:40.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:40.832 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:40.933 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:40.969 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:40.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:40.970 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:41.071 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:41.109 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:41.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:41.110 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:41.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:41.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:41.237 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:41.238 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:41.338 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:41.368 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:41.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:41.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:41.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:41.499 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:41.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:41.500 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:41.601 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:41.640 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:41.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:41.641 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:41.742 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:41.745 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:51:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:51:41] "GET /metrics HTTP/1.1" 200 224529 "" "Prometheus/2.43.0" 2024-09-11T08:51:41.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:41.771 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:41.771 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:41.873 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:41.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:41.908 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:41.908 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:42.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:42.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:42.038 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:42.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:42.176 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:42.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:42.178 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:42.226 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:51:42.226 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:51:42.279 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:42.314 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:42.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:42.315 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:42.416 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:42.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:42.455 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:42.455 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:42.556 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:42.588 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:42.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:42.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:42 smithi145 ceph-mon[33384]: pgmap v571: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 4.7 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 824 op/s; 6176/233316 objects misplaced (2.647%) 2024-09-11T08:51:42.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:42 smithi092 ceph-mon[29647]: pgmap v571: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 4.7 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 824 op/s; 6176/233316 objects misplaced (2.647%) 2024-09-11T08:51:42.667 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:42 smithi092 ceph-mon[41114]: pgmap v571: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 4.7 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 824 op/s; 6176/233316 objects misplaced (2.647%) 2024-09-11T08:51:42.690 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:42.726 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:42.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:42.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:42.827 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:42.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:42.861 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:42.862 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:42.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:43.001 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:43.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:43.002 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:43.102 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:43.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:43.140 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:43.140 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:43.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:51:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:51:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:51:43.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:43.272 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:43.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:43.273 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:43.374 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:43.405 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:43.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:43.406 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:43.507 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:43.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:43.540 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:43.540 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:43.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:43.654 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:43 smithi092 ceph-mon[29647]: 9.18 deep-scrub ok 2024-09-11T08:51:43.654 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:43 smithi092 ceph-mon[41114]: 9.18 deep-scrub ok 2024-09-11T08:51:43.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:43 smithi145 ceph-mon[33384]: 9.18 deep-scrub ok 2024-09-11T08:51:43.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:43.678 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:43.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:43.812 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:43.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:43.814 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:43.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:43.951 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:43.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:43.952 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:43.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-11T08:51:44.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:44.101 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:44.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:44.102 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:44.203 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:44.238 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:44.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:44.239 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:44.340 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:44.369 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:44.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:44.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:44.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:44.505 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:44.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:44.507 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:44.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:44.627 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:44 smithi092 ceph-mon[29647]: pgmap v572: 168 pgs: 1 active+clean+scrubbing+deep, 2 active+remapped+backfill_toofull, 165 active+clean; 4.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.24k op/s; 5705/218175 objects misplaced (2.615%) 2024-09-11T08:51:44.627 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:44 smithi092 ceph-mon[29647]: osdmap e265: 8 total, 8 up, 7 in 2024-09-11T08:51:44.627 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:44 smithi092 ceph-mon[41114]: pgmap v572: 168 pgs: 1 active+clean+scrubbing+deep, 2 active+remapped+backfill_toofull, 165 active+clean; 4.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.24k op/s; 5705/218175 objects misplaced (2.615%) 2024-09-11T08:51:44.627 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:44 smithi092 ceph-mon[41114]: osdmap e265: 8 total, 8 up, 7 in 2024-09-11T08:51:44.644 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:44.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:44.645 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:44 smithi145 ceph-mon[33384]: pgmap v572: 168 pgs: 1 active+clean+scrubbing+deep, 2 active+remapped+backfill_toofull, 165 active+clean; 4.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.24k op/s; 5705/218175 objects misplaced (2.615%) 2024-09-11T08:51:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:44 smithi145 ceph-mon[33384]: osdmap e265: 8 total, 8 up, 7 in 2024-09-11T08:51:44.746 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:44.782 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:44.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:44.783 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:44.884 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:44.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:44.920 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:44.920 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:45.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:45.053 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:45.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:45.054 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:45.155 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:45.188 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:45.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:45.190 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:45.290 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:45.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:45.324 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:45.324 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:45.425 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:45.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:45.459 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:45.459 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:45.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:45.595 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:45.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:45.596 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:45.673 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:51:45.697 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:45.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:45.730 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:45.730 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:45.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:51:45.872 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:51:45.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:45.874 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:45.874 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:45.975 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:46.014 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:46.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:46.015 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:46.116 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:46.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:46.151 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:46.151 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:46.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:46.283 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:46.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:46.284 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:46.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:46 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:46.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:46 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:51:46.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:46 smithi092 ceph-mon[29647]: pgmap v574: 168 pgs: 1 active+clean+scrubbing+deep, 2 active+remapped+backfill_toofull, 165 active+clean; 4.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 714 op/s; 5705/218175 objects misplaced (2.615%) 2024-09-11T08:51:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:46 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:46 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:51:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:46 smithi092 ceph-mon[41114]: pgmap v574: 168 pgs: 1 active+clean+scrubbing+deep, 2 active+remapped+backfill_toofull, 165 active+clean; 4.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 714 op/s; 5705/218175 objects misplaced (2.615%) 2024-09-11T08:51:46.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:46.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:46 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:51:46.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:46 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:51:46.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:46 smithi145 ceph-mon[33384]: pgmap v574: 168 pgs: 1 active+clean+scrubbing+deep, 2 active+remapped+backfill_toofull, 165 active+clean; 4.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 714 op/s; 5705/218175 objects misplaced (2.615%) 2024-09-11T08:51:46.422 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:46.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:46.423 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:46.524 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:46.538 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:51:46.567 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:46.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:46.568 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:46.669 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:46.704 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:46.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:46.705 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:46.806 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:46.833 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":574,"stamp":"2024-09-11T08:51:45.274495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4753683880,"num_objects":72725,"num_object_clones":0,"num_object_copies":218175,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5705,"num_objects_unfound":0,"num_objects_dirty":72725,"num_whiteouts":0,"num_read":2020,"num_read_kb":1857,"num_write":215072,"num_write_kb":9178656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28787,"num_bytes_recovered":1876354578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119439,"ondisk_log_size":119439,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":516,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":18960352,"kb_used_data":15650068,"kb_used_omap":11,"kb_used_meta":3309940,"kb_avail":637255712,"statfs":{"total":671965249536,"available":652549849088,"internally_reserved":0,"allocated":16025669632,"data_stored":16021841583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":3389378920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-374538240,"num_objects":-5715,"num_object_clones":0,"num_object_copies":-17145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-471,"num_objects_unfound":0,"num_objects_dirty":-5715,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5715,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002681"},"pg_stats":[{"pgid":"9.15","version":"264'10354","reported_seq":18743,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.412158+0000","last_change":"2024-09-11T08:50:59.945042+0000","last_active":"2024-09-11T08:51:41.412158+0000","last_peered":"2024-09-11T08:51:41.412158+0000","last_clean":"2024-09-11T08:51:41.412158+0000","last_became_active":"2024-09-11T08:50:59.918996+0000","last_became_peered":"2024-09-11T08:50:59.918996+0000","last_unstale":"2024-09-11T08:51:41.412158+0000","last_undegraded":"2024-09-11T08:51:41.412158+0000","last_fullsized":"2024-09-11T08:51:41.412158+0000","mapping_epoch":247,"log_start":"226'5603","ondisk_log_start":"226'5603","created":235,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4751,"log_dups_size":0,"ondisk_log_size":4751,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148897792,"num_objects":2272,"num_object_clones":0,"num_object_copies":6816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6618,"num_write_kb":284480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4144,"num_bytes_recovered":271581184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":76,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.920702+0000","last_change":"2024-09-11T08:51:29.431141+0000","last_active":"2024-09-11T08:51:33.920702+0000","last_peered":"2024-09-11T08:51:33.920702+0000","last_clean":"2024-09-11T08:51:33.920702+0000","last_became_active":"2024-09-11T08:49:09.905661+0000","last_became_peered":"2024-09-11T08:49:09.905661+0000","last_unstale":"2024-09-11T08:51:33.920702+0000","last_undegraded":"2024-09-11T08:51:33.920702+0000","last_fullsized":"2024-09-11T08:51:33.920702+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:29.430953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:29.430953+0000","last_clean_scrub_stamp":"2024-09-11T08:51:29.430953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:45.539942+0000","scrub_duration":0.0052200279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"200'19","reported_seq":324,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.925987+0000","last_change":"2024-09-11T08:49:34.732293+0000","last_active":"2024-09-11T08:51:33.925987+0000","last_peered":"2024-09-11T08:51:33.925987+0000","last_clean":"2024-09-11T08:51:33.925987+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:51:33.925987+0000","last_undegraded":"2024-09-11T08:51:33.925987+0000","last_fullsized":"2024-09-11T08:51:33.925987+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:49:34.732198+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:49:34.732198+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076844679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":272,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.911509+0000","last_change":"2024-09-11T08:50:18.611806+0000","last_active":"2024-09-11T08:51:33.911509+0000","last_peered":"2024-09-11T08:51:33.911509+0000","last_clean":"2024-09-11T08:51:33.911509+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:51:33.911509+0000","last_undegraded":"2024-09-11T08:51:33.911509+0000","last_fullsized":"2024-09-11T08:51:33.911509+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:18.611725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:18.611725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:47.446057+0000","scrub_duration":0.0072923490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":251,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:37.048994+0000","last_change":"2024-09-11T08:51:37.048922+0000","last_active":"2024-09-11T08:51:37.048994+0000","last_peered":"2024-09-11T08:51:37.048994+0000","last_clean":"2024-09-11T08:51:37.048994+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:51:37.048994+0000","last_undegraded":"2024-09-11T08:51:37.048994+0000","last_fullsized":"2024-09-11T08:51:37.048994+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:37.048848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:51:37.048848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:43.265961+0000","scrub_duration":0.0029662830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"264'10526","reported_seq":10601,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.726241+0000","last_change":"2024-09-11T08:50:26.556742+0000","last_active":"2024-09-11T08:51:41.726241+0000","last_peered":"2024-09-11T08:51:41.726241+0000","last_clean":"2024-09-11T08:51:41.726241+0000","last_became_active":"2024-09-11T08:50:26.553605+0000","last_became_peered":"2024-09-11T08:50:26.553605+0000","last_unstale":"2024-09-11T08:51:41.726241+0000","last_undegraded":"2024-09-11T08:51:41.726241+0000","last_fullsized":"2024-09-11T08:51:41.726241+0000","mapping_epoch":235,"log_start":"227'5995","ondisk_log_start":"227'5995","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4531,"log_dups_size":0,"ondisk_log_size":4531,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155418624,"num_objects":2372,"num_object_clones":0,"num_object_copies":7116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6718,"num_write_kb":290848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":280,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:35.370552+0000","last_change":"2024-09-11T08:51:35.370494+0000","last_active":"2024-09-11T08:51:35.370552+0000","last_peered":"2024-09-11T08:51:35.370552+0000","last_clean":"2024-09-11T08:51:35.370552+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:51:35.370552+0000","last_undegraded":"2024-09-11T08:51:35.370552+0000","last_fullsized":"2024-09-11T08:51:35.370552+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:35.370438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:51:35.370438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:53.655448+0000","scrub_duration":0.0053884329999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":286,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916751+0000","last_change":"2024-09-11T08:51:28.347060+0000","last_active":"2024-09-11T08:51:33.916751+0000","last_peered":"2024-09-11T08:51:33.916751+0000","last_clean":"2024-09-11T08:51:33.916751+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:51:33.916751+0000","last_undegraded":"2024-09-11T08:51:33.916751+0000","last_fullsized":"2024-09-11T08:51:33.916751+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:28.346979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:51:28.346979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:38.146440+0000","scrub_duration":0.0020416560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"200'34","reported_seq":340,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916837+0000","last_change":"2024-09-11T08:51:31.358835+0000","last_active":"2024-09-11T08:51:33.916837+0000","last_peered":"2024-09-11T08:51:33.916837+0000","last_clean":"2024-09-11T08:51:33.916837+0000","last_became_active":"2024-09-11T08:49:09.909107+0000","last_became_peered":"2024-09-11T08:49:09.909107+0000","last_unstale":"2024-09-11T08:51:33.916837+0000","last_undegraded":"2024-09-11T08:51:33.916837+0000","last_fullsized":"2024-09-11T08:51:33.916837+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:51:31.358660+0000","last_deep_scrub":"200'34","last_deep_scrub_stamp":"2024-09-11T08:51:31.358660+0000","last_clean_scrub_stamp":"2024-09-11T08:51:31.358660+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:35.173645+0000","scrub_duration":0.0071678469999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":253,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.925933+0000","last_change":"2024-09-11T08:51:19.004758+0000","last_active":"2024-09-11T08:51:33.925933+0000","last_peered":"2024-09-11T08:51:33.925933+0000","last_clean":"2024-09-11T08:51:33.925933+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:51:33.925933+0000","last_undegraded":"2024-09-11T08:51:33.925933+0000","last_fullsized":"2024-09-11T08:51:33.925933+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:19.004676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:51:19.004676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:45.688554+0000","scrub_duration":0.006451526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.17","version":"264'10441","reported_seq":22949,"reported_epoch":264,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T08:51:41.709740+0000","last_change":"2024-09-11T08:51:18.648932+0000","last_active":"2024-09-11T08:51:41.709740+0000","last_peered":"2024-09-11T08:51:41.709740+0000","last_clean":"2024-09-11T08:51:16.491184+0000","last_became_active":"2024-09-11T08:51:18.635420+0000","last_became_peered":"2024-09-11T08:51:18.635420+0000","last_unstale":"2024-09-11T08:51:41.709740+0000","last_undegraded":"2024-09-11T08:51:41.709740+0000","last_fullsized":"2024-09-11T08:51:41.709740+0000","mapping_epoch":255,"log_start":"227'5996","ondisk_log_start":"227'5996","created":235,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4445,"log_dups_size":0,"ondisk_log_size":4445,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":144113664,"num_objects":2199,"num_object_clones":0,"num_object_copies":6597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2199,"num_objects_unfound":0,"num_objects_dirty":2199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6683,"num_write_kb":284224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4133,"num_bytes_recovered":270860288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[1,2,7],"avail_no_missing":["1","2","3","5","7"],"object_location_counts":[{"shards":"1,2,7","objects":2199}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":292,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.923323+0000","last_change":"2024-09-11T08:51:18.275081+0000","last_active":"2024-09-11T08:51:33.923323+0000","last_peered":"2024-09-11T08:51:33.923323+0000","last_clean":"2024-09-11T08:51:33.923323+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:51:33.923323+0000","last_undegraded":"2024-09-11T08:51:33.923323+0000","last_fullsized":"2024-09-11T08:51:33.923323+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:18.275025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:51:18.275025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:19.965593+0000","scrub_duration":0.0011943050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"200'19","reported_seq":314,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:34.003977+0000","last_change":"2024-09-11T08:51:34.003898+0000","last_active":"2024-09-11T08:51:34.003977+0000","last_peered":"2024-09-11T08:51:34.003977+0000","last_clean":"2024-09-11T08:51:34.003977+0000","last_became_active":"2024-09-11T08:49:09.910441+0000","last_became_peered":"2024-09-11T08:49:09.910441+0000","last_unstale":"2024-09-11T08:51:34.003977+0000","last_undegraded":"2024-09-11T08:51:34.003977+0000","last_fullsized":"2024-09-11T08:51:34.003977+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:34.003806+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:51:34.003806+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:46.075623+0000","scrub_duration":0.0022468330000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":278,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.919831+0000","last_change":"2024-09-11T08:50:25.373627+0000","last_active":"2024-09-11T08:51:33.919831+0000","last_peered":"2024-09-11T08:51:33.919831+0000","last_clean":"2024-09-11T08:51:33.919831+0000","last_became_active":"2024-09-11T08:49:09.899188+0000","last_became_peered":"2024-09-11T08:49:09.899188+0000","last_unstale":"2024-09-11T08:51:33.919831+0000","last_undegraded":"2024-09-11T08:51:33.919831+0000","last_fullsized":"2024-09-11T08:51:33.919831+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:25.373549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:25.373549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026496549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":76,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.912017+0000","last_change":"2024-09-11T08:49:33.520537+0000","last_active":"2024-09-11T08:51:33.912017+0000","last_peered":"2024-09-11T08:51:33.912017+0000","last_clean":"2024-09-11T08:51:33.912017+0000","last_became_active":"2024-09-11T08:49:09.907856+0000","last_became_peered":"2024-09-11T08:49:09.907856+0000","last_unstale":"2024-09-11T08:51:33.912017+0000","last_undegraded":"2024-09-11T08:51:33.912017+0000","last_fullsized":"2024-09-11T08:51:33.912017+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:33.520462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:33.520462+0000","last_clean_scrub_stamp":"2024-09-11T08:49:33.520462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043744099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.16","version":"264'10323","reported_seq":18791,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.150386+0000","last_change":"2024-09-11T08:50:53.929053+0000","last_active":"2024-09-11T08:51:41.150386+0000","last_peered":"2024-09-11T08:51:41.150386+0000","last_clean":"2024-09-11T08:51:41.150386+0000","last_became_active":"2024-09-11T08:50:53.902271+0000","last_became_peered":"2024-09-11T08:50:53.902271+0000","last_unstale":"2024-09-11T08:51:41.150386+0000","last_undegraded":"2024-09-11T08:51:41.150386+0000","last_fullsized":"2024-09-11T08:51:41.150386+0000","mapping_epoch":243,"log_start":"223'5043","ondisk_log_start":"223'5043","created":235,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":5280,"log_dups_size":0,"ondisk_log_size":5280,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147685376,"num_objects":2254,"num_object_clones":0,"num_object_copies":6762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6598,"num_write_kb":283232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4189,"num_bytes_recovered":274530304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":272,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.919306+0000","last_change":"2024-09-11T08:51:30.465078+0000","last_active":"2024-09-11T08:51:33.919306+0000","last_peered":"2024-09-11T08:51:33.919306+0000","last_clean":"2024-09-11T08:51:33.919306+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:51:33.919306+0000","last_undegraded":"2024-09-11T08:51:33.919306+0000","last_fullsized":"2024-09-11T08:51:33.919306+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:30.465003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:30.465003+0000","last_clean_scrub_stamp":"2024-09-11T08:51:30.465003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:51.931388+0000","scrub_duration":0.0019306950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"200'7","reported_seq":261,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.927232+0000","last_change":"2024-09-11T08:51:09.084860+0000","last_active":"2024-09-11T08:51:33.927232+0000","last_peered":"2024-09-11T08:51:33.927232+0000","last_clean":"2024-09-11T08:51:33.927232+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:51:33.927232+0000","last_undegraded":"2024-09-11T08:51:33.927232+0000","last_fullsized":"2024-09-11T08:51:33.927232+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:47:44.866057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:47:44.866057+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020690169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":272,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.915501+0000","last_change":"2024-09-11T08:50:18.199730+0000","last_active":"2024-09-11T08:51:33.915501+0000","last_peered":"2024-09-11T08:51:33.915501+0000","last_clean":"2024-09-11T08:51:33.915501+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:51:33.915501+0000","last_undegraded":"2024-09-11T08:51:33.915501+0000","last_fullsized":"2024-09-11T08:51:33.915501+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:18.199651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:18.199651+0000","last_clean_scrub_stamp":"2024-09-11T08:50:18.199651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048782360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":256,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.915319+0000","last_change":"2024-09-11T08:49:28.283272+0000","last_active":"2024-09-11T08:51:33.915319+0000","last_peered":"2024-09-11T08:51:33.915319+0000","last_clean":"2024-09-11T08:51:33.915319+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:51:33.915319+0000","last_undegraded":"2024-09-11T08:51:33.915319+0000","last_fullsized":"2024-09-11T08:51:33.915319+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:28.283167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:49:28.283167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040125320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"264'10562","reported_seq":10644,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.392799+0000","last_change":"2024-09-11T08:50:28.541687+0000","last_active":"2024-09-11T08:51:41.392799+0000","last_peered":"2024-09-11T08:51:41.392799+0000","last_clean":"2024-09-11T08:51:41.392799+0000","last_became_active":"2024-09-11T08:50:28.539715+0000","last_became_peered":"2024-09-11T08:50:28.539715+0000","last_unstale":"2024-09-11T08:51:41.392799+0000","last_undegraded":"2024-09-11T08:51:41.392799+0000","last_fullsized":"2024-09-11T08:51:41.392799+0000","mapping_epoch":237,"log_start":"227'5860","ondisk_log_start":"227'5860","created":235,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4702,"log_dups_size":0,"ondisk_log_size":4702,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":152961024,"num_objects":2334,"num_object_clones":0,"num_object_copies":7002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6758,"num_write_kb":290944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":271,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.929430+0000","last_change":"2024-09-11T08:50:13.762434+0000","last_active":"2024-09-11T08:51:33.929430+0000","last_peered":"2024-09-11T08:51:33.929430+0000","last_clean":"2024-09-11T08:51:33.929430+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:51:33.929430+0000","last_undegraded":"2024-09-11T08:51:33.929430+0000","last_fullsized":"2024-09-11T08:51:33.929430+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:13.762372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:13.762372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:41.024766+0000","scrub_duration":0.001701496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":76,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.925805+0000","last_change":"2024-09-11T08:50:24.573824+0000","last_active":"2024-09-11T08:51:33.925805+0000","last_peered":"2024-09-11T08:51:33.925805+0000","last_clean":"2024-09-11T08:51:33.925805+0000","last_became_active":"2024-09-11T08:49:09.908237+0000","last_became_peered":"2024-09-11T08:49:09.908237+0000","last_unstale":"2024-09-11T08:51:33.925805+0000","last_undegraded":"2024-09-11T08:51:33.925805+0000","last_fullsized":"2024-09-11T08:51:33.925805+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:24.573732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:24.573732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:48.045877+0000","scrub_duration":0.002943612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"200'34","reported_seq":350,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.918987+0000","last_change":"2024-09-11T08:51:23.163719+0000","last_active":"2024-09-11T08:51:33.918987+0000","last_peered":"2024-09-11T08:51:33.918987+0000","last_clean":"2024-09-11T08:51:33.918987+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:51:33.918987+0000","last_undegraded":"2024-09-11T08:51:33.918987+0000","last_fullsized":"2024-09-11T08:51:33.918987+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:51:23.163637+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:51:23.163637+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:34.786278+0000","scrub_duration":0.0059142170000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":282,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.911845+0000","last_change":"2024-09-11T08:51:29.793970+0000","last_active":"2024-09-11T08:51:33.911845+0000","last_peered":"2024-09-11T08:51:33.911845+0000","last_clean":"2024-09-11T08:51:33.911845+0000","last_became_active":"2024-09-11T08:49:09.903078+0000","last_became_peered":"2024-09-11T08:49:09.903078+0000","last_unstale":"2024-09-11T08:51:33.911845+0000","last_undegraded":"2024-09-11T08:51:33.911845+0000","last_fullsized":"2024-09-11T08:51:33.911845+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:29.793890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:29.793890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:56.913485+0000","scrub_duration":0.0020976990000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"264'10357","reported_seq":13203,"reported_epoch":264,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T08:51:41.635983+0000","last_change":"2024-09-11T08:51:33.033583+0000","last_active":"2024-09-11T08:51:41.635983+0000","last_peered":"2024-09-11T08:51:41.635983+0000","last_clean":"2024-09-11T08:50:27.522886+0000","last_became_active":"2024-09-11T08:50:28.966712+0000","last_became_peered":"2024-09-11T08:50:28.966712+0000","last_unstale":"2024-09-11T08:51:41.635983+0000","last_undegraded":"2024-09-11T08:51:41.635983+0000","last_fullsized":"2024-09-11T08:51:41.635983+0000","mapping_epoch":238,"log_start":"227'5821","ondisk_log_start":"227'5821","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4536,"log_dups_size":0,"ondisk_log_size":4536,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148897792,"num_objects":2272,"num_object_clones":0,"num_object_copies":6816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3506,"num_objects_unfound":0,"num_objects_dirty":2272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6596,"num_write_kb":283776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":916,"num_bytes_recovered":60030976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[1,2,6],"avail_no_missing":["1","2","6"],"object_location_counts":[{"shards":"1,2,6","objects":2272}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":279,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.917173+0000","last_change":"2024-09-11T08:50:07.275539+0000","last_active":"2024-09-11T08:51:33.917173+0000","last_peered":"2024-09-11T08:51:33.917173+0000","last_clean":"2024-09-11T08:51:33.917173+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:51:33.917173+0000","last_undegraded":"2024-09-11T08:51:33.917173+0000","last_fullsized":"2024-09-11T08:51:33.917173+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:50:07.275301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:07.275301+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019610208000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":257,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.926528+0000","last_change":"2024-09-11T08:50:17.743118+0000","last_active":"2024-09-11T08:51:33.926528+0000","last_peered":"2024-09-11T08:51:33.926528+0000","last_clean":"2024-09-11T08:51:33.926528+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:51:33.926528+0000","last_undegraded":"2024-09-11T08:51:33.926528+0000","last_fullsized":"2024-09-11T08:51:33.926528+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:17.743026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:17.743026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:40.993419+0000","scrub_duration":0.0061030829999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"200'18","reported_seq":308,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916598+0000","last_change":"2024-09-11T08:49:26.388924+0000","last_active":"2024-09-11T08:51:33.916598+0000","last_peered":"2024-09-11T08:51:33.916598+0000","last_clean":"2024-09-11T08:51:33.916598+0000","last_became_active":"2024-09-11T08:49:16.943886+0000","last_became_peered":"2024-09-11T08:49:16.943886+0000","last_unstale":"2024-09-11T08:51:33.916598+0000","last_undegraded":"2024-09-11T08:51:33.916598+0000","last_fullsized":"2024-09-11T08:51:33.916598+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'18","last_scrub_stamp":"2024-09-11T08:49:26.388870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:49:26.388870+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001482014,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":253,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.927800+0000","last_change":"2024-09-11T08:50:09.746351+0000","last_active":"2024-09-11T08:51:33.927800+0000","last_peered":"2024-09-11T08:51:33.927800+0000","last_clean":"2024-09-11T08:51:33.927800+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:51:33.927800+0000","last_undegraded":"2024-09-11T08:51:33.927800+0000","last_fullsized":"2024-09-11T08:51:33.927800+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:09.745640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:09.745640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010457068999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"264'10501","reported_seq":22832,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.377003+0000","last_change":"2024-09-11T08:51:29.165502+0000","last_active":"2024-09-11T08:51:41.377003+0000","last_peered":"2024-09-11T08:51:41.377003+0000","last_clean":"2024-09-11T08:51:41.377003+0000","last_became_active":"2024-09-11T08:51:29.125323+0000","last_became_peered":"2024-09-11T08:51:29.125323+0000","last_unstale":"2024-09-11T08:51:41.377003+0000","last_undegraded":"2024-09-11T08:51:41.377003+0000","last_fullsized":"2024-09-11T08:51:41.377003+0000","mapping_epoch":261,"log_start":"227'5776","ondisk_log_start":"227'5776","created":235,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4725,"log_dups_size":0,"ondisk_log_size":4725,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":151191552,"num_objects":2307,"num_object_clones":0,"num_object_copies":6921,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6713,"num_write_kb":288640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4074,"num_bytes_recovered":266993664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":261,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.915670+0000","last_change":"2024-09-11T08:50:09.077952+0000","last_active":"2024-09-11T08:51:33.915670+0000","last_peered":"2024-09-11T08:51:33.915670+0000","last_clean":"2024-09-11T08:51:33.915670+0000","last_became_active":"2024-09-11T08:38:44.992891+0000","last_became_peered":"2024-09-11T08:38:44.992891+0000","last_unstale":"2024-09-11T08:51:33.915670+0000","last_undegraded":"2024-09-11T08:51:33.915670+0000","last_fullsized":"2024-09-11T08:51:33.915670+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:50:09.077871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:09.077871+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059438809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":276,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.911578+0000","last_change":"2024-09-11T08:51:28.824144+0000","last_active":"2024-09-11T08:51:33.911578+0000","last_peered":"2024-09-11T08:51:33.911578+0000","last_clean":"2024-09-11T08:51:33.911578+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:51:33.911578+0000","last_undegraded":"2024-09-11T08:51:33.911578+0000","last_fullsized":"2024-09-11T08:51:33.911578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:28.824093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:51:28.824093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:48.361844+0000","scrub_duration":0.0017179579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"200'20","reported_seq":303,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916070+0000","last_change":"2024-09-11T08:51:15.344788+0000","last_active":"2024-09-11T08:51:33.916070+0000","last_peered":"2024-09-11T08:51:33.916070+0000","last_clean":"2024-09-11T08:51:33.916070+0000","last_became_active":"2024-09-11T08:49:09.913968+0000","last_became_peered":"2024-09-11T08:49:09.913968+0000","last_unstale":"2024-09-11T08:51:33.916070+0000","last_undegraded":"2024-09-11T08:51:33.916070+0000","last_fullsized":"2024-09-11T08:51:33.916070+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:48:19.705185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:19.705185+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050623580000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":439,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:37.824262+0000","last_change":"2024-09-11T08:50:06.195464+0000","last_active":"2024-09-11T08:51:37.824262+0000","last_peered":"2024-09-11T08:51:37.824262+0000","last_clean":"2024-09-11T08:51:37.824262+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:51:37.824262+0000","last_undegraded":"2024-09-11T08:51:37.824262+0000","last_fullsized":"2024-09-11T08:51:37.824262+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:50:06.195410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:06.195410+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013222749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"199'81","reported_seq":23,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.914582+0000","last_change":"2024-09-11T08:51:03.024010+0000","last_active":"2024-09-11T08:51:33.914582+0000","last_peered":"2024-09-11T08:51:33.914582+0000","last_clean":"2024-09-11T08:51:33.914582+0000","last_became_active":"2024-09-11T08:51:03.023615+0000","last_became_peered":"2024-09-11T08:51:03.023615+0000","last_unstale":"2024-09-11T08:51:33.914582+0000","last_undegraded":"2024-09-11T08:51:33.914582+0000","last_fullsized":"2024-09-11T08:51:33.914582+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"264'10384","reported_seq":10461,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.824731+0000","last_change":"2024-09-11T08:50:26.529378+0000","last_active":"2024-09-11T08:51:41.824731+0000","last_peered":"2024-09-11T08:51:41.824731+0000","last_clean":"2024-09-11T08:51:41.824731+0000","last_became_active":"2024-09-11T08:50:26.526948+0000","last_became_peered":"2024-09-11T08:50:26.526948+0000","last_unstale":"2024-09-11T08:51:41.824731+0000","last_undegraded":"2024-09-11T08:51:41.824731+0000","last_fullsized":"2024-09-11T08:51:41.824731+0000","mapping_epoch":206,"log_start":"227'6257","ondisk_log_start":"227'6257","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4127,"log_dups_size":0,"ondisk_log_size":4127,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145555456,"num_objects":2221,"num_object_clones":0,"num_object_copies":6663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6671,"num_write_kb":284544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"200'34","reported_seq":314,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.913182+0000","last_change":"2024-09-11T08:50:03.458529+0000","last_active":"2024-09-11T08:51:33.913182+0000","last_peered":"2024-09-11T08:51:33.913182+0000","last_clean":"2024-09-11T08:51:33.913182+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:51:33.913182+0000","last_undegraded":"2024-09-11T08:51:33.913182+0000","last_fullsized":"2024-09-11T08:51:33.913182+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:50:03.458381+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:50:03.458381+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023474210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":256,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916281+0000","last_change":"2024-09-11T08:49:25.383955+0000","last_active":"2024-09-11T08:51:33.916281+0000","last_peered":"2024-09-11T08:51:33.916281+0000","last_clean":"2024-09-11T08:51:33.916281+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:51:33.916281+0000","last_undegraded":"2024-09-11T08:51:33.916281+0000","last_fullsized":"2024-09-11T08:51:33.916281+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:25.383554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:25.383554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0084378879999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"211'10","reported_seq":432,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:37.828727+0000","last_change":"2024-09-11T08:49:35.694375+0000","last_active":"2024-09-11T08:51:37.828727+0000","last_peered":"2024-09-11T08:51:37.828727+0000","last_clean":"2024-09-11T08:51:37.828727+0000","last_became_active":"2024-09-11T08:49:09.905262+0000","last_became_peered":"2024-09-11T08:49:09.905262+0000","last_unstale":"2024-09-11T08:51:37.828727+0000","last_undegraded":"2024-09-11T08:51:37.828727+0000","last_fullsized":"2024-09-11T08:51:37.828727+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'10","last_scrub_stamp":"2024-09-11T08:49:35.694282+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:49:35.694282+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051461679999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":263,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.922894+0000","last_change":"2024-09-11T08:49:20.335925+0000","last_active":"2024-09-11T08:51:33.922894+0000","last_peered":"2024-09-11T08:51:33.922894+0000","last_clean":"2024-09-11T08:51:33.922894+0000","last_became_active":"2024-09-11T08:49:09.906872+0000","last_became_peered":"2024-09-11T08:49:09.906872+0000","last_unstale":"2024-09-11T08:51:33.922894+0000","last_undegraded":"2024-09-11T08:51:33.922894+0000","last_fullsized":"2024-09-11T08:51:33.922894+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:20.335841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:20.335841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041148479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"199'81","reported_seq":23,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.929156+0000","last_change":"2024-09-11T08:51:03.023733+0000","last_active":"2024-09-11T08:51:33.929156+0000","last_peered":"2024-09-11T08:51:33.929156+0000","last_clean":"2024-09-11T08:51:33.929156+0000","last_became_active":"2024-09-11T08:51:03.023365+0000","last_became_peered":"2024-09-11T08:51:03.023365+0000","last_unstale":"2024-09-11T08:51:33.929156+0000","last_undegraded":"2024-09-11T08:51:33.929156+0000","last_fullsized":"2024-09-11T08:51:33.929156+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"264'13483","reported_seq":13754,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.403589+0000","last_change":"2024-09-11T08:51:38.165835+0000","last_active":"2024-09-11T08:51:41.403589+0000","last_peered":"2024-09-11T08:51:41.403589+0000","last_clean":"2024-09-11T08:51:41.403589+0000","last_became_active":"2024-09-11T08:49:05.875264+0000","last_became_peered":"2024-09-11T08:49:05.875264+0000","last_unstale":"2024-09-11T08:51:41.403589+0000","last_undegraded":"2024-09-11T08:51:41.403589+0000","last_fullsized":"2024-09-11T08:51:41.403589+0000","mapping_epoch":206,"log_start":"242'9602","ondisk_log_start":"242'9602","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'13254","last_scrub_stamp":"2024-09-11T08:51:38.165747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:51:38.165747+0000","objects_scrubbed":4839,"log_size":3881,"log_dups_size":0,"ondisk_log_size":3881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:59.271449+0000","scrub_duration":1.68855479,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":298188824,"num_objects":4551,"num_object_clones":0,"num_object_copies":13653,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4551,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13483,"num_write_kb":577025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"200'38","reported_seq":340,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916151+0000","last_change":"2024-09-11T08:49:09.936915+0000","last_active":"2024-09-11T08:51:33.916151+0000","last_peered":"2024-09-11T08:51:33.916151+0000","last_clean":"2024-09-11T08:51:33.916151+0000","last_became_active":"2024-09-11T08:49:09.917354+0000","last_became_peered":"2024-09-11T08:49:09.917354+0000","last_unstale":"2024-09-11T08:51:33.916151+0000","last_undegraded":"2024-09-11T08:51:33.916151+0000","last_fullsized":"2024-09-11T08:51:33.916151+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'26","last_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_deep_scrub":"199'26","last_deep_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_clean_scrub_stamp":"2024-09-11T08:48:44.093994+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.026938573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":269,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.917507+0000","last_change":"2024-09-11T08:51:27.328779+0000","last_active":"2024-09-11T08:51:33.917507+0000","last_peered":"2024-09-11T08:51:33.917507+0000","last_clean":"2024-09-11T08:51:33.917507+0000","last_became_active":"2024-09-11T08:49:09.915191+0000","last_became_peered":"2024-09-11T08:49:09.915191+0000","last_unstale":"2024-09-11T08:51:33.917507+0000","last_undegraded":"2024-09-11T08:51:33.917507+0000","last_fullsized":"2024-09-11T08:51:33.917507+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:38.041303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:48:38.041303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073485030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":253,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.926642+0000","last_change":"2024-09-11T08:49:29.687666+0000","last_active":"2024-09-11T08:51:33.926642+0000","last_peered":"2024-09-11T08:51:33.926642+0000","last_clean":"2024-09-11T08:51:33.926642+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:51:33.926642+0000","last_undegraded":"2024-09-11T08:51:33.926642+0000","last_fullsized":"2024-09-11T08:51:33.926642+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:29.687577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:29.687577+0000","last_clean_scrub_stamp":"2024-09-11T08:49:29.687577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082090140000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":251,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.926397+0000","last_change":"2024-09-11T08:50:19.706569+0000","last_active":"2024-09-11T08:51:33.926397+0000","last_peered":"2024-09-11T08:51:33.926397+0000","last_clean":"2024-09-11T08:51:33.926397+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:51:33.926397+0000","last_undegraded":"2024-09-11T08:51:33.926397+0000","last_fullsized":"2024-09-11T08:51:33.926397+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:19.706509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:19.706509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:44.665049+0000","scrub_duration":0.045176716999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"199'81","reported_seq":28,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.917100+0000","last_change":"2024-09-11T08:51:03.053381+0000","last_active":"2024-09-11T08:51:33.917100+0000","last_peered":"2024-09-11T08:51:33.917100+0000","last_clean":"2024-09-11T08:51:33.917100+0000","last_became_active":"2024-09-11T08:51:03.009983+0000","last_became_peered":"2024-09-11T08:51:03.009983+0000","last_unstale":"2024-09-11T08:51:33.917100+0000","last_undegraded":"2024-09-11T08:51:33.917100+0000","last_fullsized":"2024-09-11T08:51:33.917100+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":381350,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"264'13289","reported_seq":13355,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.887704+0000","last_change":"2024-09-11T08:49:05.874640+0000","last_active":"2024-09-11T08:51:41.887704+0000","last_peered":"2024-09-11T08:51:41.887704+0000","last_clean":"2024-09-11T08:51:41.887704+0000","last_became_active":"2024-09-11T08:49:05.874365+0000","last_became_peered":"2024-09-11T08:49:05.874365+0000","last_unstale":"2024-09-11T08:51:41.887704+0000","last_undegraded":"2024-09-11T08:51:41.887704+0000","last_fullsized":"2024-09-11T08:51:41.887704+0000","mapping_epoch":206,"log_start":"245'9677","ondisk_log_start":"245'9677","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3612,"log_dups_size":0,"ondisk_log_size":3612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":285802496,"num_objects":4361,"num_object_clones":0,"num_object_copies":13083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13289,"num_write_kb":564800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"200'35","reported_seq":297,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.926875+0000","last_change":"2024-09-11T08:51:14.084540+0000","last_active":"2024-09-11T08:51:33.926875+0000","last_peered":"2024-09-11T08:51:33.926875+0000","last_clean":"2024-09-11T08:51:33.926875+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:51:33.926875+0000","last_undegraded":"2024-09-11T08:51:33.926875+0000","last_fullsized":"2024-09-11T08:51:33.926875+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'21","last_scrub_stamp":"2024-09-11T08:48:23.862482+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:48:23.862482+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001795776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":76,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.912765+0000","last_change":"2024-09-11T08:49:24.594111+0000","last_active":"2024-09-11T08:51:33.912765+0000","last_peered":"2024-09-11T08:51:33.912765+0000","last_clean":"2024-09-11T08:51:33.912765+0000","last_became_active":"2024-09-11T08:49:09.908041+0000","last_became_peered":"2024-09-11T08:49:09.908041+0000","last_unstale":"2024-09-11T08:51:33.912765+0000","last_undegraded":"2024-09-11T08:51:33.912765+0000","last_fullsized":"2024-09-11T08:51:33.912765+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:24.593902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:24.593902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.023705506000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":419,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:37.824336+0000","last_change":"2024-09-11T08:51:29.395679+0000","last_active":"2024-09-11T08:51:37.824336+0000","last_peered":"2024-09-11T08:51:37.824336+0000","last_clean":"2024-09-11T08:51:37.824336+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:51:37.824336+0000","last_undegraded":"2024-09-11T08:51:37.824336+0000","last_fullsized":"2024-09-11T08:51:37.824336+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:51:29.395598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:29.395598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:35.354158+0000","scrub_duration":0.0041488640000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":280,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.915695+0000","last_change":"2024-09-11T08:49:31.758729+0000","last_active":"2024-09-11T08:51:33.915695+0000","last_peered":"2024-09-11T08:51:33.915695+0000","last_clean":"2024-09-11T08:51:33.915695+0000","last_became_active":"2024-09-11T08:46:01.983335+0000","last_became_peered":"2024-09-11T08:46:01.983335+0000","last_unstale":"2024-09-11T08:51:33.915695+0000","last_undegraded":"2024-09-11T08:51:33.915695+0000","last_fullsized":"2024-09-11T08:51:33.915695+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:31.758573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:31.758573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0084881980000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"199'81","reported_seq":275,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.918928+0000","last_change":"2024-09-11T08:51:22.301835+0000","last_active":"2024-09-11T08:51:33.918928+0000","last_peered":"2024-09-11T08:51:33.918928+0000","last_clean":"2024-09-11T08:51:33.918928+0000","last_became_active":"2024-09-11T08:51:03.003800+0000","last_became_peered":"2024-09-11T08:51:03.003800+0000","last_unstale":"2024-09-11T08:51:33.918928+0000","last_undegraded":"2024-09-11T08:51:33.918928+0000","last_fullsized":"2024-09-11T08:51:33.918928+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:51:22.301752+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:51:22.301752+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:45.881190+0000","scrub_duration":0.0061576410000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"264'13388","reported_seq":13453,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.426182+0000","last_change":"2024-09-11T08:49:05.871535+0000","last_active":"2024-09-11T08:51:41.426182+0000","last_peered":"2024-09-11T08:51:41.426182+0000","last_clean":"2024-09-11T08:51:41.426182+0000","last_became_active":"2024-09-11T08:49:05.871255+0000","last_became_peered":"2024-09-11T08:49:05.871255+0000","last_unstale":"2024-09-11T08:51:41.426182+0000","last_undegraded":"2024-09-11T08:51:41.426182+0000","last_fullsized":"2024-09-11T08:51:41.426182+0000","mapping_epoch":206,"log_start":"241'9369","ondisk_log_start":"241'9369","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4019,"log_dups_size":0,"ondisk_log_size":4019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":303562752,"num_objects":4632,"num_object_clones":0,"num_object_copies":13896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13388,"num_write_kb":576640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"200'25","reported_seq":309,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.911621+0000","last_change":"2024-09-11T08:51:31.799902+0000","last_active":"2024-09-11T08:51:33.911621+0000","last_peered":"2024-09-11T08:51:33.911621+0000","last_clean":"2024-09-11T08:51:33.911621+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:51:33.911621+0000","last_undegraded":"2024-09-11T08:51:33.911621+0000","last_fullsized":"2024-09-11T08:51:33.911621+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:51:31.799844+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_clean_scrub_stamp":"2024-09-11T08:51:31.799844+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:59.532144+0000","scrub_duration":0.0075128269999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":262,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.923393+0000","last_change":"2024-09-11T08:49:26.248106+0000","last_active":"2024-09-11T08:51:33.923393+0000","last_peered":"2024-09-11T08:51:33.923393+0000","last_clean":"2024-09-11T08:51:33.923393+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:51:33.923393+0000","last_undegraded":"2024-09-11T08:51:33.923393+0000","last_fullsized":"2024-09-11T08:51:33.923393+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:26.247212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:26.247212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056073520000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"211'9","reported_seq":129,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:37.825691+0000","last_change":"2024-09-11T08:50:12.144286+0000","last_active":"2024-09-11T08:51:37.825691+0000","last_peered":"2024-09-11T08:51:37.825691+0000","last_clean":"2024-09-11T08:51:37.825691+0000","last_became_active":"2024-09-11T08:49:09.910279+0000","last_became_peered":"2024-09-11T08:49:09.910279+0000","last_unstale":"2024-09-11T08:51:37.825691+0000","last_undegraded":"2024-09-11T08:51:37.825691+0000","last_fullsized":"2024-09-11T08:51:37.825691+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'9","last_scrub_stamp":"2024-09-11T08:50:12.144196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:12.144196+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013090123,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":76,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.915992+0000","last_change":"2024-09-11T08:51:20.428797+0000","last_active":"2024-09-11T08:51:33.915992+0000","last_peered":"2024-09-11T08:51:33.915992+0000","last_clean":"2024-09-11T08:51:33.915992+0000","last_became_active":"2024-09-11T08:49:09.907944+0000","last_became_peered":"2024-09-11T08:49:09.907944+0000","last_unstale":"2024-09-11T08:51:33.915992+0000","last_undegraded":"2024-09-11T08:51:33.915992+0000","last_fullsized":"2024-09-11T08:51:33.915992+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:20.428724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:51:20.428724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:33.509031+0000","scrub_duration":0.002261304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"199'81","reported_seq":275,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.918512+0000","last_change":"2024-09-11T08:51:28.247210+0000","last_active":"2024-09-11T08:51:33.918512+0000","last_peered":"2024-09-11T08:51:33.918512+0000","last_clean":"2024-09-11T08:51:33.918512+0000","last_became_active":"2024-09-11T08:51:03.003509+0000","last_became_peered":"2024-09-11T08:51:03.003509+0000","last_unstale":"2024-09-11T08:51:33.918512+0000","last_undegraded":"2024-09-11T08:51:33.918512+0000","last_fullsized":"2024-09-11T08:51:33.918512+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:51:28.247128+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:51:28.247128+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:29.305869+0000","scrub_duration":0.0025551829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"264'13643","reported_seq":13709,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.412097+0000","last_change":"2024-09-11T08:49:05.875096+0000","last_active":"2024-09-11T08:51:41.412097+0000","last_peered":"2024-09-11T08:51:41.412097+0000","last_clean":"2024-09-11T08:51:41.412097+0000","last_became_active":"2024-09-11T08:49:05.874803+0000","last_became_peered":"2024-09-11T08:49:05.874803+0000","last_unstale":"2024-09-11T08:51:41.412097+0000","last_undegraded":"2024-09-11T08:51:41.412097+0000","last_fullsized":"2024-09-11T08:51:41.412097+0000","mapping_epoch":206,"log_start":"242'9778","ondisk_log_start":"242'9778","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":3865,"log_dups_size":0,"ondisk_log_size":3865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302317568,"num_objects":4613,"num_object_clones":0,"num_object_copies":13839,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13643,"num_write_kb":584192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"200'29","reported_seq":324,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.919983+0000","last_change":"2024-09-11T08:50:06.372303+0000","last_active":"2024-09-11T08:51:33.919983+0000","last_peered":"2024-09-11T08:51:33.919983+0000","last_clean":"2024-09-11T08:51:33.919983+0000","last_became_active":"2024-09-11T08:49:09.901065+0000","last_became_peered":"2024-09-11T08:49:09.901065+0000","last_unstale":"2024-09-11T08:51:33.919983+0000","last_undegraded":"2024-09-11T08:51:33.919983+0000","last_fullsized":"2024-09-11T08:51:33.919983+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'29","last_scrub_stamp":"2024-09-11T08:50:06.372228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:06.372228+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050905730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":276,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.912652+0000","last_change":"2024-09-11T08:50:23.612190+0000","last_active":"2024-09-11T08:51:33.912652+0000","last_peered":"2024-09-11T08:51:33.912652+0000","last_clean":"2024-09-11T08:51:33.912652+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:51:33.912652+0000","last_undegraded":"2024-09-11T08:51:33.912652+0000","last_fullsized":"2024-09-11T08:51:33.912652+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:23.612108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:23.612108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044265900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":292,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.918502+0000","last_change":"2024-09-11T08:51:21.316509+0000","last_active":"2024-09-11T08:51:33.918502+0000","last_peered":"2024-09-11T08:51:33.918502+0000","last_clean":"2024-09-11T08:51:33.918502+0000","last_became_active":"2024-09-11T08:49:09.909306+0000","last_became_peered":"2024-09-11T08:49:09.909306+0000","last_unstale":"2024-09-11T08:51:33.918502+0000","last_undegraded":"2024-09-11T08:51:33.918502+0000","last_fullsized":"2024-09-11T08:51:33.918502+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:21.316318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_clean_scrub_stamp":"2024-09-11T08:51:21.316318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:39.300409+0000","scrub_duration":0.011823264,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":290,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.911498+0000","last_change":"2024-09-11T08:50:19.562077+0000","last_active":"2024-09-11T08:51:33.911498+0000","last_peered":"2024-09-11T08:51:33.911498+0000","last_clean":"2024-09-11T08:51:33.911498+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:51:33.911498+0000","last_undegraded":"2024-09-11T08:51:33.911498+0000","last_fullsized":"2024-09-11T08:51:33.911498+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:19.561995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:50:19.561995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:40.285903+0000","scrub_duration":0.0048084249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"199'81","reported_seq":23,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.914989+0000","last_change":"2024-09-11T08:51:03.021235+0000","last_active":"2024-09-11T08:51:33.914989+0000","last_peered":"2024-09-11T08:51:33.914989+0000","last_clean":"2024-09-11T08:51:33.914989+0000","last_became_active":"2024-09-11T08:51:03.020785+0000","last_became_peered":"2024-09-11T08:51:03.020785+0000","last_unstale":"2024-09-11T08:51:33.914989+0000","last_undegraded":"2024-09-11T08:51:33.914989+0000","last_fullsized":"2024-09-11T08:51:33.914989+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"264'10555","reported_seq":10632,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.357370+0000","last_change":"2024-09-11T08:50:26.529551+0000","last_active":"2024-09-11T08:51:41.357370+0000","last_peered":"2024-09-11T08:51:41.357370+0000","last_clean":"2024-09-11T08:51:41.357370+0000","last_became_active":"2024-09-11T08:50:26.526862+0000","last_became_peered":"2024-09-11T08:50:26.526862+0000","last_unstale":"2024-09-11T08:51:41.357370+0000","last_undegraded":"2024-09-11T08:51:41.357370+0000","last_fullsized":"2024-09-11T08:51:41.357370+0000","mapping_epoch":210,"log_start":"227'5828","ondisk_log_start":"227'5828","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4727,"log_dups_size":0,"ondisk_log_size":4727,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145096704,"num_objects":2214,"num_object_clones":0,"num_object_copies":6642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6758,"num_write_kb":287104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"200'50","reported_seq":506,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:37.825574+0000","last_change":"2024-09-11T08:50:17.612220+0000","last_active":"2024-09-11T08:51:37.825574+0000","last_peered":"2024-09-11T08:51:37.825574+0000","last_clean":"2024-09-11T08:51:37.825574+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:51:37.825574+0000","last_undegraded":"2024-09-11T08:51:37.825574+0000","last_fullsized":"2024-09-11T08:51:37.825574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'50","last_scrub_stamp":"2024-09-11T08:50:17.611978+0000","last_deep_scrub":"200'50","last_deep_scrub_stamp":"2024-09-11T08:50:17.611978+0000","last_clean_scrub_stamp":"2024-09-11T08:50:17.611978+0000","objects_scrubbed":10,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077394660000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":277,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.925713+0000","last_change":"2024-09-11T08:51:33.010221+0000","last_active":"2024-09-11T08:51:33.925713+0000","last_peered":"2024-09-11T08:51:33.925713+0000","last_clean":"2024-09-11T08:51:33.925713+0000","last_became_active":"2024-09-11T08:49:09.907850+0000","last_became_peered":"2024-09-11T08:49:09.907850+0000","last_unstale":"2024-09-11T08:51:33.925713+0000","last_undegraded":"2024-09-11T08:51:33.925713+0000","last_fullsized":"2024-09-11T08:51:33.925713+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:33.010134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_clean_scrub_stamp":"2024-09-11T08:51:33.010134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:47.514215+0000","scrub_duration":0.0033959530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":307,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916412+0000","last_change":"2024-09-11T08:51:19.322458+0000","last_active":"2024-09-11T08:51:33.916412+0000","last_peered":"2024-09-11T08:51:33.916412+0000","last_clean":"2024-09-11T08:51:33.916412+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:51:33.916412+0000","last_undegraded":"2024-09-11T08:51:33.916412+0000","last_fullsized":"2024-09-11T08:51:33.916412+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:51:19.321966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:51:19.321966+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:24.421216+0000","scrub_duration":0.0077550900000000001,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":76,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.911818+0000","last_change":"2024-09-11T08:50:24.593208+0000","last_active":"2024-09-11T08:51:33.911818+0000","last_peered":"2024-09-11T08:51:33.911818+0000","last_clean":"2024-09-11T08:51:33.911818+0000","last_became_active":"2024-09-11T08:49:09.903614+0000","last_became_peered":"2024-09-11T08:49:09.903614+0000","last_unstale":"2024-09-11T08:51:33.911818+0000","last_undegraded":"2024-09-11T08:51:33.911818+0000","last_fullsized":"2024-09-11T08:51:33.911818+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:24.593021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:24.593021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022715140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":250,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916603+0000","last_change":"2024-09-11T08:50:02.248519+0000","last_active":"2024-09-11T08:51:33.916603+0000","last_peered":"2024-09-11T08:51:33.916603+0000","last_clean":"2024-09-11T08:51:33.916603+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:51:33.916603+0000","last_undegraded":"2024-09-11T08:51:33.916603+0000","last_fullsized":"2024-09-11T08:51:33.916603+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:02.248447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:02.248447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029743439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"199'81","reported_seq":348,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.915040+0000","last_change":"2024-09-11T08:51:03.025436+0000","last_active":"2024-09-11T08:51:33.915040+0000","last_peered":"2024-09-11T08:51:33.915040+0000","last_clean":"2024-09-11T08:51:33.915040+0000","last_became_active":"2024-09-11T08:51:03.024999+0000","last_became_peered":"2024-09-11T08:51:03.024999+0000","last_unstale":"2024-09-11T08:51:33.915040+0000","last_undegraded":"2024-09-11T08:51:33.915040+0000","last_fullsized":"2024-09-11T08:51:33.915040+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"264'13462","reported_seq":13734,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.436116+0000","last_change":"2024-09-11T08:51:37.038018+0000","last_active":"2024-09-11T08:51:41.436116+0000","last_peered":"2024-09-11T08:51:41.436116+0000","last_clean":"2024-09-11T08:51:41.436116+0000","last_became_active":"2024-09-11T08:49:05.872392+0000","last_became_peered":"2024-09-11T08:49:05.872392+0000","last_unstale":"2024-09-11T08:51:41.436116+0000","last_undegraded":"2024-09-11T08:51:41.436116+0000","last_fullsized":"2024-09-11T08:51:41.436116+0000","mapping_epoch":206,"log_start":"241'9465","ondisk_log_start":"241'9465","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'13123","last_scrub_stamp":"2024-09-11T08:51:37.037928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:51:37.037928+0000","objects_scrubbed":4937,"log_size":3997,"log_dups_size":0,"ondisk_log_size":3997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:00.936144+0000","scrub_duration":1.6254521159999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297140224,"num_objects":4534,"num_object_clones":0,"num_object_copies":13602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13462,"num_write_kb":575872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"200'17","reported_seq":329,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.911997+0000","last_change":"2024-09-11T08:50:15.565377+0000","last_active":"2024-09-11T08:51:33.911997+0000","last_peered":"2024-09-11T08:51:33.911997+0000","last_clean":"2024-09-11T08:51:33.911997+0000","last_became_active":"2024-09-11T08:49:09.902884+0000","last_became_peered":"2024-09-11T08:49:09.902884+0000","last_unstale":"2024-09-11T08:51:33.911997+0000","last_undegraded":"2024-09-11T08:51:33.911997+0000","last_fullsized":"2024-09-11T08:51:33.911997+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'17","last_scrub_stamp":"2024-09-11T08:50:15.565290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:15.565290+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060447950000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":292,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.911400+0000","last_change":"2024-09-11T08:49:23.588233+0000","last_active":"2024-09-11T08:51:33.911400+0000","last_peered":"2024-09-11T08:51:33.911400+0000","last_clean":"2024-09-11T08:51:33.911400+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:51:33.911400+0000","last_undegraded":"2024-09-11T08:51:33.911400+0000","last_fullsized":"2024-09-11T08:51:33.911400+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:23.588182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:23.588182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0010197089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"263'2","reported_seq":324,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:39.038411+0000","last_change":"2024-09-11T08:51:39.038357+0000","last_active":"2024-09-11T08:51:39.038411+0000","last_peered":"2024-09-11T08:51:39.038411+0000","last_clean":"2024-09-11T08:51:39.038411+0000","last_became_active":"2024-09-11T08:49:09.909804+0000","last_became_peered":"2024-09-11T08:49:09.909804+0000","last_unstale":"2024-09-11T08:51:39.038411+0000","last_undegraded":"2024-09-11T08:51:39.038411+0000","last_fullsized":"2024-09-11T08:51:39.038411+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'2","last_scrub_stamp":"2024-09-11T08:51:39.038303+0000","last_deep_scrub":"263'2","last_deep_scrub_stamp":"2024-09-11T08:51:39.038303+0000","last_clean_scrub_stamp":"2024-09-11T08:51:39.038303+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:54.497753+0000","scrub_duration":0.0087343769999999998,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":288,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.919355+0000","last_change":"2024-09-11T08:51:26.072381+0000","last_active":"2024-09-11T08:51:33.919355+0000","last_peered":"2024-09-11T08:51:33.919355+0000","last_clean":"2024-09-11T08:51:33.919355+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:51:33.919355+0000","last_undegraded":"2024-09-11T08:51:33.919355+0000","last_fullsized":"2024-09-11T08:51:33.919355+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:15.477483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_clean_scrub_stamp":"2024-09-11T08:49:15.477483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049012409999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":290,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916825+0000","last_change":"2024-09-11T08:51:31.465914+0000","last_active":"2024-09-11T08:51:33.916825+0000","last_peered":"2024-09-11T08:51:33.916825+0000","last_clean":"2024-09-11T08:51:33.916825+0000","last_became_active":"2024-09-11T08:46:01.983095+0000","last_became_peered":"2024-09-11T08:46:01.983095+0000","last_unstale":"2024-09-11T08:51:33.916825+0000","last_undegraded":"2024-09-11T08:51:33.916825+0000","last_fullsized":"2024-09-11T08:51:33.916825+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:31.465384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:51:31.465384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:54.378772+0000","scrub_duration":0.0052547410000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"199'81","reported_seq":286,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.912488+0000","last_change":"2024-09-11T08:51:03.027587+0000","last_active":"2024-09-11T08:51:33.912488+0000","last_peered":"2024-09-11T08:51:33.912488+0000","last_clean":"2024-09-11T08:51:33.912488+0000","last_became_active":"2024-09-11T08:51:03.027281+0000","last_became_peered":"2024-09-11T08:51:03.027281+0000","last_unstale":"2024-09-11T08:51:33.912488+0000","last_undegraded":"2024-09-11T08:51:33.912488+0000","last_fullsized":"2024-09-11T08:51:33.912488+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"264'13347","reported_seq":13709,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.210928+0000","last_change":"2024-09-11T08:50:22.510256+0000","last_active":"2024-09-11T08:51:41.210928+0000","last_peered":"2024-09-11T08:51:41.210928+0000","last_clean":"2024-09-11T08:51:41.210928+0000","last_became_active":"2024-09-11T08:49:09.909429+0000","last_became_peered":"2024-09-11T08:49:09.909429+0000","last_unstale":"2024-09-11T08:51:41.210928+0000","last_undegraded":"2024-09-11T08:51:41.210928+0000","last_fullsized":"2024-09-11T08:51:41.210928+0000","mapping_epoch":210,"log_start":"240'8340","ondisk_log_start":"240'8340","created":206,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'7271","last_scrub_stamp":"2024-09-11T08:50:22.510171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:50:22.510171+0000","objects_scrubbed":7171,"log_size":5007,"log_dups_size":0,"ondisk_log_size":5007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:45.661528+0000","scrub_duration":2.357543234,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302055424,"num_objects":4609,"num_object_clones":0,"num_object_copies":13827,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13347,"num_write_kb":574592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"200'24","reported_seq":315,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.913269+0000","last_change":"2024-09-11T08:51:32.758849+0000","last_active":"2024-09-11T08:51:33.913269+0000","last_peered":"2024-09-11T08:51:33.913269+0000","last_clean":"2024-09-11T08:51:33.913269+0000","last_became_active":"2024-09-11T08:49:09.911956+0000","last_became_peered":"2024-09-11T08:49:09.911956+0000","last_unstale":"2024-09-11T08:51:33.913269+0000","last_undegraded":"2024-09-11T08:51:33.913269+0000","last_fullsized":"2024-09-11T08:51:33.913269+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'24","last_scrub_stamp":"2024-09-11T08:51:32.758745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:32.758745+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:52.667203+0000","scrub_duration":0.0034796879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":76,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.925676+0000","last_change":"2024-09-11T08:50:12.796879+0000","last_active":"2024-09-11T08:51:33.925676+0000","last_peered":"2024-09-11T08:51:33.925676+0000","last_clean":"2024-09-11T08:51:33.925676+0000","last_became_active":"2024-09-11T08:49:09.909803+0000","last_became_peered":"2024-09-11T08:49:09.909803+0000","last_unstale":"2024-09-11T08:51:33.925676+0000","last_undegraded":"2024-09-11T08:51:33.925676+0000","last_fullsized":"2024-09-11T08:51:33.925676+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:12.796804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:12.796804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056357270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":1072,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:40.349952+0000","last_change":"2024-09-11T08:49:27.805277+0000","last_active":"2024-09-11T08:51:40.349952+0000","last_peered":"2024-09-11T08:51:40.349952+0000","last_clean":"2024-09-11T08:51:40.349952+0000","last_became_active":"2024-09-11T08:49:09.912659+0000","last_became_peered":"2024-09-11T08:49:09.912659+0000","last_unstale":"2024-09-11T08:51:40.349952+0000","last_undegraded":"2024-09-11T08:51:40.349952+0000","last_fullsized":"2024-09-11T08:51:40.349952+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:49:27.805050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:27.805050+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046228930000000003,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":773,"num_read_kb":768,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":278,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.920635+0000","last_change":"2024-09-11T08:51:32.443796+0000","last_active":"2024-09-11T08:51:33.920635+0000","last_peered":"2024-09-11T08:51:33.920635+0000","last_clean":"2024-09-11T08:51:33.920635+0000","last_became_active":"2024-09-11T08:49:09.903283+0000","last_became_peered":"2024-09-11T08:49:09.903283+0000","last_unstale":"2024-09-11T08:51:33.920635+0000","last_undegraded":"2024-09-11T08:51:33.920635+0000","last_fullsized":"2024-09-11T08:51:33.920635+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:32.443727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:32.443727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:35.801037+0000","scrub_duration":0.0056199379999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":270,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:35.779910+0000","last_change":"2024-09-11T08:51:35.779870+0000","last_active":"2024-09-11T08:51:35.779910+0000","last_peered":"2024-09-11T08:51:35.779910+0000","last_clean":"2024-09-11T08:51:35.779910+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:51:35.779910+0000","last_undegraded":"2024-09-11T08:51:35.779910+0000","last_fullsized":"2024-09-11T08:51:35.779910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:35.779796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:51:35.779796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:36.119459+0000","scrub_duration":0.0048768270000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"264'10368","reported_seq":10455,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.792965+0000","last_change":"2024-09-11T08:50:26.518231+0000","last_active":"2024-09-11T08:51:41.792965+0000","last_peered":"2024-09-11T08:51:41.792965+0000","last_clean":"2024-09-11T08:51:41.792965+0000","last_became_active":"2024-09-11T08:50:26.515366+0000","last_became_peered":"2024-09-11T08:50:26.515366+0000","last_unstale":"2024-09-11T08:51:41.792965+0000","last_undegraded":"2024-09-11T08:51:41.792965+0000","last_fullsized":"2024-09-11T08:51:41.792965+0000","mapping_epoch":210,"log_start":"227'6051","ondisk_log_start":"227'6051","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4317,"log_dups_size":0,"ondisk_log_size":4317,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147947520,"num_objects":2257,"num_object_clones":0,"num_object_copies":6771,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6609,"num_write_kb":283744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"201'27","reported_seq":171,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.918729+0000","last_change":"2024-09-11T08:50:16.136431+0000","last_active":"2024-09-11T08:51:33.918729+0000","last_peered":"2024-09-11T08:51:33.918729+0000","last_clean":"2024-09-11T08:51:33.918729+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:51:33.918729+0000","last_undegraded":"2024-09-11T08:51:33.918729+0000","last_fullsized":"2024-09-11T08:51:33.918729+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'27","last_scrub_stamp":"2024-09-11T08:50:16.136356+0000","last_deep_scrub":"201'27","last_deep_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_clean_scrub_stamp":"2024-09-11T08:50:16.136356+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:41.100937+0000","scrub_duration":0.0074356769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":265,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.918276+0000","last_change":"2024-09-11T08:50:08.276283+0000","last_active":"2024-09-11T08:51:33.918276+0000","last_peered":"2024-09-11T08:51:33.918276+0000","last_clean":"2024-09-11T08:51:33.918276+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:51:33.918276+0000","last_undegraded":"2024-09-11T08:51:33.918276+0000","last_fullsized":"2024-09-11T08:51:33.918276+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:50:08.276232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:08.276232+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018088246999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"199'81","reported_seq":339,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.920390+0000","last_change":"2024-09-11T08:51:00.928498+0000","last_active":"2024-09-11T08:51:33.920390+0000","last_peered":"2024-09-11T08:51:33.920390+0000","last_clean":"2024-09-11T08:51:33.920390+0000","last_became_active":"2024-09-11T08:51:00.928184+0000","last_became_peered":"2024-09-11T08:51:00.928184+0000","last_unstale":"2024-09-11T08:51:33.920390+0000","last_undegraded":"2024-09-11T08:51:33.920390+0000","last_fullsized":"2024-09-11T08:51:33.920390+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":272,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.915814+0000","last_change":"2024-09-11T08:50:15.173370+0000","last_active":"2024-09-11T08:51:33.915814+0000","last_peered":"2024-09-11T08:51:33.915814+0000","last_clean":"2024-09-11T08:51:33.915814+0000","last_became_active":"2024-09-11T08:38:42.986106+0000","last_became_peered":"2024-09-11T08:38:42.986106+0000","last_unstale":"2024-09-11T08:51:33.915814+0000","last_undegraded":"2024-09-11T08:51:33.915814+0000","last_fullsized":"2024-09-11T08:51:33.915814+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:15.173273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:15.173273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003216324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":266,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.920513+0000","last_change":"2024-09-11T08:50:13.347446+0000","last_active":"2024-09-11T08:51:33.920513+0000","last_peered":"2024-09-11T08:51:33.920513+0000","last_clean":"2024-09-11T08:51:33.920513+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:51:33.920513+0000","last_undegraded":"2024-09-11T08:51:33.920513+0000","last_fullsized":"2024-09-11T08:51:33.920513+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:13.347374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:13.347374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062539859999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"264'10508","reported_seq":10585,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.380421+0000","last_change":"2024-09-11T08:50:26.527617+0000","last_active":"2024-09-11T08:51:41.380421+0000","last_peered":"2024-09-11T08:51:41.380421+0000","last_clean":"2024-09-11T08:51:41.380421+0000","last_became_active":"2024-09-11T08:50:26.523754+0000","last_became_peered":"2024-09-11T08:50:26.523754+0000","last_unstale":"2024-09-11T08:51:41.380421+0000","last_undegraded":"2024-09-11T08:51:41.380421+0000","last_fullsized":"2024-09-11T08:51:41.380421+0000","mapping_epoch":206,"log_start":"227'5706","ondisk_log_start":"227'5706","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":151126016,"num_objects":2306,"num_object_clones":0,"num_object_copies":6918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6720,"num_write_kb":288832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"200'19","reported_seq":355,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.929518+0000","last_change":"2024-09-11T08:51:21.979895+0000","last_active":"2024-09-11T08:51:33.929518+0000","last_peered":"2024-09-11T08:51:33.929518+0000","last_clean":"2024-09-11T08:51:33.929518+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:51:33.929518+0000","last_undegraded":"2024-09-11T08:51:33.929518+0000","last_fullsized":"2024-09-11T08:51:33.929518+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:21.979796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:21.979796+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:49.011962+0000","scrub_duration":0.0045321210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":257,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:38.083294+0000","last_change":"2024-09-11T08:51:38.083246+0000","last_active":"2024-09-11T08:51:38.083294+0000","last_peered":"2024-09-11T08:51:38.083294+0000","last_clean":"2024-09-11T08:51:38.083294+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:51:38.083294+0000","last_undegraded":"2024-09-11T08:51:38.083294+0000","last_fullsized":"2024-09-11T08:51:38.083294+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:38.083168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:51:38.083168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:55.119368+0000","scrub_duration":0.0029716930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":265,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.920293+0000","last_change":"2024-09-11T08:49:29.269384+0000","last_active":"2024-09-11T08:51:33.920293+0000","last_peered":"2024-09-11T08:51:33.920293+0000","last_clean":"2024-09-11T08:51:33.920293+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:51:33.920293+0000","last_undegraded":"2024-09-11T08:51:33.920293+0000","last_fullsized":"2024-09-11T08:51:33.920293+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:29.269302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:49:29.269302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0083588919999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":270,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.917181+0000","last_change":"2024-09-11T08:50:06.234445+0000","last_active":"2024-09-11T08:51:33.917181+0000","last_peered":"2024-09-11T08:51:33.917181+0000","last_clean":"2024-09-11T08:51:33.917181+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:51:33.917181+0000","last_undegraded":"2024-09-11T08:51:33.917181+0000","last_fullsized":"2024-09-11T08:51:33.917181+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:06.234119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:06.234119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034529980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"199'81","reported_seq":30,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.914423+0000","last_change":"2024-09-11T08:51:03.042396+0000","last_active":"2024-09-11T08:51:33.914423+0000","last_peered":"2024-09-11T08:51:33.914423+0000","last_clean":"2024-09-11T08:51:33.914423+0000","last_became_active":"2024-09-11T08:51:03.020941+0000","last_became_peered":"2024-09-11T08:51:03.020941+0000","last_unstale":"2024-09-11T08:51:33.914423+0000","last_undegraded":"2024-09-11T08:51:33.914423+0000","last_fullsized":"2024-09-11T08:51:33.914423+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":316358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.0","version":"264'10511","reported_seq":10586,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.681603+0000","last_change":"2024-09-11T08:50:26.528932+0000","last_active":"2024-09-11T08:51:41.681603+0000","last_peered":"2024-09-11T08:51:41.681603+0000","last_clean":"2024-09-11T08:51:41.681603+0000","last_became_active":"2024-09-11T08:50:26.524700+0000","last_became_peered":"2024-09-11T08:50:26.524700+0000","last_unstale":"2024-09-11T08:51:41.681603+0000","last_undegraded":"2024-09-11T08:51:41.681603+0000","last_fullsized":"2024-09-11T08:51:41.681603+0000","mapping_epoch":206,"log_start":"227'6049","ondisk_log_start":"227'6049","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4462,"log_dups_size":0,"ondisk_log_size":4462,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149684224,"num_objects":2284,"num_object_clones":0,"num_object_copies":6852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6750,"num_write_kb":289088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"200'31","reported_seq":314,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.929120+0000","last_change":"2024-09-11T08:51:24.989802+0000","last_active":"2024-09-11T08:51:33.929120+0000","last_peered":"2024-09-11T08:51:33.929120+0000","last_clean":"2024-09-11T08:51:33.929120+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:51:33.929120+0000","last_undegraded":"2024-09-11T08:51:33.929120+0000","last_fullsized":"2024-09-11T08:51:33.929120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'31","last_scrub_stamp":"2024-09-11T08:51:24.989715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:24.989715+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:38.972392+0000","scrub_duration":0.0037630540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":296,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.918791+0000","last_change":"2024-09-11T08:51:30.184847+0000","last_active":"2024-09-11T08:51:33.918791+0000","last_peered":"2024-09-11T08:51:33.918791+0000","last_clean":"2024-09-11T08:51:33.918791+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:51:33.918791+0000","last_undegraded":"2024-09-11T08:51:33.918791+0000","last_fullsized":"2024-09-11T08:51:33.918791+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:30.184759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_clean_scrub_stamp":"2024-09-11T08:51:30.184759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:41.855053+0000","scrub_duration":0.003639691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":282,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.912377+0000","last_change":"2024-09-11T08:51:22.782518+0000","last_active":"2024-09-11T08:51:33.912377+0000","last_peered":"2024-09-11T08:51:33.912377+0000","last_clean":"2024-09-11T08:51:33.912377+0000","last_became_active":"2024-09-11T08:49:09.903589+0000","last_became_peered":"2024-09-11T08:49:09.903589+0000","last_unstale":"2024-09-11T08:51:33.912377+0000","last_undegraded":"2024-09-11T08:51:33.912377+0000","last_fullsized":"2024-09-11T08:51:33.912377+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:22.782448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:22.782448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:24.581122+0000","scrub_duration":0.0071094909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":299,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.919433+0000","last_change":"2024-09-11T08:51:31.053086+0000","last_active":"2024-09-11T08:51:33.919433+0000","last_peered":"2024-09-11T08:51:33.919433+0000","last_clean":"2024-09-11T08:51:33.919433+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:51:33.919433+0000","last_undegraded":"2024-09-11T08:51:33.919433+0000","last_fullsized":"2024-09-11T08:51:33.919433+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:31.053010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:51:31.053010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:58.526195+0000","scrub_duration":0.002015668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"199'81","reported_seq":348,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.917470+0000","last_change":"2024-09-11T08:51:03.021042+0000","last_active":"2024-09-11T08:51:33.917470+0000","last_peered":"2024-09-11T08:51:33.917470+0000","last_clean":"2024-09-11T08:51:33.917470+0000","last_became_active":"2024-09-11T08:51:03.020720+0000","last_became_peered":"2024-09-11T08:51:03.020720+0000","last_unstale":"2024-09-11T08:51:33.917470+0000","last_undegraded":"2024-09-11T08:51:33.917470+0000","last_fullsized":"2024-09-11T08:51:33.917470+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1","version":"264'10507","reported_seq":10582,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.813547+0000","last_change":"2024-09-11T08:50:26.528936+0000","last_active":"2024-09-11T08:51:41.813547+0000","last_peered":"2024-09-11T08:51:41.813547+0000","last_clean":"2024-09-11T08:51:41.813547+0000","last_became_active":"2024-09-11T08:50:26.523855+0000","last_became_peered":"2024-09-11T08:50:26.523855+0000","last_unstale":"2024-09-11T08:51:41.813547+0000","last_undegraded":"2024-09-11T08:51:41.813547+0000","last_fullsized":"2024-09-11T08:51:41.813547+0000","mapping_epoch":206,"log_start":"227'6225","ondisk_log_start":"227'6225","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4282,"log_dups_size":0,"ondisk_log_size":4282,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150667264,"num_objects":2299,"num_object_clones":0,"num_object_copies":6897,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6703,"num_write_kb":288064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"200'20","reported_seq":325,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.927860+0000","last_change":"2024-09-11T08:51:12.027457+0000","last_active":"2024-09-11T08:51:33.927860+0000","last_peered":"2024-09-11T08:51:33.927860+0000","last_clean":"2024-09-11T08:51:33.927860+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:51:33.927860+0000","last_undegraded":"2024-09-11T08:51:33.927860+0000","last_fullsized":"2024-09-11T08:51:33.927860+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-11T08:48:21.885681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:21.885681+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051165580000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":266,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916544+0000","last_change":"2024-09-11T08:50:04.191785+0000","last_active":"2024-09-11T08:51:33.916544+0000","last_peered":"2024-09-11T08:51:33.916544+0000","last_clean":"2024-09-11T08:51:33.916544+0000","last_became_active":"2024-09-11T08:49:09.911101+0000","last_became_peered":"2024-09-11T08:49:09.911101+0000","last_unstale":"2024-09-11T08:51:33.916544+0000","last_undegraded":"2024-09-11T08:51:33.916544+0000","last_fullsized":"2024-09-11T08:51:33.916544+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:04.191702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:04.191702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034932320000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":459,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:37.824438+0000","last_change":"2024-09-11T08:51:24.167737+0000","last_active":"2024-09-11T08:51:37.824438+0000","last_peered":"2024-09-11T08:51:37.824438+0000","last_clean":"2024-09-11T08:51:37.824438+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:51:37.824438+0000","last_undegraded":"2024-09-11T08:51:37.824438+0000","last_fullsized":"2024-09-11T08:51:37.824438+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:51:24.167652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:24.167652+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:52.001173+0000","scrub_duration":0.003586816,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":251,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.929405+0000","last_change":"2024-09-11T08:50:05.815267+0000","last_active":"2024-09-11T08:51:33.929405+0000","last_peered":"2024-09-11T08:51:33.929405+0000","last_clean":"2024-09-11T08:51:33.929405+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:51:33.929405+0000","last_undegraded":"2024-09-11T08:51:33.929405+0000","last_fullsized":"2024-09-11T08:51:33.929405+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:05.815178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:50:05.815178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0078777670000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"264'10360","reported_seq":10432,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.419919+0000","last_change":"2024-09-11T08:50:26.533353+0000","last_active":"2024-09-11T08:51:41.419919+0000","last_peered":"2024-09-11T08:51:41.419919+0000","last_clean":"2024-09-11T08:51:41.419919+0000","last_became_active":"2024-09-11T08:50:26.531198+0000","last_became_peered":"2024-09-11T08:50:26.531198+0000","last_unstale":"2024-09-11T08:51:41.419919+0000","last_undegraded":"2024-09-11T08:51:41.419919+0000","last_fullsized":"2024-09-11T08:51:41.419919+0000","mapping_epoch":206,"log_start":"225'5288","ondisk_log_start":"225'5288","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":5072,"log_dups_size":0,"ondisk_log_size":5072,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148275200,"num_objects":2262,"num_object_clones":0,"num_object_copies":6786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6634,"num_write_kb":284704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"200'15","reported_seq":290,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.912430+0000","last_change":"2024-09-11T08:49:09.925808+0000","last_active":"2024-09-11T08:51:33.912430+0000","last_peered":"2024-09-11T08:51:33.912430+0000","last_clean":"2024-09-11T08:51:33.912430+0000","last_became_active":"2024-09-11T08:49:09.915666+0000","last_became_peered":"2024-09-11T08:49:09.915666+0000","last_unstale":"2024-09-11T08:51:33.912430+0000","last_undegraded":"2024-09-11T08:51:33.912430+0000","last_fullsized":"2024-09-11T08:51:33.912430+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-11T08:48:41.787219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.787219+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060192070000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":269,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.919418+0000","last_change":"2024-09-11T08:49:30.234576+0000","last_active":"2024-09-11T08:51:33.919418+0000","last_peered":"2024-09-11T08:51:33.919418+0000","last_clean":"2024-09-11T08:51:33.919418+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:51:33.919418+0000","last_undegraded":"2024-09-11T08:51:33.919418+0000","last_fullsized":"2024-09-11T08:51:33.919418+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:30.234078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:49:30.234078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025717829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":286,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.920040+0000","last_change":"2024-09-11T08:51:22.145686+0000","last_active":"2024-09-11T08:51:33.920040+0000","last_peered":"2024-09-11T08:51:33.920040+0000","last_clean":"2024-09-11T08:51:33.920040+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:51:33.920040+0000","last_undegraded":"2024-09-11T08:51:33.920040+0000","last_fullsized":"2024-09-11T08:51:33.920040+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:22.145548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:22.145548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:40.410287+0000","scrub_duration":0.0036550889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":69,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.917545+0000","last_change":"2024-09-11T08:51:13.387398+0000","last_active":"2024-09-11T08:51:33.917545+0000","last_peered":"2024-09-11T08:51:33.917545+0000","last_clean":"2024-09-11T08:51:33.917545+0000","last_became_active":"2024-09-11T08:49:09.912556+0000","last_became_peered":"2024-09-11T08:49:09.912556+0000","last_unstale":"2024-09-11T08:51:33.917545+0000","last_undegraded":"2024-09-11T08:51:33.917545+0000","last_fullsized":"2024-09-11T08:51:33.917545+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:47.699453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:47:47.699453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"264'10488","reported_seq":10564,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.418876+0000","last_change":"2024-09-11T08:50:26.521672+0000","last_active":"2024-09-11T08:51:41.418876+0000","last_peered":"2024-09-11T08:51:41.418876+0000","last_clean":"2024-09-11T08:51:41.418876+0000","last_became_active":"2024-09-11T08:50:26.519761+0000","last_became_peered":"2024-09-11T08:50:26.519761+0000","last_unstale":"2024-09-11T08:51:41.418876+0000","last_undegraded":"2024-09-11T08:51:41.418876+0000","last_fullsized":"2024-09-11T08:51:41.418876+0000","mapping_epoch":206,"log_start":"226'5547","ondisk_log_start":"226'5547","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4941,"log_dups_size":0,"ondisk_log_size":4941,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146407424,"num_objects":2234,"num_object_clones":0,"num_object_copies":6702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6730,"num_write_kb":286848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"200'35","reported_seq":183,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:35.255175+0000","last_change":"2024-09-11T08:51:35.255093+0000","last_active":"2024-09-11T08:51:35.255175+0000","last_peered":"2024-09-11T08:51:35.255175+0000","last_clean":"2024-09-11T08:51:35.255175+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:51:35.255175+0000","last_undegraded":"2024-09-11T08:51:35.255175+0000","last_fullsized":"2024-09-11T08:51:35.255175+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'35","last_scrub_stamp":"2024-09-11T08:51:35.255013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:35.255013+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:49.222297+0000","scrub_duration":0.0048258839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":276,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.917207+0000","last_change":"2024-09-11T08:50:16.196911+0000","last_active":"2024-09-11T08:51:33.917207+0000","last_peered":"2024-09-11T08:51:33.917207+0000","last_clean":"2024-09-11T08:51:33.917207+0000","last_became_active":"2024-09-11T08:38:38.951683+0000","last_became_peered":"2024-09-11T08:38:38.951683+0000","last_unstale":"2024-09-11T08:51:33.917207+0000","last_undegraded":"2024-09-11T08:51:33.917207+0000","last_fullsized":"2024-09-11T08:51:33.917207+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:16.196521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:16.196521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:43.309271+0000","scrub_duration":0.0027210310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":305,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:35.022393+0000","last_change":"2024-09-11T08:51:35.022334+0000","last_active":"2024-09-11T08:51:35.022393+0000","last_peered":"2024-09-11T08:51:35.022393+0000","last_clean":"2024-09-11T08:51:35.022393+0000","last_became_active":"2024-09-11T08:49:09.907542+0000","last_became_peered":"2024-09-11T08:49:09.907542+0000","last_unstale":"2024-09-11T08:51:35.022393+0000","last_undegraded":"2024-09-11T08:51:35.022393+0000","last_fullsized":"2024-09-11T08:51:35.022393+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:35.022256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:35.022256+0000","last_clean_scrub_stamp":"2024-09-11T08:51:35.022256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:41.307198+0000","scrub_duration":0.0058631000000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":263,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.920314+0000","last_change":"2024-09-11T08:51:28.410954+0000","last_active":"2024-09-11T08:51:33.920314+0000","last_peered":"2024-09-11T08:51:33.920314+0000","last_clean":"2024-09-11T08:51:33.920314+0000","last_became_active":"2024-09-11T08:38:44.993206+0000","last_became_peered":"2024-09-11T08:38:44.993206+0000","last_unstale":"2024-09-11T08:51:33.920314+0000","last_undegraded":"2024-09-11T08:51:33.920314+0000","last_fullsized":"2024-09-11T08:51:33.920314+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:28.410895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:51:28.410895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:45.805224+0000","scrub_duration":0.0016251970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"264'10573","reported_seq":10649,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.720730+0000","last_change":"2024-09-11T08:50:26.523606+0000","last_active":"2024-09-11T08:51:41.720730+0000","last_peered":"2024-09-11T08:51:41.720730+0000","last_clean":"2024-09-11T08:51:41.720730+0000","last_became_active":"2024-09-11T08:50:26.521352+0000","last_became_peered":"2024-09-11T08:50:26.521352+0000","last_unstale":"2024-09-11T08:51:41.720730+0000","last_undegraded":"2024-09-11T08:51:41.720730+0000","last_fullsized":"2024-09-11T08:51:41.720730+0000","mapping_epoch":206,"log_start":"227'6168","ondisk_log_start":"227'6168","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4405,"log_dups_size":0,"ondisk_log_size":4405,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145653760,"num_objects":2222,"num_object_clones":0,"num_object_copies":6666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6764,"num_write_kb":287584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"200'18","reported_seq":287,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.926769+0000","last_change":"2024-09-11T08:50:03.790643+0000","last_active":"2024-09-11T08:51:33.926769+0000","last_peered":"2024-09-11T08:51:33.926769+0000","last_clean":"2024-09-11T08:51:33.926769+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:51:33.926769+0000","last_undegraded":"2024-09-11T08:51:33.926769+0000","last_fullsized":"2024-09-11T08:51:33.926769+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'18","last_scrub_stamp":"2024-09-11T08:50:03.790561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:03.790561+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.050104567000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":276,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916139+0000","last_change":"2024-09-11T08:51:24.332051+0000","last_active":"2024-09-11T08:51:33.916139+0000","last_peered":"2024-09-11T08:51:33.916139+0000","last_clean":"2024-09-11T08:51:33.916139+0000","last_became_active":"2024-09-11T08:38:38.951035+0000","last_became_peered":"2024-09-11T08:38:38.951035+0000","last_unstale":"2024-09-11T08:51:33.916139+0000","last_undegraded":"2024-09-11T08:51:33.916139+0000","last_fullsized":"2024-09-11T08:51:33.916139+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:24.331966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:24.331966+0000","last_clean_scrub_stamp":"2024-09-11T08:51:24.331966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:35.147885+0000","scrub_duration":0.0044900610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":295,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.918406+0000","last_change":"2024-09-11T08:50:06.808228+0000","last_active":"2024-09-11T08:51:33.918406+0000","last_peered":"2024-09-11T08:51:33.918406+0000","last_clean":"2024-09-11T08:51:33.918406+0000","last_became_active":"2024-09-11T08:49:09.907535+0000","last_became_peered":"2024-09-11T08:49:09.907535+0000","last_unstale":"2024-09-11T08:51:33.918406+0000","last_undegraded":"2024-09-11T08:51:33.918406+0000","last_fullsized":"2024-09-11T08:51:33.918406+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:06.808051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:06.808051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0080726240000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":250,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916378+0000","last_change":"2024-09-11T08:50:15.277171+0000","last_active":"2024-09-11T08:51:33.916378+0000","last_peered":"2024-09-11T08:51:33.916378+0000","last_clean":"2024-09-11T08:51:33.916378+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:51:33.916378+0000","last_undegraded":"2024-09-11T08:51:33.916378+0000","last_fullsized":"2024-09-11T08:51:33.916378+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:15.277094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:50:15.277094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004775751,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"264'10440","reported_seq":10517,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.401280+0000","last_change":"2024-09-11T08:50:26.517074+0000","last_active":"2024-09-11T08:51:41.401280+0000","last_peered":"2024-09-11T08:51:41.401280+0000","last_clean":"2024-09-11T08:51:41.401280+0000","last_became_active":"2024-09-11T08:50:26.511250+0000","last_became_peered":"2024-09-11T08:50:26.511250+0000","last_unstale":"2024-09-11T08:51:41.401280+0000","last_undegraded":"2024-09-11T08:51:41.401280+0000","last_fullsized":"2024-09-11T08:51:41.401280+0000","mapping_epoch":210,"log_start":"225'5440","ondisk_log_start":"225'5440","created":206,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":5000,"log_dups_size":0,"ondisk_log_size":5000,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143261696,"num_objects":2186,"num_object_clones":0,"num_object_copies":6558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6704,"num_write_kb":284480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"200'34","reported_seq":372,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916027+0000","last_change":"2024-09-11T08:50:11.325671+0000","last_active":"2024-09-11T08:51:33.916027+0000","last_peered":"2024-09-11T08:51:33.916027+0000","last_clean":"2024-09-11T08:51:33.916027+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:51:33.916027+0000","last_undegraded":"2024-09-11T08:51:33.916027+0000","last_fullsized":"2024-09-11T08:51:33.916027+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:50:11.325616+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:50:11.325616+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023307359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":299,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.917380+0000","last_change":"2024-09-11T08:49:17.884190+0000","last_active":"2024-09-11T08:51:33.917380+0000","last_peered":"2024-09-11T08:51:33.917380+0000","last_clean":"2024-09-11T08:51:33.917380+0000","last_became_active":"2024-09-11T08:49:09.904656+0000","last_became_peered":"2024-09-11T08:49:09.904656+0000","last_unstale":"2024-09-11T08:51:33.917380+0000","last_undegraded":"2024-09-11T08:51:33.917380+0000","last_fullsized":"2024-09-11T08:51:33.917380+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:49:17.884108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:17.884108+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031968909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"168'10","reported_seq":464,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:37.824559+0000","last_change":"2024-09-11T08:51:35.452381+0000","last_active":"2024-09-11T08:51:37.824559+0000","last_peered":"2024-09-11T08:51:37.824559+0000","last_clean":"2024-09-11T08:51:37.824559+0000","last_became_active":"2024-09-11T08:46:01.983055+0000","last_became_peered":"2024-09-11T08:46:01.983055+0000","last_unstale":"2024-09-11T08:51:37.824559+0000","last_undegraded":"2024-09-11T08:51:37.824559+0000","last_fullsized":"2024-09-11T08:51:37.824559+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"168'10","last_scrub_stamp":"2024-09-11T08:51:35.452290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:35.452290+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:49.235850+0000","scrub_duration":0.0071142979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":266,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916064+0000","last_change":"2024-09-11T08:49:18.301286+0000","last_active":"2024-09-11T08:51:33.916064+0000","last_peered":"2024-09-11T08:51:33.916064+0000","last_clean":"2024-09-11T08:51:33.916064+0000","last_became_active":"2024-09-11T08:49:09.906201+0000","last_became_peered":"2024-09-11T08:49:09.906201+0000","last_unstale":"2024-09-11T08:51:33.916064+0000","last_undegraded":"2024-09-11T08:51:33.916064+0000","last_fullsized":"2024-09-11T08:51:33.916064+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:18.301203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:18.301203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032551070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":306,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.920349+0000","last_change":"2024-09-11T08:51:20.276946+0000","last_active":"2024-09-11T08:51:33.920349+0000","last_peered":"2024-09-11T08:51:33.920349+0000","last_clean":"2024-09-11T08:51:33.920349+0000","last_became_active":"2024-09-11T08:49:09.910329+0000","last_became_peered":"2024-09-11T08:49:09.910329+0000","last_unstale":"2024-09-11T08:51:33.920349+0000","last_undegraded":"2024-09-11T08:51:33.920349+0000","last_fullsized":"2024-09-11T08:51:33.920349+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:20.276875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:51:20.276875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:25.035770+0000","scrub_duration":0.014836656,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"200'12","reported_seq":290,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.917400+0000","last_change":"2024-09-11T08:49:15.347679+0000","last_active":"2024-09-11T08:51:33.917400+0000","last_peered":"2024-09-11T08:51:33.917400+0000","last_clean":"2024-09-11T08:51:33.917400+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:51:33.917400+0000","last_undegraded":"2024-09-11T08:51:33.917400+0000","last_fullsized":"2024-09-11T08:51:33.917400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'12","last_scrub_stamp":"2024-09-11T08:49:15.347591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:49:15.347591+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048379369999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":262,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.920046+0000","last_change":"2024-09-11T08:50:07.390163+0000","last_active":"2024-09-11T08:51:33.920046+0000","last_peered":"2024-09-11T08:51:33.920046+0000","last_clean":"2024-09-11T08:51:33.920046+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:51:33.920046+0000","last_undegraded":"2024-09-11T08:51:33.920046+0000","last_fullsized":"2024-09-11T08:51:33.920046+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:07.389973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:07.389973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014729302999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":280,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916756+0000","last_change":"2024-09-11T08:50:08.089714+0000","last_active":"2024-09-11T08:51:33.916756+0000","last_peered":"2024-09-11T08:51:33.916756+0000","last_clean":"2024-09-11T08:51:33.916756+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:51:33.916756+0000","last_undegraded":"2024-09-11T08:51:33.916756+0000","last_fullsized":"2024-09-11T08:51:33.916756+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:08.089015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:08.089015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075245379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"200'15","reported_seq":300,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916681+0000","last_change":"2024-09-11T08:50:24.263773+0000","last_active":"2024-09-11T08:51:33.916681+0000","last_peered":"2024-09-11T08:51:33.916681+0000","last_clean":"2024-09-11T08:51:33.916681+0000","last_became_active":"2024-09-11T08:49:09.916927+0000","last_became_peered":"2024-09-11T08:49:09.916927+0000","last_unstale":"2024-09-11T08:51:33.916681+0000","last_undegraded":"2024-09-11T08:51:33.916681+0000","last_fullsized":"2024-09-11T08:51:33.916681+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:50:24.263662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:24.263662+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052890580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":279,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.917304+0000","last_change":"2024-09-11T08:51:20.368106+0000","last_active":"2024-09-11T08:51:33.917304+0000","last_peered":"2024-09-11T08:51:33.917304+0000","last_clean":"2024-09-11T08:51:33.917304+0000","last_became_active":"2024-09-11T08:38:38.952300+0000","last_became_peered":"2024-09-11T08:38:38.952300+0000","last_unstale":"2024-09-11T08:51:33.917304+0000","last_undegraded":"2024-09-11T08:51:33.917304+0000","last_fullsized":"2024-09-11T08:51:33.917304+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:20.368038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:51:20.368038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:30.905453+0000","scrub_duration":0.011810308,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":252,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.917201+0000","last_change":"2024-09-11T08:50:19.288275+0000","last_active":"2024-09-11T08:51:33.917201+0000","last_peered":"2024-09-11T08:51:33.917201+0000","last_clean":"2024-09-11T08:51:33.917201+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:51:33.917201+0000","last_undegraded":"2024-09-11T08:51:33.917201+0000","last_fullsized":"2024-09-11T08:51:33.917201+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:19.288216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:19.288216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:41.431437+0000","scrub_duration":0.0055736010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":281,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.918737+0000","last_change":"2024-09-11T08:50:17.717047+0000","last_active":"2024-09-11T08:51:33.918737+0000","last_peered":"2024-09-11T08:51:33.918737+0000","last_clean":"2024-09-11T08:51:33.918737+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:51:33.918737+0000","last_undegraded":"2024-09-11T08:51:33.918737+0000","last_fullsized":"2024-09-11T08:51:33.918737+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:17.716977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_clean_scrub_stamp":"2024-09-11T08:50:17.716977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032296289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"264'12590","reported_seq":19950,"reported_epoch":264,"state":"active+clean+scrubbing+deep","last_fresh":"2024-09-11T08:51:41.128248+0000","last_change":"2024-09-11T08:51:29.210904+0000","last_active":"2024-09-11T08:51:41.128248+0000","last_peered":"2024-09-11T08:51:41.128248+0000","last_clean":"2024-09-11T08:51:41.128248+0000","last_became_active":"2024-09-11T08:51:18.598650+0000","last_became_peered":"2024-09-11T08:51:18.598650+0000","last_unstale":"2024-09-11T08:51:41.128248+0000","last_undegraded":"2024-09-11T08:51:41.128248+0000","last_fullsized":"2024-09-11T08:51:41.128248+0000","mapping_epoch":255,"log_start":"239'7634","ondisk_log_start":"239'7634","created":235,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":2241,"log_size":4956,"log_dups_size":0,"ondisk_log_size":4956,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 11s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147947520,"num_objects":2258,"num_object_clones":0,"num_object_copies":6774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6616,"num_write_kb":283936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3577,"num_bytes_recovered":234422272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"200'15","reported_seq":295,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.923248+0000","last_change":"2024-09-11T08:50:26.391719+0000","last_active":"2024-09-11T08:51:33.923248+0000","last_peered":"2024-09-11T08:51:33.923248+0000","last_clean":"2024-09-11T08:51:33.923248+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:51:33.923248+0000","last_undegraded":"2024-09-11T08:51:33.923248+0000","last_fullsized":"2024-09-11T08:51:33.923248+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:50:26.391622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:26.391622+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021692130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":69,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.915948+0000","last_change":"2024-09-11T08:51:14.340049+0000","last_active":"2024-09-11T08:51:33.915948+0000","last_peered":"2024-09-11T08:51:33.915948+0000","last_clean":"2024-09-11T08:51:33.915948+0000","last_became_active":"2024-09-11T08:49:09.912417+0000","last_became_peered":"2024-09-11T08:49:09.912417+0000","last_unstale":"2024-09-11T08:51:33.915948+0000","last_undegraded":"2024-09-11T08:51:33.915948+0000","last_fullsized":"2024-09-11T08:51:33.915948+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:47:45.751518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:47:45.751518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":416,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:37.824365+0000","last_change":"2024-09-11T08:50:16.313863+0000","last_active":"2024-09-11T08:51:37.824365+0000","last_peered":"2024-09-11T08:51:37.824365+0000","last_clean":"2024-09-11T08:51:37.824365+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:51:37.824365+0000","last_undegraded":"2024-09-11T08:51:37.824365+0000","last_fullsized":"2024-09-11T08:51:37.824365+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:50:16.313779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:16.313779+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:39.660142+0000","scrub_duration":0.0049551839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":244,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.929530+0000","last_change":"2024-09-11T08:51:27.009371+0000","last_active":"2024-09-11T08:51:33.929530+0000","last_peered":"2024-09-11T08:51:33.929530+0000","last_clean":"2024-09-11T08:51:33.929530+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:51:33.929530+0000","last_undegraded":"2024-09-11T08:51:33.929530+0000","last_fullsized":"2024-09-11T08:51:33.929530+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:41.648756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:48:41.648756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019862030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.19","version":"264'10592","reported_seq":19094,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.832786+0000","last_change":"2024-09-11T08:50:54.909109+0000","last_active":"2024-09-11T08:51:41.832786+0000","last_peered":"2024-09-11T08:51:41.832786+0000","last_clean":"2024-09-11T08:51:41.832786+0000","last_became_active":"2024-09-11T08:50:54.902266+0000","last_became_peered":"2024-09-11T08:50:54.902266+0000","last_unstale":"2024-09-11T08:51:41.832786+0000","last_undegraded":"2024-09-11T08:51:41.832786+0000","last_fullsized":"2024-09-11T08:51:41.832786+0000","mapping_epoch":244,"log_start":"227'6456","ondisk_log_start":"227'6456","created":235,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":4136,"log_dups_size":0,"ondisk_log_size":4136,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146866176,"num_objects":2241,"num_object_clones":0,"num_object_copies":6723,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6795,"num_write_kb":289152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4201,"num_bytes_recovered":275316736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"200'25","reported_seq":289,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.917669+0000","last_change":"2024-09-11T08:49:28.382609+0000","last_active":"2024-09-11T08:51:33.917669+0000","last_peered":"2024-09-11T08:51:33.917669+0000","last_clean":"2024-09-11T08:51:33.917669+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:51:33.917669+0000","last_undegraded":"2024-09-11T08:51:33.917669+0000","last_fullsized":"2024-09-11T08:51:33.917669+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:49:28.382409+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:49:28.382409+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046362419999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":276,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:34.470871+0000","last_change":"2024-09-11T08:51:34.470806+0000","last_active":"2024-09-11T08:51:34.470871+0000","last_peered":"2024-09-11T08:51:34.470871+0000","last_clean":"2024-09-11T08:51:34.470871+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:51:34.470871+0000","last_undegraded":"2024-09-11T08:51:34.470871+0000","last_fullsized":"2024-09-11T08:51:34.470871+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:34.470720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_clean_scrub_stamp":"2024-09-11T08:51:34.470720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:57.507880+0000","scrub_duration":0.0041196690000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"211'9","reported_seq":440,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:37.832159+0000","last_change":"2024-09-11T08:50:08.389210+0000","last_active":"2024-09-11T08:51:37.832159+0000","last_peered":"2024-09-11T08:51:37.832159+0000","last_clean":"2024-09-11T08:51:37.832159+0000","last_became_active":"2024-09-11T08:49:09.908380+0000","last_became_peered":"2024-09-11T08:49:09.908380+0000","last_unstale":"2024-09-11T08:51:37.832159+0000","last_undegraded":"2024-09-11T08:51:37.832159+0000","last_fullsized":"2024-09-11T08:51:37.832159+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'9","last_scrub_stamp":"2024-09-11T08:50:08.388991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:08.388991+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019634517000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":76,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.915731+0000","last_change":"2024-09-11T08:49:21.855032+0000","last_active":"2024-09-11T08:51:33.915731+0000","last_peered":"2024-09-11T08:51:33.915731+0000","last_clean":"2024-09-11T08:51:33.915731+0000","last_became_active":"2024-09-11T08:49:09.914396+0000","last_became_peered":"2024-09-11T08:49:09.914396+0000","last_unstale":"2024-09-11T08:51:33.915731+0000","last_undegraded":"2024-09-11T08:51:33.915731+0000","last_fullsized":"2024-09-11T08:51:33.915731+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:21.854973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:21.854973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064444309999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"200'19","reported_seq":308,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.927303+0000","last_change":"2024-09-11T08:51:31.019004+0000","last_active":"2024-09-11T08:51:33.927303+0000","last_peered":"2024-09-11T08:51:33.927303+0000","last_clean":"2024-09-11T08:51:33.927303+0000","last_became_active":"2024-09-11T08:49:16.937185+0000","last_became_peered":"2024-09-11T08:49:16.937185+0000","last_unstale":"2024-09-11T08:51:33.927303+0000","last_undegraded":"2024-09-11T08:51:33.927303+0000","last_fullsized":"2024-09-11T08:51:33.927303+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:31.018893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:31.018893+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:45.174641+0000","scrub_duration":0.0062729989999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":257,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.927941+0000","last_change":"2024-09-11T08:50:04.791525+0000","last_active":"2024-09-11T08:51:33.927941+0000","last_peered":"2024-09-11T08:51:33.927941+0000","last_clean":"2024-09-11T08:51:33.927941+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:51:33.927941+0000","last_undegraded":"2024-09-11T08:51:33.927941+0000","last_fullsized":"2024-09-11T08:51:33.927941+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:04.791327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:04.791327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063425010000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":262,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.911696+0000","last_change":"2024-09-11T08:48:30.985518+0000","last_active":"2024-09-11T08:51:33.911696+0000","last_peered":"2024-09-11T08:51:33.911696+0000","last_clean":"2024-09-11T08:51:33.911696+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:51:33.911696+0000","last_undegraded":"2024-09-11T08:51:33.911696+0000","last_fullsized":"2024-09-11T08:51:33.911696+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:48:30.985271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:48:30.985271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099058919999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":260,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.917069+0000","last_change":"2024-09-11T08:50:21.261969+0000","last_active":"2024-09-11T08:51:33.917069+0000","last_peered":"2024-09-11T08:51:33.917069+0000","last_clean":"2024-09-11T08:51:33.917069+0000","last_became_active":"2024-09-11T08:49:09.905289+0000","last_became_peered":"2024-09-11T08:49:09.905289+0000","last_unstale":"2024-09-11T08:51:33.917069+0000","last_undegraded":"2024-09-11T08:51:33.917069+0000","last_fullsized":"2024-09-11T08:51:33.917069+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:21.261891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:21.261891+0000","last_clean_scrub_stamp":"2024-09-11T08:50:21.261891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044878879999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"200'15","reported_seq":302,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.915930+0000","last_change":"2024-09-11T08:50:03.319748+0000","last_active":"2024-09-11T08:51:33.915930+0000","last_peered":"2024-09-11T08:51:33.915930+0000","last_clean":"2024-09-11T08:51:33.915930+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:51:33.915930+0000","last_undegraded":"2024-09-11T08:51:33.915930+0000","last_fullsized":"2024-09-11T08:51:33.915930+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:50:03.319664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:03.319664+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046538380000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":276,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.917086+0000","last_change":"2024-09-11T08:50:21.208754+0000","last_active":"2024-09-11T08:51:33.917086+0000","last_peered":"2024-09-11T08:51:33.917086+0000","last_clean":"2024-09-11T08:51:33.917086+0000","last_became_active":"2024-09-11T08:38:38.954325+0000","last_became_peered":"2024-09-11T08:38:38.954325+0000","last_unstale":"2024-09-11T08:51:33.917086+0000","last_undegraded":"2024-09-11T08:51:33.917086+0000","last_fullsized":"2024-09-11T08:51:33.917086+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:21.208684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:21.208684+0000","last_clean_scrub_stamp":"2024-09-11T08:50:21.208684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075563440000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":262,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916441+0000","last_change":"2024-09-11T08:49:27.342962+0000","last_active":"2024-09-11T08:51:33.916441+0000","last_peered":"2024-09-11T08:51:33.916441+0000","last_clean":"2024-09-11T08:51:33.916441+0000","last_became_active":"2024-09-11T08:49:09.910548+0000","last_became_peered":"2024-09-11T08:49:09.910548+0000","last_unstale":"2024-09-11T08:51:33.916441+0000","last_undegraded":"2024-09-11T08:51:33.916441+0000","last_fullsized":"2024-09-11T08:51:33.916441+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:27.342908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:49:27.342908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020004860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":76,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.914718+0000","last_change":"2024-09-11T08:50:24.401692+0000","last_active":"2024-09-11T08:51:33.914718+0000","last_peered":"2024-09-11T08:51:33.914718+0000","last_clean":"2024-09-11T08:51:33.914718+0000","last_became_active":"2024-09-11T08:49:09.904622+0000","last_became_peered":"2024-09-11T08:49:09.904622+0000","last_unstale":"2024-09-11T08:51:33.914718+0000","last_undegraded":"2024-09-11T08:51:33.914718+0000","last_fullsized":"2024-09-11T08:51:33.914718+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:24.401480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:24.401480+0000","last_clean_scrub_stamp":"2024-09-11T08:50:24.401480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:43.701178+0000","scrub_duration":0.010533496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"200'19","reported_seq":285,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916920+0000","last_change":"2024-09-11T08:51:21.423287+0000","last_active":"2024-09-11T08:51:33.916920+0000","last_peered":"2024-09-11T08:51:33.916920+0000","last_clean":"2024-09-11T08:51:33.916920+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:51:33.916920+0000","last_undegraded":"2024-09-11T08:51:33.916920+0000","last_fullsized":"2024-09-11T08:51:33.916920+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:21.423213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:21.423213+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:44.934999+0000","scrub_duration":0.0086807859999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":266,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.920118+0000","last_change":"2024-09-11T08:49:33.209392+0000","last_active":"2024-09-11T08:51:33.920118+0000","last_peered":"2024-09-11T08:51:33.920118+0000","last_clean":"2024-09-11T08:51:33.920118+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:51:33.920118+0000","last_undegraded":"2024-09-11T08:51:33.920118+0000","last_fullsized":"2024-09-11T08:51:33.920118+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:33.209345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:49:33.209345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052610230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":275,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916252+0000","last_change":"2024-09-11T08:51:17.394463+0000","last_active":"2024-09-11T08:51:33.916252+0000","last_peered":"2024-09-11T08:51:33.916252+0000","last_clean":"2024-09-11T08:51:33.916252+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:51:33.916252+0000","last_undegraded":"2024-09-11T08:51:33.916252+0000","last_fullsized":"2024-09-11T08:51:33.916252+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:17.394413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:17.394413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:19.867235+0000","scrub_duration":0.0012982250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":280,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916509+0000","last_change":"2024-09-11T08:50:25.255389+0000","last_active":"2024-09-11T08:51:33.916509+0000","last_peered":"2024-09-11T08:51:33.916509+0000","last_clean":"2024-09-11T08:51:33.916509+0000","last_became_active":"2024-09-11T08:49:09.905634+0000","last_became_peered":"2024-09-11T08:49:09.905634+0000","last_unstale":"2024-09-11T08:51:33.916509+0000","last_undegraded":"2024-09-11T08:51:33.916509+0000","last_fullsized":"2024-09-11T08:51:33.916509+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:25.255332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:25.255332+0000","last_clean_scrub_stamp":"2024-09-11T08:50:25.255332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:39.868191+0000","scrub_duration":0.0046648280000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"200'10","reported_seq":303,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916074+0000","last_change":"2024-09-11T08:51:25.335052+0000","last_active":"2024-09-11T08:51:33.916074+0000","last_peered":"2024-09-11T08:51:33.916074+0000","last_clean":"2024-09-11T08:51:33.916074+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:51:33.916074+0000","last_undegraded":"2024-09-11T08:51:33.916074+0000","last_fullsized":"2024-09-11T08:51:33.916074+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'6","last_scrub_stamp":"2024-09-11T08:48:32.019991+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:48:32.019991+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010394439,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":88,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.920776+0000","last_change":"2024-09-11T08:50:09.396223+0000","last_active":"2024-09-11T08:51:33.920776+0000","last_peered":"2024-09-11T08:51:33.920776+0000","last_clean":"2024-09-11T08:51:33.920776+0000","last_became_active":"2024-09-11T08:49:09.907483+0000","last_became_peered":"2024-09-11T08:49:09.907483+0000","last_unstale":"2024-09-11T08:51:33.920776+0000","last_undegraded":"2024-09-11T08:51:33.920776+0000","last_fullsized":"2024-09-11T08:51:33.920776+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:50:09.396148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:09.396148+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013176124000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":136,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.918257+0000","last_change":"2024-09-11T08:51:33.142487+0000","last_active":"2024-09-11T08:51:33.918257+0000","last_peered":"2024-09-11T08:51:33.918257+0000","last_clean":"2024-09-11T08:51:33.918257+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:51:33.918257+0000","last_undegraded":"2024-09-11T08:51:33.918257+0000","last_fullsized":"2024-09-11T08:51:33.918257+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:33.142415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:33.142415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:47.140636+0000","scrub_duration":0.0057010619999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":270,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.915868+0000","last_change":"2024-09-11T08:49:30.788379+0000","last_active":"2024-09-11T08:51:33.915868+0000","last_peered":"2024-09-11T08:51:33.915868+0000","last_clean":"2024-09-11T08:51:33.915868+0000","last_became_active":"2024-09-11T08:38:44.992831+0000","last_became_peered":"2024-09-11T08:38:44.992831+0000","last_unstale":"2024-09-11T08:51:33.915868+0000","last_undegraded":"2024-09-11T08:51:33.915868+0000","last_fullsized":"2024-09-11T08:51:33.915868+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:30.788303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:30.788303+0000","last_clean_scrub_stamp":"2024-09-11T08:49:30.788303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038913020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.12","version":"264'10226","reported_seq":17084,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-11T08:51:41.399136+0000","last_change":"2024-09-11T08:51:18.922747+0000","last_active":"2024-09-11T08:51:41.399136+0000","last_peered":"2024-09-11T08:51:41.399136+0000","last_clean":"2024-09-11T08:51:41.399136+0000","last_became_active":"2024-09-11T08:51:18.911073+0000","last_became_peered":"2024-09-11T08:51:18.911073+0000","last_unstale":"2024-09-11T08:51:41.399136+0000","last_undegraded":"2024-09-11T08:51:41.399136+0000","last_fullsized":"2024-09-11T08:51:41.399136+0000","mapping_epoch":256,"log_start":"223'5092","ondisk_log_start":"223'5092","created":235,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:04.855764+0000","last_clean_scrub_stamp":"2024-09-11T08:49:04.855764+0000","objects_scrubbed":0,"log_size":5134,"log_dups_size":0,"ondisk_log_size":5134,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146472960,"num_objects":2235,"num_object_clones":0,"num_object_copies":6705,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6513,"num_write_kb":279936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3378,"num_bytes_recovered":221380608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":270,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.912566+0000","last_change":"2024-09-11T08:49:29.487522+0000","last_active":"2024-09-11T08:51:33.912566+0000","last_peered":"2024-09-11T08:51:33.912566+0000","last_clean":"2024-09-11T08:51:33.912566+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:51:33.912566+0000","last_undegraded":"2024-09-11T08:51:33.912566+0000","last_fullsized":"2024-09-11T08:51:33.912566+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:29.487391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:49:29.487391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0091739200000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":276,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916257+0000","last_change":"2024-09-11T08:51:32.393279+0000","last_active":"2024-09-11T08:51:33.916257+0000","last_peered":"2024-09-11T08:51:33.916257+0000","last_clean":"2024-09-11T08:51:33.916257+0000","last_became_active":"2024-09-11T08:49:09.918992+0000","last_became_peered":"2024-09-11T08:49:09.918992+0000","last_unstale":"2024-09-11T08:51:33.916257+0000","last_undegraded":"2024-09-11T08:51:33.916257+0000","last_fullsized":"2024-09-11T08:51:33.916257+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:32.393197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:32.393197+0000","last_clean_scrub_stamp":"2024-09-11T08:51:32.393197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:48.941330+0000","scrub_duration":0.001696321,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"200'19","reported_seq":160,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.919179+0000","last_change":"2024-09-11T08:51:32.119010+0000","last_active":"2024-09-11T08:51:33.919179+0000","last_peered":"2024-09-11T08:51:33.919179+0000","last_clean":"2024-09-11T08:51:33.919179+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:51:33.919179+0000","last_undegraded":"2024-09-11T08:51:33.919179+0000","last_fullsized":"2024-09-11T08:51:33.919179+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:32.118917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:32.118917+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:52:56.212024+0000","scrub_duration":0.0023296060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":76,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-11T08:51:33.916271+0000","last_change":"2024-09-11T08:50:26.239880+0000","last_active":"2024-09-11T08:51:33.916271+0000","last_peered":"2024-09-11T08:51:33.916271+0000","last_clean":"2024-09-11T08:51:33.916271+0000","last_became_active":"2024-09-11T08:49:09.904721+0000","last_became_peered":"2024-09-11T08:49:09.904721+0000","last_unstale":"2024-09-11T08:51:33.916271+0000","last_undegraded":"2024-09-11T08:51:33.916271+0000","last_fullsized":"2024-09-11T08:51:33.916271+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:26.239829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:26.239829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:50.757374+0000","scrub_duration":0.001788675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":26,"stat_sum":{"num_bytes":4753195032,"num_objects":72529,"num_object_clones":0,"num_object_copies":217587,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5705,"num_objects_unfound":0,"num_objects_dirty":72529,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":214263,"num_write_kb":9177281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28612,"num_bytes_recovered":1875116032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15928864768,"data_stored":15928852552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117712,"ondisk_log_size":117712,"up":78,"acting":78,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1119,"num_read_kb":876,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":3234,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":12426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":751,"ondisk_log_size":751,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":773,"num_read_kb":768,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":483028,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":189,"num_write":93,"num_write_kb":1332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1234405,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299554,"num_pgs":78,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3314932,"kb_used_data":3005096,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":90430220,"statfs":{"total":95995035648,"available":92600545280,"internally_reserved":0,"allocated":3077218304,"data_stored":3076670575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:50:49 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.2889999999999999,"15min":0.89500000000000002},"min":{"1min":0.29199999999999998,"5min":0.23200000000000001,"15min":0.22},"max":{"1min":4.8730000000000002,"5min":32.171999999999997,"15min":32.171999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.4750000000000001,"15min":1.0029999999999999},"min":{"1min":0.28199999999999997,"5min":0.221,"15min":0.21199999999999999},"max":{"1min":2.8540000000000001,"5min":32.149999999999999,"15min":32.149999999999999},"last":4.0069999999999997}]},{"osd":1,"last update":"Wed Sep 11 08:50:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.603,"15min":1.056},"min":{"1min":0.27300000000000002,"5min":0.224,"15min":0.224},"max":{"1min":4.8970000000000002,"5min":33.692999999999998,"15min":33.692999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.5569999999999999,"15min":0.98799999999999999},"min":{"1min":0.254,"5min":0.23799999999999999,"15min":0.23300000000000001},"max":{"1min":2.6930000000000001,"5min":35.591000000000001,"15min":35.591000000000001},"last":3.9780000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:50:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.05,"15min":0.84199999999999997},"min":{"1min":0.36499999999999999,"5min":0.249,"15min":0.249},"max":{"1min":4.2549999999999999,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":1.105,"5min":1.0740000000000001,"15min":0.84499999999999997},"min":{"1min":0.35099999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":4.9189999999999996,"5min":7.5259999999999998,"15min":7.5259999999999998},"last":0.51900000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:50:49 2024","interfaces":[{"interface":"back","average":{"1min":4.1639999999999997,"5min":1.6779999999999999,"15min":1.0920000000000001},"min":{"1min":0.31,"5min":0.27200000000000002,"15min":0.25},"max":{"1min":56.265999999999998,"5min":56.265999999999998,"15min":56.265999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":4.5149999999999997,"5min":1.8009999999999999,"15min":1.1890000000000001},"min":{"1min":0.39600000000000002,"5min":0.27700000000000002,"15min":0.27400000000000002},"max":{"1min":57.207000000000001,"5min":57.207000000000001,"15min":57.207000000000001},"last":0.58299999999999996}]},{"osd":4,"last update":"Wed Sep 11 08:50:49 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":0.83899999999999997,"15min":0.58399999999999996},"min":{"1min":0.255,"5min":0.24099999999999999,"15min":0.23400000000000001},"max":{"1min":3.8599999999999999,"5min":7.1520000000000001,"15min":7.1520000000000001},"last":3.9129999999999998},{"interface":"front","average":{"1min":1.349,"5min":0.93999999999999995,"15min":0.61399999999999999},"min":{"1min":0.33800000000000002,"5min":0.216,"15min":0.216},"max":{"1min":4.641,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.36899999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:50:49 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":0.93400000000000005,"15min":0.624},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":5.0110000000000001,"5min":5.1059999999999999,"15min":5.1059999999999999},"last":1.956},{"interface":"front","average":{"1min":1.276,"5min":1.002,"15min":0.68300000000000005},"min":{"1min":0.44800000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.9769999999999999,"5min":6.3650000000000002,"15min":6.3650000000000002},"last":4.032}]},{"osd":6,"last update":"Wed Sep 11 08:50:49 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.86799999999999999,"15min":0.65600000000000003},"min":{"1min":0.34300000000000003,"5min":0.317,"15min":0.315},"max":{"1min":2.726,"5min":3.512,"15min":3.5979999999999999},"last":4.0609999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.94599999999999995,"15min":0.67300000000000004},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.27900000000000003},"max":{"1min":4.6719999999999997,"5min":4.6719999999999997,"15min":4.6719999999999997},"last":0.45600000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528487,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1630744,"kb_used_data":1329168,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":92114408,"statfs":{"total":95995035648,"available":94325153792,"internally_reserved":0,"allocated":1361068032,"data_stored":1360544723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:51:30 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.4770000000000001,"15min":1.1459999999999999},"min":{"1min":0.27900000000000003,"5min":0.23300000000000001,"15min":0.217},"max":{"1min":2.9380000000000002,"5min":27.908000000000001,"15min":28.302},"last":0.26700000000000002},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.458,"15min":1.139},"min":{"1min":0.24399999999999999,"5min":0.24299999999999999,"15min":0.23300000000000001},"max":{"1min":4.8609999999999998,"5min":27.771000000000001,"15min":28.314},"last":0.97599999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:51:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.6659999999999999,"15min":1.125},"min":{"1min":0.27400000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.6760000000000002,"5min":27.984000000000002,"15min":27.984000000000002},"last":0.23000000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.6679999999999999,"15min":1.089},"min":{"1min":0.33700000000000002,"5min":0.25700000000000001,"15min":0.253},"max":{"1min":7.3840000000000003,"5min":28.106000000000002,"15min":28.106000000000002},"last":0.44500000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:51:30 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":0.97399999999999998,"15min":0.81399999999999995},"min":{"1min":0.38900000000000001,"5min":0.28399999999999997,"15min":0.23200000000000001},"max":{"1min":3.899,"5min":3.899,"15min":14.313000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.0409999999999999,"15min":0.83699999999999997},"min":{"1min":0.28899999999999998,"5min":0.27700000000000002,"15min":0.251},"max":{"1min":3.0009999999999999,"5min":4.9020000000000001,"15min":6.5979999999999999},"last":0.997}]},{"osd":3,"last update":"Wed Sep 11 08:51:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.675,"15min":1.147},"min":{"1min":0.26900000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.484,"5min":27.795000000000002,"15min":27.795000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.8200000000000001,"15min":1.1559999999999999},"min":{"1min":0.36299999999999999,"5min":0.32100000000000001,"15min":0.224},"max":{"1min":4.2770000000000001,"5min":27.841999999999999,"15min":27.841999999999999},"last":0.40100000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:51:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":0.78400000000000003,"15min":0.55900000000000005},"min":{"1min":0.252,"5min":0.252,"15min":0.20799999999999999},"max":{"1min":2.9409999999999998,"5min":4.8360000000000003,"15min":4.8360000000000003},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.004,"5min":0.78200000000000003,"15min":0.54900000000000004},"min":{"1min":0.30099999999999999,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":3.0830000000000002,"5min":5.5330000000000004,"15min":5.5330000000000004},"last":0.56200000000000006}]},{"osd":5,"last update":"Wed Sep 11 08:51:30 2024","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.0569999999999999,"15min":0.68000000000000005},"min":{"1min":0.41799999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":4.702,"5min":8.6829999999999998,"15min":8.6829999999999998},"last":1.351},{"interface":"front","average":{"1min":1.0389999999999999,"5min":0.91600000000000004,"15min":0.623},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.6499999999999999,"5min":5.0819999999999999,"15min":5.0819999999999999},"last":0.375}]},{"osd":7,"last update":"Wed Sep 11 08:50:43 2024","interfaces":[{"interface":"back","average":{"1min":2.819,"5min":1.2509999999999999,"15min":0.73499999999999999},"min":{"1min":0.39600000000000002,"5min":0.29899999999999999,"15min":0.247},"max":{"1min":35.805,"5min":35.805,"15min":35.805},"last":0.76200000000000001},{"interface":"front","average":{"1min":2.5470000000000002,"5min":1.121,"15min":0.71599999999999997},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.27700000000000002},"max":{"1min":35.045999999999999,"5min":35.045999999999999,"15min":35.045999999999999},"last":0.79500000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757419,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3614964,"kb_used_data":3020904,"kb_used_omap":1,"kb_used_meta":594046,"kb_avail":90130188,"statfs":{"total":95995035648,"available":92293312512,"internally_reserved":0,"allocated":3093405696,"data_stored":3092859034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":608303448},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:51:10 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.0980000000000001,"15min":0.84799999999999998},"min":{"1min":0.26700000000000002,"5min":0.21099999999999999,"15min":0.20599999999999999},"max":{"1min":4.1929999999999996,"5min":4.6420000000000003,"15min":7.3899999999999997},"last":1.6919999999999999},{"interface":"front","average":{"1min":0.92200000000000004,"5min":1.282,"15min":0.97199999999999998},"min":{"1min":0.253,"5min":0.23400000000000001,"15min":0.23100000000000001},"max":{"1min":3.1099999999999999,"5min":8.0410000000000004,"15min":8.0410000000000004},"last":1.5369999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:51:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.3009999999999999,"15min":0.95899999999999996},"min":{"1min":0.29299999999999998,"5min":0.23799999999999999,"15min":0.22800000000000001},"max":{"1min":3.629,"5min":8,"15min":8},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2889999999999999,"15min":0.92100000000000004},"min":{"1min":0.34799999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.6240000000000001,"5min":5.3949999999999996,"15min":6.6719999999999997},"last":0.45400000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:51:10 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":1.234,"15min":0.92100000000000004},"min":{"1min":0.36599999999999999,"5min":0.27800000000000002,"15min":0.23799999999999999},"max":{"1min":3.4870000000000001,"5min":6.4080000000000004,"15min":6.4080000000000004},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.2310000000000001,"15min":0.89300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.6589999999999998,"5min":8.0640000000000001,"15min":8.0640000000000001},"last":1.569}]},{"osd":3,"last update":"Wed Sep 11 08:51:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.2230000000000001,"15min":0.91400000000000003},"min":{"1min":0.33800000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":4.1500000000000004,"5min":6.4299999999999997,"15min":6.4299999999999997},"last":1.774},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.4810000000000001,"15min":1.0760000000000001},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":3.681,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":1.6319999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:51:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.006,"15min":0.61299999999999999},"min":{"1min":0.249,"5min":0.247,"15min":0.20699999999999999},"max":{"1min":3.6920000000000002,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.88100000000000001,"15min":0.59299999999999997},"min":{"1min":0.308,"5min":0.25800000000000001,"15min":0.20499999999999999},"max":{"1min":3.5329999999999999,"5min":6.3769999999999998,"15min":6.3769999999999998},"last":1.651}]},{"osd":6,"last update":"Wed Sep 11 08:51:33 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.996,"15min":0.65400000000000003},"min":{"1min":0.371,"5min":0.29399999999999998,"15min":0.23599999999999999},"max":{"1min":3.4500000000000002,"5min":6.4669999999999996,"15min":6.4669999999999996},"last":1.8160000000000001},{"interface":"front","average":{"1min":1.0660000000000001,"5min":0.97899999999999998,"15min":0.64500000000000002},"min":{"1min":0.32300000000000001,"5min":0.27300000000000002,"15min":0.251},"max":{"1min":4.9820000000000002,"5min":8.1920000000000002,"15min":8.1920000000000002},"last":1.6799999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:50:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.859,"15min":0.94199999999999995},"min":{"1min":0.31900000000000001,"5min":0.29999999999999999,"15min":0.27800000000000002},"max":{"1min":3.5979999999999999,"5min":88.513000000000005,"15min":88.513000000000005},"last":1.744},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.8740000000000001,"15min":0.97799999999999998},"min":{"1min":0.35199999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.681,"5min":86.963999999999999,"15min":86.963999999999999},"last":1.845}]}]},{"osd":4,"up_from":31,"seq":133143986351,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1360100,"kb_used_data":1058456,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":92385052,"statfs":{"total":95995035648,"available":94602293248,"internally_reserved":0,"allocated":1083858944,"data_stored":1083326707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":1.5880000000000001,"15min":1.5820000000000001},"min":{"1min":0.26000000000000001,"5min":0.21199999999999999,"15min":0.20399999999999999},"max":{"1min":2.75,"5min":44.140999999999998,"15min":89.510000000000005},"last":2.1880000000000002},{"interface":"front","average":{"1min":0.82599999999999996,"5min":1.7869999999999999,"15min":1.6830000000000001},"min":{"1min":0.248,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.8759999999999999,"5min":44.450000000000003,"15min":89.765000000000001},"last":0.45000000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:50:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":2.3340000000000001,"15min":1.3440000000000001},"min":{"1min":0.28100000000000003,"5min":0.27300000000000002,"15min":0.24399999999999999},"max":{"1min":4.3879999999999999,"5min":48.976999999999997,"15min":48.976999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":1.048,"5min":2.3140000000000001,"15min":1.288},"min":{"1min":0.373,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.085,"5min":45.787999999999997,"15min":45.787999999999997},"last":2.2919999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.109,"15min":0.83399999999999996},"min":{"1min":0.28899999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":3.9430000000000001,"5min":17.303000000000001,"15min":17.303000000000001},"last":2.2160000000000002},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.0880000000000001,"15min":0.83499999999999996},"min":{"1min":0.30499999999999999,"5min":0.27500000000000002,"15min":0.25800000000000001},"max":{"1min":2.7490000000000001,"5min":20.702999999999999,"15min":20.702999999999999},"last":0.42999999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":2.0249999999999999,"15min":1.8600000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":2.4670000000000001,"5min":45.268000000000001,"15min":89.881},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":2.141,"15min":1.8160000000000001},"min":{"1min":0.318,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.504,"5min":44.234000000000002,"15min":89.715000000000003},"last":0.46200000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:51:07 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.92300000000000004,"15min":0.61199999999999999},"min":{"1min":0.35199999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":2.9060000000000001,"5min":4.2069999999999999,"15min":4.2069999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.0760000000000001,"5min":0.81399999999999995,"15min":0.57099999999999995},"min":{"1min":0.254,"5min":0.254,"15min":0.23699999999999999},"max":{"1min":2.9180000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":2.1200000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:51:38 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.77100000000000002,"15min":0.56999999999999995},"min":{"1min":0.30399999999999999,"5min":0.27300000000000002,"15min":0.22900000000000001},"max":{"1min":2.363,"5min":3.02,"15min":3.02},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.753,"15min":0.60199999999999998},"min":{"1min":0.27800000000000002,"5min":0.22800000000000001,"15min":0.221},"max":{"1min":2.5840000000000001,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":0.61099999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.74199999999999999,"15min":0.57799999999999996},"min":{"1min":0.34300000000000003,"5min":0.252,"15min":0.252},"max":{"1min":3.0350000000000001,"5min":3.339,"15min":3.339},"last":2.157},{"interface":"front","average":{"1min":1.1200000000000001,"5min":0.90200000000000002,"15min":0.63200000000000001},"min":{"1min":0.38100000000000001,"5min":0.29599999999999999,"15min":0.28799999999999998},"max":{"1min":2.984,"5min":5.1970000000000001,"15min":5.1970000000000001},"last":2.399}]}]},{"osd":3,"up_from":26,"seq":111669149877,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3774600,"kb_used_data":2871228,"kb_used_omap":1,"kb_used_meta":903358,"kb_avail":89970552,"statfs":{"total":95995035648,"available":92129845248,"internally_reserved":0,"allocated":2940137472,"data_stored":2939570995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":925038936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:51:30 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":1.1399999999999999,"15min":1.0549999999999999},"min":{"1min":0.215,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":3.0310000000000001,"5min":24.280000000000001,"15min":24.280000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.754,"5min":1.371,"15min":1.2729999999999999},"min":{"1min":0.26800000000000002,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":2.2599999999999998,"5min":22.225000000000001,"15min":22.225000000000001},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:51:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.5429999999999999,"15min":1.6639999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.19800000000000001},"max":{"1min":5.2519999999999998,"5min":8.2439999999999998,"15min":115.601},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.2589999999999999,"15min":1.542},"min":{"1min":0.30499999999999999,"5min":0.22700000000000001,"15min":0.17999999999999999},"max":{"1min":4.577,"5min":8.4420000000000002,"15min":115.474},"last":1.7370000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:51:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":1.042,"15min":1.1100000000000001},"min":{"1min":0.253,"5min":0.24299999999999999,"15min":0.23200000000000001},"max":{"1min":2.1949999999999998,"5min":8.2699999999999996,"15min":8.9510000000000005},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.23799999999999999,"5min":0.22700000000000001,"15min":0.20799999999999999},"max":{"1min":4.032,"5min":5.5999999999999996,"15min":10.695},"last":4.4470000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:51:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.379,"15min":1.1259999999999999},"min":{"1min":0.35599999999999998,"5min":0.33700000000000002,"15min":0.26700000000000002},"max":{"1min":4.298,"5min":8.3279999999999994,"15min":8.9969999999999999},"last":4.1580000000000004},{"interface":"front","average":{"1min":0.90000000000000002,"5min":1.0880000000000001,"15min":0.95699999999999996},"min":{"1min":0.40999999999999998,"5min":0.29099999999999998,"15min":0.27700000000000002},"max":{"1min":2.2629999999999999,"5min":7.234,"15min":7.234},"last":0.442}]},{"osd":5,"last update":"Wed Sep 11 08:50:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.385,"15min":1.1100000000000001},"min":{"1min":0.38500000000000001,"5min":0.29799999999999999,"15min":0.28299999999999997},"max":{"1min":2.8639999999999999,"5min":8.3599999999999994,"15min":11.526999999999999},"last":2.3069999999999999},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.2230000000000001,"15min":1.0029999999999999},"min":{"1min":0.38100000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.601,"5min":7.1689999999999996,"15min":7.1689999999999996},"last":0.46800000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:51:30 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.7130000000000001,"15min":1.294},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.29999999999999999},"max":{"1min":4.2569999999999997,"5min":33.106999999999999,"15min":33.106999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.665,"15min":1.2769999999999999},"min":{"1min":0.42999999999999999,"5min":0.36299999999999999,"15min":0.311},"max":{"1min":2.8399999999999999,"5min":33.076000000000001,"15min":33.076000000000001},"last":1.1479999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:50:48 2024","interfaces":[{"interface":"back","average":{"1min":3.7200000000000002,"5min":1.986,"15min":1.341},"min":{"1min":0.41799999999999998,"5min":0.36399999999999999,"15min":0.35299999999999998},"max":{"1min":35.073,"5min":35.073,"15min":35.073},"last":0.89800000000000002},{"interface":"front","average":{"1min":3.202,"5min":1.7529999999999999,"15min":1.1990000000000001},"min":{"1min":0.434,"5min":0.377,"15min":0.34699999999999998},"max":{"1min":34.844999999999999,"5min":34.844999999999999,"15min":34.844999999999999},"last":1.8819999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411513,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1624548,"kb_used_data":1314712,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92120604,"statfs":{"total":95995035648,"available":94331498496,"internally_reserved":0,"allocated":1346265088,"data_stored":1345733546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:51:06 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":2.2709999999999999,"15min":1.5289999999999999},"min":{"1min":0.192,"5min":0.16500000000000001,"15min":0.159},"max":{"1min":2.7530000000000001,"5min":34.713999999999999,"15min":34.713999999999999},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.622,"5min":2.1389999999999998,"15min":1.613},"min":{"1min":0.217,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.6020000000000001,"5min":34.399000000000001,"15min":34.399000000000001},"last":0.55700000000000005}]},{"osd":1,"last update":"Wed Sep 11 08:51:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":3.0859999999999999,"15min":1.6180000000000001},"min":{"1min":0.25700000000000001,"5min":0.23799999999999999,"15min":0.19},"max":{"1min":11.552,"5min":72.867000000000004,"15min":72.867000000000004},"last":0.32400000000000001},{"interface":"front","average":{"1min":1.498,"5min":3.1459999999999999,"15min":1.722},"min":{"1min":0.27200000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":11.616,"5min":67.799000000000007,"15min":67.799000000000007},"last":0.68300000000000005}]},{"osd":3,"last update":"Wed Sep 11 08:51:13 2024","interfaces":[{"interface":"back","average":{"1min":1.6120000000000001,"5min":2.254,"15min":1.522},"min":{"1min":0.27900000000000003,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":11.878,"5min":34.75,"15min":34.75},"last":0.36099999999999999},{"interface":"front","average":{"1min":1.6779999999999999,"5min":2.0760000000000001,"15min":1.46},"min":{"1min":0.29599999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":9.2759999999999998,"5min":32.802,"15min":32.802},"last":0.77400000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:51:34 2024","interfaces":[{"interface":"back","average":{"1min":3.4020000000000001,"5min":1.5620000000000001,"15min":1.163},"min":{"1min":0.28799999999999998,"5min":0.255,"15min":0.255},"max":{"1min":52.113999999999997,"5min":52.113999999999997,"15min":52.113999999999997},"last":0.28699999999999998},{"interface":"front","average":{"1min":3.2320000000000002,"5min":1.4750000000000001,"15min":1.111},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.28399999999999997},"max":{"1min":52.097999999999999,"5min":52.097999999999999,"15min":52.097999999999999},"last":0.33900000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:51:06 2024","interfaces":[{"interface":"back","average":{"1min":3.032,"5min":1.5660000000000001,"15min":1.1100000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":52.145000000000003,"5min":52.145000000000003,"15min":52.145000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":2.9420000000000002,"5min":1.4139999999999999,"15min":0.99099999999999999},"min":{"1min":0.33100000000000002,"5min":0.28999999999999998,"15min":0.28399999999999997},"max":{"1min":52.216000000000001,"5min":52.216000000000001,"15min":52.216000000000001},"last":2.1970000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:51:26 2024","interfaces":[{"interface":"back","average":{"1min":3.5030000000000001,"5min":1.6459999999999999,"15min":1.212},"min":{"1min":0.40100000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":52.235999999999997,"5min":52.235999999999997,"15min":52.235999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":3.4079999999999999,"5min":1.51,"15min":1.1180000000000001},"min":{"1min":0.35299999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":52.186,"5min":52.186,"15min":52.186},"last":0.377}]},{"osd":7,"last update":"Wed Sep 11 08:50:36 2024","interfaces":[{"interface":"back","average":{"1min":4.5010000000000003,"5min":1.7450000000000001,"15min":1.1120000000000001},"min":{"1min":0.372,"5min":0.32200000000000001,"15min":0.30399999999999999},"max":{"1min":84.108999999999995,"5min":84.108999999999995,"15min":84.108999999999995},"last":1.2130000000000001},{"interface":"front","average":{"1min":4.7640000000000002,"5min":2.0979999999999999,"15min":1.3089999999999999},"min":{"1min":0.432,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":85.938000000000002,"5min":85.938000000000002,"15min":85.938000000000002},"last":0.41199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":81,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3640464,"kb_used_data":3050504,"kb_used_omap":1,"kb_used_meta":589886,"kb_avail":90104688,"statfs":{"total":95995035648,"available":92267200512,"internally_reserved":0,"allocated":3123716096,"data_stored":3123136003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":604043608},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:51:40 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":1.0609999999999999,"15min":1.4399999999999999},"min":{"1min":0.189,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":3.8639999999999999,"5min":4.6319999999999997,"15min":117.01600000000001},"last":0.20100000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":1.167,"15min":1.73},"min":{"1min":0.217,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":2.8660000000000001,"5min":8.1549999999999994,"15min":117.62},"last":0.38500000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":1.228,"15min":1.1259999999999999},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.191},"max":{"1min":2.21,"5min":8.1329999999999991,"15min":20.628},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.872,"5min":1.355,"15min":1.139},"min":{"1min":0.25800000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":4.8470000000000004,"5min":9.7200000000000006,"15min":20.148},"last":0.29699999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:51:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.53,"15min":1.4850000000000001},"min":{"1min":0.223,"5min":0.22,"15min":0.191},"max":{"1min":5.0220000000000002,"5min":5.8490000000000002,"15min":116.98},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.266,"5min":1.345,"15min":1.554},"min":{"1min":0.318,"5min":0.246,"15min":0.22},"max":{"1min":4.4409999999999998,"5min":4.6719999999999997,"15min":117.56699999999999},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed Sep 11 08:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.246,"15min":0.996},"min":{"1min":0.28799999999999998,"5min":0.27500000000000002,"15min":0.26200000000000001},"max":{"1min":4.7930000000000001,"5min":4.7930000000000001,"15min":11.19},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.103,"15min":0.91300000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.251},"max":{"1min":3.9009999999999998,"5min":4.0730000000000004,"15min":7.7169999999999996},"last":0.437}]},{"osd":5,"last update":"Wed Sep 11 08:51:05 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.329,"15min":1.468},"min":{"1min":0.39300000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.9279999999999999,"5min":8.0999999999999996,"15min":103.337},"last":0.50700000000000001},{"interface":"front","average":{"1min":1.827,"5min":1.45,"15min":1.478},"min":{"1min":0.33100000000000002,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":5.1050000000000004,"5min":5.9020000000000001,"15min":103.163},"last":0.74099999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:51:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.2230000000000001,"15min":0.94999999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29399999999999998},"max":{"1min":5.8250000000000002,"5min":5.8250000000000002,"15min":8.9149999999999991},"last":2.7149999999999999},{"interface":"front","average":{"1min":1.536,"5min":1.367,"15min":1.1140000000000001},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.31},"max":{"1min":4.0800000000000001,"5min":4.7080000000000002,"15min":9.5199999999999996},"last":2.8759999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.339,"15min":1.0649999999999999},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":4.6289999999999996,"5min":4.6289999999999996,"15min":11.315},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.512,"5min":1.359,"15min":1.0249999999999999},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.315},"max":{"1min":5.9089999999999998,"5min":10.266,"15min":10.266},"last":1.6240000000000001}]}]},{"osd":0,"up_from":0,"seq":34359738562,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3121680384,"data_stored":3121676312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1337327616,"data_stored":1337327616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2912026624,"data_stored":2912026624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1081409536,"data_stored":1081409536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3087798272,"data_stored":3087794200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1359151104,"data_stored":1359151104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3029471232,"data_stored":3029467160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:51:46.842 INFO:tasks.thrashosds.thrasher:Setting 3.5 to [1, 6, 0, 3, 5, 2] 2024-09-11T08:51:46.842 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.5', '1', '6', '0', '3', '5', '2'] 2024-09-11T08:51:46.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pg-upmap-items 3.5 1 6 0 3 5 2 2024-09-11T08:51:46.845 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:46.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:46.846 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:46.947 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:46.979 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:46.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:46.981 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:47.032 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:51:47.082 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:47.105 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:51:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:51:47.107 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:47 smithi092 ceph-mon[29647]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:51:47.108 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:47 smithi092 ceph-mon[41114]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:51:47.112 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:47.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:47.113 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:47.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:47.234 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:47.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:47.235 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:47.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:47.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:47.373 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:47.373 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:47.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:47 smithi145 ceph-mon[33384]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:51:47.474 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:47.511 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:47.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:47.512 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:47.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:47.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:47.646 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:47.747 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:47.784 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:47.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:47.786 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:47.886 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:47.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:47.919 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:47.919 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:48.020 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:48.057 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:48.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:48.058 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:48.159 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:48.187 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:48.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:48.188 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:48.288 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:48.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:48 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:48.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:48 smithi092 ceph-mon[29647]: pgmap v575: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 863 op/s; 5046/195291 objects misplaced (2.584%) 2024-09-11T08:51:48.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:48 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:48.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:48 smithi092 ceph-mon[41114]: pgmap v575: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 863 op/s; 5046/195291 objects misplaced (2.584%) 2024-09-11T08:51:48.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:48.325 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:48.326 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:48.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:48 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:48.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:48 smithi145 ceph-mon[33384]: pgmap v575: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 863 op/s; 5046/195291 objects misplaced (2.584%) 2024-09-11T08:51:48.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:48.461 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:48.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:48.462 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:48.525 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:51:48.536 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:51:48.563 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:48.567 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:51:48.627 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:48.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:48.628 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:48.729 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:48.767 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:48.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:48.768 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:48.869 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:48.910 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:48.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:48.911 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:49.012 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:49.051 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:49.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:49.053 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:49.154 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:49.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:49.193 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:49.194 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:49.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:49.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:49.333 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:49.334 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:49.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:49.466 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:49.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:49.467 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:49.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:49.599 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:49.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:49.600 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:49.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:49.742 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:49.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:49.743 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:49.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:49.875 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:49.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:49.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:49.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:50.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:50.015 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:50.015 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:50.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:49 smithi092 ceph-mon[29647]: pgmap v576: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 692 op/s; 5046/195291 objects misplaced (2.584%) 2024-09-11T08:51:50.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:49 smithi092 ceph-mon[41114]: pgmap v576: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 692 op/s; 5046/195291 objects misplaced (2.584%) 2024-09-11T08:51:50.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:50.151 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:50.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:50.152 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:50.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:50.286 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:50.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:50.287 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:50.388 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:50.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:49 smithi145 ceph-mon[33384]: pgmap v576: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 692 op/s; 5046/195291 objects misplaced (2.584%) 2024-09-11T08:51:50.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:50.433 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:50.433 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:50.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:50.570 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:50.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:50.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:50.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:50.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:50.709 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:50.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:50.839 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:50.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:50.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:50.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:50.978 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:50.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:50.979 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:51.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:51.111 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:51.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:51.112 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:51.212 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:51.250 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:51.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:51.251 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:51.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:51.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:51.385 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:51.385 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:51.486 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:51.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:51.526 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:51.526 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:51.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:51:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:51:51] "GET /metrics HTTP/1.1" 200 224542 "" "Prometheus/2.43.0" 2024-09-11T08:51:51.627 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:51.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:51.665 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:51.666 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:51.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:51.804 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:51.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:51.805 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:51.906 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:51.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:51.945 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:51.946 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:52.000 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:51:52.001 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:51:52.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:52.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:52.081 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:52.081 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:52.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:52.212 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:52.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:52.213 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:52.314 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:52.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:52.357 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:52.357 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:52.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:52.504 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:52.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:52.505 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:52.605 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:52.618 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:52 smithi092 ceph-mon[29647]: pgmap v577: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 3.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 722 op/s; 5046/192402 objects misplaced (2.623%) 2024-09-11T08:51:52.619 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:52 smithi092 ceph-mon[41114]: pgmap v577: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 3.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 722 op/s; 5046/192402 objects misplaced (2.623%) 2024-09-11T08:51:52.642 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:52.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:52.643 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:52 smithi145 ceph-mon[33384]: pgmap v577: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 3.9 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 722 op/s; 5046/192402 objects misplaced (2.623%) 2024-09-11T08:51:52.744 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:52.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:52.779 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:52.779 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:52.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:52.910 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:52.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:52.911 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:53.012 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:53.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:53.051 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:53.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:53.152 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:53.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:51:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:51:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:51:53.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:53.188 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:53.188 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:53.289 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:53.319 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:53.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:53.321 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:53.422 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:53.456 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:53.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:53.457 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:53.558 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:53.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-11T08:51:53.592 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:53.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:53.593 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:53.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:53 smithi145 ceph-mon[33384]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:51:53.694 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:53.726 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:53.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:53.727 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:53.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:53 smithi092 ceph-mon[29647]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:51:53.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:53 smithi092 ceph-mon[41114]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:51:53.828 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:53.863 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:53.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:53.864 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:53.965 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:54.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:54.006 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:54.006 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:54.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:54.131 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:54.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:54.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:54.233 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:54.269 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:54.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:54.270 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:54.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:54.406 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:54.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:54.408 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:54.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:54.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:54.542 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:54.642 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:54.660 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:54 smithi092 ceph-mon[29647]: pgmap v578: 168 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 166 active+clean; 3.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 884 op/s; 3875/172389 objects misplaced (2.248%) 2024-09-11T08:51:54.660 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:54 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:51:54.660 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:54 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:51:54.660 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:54 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:51:54.660 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:54 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:51:54.660 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:54 smithi092 ceph-mon[41114]: pgmap v578: 168 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 166 active+clean; 3.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 884 op/s; 3875/172389 objects misplaced (2.248%) 2024-09-11T08:51:54.660 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:54 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:51:54.660 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:54 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:51:54.660 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:54 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:51:54.661 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:54 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:51:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:54 smithi145 ceph-mon[33384]: pgmap v578: 168 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 166 active+clean; 3.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 884 op/s; 3875/172389 objects misplaced (2.248%) 2024-09-11T08:51:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:54 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:51:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:54 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:51:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:54 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:51:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:54 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:51:54.674 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:54.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:54.675 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:54.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:54.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:54.817 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:54.817 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:54.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:54.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:54.955 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:54.955 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:55.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:55.090 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:55.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:55.091 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:55.192 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:55.227 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:55.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:55.229 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:55.329 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:55.368 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:55.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:55.369 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:55.470 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:55.509 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:55.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:55.510 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:55.611 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:55.649 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:55.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:55.650 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:55.730 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:55 smithi092 ceph-mon[41114]: osdmap e266: 8 total, 8 up, 7 in 2024-09-11T08:51:55.730 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:55 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3424211866' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:51:55.730 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:51:55.730 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:55 smithi092 ceph-mon[29647]: osdmap e266: 8 total, 8 up, 7 in 2024-09-11T08:51:55.730 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:55 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3424211866' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:51:55.751 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:55.774 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:55.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:55.775 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:55.876 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:55.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:55 smithi145 ceph-mon[33384]: osdmap e266: 8 total, 8 up, 7 in 2024-09-11T08:51:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:55 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3424211866' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:51:55.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:55.915 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:55.915 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:56.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:56.046 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:56.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:56.047 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:56.147 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:56.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:56.186 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:56.186 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:51:56.287 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:56.326 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:56.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:56.327 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:56.428 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:56.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:56.463 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:56.463 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:56.564 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:56.594 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:56.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:56.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:56.696 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:56.729 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:56.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:56.730 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:56.807 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:56 smithi145 ceph-mon[33384]: pgmap v580: 168 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 166 active+clean; 3.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 884 op/s; 3875/172389 objects misplaced (2.248%) 2024-09-11T08:51:56.807 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:56 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:51:56.807 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:56 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3424211866' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:51:56.807 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:56 smithi145 ceph-mon[33384]: osdmap e267: 8 total, 8 up, 7 in 2024-09-11T08:51:56.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:56 smithi092 ceph-mon[29647]: pgmap v580: 168 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 166 active+clean; 3.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 884 op/s; 3875/172389 objects misplaced (2.248%) 2024-09-11T08:51:56.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:56 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:51:56.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:56 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3424211866' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:51:56.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:56 smithi092 ceph-mon[29647]: osdmap e267: 8 total, 8 up, 7 in 2024-09-11T08:51:56.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:56 smithi092 ceph-mon[41114]: pgmap v580: 168 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 166 active+clean; 3.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 884 op/s; 3875/172389 objects misplaced (2.248%) 2024-09-11T08:51:56.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:56 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:51:56.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:56 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3424211866' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:51:56.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:56 smithi092 ceph-mon[41114]: osdmap e267: 8 total, 8 up, 7 in 2024-09-11T08:51:56.831 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:56.869 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:56.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:56.870 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:56.970 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:57.006 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:57.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:57.007 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:57.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:57.142 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:57.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:57.143 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:57.161 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:51:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:51:57.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:51:57.278 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:57.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:57.279 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:51:57.380 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:57.414 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:57.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:57.415 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:57.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:57.547 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:57.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:57.549 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:57.649 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:57.687 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:57.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:57.688 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:57.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:57.819 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:57.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:57.820 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:57.921 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:57.958 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:57.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:57.959 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:58.060 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:58.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:58.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:58.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:58.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:58.226 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:58.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:58.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:51:58.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:58.366 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:58.366 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:51:58.467 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:58.502 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:58.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:58.504 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:58.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:58 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:58.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:58 smithi092 ceph-mon[29647]: pgmap v582: 168 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 166 active+clean; 3.0 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.90k op/s; 2499/149745 objects misplaced (1.669%); 12 MiB/s, 188 objects/s recovering 2024-09-11T08:51:58.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:58 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:58.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:58 smithi092 ceph-mon[41114]: pgmap v582: 168 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 166 active+clean; 3.0 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.90k op/s; 2499/149745 objects misplaced (1.669%); 12 MiB/s, 188 objects/s recovering 2024-09-11T08:51:58.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:58.631 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:58.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:58.632 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:58.732 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:58.754 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:58.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:58.755 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:58.856 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:58.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:58.878 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:58.878 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:58.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:58 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:51:58.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:58 smithi145 ceph-mon[33384]: pgmap v582: 168 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 166 active+clean; 3.0 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.90k op/s; 2499/149745 objects misplaced (1.669%); 12 MiB/s, 188 objects/s recovering 2024-09-11T08:51:58.979 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:59.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:59.015 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:59.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:59.116 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:59.131 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:51:59.132 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:51:59.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:59.154 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:59.154 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:59.163 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:51:59.164 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:51:59.197 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:51:59.197 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:51:59.225 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:51:59.226 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:51:59.250 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:51:59.251 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:51:59.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:51:59.283 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:51:59.283 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:51:59.290 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:59.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:59.291 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:51:59.314 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:51:59.314 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:51:59.341 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:51:59.341 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:51:59.393 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:51:59.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:59.429 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:59.430 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:51:59.531 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:51:59.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:59 smithi092 ceph-mon[29647]: osdmap e268: 8 total, 8 up, 7 in 2024-09-11T08:51:59.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:59 smithi092 ceph-mon[29647]: osdmap e269: 8 total, 8 up, 7 in 2024-09-11T08:51:59.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:51:59 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1791177374' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:51:59.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:59 smithi092 ceph-mon[41114]: osdmap e268: 8 total, 8 up, 7 in 2024-09-11T08:51:59.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:59 smithi092 ceph-mon[41114]: osdmap e269: 8 total, 8 up, 7 in 2024-09-11T08:51:59.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:51:59 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1791177374' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:51:59.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:59.569 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:59.569 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:51:59.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:51:59.701 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:51:59.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:59.702 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:51:59.804 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:51:59.841 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:59.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:59.843 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:51:59.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:59 smithi145 ceph-mon[33384]: osdmap e268: 8 total, 8 up, 7 in 2024-09-11T08:51:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:59 smithi145 ceph-mon[33384]: osdmap e269: 8 total, 8 up, 7 in 2024-09-11T08:51:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:51:59 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1791177374' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:51:59.943 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:51:59.984 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:51:59.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:51:59.985 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:00.085 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:00.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:00.124 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:00.124 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:00.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:00.254 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:00.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:00.255 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:00.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:00.387 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:00.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:00.388 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:00.489 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:00.523 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:00.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:00.524 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:00.625 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:00.664 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:00.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:00.665 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:00.717 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:00 smithi092 ceph-mon[29647]: pgmap v585: 168 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 166 active+clean; 3.0 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.54k op/s; 2499/149745 objects misplaced (1.669%); 12 MiB/s, 189 objects/s recovering 2024-09-11T08:52:00.717 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:52:00.717 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:00 smithi092 ceph-mon[41114]: pgmap v585: 168 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 166 active+clean; 3.0 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.54k op/s; 2499/149745 objects misplaced (1.669%); 12 MiB/s, 189 objects/s recovering 2024-09-11T08:52:00.717 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:52:00.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:00.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:00.801 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:00.801 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:00.902 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:00.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:00 smithi145 ceph-mon[33384]: pgmap v585: 168 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 166 active+clean; 3.0 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.54k op/s; 2499/149745 objects misplaced (1.669%); 12 MiB/s, 189 objects/s recovering 2024-09-11T08:52:00.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:52:00.935 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:00.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:00.936 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:01.037 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:01.071 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:01.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:01.072 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:01.172 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:01.209 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:01.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:01.210 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:01.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:01.353 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:01.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:01.354 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:01.455 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:01.490 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:01.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:01.491 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:01.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:01.623 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:52:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:52:01] "GET /metrics HTTP/1.1" 200 224567 "" "Prometheus/2.43.0" 2024-09-11T08:52:01.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:01.631 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:01.631 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:01.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:01.767 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:01.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:01.768 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:01.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:52:01.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:01.910 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:01.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:01.912 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:02.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:02.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:02.054 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:02.055 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:02.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:02.199 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:02.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:02.200 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:02.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:02.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:02.333 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:02.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:02.464 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:02.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:02.465 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:02.563 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:52:02.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:02 smithi092 ceph-mon[29647]: pgmap v586: 168 pgs: 1 active+remapped+backfill_toofull, 167 active+clean; 3.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.45k op/s; 2378/146340 objects misplaced (1.625%); 11 MiB/s, 177 objects/s recovering 2024-09-11T08:52:02.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:02 smithi092 ceph-mon[41114]: pgmap v586: 168 pgs: 1 active+remapped+backfill_toofull, 167 active+clean; 3.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.45k op/s; 2378/146340 objects misplaced (1.625%); 11 MiB/s, 177 objects/s recovering 2024-09-11T08:52:02.566 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:02.604 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:02.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:02.605 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:02.706 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:02.723 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:02 smithi145 ceph-mon[33384]: pgmap v586: 168 pgs: 1 active+remapped+backfill_toofull, 167 active+clean; 3.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.45k op/s; 2378/146340 objects misplaced (1.625%); 11 MiB/s, 177 objects/s recovering 2024-09-11T08:52:02.743 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:02.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:02.744 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:02.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:02.877 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:02.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:02.878 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:02.979 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:03.017 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:03.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:03.018 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:03.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:03.161 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:03.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:03.162 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:03.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:52:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:52:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:52:03.262 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:03.296 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:03.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:03.297 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:03.398 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:03.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:03.434 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:03.434 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:03.455 INFO:teuthology.orchestra.run.smithi092.stderr:set 3.5 pg_upmap_items mapping to [1->6,0->3,5->2] 2024-09-11T08:52:03.535 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:03.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:03.572 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:03.572 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:03.673 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:03.699 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:03.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:03.700 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:03.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:03 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2791276790' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "3.5", "id": [1, 6, 0, 3, 5, 2]}]: dispatch 2024-09-11T08:52:03.894 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:03 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "3.5", "id": [1, 6, 0, 3, 5, 2]}]: dispatch 2024-09-11T08:52:03.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:03 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2791276790' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "3.5", "id": [1, 6, 0, 3, 5, 2]}]: dispatch 2024-09-11T08:52:03.894 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:03 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "3.5", "id": [1, 6, 0, 3, 5, 2]}]: dispatch 2024-09-11T08:52:03.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:03.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:03 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2791276790' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "3.5", "id": [1, 6, 0, 3, 5, 2]}]: dispatch 2024-09-11T08:52:03.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:03 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "3.5", "id": [1, 6, 0, 3, 5, 2]}]: dispatch 2024-09-11T08:52:03.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:03.937 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:03.937 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:04.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:04.071 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:04.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:04.072 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:04.173 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:04.194 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:04.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:04.195 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:04.296 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:04.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:04.330 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:04.330 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:04.395 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:52:04.395 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-11T08:52:04.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-11T08:52:04.395 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:52:04.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.395 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:242089 7.a 7:562e1108:::benchmark_data_smithi092_84565_object116561:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-11T08:52:04.395 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:36.307092+0000", 2024-09-11T08:52:04.395 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 448.08551337599999, 2024-09-11T08:52:04.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.123527491, 2024-09-11T08:52:04.395 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.395 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:04.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:52:04.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:52:04.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:52:04.396 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 242089 2024-09-11T08:52:04.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.396 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.307092+0000", 2024-09-11T08:52:04.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.307092+0000", 2024-09-11T08:52:04.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.307091+0000", 2024-09-11T08:52:04.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:04.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.307096+0000", 2024-09-11T08:52:04.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2024-09-11T08:52:04.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.307108+0000", 2024-09-11T08:52:04.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1867e-05 2024-09-11T08:52:04.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.307113+0000", 2024-09-11T08:52:04.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.575e-06 2024-09-11T08:52:04.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.308471+0000", 2024-09-11T08:52:04.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001357673 2024-09-11T08:52:04.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:52:04.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.308577+0000", 2024-09-11T08:52:04.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106085 2024-09-11T08:52:04.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,5", 2024-09-11T08:52:04.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.308681+0000", 2024-09-11T08:52:04.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010356099999999999 2024-09-11T08:52:04.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.318220+0000", 2024-09-11T08:52:04.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095396560000000005 2024-09-11T08:52:04.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.318330+0000", 2024-09-11T08:52:04.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010933399999999999 2024-09-11T08:52:04.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:52:04.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430532+0000", 2024-09-11T08:52:04.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.112202185 2024-09-11T08:52:04.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:52:04.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430580+0000", 2024-09-11T08:52:04.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8353999999999998e-05 2024-09-11T08:52:04.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:36.430620+0000", 2024-09-11T08:52:04.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9521000000000003e-05 2024-09-11T08:52:04.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.401 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.402 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:125090 8.f 8:fb475083:::benchmark_data_smithi092_117541_object5129:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e184)", 2024-09-11T08:52:04.402 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:47:04.928024+0000", 2024-09-11T08:52:04.402 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 299.46458184800002, 2024-09-11T08:52:04.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.130102088, 2024-09-11T08:52:04.402 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.402 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:04.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:52:04.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:52:04.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:52:04.402 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 125090 2024-09-11T08:52:04.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.402 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928024+0000", 2024-09-11T08:52:04.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928024+0000", 2024-09-11T08:52:04.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928023+0000", 2024-09-11T08:52:04.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:04.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928028+0000", 2024-09-11T08:52:04.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5539999999999997e-06 2024-09-11T08:52:04.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928042+0000", 2024-09-11T08:52:04.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.359e-05 2024-09-11T08:52:04.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928047+0000", 2024-09-11T08:52:04.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5820000000000001e-06 2024-09-11T08:52:04.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928163+0000", 2024-09-11T08:52:04.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116062 2024-09-11T08:52:04.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:52:04.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928549+0000", 2024-09-11T08:52:04.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000385604 2024-09-11T08:52:04.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:52:04.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.928912+0000", 2024-09-11T08:52:04.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036309800000000001 2024-09-11T08:52:04.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:52:04.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.936819+0000", 2024-09-11T08:52:04.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079072350000000003 2024-09-11T08:52:04.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:04.938487+0000", 2024-09-11T08:52:04.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016671349999999999 2024-09-11T08:52:04.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:05.058044+0000", 2024-09-11T08:52:04.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.119557371 2024-09-11T08:52:04.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:52:04.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:05.058090+0000", 2024-09-11T08:52:04.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6236000000000003e-05 2024-09-11T08:52:04.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:47:05.058126+0000", 2024-09-11T08:52:04.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5846000000000002e-05 2024-09-11T08:52:04.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.408 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.408 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120753 9.1 9:80027676:::benchmark_data_smithi092_145083_object120752:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e234)", 2024-09-11T08:52:04.408 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.482645+0000", 2024-09-11T08:52:04.408 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 98.909960436999995, 2024-09-11T08:52:04.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.202329761, 2024-09-11T08:52:04.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.408 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:52:04.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:52:04.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:52:04.409 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:04.409 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120753 2024-09-11T08:52:04.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.409 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482645+0000", 2024-09-11T08:52:04.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482645+0000", 2024-09-11T08:52:04.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482644+0000", 2024-09-11T08:52:04.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:52:04.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482670+0000", 2024-09-11T08:52:04.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6781999999999999e-05 2024-09-11T08:52:04.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482682+0000", 2024-09-11T08:52:04.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2051e-05 2024-09-11T08:52:04.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482688+0000", 2024-09-11T08:52:04.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0870000000000003e-06 2024-09-11T08:52:04.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.487390+0000", 2024-09-11T08:52:04.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047011450000000003 2024-09-11T08:52:04.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:52:04.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.487514+0000", 2024-09-11T08:52:04.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000124664 2024-09-11T08:52:04.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6,7", 2024-09-11T08:52:04.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.487660+0000", 2024-09-11T08:52:04.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014545500000000001 2024-09-11T08:52:04.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.496777+0000", 2024-09-11T08:52:04.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0091171150000000003 2024-09-11T08:52:04.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:52:04.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.497628+0000", 2024-09-11T08:52:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00085127100000000002 2024-09-11T08:52:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684968+0000", 2024-09-11T08:52:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.187339704 2024-09-11T08:52:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684975+0000", 2024-09-11T08:52:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-11T08:52:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.415 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120754 9.1 9:8766ca32:::benchmark_data_smithi092_145083_object120753:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e234)", 2024-09-11T08:52:04.415 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.483486+0000", 2024-09-11T08:52:04.415 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 98.909119595999996, 2024-09-11T08:52:04.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20167465100000001, 2024-09-11T08:52:04.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.415 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:52:04.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:52:04.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:52:04.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:04.415 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120754 2024-09-11T08:52:04.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.415 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.483486+0000", 2024-09-11T08:52:04.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.483486+0000", 2024-09-11T08:52:04.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.483485+0000", 2024-09-11T08:52:04.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:52:04.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.483509+0000", 2024-09-11T08:52:04.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3549000000000001e-05 2024-09-11T08:52:04.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.483521+0000", 2024-09-11T08:52:04.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2855e-05 2024-09-11T08:52:04.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.483527+0000", 2024-09-11T08:52:04.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8300000000000001e-06 2024-09-11T08:52:04.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.492869+0000", 2024-09-11T08:52:04.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093416190000000007 2024-09-11T08:52:04.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:52:04.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.493003+0000", 2024-09-11T08:52:04.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013431599999999999 2024-09-11T08:52:04.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6,7", 2024-09-11T08:52:04.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.493150+0000", 2024-09-11T08:52:04.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014698099999999999 2024-09-11T08:52:04.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:52:04.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.497632+0000", 2024-09-11T08:52:04.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044822689999999997 2024-09-11T08:52:04.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685155+0000", 2024-09-11T08:52:04.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18752247599999999 2024-09-11T08:52:04.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685161+0000", 2024-09-11T08:52:04.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.908e-06 2024-09-11T08:52:04.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.420 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.420 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.15366.0:120754 9.1 e234/206)", 2024-09-11T08:52:04.420 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.496702+0000", 2024-09-11T08:52:04.420 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 98.895903270999995, 2024-09-11T08:52:04.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18818669199999999, 2024-09-11T08:52:04.421 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.421 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:52:04.421 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.496702+0000", 2024-09-11T08:52:04.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.496702+0000", 2024-09-11T08:52:04.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.496702+0000", 2024-09-11T08:52:04.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:04.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.496705+0000", 2024-09-11T08:52:04.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.518e-06 2024-09-11T08:52:04.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.496716+0000", 2024-09-11T08:52:04.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0454e-05 2024-09-11T08:52:04.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.496720+0000", 2024-09-11T08:52:04.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1779999999999996e-06 2024-09-11T08:52:04.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684876+0000", 2024-09-11T08:52:04.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18815546899999999 2024-09-11T08:52:04.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684889+0000", 2024-09-11T08:52:04.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3625000000000001e-05 2024-09-11T08:52:04.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.424 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.424 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120753 9.1 9:80027676:::benchmark_data_smithi092_145083_object120752:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:52:04.424 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.504067+0000", 2024-09-11T08:52:04.424 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 98.888538268999994, 2024-09-11T08:52:04.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0247133180000001, 2024-09-11T08:52:04.424 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:52:04.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:52:04.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:52:04.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:04.425 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120753 2024-09-11T08:52:04.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.425 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504067+0000", 2024-09-11T08:52:04.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504067+0000", 2024-09-11T08:52:04.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504066+0000", 2024-09-11T08:52:04.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:04.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504100+0000", 2024-09-11T08:52:04.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4513e-05 2024-09-11T08:52:04.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504114+0000", 2024-09-11T08:52:04.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305999999999999e-05 2024-09-11T08:52:04.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504120+0000", 2024-09-11T08:52:04.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2450000000000003e-06 2024-09-11T08:52:04.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685178+0000", 2024-09-11T08:52:04.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.181057575 2024-09-11T08:52:04.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:52:04.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685184+0000", 2024-09-11T08:52:04.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9600000000000003e-06 2024-09-11T08:52:04.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.528703+0000", 2024-09-11T08:52:04.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84351812699999995 2024-09-11T08:52:04.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.528781+0000", 2024-09-11T08:52:04.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8183999999999995e-05 2024-09-11T08:52:04.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.429 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.429 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120754 9.1 9:8766ca32:::benchmark_data_smithi092_145083_object120753:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:52:04.429 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.504271+0000", 2024-09-11T08:52:04.429 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 98.888334684, 2024-09-11T08:52:04.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0247913390000001, 2024-09-11T08:52:04.429 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.429 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:52:04.429 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:52:04.429 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:52:04.429 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120754 2024-09-11T08:52:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504271+0000", 2024-09-11T08:52:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504271+0000", 2024-09-11T08:52:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504270+0000", 2024-09-11T08:52:04.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:04.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504292+0000", 2024-09-11T08:52:04.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1151000000000001e-05 2024-09-11T08:52:04.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504299+0000", 2024-09-11T08:52:04.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9079999999999995e-06 2024-09-11T08:52:04.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504304+0000", 2024-09-11T08:52:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5109999999999997e-06 2024-09-11T08:52:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685200+0000", 2024-09-11T08:52:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18089587500000001 2024-09-11T08:52:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:52:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685205+0000", 2024-09-11T08:52:04.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9439999999999996e-06 2024-09-11T08:52:04.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529010+0000", 2024-09-11T08:52:04.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84380478599999997 2024-09-11T08:52:04.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529062+0000", 2024-09-11T08:52:04.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2797999999999998e-05 2024-09-11T08:52:04.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.433 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.434 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.15366.0:120753 9.1 e234/206)", 2024-09-11T08:52:04.434 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.507259+0000", 2024-09-11T08:52:04.434 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 98.885346698000006, 2024-09-11T08:52:04.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.17765646299999999, 2024-09-11T08:52:04.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:52:04.434 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507259+0000", 2024-09-11T08:52:04.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507259+0000", 2024-09-11T08:52:04.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507259+0000", 2024-09-11T08:52:04.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:04.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507262+0000", 2024-09-11T08:52:04.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7000000000000002e-06 2024-09-11T08:52:04.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507274+0000", 2024-09-11T08:52:04.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1742999999999999e-05 2024-09-11T08:52:04.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507281+0000", 2024-09-11T08:52:04.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5420000000000002e-06 2024-09-11T08:52:04.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684905+0000", 2024-09-11T08:52:04.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.177624169 2024-09-11T08:52:04.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684916+0000", 2024-09-11T08:52:04.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0679000000000001e-05 2024-09-11T08:52:04.437 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.437 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.437 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.437 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.15366.0:120754 9.1 e234/206)", 2024-09-11T08:52:04.437 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.507355+0000", 2024-09-11T08:52:04.438 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 98.885250944000006, 2024-09-11T08:52:04.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.17758486300000001, 2024-09-11T08:52:04.438 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.438 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:52:04.438 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507355+0000", 2024-09-11T08:52:04.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:04.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507355+0000", 2024-09-11T08:52:04.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507354+0000", 2024-09-11T08:52:04.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:04.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507358+0000", 2024-09-11T08:52:04.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7610000000000001e-06 2024-09-11T08:52:04.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507365+0000", 2024-09-11T08:52:04.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4860000000000002e-06 2024-09-11T08:52:04.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507369+0000", 2024-09-11T08:52:04.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9779999999999999e-06 2024-09-11T08:52:04.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684929+0000", 2024-09-11T08:52:04.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.17756022599999999 2024-09-11T08:52:04.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684940+0000", 2024-09-11T08:52:04.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0879e-05 2024-09-11T08:52:04.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.442 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.443 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120773 9.1 9:84313f40:::benchmark_data_smithi092_145083_object120772:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:52:04.443 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.524403+0000", 2024-09-11T08:52:04.443 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 98.868202671999995, 2024-09-11T08:52:04.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0196523550000001, 2024-09-11T08:52:04.443 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.443 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:04.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:52:04.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:52:04.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:04.443 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120773 2024-09-11T08:52:04.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.444 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524403+0000", 2024-09-11T08:52:04.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524403+0000", 2024-09-11T08:52:04.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524402+0000", 2024-09-11T08:52:04.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:04.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524428+0000", 2024-09-11T08:52:04.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6698000000000001e-05 2024-09-11T08:52:04.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524442+0000", 2024-09-11T08:52:04.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3806e-05 2024-09-11T08:52:04.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524448+0000", 2024-09-11T08:52:04.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3109999999999996e-06 2024-09-11T08:52:04.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685220+0000", 2024-09-11T08:52:04.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16077285399999999 2024-09-11T08:52:04.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:52:04.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685225+0000", 2024-09-11T08:52:04.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8810000000000004e-06 2024-09-11T08:52:04.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529083+0000", 2024-09-11T08:52:04.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84385757900000002 2024-09-11T08:52:04.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:52:04.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529204+0000", 2024-09-11T08:52:04.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121352 2024-09-11T08:52:04.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6,7", 2024-09-11T08:52:04.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529650+0000", 2024-09-11T08:52:04.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044596399999999997 2024-09-11T08:52:04.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:52:04.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.533815+0000", 2024-09-11T08:52:04.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041648370000000002 2024-09-11T08:52:04.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.536911+0000", 2024-09-11T08:52:04.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030961280000000001 2024-09-11T08:52:04.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.543805+0000", 2024-09-11T08:52:04.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006893939 2024-09-11T08:52:04.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:52:04.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.544015+0000", 2024-09-11T08:52:04.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020939999999999999 2024-09-11T08:52:04.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.544055+0000", 2024-09-11T08:52:04.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0908000000000002e-05 2024-09-11T08:52:04.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.450 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.450 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:122641 9.19 9:9f7f102d:::benchmark_data_smithi092_145083_object122640:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:52:04.450 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:27.538908+0000", 2024-09-11T08:52:04.450 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.853697273999998, 2024-09-11T08:52:04.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.439093481, 2024-09-11T08:52:04.450 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.450 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:04.450 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:52:04.450 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:52:04.451 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:04.451 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122641 2024-09-11T08:52:04.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.451 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538908+0000", 2024-09-11T08:52:04.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538908+0000", 2024-09-11T08:52:04.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538907+0000", 2024-09-11T08:52:04.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:04.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538933+0000", 2024-09-11T08:52:04.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5627e-05 2024-09-11T08:52:04.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538945+0000", 2024-09-11T08:52:04.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2721000000000001e-05 2024-09-11T08:52:04.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538951+0000", 2024-09-11T08:52:04.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5849999999999999e-06 2024-09-11T08:52:04.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.539347+0000", 2024-09-11T08:52:04.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000395936 2024-09-11T08:52:04.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:52:04.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.539355+0000", 2024-09-11T08:52:04.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6629999999999999e-06 2024-09-11T08:52:04.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.534990+0000", 2024-09-11T08:52:04.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99563493400000003 2024-09-11T08:52:04.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:52:04.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535002+0000", 2024-09-11T08:52:04.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1333e-05 2024-09-11T08:52:04.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.963852+0000", 2024-09-11T08:52:04.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.42885004700000001 2024-09-11T08:52:04.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:52:04.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.964026+0000", 2024-09-11T08:52:04.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017450399999999999 2024-09-11T08:52:04.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6,7", 2024-09-11T08:52:04.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.964280+0000", 2024-09-11T08:52:04.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025399599999999999 2024-09-11T08:52:04.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:52:04.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.965296+0000", 2024-09-11T08:52:04.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010152290000000001 2024-09-11T08:52:04.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.969974+0000", 2024-09-11T08:52:04.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046781729999999999 2024-09-11T08:52:04.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.971213+0000", 2024-09-11T08:52:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001239431 2024-09-11T08:52:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.977816+0000", 2024-09-11T08:52:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066024580000000003 2024-09-11T08:52:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:52:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.977893+0000", 2024-09-11T08:52:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7600000000000002e-05 2024-09-11T08:52:04.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.978002+0000", 2024-09-11T08:52:04.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010877199999999999 2024-09-11T08:52:04.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.458 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.458 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:122656 9.19 9:9b532791:::benchmark_data_smithi092_145083_object122655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:52:04.458 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:27.549187+0000", 2024-09-11T08:52:04.458 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.843418482999994, 2024-09-11T08:52:04.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4294275089999999, 2024-09-11T08:52:04.459 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.459 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:04.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:52:04.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:52:04.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:04.459 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122656 2024-09-11T08:52:04.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.459 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549187+0000", 2024-09-11T08:52:04.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549187+0000", 2024-09-11T08:52:04.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549186+0000", 2024-09-11T08:52:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549208+0000", 2024-09-11T08:52:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2378000000000002e-05 2024-09-11T08:52:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549220+0000", 2024-09-11T08:52:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1711e-05 2024-09-11T08:52:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549226+0000", 2024-09-11T08:52:04.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7810000000000002e-06 2024-09-11T08:52:04.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549320+0000", 2024-09-11T08:52:04.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3955000000000004e-05 2024-09-11T08:52:04.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:52:04.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549329+0000", 2024-09-11T08:52:04.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.952e-06 2024-09-11T08:52:04.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535136+0000", 2024-09-11T08:52:04.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98580697900000003 2024-09-11T08:52:04.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:52:04.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535143+0000", 2024-09-11T08:52:04.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4440000000000002e-06 2024-09-11T08:52:04.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.965346+0000", 2024-09-11T08:52:04.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43020250500000001 2024-09-11T08:52:04.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:52:04.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.965438+0000", 2024-09-11T08:52:04.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2334000000000001e-05 2024-09-11T08:52:04.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6,7", 2024-09-11T08:52:04.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.965597+0000", 2024-09-11T08:52:04.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015942300000000001 2024-09-11T08:52:04.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:52:04.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.966745+0000", 2024-09-11T08:52:04.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001147482 2024-09-11T08:52:04.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.971264+0000", 2024-09-11T08:52:04.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045190960000000002 2024-09-11T08:52:04.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.978072+0000", 2024-09-11T08:52:04.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068077540000000001 2024-09-11T08:52:04.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.978396+0000", 2024-09-11T08:52:04.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000324247 2024-09-11T08:52:04.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:52:04.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.978508+0000", 2024-09-11T08:52:04.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112379 2024-09-11T08:52:04.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.978615+0000", 2024-09-11T08:52:04.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010634800000000001 2024-09-11T08:52:04.467 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.467 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.467 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.467 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:165712 9.c 9:35622153:::benchmark_data_smithi092_145083_object22314:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-11T08:52:04.467 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:51:02.878895+0000", 2024-09-11T08:52:04.467 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.513710731000003, 2024-09-11T08:52:04.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.123753609, 2024-09-11T08:52:04.467 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.467 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:04.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:52:04.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:52:04.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:04.468 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 165712 2024-09-11T08:52:04.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.468 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.878895+0000", 2024-09-11T08:52:04.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.878895+0000", 2024-09-11T08:52:04.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.878894+0000", 2024-09-11T08:52:04.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:04.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.878898+0000", 2024-09-11T08:52:04.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6669999999999998e-06 2024-09-11T08:52:04.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.878906+0000", 2024-09-11T08:52:04.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.943e-06 2024-09-11T08:52:04.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.878910+0000", 2024-09-11T08:52:04.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3250000000000001e-06 2024-09-11T08:52:04.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.887025+0000", 2024-09-11T08:52:04.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0081146870000000006 2024-09-11T08:52:04.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:52:04.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.887132+0000", 2024-09-11T08:52:04.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010688699999999999 2024-09-11T08:52:04.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,7", 2024-09-11T08:52:04.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.887242+0000", 2024-09-11T08:52:04.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110163 2024-09-11T08:52:04.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:52:04.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.889344+0000", 2024-09-11T08:52:04.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002102179 2024-09-11T08:52:04.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.890337+0000", 2024-09-11T08:52:04.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00099273299999999998 2024-09-11T08:52:04.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.002569+0000", 2024-09-11T08:52:04.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.112232304 2024-09-11T08:52:04.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:52:04.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.002611+0000", 2024-09-11T08:52:04.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.197e-05 2024-09-11T08:52:04.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.002649+0000", 2024-09-11T08:52:04.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7465999999999997e-05 2024-09-11T08:52:04.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.473 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.473 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:165732 9.2 9:47194da5:::benchmark_data_smithi092_145083_object22334:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-11T08:52:04.473 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:51:02.891989+0000", 2024-09-11T08:52:04.473 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.500617212999998, 2024-09-11T08:52:04.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.13579332399999999, 2024-09-11T08:52:04.474 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.474 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:04.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:52:04.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:52:04.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:04.474 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 165732 2024-09-11T08:52:04.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.474 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.891989+0000", 2024-09-11T08:52:04.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.891989+0000", 2024-09-11T08:52:04.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.891988+0000", 2024-09-11T08:52:04.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:52:04.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.891993+0000", 2024-09-11T08:52:04.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.48e-06 2024-09-11T08:52:04.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.996787+0000", 2024-09-11T08:52:04.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104793624 2024-09-11T08:52:04.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.996795+0000", 2024-09-11T08:52:04.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7520000000000002e-06 2024-09-11T08:52:04.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.997074+0000", 2024-09-11T08:52:04.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027848 2024-09-11T08:52:04.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:52:04.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.997387+0000", 2024-09-11T08:52:04.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031345599999999999 2024-09-11T08:52:04.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:52:04.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.998385+0000", 2024-09-11T08:52:04.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00099731600000000004 2024-09-11T08:52:04.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:52:04.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.000687+0000", 2024-09-11T08:52:04.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023024920000000002 2024-09-11T08:52:04.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.024796+0000", 2024-09-11T08:52:04.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024108950000000001 2024-09-11T08:52:04.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.027696+0000", 2024-09-11T08:52:04.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002899413 2024-09-11T08:52:04.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:52:04.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.027745+0000", 2024-09-11T08:52:04.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8912000000000002e-05 2024-09-11T08:52:04.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.027782+0000", 2024-09-11T08:52:04.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7345000000000002e-05 2024-09-11T08:52:04.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.479 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.479 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:166596 9.2 9:45208a3d:::benchmark_data_smithi092_145083_object23198:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-11T08:52:04.479 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:51:03.529849+0000", 2024-09-11T08:52:04.480 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.862757201999997, 2024-09-11T08:52:04.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.123370784, 2024-09-11T08:52:04.480 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.480 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:04.480 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:52:04.480 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:52:04.480 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:04.480 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 166596 2024-09-11T08:52:04.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.480 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.529849+0000", 2024-09-11T08:52:04.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.529849+0000", 2024-09-11T08:52:04.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.529847+0000", 2024-09-11T08:52:04.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:04.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.529852+0000", 2024-09-11T08:52:04.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3480000000000003e-06 2024-09-11T08:52:04.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.529862+0000", 2024-09-11T08:52:04.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2450000000000004e-06 2024-09-11T08:52:04.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.529866+0000", 2024-09-11T08:52:04.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4769999999999997e-06 2024-09-11T08:52:04.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.529912+0000", 2024-09-11T08:52:04.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5812000000000001e-05 2024-09-11T08:52:04.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:52:04.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.530022+0000", 2024-09-11T08:52:04.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010983400000000001 2024-09-11T08:52:04.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:52:04.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.530389+0000", 2024-09-11T08:52:04.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036710200000000001 2024-09-11T08:52:04.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.535081+0000", 2024-09-11T08:52:04.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046917410000000001 2024-09-11T08:52:04.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:52:04.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.536063+0000", 2024-09-11T08:52:04.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00098251799999999993 2024-09-11T08:52:04.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.653136+0000", 2024-09-11T08:52:04.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11707337900000001 2024-09-11T08:52:04.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:52:04.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.653180+0000", 2024-09-11T08:52:04.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3770999999999998e-05 2024-09-11T08:52:04.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.653219+0000", 2024-09-11T08:52:04.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9140999999999997e-05 2024-09-11T08:52:04.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.486 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.487 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.487 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:173079 9.c 9:393dc044:::benchmark_data_smithi092_145083_object29681:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-11T08:52:04.487 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:51:08.290684+0000", 2024-09-11T08:52:04.487 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.101921587, 2024-09-11T08:52:04.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12909563199999999, 2024-09-11T08:52:04.487 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.487 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:04.487 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:52:04.487 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:52:04.487 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:04.487 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 173079 2024-09-11T08:52:04.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.488 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.290684+0000", 2024-09-11T08:52:04.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.290684+0000", 2024-09-11T08:52:04.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.290683+0000", 2024-09-11T08:52:04.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:52:04.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.290689+0000", 2024-09-11T08:52:04.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3179999999999997e-06 2024-09-11T08:52:04.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.290699+0000", 2024-09-11T08:52:04.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0399e-05 2024-09-11T08:52:04.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.290705+0000", 2024-09-11T08:52:04.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0669999999999997e-06 2024-09-11T08:52:04.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.290736+0000", 2024-09-11T08:52:04.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1389000000000001e-05 2024-09-11T08:52:04.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:52:04.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.290844+0000", 2024-09-11T08:52:04.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107899 2024-09-11T08:52:04.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,7", 2024-09-11T08:52:04.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.290950+0000", 2024-09-11T08:52:04.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597799999999999 2024-09-11T08:52:04.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:52:04.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.293233+0000", 2024-09-11T08:52:04.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022830849999999998 2024-09-11T08:52:04.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.293840+0000", 2024-09-11T08:52:04.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060659500000000003 2024-09-11T08:52:04.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.419701+0000", 2024-09-11T08:52:04.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.125861 2024-09-11T08:52:04.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:52:04.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.419747+0000", 2024-09-11T08:52:04.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5686000000000003e-05 2024-09-11T08:52:04.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.419780+0000", 2024-09-11T08:52:04.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3170999999999998e-05 2024-09-11T08:52:04.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.493 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.493 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:173084 9.c 9:3261fbae:::benchmark_data_smithi092_145083_object29686:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-11T08:52:04.493 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:51:08.295456+0000", 2024-09-11T08:52:04.493 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.097149692999999, 2024-09-11T08:52:04.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12789482599999999, 2024-09-11T08:52:04.493 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.493 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:04.494 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:52:04.494 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:52:04.494 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:04.494 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 173084 2024-09-11T08:52:04.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.494 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.295456+0000", 2024-09-11T08:52:04.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.295456+0000", 2024-09-11T08:52:04.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.295454+0000", 2024-09-11T08:52:04.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:04.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.295461+0000", 2024-09-11T08:52:04.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.088e-06 2024-09-11T08:52:04.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.295469+0000", 2024-09-11T08:52:04.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7800000000000006e-06 2024-09-11T08:52:04.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.295473+0000", 2024-09-11T08:52:04.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1450000000000001e-06 2024-09-11T08:52:04.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.295503+0000", 2024-09-11T08:52:04.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9895000000000001e-05 2024-09-11T08:52:04.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:52:04.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.295578+0000", 2024-09-11T08:52:04.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5050999999999997e-05 2024-09-11T08:52:04.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,7", 2024-09-11T08:52:04.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.295656+0000", 2024-09-11T08:52:04.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7883e-05 2024-09-11T08:52:04.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.419676+0000", 2024-09-11T08:52:04.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12402015800000001 2024-09-11T08:52:04.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.419837+0000", 2024-09-11T08:52:04.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160178 2024-09-11T08:52:04.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:52:04.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.423280+0000", 2024-09-11T08:52:04.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034431050000000001 2024-09-11T08:52:04.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:52:04.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.423315+0000", 2024-09-11T08:52:04.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4861999999999999e-05 2024-09-11T08:52:04.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.423351+0000", 2024-09-11T08:52:04.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6312000000000001e-05 2024-09-11T08:52:04.499 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.499 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.499 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.500 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15366.0:173088 9.14 e251/235 9:2d459e2a:::benchmark_data_smithi092_145083_object29690:head v 251'9263, mlcod=251'9261)", 2024-09-11T08:52:04.500 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:51:08.296694+0000", 2024-09-11T08:52:04.500 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.095912017000003, 2024-09-11T08:52:04.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.126694372, 2024-09-11T08:52:04.500 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.500 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:04.500 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.296694+0000", 2024-09-11T08:52:04.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.296694+0000", 2024-09-11T08:52:04.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.296693+0000", 2024-09-11T08:52:04.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:04.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.296699+0000", 2024-09-11T08:52:04.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.896e-06 2024-09-11T08:52:04.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.296709+0000", 2024-09-11T08:52:04.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.747e-06 2024-09-11T08:52:04.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.296714+0000", 2024-09-11T08:52:04.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2569999999999998e-06 2024-09-11T08:52:04.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.296754+0000", 2024-09-11T08:52:04.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9520000000000001e-05 2024-09-11T08:52:04.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:52:04.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.296771+0000", 2024-09-11T08:52:04.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7824000000000001e-05 2024-09-11T08:52:04.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:52:04.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.423355+0000", 2024-09-11T08:52:04.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12658340900000001 2024-09-11T08:52:04.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.423388+0000", 2024-09-11T08:52:04.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3352000000000002e-05 2024-09-11T08:52:04.505 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.505 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.505 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.506 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:173089 9.c 9:30ba2d1b:::benchmark_data_smithi092_145083_object29691:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-11T08:52:04.506 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:51:08.296803+0000", 2024-09-11T08:52:04.506 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.095803101000001, 2024-09-11T08:52:04.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12666161000000001, 2024-09-11T08:52:04.506 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.506 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:04.506 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:52:04.506 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:52:04.506 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:04.506 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 173089 2024-09-11T08:52:04.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.296803+0000", 2024-09-11T08:52:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.296803+0000", 2024-09-11T08:52:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.296802+0000", 2024-09-11T08:52:04.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:52:04.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.296807+0000", 2024-09-11T08:52:04.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2499999999999997e-06 2024-09-11T08:52:04.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.296818+0000", 2024-09-11T08:52:04.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1032e-05 2024-09-11T08:52:04.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.296832+0000", 2024-09-11T08:52:04.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3709999999999999e-05 2024-09-11T08:52:04.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.297587+0000", 2024-09-11T08:52:04.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00075522100000000004 2024-09-11T08:52:04.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:52:04.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.297689+0000", 2024-09-11T08:52:04.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010181199999999999 2024-09-11T08:52:04.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,7", 2024-09-11T08:52:04.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.297790+0000", 2024-09-11T08:52:04.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010142200000000001 2024-09-11T08:52:04.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.419808+0000", 2024-09-11T08:52:04.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.122018127 2024-09-11T08:52:04.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:04.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.420911+0000", 2024-09-11T08:52:04.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011027330000000001 2024-09-11T08:52:04.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:52:04.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.423396+0000", 2024-09-11T08:52:04.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024850689999999999 2024-09-11T08:52:04.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:52:04.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.423434+0000", 2024-09-11T08:52:04.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8141e-05 2024-09-11T08:52:04.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.423464+0000", 2024-09-11T08:52:04.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0063e-05 2024-09-11T08:52:04.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.512 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.512 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15366.0:173087 9.f e251/206 9:fa023c43:::benchmark_data_smithi092_145083_object29689:head v 251'10879, mlcod=251'10877)", 2024-09-11T08:52:04.512 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:51:08.298184+0000", 2024-09-11T08:52:04.512 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.094421298, 2024-09-11T08:52:04.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12555068799999999, 2024-09-11T08:52:04.512 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:52:04.512 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:04.513 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:52:04.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:52:04.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.298184+0000", 2024-09-11T08:52:04.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:52:04.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.298184+0000", 2024-09-11T08:52:04.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:52:04.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:52:04.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.298184+0000", 2024-09-11T08:52:04.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:04.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:52:04.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.298189+0000", 2024-09-11T08:52:04.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8609999999999997e-06 2024-09-11T08:52:04.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:52:04.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.416580+0000", 2024-09-11T08:52:04.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11839098300000001 2024-09-11T08:52:04.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:52:04.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.416594+0000", 2024-09-11T08:52:04.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.381e-05 2024-09-11T08:52:04.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:52:04.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.416938+0000", 2024-09-11T08:52:04.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034480300000000003 2024-09-11T08:52:04.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:52:04.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.416958+0000", 2024-09-11T08:52:04.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9905999999999999e-05 2024-09-11T08:52:04.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:52:04.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.423706+0000", 2024-09-11T08:52:04.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067472510000000001 2024-09-11T08:52:04.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:52:04.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:52:04.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:52:04.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:08.423735+0000", 2024-09-11T08:52:04.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9527e-05 2024-09-11T08:52:04.516 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.517 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:52:04.517 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:52:04.517 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:52:04.520 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:04.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:04.521 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:04.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:04 smithi092 ceph-mon[29647]: pgmap v587: 168 pgs: 1 active+remapped+backfill_toofull, 167 active+clean; 2.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.90k op/s; 2020/127506 objects misplaced (1.584%); 8.5 MiB/s, 135 objects/s recovering 2024-09-11T08:52:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:04 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "3.5", "id": [1, 6, 0, 3, 5, 2]}]': finished 2024-09-11T08:52:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:04 smithi092 ceph-mon[29647]: osdmap e270: 8 total, 8 up, 7 in 2024-09-11T08:52:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:04 smithi092 ceph-mon[29647]: osdmap e271: 8 total, 8 up, 7 in 2024-09-11T08:52:04.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:04 smithi092 ceph-mon[41114]: pgmap v587: 168 pgs: 1 active+remapped+backfill_toofull, 167 active+clean; 2.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.90k op/s; 2020/127506 objects misplaced (1.584%); 8.5 MiB/s, 135 objects/s recovering 2024-09-11T08:52:04.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:04 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "3.5", "id": [1, 6, 0, 3, 5, 2]}]': finished 2024-09-11T08:52:04.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:04 smithi092 ceph-mon[41114]: osdmap e270: 8 total, 8 up, 7 in 2024-09-11T08:52:04.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:04 smithi092 ceph-mon[41114]: osdmap e271: 8 total, 8 up, 7 in 2024-09-11T08:52:04.622 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:04.659 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:04.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:04.660 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:04.760 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:04.805 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:04.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:04.806 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:04.907 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:04 smithi145 ceph-mon[33384]: pgmap v587: 168 pgs: 1 active+remapped+backfill_toofull, 167 active+clean; 2.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.90k op/s; 2020/127506 objects misplaced (1.584%); 8.5 MiB/s, 135 objects/s recovering 2024-09-11T08:52:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:04 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "3.5", "id": [1, 6, 0, 3, 5, 2]}]': finished 2024-09-11T08:52:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:04 smithi145 ceph-mon[33384]: osdmap e270: 8 total, 8 up, 7 in 2024-09-11T08:52:04.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:04 smithi145 ceph-mon[33384]: osdmap e271: 8 total, 8 up, 7 in 2024-09-11T08:52:04.945 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:04.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:04.946 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:05.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:05.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:05.084 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:05.084 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:05.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:05.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:05.221 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:05.221 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:05.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:05.352 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:05.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:05.353 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:05.454 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:05.490 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:05.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:05.491 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:05.592 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:05.627 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:05.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:05.628 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:05.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:05.759 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:05.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:05.760 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:05.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:05 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:52:05.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:05 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:52:05.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:05 smithi092 ceph-mon[29647]: osdmap e272: 8 total, 8 up, 7 in 2024-09-11T08:52:05.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:05 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:52:05.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:05 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:52:05.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:05 smithi092 ceph-mon[41114]: osdmap e272: 8 total, 8 up, 7 in 2024-09-11T08:52:05.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:05.889 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:05.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:05.891 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:05.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:05 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:52:05.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:05 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:52:05.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:05 smithi145 ceph-mon[33384]: osdmap e272: 8 total, 8 up, 7 in 2024-09-11T08:52:05.991 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:06.027 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:06.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:06.028 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:06.126 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:06.129 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:06.154 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:06.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:06.155 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:06.255 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:06.294 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:06.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:06.295 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:06.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:06.428 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:06.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:06.429 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:06.530 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:06.564 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:06.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:06.565 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:06.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:06 smithi092 ceph-mon[29647]: pgmap v591: 168 pgs: 1 active+remapped+backfill_toofull, 167 active+clean; 2.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.24k op/s; 2020/127506 objects misplaced (1.584%); 1.5 MiB/s, 23 objects/s recovering 2024-09-11T08:52:06.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:06 smithi092 ceph-mon[41114]: pgmap v591: 168 pgs: 1 active+remapped+backfill_toofull, 167 active+clean; 2.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.24k op/s; 2020/127506 objects misplaced (1.584%); 1.5 MiB/s, 23 objects/s recovering 2024-09-11T08:52:06.610 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:06.645 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-11T08:52:06.666 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:06.745 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:06.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:06.746 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:06.819 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:06 smithi145 ceph-mon[33384]: pgmap v591: 168 pgs: 1 active+remapped+backfill_toofull, 167 active+clean; 2.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.24k op/s; 2020/127506 objects misplaced (1.584%); 1.5 MiB/s, 23 objects/s recovering 2024-09-11T08:52:06.848 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:06.892 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:06.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:06.893 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:06.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:07.023 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:07.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:07.024 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:07.086 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:52:07.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:07.162 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:07.162 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:52:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:52:07.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:07.163 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:07.264 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:07.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:07.300 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:07.300 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:07.401 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:07.428 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:07.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:07.429 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:07.530 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:07.566 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:07.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:07.567 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:07.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:07.699 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:07.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:07.700 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:07.801 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:07.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:07 smithi092 ceph-mon[29647]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-11T08:52:07.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:07 smithi092 ceph-mon[41114]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-11T08:52:07.839 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:07.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:07.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:07.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:07 smithi145 ceph-mon[33384]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-11T08:52:07.940 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:07.981 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:07.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:07.982 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:08.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:08.114 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:08.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:08.115 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:08.216 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:08.245 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:08.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:08.246 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:08.347 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:08.369 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:08.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:08.370 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:08.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:08.502 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:08.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:08.503 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:08.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:08 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:08.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:08 smithi092 ceph-mon[41114]: pgmap v592: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 2.1 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2.29k op/s; 906/105054 objects misplaced (0.862%); 4.2 MiB/s, 67 objects/s recovering 2024-09-11T08:52:08.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:08 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:52:08.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:08 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:52:08.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:08 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:08.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:08 smithi092 ceph-mon[29647]: pgmap v592: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 2.1 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2.29k op/s; 906/105054 objects misplaced (0.862%); 4.2 MiB/s, 67 objects/s recovering 2024-09-11T08:52:08.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:08 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:52:08.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:08 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:52:08.605 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:08.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:08.640 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:08.640 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:08.741 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:08.779 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:08.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:08.780 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:08.881 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:08 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:08 smithi145 ceph-mon[33384]: pgmap v592: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 2.1 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2.29k op/s; 906/105054 objects misplaced (0.862%); 4.2 MiB/s, 67 objects/s recovering 2024-09-11T08:52:08.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:08 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:52:08.914 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:08 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:52:08.916 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:08.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:08.917 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:09.018 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:09.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:09.057 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:09.057 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:09.158 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:09.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:09.194 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:09.195 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:09.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:09.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:09.325 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:09.326 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:09.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:09.460 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:09.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:09.461 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:09.562 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:09.599 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:09.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:09.600 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:09.701 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:09.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:09.741 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:09.741 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:09.779 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:09 smithi092 ceph-mon[29647]: osdmap e273: 8 total, 8 up, 7 in 2024-09-11T08:52:09.779 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:09 smithi092 ceph-mon[41114]: osdmap e273: 8 total, 8 up, 7 in 2024-09-11T08:52:09.784 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:09 smithi145 ceph-mon[33384]: osdmap e273: 8 total, 8 up, 7 in 2024-09-11T08:52:09.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:09.875 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:09.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:09.876 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:09.977 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:10.016 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:10.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:10.017 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:10.118 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:10.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:10.162 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:10.163 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:10.191 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:52:10.192 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:52:10.192 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-11T08:52:10.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd in 0 2024-09-11T08:52:10.264 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:10.295 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:10.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:10.297 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:10.310 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:52:10.310 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:52:10.310 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.310 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15366.0:257345 9.14 e273/235 9:289fa75f:::benchmark_data_smithi092_145083_object113947:head v 273'11922, mlcod=273'11919)", 2024-09-11T08:52:10.311 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:10.258622+0000", 2024-09-11T08:52:10.311 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.0018550260000000001, 2024-09-11T08:52:10.311 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001883373, 2024-09-11T08:52:10.311 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:10.311 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T08:52:10.311 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:10.311 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.311 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:10.311 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258622+0000", 2024-09-11T08:52:10.311 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:10.312 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:10.312 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.312 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:10.312 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258622+0000", 2024-09-11T08:52:10.312 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:10.312 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:10.312 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.312 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:10.312 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258621+0000", 2024-09-11T08:52:10.312 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:52:10.312 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:10.312 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.312 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:10.313 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258631+0000", 2024-09-11T08:52:10.313 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0168e-05 2024-09-11T08:52:10.313 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:10.313 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.313 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:10.313 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258646+0000", 2024-09-11T08:52:10.313 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4630000000000001e-05 2024-09-11T08:52:10.313 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:10.313 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.313 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:10.313 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258651+0000", 2024-09-11T08:52:10.313 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.2100000000000001e-06 2024-09-11T08:52:10.313 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:10.314 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.314 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:10.314 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258678+0000", 2024-09-11T08:52:10.314 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.6701999999999999e-05 2024-09-11T08:52:10.314 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:10.314 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.314 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:10.314 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258691+0000", 2024-09-11T08:52:10.314 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3570000000000001e-05 2024-09-11T08:52:10.314 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:10.315 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:10.315 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:10.315 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:10.315 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.315 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15366.0:257347 9.10 e273/238 9:0d7e57a7:::benchmark_data_smithi092_145083_object113949:head v 273'11712, mlcod=273'11710)", 2024-09-11T08:52:10.315 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:10.258662+0000", 2024-09-11T08:52:10.315 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.0018151109999999999, 2024-09-11T08:52:10.315 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001913728, 2024-09-11T08:52:10.315 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:10.315 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T08:52:10.315 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:10.315 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.315 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:10.316 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258662+0000", 2024-09-11T08:52:10.316 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:10.316 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:10.316 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.317 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:10.317 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258662+0000", 2024-09-11T08:52:10.317 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:10.317 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:10.317 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.317 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:10.317 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258662+0000", 2024-09-11T08:52:10.317 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967296 2024-09-11T08:52:10.317 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:10.317 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.317 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:10.317 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258665+0000", 2024-09-11T08:52:10.317 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7230000000000002e-06 2024-09-11T08:52:10.318 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:10.318 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.318 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:10.318 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258671+0000", 2024-09-11T08:52:10.318 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0410000000000003e-06 2024-09-11T08:52:10.318 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:10.318 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.319 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:10.319 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258675+0000", 2024-09-11T08:52:10.319 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7579999999999998e-06 2024-09-11T08:52:10.319 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:10.319 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.319 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:10.320 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258728+0000", 2024-09-11T08:52:10.320 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.2541999999999999e-05 2024-09-11T08:52:10.320 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:10.320 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:10.320 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:10.320 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:10.258751+0000", 2024-09-11T08:52:10.320 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3258999999999999e-05 2024-09-11T08:52:10.320 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:10.320 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:10.320 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:10.320 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:10.320 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2024-09-11T08:52:10.320 INFO:teuthology.orchestra.run.smithi145.stdout: "num_ops": 2 2024-09-11T08:52:10.320 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:52:10.339 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:52:10.339 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:52:10.339 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:52:10.339 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:52:10.339 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:52:10.340 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:52:10.340 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:52:10.340 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:52:10.340 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:52:10.340 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:52:10.355 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:52:10.355 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:52:10.394 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:52:10.394 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:52:10.398 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:10.425 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:52:10.425 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:52:10.430 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:10.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:10.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:10.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:10.575 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:10.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:10.576 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:10.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:10.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:10.699 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:10.699 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:10.800 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:10 smithi092 ceph-mon[29647]: pgmap v594: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 2.1 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 702 B/s rd, 0 B/s wr, 1.28k op/s; 906/105054 objects misplaced (0.862%); 4.3 MiB/s, 69 objects/s recovering 2024-09-11T08:52:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:10 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2456965285' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:52:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:10 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1074053945' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:52:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:10 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:52:10.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:10 smithi092 ceph-mon[41114]: pgmap v594: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 2.1 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 702 B/s rd, 0 B/s wr, 1.28k op/s; 906/105054 objects misplaced (0.862%); 4.3 MiB/s, 69 objects/s recovering 2024-09-11T08:52:10.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:10 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2456965285' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:52:10.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:10 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1074053945' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:52:10.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:10 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:52:10.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:10.822 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:10.822 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:10.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:10 smithi145 ceph-mon[33384]: pgmap v594: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 2.1 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 702 B/s rd, 0 B/s wr, 1.28k op/s; 906/105054 objects misplaced (0.862%); 4.3 MiB/s, 69 objects/s recovering 2024-09-11T08:52:10.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:10 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2456965285' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:52:10.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:10 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1074053945' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:52:10.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:10 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:52:10.923 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:10.936 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:52:10.946 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:10.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:10.947 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:11.048 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:11.083 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:11.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:11.084 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:11.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:11.209 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:11.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:11.210 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:11.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:11.338 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:11.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:11.339 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:11.422 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-11T08:52:11.440 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:11.486 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:11.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:11.487 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:11.588 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:11.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:11.611 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:11.611 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:11.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:11 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:52:11.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:11 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:52:11.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:11 smithi145 ceph-mon[33384]: osdmap e274: 8 total, 8 up, 7 in 2024-09-11T08:52:11.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:11 smithi145 ceph-mon[33384]: 4.1b scrub starts 2024-09-11T08:52:11.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:11 smithi145 ceph-mon[33384]: 4.1b scrub ok 2024-09-11T08:52:11.668 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:52:11.712 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:11.733 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:11.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:11.734 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:11.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:52:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:52:11] "GET /metrics HTTP/1.1" 200 224567 "" "Prometheus/2.43.0" 2024-09-11T08:52:11.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:11 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:52:11.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:11 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:52:11.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:11 smithi092 ceph-mon[29647]: osdmap e274: 8 total, 8 up, 7 in 2024-09-11T08:52:11.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:11 smithi092 ceph-mon[29647]: 4.1b scrub starts 2024-09-11T08:52:11.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:11 smithi092 ceph-mon[29647]: 4.1b scrub ok 2024-09-11T08:52:11.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:11 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:52:11.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:11 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:52:11.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:11 smithi092 ceph-mon[41114]: osdmap e274: 8 total, 8 up, 7 in 2024-09-11T08:52:11.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:11 smithi092 ceph-mon[41114]: 4.1b scrub starts 2024-09-11T08:52:11.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:11 smithi092 ceph-mon[41114]: 4.1b scrub ok 2024-09-11T08:52:11.835 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:11.856 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:11.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:11.857 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:11.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:11.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:11.982 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:11.982 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:12.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:12.111 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:12.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:12.112 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:12.213 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:12.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:12.235 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:12.235 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:12.336 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:12.373 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:12.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:12.374 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:12.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:12.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:12.500 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:12.601 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:12.639 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:12.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:12.640 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:12.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:12.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:12.772 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:12.772 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:12 smithi092 ceph-mon[29647]: pgmap v596: 168 pgs: 1 peering, 167 active+clean; 2.1 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 643 B/s rd, 0 B/s wr, 1.37k op/s; 7.7 MiB/s, 123 objects/s recovering 2024-09-11T08:52:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:12 smithi092 ceph-mon[29647]: 4.10 scrub starts 2024-09-11T08:52:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:12 smithi092 ceph-mon[29647]: 4.10 scrub ok 2024-09-11T08:52:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:12 smithi092 ceph-mon[29647]: osdmap e275: 8 total, 8 up, 7 in 2024-09-11T08:52:12.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:12 smithi092 ceph-mon[41114]: pgmap v596: 168 pgs: 1 peering, 167 active+clean; 2.1 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 643 B/s rd, 0 B/s wr, 1.37k op/s; 7.7 MiB/s, 123 objects/s recovering 2024-09-11T08:52:12.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:12 smithi092 ceph-mon[41114]: 4.10 scrub starts 2024-09-11T08:52:12.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:12 smithi092 ceph-mon[41114]: 4.10 scrub ok 2024-09-11T08:52:12.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:12 smithi092 ceph-mon[41114]: osdmap e275: 8 total, 8 up, 7 in 2024-09-11T08:52:12.874 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:12.893 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:52:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:52:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:52:12.893 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:12 smithi145 ceph-mon[33384]: pgmap v596: 168 pgs: 1 peering, 167 active+clean; 2.1 GiB data, 11 GiB used, 614 GiB / 626 GiB avail; 643 B/s rd, 0 B/s wr, 1.37k op/s; 7.7 MiB/s, 123 objects/s recovering 2024-09-11T08:52:12.893 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:12 smithi145 ceph-mon[33384]: 4.10 scrub starts 2024-09-11T08:52:12.893 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:12 smithi145 ceph-mon[33384]: 4.10 scrub ok 2024-09-11T08:52:12.894 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:12 smithi145 ceph-mon[33384]: osdmap e275: 8 total, 8 up, 7 in 2024-09-11T08:52:12.902 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:12.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:12.903 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:13.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:13.031 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:13.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:13.032 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:13.133 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:13.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:13.175 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:13.176 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:13.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:13.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:13.308 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:13.309 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:13.409 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:13.452 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:13.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:13.454 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:13.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:13.572 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:52:13.572 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:52:13.581 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:13.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:13.582 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:13.683 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:13.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:13.717 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:13.717 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:13.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:13 smithi092 ceph-mon[29647]: 5.1f scrub starts 2024-09-11T08:52:13.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:13 smithi092 ceph-mon[29647]: 5.1f scrub ok 2024-09-11T08:52:13.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:13 smithi092 ceph-mon[41114]: 5.1f scrub starts 2024-09-11T08:52:13.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:13 smithi092 ceph-mon[41114]: 5.1f scrub ok 2024-09-11T08:52:13.818 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:13.859 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:13.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:13.860 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:13.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:13 smithi145 ceph-mon[33384]: 5.1f scrub starts 2024-09-11T08:52:13.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:13 smithi145 ceph-mon[33384]: 5.1f scrub ok 2024-09-11T08:52:13.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:13.989 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:13.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:13.990 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:14.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:14.121 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:14.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:14.123 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:14.223 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:14.255 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:14.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:14.256 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:14.286 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-11T08:52:14.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:14.386 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:14.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:14.387 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:14.487 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:14.511 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:14.528 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:14.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:14.529 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:14.630 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:14.684 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:14.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:14.685 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:14.731 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:52:14.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:14.793 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:14 smithi092 ceph-mon[29647]: pgmap v598: 168 pgs: 1 peering, 167 active+clean; 1.7 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.05k op/s; 4.0 MiB/s, 63 objects/s recovering 2024-09-11T08:52:14.793 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:14 smithi092 ceph-mon[29647]: 3.7 scrub starts 2024-09-11T08:52:14.793 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:14 smithi092 ceph-mon[29647]: 3.7 scrub ok 2024-09-11T08:52:14.793 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:14 smithi092 ceph-mon[41114]: pgmap v598: 168 pgs: 1 peering, 167 active+clean; 1.7 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.05k op/s; 4.0 MiB/s, 63 objects/s recovering 2024-09-11T08:52:14.793 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:14 smithi092 ceph-mon[41114]: 3.7 scrub starts 2024-09-11T08:52:14.793 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:14 smithi092 ceph-mon[41114]: 3.7 scrub ok 2024-09-11T08:52:14.818 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:14.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:14.819 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:14.873 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:14 smithi145 ceph-mon[33384]: pgmap v598: 168 pgs: 1 peering, 167 active+clean; 1.7 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.05k op/s; 4.0 MiB/s, 63 objects/s recovering 2024-09-11T08:52:14.873 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:14 smithi145 ceph-mon[33384]: 3.7 scrub starts 2024-09-11T08:52:14.873 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:14 smithi145 ceph-mon[33384]: 3.7 scrub ok 2024-09-11T08:52:14.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:14.948 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:14.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:14.950 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:15.050 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:15.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:15.087 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:15.087 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:15.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:15.217 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:15.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:15.218 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:15.319 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:15.355 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:15.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:15.356 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:15.457 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:15.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:15.490 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:15.490 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:15.590 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:15.630 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:15.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:15.631 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:15.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:15.761 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:15.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:15.763 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:15.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:15 smithi092 ceph-mon[29647]: 4.0 scrub starts 2024-09-11T08:52:15.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:15 smithi092 ceph-mon[29647]: 4.0 scrub ok 2024-09-11T08:52:15.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:15 smithi092 ceph-mon[29647]: 6.9 scrub starts 2024-09-11T08:52:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:15 smithi092 ceph-mon[29647]: 6.9 scrub ok 2024-09-11T08:52:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:15 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:52:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:52:15.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:15 smithi092 ceph-mon[41114]: 4.0 scrub starts 2024-09-11T08:52:15.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:15 smithi092 ceph-mon[41114]: 4.0 scrub ok 2024-09-11T08:52:15.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:15 smithi092 ceph-mon[41114]: 6.9 scrub starts 2024-09-11T08:52:15.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:15 smithi092 ceph-mon[41114]: 6.9 scrub ok 2024-09-11T08:52:15.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:15 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:52:15.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:52:15.864 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:15.885 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:15.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:15.886 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:15.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:15 smithi145 ceph-mon[33384]: 4.0 scrub starts 2024-09-11T08:52:15.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:15 smithi145 ceph-mon[33384]: 4.0 scrub ok 2024-09-11T08:52:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:15 smithi145 ceph-mon[33384]: 6.9 scrub starts 2024-09-11T08:52:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:15 smithi145 ceph-mon[33384]: 6.9 scrub ok 2024-09-11T08:52:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:15 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:52:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:52:15.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:52:15.987 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:16.020 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:16.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:16.021 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:16.122 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:16.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:16.159 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:16.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:16.260 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:16.293 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:16.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:16.294 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:16.395 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:16.428 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:16.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:16.430 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:16.488 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:52:16.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:16.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:16.579 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:16.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:16.592 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:16 smithi145 ceph-mon[33384]: pgmap v599: 168 pgs: 1 peering, 167 active+clean; 1.7 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 755 B/s rd, 0 B/s wr, 933 op/s; 3.5 MiB/s, 56 objects/s recovering 2024-09-11T08:52:16.592 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:16 smithi145 ceph-mon[33384]: 5.12 scrub starts 2024-09-11T08:52:16.592 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:16 smithi145 ceph-mon[33384]: 5.12 scrub ok 2024-09-11T08:52:16.593 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:16 smithi145 ceph-mon[33384]: 5.6 scrub starts 2024-09-11T08:52:16.593 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:16 smithi145 ceph-mon[33384]: 5.6 scrub ok 2024-09-11T08:52:16.597 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:16 smithi092 ceph-mon[29647]: pgmap v599: 168 pgs: 1 peering, 167 active+clean; 1.7 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 755 B/s rd, 0 B/s wr, 933 op/s; 3.5 MiB/s, 56 objects/s recovering 2024-09-11T08:52:16.597 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:16 smithi092 ceph-mon[29647]: 5.12 scrub starts 2024-09-11T08:52:16.597 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:16 smithi092 ceph-mon[29647]: 5.12 scrub ok 2024-09-11T08:52:16.597 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:16 smithi092 ceph-mon[29647]: 5.6 scrub starts 2024-09-11T08:52:16.597 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:16 smithi092 ceph-mon[29647]: 5.6 scrub ok 2024-09-11T08:52:16.598 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:16 smithi092 ceph-mon[41114]: pgmap v599: 168 pgs: 1 peering, 167 active+clean; 1.7 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 755 B/s rd, 0 B/s wr, 933 op/s; 3.5 MiB/s, 56 objects/s recovering 2024-09-11T08:52:16.599 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:16 smithi092 ceph-mon[41114]: 5.12 scrub starts 2024-09-11T08:52:16.599 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:16 smithi092 ceph-mon[41114]: 5.12 scrub ok 2024-09-11T08:52:16.599 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:16 smithi092 ceph-mon[41114]: 5.6 scrub starts 2024-09-11T08:52:16.599 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:16 smithi092 ceph-mon[41114]: 5.6 scrub ok 2024-09-11T08:52:16.672 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:16.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:16.738 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:16.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:16.739 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:16.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:16.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:16.884 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:16.911 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:52:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:52:16.985 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:17.022 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:17.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:17.023 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:17.124 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:17.162 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:16 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a[29624]: 2024-09-11T08:52:16.965+0000 7f9ae2e34640 -1 mon.a@0(leader).osd e275 definitely_dead 0 2024-09-11T08:52:17.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:17.163 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:17.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:17.265 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:17.297 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:17.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:17.298 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:17.399 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:17.449 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:17.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:17.450 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:17.527 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:17.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:17.564 INFO:teuthology.orchestra.run.smithi092.stderr:marked in osd.0. 2024-09-11T08:52:17.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:17.605 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:17.606 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:17 smithi145 ceph-mon[33384]: 3.12 scrub starts 2024-09-11T08:52:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:17 smithi145 ceph-mon[33384]: 3.12 scrub ok 2024-09-11T08:52:17.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:17 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3742771207' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-09-11T08:52:17.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:17 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-09-11T08:52:17.706 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:17.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:17.728 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:17.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:17.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:17 smithi092 ceph-mon[29647]: 3.12 scrub starts 2024-09-11T08:52:17.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:17 smithi092 ceph-mon[29647]: 3.12 scrub ok 2024-09-11T08:52:17.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:17 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3742771207' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-09-11T08:52:17.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:17 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-09-11T08:52:17.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:17 smithi092 ceph-mon[41114]: 3.12 scrub starts 2024-09-11T08:52:17.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:17 smithi092 ceph-mon[41114]: 3.12 scrub ok 2024-09-11T08:52:17.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:17 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3742771207' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-09-11T08:52:17.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:17 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-09-11T08:52:17.829 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:17.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:17.853 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:17.944 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:52:17.945 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:52:17.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:17.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:17.977 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:18.078 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:18.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:18.101 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:18.101 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:18.202 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:18.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:18.224 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:18.224 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:18.325 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:18.346 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:18.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:18.347 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:18.448 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:18.468 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:18.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:18.469 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:18.570 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:18.600 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:18.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:18.601 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:18.609 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-11T08:52:18.702 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:18.723 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:18.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:18.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:18.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:18 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:18.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:18 smithi092 ceph-mon[29647]: pgmap v600: 168 pgs: 168 active+clean; 1.3 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.66k op/s; 3.0 MiB/s, 47 objects/s recovering 2024-09-11T08:52:18.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:18 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["0"]}]': finished 2024-09-11T08:52:18.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:18 smithi092 ceph-mon[29647]: osdmap e276: 8 total, 8 up, 8 in 2024-09-11T08:52:18.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:18 smithi092 ceph-mon[29647]: 4.c scrub starts 2024-09-11T08:52:18.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:18 smithi092 ceph-mon[29647]: 4.c scrub ok 2024-09-11T08:52:18.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:18 smithi092 ceph-mon[29647]: osdmap e277: 8 total, 8 up, 8 in 2024-09-11T08:52:18.808 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:18 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:18.808 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:18 smithi092 ceph-mon[41114]: pgmap v600: 168 pgs: 168 active+clean; 1.3 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.66k op/s; 3.0 MiB/s, 47 objects/s recovering 2024-09-11T08:52:18.808 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:18 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["0"]}]': finished 2024-09-11T08:52:18.808 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:18 smithi092 ceph-mon[41114]: osdmap e276: 8 total, 8 up, 8 in 2024-09-11T08:52:18.808 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:18 smithi092 ceph-mon[41114]: 4.c scrub starts 2024-09-11T08:52:18.808 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:18 smithi092 ceph-mon[41114]: 4.c scrub ok 2024-09-11T08:52:18.808 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:18 smithi092 ceph-mon[41114]: osdmap e277: 8 total, 8 up, 8 in 2024-09-11T08:52:18.825 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:18.846 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:18.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:18.847 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:18.893 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:52:18.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:18 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:18.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:18 smithi145 ceph-mon[33384]: pgmap v600: 168 pgs: 168 active+clean; 1.3 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.66k op/s; 3.0 MiB/s, 47 objects/s recovering 2024-09-11T08:52:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:18 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["0"]}]': finished 2024-09-11T08:52:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:18 smithi145 ceph-mon[33384]: osdmap e276: 8 total, 8 up, 8 in 2024-09-11T08:52:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:18 smithi145 ceph-mon[33384]: 4.c scrub starts 2024-09-11T08:52:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:18 smithi145 ceph-mon[33384]: 4.c scrub ok 2024-09-11T08:52:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:18 smithi145 ceph-mon[33384]: osdmap e277: 8 total, 8 up, 8 in 2024-09-11T08:52:18.948 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:18.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:18.971 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:19.072 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:19.093 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:19.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:19.094 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:19.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:19.219 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:19.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:19.220 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:19.321 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:19.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:19.344 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:19.345 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:19.445 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:19.467 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:19.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:19.468 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:19.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:19.601 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:19.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:19.602 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:19.703 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:19.734 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:19.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:19.736 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:19.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:19.870 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:19.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:19.871 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:19 smithi145 ceph-mon[33384]: 9.f scrub starts 2024-09-11T08:52:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:19 smithi145 ceph-mon[33384]: 9.f scrub ok 2024-09-11T08:52:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:19 smithi145 ceph-mon[33384]: 4.5 scrub starts 2024-09-11T08:52:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:19 smithi145 ceph-mon[33384]: 4.5 scrub ok 2024-09-11T08:52:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:19 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:52:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:19 smithi145 ceph-mon[33384]: 5.8 scrub ok 2024-09-11T08:52:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:19 smithi145 ceph-mon[33384]: osdmap e278: 8 total, 8 up, 8 in 2024-09-11T08:52:19.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:20.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:20.012 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:20.012 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:20.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:19 smithi092 ceph-mon[29647]: 9.f scrub starts 2024-09-11T08:52:20.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:19 smithi092 ceph-mon[29647]: 9.f scrub ok 2024-09-11T08:52:20.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:19 smithi092 ceph-mon[29647]: 4.5 scrub starts 2024-09-11T08:52:20.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:19 smithi092 ceph-mon[29647]: 4.5 scrub ok 2024-09-11T08:52:20.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:19 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:52:20.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:19 smithi092 ceph-mon[29647]: 5.8 scrub ok 2024-09-11T08:52:20.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:19 smithi092 ceph-mon[29647]: osdmap e278: 8 total, 8 up, 8 in 2024-09-11T08:52:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:19 smithi092 ceph-mon[41114]: 9.f scrub starts 2024-09-11T08:52:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:19 smithi092 ceph-mon[41114]: 9.f scrub ok 2024-09-11T08:52:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:19 smithi092 ceph-mon[41114]: 4.5 scrub starts 2024-09-11T08:52:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:19 smithi092 ceph-mon[41114]: 4.5 scrub ok 2024-09-11T08:52:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:19 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:52:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:19 smithi092 ceph-mon[41114]: 5.8 scrub ok 2024-09-11T08:52:20.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:19 smithi092 ceph-mon[41114]: osdmap e278: 8 total, 8 up, 8 in 2024-09-11T08:52:20.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:20.147 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:20.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:20.148 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:20.222 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-11T08:52:20.249 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:20.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:20.283 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:20.283 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:20.385 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:20.426 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:20.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:20.428 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:20.528 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:20.562 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:20.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:20.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:20.643 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:20 smithi145 ceph-mon[33384]: pgmap v603: 168 pgs: 168 active+clean; 1.3 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 697 B/s rd, 0 B/s wr, 947 op/s 2024-09-11T08:52:20.643 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:20 smithi145 ceph-mon[33384]: 9.1 scrub starts 2024-09-11T08:52:20.644 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:20 smithi145 ceph-mon[33384]: 9.1 scrub ok 2024-09-11T08:52:20.644 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:20 smithi145 ceph-mon[33384]: 9.b starting backfill to osd.0 from (0'0,0'0] MAX to 275'16671 2024-09-11T08:52:20.644 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:20 smithi145 ceph-mon[33384]: 9.10 starting backfill to osd.0 from (0'0,0'0] MAX to 275'11995 2024-09-11T08:52:20.644 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:20 smithi145 ceph-mon[33384]: 9.17 starting backfill to osd.0 from (0'0,0'0] MAX to 275'12074 2024-09-11T08:52:20.644 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:20 smithi145 ceph-mon[33384]: 9.5 starting backfill to osd.0 from (0'0,0'0] MAX to 275'11990 2024-09-11T08:52:20.644 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:20 smithi145 ceph-mon[33384]: 9.12 starting backfill to osd.0 from (0'0,0'0] MAX to 275'11784 2024-09-11T08:52:20.644 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:20 smithi145 ceph-mon[33384]: 9.9 starting backfill to osd.0 from (0'0,0'0] MAX to 275'12183 2024-09-11T08:52:20.644 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:20 smithi145 ceph-mon[33384]: 9.8 starting backfill to osd.0 from (0'0,0'0] MAX to 275'11992 2024-09-11T08:52:20.664 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:20.698 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:20.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:20.699 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:20.800 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:20.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:20.839 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:20.839 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:20.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:20.941 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[29647]: pgmap v603: 168 pgs: 168 active+clean; 1.3 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 697 B/s rd, 0 B/s wr, 947 op/s 2024-09-11T08:52:20.941 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[29647]: 9.1 scrub starts 2024-09-11T08:52:20.941 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[29647]: 9.1 scrub ok 2024-09-11T08:52:20.941 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[29647]: 9.b starting backfill to osd.0 from (0'0,0'0] MAX to 275'16671 2024-09-11T08:52:20.941 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[29647]: 9.10 starting backfill to osd.0 from (0'0,0'0] MAX to 275'11995 2024-09-11T08:52:20.941 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[29647]: 9.17 starting backfill to osd.0 from (0'0,0'0] MAX to 275'12074 2024-09-11T08:52:20.941 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[29647]: 9.5 starting backfill to osd.0 from (0'0,0'0] MAX to 275'11990 2024-09-11T08:52:20.941 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[29647]: 9.12 starting backfill to osd.0 from (0'0,0'0] MAX to 275'11784 2024-09-11T08:52:20.942 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[29647]: 9.9 starting backfill to osd.0 from (0'0,0'0] MAX to 275'12183 2024-09-11T08:52:20.942 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[29647]: 9.8 starting backfill to osd.0 from (0'0,0'0] MAX to 275'11992 2024-09-11T08:52:20.942 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[41114]: pgmap v603: 168 pgs: 168 active+clean; 1.3 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 697 B/s rd, 0 B/s wr, 947 op/s 2024-09-11T08:52:20.942 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[41114]: 9.1 scrub starts 2024-09-11T08:52:20.942 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[41114]: 9.1 scrub ok 2024-09-11T08:52:20.943 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[41114]: 9.b starting backfill to osd.0 from (0'0,0'0] MAX to 275'16671 2024-09-11T08:52:20.943 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[41114]: 9.10 starting backfill to osd.0 from (0'0,0'0] MAX to 275'11995 2024-09-11T08:52:20.943 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[41114]: 9.17 starting backfill to osd.0 from (0'0,0'0] MAX to 275'12074 2024-09-11T08:52:20.943 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[41114]: 9.5 starting backfill to osd.0 from (0'0,0'0] MAX to 275'11990 2024-09-11T08:52:20.943 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[41114]: 9.12 starting backfill to osd.0 from (0'0,0'0] MAX to 275'11784 2024-09-11T08:52:20.943 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[41114]: 9.9 starting backfill to osd.0 from (0'0,0'0] MAX to 275'12183 2024-09-11T08:52:20.943 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:20 smithi092 ceph-mon[41114]: 9.8 starting backfill to osd.0 from (0'0,0'0] MAX to 275'11992 2024-09-11T08:52:20.990 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:20.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:20.991 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:21.080 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:52:21.080 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:52:21.092 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:21.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:21.132 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:21.132 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:21.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:21.267 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:21.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:21.268 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:21.369 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:21.402 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:21.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:21.403 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:21.504 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:21.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:21.545 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:21.609 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:52:21.646 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:21.672 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:52:21.672 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:52:21.681 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:21.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:21.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:21.714 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:21 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/663552329' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:52:21.714 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:21 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:52:21.714 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:52:21.715 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:52:21.715 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:21 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/663552329' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:52:21.715 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:21 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:52:21.715 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:52:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:52:21] "GET /metrics HTTP/1.1" 200 224569 "" "Prometheus/2.43.0" 2024-09-11T08:52:21.749 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:52:21.750 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:52:21.761 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:21 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/663552329' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:52:21.761 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:21 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:52:21.782 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:52:21.782 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:52:21.783 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:21.803 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-11T08:52:21.812 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:52:21.813 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:52:21.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:21.819 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:21.819 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:21.846 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:52:21.846 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:52:21.876 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:52:21.877 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:52:21.906 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:52:21.907 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:52:21.920 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:21.955 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:21.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:21.956 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:22.057 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:22.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:22.098 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:22.098 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:22.199 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:22.252 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:52:22.257 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:22.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:22.258 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:22.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:22.401 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:22.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:22.403 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:22.504 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:22.543 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:22.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:22.544 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:22.645 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:22.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:22 smithi145 ceph-mon[33384]: pgmap v605: 168 pgs: 1 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 166 active+clean; 1.3 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.22k op/s; 1088/64131 objects misplaced (1.697%); 18 B/s, 0 objects/s recovering 2024-09-11T08:52:22.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:22 smithi145 ceph-mon[33384]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:52:22.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:22 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:52:22.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:22 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:52:22.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:22 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:52:22.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:22 smithi145 ceph-mon[33384]: osdmap e279: 8 total, 8 up, 8 in 2024-09-11T08:52:22.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:22 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4280772150' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:52:22.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:22 smithi145 ceph-mon[33384]: 9.e scrub starts 2024-09-11T08:52:22.682 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:22.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:22.683 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:22.784 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:22.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[29647]: pgmap v605: 168 pgs: 1 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 166 active+clean; 1.3 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.22k op/s; 1088/64131 objects misplaced (1.697%); 18 B/s, 0 objects/s recovering 2024-09-11T08:52:22.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[29647]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:52:22.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:52:22.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:52:22.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:52:22.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[29647]: osdmap e279: 8 total, 8 up, 8 in 2024-09-11T08:52:22.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4280772150' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:52:22.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[29647]: 9.e scrub starts 2024-09-11T08:52:22.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[41114]: pgmap v605: 168 pgs: 1 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 166 active+clean; 1.3 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.22k op/s; 1088/64131 objects misplaced (1.697%); 18 B/s, 0 objects/s recovering 2024-09-11T08:52:22.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[41114]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:52:22.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:52:22.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:52:22.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:52:22.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[41114]: osdmap e279: 8 total, 8 up, 8 in 2024-09-11T08:52:22.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4280772150' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:52:22.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:22 smithi092 ceph-mon[41114]: 9.e scrub starts 2024-09-11T08:52:22.829 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:22.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:22.830 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:22.931 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:22.968 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:22.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:22.970 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:23.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:23.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:23.103 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:23.103 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:23.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:52:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:52:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:52:23.204 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:23.234 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:23.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:23.235 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:23.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:23.371 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:23.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:23.373 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:23.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:23.512 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:23.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:23.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:23.614 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:23.653 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:23.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:23.655 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:23.755 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:23.789 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:23.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:23.790 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:23.891 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:23.916 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:23.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:23.917 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:24.018 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:24.043 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:24.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:24.044 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:24.145 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:24.167 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:52:24.167 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T08:52:24.167 INFO:teuthology.orchestra.run.smithi145.stdout: "complaint_time": 30, 2024-09-11T08:52:24.167 INFO:teuthology.orchestra.run.smithi145.stdout: "num_blocked_ops": 0 2024-09-11T08:52:24.167 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:52:24.168 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:24.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:24.169 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:24.270 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:24.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:24.293 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:24.293 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:24.394 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:24.416 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:24.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:24.417 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:24.518 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:24.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:52:24.539 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:24.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:24.540 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:24.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:24.679 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:24.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:24.680 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:24.780 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:24.803 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:24.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:24.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:24.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:24 smithi092 ceph-mon[29647]: pgmap v607: 168 pgs: 3 active+remapped+backfilling, 2 active+remapped+backfill_wait, 2 active+remapped+backfill_toofull, 161 active+clean; 1.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 178 B/s rd, 0 op/s; 3599/53169 objects misplaced (6.769%); 9.5 MiB/s, 159 objects/s recovering 2024-09-11T08:52:24.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:24 smithi092 ceph-mon[29647]: osdmap e280: 8 total, 8 up, 8 in 2024-09-11T08:52:24.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:24 smithi092 ceph-mon[29647]: 9.3 scrub starts 2024-09-11T08:52:24.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:24 smithi092 ceph-mon[41114]: pgmap v607: 168 pgs: 3 active+remapped+backfilling, 2 active+remapped+backfill_wait, 2 active+remapped+backfill_toofull, 161 active+clean; 1.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 178 B/s rd, 0 op/s; 3599/53169 objects misplaced (6.769%); 9.5 MiB/s, 159 objects/s recovering 2024-09-11T08:52:24.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:24 smithi092 ceph-mon[41114]: osdmap e280: 8 total, 8 up, 8 in 2024-09-11T08:52:24.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:24 smithi092 ceph-mon[41114]: 9.3 scrub starts 2024-09-11T08:52:24.824 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:24.904 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:24.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:24 smithi145 ceph-mon[33384]: pgmap v607: 168 pgs: 3 active+remapped+backfilling, 2 active+remapped+backfill_wait, 2 active+remapped+backfill_toofull, 161 active+clean; 1.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 178 B/s rd, 0 op/s; 3599/53169 objects misplaced (6.769%); 9.5 MiB/s, 159 objects/s recovering 2024-09-11T08:52:24.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:24 smithi145 ceph-mon[33384]: osdmap e280: 8 total, 8 up, 8 in 2024-09-11T08:52:24.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:24 smithi145 ceph-mon[33384]: 9.3 scrub starts 2024-09-11T08:52:24.926 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:24.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:24.927 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:25.028 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:25.049 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:25.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:25.050 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:25.151 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:25.172 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-11T08:52:25.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:25.174 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:25.174 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:25.223 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:52:25.223 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:52:25.223 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-11T08:52:25.223 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 0.235612 2024-09-11T08:52:25.223 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 7 0.2356119650261148 2024-09-11T08:52:25.224 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:52:25.275 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:25.296 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:25.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:25.297 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:25.398 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:25.417 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:52:25.420 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:25.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:25.421 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:25.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:25.544 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:25.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:25.545 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:25.646 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:25.673 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:25.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:25.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:25.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:25.813 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:25.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:25.814 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:25 smithi145 ceph-mon[33384]: 4.1e deep-scrub starts 2024-09-11T08:52:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:25 smithi145 ceph-mon[33384]: 4.1e deep-scrub ok 2024-09-11T08:52:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:25 smithi145 ceph-mon[33384]: osdmap e281: 8 total, 8 up, 8 in 2024-09-11T08:52:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:25 smithi145 ceph-mon[33384]: 6.14 scrub starts 2024-09-11T08:52:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:25 smithi145 ceph-mon[33384]: 6.14 scrub ok 2024-09-11T08:52:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:25 smithi145 ceph-mon[33384]: 4.1 scrub starts 2024-09-11T08:52:25.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:25 smithi145 ceph-mon[33384]: 4.1 scrub ok 2024-09-11T08:52:25.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:25.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:25.947 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:26.048 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:26.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:25 smithi092 ceph-mon[29647]: 4.1e deep-scrub starts 2024-09-11T08:52:26.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:25 smithi092 ceph-mon[29647]: 4.1e deep-scrub ok 2024-09-11T08:52:26.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:25 smithi092 ceph-mon[29647]: osdmap e281: 8 total, 8 up, 8 in 2024-09-11T08:52:26.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:25 smithi092 ceph-mon[29647]: 6.14 scrub starts 2024-09-11T08:52:26.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:25 smithi092 ceph-mon[29647]: 6.14 scrub ok 2024-09-11T08:52:26.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:25 smithi092 ceph-mon[29647]: 4.1 scrub starts 2024-09-11T08:52:26.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:25 smithi092 ceph-mon[29647]: 4.1 scrub ok 2024-09-11T08:52:26.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:25 smithi092 ceph-mon[41114]: 4.1e deep-scrub starts 2024-09-11T08:52:26.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:25 smithi092 ceph-mon[41114]: 4.1e deep-scrub ok 2024-09-11T08:52:26.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:25 smithi092 ceph-mon[41114]: osdmap e281: 8 total, 8 up, 8 in 2024-09-11T08:52:26.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:25 smithi092 ceph-mon[41114]: 6.14 scrub starts 2024-09-11T08:52:26.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:25 smithi092 ceph-mon[41114]: 6.14 scrub ok 2024-09-11T08:52:26.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:25 smithi092 ceph-mon[41114]: 4.1 scrub starts 2024-09-11T08:52:26.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:25 smithi092 ceph-mon[41114]: 4.1 scrub ok 2024-09-11T08:52:26.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:26.092 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:26.092 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:26.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:26.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:26.219 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:26.219 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:26.320 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:26.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:26.364 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:26.365 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:26.465 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:26.498 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:26.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:26.499 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:26.600 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:26.625 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:26.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:26.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:26.727 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:26.753 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:26.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:26.754 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:26.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:26 smithi092 ceph-mon[29647]: pgmap v610: 168 pgs: 3 active+remapped+backfilling, 2 active+remapped+backfill_wait, 2 active+remapped+backfill_toofull, 161 active+clean; 1.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 178 B/s rd, 0 op/s; 3599/53169 objects misplaced (6.769%); 9.5 MiB/s, 159 objects/s recovering 2024-09-11T08:52:26.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:26 smithi092 ceph-mon[29647]: osdmap e282: 8 total, 8 up, 8 in 2024-09-11T08:52:26.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:26 smithi092 ceph-mon[41114]: pgmap v610: 168 pgs: 3 active+remapped+backfilling, 2 active+remapped+backfill_wait, 2 active+remapped+backfill_toofull, 161 active+clean; 1.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 178 B/s rd, 0 op/s; 3599/53169 objects misplaced (6.769%); 9.5 MiB/s, 159 objects/s recovering 2024-09-11T08:52:26.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:26 smithi092 ceph-mon[41114]: osdmap e282: 8 total, 8 up, 8 in 2024-09-11T08:52:26.854 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:26.880 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:26.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:26.882 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:26.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:26 smithi145 ceph-mon[33384]: pgmap v610: 168 pgs: 3 active+remapped+backfilling, 2 active+remapped+backfill_wait, 2 active+remapped+backfill_toofull, 161 active+clean; 1.1 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 178 B/s rd, 0 op/s; 3599/53169 objects misplaced (6.769%); 9.5 MiB/s, 159 objects/s recovering 2024-09-11T08:52:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:26 smithi145 ceph-mon[33384]: osdmap e282: 8 total, 8 up, 8 in 2024-09-11T08:52:26.912 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:52:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:52:26.983 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:27.021 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:27.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:27.023 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:27.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:27.154 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:27.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:27.155 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:27.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:27.286 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:27.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:27.287 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:27.388 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:27.430 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:27.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:27.431 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:27.532 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:27.556 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:27.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:27.557 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:27.658 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:27.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:27.693 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:27.693 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:27.750 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:27 smithi145 ceph-mon[33384]: 9.2 scrub starts 2024-09-11T08:52:27.751 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:27 smithi145 ceph-mon[33384]: 9.2 scrub ok 2024-09-11T08:52:27.751 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:27 smithi145 ceph-mon[33384]: 4.b scrub starts 2024-09-11T08:52:27.751 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:27 smithi145 ceph-mon[33384]: 4.b scrub ok 2024-09-11T08:52:27.751 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:27 smithi145 ceph-mon[33384]: 6.17 scrub starts 2024-09-11T08:52:27.751 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:27 smithi145 ceph-mon[33384]: 6.17 scrub ok 2024-09-11T08:52:27.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:27.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:27.826 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:27.826 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:27.927 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:27.962 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:27.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:27.963 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:28.064 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:28.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:27 smithi092 ceph-mon[29647]: 9.2 scrub starts 2024-09-11T08:52:28.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:27 smithi092 ceph-mon[29647]: 9.2 scrub ok 2024-09-11T08:52:28.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:27 smithi092 ceph-mon[29647]: 4.b scrub starts 2024-09-11T08:52:28.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:27 smithi092 ceph-mon[29647]: 4.b scrub ok 2024-09-11T08:52:28.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:27 smithi092 ceph-mon[29647]: 6.17 scrub starts 2024-09-11T08:52:28.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:27 smithi092 ceph-mon[29647]: 6.17 scrub ok 2024-09-11T08:52:28.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:27 smithi092 ceph-mon[41114]: 9.2 scrub starts 2024-09-11T08:52:28.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:27 smithi092 ceph-mon[41114]: 9.2 scrub ok 2024-09-11T08:52:28.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:27 smithi092 ceph-mon[41114]: 4.b scrub starts 2024-09-11T08:52:28.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:27 smithi092 ceph-mon[41114]: 4.b scrub ok 2024-09-11T08:52:28.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:27 smithi092 ceph-mon[41114]: 6.17 scrub starts 2024-09-11T08:52:28.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:27 smithi092 ceph-mon[41114]: 6.17 scrub ok 2024-09-11T08:52:28.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:28.101 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:28.101 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:28.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:28.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:28.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:28.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:28.359 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:28.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:28.360 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:28.426 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:52:28.426 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T08:52:28.426 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T08:52:28.426 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:52:28.426 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.426 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120742 9.16 9:6eef19e6:::benchmark_data_smithi092_145083_object120741:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:52:28.426 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.505791+0000", 2024-09-11T08:52:28.427 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 122.91633058799999, 2024-09-11T08:52:28.427 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0374483940000001, 2024-09-11T08:52:28.427 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.427 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:52:28.427 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.427 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.427 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.427 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120742 2024-09-11T08:52:28.427 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.427 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.427 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.427 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.427 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505791+0000", 2024-09-11T08:52:28.428 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.428 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.428 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505791+0000", 2024-09-11T08:52:28.428 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.428 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.428 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505788+0000", 2024-09-11T08:52:28.428 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999976 2024-09-11T08:52:28.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.429 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.429 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505833+0000", 2024-09-11T08:52:28.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4573000000000002e-05 2024-09-11T08:52:28.429 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.429 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.429 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.429 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505846+0000", 2024-09-11T08:52:28.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3033e-05 2024-09-11T08:52:28.429 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.429 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.429 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.429 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505851+0000", 2024-09-11T08:52:28.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.2719999999999997e-06 2024-09-11T08:52:28.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.430 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.430 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.430 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642292+0000", 2024-09-11T08:52:28.430 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.136440541 2024-09-11T08:52:28.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.430 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.430 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.430 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642298+0000", 2024-09-11T08:52:28.430 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5309999999999998e-06 2024-09-11T08:52:28.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.430 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.430 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543169+0000", 2024-09-11T08:52:28.431 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90087110199999998 2024-09-11T08:52:28.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.431 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.431 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543239+0000", 2024-09-11T08:52:28.431 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9496e-05 2024-09-11T08:52:28.431 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.431 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.431 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.431 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.432 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120746 9.6 9:633191e9:::benchmark_data_smithi092_145083_object120745:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:52:28.432 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.505860+0000", 2024-09-11T08:52:28.432 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 122.916260945, 2024-09-11T08:52:28.432 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0273401719999999, 2024-09-11T08:52:28.432 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.432 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:52:28.432 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.432 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.432 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.432 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120746 2024-09-11T08:52:28.432 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.432 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.433 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.433 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.433 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505860+0000", 2024-09-11T08:52:28.433 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.433 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.433 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.433 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505860+0000", 2024-09-11T08:52:28.433 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.433 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.434 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.434 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505860+0000", 2024-09-11T08:52:28.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:28.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.434 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.434 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.434 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505904+0000", 2024-09-11T08:52:28.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4440999999999997e-05 2024-09-11T08:52:28.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.434 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.434 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.434 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505910+0000", 2024-09-11T08:52:28.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1469999999999998e-06 2024-09-11T08:52:28.435 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.435 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.435 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505914+0000", 2024-09-11T08:52:28.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4620000000000001e-06 2024-09-11T08:52:28.435 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.435 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.435 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.641584+0000", 2024-09-11T08:52:28.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.13567022500000001 2024-09-11T08:52:28.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.436 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.436 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.436 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.641609+0000", 2024-09-11T08:52:28.436 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4556999999999999e-05 2024-09-11T08:52:28.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.436 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.436 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.436 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.533129+0000", 2024-09-11T08:52:28.436 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89152070299999997 2024-09-11T08:52:28.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.436 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.436 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.437 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.533200+0000", 2024-09-11T08:52:28.437 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1083999999999999e-05 2024-09-11T08:52:28.437 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.437 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.437 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.437 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.437 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.437 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120752 9.16 9:6c917c0c:::benchmark_data_smithi092_145083_object120751:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:52:28.437 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.505918+0000", 2024-09-11T08:52:28.437 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 122.916203416, 2024-09-11T08:52:28.437 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.037401907, 2024-09-11T08:52:28.437 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.437 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:52:28.438 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.438 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.438 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.438 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120752 2024-09-11T08:52:28.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.438 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.438 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.438 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.438 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505918+0000", 2024-09-11T08:52:28.438 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.438 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.438 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.438 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505918+0000", 2024-09-11T08:52:28.439 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.439 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.439 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505917+0000", 2024-09-11T08:52:28.439 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:28.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.439 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.440 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505987+0000", 2024-09-11T08:52:28.440 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9468000000000006e-05 2024-09-11T08:52:28.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.440 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.441 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505994+0000", 2024-09-11T08:52:28.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2119999999999997e-06 2024-09-11T08:52:28.441 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.441 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.441 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.441 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505998+0000", 2024-09-11T08:52:28.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6629999999999999e-06 2024-09-11T08:52:28.441 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.441 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.441 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.441 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642315+0000", 2024-09-11T08:52:28.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.136317302 2024-09-11T08:52:28.441 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.442 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.442 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.442 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642320+0000", 2024-09-11T08:52:28.442 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6290000000000001e-06 2024-09-11T08:52:28.442 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.442 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.442 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.442 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543263+0000", 2024-09-11T08:52:28.442 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90094333100000001 2024-09-11T08:52:28.442 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.442 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.442 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.442 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543320+0000", 2024-09-11T08:52:28.443 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6672999999999998e-05 2024-09-11T08:52:28.443 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.443 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.443 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.443 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.443 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.443 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120762 9.16 9:6d28e92c:::benchmark_data_smithi092_145083_object120761:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:52:28.443 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.506002+0000", 2024-09-11T08:52:28.443 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 122.91611921400001, 2024-09-11T08:52:28.443 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.037390528, 2024-09-11T08:52:28.443 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.443 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:52:28.444 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.444 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.444 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.444 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120762 2024-09-11T08:52:28.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.444 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.444 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506002+0000", 2024-09-11T08:52:28.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506002+0000", 2024-09-11T08:52:28.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.445 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.445 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506002+0000", 2024-09-11T08:52:28.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:28.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.445 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.445 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506056+0000", 2024-09-11T08:52:28.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4002999999999998e-05 2024-09-11T08:52:28.446 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506062+0000", 2024-09-11T08:52:28.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4069999999999998e-06 2024-09-11T08:52:28.446 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506065+0000", 2024-09-11T08:52:28.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4709999999999999e-06 2024-09-11T08:52:28.446 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642336+0000", 2024-09-11T08:52:28.447 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.136270262 2024-09-11T08:52:28.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642339+0000", 2024-09-11T08:52:28.447 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5209999999999998e-06 2024-09-11T08:52:28.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543343+0000", 2024-09-11T08:52:28.447 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90100398000000004 2024-09-11T08:52:28.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.448 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.448 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.448 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543392+0000", 2024-09-11T08:52:28.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9227999999999997e-05 2024-09-11T08:52:28.448 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.448 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.448 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.448 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.448 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.448 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120768 9.17 9:e9fb5cfc:::benchmark_data_smithi092_145083_object120767:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:52:28.448 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.506070+0000", 2024-09-11T08:52:28.448 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 122.91605155800001, 2024-09-11T08:52:28.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.039930231, 2024-09-11T08:52:28.449 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.449 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:28.449 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.449 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.449 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.449 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120768 2024-09-11T08:52:28.449 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.449 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.449 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.449 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.449 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506070+0000", 2024-09-11T08:52:28.449 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.449 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.449 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506070+0000", 2024-09-11T08:52:28.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506069+0000", 2024-09-11T08:52:28.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:28.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506110+0000", 2024-09-11T08:52:28.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0395999999999997e-05 2024-09-11T08:52:28.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506116+0000", 2024-09-11T08:52:28.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.19e-06 2024-09-11T08:52:28.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506120+0000", 2024-09-11T08:52:28.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9620000000000004e-06 2024-09-11T08:52:28.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.653509+0000", 2024-09-11T08:52:28.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.147388988 2024-09-11T08:52:28.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.653516+0000", 2024-09-11T08:52:28.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9909999999999996e-06 2024-09-11T08:52:28.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.541363+0000", 2024-09-11T08:52:28.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.88784689000000006 2024-09-11T08:52:28.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:28.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.541484+0000", 2024-09-11T08:52:28.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012157100000000001 2024-09-11T08:52:28.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,7", 2024-09-11T08:52:28.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.541711+0000", 2024-09-11T08:52:28.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000226314 2024-09-11T08:52:28.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:28.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543128+0000", 2024-09-11T08:52:28.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001417115 2024-09-11T08:52:28.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.545051+0000", 2024-09-11T08:52:28.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0019233410000000001 2024-09-11T08:52:28.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.545904+0000", 2024-09-11T08:52:28.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00085261600000000005 2024-09-11T08:52:28.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:28.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.545953+0000", 2024-09-11T08:52:28.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9363000000000002e-05 2024-09-11T08:52:28.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.455 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.455 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.546000+0000", 2024-09-11T08:52:28.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6848999999999998e-05 2024-09-11T08:52:28.455 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.455 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.455 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.455 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120769 9.16 9:6ce1939e:::benchmark_data_smithi092_145083_object120768:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:52:28.455 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.506125+0000", 2024-09-11T08:52:28.455 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 122.915996264, 2024-09-11T08:52:28.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0425137600000001, 2024-09-11T08:52:28.456 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.456 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:28.456 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.456 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.457 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.457 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120769 2024-09-11T08:52:28.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.457 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506125+0000", 2024-09-11T08:52:28.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506125+0000", 2024-09-11T08:52:28.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506124+0000", 2024-09-11T08:52:28.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:28.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506160+0000", 2024-09-11T08:52:28.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5710000000000002e-05 2024-09-11T08:52:28.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506167+0000", 2024-09-11T08:52:28.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3389999999999998e-06 2024-09-11T08:52:28.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506170+0000", 2024-09-11T08:52:28.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7139999999999999e-06 2024-09-11T08:52:28.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642352+0000", 2024-09-11T08:52:28.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.13618196299999999 2024-09-11T08:52:28.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642356+0000", 2024-09-11T08:52:28.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4209999999999999e-06 2024-09-11T08:52:28.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543476+0000", 2024-09-11T08:52:28.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90112084699999995 2024-09-11T08:52:28.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:28.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543604+0000", 2024-09-11T08:52:28.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000127396 2024-09-11T08:52:28.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:52:28.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543929+0000", 2024-09-11T08:52:28.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00032481300000000002 2024-09-11T08:52:28.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:28.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.546522+0000", 2024-09-11T08:52:28.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0025933559999999998 2024-09-11T08:52:28.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547191+0000", 2024-09-11T08:52:28.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00066896499999999999 2024-09-11T08:52:28.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.548558+0000", 2024-09-11T08:52:28.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001367385 2024-09-11T08:52:28.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:28.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.548605+0000", 2024-09-11T08:52:28.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6139e-05 2024-09-11T08:52:28.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.548639+0000", 2024-09-11T08:52:28.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4088999999999998e-05 2024-09-11T08:52:28.463 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.463 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.463 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.464 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120771 9.5 9:a002ebf5:::benchmark_data_smithi092_145083_object120770:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:52:28.464 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.506174+0000", 2024-09-11T08:52:28.464 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 122.91594693499999, 2024-09-11T08:52:28.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0255135049999999, 2024-09-11T08:52:28.464 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.464 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:28.464 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.464 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.464 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.464 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120771 2024-09-11T08:52:28.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.464 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506174+0000", 2024-09-11T08:52:28.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506174+0000", 2024-09-11T08:52:28.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506174+0000", 2024-09-11T08:52:28.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:28.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.466 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506204+0000", 2024-09-11T08:52:28.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0658999999999999e-05 2024-09-11T08:52:28.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.466 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506211+0000", 2024-09-11T08:52:28.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.127e-06 2024-09-11T08:52:28.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.467 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506214+0000", 2024-09-11T08:52:28.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6820000000000001e-06 2024-09-11T08:52:28.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.467 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.653029+0000", 2024-09-11T08:52:28.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.146814588 2024-09-11T08:52:28.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.467 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.653040+0000", 2024-09-11T08:52:28.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0954e-05 2024-09-11T08:52:28.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.513562+0000", 2024-09-11T08:52:28.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.86052253899999998 2024-09-11T08:52:28.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:28.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.513719+0000", 2024-09-11T08:52:28.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015641799999999999 2024-09-11T08:52:28.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.469 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,7", 2024-09-11T08:52:28.469 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.513988+0000", 2024-09-11T08:52:28.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00026880900000000001 2024-09-11T08:52:28.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.469 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:28.469 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.517404+0000", 2024-09-11T08:52:28.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003416681 2024-09-11T08:52:28.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.521873+0000", 2024-09-11T08:52:28.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0044682899999999998 2024-09-11T08:52:28.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.531603+0000", 2024-09-11T08:52:28.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0097300000000000008 2024-09-11T08:52:28.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:28.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.531650+0000", 2024-09-11T08:52:28.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7855999999999997e-05 2024-09-11T08:52:28.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.531688+0000", 2024-09-11T08:52:28.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7243999999999998e-05 2024-09-11T08:52:28.471 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.471 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.471 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.471 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120772 9.5 9:a5113f5f:::benchmark_data_smithi092_145083_object120771:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:52:28.471 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.506219+0000", 2024-09-11T08:52:28.471 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 122.915901789, 2024-09-11T08:52:28.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.032593828, 2024-09-11T08:52:28.472 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.472 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:28.472 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.472 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.472 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.472 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120772 2024-09-11T08:52:28.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.472 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506219+0000", 2024-09-11T08:52:28.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506219+0000", 2024-09-11T08:52:28.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506219+0000", 2024-09-11T08:52:28.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:28.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506254+0000", 2024-09-11T08:52:28.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4820000000000002e-05 2024-09-11T08:52:28.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506260+0000", 2024-09-11T08:52:28.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3090000000000001e-06 2024-09-11T08:52:28.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506264+0000", 2024-09-11T08:52:28.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5319999999999998e-06 2024-09-11T08:52:28.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.657026+0000", 2024-09-11T08:52:28.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.15076241700000001 2024-09-11T08:52:28.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.657040+0000", 2024-09-11T08:52:28.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3667e-05 2024-09-11T08:52:28.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.515201+0000", 2024-09-11T08:52:28.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.85816157100000001 2024-09-11T08:52:28.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:28.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.515331+0000", 2024-09-11T08:52:28.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000130211 2024-09-11T08:52:28.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,7", 2024-09-11T08:52:28.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.515639+0000", 2024-09-11T08:52:28.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00030726899999999999 2024-09-11T08:52:28.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:28.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.517591+0000", 2024-09-11T08:52:28.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001952147 2024-09-11T08:52:28.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.526102+0000", 2024-09-11T08:52:28.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0085114309999999999 2024-09-11T08:52:28.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538708+0000", 2024-09-11T08:52:28.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.01260591 2024-09-11T08:52:28.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:28.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538778+0000", 2024-09-11T08:52:28.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9960000000000001e-05 2024-09-11T08:52:28.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.479 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.479 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538813+0000", 2024-09-11T08:52:28.479 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4953000000000002e-05 2024-09-11T08:52:28.479 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.479 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.479 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.479 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.479 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.479 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122677 9.12 9.94d43692 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:52:28.479 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.562235+0000", 2024-09-11T08:52:28.479 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 120.859886133, 2024-09-11T08:52:28.479 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97037142899999995, 2024-09-11T08:52:28.479 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.479 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:52:28.480 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.480 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.480 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.480 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122677 2024-09-11T08:52:28.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.480 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.480 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.480 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.480 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562235+0000", 2024-09-11T08:52:28.480 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.481 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562235+0000", 2024-09-11T08:52:28.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.481 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562233+0000", 2024-09-11T08:52:28.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:52:28.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562285+0000", 2024-09-11T08:52:28.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1749000000000002e-05 2024-09-11T08:52:28.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562296+0000", 2024-09-11T08:52:28.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1205e-05 2024-09-11T08:52:28.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562301+0000", 2024-09-11T08:52:28.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6040000000000003e-06 2024-09-11T08:52:28.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562340+0000", 2024-09-11T08:52:28.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8985e-05 2024-09-11T08:52:28.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562345+0000", 2024-09-11T08:52:28.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9629999999999997e-06 2024-09-11T08:52:28.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.532595+0000", 2024-09-11T08:52:28.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97025015999999997 2024-09-11T08:52:28.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.484 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.484 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.532606+0000", 2024-09-11T08:52:28.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1637999999999999e-05 2024-09-11T08:52:28.484 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.484 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.484 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.484 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.484 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272038 9.12 9:4c1395bf:::benchmark_data_smithi092_145083_object128640:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:52:28.484 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.599637+0000", 2024-09-11T08:52:28.484 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 4.8224839150000003, 2024-09-11T08:52:28.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0332527600000001, 2024-09-11T08:52:28.485 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.485 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:28.485 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.485 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.485 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.485 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272038 2024-09-11T08:52:28.485 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.485 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.485 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.485 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.485 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599637+0000", 2024-09-11T08:52:28.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599637+0000", 2024-09-11T08:52:28.486 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599635+0000", 2024-09-11T08:52:28.486 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:52:28.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599643+0000", 2024-09-11T08:52:28.487 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.1759999999999993e-06 2024-09-11T08:52:28.487 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.487 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.487 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.487 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599660+0000", 2024-09-11T08:52:28.487 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7118000000000001e-05 2024-09-11T08:52:28.487 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.487 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.487 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.487 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599667+0000", 2024-09-11T08:52:28.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8260000000000004e-06 2024-09-11T08:52:28.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599717+0000", 2024-09-11T08:52:28.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0219000000000001e-05 2024-09-11T08:52:28.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op must wait for map", 2024-09-11T08:52:28.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599724+0000", 2024-09-11T08:52:28.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4130000000000003e-06 2024-09-11T08:52:28.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.611082+0000", 2024-09-11T08:52:28.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.01135824 2024-09-11T08:52:28.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.611093+0000", 2024-09-11T08:52:28.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0729e-05 2024-09-11T08:52:28.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.627111+0000", 2024-09-11T08:52:28.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0160185660000001 2024-09-11T08:52:28.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:28.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.627211+0000", 2024-09-11T08:52:28.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.9397999999999999e-05 2024-09-11T08:52:28.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:52:28.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.627282+0000", 2024-09-11T08:52:28.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1217000000000006e-05 2024-09-11T08:52:28.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.491 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.491 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:28.491 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.631283+0000", 2024-09-11T08:52:28.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0040011389999999999 2024-09-11T08:52:28.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.491 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.491 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.491 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.632714+0000", 2024-09-11T08:52:28.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0014306360000000001 2024-09-11T08:52:28.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.491 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.491 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.491 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.632844+0000", 2024-09-11T08:52:28.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013048000000000001 2024-09-11T08:52:28.492 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.492 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:28.492 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.632869+0000", 2024-09-11T08:52:28.492 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4278999999999999e-05 2024-09-11T08:52:28.492 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.492 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.492 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.632890+0000", 2024-09-11T08:52:28.492 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.1382e-05 2024-09-11T08:52:28.492 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.492 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.493 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.493 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.493 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272048 9.12 9:4dde8efd:::benchmark_data_smithi092_145083_object128650:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:52:28.493 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.614452+0000", 2024-09-11T08:52:28.493 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 4.8076688750000001, 2024-09-11T08:52:28.493 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.031362157, 2024-09-11T08:52:28.493 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.493 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:28.493 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.493 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.493 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.493 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272048 2024-09-11T08:52:28.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.494 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.494 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.494 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614452+0000", 2024-09-11T08:52:28.494 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.494 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.494 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614452+0000", 2024-09-11T08:52:28.494 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614451+0000", 2024-09-11T08:52:28.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:28.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614458+0000", 2024-09-11T08:52:28.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4889999999999998e-06 2024-09-11T08:52:28.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614474+0000", 2024-09-11T08:52:28.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5764e-05 2024-09-11T08:52:28.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614481+0000", 2024-09-11T08:52:28.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.7859999999999999e-06 2024-09-11T08:52:28.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614573+0000", 2024-09-11T08:52:28.497 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.2504999999999996e-05 2024-09-11T08:52:28.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.497 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.497 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.497 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614581+0000", 2024-09-11T08:52:28.497 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7680000000000008e-06 2024-09-11T08:52:28.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.497 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.498 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.498 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.629158+0000", 2024-09-11T08:52:28.498 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.014577072 2024-09-11T08:52:28.498 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.498 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.498 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:28.498 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.629222+0000", 2024-09-11T08:52:28.498 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3751000000000007e-05 2024-09-11T08:52:28.498 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.498 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.498 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:52:28.498 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.629291+0000", 2024-09-11T08:52:28.498 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9078999999999994e-05 2024-09-11T08:52:28.498 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.499 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.499 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.499 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.634043+0000", 2024-09-11T08:52:28.499 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0047520009999999996 2024-09-11T08:52:28.499 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.499 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.499 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:28.499 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.634048+0000", 2024-09-11T08:52:28.499 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0679999999999996e-06 2024-09-11T08:52:28.499 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.499 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.499 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.499 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.645758+0000", 2024-09-11T08:52:28.500 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011710124000000001 2024-09-11T08:52:28.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.500 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.500 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:28.500 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.645789+0000", 2024-09-11T08:52:28.500 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1124000000000002e-05 2024-09-11T08:52:28.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.500 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.500 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.500 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.645814+0000", 2024-09-11T08:52:28.500 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5244000000000001e-05 2024-09-11T08:52:28.501 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.501 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.501 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.501 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.501 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.501 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272066 9.12 9:4968a467:::benchmark_data_smithi092_145083_object128668:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:52:28.501 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.629493+0000", 2024-09-11T08:52:28.501 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 4.7926280200000004, 2024-09-11T08:52:28.501 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0189994769999999, 2024-09-11T08:52:28.501 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.501 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:28.501 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.501 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.502 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.502 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272066 2024-09-11T08:52:28.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.502 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.502 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.502 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.502 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629493+0000", 2024-09-11T08:52:28.502 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.502 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.502 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.502 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629493+0000", 2024-09-11T08:52:28.502 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.503 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.503 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:28.504 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.504 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629492+0000", 2024-09-11T08:52:28.504 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:28.505 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.505 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.505 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.505 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629499+0000", 2024-09-11T08:52:28.505 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4390000000000003e-06 2024-09-11T08:52:28.505 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.505 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.505 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.505 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629516+0000", 2024-09-11T08:52:28.505 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6617e-05 2024-09-11T08:52:28.505 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.505 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.505 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.506 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629522+0000", 2024-09-11T08:52:28.506 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2009999999999996e-06 2024-09-11T08:52:28.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.506 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.506 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.506 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629586+0000", 2024-09-11T08:52:28.506 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4300000000000004e-05 2024-09-11T08:52:28.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.506 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.506 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.506 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629594+0000", 2024-09-11T08:52:28.506 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.5959999999999998e-06 2024-09-11T08:52:28.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.507 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.507 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.507 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.630309+0000", 2024-09-11T08:52:28.507 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0007151240000001 2024-09-11T08:52:28.507 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.507 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.507 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:28.507 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.630368+0000", 2024-09-11T08:52:28.507 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9344e-05 2024-09-11T08:52:28.507 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.507 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.507 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:52:28.507 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.630435+0000", 2024-09-11T08:52:28.507 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6285999999999995e-05 2024-09-11T08:52:28.508 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.508 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.508 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:28.508 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.635154+0000", 2024-09-11T08:52:28.508 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0047192019999999996 2024-09-11T08:52:28.508 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.508 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.508 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.508 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.635174+0000", 2024-09-11T08:52:28.508 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0749000000000001e-05 2024-09-11T08:52:28.509 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.509 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.509 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.509 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648406+0000", 2024-09-11T08:52:28.509 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013231463000000001 2024-09-11T08:52:28.509 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.509 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.509 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:28.509 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648455+0000", 2024-09-11T08:52:28.509 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9174e-05 2024-09-11T08:52:28.509 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.509 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.509 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.510 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648493+0000", 2024-09-11T08:52:28.510 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7438999999999998e-05 2024-09-11T08:52:28.510 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.510 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.510 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.510 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.510 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.510 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272071 9.12 9:4c320382:::benchmark_data_smithi092_145083_object128673:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:52:28.510 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.632070+0000", 2024-09-11T08:52:28.510 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 4.7900507399999999, 2024-09-11T08:52:28.510 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.016575234, 2024-09-11T08:52:28.510 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.510 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:28.511 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.511 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.511 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.511 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272071 2024-09-11T08:52:28.511 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.511 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.511 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.511 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.511 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632070+0000", 2024-09-11T08:52:28.511 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.511 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.512 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.512 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.512 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632070+0000", 2024-09-11T08:52:28.512 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.512 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.512 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.512 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.512 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632070+0000", 2024-09-11T08:52:28.512 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:52:28.512 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.512 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.512 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.512 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632074+0000", 2024-09-11T08:52:28.513 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9110000000000001e-06 2024-09-11T08:52:28.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.513 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.513 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.513 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632083+0000", 2024-09-11T08:52:28.513 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.6370000000000005e-06 2024-09-11T08:52:28.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.513 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.513 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.513 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632089+0000", 2024-09-11T08:52:28.514 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6810000000000003e-06 2024-09-11T08:52:28.514 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.514 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.514 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.514 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632285+0000", 2024-09-11T08:52:28.514 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00019620999999999999 2024-09-11T08:52:28.514 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.515 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.515 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.515 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632292+0000", 2024-09-11T08:52:28.515 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5289999999999997e-06 2024-09-11T08:52:28.515 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.515 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.515 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.515 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.631306+0000", 2024-09-11T08:52:28.515 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99901447799999998 2024-09-11T08:52:28.515 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.515 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.516 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:28.516 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.631400+0000", 2024-09-11T08:52:28.516 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.4510000000000001e-05 2024-09-11T08:52:28.516 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.516 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.516 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:52:28.516 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.631493+0000", 2024-09-11T08:52:28.516 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.2555999999999999e-05 2024-09-11T08:52:28.516 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.516 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.517 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:28.517 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.635158+0000", 2024-09-11T08:52:28.517 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0036650089999999999 2024-09-11T08:52:28.517 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.517 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.517 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.517 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.643385+0000", 2024-09-11T08:52:28.517 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0082264529999999999 2024-09-11T08:52:28.517 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.517 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.517 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.517 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648566+0000", 2024-09-11T08:52:28.517 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0051818469999999998 2024-09-11T08:52:28.517 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.518 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.518 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:28.518 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648610+0000", 2024-09-11T08:52:28.518 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3442999999999997e-05 2024-09-11T08:52:28.518 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.518 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.518 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.518 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648646+0000", 2024-09-11T08:52:28.518 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5800000000000003e-05 2024-09-11T08:52:28.518 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.519 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.519 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.519 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.519 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.519 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272083 9.12 9:4e00b3e7:::benchmark_data_smithi092_145083_object128685:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:52:28.519 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.640138+0000", 2024-09-11T08:52:28.519 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 4.781982664, 2024-09-11T08:52:28.519 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.008620208, 2024-09-11T08:52:28.519 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.519 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:28.519 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.519 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.519 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.519 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272083 2024-09-11T08:52:28.520 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.520 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.520 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.520 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.520 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640138+0000", 2024-09-11T08:52:28.520 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.520 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.520 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.520 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.520 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640138+0000", 2024-09-11T08:52:28.520 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.520 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.520 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.521 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.521 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640137+0000", 2024-09-11T08:52:28.521 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:28.521 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.521 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.521 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.521 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640144+0000", 2024-09-11T08:52:28.521 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4889999999999998e-06 2024-09-11T08:52:28.521 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.521 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.521 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.521 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640156+0000", 2024-09-11T08:52:28.521 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1718e-05 2024-09-11T08:52:28.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.522 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.522 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.522 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640162+0000", 2024-09-11T08:52:28.522 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8159999999999999e-06 2024-09-11T08:52:28.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.522 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.522 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.522 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640202+0000", 2024-09-11T08:52:28.522 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0370999999999997e-05 2024-09-11T08:52:28.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.522 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.522 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.522 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640210+0000", 2024-09-11T08:52:28.523 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4039999999999998e-06 2024-09-11T08:52:28.523 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.523 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.523 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.523 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.632939+0000", 2024-09-11T08:52:28.523 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99272926299999997 2024-09-11T08:52:28.523 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.523 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.523 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:28.523 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.633019+0000", 2024-09-11T08:52:28.523 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.9963000000000007e-05 2024-09-11T08:52:28.523 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.523 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.523 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:52:28.524 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.633114+0000", 2024-09-11T08:52:28.524 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.5613999999999996e-05 2024-09-11T08:52:28.524 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.524 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.524 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:28.524 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.636389+0000", 2024-09-11T08:52:28.524 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0032742420000000001 2024-09-11T08:52:28.524 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.524 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.524 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.524 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.643483+0000", 2024-09-11T08:52:28.524 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0070939260000000004 2024-09-11T08:52:28.524 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.524 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.525 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.525 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648683+0000", 2024-09-11T08:52:28.525 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0052004089999999996 2024-09-11T08:52:28.525 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.525 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.525 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:28.525 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648726+0000", 2024-09-11T08:52:28.525 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3457000000000001e-05 2024-09-11T08:52:28.525 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.525 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.525 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.525 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648759+0000", 2024-09-11T08:52:28.526 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2200999999999999e-05 2024-09-11T08:52:28.526 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.526 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.526 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.526 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.526 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.526 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272127 9.12 9:4a2c3ebc:::benchmark_data_smithi092_145083_object128729:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:52:28.526 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.679728+0000", 2024-09-11T08:52:28.526 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 4.7423934809999997, 2024-09-11T08:52:28.526 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.96911529500000004, 2024-09-11T08:52:28.526 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.526 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:28.526 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.527 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.527 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.527 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272127 2024-09-11T08:52:28.527 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.527 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.527 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.527 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.527 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.679728+0000", 2024-09-11T08:52:28.527 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.527 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.527 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.527 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.527 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.679728+0000", 2024-09-11T08:52:28.527 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.528 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.528 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.528 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.528 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.679726+0000", 2024-09-11T08:52:28.528 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:52:28.528 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.528 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.528 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.528 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.679733+0000", 2024-09-11T08:52:28.528 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7950000000000008e-06 2024-09-11T08:52:28.528 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.528 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.529 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.529 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.679746+0000", 2024-09-11T08:52:28.529 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2954e-05 2024-09-11T08:52:28.529 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.529 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.529 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.529 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.679752+0000", 2024-09-11T08:52:28.529 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7649999999999999e-06 2024-09-11T08:52:28.529 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.529 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.530 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.530 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.680844+0000", 2024-09-11T08:52:28.530 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00109219 2024-09-11T08:52:28.530 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.530 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.530 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.530 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.680851+0000", 2024-09-11T08:52:28.530 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.7909999999999999e-06 2024-09-11T08:52:28.530 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.530 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.530 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.530 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.634057+0000", 2024-09-11T08:52:28.530 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.95320627499999999 2024-09-11T08:52:28.530 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.531 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.531 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:28.531 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.634119+0000", 2024-09-11T08:52:28.531 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1271000000000003e-05 2024-09-11T08:52:28.531 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.531 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.531 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:52:28.531 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.634189+0000", 2024-09-11T08:52:28.531 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9933999999999998e-05 2024-09-11T08:52:28.531 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.531 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.531 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:28.531 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.640400+0000", 2024-09-11T08:52:28.531 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0062117580000000004 2024-09-11T08:52:28.532 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.532 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.532 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.532 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.643525+0000", 2024-09-11T08:52:28.532 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0031246580000000002 2024-09-11T08:52:28.532 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.532 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.532 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.532 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648789+0000", 2024-09-11T08:52:28.532 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0052638069999999997 2024-09-11T08:52:28.532 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.532 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.532 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:28.533 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648812+0000", 2024-09-11T08:52:28.533 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3360999999999999e-05 2024-09-11T08:52:28.533 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.533 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.533 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.533 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648843+0000", 2024-09-11T08:52:28.533 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0741999999999998e-05 2024-09-11T08:52:28.533 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.533 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.533 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.533 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.533 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.533 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272200 9.12 9:4946c904:::benchmark_data_smithi092_145083_object128802:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:52:28.534 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.737018+0000", 2024-09-11T08:52:28.534 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 4.6851031069999998, 2024-09-11T08:52:28.534 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.91195315600000004, 2024-09-11T08:52:28.534 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.534 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:28.534 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.534 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.534 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.534 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272200 2024-09-11T08:52:28.534 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.534 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.534 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.535 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.535 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.737018+0000", 2024-09-11T08:52:28.535 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.535 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.535 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.535 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.535 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.737018+0000", 2024-09-11T08:52:28.535 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.535 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.535 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.535 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.535 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.737017+0000", 2024-09-11T08:52:28.535 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:52:28.536 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.536 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.536 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.536 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.737026+0000", 2024-09-11T08:52:28.536 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8009999999999993e-06 2024-09-11T08:52:28.536 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.536 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.536 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.536 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.737038+0000", 2024-09-11T08:52:28.536 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1704e-05 2024-09-11T08:52:28.536 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.536 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.536 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.536 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.737043+0000", 2024-09-11T08:52:28.537 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3410000000000002e-06 2024-09-11T08:52:28.537 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.537 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.537 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.537 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.737083+0000", 2024-09-11T08:52:28.537 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0114999999999998e-05 2024-09-11T08:52:28.537 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.537 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.537 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.537 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.737090+0000", 2024-09-11T08:52:28.537 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.7530000000000004e-06 2024-09-11T08:52:28.537 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.537 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.537 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.538 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.635186+0000", 2024-09-11T08:52:28.538 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89809583900000001 2024-09-11T08:52:28.538 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.538 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.538 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:28.538 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.635274+0000", 2024-09-11T08:52:28.538 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8005999999999997e-05 2024-09-11T08:52:28.538 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.538 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.538 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:52:28.538 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.635344+0000", 2024-09-11T08:52:28.538 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9968999999999994e-05 2024-09-11T08:52:28.538 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.538 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.539 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:28.539 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.643336+0000", 2024-09-11T08:52:28.539 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0079919250000000004 2024-09-11T08:52:28.539 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.539 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.539 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.539 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.644283+0000", 2024-09-11T08:52:28.539 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00094704800000000003 2024-09-11T08:52:28.539 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.539 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.539 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.540 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648883+0000", 2024-09-11T08:52:28.540 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0046006720000000001 2024-09-11T08:52:28.540 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.540 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.540 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:28.540 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648927+0000", 2024-09-11T08:52:28.540 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3281000000000001e-05 2024-09-11T08:52:28.540 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.540 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.540 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.541 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648971+0000", 2024-09-11T08:52:28.541 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4567999999999997e-05 2024-09-11T08:52:28.541 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.541 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.541 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.541 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.541 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.541 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272202 9.12 9:4a35500b:::benchmark_data_smithi092_145083_object128804:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:52:28.541 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.739853+0000", 2024-09-11T08:52:28.541 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 4.6822683789999999, 2024-09-11T08:52:28.541 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90921051799999997, 2024-09-11T08:52:28.541 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.541 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:28.542 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.542 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.542 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.542 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272202 2024-09-11T08:52:28.542 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.542 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.542 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.542 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.542 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.739853+0000", 2024-09-11T08:52:28.542 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.542 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.542 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.542 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.542 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.739853+0000", 2024-09-11T08:52:28.543 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.543 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.543 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.543 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.543 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.739852+0000", 2024-09-11T08:52:28.543 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:52:28.543 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.543 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.543 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.543 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.739857+0000", 2024-09-11T08:52:28.543 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7780000000000004e-06 2024-09-11T08:52:28.543 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.543 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.544 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.544 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.739868+0000", 2024-09-11T08:52:28.544 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0186999999999999e-05 2024-09-11T08:52:28.544 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.544 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.544 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.544 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.739873+0000", 2024-09-11T08:52:28.544 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5759999999999996e-06 2024-09-11T08:52:28.544 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.544 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.544 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.545 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.739918+0000", 2024-09-11T08:52:28.545 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5197000000000001e-05 2024-09-11T08:52:28.545 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.545 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.545 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.545 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.739926+0000", 2024-09-11T08:52:28.545 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4059999999999999e-06 2024-09-11T08:52:28.545 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.545 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.545 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.545 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.636421+0000", 2024-09-11T08:52:28.545 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89649537000000001 2024-09-11T08:52:28.545 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.545 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.546 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:28.546 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.636518+0000", 2024-09-11T08:52:28.546 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.7099e-05 2024-09-11T08:52:28.546 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.546 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.546 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:52:28.546 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.636622+0000", 2024-09-11T08:52:28.546 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010404 2024-09-11T08:52:28.546 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.546 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.546 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:28.546 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.643349+0000", 2024-09-11T08:52:28.546 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0067265809999999997 2024-09-11T08:52:28.546 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.546 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.547 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.547 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.649000+0000", 2024-09-11T08:52:28.547 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0056514629999999998 2024-09-11T08:52:28.547 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.547 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.547 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.547 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.649022+0000", 2024-09-11T08:52:28.547 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.1861e-05 2024-09-11T08:52:28.547 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.547 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.548 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:28.548 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.649045+0000", 2024-09-11T08:52:28.548 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.2948000000000001e-05 2024-09-11T08:52:28.548 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.548 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.548 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.548 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.649063+0000", 2024-09-11T08:52:28.548 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.8079000000000001e-05 2024-09-11T08:52:28.548 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.548 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.548 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.548 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.548 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.548 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272253 9.12 9:4fd1fa8d:::benchmark_data_smithi092_145083_object128855:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:52:28.549 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.784652+0000", 2024-09-11T08:52:28.549 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 4.6374690530000002, 2024-09-11T08:52:28.549 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.86449383099999999, 2024-09-11T08:52:28.549 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.549 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:28.549 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.549 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.549 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.549 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272253 2024-09-11T08:52:28.549 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.549 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.550 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.550 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.550 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.784652+0000", 2024-09-11T08:52:28.550 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.550 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.550 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.550 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.550 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.784652+0000", 2024-09-11T08:52:28.550 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.550 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.550 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.550 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.550 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.784651+0000", 2024-09-11T08:52:28.550 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:52:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.784656+0000", 2024-09-11T08:52:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0170000000000004e-06 2024-09-11T08:52:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.784666+0000", 2024-09-11T08:52:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.4380000000000001e-06 2024-09-11T08:52:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.551 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.784671+0000", 2024-09-11T08:52:28.552 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9609999999999996e-06 2024-09-11T08:52:28.552 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.552 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.552 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.552 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.787845+0000", 2024-09-11T08:52:28.552 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0031747939999999999 2024-09-11T08:52:28.552 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.552 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.552 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.552 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.787852+0000", 2024-09-11T08:52:28.552 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3990000000000001e-06 2024-09-11T08:52:28.552 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.552 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.552 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.553 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.637818+0000", 2024-09-11T08:52:28.553 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.84996587300000004 2024-09-11T08:52:28.553 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.553 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.553 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:28.553 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.637889+0000", 2024-09-11T08:52:28.553 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1074000000000004e-05 2024-09-11T08:52:28.553 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.553 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.553 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:52:28.553 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.637970+0000", 2024-09-11T08:52:28.553 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.1106000000000006e-05 2024-09-11T08:52:28.553 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.554 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.554 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:28.554 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.643359+0000", 2024-09-11T08:52:28.554 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0053888369999999996 2024-09-11T08:52:28.554 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.554 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.554 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.554 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.649088+0000", 2024-09-11T08:52:28.554 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.005729601 2024-09-11T08:52:28.554 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.554 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.554 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.554 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.649109+0000", 2024-09-11T08:52:28.554 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0979999999999999e-05 2024-09-11T08:52:28.555 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.555 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.555 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:28.555 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.649130+0000", 2024-09-11T08:52:28.555 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0330999999999999e-05 2024-09-11T08:52:28.555 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.555 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.555 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.555 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.649146+0000", 2024-09-11T08:52:28.555 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.63e-05 2024-09-11T08:52:28.555 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.555 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.555 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.555 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.555 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.556 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272298 9.12 9:4c8ab37c:::benchmark_data_smithi092_145083_object128900:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:52:28.556 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.830337+0000", 2024-09-11T08:52:28.556 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 4.5917842100000001, 2024-09-11T08:52:28.556 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.818889062, 2024-09-11T08:52:28.556 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.556 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:28.556 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.556 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.556 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.556 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272298 2024-09-11T08:52:28.556 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.556 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.556 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.557 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.557 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.830337+0000", 2024-09-11T08:52:28.557 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.557 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.557 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.557 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.557 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.830337+0000", 2024-09-11T08:52:28.557 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.557 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.557 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.557 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.557 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.830336+0000", 2024-09-11T08:52:28.557 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:52:28.557 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.558 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.558 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.558 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.830343+0000", 2024-09-11T08:52:28.558 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5919999999999997e-06 2024-09-11T08:52:28.558 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.558 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.558 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.558 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.830357+0000", 2024-09-11T08:52:28.558 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4306e-05 2024-09-11T08:52:28.558 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.558 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.558 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.558 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.830363+0000", 2024-09-11T08:52:28.559 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3879999999999997e-06 2024-09-11T08:52:28.559 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.559 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.559 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.559 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.830406+0000", 2024-09-11T08:52:28.559 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3129e-05 2024-09-11T08:52:28.559 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.559 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.559 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.559 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.830414+0000", 2024-09-11T08:52:28.559 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.323e-06 2024-09-11T08:52:28.559 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.559 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.559 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.560 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.639006+0000", 2024-09-11T08:52:28.560 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.808592062 2024-09-11T08:52:28.560 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.560 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.560 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:28.560 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.639079+0000", 2024-09-11T08:52:28.560 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2768999999999994e-05 2024-09-11T08:52:28.560 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.560 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.560 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:52:28.561 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.639172+0000", 2024-09-11T08:52:28.561 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.3567999999999996e-05 2024-09-11T08:52:28.561 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.561 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.561 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:28.561 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.643394+0000", 2024-09-11T08:52:28.561 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00422171 2024-09-11T08:52:28.561 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.561 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.561 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.561 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.649170+0000", 2024-09-11T08:52:28.561 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0057757579999999998 2024-09-11T08:52:28.562 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.562 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.562 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.562 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.649191+0000", 2024-09-11T08:52:28.562 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.1316999999999999e-05 2024-09-11T08:52:28.562 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.562 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.562 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:28.562 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.649210+0000", 2024-09-11T08:52:28.562 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.9440999999999999e-05 2024-09-11T08:52:28.562 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.562 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.563 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.563 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.649226+0000", 2024-09-11T08:52:28.563 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5656999999999999e-05 2024-09-11T08:52:28.563 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.563 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.563 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.563 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.563 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.563 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272319 9.12 9:490a8cd1:::benchmark_data_smithi092_145083_object128921:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:52:28.563 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.858016+0000", 2024-09-11T08:52:28.563 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 4.5641050600000002, 2024-09-11T08:52:28.563 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.80114785399999999, 2024-09-11T08:52:28.563 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:28.563 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:28.564 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:28.564 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:28.564 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:28.564 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272319 2024-09-11T08:52:28.565 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.565 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:28.565 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.565 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:28.565 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.858016+0000", 2024-09-11T08:52:28.565 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.565 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.565 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.565 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:28.565 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.858016+0000", 2024-09-11T08:52:28.565 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:28.565 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.565 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.858015+0000", 2024-09-11T08:52:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.858021+0000", 2024-09-11T08:52:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0100000000000001e-06 2024-09-11T08:52:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.858035+0000", 2024-09-11T08:52:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.417e-05 2024-09-11T08:52:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.858040+0000", 2024-09-11T08:52:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.2580000000000003e-06 2024-09-11T08:52:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.858085+0000", 2024-09-11T08:52:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4767e-05 2024-09-11T08:52:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.858091+0000", 2024-09-11T08:52:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5300000000000002e-06 2024-09-11T08:52:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.640430+0000", 2024-09-11T08:52:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.78233838300000003 2024-09-11T08:52:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.640525+0000", 2024-09-11T08:52:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.5013000000000007e-05 2024-09-11T08:52:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:52:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.640624+0000", 2024-09-11T08:52:28.569 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.9619999999999998e-05 2024-09-11T08:52:28.569 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.569 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.569 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:28.569 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.643400+0000", 2024-09-11T08:52:28.569 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0027757860000000001 2024-09-11T08:52:28.569 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.569 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.569 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.569 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.649251+0000", 2024-09-11T08:52:28.569 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0058510300000000001 2024-09-11T08:52:28.570 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.570 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.570 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:28.570 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.659090+0000", 2024-09-11T08:52:28.570 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.009838823 2024-09-11T08:52:28.570 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.570 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.570 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:28.570 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.659131+0000", 2024-09-11T08:52:28.570 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0550000000000003e-05 2024-09-11T08:52:28.570 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:28.571 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:28.571 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:28.571 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.659164+0000", 2024-09-11T08:52:28.571 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3362999999999999e-05 2024-09-11T08:52:28.571 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.571 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.571 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.571 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:28.571 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:28.571 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:52:28.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:28.601 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:28.602 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:28.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:28 smithi145 ceph-mon[33384]: 6.a scrub starts 2024-09-11T08:52:28.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:28 smithi145 ceph-mon[33384]: 6.a scrub ok 2024-09-11T08:52:28.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:28 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:28.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:28 smithi145 ceph-mon[33384]: pgmap v612: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 820 MiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 900 B/s rd, 0 op/s; 741/39930 objects misplaced (1.856%); 27 MiB/s, 436 objects/s recovering 2024-09-11T08:52:28.702 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:28.754 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:28.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:28.755 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:28.855 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:28.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:28.884 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:28.884 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:28.985 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:29.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:29.019 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:29.019 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:29.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:28 smithi092 ceph-mon[29647]: 6.a scrub starts 2024-09-11T08:52:29.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:28 smithi092 ceph-mon[29647]: 6.a scrub ok 2024-09-11T08:52:29.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:28 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:29.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:28 smithi092 ceph-mon[29647]: pgmap v612: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 820 MiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 900 B/s rd, 0 op/s; 741/39930 objects misplaced (1.856%); 27 MiB/s, 436 objects/s recovering 2024-09-11T08:52:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:28 smithi092 ceph-mon[41114]: 6.a scrub starts 2024-09-11T08:52:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:28 smithi092 ceph-mon[41114]: 6.a scrub ok 2024-09-11T08:52:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:28 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:28 smithi092 ceph-mon[41114]: pgmap v612: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 820 MiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 900 B/s rd, 0 op/s; 741/39930 objects misplaced (1.856%); 27 MiB/s, 436 objects/s recovering 2024-09-11T08:52:29.120 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:29.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:29.160 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:29.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:29.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:29.290 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:29.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:29.291 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:29.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:29.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:29.415 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:29.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:29.476 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-11T08:52:29.517 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:29.559 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:29.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:29.561 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:29.662 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:29.712 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:29.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:29.713 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:29.814 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:29.841 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:29.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:29.842 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:29 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:52:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:29 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:52:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:29 smithi145 ceph-mon[33384]: 9.7 scrub starts 2024-09-11T08:52:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:29 smithi145 ceph-mon[33384]: 9.7 scrub ok 2024-09-11T08:52:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:29 smithi145 ceph-mon[33384]: 3.6 deep-scrub starts 2024-09-11T08:52:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:29 smithi145 ceph-mon[33384]: 3.6 deep-scrub ok 2024-09-11T08:52:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:29 smithi145 ceph-mon[33384]: osdmap e283: 8 total, 8 up, 8 in 2024-09-11T08:52:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:29 smithi145 ceph-mon[33384]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:52:29.930 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:52:29.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:29.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:29.972 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:29.972 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:30.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:52:30.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:52:30.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[29647]: 9.7 scrub starts 2024-09-11T08:52:30.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[29647]: 9.7 scrub ok 2024-09-11T08:52:30.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[29647]: 3.6 deep-scrub starts 2024-09-11T08:52:30.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[29647]: 3.6 deep-scrub ok 2024-09-11T08:52:30.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[29647]: osdmap e283: 8 total, 8 up, 8 in 2024-09-11T08:52:30.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[29647]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:52:30.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:52:30.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:52:30.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[41114]: 9.7 scrub starts 2024-09-11T08:52:30.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[41114]: 9.7 scrub ok 2024-09-11T08:52:30.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[41114]: 3.6 deep-scrub starts 2024-09-11T08:52:30.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[41114]: 3.6 deep-scrub ok 2024-09-11T08:52:30.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[41114]: osdmap e283: 8 total, 8 up, 8 in 2024-09-11T08:52:30.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:29 smithi092 ceph-mon[41114]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:52:30.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:30.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:30.107 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:30.107 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:30.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:30.243 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:30.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:30.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:30.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:30.370 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:30.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:30.371 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:30.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:30.488 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:30.529 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:30.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:30.530 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:30.541 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:30.631 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:30.660 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:30.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:30.661 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:30.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[29647]: 9.6 scrub starts 2024-09-11T08:52:30.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[29647]: 9.6 scrub ok 2024-09-11T08:52:30.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[29647]: pgmap v614: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 820 MiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 897 B/s rd, 0 op/s; 741/39930 objects misplaced (1.856%); 23 MiB/s, 370 objects/s recovering 2024-09-11T08:52:30.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[29647]: 9.4 scrub starts 2024-09-11T08:52:30.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[29647]: 9.4 scrub ok 2024-09-11T08:52:30.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:52:30.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:52:30.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[29647]: 4.18 scrub starts 2024-09-11T08:52:30.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[29647]: 4.18 scrub ok 2024-09-11T08:52:30.672 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[41114]: 9.6 scrub starts 2024-09-11T08:52:30.672 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[41114]: 9.6 scrub ok 2024-09-11T08:52:30.672 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[41114]: pgmap v614: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 820 MiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 897 B/s rd, 0 op/s; 741/39930 objects misplaced (1.856%); 23 MiB/s, 370 objects/s recovering 2024-09-11T08:52:30.672 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[41114]: 9.4 scrub starts 2024-09-11T08:52:30.672 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[41114]: 9.4 scrub ok 2024-09-11T08:52:30.672 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:52:30.672 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:52:30.672 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[41114]: 4.18 scrub starts 2024-09-11T08:52:30.673 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:30 smithi092 ceph-mon[41114]: 4.18 scrub ok 2024-09-11T08:52:30.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:30.799 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:30.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:30.800 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:30.880 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:30 smithi145 ceph-mon[33384]: 9.6 scrub starts 2024-09-11T08:52:30.880 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:30 smithi145 ceph-mon[33384]: 9.6 scrub ok 2024-09-11T08:52:30.880 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:30 smithi145 ceph-mon[33384]: pgmap v614: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 820 MiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 897 B/s rd, 0 op/s; 741/39930 objects misplaced (1.856%); 23 MiB/s, 370 objects/s recovering 2024-09-11T08:52:30.880 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:30 smithi145 ceph-mon[33384]: 9.4 scrub starts 2024-09-11T08:52:30.880 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:30 smithi145 ceph-mon[33384]: 9.4 scrub ok 2024-09-11T08:52:30.880 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:30 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:52:30.881 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:52:30.881 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:30 smithi145 ceph-mon[33384]: 4.18 scrub starts 2024-09-11T08:52:30.881 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:30 smithi145 ceph-mon[33384]: 4.18 scrub ok 2024-09-11T08:52:30.901 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:30.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:30.932 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:30.933 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:31.033 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:31.069 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:31.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:31.070 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:31.171 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:31.207 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:31.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:31.208 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:31.309 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:31.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:31.332 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:31.332 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:31.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:31.465 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:31.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:31.466 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:31.533 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:52:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:52:31] "GET /metrics HTTP/1.1" 200 224620 "" "Prometheus/2.43.0" 2024-09-11T08:52:31.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:31.605 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:31.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:31.606 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:31.634 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:52:31.707 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:31.728 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:31.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:31.729 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:31.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:31 smithi092 ceph-mon[29647]: 6.18 scrub starts 2024-09-11T08:52:31.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:31 smithi092 ceph-mon[29647]: 6.18 scrub ok 2024-09-11T08:52:31.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:31 smithi092 ceph-mon[29647]: 9.0 scrub starts 2024-09-11T08:52:31.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:31 smithi092 ceph-mon[29647]: 9.0 scrub ok 2024-09-11T08:52:31.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:31 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1918119779' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:52:31.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:31 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:52:31.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:31 smithi092 ceph-mon[41114]: 6.18 scrub starts 2024-09-11T08:52:31.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:31 smithi092 ceph-mon[41114]: 6.18 scrub ok 2024-09-11T08:52:31.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:31 smithi092 ceph-mon[41114]: 9.0 scrub starts 2024-09-11T08:52:31.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:31 smithi092 ceph-mon[41114]: 9.0 scrub ok 2024-09-11T08:52:31.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:31 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1918119779' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:52:31.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:31 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:52:31.830 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:31.866 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:31.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:31.867 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:31.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:31 smithi145 ceph-mon[33384]: 6.18 scrub starts 2024-09-11T08:52:31.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:31 smithi145 ceph-mon[33384]: 6.18 scrub ok 2024-09-11T08:52:31.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:31 smithi145 ceph-mon[33384]: 9.0 scrub starts 2024-09-11T08:52:31.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:31 smithi145 ceph-mon[33384]: 9.0 scrub ok 2024-09-11T08:52:31.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:31 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1918119779' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:52:31.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:31 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:52:31.968 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:31.989 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:31.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:31.990 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:32.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:32.124 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:32.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:32.125 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:32.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:32.246 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:32.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:32.247 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:32.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:32.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:32.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:32.471 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:32.509 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:32.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:32.510 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:32.611 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:32.647 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:32.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:32.648 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:32.749 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:32.782 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:32.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:32.783 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:32.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[29647]: pgmap v615: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 785 MiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 765 B/s rd, 0 op/s; 545/38238 objects misplaced (1.425%); 20 MiB/s, 315 objects/s recovering 2024-09-11T08:52:32.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[29647]: 3.10 scrub starts 2024-09-11T08:52:32.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[29647]: 3.10 scrub ok 2024-09-11T08:52:32.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[29647]: 9.11 scrub starts 2024-09-11T08:52:32.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:52:32.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:52:32.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[29647]: osdmap e284: 8 total, 8 up, 8 in 2024-09-11T08:52:32.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:52:32.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:52:32.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[41114]: pgmap v615: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 785 MiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 765 B/s rd, 0 op/s; 545/38238 objects misplaced (1.425%); 20 MiB/s, 315 objects/s recovering 2024-09-11T08:52:32.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[41114]: 3.10 scrub starts 2024-09-11T08:52:32.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[41114]: 3.10 scrub ok 2024-09-11T08:52:32.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[41114]: 9.11 scrub starts 2024-09-11T08:52:32.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:52:32.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:52:32.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[41114]: osdmap e284: 8 total, 8 up, 8 in 2024-09-11T08:52:32.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:52:32.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:32 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:52:32.826 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:52:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:52:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:52:32.826 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:32 smithi145 ceph-mon[33384]: pgmap v615: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 785 MiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 765 B/s rd, 0 op/s; 545/38238 objects misplaced (1.425%); 20 MiB/s, 315 objects/s recovering 2024-09-11T08:52:32.826 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:32 smithi145 ceph-mon[33384]: 3.10 scrub starts 2024-09-11T08:52:32.826 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:32 smithi145 ceph-mon[33384]: 3.10 scrub ok 2024-09-11T08:52:32.827 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:32 smithi145 ceph-mon[33384]: 9.11 scrub starts 2024-09-11T08:52:32.827 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:32 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:52:32.827 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:32 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:52:32.827 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:32 smithi145 ceph-mon[33384]: osdmap e284: 8 total, 8 up, 8 in 2024-09-11T08:52:32.827 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:32 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:52:32.827 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:32 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:52:32.884 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:32.923 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:32.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:32.923 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:33.024 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:33.060 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:33.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:33.061 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:33.162 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:33.215 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:33.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:33.216 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:33.317 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:33.349 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:33.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:33.350 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:33.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:33.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:33.486 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:33.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:33.614 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:33.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:33.616 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:33.716 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:33.725 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:33 smithi145 ceph-mon[33384]: osdmap e285: 8 total, 8 up, 8 in 2024-09-11T08:52:33.756 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:33.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:33.757 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:33.858 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:33.885 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:52:33.885 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:52:33.885 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.885 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15366.0:282861 9.b e285/281 9:d8dfc905:::benchmark_data_smithi092_145083_object139463:head v 285'17677, mlcod=285'17675)", 2024-09-11T08:52:33.885 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:33.883071+0000", 2024-09-11T08:52:33.885 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.0019277179999999999, 2024-09-11T08:52:33.885 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0019576730000000001, 2024-09-11T08:52:33.885 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:33.886 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T08:52:33.886 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:33.886 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.886 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:33.886 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883071+0000", 2024-09-11T08:52:33.886 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:33.886 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:33.886 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.886 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:33.886 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883071+0000", 2024-09-11T08:52:33.886 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:33.887 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:33.887 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.887 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:33.887 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883070+0000", 2024-09-11T08:52:33.887 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:33.887 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:33.887 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.887 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:33.887 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883078+0000", 2024-09-11T08:52:33.887 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.9570000000000002e-06 2024-09-11T08:52:33.888 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:33.888 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.888 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:33.888 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883097+0000", 2024-09-11T08:52:33.888 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.8847000000000001e-05 2024-09-11T08:52:33.888 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:33.888 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.888 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:33.888 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883103+0000", 2024-09-11T08:52:33.888 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9850000000000003e-06 2024-09-11T08:52:33.888 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:33.889 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.889 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:33.889 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883262+0000", 2024-09-11T08:52:33.889 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015902200000000001 2024-09-11T08:52:33.889 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:33.889 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.889 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:33.889 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883311+0000", 2024-09-11T08:52:33.889 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9638999999999997e-05 2024-09-11T08:52:33.889 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:33.889 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:33.890 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:33.890 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:33.890 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.890 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15366.0:282860 9.11 e285/237 9:8983b53d:::benchmark_data_smithi092_145083_object139462:head v 285'12794, mlcod=285'12792)", 2024-09-11T08:52:33.890 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:33.883739+0000", 2024-09-11T08:52:33.890 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.001259584, 2024-09-11T08:52:33.890 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001351552, 2024-09-11T08:52:33.890 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:33.890 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T08:52:33.890 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:33.890 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.891 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:33.891 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883739+0000", 2024-09-11T08:52:33.891 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:33.891 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:33.891 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.891 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:33.891 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883739+0000", 2024-09-11T08:52:33.891 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:33.891 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:33.891 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.891 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:33.892 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883739+0000", 2024-09-11T08:52:33.892 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:52:33.892 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:33.892 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.892 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:33.892 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883746+0000", 2024-09-11T08:52:33.892 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.058e-06 2024-09-11T08:52:33.892 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:33.892 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.892 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:33.892 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883761+0000", 2024-09-11T08:52:33.893 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5299999999999999e-05 2024-09-11T08:52:33.893 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:33.893 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.893 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:33.893 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883768+0000", 2024-09-11T08:52:33.893 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.7220000000000002e-06 2024-09-11T08:52:33.893 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:33.893 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.893 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:33.893 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883808+0000", 2024-09-11T08:52:33.893 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0064999999999997e-05 2024-09-11T08:52:33.893 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:33.894 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:33.894 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:33.894 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:33.883828+0000", 2024-09-11T08:52:33.894 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0103999999999999e-05 2024-09-11T08:52:33.894 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:33.894 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:33.894 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:33.894 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:33.894 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2024-09-11T08:52:33.894 INFO:teuthology.orchestra.run.smithi145.stdout: "num_ops": 2 2024-09-11T08:52:33.894 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:52:33.897 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:33.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:33.898 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:33.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:34.031 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:34.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:34.032 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:34.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:33 smithi092 ceph-mon[29647]: osdmap e285: 8 total, 8 up, 8 in 2024-09-11T08:52:34.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:33 smithi092 ceph-mon[41114]: osdmap e285: 8 total, 8 up, 8 in 2024-09-11T08:52:34.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:34.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:34.199 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:34.199 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:34.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:34.334 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:34.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:34.335 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:34.436 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:34.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:34.474 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:34.475 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:34.575 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:34.665 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-11T08:52:34.741 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:34.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:34.742 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:34.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:34.871 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:34.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:34.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:34.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:34 smithi145 ceph-mon[33384]: pgmap v618: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 438 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 374/21612 objects misplaced (1.731%) 2024-09-11T08:52:34.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:35.002 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:35.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:35.004 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:35.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:34 smithi092 ceph-mon[29647]: pgmap v618: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 438 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 374/21612 objects misplaced (1.731%) 2024-09-11T08:52:35.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:34 smithi092 ceph-mon[41114]: pgmap v618: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 438 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 374/21612 objects misplaced (1.731%) 2024-09-11T08:52:35.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:35.134 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:52:35.144 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:35.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:35.145 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:35.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:35.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:35.283 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:35.283 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:35.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:35.415 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:35.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:35.416 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:35.517 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:35.562 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:35.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:35.565 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:35.666 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:35.721 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:35.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:35.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:35.731 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:35 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:52:35.731 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:35 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:52:35.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:35.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:35.859 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:35.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:35.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:35 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:52:35.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:36.004 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:36.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:36.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:36.010 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:52:36.010 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:52:36.036 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:52:36.036 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:52:36.062 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:52:36.062 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:52:36.092 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:52:36.093 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:52:36.106 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:36.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:52:36.132 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:52:36.132 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:52:36.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:36.149 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:36.149 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:36.176 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:52:36.177 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:52:36.218 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:52:36.218 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:52:36.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:36.256 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:52:36.257 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:52:36.364 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:36.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:36.365 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:36.466 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:36.498 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:36.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:36.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:36.600 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:36.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:36.639 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:36.640 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:36.740 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:36.764 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Removed 143395 objects 2024-09-11T08:52:36.764 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Clean up completed and total clean up time :112.021 2024-09-11T08:52:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:36.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:36.772 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:36.873 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:36.894 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:36.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:36.896 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:36.911 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:52:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:52:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:36 smithi145 ceph-mon[33384]: pgmap v619: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 438 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 768 B/s rd, 0 op/s; 374/21612 objects misplaced (1.731%) 2024-09-11T08:52:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:36 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2600849700' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:52:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:36 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:52:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:36 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:52:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:36 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:52:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:36 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1392037464' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.2356119650261148}]: dispatch 2024-09-11T08:52:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:36 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.2356119650261148}]: dispatch 2024-09-11T08:52:36.996 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:37.018 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:37.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:37.019 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:37.022 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:36 smithi092 ceph-mon[29647]: pgmap v619: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 438 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 768 B/s rd, 0 op/s; 374/21612 objects misplaced (1.731%) 2024-09-11T08:52:37.022 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:36 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2600849700' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:52:37.022 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:36 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:52:37.023 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:36 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:52:37.023 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:36 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:52:37.023 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:36 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1392037464' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.2356119650261148}]: dispatch 2024-09-11T08:52:37.023 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:36 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.2356119650261148}]: dispatch 2024-09-11T08:52:37.023 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:36 smithi092 ceph-mon[41114]: pgmap v619: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 438 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 768 B/s rd, 0 op/s; 374/21612 objects misplaced (1.731%) 2024-09-11T08:52:37.023 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:36 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2600849700' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:52:37.023 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:36 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:52:37.023 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:36 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:52:37.023 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:36 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:52:37.023 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:36 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1392037464' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.2356119650261148}]: dispatch 2024-09-11T08:52:37.023 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:36 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.2356119650261148}]: dispatch 2024-09-11T08:52:37.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:37.141 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:37.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:37.142 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:37.146 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:37.203 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:52:37.204 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:52:37.243 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:37.266 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:37.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:37.267 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:37.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:37.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:37.393 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:37.394 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:37.475 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.7 primary-affinity to 0.235612 (8154412) 2024-09-11T08:52:37.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:37.516 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:37.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:37.517 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:37.618 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:37.640 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:37.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:37.641 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:37.742 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:37.763 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:37.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:37.764 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:37.865 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:37.887 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:37.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:37.888 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:37.989 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:38.011 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:38.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:38.012 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:38.113 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:38.135 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:38.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:38.136 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:38.237 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:38.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:38.262 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:38.263 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:38.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:38.385 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:38.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:38.387 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:38.487 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:38.514 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:38.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:38.516 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:38 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:38.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:38 smithi092 ceph-mon[29647]: pgmap v620: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 26 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:52:38.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:38 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.2356119650261148}]': finished 2024-09-11T08:52:38.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:38 smithi092 ceph-mon[29647]: osdmap e286: 8 total, 8 up, 8 in 2024-09-11T08:52:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:38 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:38 smithi092 ceph-mon[41114]: pgmap v620: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 26 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:52:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:38 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.2356119650261148}]': finished 2024-09-11T08:52:38.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:38 smithi092 ceph-mon[41114]: osdmap e286: 8 total, 8 up, 8 in 2024-09-11T08:52:38.618 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:38.643 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:38.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:38.644 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:38.658 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-11T08:52:38.745 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:38.767 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:38.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:38.768 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:38.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:38.891 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:38.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:38.892 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:38.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:38 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:38.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:38 smithi145 ceph-mon[33384]: pgmap v620: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 26 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:52:38.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:38 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.2356119650261148}]': finished 2024-09-11T08:52:38.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:38 smithi145 ceph-mon[33384]: osdmap e286: 8 total, 8 up, 8 in 2024-09-11T08:52:38.949 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:52:38.993 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:39.015 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:39.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:39.016 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:39.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:52:39.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:39.137 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:39.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:39.138 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:39.239 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:39.262 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:39.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:39.263 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:39.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:39.387 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:39.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:39.388 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:39.489 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:39.510 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:39.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:39.511 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:39.534 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:39.612 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:39.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:39.635 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:39.635 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:39.660 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:39 smithi145 ceph-mon[33384]: osdmap e287: 8 total, 8 up, 8 in 2024-09-11T08:52:39.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:39.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:39.760 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:39.760 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:39.766 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:39 smithi092 ceph-mon[29647]: osdmap e287: 8 total, 8 up, 8 in 2024-09-11T08:52:39.766 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:39 smithi092 ceph-mon[41114]: osdmap e287: 8 total, 8 up, 8 in 2024-09-11T08:52:39.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:39.883 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:39.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:39.884 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:39.985 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:40.009 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:40.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:40.010 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:40.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:40.133 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:40.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:40.134 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:40.235 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:40.256 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:40.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:40.257 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:40.358 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:40.380 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:40.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:40.381 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:40.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:40.494 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:52:40.506 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:40.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:40.507 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:40.608 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:40.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:40 smithi145 ceph-mon[33384]: pgmap v623: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 26 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 770 B/s rd, 0 B/s wr, 1.00k op/s 2024-09-11T08:52:40.612 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:40 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1992502899' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:52:40.630 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:40.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:40.631 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:40.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:40.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:40.755 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:40.763 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:40 smithi092 ceph-mon[41114]: pgmap v623: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 26 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 770 B/s rd, 0 B/s wr, 1.00k op/s 2024-09-11T08:52:40.763 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:40 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1992502899' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:52:40.763 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:40 smithi092 ceph-mon[29647]: pgmap v623: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 26 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 770 B/s rd, 0 B/s wr, 1.00k op/s 2024-09-11T08:52:40.763 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:40 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1992502899' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:52:40.816 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:52:40.817 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:52:40.856 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:40.877 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:40.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:40.881 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:40.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:41.004 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:41.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:41.005 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:41.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:41.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:41.128 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:41.128 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:41.229 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:41.250 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:41.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:41.251 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:41.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:41.374 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:41.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:41.375 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:41.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:41.499 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:41.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:41.500 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:41.588 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T08:52:41.600 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:41.622 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:41.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:41.623 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:41.723 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:41.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:41.746 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:41.746 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:41.788 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:41 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:52:41.788 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:41 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1992502899' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:52:41.788 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:41 smithi092 ceph-mon[29647]: osdmap e288: 8 total, 8 up, 8 in 2024-09-11T08:52:41.788 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:52:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:52:41] "GET /metrics HTTP/1.1" 200 224620 "" "Prometheus/2.43.0" 2024-09-11T08:52:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:41 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:52:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:41 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1992502899' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:52:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:41 smithi092 ceph-mon[41114]: osdmap e288: 8 total, 8 up, 8 in 2024-09-11T08:52:41.830 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:52:41.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:41.868 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:41.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:41.869 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:41.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:41 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:52:41.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:41 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1992502899' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:52:41.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:41 smithi145 ceph-mon[33384]: osdmap e288: 8 total, 8 up, 8 in 2024-09-11T08:52:41.971 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:41.992 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:41.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:41.993 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:42.094 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:42.097 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:52:42.097 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:52:42.108 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:52:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:52:42.116 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:42.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:42.117 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:42.120 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_2 2024-09-11T08:52:42.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2024-09-11T08:52:42.123 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:52:42.123 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:52:42.138 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:52:42.139 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:52:42.154 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:52:42.154 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:52:42.170 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:52:42.170 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:52:42.185 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:52:42.185 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:52:42.200 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:52:42.201 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:52:42.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:42.239 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:42.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:42.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:42.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:42.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:42.362 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:42.463 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:42.484 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:42.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:42.485 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:42.586 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:42.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:42.608 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:42.608 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:42 smithi145 ceph-mon[33384]: pgmap v625: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 22 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 1.13k op/s 2024-09-11T08:52:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:42 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1894459459' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:52:42.709 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:42.731 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:42.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:42.732 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:42.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:42 smithi092 ceph-mon[41114]: pgmap v625: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 22 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 1.13k op/s 2024-09-11T08:52:42.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:42 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1894459459' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:52:42.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:42 smithi092 ceph-mon[29647]: pgmap v625: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 22 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 1.13k op/s 2024-09-11T08:52:42.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:42 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1894459459' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:52:42.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:42.854 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:42.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:42.855 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:42.955 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:42.976 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:42.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:42.977 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:43.004 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:52:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:52:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:52:43.064 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:43.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:43.102 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:43.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:43.103 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:43.204 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:43.226 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:43.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:43.227 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:43.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:43.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:43.350 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:43.350 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:43.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:43.473 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:43.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:43.474 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:43.575 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:43.598 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:43.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:43.599 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:43.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:43.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:43.722 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:43.791 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:52:43.791 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T08:52:43.791 INFO:teuthology.orchestra.run.smithi145.stdout: "num_ops": 0 2024-09-11T08:52:43.791 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:52:43.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:52:43.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:43.876 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:43.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:43.877 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:43.957 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:52:43.958 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:52:43.958 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-11T08:52:43.958 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.171193 2024-09-11T08:52:43.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 0 0.1711928788699405 2024-09-11T08:52:43.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:44.022 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:44.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:44.023 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:44.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:44.146 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:44.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:44.147 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:44.247 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:44.271 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:44.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:44.272 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:44.373 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:44.396 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:44.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:44.397 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:44.420 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-11T08:52:44.499 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:44.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:44.525 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:44.525 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:44.626 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:44.648 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:44.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:44.649 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:44 smithi145 ceph-mon[33384]: pgmap v626: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 15 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 41 op/s 2024-09-11T08:52:44.665 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:44 smithi092 ceph-mon[29647]: pgmap v626: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 15 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 41 op/s 2024-09-11T08:52:44.665 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:44 smithi092 ceph-mon[41114]: pgmap v626: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 15 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 41 op/s 2024-09-11T08:52:44.705 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:52:44.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:44.773 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:44.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:44.774 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:44.874 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:44.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:44.898 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:44.898 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:44.912 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:44.948 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:44.999 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:45.021 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:45.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:45.022 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:45.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:52:45.123 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:45.144 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:45.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:45.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:45.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:45.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:45.268 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:45.369 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:45.390 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:45.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:45.391 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:45.492 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:45.514 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'unique_pool_2' removed 2024-09-11T08:52:45.518 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:45.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:45.519 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:45.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:45 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3361330605' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:52:45.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:45 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:52:45.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:45 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:52:45.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:52:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:45 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3361330605' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:52:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:45 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:52:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:45 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:52:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:52:45.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:45.642 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:45.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:45.643 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:45.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:45.765 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:45.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:45.766 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:45.867 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:45.889 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:45.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:45.890 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:45.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:45 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3361330605' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:52:45.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:45 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:52:45.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:45 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:52:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:52:45.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:46.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:46.027 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:46.027 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:46.128 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:46.149 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:46.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:46.150 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:46.251 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:46.272 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:46.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:46.273 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:46.281 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-11T08:52:46.282 INFO:tasks.radosbench:Beginning radosbench... 2024-09-11T08:52:46.283 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_3 2024-09-11T08:52:46.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool create unique_pool_3 16 2024-09-11T08:52:46.374 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:46.395 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:46.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:46.396 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:46.497 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:46.519 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:46.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:46.521 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:46 smithi092 ceph-mon[29647]: pgmap v627: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 15 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 31 op/s 2024-09-11T08:52:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:46 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:52:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:46 smithi092 ceph-mon[29647]: osdmap e289: 8 total, 8 up, 8 in 2024-09-11T08:52:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:46 smithi092 ceph-mon[41114]: pgmap v627: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 15 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 31 op/s 2024-09-11T08:52:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:46 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:52:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:46 smithi092 ceph-mon[41114]: osdmap e289: 8 total, 8 up, 8 in 2024-09-11T08:52:46.575 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:46 smithi145 ceph-mon[33384]: pgmap v627: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 15 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 31 op/s 2024-09-11T08:52:46.575 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:46 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:52:46.575 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:46 smithi145 ceph-mon[33384]: osdmap e289: 8 total, 8 up, 8 in 2024-09-11T08:52:46.581 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:52:46.582 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T08:52:46.582 INFO:teuthology.orchestra.run.smithi145.stdout: "complaint_time": 30, 2024-09-11T08:52:46.582 INFO:teuthology.orchestra.run.smithi145.stdout: "num_blocked_ops": 0 2024-09-11T08:52:46.582 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:52:46.621 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:46.643 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:46.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:46.644 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:46.745 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:46.766 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:46.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:46.767 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:46.867 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:46.888 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:46.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:46.890 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:46.990 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:47.011 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:47.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:47.012 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:47.113 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:47.135 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:47.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:47.136 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:47.161 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:52:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:52:47.237 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:47.252 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-11T08:52:47.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:47.259 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:47.260 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:47.361 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:47.383 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:47.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:47.384 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:47.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:47.507 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:47.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:47.508 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:47.510 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:52:47.609 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:47.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:47.632 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:47.632 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:47.652 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:47 smithi092 ceph-mon[29647]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-11T08:52:47.652 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:47 smithi092 ceph-mon[41114]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-11T08:52:47.733 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:47.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:47.755 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:47.755 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:47.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:47.878 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:47.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:47.879 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:47.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:47 smithi145 ceph-mon[33384]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-11T08:52:47.980 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:48.002 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:48.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:48.003 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:48.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:48.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:48.126 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:48.127 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:48.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:48.248 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:48.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:48.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:48.350 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:48.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:48.375 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:48.375 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:48.476 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:48.499 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:48.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:48.501 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:48.537 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:48.561 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.0 primary-affinity to 0.171193 (8112192) 2024-09-11T08:52:48.601 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:48.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:48.623 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:48.623 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:48.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:48.745 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:48.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:48.746 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:48.794 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:48 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:48.794 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:48 smithi145 ceph-mon[33384]: pgmap v629: 142 pgs: 142 active+clean; 477 KiB data, 4.7 GiB used, 711 GiB / 715 GiB avail 2024-09-11T08:52:48.795 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:48 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4044800497' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 0.1711928788699405}]: dispatch 2024-09-11T08:52:48.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:48 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:48.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:48 smithi092 ceph-mon[29647]: pgmap v629: 142 pgs: 142 active+clean; 477 KiB data, 4.7 GiB used, 711 GiB / 715 GiB avail 2024-09-11T08:52:48.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:48 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4044800497' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 0.1711928788699405}]: dispatch 2024-09-11T08:52:48.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:48 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:48.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:48 smithi092 ceph-mon[41114]: pgmap v629: 142 pgs: 142 active+clean; 477 KiB data, 4.7 GiB used, 711 GiB / 715 GiB avail 2024-09-11T08:52:48.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:48 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4044800497' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 0.1711928788699405}]: dispatch 2024-09-11T08:52:48.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:48.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:48.869 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:48.870 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:48.971 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:48.992 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:48.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:48.993 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:49.094 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:49.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:49.117 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:49.117 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:49.218 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:49.239 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:49.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:49.240 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:49.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:49.359 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:52:49.359 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T08:52:49.359 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T08:52:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:52:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120775 9.19 9:9f3327b4:::benchmark_data_smithi092_145083_object120774:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:52:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.527015+0000", 2024-09-11T08:52:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 143.783705239, 2024-09-11T08:52:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0204559310000001, 2024-09-11T08:52:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120775 2024-09-11T08:52:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527015+0000", 2024-09-11T08:52:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527015+0000", 2024-09-11T08:52:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527008+0000", 2024-09-11T08:52:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999933 2024-09-11T08:52:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527071+0000", 2024-09-11T08:52:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2667000000000002e-05 2024-09-11T08:52:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527089+0000", 2024-09-11T08:52:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.8498000000000001e-05 2024-09-11T08:52:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527099+0000", 2024-09-11T08:52:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0016e-05 2024-09-11T08:52:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.676418+0000", 2024-09-11T08:52:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.14931815200000001 2024-09-11T08:52:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.676427+0000", 2024-09-11T08:52:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.8320000000000001e-06 2024-09-11T08:52:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538222+0000", 2024-09-11T08:52:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.86179418299999999 2024-09-11T08:52:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538346+0000", 2024-09-11T08:52:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012482300000000001 2024-09-11T08:52:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,5", 2024-09-11T08:52:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538574+0000", 2024-09-11T08:52:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00022776599999999999 2024-09-11T08:52:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.539918+0000", 2024-09-11T08:52:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001344065 2024-09-11T08:52:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.539927+0000", 2024-09-11T08:52:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.4710000000000007e-06 2024-09-11T08:52:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547401+0000", 2024-09-11T08:52:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.007474217 2024-09-11T08:52:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547440+0000", 2024-09-11T08:52:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9490999999999998e-05 2024-09-11T08:52:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547471+0000", 2024-09-11T08:52:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0651000000000002e-05 2024-09-11T08:52:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120776 9.13 9:cecf0514:::benchmark_data_smithi092_145083_object120775:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:52:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.655311+0000", 2024-09-11T08:52:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 143.655408897, 2024-09-11T08:52:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89210484800000001, 2024-09-11T08:52:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120776 2024-09-11T08:52:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655311+0000", 2024-09-11T08:52:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655311+0000", 2024-09-11T08:52:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655309+0000", 2024-09-11T08:52:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:52:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655369+0000", 2024-09-11T08:52:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0052e-05 2024-09-11T08:52:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655395+0000", 2024-09-11T08:52:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5205999999999999e-05 2024-09-11T08:52:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655405+0000", 2024-09-11T08:52:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0399e-05 2024-09-11T08:52:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.659087+0000", 2024-09-11T08:52:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0036818240000000002 2024-09-11T08:52:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.659098+0000", 2024-09-11T08:52:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1600000000000001e-05 2024-09-11T08:52:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.537736+0000", 2024-09-11T08:52:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.87863754699999996 2024-09-11T08:52:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.537834+0000", 2024-09-11T08:52:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.7986999999999995e-05 2024-09-11T08:52:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:52:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538018+0000", 2024-09-11T08:52:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00018384599999999999 2024-09-11T08:52:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538962+0000", 2024-09-11T08:52:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00094433400000000002 2024-09-11T08:52:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.540658+0000", 2024-09-11T08:52:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0016956969999999999 2024-09-11T08:52:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547326+0000", 2024-09-11T08:52:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0066684819999999999 2024-09-11T08:52:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547381+0000", 2024-09-11T08:52:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4537999999999999e-05 2024-09-11T08:52:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547416+0000", 2024-09-11T08:52:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5389000000000003e-05 2024-09-11T08:52:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120777 9.3 9:c63642ab:::benchmark_data_smithi092_145083_object120776:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:52:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.681266+0000", 2024-09-11T08:52:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 143.62945449099999, 2024-09-11T08:52:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.85751633400000005, 2024-09-11T08:52:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120777 2024-09-11T08:52:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681266+0000", 2024-09-11T08:52:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681266+0000", 2024-09-11T08:52:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681264+0000", 2024-09-11T08:52:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681297+0000", 2024-09-11T08:52:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2461e-05 2024-09-11T08:52:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681309+0000", 2024-09-11T08:52:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2519e-05 2024-09-11T08:52:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681314+0000", 2024-09-11T08:52:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1279999999999999e-06 2024-09-11T08:52:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681360+0000", 2024-09-11T08:52:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5438000000000001e-05 2024-09-11T08:52:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681368+0000", 2024-09-11T08:52:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2949999999999994e-06 2024-09-11T08:52:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.525926+0000", 2024-09-11T08:52:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.84455778599999998 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.526080+0000", 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015359799999999999 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.526368+0000", 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000288027 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.527948+0000", 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0015808269999999999 2024-09-11T08:52:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.535215+0000", 2024-09-11T08:52:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0072669259999999999 2024-09-11T08:52:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538703+0000", 2024-09-11T08:52:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0034881640000000002 2024-09-11T08:52:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538748+0000", 2024-09-11T08:52:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4304999999999997e-05 2024-09-11T08:52:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538782+0000", 2024-09-11T08:52:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4400000000000003e-05 2024-09-11T08:52:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122628 9.13 9:cbe7bc43:::benchmark_data_smithi092_145083_object122627:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236)", 2024-09-11T08:52:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.520279+0000", 2024-09-11T08:52:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 141.790441227, 2024-09-11T08:52:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.435847541, 2024-09-11T08:52:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:52:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122628 2024-09-11T08:52:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520279+0000", 2024-09-11T08:52:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520279+0000", 2024-09-11T08:52:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520278+0000", 2024-09-11T08:52:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520339+0000", 2024-09-11T08:52:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1049000000000005e-05 2024-09-11T08:52:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520349+0000", 2024-09-11T08:52:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.6190000000000006e-06 2024-09-11T08:52:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520353+0000", 2024-09-11T08:52:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3220000000000003e-06 2024-09-11T08:52:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520397+0000", 2024-09-11T08:52:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3739000000000003e-05 2024-09-11T08:52:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520511+0000", 2024-09-11T08:52:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011360499999999999 2024-09-11T08:52:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:52:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520760+0000", 2024-09-11T08:52:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00024918800000000001 2024-09-11T08:52:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.522187+0000", 2024-09-11T08:52:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0014270960000000001 2024-09-11T08:52:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.527785+0000", 2024-09-11T08:52:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0055977650000000002 2024-09-11T08:52:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.536750+0000", 2024-09-11T08:52:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0089646959999999994 2024-09-11T08:52:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.536758+0000", 2024-09-11T08:52:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.1410000000000005e-06 2024-09-11T08:52:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537393+0000", 2024-09-11T08:52:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.000635798 2024-09-11T08:52:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537404+0000", 2024-09-11T08:52:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0471e-05 2024-09-11T08:52:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.956039+0000", 2024-09-11T08:52:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.41863534800000002 2024-09-11T08:52:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.956127+0000", 2024-09-11T08:52:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.7373999999999994e-05 2024-09-11T08:52:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122632 9.13 9:cb01d006:::benchmark_data_smithi092_145083_object122631:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236)", 2024-09-11T08:52:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.523542+0000", 2024-09-11T08:52:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 141.78717808100001, 2024-09-11T08:52:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.43267765, 2024-09-11T08:52:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:52:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122632 2024-09-11T08:52:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523542+0000", 2024-09-11T08:52:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523542+0000", 2024-09-11T08:52:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523541+0000", 2024-09-11T08:52:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:52:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523573+0000", 2024-09-11T08:52:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2162000000000001e-05 2024-09-11T08:52:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523585+0000", 2024-09-11T08:52:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1269e-05 2024-09-11T08:52:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523590+0000", 2024-09-11T08:52:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5720000000000002e-06 2024-09-11T08:52:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523636+0000", 2024-09-11T08:52:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6233999999999998e-05 2024-09-11T08:52:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523944+0000", 2024-09-11T08:52:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00030767799999999998 2024-09-11T08:52:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:52:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.524202+0000", 2024-09-11T08:52:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00025762300000000001 2024-09-11T08:52:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.528196+0000", 2024-09-11T08:52:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0039945980000000002 2024-09-11T08:52:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.530011+0000", 2024-09-11T08:52:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001814525 2024-09-11T08:52:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.537490+0000", 2024-09-11T08:52:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0074790059999999998 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.537523+0000", 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3389000000000002e-05 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537576+0000", 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0000530000000001 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537583+0000", 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9650000000000002e-06 2024-09-11T08:52:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.956160+0000", 2024-09-11T08:52:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.41857655500000002 2024-09-11T08:52:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.956220+0000", 2024-09-11T08:52:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0102000000000002e-05 2024-09-11T08:52:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122660 9.17 9:e8b491cd:::benchmark_data_smithi092_145083_object122659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:52:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.551000+0000", 2024-09-11T08:52:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 141.759720493, 2024-09-11T08:52:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4263746720000001, 2024-09-11T08:52:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122660 2024-09-11T08:52:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551000+0000", 2024-09-11T08:52:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551000+0000", 2024-09-11T08:52:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.550999+0000", 2024-09-11T08:52:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:52:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551060+0000", 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1168999999999997e-05 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551072+0000", 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2099000000000001e-05 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551078+0000", 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5629999999999999e-06 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551141+0000", 2024-09-11T08:52:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3082000000000004e-05 2024-09-11T08:52:49.399 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.399 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.399 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.399 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551150+0000", 2024-09-11T08:52:49.399 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8459999999999997e-06 2024-09-11T08:52:49.399 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.399 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.399 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.399 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537577+0000", 2024-09-11T08:52:49.399 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.98642732899999996 2024-09-11T08:52:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537585+0000", 2024-09-11T08:52:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2230000000000007e-06 2024-09-11T08:52:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.961506+0000", 2024-09-11T08:52:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.42392071100000001 2024-09-11T08:52:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.961706+0000", 2024-09-11T08:52:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00020044799999999999 2024-09-11T08:52:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2,3,5", 2024-09-11T08:52:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.962037+0000", 2024-09-11T08:52:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00033083199999999998 2024-09-11T08:52:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.964043+0000", 2024-09-11T08:52:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0020053179999999999 2024-09-11T08:52:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.965851+0000", 2024-09-11T08:52:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0018087610000000001 2024-09-11T08:52:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.967059+0000", 2024-09-11T08:52:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0012074080000000001 2024-09-11T08:52:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.967101+0000", 2024-09-11T08:52:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2601000000000001e-05 2024-09-11T08:52:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.977211+0000", 2024-09-11T08:52:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.010109315000000001 2024-09-11T08:52:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.977264+0000", 2024-09-11T08:52:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3288999999999998e-05 2024-09-11T08:52:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.977375+0000", 2024-09-11T08:52:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000110554 2024-09-11T08:52:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122671 9.15 9:ab433584:::benchmark_data_smithi092_145083_object122670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:52:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.558491+0000", 2024-09-11T08:52:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 141.752229675, 2024-09-11T08:52:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3936105059999999, 2024-09-11T08:52:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122671 2024-09-11T08:52:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558491+0000", 2024-09-11T08:52:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558491+0000", 2024-09-11T08:52:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558490+0000", 2024-09-11T08:52:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:52:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558704+0000", 2024-09-11T08:52:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00021395600000000001 2024-09-11T08:52:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558716+0000", 2024-09-11T08:52:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.278e-05 2024-09-11T08:52:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558722+0000", 2024-09-11T08:52:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4999999999999999e-06 2024-09-11T08:52:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558765+0000", 2024-09-11T08:52:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2815999999999998e-05 2024-09-11T08:52:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558770+0000", 2024-09-11T08:52:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1660000000000002e-06 2024-09-11T08:52:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537512+0000", 2024-09-11T08:52:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97874157399999995 2024-09-11T08:52:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537520+0000", 2024-09-11T08:52:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.5860000000000005e-06 2024-09-11T08:52:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.932889+0000", 2024-09-11T08:52:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.39536916900000002 2024-09-11T08:52:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.933050+0000", 2024-09-11T08:52:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00016028799999999999 2024-09-11T08:52:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3,5", 2024-09-11T08:52:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.933393+0000", 2024-09-11T08:52:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00034307499999999998 2024-09-11T08:52:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.937335+0000", 2024-09-11T08:52:49.410 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0039422570000000002 2024-09-11T08:52:49.410 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.410 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.410 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.410 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.939190+0000", 2024-09-11T08:52:49.410 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0018552729999999999 2024-09-11T08:52:49.410 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.410 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.410 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.410 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.942079+0000", 2024-09-11T08:52:49.410 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.002888767 2024-09-11T08:52:49.410 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.951900+0000", 2024-09-11T08:52:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0098207899999999994 2024-09-11T08:52:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.951972+0000", 2024-09-11T08:52:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2611e-05 2024-09-11T08:52:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.952101+0000", 2024-09-11T08:52:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012884499999999999 2024-09-11T08:52:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:161920 9.15 9:afdf2969:::benchmark_data_smithi092_145083_object18522:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:52:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:58.943979+0000", 2024-09-11T08:52:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 110.366741682, 2024-09-11T08:52:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.001253038, 2024-09-11T08:52:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 161920 2024-09-11T08:52:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.943979+0000", 2024-09-11T08:52:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.943979+0000", 2024-09-11T08:52:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.943978+0000", 2024-09-11T08:52:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:52:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.943982+0000", 2024-09-11T08:52:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5090000000000004e-06 2024-09-11T08:52:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.944019+0000", 2024-09-11T08:52:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6127999999999997e-05 2024-09-11T08:52:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.944024+0000", 2024-09-11T08:52:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9320000000000001e-06 2024-09-11T08:52:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.945966+0000", 2024-09-11T08:52:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001941047 2024-09-11T08:52:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.945973+0000", 2024-09-11T08:52:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.267e-06 2024-09-11T08:52:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.924511+0000", 2024-09-11T08:52:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97853800700000004 2024-09-11T08:52:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.924629+0000", 2024-09-11T08:52:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000117945 2024-09-11T08:52:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:52:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.924992+0000", 2024-09-11T08:52:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00036293600000000001 2024-09-11T08:52:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.931573+0000", 2024-09-11T08:52:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0065809639999999999 2024-09-11T08:52:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945131+0000", 2024-09-11T08:52:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013558374999999999 2024-09-11T08:52:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945175+0000", 2024-09-11T08:52:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3695000000000001e-05 2024-09-11T08:52:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945208+0000", 2024-09-11T08:52:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3596000000000002e-05 2024-09-11T08:52:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945232+0000", 2024-09-11T08:52:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3382000000000001e-05 2024-09-11T08:52:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:161947 9.15 9:a9036c16:::benchmark_data_smithi092_145083_object18549:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:52:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:58.962388+0000", 2024-09-11T08:52:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 110.348332498, 2024-09-11T08:52:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.98294829399999994, 2024-09-11T08:52:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 161947 2024-09-11T08:52:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962388+0000", 2024-09-11T08:52:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962388+0000", 2024-09-11T08:52:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962387+0000", 2024-09-11T08:52:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962392+0000", 2024-09-11T08:52:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8999999999999997e-06 2024-09-11T08:52:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962400+0000", 2024-09-11T08:52:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6109999999999996e-06 2024-09-11T08:52:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962404+0000", 2024-09-11T08:52:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5549999999999996e-06 2024-09-11T08:52:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962462+0000", 2024-09-11T08:52:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7262999999999997e-05 2024-09-11T08:52:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962468+0000", 2024-09-11T08:52:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3049999999999998e-06 2024-09-11T08:52:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.926681+0000", 2024-09-11T08:52:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.96421365999999997 2024-09-11T08:52:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.926819+0000", 2024-09-11T08:52:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013747 2024-09-11T08:52:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:52:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.927081+0000", 2024-09-11T08:52:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00026203400000000002 2024-09-11T08:52:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.931577+0000", 2024-09-11T08:52:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.004495644 2024-09-11T08:52:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945265+0000", 2024-09-11T08:52:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013688709 2024-09-11T08:52:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945295+0000", 2024-09-11T08:52:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9221e-05 2024-09-11T08:52:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945318+0000", 2024-09-11T08:52:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3699000000000001e-05 2024-09-11T08:52:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945336+0000", 2024-09-11T08:52:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7876999999999998e-05 2024-09-11T08:52:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:161955 9.15 9:aef2dade:::benchmark_data_smithi092_145083_object18557:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:52:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:58.968477+0000", 2024-09-11T08:52:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 110.342243279, 2024-09-11T08:52:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97706887499999995, 2024-09-11T08:52:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 161955 2024-09-11T08:52:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968477+0000", 2024-09-11T08:52:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968477+0000", 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968477+0000", 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968479+0000", 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5830000000000001e-06 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968485+0000", 2024-09-11T08:52:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0410000000000003e-06 2024-09-11T08:52:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968489+0000", 2024-09-11T08:52:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4520000000000002e-06 2024-09-11T08:52:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968632+0000", 2024-09-11T08:52:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014362199999999999 2024-09-11T08:52:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968639+0000", 2024-09-11T08:52:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8290000000000002e-06 2024-09-11T08:52:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.928106+0000", 2024-09-11T08:52:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.95946673199999999 2024-09-11T08:52:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.928211+0000", 2024-09-11T08:52:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000104817 2024-09-11T08:52:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:52:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.928417+0000", 2024-09-11T08:52:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00020603400000000001 2024-09-11T08:52:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.933865+0000", 2024-09-11T08:52:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0054485180000000003 2024-09-11T08:52:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945414+0000", 2024-09-11T08:52:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011548388999999999 2024-09-11T08:52:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945462+0000", 2024-09-11T08:52:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8201000000000001e-05 2024-09-11T08:52:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945511+0000", 2024-09-11T08:52:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9518999999999998e-05 2024-09-11T08:52:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945546+0000", 2024-09-11T08:52:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4551999999999997e-05 2024-09-11T08:52:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:161996 9.15 9:a871d7df:::benchmark_data_smithi092_145083_object18598:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:52:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:59.038626+0000", 2024-09-11T08:52:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 110.272094157, 2024-09-11T08:52:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90711806399999995, 2024-09-11T08:52:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 161996 2024-09-11T08:52:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038626+0000", 2024-09-11T08:52:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038626+0000", 2024-09-11T08:52:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038625+0000", 2024-09-11T08:52:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:52:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038633+0000", 2024-09-11T08:52:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.9629999999999998e-06 2024-09-11T08:52:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038645+0000", 2024-09-11T08:52:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2421e-05 2024-09-11T08:52:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038651+0000", 2024-09-11T08:52:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3399999999999997e-06 2024-09-11T08:52:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038936+0000", 2024-09-11T08:52:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00028560199999999998 2024-09-11T08:52:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038943+0000", 2024-09-11T08:52:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.7309999999999996e-06 2024-09-11T08:52:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.929410+0000", 2024-09-11T08:52:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89046684799999998 2024-09-11T08:52:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.929473+0000", 2024-09-11T08:52:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3482e-05 2024-09-11T08:52:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:52:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.929657+0000", 2024-09-11T08:52:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00018351000000000001 2024-09-11T08:52:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.933869+0000", 2024-09-11T08:52:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0042121370000000003 2024-09-11T08:52:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945597+0000", 2024-09-11T08:52:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011727609999999999 2024-09-11T08:52:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945665+0000", 2024-09-11T08:52:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8650000000000002e-05 2024-09-11T08:52:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945708+0000", 2024-09-11T08:52:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2342000000000002e-05 2024-09-11T08:52:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945744+0000", 2024-09-11T08:52:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6569999999999997e-05 2024-09-11T08:52:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:161999 9.15 9:acecc340:::benchmark_data_smithi092_145083_object18601:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:52:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:59.040350+0000", 2024-09-11T08:52:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 110.27037027199999, 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90555341600000006, 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 161999 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040350+0000", 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040350+0000", 2024-09-11T08:52:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040349+0000", 2024-09-11T08:52:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:52:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040354+0000", 2024-09-11T08:52:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8829999999999997e-06 2024-09-11T08:52:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040366+0000", 2024-09-11T08:52:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.238e-05 2024-09-11T08:52:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040372+0000", 2024-09-11T08:52:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8139999999999997e-06 2024-09-11T08:52:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040516+0000", 2024-09-11T08:52:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000143313 2024-09-11T08:52:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040520+0000", 2024-09-11T08:52:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4089999999999996e-06 2024-09-11T08:52:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.930467+0000", 2024-09-11T08:52:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.88994658900000001 2024-09-11T08:52:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.930588+0000", 2024-09-11T08:52:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012184 2024-09-11T08:52:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:52:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.930801+0000", 2024-09-11T08:52:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000212124 2024-09-11T08:52:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.937385+0000", 2024-09-11T08:52:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0065848369999999996 2024-09-11T08:52:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945800+0000", 2024-09-11T08:52:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0084145890000000001 2024-09-11T08:52:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945832+0000", 2024-09-11T08:52:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2202000000000001e-05 2024-09-11T08:52:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945872+0000", 2024-09-11T08:52:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0256000000000002e-05 2024-09-11T08:52:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945903+0000", 2024-09-11T08:52:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1078000000000003e-05 2024-09-11T08:52:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162018 9.15 9:ac3e5f86:::benchmark_data_smithi092_145083_object18620:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:52:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:59.052029+0000", 2024-09-11T08:52:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 110.258691539, 2024-09-11T08:52:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89400514099999995, 2024-09-11T08:52:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162018 2024-09-11T08:52:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052029+0000", 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052029+0000", 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052027+0000", 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052034+0000", 2024-09-11T08:52:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3389999999999998e-06 2024-09-11T08:52:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052046+0000", 2024-09-11T08:52:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2517999999999999e-05 2024-09-11T08:52:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052056+0000", 2024-09-11T08:52:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.023e-05 2024-09-11T08:52:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052119+0000", 2024-09-11T08:52:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2984999999999994e-05 2024-09-11T08:52:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052127+0000", 2024-09-11T08:52:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.0330000000000008e-06 2024-09-11T08:52:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.931615+0000", 2024-09-11T08:52:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.87948807200000001 2024-09-11T08:52:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.931701+0000", 2024-09-11T08:52:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.5131999999999997e-05 2024-09-11T08:52:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:52:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.931887+0000", 2024-09-11T08:52:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00018613399999999999 2024-09-11T08:52:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.937393+0000", 2024-09-11T08:52:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0055058060000000002 2024-09-11T08:52:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945937+0000", 2024-09-11T08:52:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0085445950000000003 2024-09-11T08:52:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945969+0000", 2024-09-11T08:52:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2131000000000001e-05 2024-09-11T08:52:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946003+0000", 2024-09-11T08:52:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4224000000000003e-05 2024-09-11T08:52:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946034+0000", 2024-09-11T08:52:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0470000000000001e-05 2024-09-11T08:52:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162035 9.15 9:ac737bbe:::benchmark_data_smithi092_145083_object18637:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:52:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:59.065230+0000", 2024-09-11T08:52:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 110.245490547, 2024-09-11T08:52:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.88095609799999997, 2024-09-11T08:52:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162035 2024-09-11T08:52:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065230+0000", 2024-09-11T08:52:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065230+0000", 2024-09-11T08:52:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065229+0000", 2024-09-11T08:52:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065232+0000", 2024-09-11T08:52:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9340000000000002e-06 2024-09-11T08:52:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065239+0000", 2024-09-11T08:52:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.708e-06 2024-09-11T08:52:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065243+0000", 2024-09-11T08:52:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6430000000000001e-06 2024-09-11T08:52:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065366+0000", 2024-09-11T08:52:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000123511 2024-09-11T08:52:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065372+0000", 2024-09-11T08:52:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1909999999999997e-06 2024-09-11T08:52:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.932683+0000", 2024-09-11T08:52:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.86731089299999997 2024-09-11T08:52:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.932780+0000", 2024-09-11T08:52:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.6373999999999996e-05 2024-09-11T08:52:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:52:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.933044+0000", 2024-09-11T08:52:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00026401899999999999 2024-09-11T08:52:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.939749+0000", 2024-09-11T08:52:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.006705529 2024-09-11T08:52:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946068+0000", 2024-09-11T08:52:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0063190540000000002 2024-09-11T08:52:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946106+0000", 2024-09-11T08:52:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7839000000000001e-05 2024-09-11T08:52:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946144+0000", 2024-09-11T08:52:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8343000000000002e-05 2024-09-11T08:52:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946186+0000", 2024-09-11T08:52:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1564999999999999e-05 2024-09-11T08:52:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162098 9.15 9:ab1b879d:::benchmark_data_smithi092_145083_object18700:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:52:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:59.112673+0000", 2024-09-11T08:52:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 110.19804753699999, 2024-09-11T08:52:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.83383003600000005, 2024-09-11T08:52:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162098 2024-09-11T08:52:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112673+0000", 2024-09-11T08:52:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112673+0000", 2024-09-11T08:52:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112672+0000", 2024-09-11T08:52:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:52:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112676+0000", 2024-09-11T08:52:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4020000000000004e-06 2024-09-11T08:52:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112685+0000", 2024-09-11T08:52:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.9390000000000003e-06 2024-09-11T08:52:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112689+0000", 2024-09-11T08:52:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5210000000000004e-06 2024-09-11T08:52:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112725+0000", 2024-09-11T08:52:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5176000000000003e-05 2024-09-11T08:52:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112731+0000", 2024-09-11T08:52:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.088e-06 2024-09-11T08:52:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.933898+0000", 2024-09-11T08:52:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.82116687399999999 2024-09-11T08:52:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.933969+0000", 2024-09-11T08:52:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1109e-05 2024-09-11T08:52:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:52:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.934117+0000", 2024-09-11T08:52:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000148527 2024-09-11T08:52:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.939757+0000", 2024-09-11T08:52:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0056393290000000002 2024-09-11T08:52:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946218+0000", 2024-09-11T08:52:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0064610709999999997 2024-09-11T08:52:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946415+0000", 2024-09-11T08:52:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00019775899999999999 2024-09-11T08:52:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946453+0000", 2024-09-11T08:52:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8075999999999999e-05 2024-09-11T08:52:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946503+0000", 2024-09-11T08:52:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9345999999999998e-05 2024-09-11T08:52:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162378 9.17 9:eb84acd7:::benchmark_data_smithi092_145083_object18980:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-11T08:52:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:51:00.001027+0000", 2024-09-11T08:52:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 109.30969319499999, 2024-09-11T08:52:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.95130415599999996, 2024-09-11T08:52:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162378 2024-09-11T08:52:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.466 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001027+0000", 2024-09-11T08:52:49.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001027+0000", 2024-09-11T08:52:49.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001027+0000", 2024-09-11T08:52:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001029+0000", 2024-09-11T08:52:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0640000000000001e-06 2024-09-11T08:52:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001035+0000", 2024-09-11T08:52:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1e-06 2024-09-11T08:52:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001039+0000", 2024-09-11T08:52:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7749999999999999e-06 2024-09-11T08:52:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001115+0000", 2024-09-11T08:52:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6666000000000001e-05 2024-09-11T08:52:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001124+0000", 2024-09-11T08:52:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8990000000000007e-06 2024-09-11T08:52:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.933510+0000", 2024-09-11T08:52:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.93238578100000002 2024-09-11T08:52:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.933633+0000", 2024-09-11T08:52:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012321599999999999 2024-09-11T08:52:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:52:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.933908+0000", 2024-09-11T08:52:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00027423600000000002 2024-09-11T08:52:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.939240+0000", 2024-09-11T08:52:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0053320060000000002 2024-09-11T08:52:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952168+0000", 2024-09-11T08:52:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.012928439 2024-09-11T08:52:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952206+0000", 2024-09-11T08:52:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8479999999999997e-05 2024-09-11T08:52:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952254+0000", 2024-09-11T08:52:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7126999999999998e-05 2024-09-11T08:52:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952331+0000", 2024-09-11T08:52:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7713999999999995e-05 2024-09-11T08:52:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162412 9.17 9:ee197516:::benchmark_data_smithi092_145083_object19014:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-11T08:52:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:51:00.024372+0000", 2024-09-11T08:52:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 109.28634856799999, 2024-09-11T08:52:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.92838030800000004, 2024-09-11T08:52:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162412 2024-09-11T08:52:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024372+0000", 2024-09-11T08:52:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024372+0000", 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024370+0000", 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024377+0000", 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1179999999999997e-06 2024-09-11T08:52:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024388+0000", 2024-09-11T08:52:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1260999999999999e-05 2024-09-11T08:52:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024393+0000", 2024-09-11T08:52:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5600000000000001e-06 2024-09-11T08:52:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024653+0000", 2024-09-11T08:52:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00025985299999999999 2024-09-11T08:52:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024661+0000", 2024-09-11T08:52:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2760000000000003e-06 2024-09-11T08:52:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.935055+0000", 2024-09-11T08:52:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.91039429400000005 2024-09-11T08:52:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.935124+0000", 2024-09-11T08:52:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8872000000000001e-05 2024-09-11T08:52:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:52:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.935310+0000", 2024-09-11T08:52:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000185823 2024-09-11T08:52:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.939246+0000", 2024-09-11T08:52:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0039369030000000003 2024-09-11T08:52:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952414+0000", 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013167935 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952690+0000", 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00027535 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952724+0000", 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4767000000000001e-05 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952752+0000", 2024-09-11T08:52:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7591999999999999e-05 2024-09-11T08:52:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162425 9.17 9:ee96b337:::benchmark_data_smithi092_145083_object19027:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-11T08:52:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:51:00.030921+0000", 2024-09-11T08:52:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 109.279799551, 2024-09-11T08:52:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.921978466, 2024-09-11T08:52:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162425 2024-09-11T08:52:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030921+0000", 2024-09-11T08:52:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030921+0000", 2024-09-11T08:52:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030920+0000", 2024-09-11T08:52:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030923+0000", 2024-09-11T08:52:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.362e-06 2024-09-11T08:52:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030929+0000", 2024-09-11T08:52:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8769999999999998e-06 2024-09-11T08:52:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030932+0000", 2024-09-11T08:52:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.32e-06 2024-09-11T08:52:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030982+0000", 2024-09-11T08:52:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9997999999999998e-05 2024-09-11T08:52:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030989+0000", 2024-09-11T08:52:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8190000000000003e-06 2024-09-11T08:52:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.936138+0000", 2024-09-11T08:52:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90514870999999997 2024-09-11T08:52:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.936278+0000", 2024-09-11T08:52:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014009 2024-09-11T08:52:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:52:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.936509+0000", 2024-09-11T08:52:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00023174599999999999 2024-09-11T08:52:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.941634+0000", 2024-09-11T08:52:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0051242279999999998 2024-09-11T08:52:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952645+0000", 2024-09-11T08:52:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011011099 2024-09-11T08:52:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952820+0000", 2024-09-11T08:52:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00017567400000000001 2024-09-11T08:52:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952863+0000", 2024-09-11T08:52:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3123e-05 2024-09-11T08:52:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952899+0000", 2024-09-11T08:52:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.578e-05 2024-09-11T08:52:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162450 9.17 9:e99b70b0:::benchmark_data_smithi092_145083_object19052:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-11T08:52:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:51:00.045692+0000", 2024-09-11T08:52:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 109.26502793900001, 2024-09-11T08:52:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90749497800000001, 2024-09-11T08:52:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162450 2024-09-11T08:52:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045692+0000", 2024-09-11T08:52:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045692+0000", 2024-09-11T08:52:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045692+0000", 2024-09-11T08:52:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:52:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045696+0000", 2024-09-11T08:52:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0729999999999998e-06 2024-09-11T08:52:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045706+0000", 2024-09-11T08:52:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.9299999999999998e-06 2024-09-11T08:52:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045712+0000", 2024-09-11T08:52:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6829999999999996e-06 2024-09-11T08:52:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045759+0000", 2024-09-11T08:52:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7027999999999998e-05 2024-09-11T08:52:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045766+0000", 2024-09-11T08:52:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9979999999999997e-06 2024-09-11T08:52:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.937223+0000", 2024-09-11T08:52:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89145709200000001 2024-09-11T08:52:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.937283+0000", 2024-09-11T08:52:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0676e-05 2024-09-11T08:52:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:52:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.937427+0000", 2024-09-11T08:52:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000143642 2024-09-11T08:52:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.941640+0000", 2024-09-11T08:52:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0042131299999999998 2024-09-11T08:52:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952942+0000", 2024-09-11T08:52:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011301774000000001 2024-09-11T08:52:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953099+0000", 2024-09-11T08:52:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015739500000000001 2024-09-11T08:52:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953141+0000", 2024-09-11T08:52:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1414e-05 2024-09-11T08:52:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953187+0000", 2024-09-11T08:52:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6687999999999998e-05 2024-09-11T08:52:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162513 9.17 9:e993ae52:::benchmark_data_smithi092_145083_object19115:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-11T08:52:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:51:00.087447+0000", 2024-09-11T08:52:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 109.223273414, 2024-09-11T08:52:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.86587709199999996, 2024-09-11T08:52:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:52:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:52:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:52:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:52:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:52:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162513 2024-09-11T08:52:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:52:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:52:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087447+0000", 2024-09-11T08:52:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:52:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087447+0000", 2024-09-11T08:52:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:52:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:52:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087446+0000", 2024-09-11T08:52:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:52:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:52:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087452+0000", 2024-09-11T08:52:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6439999999999997e-06 2024-09-11T08:52:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:52:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087466+0000", 2024-09-11T08:52:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3966e-05 2024-09-11T08:52:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:52:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087471+0000", 2024-09-11T08:52:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6849999999999998e-06 2024-09-11T08:52:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087515+0000", 2024-09-11T08:52:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3890000000000002e-05 2024-09-11T08:52:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:52:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087522+0000", 2024-09-11T08:52:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5899999999999996e-06 2024-09-11T08:52:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:52:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.938122+0000", 2024-09-11T08:52:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.85060005800000005 2024-09-11T08:52:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:52:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.938185+0000", 2024-09-11T08:52:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2726999999999998e-05 2024-09-11T08:52:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:52:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.938349+0000", 2024-09-11T08:52:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000164535 2024-09-11T08:52:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:52:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.944372+0000", 2024-09-11T08:52:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0060229369999999999 2024-09-11T08:52:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952983+0000", 2024-09-11T08:52:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0086107600000000003 2024-09-11T08:52:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:52:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953253+0000", 2024-09-11T08:52:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00027028600000000001 2024-09-11T08:52:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:52:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953290+0000", 2024-09-11T08:52:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6860000000000003e-05 2024-09-11T08:52:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:52:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:52:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:52:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953324+0000", 2024-09-11T08:52:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4007999999999997e-05 2024-09-11T08:52:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.501 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:52:49.501 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:52:49.501 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:52:49.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:49.502 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:49.502 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:49.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:49.624 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:49.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:49.625 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:49 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4044800497' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 0.1711928788699405}]': finished 2024-09-11T08:52:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:49 smithi145 ceph-mon[33384]: osdmap e290: 8 total, 8 up, 8 in 2024-09-11T08:52:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:49 smithi145 ceph-mon[33384]: osdmap e291: 8 total, 8 up, 8 in 2024-09-11T08:52:49.726 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:49.747 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:49.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:49.748 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:49.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:49.870 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:49.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:49.871 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:49.948 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:49.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:49.995 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:49.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:49.996 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:50.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:49 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4044800497' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 0.1711928788699405}]': finished 2024-09-11T08:52:50.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:49 smithi092 ceph-mon[29647]: osdmap e290: 8 total, 8 up, 8 in 2024-09-11T08:52:50.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:49 smithi092 ceph-mon[29647]: osdmap e291: 8 total, 8 up, 8 in 2024-09-11T08:52:50.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:49 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4044800497' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 0.1711928788699405}]': finished 2024-09-11T08:52:50.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:49 smithi092 ceph-mon[41114]: osdmap e290: 8 total, 8 up, 8 in 2024-09-11T08:52:50.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:49 smithi092 ceph-mon[41114]: osdmap e291: 8 total, 8 up, 8 in 2024-09-11T08:52:50.097 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:50.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:50.119 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:50.119 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:50.220 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:50.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:50.242 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:50.243 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:50.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:50.365 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:50.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:50.366 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:50.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-11T08:52:50.467 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:50.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:50.493 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:50.494 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:50.581 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:52:50.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:50.617 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:50.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:50.618 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:50.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:50.723 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:50 smithi092 ceph-mon[41114]: pgmap v631: 142 pgs: 142 active+clean; 477 KiB data, 4.7 GiB used, 711 GiB / 715 GiB avail 2024-09-11T08:52:50.723 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:50 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2620501988' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:52:50.724 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:50 smithi092 ceph-mon[29647]: pgmap v631: 142 pgs: 142 active+clean; 477 KiB data, 4.7 GiB used, 711 GiB / 715 GiB avail 2024-09-11T08:52:50.724 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:50 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2620501988' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:52:50.742 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:50.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:50.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:50.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:50.866 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:50.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:50.867 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:50.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:50 smithi145 ceph-mon[33384]: pgmap v631: 142 pgs: 142 active+clean; 477 KiB data, 4.7 GiB used, 711 GiB / 715 GiB avail 2024-09-11T08:52:50.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:50 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2620501988' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:52:50.967 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:50.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:50.990 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:50.991 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:51.091 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:51.114 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:51.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:51.115 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:51.216 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:51.238 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:51.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:51.239 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:51.339 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:51.361 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:51.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:51.363 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:51.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:51.485 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:51.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:51.487 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:51.587 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:51.609 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:51.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:51.610 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:51.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:51 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:52:51.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:51 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2620501988' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:52:51.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:51 smithi092 ceph-mon[29647]: osdmap e292: 8 total, 8 up, 8 in 2024-09-11T08:52:51.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:51 smithi092 ceph-mon[29647]: 6.19 scrub starts 2024-09-11T08:52:51.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:51 smithi092 ceph-mon[29647]: 6.19 scrub ok 2024-09-11T08:52:51.700 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:52:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:52:51] "GET /metrics HTTP/1.1" 200 224527 "" "Prometheus/2.43.0" 2024-09-11T08:52:51.700 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:51 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:52:51.700 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:51 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2620501988' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:52:51.700 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:51 smithi092 ceph-mon[41114]: osdmap e292: 8 total, 8 up, 8 in 2024-09-11T08:52:51.700 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:51 smithi092 ceph-mon[41114]: 6.19 scrub starts 2024-09-11T08:52:51.700 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:51 smithi092 ceph-mon[41114]: 6.19 scrub ok 2024-09-11T08:52:51.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:51.733 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:51.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:51.734 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:51.835 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:51.857 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:51.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:51.858 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:51 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:52:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:51 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2620501988' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:52:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:51 smithi145 ceph-mon[33384]: osdmap e292: 8 total, 8 up, 8 in 2024-09-11T08:52:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:51 smithi145 ceph-mon[33384]: 6.19 scrub starts 2024-09-11T08:52:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:51 smithi145 ceph-mon[33384]: 6.19 scrub ok 2024-09-11T08:52:51.960 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:51.981 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:51.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:51.982 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:52.083 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:52.105 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:52.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:52.106 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:52.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:52.228 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:52.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:52.229 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:52.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:52.351 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:52.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:52.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:52.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:52.475 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:52.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:52.476 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:52.577 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:52.602 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:52.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:52.603 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:52.704 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:52.730 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:52.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:52.732 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:52.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:52 smithi092 ceph-mon[29647]: pgmap v634: 142 pgs: 142 active+clean; 477 KiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 884 B/s rd, 0 op/s 2024-09-11T08:52:52.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:52 smithi092 ceph-mon[29647]: 6.3 scrub starts 2024-09-11T08:52:52.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:52 smithi092 ceph-mon[29647]: 6.3 scrub ok 2024-09-11T08:52:52.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:52 smithi092 ceph-mon[29647]: 6.12 scrub starts 2024-09-11T08:52:52.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:52 smithi092 ceph-mon[29647]: 6.12 scrub ok 2024-09-11T08:52:52.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:52 smithi092 ceph-mon[41114]: pgmap v634: 142 pgs: 142 active+clean; 477 KiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 884 B/s rd, 0 op/s 2024-09-11T08:52:52.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:52 smithi092 ceph-mon[41114]: 6.3 scrub starts 2024-09-11T08:52:52.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:52 smithi092 ceph-mon[41114]: 6.3 scrub ok 2024-09-11T08:52:52.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:52 smithi092 ceph-mon[41114]: 6.12 scrub starts 2024-09-11T08:52:52.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:52 smithi092 ceph-mon[41114]: 6.12 scrub ok 2024-09-11T08:52:52.833 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:52.858 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:52.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:52.859 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:52.912 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:52:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:52:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:52:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:52 smithi145 ceph-mon[33384]: pgmap v634: 142 pgs: 142 active+clean; 477 KiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 884 B/s rd, 0 op/s 2024-09-11T08:52:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:52 smithi145 ceph-mon[33384]: 6.3 scrub starts 2024-09-11T08:52:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:52 smithi145 ceph-mon[33384]: 6.3 scrub ok 2024-09-11T08:52:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:52 smithi145 ceph-mon[33384]: 6.12 scrub starts 2024-09-11T08:52:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:52 smithi145 ceph-mon[33384]: 6.12 scrub ok 2024-09-11T08:52:52.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:52.982 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:52.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:52.983 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:53.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:53.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:53.106 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:53.106 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:53.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:53.229 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:53.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:53.230 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:53.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:53.352 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:53.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:53.354 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:53.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:53.476 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:53.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:53.477 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:53.578 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:53.591 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:53.603 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:53.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:53.604 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:53.704 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:53.713 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:53 smithi092 ceph-mon[29647]: 5.1a scrub starts 2024-09-11T08:52:53.713 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:53 smithi092 ceph-mon[29647]: 5.1a scrub ok 2024-09-11T08:52:53.713 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:53 smithi092 ceph-mon[29647]: 4.6 scrub starts 2024-09-11T08:52:53.713 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:53 smithi092 ceph-mon[29647]: 4.6 scrub ok 2024-09-11T08:52:53.713 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:53 smithi092 ceph-mon[29647]: 3.0 scrub starts 2024-09-11T08:52:53.713 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:53 smithi092 ceph-mon[29647]: 3.0 scrub ok 2024-09-11T08:52:53.714 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:53 smithi092 ceph-mon[41114]: 5.1a scrub starts 2024-09-11T08:52:53.714 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:53 smithi092 ceph-mon[41114]: 5.1a scrub ok 2024-09-11T08:52:53.714 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:53 smithi092 ceph-mon[41114]: 4.6 scrub starts 2024-09-11T08:52:53.714 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:53 smithi092 ceph-mon[41114]: 4.6 scrub ok 2024-09-11T08:52:53.714 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:53 smithi092 ceph-mon[41114]: 3.0 scrub starts 2024-09-11T08:52:53.714 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:53 smithi092 ceph-mon[41114]: 3.0 scrub ok 2024-09-11T08:52:53.730 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:53.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:53.731 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:53.832 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:53.857 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:53.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:53.858 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:53 smithi145 ceph-mon[33384]: 5.1a scrub starts 2024-09-11T08:52:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:53 smithi145 ceph-mon[33384]: 5.1a scrub ok 2024-09-11T08:52:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:53 smithi145 ceph-mon[33384]: 4.6 scrub starts 2024-09-11T08:52:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:53 smithi145 ceph-mon[33384]: 4.6 scrub ok 2024-09-11T08:52:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:53 smithi145 ceph-mon[33384]: 3.0 scrub starts 2024-09-11T08:52:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:53 smithi145 ceph-mon[33384]: 3.0 scrub ok 2024-09-11T08:52:53.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:53.980 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:53.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:53.981 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:54.082 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:54.104 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:54.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:54.105 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:54.206 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:54.231 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:54.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:54.232 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:54.333 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:54.358 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:54.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:54.359 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:54.460 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:54.482 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:54.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:54.483 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:54.558 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:52:54.558 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:52:54.563 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:52:54.563 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:52:54.578 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:52:54.578 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:52:54.584 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:54.595 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'unique_pool_3' created 2024-09-11T08:52:54.596 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:52:54.596 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:52:54.613 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:54.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:54.614 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:54.616 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:52:54.616 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:52:54.632 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:52:54.633 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:52:54.648 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:52:54.648 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:52:54.664 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:52:54.664 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:52:54.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:54.736 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:54.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:54.737 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:54.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[29647]: pgmap v635: 142 pgs: 142 active+clean; 477 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:52:54.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[29647]: 4.14 scrub starts 2024-09-11T08:52:54.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[29647]: 4.14 scrub ok 2024-09-11T08:52:54.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[29647]: 5.1e scrub starts 2024-09-11T08:52:54.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[29647]: 5.1e scrub ok 2024-09-11T08:52:54.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3486862740' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-11T08:52:54.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-11T08:52:54.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[29647]: 5.4 scrub starts 2024-09-11T08:52:54.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[29647]: 5.4 scrub ok 2024-09-11T08:52:54.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3786115587' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:52:54.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[41114]: pgmap v635: 142 pgs: 142 active+clean; 477 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:52:54.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[41114]: 4.14 scrub starts 2024-09-11T08:52:54.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[41114]: 4.14 scrub ok 2024-09-11T08:52:54.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[41114]: 5.1e scrub starts 2024-09-11T08:52:54.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[41114]: 5.1e scrub ok 2024-09-11T08:52:54.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3486862740' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-11T08:52:54.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-11T08:52:54.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[41114]: 5.4 scrub starts 2024-09-11T08:52:54.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[41114]: 5.4 scrub ok 2024-09-11T08:52:54.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:54 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3786115587' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:52:54.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:54.859 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:54.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:54.860 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:54.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:52:54.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:54 smithi145 ceph-mon[33384]: pgmap v635: 142 pgs: 142 active+clean; 477 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:52:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:54 smithi145 ceph-mon[33384]: 4.14 scrub starts 2024-09-11T08:52:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:54 smithi145 ceph-mon[33384]: 4.14 scrub ok 2024-09-11T08:52:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:54 smithi145 ceph-mon[33384]: 5.1e scrub starts 2024-09-11T08:52:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:54 smithi145 ceph-mon[33384]: 5.1e scrub ok 2024-09-11T08:52:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:54 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3486862740' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-11T08:52:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:54 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-11T08:52:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:54 smithi145 ceph-mon[33384]: 5.4 scrub starts 2024-09-11T08:52:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:54 smithi145 ceph-mon[33384]: 5.4 scrub ok 2024-09-11T08:52:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:54 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3786115587' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:52:54.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:54.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:54.985 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:54.985 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:55.086 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:55.107 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:55.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:55.108 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:55.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:55.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:55.233 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:55.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:55.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:55.356 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:55.356 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:55.457 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:55.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:55.480 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:55.481 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:55.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:55.606 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:55.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:55.607 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:55.708 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:55.729 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:55.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:55.730 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:55.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[29647]: 5.13 scrub starts 2024-09-11T08:52:55.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[29647]: 5.13 scrub ok 2024-09-11T08:52:55.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-11T08:52:55.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[29647]: osdmap e293: 8 total, 8 up, 8 in 2024-09-11T08:52:55.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[29647]: 6.1d scrub starts 2024-09-11T08:52:55.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[29647]: 6.1d scrub ok 2024-09-11T08:52:55.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[29647]: 3.f scrub starts 2024-09-11T08:52:55.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[29647]: 3.f scrub ok 2024-09-11T08:52:55.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[29647]: 6.15 scrub starts 2024-09-11T08:52:55.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[29647]: 6.15 scrub ok 2024-09-11T08:52:55.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[41114]: 5.13 scrub starts 2024-09-11T08:52:55.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[41114]: 5.13 scrub ok 2024-09-11T08:52:55.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-11T08:52:55.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[41114]: osdmap e293: 8 total, 8 up, 8 in 2024-09-11T08:52:55.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[41114]: 6.1d scrub starts 2024-09-11T08:52:55.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[41114]: 6.1d scrub ok 2024-09-11T08:52:55.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[41114]: 3.f scrub starts 2024-09-11T08:52:55.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[41114]: 3.f scrub ok 2024-09-11T08:52:55.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[41114]: 6.15 scrub starts 2024-09-11T08:52:55.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:55 smithi092 ceph-mon[41114]: 6.15 scrub ok 2024-09-11T08:52:55.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:55.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:55.853 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:55.853 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:55 smithi145 ceph-mon[33384]: 5.13 scrub starts 2024-09-11T08:52:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:55 smithi145 ceph-mon[33384]: 5.13 scrub ok 2024-09-11T08:52:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:55 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-11T08:52:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:55 smithi145 ceph-mon[33384]: osdmap e293: 8 total, 8 up, 8 in 2024-09-11T08:52:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:55 smithi145 ceph-mon[33384]: 6.1d scrub starts 2024-09-11T08:52:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:55 smithi145 ceph-mon[33384]: 6.1d scrub ok 2024-09-11T08:52:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:55 smithi145 ceph-mon[33384]: 3.f scrub starts 2024-09-11T08:52:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:55 smithi145 ceph-mon[33384]: 3.f scrub ok 2024-09-11T08:52:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:55 smithi145 ceph-mon[33384]: 6.15 scrub starts 2024-09-11T08:52:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:55 smithi145 ceph-mon[33384]: 6.15 scrub ok 2024-09-11T08:52:55.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:55.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:55.977 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:56.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:56.099 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:56.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:56.101 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:56.151 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:52:56.151 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:52:56.152 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-11T08:52:56.152 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T08:52:56.152 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T08:52:56.152 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-09-11T08:52:56.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:56.223 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:56.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:56.224 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:56.325 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:56.347 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:56.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:56.348 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:56.415 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:52:56.450 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:56.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:56.473 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:56.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:56.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2024-09-11T08:52:56.542 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:56.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:56.596 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:56.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:56.597 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:56.698 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:56.721 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:56.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:56.722 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:56.773 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:56 smithi092 ceph-mon[29647]: pgmap v637: 158 pgs: 16 unknown, 142 active+clean; 477 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:52:56.773 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:56 smithi092 ceph-mon[29647]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:52:56.773 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:56 smithi092 ceph-mon[29647]: osdmap e294: 8 total, 8 up, 8 in 2024-09-11T08:52:56.773 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:56 smithi092 ceph-mon[29647]: 3.5 scrub starts 2024-09-11T08:52:56.774 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:56 smithi092 ceph-mon[29647]: 3.5 scrub ok 2024-09-11T08:52:56.774 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:56 smithi092 ceph-mon[41114]: pgmap v637: 158 pgs: 16 unknown, 142 active+clean; 477 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:52:56.774 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:56 smithi092 ceph-mon[41114]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:52:56.774 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:56 smithi092 ceph-mon[41114]: osdmap e294: 8 total, 8 up, 8 in 2024-09-11T08:52:56.774 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:56 smithi092 ceph-mon[41114]: 3.5 scrub starts 2024-09-11T08:52:56.774 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:56 smithi092 ceph-mon[41114]: 3.5 scrub ok 2024-09-11T08:52:56.823 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:56.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:56.845 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:56.845 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:56.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:56 smithi145 ceph-mon[33384]: pgmap v637: 158 pgs: 16 unknown, 142 active+clean; 477 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:52:56.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:56 smithi145 ceph-mon[33384]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:52:56.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:56 smithi145 ceph-mon[33384]: osdmap e294: 8 total, 8 up, 8 in 2024-09-11T08:52:56.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:56 smithi145 ceph-mon[33384]: 3.5 scrub starts 2024-09-11T08:52:56.861 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:56 smithi145 ceph-mon[33384]: 3.5 scrub ok 2024-09-11T08:52:56.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:56.967 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:56.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:56.968 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:57.069 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:57.091 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:57.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:57.092 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:57.135 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:52:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:52:57.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:57.214 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:57.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:57.215 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:57.252 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:52:57.252 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:52:57.317 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:57.338 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:57.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:57.339 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:57.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:57.462 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:57.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:57.463 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:57.564 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:57.566 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:52:57.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:57.587 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:57.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:57.688 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:57.711 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:57.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:57.712 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:57.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:57 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:52:57.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:57 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:52:57.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:57 smithi145 ceph-mon[33384]: osdmap e295: 8 total, 8 up, 8 in 2024-09-11T08:52:57.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:57 smithi145 ceph-mon[33384]: 5.f scrub starts 2024-09-11T08:52:57.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:57 smithi145 ceph-mon[33384]: 5.f scrub ok 2024-09-11T08:52:57.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:57 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T08:52:57.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:57 smithi145 ceph-mon[33384]: 4.1c scrub ok 2024-09-11T08:52:57.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:57 smithi145 ceph-mon[33384]: 5.e scrub starts 2024-09-11T08:52:57.757 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:57 smithi145 ceph-mon[33384]: 5.e scrub ok 2024-09-11T08:52:57.813 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:57.834 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:57.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:57.835 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:57.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:57.958 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:57.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:57.959 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:58.060 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:58.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:52:58.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:52:58.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[29647]: osdmap e295: 8 total, 8 up, 8 in 2024-09-11T08:52:58.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[29647]: 5.f scrub starts 2024-09-11T08:52:58.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[29647]: 5.f scrub ok 2024-09-11T08:52:58.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T08:52:58.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[29647]: 4.1c scrub ok 2024-09-11T08:52:58.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[29647]: 5.e scrub starts 2024-09-11T08:52:58.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[29647]: 5.e scrub ok 2024-09-11T08:52:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:52:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:52:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[41114]: osdmap e295: 8 total, 8 up, 8 in 2024-09-11T08:52:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[41114]: 5.f scrub starts 2024-09-11T08:52:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[41114]: 5.f scrub ok 2024-09-11T08:52:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T08:52:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[41114]: 4.1c scrub ok 2024-09-11T08:52:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[41114]: 5.e scrub starts 2024-09-11T08:52:58.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:57 smithi092 ceph-mon[41114]: 5.e scrub ok 2024-09-11T08:52:58.082 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:58.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:58.083 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:58.163 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:52:58.184 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:58.206 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:58.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:58.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:58.293 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:52:58.293 INFO:teuthology.orchestra.run.smithi145.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T08:52:58.293 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:52:58.308 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:52:58.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:58.331 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:58.331 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:52:58.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:58.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:58.462 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:58.463 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:58.563 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:58.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:58.587 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:58.688 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:58.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:58.710 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:58.710 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:58.811 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:58.834 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:58.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:58.835 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:58.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:58 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:58.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:58 smithi092 ceph-mon[29647]: pgmap v640: 158 pgs: 6 creating+activating, 152 active+clean; 477 KiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:52:58.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:58 smithi092 ceph-mon[29647]: 6.d scrub starts 2024-09-11T08:52:58.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:58 smithi092 ceph-mon[29647]: 6.d scrub ok 2024-09-11T08:52:58.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:58 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:58.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:58 smithi092 ceph-mon[41114]: pgmap v640: 158 pgs: 6 creating+activating, 152 active+clean; 477 KiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:52:58.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:58 smithi092 ceph-mon[41114]: 6.d scrub starts 2024-09-11T08:52:58.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:58 smithi092 ceph-mon[41114]: 6.d scrub ok 2024-09-11T08:52:58.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:58 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:52:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:58 smithi145 ceph-mon[33384]: pgmap v640: 158 pgs: 6 creating+activating, 152 active+clean; 477 KiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:52:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:58 smithi145 ceph-mon[33384]: 6.d scrub starts 2024-09-11T08:52:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:58 smithi145 ceph-mon[33384]: 6.d scrub ok 2024-09-11T08:52:58.936 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:52:58.958 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:58.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:58.959 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:52:59.060 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:52:59.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:59.083 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:59.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:52:59.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:59.205 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:59.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:59.206 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:59.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:52:59.329 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:59.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:59.331 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:52:59.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-11T08:52:59.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:52:59.454 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:59.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:59.455 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:52:59.556 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:52:59.578 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:59.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:59.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:52:59.680 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:59.702 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:59.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:59.703 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:52:59.804 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:52:59.826 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:52:59.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:59.827 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:52:59.887 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:59 smithi092 ceph-mon[29647]: 4.17 scrub starts 2024-09-11T08:52:59.887 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:52:59 smithi092 ceph-mon[29647]: 4.17 scrub ok 2024-09-11T08:52:59.888 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:59 smithi092 ceph-mon[41114]: 4.17 scrub starts 2024-09-11T08:52:59.888 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:52:59 smithi092 ceph-mon[41114]: 4.17 scrub ok 2024-09-11T08:52:59.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:59 smithi145 ceph-mon[33384]: 4.17 scrub starts 2024-09-11T08:52:59.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:52:59 smithi145 ceph-mon[33384]: 4.17 scrub ok 2024-09-11T08:52:59.928 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:52:59.949 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:52:59.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:52:59.950 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:00.051 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:00.073 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:00.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:00.074 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:00.174 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:00.196 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:00.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:00.197 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:00.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:00.320 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:00.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:00.321 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:00.422 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:00.442 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:00.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:00.443 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:00.544 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:00.565 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:00.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:00.567 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:00.667 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:00.689 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:00.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:00.691 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:00.792 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:00.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:00.813 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:00.814 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[29647]: pgmap v641: 158 pgs: 6 creating+activating, 152 active+clean; 477 KiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:53:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[29647]: 2.0 scrub starts 2024-09-11T08:53:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[29647]: 2.0 scrub ok 2024-09-11T08:53:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[29647]: 6.8 scrub starts 2024-09-11T08:53:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[29647]: 6.8 scrub ok 2024-09-11T08:53:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[29647]: 5.1 scrub starts 2024-09-11T08:53:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[29647]: 5.1 scrub ok 2024-09-11T08:53:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:53:00.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:53:00.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[41114]: pgmap v641: 158 pgs: 6 creating+activating, 152 active+clean; 477 KiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:53:00.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[41114]: 2.0 scrub starts 2024-09-11T08:53:00.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[41114]: 2.0 scrub ok 2024-09-11T08:53:00.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[41114]: 6.8 scrub starts 2024-09-11T08:53:00.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[41114]: 6.8 scrub ok 2024-09-11T08:53:00.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[41114]: 5.1 scrub starts 2024-09-11T08:53:00.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[41114]: 5.1 scrub ok 2024-09-11T08:53:00.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:53:00.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:53:00.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:00 smithi145 ceph-mon[33384]: pgmap v641: 158 pgs: 6 creating+activating, 152 active+clean; 477 KiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:53:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:00 smithi145 ceph-mon[33384]: 2.0 scrub starts 2024-09-11T08:53:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:00 smithi145 ceph-mon[33384]: 2.0 scrub ok 2024-09-11T08:53:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:00 smithi145 ceph-mon[33384]: 6.8 scrub starts 2024-09-11T08:53:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:00 smithi145 ceph-mon[33384]: 6.8 scrub ok 2024-09-11T08:53:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:00 smithi145 ceph-mon[33384]: 5.1 scrub starts 2024-09-11T08:53:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:00 smithi145 ceph-mon[33384]: 5.1 scrub ok 2024-09-11T08:53:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:00 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:53:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:53:00.916 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:00.937 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:00.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:00.938 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:00.941 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:01.039 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:01.061 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:01.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:01.062 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:01.163 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:01.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:01.185 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:01.185 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:01.286 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:01.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:01.309 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:01.309 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:01.410 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:01.434 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:01.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:01.435 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:01.536 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:01.558 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:01.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:01.559 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:01.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:53:01] "GET /metrics HTTP/1.1" 200 224534 "" "Prometheus/2.43.0" 2024-09-11T08:53:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:enabled application 'rados' on pool 'unique_pool_3' 2024-09-11T08:53:01.659 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:01.682 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:01.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:01.683 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:01.784 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:01.805 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:01.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:01.806 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:01.875 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:53:01.875 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:53:01.875 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[29647]: 5.10 scrub starts 2024-09-11T08:53:01.875 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[29647]: 5.10 scrub ok 2024-09-11T08:53:01.875 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[29647]: osdmap e296: 8 total, 8 up, 8 in 2024-09-11T08:53:01.875 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[29647]: 5.a scrub starts 2024-09-11T08:53:01.875 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[29647]: 5.a scrub ok 2024-09-11T08:53:01.875 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[29647]: 1.5 scrub starts 2024-09-11T08:53:01.875 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[29647]: 1.5 scrub ok 2024-09-11T08:53:01.875 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[29647]: 3.13 scrub starts 2024-09-11T08:53:01.876 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[29647]: 3.13 scrub ok 2024-09-11T08:53:01.876 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/552053669' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:53:01.876 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:53:01.876 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:53:01.876 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[41114]: 5.10 scrub starts 2024-09-11T08:53:01.876 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[41114]: 5.10 scrub ok 2024-09-11T08:53:01.876 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[41114]: osdmap e296: 8 total, 8 up, 8 in 2024-09-11T08:53:01.876 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[41114]: 5.a scrub starts 2024-09-11T08:53:01.876 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[41114]: 5.a scrub ok 2024-09-11T08:53:01.876 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[41114]: 1.5 scrub starts 2024-09-11T08:53:01.876 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[41114]: 1.5 scrub ok 2024-09-11T08:53:01.876 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[41114]: 3.13 scrub starts 2024-09-11T08:53:01.877 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[41114]: 3.13 scrub ok 2024-09-11T08:53:01.877 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:01 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/552053669' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:53:01.907 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:01.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:01 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:53:01.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:01 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:53:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:01 smithi145 ceph-mon[33384]: 5.10 scrub starts 2024-09-11T08:53:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:01 smithi145 ceph-mon[33384]: 5.10 scrub ok 2024-09-11T08:53:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:01 smithi145 ceph-mon[33384]: osdmap e296: 8 total, 8 up, 8 in 2024-09-11T08:53:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:01 smithi145 ceph-mon[33384]: 5.a scrub starts 2024-09-11T08:53:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:01 smithi145 ceph-mon[33384]: 5.a scrub ok 2024-09-11T08:53:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:01 smithi145 ceph-mon[33384]: 1.5 scrub starts 2024-09-11T08:53:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:01 smithi145 ceph-mon[33384]: 1.5 scrub ok 2024-09-11T08:53:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:01 smithi145 ceph-mon[33384]: 3.13 scrub starts 2024-09-11T08:53:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:01 smithi145 ceph-mon[33384]: 3.13 scrub ok 2024-09-11T08:53:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:01 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/552053669' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:53:01.928 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:01.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:01.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:02.030 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:02.051 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:02.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:02.052 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:02.109 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:02.152 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:02.174 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:02.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:02.175 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:02.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:02.301 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:02.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:02.302 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:02.403 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:02.425 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:02.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:02.426 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:02.527 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:02.548 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:02.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:02.549 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:02.650 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:02.672 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:02.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:02.673 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:02.774 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:02.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:02.797 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:02.797 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:02.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:02.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:53:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:53:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:53:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:02 smithi145 ceph-mon[33384]: pgmap v643: 158 pgs: 158 active+clean; 425 KiB data, 4.8 GiB used, 710 GiB / 715 GiB avail 2024-09-11T08:53:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:02 smithi145 ceph-mon[33384]: 5.3 scrub starts 2024-09-11T08:53:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:02 smithi145 ceph-mon[33384]: 5.3 scrub ok 2024-09-11T08:53:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:02 smithi145 ceph-mon[33384]: 6.16 scrub starts 2024-09-11T08:53:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:02 smithi145 ceph-mon[33384]: 6.16 scrub ok 2024-09-11T08:53:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:02 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:53:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:02 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:53:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:02 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/552053669' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:53:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:02 smithi145 ceph-mon[33384]: osdmap e297: 8 total, 8 up, 8 in 2024-09-11T08:53:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:02 smithi145 ceph-mon[33384]: 1.3 scrub starts 2024-09-11T08:53:02.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:02 smithi145 ceph-mon[33384]: 1.3 scrub ok 2024-09-11T08:53:02.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:02 smithi145 ceph-mon[33384]: 5.b scrub starts 2024-09-11T08:53:02.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:02 smithi145 ceph-mon[33384]: 5.b scrub ok 2024-09-11T08:53:02.922 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:02.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:02.923 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:03.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:03.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:03.046 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:03.046 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:03.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[41114]: pgmap v643: 158 pgs: 158 active+clean; 425 KiB data, 4.8 GiB used, 710 GiB / 715 GiB avail 2024-09-11T08:53:03.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[41114]: 5.3 scrub starts 2024-09-11T08:53:03.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[41114]: 5.3 scrub ok 2024-09-11T08:53:03.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[41114]: 6.16 scrub starts 2024-09-11T08:53:03.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[41114]: 6.16 scrub ok 2024-09-11T08:53:03.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:53:03.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:53:03.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/552053669' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:53:03.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[41114]: osdmap e297: 8 total, 8 up, 8 in 2024-09-11T08:53:03.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[41114]: 1.3 scrub starts 2024-09-11T08:53:03.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[41114]: 1.3 scrub ok 2024-09-11T08:53:03.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[41114]: 5.b scrub starts 2024-09-11T08:53:03.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[41114]: 5.b scrub ok 2024-09-11T08:53:03.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[29647]: pgmap v643: 158 pgs: 158 active+clean; 425 KiB data, 4.8 GiB used, 710 GiB / 715 GiB avail 2024-09-11T08:53:03.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[29647]: 5.3 scrub starts 2024-09-11T08:53:03.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[29647]: 5.3 scrub ok 2024-09-11T08:53:03.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[29647]: 6.16 scrub starts 2024-09-11T08:53:03.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[29647]: 6.16 scrub ok 2024-09-11T08:53:03.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:53:03.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:53:03.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/552053669' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:53:03.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[29647]: osdmap e297: 8 total, 8 up, 8 in 2024-09-11T08:53:03.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[29647]: 1.3 scrub starts 2024-09-11T08:53:03.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[29647]: 1.3 scrub ok 2024-09-11T08:53:03.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[29647]: 5.b scrub starts 2024-09-11T08:53:03.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:02 smithi092 ceph-mon[29647]: 5.b scrub ok 2024-09-11T08:53:03.147 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:03.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:03.169 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:03.170 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:03.270 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:03.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:03.293 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:03.294 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:03.394 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:03.416 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:03.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:03.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:03.518 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:03.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:03.541 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:03.541 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:03.639 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:53:03.642 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:03.663 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:03.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:03.664 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:03.765 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:03.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:03.790 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:03.790 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:03.891 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:03.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:03 smithi145 ceph-mon[33384]: 3.8 scrub starts 2024-09-11T08:53:03.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:03 smithi145 ceph-mon[33384]: 3.8 scrub ok 2024-09-11T08:53:03.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:03 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1703595036' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:53:03.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:03 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:53:03.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:03.913 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:03.913 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:03.948 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:53:03.948 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:53:03.981 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:03 smithi092 ceph-mon[29647]: 3.8 scrub starts 2024-09-11T08:53:03.982 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:03 smithi092 ceph-mon[29647]: 3.8 scrub ok 2024-09-11T08:53:03.982 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:03 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1703595036' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:53:03.982 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:03 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:53:03.982 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:03 smithi092 ceph-mon[41114]: 3.8 scrub starts 2024-09-11T08:53:03.982 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:03 smithi092 ceph-mon[41114]: 3.8 scrub ok 2024-09-11T08:53:03.982 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:03 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1703595036' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:53:03.982 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:03 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:53:04.014 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:04.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:04.036 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:04.036 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:04.137 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:04.158 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:04.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:04.159 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:04.260 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:04.281 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:04.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:04.282 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:04.383 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:04.408 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:04.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:04.409 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:04.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:04.536 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:04.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:04.537 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:04.638 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:04.659 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:04.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:04.660 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:04.712 DEBUG:teuthology.orchestra.run.smithi092:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_3 bench 90 write' 2024-09-11T08:53:04.712 INFO:tasks.thrashosds.thrasher:Shrinking pool .mgr 2024-09-11T08:53:04.713 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool .mgr by 10 min 8 2024-09-11T08:53:04.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:53:04.715 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-11T08:53:04.715 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-11T08:53:04.761 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:04.783 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:04.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:04.784 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:04.878 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:hints = 1 2024-09-11T08:53:04.878 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-11T08:53:04.878 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Object prefix: benchmark_data_smithi092_173109 2024-09-11T08:53:04.878 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:53:04.878 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 0 0 0 0 0 0 - 0 2024-09-11T08:53:04.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:04 smithi145 ceph-mon[33384]: pgmap v645: 158 pgs: 158 active+clean; 373 KiB data, 4.7 GiB used, 711 GiB / 715 GiB avail 2024-09-11T08:53:04.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:04 smithi145 ceph-mon[33384]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:53:04.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:04 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:53:04.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:04 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:53:04.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:04 smithi145 ceph-mon[33384]: osdmap e298: 8 total, 8 up, 8 in 2024-09-11T08:53:04.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:04 smithi145 ceph-mon[33384]: 4.f scrub starts 2024-09-11T08:53:04.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:04 smithi145 ceph-mon[33384]: 4.f scrub ok 2024-09-11T08:53:04.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:04 smithi145 ceph-mon[33384]: osdmap e299: 8 total, 8 up, 8 in 2024-09-11T08:53:04.936 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:04.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:04.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:05.025 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[29647]: pgmap v645: 158 pgs: 158 active+clean; 373 KiB data, 4.7 GiB used, 711 GiB / 715 GiB avail 2024-09-11T08:53:05.025 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[29647]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:53:05.025 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:53:05.025 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:53:05.025 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[29647]: osdmap e298: 8 total, 8 up, 8 in 2024-09-11T08:53:05.025 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[29647]: 4.f scrub starts 2024-09-11T08:53:05.025 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[29647]: 4.f scrub ok 2024-09-11T08:53:05.026 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[29647]: osdmap e299: 8 total, 8 up, 8 in 2024-09-11T08:53:05.026 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[41114]: pgmap v645: 158 pgs: 158 active+clean; 373 KiB data, 4.7 GiB used, 711 GiB / 715 GiB avail 2024-09-11T08:53:05.026 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[41114]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:53:05.026 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:53:05.026 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:53:05.026 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[41114]: osdmap e298: 8 total, 8 up, 8 in 2024-09-11T08:53:05.026 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[41114]: 4.f scrub starts 2024-09-11T08:53:05.026 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[41114]: 4.f scrub ok 2024-09-11T08:53:05.026 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:04 smithi092 ceph-mon[41114]: osdmap e299: 8 total, 8 up, 8 in 2024-09-11T08:53:05.038 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:05.072 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:05.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:05.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:05.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:05.209 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:05.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:05.211 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:05.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:05.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:05.347 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:05.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:05.482 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:05.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:05.483 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:05.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:05.618 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:05.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:05.619 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:05.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:05.754 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:05.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:05.756 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:05.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:05.878 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 1 16 4086 4070 254.361 254.375 0.00394893 0.00391852 2024-09-11T08:53:05.893 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:05.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:05.894 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:05.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:05 smithi145 ceph-mon[33384]: 3.16 scrub starts 2024-09-11T08:53:05.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:05 smithi145 ceph-mon[33384]: 3.16 scrub ok 2024-09-11T08:53:05.924 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:05 smithi092 ceph-mon[29647]: 3.16 scrub starts 2024-09-11T08:53:05.924 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:05 smithi092 ceph-mon[29647]: 3.16 scrub ok 2024-09-11T08:53:05.924 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:05 smithi092 ceph-mon[41114]: 3.16 scrub starts 2024-09-11T08:53:05.924 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:05 smithi092 ceph-mon[41114]: 3.16 scrub ok 2024-09-11T08:53:05.996 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:06.027 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:06.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:06.028 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:06.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:06.166 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:06.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:06.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:06.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:06.306 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:06.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:06.308 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:06.408 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:06.444 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:06.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:06.445 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:06.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:06.590 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:06.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:06.591 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:06.692 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:06.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:06 smithi092 ceph-mon[29647]: pgmap v648: 158 pgs: 158 active+clean; 373 KiB data, 4.7 GiB used, 711 GiB / 715 GiB avail 2024-09-11T08:53:06.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:06 smithi092 ceph-mon[29647]: 5.d deep-scrub starts 2024-09-11T08:53:06.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:06 smithi092 ceph-mon[29647]: 5.d deep-scrub ok 2024-09-11T08:53:06.724 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:06 smithi092 ceph-mon[41114]: pgmap v648: 158 pgs: 158 active+clean; 373 KiB data, 4.7 GiB used, 711 GiB / 715 GiB avail 2024-09-11T08:53:06.724 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:06 smithi092 ceph-mon[41114]: 5.d deep-scrub starts 2024-09-11T08:53:06.724 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:06 smithi092 ceph-mon[41114]: 5.d deep-scrub ok 2024-09-11T08:53:06.724 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:06.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:06.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:06.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:06.858 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:06.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:06.859 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:06.878 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 2 15 7756 7741 241.89 229.438 0.00391344 0.00412521 2024-09-11T08:53:06.911 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:53:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:53:06.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:06 smithi145 ceph-mon[33384]: pgmap v648: 158 pgs: 158 active+clean; 373 KiB data, 4.7 GiB used, 711 GiB / 715 GiB avail 2024-09-11T08:53:06.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:06 smithi145 ceph-mon[33384]: 5.d deep-scrub starts 2024-09-11T08:53:06.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:06 smithi145 ceph-mon[33384]: 5.d deep-scrub ok 2024-09-11T08:53:06.960 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:06.992 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:06.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:06.993 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:07.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:53:07.094 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:07.126 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:07.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:07.127 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:07.228 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:07.259 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:07.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:07.260 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:07.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:07.388 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:53:07.388 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:53:07.388 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:53:07.388 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:53:07.388 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:53:07.423 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:07.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:07.425 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:07.525 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:07.560 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:07.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:07.561 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:07.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:07.693 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:07.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:07.694 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:07.795 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:07.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:07 smithi092 ceph-mon[29647]: 6.e scrub starts 2024-09-11T08:53:07.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:07 smithi092 ceph-mon[29647]: 6.e scrub ok 2024-09-11T08:53:07.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:07 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/393701954' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:53:07.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:07 smithi092 ceph-mon[41114]: 6.e scrub starts 2024-09-11T08:53:07.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:07 smithi092 ceph-mon[41114]: 6.e scrub ok 2024-09-11T08:53:07.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:07 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/393701954' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:53:07.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:07.830 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:07.830 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:07.878 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 3 16 10914 10898 227.025 197.312 0.00418684 0.00439807 2024-09-11T08:53:07.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:07 smithi145 ceph-mon[33384]: 6.e scrub starts 2024-09-11T08:53:07.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:07 smithi145 ceph-mon[33384]: 6.e scrub ok 2024-09-11T08:53:07.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:07 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/393701954' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:53:07.931 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:07.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:07.967 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:07.967 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:08.013 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:53:08.014 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:53:08.014 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:53:08.014 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:53:08.014 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:53:08.014 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:53:08.014 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:53:08.014 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:53:08.014 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:53:08.014 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:53:08.014 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:53:08.014 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:53:08.015 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:53:08.015 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:53:08.015 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:53:08.015 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:53:08.069 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:08.103 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:08.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:08.105 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:08.205 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:08.242 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:08.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:08.243 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:08.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:08.379 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:08.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:08.381 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:08.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:08.519 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:08.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:08.520 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:08.621 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:08.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:08.658 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:08.658 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:08.759 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:08.795 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:08.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:08.796 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:08.878 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 4 16 14235 14219 222.155 207.562 0.00675189 0.00449539 2024-09-11T08:53:08.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:08.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:08 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:08.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:08 smithi092 ceph-mon[29647]: pgmap v649: 158 pgs: 158 active+clean; 370 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 62 MiB/s wr, 983 op/s 2024-09-11T08:53:08.907 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:08 smithi092 ceph-mon[29647]: 4.1d scrub starts 2024-09-11T08:53:08.907 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:08 smithi092 ceph-mon[29647]: 4.1d scrub ok 2024-09-11T08:53:08.907 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:08 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T08:53:08.907 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:08 smithi092 ceph-mon[29647]: 6.f scrub ok 2024-09-11T08:53:08.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:08 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:08.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:08 smithi092 ceph-mon[41114]: pgmap v649: 158 pgs: 158 active+clean; 370 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 62 MiB/s wr, 983 op/s 2024-09-11T08:53:08.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:08 smithi092 ceph-mon[41114]: 4.1d scrub starts 2024-09-11T08:53:08.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:08 smithi092 ceph-mon[41114]: 4.1d scrub ok 2024-09-11T08:53:08.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:08 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T08:53:08.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:08 smithi092 ceph-mon[41114]: 6.f scrub ok 2024-09-11T08:53:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:08 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:08 smithi145 ceph-mon[33384]: pgmap v649: 158 pgs: 158 active+clean; 370 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 62 MiB/s wr, 983 op/s 2024-09-11T08:53:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:08 smithi145 ceph-mon[33384]: 4.1d scrub starts 2024-09-11T08:53:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:08 smithi145 ceph-mon[33384]: 4.1d scrub ok 2024-09-11T08:53:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:08 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T08:53:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:08 smithi145 ceph-mon[33384]: 6.f scrub ok 2024-09-11T08:53:08.939 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:08.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:08.940 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:09.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:09.079 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:09.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:09.080 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:09.181 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:09.218 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:09.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:09.219 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:09.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:09.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:09.356 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:09.356 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:09.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-11T08:53:09.457 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:09.475 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:09.487 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:09.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:09.488 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:09.589 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:09.624 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:09.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:09.625 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:09.726 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:09.757 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:09.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:09.758 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:09.859 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:09.878 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 5 16 17095 17079 213.47 178.75 0.00452939 0.00467813 2024-09-11T08:53:09.895 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:09.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:09.896 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:09.987 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:09 smithi092 ceph-mon[29647]: 1.4 scrub starts 2024-09-11T08:53:09.987 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:09 smithi092 ceph-mon[29647]: 1.4 scrub ok 2024-09-11T08:53:09.988 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:09 smithi092 ceph-mon[29647]: 2.1 deep-scrub starts 2024-09-11T08:53:09.988 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:09 smithi092 ceph-mon[29647]: 2.1 deep-scrub ok 2024-09-11T08:53:09.988 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:09 smithi092 ceph-mon[29647]: osdmap e300: 8 total, 8 up, 8 in 2024-09-11T08:53:09.988 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:09 smithi092 ceph-mon[41114]: 1.4 scrub starts 2024-09-11T08:53:09.988 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:09 smithi092 ceph-mon[41114]: 1.4 scrub ok 2024-09-11T08:53:09.988 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:09 smithi092 ceph-mon[41114]: 2.1 deep-scrub starts 2024-09-11T08:53:09.988 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:09 smithi092 ceph-mon[41114]: 2.1 deep-scrub ok 2024-09-11T08:53:09.988 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:09 smithi092 ceph-mon[41114]: osdmap e300: 8 total, 8 up, 8 in 2024-09-11T08:53:09.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:10.033 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:10.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:10.034 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:10.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:10.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:09 smithi145 ceph-mon[33384]: 1.4 scrub starts 2024-09-11T08:53:10.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:09 smithi145 ceph-mon[33384]: 1.4 scrub ok 2024-09-11T08:53:10.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:09 smithi145 ceph-mon[33384]: 2.1 deep-scrub starts 2024-09-11T08:53:10.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:09 smithi145 ceph-mon[33384]: 2.1 deep-scrub ok 2024-09-11T08:53:10.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:09 smithi145 ceph-mon[33384]: osdmap e300: 8 total, 8 up, 8 in 2024-09-11T08:53:10.172 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:10.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:10.173 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:10.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:10.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:10.308 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:10.308 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:10.361 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:10.409 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:10.432 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:10.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:10.433 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:10.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:10.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:10.569 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:10.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:10.670 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:10.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:10.708 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:10.708 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:10.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:10.841 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:10.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:10.842 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:10.879 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 6 16 19835 19819 206.431 171.25 0.00481343 0.00483166 2024-09-11T08:53:10.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:10.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:53:10.978 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:10.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:10.979 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:11.013 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:10 smithi092 ceph-mon[29647]: pgmap v651: 158 pgs: 158 active+clean; 370 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 62 MiB/s wr, 985 op/s 2024-09-11T08:53:11.013 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:10 smithi092 ceph-mon[29647]: 5.2 scrub starts 2024-09-11T08:53:11.013 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:10 smithi092 ceph-mon[29647]: 5.2 scrub ok 2024-09-11T08:53:11.013 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:10 smithi092 ceph-mon[29647]: 4.4 deep-scrub starts 2024-09-11T08:53:11.013 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:10 smithi092 ceph-mon[29647]: 4.4 deep-scrub ok 2024-09-11T08:53:11.013 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:10 smithi092 ceph-mon[41114]: pgmap v651: 158 pgs: 158 active+clean; 370 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 62 MiB/s wr, 985 op/s 2024-09-11T08:53:11.013 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:10 smithi092 ceph-mon[41114]: 5.2 scrub starts 2024-09-11T08:53:11.014 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:10 smithi092 ceph-mon[41114]: 5.2 scrub ok 2024-09-11T08:53:11.014 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:10 smithi092 ceph-mon[41114]: 4.4 deep-scrub starts 2024-09-11T08:53:11.014 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:10 smithi092 ceph-mon[41114]: 4.4 deep-scrub ok 2024-09-11T08:53:11.080 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:11.116 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:11.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:11.118 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:11.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:10 smithi145 ceph-mon[33384]: pgmap v651: 158 pgs: 158 active+clean; 370 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 62 MiB/s wr, 985 op/s 2024-09-11T08:53:11.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:10 smithi145 ceph-mon[33384]: 5.2 scrub starts 2024-09-11T08:53:11.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:10 smithi145 ceph-mon[33384]: 5.2 scrub ok 2024-09-11T08:53:11.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:10 smithi145 ceph-mon[33384]: 4.4 deep-scrub starts 2024-09-11T08:53:11.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:10 smithi145 ceph-mon[33384]: 4.4 deep-scrub ok 2024-09-11T08:53:11.219 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:11.249 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:11.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:11.251 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:11.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:11.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:11.389 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:11.390 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:11.490 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:11.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:11.525 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:11.526 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:11.626 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:11.665 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:11.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:11.666 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:11.767 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:11.803 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:11.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:11.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:11.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:11 smithi092 ceph-mon[29647]: 3.a scrub starts 2024-09-11T08:53:11.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:11 smithi092 ceph-mon[29647]: 3.a scrub ok 2024-09-11T08:53:11.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:11 smithi092 ceph-mon[41114]: 3.a scrub starts 2024-09-11T08:53:11.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:11 smithi092 ceph-mon[41114]: 3.a scrub ok 2024-09-11T08:53:11.816 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:53:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:53:11] "GET /metrics HTTP/1.1" 200 224534 "" "Prometheus/2.43.0" 2024-09-11T08:53:11.879 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 7 16 22174 22158 197.823 146.188 0.0073064 0.00504947 2024-09-11T08:53:11.904 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:11.940 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:11.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:11.941 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:12.042 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:12.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:12.077 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:12.077 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:12.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:11 smithi145 ceph-mon[33384]: 3.a scrub starts 2024-09-11T08:53:12.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:11 smithi145 ceph-mon[33384]: 3.a scrub ok 2024-09-11T08:53:12.178 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:12.215 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:12.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:12.216 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:12.317 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:12.338 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:12.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:12.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:12.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:12.474 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:12.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:12.476 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:12.576 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:12.598 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:12.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:12.599 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:12.699 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:12.728 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:12.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:12.729 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:12.830 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:12.862 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:12.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:12.863 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:12.879 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 8 16 23980 23964 187.203 112.875 0.00717631 0.00533684 2024-09-11T08:53:12.964 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:12.989 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:12.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:12.990 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:13.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:12 smithi092 ceph-mon[29647]: pgmap v652: 158 pgs: 158 active+clean; 961 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 667 B/s rd, 125 MiB/s wr, 2.00k op/s 2024-09-11T08:53:13.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:12 smithi092 ceph-mon[29647]: 5.9 scrub starts 2024-09-11T08:53:13.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:12 smithi092 ceph-mon[29647]: 5.9 scrub ok 2024-09-11T08:53:13.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:12 smithi092 ceph-mon[29647]: 6.6 scrub starts 2024-09-11T08:53:13.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:12 smithi092 ceph-mon[29647]: 6.6 scrub ok 2024-09-11T08:53:13.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:12 smithi092 ceph-mon[41114]: pgmap v652: 158 pgs: 158 active+clean; 961 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 667 B/s rd, 125 MiB/s wr, 2.00k op/s 2024-09-11T08:53:13.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:12 smithi092 ceph-mon[41114]: 5.9 scrub starts 2024-09-11T08:53:13.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:12 smithi092 ceph-mon[41114]: 5.9 scrub ok 2024-09-11T08:53:13.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:12 smithi092 ceph-mon[41114]: 6.6 scrub starts 2024-09-11T08:53:13.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:12 smithi092 ceph-mon[41114]: 6.6 scrub ok 2024-09-11T08:53:13.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:13.128 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:13.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:13.129 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:13.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:53:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:53:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:53:13.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:12 smithi145 ceph-mon[33384]: pgmap v652: 158 pgs: 158 active+clean; 961 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 667 B/s rd, 125 MiB/s wr, 2.00k op/s 2024-09-11T08:53:13.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:12 smithi145 ceph-mon[33384]: 5.9 scrub starts 2024-09-11T08:53:13.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:12 smithi145 ceph-mon[33384]: 5.9 scrub ok 2024-09-11T08:53:13.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:12 smithi145 ceph-mon[33384]: 6.6 scrub starts 2024-09-11T08:53:13.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:12 smithi145 ceph-mon[33384]: 6.6 scrub ok 2024-09-11T08:53:13.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:13.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:13.259 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:13.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:13.395 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:13.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:13.396 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:13.497 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:13.531 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:13.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:13.533 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:13.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:13.670 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:13.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:13.671 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:13.772 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:13.805 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:13.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:13.806 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:13.879 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 9 16 26399 26383 183.2 151.188 0.00415845 0.00544574 2024-09-11T08:53:13.907 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:13.929 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:13.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:13.930 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:14.031 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:14.064 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:13 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:53:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:13 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:53:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:13 smithi092 ceph-mon[29647]: osdmap e301: 8 total, 8 up, 8 in 2024-09-11T08:53:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:13 smithi092 ceph-mon[29647]: 5.1c scrub starts 2024-09-11T08:53:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:13 smithi092 ceph-mon[29647]: 5.1c scrub ok 2024-09-11T08:53:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:13 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:53:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:13 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:53:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:13 smithi092 ceph-mon[41114]: osdmap e301: 8 total, 8 up, 8 in 2024-09-11T08:53:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:13 smithi092 ceph-mon[41114]: 5.1c scrub starts 2024-09-11T08:53:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:13 smithi092 ceph-mon[41114]: 5.1c scrub ok 2024-09-11T08:53:14.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:14.067 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:14.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:13 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:53:14.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:13 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:53:14.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:13 smithi145 ceph-mon[33384]: osdmap e301: 8 total, 8 up, 8 in 2024-09-11T08:53:14.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:13 smithi145 ceph-mon[33384]: 5.1c scrub starts 2024-09-11T08:53:14.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:13 smithi145 ceph-mon[33384]: 5.1c scrub ok 2024-09-11T08:53:14.167 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:14.204 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:14.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:14.205 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:14.306 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:14.339 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:14.341 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:14.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:14.342 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:14.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:14.483 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:14.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:14.484 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:14.585 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:14.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:14.620 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:14.620 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:14.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:14.754 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:14.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:14.755 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:14.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:14.879 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 10 16 28611 28595 178.702 138.25 0.0060894 0.00559017 2024-09-11T08:53:14.889 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:14.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:14.890 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:14.917 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[29647]: pgmap v654: 158 pgs: 158 active+clean; 1.3 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 164 MiB/s wr, 2.62k op/s 2024-09-11T08:53:14.918 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[29647]: 1.7 scrub starts 2024-09-11T08:53:14.918 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[29647]: 1.7 scrub ok 2024-09-11T08:53:14.918 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[29647]: 3.4 scrub starts 2024-09-11T08:53:14.918 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[29647]: 3.4 scrub ok 2024-09-11T08:53:14.918 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[29647]: 3.c scrub starts 2024-09-11T08:53:14.918 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[29647]: 3.c scrub ok 2024-09-11T08:53:14.918 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[29647]: osdmap e302: 8 total, 8 up, 8 in 2024-09-11T08:53:14.918 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[41114]: pgmap v654: 158 pgs: 158 active+clean; 1.3 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 164 MiB/s wr, 2.62k op/s 2024-09-11T08:53:14.918 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[41114]: 1.7 scrub starts 2024-09-11T08:53:14.918 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[41114]: 1.7 scrub ok 2024-09-11T08:53:14.918 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[41114]: 3.4 scrub starts 2024-09-11T08:53:14.919 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[41114]: 3.4 scrub ok 2024-09-11T08:53:14.919 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[41114]: 3.c scrub starts 2024-09-11T08:53:14.919 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[41114]: 3.c scrub ok 2024-09-11T08:53:14.919 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:14 smithi092 ceph-mon[41114]: osdmap e302: 8 total, 8 up, 8 in 2024-09-11T08:53:14.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:15.028 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:15.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:15.029 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:15.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:15.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:14 smithi145 ceph-mon[33384]: pgmap v654: 158 pgs: 158 active+clean; 1.3 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 164 MiB/s wr, 2.62k op/s 2024-09-11T08:53:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:14 smithi145 ceph-mon[33384]: 1.7 scrub starts 2024-09-11T08:53:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:14 smithi145 ceph-mon[33384]: 1.7 scrub ok 2024-09-11T08:53:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:14 smithi145 ceph-mon[33384]: 3.4 scrub starts 2024-09-11T08:53:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:14 smithi145 ceph-mon[33384]: 3.4 scrub ok 2024-09-11T08:53:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:14 smithi145 ceph-mon[33384]: 3.c scrub starts 2024-09-11T08:53:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:14 smithi145 ceph-mon[33384]: 3.c scrub ok 2024-09-11T08:53:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:14 smithi145 ceph-mon[33384]: osdmap e302: 8 total, 8 up, 8 in 2024-09-11T08:53:15.171 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:15.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:15.172 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:15.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:15.310 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:15.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:15.311 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:15.412 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:15.445 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:15.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:15.447 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:15.548 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:15.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:15.582 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:15.583 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:15.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:15.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:15.725 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:15.725 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:15.808 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:53:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:53:15.825 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:15.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:15.866 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:15.866 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:15.879 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 11 16 30876 30860 175.325 141.562 0.00981079 0.0056986 2024-09-11T08:53:15.967 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:16.004 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:16.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:16.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:16.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:15 smithi092 ceph-mon[29647]: 5.11 scrub starts 2024-09-11T08:53:16.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:15 smithi092 ceph-mon[29647]: 5.11 scrub ok 2024-09-11T08:53:16.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:15 smithi092 ceph-mon[29647]: 6.1f scrub starts 2024-09-11T08:53:16.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:15 smithi092 ceph-mon[29647]: 6.1f scrub ok 2024-09-11T08:53:16.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:15 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:53:16.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:53:16.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:15 smithi092 ceph-mon[29647]: pgmap v656: 158 pgs: 158 active+clean; 1.3 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 149 MiB/s wr, 2.39k op/s 2024-09-11T08:53:16.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:15 smithi092 ceph-mon[41114]: 5.11 scrub starts 2024-09-11T08:53:16.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:15 smithi092 ceph-mon[41114]: 5.11 scrub ok 2024-09-11T08:53:16.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:15 smithi092 ceph-mon[41114]: 6.1f scrub starts 2024-09-11T08:53:16.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:15 smithi092 ceph-mon[41114]: 6.1f scrub ok 2024-09-11T08:53:16.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:15 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:53:16.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:53:16.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:15 smithi092 ceph-mon[41114]: pgmap v656: 158 pgs: 158 active+clean; 1.3 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 149 MiB/s wr, 2.39k op/s 2024-09-11T08:53:16.105 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:16.142 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:16.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:16.143 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:16.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:15 smithi145 ceph-mon[33384]: 5.11 scrub starts 2024-09-11T08:53:16.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:15 smithi145 ceph-mon[33384]: 5.11 scrub ok 2024-09-11T08:53:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:15 smithi145 ceph-mon[33384]: 6.1f scrub starts 2024-09-11T08:53:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:15 smithi145 ceph-mon[33384]: 6.1f scrub ok 2024-09-11T08:53:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:15 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:53:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:53:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:15 smithi145 ceph-mon[33384]: pgmap v656: 158 pgs: 158 active+clean; 1.3 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 149 MiB/s wr, 2.39k op/s 2024-09-11T08:53:16.244 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:16.278 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:16.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:16.280 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:16.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:16.416 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:16.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:16.417 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:16.518 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:16.554 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:16.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:16.555 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:16.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:16.695 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:16.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:16.696 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:16.797 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:16.829 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:16.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:16.830 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:16.880 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 12 16 33209 33193 172.864 145.812 0.00466663 0.00577999 2024-09-11T08:53:16.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:16.965 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:16.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:16.966 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:17.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:16 smithi092 ceph-mon[29647]: 1.8 deep-scrub starts 2024-09-11T08:53:17.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:16 smithi092 ceph-mon[29647]: 1.8 deep-scrub ok 2024-09-11T08:53:17.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:16 smithi092 ceph-mon[29647]: 1.0 scrub starts 2024-09-11T08:53:17.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:16 smithi092 ceph-mon[29647]: 1.0 scrub ok 2024-09-11T08:53:17.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:16 smithi092 ceph-mon[29647]: 6.c scrub starts 2024-09-11T08:53:17.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:16 smithi092 ceph-mon[29647]: 6.c scrub ok 2024-09-11T08:53:17.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:16 smithi092 ceph-mon[29647]: from='client.25466 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:53:17.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:16 smithi092 ceph-mon[41114]: 1.8 deep-scrub starts 2024-09-11T08:53:17.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:16 smithi092 ceph-mon[41114]: 1.8 deep-scrub ok 2024-09-11T08:53:17.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:16 smithi092 ceph-mon[41114]: 1.0 scrub starts 2024-09-11T08:53:17.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:16 smithi092 ceph-mon[41114]: 1.0 scrub ok 2024-09-11T08:53:17.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:16 smithi092 ceph-mon[41114]: 6.c scrub starts 2024-09-11T08:53:17.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:16 smithi092 ceph-mon[41114]: 6.c scrub ok 2024-09-11T08:53:17.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:16 smithi092 ceph-mon[41114]: from='client.25466 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:53:17.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:17.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:17.102 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:17.102 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:17.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:16 smithi145 ceph-mon[33384]: 1.8 deep-scrub starts 2024-09-11T08:53:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:16 smithi145 ceph-mon[33384]: 1.8 deep-scrub ok 2024-09-11T08:53:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:16 smithi145 ceph-mon[33384]: 1.0 scrub starts 2024-09-11T08:53:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:16 smithi145 ceph-mon[33384]: 1.0 scrub ok 2024-09-11T08:53:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:16 smithi145 ceph-mon[33384]: 6.c scrub starts 2024-09-11T08:53:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:16 smithi145 ceph-mon[33384]: 6.c scrub ok 2024-09-11T08:53:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:16 smithi145 ceph-mon[33384]: from='client.25466 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:53:17.162 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:53:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:53:17.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:17.236 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:17.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:17.237 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:17.338 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:17.371 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:17.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:17.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:17.473 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:17.495 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:17.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:17.496 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:17.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:17.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:17.634 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:17.635 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:17.735 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:17.745 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:17.766 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:17.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:17.767 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:17.868 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:17.882 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 13 16 35341 35325 169.783 133.25 0.0143299 0.00588055 2024-09-11T08:53:17.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:17.909 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:17.909 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:18.010 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:18.042 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:18.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:18.044 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:18.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:17 smithi092 ceph-mon[29647]: 3.15 scrub starts 2024-09-11T08:53:18.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:17 smithi092 ceph-mon[29647]: 3.15 scrub ok 2024-09-11T08:53:18.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:17 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:18.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:17 smithi092 ceph-mon[29647]: 4.9 deep-scrub starts 2024-09-11T08:53:18.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:17 smithi092 ceph-mon[29647]: 4.9 deep-scrub ok 2024-09-11T08:53:18.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:17 smithi092 ceph-mon[29647]: pgmap v657: 158 pgs: 158 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 204 MiB/s wr, 3.26k op/s 2024-09-11T08:53:18.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:17 smithi092 ceph-mon[41114]: 3.15 scrub starts 2024-09-11T08:53:18.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:17 smithi092 ceph-mon[41114]: 3.15 scrub ok 2024-09-11T08:53:18.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:17 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:18.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:17 smithi092 ceph-mon[41114]: 4.9 deep-scrub starts 2024-09-11T08:53:18.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:17 smithi092 ceph-mon[41114]: 4.9 deep-scrub ok 2024-09-11T08:53:18.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:17 smithi092 ceph-mon[41114]: pgmap v657: 158 pgs: 158 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 204 MiB/s wr, 3.26k op/s 2024-09-11T08:53:18.145 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:18.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:17 smithi145 ceph-mon[33384]: 3.15 scrub starts 2024-09-11T08:53:18.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:17 smithi145 ceph-mon[33384]: 3.15 scrub ok 2024-09-11T08:53:18.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:17 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:17 smithi145 ceph-mon[33384]: 4.9 deep-scrub starts 2024-09-11T08:53:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:17 smithi145 ceph-mon[33384]: 4.9 deep-scrub ok 2024-09-11T08:53:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:17 smithi145 ceph-mon[33384]: pgmap v657: 158 pgs: 158 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 204 MiB/s wr, 3.26k op/s 2024-09-11T08:53:18.175 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:18.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:18.176 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:18.277 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:18.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:18.314 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:18.315 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:18.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:18.452 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:18.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:18.453 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:18.553 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:18.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:18.590 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:18.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:18.723 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:18.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:18.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:18.825 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:18.860 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:18.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:18.861 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:18.882 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 14 16 37480 37464 167.203 133.688 0.00826137 0.00596559 2024-09-11T08:53:18.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:18.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:18.984 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:19.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:18 smithi092 ceph-mon[29647]: 1.a scrub starts 2024-09-11T08:53:19.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:18 smithi092 ceph-mon[29647]: 1.a scrub ok 2024-09-11T08:53:19.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:18 smithi092 ceph-mon[41114]: 1.a scrub starts 2024-09-11T08:53:19.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:18 smithi092 ceph-mon[41114]: 1.a scrub ok 2024-09-11T08:53:19.085 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:19.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:19.124 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:19.125 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:19.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:18 smithi145 ceph-mon[33384]: 1.a scrub starts 2024-09-11T08:53:19.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:18 smithi145 ceph-mon[33384]: 1.a scrub ok 2024-09-11T08:53:19.225 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:19.260 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:19.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:19.261 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:19.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:19.399 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:19.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:19.400 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:19.501 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:19.537 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:19.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:19.539 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:19.640 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:19.675 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:19.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:19.676 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:19.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:19.821 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":656,"stamp":"2024-09-11T08:53:15.304485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1375928478,"num_objects":21182,"num_object_clones":0,"num_object_copies":63546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21182,"num_whiteouts":0,"num_read":1235,"num_read_kb":1045,"num_write":21763,"num_write_kb":1343971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":696585,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22723,"ondisk_log_size":22723,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":480,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":8723140,"kb_used_data":4201220,"kb_used_omap":13,"kb_used_meta":4521842,"kb_avail":741238076,"statfs":{"total":767960285184,"available":759027789824,"internally_reserved":0,"allocated":4302049280,"data_stored":4299147220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":4630366912},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[13,6,12,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":988163674,"num_objects":15075,"num_object_clones":0,"num_object_copies":45225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15075,"num_whiteouts":0,"num_read":-7,"num_read_kb":-16,"num_write":15064,"num_write_kb":964588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-381350,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.303473"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":108,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998389+0000","last_change":"2024-09-11T08:52:49.568717+0000","last_active":"2024-09-11T08:53:03.998389+0000","last_peered":"2024-09-11T08:53:03.998389+0000","last_clean":"2024-09-11T08:53:03.998389+0000","last_became_active":"2024-09-11T08:52:49.568416+0000","last_became_peered":"2024-09-11T08:52:49.568416+0000","last_unstale":"2024-09-11T08:53:03.998389+0000","last_undegraded":"2024-09-11T08:53:03.998389+0000","last_fullsized":"2024-09-11T08:53:03.998389+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:29.430953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:29.430953+0000","last_clean_scrub_stamp":"2024-09-11T08:51:29.430953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"200'19","reported_seq":370,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996263+0000","last_change":"2024-09-11T08:52:30.312273+0000","last_active":"2024-09-11T08:53:03.996263+0000","last_peered":"2024-09-11T08:53:03.996263+0000","last_clean":"2024-09-11T08:53:03.996263+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:53:03.996263+0000","last_undegraded":"2024-09-11T08:53:03.996263+0000","last_fullsized":"2024-09-11T08:53:03.996263+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:52:30.312197+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:52:30.312197+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:43.338489+0000","scrub_duration":0.002912516,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":308,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.078145+0000","last_change":"2024-09-11T08:50:18.611806+0000","last_active":"2024-09-11T08:53:04.078145+0000","last_peered":"2024-09-11T08:53:04.078145+0000","last_clean":"2024-09-11T08:53:04.078145+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:53:04.078145+0000","last_undegraded":"2024-09-11T08:53:04.078145+0000","last_fullsized":"2024-09-11T08:53:04.078145+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:18.611725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:18.611725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072923490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":287,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996457+0000","last_change":"2024-09-11T08:51:37.048922+0000","last_active":"2024-09-11T08:53:03.996457+0000","last_peered":"2024-09-11T08:53:03.996457+0000","last_clean":"2024-09-11T08:53:03.996457+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:53:03.996457+0000","last_undegraded":"2024-09-11T08:53:03.996457+0000","last_fullsized":"2024-09-11T08:53:03.996457+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:37.048848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:51:37.048848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029662830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":316,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997428+0000","last_change":"2024-09-11T08:51:35.370494+0000","last_active":"2024-09-11T08:53:03.997428+0000","last_peered":"2024-09-11T08:53:03.997428+0000","last_clean":"2024-09-11T08:53:03.997428+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:53:03.997428+0000","last_undegraded":"2024-09-11T08:53:03.997428+0000","last_fullsized":"2024-09-11T08:53:03.997428+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:35.370438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:51:35.370438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053884329999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":322,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998126+0000","last_change":"2024-09-11T08:51:28.347060+0000","last_active":"2024-09-11T08:53:03.998126+0000","last_peered":"2024-09-11T08:53:03.998126+0000","last_clean":"2024-09-11T08:53:03.998126+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:53:03.998126+0000","last_undegraded":"2024-09-11T08:53:03.998126+0000","last_fullsized":"2024-09-11T08:53:03.998126+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:28.346979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:51:28.346979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020416560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"200'34","reported_seq":402,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997185+0000","last_change":"2024-09-11T08:52:18.609763+0000","last_active":"2024-09-11T08:53:03.997185+0000","last_peered":"2024-09-11T08:53:03.997185+0000","last_clean":"2024-09-11T08:53:03.997185+0000","last_became_active":"2024-09-11T08:52:18.585600+0000","last_became_peered":"2024-09-11T08:52:18.585600+0000","last_unstale":"2024-09-11T08:53:03.997185+0000","last_undegraded":"2024-09-11T08:53:03.997185+0000","last_fullsized":"2024-09-11T08:53:03.997185+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:51:31.358660+0000","last_deep_scrub":"200'34","last_deep_scrub_stamp":"2024-09-11T08:51:31.358660+0000","last_clean_scrub_stamp":"2024-09-11T08:51:31.358660+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071678469999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":289,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996212+0000","last_change":"2024-09-11T08:51:19.004758+0000","last_active":"2024-09-11T08:53:03.996212+0000","last_peered":"2024-09-11T08:53:03.996212+0000","last_clean":"2024-09-11T08:53:03.996212+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:53:03.996212+0000","last_undegraded":"2024-09-11T08:53:03.996212+0000","last_fullsized":"2024-09-11T08:53:03.996212+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:19.004676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:51:19.004676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006451526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":328,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998279+0000","last_change":"2024-09-11T08:51:18.275081+0000","last_active":"2024-09-11T08:53:03.998279+0000","last_peered":"2024-09-11T08:53:03.998279+0000","last_clean":"2024-09-11T08:53:03.998279+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:53:03.998279+0000","last_undegraded":"2024-09-11T08:53:03.998279+0000","last_fullsized":"2024-09-11T08:53:03.998279+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:18.275025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:51:18.275025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011943050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"200'19","reported_seq":370,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996603+0000","last_change":"2024-09-11T08:52:18.598556+0000","last_active":"2024-09-11T08:53:03.996603+0000","last_peered":"2024-09-11T08:53:03.996603+0000","last_clean":"2024-09-11T08:53:03.996603+0000","last_became_active":"2024-09-11T08:52:18.577411+0000","last_became_peered":"2024-09-11T08:52:18.577411+0000","last_unstale":"2024-09-11T08:53:03.996603+0000","last_undegraded":"2024-09-11T08:53:03.996603+0000","last_fullsized":"2024-09-11T08:53:03.996603+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:34.003806+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:51:34.003806+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022468330000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":324,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997726+0000","last_change":"2024-09-11T08:52:18.573865+0000","last_active":"2024-09-11T08:53:03.997726+0000","last_peered":"2024-09-11T08:53:03.997726+0000","last_clean":"2024-09-11T08:53:03.997726+0000","last_became_active":"2024-09-11T08:52:18.573461+0000","last_became_peered":"2024-09-11T08:52:18.573461+0000","last_unstale":"2024-09-11T08:53:03.997726+0000","last_undegraded":"2024-09-11T08:53:03.997726+0000","last_fullsized":"2024-09-11T08:53:03.997726+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:25.373549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:25.373549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026496549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":108,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.179076+0000","last_change":"2024-09-11T08:52:49.568661+0000","last_active":"2024-09-11T08:53:04.179076+0000","last_peered":"2024-09-11T08:53:04.179076+0000","last_clean":"2024-09-11T08:53:04.179076+0000","last_became_active":"2024-09-11T08:52:49.568147+0000","last_became_peered":"2024-09-11T08:52:49.568147+0000","last_unstale":"2024-09-11T08:53:04.179076+0000","last_undegraded":"2024-09-11T08:53:04.179076+0000","last_fullsized":"2024-09-11T08:53:04.179076+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:30.508427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:33.520462+0000","last_clean_scrub_stamp":"2024-09-11T08:52:30.508427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:59.023014+0000","scrub_duration":0.0043744099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":308,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998054+0000","last_change":"2024-09-11T08:51:30.465078+0000","last_active":"2024-09-11T08:53:03.998054+0000","last_peered":"2024-09-11T08:53:03.998054+0000","last_clean":"2024-09-11T08:53:03.998054+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:53:03.998054+0000","last_undegraded":"2024-09-11T08:53:03.998054+0000","last_fullsized":"2024-09-11T08:53:03.998054+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:30.465003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:30.465003+0000","last_clean_scrub_stamp":"2024-09-11T08:51:30.465003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019306950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"200'7","reported_seq":307,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996538+0000","last_change":"2024-09-11T08:52:11.155994+0000","last_active":"2024-09-11T08:53:03.996538+0000","last_peered":"2024-09-11T08:53:03.996538+0000","last_clean":"2024-09-11T08:53:03.996538+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:53:03.996538+0000","last_undegraded":"2024-09-11T08:53:03.996538+0000","last_fullsized":"2024-09-11T08:53:03.996538+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'7","last_scrub_stamp":"2024-09-11T08:52:11.155898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:52:11.155898+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:24.232328+0000","scrub_duration":0.002524072,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":317,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.078951+0000","last_change":"2024-09-11T08:52:52.458278+0000","last_active":"2024-09-11T08:53:04.078951+0000","last_peered":"2024-09-11T08:53:04.078951+0000","last_clean":"2024-09-11T08:53:04.078951+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:53:04.078951+0000","last_undegraded":"2024-09-11T08:53:04.078951+0000","last_fullsized":"2024-09-11T08:53:04.078951+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:52.458219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:18.199651+0000","last_clean_scrub_stamp":"2024-09-11T08:52:52.458219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:17.396094+0000","scrub_duration":0.0013148910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":302,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996886+0000","last_change":"2024-09-11T08:52:50.730859+0000","last_active":"2024-09-11T08:53:03.996886+0000","last_peered":"2024-09-11T08:53:03.996886+0000","last_clean":"2024-09-11T08:53:03.996886+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:53:03.996886+0000","last_undegraded":"2024-09-11T08:53:03.996886+0000","last_fullsized":"2024-09-11T08:53:03.996886+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:50.730804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:52:50.730804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:14.400201+0000","scrub_duration":0.001426482,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":307,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997151+0000","last_change":"2024-09-11T08:50:13.762434+0000","last_active":"2024-09-11T08:53:03.997151+0000","last_peered":"2024-09-11T08:53:03.997151+0000","last_clean":"2024-09-11T08:53:03.997151+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:53:03.997151+0000","last_undegraded":"2024-09-11T08:53:03.997151+0000","last_fullsized":"2024-09-11T08:53:03.997151+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:13.762372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:13.762372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001701496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":108,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996696+0000","last_change":"2024-09-11T08:52:49.566922+0000","last_active":"2024-09-11T08:53:03.996696+0000","last_peered":"2024-09-11T08:53:03.996696+0000","last_clean":"2024-09-11T08:53:03.996696+0000","last_became_active":"2024-09-11T08:52:49.566575+0000","last_became_peered":"2024-09-11T08:52:49.566575+0000","last_unstale":"2024-09-11T08:53:03.996696+0000","last_undegraded":"2024-09-11T08:53:03.996696+0000","last_fullsized":"2024-09-11T08:53:03.996696+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:24.573732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:24.573732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"200'34","reported_seq":395,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.202559+0000","last_change":"2024-09-11T08:52:56.872065+0000","last_active":"2024-09-11T08:53:04.202559+0000","last_peered":"2024-09-11T08:53:04.202559+0000","last_clean":"2024-09-11T08:53:04.202559+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:53:04.202559+0000","last_undegraded":"2024-09-11T08:53:04.202559+0000","last_fullsized":"2024-09-11T08:53:04.202559+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:52:56.872009+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:52:56.872009+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:02.594140+0000","scrub_duration":0.001869905,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":328,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.179016+0000","last_change":"2024-09-11T08:52:18.579843+0000","last_active":"2024-09-11T08:53:04.179016+0000","last_peered":"2024-09-11T08:53:04.179016+0000","last_clean":"2024-09-11T08:53:04.179016+0000","last_became_active":"2024-09-11T08:52:18.579026+0000","last_became_peered":"2024-09-11T08:52:18.579026+0000","last_unstale":"2024-09-11T08:53:04.179016+0000","last_undegraded":"2024-09-11T08:53:04.179016+0000","last_fullsized":"2024-09-11T08:53:04.179016+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:29.793890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:29.793890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020976990000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":315,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998145+0000","last_change":"2024-09-11T08:50:07.275539+0000","last_active":"2024-09-11T08:53:03.998145+0000","last_peered":"2024-09-11T08:53:03.998145+0000","last_clean":"2024-09-11T08:53:03.998145+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:53:03.998145+0000","last_undegraded":"2024-09-11T08:53:03.998145+0000","last_fullsized":"2024-09-11T08:53:03.998145+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:50:07.275301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:07.275301+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019610208000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":293,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996672+0000","last_change":"2024-09-11T08:50:17.743118+0000","last_active":"2024-09-11T08:53:03.996672+0000","last_peered":"2024-09-11T08:53:03.996672+0000","last_clean":"2024-09-11T08:53:03.996672+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:53:03.996672+0000","last_undegraded":"2024-09-11T08:53:03.996672+0000","last_fullsized":"2024-09-11T08:53:03.996672+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:17.743026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:17.743026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061030829999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"200'18","reported_seq":354,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:07.440600+0000","last_change":"2024-09-11T08:53:07.440507+0000","last_active":"2024-09-11T08:53:07.440600+0000","last_peered":"2024-09-11T08:53:07.440600+0000","last_clean":"2024-09-11T08:53:07.440600+0000","last_became_active":"2024-09-11T08:49:16.943886+0000","last_became_peered":"2024-09-11T08:49:16.943886+0000","last_unstale":"2024-09-11T08:53:07.440600+0000","last_undegraded":"2024-09-11T08:53:07.440600+0000","last_fullsized":"2024-09-11T08:53:07.440600+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'18","last_scrub_stamp":"2024-09-11T08:53:07.440404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:07.440404+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:28.528108+0000","scrub_duration":0.0041402180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":289,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996948+0000","last_change":"2024-09-11T08:50:09.746351+0000","last_active":"2024-09-11T08:53:03.996948+0000","last_peered":"2024-09-11T08:53:03.996948+0000","last_clean":"2024-09-11T08:53:03.996948+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:53:03.996948+0000","last_undegraded":"2024-09-11T08:53:03.996948+0000","last_fullsized":"2024-09-11T08:53:03.996948+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:09.745640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:09.745640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010457068999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":35,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997898+0000","last_change":"2024-09-11T08:52:38.483970+0000","last_active":"2024-09-11T08:53:03.997898+0000","last_peered":"2024-09-11T08:53:03.997898+0000","last_clean":"2024-09-11T08:53:03.997898+0000","last_became_active":"2024-09-11T08:52:38.483603+0000","last_became_peered":"2024-09-11T08:52:38.483603+0000","last_unstale":"2024-09-11T08:53:03.997898+0000","last_undegraded":"2024-09-11T08:53:03.997898+0000","last_fullsized":"2024-09-11T08:53:03.997898+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:50:09.077871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:09.077871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":312,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.078216+0000","last_change":"2024-09-11T08:51:28.824144+0000","last_active":"2024-09-11T08:53:04.078216+0000","last_peered":"2024-09-11T08:53:04.078216+0000","last_clean":"2024-09-11T08:53:04.078216+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:53:04.078216+0000","last_undegraded":"2024-09-11T08:53:04.078216+0000","last_fullsized":"2024-09-11T08:53:04.078216+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:28.824093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:51:28.824093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017179579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"280'25","reported_seq":51,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997891+0000","last_change":"2024-09-11T08:52:49.568462+0000","last_active":"2024-09-11T08:53:03.997891+0000","last_peered":"2024-09-11T08:53:03.997891+0000","last_clean":"2024-09-11T08:53:03.997891+0000","last_became_active":"2024-09-11T08:52:49.568074+0000","last_became_peered":"2024-09-11T08:52:49.568074+0000","last_unstale":"2024-09-11T08:53:03.997891+0000","last_undegraded":"2024-09-11T08:53:03.997891+0000","last_fullsized":"2024-09-11T08:53:03.997891+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"280'25","last_scrub_stamp":"2024-09-11T08:52:24.584533+0000","last_deep_scrub":"280'25","last_deep_scrub_stamp":"2024-09-11T08:52:24.584533+0000","last_clean_scrub_stamp":"2024-09-11T08:52:24.584533+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:51.336883+0000","scrub_duration":0.0050623580000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":502,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:07.840180+0000","last_change":"2024-09-11T08:52:12.368313+0000","last_active":"2024-09-11T08:53:07.840180+0000","last_peered":"2024-09-11T08:53:07.840180+0000","last_clean":"2024-09-11T08:53:07.840180+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:53:07.840180+0000","last_undegraded":"2024-09-11T08:53:07.840180+0000","last_fullsized":"2024-09-11T08:53:07.840180+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:52:12.368231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:12.368231+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:21.567571+0000","scrub_duration":0.0073151179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"300'1321","reported_seq":1336,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.187183+0000","last_change":"2024-09-11T08:52:56.415129+0000","last_active":"2024-09-11T08:53:11.187183+0000","last_peered":"2024-09-11T08:53:11.187183+0000","last_clean":"2024-09-11T08:53:11.187183+0000","last_became_active":"2024-09-11T08:52:56.414844+0000","last_became_peered":"2024-09-11T08:52:56.414844+0000","last_unstale":"2024-09-11T08:53:11.187183+0000","last_undegraded":"2024-09-11T08:53:11.187183+0000","last_fullsized":"2024-09-11T08:53:11.187183+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1321,"log_dups_size":0,"ondisk_log_size":1321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:18.963092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86573056,"num_objects":1321,"num_object_clones":0,"num_object_copies":3963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1321,"num_write_kb":84544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"199'81","reported_seq":59,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997351+0000","last_change":"2024-09-11T08:51:03.024010+0000","last_active":"2024-09-11T08:53:03.997351+0000","last_peered":"2024-09-11T08:53:03.997351+0000","last_clean":"2024-09-11T08:53:03.997351+0000","last_became_active":"2024-09-11T08:51:03.023615+0000","last_became_peered":"2024-09-11T08:51:03.023615+0000","last_unstale":"2024-09-11T08:53:03.997351+0000","last_undegraded":"2024-09-11T08:53:03.997351+0000","last_fullsized":"2024-09-11T08:53:03.997351+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"200'34","reported_seq":360,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.178616+0000","last_change":"2024-09-11T08:53:03.680968+0000","last_active":"2024-09-11T08:53:04.178616+0000","last_peered":"2024-09-11T08:53:04.178616+0000","last_clean":"2024-09-11T08:53:04.178616+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:53:04.178616+0000","last_undegraded":"2024-09-11T08:53:04.178616+0000","last_fullsized":"2024-09-11T08:53:04.178616+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:53:03.680906+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:53:03.680906+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:26.793062+0000","scrub_duration":0.0038860740000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":302,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998067+0000","last_change":"2024-09-11T08:53:02.487714+0000","last_active":"2024-09-11T08:53:03.998067+0000","last_peered":"2024-09-11T08:53:03.998067+0000","last_clean":"2024-09-11T08:53:03.998067+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:53:03.998067+0000","last_undegraded":"2024-09-11T08:53:03.998067+0000","last_fullsized":"2024-09-11T08:53:03.998067+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:02.487657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:02.487657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:07.383897+0000","scrub_duration":0.0011084630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"277'11","reported_seq":510,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:07.839920+0000","last_change":"2024-09-11T08:52:56.987065+0000","last_active":"2024-09-11T08:53:07.839920+0000","last_peered":"2024-09-11T08:53:07.839920+0000","last_clean":"2024-09-11T08:53:07.839920+0000","last_became_active":"2024-09-11T08:52:18.578328+0000","last_became_peered":"2024-09-11T08:52:18.578328+0000","last_unstale":"2024-09-11T08:53:07.839920+0000","last_undegraded":"2024-09-11T08:53:07.839920+0000","last_fullsized":"2024-09-11T08:53:07.839920+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'11","last_scrub_stamp":"2024-09-11T08:52:56.987010+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:52:56.987010+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:03.157069+0000","scrub_duration":0.0033882560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":319,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997873+0000","last_change":"2024-09-11T08:52:57.679482+0000","last_active":"2024-09-11T08:53:03.997873+0000","last_peered":"2024-09-11T08:53:03.997873+0000","last_clean":"2024-09-11T08:53:03.997873+0000","last_became_active":"2024-09-11T08:52:18.576898+0000","last_became_peered":"2024-09-11T08:52:18.576898+0000","last_unstale":"2024-09-11T08:53:03.997873+0000","last_undegraded":"2024-09-11T08:53:03.997873+0000","last_fullsized":"2024-09-11T08:53:03.997873+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:57.679430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:57.679430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:59.791070+0000","scrub_duration":0.0013664560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"300'1181","reported_seq":1196,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.179130+0000","last_change":"2024-09-11T08:52:55.610689+0000","last_active":"2024-09-11T08:53:11.179130+0000","last_peered":"2024-09-11T08:53:11.179130+0000","last_clean":"2024-09-11T08:53:11.179130+0000","last_became_active":"2024-09-11T08:52:55.610383+0000","last_became_peered":"2024-09-11T08:52:55.610383+0000","last_unstale":"2024-09-11T08:53:11.179130+0000","last_undegraded":"2024-09-11T08:53:11.179130+0000","last_fullsized":"2024-09-11T08:53:11.179130+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:17.453638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77398016,"num_objects":1181,"num_object_clones":0,"num_object_copies":3543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1181,"num_write_kb":75584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"199'81","reported_seq":69,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997277+0000","last_change":"2024-09-11T08:53:00.953735+0000","last_active":"2024-09-11T08:53:03.997277+0000","last_peered":"2024-09-11T08:53:03.997277+0000","last_clean":"2024-09-11T08:53:03.997277+0000","last_became_active":"2024-09-11T08:51:03.023365+0000","last_became_peered":"2024-09-11T08:51:03.023365+0000","last_unstale":"2024-09-11T08:53:03.997277+0000","last_undegraded":"2024-09-11T08:53:03.997277+0000","last_fullsized":"2024-09-11T08:53:03.997277+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:00.953685+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:00.953685+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:07.440648+0000","scrub_duration":0.001183323,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"200'38","reported_seq":414,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997233+0000","last_change":"2024-09-11T08:52:18.611214+0000","last_active":"2024-09-11T08:53:03.997233+0000","last_peered":"2024-09-11T08:53:03.997233+0000","last_clean":"2024-09-11T08:53:03.997233+0000","last_became_active":"2024-09-11T08:52:18.595013+0000","last_became_peered":"2024-09-11T08:52:18.595013+0000","last_unstale":"2024-09-11T08:53:03.997233+0000","last_undegraded":"2024-09-11T08:53:03.997233+0000","last_fullsized":"2024-09-11T08:53:03.997233+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'38","last_scrub_stamp":"2024-09-11T08:52:14.460428+0000","last_deep_scrub":"199'26","last_deep_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_clean_scrub_stamp":"2024-09-11T08:52:14.460428+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:35.304464+0000","scrub_duration":0.012981754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":325,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998367+0000","last_change":"2024-09-11T08:52:18.587201+0000","last_active":"2024-09-11T08:53:03.998367+0000","last_peered":"2024-09-11T08:53:03.998367+0000","last_clean":"2024-09-11T08:53:03.998367+0000","last_became_active":"2024-09-11T08:52:18.586084+0000","last_became_peered":"2024-09-11T08:52:18.586084+0000","last_unstale":"2024-09-11T08:53:03.998367+0000","last_undegraded":"2024-09-11T08:53:03.998367+0000","last_fullsized":"2024-09-11T08:53:03.998367+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:13.467288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:13.467288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:29.717990+0000","scrub_duration":0.0046940990000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":299,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996319+0000","last_change":"2024-09-11T08:52:59.933645+0000","last_active":"2024-09-11T08:53:03.996319+0000","last_peered":"2024-09-11T08:53:03.996319+0000","last_clean":"2024-09-11T08:53:03.996319+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:53:03.996319+0000","last_undegraded":"2024-09-11T08:53:03.996319+0000","last_fullsized":"2024-09-11T08:53:03.996319+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:59.933599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:29.687577+0000","last_clean_scrub_stamp":"2024-09-11T08:52:59.933599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:28.572636+0000","scrub_duration":0.001022233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":287,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996553+0000","last_change":"2024-09-11T08:50:19.706569+0000","last_active":"2024-09-11T08:53:03.996553+0000","last_peered":"2024-09-11T08:53:03.996553+0000","last_clean":"2024-09-11T08:53:03.996553+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:53:03.996553+0000","last_undegraded":"2024-09-11T08:53:03.996553+0000","last_fullsized":"2024-09-11T08:53:03.996553+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:19.706509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:19.706509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.045176716999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"300'1340","reported_seq":1354,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.269743+0000","last_change":"2024-09-11T08:52:55.612830+0000","last_active":"2024-09-11T08:53:11.269743+0000","last_peered":"2024-09-11T08:53:11.269743+0000","last_clean":"2024-09-11T08:53:11.269743+0000","last_became_active":"2024-09-11T08:52:55.612351+0000","last_became_peered":"2024-09-11T08:52:55.612351+0000","last_unstale":"2024-09-11T08:53:11.269743+0000","last_undegraded":"2024-09-11T08:53:11.269743+0000","last_fullsized":"2024-09-11T08:53:11.269743+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1340,"log_dups_size":0,"ondisk_log_size":1340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:17.328173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87818240,"num_objects":1340,"num_object_clones":0,"num_object_copies":4020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1340,"num_write_kb":85760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"199'81","reported_seq":74,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:08.418452+0000","last_change":"2024-09-11T08:53:08.418382+0000","last_active":"2024-09-11T08:53:08.418452+0000","last_peered":"2024-09-11T08:53:08.418452+0000","last_clean":"2024-09-11T08:53:08.418452+0000","last_became_active":"2024-09-11T08:51:03.009983+0000","last_became_peered":"2024-09-11T08:51:03.009983+0000","last_unstale":"2024-09-11T08:53:08.418452+0000","last_undegraded":"2024-09-11T08:53:08.418452+0000","last_fullsized":"2024-09-11T08:53:08.418452+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:08.418288+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:08.418288+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:23.484408+0000","scrub_duration":0.003188832,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"200'35","reported_seq":343,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996665+0000","last_change":"2024-09-11T08:52:25.186590+0000","last_active":"2024-09-11T08:53:03.996665+0000","last_peered":"2024-09-11T08:53:03.996665+0000","last_clean":"2024-09-11T08:53:03.996665+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:53:03.996665+0000","last_undegraded":"2024-09-11T08:53:03.996665+0000","last_fullsized":"2024-09-11T08:53:03.996665+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'35","last_scrub_stamp":"2024-09-11T08:52:25.186528+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:52:25.186528+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:40.024410+0000","scrub_duration":0.0018614930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":42,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.179070+0000","last_change":"2024-09-11T08:52:28.487656+0000","last_active":"2024-09-11T08:53:04.179070+0000","last_peered":"2024-09-11T08:53:04.179070+0000","last_clean":"2024-09-11T08:53:04.179070+0000","last_became_active":"2024-09-11T08:52:18.586549+0000","last_became_peered":"2024-09-11T08:52:18.586549+0000","last_unstale":"2024-09-11T08:53:04.179070+0000","last_undegraded":"2024-09-11T08:53:04.179070+0000","last_fullsized":"2024-09-11T08:53:04.179070+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:28.487573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:28.487573+0000","last_clean_scrub_stamp":"2024-09-11T08:52:28.487573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:58.237961+0000","scrub_duration":0.0049500239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":473,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:07.839419+0000","last_change":"2024-09-11T08:51:29.395679+0000","last_active":"2024-09-11T08:53:07.839419+0000","last_peered":"2024-09-11T08:53:07.839419+0000","last_clean":"2024-09-11T08:53:07.839419+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:53:07.839419+0000","last_undegraded":"2024-09-11T08:53:07.839419+0000","last_fullsized":"2024-09-11T08:53:07.839419+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:51:29.395598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:29.395598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041488640000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":200,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.079380+0000","last_change":"2024-09-11T08:52:51.396282+0000","last_active":"2024-09-11T08:53:04.079380+0000","last_peered":"2024-09-11T08:53:04.079380+0000","last_clean":"2024-09-11T08:53:04.079380+0000","last_became_active":"2024-09-11T08:52:38.488567+0000","last_became_peered":"2024-09-11T08:52:38.488567+0000","last_unstale":"2024-09-11T08:53:04.079380+0000","last_undegraded":"2024-09-11T08:53:04.079380+0000","last_fullsized":"2024-09-11T08:53:04.079380+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:51.396225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:51.396225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:52.673765+0000","scrub_duration":0.001381873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"300'1298","reported_seq":1313,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.193671+0000","last_change":"2024-09-11T08:52:56.414997+0000","last_active":"2024-09-11T08:53:11.193671+0000","last_peered":"2024-09-11T08:53:11.193671+0000","last_clean":"2024-09-11T08:53:11.193671+0000","last_became_active":"2024-09-11T08:52:56.414726+0000","last_became_peered":"2024-09-11T08:52:56.414726+0000","last_unstale":"2024-09-11T08:53:11.193671+0000","last_undegraded":"2024-09-11T08:53:11.193671+0000","last_fullsized":"2024-09-11T08:53:11.193671+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1298,"log_dups_size":0,"ondisk_log_size":1298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:17.557033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85065728,"num_objects":1298,"num_object_clones":0,"num_object_copies":3894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1298,"num_write_kb":83072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"199'81","reported_seq":310,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.080151+0000","last_change":"2024-09-11T08:51:22.301835+0000","last_active":"2024-09-11T08:53:04.080151+0000","last_peered":"2024-09-11T08:53:04.080151+0000","last_clean":"2024-09-11T08:53:04.080151+0000","last_became_active":"2024-09-11T08:51:03.003800+0000","last_became_peered":"2024-09-11T08:51:03.003800+0000","last_unstale":"2024-09-11T08:53:04.080151+0000","last_undegraded":"2024-09-11T08:53:04.080151+0000","last_fullsized":"2024-09-11T08:53:04.080151+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:51:22.301752+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:51:22.301752+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061576410000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"200'25","reported_seq":345,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.078521+0000","last_change":"2024-09-11T08:51:31.799902+0000","last_active":"2024-09-11T08:53:04.078521+0000","last_peered":"2024-09-11T08:53:04.078521+0000","last_clean":"2024-09-11T08:53:04.078521+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:53:04.078521+0000","last_undegraded":"2024-09-11T08:53:04.078521+0000","last_fullsized":"2024-09-11T08:53:04.078521+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:51:31.799844+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_clean_scrub_stamp":"2024-09-11T08:51:31.799844+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075128269999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":308,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998352+0000","last_change":"2024-09-11T08:52:55.701978+0000","last_active":"2024-09-11T08:53:03.998352+0000","last_peered":"2024-09-11T08:53:03.998352+0000","last_clean":"2024-09-11T08:53:03.998352+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:53:03.998352+0000","last_undegraded":"2024-09-11T08:53:03.998352+0000","last_fullsized":"2024-09-11T08:53:03.998352+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:55.701902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:55.701902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:21.824877+0000","scrub_duration":0.0016250119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"292'11","reported_seq":181,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:07.839296+0000","last_change":"2024-09-11T08:53:01.380925+0000","last_active":"2024-09-11T08:53:07.839296+0000","last_peered":"2024-09-11T08:53:07.839296+0000","last_clean":"2024-09-11T08:53:07.839296+0000","last_became_active":"2024-09-11T08:52:49.564770+0000","last_became_peered":"2024-09-11T08:52:49.564770+0000","last_unstale":"2024-09-11T08:53:07.839296+0000","last_undegraded":"2024-09-11T08:53:07.839296+0000","last_fullsized":"2024-09-11T08:53:07.839296+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"292'11","last_scrub_stamp":"2024-09-11T08:53:01.380869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:01.380869+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:27.934762+0000","scrub_duration":0.0014786319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":108,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997047+0000","last_change":"2024-09-11T08:52:49.572126+0000","last_active":"2024-09-11T08:53:03.997047+0000","last_peered":"2024-09-11T08:53:03.997047+0000","last_clean":"2024-09-11T08:53:03.997047+0000","last_became_active":"2024-09-11T08:52:49.571683+0000","last_became_peered":"2024-09-11T08:52:49.571683+0000","last_unstale":"2024-09-11T08:53:03.997047+0000","last_undegraded":"2024-09-11T08:53:03.997047+0000","last_fullsized":"2024-09-11T08:53:03.997047+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:20.428724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:51:20.428724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.d","version":"300'1232","reported_seq":1246,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.288518+0000","last_change":"2024-09-11T08:52:56.415069+0000","last_active":"2024-09-11T08:53:11.288518+0000","last_peered":"2024-09-11T08:53:11.288518+0000","last_clean":"2024-09-11T08:53:11.288518+0000","last_became_active":"2024-09-11T08:52:56.414830+0000","last_became_peered":"2024-09-11T08:52:56.414830+0000","last_unstale":"2024-09-11T08:53:11.288518+0000","last_undegraded":"2024-09-11T08:53:11.288518+0000","last_fullsized":"2024-09-11T08:53:11.288518+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1232,"log_dups_size":0,"ondisk_log_size":1232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:09.860125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":80740352,"num_objects":1232,"num_object_clones":0,"num_object_copies":3696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1232,"num_write_kb":78848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"199'81","reported_seq":320,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.080512+0000","last_change":"2024-09-11T08:52:18.585310+0000","last_active":"2024-09-11T08:53:04.080512+0000","last_peered":"2024-09-11T08:53:04.080512+0000","last_clean":"2024-09-11T08:53:04.080512+0000","last_became_active":"2024-09-11T08:52:18.584896+0000","last_became_peered":"2024-09-11T08:52:18.584896+0000","last_unstale":"2024-09-11T08:53:04.080512+0000","last_undegraded":"2024-09-11T08:53:04.080512+0000","last_fullsized":"2024-09-11T08:53:04.080512+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:51:28.247128+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:51:28.247128+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025551829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"200'29","reported_seq":356,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997854+0000","last_change":"2024-09-11T08:52:49.564105+0000","last_active":"2024-09-11T08:53:03.997854+0000","last_peered":"2024-09-11T08:53:03.997854+0000","last_clean":"2024-09-11T08:53:03.997854+0000","last_became_active":"2024-09-11T08:52:49.563364+0000","last_became_peered":"2024-09-11T08:52:49.563364+0000","last_unstale":"2024-09-11T08:53:03.997854+0000","last_undegraded":"2024-09-11T08:53:03.997854+0000","last_fullsized":"2024-09-11T08:53:03.997854+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'29","last_scrub_stamp":"2024-09-11T08:50:06.372228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:06.372228+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050905730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":312,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.178914+0000","last_change":"2024-09-11T08:50:23.612190+0000","last_active":"2024-09-11T08:53:04.178914+0000","last_peered":"2024-09-11T08:53:04.178914+0000","last_clean":"2024-09-11T08:53:04.178914+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:53:04.178914+0000","last_undegraded":"2024-09-11T08:53:04.178914+0000","last_fullsized":"2024-09-11T08:53:04.178914+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:23.612108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:23.612108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044265900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":347,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:09.384179+0000","last_change":"2024-09-11T08:53:09.384145+0000","last_active":"2024-09-11T08:53:09.384179+0000","last_peered":"2024-09-11T08:53:09.384179+0000","last_clean":"2024-09-11T08:53:09.384179+0000","last_became_active":"2024-09-11T08:52:18.573235+0000","last_became_peered":"2024-09-11T08:52:18.573235+0000","last_unstale":"2024-09-11T08:53:09.384179+0000","last_undegraded":"2024-09-11T08:53:09.384179+0000","last_fullsized":"2024-09-11T08:53:09.384179+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:09.384067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_clean_scrub_stamp":"2024-09-11T08:53:09.384067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:28.457187+0000","scrub_duration":0.0035749929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":326,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.078092+0000","last_change":"2024-09-11T08:50:19.562077+0000","last_active":"2024-09-11T08:53:04.078092+0000","last_peered":"2024-09-11T08:53:04.078092+0000","last_clean":"2024-09-11T08:53:04.078092+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:53:04.078092+0000","last_undegraded":"2024-09-11T08:53:04.078092+0000","last_fullsized":"2024-09-11T08:53:04.078092+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:19.561995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:50:19.561995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048084249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"300'1300","reported_seq":1314,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.273739+0000","last_change":"2024-09-11T08:52:56.415080+0000","last_active":"2024-09-11T08:53:11.273739+0000","last_peered":"2024-09-11T08:53:11.273739+0000","last_clean":"2024-09-11T08:53:11.273739+0000","last_became_active":"2024-09-11T08:52:56.414809+0000","last_became_peered":"2024-09-11T08:52:56.414809+0000","last_unstale":"2024-09-11T08:53:11.273739+0000","last_undegraded":"2024-09-11T08:53:11.273739+0000","last_fullsized":"2024-09-11T08:53:11.273739+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1300,"log_dups_size":0,"ondisk_log_size":1300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:06.098380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85196800,"num_objects":1300,"num_object_clones":0,"num_object_copies":3900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1300,"num_write_kb":83200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"199'81","reported_seq":69,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996720+0000","last_change":"2024-09-11T08:52:18.575075+0000","last_active":"2024-09-11T08:53:03.996720+0000","last_peered":"2024-09-11T08:53:03.996720+0000","last_clean":"2024-09-11T08:53:03.996720+0000","last_became_active":"2024-09-11T08:52:18.574468+0000","last_became_peered":"2024-09-11T08:52:18.574468+0000","last_unstale":"2024-09-11T08:53:03.996720+0000","last_undegraded":"2024-09-11T08:53:03.996720+0000","last_fullsized":"2024-09-11T08:53:03.996720+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"200'50","reported_seq":570,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:09.745483+0000","last_change":"2024-09-11T08:53:09.745429+0000","last_active":"2024-09-11T08:53:09.745483+0000","last_peered":"2024-09-11T08:53:09.745483+0000","last_clean":"2024-09-11T08:53:09.745483+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:53:09.745483+0000","last_undegraded":"2024-09-11T08:53:09.745483+0000","last_fullsized":"2024-09-11T08:53:09.745483+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'50","last_scrub_stamp":"2024-09-11T08:53:09.745322+0000","last_deep_scrub":"200'50","last_deep_scrub_stamp":"2024-09-11T08:53:09.745322+0000","last_clean_scrub_stamp":"2024-09-11T08:53:09.745322+0000","objects_scrubbed":10,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:24.698208+0000","scrub_duration":0.0095378719999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":323,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996876+0000","last_change":"2024-09-11T08:52:18.579773+0000","last_active":"2024-09-11T08:53:03.996876+0000","last_peered":"2024-09-11T08:53:03.996876+0000","last_clean":"2024-09-11T08:53:03.996876+0000","last_became_active":"2024-09-11T08:52:18.579509+0000","last_became_peered":"2024-09-11T08:52:18.579509+0000","last_unstale":"2024-09-11T08:53:03.996876+0000","last_undegraded":"2024-09-11T08:53:03.996876+0000","last_fullsized":"2024-09-11T08:53:03.996876+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:33.010134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_clean_scrub_stamp":"2024-09-11T08:51:33.010134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033959530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":343,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997781+0000","last_change":"2024-09-11T08:51:19.322458+0000","last_active":"2024-09-11T08:53:03.997781+0000","last_peered":"2024-09-11T08:53:03.997781+0000","last_clean":"2024-09-11T08:53:03.997781+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:53:03.997781+0000","last_undegraded":"2024-09-11T08:53:03.997781+0000","last_fullsized":"2024-09-11T08:53:03.997781+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:51:19.321966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:51:19.321966+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077550900000000001,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":108,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.178885+0000","last_change":"2024-09-11T08:52:49.569424+0000","last_active":"2024-09-11T08:53:04.178885+0000","last_peered":"2024-09-11T08:53:04.178885+0000","last_clean":"2024-09-11T08:53:04.178885+0000","last_became_active":"2024-09-11T08:52:49.569137+0000","last_became_peered":"2024-09-11T08:52:49.569137+0000","last_unstale":"2024-09-11T08:53:04.178885+0000","last_undegraded":"2024-09-11T08:53:04.178885+0000","last_fullsized":"2024-09-11T08:53:04.178885+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:24.593021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:24.593021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022715140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":296,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.420261+0000","last_change":"2024-09-11T08:53:11.420230+0000","last_active":"2024-09-11T08:53:11.420261+0000","last_peered":"2024-09-11T08:53:11.420261+0000","last_clean":"2024-09-11T08:53:11.420261+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:53:11.420261+0000","last_undegraded":"2024-09-11T08:53:11.420261+0000","last_fullsized":"2024-09-11T08:53:11.420261+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:11.420182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:11.420182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:27.596432+0000","scrub_duration":0.0034052280000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"300'1311","reported_seq":1326,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.517280+0000","last_change":"2024-09-11T08:52:56.413844+0000","last_active":"2024-09-11T08:53:11.517280+0000","last_peered":"2024-09-11T08:53:11.517280+0000","last_clean":"2024-09-11T08:53:11.517280+0000","last_became_active":"2024-09-11T08:52:56.413594+0000","last_became_peered":"2024-09-11T08:52:56.413594+0000","last_unstale":"2024-09-11T08:53:11.517280+0000","last_undegraded":"2024-09-11T08:53:11.517280+0000","last_fullsized":"2024-09-11T08:53:11.517280+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1311,"log_dups_size":0,"ondisk_log_size":1311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:17.749632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85917696,"num_objects":1311,"num_object_clones":0,"num_object_copies":3933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1311,"num_write_kb":83904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"199'81","reported_seq":348,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996317+0000","last_change":"2024-09-11T08:52:49.570289+0000","last_active":"2024-09-11T08:53:03.996317+0000","last_peered":"2024-09-11T08:53:03.996317+0000","last_clean":"2024-09-11T08:53:03.996317+0000","last_became_active":"2024-09-11T08:52:49.569934+0000","last_became_peered":"2024-09-11T08:52:49.569934+0000","last_unstale":"2024-09-11T08:53:03.996317+0000","last_undegraded":"2024-09-11T08:53:03.996317+0000","last_fullsized":"2024-09-11T08:53:03.996317+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"200'17","reported_seq":387,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.178706+0000","last_change":"2024-09-11T08:52:18.598436+0000","last_active":"2024-09-11T08:53:04.178706+0000","last_peered":"2024-09-11T08:53:04.178706+0000","last_clean":"2024-09-11T08:53:04.178706+0000","last_became_active":"2024-09-11T08:52:18.580008+0000","last_became_peered":"2024-09-11T08:52:18.580008+0000","last_unstale":"2024-09-11T08:53:04.178706+0000","last_undegraded":"2024-09-11T08:53:04.178706+0000","last_fullsized":"2024-09-11T08:53:04.178706+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'17","last_scrub_stamp":"2024-09-11T08:50:15.565290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:15.565290+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060447950000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":338,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.078000+0000","last_change":"2024-09-11T08:52:52.843079+0000","last_active":"2024-09-11T08:53:04.078000+0000","last_peered":"2024-09-11T08:53:04.078000+0000","last_clean":"2024-09-11T08:53:04.078000+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:53:04.078000+0000","last_undegraded":"2024-09-11T08:53:04.078000+0000","last_fullsized":"2024-09-11T08:53:04.078000+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:52.843019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:52.843019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:12.212376+0000","scrub_duration":0.001298305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"263'2","reported_seq":383,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:08.773166+0000","last_change":"2024-09-11T08:53:08.773090+0000","last_active":"2024-09-11T08:53:08.773166+0000","last_peered":"2024-09-11T08:53:08.773166+0000","last_clean":"2024-09-11T08:53:08.773166+0000","last_became_active":"2024-09-11T08:52:18.586545+0000","last_became_peered":"2024-09-11T08:52:18.586545+0000","last_unstale":"2024-09-11T08:53:08.773166+0000","last_undegraded":"2024-09-11T08:53:08.773166+0000","last_fullsized":"2024-09-11T08:53:08.773166+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'2","last_scrub_stamp":"2024-09-11T08:53:08.772979+0000","last_deep_scrub":"263'2","last_deep_scrub_stamp":"2024-09-11T08:53:08.772979+0000","last_clean_scrub_stamp":"2024-09-11T08:53:08.772979+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:16.601208+0000","scrub_duration":0.0027552229999999998,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":333,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.202506+0000","last_change":"2024-09-11T08:52:16.023858+0000","last_active":"2024-09-11T08:53:04.202506+0000","last_peered":"2024-09-11T08:53:04.202506+0000","last_clean":"2024-09-11T08:53:04.202506+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:53:04.202506+0000","last_undegraded":"2024-09-11T08:53:04.202506+0000","last_fullsized":"2024-09-11T08:53:04.202506+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:16.023763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_clean_scrub_stamp":"2024-09-11T08:52:16.023763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:40.228929+0000","scrub_duration":0.0032804420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":190,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.080091+0000","last_change":"2024-09-11T08:52:38.486638+0000","last_active":"2024-09-11T08:53:04.080091+0000","last_peered":"2024-09-11T08:53:04.080091+0000","last_clean":"2024-09-11T08:53:04.080091+0000","last_became_active":"2024-09-11T08:52:38.484800+0000","last_became_peered":"2024-09-11T08:52:38.484800+0000","last_unstale":"2024-09-11T08:53:04.080091+0000","last_undegraded":"2024-09-11T08:53:04.080091+0000","last_fullsized":"2024-09-11T08:53:04.080091+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:31.465384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:51:31.465384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015184790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"300'1346","reported_seq":1360,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.410506+0000","last_change":"2024-09-11T08:52:56.414016+0000","last_active":"2024-09-11T08:53:11.410506+0000","last_peered":"2024-09-11T08:53:11.410506+0000","last_clean":"2024-09-11T08:53:11.410506+0000","last_became_active":"2024-09-11T08:52:56.413725+0000","last_became_peered":"2024-09-11T08:52:56.413725+0000","last_unstale":"2024-09-11T08:53:11.410506+0000","last_undegraded":"2024-09-11T08:53:11.410506+0000","last_fullsized":"2024-09-11T08:53:11.410506+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1346,"log_dups_size":0,"ondisk_log_size":1346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:04.942971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88211456,"num_objects":1346,"num_object_clones":0,"num_object_copies":4038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":86144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"199'81","reported_seq":332,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.078508+0000","last_change":"2024-09-11T08:53:01.761090+0000","last_active":"2024-09-11T08:53:04.078508+0000","last_peered":"2024-09-11T08:53:04.078508+0000","last_clean":"2024-09-11T08:53:04.078508+0000","last_became_active":"2024-09-11T08:51:03.027281+0000","last_became_peered":"2024-09-11T08:51:03.027281+0000","last_unstale":"2024-09-11T08:53:04.078508+0000","last_undegraded":"2024-09-11T08:53:04.078508+0000","last_fullsized":"2024-09-11T08:53:04.078508+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:01.760951+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:01.760951+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:19.773549+0000","scrub_duration":0.0017313369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"200'24","reported_seq":56,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.178838+0000","last_change":"2024-09-11T08:52:52.548054+0000","last_active":"2024-09-11T08:53:04.178838+0000","last_peered":"2024-09-11T08:53:04.178838+0000","last_clean":"2024-09-11T08:53:04.178838+0000","last_became_active":"2024-09-11T08:52:18.580950+0000","last_became_peered":"2024-09-11T08:52:18.580950+0000","last_unstale":"2024-09-11T08:53:04.178838+0000","last_undegraded":"2024-09-11T08:53:04.178838+0000","last_fullsized":"2024-09-11T08:53:04.178838+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'24","last_scrub_stamp":"2024-09-11T08:52:52.547998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:52:52.547998+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:14.066702+0000","scrub_duration":0.001854198,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":108,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996366+0000","last_change":"2024-09-11T08:52:49.566726+0000","last_active":"2024-09-11T08:53:03.996366+0000","last_peered":"2024-09-11T08:53:03.996366+0000","last_clean":"2024-09-11T08:53:03.996366+0000","last_became_active":"2024-09-11T08:52:49.566437+0000","last_became_peered":"2024-09-11T08:52:49.566437+0000","last_unstale":"2024-09-11T08:53:03.996366+0000","last_undegraded":"2024-09-11T08:53:03.996366+0000","last_fullsized":"2024-09-11T08:53:03.996366+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:12.796804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:12.796804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":97,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:10.534451+0000","last_change":"2024-09-11T08:52:59.810864+0000","last_active":"2024-09-11T08:53:10.534451+0000","last_peered":"2024-09-11T08:53:10.534451+0000","last_clean":"2024-09-11T08:53:10.534451+0000","last_became_active":"2024-09-11T08:52:38.484664+0000","last_became_peered":"2024-09-11T08:52:38.484664+0000","last_unstale":"2024-09-11T08:53:10.534451+0000","last_undegraded":"2024-09-11T08:53:10.534451+0000","last_fullsized":"2024-09-11T08:53:10.534451+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:52:59.810791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:59.810791+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:05.917891+0000","scrub_duration":0.0017088629999999999,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":40,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":324,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998339+0000","last_change":"2024-09-11T08:52:18.578912+0000","last_active":"2024-09-11T08:53:03.998339+0000","last_peered":"2024-09-11T08:53:03.998339+0000","last_clean":"2024-09-11T08:53:03.998339+0000","last_became_active":"2024-09-11T08:52:18.578530+0000","last_became_peered":"2024-09-11T08:52:18.578530+0000","last_unstale":"2024-09-11T08:53:03.998339+0000","last_undegraded":"2024-09-11T08:53:03.998339+0000","last_fullsized":"2024-09-11T08:53:03.998339+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:32.443727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:32.443727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056199379999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":306,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.078618+0000","last_change":"2024-09-11T08:51:35.779870+0000","last_active":"2024-09-11T08:53:04.078618+0000","last_peered":"2024-09-11T08:53:04.078618+0000","last_clean":"2024-09-11T08:53:04.078618+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:53:04.078618+0000","last_undegraded":"2024-09-11T08:53:04.078618+0000","last_fullsized":"2024-09-11T08:53:04.078618+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:35.779796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:51:35.779796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048768270000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"300'1324","reported_seq":1338,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.281750+0000","last_change":"2024-09-11T08:52:56.413621+0000","last_active":"2024-09-11T08:53:11.281750+0000","last_peered":"2024-09-11T08:53:11.281750+0000","last_clean":"2024-09-11T08:53:11.281750+0000","last_became_active":"2024-09-11T08:52:56.413344+0000","last_became_peered":"2024-09-11T08:52:56.413344+0000","last_unstale":"2024-09-11T08:53:11.281750+0000","last_undegraded":"2024-09-11T08:53:11.281750+0000","last_fullsized":"2024-09-11T08:53:11.281750+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1324,"log_dups_size":0,"ondisk_log_size":1324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:10.229971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86769664,"num_objects":1324,"num_object_clones":0,"num_object_copies":3972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1324,"num_write_kb":84736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"201'27","reported_seq":216,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.080210+0000","last_change":"2024-09-11T08:52:18.228592+0000","last_active":"2024-09-11T08:53:04.080210+0000","last_peered":"2024-09-11T08:53:04.080210+0000","last_clean":"2024-09-11T08:53:04.080210+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:53:04.080210+0000","last_undegraded":"2024-09-11T08:53:04.080210+0000","last_fullsized":"2024-09-11T08:53:04.080210+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'27","last_scrub_stamp":"2024-09-11T08:52:18.228537+0000","last_deep_scrub":"201'27","last_deep_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_clean_scrub_stamp":"2024-09-11T08:52:18.228537+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:27.083005+0000","scrub_duration":0.0018286559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":301,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997589+0000","last_change":"2024-09-11T08:50:08.276283+0000","last_active":"2024-09-11T08:53:03.997589+0000","last_peered":"2024-09-11T08:53:03.997589+0000","last_clean":"2024-09-11T08:53:03.997589+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:53:03.997589+0000","last_undegraded":"2024-09-11T08:53:03.997589+0000","last_fullsized":"2024-09-11T08:53:03.997589+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:50:08.276232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:08.276232+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018088246999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"199'81","reported_seq":260,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.079003+0000","last_change":"2024-09-11T08:52:49.565423+0000","last_active":"2024-09-11T08:53:04.079003+0000","last_peered":"2024-09-11T08:53:04.079003+0000","last_clean":"2024-09-11T08:53:04.079003+0000","last_became_active":"2024-09-11T08:52:49.565126+0000","last_became_peered":"2024-09-11T08:52:49.565126+0000","last_unstale":"2024-09-11T08:53:04.079003+0000","last_undegraded":"2024-09-11T08:53:04.079003+0000","last_fullsized":"2024-09-11T08:53:04.079003+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002308867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":35,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.079094+0000","last_change":"2024-09-11T08:52:53.870726+0000","last_active":"2024-09-11T08:53:04.079094+0000","last_peered":"2024-09-11T08:53:04.079094+0000","last_clean":"2024-09-11T08:53:04.079094+0000","last_became_active":"2024-09-11T08:52:38.485446+0000","last_became_peered":"2024-09-11T08:52:38.485446+0000","last_unstale":"2024-09-11T08:53:04.079094+0000","last_undegraded":"2024-09-11T08:53:04.079094+0000","last_fullsized":"2024-09-11T08:53:04.079094+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:53.870644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:53.870644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:19.224184+0000","scrub_duration":0.001623457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":302,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998287+0000","last_change":"2024-09-11T08:50:13.347446+0000","last_active":"2024-09-11T08:53:03.998287+0000","last_peered":"2024-09-11T08:53:03.998287+0000","last_clean":"2024-09-11T08:53:03.998287+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:53:03.998287+0000","last_undegraded":"2024-09-11T08:53:03.998287+0000","last_fullsized":"2024-09-11T08:53:03.998287+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:13.347374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:13.347374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062539859999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"300'1362","reported_seq":1376,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.282975+0000","last_change":"2024-09-11T08:52:55.612239+0000","last_active":"2024-09-11T08:53:11.282975+0000","last_peered":"2024-09-11T08:53:11.282975+0000","last_clean":"2024-09-11T08:53:11.282975+0000","last_became_active":"2024-09-11T08:52:55.611835+0000","last_became_peered":"2024-09-11T08:52:55.611835+0000","last_unstale":"2024-09-11T08:53:11.282975+0000","last_undegraded":"2024-09-11T08:53:11.282975+0000","last_fullsized":"2024-09-11T08:53:11.282975+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:18.573270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89260032,"num_objects":1362,"num_object_clones":0,"num_object_copies":4086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1362,"num_write_kb":87168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"200'19","reported_seq":391,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997394+0000","last_change":"2024-09-11T08:51:21.979895+0000","last_active":"2024-09-11T08:53:03.997394+0000","last_peered":"2024-09-11T08:53:03.997394+0000","last_clean":"2024-09-11T08:53:03.997394+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:53:03.997394+0000","last_undegraded":"2024-09-11T08:53:03.997394+0000","last_fullsized":"2024-09-11T08:53:03.997394+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:21.979796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:21.979796+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045321210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":293,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996571+0000","last_change":"2024-09-11T08:51:38.083246+0000","last_active":"2024-09-11T08:53:03.996571+0000","last_peered":"2024-09-11T08:53:03.996571+0000","last_clean":"2024-09-11T08:53:03.996571+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:53:03.996571+0000","last_undegraded":"2024-09-11T08:53:03.996571+0000","last_fullsized":"2024-09-11T08:53:03.996571+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:38.083168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:51:38.083168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029716930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":311,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998225+0000","last_change":"2024-09-11T08:52:56.675410+0000","last_active":"2024-09-11T08:53:03.998225+0000","last_peered":"2024-09-11T08:53:03.998225+0000","last_clean":"2024-09-11T08:53:03.998225+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:53:03.998225+0000","last_undegraded":"2024-09-11T08:53:03.998225+0000","last_fullsized":"2024-09-11T08:53:03.998225+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:56.675326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:52:56.675326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:04.423103+0000","scrub_duration":0.0033910770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":306,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997703+0000","last_change":"2024-09-11T08:50:06.234445+0000","last_active":"2024-09-11T08:53:03.997703+0000","last_peered":"2024-09-11T08:53:03.997703+0000","last_clean":"2024-09-11T08:53:03.997703+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:53:03.997703+0000","last_undegraded":"2024-09-11T08:53:03.997703+0000","last_fullsized":"2024-09-11T08:53:03.997703+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:06.234119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:06.234119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034529980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"300'1298","reported_seq":1312,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.410569+0000","last_change":"2024-09-11T08:52:55.611140+0000","last_active":"2024-09-11T08:53:11.410569+0000","last_peered":"2024-09-11T08:53:11.410569+0000","last_clean":"2024-09-11T08:53:11.410569+0000","last_became_active":"2024-09-11T08:52:55.610852+0000","last_became_peered":"2024-09-11T08:52:55.610852+0000","last_unstale":"2024-09-11T08:53:11.410569+0000","last_undegraded":"2024-09-11T08:53:11.410569+0000","last_fullsized":"2024-09-11T08:53:11.410569+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1298,"log_dups_size":0,"ondisk_log_size":1298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:11.182978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85065728,"num_objects":1298,"num_object_clones":0,"num_object_copies":3894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1298,"num_write_kb":83072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"199'81","reported_seq":66,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997018+0000","last_change":"2024-09-11T08:51:03.042396+0000","last_active":"2024-09-11T08:53:03.997018+0000","last_peered":"2024-09-11T08:53:03.997018+0000","last_clean":"2024-09-11T08:53:03.997018+0000","last_became_active":"2024-09-11T08:51:03.020941+0000","last_became_peered":"2024-09-11T08:51:03.020941+0000","last_unstale":"2024-09-11T08:53:03.997018+0000","last_undegraded":"2024-09-11T08:53:03.997018+0000","last_fullsized":"2024-09-11T08:53:03.997018+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":316358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"200'31","reported_seq":350,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997109+0000","last_change":"2024-09-11T08:51:24.989802+0000","last_active":"2024-09-11T08:53:03.997109+0000","last_peered":"2024-09-11T08:53:03.997109+0000","last_clean":"2024-09-11T08:53:03.997109+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:53:03.997109+0000","last_undegraded":"2024-09-11T08:53:03.997109+0000","last_fullsized":"2024-09-11T08:53:03.997109+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'31","last_scrub_stamp":"2024-09-11T08:51:24.989715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:24.989715+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037630540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":341,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:10.339185+0000","last_change":"2024-09-11T08:53:10.339152+0000","last_active":"2024-09-11T08:53:10.339185+0000","last_peered":"2024-09-11T08:53:10.339185+0000","last_clean":"2024-09-11T08:53:10.339185+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:53:10.339185+0000","last_undegraded":"2024-09-11T08:53:10.339185+0000","last_fullsized":"2024-09-11T08:53:10.339185+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:10.339096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_clean_scrub_stamp":"2024-09-11T08:53:10.339096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:31.094115+0000","scrub_duration":0.0027644290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":328,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.178670+0000","last_change":"2024-09-11T08:52:18.577488+0000","last_active":"2024-09-11T08:53:04.178670+0000","last_peered":"2024-09-11T08:53:04.178670+0000","last_clean":"2024-09-11T08:53:04.178670+0000","last_became_active":"2024-09-11T08:52:18.577201+0000","last_became_peered":"2024-09-11T08:52:18.577201+0000","last_unstale":"2024-09-11T08:53:04.178670+0000","last_undegraded":"2024-09-11T08:53:04.178670+0000","last_fullsized":"2024-09-11T08:53:04.178670+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:22.782448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:22.782448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071094909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":344,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:07.808293+0000","last_change":"2024-09-11T08:53:07.808224+0000","last_active":"2024-09-11T08:53:07.808293+0000","last_peered":"2024-09-11T08:53:07.808293+0000","last_clean":"2024-09-11T08:53:07.808293+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:53:07.808293+0000","last_undegraded":"2024-09-11T08:53:07.808293+0000","last_fullsized":"2024-09-11T08:53:07.808293+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:07.808138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:07.808138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:18.430717+0000","scrub_duration":0.003891982,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"300'1405","reported_seq":1419,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.789155+0000","last_change":"2024-09-11T08:52:56.415049+0000","last_active":"2024-09-11T08:53:11.789155+0000","last_peered":"2024-09-11T08:53:11.789155+0000","last_clean":"2024-09-11T08:53:11.789155+0000","last_became_active":"2024-09-11T08:52:56.414765+0000","last_became_peered":"2024-09-11T08:52:56.414765+0000","last_unstale":"2024-09-11T08:53:11.789155+0000","last_undegraded":"2024-09-11T08:53:11.789155+0000","last_fullsized":"2024-09-11T08:53:11.789155+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1405,"log_dups_size":0,"ondisk_log_size":1405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:10.761677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92078080,"num_objects":1405,"num_object_clones":0,"num_object_copies":4215,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1405,"num_write_kb":89920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"199'81","reported_seq":352,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997290+0000","last_change":"2024-09-11T08:52:38.482411+0000","last_active":"2024-09-11T08:53:03.997290+0000","last_peered":"2024-09-11T08:53:03.997290+0000","last_clean":"2024-09-11T08:53:03.997290+0000","last_became_active":"2024-09-11T08:52:38.481475+0000","last_became_peered":"2024-09-11T08:52:38.481475+0000","last_unstale":"2024-09-11T08:53:03.997290+0000","last_undegraded":"2024-09-11T08:53:03.997290+0000","last_fullsized":"2024-09-11T08:53:03.997290+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"200'20","reported_seq":371,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997031+0000","last_change":"2024-09-11T08:52:18.090235+0000","last_active":"2024-09-11T08:53:03.997031+0000","last_peered":"2024-09-11T08:53:03.997031+0000","last_clean":"2024-09-11T08:53:03.997031+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:53:03.997031+0000","last_undegraded":"2024-09-11T08:53:03.997031+0000","last_fullsized":"2024-09-11T08:53:03.997031+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'20","last_scrub_stamp":"2024-09-11T08:52:18.090140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:52:18.090140+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:36.215148+0000","scrub_duration":0.0018601010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":312,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997714+0000","last_change":"2024-09-11T08:52:18.586953+0000","last_active":"2024-09-11T08:53:03.997714+0000","last_peered":"2024-09-11T08:53:03.997714+0000","last_clean":"2024-09-11T08:53:03.997714+0000","last_became_active":"2024-09-11T08:52:18.586343+0000","last_became_peered":"2024-09-11T08:52:18.586343+0000","last_unstale":"2024-09-11T08:53:03.997714+0000","last_undegraded":"2024-09-11T08:53:03.997714+0000","last_fullsized":"2024-09-11T08:53:03.997714+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:04.191702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:04.191702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034932320000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"299'9","reported_seq":523,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:07.839844+0000","last_change":"2024-09-11T08:53:05.835268+0000","last_active":"2024-09-11T08:53:07.839844+0000","last_peered":"2024-09-11T08:53:07.839844+0000","last_clean":"2024-09-11T08:53:07.839844+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:53:07.839844+0000","last_undegraded":"2024-09-11T08:53:07.839844+0000","last_fullsized":"2024-09-11T08:53:07.839844+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'9","last_scrub_stamp":"2024-09-11T08:53:05.835177+0000","last_deep_scrub":"299'9","last_deep_scrub_stamp":"2024-09-11T08:53:05.835177+0000","last_clean_scrub_stamp":"2024-09-11T08:53:05.835177+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:24.501292+0000","scrub_duration":0.0089053550000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":297,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:07.008340+0000","last_change":"2024-09-11T08:53:07.008184+0000","last_active":"2024-09-11T08:53:07.008340+0000","last_peered":"2024-09-11T08:53:07.008340+0000","last_clean":"2024-09-11T08:53:07.008340+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:53:07.008340+0000","last_undegraded":"2024-09-11T08:53:07.008340+0000","last_fullsized":"2024-09-11T08:53:07.008340+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:07.008088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:53:07.008088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:15.188670+0000","scrub_duration":0.0041216330000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"300'1385","reported_seq":1399,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.403610+0000","last_change":"2024-09-11T08:52:56.413212+0000","last_active":"2024-09-11T08:53:11.403610+0000","last_peered":"2024-09-11T08:53:11.403610+0000","last_clean":"2024-09-11T08:53:11.403610+0000","last_became_active":"2024-09-11T08:52:56.412642+0000","last_became_peered":"2024-09-11T08:52:56.412642+0000","last_unstale":"2024-09-11T08:53:11.403610+0000","last_undegraded":"2024-09-11T08:53:11.403610+0000","last_fullsized":"2024-09-11T08:53:11.403610+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1385,"log_dups_size":0,"ondisk_log_size":1385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:59.474410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90767360,"num_objects":1385,"num_object_clones":0,"num_object_copies":4155,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1385,"num_write_kb":88640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"200'15","reported_seq":52,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.179423+0000","last_change":"2024-09-11T08:52:26.532447+0000","last_active":"2024-09-11T08:53:04.179423+0000","last_peered":"2024-09-11T08:53:04.179423+0000","last_clean":"2024-09-11T08:53:04.179423+0000","last_became_active":"2024-09-11T08:52:18.588329+0000","last_became_peered":"2024-09-11T08:52:18.588329+0000","last_unstale":"2024-09-11T08:53:04.179423+0000","last_undegraded":"2024-09-11T08:53:04.179423+0000","last_fullsized":"2024-09-11T08:53:04.179423+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:52:26.532385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:52:26.532385+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:41.095587+0000","scrub_duration":0.003040849,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":305,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998100+0000","last_change":"2024-09-11T08:49:30.234576+0000","last_active":"2024-09-11T08:53:03.998100+0000","last_peered":"2024-09-11T08:53:03.998100+0000","last_clean":"2024-09-11T08:53:03.998100+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:53:03.998100+0000","last_undegraded":"2024-09-11T08:53:03.998100+0000","last_fullsized":"2024-09-11T08:53:03.998100+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:49:30.234078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:49:30.234078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025717829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":331,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.202035+0000","last_change":"2024-09-11T08:53:00.940980+0000","last_active":"2024-09-11T08:53:04.202035+0000","last_peered":"2024-09-11T08:53:04.202035+0000","last_clean":"2024-09-11T08:53:04.202035+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:53:04.202035+0000","last_undegraded":"2024-09-11T08:53:04.202035+0000","last_fullsized":"2024-09-11T08:53:04.202035+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:00.940914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:00.940914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:04.012934+0000","scrub_duration":0.0014530680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":25,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.078939+0000","last_change":"2024-09-11T08:52:49.570019+0000","last_active":"2024-09-11T08:53:04.078939+0000","last_peered":"2024-09-11T08:53:04.078939+0000","last_clean":"2024-09-11T08:53:04.078939+0000","last_became_active":"2024-09-11T08:52:49.569749+0000","last_became_peered":"2024-09-11T08:52:49.569749+0000","last_unstale":"2024-09-11T08:53:04.078939+0000","last_undegraded":"2024-09-11T08:53:04.078939+0000","last_fullsized":"2024-09-11T08:53:04.078939+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:14.645975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:14.645975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:26.300616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"300'1229","reported_seq":1242,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.193686+0000","last_change":"2024-09-11T08:52:56.413137+0000","last_active":"2024-09-11T08:53:11.193686+0000","last_peered":"2024-09-11T08:53:11.193686+0000","last_clean":"2024-09-11T08:53:11.193686+0000","last_became_active":"2024-09-11T08:52:56.412780+0000","last_became_peered":"2024-09-11T08:52:56.412780+0000","last_unstale":"2024-09-11T08:53:11.193686+0000","last_undegraded":"2024-09-11T08:53:11.193686+0000","last_fullsized":"2024-09-11T08:53:11.193686+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:08.686334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":80543744,"num_objects":1229,"num_object_clones":0,"num_object_copies":3687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1229,"num_write_kb":78656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"200'35","reported_seq":218,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.079058+0000","last_change":"2024-09-11T08:51:35.255093+0000","last_active":"2024-09-11T08:53:04.079058+0000","last_peered":"2024-09-11T08:53:04.079058+0000","last_clean":"2024-09-11T08:53:04.079058+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:53:04.079058+0000","last_undegraded":"2024-09-11T08:53:04.079058+0000","last_fullsized":"2024-09-11T08:53:04.079058+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'35","last_scrub_stamp":"2024-09-11T08:51:35.255013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:35.255013+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048258839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":26,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998098+0000","last_change":"2024-09-11T08:52:38.485375+0000","last_active":"2024-09-11T08:53:03.998098+0000","last_peered":"2024-09-11T08:53:03.998098+0000","last_clean":"2024-09-11T08:53:03.998098+0000","last_became_active":"2024-09-11T08:52:38.484857+0000","last_became_peered":"2024-09-11T08:52:38.484857+0000","last_unstale":"2024-09-11T08:53:03.998098+0000","last_undegraded":"2024-09-11T08:53:03.998098+0000","last_fullsized":"2024-09-11T08:53:03.998098+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:16.196521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:16.196521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":360,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.202247+0000","last_change":"2024-09-11T08:53:01.907502+0000","last_active":"2024-09-11T08:53:04.202247+0000","last_peered":"2024-09-11T08:53:04.202247+0000","last_clean":"2024-09-11T08:53:04.202247+0000","last_became_active":"2024-09-11T08:52:18.584748+0000","last_became_peered":"2024-09-11T08:52:18.584748+0000","last_unstale":"2024-09-11T08:53:04.202247+0000","last_undegraded":"2024-09-11T08:53:04.202247+0000","last_fullsized":"2024-09-11T08:53:04.202247+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:01.907441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:35.022256+0000","last_clean_scrub_stamp":"2024-09-11T08:53:01.907441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:28.038717+0000","scrub_duration":0.001627903,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":35,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.078463+0000","last_change":"2024-09-11T08:52:59.930178+0000","last_active":"2024-09-11T08:53:04.078463+0000","last_peered":"2024-09-11T08:53:04.078463+0000","last_clean":"2024-09-11T08:53:04.078463+0000","last_became_active":"2024-09-11T08:52:38.480684+0000","last_became_peered":"2024-09-11T08:52:38.480684+0000","last_unstale":"2024-09-11T08:53:04.078463+0000","last_undegraded":"2024-09-11T08:53:04.078463+0000","last_fullsized":"2024-09-11T08:53:04.078463+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:59.930119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:59.930119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:09.598207+0000","scrub_duration":0.0027767069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.7","version":"300'1374","reported_seq":1388,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.289653+0000","last_change":"2024-09-11T08:52:55.613497+0000","last_active":"2024-09-11T08:53:11.289653+0000","last_peered":"2024-09-11T08:53:11.289653+0000","last_clean":"2024-09-11T08:53:11.289653+0000","last_became_active":"2024-09-11T08:52:55.613121+0000","last_became_peered":"2024-09-11T08:52:55.613121+0000","last_unstale":"2024-09-11T08:53:11.289653+0000","last_undegraded":"2024-09-11T08:53:11.289653+0000","last_fullsized":"2024-09-11T08:53:11.289653+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1374,"log_dups_size":0,"ondisk_log_size":1374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:07.376802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90046464,"num_objects":1374,"num_object_clones":0,"num_object_copies":4122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1374,"num_write_kb":87936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"200'18","reported_seq":323,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996402+0000","last_change":"2024-09-11T08:50:03.790643+0000","last_active":"2024-09-11T08:53:03.996402+0000","last_peered":"2024-09-11T08:53:03.996402+0000","last_clean":"2024-09-11T08:53:03.996402+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:53:03.996402+0000","last_undegraded":"2024-09-11T08:53:03.996402+0000","last_fullsized":"2024-09-11T08:53:03.996402+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'18","last_scrub_stamp":"2024-09-11T08:50:03.790561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:03.790561+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.050104567000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997460+0000","last_change":"2024-09-11T08:52:38.488564+0000","last_active":"2024-09-11T08:53:03.997460+0000","last_peered":"2024-09-11T08:53:03.997460+0000","last_clean":"2024-09-11T08:53:03.997460+0000","last_became_active":"2024-09-11T08:52:38.487881+0000","last_became_peered":"2024-09-11T08:52:38.487881+0000","last_unstale":"2024-09-11T08:53:03.997460+0000","last_undegraded":"2024-09-11T08:53:03.997460+0000","last_fullsized":"2024-09-11T08:53:03.997460+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:24.331966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:24.331966+0000","last_clean_scrub_stamp":"2024-09-11T08:51:24.331966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":350,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.078558+0000","last_change":"2024-09-11T08:52:18.995061+0000","last_active":"2024-09-11T08:53:04.078558+0000","last_peered":"2024-09-11T08:53:04.078558+0000","last_clean":"2024-09-11T08:53:04.078558+0000","last_became_active":"2024-09-11T08:52:18.585136+0000","last_became_peered":"2024-09-11T08:52:18.585136+0000","last_unstale":"2024-09-11T08:53:04.078558+0000","last_undegraded":"2024-09-11T08:53:04.078558+0000","last_fullsized":"2024-09-11T08:53:04.078558+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:18.995003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:18.995003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:31.559060+0000","scrub_duration":0.0012036359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":286,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997506+0000","last_change":"2024-09-11T08:50:15.277171+0000","last_active":"2024-09-11T08:53:03.997506+0000","last_peered":"2024-09-11T08:53:03.997506+0000","last_clean":"2024-09-11T08:53:03.997506+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:53:03.997506+0000","last_undegraded":"2024-09-11T08:53:03.997506+0000","last_fullsized":"2024-09-11T08:53:03.997506+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:15.277094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:50:15.277094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004775751,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"300'1279","reported_seq":1293,"reported_epoch":300,"state":"active+clean","last_fresh":"2024-09-11T08:53:11.528141+0000","last_change":"2024-09-11T08:52:56.413103+0000","last_active":"2024-09-11T08:53:11.528141+0000","last_peered":"2024-09-11T08:53:11.528141+0000","last_clean":"2024-09-11T08:53:11.528141+0000","last_became_active":"2024-09-11T08:52:56.412787+0000","last_became_peered":"2024-09-11T08:52:56.412787+0000","last_unstale":"2024-09-11T08:53:11.528141+0000","last_undegraded":"2024-09-11T08:53:11.528141+0000","last_fullsized":"2024-09-11T08:53:11.528141+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":1279,"log_dups_size":0,"ondisk_log_size":1279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:10.489342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83820544,"num_objects":1279,"num_object_clones":0,"num_object_copies":3837,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1279,"num_write_kb":81856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"200'34","reported_seq":408,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997174+0000","last_change":"2024-09-11T08:50:11.325671+0000","last_active":"2024-09-11T08:53:03.997174+0000","last_peered":"2024-09-11T08:53:03.997174+0000","last_clean":"2024-09-11T08:53:03.997174+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:53:03.997174+0000","last_undegraded":"2024-09-11T08:53:03.997174+0000","last_fullsized":"2024-09-11T08:53:03.997174+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:50:11.325616+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:50:11.325616+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023307359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":45,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997397+0000","last_change":"2024-09-11T08:52:54.988951+0000","last_active":"2024-09-11T08:53:03.997397+0000","last_peered":"2024-09-11T08:53:03.997397+0000","last_clean":"2024-09-11T08:53:03.997397+0000","last_became_active":"2024-09-11T08:52:38.484199+0000","last_became_peered":"2024-09-11T08:52:38.484199+0000","last_unstale":"2024-09-11T08:53:03.997397+0000","last_undegraded":"2024-09-11T08:53:03.997397+0000","last_fullsized":"2024-09-11T08:53:03.997397+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:52:54.988868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.988868+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:06.018653+0000","scrub_duration":0.0019261269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"287'11","reported_seq":299,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:07.839491+0000","last_change":"2024-09-11T08:52:38.489283+0000","last_active":"2024-09-11T08:53:07.839491+0000","last_peered":"2024-09-11T08:53:07.839491+0000","last_clean":"2024-09-11T08:53:07.839491+0000","last_became_active":"2024-09-11T08:52:38.488856+0000","last_became_peered":"2024-09-11T08:52:38.488856+0000","last_unstale":"2024-09-11T08:53:07.839491+0000","last_undegraded":"2024-09-11T08:53:07.839491+0000","last_fullsized":"2024-09-11T08:53:07.839491+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"168'10","last_scrub_stamp":"2024-09-11T08:51:35.452290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:35.452290+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001669032,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":322,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997592+0000","last_change":"2024-09-11T08:52:26.813313+0000","last_active":"2024-09-11T08:53:03.997592+0000","last_peered":"2024-09-11T08:53:03.997592+0000","last_clean":"2024-09-11T08:53:03.997592+0000","last_became_active":"2024-09-11T08:52:18.573947+0000","last_became_peered":"2024-09-11T08:52:18.573947+0000","last_unstale":"2024-09-11T08:53:03.997592+0000","last_undegraded":"2024-09-11T08:53:03.997592+0000","last_fullsized":"2024-09-11T08:53:03.997592+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:26.813220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:26.813220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:45.083545+0000","scrub_duration":0.001765602,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":361,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.078878+0000","last_change":"2024-09-11T08:52:31.391319+0000","last_active":"2024-09-11T08:53:04.078878+0000","last_peered":"2024-09-11T08:53:04.078878+0000","last_clean":"2024-09-11T08:53:04.078878+0000","last_became_active":"2024-09-11T08:52:18.584768+0000","last_became_peered":"2024-09-11T08:52:18.584768+0000","last_unstale":"2024-09-11T08:53:04.078878+0000","last_undegraded":"2024-09-11T08:53:04.078878+0000","last_fullsized":"2024-09-11T08:53:04.078878+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:31.391081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:52:31.391081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:38.170386+0000","scrub_duration":0.0044936749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"200'12","reported_seq":336,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998302+0000","last_change":"2024-09-11T08:52:58.504342+0000","last_active":"2024-09-11T08:53:03.998302+0000","last_peered":"2024-09-11T08:53:03.998302+0000","last_clean":"2024-09-11T08:53:03.998302+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:53:03.998302+0000","last_undegraded":"2024-09-11T08:53:03.998302+0000","last_fullsized":"2024-09-11T08:53:03.998302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'12","last_scrub_stamp":"2024-09-11T08:52:58.504277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:52:58.504277+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:02.199526+0000","scrub_duration":0.0016055349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":298,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998199+0000","last_change":"2024-09-11T08:50:07.390163+0000","last_active":"2024-09-11T08:53:03.998199+0000","last_peered":"2024-09-11T08:53:03.998199+0000","last_clean":"2024-09-11T08:53:03.998199+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:53:03.998199+0000","last_undegraded":"2024-09-11T08:53:03.998199+0000","last_fullsized":"2024-09-11T08:53:03.998199+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:07.389973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:07.389973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014729302999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":325,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.079577+0000","last_change":"2024-09-11T08:52:55.411931+0000","last_active":"2024-09-11T08:53:04.079577+0000","last_peered":"2024-09-11T08:53:04.079577+0000","last_clean":"2024-09-11T08:53:04.079577+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:53:04.079577+0000","last_undegraded":"2024-09-11T08:53:04.079577+0000","last_fullsized":"2024-09-11T08:53:04.079577+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:55.411867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:55.411867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:10.675551+0000","scrub_duration":0.001452861,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"200'15","reported_seq":332,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998053+0000","last_change":"2024-09-11T08:52:49.570164+0000","last_active":"2024-09-11T08:53:03.998053+0000","last_peered":"2024-09-11T08:53:03.998053+0000","last_clean":"2024-09-11T08:53:03.998053+0000","last_became_active":"2024-09-11T08:52:49.569850+0000","last_became_peered":"2024-09-11T08:52:49.569850+0000","last_unstale":"2024-09-11T08:53:03.998053+0000","last_undegraded":"2024-09-11T08:53:03.998053+0000","last_fullsized":"2024-09-11T08:53:03.998053+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:50:24.263662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:24.263662+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052890580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":26,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996500+0000","last_change":"2024-09-11T08:52:38.486022+0000","last_active":"2024-09-11T08:53:03.996500+0000","last_peered":"2024-09-11T08:53:03.996500+0000","last_clean":"2024-09-11T08:53:03.996500+0000","last_became_active":"2024-09-11T08:52:38.485708+0000","last_became_peered":"2024-09-11T08:52:38.485708+0000","last_unstale":"2024-09-11T08:53:03.996500+0000","last_undegraded":"2024-09-11T08:53:03.996500+0000","last_fullsized":"2024-09-11T08:53:03.996500+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:20.368038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:51:20.368038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":288,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998214+0000","last_change":"2024-09-11T08:50:19.288275+0000","last_active":"2024-09-11T08:53:03.998214+0000","last_peered":"2024-09-11T08:53:03.998214+0000","last_clean":"2024-09-11T08:53:03.998214+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:53:03.998214+0000","last_undegraded":"2024-09-11T08:53:03.998214+0000","last_fullsized":"2024-09-11T08:53:03.998214+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:19.288216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:19.288216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055736010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":326,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.079122+0000","last_change":"2024-09-11T08:52:24.921838+0000","last_active":"2024-09-11T08:53:04.079122+0000","last_peered":"2024-09-11T08:53:04.079122+0000","last_clean":"2024-09-11T08:53:04.079122+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:53:04.079122+0000","last_undegraded":"2024-09-11T08:53:04.079122+0000","last_fullsized":"2024-09-11T08:53:04.079122+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:24.921781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_clean_scrub_stamp":"2024-09-11T08:52:24.921781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:33.057567+0000","scrub_duration":0.001256795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"200'15","reported_seq":331,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998202+0000","last_change":"2024-09-11T08:50:26.391719+0000","last_active":"2024-09-11T08:53:03.998202+0000","last_peered":"2024-09-11T08:53:03.998202+0000","last_clean":"2024-09-11T08:53:03.998202+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:53:03.998202+0000","last_undegraded":"2024-09-11T08:53:03.998202+0000","last_fullsized":"2024-09-11T08:53:03.998202+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:50:26.391622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:26.391622+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021692130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":26,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997405+0000","last_change":"2024-09-11T08:52:49.567663+0000","last_active":"2024-09-11T08:53:03.997405+0000","last_peered":"2024-09-11T08:53:03.997405+0000","last_clean":"2024-09-11T08:53:03.997405+0000","last_became_active":"2024-09-11T08:52:49.566759+0000","last_became_peered":"2024-09-11T08:52:49.566759+0000","last_unstale":"2024-09-11T08:53:03.997405+0000","last_undegraded":"2024-09-11T08:53:03.997405+0000","last_fullsized":"2024-09-11T08:53:03.997405+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:16.609826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:16.609826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:22.186111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":470,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:07.839334+0000","last_change":"2024-09-11T08:50:16.313863+0000","last_active":"2024-09-11T08:53:07.839334+0000","last_peered":"2024-09-11T08:53:07.839334+0000","last_clean":"2024-09-11T08:53:07.839334+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:53:07.839334+0000","last_undegraded":"2024-09-11T08:53:07.839334+0000","last_fullsized":"2024-09-11T08:53:07.839334+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:50:16.313779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:16.313779+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049551839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":290,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996984+0000","last_change":"2024-09-11T08:52:27.235517+0000","last_active":"2024-09-11T08:53:03.996984+0000","last_peered":"2024-09-11T08:53:03.996984+0000","last_clean":"2024-09-11T08:53:03.996984+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:53:03.996984+0000","last_undegraded":"2024-09-11T08:53:03.996984+0000","last_fullsized":"2024-09-11T08:53:03.996984+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:27.235448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:27.235448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:30.171192+0000","scrub_duration":0.0017172019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"200'25","reported_seq":335,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997919+0000","last_change":"2024-09-11T08:52:53.432742+0000","last_active":"2024-09-11T08:53:03.997919+0000","last_peered":"2024-09-11T08:53:03.997919+0000","last_clean":"2024-09-11T08:53:03.997919+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:53:03.997919+0000","last_undegraded":"2024-09-11T08:53:03.997919+0000","last_fullsized":"2024-09-11T08:53:03.997919+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:52:53.432685+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:52:53.432685+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:02.336717+0000","scrub_duration":0.001930122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":321,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.079870+0000","last_change":"2024-09-11T08:53:01.430368+0000","last_active":"2024-09-11T08:53:04.079870+0000","last_peered":"2024-09-11T08:53:04.079870+0000","last_clean":"2024-09-11T08:53:04.079870+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:53:04.079870+0000","last_undegraded":"2024-09-11T08:53:04.079870+0000","last_fullsized":"2024-09-11T08:53:04.079870+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:01.430318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_clean_scrub_stamp":"2024-09-11T08:53:01.430318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:11.060354+0000","scrub_duration":0.00111052,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"277'10","reported_seq":508,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:07.839493+0000","last_change":"2024-09-11T08:52:18.599864+0000","last_active":"2024-09-11T08:53:07.839493+0000","last_peered":"2024-09-11T08:53:07.839493+0000","last_clean":"2024-09-11T08:53:07.839493+0000","last_became_active":"2024-09-11T08:52:18.585914+0000","last_became_peered":"2024-09-11T08:52:18.585914+0000","last_unstale":"2024-09-11T08:53:07.839493+0000","last_undegraded":"2024-09-11T08:53:07.839493+0000","last_fullsized":"2024-09-11T08:53:07.839493+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'9","last_scrub_stamp":"2024-09-11T08:50:08.388991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:08.388991+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019634517000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":36,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998044+0000","last_change":"2024-09-11T08:53:01.446031+0000","last_active":"2024-09-11T08:53:03.998044+0000","last_peered":"2024-09-11T08:53:03.998044+0000","last_clean":"2024-09-11T08:53:03.998044+0000","last_became_active":"2024-09-11T08:52:49.567693+0000","last_became_peered":"2024-09-11T08:52:49.567693+0000","last_unstale":"2024-09-11T08:53:03.998044+0000","last_undegraded":"2024-09-11T08:53:03.998044+0000","last_fullsized":"2024-09-11T08:53:03.998044+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:01.445935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:01.445935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:10.808558+0000","scrub_duration":0.0019176729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"200'19","reported_seq":344,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.996620+0000","last_change":"2024-09-11T08:51:31.019004+0000","last_active":"2024-09-11T08:53:03.996620+0000","last_peered":"2024-09-11T08:53:03.996620+0000","last_clean":"2024-09-11T08:53:03.996620+0000","last_became_active":"2024-09-11T08:49:16.937185+0000","last_became_peered":"2024-09-11T08:49:16.937185+0000","last_unstale":"2024-09-11T08:53:03.996620+0000","last_undegraded":"2024-09-11T08:53:03.996620+0000","last_fullsized":"2024-09-11T08:53:03.996620+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:31.018893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:31.018893+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062729989999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":293,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997100+0000","last_change":"2024-09-11T08:50:04.791525+0000","last_active":"2024-09-11T08:53:03.997100+0000","last_peered":"2024-09-11T08:53:03.997100+0000","last_clean":"2024-09-11T08:53:03.997100+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:53:03.997100+0000","last_undegraded":"2024-09-11T08:53:03.997100+0000","last_fullsized":"2024-09-11T08:53:03.997100+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:04.791327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:04.791327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063425010000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":308,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.078677+0000","last_change":"2024-09-11T08:52:15.686017+0000","last_active":"2024-09-11T08:53:04.078677+0000","last_peered":"2024-09-11T08:53:04.078677+0000","last_clean":"2024-09-11T08:53:04.078677+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:53:04.078677+0000","last_undegraded":"2024-09-11T08:53:04.078677+0000","last_fullsized":"2024-09-11T08:53:04.078677+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:15.685694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:15.685694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:41.819033+0000","scrub_duration":0.0028796590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":306,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997414+0000","last_change":"2024-09-11T08:52:18.586021+0000","last_active":"2024-09-11T08:53:03.997414+0000","last_peered":"2024-09-11T08:53:03.997414+0000","last_clean":"2024-09-11T08:53:03.997414+0000","last_became_active":"2024-09-11T08:52:18.585089+0000","last_became_peered":"2024-09-11T08:52:18.585089+0000","last_unstale":"2024-09-11T08:53:03.997414+0000","last_undegraded":"2024-09-11T08:53:03.997414+0000","last_fullsized":"2024-09-11T08:53:03.997414+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:21.261891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:21.261891+0000","last_clean_scrub_stamp":"2024-09-11T08:50:21.261891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044878879999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"200'15","reported_seq":338,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997433+0000","last_change":"2024-09-11T08:50:03.319748+0000","last_active":"2024-09-11T08:53:03.997433+0000","last_peered":"2024-09-11T08:53:03.997433+0000","last_clean":"2024-09-11T08:53:03.997433+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:53:03.997433+0000","last_undegraded":"2024-09-11T08:53:03.997433+0000","last_fullsized":"2024-09-11T08:53:03.997433+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:50:03.319664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:03.319664+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046538380000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":26,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997969+0000","last_change":"2024-09-11T08:52:38.486073+0000","last_active":"2024-09-11T08:53:03.997969+0000","last_peered":"2024-09-11T08:53:03.997969+0000","last_clean":"2024-09-11T08:53:03.997969+0000","last_became_active":"2024-09-11T08:52:38.485748+0000","last_became_peered":"2024-09-11T08:52:38.485748+0000","last_unstale":"2024-09-11T08:53:03.997969+0000","last_undegraded":"2024-09-11T08:53:03.997969+0000","last_fullsized":"2024-09-11T08:53:03.997969+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:21.208684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:21.208684+0000","last_clean_scrub_stamp":"2024-09-11T08:50:21.208684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":318,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997582+0000","last_change":"2024-09-11T08:52:54.441725+0000","last_active":"2024-09-11T08:53:03.997582+0000","last_peered":"2024-09-11T08:53:03.997582+0000","last_clean":"2024-09-11T08:53:03.997582+0000","last_became_active":"2024-09-11T08:52:18.585691+0000","last_became_peered":"2024-09-11T08:52:18.585691+0000","last_unstale":"2024-09-11T08:53:03.997582+0000","last_undegraded":"2024-09-11T08:53:03.997582+0000","last_fullsized":"2024-09-11T08:53:03.997582+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.441666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.441666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:08.658190+0000","scrub_duration":0.0012216760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":108,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997092+0000","last_change":"2024-09-11T08:52:49.567009+0000","last_active":"2024-09-11T08:53:03.997092+0000","last_peered":"2024-09-11T08:53:03.997092+0000","last_clean":"2024-09-11T08:53:03.997092+0000","last_became_active":"2024-09-11T08:52:49.566694+0000","last_became_peered":"2024-09-11T08:52:49.566694+0000","last_unstale":"2024-09-11T08:53:03.997092+0000","last_undegraded":"2024-09-11T08:53:03.997092+0000","last_fullsized":"2024-09-11T08:53:03.997092+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:24.401480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:24.401480+0000","last_clean_scrub_stamp":"2024-09-11T08:50:24.401480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"200'19","reported_seq":321,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997232+0000","last_change":"2024-09-11T08:51:21.423287+0000","last_active":"2024-09-11T08:53:03.997232+0000","last_peered":"2024-09-11T08:53:03.997232+0000","last_clean":"2024-09-11T08:53:03.997232+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:53:03.997232+0000","last_undegraded":"2024-09-11T08:53:03.997232+0000","last_fullsized":"2024-09-11T08:53:03.997232+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:21.423213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:21.423213+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086807859999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":312,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.734557+0000","last_change":"2024-09-11T08:53:04.734505+0000","last_active":"2024-09-11T08:53:04.734557+0000","last_peered":"2024-09-11T08:53:04.734557+0000","last_clean":"2024-09-11T08:53:04.734557+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:53:04.734557+0000","last_undegraded":"2024-09-11T08:53:04.734557+0000","last_fullsized":"2024-09-11T08:53:04.734557+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:04.734452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:04.734452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:19.824548+0000","scrub_duration":0.001307342,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":320,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.078466+0000","last_change":"2024-09-11T08:53:00.454714+0000","last_active":"2024-09-11T08:53:04.078466+0000","last_peered":"2024-09-11T08:53:04.078466+0000","last_clean":"2024-09-11T08:53:04.078466+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:53:04.078466+0000","last_undegraded":"2024-09-11T08:53:04.078466+0000","last_fullsized":"2024-09-11T08:53:04.078466+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:00.454658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:00.454658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:08.231173+0000","scrub_duration":0.0013095660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":326,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998053+0000","last_change":"2024-09-11T08:52:18.586241+0000","last_active":"2024-09-11T08:53:03.998053+0000","last_peered":"2024-09-11T08:53:03.998053+0000","last_clean":"2024-09-11T08:53:03.998053+0000","last_became_active":"2024-09-11T08:52:18.585315+0000","last_became_peered":"2024-09-11T08:52:18.585315+0000","last_unstale":"2024-09-11T08:53:03.998053+0000","last_undegraded":"2024-09-11T08:53:03.998053+0000","last_fullsized":"2024-09-11T08:53:03.998053+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:25.255332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:25.255332+0000","last_clean_scrub_stamp":"2024-09-11T08:50:25.255332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046648280000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"200'10","reported_seq":349,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997129+0000","last_change":"2024-09-11T08:52:11.431872+0000","last_active":"2024-09-11T08:53:03.997129+0000","last_peered":"2024-09-11T08:53:03.997129+0000","last_clean":"2024-09-11T08:53:03.997129+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:53:03.997129+0000","last_undegraded":"2024-09-11T08:53:03.997129+0000","last_fullsized":"2024-09-11T08:53:03.997129+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'10","last_scrub_stamp":"2024-09-11T08:52:11.431805+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:52:11.431805+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:20.916559+0000","scrub_duration":0.0013697399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":120,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.998421+0000","last_change":"2024-09-11T08:52:49.570339+0000","last_active":"2024-09-11T08:53:03.998421+0000","last_peered":"2024-09-11T08:53:03.998421+0000","last_clean":"2024-09-11T08:53:03.998421+0000","last_became_active":"2024-09-11T08:52:49.569954+0000","last_became_peered":"2024-09-11T08:52:49.569954+0000","last_unstale":"2024-09-11T08:53:03.998421+0000","last_undegraded":"2024-09-11T08:53:03.998421+0000","last_fullsized":"2024-09-11T08:53:03.998421+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:50:09.396148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:09.396148+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013176124000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":171,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.078660+0000","last_change":"2024-09-11T08:51:33.142487+0000","last_active":"2024-09-11T08:53:04.078660+0000","last_peered":"2024-09-11T08:53:04.078660+0000","last_clean":"2024-09-11T08:53:04.078660+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:53:04.078660+0000","last_undegraded":"2024-09-11T08:53:04.078660+0000","last_fullsized":"2024-09-11T08:53:04.078660+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:33.142415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:33.142415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057010619999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":35,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.202236+0000","last_change":"2024-09-11T08:52:51.904084+0000","last_active":"2024-09-11T08:53:04.202236+0000","last_peered":"2024-09-11T08:53:04.202236+0000","last_clean":"2024-09-11T08:53:04.202236+0000","last_became_active":"2024-09-11T08:52:38.481474+0000","last_became_peered":"2024-09-11T08:52:38.481474+0000","last_unstale":"2024-09-11T08:53:04.202236+0000","last_undegraded":"2024-09-11T08:53:04.202236+0000","last_fullsized":"2024-09-11T08:53:04.202236+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:51.904033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:30.788303+0000","last_clean_scrub_stamp":"2024-09-11T08:52:51.904033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:07.919285+0000","scrub_duration":0.0016307439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":316,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.178734+0000","last_change":"2024-09-11T08:52:54.846053+0000","last_active":"2024-09-11T08:53:04.178734+0000","last_peered":"2024-09-11T08:53:04.178734+0000","last_clean":"2024-09-11T08:53:04.178734+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:53:04.178734+0000","last_undegraded":"2024-09-11T08:53:04.178734+0000","last_fullsized":"2024-09-11T08:53:04.178734+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.845993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.845993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:59.400471+0000","scrub_duration":0.001860252,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":322,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:03.997315+0000","last_change":"2024-09-11T08:52:18.595475+0000","last_active":"2024-09-11T08:53:03.997315+0000","last_peered":"2024-09-11T08:53:03.997315+0000","last_clean":"2024-09-11T08:53:03.997315+0000","last_became_active":"2024-09-11T08:52:18.595146+0000","last_became_peered":"2024-09-11T08:52:18.595146+0000","last_unstale":"2024-09-11T08:53:03.997315+0000","last_undegraded":"2024-09-11T08:53:03.997315+0000","last_fullsized":"2024-09-11T08:53:03.997315+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:32.393197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:32.393197+0000","last_clean_scrub_stamp":"2024-09-11T08:51:32.393197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001696321,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"200'19","reported_seq":195,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.080327+0000","last_change":"2024-09-11T08:51:32.119010+0000","last_active":"2024-09-11T08:53:04.080327+0000","last_peered":"2024-09-11T08:53:04.080327+0000","last_clean":"2024-09-11T08:53:04.080327+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:53:04.080327+0000","last_undegraded":"2024-09-11T08:53:04.080327+0000","last_fullsized":"2024-09-11T08:53:04.080327+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:32.118917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:32.118917+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023296060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":117,"reported_epoch":299,"state":"active+clean","last_fresh":"2024-09-11T08:53:04.079881+0000","last_change":"2024-09-11T08:52:53.465187+0000","last_active":"2024-09-11T08:53:04.079881+0000","last_peered":"2024-09-11T08:53:04.079881+0000","last_clean":"2024-09-11T08:53:04.079881+0000","last_became_active":"2024-09-11T08:52:49.568537+0000","last_became_peered":"2024-09-11T08:52:49.568537+0000","last_unstale":"2024-09-11T08:53:04.079881+0000","last_undegraded":"2024-09-11T08:53:04.079881+0000","last_fullsized":"2024-09-11T08:53:04.079881+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:53.465128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:53.465128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:19.014737+0000","scrub_duration":0.00141025,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":1375272960,"num_objects":20985,"num_object_clones":0,"num_object_copies":62955,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20985,"num_write_kb":1343040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4223467520,"data_stored":4223467520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20985,"ondisk_log_size":20985,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1119,"num_read_kb":876,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":4114,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":756,"ondisk_log_size":756,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":35,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":649698,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":126,"num_write":62,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":692045,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299572,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":940116,"kb_used_data":350220,"kb_used_omap":1,"kb_used_meta":589886,"kb_avail":92805036,"statfs":{"total":95995035648,"available":95032356864,"internally_reserved":0,"allocated":358625280,"data_stored":358266848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":604043608},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:52:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.86299999999999999,"15min":0.93899999999999995},"min":{"1min":0.249,"5min":0.23200000000000001,"15min":0.22},"max":{"1min":4.2610000000000001,"5min":4.8730000000000002,"15min":32.171999999999997},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.94199999999999995,"15min":1.044},"min":{"1min":0.311,"5min":0.26700000000000002,"15min":0.21199999999999999},"max":{"1min":2.6480000000000001,"5min":4.0069999999999997,"15min":32.149999999999999},"last":0.496}]},{"osd":1,"last update":"Wed Sep 11 08:52:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.1519999999999999,"15min":1.1160000000000001},"min":{"1min":0.33300000000000002,"5min":0.224,"15min":0.224},"max":{"1min":4.3029999999999999,"5min":4.8970000000000002,"15min":33.692999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.083,"15min":1.0309999999999999},"min":{"1min":0.28000000000000003,"5min":0.23799999999999999,"15min":0.23300000000000001},"max":{"1min":2.9590000000000001,"5min":5.2030000000000003,"15min":35.591000000000001},"last":1.1339999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:52:50 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.0549999999999999,"15min":0.93600000000000005},"min":{"1min":0.34799999999999998,"5min":0.29799999999999999,"15min":0.249},"max":{"1min":4.9829999999999997,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0169999999999999,"15min":0.89200000000000002},"min":{"1min":0.29199999999999998,"5min":0.27900000000000003,"15min":0.25800000000000001},"max":{"1min":4.282,"5min":7.5259999999999998,"15min":7.5259999999999998},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed Sep 11 08:52:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.784,"15min":1.198},"min":{"1min":0.32200000000000001,"5min":0.27200000000000002,"15min":0.25},"max":{"1min":5.0220000000000002,"5min":56.265999999999998,"15min":56.265999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.7190000000000001,"15min":1.2549999999999999},"min":{"1min":0.36299999999999999,"5min":0.34499999999999997,"15min":0.27400000000000002},"max":{"1min":4.2329999999999997,"5min":57.207000000000001,"15min":57.207000000000001},"last":0.42199999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:52:50 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0009999999999999,"15min":0.66100000000000003},"min":{"1min":0.28100000000000003,"5min":0.255,"15min":0.23400000000000001},"max":{"1min":5.468,"5min":7.1520000000000001,"15min":7.1520000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.0980000000000001,"15min":0.70499999999999996},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.216},"max":{"1min":5.1669999999999998,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":0.57199999999999995}]},{"osd":5,"last update":"Wed Sep 11 08:52:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.052,"15min":0.70499999999999996},"min":{"1min":0.317,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":4.2050000000000001,"5min":5.1059999999999999,"15min":5.1059999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.143,"15min":0.77800000000000002},"min":{"1min":0.311,"5min":0.308,"15min":0.26700000000000002},"max":{"1min":5.2080000000000002,"5min":6.3650000000000002,"15min":6.3650000000000002},"last":0.36899999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:52:50 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":1.073,"15min":0.76000000000000001},"min":{"1min":0.35399999999999998,"5min":0.32000000000000001,"15min":0.315},"max":{"1min":2.992,"5min":4.5449999999999999,"15min":4.5449999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.224,"15min":0.79700000000000004},"min":{"1min":0.35999999999999999,"5min":0.33300000000000002,"15min":0.27900000000000003},"max":{"1min":5.0620000000000003,"5min":5.0620000000000003,"15min":5.0620000000000003},"last":0.61799999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528505,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":824200,"kb_used_data":518528,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92920952,"statfs":{"total":95995035648,"available":95151054848,"internally_reserved":0,"allocated":530972672,"data_stored":530626364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:52:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":1.3109999999999999,"15min":1.155},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.217},"max":{"1min":1.9710000000000001,"5min":27.908000000000001,"15min":28.302},"last":1.381},{"interface":"front","average":{"1min":0.63400000000000001,"5min":1.3460000000000001,"15min":1.1459999999999999},"min":{"1min":0.26100000000000001,"5min":0.24299999999999999,"15min":0.23300000000000001},"max":{"1min":1.885,"5min":27.771000000000001,"15min":28.314},"last":0.64300000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:52:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.4490000000000001,"15min":1.1619999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.7919999999999998,"5min":27.984000000000002,"15min":27.984000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.5589999999999999,"15min":1.1180000000000001},"min":{"1min":0.33500000000000002,"5min":0.25700000000000001,"15min":0.253},"max":{"1min":3.028,"5min":28.106000000000002,"15min":28.106000000000002},"last":0.34000000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:52:30 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.92100000000000004,"15min":0.82499999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.23200000000000001},"max":{"1min":2.0409999999999999,"5min":3.899,"15min":14.313000000000001},"last":0.443},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.92200000000000004,"15min":0.85399999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.251},"max":{"1min":1.8979999999999999,"5min":4.9020000000000001,"15min":6.5979999999999999},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:52:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.51,"15min":1.1830000000000001},"min":{"1min":0.34399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.6850000000000001,"5min":27.795000000000002,"15min":27.795000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.85899999999999999,"5min":1.784,"15min":1.1779999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.224},"max":{"1min":2.2240000000000002,"5min":27.841999999999999,"15min":27.841999999999999},"last":0.48399999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:52:30 2024","interfaces":[{"interface":"back","average":{"1min":3.04,"5min":1.274,"15min":0.73499999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.20799999999999999},"max":{"1min":40.927999999999997,"5min":40.927999999999997,"15min":40.927999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":3.629,"5min":1.367,"15min":0.76400000000000001},"min":{"1min":0.30499999999999999,"5min":0.27700000000000002,"15min":0.26400000000000001},"max":{"1min":40.706000000000003,"5min":40.706000000000003,"15min":40.706000000000003},"last":0.39200000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:52:30 2024","interfaces":[{"interface":"back","average":{"1min":3.1440000000000001,"5min":1.532,"15min":0.86099999999999999},"min":{"1min":0.29899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":40.847000000000001,"5min":40.847000000000001,"15min":40.847000000000001},"last":0.311},{"interface":"front","average":{"1min":3.4359999999999999,"5min":1.3939999999999999,"15min":0.82299999999999995},"min":{"1min":0.375,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":41.401000000000003,"5min":41.401000000000003,"15min":41.401000000000003},"last":0.36699999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:52:48 2024","interfaces":[{"interface":"back","average":{"1min":2.9449999999999998,"5min":1.7150000000000001,"15min":0.94199999999999995},"min":{"1min":0.32500000000000001,"5min":0.313,"15min":0.247},"max":{"1min":40.426000000000002,"5min":40.426000000000002,"15min":40.426000000000002},"last":0.38200000000000001},{"interface":"front","average":{"1min":3.3780000000000001,"5min":1.7589999999999999,"15min":0.94199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.27700000000000002},"max":{"1min":45.401000000000003,"5min":45.401000000000003,"15min":45.401000000000003},"last":0.41299999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757437,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1412572,"kb_used_data":522772,"kb_used_omap":1,"kb_used_meta":889790,"kb_avail":92332580,"statfs":{"total":95995035648,"available":94548561920,"internally_reserved":0,"allocated":535318528,"data_stored":534953191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":911145304},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:52:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.0429999999999999,"15min":0.88700000000000001},"min":{"1min":0.26300000000000001,"5min":0.21099999999999999,"15min":0.20599999999999999},"max":{"1min":3.7050000000000001,"5min":4.6420000000000003,"15min":7.3899999999999997},"last":0.221},{"interface":"front","average":{"1min":1.026,"5min":1.2250000000000001,"15min":1.0049999999999999},"min":{"1min":0.26100000000000001,"5min":0.23400000000000001,"15min":0.23100000000000001},"max":{"1min":4.2469999999999999,"5min":8.0410000000000004,"15min":8.0410000000000004},"last":0.71499999999999997}]},{"osd":1,"last update":"Wed Sep 11 08:52:11 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.266,"15min":0.98499999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":4.2140000000000004,"5min":8,"15min":8},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.2969999999999999,"15min":0.97799999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":5.984,"5min":5.984,"15min":6.6719999999999997},"last":0.53000000000000003}]},{"osd":2,"last update":"Wed Sep 11 08:52:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.1220000000000001,"15min":0.95699999999999996},"min":{"1min":0.253,"5min":0.253,"15min":0.23799999999999999},"max":{"1min":3.7250000000000001,"5min":6.4080000000000004,"15min":6.4080000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":1,"5min":1.214,"15min":0.92300000000000004},"min":{"1min":0.31900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":4.2889999999999997,"5min":8.0640000000000001,"15min":8.0640000000000001},"last":0.68799999999999994}]},{"osd":3,"last update":"Wed Sep 11 08:52:11 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.214,"15min":0.95999999999999996},"min":{"1min":0.31900000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.6099999999999999,"5min":6.4299999999999997,"15min":6.4299999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.4590000000000001,"15min":1.127},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.27400000000000002},"max":{"1min":4.2690000000000001,"5min":8.0899999999999999,"15min":8.0899999999999999},"last":0.66600000000000004}]},{"osd":4,"last update":"Wed Sep 11 08:52:11 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.0669999999999999,"15min":0.67100000000000004},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.20699999999999999},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.96699999999999997,"15min":0.63700000000000001},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.20499999999999999},"max":{"1min":3.6640000000000001,"5min":6.3769999999999998,"15min":6.3769999999999998},"last":0.435}]},{"osd":6,"last update":"Wed Sep 11 08:52:34 2024","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.22,"15min":0.751},"min":{"1min":0.373,"5min":0.30399999999999999,"15min":0.23599999999999999},"max":{"1min":8.8059999999999992,"5min":8.8059999999999992,"15min":8.8059999999999992},"last":0.41299999999999998},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.1020000000000001,"15min":0.71199999999999997},"min":{"1min":0.33700000000000002,"5min":0.27300000000000002,"15min":0.251},"max":{"1min":5.258,"5min":8.1920000000000002,"15min":8.1920000000000002},"last":0.60199999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:52:47 2024","interfaces":[{"interface":"back","average":{"1min":2.044,"5min":2.2090000000000001,"15min":1.1080000000000001},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.27800000000000002},"max":{"1min":16.062999999999999,"5min":88.513000000000005,"15min":88.513000000000005},"last":0.33300000000000002},{"interface":"front","average":{"1min":1.4299999999999999,"5min":2.1389999999999998,"15min":1.1000000000000001},"min":{"1min":0.36699999999999999,"5min":0.317,"15min":0.29599999999999999},"max":{"1min":8.8300000000000001,"5min":86.963999999999999,"15min":86.963999999999999},"last":0.47199999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986369,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":939284,"kb_used_data":629452,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92805868,"statfs":{"total":95995035648,"available":95033208832,"internally_reserved":0,"allocated":644558848,"data_stored":644199878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:52:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.579,"15min":1.6379999999999999},"min":{"1min":0.23899999999999999,"5min":0.22600000000000001,"15min":0.20399999999999999},"max":{"1min":5.2400000000000002,"5min":44.140999999999998,"15min":89.510000000000005},"last":0.33000000000000002},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.526,"15min":1.72},"min":{"1min":0.308,"5min":0.23699999999999999,"15min":0.22600000000000001},"max":{"1min":5.3220000000000001,"5min":44.450000000000003,"15min":89.765000000000001},"last":1.4490000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:52:49 2024","interfaces":[{"interface":"back","average":{"1min":1.8260000000000001,"5min":2.2810000000000001,"15min":1.456},"min":{"1min":0.27400000000000002,"5min":0.27300000000000002,"15min":0.24399999999999999},"max":{"1min":18.065999999999999,"5min":48.976999999999997,"15min":48.976999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":1.8779999999999999,"5min":2.1480000000000001,"15min":1.401},"min":{"1min":0.33700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":18.713999999999999,"5min":45.787999999999997,"15min":45.787999999999997},"last":0.498}]},{"osd":2,"last update":"Wed Sep 11 08:52:49 2024","interfaces":[{"interface":"back","average":{"1min":16.032,"5min":3.843,"15min":1.8899999999999999},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":255.88900000000001,"5min":255.88900000000001,"15min":255.88900000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":15.782999999999999,"5min":3.7810000000000001,"15min":1.8520000000000001},"min":{"1min":0.36799999999999999,"5min":0.29199999999999998,"15min":0.25800000000000001},"max":{"1min":255.41200000000001,"5min":255.41200000000001,"15min":255.41200000000001},"last":1.4239999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:52:49 2024","interfaces":[{"interface":"back","average":{"1min":1.9139999999999999,"5min":2.1030000000000002,"15min":1.97},"min":{"1min":0.41499999999999998,"5min":0.252,"15min":0.252},"max":{"1min":16.663,"5min":45.268000000000001,"15min":89.881},"last":1.391},{"interface":"front","average":{"1min":1.8160000000000001,"5min":2.02,"15min":1.915},"min":{"1min":0.32700000000000001,"5min":0.29299999999999998,"15min":0.25900000000000001},"max":{"1min":16.795999999999999,"5min":44.234000000000002,"15min":89.715000000000003},"last":0.51300000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:53:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.99299999999999999,"15min":0.69599999999999995},"min":{"1min":0.28999999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.4350000000000001,"5min":11.045999999999999,"15min":11.045999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":1.004,"15min":0.69799999999999995},"min":{"1min":0.27100000000000002,"5min":0.254,"15min":0.23699999999999999},"max":{"1min":3.1659999999999999,"5min":10.526999999999999,"15min":10.526999999999999},"last":0.35999999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:52:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":0.89100000000000001,"15min":0.61199999999999999},"min":{"1min":0.32000000000000001,"5min":0.27300000000000002,"15min":0.22900000000000001},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":0.85199999999999998,"15min":0.65400000000000003},"min":{"1min":0.35999999999999999,"5min":0.27800000000000002,"15min":0.221},"max":{"1min":6.2619999999999996,"5min":6.2619999999999996,"15min":6.2619999999999996},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:52:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.873,"15min":0.66200000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.252},"max":{"1min":3.1469999999999998,"5min":4.8079999999999998,"15min":4.8079999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.0289999999999999,"15min":0.73899999999999999},"min":{"1min":0.34399999999999997,"5min":0.32300000000000001,"15min":0.28799999999999998},"max":{"1min":5.0199999999999996,"5min":5.0199999999999996,"15min":5.1970000000000001},"last":3.0659999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149895,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1467120,"kb_used_data":559592,"kb_used_omap":1,"kb_used_meta":907518,"kb_avail":92278032,"statfs":{"total":95995035648,"available":94492704768,"internally_reserved":0,"allocated":573022208,"data_stored":572636691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":929298776},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:52:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.91600000000000004,"15min":1.0740000000000001},"min":{"1min":0.20899999999999999,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":2.4750000000000001,"5min":24.280000000000001,"15min":24.280000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":1.0169999999999999,"15min":1.286},"min":{"1min":0.23400000000000001,"5min":0.184,"15min":0.17000000000000001},"max":{"1min":3.4430000000000001,"5min":22.225000000000001,"15min":22.225000000000001},"last":0.751}]},{"osd":1,"last update":"Wed Sep 11 08:52:31 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":1.1220000000000001,"15min":1.6830000000000001},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.19800000000000001},"max":{"1min":2.464,"5min":7.4909999999999997,"15min":115.601},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.014,"15min":1.571},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.17999999999999999},"max":{"1min":2.3319999999999999,"5min":6.6639999999999997,"15min":115.474},"last":0.22900000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:52:31 2024","interfaces":[{"interface":"back","average":{"1min":5.6459999999999999,"5min":1.8340000000000001,"15min":1.4570000000000001},"min":{"1min":0.27200000000000002,"5min":0.24299999999999999,"15min":0.23200000000000001},"max":{"1min":86.766000000000005,"5min":86.766000000000005,"15min":86.766000000000005},"last":0.26000000000000001},{"interface":"front","average":{"1min":5.7290000000000001,"5min":1.901,"15min":1.4330000000000001},"min":{"1min":0.29099999999999998,"5min":0.22700000000000001,"15min":0.20799999999999999},"max":{"1min":85.367999999999995,"5min":85.367999999999995,"15min":85.367999999999995},"last":0.311}]},{"osd":4,"last update":"Wed Sep 11 08:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.093,"15min":1.1719999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.26700000000000002},"max":{"1min":4.843,"5min":6.0519999999999996,"15min":8.9969999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.143,"5min":0.90100000000000002,"15min":0.995},"min":{"1min":0.33300000000000002,"5min":0.29099999999999998,"15min":0.27700000000000002},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":7.234},"last":0.82199999999999995}]},{"osd":5,"last update":"Wed Sep 11 08:53:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0680000000000001,"15min":1.1679999999999999},"min":{"1min":0.35999999999999999,"5min":0.33900000000000002,"15min":0.28299999999999997},"max":{"1min":3.391,"5min":6.0119999999999996,"15min":11.526999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.999,"5min":1.073,"15min":1.079},"min":{"1min":0.42899999999999999,"5min":0.314,"15min":0.29399999999999998},"max":{"1min":3.9750000000000001,"5min":4.8179999999999996,"15min":7.1689999999999996},"last":0.86499999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.601,"15min":1.3460000000000001},"min":{"1min":0.39700000000000002,"5min":0.34799999999999998,"15min":0.29999999999999999},"max":{"1min":3.4209999999999998,"5min":33.106999999999999,"15min":33.106999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.5700000000000001,"15min":1.319},"min":{"1min":0.375,"5min":0.36299999999999999,"15min":0.311},"max":{"1min":2.4100000000000001,"5min":33.076000000000001,"15min":33.076000000000001},"last":0.69399999999999995}]},{"osd":7,"last update":"Wed Sep 11 08:52:55 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.6930000000000001,"15min":1.431},"min":{"1min":0.45700000000000002,"5min":0.38700000000000001,"15min":0.35299999999999998},"max":{"1min":3.7669999999999999,"5min":35.073,"15min":35.073},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.6879999999999999,"15min":1.302},"min":{"1min":0.48299999999999998,"5min":0.377,"15min":0.34699999999999998},"max":{"1min":3.754,"5min":34.844999999999999,"15min":34.844999999999999},"last":0.58199999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411531,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":784908,"kb_used_data":470916,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":92960244,"statfs":{"total":95995035648,"available":95191289856,"internally_reserved":0,"allocated":482217984,"data_stored":481867681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,4,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:53:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":1.151,"15min":1.5389999999999999},"min":{"1min":0.20799999999999999,"5min":0.191,"15min":0.159},"max":{"1min":2.0910000000000002,"5min":16.606999999999999,"15min":34.713999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":1.044,"15min":1.6140000000000001},"min":{"1min":0.23699999999999999,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.85999999999999999,"5min":16.855,"15min":34.399000000000001},"last":1.6870000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:53:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":2.0550000000000002,"15min":1.661},"min":{"1min":0.29899999999999999,"5min":0.23799999999999999,"15min":0.19},"max":{"1min":2.1920000000000002,"5min":72.867000000000004,"15min":72.867000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.46500000000000002,"5min":2.1349999999999998,"15min":1.7649999999999999},"min":{"1min":0.26800000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.1839999999999999,"5min":67.799000000000007,"15min":67.799000000000007},"last":1.7549999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:52:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":1.575,"15min":1.548},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":5.9500000000000002,"5min":16.353999999999999,"15min":34.75},"last":1.254},{"interface":"front","average":{"1min":0.84599999999999997,"5min":1.456,"15min":1.4870000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":6.0490000000000004,"5min":16.138999999999999,"15min":32.802},"last":2.1619999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:52:37 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.4910000000000001,"15min":1.1950000000000001},"min":{"1min":0.28699999999999998,"5min":0.255,"15min":0.255},"max":{"1min":4.9349999999999996,"5min":52.113999999999997,"15min":52.113999999999997},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":1.3700000000000001,"15min":1.1279999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.8029999999999999,"5min":52.097999999999999,"15min":52.097999999999999},"last":2.2930000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:53:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":1.4350000000000001,"15min":1.1599999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.96199999999999997,"5min":52.145000000000003,"15min":52.145000000000003},"last":2.3220000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":1.3200000000000001,"15min":1.0429999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.28399999999999997},"max":{"1min":3.3119999999999998,"5min":52.216000000000001,"15min":52.216000000000001},"last":1.2410000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.52,"15min":1.248},"min":{"1min":0.30299999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":3.3410000000000002,"5min":52.235999999999997,"15min":52.235999999999997},"last":1.2909999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.5029999999999999,"15min":1.1419999999999999},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.34200000000000003},"max":{"1min":2.7730000000000001,"5min":52.186,"15min":52.186},"last":2.3719999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:52:42 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.802,"15min":1.21},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.30399999999999999},"max":{"1min":4.4500000000000002,"5min":84.108999999999995,"15min":84.108999999999995},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.9510000000000001,"15min":1.3979999999999999},"min":{"1min":0.40699999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":7.0720000000000001,"5min":85.938000000000002,"15min":85.938000000000002},"last":1.3200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575056,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1164196,"kb_used_data":268828,"kb_used_omap":1,"kb_used_meta":895358,"kb_avail":92580956,"statfs":{"total":95995035648,"available":94802898944,"internally_reserved":0,"allocated":275279872,"data_stored":274905148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":916846936},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:52:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.91700000000000004,"15min":1.464},"min":{"1min":0.17499999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":2.794,"5min":3.9470000000000001,"15min":117.01600000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.94699999999999995,"15min":1.7470000000000001},"min":{"1min":0.20799999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":3.6659999999999999,"5min":8.1549999999999994,"15min":117.62},"last":0.61799999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:53:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.97699999999999998,"15min":1.171},"min":{"1min":0.221,"5min":0.21099999999999999,"15min":0.191},"max":{"1min":1.819,"5min":8.1329999999999991,"15min":20.628},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.48599999999999999,"5min":1.0880000000000001,"15min":1.1819999999999999},"min":{"1min":0.22500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":2.032,"5min":9.7200000000000006,"15min":20.148},"last":1.7430000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:52:20 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.2689999999999999,"15min":1.528},"min":{"1min":0.28799999999999998,"5min":0.22,"15min":0.191},"max":{"1min":5.1859999999999999,"5min":5.1859999999999999,"15min":116.98},"last":1.9079999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":1.2430000000000001,"15min":1.573},"min":{"1min":0.31900000000000001,"5min":0.246,"15min":0.22},"max":{"1min":3.9340000000000002,"5min":4.6719999999999997,"15min":117.56699999999999},"last":0.80800000000000005}]},{"osd":4,"last update":"Wed Sep 11 08:52:42 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.23,"15min":1.044},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.26200000000000001},"max":{"1min":7.1109999999999998,"5min":7.1109999999999998,"15min":11.19},"last":0.39400000000000002},{"interface":"front","average":{"1min":1.365,"5min":1.1719999999999999,"15min":0.96599999999999997},"min":{"1min":0.41199999999999998,"5min":0.28399999999999997,"15min":0.251},"max":{"1min":6.952,"5min":6.952,"15min":7.7169999999999996},"last":0.29899999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:53:09 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.3180000000000001,"15min":1.5700000000000001},"min":{"1min":0.315,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.3650000000000002,"5min":8.0999999999999996,"15min":103.337},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":1.3200000000000001,"15min":1.54},"min":{"1min":0.40300000000000002,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":2.0720000000000001,"5min":5.9020000000000001,"15min":103.163},"last":0.47799999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:52:37 2024","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.2569999999999999,"15min":1.0129999999999999},"min":{"1min":0.39600000000000002,"5min":0.30499999999999999,"15min":0.29399999999999998},"max":{"1min":6.2249999999999996,"5min":6.2249999999999996,"15min":8.9149999999999991},"last":0.35099999999999998},{"interface":"front","average":{"1min":1.359,"5min":1.333,"15min":1.1639999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.31},"max":{"1min":6.2869999999999999,"5min":6.2869999999999999,"15min":9.5199999999999996},"last":0.495}]},{"osd":7,"last update":"Wed Sep 11 08:52:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.212,"15min":1.087},"min":{"1min":0.371,"5min":0.37,"15min":0.32800000000000001},"max":{"1min":2.5840000000000001,"5min":4.6289999999999996,"15min":11.315},"last":0.44800000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.3080000000000001,"15min":1.0609999999999999},"min":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.315},"max":{"1min":2.9550000000000001,"5min":10.266,"15min":10.266},"last":0.373}]}]},{"osd":0,"up_from":8,"seq":34359738580,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1190744,"kb_used_data":880912,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92554408,"statfs":{"total":95995035648,"available":94775713792,"internally_reserved":0,"allocated":902053888,"data_stored":901691419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:52:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.7989999999999999,"15min":1.4670000000000001},"min":{"1min":0.247,"5min":0.19,"15min":0.16300000000000001},"max":{"1min":6.9699999999999998,"5min":56.685000000000002,"15min":56.685000000000002},"last":0.29799999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.79,"15min":1.407},"min":{"1min":0.26200000000000001,"5min":0.217,"15min":0.186},"max":{"1min":7.694,"5min":61.012999999999998,"15min":61.012999999999998},"last":1.7609999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:53:08 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.74399999999999999,"15min":1.143},"min":{"1min":0.253,"5min":0.223,"15min":0.21099999999999999},"max":{"1min":3.2850000000000001,"5min":3.9449999999999998,"15min":19.940000000000001},"last":1.889},{"interface":"front","average":{"1min":0.875,"5min":0.81200000000000006,"15min":1.2330000000000001},"min":{"1min":0.222,"5min":0.22,"15min":0.20000000000000001},"max":{"1min":3.7730000000000001,"5min":3.9300000000000002,"15min":20.757000000000001},"last":0.70899999999999996}]},{"osd":3,"last update":"Wed Sep 11 08:52:26 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":1.621,"15min":1.502},"min":{"1min":0.218,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":3.9710000000000001,"5min":56.095999999999997,"15min":56.095999999999997},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":1.786,"15min":1.4750000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.20899999999999999},"max":{"1min":2.75,"5min":59.651000000000003,"15min":59.651000000000003},"last":0.46600000000000003}]},{"osd":4,"last update":"Wed Sep 11 08:52:38 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.95899999999999996,"15min":1.0760000000000001},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.25},"max":{"1min":2.6659999999999999,"5min":4.6959999999999997,"15min":7.4690000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.222,"5min":0.95799999999999996,"15min":1.0509999999999999},"min":{"1min":0.34799999999999998,"5min":0.316,"15min":0.255},"max":{"1min":3.8050000000000002,"5min":3.9670000000000001,"15min":7.4630000000000001},"last":1.9159999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:53:08 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.022,"15min":1.153},"min":{"1min":0.40300000000000002,"5min":0.316,"15min":0.27800000000000002},"max":{"1min":3.8149999999999999,"5min":5.3550000000000004,"15min":7.4009999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.84999999999999998,"5min":1.1399999999999999,"15min":1.181},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.28299999999999997},"max":{"1min":2.5670000000000002,"5min":6.4420000000000002,"15min":7.423},"last":0.69599999999999995}]},{"osd":6,"last update":"Wed Sep 11 08:52:26 2024","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.103,"15min":1.1779999999999999},"min":{"1min":0.33600000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":3.9289999999999998,"5min":4.2270000000000003,"15min":7.4240000000000004},"last":0.878},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1220000000000001,"15min":1.208},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.9060000000000001,"5min":4.907,"15min":7.4489999999999998},"last":0.72899999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:52:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.8069999999999999,"15min":1.409},"min":{"1min":0.41599999999999998,"5min":0.33500000000000002,"15min":0.313},"max":{"1min":3.6030000000000002,"5min":56.119,"15min":56.119},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.591,"5min":1.901,"15min":1.4690000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.307},"max":{"1min":7.29,"5min":56.100000000000001,"15min":56.100000000000001},"last":0.90400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":917372928,"data_stored":917372928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":277610496,"data_stored":277610496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":518717440,"data_stored":518717440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462880768,"data_stored":462880768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":665452544,"data_stored":665452544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":433717248,"data_stored":433717248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":598278144,"data_stored":598278144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":349437952,"data_stored":349437952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:53:19.830 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 1, currently 11 2024-09-11T08:53:19.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:19.831 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:19.831 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:19.882 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 15 16 39246 39230 163.415 110.375 0.00621061 0.00611501 2024-09-11T08:53:19.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:19.966 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:19.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:19.967 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:20.068 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:20.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:20.108 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:20.108 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:20.209 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:20.247 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:20.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:20.248 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:20.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[29647]: 6.1 scrub starts 2024-09-11T08:53:20.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[29647]: 6.1 scrub ok 2024-09-11T08:53:20.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[29647]: 3.1 scrub starts 2024-09-11T08:53:20.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[29647]: 3.1 scrub ok 2024-09-11T08:53:20.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[29647]: osdmap e303: 8 total, 8 up, 8 in 2024-09-11T08:53:20.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[29647]: pgmap v659: 158 pgs: 158 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 104 MiB/s wr, 1.67k op/s 2024-09-11T08:53:20.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[29647]: 3.b scrub starts 2024-09-11T08:53:20.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[29647]: 3.b scrub ok 2024-09-11T08:53:20.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[29647]: 4.12 scrub starts 2024-09-11T08:53:20.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[29647]: 4.12 scrub ok 2024-09-11T08:53:20.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[41114]: 6.1 scrub starts 2024-09-11T08:53:20.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[41114]: 6.1 scrub ok 2024-09-11T08:53:20.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[41114]: 3.1 scrub starts 2024-09-11T08:53:20.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[41114]: 3.1 scrub ok 2024-09-11T08:53:20.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[41114]: osdmap e303: 8 total, 8 up, 8 in 2024-09-11T08:53:20.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[41114]: pgmap v659: 158 pgs: 158 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 104 MiB/s wr, 1.67k op/s 2024-09-11T08:53:20.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[41114]: 3.b scrub starts 2024-09-11T08:53:20.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[41114]: 3.b scrub ok 2024-09-11T08:53:20.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[41114]: 4.12 scrub starts 2024-09-11T08:53:20.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:20 smithi092 ceph-mon[41114]: 4.12 scrub ok 2024-09-11T08:53:20.349 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:20.385 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:20.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:20.386 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:20.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:20 smithi145 ceph-mon[33384]: 6.1 scrub starts 2024-09-11T08:53:20.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:20 smithi145 ceph-mon[33384]: 6.1 scrub ok 2024-09-11T08:53:20.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:20 smithi145 ceph-mon[33384]: 3.1 scrub starts 2024-09-11T08:53:20.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:20 smithi145 ceph-mon[33384]: 3.1 scrub ok 2024-09-11T08:53:20.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:20 smithi145 ceph-mon[33384]: osdmap e303: 8 total, 8 up, 8 in 2024-09-11T08:53:20.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:20 smithi145 ceph-mon[33384]: pgmap v659: 158 pgs: 158 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 104 MiB/s wr, 1.67k op/s 2024-09-11T08:53:20.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:20 smithi145 ceph-mon[33384]: 3.b scrub starts 2024-09-11T08:53:20.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:20 smithi145 ceph-mon[33384]: 3.b scrub ok 2024-09-11T08:53:20.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:20 smithi145 ceph-mon[33384]: 4.12 scrub starts 2024-09-11T08:53:20.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:20 smithi145 ceph-mon[33384]: 4.12 scrub ok 2024-09-11T08:53:20.487 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:20.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:20.523 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:20.624 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:20.658 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:20.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:20.659 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:20.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:20.796 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:20.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:20.797 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:20.882 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 16 16 41334 41318 161.357 130.5 0.0032803 0.00619279 2024-09-11T08:53:20.898 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:20.932 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:20.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:20.933 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:21.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:21.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:21 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:53:21.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:21 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:53:21.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:21 smithi092 ceph-mon[29647]: osdmap e304: 8 total, 8 up, 8 in 2024-09-11T08:53:21.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:21 smithi092 ceph-mon[29647]: 5.5 scrub starts 2024-09-11T08:53:21.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:21 smithi092 ceph-mon[29647]: 5.5 scrub ok 2024-09-11T08:53:21.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:21 smithi092 ceph-mon[29647]: 5.16 scrub starts 2024-09-11T08:53:21.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:21 smithi092 ceph-mon[29647]: 5.16 scrub ok 2024-09-11T08:53:21.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:21 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:53:21.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:21 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:53:21.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:21 smithi092 ceph-mon[41114]: osdmap e304: 8 total, 8 up, 8 in 2024-09-11T08:53:21.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:21 smithi092 ceph-mon[41114]: 5.5 scrub starts 2024-09-11T08:53:21.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:21 smithi092 ceph-mon[41114]: 5.5 scrub ok 2024-09-11T08:53:21.068 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:21 smithi092 ceph-mon[41114]: 5.16 scrub starts 2024-09-11T08:53:21.068 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:21 smithi092 ceph-mon[41114]: 5.16 scrub ok 2024-09-11T08:53:21.071 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:21.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:21.072 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:21.173 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:21.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:21.208 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:21.208 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:21.309 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:21.340 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:21.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:21.341 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:21.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:21 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:53:21.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:21 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:53:21.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:21 smithi145 ceph-mon[33384]: osdmap e304: 8 total, 8 up, 8 in 2024-09-11T08:53:21.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:21 smithi145 ceph-mon[33384]: 5.5 scrub starts 2024-09-11T08:53:21.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:21 smithi145 ceph-mon[33384]: 5.5 scrub ok 2024-09-11T08:53:21.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:21 smithi145 ceph-mon[33384]: 5.16 scrub starts 2024-09-11T08:53:21.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:21 smithi145 ceph-mon[33384]: 5.16 scrub ok 2024-09-11T08:53:21.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:21.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:21.482 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:21.482 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:21.583 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:21.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:21.617 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:21.617 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:21.671 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:53:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:53:21] "GET /metrics HTTP/1.1" 200 224713 "" "Prometheus/2.43.0" 2024-09-11T08:53:21.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:21.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:21.758 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:21.758 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:21.829 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:53:21.829 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-11T08:53:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-11T08:53:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:53:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:249848 7.7 7:eac2f844:::benchmark_data_smithi092_84565_object124320:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-11T08:53:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:44.824351+0000", 2024-09-11T08:53:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 517.00217590700004, 2024-09-11T08:53:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.017548627, 2024-09-11T08:53:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:53:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:53:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 249848 2024-09-11T08:53:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.824351+0000", 2024-09-11T08:53:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.824351+0000", 2024-09-11T08:53:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.824350+0000", 2024-09-11T08:53:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:53:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.824356+0000", 2024-09-11T08:53:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2779999999999999e-06 2024-09-11T08:53:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.824372+0000", 2024-09-11T08:53:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5503999999999999e-05 2024-09-11T08:53:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.824378+0000", 2024-09-11T08:53:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5370000000000003e-06 2024-09-11T08:53:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.824433+0000", 2024-09-11T08:53:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4787999999999998e-05 2024-09-11T08:53:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.824442+0000", 2024-09-11T08:53:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6099999999999995e-06 2024-09-11T08:53:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.830975+0000", 2024-09-11T08:53:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.006532835 2024-09-11T08:53:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.831075+0000", 2024-09-11T08:53:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100085 2024-09-11T08:53:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:53:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.831189+0000", 2024-09-11T08:53:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113816 2024-09-11T08:53:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.834425+0000", 2024-09-11T08:53:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032355460000000002 2024-09-11T08:53:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.841743+0000", 2024-09-11T08:53:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073183279999999998 2024-09-11T08:53:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.841775+0000", 2024-09-11T08:53:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1409999999999999e-05 2024-09-11T08:53:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.841864+0000", 2024-09-11T08:53:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9557999999999999e-05 2024-09-11T08:53:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.841900+0000", 2024-09-11T08:53:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5843999999999998e-05 2024-09-11T08:53:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:249871 7.7 7:e4357a1d:::benchmark_data_smithi092_84565_object124343:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-11T08:53:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:44.851454+0000", 2024-09-11T08:53:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 516.97507361400005, 2024-09-11T08:53:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99074989099999999, 2024-09-11T08:53:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:53:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:53:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 249871 2024-09-11T08:53:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.851454+0000", 2024-09-11T08:53:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.851454+0000", 2024-09-11T08:53:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.851452+0000", 2024-09-11T08:53:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:53:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.851459+0000", 2024-09-11T08:53:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2550000000000003e-06 2024-09-11T08:53:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.851472+0000", 2024-09-11T08:53:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3134e-05 2024-09-11T08:53:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.851478+0000", 2024-09-11T08:53:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2280000000000003e-06 2024-09-11T08:53:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.851529+0000", 2024-09-11T08:53:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1301999999999998e-05 2024-09-11T08:53:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.851541+0000", 2024-09-11T08:53:21.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1517e-05 2024-09-11T08:53:21.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.834450+0000", 2024-09-11T08:53:21.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98290926899999997 2024-09-11T08:53:21.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.834610+0000", 2024-09-11T08:53:21.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016000300000000001 2024-09-11T08:53:21.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:53:21.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.834712+0000", 2024-09-11T08:53:21.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010211400000000001 2024-09-11T08:53:21.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.841706+0000", 2024-09-11T08:53:21.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0069941939999999996 2024-09-11T08:53:21.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.841956+0000", 2024-09-11T08:53:21.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024968700000000001 2024-09-11T08:53:21.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.842030+0000", 2024-09-11T08:53:21.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4170000000000003e-05 2024-09-11T08:53:21.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.842168+0000", 2024-09-11T08:53:21.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013788299999999999 2024-09-11T08:53:21.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.842203+0000", 2024-09-11T08:53:21.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5395999999999998e-05 2024-09-11T08:53:21.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.844 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.845 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:249916 7.7 7:e8d6ea8f:::benchmark_data_smithi092_84565_object124388:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-11T08:53:21.845 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:44.887154+0000", 2024-09-11T08:53:21.845 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 516.93937307199997, 2024-09-11T08:53:21.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.96799707800000001, 2024-09-11T08:53:21.845 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.845 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:53:21.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:53:21.845 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 249916 2024-09-11T08:53:21.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.845 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.887154+0000", 2024-09-11T08:53:21.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.887154+0000", 2024-09-11T08:53:21.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.887153+0000", 2024-09-11T08:53:21.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:53:21.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.887157+0000", 2024-09-11T08:53:21.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2069999999999997e-06 2024-09-11T08:53:21.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.887166+0000", 2024-09-11T08:53:21.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2090000000000005e-06 2024-09-11T08:53:21.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.887170+0000", 2024-09-11T08:53:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3320000000000002e-06 2024-09-11T08:53:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.887217+0000", 2024-09-11T08:53:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6746999999999998e-05 2024-09-11T08:53:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.887223+0000", 2024-09-11T08:53:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.844e-06 2024-09-11T08:53:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.838162+0000", 2024-09-11T08:53:21.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.95093893299999999 2024-09-11T08:53:21.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.838246+0000", 2024-09-11T08:53:21.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3554000000000007e-05 2024-09-11T08:53:21.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:53:21.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.838310+0000", 2024-09-11T08:53:21.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4338000000000006e-05 2024-09-11T08:53:21.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.842242+0000", 2024-09-11T08:53:21.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039318449999999998 2024-09-11T08:53:21.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.852159+0000", 2024-09-11T08:53:21.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099168009999999994 2024-09-11T08:53:21.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.855072+0000", 2024-09-11T08:53:21.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029125779999999999 2024-09-11T08:53:21.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.855116+0000", 2024-09-11T08:53:21.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4265999999999999e-05 2024-09-11T08:53:21.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.855151+0000", 2024-09-11T08:53:21.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5403e-05 2024-09-11T08:53:21.851 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.851 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.851 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.852 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.14814.0:249946 7.7 7:e22f6e88:::benchmark_data_smithi092_84565_object124418:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-11T08:53:21.852 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:44:44.911174+0000", 2024-09-11T08:53:21.852 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 516.91535341400004, 2024-09-11T08:53:21.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.96507757999999999, 2024-09-11T08:53:21.852 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.852 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.852 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.852 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.14814", 2024-09-11T08:53:21.852 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3601475834", 2024-09-11T08:53:21.852 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 249946 2024-09-11T08:53:21.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.853 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.911174+0000", 2024-09-11T08:53:21.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.911174+0000", 2024-09-11T08:53:21.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.911173+0000", 2024-09-11T08:53:21.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:53:21.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.911177+0000", 2024-09-11T08:53:21.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9249999999999997e-06 2024-09-11T08:53:21.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.911188+0000", 2024-09-11T08:53:21.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0599e-05 2024-09-11T08:53:21.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.911192+0000", 2024-09-11T08:53:21.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3220000000000003e-06 2024-09-11T08:53:21.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.911222+0000", 2024-09-11T08:53:21.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9927999999999999e-05 2024-09-11T08:53:21.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:44.911228+0000", 2024-09-11T08:53:21.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9909999999999999e-06 2024-09-11T08:53:21.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.845952+0000", 2024-09-11T08:53:21.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93472377100000004 2024-09-11T08:53:21.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.846049+0000", 2024-09-11T08:53:21.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.768e-05 2024-09-11T08:53:21.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:53:21.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.846157+0000", 2024-09-11T08:53:21.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107768 2024-09-11T08:53:21.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.855166+0000", 2024-09-11T08:53:21.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090086260000000005 2024-09-11T08:53:21.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.860066+0000", 2024-09-11T08:53:21.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049003120000000004 2024-09-11T08:53:21.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.876172+0000", 2024-09-11T08:53:21.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016105633000000001 2024-09-11T08:53:21.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.876215+0000", 2024-09-11T08:53:21.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2930999999999999e-05 2024-09-11T08:53:21.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:44:45.876251+0000", 2024-09-11T08:53:21.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6779000000000001e-05 2024-09-11T08:53:21.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.858 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.859 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:62827 8.5 8:ad86915f:::benchmark_data_smithi092_117541_object62826:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-11T08:53:21.859 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:46:01.977046+0000", 2024-09-11T08:53:21.859 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 439.84948167099998, 2024-09-11T08:53:21.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.322996968, 2024-09-11T08:53:21.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.859 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.859 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.859 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:53:21.859 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:53:21.859 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62827 2024-09-11T08:53:21.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.859 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977046+0000", 2024-09-11T08:53:21.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977046+0000", 2024-09-11T08:53:21.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977043+0000", 2024-09-11T08:53:21.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999976 2024-09-11T08:53:21.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977070+0000", 2024-09-11T08:53:21.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6936000000000001e-05 2024-09-11T08:53:21.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977085+0000", 2024-09-11T08:53:21.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4797000000000001e-05 2024-09-11T08:53:21.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977092+0000", 2024-09-11T08:53:21.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5079999999999999e-06 2024-09-11T08:53:21.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977654+0000", 2024-09-11T08:53:21.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00056191599999999998 2024-09-11T08:53:21.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977663+0000", 2024-09-11T08:53:21.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0990000000000005e-06 2024-09-11T08:53:21.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.266932+0000", 2024-09-11T08:53:21.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2892690060000001 2024-09-11T08:53:21.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.267958+0000", 2024-09-11T08:53:21.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010253059999999999 2024-09-11T08:53:21.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,6,7", 2024-09-11T08:53:21.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.268067+0000", 2024-09-11T08:53:21.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010941 2024-09-11T08:53:21.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.272173+0000", 2024-09-11T08:53:21.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00410604 2024-09-11T08:53:21.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.272184+0000", 2024-09-11T08:53:21.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0798000000000001e-05 2024-09-11T08:53:21.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.272231+0000", 2024-09-11T08:53:21.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7376999999999997e-05 2024-09-11T08:53:21.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.297796+0000", 2024-09-11T08:53:21.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025564455999999999 2024-09-11T08:53:21.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.297850+0000", 2024-09-11T08:53:21.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3698e-05 2024-09-11T08:53:21.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.300042+0000", 2024-09-11T08:53:21.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021929800000000002 2024-09-11T08:53:21.866 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.866 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.866 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.866 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:62854 8.5 8:ae5f6505:::benchmark_data_smithi092_117541_object62853:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-11T08:53:21.866 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:46:01.977159+0000", 2024-09-11T08:53:21.866 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 439.84936854900002, 2024-09-11T08:53:21.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.324933583, 2024-09-11T08:53:21.867 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.867 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.867 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.867 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:53:21.867 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:53:21.867 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62854 2024-09-11T08:53:21.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.867 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977159+0000", 2024-09-11T08:53:21.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977159+0000", 2024-09-11T08:53:21.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977158+0000", 2024-09-11T08:53:21.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:53:21.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977181+0000", 2024-09-11T08:53:21.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2646e-05 2024-09-11T08:53:21.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977190+0000", 2024-09-11T08:53:21.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2040000000000003e-06 2024-09-11T08:53:21.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977195+0000", 2024-09-11T08:53:21.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8010000000000003e-06 2024-09-11T08:53:21.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977684+0000", 2024-09-11T08:53:21.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048916699999999999 2024-09-11T08:53:21.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977689+0000", 2024-09-11T08:53:21.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4960000000000004e-06 2024-09-11T08:53:21.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.270684+0000", 2024-09-11T08:53:21.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929946489999999 2024-09-11T08:53:21.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.270834+0000", 2024-09-11T08:53:21.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014989699999999999 2024-09-11T08:53:21.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,6,7", 2024-09-11T08:53:21.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.271004+0000", 2024-09-11T08:53:21.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000170576 2024-09-11T08:53:21.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.273481+0000", 2024-09-11T08:53:21.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024769240000000001 2024-09-11T08:53:21.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.277070+0000", 2024-09-11T08:53:21.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003589011 2024-09-11T08:53:21.872 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:21.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.281809+0000", 2024-09-11T08:53:21.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047383429999999999 2024-09-11T08:53:21.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.300093+0000", 2024-09-11T08:53:21.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018283938999999999 2024-09-11T08:53:21.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.300144+0000", 2024-09-11T08:53:21.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1589999999999999e-05 2024-09-11T08:53:21.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.302092+0000", 2024-09-11T08:53:21.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00194805 2024-09-11T08:53:21.875 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.875 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.875 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.875 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:62857 8.5 8:afe8003c:::benchmark_data_smithi092_117541_object62856:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-11T08:53:21.875 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:46:01.977200+0000", 2024-09-11T08:53:21.875 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 439.84932762800003, 2024-09-11T08:53:21.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305109719999999, 2024-09-11T08:53:21.875 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.875 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.875 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.875 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:53:21.875 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:53:21.876 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62857 2024-09-11T08:53:21.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.876 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977200+0000", 2024-09-11T08:53:21.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977200+0000", 2024-09-11T08:53:21.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977199+0000", 2024-09-11T08:53:21.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:53:21.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977244+0000", 2024-09-11T08:53:21.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5386999999999998e-05 2024-09-11T08:53:21.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977253+0000", 2024-09-11T08:53:21.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4780000000000008e-06 2024-09-11T08:53:21.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977257+0000", 2024-09-11T08:53:21.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3189999999999996e-06 2024-09-11T08:53:21.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977709+0000", 2024-09-11T08:53:21.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000451206 2024-09-11T08:53:21.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977714+0000", 2024-09-11T08:53:21.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3739999999999997e-06 2024-09-11T08:53:21.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.272262+0000", 2024-09-11T08:53:21.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.294548236 2024-09-11T08:53:21.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.272386+0000", 2024-09-11T08:53:21.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012377500000000001 2024-09-11T08:53:21.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,6,7", 2024-09-11T08:53:21.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.272549+0000", 2024-09-11T08:53:21.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000163299 2024-09-11T08:53:21.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.275804+0000", 2024-09-11T08:53:21.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032546390000000001 2024-09-11T08:53:21.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290089+0000", 2024-09-11T08:53:21.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014285057 2024-09-11T08:53:21.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290487+0000", 2024-09-11T08:53:21.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000398391 2024-09-11T08:53:21.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.305473+0000", 2024-09-11T08:53:21.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014985517 2024-09-11T08:53:21.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.305524+0000", 2024-09-11T08:53:21.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1171000000000002e-05 2024-09-11T08:53:21.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.307711+0000", 2024-09-11T08:53:21.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021865729999999998 2024-09-11T08:53:21.882 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.882 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.882 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.882 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:62867 8.5 8:a5c89a22:::benchmark_data_smithi092_117541_object62866:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-11T08:53:21.882 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:46:01.977263+0000", 2024-09-11T08:53:21.883 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 439.84926454999999, 2024-09-11T08:53:21.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.333047401, 2024-09-11T08:53:21.883 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.883 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.883 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.883 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:53:21.883 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:53:21.883 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62867 2024-09-11T08:53:21.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.883 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977263+0000", 2024-09-11T08:53:21.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977263+0000", 2024-09-11T08:53:21.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977262+0000", 2024-09-11T08:53:21.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:53:21.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977304+0000", 2024-09-11T08:53:21.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1588000000000002e-05 2024-09-11T08:53:21.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977313+0000", 2024-09-11T08:53:21.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2150000000000007e-06 2024-09-11T08:53:21.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977317+0000", 2024-09-11T08:53:21.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5399999999999997e-06 2024-09-11T08:53:21.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977738+0000", 2024-09-11T08:53:21.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00042099100000000001 2024-09-11T08:53:21.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977744+0000", 2024-09-11T08:53:21.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1989999999999997e-06 2024-09-11T08:53:21.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.273518+0000", 2024-09-11T08:53:21.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.295774245 2024-09-11T08:53:21.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.273603+0000", 2024-09-11T08:53:21.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5121e-05 2024-09-11T08:53:21.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,6,7", 2024-09-11T08:53:21.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.273699+0000", 2024-09-11T08:53:21.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5651999999999998e-05 2024-09-11T08:53:21.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.277022+0000", 2024-09-11T08:53:21.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033238009999999999 2024-09-11T08:53:21.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290120+0000", 2024-09-11T08:53:21.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013097200999999999 2024-09-11T08:53:21.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290588+0000", 2024-09-11T08:53:21.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046811599999999997 2024-09-11T08:53:21.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.307756+0000", 2024-09-11T08:53:21.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017168493 2024-09-11T08:53:21.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.307807+0000", 2024-09-11T08:53:21.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0328999999999999e-05 2024-09-11T08:53:21.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.310310+0000", 2024-09-11T08:53:21.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025034459999999999 2024-09-11T08:53:21.889 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.890 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.890 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:62869 8.5 8:ad538366:::benchmark_data_smithi092_117541_object62868:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-11T08:53:21.890 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:46:01.977322+0000", 2024-09-11T08:53:21.890 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 439.84920472800002, 2024-09-11T08:53:21.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3357705260000001, 2024-09-11T08:53:21.890 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.890 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.890 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.890 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:53:21.890 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:53:21.890 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62869 2024-09-11T08:53:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977322+0000", 2024-09-11T08:53:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977322+0000", 2024-09-11T08:53:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977322+0000", 2024-09-11T08:53:21.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:53:21.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977363+0000", 2024-09-11T08:53:21.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0720999999999999e-05 2024-09-11T08:53:21.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977371+0000", 2024-09-11T08:53:21.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9950000000000005e-06 2024-09-11T08:53:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977375+0000", 2024-09-11T08:53:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4360000000000004e-06 2024-09-11T08:53:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977762+0000", 2024-09-11T08:53:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038663800000000002 2024-09-11T08:53:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977767+0000", 2024-09-11T08:53:21.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1560000000000003e-06 2024-09-11T08:53:21.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.274616+0000", 2024-09-11T08:53:21.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.296849272 2024-09-11T08:53:21.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.274755+0000", 2024-09-11T08:53:21.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013862 2024-09-11T08:53:21.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,6,7", 2024-09-11T08:53:21.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.274932+0000", 2024-09-11T08:53:21.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017738699999999999 2024-09-11T08:53:21.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.278348+0000", 2024-09-11T08:53:21.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003415618 2024-09-11T08:53:21.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290141+0000", 2024-09-11T08:53:21.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011793295000000001 2024-09-11T08:53:21.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290634+0000", 2024-09-11T08:53:21.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049306800000000004 2024-09-11T08:53:21.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.310357+0000", 2024-09-11T08:53:21.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019722841000000001 2024-09-11T08:53:21.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.310547+0000", 2024-09-11T08:53:21.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019053599999999999 2024-09-11T08:53:21.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.313093+0000", 2024-09-11T08:53:21.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025455 2024-09-11T08:53:21.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.897 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.897 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:62876 8.5 8:a3fe346f:::benchmark_data_smithi092_117541_object62875:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-11T08:53:21.897 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:46:01.977380+0000", 2024-09-11T08:53:21.897 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 439.84914716499998, 2024-09-11T08:53:21.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.338348772, 2024-09-11T08:53:21.898 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.898 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:53:21.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:53:21.898 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62876 2024-09-11T08:53:21.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.898 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977380+0000", 2024-09-11T08:53:21.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977380+0000", 2024-09-11T08:53:21.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977379+0000", 2024-09-11T08:53:21.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:53:21.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977421+0000", 2024-09-11T08:53:21.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1430000000000001e-05 2024-09-11T08:53:21.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977429+0000", 2024-09-11T08:53:21.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6760000000000007e-06 2024-09-11T08:53:21.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977433+0000", 2024-09-11T08:53:21.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.549e-06 2024-09-11T08:53:21.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977785+0000", 2024-09-11T08:53:21.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035186100000000002 2024-09-11T08:53:21.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977790+0000", 2024-09-11T08:53:21.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0710000000000002e-06 2024-09-11T08:53:21.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.275838+0000", 2024-09-11T08:53:21.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2980475659999999 2024-09-11T08:53:21.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.275955+0000", 2024-09-11T08:53:21.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116894 2024-09-11T08:53:21.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,6,7", 2024-09-11T08:53:21.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.276131+0000", 2024-09-11T08:53:21.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017683799999999999 2024-09-11T08:53:21.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.281671+0000", 2024-09-11T08:53:21.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055394520000000003 2024-09-11T08:53:21.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290188+0000", 2024-09-11T08:53:21.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085170780000000008 2024-09-11T08:53:21.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290670+0000", 2024-09-11T08:53:21.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048245100000000001 2024-09-11T08:53:21.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.313457+0000", 2024-09-11T08:53:21.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022786760999999999 2024-09-11T08:53:21.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.313513+0000", 2024-09-11T08:53:21.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6263e-05 2024-09-11T08:53:21.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.315729+0000", 2024-09-11T08:53:21.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002215421 2024-09-11T08:53:21.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.905 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.905 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:62883 8.5 8:a014e9e7:::benchmark_data_smithi092_117541_object62882:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-11T08:53:21.905 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:46:01.977438+0000", 2024-09-11T08:53:21.905 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 439.84908897100001, 2024-09-11T08:53:21.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3418555489999999, 2024-09-11T08:53:21.905 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.905 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.905 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.906 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:53:21.906 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:53:21.906 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62883 2024-09-11T08:53:21.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.906 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977438+0000", 2024-09-11T08:53:21.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977438+0000", 2024-09-11T08:53:21.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977438+0000", 2024-09-11T08:53:21.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:53:21.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977479+0000", 2024-09-11T08:53:21.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1124999999999997e-05 2024-09-11T08:53:21.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977488+0000", 2024-09-11T08:53:21.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9549999999999998e-06 2024-09-11T08:53:21.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977493+0000", 2024-09-11T08:53:21.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0529999999999997e-06 2024-09-11T08:53:21.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977808+0000", 2024-09-11T08:53:21.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031535200000000003 2024-09-11T08:53:21.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977814+0000", 2024-09-11T08:53:21.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5620000000000003e-06 2024-09-11T08:53:21.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.277104+0000", 2024-09-11T08:53:21.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2992901940000001 2024-09-11T08:53:21.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.277229+0000", 2024-09-11T08:53:21.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012461699999999999 2024-09-11T08:53:21.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,6,7", 2024-09-11T08:53:21.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.277412+0000", 2024-09-11T08:53:21.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000183165 2024-09-11T08:53:21.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.281735+0000", 2024-09-11T08:53:21.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043235640000000002 2024-09-11T08:53:21.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290214+0000", 2024-09-11T08:53:21.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0084788329999999999 2024-09-11T08:53:21.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290717+0000", 2024-09-11T08:53:21.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050322300000000002 2024-09-11T08:53:21.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.317029+0000", 2024-09-11T08:53:21.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026311570999999999 2024-09-11T08:53:21.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.317106+0000", 2024-09-11T08:53:21.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7052999999999996e-05 2024-09-11T08:53:21.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.319294+0000", 2024-09-11T08:53:21.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002187797 2024-09-11T08:53:21.912 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.912 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.912 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.912 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:62909 8.5 8:a5994931:::benchmark_data_smithi092_117541_object62908:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-11T08:53:21.912 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:46:01.977498+0000", 2024-09-11T08:53:21.912 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 439.84902935899999, 2024-09-11T08:53:21.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.344509186, 2024-09-11T08:53:21.912 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.912 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.912 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:53:21.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:53:21.913 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62909 2024-09-11T08:53:21.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.913 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977498+0000", 2024-09-11T08:53:21.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977498+0000", 2024-09-11T08:53:21.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977497+0000", 2024-09-11T08:53:21.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:53:21.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977538+0000", 2024-09-11T08:53:21.915 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 17 16 43206 43190 158.748 117 0.00549942 0.00629394 2024-09-11T08:53:21.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0905999999999998e-05 2024-09-11T08:53:21.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977546+0000", 2024-09-11T08:53:21.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6930000000000007e-06 2024-09-11T08:53:21.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977550+0000", 2024-09-11T08:53:21.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4070000000000003e-06 2024-09-11T08:53:21.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977837+0000", 2024-09-11T08:53:21.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-11T08:53:21.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977842+0000", 2024-09-11T08:53:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3959999999999997e-06 2024-09-11T08:53:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.278387+0000", 2024-09-11T08:53:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.300544997 2024-09-11T08:53:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.278493+0000", 2024-09-11T08:53:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106096 2024-09-11T08:53:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,6,7", 2024-09-11T08:53:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.278629+0000", 2024-09-11T08:53:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013622100000000001 2024-09-11T08:53:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.281741+0000", 2024-09-11T08:53:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031118610000000001 2024-09-11T08:53:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290251+0000", 2024-09-11T08:53:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085100760000000001 2024-09-11T08:53:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290756+0000", 2024-09-11T08:53:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050484399999999995 2024-09-11T08:53:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.319419+0000", 2024-09-11T08:53:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028662935 2024-09-11T08:53:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.319472+0000", 2024-09-11T08:53:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2648000000000001e-05 2024-09-11T08:53:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.322007+0000", 2024-09-11T08:53:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025351589999999999 2024-09-11T08:53:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:62913 8.5 8:ab446469:::benchmark_data_smithi092_117541_object62912:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-11T08:53:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:46:01.977555+0000", 2024-09-11T08:53:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 439.84897214900002, 2024-09-11T08:53:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.347412778, 2024-09-11T08:53:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:53:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:53:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62913 2024-09-11T08:53:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977555+0000", 2024-09-11T08:53:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977555+0000", 2024-09-11T08:53:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977555+0000", 2024-09-11T08:53:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:53:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977596+0000", 2024-09-11T08:53:21.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1291000000000002e-05 2024-09-11T08:53:21.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977603+0000", 2024-09-11T08:53:21.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5789999999999998e-06 2024-09-11T08:53:21.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977608+0000", 2024-09-11T08:53:21.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3080000000000001e-06 2024-09-11T08:53:21.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977860+0000", 2024-09-11T08:53:21.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025275500000000002 2024-09-11T08:53:21.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977866+0000", 2024-09-11T08:53:21.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6749999999999999e-06 2024-09-11T08:53:21.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.280050+0000", 2024-09-11T08:53:21.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.302183686 2024-09-11T08:53:21.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.280177+0000", 2024-09-11T08:53:21.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012754699999999999 2024-09-11T08:53:21.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,6,7", 2024-09-11T08:53:21.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.280283+0000", 2024-09-11T08:53:21.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105683 2024-09-11T08:53:21.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.283494+0000", 2024-09-11T08:53:21.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032109740000000001 2024-09-11T08:53:21.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290280+0000", 2024-09-11T08:53:21.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067855499999999996 2024-09-11T08:53:21.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290781+0000", 2024-09-11T08:53:21.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050162799999999995 2024-09-11T08:53:21.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.322053+0000", 2024-09-11T08:53:21.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031271291999999999 2024-09-11T08:53:21.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.322099+0000", 2024-09-11T08:53:21.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6285000000000002e-05 2024-09-11T08:53:21.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.324968+0000", 2024-09-11T08:53:21.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028690059999999999 2024-09-11T08:53:21.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.927 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.928 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15126.0:62948 8.5 8:a56c41b7:::benchmark_data_smithi092_117541_object62947:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-11T08:53:21.928 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:46:01.977612+0000", 2024-09-11T08:53:21.928 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 439.84891494999999, 2024-09-11T08:53:21.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3552955250000001, 2024-09-11T08:53:21.928 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.928 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.928 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.928 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15126", 2024-09-11T08:53:21.928 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3541972015", 2024-09-11T08:53:21.928 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 62948 2024-09-11T08:53:21.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.928 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977612+0000", 2024-09-11T08:53:21.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977612+0000", 2024-09-11T08:53:21.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977612+0000", 2024-09-11T08:53:21.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:53:21.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977654+0000", 2024-09-11T08:53:21.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2097e-05 2024-09-11T08:53:21.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977662+0000", 2024-09-11T08:53:21.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2090000000000005e-06 2024-09-11T08:53:21.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977666+0000", 2024-09-11T08:53:21.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3800000000000004e-06 2024-09-11T08:53:21.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977884+0000", 2024-09-11T08:53:21.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021780599999999999 2024-09-11T08:53:21.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:01.977890+0000", 2024-09-11T08:53:21.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.541e-06 2024-09-11T08:53:21.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.281838+0000", 2024-09-11T08:53:21.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.30394777 2024-09-11T08:53:21.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.281943+0000", 2024-09-11T08:53:21.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105437 2024-09-11T08:53:21.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,6,7", 2024-09-11T08:53:21.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.282033+0000", 2024-09-11T08:53:21.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0381000000000001e-05 2024-09-11T08:53:21.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.283602+0000", 2024-09-11T08:53:21.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015688589999999999 2024-09-11T08:53:21.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290312+0000", 2024-09-11T08:53:21.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067101469999999996 2024-09-11T08:53:21.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.290805+0000", 2024-09-11T08:53:21.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049262999999999998 2024-09-11T08:53:21.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.329936+0000", 2024-09-11T08:53:21.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039131155000000001 2024-09-11T08:53:21.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.330573+0000", 2024-09-11T08:53:21.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00063634300000000004 2024-09-11T08:53:21.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:46:03.332908+0000", 2024-09-11T08:53:21.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023352400000000001 2024-09-11T08:53:21.934 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272053 9.5 9:a00bb8b6:::benchmark_data_smithi092_145083_object128655:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:53:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:23.616160+0000", 2024-09-11T08:53:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.210367323, 2024-09-11T08:53:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0208401620000001, 2024-09-11T08:53:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:53:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:53:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272053 2024-09-11T08:53:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616160+0000", 2024-09-11T08:53:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616160+0000", 2024-09-11T08:53:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616158+0000", 2024-09-11T08:53:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:53:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616165+0000", 2024-09-11T08:53:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7240000000000003e-06 2024-09-11T08:53:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616179+0000", 2024-09-11T08:53:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3740999999999999e-05 2024-09-11T08:53:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616185+0000", 2024-09-11T08:53:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9529999999999996e-06 2024-09-11T08:53:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616245+0000", 2024-09-11T08:53:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0341999999999999e-05 2024-09-11T08:53:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616257+0000", 2024-09-11T08:53:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1643e-05 2024-09-11T08:53:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.630442+0000", 2024-09-11T08:53:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.014185804 2024-09-11T08:53:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.630590+0000", 2024-09-11T08:53:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014730400000000001 2024-09-11T08:53:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,7", 2024-09-11T08:53:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.630711+0000", 2024-09-11T08:53:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012151599999999999 2024-09-11T08:53:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.636837+0000", 2024-09-11T08:53:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061256540000000003 2024-09-11T08:53:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.636891+0000", 2024-09-11T08:53:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3708999999999997e-05 2024-09-11T08:53:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.636903+0000", 2024-09-11T08:53:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2651e-05 2024-09-11T08:53:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.636959+0000", 2024-09-11T08:53:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6181000000000003e-05 2024-09-11T08:53:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.637000+0000", 2024-09-11T08:53:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0586000000000001e-05 2024-09-11T08:53:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272343 9.b 9:deb81e83:::benchmark_data_smithi092_145083_object128945:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:53:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.633667+0000", 2024-09-11T08:53:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 57.192860383999999, 2024-09-11T08:53:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0859924860000001, 2024-09-11T08:53:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:53:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:53:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272343 2024-09-11T08:53:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633667+0000", 2024-09-11T08:53:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633667+0000", 2024-09-11T08:53:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633665+0000", 2024-09-11T08:53:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:53:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633716+0000", 2024-09-11T08:53:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0563999999999999e-05 2024-09-11T08:53:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633726+0000", 2024-09-11T08:53:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7629999999999995e-06 2024-09-11T08:53:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633730+0000", 2024-09-11T08:53:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4610000000000001e-06 2024-09-11T08:53:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633789+0000", 2024-09-11T08:53:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9329000000000001e-05 2024-09-11T08:53:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633797+0000", 2024-09-11T08:53:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7049999999999999e-06 2024-09-11T08:53:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.707365+0000", 2024-09-11T08:53:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.073567543 2024-09-11T08:53:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.707459+0000", 2024-09-11T08:53:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4926999999999994e-05 2024-09-11T08:53:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:53:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.707557+0000", 2024-09-11T08:53:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7717e-05 2024-09-11T08:53:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.714212+0000", 2024-09-11T08:53:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066545939999999998 2024-09-11T08:53:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719510+0000", 2024-09-11T08:53:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005298395 2024-09-11T08:53:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719541+0000", 2024-09-11T08:53:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0528999999999999e-05 2024-09-11T08:53:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719606+0000", 2024-09-11T08:53:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5350000000000003e-05 2024-09-11T08:53:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719659+0000", 2024-09-11T08:53:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3238000000000001e-05 2024-09-11T08:53:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272372 9.b 9:dabfe76a:::benchmark_data_smithi092_145083_object128974:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:53:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.660754+0000", 2024-09-11T08:53:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 57.165773409000003, 2024-09-11T08:53:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.059110105, 2024-09-11T08:53:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:53:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:53:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272372 2024-09-11T08:53:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660754+0000", 2024-09-11T08:53:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660754+0000", 2024-09-11T08:53:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660752+0000", 2024-09-11T08:53:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:53:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660759+0000", 2024-09-11T08:53:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9929999999999998e-06 2024-09-11T08:53:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660778+0000", 2024-09-11T08:53:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.855e-05 2024-09-11T08:53:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660785+0000", 2024-09-11T08:53:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0960000000000003e-06 2024-09-11T08:53:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660850+0000", 2024-09-11T08:53:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5032999999999999e-05 2024-09-11T08:53:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660857+0000", 2024-09-11T08:53:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7219999999999999e-06 2024-09-11T08:53:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.709617+0000", 2024-09-11T08:53:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.048759955 2024-09-11T08:53:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.709729+0000", 2024-09-11T08:53:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111554 2024-09-11T08:53:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:53:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.709875+0000", 2024-09-11T08:53:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000145597 2024-09-11T08:53:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.716482+0000", 2024-09-11T08:53:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066076379999999999 2024-09-11T08:53:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719695+0000", 2024-09-11T08:53:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032126479999999998 2024-09-11T08:53:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719780+0000", 2024-09-11T08:53:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4653000000000004e-05 2024-09-11T08:53:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719820+0000", 2024-09-11T08:53:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0395999999999997e-05 2024-09-11T08:53:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719864+0000", 2024-09-11T08:53:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3959999999999999e-05 2024-09-11T08:53:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272378 9.b 9:da18d24a:::benchmark_data_smithi092_145083_object128980:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:53:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.664541+0000", 2024-09-11T08:53:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 57.161986349000003, 2024-09-11T08:53:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0630863310000001, 2024-09-11T08:53:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:53:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:53:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272378 2024-09-11T08:53:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664541+0000", 2024-09-11T08:53:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664541+0000", 2024-09-11T08:53:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664539+0000", 2024-09-11T08:53:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:53:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664547+0000", 2024-09-11T08:53:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1960000000000008e-06 2024-09-11T08:53:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664568+0000", 2024-09-11T08:53:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0577999999999998e-05 2024-09-11T08:53:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664574+0000", 2024-09-11T08:53:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7619999999999998e-06 2024-09-11T08:53:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664646+0000", 2024-09-11T08:53:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1848999999999996e-05 2024-09-11T08:53:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664656+0000", 2024-09-11T08:53:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6889999999999999e-06 2024-09-11T08:53:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.711155+0000", 2024-09-11T08:53:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0464992740000001 2024-09-11T08:53:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.711226+0000", 2024-09-11T08:53:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1174999999999995e-05 2024-09-11T08:53:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:53:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.711339+0000", 2024-09-11T08:53:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011281399999999999 2024-09-11T08:53:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.716490+0000", 2024-09-11T08:53:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051504020000000001 2024-09-11T08:53:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719723+0000", 2024-09-11T08:53:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032333890000000001 2024-09-11T08:53:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727554+0000", 2024-09-11T08:53:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0078313399999999991 2024-09-11T08:53:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727595+0000", 2024-09-11T08:53:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0383000000000003e-05 2024-09-11T08:53:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727627+0000", 2024-09-11T08:53:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2435999999999999e-05 2024-09-11T08:53:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272400 9.b 9:d843832b:::benchmark_data_smithi092_145083_object129002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:53:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.678382+0000", 2024-09-11T08:53:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 57.148145470999999, 2024-09-11T08:53:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.049380773, 2024-09-11T08:53:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:53:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:53:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272400 2024-09-11T08:53:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678382+0000", 2024-09-11T08:53:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678382+0000", 2024-09-11T08:53:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678380+0000", 2024-09-11T08:53:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:53:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678386+0000", 2024-09-11T08:53:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5840000000000002e-06 2024-09-11T08:53:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678401+0000", 2024-09-11T08:53:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4601e-05 2024-09-11T08:53:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678407+0000", 2024-09-11T08:53:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3409999999999999e-06 2024-09-11T08:53:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678457+0000", 2024-09-11T08:53:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9512000000000003e-05 2024-09-11T08:53:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678465+0000", 2024-09-11T08:53:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2870000000000004e-06 2024-09-11T08:53:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.712993+0000", 2024-09-11T08:53:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0345277070000001 2024-09-11T08:53:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.713060+0000", 2024-09-11T08:53:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7471999999999994e-05 2024-09-11T08:53:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:53:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.713135+0000", 2024-09-11T08:53:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5100999999999998e-05 2024-09-11T08:53:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719168+0000", 2024-09-11T08:53:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060324829999999999 2024-09-11T08:53:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727669+0000", 2024-09-11T08:53:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085014749999999997 2024-09-11T08:53:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727695+0000", 2024-09-11T08:53:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6137000000000001e-05 2024-09-11T08:53:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727732+0000", 2024-09-11T08:53:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6380000000000001e-05 2024-09-11T08:53:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727762+0000", 2024-09-11T08:53:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0917000000000002e-05 2024-09-11T08:53:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272419 9.b 9:dbacabf2:::benchmark_data_smithi092_145083_object129021:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:53:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.733096+0000", 2024-09-11T08:53:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 57.093431502999998, 2024-09-11T08:53:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99494694900000002, 2024-09-11T08:53:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:53:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:53:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:53:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:53:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:53:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272419 2024-09-11T08:53:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:53:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:53:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733096+0000", 2024-09-11T08:53:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:53:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733096+0000", 2024-09-11T08:53:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:53:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:53:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733094+0000", 2024-09-11T08:53:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:53:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:53:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733101+0000", 2024-09-11T08:53:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4499999999999998e-06 2024-09-11T08:53:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:53:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733119+0000", 2024-09-11T08:53:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7252999999999999e-05 2024-09-11T08:53:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:53:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733125+0000", 2024-09-11T08:53:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5629999999999997e-06 2024-09-11T08:53:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733204+0000", 2024-09-11T08:53:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9022000000000003e-05 2024-09-11T08:53:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:53:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733286+0000", 2024-09-11T08:53:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1546999999999997e-05 2024-09-11T08:53:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:53:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.714254+0000", 2024-09-11T08:53:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98096810099999998 2024-09-11T08:53:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:53:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.714361+0000", 2024-09-11T08:53:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010746100000000001 2024-09-11T08:53:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:53:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.714441+0000", 2024-09-11T08:53:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9348e-05 2024-09-11T08:53:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:53:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719175+0000", 2024-09-11T08:53:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047343079999999996 2024-09-11T08:53:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727872+0000", 2024-09-11T08:53:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0086973250000000005 2024-09-11T08:53:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:53:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727961+0000", 2024-09-11T08:53:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8913999999999996e-05 2024-09-11T08:53:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:53:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.728006+0000", 2024-09-11T08:53:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4579999999999997e-05 2024-09-11T08:53:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:53:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.728043+0000", 2024-09-11T08:53:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.676e-05 2024-09-11T08:53:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:21.981 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:53:21.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:21.982 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:21.982 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:22.040 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:53:22.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:22.117 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:22.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:22.118 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:22.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:22.255 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:22.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:22.256 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:22.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:22 smithi092 ceph-mon[29647]: pgmap v661: 158 pgs: 158 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 150 MiB/s wr, 2.39k op/s 2024-09-11T08:53:22.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:22 smithi092 ceph-mon[29647]: 6.11 deep-scrub starts 2024-09-11T08:53:22.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:22 smithi092 ceph-mon[29647]: 6.11 deep-scrub ok 2024-09-11T08:53:22.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:22 smithi092 ceph-mon[29647]: 1.6 scrub starts 2024-09-11T08:53:22.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:22 smithi092 ceph-mon[29647]: 1.6 scrub ok 2024-09-11T08:53:22.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:22 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/92258555' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:53:22.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:22 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:53:22.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:22 smithi092 ceph-mon[41114]: pgmap v661: 158 pgs: 158 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 150 MiB/s wr, 2.39k op/s 2024-09-11T08:53:22.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:22 smithi092 ceph-mon[41114]: 6.11 deep-scrub starts 2024-09-11T08:53:22.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:22 smithi092 ceph-mon[41114]: 6.11 deep-scrub ok 2024-09-11T08:53:22.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:22 smithi092 ceph-mon[41114]: 1.6 scrub starts 2024-09-11T08:53:22.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:22 smithi092 ceph-mon[41114]: 1.6 scrub ok 2024-09-11T08:53:22.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:22 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/92258555' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:53:22.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:22 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:53:22.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:22.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:22.394 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:22 smithi145 ceph-mon[33384]: pgmap v661: 158 pgs: 158 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 150 MiB/s wr, 2.39k op/s 2024-09-11T08:53:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:22 smithi145 ceph-mon[33384]: 6.11 deep-scrub starts 2024-09-11T08:53:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:22 smithi145 ceph-mon[33384]: 6.11 deep-scrub ok 2024-09-11T08:53:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:22 smithi145 ceph-mon[33384]: 1.6 scrub starts 2024-09-11T08:53:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:22 smithi145 ceph-mon[33384]: 1.6 scrub ok 2024-09-11T08:53:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:22 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/92258555' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:53:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:22 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:53:22.495 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:22.527 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:22.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:22.528 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:22.628 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:22.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:22.669 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:22.669 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:22.770 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:22.806 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:22.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:22.807 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:22.882 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 18 16 45154 45138 156.692 121.75 0.00603057 0.00637675 2024-09-11T08:53:22.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:22.946 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:22.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:22.947 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:23.031 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:53:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:53:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:53:23.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:23.089 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:23.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:23.090 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:23.191 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:23.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:23.230 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:23.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:23 smithi092 ceph-mon[29647]: 3.14 scrub starts 2024-09-11T08:53:23.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:23 smithi092 ceph-mon[29647]: 3.14 scrub ok 2024-09-11T08:53:23.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:23 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:53:23.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:23 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:53:23.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:23 smithi092 ceph-mon[29647]: osdmap e305: 8 total, 8 up, 8 in 2024-09-11T08:53:23.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:23 smithi092 ceph-mon[41114]: 3.14 scrub starts 2024-09-11T08:53:23.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:23 smithi092 ceph-mon[41114]: 3.14 scrub ok 2024-09-11T08:53:23.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:23 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:53:23.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:23 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:53:23.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:23 smithi092 ceph-mon[41114]: osdmap e305: 8 total, 8 up, 8 in 2024-09-11T08:53:23.331 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:23.365 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:23.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:23.366 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:23.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:23 smithi145 ceph-mon[33384]: 3.14 scrub starts 2024-09-11T08:53:23.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:23 smithi145 ceph-mon[33384]: 3.14 scrub ok 2024-09-11T08:53:23.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:23 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:53:23.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:23 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:53:23.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:23 smithi145 ceph-mon[33384]: osdmap e305: 8 total, 8 up, 8 in 2024-09-11T08:53:23.467 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:23.503 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:23.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:23.504 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:23.605 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:23.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:23.642 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:23.642 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:23.743 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:23.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:23.783 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:23.783 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:23.882 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 19 16 47103 47087 154.856 121.812 0.00559548 0.00645245 2024-09-11T08:53:23.884 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:23.921 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:23.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:23.923 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:24.023 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:24.049 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:24.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:24.050 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:24.151 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:24.185 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:24.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:24.186 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:24.287 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:24.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:24 smithi092 ceph-mon[29647]: pgmap v663: 158 pgs: 158 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 107 MiB/s wr, 1.71k op/s 2024-09-11T08:53:24.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:24 smithi092 ceph-mon[29647]: osdmap e306: 8 total, 8 up, 8 in 2024-09-11T08:53:24.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:24 smithi092 ceph-mon[41114]: pgmap v663: 158 pgs: 158 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 107 MiB/s wr, 1.71k op/s 2024-09-11T08:53:24.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:24 smithi092 ceph-mon[41114]: osdmap e306: 8 total, 8 up, 8 in 2024-09-11T08:53:24.319 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:24.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:24.320 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:24.389 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:53:24.389 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:53:24.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:24 smithi145 ceph-mon[33384]: pgmap v663: 158 pgs: 158 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 107 MiB/s wr, 1.71k op/s 2024-09-11T08:53:24.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:24 smithi145 ceph-mon[33384]: osdmap e306: 8 total, 8 up, 8 in 2024-09-11T08:53:24.420 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:53:24.420 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:53:24.420 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:24.448 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:53:24.449 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:53:24.455 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:24.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:24.457 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:24.485 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:53:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:53:24.516 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:53:24.516 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:53:24.545 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:53:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:53:24.558 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:24.573 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:53:24.574 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:53:24.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:24.594 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:24.594 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:24.599 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:53:24.599 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:53:24.695 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:24.729 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:24.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:24.730 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:24.831 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:53:24.831 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:53:24.831 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-11T08:53:24.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-11T08:53:24.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:24.870 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:24.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:24.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:24.882 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:53:24.883912+0000 min lat: 0.00119342 max lat: 0.169837 avg lat: 0.00653913 2024-09-11T08:53:24.882 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:53:24.882 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 20 16 48927 48911 152.813 114 0.0100305 0.00653913 2024-09-11T08:53:24.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-11T08:53:24.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:25.035 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:25.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:25.036 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:25.136 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:25.168 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:25.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:25.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:25.270 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:25.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:25 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/516898447' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:53:25.286 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:25 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/516898447' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:53:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:25.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:25.314 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:25.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:25 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/516898447' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:53:25.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:25.453 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:25.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:25.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:25.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:25.596 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:25.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:25.597 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:25.697 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:25.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:25.736 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:25.736 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:25.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:53:25.836 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:25.876 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:25.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:25.877 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:25.883 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 21 16 50586 50570 150.474 103.688 0.0089048 0.00664111 2024-09-11T08:53:25.977 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:26.010 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:26.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:26.012 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:26.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:26.138 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:26 smithi092 ceph-mon[29647]: pgmap v665: 158 pgs: 158 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 107 MiB/s wr, 1.71k op/s 2024-09-11T08:53:26.139 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:26 smithi092 ceph-mon[41114]: pgmap v665: 158 pgs: 158 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 107 MiB/s wr, 1.71k op/s 2024-09-11T08:53:26.153 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:26.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:26.155 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:26.252 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:26.256 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:26.286 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:26.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:26.287 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:26.290 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:26.388 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:26.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:26 smithi145 ceph-mon[33384]: pgmap v665: 158 pgs: 158 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 107 MiB/s wr, 1.71k op/s 2024-09-11T08:53:26.426 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:26.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:26.427 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:26.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:26.565 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:26.577 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:26.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:26.578 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:26.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:26.713 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:26.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:26.714 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:26.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:26.859 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:26.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:26.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:26.883 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 22 16 52267 52251 148.409 105.062 0.00522791 0.00673395 2024-09-11T08:53:26.961 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:26.999 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:27.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:27.000 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:27.101 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:27.137 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:27.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:27.138 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:27.161 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:53:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:53:27.238 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:27.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:27.273 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:27.273 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:27.374 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:27.411 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:27.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:27.412 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:27.513 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:27.546 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:27.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:27.547 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:27.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:27.679 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:27.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:27.680 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:27.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:27.815 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:27.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:27.816 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:27.883 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 23 16 54073 54057 146.864 112.875 0.00687616 0.00680433 2024-09-11T08:53:27.917 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:27.952 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:27.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:27.953 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:28.053 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:28.083 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:28.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:28.085 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:28.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:28.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:53:28.209 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:28.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:28.210 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:28.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:28.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:28.349 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:28.349 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:28.449 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:28.482 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:28.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:28.483 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:28.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:28 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:28.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:28 smithi092 ceph-mon[29647]: pgmap v666: 158 pgs: 158 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 166 MiB/s wr, 2.65k op/s 2024-09-11T08:53:28.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:28 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:28.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:28 smithi092 ceph-mon[41114]: pgmap v666: 158 pgs: 158 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 166 MiB/s wr, 2.65k op/s 2024-09-11T08:53:28.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:28.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:28.622 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:28.622 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:28.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:28 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:28.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:28 smithi145 ceph-mon[33384]: pgmap v666: 158 pgs: 158 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 166 MiB/s wr, 2.65k op/s 2024-09-11T08:53:28.723 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:28.759 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:28.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:28.760 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:28.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:28.883 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 24 16 55658 55642 144.872 99.0625 0.00430581 0.0068971 2024-09-11T08:53:28.899 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:28.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:28.901 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:29.001 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:29.034 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:29.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:29.035 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:29.136 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:29.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:29.169 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:29.170 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:29.271 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:29.306 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:29.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:29.307 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:29.408 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:29.445 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:29.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:29.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:29.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:29.587 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:29.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:29.588 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:29.689 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:29.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:29.726 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:29.726 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:29.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:29.878 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:29.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:29.879 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:29.883 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 25 16 57361 57345 143.335 106.438 0.0115182 0.00697174 2024-09-11T08:53:29.981 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:30.015 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:30.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:30.017 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:30.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:30.163 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:30.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:30.164 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:30.234 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:30 smithi092 ceph-mon[29647]: pgmap v667: 158 pgs: 158 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 101 MiB/s wr, 1.61k op/s 2024-09-11T08:53:30.235 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:30 smithi092 ceph-mon[41114]: pgmap v667: 158 pgs: 158 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 101 MiB/s wr, 1.61k op/s 2024-09-11T08:53:30.265 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:30.306 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:30.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:30.307 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:30.408 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:30.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:30 smithi145 ceph-mon[33384]: pgmap v667: 158 pgs: 158 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 101 MiB/s wr, 1.61k op/s 2024-09-11T08:53:30.441 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:30.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:30.442 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:30.543 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:30.578 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:30.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:30.581 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:30.682 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:30.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:30.718 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:30.718 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:30.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:30.877 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:53:30.877 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:53:30.884 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 26 16 58972 58956 141.687 100.688 0.00668403 0.007052 2024-09-11T08:53:30.915 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:30.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:30.916 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:31.017 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:31.053 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:31.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:31.054 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:31.155 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:31.194 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:31.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:31.195 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:31.296 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:31.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:31.333 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:31.333 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:31.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:31 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:53:31.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:31 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:53:31.434 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:31.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:31 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:53:31.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:31 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:53:31.448 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:31 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:53:31.448 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:31 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:53:31.472 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:31.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:31.473 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:31.574 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:31.613 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:31.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:31.614 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:31.715 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:31.749 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:31.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:31.750 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:31.795 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:53:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:53:31] "GET /metrics HTTP/1.1" 200 224772 "" "Prometheus/2.43.0" 2024-09-11T08:53:31.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-11T08:53:31.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:31.886 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 27 16 60633 60617 140.276 103.812 0.00779678 0.00712422 2024-09-11T08:53:31.923 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:31.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:31.924 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:32.025 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:32.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:32.065 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:32.065 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:32.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:32.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:32.202 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:32.202 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:32.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:32.337 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:32.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:32.338 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:32 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:53:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:32 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:53:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:32 smithi145 ceph-mon[33384]: osdmap e307: 8 total, 8 up, 8 in 2024-09-11T08:53:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:32 smithi145 ceph-mon[33384]: pgmap v669: 158 pgs: 158 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-11T08:53:32.438 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:32.471 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:32.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:32.472 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:32 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:53:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:32 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:53:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:32 smithi092 ceph-mon[29647]: osdmap e307: 8 total, 8 up, 8 in 2024-09-11T08:53:32.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:32 smithi092 ceph-mon[29647]: pgmap v669: 158 pgs: 158 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-11T08:53:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:32 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:53:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:32 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:53:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:32 smithi092 ceph-mon[41114]: osdmap e307: 8 total, 8 up, 8 in 2024-09-11T08:53:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:32 smithi092 ceph-mon[41114]: pgmap v669: 158 pgs: 158 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-11T08:53:32.572 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:32.605 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:32.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:32.606 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:32.707 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:32.711 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:32.738 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:32.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:32.739 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:32.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:32.888 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 28 16 62151 62135 138.647 94.875 0.0132534 0.00720745 2024-09-11T08:53:32.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:32.890 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:32.890 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:32.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:33.035 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:33.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:33.036 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:33.128 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:53:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:53:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:53:33.137 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:33.147 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:53:33.172 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:33.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:33.173 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:33.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:33.311 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:33.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:33.312 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:33.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:33 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/704815698' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:53:33.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:33 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:53:33.413 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:33.451 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:33.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:33.452 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:33.528 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:33 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/704815698' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:53:33.528 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:33 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:53:33.529 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:33 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/704815698' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:53:33.529 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:33 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:53:33.552 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:33.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:33.588 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:33.588 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:33.689 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:33.725 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:33.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:33.726 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:33.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:33.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:33.864 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:33.888 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 29 16 63576 63560 136.937 89.0625 0.0101292 0.00729807 2024-09-11T08:53:33.965 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:34.003 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:34.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:34.004 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:34.105 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:34.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:34.144 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:34.144 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:34.245 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:34.277 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:34.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:34.278 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:34.379 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:34.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:34 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:53:34.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:34 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:53:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:34 smithi145 ceph-mon[33384]: osdmap e308: 8 total, 8 up, 8 in 2024-09-11T08:53:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:34 smithi145 ceph-mon[33384]: pgmap v671: 158 pgs: 158 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 137 MiB/s wr, 2.19k op/s 2024-09-11T08:53:34.421 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:34.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:34.422 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:34.523 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:34.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:34.560 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:34.560 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:34.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:34 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:53:34.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:34 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:53:34.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:34 smithi092 ceph-mon[29647]: osdmap e308: 8 total, 8 up, 8 in 2024-09-11T08:53:34.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:34 smithi092 ceph-mon[29647]: pgmap v671: 158 pgs: 158 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 137 MiB/s wr, 2.19k op/s 2024-09-11T08:53:34.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:34 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:53:34.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:34 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:53:34.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:34 smithi092 ceph-mon[41114]: osdmap e308: 8 total, 8 up, 8 in 2024-09-11T08:53:34.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:34 smithi092 ceph-mon[41114]: pgmap v671: 158 pgs: 158 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 137 MiB/s wr, 2.19k op/s 2024-09-11T08:53:34.661 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:34.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:34.697 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:34.697 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:34.798 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:34.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:34.837 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:34.837 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:34.888 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 30 16 65027 65011 135.396 90.6875 0.0129776 0.00738135 2024-09-11T08:53:34.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:34.974 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:34.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:34.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:35.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:35.109 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:35.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:35.110 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:35.211 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:35.252 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:35.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:35.253 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:35.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:35.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:35.387 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:35.387 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:35.488 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:35.529 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:35.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:35.530 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:35.631 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:35.663 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:35.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:35.664 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:35.765 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:35.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:35.807 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:35.807 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:35.888 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 31 16 66489 66473 133.976 91.375 0.0111546 0.00745677 2024-09-11T08:53:35.907 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:35.928 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:35.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:35.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:36.030 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:36.065 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:36.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:36.066 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:36.167 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:36.196 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:36.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:36.197 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:36.298 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:36.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:36.337 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:36.338 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:36.439 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:36.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:36.476 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:36.476 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:36.577 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:36.615 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:36.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:36.616 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:36.648 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:36 smithi092 ceph-mon[29647]: pgmap v672: 158 pgs: 158 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-11T08:53:36.648 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:36 smithi092 ceph-mon[41114]: pgmap v672: 158 pgs: 158 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-11T08:53:36.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:36 smithi145 ceph-mon[33384]: pgmap v672: 158 pgs: 158 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-11T08:53:36.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:36.752 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:36.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:36.753 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:36.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:36.887 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:36.888 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 32 16 67832 67816 132.412 83.9375 0.00816383 0.00754734 2024-09-11T08:53:36.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:36.889 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:36.990 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:37.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:37.035 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:37.035 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:37.136 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:37.161 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:53:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:53:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:37.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:37.173 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:37.274 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:37.311 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:37.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:37.312 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:37.413 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:37.450 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:37.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:37.451 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:37.552 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:37.588 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:37.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:37.589 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:37.647 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:37 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:53:37.647 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:37 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:53:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:37 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:53:37.690 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:37.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:37.726 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:37.726 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:37.778 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:37.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:37.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:37.886 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:37.886 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:37.889 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 33 16 69167 69151 130.928 83.4375 0.00808052 0.00763251 2024-09-11T08:53:37.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:38.041 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:38.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:38.042 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:53:38.106 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:53:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "epoch": 308, 2024-09-11T08:53:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "fsid": "211625d4-7018-11ef-bcea-c7b262605968", 2024-09-11T08:53:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "created": "2024-09-11T08:33:32.174444+0000", 2024-09-11T08:53:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "modified": "2024-09-11T08:53:33.127006+0000", 2024-09-11T08:53:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "last_up_change": "2024-09-11T08:38:23.097480+0000", 2024-09-11T08:53:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "last_in_change": "2024-09-11T08:52:17.544985+0000", 2024-09-11T08:53:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T08:53:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_num": 5806080, 2024-09-11T08:53:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_set": [ 2024-09-11T08:53:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "nodeep-scrub", 2024-09-11T08:53:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "noscrub", 2024-09-11T08:53:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "pglog_hardlimit", 2024-09-11T08:53:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "purged_snapdirs", 2024-09-11T08:53:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_deletes", 2024-09-11T08:53:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "sortbitwise" 2024-09-11T08:53:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:53:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_version": 18, 2024-09-11T08:53:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T08:53:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T08:53:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T08:53:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_snapshot": "", 2024-09-11T08:53:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_max": 10, 2024-09-11T08:53:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd": 8, 2024-09-11T08:53:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "require_min_compat_client": "luminous", 2024-09-11T08:53:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "min_compat_client": "luminous", 2024-09-11T08:53:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "require_osd_release": "quincy", 2024-09-11T08:53:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "pools": [ 2024-09-11T08:53:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 1, 2024-09-11T08:53:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": ".mgr", 2024-09-11T08:53:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:36:29.606460+0000", 2024-09-11T08:53:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:53:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:53:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:53:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:53:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:53:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:53:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:53:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:53:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:53:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:53:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:53:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:53:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 11, 2024-09-11T08:53:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 11, 2024-09-11T08:53:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 11, 2024-09-11T08:53:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 11, 2024-09-11T08:53:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 11, 2024-09-11T08:53:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:53:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:53:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:53:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:53:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:53:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:53:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:53:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "250", 2024-09-11T08:53:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:53:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:53:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "248", 2024-09-11T08:53:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:53:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:53:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:53:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:53:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:53:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:53:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:53:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:53:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:53:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:53:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:53:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:53:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:53:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:53:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:53:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:53:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:53:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:53:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:53:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:53:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:53:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:53:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:53:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:53:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:53:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:53:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:53:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:53:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:53:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:53:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:53:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:53:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:53:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:53:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "options": { 2024-09-11T08:53:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_max": 32, 2024-09-11T08:53:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_min": 1 2024-09-11T08:53:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:53:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": {} 2024-09-11T08:53:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 2, 2024-09-11T08:53:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "datapool", 2024-09-11T08:53:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:37.674272+0000", 2024-09-11T08:53:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 8193, 2024-09-11T08:53:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T08:53:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:53:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:53:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:53:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:53:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:53:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:53:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:53:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:53:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:53:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:53:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 3, 2024-09-11T08:53:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 3, 2024-09-11T08:53:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 3, 2024-09-11T08:53:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 3, 2024-09-11T08:53:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 3, 2024-09-11T08:53:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:53:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:53:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:53:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:53:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:53:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:53:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:53:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "60", 2024-09-11T08:53:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:53:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:53:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:53:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:53:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:53:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 3, 2024-09-11T08:53:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 60, 2024-09-11T08:53:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:53:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:53:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:53:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:53:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:53:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:53:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:53:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:53:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:53:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:53:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:53:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:53:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:53:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:53:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:53:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:53:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:53:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:53:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:53:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:53:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:53:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:53:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:53:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:53:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:53:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:53:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:53:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:53:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:53:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:53:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:53:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:53:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd": {} 2024-09-11T08:53:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 3, 2024-09-11T08:53:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": ".rgw.root", 2024-09-11T08:53:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:37.912906+0000", 2024-09-11T08:53:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:53:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:53:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:53:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:53:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:53:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:53:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:53:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:53:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:53:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:53:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:53:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:53:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:53:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:53:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:53:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:53:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:53:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:53:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:53:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:53:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:53:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:53:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:53:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:53:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "57", 2024-09-11T08:53:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:53:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:53:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:53:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:53:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:53:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:53:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:53:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:53:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:53:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:53:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:53:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:53:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:53:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:53:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:53:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:53:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:53:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:53:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:53:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:53:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:53:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:53:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:53:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:53:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:53:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:53:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:53:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:53:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:53:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:53:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:53:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:53:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:53:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:53:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:53:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:53:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:53:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:53:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:53:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:53:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 4, 2024-09-11T08:53:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "default.rgw.log", 2024-09-11T08:53:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:39.234173+0000", 2024-09-11T08:53:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:53:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:53:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:53:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:53:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:53:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:53:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:53:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:53:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:53:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:53:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:53:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:53:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:53:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:53:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:53:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:53:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:53:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:53:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:53:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:53:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:53:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:53:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:53:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:53:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "59", 2024-09-11T08:53:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:53:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:53:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:53:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:53:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:53:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:53:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:53:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:53:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:53:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:53:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:53:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:53:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:53:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:53:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:53:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:53:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:53:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:53:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:53:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:53:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:53:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:53:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:53:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:53:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:53:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:53:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:53:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:53:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:53:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:53:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:53:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:53:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:53:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:53:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:53:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:53:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:53:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:53:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:53:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:53:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 5, 2024-09-11T08:53:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "default.rgw.control", 2024-09-11T08:53:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:41.216386+0000", 2024-09-11T08:53:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:53:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:53:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:53:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:53:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:53:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:53:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:53:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:53:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:53:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:53:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:53:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:53:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:53:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:53:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:53:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:53:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:53:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:53:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:53:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:53:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:53:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:53:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:53:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:53:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "61", 2024-09-11T08:53:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:53:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:53:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:53:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:53:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:53:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:53:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:53:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:53:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:53:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:53:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:53:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:53:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:53:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:53:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:53:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:53:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:53:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:53:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:53:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:53:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:53:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:53:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:53:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:53:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:53:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:53:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:53:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:53:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:53:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:53:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:53:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:53:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:53:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:53:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:53:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:53:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:53:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:53:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:53:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:53:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 6, 2024-09-11T08:53:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "default.rgw.meta", 2024-09-11T08:53:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:43.285497+0000", 2024-09-11T08:53:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:53:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:53:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:53:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:53:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:53:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:53:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:53:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:53:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:53:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:53:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:53:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:53:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:53:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:53:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:53:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:53:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:53:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:53:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:53:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:53:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:53:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:53:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:53:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:53:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "63", 2024-09-11T08:53:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:53:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:53:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:53:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:53:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:53:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:53:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:53:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:53:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:53:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:53:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:53:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:53:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:53:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:53:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:53:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:53:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:53:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:53:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:53:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:53:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:53:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:53:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:53:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:53:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:53:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:53:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:53:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:53:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:53:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:53:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:53:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:53:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:53:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:53:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:53:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:53:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:53:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "options": { 2024-09-11T08:53:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_bias": 4 2024-09-11T08:53:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:53:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:53:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 10, 2024-09-11T08:53:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "unique_pool_3", 2024-09-11T08:53:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:52:53.990481+0000", 2024-09-11T08:53:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:53:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:53:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:53:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:53:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:53:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:53:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:53:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:53:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:53:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:53:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:53:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:53:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 16, 2024-09-11T08:53:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 16, 2024-09-11T08:53:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 16, 2024-09-11T08:53:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 16, 2024-09-11T08:53:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 16, 2024-09-11T08:53:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:53:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:53:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:53:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:53:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:53:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:53:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:53:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "297", 2024-09-11T08:53:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:53:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:53:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:53:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:53:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:53:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:53:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:53:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:53:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:53:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:53:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:53:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:53:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:53:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:53:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:53:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:53:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:53:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:53:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:53:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:53:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:53:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:53:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:53:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:53:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:53:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:53:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:53:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:53:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:53:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:53:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:53:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:53:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:53:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:53:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:53:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:53:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:53:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:53:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:53:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "rados": {} 2024-09-11T08:53:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:53:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "osds": [ 2024-09-11T08:53:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-09-11T08:53:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "9d8a3811-e801-4b6a-9a63-83936283cb7d", 2024-09-11T08:53:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:53:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:53:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:53:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 0.1711883544921875, 2024-09-11T08:53:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:53:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:53:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 8, 2024-09-11T08:53:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 286, 2024-09-11T08:53:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:53:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:53:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:53:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6802", 2024-09-11T08:53:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:53:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6803", 2024-09-11T08:53:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:53:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:53:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6804", 2024-09-11T08:53:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:53:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6805", 2024-09-11T08:53:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:53:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:53:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6808", 2024-09-11T08:53:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:53:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6809", 2024-09-11T08:53:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:53:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:53:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6806", 2024-09-11T08:53:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:53:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6807", 2024-09-11T08:53:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:53:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6803/2040613580", 2024-09-11T08:53:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6805/2040613580", 2024-09-11T08:53:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6809/2040613580", 2024-09-11T08:53:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6807/2040613580", 2024-09-11T08:53:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:53:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:53:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:53:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-09-11T08:53:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "e7a18e45-0137-45ef-8b85-e458cf3737ac", 2024-09-11T08:53:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:53:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:53:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:53:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:53:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:53:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:53:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 13, 2024-09-11T08:53:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 293, 2024-09-11T08:53:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:53:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:53:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:53:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6810", 2024-09-11T08:53:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:53:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6811", 2024-09-11T08:53:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:53:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:53:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6812", 2024-09-11T08:53:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:53:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6813", 2024-09-11T08:53:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:53:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:53:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6816", 2024-09-11T08:53:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:53:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6817", 2024-09-11T08:53:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:53:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:53:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6814", 2024-09-11T08:53:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:53:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6815", 2024-09-11T08:53:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:53:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6811/544779328", 2024-09-11T08:53:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6813/544779328", 2024-09-11T08:53:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6817/544779328", 2024-09-11T08:53:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6815/544779328", 2024-09-11T08:53:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:53:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:53:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:53:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-09-11T08:53:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "017282e2-8053-4fa7-affc-a05408fa481e", 2024-09-11T08:53:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:53:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:53:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:53:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:53:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:53:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:53:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 18, 2024-09-11T08:53:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 293, 2024-09-11T08:53:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:53:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:53:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:53:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6818", 2024-09-11T08:53:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:53:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6819", 2024-09-11T08:53:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:53:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:53:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6820", 2024-09-11T08:53:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:53:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6821", 2024-09-11T08:53:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:53:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:53:38.164 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6824", 2024-09-11T08:53:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:53:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6825", 2024-09-11T08:53:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:53:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:53:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6822", 2024-09-11T08:53:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:53:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6823", 2024-09-11T08:53:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:53:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6819/933736777", 2024-09-11T08:53:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6821/933736777", 2024-09-11T08:53:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6825/933736777", 2024-09-11T08:53:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6823/933736777", 2024-09-11T08:53:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:53:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:53:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:53:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-09-11T08:53:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "58e5a8a4-9e58-40ad-9b40-90cb5ff5414b", 2024-09-11T08:53:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:53:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:53:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:53:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:53:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:53:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:53:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 26, 2024-09-11T08:53:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 290, 2024-09-11T08:53:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:53:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:53:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:53:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6826", 2024-09-11T08:53:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:53:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6827", 2024-09-11T08:53:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:53:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:53:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.169 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6828", 2024-09-11T08:53:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:53:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6829", 2024-09-11T08:53:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:53:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:53:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.170 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6832", 2024-09-11T08:53:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:53:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6833", 2024-09-11T08:53:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:53:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:53:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.171 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6830", 2024-09-11T08:53:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:53:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6831", 2024-09-11T08:53:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:53:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6827/878892459", 2024-09-11T08:53:38.172 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6829/878892459", 2024-09-11T08:53:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6833/878892459", 2024-09-11T08:53:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6831/878892459", 2024-09-11T08:53:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:53:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:53:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:53:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-09-11T08:53:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "e144e102-963b-48b7-92a9-9391706164bf", 2024-09-11T08:53:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:53:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:53:38.173 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:53:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:53:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:53:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:53:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 31, 2024-09-11T08:53:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 293, 2024-09-11T08:53:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:53:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:53:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:53:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6800", 2024-09-11T08:53:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:53:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6801", 2024-09-11T08:53:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:53:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:53:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6802", 2024-09-11T08:53:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:53:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6803", 2024-09-11T08:53:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:53:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:53:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6806", 2024-09-11T08:53:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:53:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6807", 2024-09-11T08:53:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:53:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:53:38.177 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6804", 2024-09-11T08:53:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:53:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6805", 2024-09-11T08:53:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:53:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6801/2792751669", 2024-09-11T08:53:38.178 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6803/2792751669", 2024-09-11T08:53:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6807/2792751669", 2024-09-11T08:53:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6805/2792751669", 2024-09-11T08:53:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:53:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:53:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:53:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-09-11T08:53:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "811b0dc9-b7a7-439a-86c1-5dc9d0d54e16", 2024-09-11T08:53:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:53:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:53:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:53:38.179 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:53:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:53:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:53:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 38, 2024-09-11T08:53:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 293, 2024-09-11T08:53:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:53:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:53:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:53:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6808", 2024-09-11T08:53:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:53:38.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6809", 2024-09-11T08:53:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:53:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:53:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.181 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.182 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6810", 2024-09-11T08:53:38.182 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:53:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6811", 2024-09-11T08:53:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:53:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:53:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.183 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6814", 2024-09-11T08:53:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:53:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6815", 2024-09-11T08:53:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:53:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:53:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6812", 2024-09-11T08:53:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:53:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6813", 2024-09-11T08:53:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:53:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.185 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6809/3569637188", 2024-09-11T08:53:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6811/3569637188", 2024-09-11T08:53:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6815/3569637188", 2024-09-11T08:53:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6813/3569637188", 2024-09-11T08:53:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:53:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:53:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:53:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-09-11T08:53:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "bd162f52-097c-4c5d-9d37-1289779a196b", 2024-09-11T08:53:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:53:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:53:38.186 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:53:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:53:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:53:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:53:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 45, 2024-09-11T08:53:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 293, 2024-09-11T08:53:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:53:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:53:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:53:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6816", 2024-09-11T08:53:38.187 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:53:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6817", 2024-09-11T08:53:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:53:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:53:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6818", 2024-09-11T08:53:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:53:38.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6819", 2024-09-11T08:53:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:53:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:53:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6822", 2024-09-11T08:53:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:53:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6823", 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6820", 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.190 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6821", 2024-09-11T08:53:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:53:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6817/2615047110", 2024-09-11T08:53:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6819/2615047110", 2024-09-11T08:53:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6823/2615047110", 2024-09-11T08:53:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6821/2615047110", 2024-09-11T08:53:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:53:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:53:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:53:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.191 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-09-11T08:53:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "4e4021bf-78c7-4bc6-b222-3494f724ddec", 2024-09-11T08:53:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:53:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:53:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:53:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 0.2356109619140625, 2024-09-11T08:53:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:53:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:53:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 52, 2024-09-11T08:53:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 290, 2024-09-11T08:53:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:53:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:53:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:53:38.192 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6824", 2024-09-11T08:53:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:53:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6825", 2024-09-11T08:53:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:53:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:53:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6826", 2024-09-11T08:53:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:53:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6827", 2024-09-11T08:53:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:53:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:53:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.194 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6830", 2024-09-11T08:53:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:53:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6831", 2024-09-11T08:53:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:53:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:53:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:53:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.195 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:53:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6828", 2024-09-11T08:53:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:53:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:53:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6829", 2024-09-11T08:53:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:53:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6825/3195898530", 2024-09-11T08:53:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6827/3195898530", 2024-09-11T08:53:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6831/3195898530", 2024-09-11T08:53:38.196 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6829/3195898530", 2024-09-11T08:53:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:53:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:53:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:53:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:53:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_xinfo": [ 2024-09-11T08:53:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-09-11T08:53:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:53:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:53:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:53:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:53:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:53:38.197 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:52:59.867957+0000", 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:53:01.525131+0000", 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:53:38.198 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:53:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:53:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:53:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:53:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:53:19.141725+0000", 2024-09-11T08:53:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:53:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-09-11T08:53:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:53:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:53:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:53:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:53:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:53:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:53:12.657587+0000", 2024-09-11T08:53:38.199 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:53:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-09-11T08:53:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:53:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:53:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:53:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:53:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:53:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:52:32.253007+0000", 2024-09-11T08:53:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:53:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-09-11T08:53:38.200 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:53:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:53:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:53:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:53:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:53:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:52:27.822236+0000", 2024-09-11T08:53:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:53:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-09-11T08:53:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:53:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:53:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:53:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:53:38.201 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:53:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:53:31.039705+0000", 2024-09-11T08:53:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:53:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-09-11T08:53:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:53:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:53:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:53:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:53:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:53:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:52:56.597092+0000", 2024-09-11T08:53:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:53:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:53:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap": [], 2024-09-11T08:53:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap_items": [ 2024-09-11T08:53:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "3.5", 2024-09-11T08:53:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:53:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 5, 2024-09-11T08:53:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T08:53:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.e", 2024-09-11T08:53:38.203 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:53:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:53:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T08:53:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.12", 2024-09-11T08:53:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:53:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:53:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 5 2024-09-11T08:53:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.204 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.13", 2024-09-11T08:53:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:53:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:53:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 5 2024-09-11T08:53:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.205 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.17", 2024-09-11T08:53:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:53:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:53:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T08:53:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.1d", 2024-09-11T08:53:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:53:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:53:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:53:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T08:53:38.206 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:53:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:53:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_temp": [], 2024-09-11T08:53:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_temp": [], 2024-09-11T08:53:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklist": { 2024-09-11T08:53:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6800/1861854063": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:53:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/4113910808": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:53:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/3501902574": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:53:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/1583695466": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:53:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/736380628": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:53:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6800/4060453592": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:53:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/2234574262": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:53:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6800/2480574848": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:53:38.207 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/1540079149": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:53:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/2680744738": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:53:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6801/4060453592": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:53:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6801/1861854063": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:53:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6801/2480574848": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:53:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/12697418": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:53:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/4280958069": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:53:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/2686720854": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:53:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/969221731": "2024-09-12T08:39:14.984442+0000" 2024-09-11T08:53:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "range_blocklist": {}, 2024-09-11T08:53:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profiles": { 2024-09-11T08:53:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "default": { 2024-09-11T08:53:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-failure-domain": "osd", 2024-09-11T08:53:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "k": "2", 2024-09-11T08:53:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "m": "1", 2024-09-11T08:53:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin": "jerasure", 2024-09-11T08:53:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "technique": "reed_sol_van" 2024-09-11T08:53:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:53:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps_queue": [], 2024-09-11T08:53:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "new_removed_snaps": [], 2024-09-11T08:53:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "new_purged_snaps": [], 2024-09-11T08:53:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_node_flags": {}, 2024-09-11T08:53:38.209 INFO:teuthology.orchestra.run.smithi092.stdout: "device_class_flags": {}, 2024-09-11T08:53:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode": { 2024-09-11T08:53:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_enabled": false, 2024-09-11T08:53:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_bucket_count": 0, 2024-09-11T08:53:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: "degraded_stretch_mode": 0, 2024-09-11T08:53:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: "recovering_stretch_mode": 0, 2024-09-11T08:53:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_bucket": 0 2024-09-11T08:53:38.210 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:53:38.210 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:53:38.250 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:38.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:38.251 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:38.352 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:38.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:38.389 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:38.389 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:38.490 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:38.525 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:38.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:38.527 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:38.627 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:38.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:38 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:38.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:38 smithi145 ceph-mon[33384]: pgmap v673: 158 pgs: 158 active+clean; 4.1 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 123 MiB/s wr, 1.97k op/s 2024-09-11T08:53:38.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:38 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:53:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:38 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:53:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:38 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:53:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:38 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1904299830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T08:53:38.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:38.668 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:38.668 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:38.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:38 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:38.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:38 smithi092 ceph-mon[29647]: pgmap v673: 158 pgs: 158 active+clean; 4.1 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 123 MiB/s wr, 1.97k op/s 2024-09-11T08:53:38.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:38 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:53:38.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:38 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:53:38.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:38 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:53:38.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:38 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1904299830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T08:53:38.678 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:38 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:38.678 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:38 smithi092 ceph-mon[41114]: pgmap v673: 158 pgs: 158 active+clean; 4.1 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 123 MiB/s wr, 1.97k op/s 2024-09-11T08:53:38.679 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:38 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:53:38.679 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:38 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:53:38.679 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:38 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:53:38.679 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:38 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1904299830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T08:53:38.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:38.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:38.807 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:38.807 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:38.888 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 34 16 70539 70523 129.599 85.75 0.0133593 0.00771144 2024-09-11T08:53:38.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:38.944 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:38.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:38.948 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:39.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:39.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:39.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:39.184 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:39.223 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:39.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:39.224 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:39.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:39.359 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:39.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:39.360 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:39.461 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:39.495 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:39.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:39.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:39.562 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:53:39.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:39.647 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:39.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:39.649 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:39.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:39 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2379019371' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:53:39.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:39 smithi092 ceph-mon[29647]: osdmap e309: 8 total, 8 up, 8 in 2024-09-11T08:53:39.673 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:39 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2379019371' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:53:39.673 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:39 smithi092 ceph-mon[41114]: osdmap e309: 8 total, 8 up, 8 in 2024-09-11T08:53:39.749 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:39.764 INFO:tasks.thrashosds.thrasher:j is {'epoch': 308, 'fsid': '211625d4-7018-11ef-bcea-c7b262605968', 'created': '2024-09-11T08:33:32.174444+0000', 'modified': '2024-09-11T08:53:33.127006+0000', 'last_up_change': '2024-09-11T08:38:23.097480+0000', 'last_in_change': '2024-09-11T08:52:17.544985+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 10, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T08:36:29.606460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '250', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '248', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-11T08:38:37.674272+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-11T08:38:37.912906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-11T08:38:39.234173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-11T08:38:41.216386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-11T08:38:43.285497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 10, 'pool_name': 'unique_pool_3', 'create_time': '2024-09-11T08:52:53.990481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '297', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '9d8a3811-e801-4b6a-9a63-83936283cb7d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1711883544921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 286, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 2040613580}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 2040613580}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 2040613580}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 2040613580}]}, 'public_addr': '172.21.15.92:6803/2040613580', 'cluster_addr': '172.21.15.92:6805/2040613580', 'heartbeat_back_addr': '172.21.15.92:6809/2040613580', 'heartbeat_front_addr': '172.21.15.92:6807/2040613580', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e7a18e45-0137-45ef-8b85-e458cf3737ac', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 544779328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 544779328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 544779328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 544779328}]}, 'public_addr': '172.21.15.92:6811/544779328', 'cluster_addr': '172.21.15.92:6813/544779328', 'heartbeat_back_addr': '172.21.15.92:6817/544779328', 'heartbeat_front_addr': '172.21.15.92:6815/544779328', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '017282e2-8053-4fa7-affc-a05408fa481e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 933736777}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 933736777}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 933736777}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 933736777}]}, 'public_addr': '172.21.15.92:6819/933736777', 'cluster_addr': '172.21.15.92:6821/933736777', 'heartbeat_back_addr': '172.21.15.92:6825/933736777', 'heartbeat_front_addr': '172.21.15.92:6823/933736777', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '58e5a8a4-9e58-40ad-9b40-90cb5ff5414b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 290, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 878892459}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 878892459}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 878892459}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 878892459}]}, 'public_addr': '172.21.15.92:6827/878892459', 'cluster_addr': '172.21.15.92:6829/878892459', 'heartbeat_back_addr': '172.21.15.92:6833/878892459', 'heartbeat_front_addr': '172.21.15.92:6831/878892459', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e144e102-963b-48b7-92a9-9391706164bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6800', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6801', 'nonce': 2792751669}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6802', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6803', 'nonce': 2792751669}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6806', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6807', 'nonce': 2792751669}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6804', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6805', 'nonce': 2792751669}]}, 'public_addr': '172.21.15.145:6801/2792751669', 'cluster_addr': '172.21.15.145:6803/2792751669', 'heartbeat_back_addr': '172.21.15.145:6807/2792751669', 'heartbeat_front_addr': '172.21.15.145:6805/2792751669', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '811b0dc9-b7a7-439a-86c1-5dc9d0d54e16', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 38, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6808', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6809', 'nonce': 3569637188}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6810', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6811', 'nonce': 3569637188}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6814', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6815', 'nonce': 3569637188}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6812', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6813', 'nonce': 3569637188}]}, 'public_addr': '172.21.15.145:6809/3569637188', 'cluster_addr': '172.21.15.145:6811/3569637188', 'heartbeat_back_addr': '172.21.15.145:6815/3569637188', 'heartbeat_front_addr': '172.21.15.145:6813/3569637188', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'bd162f52-097c-4c5d-9d37-1289779a196b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6816', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6817', 'nonce': 2615047110}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6818', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6819', 'nonce': 2615047110}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6822', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6823', 'nonce': 2615047110}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6820', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6821', 'nonce': 2615047110}]}, 'public_addr': '172.21.15.145:6817/2615047110', 'cluster_addr': '172.21.15.145:6819/2615047110', 'heartbeat_back_addr': '172.21.15.145:6823/2615047110', 'heartbeat_front_addr': '172.21.15.145:6821/2615047110', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '4e4021bf-78c7-4bc6-b222-3494f724ddec', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.2356109619140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 52, 'up_thru': 290, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6824', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6825', 'nonce': 3195898530}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6826', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6827', 'nonce': 3195898530}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6830', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6831', 'nonce': 3195898530}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6828', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6829', 'nonce': 3195898530}]}, 'public_addr': '172.21.15.145:6825/3195898530', 'cluster_addr': '172.21.15.145:6827/3195898530', 'heartbeat_back_addr': '172.21.15.145:6831/3195898530', 'heartbeat_front_addr': '172.21.15.145:6829/3195898530', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:52:59.867957+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:53:01.525131+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:53:19.141725+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:53:12.657587+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:52:32.253007+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:52:27.822236+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:53:31.039705+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:52:56.597092+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.5', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '4.e', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.12', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.1d', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.92:6800/1861854063': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/4113910808': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:0/3501902574': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/1583695466': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:0/736380628': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:6800/4060453592': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:0/2234574262': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:6800/2480574848': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/1540079149': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/2680744738': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:6801/4060453592': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:6801/1861854063': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:6801/2480574848': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/12697418': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/4280958069': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/2686720854': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/969221731': '2024-09-12T08:39:14.984442+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T08:53:39.764 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:53:39.793 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:39.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:39.794 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:39.890 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 35 16 71879 71863 128.281 83.75 0.0109028 0.00779007 2024-09-11T08:53:39.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:39.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:39 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2379019371' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:53:39.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:39 smithi145 ceph-mon[33384]: osdmap e309: 8 total, 8 up, 8 in 2024-09-11T08:53:39.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:39.940 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:39.940 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:40.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:40.082 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:40.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:40.083 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:40.103 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:53:40.104 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:53:40.104 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:53:40.104 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:53:40.104 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:53:40.104 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:53:40.104 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:53:40.104 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:53:40.104 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:53:40.105 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:53:40.105 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:53:40.105 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:53:40.105 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:53:40.105 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:53:40.105 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:53:40.105 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:53:40.185 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:40.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:40.221 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:40.222 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:40.322 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:40.354 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:40.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:40.356 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:40.456 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:40.498 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:40.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:40.499 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:40.599 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:40.640 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:40.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:40.641 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:40.742 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:40.777 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:40.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:40.778 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:40.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:40 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:53:40.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:40 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:53:40.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:40 smithi092 ceph-mon[29647]: pgmap v674: 158 pgs: 158 active+clean; 4.1 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 120 MiB/s wr, 1.92k op/s 2024-09-11T08:53:40.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:40 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:53:40.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:40 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:53:40.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:40 smithi092 ceph-mon[41114]: pgmap v674: 158 pgs: 158 active+clean; 4.1 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 120 MiB/s wr, 1.92k op/s 2024-09-11T08:53:40.880 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:40.892 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 36 15 73220 73205 127.042 83.875 0.0162037 0.00786589 2024-09-11T08:53:40.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:40.910 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:40.911 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:40.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:40 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:53:40.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:40 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:53:40.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:40 smithi145 ceph-mon[33384]: pgmap v674: 158 pgs: 158 active+clean; 4.1 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 120 MiB/s wr, 1.92k op/s 2024-09-11T08:53:41.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:41.052 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:41.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:41.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:41.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:41.194 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:41.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:41.195 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:41.296 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:41.334 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:41.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:41.336 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:41.404 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:41.415 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:41.436 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:41.448 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:53:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:53:41] "GET /metrics HTTP/1.1" 200 224772 "" "Prometheus/2.43.0" 2024-09-11T08:53:41.469 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:41.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:41.470 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:41.570 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:41.605 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:41.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:41.606 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:41.706 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:41.743 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:41.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:41.744 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:41.845 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:41.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:41.883 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:41.883 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:41.892 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 37 16 74643 74627 126.01 88.875 0.0144861 0.00793117 2024-09-11T08:53:41.984 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:42.017 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:42.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:42.018 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:42.118 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:42.159 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:42.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:42.161 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:42.262 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:42.305 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:42.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:42.306 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:42.406 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:42.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:42.436 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:42.437 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:42.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:42 smithi092 ceph-mon[29647]: pgmap v676: 158 pgs: 158 active+clean; 4.4 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 626 B/s rd, 89 MiB/s wr, 1.42k op/s 2024-09-11T08:53:42.511 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:42 smithi092 ceph-mon[41114]: pgmap v676: 158 pgs: 158 active+clean; 4.4 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 626 B/s rd, 89 MiB/s wr, 1.42k op/s 2024-09-11T08:53:42.538 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:42.576 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:42.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:42.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:42.678 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:42.708 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:42.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:42.709 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:42.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:42 smithi145 ceph-mon[33384]: pgmap v676: 158 pgs: 158 active+clean; 4.4 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 626 B/s rd, 89 MiB/s wr, 1.42k op/s 2024-09-11T08:53:42.810 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:42.840 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:42.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:42.841 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:42.884 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:53:42.893 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 38 16 75895 75879 124.75 78.25 0.00678734 0.00801107 2024-09-11T08:53:42.943 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:42.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:42.979 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:42.979 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:43.080 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:43.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:43.117 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:43.118 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:43.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:53:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:53:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:53:43.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:43.253 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:43.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:43.254 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:43.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:43.391 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:43.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:43.392 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:43.492 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:43.528 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:43.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:43.529 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:43.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:43.669 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:43.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:43.670 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:43.771 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:43.810 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:43.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:43.811 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:43.894 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 39 16 77260 77244 123.739 85.3125 0.00832436 0.00807657 2024-09-11T08:53:43.911 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:43.947 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:43.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:43.948 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:44.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:44.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:44.086 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:44.086 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:44.187 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:44.222 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:44.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:44.223 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:44.324 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:44.360 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:44.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:44.361 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:44.462 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:44.498 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:44.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:44.499 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:44.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:44 smithi092 ceph-mon[41114]: pgmap v677: 158 pgs: 158 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 88 MiB/s wr, 1.42k op/s 2024-09-11T08:53:44.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:44 smithi092 ceph-mon[41114]: osdmap e310: 8 total, 8 up, 8 in 2024-09-11T08:53:44.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:44 smithi092 ceph-mon[29647]: pgmap v677: 158 pgs: 158 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 88 MiB/s wr, 1.42k op/s 2024-09-11T08:53:44.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:44 smithi092 ceph-mon[29647]: osdmap e310: 8 total, 8 up, 8 in 2024-09-11T08:53:44.599 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:44.635 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:44.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:44.636 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:44.737 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:44.771 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:44.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:44.772 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:44.873 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:44.893 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:53:44.895052+0000 min lat: 0.00119342 max lat: 0.169837 avg lat: 0.00814282 2024-09-11T08:53:44.894 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:53:44.894 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 40 16 78595 78579 122.732 83.4375 0.0166057 0.00814282 2024-09-11T08:53:44.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:44.911 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:44.912 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:44 smithi145 ceph-mon[33384]: pgmap v677: 158 pgs: 158 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 88 MiB/s wr, 1.42k op/s 2024-09-11T08:53:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:44 smithi145 ceph-mon[33384]: osdmap e310: 8 total, 8 up, 8 in 2024-09-11T08:53:45.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:45.054 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:45.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:45.055 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:45.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:45.190 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:45.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:45.191 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:45.292 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:45.323 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:45.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:45.324 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:45.425 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:45.459 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:45.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:45.460 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:45.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:45.596 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:53:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:53:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:53:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:53:45.597 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:53:45.601 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:45.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:45.602 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:45.703 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:45.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:45.737 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:45.737 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:45.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:53:45.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:53:45.828 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:45.837 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:45.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:45.871 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:45.871 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:45.894 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 41 16 79921 79905 121.76 82.875 0.0109768 0.00820779 2024-09-11T08:53:45.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:53:45.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:46.007 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:46.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:46.008 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:46.109 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:46.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:46.144 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:46.144 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:46.245 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:46.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:46.303 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:46.403 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:46.446 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:46.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:46.447 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:46.548 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:46.588 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:46.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:46.590 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:46.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:46.697 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:46 smithi092 ceph-mon[29647]: pgmap v679: 158 pgs: 158 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 53 MiB/s wr, 847 op/s 2024-09-11T08:53:46.697 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:46 smithi092 ceph-mon[41114]: pgmap v679: 158 pgs: 158 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 53 MiB/s wr, 847 op/s 2024-09-11T08:53:46.728 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:46.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:46.730 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:46.830 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:46.855 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-11T08:53:46.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:46.866 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:46.866 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:46.894 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 42 16 81275 81259 120.876 84.625 0.0190483 0.00826803 2024-09-11T08:53:46.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:46 smithi145 ceph-mon[33384]: pgmap v679: 158 pgs: 158 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 53 MiB/s wr, 847 op/s 2024-09-11T08:53:46.968 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:47.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:47.017 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:47.017 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:47.118 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:47.143 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:47.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:47.144 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:47.198 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:53:47.205 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:53:47.245 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:47.276 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:47.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:47.277 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:47.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:47.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:53:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:53:47.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:47.416 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:47.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:47.518 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:47.555 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:47.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:47.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:47.656 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:47.692 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:47.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:47.693 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:47.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:47.832 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:47.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:47.834 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:47.894 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 43 16 82479 82463 119.815 75.25 0.0197373 0.00834166 2024-09-11T08:53:47.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:47.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:47.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:48.075 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:48.101 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":679,"stamp":"2024-09-11T08:53:45.315228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4844719566,"num_objects":74111,"num_object_clones":0,"num_object_copies":222333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74111,"num_whiteouts":0,"num_read":1232,"num_read_kb":1012,"num_write":74661,"num_write_kb":4730919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":272889,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74956,"ondisk_log_size":74956,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":476,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":19071848,"kb_used_data":14265700,"kb_used_omap":13,"kb_used_meta":4806066,"kb_avail":730889368,"statfs":{"total":767960285184,"available":748430712832,"internally_reserved":0,"allocated":14608076800,"data_stored":14604880564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":4921412288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":444334080,"num_objects":6780,"num_object_clones":0,"num_object_copies":20340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6780,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6780,"num_write_kb":433920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002753"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":117,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148487+0000","last_change":"2024-09-11T08:52:49.568717+0000","last_active":"2024-09-11T08:53:33.148487+0000","last_peered":"2024-09-11T08:53:33.148487+0000","last_clean":"2024-09-11T08:53:33.148487+0000","last_became_active":"2024-09-11T08:52:49.568416+0000","last_became_peered":"2024-09-11T08:52:49.568416+0000","last_unstale":"2024-09-11T08:53:33.148487+0000","last_undegraded":"2024-09-11T08:53:33.148487+0000","last_fullsized":"2024-09-11T08:53:33.148487+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:29.430953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:29.430953+0000","last_clean_scrub_stamp":"2024-09-11T08:51:29.430953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"200'19","reported_seq":379,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151062+0000","last_change":"2024-09-11T08:52:30.312273+0000","last_active":"2024-09-11T08:53:33.151062+0000","last_peered":"2024-09-11T08:53:33.151062+0000","last_clean":"2024-09-11T08:53:33.151062+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:53:33.151062+0000","last_undegraded":"2024-09-11T08:53:33.151062+0000","last_fullsized":"2024-09-11T08:53:33.151062+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:52:30.312197+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:52:30.312197+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:43.338489+0000","scrub_duration":0.002912516,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":317,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.145990+0000","last_change":"2024-09-11T08:50:18.611806+0000","last_active":"2024-09-11T08:53:33.145990+0000","last_peered":"2024-09-11T08:53:33.145990+0000","last_clean":"2024-09-11T08:53:33.145990+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:53:33.145990+0000","last_undegraded":"2024-09-11T08:53:33.145990+0000","last_fullsized":"2024-09-11T08:53:33.145990+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:18.611725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:18.611725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072923490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":296,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150647+0000","last_change":"2024-09-11T08:51:37.048922+0000","last_active":"2024-09-11T08:53:33.150647+0000","last_peered":"2024-09-11T08:53:33.150647+0000","last_clean":"2024-09-11T08:53:33.150647+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:53:33.150647+0000","last_undegraded":"2024-09-11T08:53:33.150647+0000","last_fullsized":"2024-09-11T08:53:33.150647+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:37.048848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:51:37.048848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029662830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":325,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.161540+0000","last_change":"2024-09-11T08:51:35.370494+0000","last_active":"2024-09-11T08:53:33.161540+0000","last_peered":"2024-09-11T08:53:33.161540+0000","last_clean":"2024-09-11T08:53:33.161540+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:53:33.161540+0000","last_undegraded":"2024-09-11T08:53:33.161540+0000","last_fullsized":"2024-09-11T08:53:33.161540+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:35.370438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:51:35.370438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053884329999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":331,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.162182+0000","last_change":"2024-09-11T08:51:28.347060+0000","last_active":"2024-09-11T08:53:33.162182+0000","last_peered":"2024-09-11T08:53:33.162182+0000","last_clean":"2024-09-11T08:53:33.162182+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:53:33.162182+0000","last_undegraded":"2024-09-11T08:53:33.162182+0000","last_fullsized":"2024-09-11T08:53:33.162182+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:28.346979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:51:28.346979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020416560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"200'34","reported_seq":411,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.160444+0000","last_change":"2024-09-11T08:52:18.609763+0000","last_active":"2024-09-11T08:53:33.160444+0000","last_peered":"2024-09-11T08:53:33.160444+0000","last_clean":"2024-09-11T08:53:33.160444+0000","last_became_active":"2024-09-11T08:52:18.585600+0000","last_became_peered":"2024-09-11T08:52:18.585600+0000","last_unstale":"2024-09-11T08:53:33.160444+0000","last_undegraded":"2024-09-11T08:53:33.160444+0000","last_fullsized":"2024-09-11T08:53:33.160444+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:51:31.358660+0000","last_deep_scrub":"200'34","last_deep_scrub_stamp":"2024-09-11T08:51:31.358660+0000","last_clean_scrub_stamp":"2024-09-11T08:51:31.358660+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071678469999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":298,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150988+0000","last_change":"2024-09-11T08:51:19.004758+0000","last_active":"2024-09-11T08:53:33.150988+0000","last_peered":"2024-09-11T08:53:33.150988+0000","last_clean":"2024-09-11T08:53:33.150988+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:53:33.150988+0000","last_undegraded":"2024-09-11T08:53:33.150988+0000","last_fullsized":"2024-09-11T08:53:33.150988+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:19.004676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:51:19.004676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006451526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":337,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.147747+0000","last_change":"2024-09-11T08:51:18.275081+0000","last_active":"2024-09-11T08:53:33.147747+0000","last_peered":"2024-09-11T08:53:33.147747+0000","last_clean":"2024-09-11T08:53:33.147747+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:53:33.147747+0000","last_undegraded":"2024-09-11T08:53:33.147747+0000","last_fullsized":"2024-09-11T08:53:33.147747+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:18.275025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:51:18.275025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011943050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"200'19","reported_seq":379,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150943+0000","last_change":"2024-09-11T08:52:18.598556+0000","last_active":"2024-09-11T08:53:33.150943+0000","last_peered":"2024-09-11T08:53:33.150943+0000","last_clean":"2024-09-11T08:53:33.150943+0000","last_became_active":"2024-09-11T08:52:18.577411+0000","last_became_peered":"2024-09-11T08:52:18.577411+0000","last_unstale":"2024-09-11T08:53:33.150943+0000","last_undegraded":"2024-09-11T08:53:33.150943+0000","last_fullsized":"2024-09-11T08:53:33.150943+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:34.003806+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:51:34.003806+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022468330000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":333,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.145553+0000","last_change":"2024-09-11T08:52:18.573865+0000","last_active":"2024-09-11T08:53:33.145553+0000","last_peered":"2024-09-11T08:53:33.145553+0000","last_clean":"2024-09-11T08:53:33.145553+0000","last_became_active":"2024-09-11T08:52:18.573461+0000","last_became_peered":"2024-09-11T08:52:18.573461+0000","last_unstale":"2024-09-11T08:53:33.145553+0000","last_undegraded":"2024-09-11T08:53:33.145553+0000","last_fullsized":"2024-09-11T08:53:33.145553+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:25.373549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:25.373549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026496549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":117,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.146164+0000","last_change":"2024-09-11T08:52:49.568661+0000","last_active":"2024-09-11T08:53:33.146164+0000","last_peered":"2024-09-11T08:53:33.146164+0000","last_clean":"2024-09-11T08:53:33.146164+0000","last_became_active":"2024-09-11T08:52:49.568147+0000","last_became_peered":"2024-09-11T08:52:49.568147+0000","last_unstale":"2024-09-11T08:53:33.146164+0000","last_undegraded":"2024-09-11T08:53:33.146164+0000","last_fullsized":"2024-09-11T08:53:33.146164+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:30.508427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:33.520462+0000","last_clean_scrub_stamp":"2024-09-11T08:52:30.508427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:59.023014+0000","scrub_duration":0.0043744099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":317,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.147178+0000","last_change":"2024-09-11T08:51:30.465078+0000","last_active":"2024-09-11T08:53:33.147178+0000","last_peered":"2024-09-11T08:53:33.147178+0000","last_clean":"2024-09-11T08:53:33.147178+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:53:33.147178+0000","last_undegraded":"2024-09-11T08:53:33.147178+0000","last_fullsized":"2024-09-11T08:53:33.147178+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:30.465003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:30.465003+0000","last_clean_scrub_stamp":"2024-09-11T08:51:30.465003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019306950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"200'7","reported_seq":316,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150043+0000","last_change":"2024-09-11T08:52:11.155994+0000","last_active":"2024-09-11T08:53:33.150043+0000","last_peered":"2024-09-11T08:53:33.150043+0000","last_clean":"2024-09-11T08:53:33.150043+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:53:33.150043+0000","last_undegraded":"2024-09-11T08:53:33.150043+0000","last_fullsized":"2024-09-11T08:53:33.150043+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'7","last_scrub_stamp":"2024-09-11T08:52:11.155898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:52:11.155898+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002524072,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":326,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148266+0000","last_change":"2024-09-11T08:52:52.458278+0000","last_active":"2024-09-11T08:53:33.148266+0000","last_peered":"2024-09-11T08:53:33.148266+0000","last_clean":"2024-09-11T08:53:33.148266+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:53:33.148266+0000","last_undegraded":"2024-09-11T08:53:33.148266+0000","last_fullsized":"2024-09-11T08:53:33.148266+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:52.458219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:18.199651+0000","last_clean_scrub_stamp":"2024-09-11T08:52:52.458219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:17.396094+0000","scrub_duration":0.0013148910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":311,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.144052+0000","last_change":"2024-09-11T08:52:50.730859+0000","last_active":"2024-09-11T08:53:33.144052+0000","last_peered":"2024-09-11T08:53:33.144052+0000","last_clean":"2024-09-11T08:53:33.144052+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:53:33.144052+0000","last_undegraded":"2024-09-11T08:53:33.144052+0000","last_fullsized":"2024-09-11T08:53:33.144052+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:50.730804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:52:50.730804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:14.400201+0000","scrub_duration":0.001426482,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":316,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150984+0000","last_change":"2024-09-11T08:50:13.762434+0000","last_active":"2024-09-11T08:53:33.150984+0000","last_peered":"2024-09-11T08:53:33.150984+0000","last_clean":"2024-09-11T08:53:33.150984+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:53:33.150984+0000","last_undegraded":"2024-09-11T08:53:33.150984+0000","last_fullsized":"2024-09-11T08:53:33.150984+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:13.762372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:13.762372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001701496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":117,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150186+0000","last_change":"2024-09-11T08:52:49.566922+0000","last_active":"2024-09-11T08:53:33.150186+0000","last_peered":"2024-09-11T08:53:33.150186+0000","last_clean":"2024-09-11T08:53:33.150186+0000","last_became_active":"2024-09-11T08:52:49.566575+0000","last_became_peered":"2024-09-11T08:52:49.566575+0000","last_unstale":"2024-09-11T08:53:33.150186+0000","last_undegraded":"2024-09-11T08:53:33.150186+0000","last_fullsized":"2024-09-11T08:53:33.150186+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:24.573732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:24.573732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"200'34","reported_seq":404,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151768+0000","last_change":"2024-09-11T08:52:56.872065+0000","last_active":"2024-09-11T08:53:33.151768+0000","last_peered":"2024-09-11T08:53:33.151768+0000","last_clean":"2024-09-11T08:53:33.151768+0000","last_became_active":"2024-09-11T08:38:40.975958+0000","last_became_peered":"2024-09-11T08:38:40.975958+0000","last_unstale":"2024-09-11T08:53:33.151768+0000","last_undegraded":"2024-09-11T08:53:33.151768+0000","last_fullsized":"2024-09-11T08:53:33.151768+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:52:56.872009+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:52:56.872009+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:02.594140+0000","scrub_duration":0.001869905,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":337,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148239+0000","last_change":"2024-09-11T08:52:18.579843+0000","last_active":"2024-09-11T08:53:33.148239+0000","last_peered":"2024-09-11T08:53:33.148239+0000","last_clean":"2024-09-11T08:53:33.148239+0000","last_became_active":"2024-09-11T08:52:18.579026+0000","last_became_peered":"2024-09-11T08:52:18.579026+0000","last_unstale":"2024-09-11T08:53:33.148239+0000","last_undegraded":"2024-09-11T08:53:33.148239+0000","last_fullsized":"2024-09-11T08:53:33.148239+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:29.793890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:29.793890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020976990000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":334,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.161913+0000","last_change":"2024-09-11T08:53:14.430444+0000","last_active":"2024-09-11T08:53:33.161913+0000","last_peered":"2024-09-11T08:53:33.161913+0000","last_clean":"2024-09-11T08:53:33.161913+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:53:33.161913+0000","last_undegraded":"2024-09-11T08:53:33.161913+0000","last_fullsized":"2024-09-11T08:53:33.161913+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:53:14.430366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:14.430366+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:16.232872+0000","scrub_duration":0.0024769649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":302,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151130+0000","last_change":"2024-09-11T08:50:17.743118+0000","last_active":"2024-09-11T08:53:33.151130+0000","last_peered":"2024-09-11T08:53:33.151130+0000","last_clean":"2024-09-11T08:53:33.151130+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:53:33.151130+0000","last_undegraded":"2024-09-11T08:53:33.151130+0000","last_fullsized":"2024-09-11T08:53:33.151130+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:17.743026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:17.743026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061030829999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"200'18","reported_seq":363,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.161900+0000","last_change":"2024-09-11T08:53:07.440507+0000","last_active":"2024-09-11T08:53:33.161900+0000","last_peered":"2024-09-11T08:53:33.161900+0000","last_clean":"2024-09-11T08:53:33.161900+0000","last_became_active":"2024-09-11T08:49:16.943886+0000","last_became_peered":"2024-09-11T08:49:16.943886+0000","last_unstale":"2024-09-11T08:53:33.161900+0000","last_undegraded":"2024-09-11T08:53:33.161900+0000","last_fullsized":"2024-09-11T08:53:33.161900+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'18","last_scrub_stamp":"2024-09-11T08:53:07.440404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:07.440404+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:28.528108+0000","scrub_duration":0.0041402180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":308,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151461+0000","last_change":"2024-09-11T08:53:12.948801+0000","last_active":"2024-09-11T08:53:33.151461+0000","last_peered":"2024-09-11T08:53:33.151461+0000","last_clean":"2024-09-11T08:53:33.151461+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:53:33.151461+0000","last_undegraded":"2024-09-11T08:53:33.151461+0000","last_fullsized":"2024-09-11T08:53:33.151461+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:12.948659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:12.948659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:19.726307+0000","scrub_duration":0.0039731519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":44,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.144180+0000","last_change":"2024-09-11T08:52:38.483970+0000","last_active":"2024-09-11T08:53:33.144180+0000","last_peered":"2024-09-11T08:53:33.144180+0000","last_clean":"2024-09-11T08:53:33.144180+0000","last_became_active":"2024-09-11T08:52:38.483603+0000","last_became_peered":"2024-09-11T08:52:38.483603+0000","last_unstale":"2024-09-11T08:53:33.144180+0000","last_undegraded":"2024-09-11T08:53:33.144180+0000","last_fullsized":"2024-09-11T08:53:33.144180+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:50:09.077871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:09.077871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":321,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.146113+0000","last_change":"2024-09-11T08:51:28.824144+0000","last_active":"2024-09-11T08:53:33.146113+0000","last_peered":"2024-09-11T08:53:33.146113+0000","last_clean":"2024-09-11T08:53:33.146113+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:53:33.146113+0000","last_undegraded":"2024-09-11T08:53:33.146113+0000","last_fullsized":"2024-09-11T08:53:33.146113+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:28.824093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:51:28.824093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017179579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"280'25","reported_seq":60,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.161962+0000","last_change":"2024-09-11T08:52:49.568462+0000","last_active":"2024-09-11T08:53:33.161962+0000","last_peered":"2024-09-11T08:53:33.161962+0000","last_clean":"2024-09-11T08:53:33.161962+0000","last_became_active":"2024-09-11T08:52:49.568074+0000","last_became_peered":"2024-09-11T08:52:49.568074+0000","last_unstale":"2024-09-11T08:53:33.161962+0000","last_undegraded":"2024-09-11T08:53:33.161962+0000","last_fullsized":"2024-09-11T08:53:33.161962+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"280'25","last_scrub_stamp":"2024-09-11T08:52:24.584533+0000","last_deep_scrub":"280'25","last_deep_scrub_stamp":"2024-09-11T08:52:24.584533+0000","last_clean_scrub_stamp":"2024-09-11T08:52:24.584533+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:51.336883+0000","scrub_duration":0.0050623580000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":517,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:37.844545+0000","last_change":"2024-09-11T08:52:12.368313+0000","last_active":"2024-09-11T08:53:37.844545+0000","last_peered":"2024-09-11T08:53:37.844545+0000","last_clean":"2024-09-11T08:53:37.844545+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:53:37.844545+0000","last_undegraded":"2024-09-11T08:53:37.844545+0000","last_fullsized":"2024-09-11T08:53:37.844545+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:52:12.368231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:12.368231+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073151179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"309'4673","reported_seq":4697,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.105502+0000","last_change":"2024-09-11T08:52:56.415129+0000","last_active":"2024-09-11T08:53:41.105502+0000","last_peered":"2024-09-11T08:53:41.105502+0000","last_clean":"2024-09-11T08:53:41.105502+0000","last_became_active":"2024-09-11T08:52:56.414844+0000","last_became_peered":"2024-09-11T08:52:56.414844+0000","last_unstale":"2024-09-11T08:53:41.105502+0000","last_undegraded":"2024-09-11T08:53:41.105502+0000","last_fullsized":"2024-09-11T08:53:41.105502+0000","mapping_epoch":293,"log_start":"299'300","ondisk_log_start":"299'300","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4373,"log_dups_size":0,"ondisk_log_size":4373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:18.963092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306249728,"num_objects":4673,"num_object_clones":0,"num_object_copies":14019,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4673,"num_write_kb":299072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"199'81","reported_seq":78,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.143830+0000","last_change":"2024-09-11T08:53:16.826016+0000","last_active":"2024-09-11T08:53:33.143830+0000","last_peered":"2024-09-11T08:53:33.143830+0000","last_clean":"2024-09-11T08:53:33.143830+0000","last_became_active":"2024-09-11T08:51:03.023615+0000","last_became_peered":"2024-09-11T08:51:03.023615+0000","last_unstale":"2024-09-11T08:53:33.143830+0000","last_undegraded":"2024-09-11T08:53:33.143830+0000","last_fullsized":"2024-09-11T08:53:33.143830+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:16.825859+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:16.825859+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:32.758862+0000","scrub_duration":0.0087576560000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"200'34","reported_seq":369,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148378+0000","last_change":"2024-09-11T08:53:03.680968+0000","last_active":"2024-09-11T08:53:33.148378+0000","last_peered":"2024-09-11T08:53:33.148378+0000","last_clean":"2024-09-11T08:53:33.148378+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:53:33.148378+0000","last_undegraded":"2024-09-11T08:53:33.148378+0000","last_fullsized":"2024-09-11T08:53:33.148378+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:53:03.680906+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:53:03.680906+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:26.793062+0000","scrub_duration":0.0038860740000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":311,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.160187+0000","last_change":"2024-09-11T08:53:02.487714+0000","last_active":"2024-09-11T08:53:33.160187+0000","last_peered":"2024-09-11T08:53:33.160187+0000","last_clean":"2024-09-11T08:53:33.160187+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:53:33.160187+0000","last_undegraded":"2024-09-11T08:53:33.160187+0000","last_fullsized":"2024-09-11T08:53:33.160187+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:02.487657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:02.487657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:07.383897+0000","scrub_duration":0.0011084630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"277'11","reported_seq":525,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:37.844360+0000","last_change":"2024-09-11T08:52:56.987065+0000","last_active":"2024-09-11T08:53:37.844360+0000","last_peered":"2024-09-11T08:53:37.844360+0000","last_clean":"2024-09-11T08:53:37.844360+0000","last_became_active":"2024-09-11T08:52:18.578328+0000","last_became_peered":"2024-09-11T08:52:18.578328+0000","last_unstale":"2024-09-11T08:53:37.844360+0000","last_undegraded":"2024-09-11T08:53:37.844360+0000","last_fullsized":"2024-09-11T08:53:37.844360+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'11","last_scrub_stamp":"2024-09-11T08:52:56.987010+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:52:56.987010+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:03.157069+0000","scrub_duration":0.0033882560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":328,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.147389+0000","last_change":"2024-09-11T08:52:57.679482+0000","last_active":"2024-09-11T08:53:33.147389+0000","last_peered":"2024-09-11T08:53:33.147389+0000","last_clean":"2024-09-11T08:53:33.147389+0000","last_became_active":"2024-09-11T08:52:18.576898+0000","last_became_peered":"2024-09-11T08:52:18.576898+0000","last_unstale":"2024-09-11T08:53:33.147389+0000","last_undegraded":"2024-09-11T08:53:33.147389+0000","last_fullsized":"2024-09-11T08:53:33.147389+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:57.679430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:57.679430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:59.791070+0000","scrub_duration":0.0013664560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"309'4498","reported_seq":4522,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.195557+0000","last_change":"2024-09-11T08:52:55.610689+0000","last_active":"2024-09-11T08:53:41.195557+0000","last_peered":"2024-09-11T08:53:41.195557+0000","last_clean":"2024-09-11T08:53:41.195557+0000","last_became_active":"2024-09-11T08:52:55.610383+0000","last_became_peered":"2024-09-11T08:52:55.610383+0000","last_unstale":"2024-09-11T08:53:41.195557+0000","last_undegraded":"2024-09-11T08:53:41.195557+0000","last_fullsized":"2024-09-11T08:53:41.195557+0000","mapping_epoch":293,"log_start":"299'100","ondisk_log_start":"299'100","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4398,"log_dups_size":0,"ondisk_log_size":4398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:17.453638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":294780928,"num_objects":4498,"num_object_clones":0,"num_object_copies":13494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4498,"num_write_kb":287872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"199'81","reported_seq":78,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.153213+0000","last_change":"2024-09-11T08:53:00.953735+0000","last_active":"2024-09-11T08:53:33.153213+0000","last_peered":"2024-09-11T08:53:33.153213+0000","last_clean":"2024-09-11T08:53:33.153213+0000","last_became_active":"2024-09-11T08:51:03.023365+0000","last_became_peered":"2024-09-11T08:51:03.023365+0000","last_unstale":"2024-09-11T08:53:33.153213+0000","last_undegraded":"2024-09-11T08:53:33.153213+0000","last_fullsized":"2024-09-11T08:53:33.153213+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:00.953685+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:00.953685+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:07.440648+0000","scrub_duration":0.001183323,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"200'38","reported_seq":423,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.160002+0000","last_change":"2024-09-11T08:52:18.611214+0000","last_active":"2024-09-11T08:53:33.160002+0000","last_peered":"2024-09-11T08:53:33.160002+0000","last_clean":"2024-09-11T08:53:33.160002+0000","last_became_active":"2024-09-11T08:52:18.595013+0000","last_became_peered":"2024-09-11T08:52:18.595013+0000","last_unstale":"2024-09-11T08:53:33.160002+0000","last_undegraded":"2024-09-11T08:53:33.160002+0000","last_fullsized":"2024-09-11T08:53:33.160002+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'38","last_scrub_stamp":"2024-09-11T08:52:14.460428+0000","last_deep_scrub":"199'26","last_deep_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_clean_scrub_stamp":"2024-09-11T08:52:14.460428+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012981754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":334,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.162350+0000","last_change":"2024-09-11T08:52:18.587201+0000","last_active":"2024-09-11T08:53:33.162350+0000","last_peered":"2024-09-11T08:53:33.162350+0000","last_clean":"2024-09-11T08:53:33.162350+0000","last_became_active":"2024-09-11T08:52:18.586084+0000","last_became_peered":"2024-09-11T08:52:18.586084+0000","last_unstale":"2024-09-11T08:53:33.162350+0000","last_undegraded":"2024-09-11T08:53:33.162350+0000","last_fullsized":"2024-09-11T08:53:33.162350+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:13.467288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:13.467288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046940990000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":308,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.149796+0000","last_change":"2024-09-11T08:52:59.933645+0000","last_active":"2024-09-11T08:53:33.149796+0000","last_peered":"2024-09-11T08:53:33.149796+0000","last_clean":"2024-09-11T08:53:33.149796+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:53:33.149796+0000","last_undegraded":"2024-09-11T08:53:33.149796+0000","last_fullsized":"2024-09-11T08:53:33.149796+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:59.933599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:29.687577+0000","last_clean_scrub_stamp":"2024-09-11T08:52:59.933599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:28.572636+0000","scrub_duration":0.001022233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":296,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150867+0000","last_change":"2024-09-11T08:50:19.706569+0000","last_active":"2024-09-11T08:53:33.150867+0000","last_peered":"2024-09-11T08:53:33.150867+0000","last_clean":"2024-09-11T08:53:33.150867+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:53:33.150867+0000","last_undegraded":"2024-09-11T08:53:33.150867+0000","last_fullsized":"2024-09-11T08:53:33.150867+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:19.706509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:19.706509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.045176716999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"309'4617","reported_seq":4640,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.296288+0000","last_change":"2024-09-11T08:52:55.612830+0000","last_active":"2024-09-11T08:53:41.296288+0000","last_peered":"2024-09-11T08:53:41.296288+0000","last_clean":"2024-09-11T08:53:41.296288+0000","last_became_active":"2024-09-11T08:52:55.612351+0000","last_became_peered":"2024-09-11T08:52:55.612351+0000","last_unstale":"2024-09-11T08:53:41.296288+0000","last_undegraded":"2024-09-11T08:53:41.296288+0000","last_fullsized":"2024-09-11T08:53:41.296288+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4617,"log_dups_size":0,"ondisk_log_size":4617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:17.328173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302579712,"num_objects":4617,"num_object_clones":0,"num_object_copies":13851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4617,"num_write_kb":295488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"199'81","reported_seq":83,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.161265+0000","last_change":"2024-09-11T08:53:08.418382+0000","last_active":"2024-09-11T08:53:33.161265+0000","last_peered":"2024-09-11T08:53:33.161265+0000","last_clean":"2024-09-11T08:53:33.161265+0000","last_became_active":"2024-09-11T08:51:03.009983+0000","last_became_peered":"2024-09-11T08:51:03.009983+0000","last_unstale":"2024-09-11T08:53:33.161265+0000","last_undegraded":"2024-09-11T08:53:33.161265+0000","last_fullsized":"2024-09-11T08:53:33.161265+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:08.418288+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:08.418288+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:23.484408+0000","scrub_duration":0.003188832,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"200'35","reported_seq":352,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150156+0000","last_change":"2024-09-11T08:52:25.186590+0000","last_active":"2024-09-11T08:53:33.150156+0000","last_peered":"2024-09-11T08:53:33.150156+0000","last_clean":"2024-09-11T08:53:33.150156+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:53:33.150156+0000","last_undegraded":"2024-09-11T08:53:33.150156+0000","last_fullsized":"2024-09-11T08:53:33.150156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'35","last_scrub_stamp":"2024-09-11T08:52:25.186528+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:52:25.186528+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:40.024410+0000","scrub_duration":0.0018614930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":51,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.149431+0000","last_change":"2024-09-11T08:52:28.487656+0000","last_active":"2024-09-11T08:53:33.149431+0000","last_peered":"2024-09-11T08:53:33.149431+0000","last_clean":"2024-09-11T08:53:33.149431+0000","last_became_active":"2024-09-11T08:52:18.586549+0000","last_became_peered":"2024-09-11T08:52:18.586549+0000","last_unstale":"2024-09-11T08:53:33.149431+0000","last_undegraded":"2024-09-11T08:53:33.149431+0000","last_fullsized":"2024-09-11T08:53:33.149431+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:28.487573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:28.487573+0000","last_clean_scrub_stamp":"2024-09-11T08:52:28.487573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:58.237961+0000","scrub_duration":0.0049500239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":488,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:37.846679+0000","last_change":"2024-09-11T08:51:29.395679+0000","last_active":"2024-09-11T08:53:37.846679+0000","last_peered":"2024-09-11T08:53:37.846679+0000","last_clean":"2024-09-11T08:53:37.846679+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:53:37.846679+0000","last_undegraded":"2024-09-11T08:53:37.846679+0000","last_fullsized":"2024-09-11T08:53:37.846679+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:51:29.395598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:29.395598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041488640000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":209,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.154125+0000","last_change":"2024-09-11T08:52:51.396282+0000","last_active":"2024-09-11T08:53:33.154125+0000","last_peered":"2024-09-11T08:53:33.154125+0000","last_clean":"2024-09-11T08:53:33.154125+0000","last_became_active":"2024-09-11T08:52:38.488567+0000","last_became_peered":"2024-09-11T08:52:38.488567+0000","last_unstale":"2024-09-11T08:53:33.154125+0000","last_undegraded":"2024-09-11T08:53:33.154125+0000","last_fullsized":"2024-09-11T08:53:33.154125+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:51.396225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:51.396225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:52.673765+0000","scrub_duration":0.001381873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"309'4617","reported_seq":4641,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.199695+0000","last_change":"2024-09-11T08:52:56.414997+0000","last_active":"2024-09-11T08:53:41.199695+0000","last_peered":"2024-09-11T08:53:41.199695+0000","last_clean":"2024-09-11T08:53:41.199695+0000","last_became_active":"2024-09-11T08:52:56.414726+0000","last_became_peered":"2024-09-11T08:52:56.414726+0000","last_unstale":"2024-09-11T08:53:41.199695+0000","last_undegraded":"2024-09-11T08:53:41.199695+0000","last_fullsized":"2024-09-11T08:53:41.199695+0000","mapping_epoch":293,"log_start":"299'200","ondisk_log_start":"299'200","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4417,"log_dups_size":0,"ondisk_log_size":4417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:17.557033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302579712,"num_objects":4617,"num_object_clones":0,"num_object_copies":13851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4617,"num_write_kb":295488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"199'81","reported_seq":329,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.154772+0000","last_change":"2024-09-11T08:53:13.350933+0000","last_active":"2024-09-11T08:53:33.154772+0000","last_peered":"2024-09-11T08:53:33.154772+0000","last_clean":"2024-09-11T08:53:33.154772+0000","last_became_active":"2024-09-11T08:51:03.003800+0000","last_became_peered":"2024-09-11T08:51:03.003800+0000","last_unstale":"2024-09-11T08:53:33.154772+0000","last_undegraded":"2024-09-11T08:53:33.154772+0000","last_fullsized":"2024-09-11T08:53:33.154772+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:13.350843+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:13.350843+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:33.425416+0000","scrub_duration":0.0032054900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"200'25","reported_seq":354,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.147901+0000","last_change":"2024-09-11T08:51:31.799902+0000","last_active":"2024-09-11T08:53:33.147901+0000","last_peered":"2024-09-11T08:53:33.147901+0000","last_clean":"2024-09-11T08:53:33.147901+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:53:33.147901+0000","last_undegraded":"2024-09-11T08:53:33.147901+0000","last_fullsized":"2024-09-11T08:53:33.147901+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:51:31.799844+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_clean_scrub_stamp":"2024-09-11T08:51:31.799844+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075128269999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":317,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.147788+0000","last_change":"2024-09-11T08:52:55.701978+0000","last_active":"2024-09-11T08:53:33.147788+0000","last_peered":"2024-09-11T08:53:33.147788+0000","last_clean":"2024-09-11T08:53:33.147788+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:53:33.147788+0000","last_undegraded":"2024-09-11T08:53:33.147788+0000","last_fullsized":"2024-09-11T08:53:33.147788+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:55.701902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:55.701902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:21.824877+0000","scrub_duration":0.0016250119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"292'11","reported_seq":196,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:37.844816+0000","last_change":"2024-09-11T08:53:01.380925+0000","last_active":"2024-09-11T08:53:37.844816+0000","last_peered":"2024-09-11T08:53:37.844816+0000","last_clean":"2024-09-11T08:53:37.844816+0000","last_became_active":"2024-09-11T08:52:49.564770+0000","last_became_peered":"2024-09-11T08:52:49.564770+0000","last_unstale":"2024-09-11T08:53:37.844816+0000","last_undegraded":"2024-09-11T08:53:37.844816+0000","last_fullsized":"2024-09-11T08:53:37.844816+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"292'11","last_scrub_stamp":"2024-09-11T08:53:01.380869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:01.380869+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:27.934762+0000","scrub_duration":0.0014786319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":117,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.159768+0000","last_change":"2024-09-11T08:52:49.572126+0000","last_active":"2024-09-11T08:53:33.159768+0000","last_peered":"2024-09-11T08:53:33.159768+0000","last_clean":"2024-09-11T08:53:33.159768+0000","last_became_active":"2024-09-11T08:52:49.571683+0000","last_became_peered":"2024-09-11T08:52:49.571683+0000","last_unstale":"2024-09-11T08:53:33.159768+0000","last_undegraded":"2024-09-11T08:53:33.159768+0000","last_fullsized":"2024-09-11T08:53:33.159768+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:20.428724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:51:20.428724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.d","version":"309'4558","reported_seq":4581,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.276903+0000","last_change":"2024-09-11T08:52:56.415069+0000","last_active":"2024-09-11T08:53:41.276903+0000","last_peered":"2024-09-11T08:53:41.276903+0000","last_clean":"2024-09-11T08:53:41.276903+0000","last_became_active":"2024-09-11T08:52:56.414830+0000","last_became_peered":"2024-09-11T08:52:56.414830+0000","last_unstale":"2024-09-11T08:53:41.276903+0000","last_undegraded":"2024-09-11T08:53:41.276903+0000","last_fullsized":"2024-09-11T08:53:41.276903+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4558,"log_dups_size":0,"ondisk_log_size":4558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:09.860125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":298713088,"num_objects":4558,"num_object_clones":0,"num_object_copies":13674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4558,"num_write_kb":291712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"199'81","reported_seq":339,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.154470+0000","last_change":"2024-09-11T08:53:21.460030+0000","last_active":"2024-09-11T08:53:33.154470+0000","last_peered":"2024-09-11T08:53:33.154470+0000","last_clean":"2024-09-11T08:53:33.154470+0000","last_became_active":"2024-09-11T08:52:18.584896+0000","last_became_peered":"2024-09-11T08:52:18.584896+0000","last_unstale":"2024-09-11T08:53:33.154470+0000","last_undegraded":"2024-09-11T08:53:33.154470+0000","last_fullsized":"2024-09-11T08:53:33.154470+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:21.459982+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:21.459982+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:39.076812+0000","scrub_duration":0.0023340290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"200'29","reported_seq":365,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.145688+0000","last_change":"2024-09-11T08:52:49.564105+0000","last_active":"2024-09-11T08:53:33.145688+0000","last_peered":"2024-09-11T08:53:33.145688+0000","last_clean":"2024-09-11T08:53:33.145688+0000","last_became_active":"2024-09-11T08:52:49.563364+0000","last_became_peered":"2024-09-11T08:52:49.563364+0000","last_unstale":"2024-09-11T08:53:33.145688+0000","last_undegraded":"2024-09-11T08:53:33.145688+0000","last_fullsized":"2024-09-11T08:53:33.145688+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'29","last_scrub_stamp":"2024-09-11T08:50:06.372228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:06.372228+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050905730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":331,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148264+0000","last_change":"2024-09-11T08:53:13.677539+0000","last_active":"2024-09-11T08:53:33.148264+0000","last_peered":"2024-09-11T08:53:33.148264+0000","last_clean":"2024-09-11T08:53:33.148264+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:53:33.148264+0000","last_undegraded":"2024-09-11T08:53:33.148264+0000","last_fullsized":"2024-09-11T08:53:33.148264+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:13.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:13.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:17.980682+0000","scrub_duration":0.0029413159999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":356,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.153625+0000","last_change":"2024-09-11T08:53:09.384145+0000","last_active":"2024-09-11T08:53:33.153625+0000","last_peered":"2024-09-11T08:53:33.153625+0000","last_clean":"2024-09-11T08:53:33.153625+0000","last_became_active":"2024-09-11T08:52:18.573235+0000","last_became_peered":"2024-09-11T08:52:18.573235+0000","last_unstale":"2024-09-11T08:53:33.153625+0000","last_undegraded":"2024-09-11T08:53:33.153625+0000","last_fullsized":"2024-09-11T08:53:33.153625+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:09.384067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_clean_scrub_stamp":"2024-09-11T08:53:09.384067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:28.457187+0000","scrub_duration":0.0035749929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":345,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.145927+0000","last_change":"2024-09-11T08:53:18.654523+0000","last_active":"2024-09-11T08:53:33.145927+0000","last_peered":"2024-09-11T08:53:33.145927+0000","last_clean":"2024-09-11T08:53:33.145927+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:53:33.145927+0000","last_undegraded":"2024-09-11T08:53:33.145927+0000","last_fullsized":"2024-09-11T08:53:33.145927+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:18.654437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:53:18.654437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:21.341535+0000","scrub_duration":0.016784286999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"309'4654","reported_seq":4677,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.281889+0000","last_change":"2024-09-11T08:52:56.415080+0000","last_active":"2024-09-11T08:53:41.281889+0000","last_peered":"2024-09-11T08:53:41.281889+0000","last_clean":"2024-09-11T08:53:41.281889+0000","last_became_active":"2024-09-11T08:52:56.414809+0000","last_became_peered":"2024-09-11T08:52:56.414809+0000","last_unstale":"2024-09-11T08:53:41.281889+0000","last_undegraded":"2024-09-11T08:53:41.281889+0000","last_fullsized":"2024-09-11T08:53:41.281889+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4654,"log_dups_size":0,"ondisk_log_size":4654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:06.098380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":305004544,"num_objects":4654,"num_object_clones":0,"num_object_copies":13962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4654,"num_write_kb":297856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"199'81","reported_seq":78,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.143900+0000","last_change":"2024-09-11T08:52:18.575075+0000","last_active":"2024-09-11T08:53:33.143900+0000","last_peered":"2024-09-11T08:53:33.143900+0000","last_clean":"2024-09-11T08:53:33.143900+0000","last_became_active":"2024-09-11T08:52:18.574468+0000","last_became_peered":"2024-09-11T08:52:18.574468+0000","last_unstale":"2024-09-11T08:53:33.143900+0000","last_undegraded":"2024-09-11T08:53:33.143900+0000","last_fullsized":"2024-09-11T08:53:33.143900+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"200'50","reported_seq":585,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:37.844875+0000","last_change":"2024-09-11T08:53:09.745429+0000","last_active":"2024-09-11T08:53:37.844875+0000","last_peered":"2024-09-11T08:53:37.844875+0000","last_clean":"2024-09-11T08:53:37.844875+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:53:37.844875+0000","last_undegraded":"2024-09-11T08:53:37.844875+0000","last_fullsized":"2024-09-11T08:53:37.844875+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'50","last_scrub_stamp":"2024-09-11T08:53:09.745322+0000","last_deep_scrub":"200'50","last_deep_scrub_stamp":"2024-09-11T08:53:09.745322+0000","last_clean_scrub_stamp":"2024-09-11T08:53:09.745322+0000","objects_scrubbed":10,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:24.698208+0000","scrub_duration":0.0095378719999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":332,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150369+0000","last_change":"2024-09-11T08:52:18.579773+0000","last_active":"2024-09-11T08:53:33.150369+0000","last_peered":"2024-09-11T08:53:33.150369+0000","last_clean":"2024-09-11T08:53:33.150369+0000","last_became_active":"2024-09-11T08:52:18.579509+0000","last_became_peered":"2024-09-11T08:52:18.579509+0000","last_unstale":"2024-09-11T08:53:33.150369+0000","last_undegraded":"2024-09-11T08:53:33.150369+0000","last_fullsized":"2024-09-11T08:53:33.150369+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:33.010134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_clean_scrub_stamp":"2024-09-11T08:51:33.010134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033959530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":352,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.146907+0000","last_change":"2024-09-11T08:51:19.322458+0000","last_active":"2024-09-11T08:53:33.146907+0000","last_peered":"2024-09-11T08:53:33.146907+0000","last_clean":"2024-09-11T08:53:33.146907+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:53:33.146907+0000","last_undegraded":"2024-09-11T08:53:33.146907+0000","last_fullsized":"2024-09-11T08:53:33.146907+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:51:19.321966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:51:19.321966+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077550900000000001,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":127,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148117+0000","last_change":"2024-09-11T08:53:20.721615+0000","last_active":"2024-09-11T08:53:33.148117+0000","last_peered":"2024-09-11T08:53:33.148117+0000","last_clean":"2024-09-11T08:53:33.148117+0000","last_became_active":"2024-09-11T08:52:49.569137+0000","last_became_peered":"2024-09-11T08:52:49.569137+0000","last_unstale":"2024-09-11T08:53:33.148117+0000","last_undegraded":"2024-09-11T08:53:33.148117+0000","last_fullsized":"2024-09-11T08:53:33.148117+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:20.721530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:20.721530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:27.624638+0000","scrub_duration":0.0026422770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":305,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.161885+0000","last_change":"2024-09-11T08:53:11.420230+0000","last_active":"2024-09-11T08:53:33.161885+0000","last_peered":"2024-09-11T08:53:33.161885+0000","last_clean":"2024-09-11T08:53:33.161885+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:53:33.161885+0000","last_undegraded":"2024-09-11T08:53:33.161885+0000","last_fullsized":"2024-09-11T08:53:33.161885+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:11.420182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:11.420182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:27.596432+0000","scrub_duration":0.0034052280000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"309'4673","reported_seq":4696,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.527951+0000","last_change":"2024-09-11T08:52:56.413844+0000","last_active":"2024-09-11T08:53:41.527951+0000","last_peered":"2024-09-11T08:53:41.527951+0000","last_clean":"2024-09-11T08:53:41.527951+0000","last_became_active":"2024-09-11T08:52:56.413594+0000","last_became_peered":"2024-09-11T08:52:56.413594+0000","last_unstale":"2024-09-11T08:53:41.527951+0000","last_undegraded":"2024-09-11T08:53:41.527951+0000","last_fullsized":"2024-09-11T08:53:41.527951+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4673,"log_dups_size":0,"ondisk_log_size":4673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:17.749632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306249728,"num_objects":4673,"num_object_clones":0,"num_object_copies":14019,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4673,"num_write_kb":299072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"199'81","reported_seq":357,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150474+0000","last_change":"2024-09-11T08:52:49.570289+0000","last_active":"2024-09-11T08:53:33.150474+0000","last_peered":"2024-09-11T08:53:33.150474+0000","last_clean":"2024-09-11T08:53:33.150474+0000","last_became_active":"2024-09-11T08:52:49.569934+0000","last_became_peered":"2024-09-11T08:52:49.569934+0000","last_unstale":"2024-09-11T08:53:33.150474+0000","last_undegraded":"2024-09-11T08:53:33.150474+0000","last_fullsized":"2024-09-11T08:53:33.150474+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"200'17","reported_seq":396,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148497+0000","last_change":"2024-09-11T08:52:18.598436+0000","last_active":"2024-09-11T08:53:33.148497+0000","last_peered":"2024-09-11T08:53:33.148497+0000","last_clean":"2024-09-11T08:53:33.148497+0000","last_became_active":"2024-09-11T08:52:18.580008+0000","last_became_peered":"2024-09-11T08:52:18.580008+0000","last_unstale":"2024-09-11T08:53:33.148497+0000","last_undegraded":"2024-09-11T08:53:33.148497+0000","last_fullsized":"2024-09-11T08:53:33.148497+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'17","last_scrub_stamp":"2024-09-11T08:50:15.565290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:15.565290+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060447950000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":347,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.145846+0000","last_change":"2024-09-11T08:52:52.843079+0000","last_active":"2024-09-11T08:53:33.145846+0000","last_peered":"2024-09-11T08:53:33.145846+0000","last_clean":"2024-09-11T08:53:33.145846+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:53:33.145846+0000","last_undegraded":"2024-09-11T08:53:33.145846+0000","last_fullsized":"2024-09-11T08:53:33.145846+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:52.843019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:52.843019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:12.212376+0000","scrub_duration":0.001298305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"263'2","reported_seq":392,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151767+0000","last_change":"2024-09-11T08:53:08.773090+0000","last_active":"2024-09-11T08:53:33.151767+0000","last_peered":"2024-09-11T08:53:33.151767+0000","last_clean":"2024-09-11T08:53:33.151767+0000","last_became_active":"2024-09-11T08:52:18.586545+0000","last_became_peered":"2024-09-11T08:52:18.586545+0000","last_unstale":"2024-09-11T08:53:33.151767+0000","last_undegraded":"2024-09-11T08:53:33.151767+0000","last_fullsized":"2024-09-11T08:53:33.151767+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'2","last_scrub_stamp":"2024-09-11T08:53:08.772979+0000","last_deep_scrub":"263'2","last_deep_scrub_stamp":"2024-09-11T08:53:08.772979+0000","last_clean_scrub_stamp":"2024-09-11T08:53:08.772979+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:16.601208+0000","scrub_duration":0.0027552229999999998,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":342,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151727+0000","last_change":"2024-09-11T08:52:16.023858+0000","last_active":"2024-09-11T08:53:33.151727+0000","last_peered":"2024-09-11T08:53:33.151727+0000","last_clean":"2024-09-11T08:53:33.151727+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:53:33.151727+0000","last_undegraded":"2024-09-11T08:53:33.151727+0000","last_fullsized":"2024-09-11T08:53:33.151727+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:16.023763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_clean_scrub_stamp":"2024-09-11T08:52:16.023763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:40.228929+0000","scrub_duration":0.0032804420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":199,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.157729+0000","last_change":"2024-09-11T08:52:38.486638+0000","last_active":"2024-09-11T08:53:33.157729+0000","last_peered":"2024-09-11T08:53:33.157729+0000","last_clean":"2024-09-11T08:53:33.157729+0000","last_became_active":"2024-09-11T08:52:38.484800+0000","last_became_peered":"2024-09-11T08:52:38.484800+0000","last_unstale":"2024-09-11T08:53:33.157729+0000","last_undegraded":"2024-09-11T08:53:33.157729+0000","last_fullsized":"2024-09-11T08:53:33.157729+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:31.465384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:51:31.465384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015184790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"309'4638","reported_seq":4661,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.494471+0000","last_change":"2024-09-11T08:52:56.414016+0000","last_active":"2024-09-11T08:53:41.494471+0000","last_peered":"2024-09-11T08:53:41.494471+0000","last_clean":"2024-09-11T08:53:41.494471+0000","last_became_active":"2024-09-11T08:52:56.413725+0000","last_became_peered":"2024-09-11T08:52:56.413725+0000","last_unstale":"2024-09-11T08:53:41.494471+0000","last_undegraded":"2024-09-11T08:53:41.494471+0000","last_fullsized":"2024-09-11T08:53:41.494471+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4638,"log_dups_size":0,"ondisk_log_size":4638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:04.942971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":303955968,"num_objects":4638,"num_object_clones":0,"num_object_copies":13914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4638,"num_write_kb":296832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"199'81","reported_seq":341,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.149087+0000","last_change":"2024-09-11T08:53:01.761090+0000","last_active":"2024-09-11T08:53:33.149087+0000","last_peered":"2024-09-11T08:53:33.149087+0000","last_clean":"2024-09-11T08:53:33.149087+0000","last_became_active":"2024-09-11T08:51:03.027281+0000","last_became_peered":"2024-09-11T08:51:03.027281+0000","last_unstale":"2024-09-11T08:53:33.149087+0000","last_undegraded":"2024-09-11T08:53:33.149087+0000","last_fullsized":"2024-09-11T08:53:33.149087+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:01.760951+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:01.760951+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:19.773549+0000","scrub_duration":0.0017313369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"200'24","reported_seq":65,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.149275+0000","last_change":"2024-09-11T08:52:52.548054+0000","last_active":"2024-09-11T08:53:33.149275+0000","last_peered":"2024-09-11T08:53:33.149275+0000","last_clean":"2024-09-11T08:53:33.149275+0000","last_became_active":"2024-09-11T08:52:18.580950+0000","last_became_peered":"2024-09-11T08:52:18.580950+0000","last_unstale":"2024-09-11T08:53:33.149275+0000","last_undegraded":"2024-09-11T08:53:33.149275+0000","last_fullsized":"2024-09-11T08:53:33.149275+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'24","last_scrub_stamp":"2024-09-11T08:52:52.547998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:52:52.547998+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:14.066702+0000","scrub_duration":0.001854198,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":127,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.149893+0000","last_change":"2024-09-11T08:53:18.845981+0000","last_active":"2024-09-11T08:53:33.149893+0000","last_peered":"2024-09-11T08:53:33.149893+0000","last_clean":"2024-09-11T08:53:33.149893+0000","last_became_active":"2024-09-11T08:52:49.566437+0000","last_became_peered":"2024-09-11T08:52:49.566437+0000","last_unstale":"2024-09-11T08:53:33.149893+0000","last_undegraded":"2024-09-11T08:53:33.149893+0000","last_fullsized":"2024-09-11T08:53:33.149893+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:18.845896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:18.845896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:35.649215+0000","scrub_duration":0.0053148020000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":136,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:40.670086+0000","last_change":"2024-09-11T08:52:59.810864+0000","last_active":"2024-09-11T08:53:40.670086+0000","last_peered":"2024-09-11T08:53:40.670086+0000","last_clean":"2024-09-11T08:53:40.670086+0000","last_became_active":"2024-09-11T08:52:38.484664+0000","last_became_peered":"2024-09-11T08:52:38.484664+0000","last_unstale":"2024-09-11T08:53:40.670086+0000","last_undegraded":"2024-09-11T08:53:40.670086+0000","last_fullsized":"2024-09-11T08:53:40.670086+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-11T08:52:59.810791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:59.810791+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:05.917891+0000","scrub_duration":0.0017088629999999999,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":333,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148427+0000","last_change":"2024-09-11T08:52:18.578912+0000","last_active":"2024-09-11T08:53:33.148427+0000","last_peered":"2024-09-11T08:53:33.148427+0000","last_clean":"2024-09-11T08:53:33.148427+0000","last_became_active":"2024-09-11T08:52:18.578530+0000","last_became_peered":"2024-09-11T08:52:18.578530+0000","last_unstale":"2024-09-11T08:53:33.148427+0000","last_undegraded":"2024-09-11T08:53:33.148427+0000","last_fullsized":"2024-09-11T08:53:33.148427+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:32.443727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:32.443727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056199379999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":315,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.147951+0000","last_change":"2024-09-11T08:51:35.779870+0000","last_active":"2024-09-11T08:53:33.147951+0000","last_peered":"2024-09-11T08:53:33.147951+0000","last_clean":"2024-09-11T08:53:33.147951+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:53:33.147951+0000","last_undegraded":"2024-09-11T08:53:33.147951+0000","last_fullsized":"2024-09-11T08:53:33.147951+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:35.779796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:51:35.779796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048768270000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"309'4623","reported_seq":4646,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.269953+0000","last_change":"2024-09-11T08:52:56.413621+0000","last_active":"2024-09-11T08:53:41.269953+0000","last_peered":"2024-09-11T08:53:41.269953+0000","last_clean":"2024-09-11T08:53:41.269953+0000","last_became_active":"2024-09-11T08:52:56.413344+0000","last_became_peered":"2024-09-11T08:52:56.413344+0000","last_unstale":"2024-09-11T08:53:41.269953+0000","last_undegraded":"2024-09-11T08:53:41.269953+0000","last_fullsized":"2024-09-11T08:53:41.269953+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4623,"log_dups_size":0,"ondisk_log_size":4623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:10.229971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302972928,"num_objects":4623,"num_object_clones":0,"num_object_copies":13869,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4623,"num_write_kb":295872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"201'27","reported_seq":225,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.157893+0000","last_change":"2024-09-11T08:52:18.228592+0000","last_active":"2024-09-11T08:53:33.157893+0000","last_peered":"2024-09-11T08:53:33.157893+0000","last_clean":"2024-09-11T08:53:33.157893+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:53:33.157893+0000","last_undegraded":"2024-09-11T08:53:33.157893+0000","last_fullsized":"2024-09-11T08:53:33.157893+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'27","last_scrub_stamp":"2024-09-11T08:52:18.228537+0000","last_deep_scrub":"201'27","last_deep_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_clean_scrub_stamp":"2024-09-11T08:52:18.228537+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018286559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":310,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.162216+0000","last_change":"2024-09-11T08:50:08.276283+0000","last_active":"2024-09-11T08:53:33.162216+0000","last_peered":"2024-09-11T08:53:33.162216+0000","last_clean":"2024-09-11T08:53:33.162216+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:53:33.162216+0000","last_undegraded":"2024-09-11T08:53:33.162216+0000","last_fullsized":"2024-09-11T08:53:33.162216+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:50:08.276232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:08.276232+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018088246999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"199'81","reported_seq":279,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.153758+0000","last_change":"2024-09-11T08:53:15.431689+0000","last_active":"2024-09-11T08:53:33.153758+0000","last_peered":"2024-09-11T08:53:33.153758+0000","last_clean":"2024-09-11T08:53:33.153758+0000","last_became_active":"2024-09-11T08:52:49.565126+0000","last_became_peered":"2024-09-11T08:52:49.565126+0000","last_unstale":"2024-09-11T08:53:33.153758+0000","last_undegraded":"2024-09-11T08:53:33.153758+0000","last_fullsized":"2024-09-11T08:53:33.153758+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:15.431514+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:15.431514+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:33.145059+0000","scrub_duration":0.0073442550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":44,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151131+0000","last_change":"2024-09-11T08:52:53.870726+0000","last_active":"2024-09-11T08:53:33.151131+0000","last_peered":"2024-09-11T08:53:33.151131+0000","last_clean":"2024-09-11T08:53:33.151131+0000","last_became_active":"2024-09-11T08:52:38.485446+0000","last_became_peered":"2024-09-11T08:52:38.485446+0000","last_unstale":"2024-09-11T08:53:33.151131+0000","last_undegraded":"2024-09-11T08:53:33.151131+0000","last_fullsized":"2024-09-11T08:53:33.151131+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:53.870644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:53.870644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:19.224184+0000","scrub_duration":0.001623457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":311,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148318+0000","last_change":"2024-09-11T08:50:13.347446+0000","last_active":"2024-09-11T08:53:33.148318+0000","last_peered":"2024-09-11T08:53:33.148318+0000","last_clean":"2024-09-11T08:53:33.148318+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:53:33.148318+0000","last_undegraded":"2024-09-11T08:53:33.148318+0000","last_fullsized":"2024-09-11T08:53:33.148318+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:13.347374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:50:13.347374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062539859999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"309'4734","reported_seq":4756,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.290213+0000","last_change":"2024-09-11T08:52:55.612239+0000","last_active":"2024-09-11T08:53:41.290213+0000","last_peered":"2024-09-11T08:53:41.290213+0000","last_clean":"2024-09-11T08:53:41.290213+0000","last_became_active":"2024-09-11T08:52:55.611835+0000","last_became_peered":"2024-09-11T08:52:55.611835+0000","last_unstale":"2024-09-11T08:53:41.290213+0000","last_undegraded":"2024-09-11T08:53:41.290213+0000","last_fullsized":"2024-09-11T08:53:41.290213+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4734,"log_dups_size":0,"ondisk_log_size":4734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:18.573270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310247424,"num_objects":4734,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4734,"num_write_kb":302976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"200'19","reported_seq":400,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151131+0000","last_change":"2024-09-11T08:51:21.979895+0000","last_active":"2024-09-11T08:53:33.151131+0000","last_peered":"2024-09-11T08:53:33.151131+0000","last_clean":"2024-09-11T08:53:33.151131+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:53:33.151131+0000","last_undegraded":"2024-09-11T08:53:33.151131+0000","last_fullsized":"2024-09-11T08:53:33.151131+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:21.979796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:21.979796+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045321210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":302,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150284+0000","last_change":"2024-09-11T08:51:38.083246+0000","last_active":"2024-09-11T08:53:33.150284+0000","last_peered":"2024-09-11T08:53:33.150284+0000","last_clean":"2024-09-11T08:53:33.150284+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:53:33.150284+0000","last_undegraded":"2024-09-11T08:53:33.150284+0000","last_fullsized":"2024-09-11T08:53:33.150284+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:38.083168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:51:38.083168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029716930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":320,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148199+0000","last_change":"2024-09-11T08:52:56.675410+0000","last_active":"2024-09-11T08:53:33.148199+0000","last_peered":"2024-09-11T08:53:33.148199+0000","last_clean":"2024-09-11T08:53:33.148199+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:53:33.148199+0000","last_undegraded":"2024-09-11T08:53:33.148199+0000","last_fullsized":"2024-09-11T08:53:33.148199+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:56.675326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:52:56.675326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:04.423103+0000","scrub_duration":0.0033910770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":325,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.162623+0000","last_change":"2024-09-11T08:53:15.469630+0000","last_active":"2024-09-11T08:53:33.162623+0000","last_peered":"2024-09-11T08:53:33.162623+0000","last_clean":"2024-09-11T08:53:33.162623+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:53:33.162623+0000","last_undegraded":"2024-09-11T08:53:33.162623+0000","last_fullsized":"2024-09-11T08:53:33.162623+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:15.469262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:15.469262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:34.242644+0000","scrub_duration":0.0071803099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"309'4610","reported_seq":4632,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.488884+0000","last_change":"2024-09-11T08:52:55.611140+0000","last_active":"2024-09-11T08:53:41.488884+0000","last_peered":"2024-09-11T08:53:41.488884+0000","last_clean":"2024-09-11T08:53:41.488884+0000","last_became_active":"2024-09-11T08:52:55.610852+0000","last_became_peered":"2024-09-11T08:52:55.610852+0000","last_unstale":"2024-09-11T08:53:41.488884+0000","last_undegraded":"2024-09-11T08:53:41.488884+0000","last_fullsized":"2024-09-11T08:53:41.488884+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4610,"log_dups_size":0,"ondisk_log_size":4610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:11.182978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302120960,"num_objects":4610,"num_object_clones":0,"num_object_copies":13830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4610,"num_write_kb":295040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"199'81","reported_seq":85,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.143822+0000","last_change":"2024-09-11T08:53:14.810007+0000","last_active":"2024-09-11T08:53:33.143822+0000","last_peered":"2024-09-11T08:53:33.143822+0000","last_clean":"2024-09-11T08:53:33.143822+0000","last_became_active":"2024-09-11T08:51:03.020941+0000","last_became_peered":"2024-09-11T08:51:03.020941+0000","last_unstale":"2024-09-11T08:53:33.143822+0000","last_undegraded":"2024-09-11T08:53:33.143822+0000","last_fullsized":"2024-09-11T08:53:33.143822+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:14.809894+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:53:14.809894+0000","last_clean_scrub_stamp":"2024-09-11T08:53:14.809894+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:16.563088+0000","scrub_duration":0.0065883419999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"200'31","reported_seq":359,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.152944+0000","last_change":"2024-09-11T08:51:24.989802+0000","last_active":"2024-09-11T08:53:33.152944+0000","last_peered":"2024-09-11T08:53:33.152944+0000","last_clean":"2024-09-11T08:53:33.152944+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:53:33.152944+0000","last_undegraded":"2024-09-11T08:53:33.152944+0000","last_fullsized":"2024-09-11T08:53:33.152944+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'31","last_scrub_stamp":"2024-09-11T08:51:24.989715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:24.989715+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037630540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":350,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.153878+0000","last_change":"2024-09-11T08:53:10.339152+0000","last_active":"2024-09-11T08:53:33.153878+0000","last_peered":"2024-09-11T08:53:33.153878+0000","last_clean":"2024-09-11T08:53:33.153878+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:53:33.153878+0000","last_undegraded":"2024-09-11T08:53:33.153878+0000","last_fullsized":"2024-09-11T08:53:33.153878+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:10.339096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_clean_scrub_stamp":"2024-09-11T08:53:10.339096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:31.094115+0000","scrub_duration":0.0027644290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":337,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148028+0000","last_change":"2024-09-11T08:52:18.577488+0000","last_active":"2024-09-11T08:53:33.148028+0000","last_peered":"2024-09-11T08:53:33.148028+0000","last_clean":"2024-09-11T08:53:33.148028+0000","last_became_active":"2024-09-11T08:52:18.577201+0000","last_became_peered":"2024-09-11T08:52:18.577201+0000","last_unstale":"2024-09-11T08:53:33.148028+0000","last_undegraded":"2024-09-11T08:53:33.148028+0000","last_fullsized":"2024-09-11T08:53:33.148028+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:22.782448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:51:22.782448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071094909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":353,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151718+0000","last_change":"2024-09-11T08:53:07.808224+0000","last_active":"2024-09-11T08:53:33.151718+0000","last_peered":"2024-09-11T08:53:33.151718+0000","last_clean":"2024-09-11T08:53:33.151718+0000","last_became_active":"2024-09-11T08:38:44.993862+0000","last_became_peered":"2024-09-11T08:38:44.993862+0000","last_unstale":"2024-09-11T08:53:33.151718+0000","last_undegraded":"2024-09-11T08:53:33.151718+0000","last_fullsized":"2024-09-11T08:53:33.151718+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:07.808138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:07.808138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:18.430717+0000","scrub_duration":0.003891982,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"309'4699","reported_seq":4723,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.792005+0000","last_change":"2024-09-11T08:52:56.415049+0000","last_active":"2024-09-11T08:53:41.792005+0000","last_peered":"2024-09-11T08:53:41.792005+0000","last_clean":"2024-09-11T08:53:41.792005+0000","last_became_active":"2024-09-11T08:52:56.414765+0000","last_became_peered":"2024-09-11T08:52:56.414765+0000","last_unstale":"2024-09-11T08:53:41.792005+0000","last_undegraded":"2024-09-11T08:53:41.792005+0000","last_fullsized":"2024-09-11T08:53:41.792005+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4699,"log_dups_size":0,"ondisk_log_size":4699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:10.761677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":307953664,"num_objects":4699,"num_object_clones":0,"num_object_copies":14097,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4699,"num_write_kb":300736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"199'81","reported_seq":361,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.160583+0000","last_change":"2024-09-11T08:52:38.482411+0000","last_active":"2024-09-11T08:53:33.160583+0000","last_peered":"2024-09-11T08:53:33.160583+0000","last_clean":"2024-09-11T08:53:33.160583+0000","last_became_active":"2024-09-11T08:52:38.481475+0000","last_became_peered":"2024-09-11T08:52:38.481475+0000","last_unstale":"2024-09-11T08:53:33.160583+0000","last_undegraded":"2024-09-11T08:53:33.160583+0000","last_fullsized":"2024-09-11T08:53:33.160583+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"200'20","reported_seq":380,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151530+0000","last_change":"2024-09-11T08:52:18.090235+0000","last_active":"2024-09-11T08:53:33.151530+0000","last_peered":"2024-09-11T08:53:33.151530+0000","last_clean":"2024-09-11T08:53:33.151530+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:53:33.151530+0000","last_undegraded":"2024-09-11T08:53:33.151530+0000","last_fullsized":"2024-09-11T08:53:33.151530+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'20","last_scrub_stamp":"2024-09-11T08:52:18.090140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:52:18.090140+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018601010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":331,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.162310+0000","last_change":"2024-09-11T08:53:19.389486+0000","last_active":"2024-09-11T08:53:33.162310+0000","last_peered":"2024-09-11T08:53:33.162310+0000","last_clean":"2024-09-11T08:53:33.162310+0000","last_became_active":"2024-09-11T08:52:18.586343+0000","last_became_peered":"2024-09-11T08:52:18.586343+0000","last_unstale":"2024-09-11T08:53:33.162310+0000","last_undegraded":"2024-09-11T08:53:33.162310+0000","last_fullsized":"2024-09-11T08:53:33.162310+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:19.389411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:19.389411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:25.181487+0000","scrub_duration":0.0058663470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"299'9","reported_seq":538,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:37.845107+0000","last_change":"2024-09-11T08:53:05.835268+0000","last_active":"2024-09-11T08:53:37.845107+0000","last_peered":"2024-09-11T08:53:37.845107+0000","last_clean":"2024-09-11T08:53:37.845107+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:53:37.845107+0000","last_undegraded":"2024-09-11T08:53:37.845107+0000","last_fullsized":"2024-09-11T08:53:37.845107+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'9","last_scrub_stamp":"2024-09-11T08:53:05.835177+0000","last_deep_scrub":"299'9","last_deep_scrub_stamp":"2024-09-11T08:53:05.835177+0000","last_clean_scrub_stamp":"2024-09-11T08:53:05.835177+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:24.501292+0000","scrub_duration":0.0089053550000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":306,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150904+0000","last_change":"2024-09-11T08:53:07.008184+0000","last_active":"2024-09-11T08:53:33.150904+0000","last_peered":"2024-09-11T08:53:33.150904+0000","last_clean":"2024-09-11T08:53:33.150904+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:53:33.150904+0000","last_undegraded":"2024-09-11T08:53:33.150904+0000","last_fullsized":"2024-09-11T08:53:33.150904+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:07.008088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:53:07.008088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:15.188670+0000","scrub_duration":0.0041216330000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"309'4718","reported_seq":4741,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.458190+0000","last_change":"2024-09-11T08:52:56.413212+0000","last_active":"2024-09-11T08:53:41.458190+0000","last_peered":"2024-09-11T08:53:41.458190+0000","last_clean":"2024-09-11T08:53:41.458190+0000","last_became_active":"2024-09-11T08:52:56.412642+0000","last_became_peered":"2024-09-11T08:52:56.412642+0000","last_unstale":"2024-09-11T08:53:41.458190+0000","last_undegraded":"2024-09-11T08:53:41.458190+0000","last_fullsized":"2024-09-11T08:53:41.458190+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4718,"log_dups_size":0,"ondisk_log_size":4718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:59.474410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":309198848,"num_objects":4718,"num_object_clones":0,"num_object_copies":14154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4718,"num_write_kb":301952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"200'15","reported_seq":61,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148772+0000","last_change":"2024-09-11T08:52:26.532447+0000","last_active":"2024-09-11T08:53:33.148772+0000","last_peered":"2024-09-11T08:53:33.148772+0000","last_clean":"2024-09-11T08:53:33.148772+0000","last_became_active":"2024-09-11T08:52:18.588329+0000","last_became_peered":"2024-09-11T08:52:18.588329+0000","last_unstale":"2024-09-11T08:53:33.148772+0000","last_undegraded":"2024-09-11T08:53:33.148772+0000","last_fullsized":"2024-09-11T08:53:33.148772+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:52:26.532385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:52:26.532385+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:41.095587+0000","scrub_duration":0.003040849,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":324,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.147301+0000","last_change":"2024-09-11T08:53:13.771378+0000","last_active":"2024-09-11T08:53:33.147301+0000","last_peered":"2024-09-11T08:53:33.147301+0000","last_clean":"2024-09-11T08:53:33.147301+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:53:33.147301+0000","last_undegraded":"2024-09-11T08:53:33.147301+0000","last_fullsized":"2024-09-11T08:53:33.147301+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:13.771300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:53:13.771300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:35.854425+0000","scrub_duration":0.0068129849999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":340,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151183+0000","last_change":"2024-09-11T08:53:00.940980+0000","last_active":"2024-09-11T08:53:33.151183+0000","last_peered":"2024-09-11T08:53:33.151183+0000","last_clean":"2024-09-11T08:53:33.151183+0000","last_became_active":"2024-09-11T08:38:42.982834+0000","last_became_peered":"2024-09-11T08:38:42.982834+0000","last_unstale":"2024-09-11T08:53:33.151183+0000","last_undegraded":"2024-09-11T08:53:33.151183+0000","last_fullsized":"2024-09-11T08:53:33.151183+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:00.940914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:00.940914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:04.012934+0000","scrub_duration":0.0014530680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":34,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151425+0000","last_change":"2024-09-11T08:52:49.570019+0000","last_active":"2024-09-11T08:53:33.151425+0000","last_peered":"2024-09-11T08:53:33.151425+0000","last_clean":"2024-09-11T08:53:33.151425+0000","last_became_active":"2024-09-11T08:52:49.569749+0000","last_became_peered":"2024-09-11T08:52:49.569749+0000","last_unstale":"2024-09-11T08:53:33.151425+0000","last_undegraded":"2024-09-11T08:53:33.151425+0000","last_fullsized":"2024-09-11T08:53:33.151425+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:14.645975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:14.645975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"309'4468","reported_seq":4492,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.210173+0000","last_change":"2024-09-11T08:52:56.413137+0000","last_active":"2024-09-11T08:53:41.210173+0000","last_peered":"2024-09-11T08:53:41.210173+0000","last_clean":"2024-09-11T08:53:41.210173+0000","last_became_active":"2024-09-11T08:52:56.412780+0000","last_became_peered":"2024-09-11T08:52:56.412780+0000","last_unstale":"2024-09-11T08:53:41.210173+0000","last_undegraded":"2024-09-11T08:53:41.210173+0000","last_fullsized":"2024-09-11T08:53:41.210173+0000","mapping_epoch":293,"log_start":"299'100","ondisk_log_start":"299'100","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4368,"log_dups_size":0,"ondisk_log_size":4368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:08.686334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":292814848,"num_objects":4468,"num_object_clones":0,"num_object_copies":13404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4468,"num_write_kb":285952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"200'35","reported_seq":227,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.153696+0000","last_change":"2024-09-11T08:51:35.255093+0000","last_active":"2024-09-11T08:53:33.153696+0000","last_peered":"2024-09-11T08:53:33.153696+0000","last_clean":"2024-09-11T08:53:33.153696+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:53:33.153696+0000","last_undegraded":"2024-09-11T08:53:33.153696+0000","last_fullsized":"2024-09-11T08:53:33.153696+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'35","last_scrub_stamp":"2024-09-11T08:51:35.255013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:35.255013+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048258839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.147642+0000","last_change":"2024-09-11T08:52:38.485375+0000","last_active":"2024-09-11T08:53:33.147642+0000","last_peered":"2024-09-11T08:53:33.147642+0000","last_clean":"2024-09-11T08:53:33.147642+0000","last_became_active":"2024-09-11T08:52:38.484857+0000","last_became_peered":"2024-09-11T08:52:38.484857+0000","last_unstale":"2024-09-11T08:53:33.147642+0000","last_undegraded":"2024-09-11T08:53:33.147642+0000","last_fullsized":"2024-09-11T08:53:33.147642+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:16.196521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:16.196521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":369,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.152047+0000","last_change":"2024-09-11T08:53:01.907502+0000","last_active":"2024-09-11T08:53:33.152047+0000","last_peered":"2024-09-11T08:53:33.152047+0000","last_clean":"2024-09-11T08:53:33.152047+0000","last_became_active":"2024-09-11T08:52:18.584748+0000","last_became_peered":"2024-09-11T08:52:18.584748+0000","last_unstale":"2024-09-11T08:53:33.152047+0000","last_undegraded":"2024-09-11T08:53:33.152047+0000","last_fullsized":"2024-09-11T08:53:33.152047+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:01.907441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:35.022256+0000","last_clean_scrub_stamp":"2024-09-11T08:53:01.907441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:28.038717+0000","scrub_duration":0.001627903,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":44,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151154+0000","last_change":"2024-09-11T08:52:59.930178+0000","last_active":"2024-09-11T08:53:33.151154+0000","last_peered":"2024-09-11T08:53:33.151154+0000","last_clean":"2024-09-11T08:53:33.151154+0000","last_became_active":"2024-09-11T08:52:38.480684+0000","last_became_peered":"2024-09-11T08:52:38.480684+0000","last_unstale":"2024-09-11T08:53:33.151154+0000","last_undegraded":"2024-09-11T08:53:33.151154+0000","last_fullsized":"2024-09-11T08:53:33.151154+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:59.930119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:59.930119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:09.598207+0000","scrub_duration":0.0027767069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.7","version":"309'4620","reported_seq":4643,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.288155+0000","last_change":"2024-09-11T08:52:55.613497+0000","last_active":"2024-09-11T08:53:41.288155+0000","last_peered":"2024-09-11T08:53:41.288155+0000","last_clean":"2024-09-11T08:53:41.288155+0000","last_became_active":"2024-09-11T08:52:55.613121+0000","last_became_peered":"2024-09-11T08:52:55.613121+0000","last_unstale":"2024-09-11T08:53:41.288155+0000","last_undegraded":"2024-09-11T08:53:41.288155+0000","last_fullsized":"2024-09-11T08:53:41.288155+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4620,"log_dups_size":0,"ondisk_log_size":4620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:07.376802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302776320,"num_objects":4620,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4620,"num_write_kb":295680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"302'23","reported_seq":347,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150034+0000","last_change":"2024-09-11T08:53:16.897477+0000","last_active":"2024-09-11T08:53:33.150034+0000","last_peered":"2024-09-11T08:53:33.150034+0000","last_clean":"2024-09-11T08:53:33.150034+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:53:33.150034+0000","last_undegraded":"2024-09-11T08:53:33.150034+0000","last_fullsized":"2024-09-11T08:53:33.150034+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'23","last_scrub_stamp":"2024-09-11T08:53:16.896643+0000","last_deep_scrub":"302'23","last_deep_scrub_stamp":"2024-09-11T08:53:16.896643+0000","last_clean_scrub_stamp":"2024-09-11T08:53:16.896643+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:45.449138+0000","scrub_duration":0.0090878710000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151560+0000","last_change":"2024-09-11T08:52:38.488564+0000","last_active":"2024-09-11T08:53:33.151560+0000","last_peered":"2024-09-11T08:53:33.151560+0000","last_clean":"2024-09-11T08:53:33.151560+0000","last_became_active":"2024-09-11T08:52:38.487881+0000","last_became_peered":"2024-09-11T08:52:38.487881+0000","last_unstale":"2024-09-11T08:53:33.151560+0000","last_undegraded":"2024-09-11T08:53:33.151560+0000","last_fullsized":"2024-09-11T08:53:33.151560+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:24.331966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:24.331966+0000","last_clean_scrub_stamp":"2024-09-11T08:51:24.331966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":359,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151222+0000","last_change":"2024-09-11T08:52:18.995061+0000","last_active":"2024-09-11T08:53:33.151222+0000","last_peered":"2024-09-11T08:53:33.151222+0000","last_clean":"2024-09-11T08:53:33.151222+0000","last_became_active":"2024-09-11T08:52:18.585136+0000","last_became_peered":"2024-09-11T08:52:18.585136+0000","last_unstale":"2024-09-11T08:53:33.151222+0000","last_undegraded":"2024-09-11T08:53:33.151222+0000","last_fullsized":"2024-09-11T08:53:33.151222+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:18.995003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:18.995003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012036359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":295,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.162095+0000","last_change":"2024-09-11T08:50:15.277171+0000","last_active":"2024-09-11T08:53:33.162095+0000","last_peered":"2024-09-11T08:53:33.162095+0000","last_clean":"2024-09-11T08:53:33.162095+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:53:33.162095+0000","last_undegraded":"2024-09-11T08:53:33.162095+0000","last_fullsized":"2024-09-11T08:53:33.162095+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:15.277094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:50:15.277094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004775751,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"309'4513","reported_seq":4537,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-11T08:53:41.524326+0000","last_change":"2024-09-11T08:52:56.413103+0000","last_active":"2024-09-11T08:53:41.524326+0000","last_peered":"2024-09-11T08:53:41.524326+0000","last_clean":"2024-09-11T08:53:41.524326+0000","last_became_active":"2024-09-11T08:52:56.412787+0000","last_became_peered":"2024-09-11T08:52:56.412787+0000","last_unstale":"2024-09-11T08:53:41.524326+0000","last_undegraded":"2024-09-11T08:53:41.524326+0000","last_fullsized":"2024-09-11T08:53:41.524326+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4513,"log_dups_size":0,"ondisk_log_size":4513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:10.489342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":295763968,"num_objects":4513,"num_object_clones":0,"num_object_copies":13539,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4513,"num_write_kb":288832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"200'34","reported_seq":417,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.144050+0000","last_change":"2024-09-11T08:50:11.325671+0000","last_active":"2024-09-11T08:53:33.144050+0000","last_peered":"2024-09-11T08:53:33.144050+0000","last_clean":"2024-09-11T08:53:33.144050+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:53:33.144050+0000","last_undegraded":"2024-09-11T08:53:33.144050+0000","last_fullsized":"2024-09-11T08:53:33.144050+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:50:11.325616+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:50:11.325616+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023307359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":54,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151023+0000","last_change":"2024-09-11T08:52:54.988951+0000","last_active":"2024-09-11T08:53:33.151023+0000","last_peered":"2024-09-11T08:53:33.151023+0000","last_clean":"2024-09-11T08:53:33.151023+0000","last_became_active":"2024-09-11T08:52:38.484199+0000","last_became_peered":"2024-09-11T08:52:38.484199+0000","last_unstale":"2024-09-11T08:53:33.151023+0000","last_undegraded":"2024-09-11T08:53:33.151023+0000","last_fullsized":"2024-09-11T08:53:33.151023+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:52:54.988868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.988868+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:06.018653+0000","scrub_duration":0.0019261269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"287'11","reported_seq":324,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:37.844557+0000","last_change":"2024-09-11T08:53:11.314150+0000","last_active":"2024-09-11T08:53:37.844557+0000","last_peered":"2024-09-11T08:53:37.844557+0000","last_clean":"2024-09-11T08:53:37.844557+0000","last_became_active":"2024-09-11T08:52:38.488856+0000","last_became_peered":"2024-09-11T08:52:38.488856+0000","last_unstale":"2024-09-11T08:53:37.844557+0000","last_undegraded":"2024-09-11T08:53:37.844557+0000","last_fullsized":"2024-09-11T08:53:37.844557+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"287'11","last_scrub_stamp":"2024-09-11T08:53:11.314069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:11.314069+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:12.315549+0000","scrub_duration":0.0041108129999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":331,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.146561+0000","last_change":"2024-09-11T08:52:26.813313+0000","last_active":"2024-09-11T08:53:33.146561+0000","last_peered":"2024-09-11T08:53:33.146561+0000","last_clean":"2024-09-11T08:53:33.146561+0000","last_became_active":"2024-09-11T08:52:18.573947+0000","last_became_peered":"2024-09-11T08:52:18.573947+0000","last_unstale":"2024-09-11T08:53:33.146561+0000","last_undegraded":"2024-09-11T08:53:33.146561+0000","last_fullsized":"2024-09-11T08:53:33.146561+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:26.813220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:26.813220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:45.083545+0000","scrub_duration":0.001765602,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":370,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.153690+0000","last_change":"2024-09-11T08:52:31.391319+0000","last_active":"2024-09-11T08:53:33.153690+0000","last_peered":"2024-09-11T08:53:33.153690+0000","last_clean":"2024-09-11T08:53:33.153690+0000","last_became_active":"2024-09-11T08:52:18.584768+0000","last_became_peered":"2024-09-11T08:52:18.584768+0000","last_unstale":"2024-09-11T08:53:33.153690+0000","last_undegraded":"2024-09-11T08:53:33.153690+0000","last_fullsized":"2024-09-11T08:53:33.153690+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:31.391081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:52:31.391081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044936749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"200'12","reported_seq":345,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.162412+0000","last_change":"2024-09-11T08:52:58.504342+0000","last_active":"2024-09-11T08:53:33.162412+0000","last_peered":"2024-09-11T08:53:33.162412+0000","last_clean":"2024-09-11T08:53:33.162412+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:53:33.162412+0000","last_undegraded":"2024-09-11T08:53:33.162412+0000","last_fullsized":"2024-09-11T08:53:33.162412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'12","last_scrub_stamp":"2024-09-11T08:52:58.504277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:52:58.504277+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:02.199526+0000","scrub_duration":0.0016055349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":317,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148676+0000","last_change":"2024-09-11T08:53:20.732004+0000","last_active":"2024-09-11T08:53:33.148676+0000","last_peered":"2024-09-11T08:53:33.148676+0000","last_clean":"2024-09-11T08:53:33.148676+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:53:33.148676+0000","last_undegraded":"2024-09-11T08:53:33.148676+0000","last_fullsized":"2024-09-11T08:53:33.148676+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:20.731788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:20.731788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:36.734097+0000","scrub_duration":0.0022864600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":334,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.149053+0000","last_change":"2024-09-11T08:52:55.411931+0000","last_active":"2024-09-11T08:53:33.149053+0000","last_peered":"2024-09-11T08:53:33.149053+0000","last_clean":"2024-09-11T08:53:33.149053+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:53:33.149053+0000","last_undegraded":"2024-09-11T08:53:33.149053+0000","last_fullsized":"2024-09-11T08:53:33.149053+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:55.411867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:55.411867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:10.675551+0000","scrub_duration":0.001452861,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"200'15","reported_seq":341,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.162038+0000","last_change":"2024-09-11T08:52:49.570164+0000","last_active":"2024-09-11T08:53:33.162038+0000","last_peered":"2024-09-11T08:53:33.162038+0000","last_clean":"2024-09-11T08:53:33.162038+0000","last_became_active":"2024-09-11T08:52:49.569850+0000","last_became_peered":"2024-09-11T08:52:49.569850+0000","last_unstale":"2024-09-11T08:53:33.162038+0000","last_undegraded":"2024-09-11T08:53:33.162038+0000","last_fullsized":"2024-09-11T08:53:33.162038+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:50:24.263662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:24.263662+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052890580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150221+0000","last_change":"2024-09-11T08:52:38.486022+0000","last_active":"2024-09-11T08:53:33.150221+0000","last_peered":"2024-09-11T08:53:33.150221+0000","last_clean":"2024-09-11T08:53:33.150221+0000","last_became_active":"2024-09-11T08:52:38.485708+0000","last_became_peered":"2024-09-11T08:52:38.485708+0000","last_unstale":"2024-09-11T08:53:33.150221+0000","last_undegraded":"2024-09-11T08:53:33.150221+0000","last_fullsized":"2024-09-11T08:53:33.150221+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:20.368038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:51:20.368038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":297,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.162035+0000","last_change":"2024-09-11T08:50:19.288275+0000","last_active":"2024-09-11T08:53:33.162035+0000","last_peered":"2024-09-11T08:53:33.162035+0000","last_clean":"2024-09-11T08:53:33.162035+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:53:33.162035+0000","last_undegraded":"2024-09-11T08:53:33.162035+0000","last_fullsized":"2024-09-11T08:53:33.162035+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:19.288216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:19.288216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055736010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":335,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151060+0000","last_change":"2024-09-11T08:52:24.921838+0000","last_active":"2024-09-11T08:53:33.151060+0000","last_peered":"2024-09-11T08:53:33.151060+0000","last_clean":"2024-09-11T08:53:33.151060+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:53:33.151060+0000","last_undegraded":"2024-09-11T08:53:33.151060+0000","last_fullsized":"2024-09-11T08:53:33.151060+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:24.921781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_clean_scrub_stamp":"2024-09-11T08:52:24.921781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001256795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"200'15","reported_seq":340,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.147712+0000","last_change":"2024-09-11T08:50:26.391719+0000","last_active":"2024-09-11T08:53:33.147712+0000","last_peered":"2024-09-11T08:53:33.147712+0000","last_clean":"2024-09-11T08:53:33.147712+0000","last_became_active":"2024-09-11T08:38:41.041727+0000","last_became_peered":"2024-09-11T08:38:41.041727+0000","last_unstale":"2024-09-11T08:53:33.147712+0000","last_undegraded":"2024-09-11T08:53:33.147712+0000","last_fullsized":"2024-09-11T08:53:33.147712+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:50:26.391622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:50:26.391622+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021692130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":35,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.161221+0000","last_change":"2024-09-11T08:52:49.567663+0000","last_active":"2024-09-11T08:53:33.161221+0000","last_peered":"2024-09-11T08:53:33.161221+0000","last_clean":"2024-09-11T08:53:33.161221+0000","last_became_active":"2024-09-11T08:52:49.566759+0000","last_became_peered":"2024-09-11T08:52:49.566759+0000","last_unstale":"2024-09-11T08:53:33.161221+0000","last_undegraded":"2024-09-11T08:53:33.161221+0000","last_fullsized":"2024-09-11T08:53:33.161221+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:16.609826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:16.609826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":485,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:37.847348+0000","last_change":"2024-09-11T08:50:16.313863+0000","last_active":"2024-09-11T08:53:37.847348+0000","last_peered":"2024-09-11T08:53:37.847348+0000","last_clean":"2024-09-11T08:53:37.847348+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:53:37.847348+0000","last_undegraded":"2024-09-11T08:53:37.847348+0000","last_fullsized":"2024-09-11T08:53:37.847348+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:50:16.313779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:16.313779+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049551839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":299,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150754+0000","last_change":"2024-09-11T08:52:27.235517+0000","last_active":"2024-09-11T08:53:33.150754+0000","last_peered":"2024-09-11T08:53:33.150754+0000","last_clean":"2024-09-11T08:53:33.150754+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:53:33.150754+0000","last_undegraded":"2024-09-11T08:53:33.150754+0000","last_fullsized":"2024-09-11T08:53:33.150754+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:27.235448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:27.235448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017172019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"200'25","reported_seq":344,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.162985+0000","last_change":"2024-09-11T08:52:53.432742+0000","last_active":"2024-09-11T08:53:33.162985+0000","last_peered":"2024-09-11T08:53:33.162985+0000","last_clean":"2024-09-11T08:53:33.162985+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:53:33.162985+0000","last_undegraded":"2024-09-11T08:53:33.162985+0000","last_fullsized":"2024-09-11T08:53:33.162985+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:52:53.432685+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:52:53.432685+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:02.336717+0000","scrub_duration":0.001930122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":330,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148753+0000","last_change":"2024-09-11T08:53:01.430368+0000","last_active":"2024-09-11T08:53:33.148753+0000","last_peered":"2024-09-11T08:53:33.148753+0000","last_clean":"2024-09-11T08:53:33.148753+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:53:33.148753+0000","last_undegraded":"2024-09-11T08:53:33.148753+0000","last_fullsized":"2024-09-11T08:53:33.148753+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:01.430318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:27.436552+0000","last_clean_scrub_stamp":"2024-09-11T08:53:01.430318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:11.060354+0000","scrub_duration":0.00111052,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"277'10","reported_seq":523,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:37.844439+0000","last_change":"2024-09-11T08:52:18.599864+0000","last_active":"2024-09-11T08:53:37.844439+0000","last_peered":"2024-09-11T08:53:37.844439+0000","last_clean":"2024-09-11T08:53:37.844439+0000","last_became_active":"2024-09-11T08:52:18.585914+0000","last_became_peered":"2024-09-11T08:52:18.585914+0000","last_unstale":"2024-09-11T08:53:37.844439+0000","last_undegraded":"2024-09-11T08:53:37.844439+0000","last_fullsized":"2024-09-11T08:53:37.844439+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'9","last_scrub_stamp":"2024-09-11T08:50:08.388991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:50:08.388991+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019634517000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":45,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.162111+0000","last_change":"2024-09-11T08:53:01.446031+0000","last_active":"2024-09-11T08:53:33.162111+0000","last_peered":"2024-09-11T08:53:33.162111+0000","last_clean":"2024-09-11T08:53:33.162111+0000","last_became_active":"2024-09-11T08:52:49.567693+0000","last_became_peered":"2024-09-11T08:52:49.567693+0000","last_unstale":"2024-09-11T08:53:33.162111+0000","last_undegraded":"2024-09-11T08:53:33.162111+0000","last_fullsized":"2024-09-11T08:53:33.162111+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:01.445935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:01.445935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:10.808558+0000","scrub_duration":0.0019176729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"200'19","reported_seq":353,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.150117+0000","last_change":"2024-09-11T08:51:31.019004+0000","last_active":"2024-09-11T08:53:33.150117+0000","last_peered":"2024-09-11T08:53:33.150117+0000","last_clean":"2024-09-11T08:53:33.150117+0000","last_became_active":"2024-09-11T08:49:16.937185+0000","last_became_peered":"2024-09-11T08:49:16.937185+0000","last_unstale":"2024-09-11T08:53:33.150117+0000","last_undegraded":"2024-09-11T08:53:33.150117+0000","last_fullsized":"2024-09-11T08:53:33.150117+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:31.018893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:31.018893+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062729989999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":312,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151595+0000","last_change":"2024-09-11T08:53:21.891151+0000","last_active":"2024-09-11T08:53:33.151595+0000","last_peered":"2024-09-11T08:53:33.151595+0000","last_clean":"2024-09-11T08:53:33.151595+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:53:33.151595+0000","last_undegraded":"2024-09-11T08:53:33.151595+0000","last_fullsized":"2024-09-11T08:53:33.151595+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:21.891073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:21.891073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:51.054430+0000","scrub_duration":0.0024938569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":317,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.147980+0000","last_change":"2024-09-11T08:52:15.686017+0000","last_active":"2024-09-11T08:53:33.147980+0000","last_peered":"2024-09-11T08:53:33.147980+0000","last_clean":"2024-09-11T08:53:33.147980+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:53:33.147980+0000","last_undegraded":"2024-09-11T08:53:33.147980+0000","last_fullsized":"2024-09-11T08:53:33.147980+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:15.685694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:15.685694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:41.819033+0000","scrub_duration":0.0028796590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":325,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.160703+0000","last_change":"2024-09-11T08:53:21.391938+0000","last_active":"2024-09-11T08:53:33.160703+0000","last_peered":"2024-09-11T08:53:33.160703+0000","last_clean":"2024-09-11T08:53:33.160703+0000","last_became_active":"2024-09-11T08:52:18.585089+0000","last_became_peered":"2024-09-11T08:52:18.585089+0000","last_unstale":"2024-09-11T08:53:33.160703+0000","last_undegraded":"2024-09-11T08:53:33.160703+0000","last_fullsized":"2024-09-11T08:53:33.160703+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:21.391849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:53:21.391849+0000","last_clean_scrub_stamp":"2024-09-11T08:53:21.391849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:40.478905+0000","scrub_duration":0.003344732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"200'15","reported_seq":357,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.146427+0000","last_change":"2024-09-11T08:53:19.736495+0000","last_active":"2024-09-11T08:53:33.146427+0000","last_peered":"2024-09-11T08:53:33.146427+0000","last_clean":"2024-09-11T08:53:33.146427+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:53:33.146427+0000","last_undegraded":"2024-09-11T08:53:33.146427+0000","last_fullsized":"2024-09-11T08:53:33.146427+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:53:19.736363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:19.736363+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:27.755432+0000","scrub_duration":0.0040364540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":45,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.161971+0000","last_change":"2024-09-11T08:53:16.431184+0000","last_active":"2024-09-11T08:53:33.161971+0000","last_peered":"2024-09-11T08:53:33.161971+0000","last_clean":"2024-09-11T08:53:33.161971+0000","last_became_active":"2024-09-11T08:52:38.485748+0000","last_became_peered":"2024-09-11T08:52:38.485748+0000","last_unstale":"2024-09-11T08:53:33.161971+0000","last_undegraded":"2024-09-11T08:53:33.161971+0000","last_fullsized":"2024-09-11T08:53:33.161971+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:16.430996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:21.208684+0000","last_clean_scrub_stamp":"2024-09-11T08:53:16.430996+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:26.673701+0000","scrub_duration":0.0051148779999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":327,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.162223+0000","last_change":"2024-09-11T08:52:54.441725+0000","last_active":"2024-09-11T08:53:33.162223+0000","last_peered":"2024-09-11T08:53:33.162223+0000","last_clean":"2024-09-11T08:53:33.162223+0000","last_became_active":"2024-09-11T08:52:18.585691+0000","last_became_peered":"2024-09-11T08:52:18.585691+0000","last_unstale":"2024-09-11T08:53:33.162223+0000","last_undegraded":"2024-09-11T08:53:33.162223+0000","last_fullsized":"2024-09-11T08:53:33.162223+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.441666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.441666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:08.658190+0000","scrub_duration":0.0012216760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":117,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.143915+0000","last_change":"2024-09-11T08:52:49.567009+0000","last_active":"2024-09-11T08:53:33.143915+0000","last_peered":"2024-09-11T08:53:33.143915+0000","last_clean":"2024-09-11T08:53:33.143915+0000","last_became_active":"2024-09-11T08:52:49.566694+0000","last_became_peered":"2024-09-11T08:52:49.566694+0000","last_unstale":"2024-09-11T08:53:33.143915+0000","last_undegraded":"2024-09-11T08:53:33.143915+0000","last_fullsized":"2024-09-11T08:53:33.143915+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:24.401480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:24.401480+0000","last_clean_scrub_stamp":"2024-09-11T08:50:24.401480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"200'19","reported_seq":330,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.160512+0000","last_change":"2024-09-11T08:51:21.423287+0000","last_active":"2024-09-11T08:53:33.160512+0000","last_peered":"2024-09-11T08:53:33.160512+0000","last_clean":"2024-09-11T08:53:33.160512+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:53:33.160512+0000","last_undegraded":"2024-09-11T08:53:33.160512+0000","last_fullsized":"2024-09-11T08:53:33.160512+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:21.423213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:21.423213+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086807859999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":321,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148741+0000","last_change":"2024-09-11T08:53:04.734505+0000","last_active":"2024-09-11T08:53:33.148741+0000","last_peered":"2024-09-11T08:53:33.148741+0000","last_clean":"2024-09-11T08:53:33.148741+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:53:33.148741+0000","last_undegraded":"2024-09-11T08:53:33.148741+0000","last_fullsized":"2024-09-11T08:53:33.148741+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:04.734452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:04.734452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:19.824548+0000","scrub_duration":0.001307342,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":329,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.147976+0000","last_change":"2024-09-11T08:53:00.454714+0000","last_active":"2024-09-11T08:53:33.147976+0000","last_peered":"2024-09-11T08:53:33.147976+0000","last_clean":"2024-09-11T08:53:33.147976+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:53:33.147976+0000","last_undegraded":"2024-09-11T08:53:33.147976+0000","last_fullsized":"2024-09-11T08:53:33.147976+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:00.454658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:00.454658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:08.231173+0000","scrub_duration":0.0013095660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":335,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.162307+0000","last_change":"2024-09-11T08:52:18.586241+0000","last_active":"2024-09-11T08:53:33.162307+0000","last_peered":"2024-09-11T08:53:33.162307+0000","last_clean":"2024-09-11T08:53:33.162307+0000","last_became_active":"2024-09-11T08:52:18.585315+0000","last_became_peered":"2024-09-11T08:52:18.585315+0000","last_unstale":"2024-09-11T08:53:33.162307+0000","last_undegraded":"2024-09-11T08:53:33.162307+0000","last_fullsized":"2024-09-11T08:53:33.162307+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:50:25.255332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:25.255332+0000","last_clean_scrub_stamp":"2024-09-11T08:50:25.255332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046648280000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"200'10","reported_seq":358,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.159934+0000","last_change":"2024-09-11T08:52:11.431872+0000","last_active":"2024-09-11T08:53:33.159934+0000","last_peered":"2024-09-11T08:53:33.159934+0000","last_clean":"2024-09-11T08:53:33.159934+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:53:33.159934+0000","last_undegraded":"2024-09-11T08:53:33.159934+0000","last_fullsized":"2024-09-11T08:53:33.159934+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'10","last_scrub_stamp":"2024-09-11T08:52:11.431805+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:52:11.431805+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013697399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":129,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148553+0000","last_change":"2024-09-11T08:52:49.570339+0000","last_active":"2024-09-11T08:53:33.148553+0000","last_peered":"2024-09-11T08:53:33.148553+0000","last_clean":"2024-09-11T08:53:33.148553+0000","last_became_active":"2024-09-11T08:52:49.569954+0000","last_became_peered":"2024-09-11T08:52:49.569954+0000","last_unstale":"2024-09-11T08:53:33.148553+0000","last_undegraded":"2024-09-11T08:53:33.148553+0000","last_fullsized":"2024-09-11T08:53:33.148553+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:50:09.396148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:50:09.396148+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013176124000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":190,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.152985+0000","last_change":"2024-09-11T08:53:14.382649+0000","last_active":"2024-09-11T08:53:33.152985+0000","last_peered":"2024-09-11T08:53:33.152985+0000","last_clean":"2024-09-11T08:53:33.152985+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:53:33.152985+0000","last_undegraded":"2024-09-11T08:53:33.152985+0000","last_fullsized":"2024-09-11T08:53:33.152985+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:14.382556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:14.382556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:22.938992+0000","scrub_duration":0.0028503370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":44,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.151115+0000","last_change":"2024-09-11T08:52:51.904084+0000","last_active":"2024-09-11T08:53:33.151115+0000","last_peered":"2024-09-11T08:53:33.151115+0000","last_clean":"2024-09-11T08:53:33.151115+0000","last_became_active":"2024-09-11T08:52:38.481474+0000","last_became_peered":"2024-09-11T08:52:38.481474+0000","last_unstale":"2024-09-11T08:53:33.151115+0000","last_undegraded":"2024-09-11T08:53:33.151115+0000","last_fullsized":"2024-09-11T08:53:33.151115+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:51.904033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:30.788303+0000","last_clean_scrub_stamp":"2024-09-11T08:52:51.904033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:07.919285+0000","scrub_duration":0.0016307439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":325,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.148336+0000","last_change":"2024-09-11T08:52:54.846053+0000","last_active":"2024-09-11T08:53:33.148336+0000","last_peered":"2024-09-11T08:53:33.148336+0000","last_clean":"2024-09-11T08:53:33.148336+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:53:33.148336+0000","last_undegraded":"2024-09-11T08:53:33.148336+0000","last_fullsized":"2024-09-11T08:53:33.148336+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.845993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.845993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:53:59.400471+0000","scrub_duration":0.001860252,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":331,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.160118+0000","last_change":"2024-09-11T08:52:18.595475+0000","last_active":"2024-09-11T08:53:33.160118+0000","last_peered":"2024-09-11T08:53:33.160118+0000","last_clean":"2024-09-11T08:53:33.160118+0000","last_became_active":"2024-09-11T08:52:18.595146+0000","last_became_peered":"2024-09-11T08:52:18.595146+0000","last_unstale":"2024-09-11T08:53:33.160118+0000","last_undegraded":"2024-09-11T08:53:33.160118+0000","last_fullsized":"2024-09-11T08:53:33.160118+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:32.393197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:32.393197+0000","last_clean_scrub_stamp":"2024-09-11T08:51:32.393197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001696321,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"200'19","reported_seq":204,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.154892+0000","last_change":"2024-09-11T08:51:32.119010+0000","last_active":"2024-09-11T08:53:33.154892+0000","last_peered":"2024-09-11T08:53:33.154892+0000","last_clean":"2024-09-11T08:53:33.154892+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:53:33.154892+0000","last_undegraded":"2024-09-11T08:53:33.154892+0000","last_fullsized":"2024-09-11T08:53:33.154892+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:32.118917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:32.118917+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023296060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":126,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-09-11T08:53:33.149324+0000","last_change":"2024-09-11T08:52:53.465187+0000","last_active":"2024-09-11T08:53:33.149324+0000","last_peered":"2024-09-11T08:53:33.149324+0000","last_clean":"2024-09-11T08:53:33.149324+0000","last_became_active":"2024-09-11T08:52:49.568537+0000","last_became_peered":"2024-09-11T08:52:49.568537+0000","last_unstale":"2024-09-11T08:53:33.149324+0000","last_undegraded":"2024-09-11T08:53:33.149324+0000","last_fullsized":"2024-09-11T08:53:33.149324+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:53.465128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:53.465128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:19.014737+0000","scrub_duration":0.00141025,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":4843962368,"num_objects":73913,"num_object_clones":0,"num_object_copies":221739,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73913,"num_write_kb":4730432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14586281984,"data_stored":14586281984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73213,"ondisk_log_size":73213,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1119,"num_read_kb":876,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":4114,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":761,"ondisk_log_size":761,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":65,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":751378,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":63,"num_write":32,"num_write_kb":444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":268349,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299578,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1777664,"kb_used_data":1187768,"kb_used_omap":1,"kb_used_meta":589886,"kb_avail":91967488,"statfs":{"total":95995035648,"available":94174707712,"internally_reserved":0,"allocated":1216274432,"data_stored":1215879228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":604043608},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:52:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.86299999999999999,"15min":0.93899999999999995},"min":{"1min":0.249,"5min":0.23200000000000001,"15min":0.22},"max":{"1min":4.2610000000000001,"5min":4.8730000000000002,"15min":32.171999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.94199999999999995,"15min":1.044},"min":{"1min":0.311,"5min":0.26700000000000002,"15min":0.21199999999999999},"max":{"1min":2.6480000000000001,"5min":4.0069999999999997,"15min":32.149999999999999},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:52:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.1519999999999999,"15min":1.1160000000000001},"min":{"1min":0.33300000000000002,"5min":0.224,"15min":0.224},"max":{"1min":4.3029999999999999,"5min":4.8970000000000002,"15min":33.692999999999998},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.083,"15min":1.0309999999999999},"min":{"1min":0.28000000000000003,"5min":0.23799999999999999,"15min":0.23300000000000001},"max":{"1min":2.9590000000000001,"5min":5.2030000000000003,"15min":35.591000000000001},"last":0.59199999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:52:50 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.0549999999999999,"15min":0.93600000000000005},"min":{"1min":0.34799999999999998,"5min":0.29799999999999999,"15min":0.249},"max":{"1min":4.9829999999999997,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.30499999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0169999999999999,"15min":0.89200000000000002},"min":{"1min":0.29199999999999998,"5min":0.27900000000000003,"15min":0.25800000000000001},"max":{"1min":4.282,"5min":7.5259999999999998,"15min":7.5259999999999998},"last":0.434}]},{"osd":3,"last update":"Wed Sep 11 08:52:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.784,"15min":1.198},"min":{"1min":0.32200000000000001,"5min":0.27200000000000002,"15min":0.25},"max":{"1min":5.0220000000000002,"5min":56.265999999999998,"15min":56.265999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.7190000000000001,"15min":1.2549999999999999},"min":{"1min":0.36299999999999999,"5min":0.34499999999999997,"15min":0.27400000000000002},"max":{"1min":4.2329999999999997,"5min":57.207000000000001,"15min":57.207000000000001},"last":0.378}]},{"osd":4,"last update":"Wed Sep 11 08:52:50 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0009999999999999,"15min":0.66100000000000003},"min":{"1min":0.28100000000000003,"5min":0.255,"15min":0.23400000000000001},"max":{"1min":5.468,"5min":7.1520000000000001,"15min":7.1520000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.0980000000000001,"15min":0.70499999999999996},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.216},"max":{"1min":5.1669999999999998,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.4139999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:52:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.052,"15min":0.70499999999999996},"min":{"1min":0.317,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":4.2050000000000001,"5min":5.1059999999999999,"15min":5.1059999999999999},"last":1.089},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.143,"15min":0.77800000000000002},"min":{"1min":0.311,"5min":0.308,"15min":0.26700000000000002},"max":{"1min":5.2080000000000002,"5min":6.3650000000000002,"15min":6.3650000000000002},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Sep 11 08:52:50 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":1.073,"15min":0.76000000000000001},"min":{"1min":0.35399999999999998,"5min":0.32000000000000001,"15min":0.315},"max":{"1min":2.992,"5min":4.5449999999999999,"15min":4.5449999999999999},"last":0.625},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.224,"15min":0.79700000000000004},"min":{"1min":0.35999999999999999,"5min":0.33300000000000002,"15min":0.27900000000000003},"max":{"1min":5.0620000000000003,"5min":5.0620000000000003,"15min":5.0620000000000003},"last":0.747}]}]},{"osd":6,"up_from":45,"seq":193273528511,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2636280,"kb_used_data":2038060,"kb_used_omap":1,"kb_used_meta":598206,"kb_avail":91108872,"statfs":{"total":95995035648,"available":93295484928,"internally_reserved":0,"allocated":2086973440,"data_stored":2086590360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":612563288},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":6.7320000000000002,"5min":2.2719999999999998,"15min":1.5680000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.217},"max":{"1min":121.208,"5min":121.208,"15min":121.208},"last":0.51000000000000001},{"interface":"front","average":{"1min":6.8529999999999998,"5min":2.4550000000000001,"15min":1.5669999999999999},"min":{"1min":0.44800000000000001,"5min":0.24399999999999999,"15min":0.23300000000000001},"max":{"1min":121.05200000000001,"5min":121.05200000000001,"15min":121.05200000000001},"last":3.3439999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":1.3220000000000001,"15min":1.173},"min":{"1min":0.247,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.4159999999999999,"5min":27.984000000000002,"15min":27.984000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":1.327,"15min":1.1299999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.253},"max":{"1min":2.1520000000000001,"5min":28.106000000000002,"15min":28.106000000000002},"last":1.925}]},{"osd":2,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.72199999999999998,"15min":0.82699999999999996},"min":{"1min":0.32200000000000001,"5min":0.27500000000000002,"15min":0.23200000000000001},"max":{"1min":1.4570000000000001,"5min":3.899,"15min":14.313000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.82899999999999996,"15min":0.876},"min":{"1min":0.42499999999999999,"5min":0.27400000000000002,"15min":0.251},"max":{"1min":1.909,"5min":4.9020000000000001,"15min":6.5979999999999999},"last":0.65900000000000003}]},{"osd":3,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":1.357,"15min":1.206},"min":{"1min":0.27200000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.4550000000000001,"5min":27.795000000000002,"15min":27.795000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":1.3959999999999999,"15min":1.194},"min":{"1min":0.309,"5min":0.30499999999999999,"15min":0.224},"max":{"1min":1.9730000000000001,"5min":27.841999999999999,"15min":27.841999999999999},"last":1.175}]},{"osd":4,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.325,"15min":0.78100000000000003},"min":{"1min":0.27400000000000002,"5min":0.24299999999999999,"15min":0.20799999999999999},"max":{"1min":5.8689999999999998,"5min":40.927999999999997,"15min":40.927999999999997},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":1.337,"15min":0.78800000000000003},"min":{"1min":0.308,"5min":0.27700000000000002,"15min":0.26400000000000001},"max":{"1min":2.5209999999999999,"5min":40.706000000000003,"15min":40.706000000000003},"last":0.41799999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.5029999999999999,"15min":0.91100000000000003},"min":{"1min":0.311,"5min":0.27600000000000002,"15min":0.23899999999999999},"max":{"1min":3.8540000000000001,"5min":40.847000000000001,"15min":40.847000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.3759999999999999,"15min":0.85799999999999998},"min":{"1min":0.29899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.758,"5min":41.401000000000003,"15min":41.401000000000003},"last":0.46000000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:52:48 2024","interfaces":[{"interface":"back","average":{"1min":2.9449999999999998,"5min":1.7150000000000001,"15min":0.94199999999999995},"min":{"1min":0.32500000000000001,"5min":0.313,"15min":0.247},"max":{"1min":40.426000000000002,"5min":40.426000000000002,"15min":40.426000000000002},"last":0.379},{"interface":"front","average":{"1min":3.3780000000000001,"5min":1.7589999999999999,"15min":0.94199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.27700000000000002},"max":{"1min":45.401000000000003,"5min":45.401000000000003,"15min":45.401000000000003},"last":0.46899999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757443,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2432264,"kb_used_data":1529984,"kb_used_omap":1,"kb_used_meta":902270,"kb_avail":91312888,"statfs":{"total":95995035648,"available":93504397312,"internally_reserved":0,"allocated":1566703616,"data_stored":1566301507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":923924824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.105,"15min":0.93200000000000005},"min":{"1min":0.221,"5min":0.21099999999999999,"15min":0.20599999999999999},"max":{"1min":8.7349999999999994,"5min":8.7349999999999994,"15min":8.7349999999999994},"last":1.948},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.0389999999999999,"15min":1.026},"min":{"1min":0.23899999999999999,"5min":0.23400000000000001,"15min":0.23100000000000001},"max":{"1min":3.9780000000000002,"5min":5.569,"15min":8.0410000000000004},"last":0.45700000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0609999999999999,"15min":1.01},"min":{"1min":0.27600000000000002,"5min":0.216,"15min":0.216},"max":{"1min":5.2060000000000004,"5min":5.3280000000000003,"15min":8},"last":2.552},{"interface":"front","average":{"1min":1.236,"5min":1.23,"15min":1.026},"min":{"1min":0.307,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":12.173,"5min":12.173,"15min":12.173},"last":2.4260000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.099,"15min":0.998},"min":{"1min":0.28199999999999997,"5min":0.253,"15min":0.23799999999999999},"max":{"1min":8.7789999999999999,"5min":8.7789999999999999,"15min":8.7789999999999999},"last":2.0630000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.98399999999999999,"15min":0.93700000000000006},"min":{"1min":0.33700000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.7509999999999999,"5min":5.2329999999999997,"15min":8.0640000000000001},"last":0.56999999999999995}]},{"osd":3,"last update":"Wed Sep 11 08:53:12 2024","interfaces":[{"interface":"back","average":{"1min":5.2060000000000004,"5min":2.0089999999999999,"15min":1.2709999999999999},"min":{"1min":0.30099999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":85.299999999999997,"5min":85.299999999999997,"15min":85.299999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":5.0019999999999998,"5min":2.1309999999999998,"15min":1.423},"min":{"1min":0.43099999999999999,"5min":0.29099999999999998,"15min":0.27400000000000002},"max":{"1min":85.941000000000003,"5min":85.941000000000003,"15min":85.941000000000003},"last":1.0600000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.149,"15min":0.73699999999999999},"min":{"1min":0.28699999999999998,"5min":0.24399999999999999,"15min":0.20699999999999999},"max":{"1min":16.021999999999998,"5min":16.021999999999998,"15min":16.021999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.034,"15min":0.68500000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.20499999999999999},"max":{"1min":9.6140000000000008,"5min":9.6140000000000008,"15min":9.6140000000000008},"last":0.29499999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:53:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":1.1259999999999999,"15min":0.76500000000000001},"min":{"1min":0.33800000000000002,"5min":0.30399999999999999,"15min":0.23599999999999999},"max":{"1min":1.855,"5min":8.8059999999999992,"15min":8.8059999999999992},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":1.0469999999999999,"15min":0.72799999999999998},"min":{"1min":0.27400000000000002,"5min":0.27300000000000002,"15min":0.251},"max":{"1min":3.2170000000000001,"5min":5.3090000000000002,"15min":8.1920000000000002},"last":0.42599999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:52:47 2024","interfaces":[{"interface":"back","average":{"1min":2.044,"5min":2.2090000000000001,"15min":1.1080000000000001},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.27800000000000002},"max":{"1min":16.062999999999999,"5min":88.513000000000005,"15min":88.513000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":2.1389999999999998,"15min":1.1000000000000001},"min":{"1min":0.36699999999999999,"5min":0.317,"15min":0.29599999999999999},"max":{"1min":8.8300000000000001,"5min":86.963999999999999,"15min":86.963999999999999},"last":0.47499999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986375,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2636480,"kb_used_data":2330808,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91108672,"statfs":{"total":95995035648,"available":93295280128,"internally_reserved":0,"allocated":2386747392,"data_stored":2386351650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:52:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.579,"15min":1.6379999999999999},"min":{"1min":0.23899999999999999,"5min":0.22600000000000001,"15min":0.20399999999999999},"max":{"1min":5.2400000000000002,"5min":44.140999999999998,"15min":89.510000000000005},"last":1.877},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.526,"15min":1.72},"min":{"1min":0.308,"5min":0.23699999999999999,"15min":0.22600000000000001},"max":{"1min":5.3220000000000001,"5min":44.450000000000003,"15min":89.765000000000001},"last":0.91800000000000004}]},{"osd":1,"last update":"Wed Sep 11 08:52:49 2024","interfaces":[{"interface":"back","average":{"1min":1.8260000000000001,"5min":2.2810000000000001,"15min":1.456},"min":{"1min":0.27400000000000002,"5min":0.27300000000000002,"15min":0.24399999999999999},"max":{"1min":18.065999999999999,"5min":48.976999999999997,"15min":48.976999999999997},"last":0.437},{"interface":"front","average":{"1min":1.8779999999999999,"5min":2.1480000000000001,"15min":1.401},"min":{"1min":0.33700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":18.713999999999999,"5min":45.787999999999997,"15min":45.787999999999997},"last":1.968}]},{"osd":2,"last update":"Wed Sep 11 08:52:49 2024","interfaces":[{"interface":"back","average":{"1min":16.032,"5min":3.843,"15min":1.8899999999999999},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":255.88900000000001,"5min":255.88900000000001,"15min":255.88900000000001},"last":1.915},{"interface":"front","average":{"1min":15.782999999999999,"5min":3.7810000000000001,"15min":1.8520000000000001},"min":{"1min":0.36799999999999999,"5min":0.29199999999999998,"15min":0.25800000000000001},"max":{"1min":255.41200000000001,"5min":255.41200000000001,"15min":255.41200000000001},"last":1.004}]},{"osd":3,"last update":"Wed Sep 11 08:52:49 2024","interfaces":[{"interface":"back","average":{"1min":1.9139999999999999,"5min":2.1030000000000002,"15min":1.97},"min":{"1min":0.41499999999999998,"5min":0.252,"15min":0.252},"max":{"1min":16.663,"5min":45.268000000000001,"15min":89.881},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.8160000000000001,"5min":2.02,"15min":1.915},"min":{"1min":0.32700000000000001,"5min":0.29299999999999998,"15min":0.25900000000000001},"max":{"1min":16.795999999999999,"5min":44.234000000000002,"15min":89.715000000000003},"last":0.46200000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:53:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.99299999999999999,"15min":0.69599999999999995},"min":{"1min":0.28999999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.4350000000000001,"5min":11.045999999999999,"15min":11.045999999999999},"last":1.214},{"interface":"front","average":{"1min":0.70299999999999996,"5min":1.004,"15min":0.69799999999999995},"min":{"1min":0.27100000000000002,"5min":0.254,"15min":0.23699999999999999},"max":{"1min":3.1659999999999999,"5min":10.526999999999999,"15min":10.526999999999999},"last":2.6190000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:52:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":0.89100000000000001,"15min":0.61199999999999999},"min":{"1min":0.32000000000000001,"5min":0.27300000000000002,"15min":0.22900000000000001},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":0.85199999999999998,"15min":0.65400000000000003},"min":{"1min":0.35999999999999999,"5min":0.27800000000000002,"15min":0.221},"max":{"1min":6.2619999999999996,"5min":6.2619999999999996,"15min":6.2619999999999996},"last":0.39100000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:52:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.873,"15min":0.66200000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.252},"max":{"1min":3.1469999999999998,"5min":4.8079999999999998,"15min":4.8079999999999998},"last":1.944},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.0289999999999999,"15min":0.73899999999999999},"min":{"1min":0.34399999999999997,"5min":0.32300000000000001,"15min":0.28799999999999998},"max":{"1min":5.0199999999999996,"5min":5.0199999999999996,"15min":5.1970000000000001},"last":0.93700000000000006}]}]},{"osd":3,"up_from":26,"seq":111669149901,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2453596,"kb_used_data":1550228,"kb_used_omap":1,"kb_used_meta":903358,"kb_avail":91291556,"statfs":{"total":95995035648,"available":93482553344,"internally_reserved":0,"allocated":1587433472,"data_stored":1587011183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":925038936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.66400000000000003,"15min":1.1100000000000001},"min":{"1min":0.26400000000000001,"5min":0.20100000000000001,"15min":0.16500000000000001},"max":{"1min":2.8639999999999999,"5min":3.0310000000000001,"15min":24.280000000000001},"last":1.605},{"interface":"front","average":{"1min":1.125,"5min":0.79200000000000004,"15min":1.3340000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.17000000000000001},"max":{"1min":6.0780000000000003,"5min":6.0780000000000003,"15min":22.225000000000001},"last":0.66200000000000003}]},{"osd":1,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.98299999999999998,"15min":1.7},"min":{"1min":0.25700000000000001,"5min":0.20100000000000001,"15min":0.19800000000000001},"max":{"1min":2.2549999999999999,"5min":7.4909999999999997,"15min":115.601},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.879,"15min":1.581},"min":{"1min":0.22900000000000001,"5min":0.20499999999999999,"15min":0.17999999999999999},"max":{"1min":1.4450000000000001,"5min":6.6639999999999997,"15min":115.474},"last":0.51600000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":1.694,"15min":1.4850000000000001},"min":{"1min":0.26000000000000001,"5min":0.24299999999999999,"15min":0.23200000000000001},"max":{"1min":2.625,"5min":86.766000000000005,"15min":86.766000000000005},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":1.754,"15min":1.4550000000000001},"min":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.20799999999999999},"max":{"1min":3.5760000000000001,"5min":85.367999999999995,"15min":85.367999999999995},"last":1.625}]},{"osd":4,"last update":"Wed Sep 11 08:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.093,"15min":1.1719999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.26700000000000002},"max":{"1min":4.843,"5min":6.0519999999999996,"15min":8.9969999999999999},"last":1.7989999999999999},{"interface":"front","average":{"1min":1.143,"5min":0.90100000000000002,"15min":0.995},"min":{"1min":0.33300000000000002,"5min":0.29099999999999998,"15min":0.27700000000000002},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":7.234},"last":1.5249999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:53:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0680000000000001,"15min":1.1679999999999999},"min":{"1min":0.35999999999999999,"5min":0.33900000000000002,"15min":0.28299999999999997},"max":{"1min":3.391,"5min":6.0119999999999996,"15min":11.526999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.999,"5min":1.073,"15min":1.079},"min":{"1min":0.42899999999999999,"5min":0.314,"15min":0.29399999999999998},"max":{"1min":3.9750000000000001,"5min":4.8179999999999996,"15min":7.1689999999999996},"last":1.5549999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":1.4239999999999999,"15min":1.3560000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.6970000000000001,"5min":33.106999999999999,"15min":33.106999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":1.4019999999999999,"15min":1.335},"min":{"1min":0.371,"5min":0.36499999999999999,"15min":0.311},"max":{"1min":3.2730000000000001,"5min":33.076000000000001,"15min":33.076000000000001},"last":2.1539999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:52:55 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.6930000000000001,"15min":1.431},"min":{"1min":0.45700000000000002,"5min":0.38700000000000001,"15min":0.35299999999999998},"max":{"1min":3.7669999999999999,"5min":35.073,"15min":35.073},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.6879999999999999,"15min":1.302},"min":{"1min":0.48299999999999998,"5min":0.377,"15min":0.34699999999999998},"max":{"1min":3.754,"5min":34.844999999999999,"15min":34.844999999999999},"last":1.5840000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411537,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2052664,"kb_used_data":1751152,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91692488,"statfs":{"total":95995035648,"available":93893107712,"internally_reserved":0,"allocated":1793179648,"data_stored":1792792573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:53:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":1.151,"15min":1.5389999999999999},"min":{"1min":0.20799999999999999,"5min":0.191,"15min":0.159},"max":{"1min":2.0910000000000002,"5min":16.606999999999999,"15min":34.713999999999999},"last":1.746},{"interface":"front","average":{"1min":0.40699999999999997,"5min":1.044,"15min":1.6140000000000001},"min":{"1min":0.23699999999999999,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.85999999999999999,"5min":16.855,"15min":34.399000000000001},"last":1.5}]},{"osd":1,"last update":"Wed Sep 11 08:53:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":2.0550000000000002,"15min":1.661},"min":{"1min":0.29899999999999999,"5min":0.23799999999999999,"15min":0.19},"max":{"1min":2.1920000000000002,"5min":72.867000000000004,"15min":72.867000000000004},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.46500000000000002,"5min":2.1349999999999998,"15min":1.7649999999999999},"min":{"1min":0.26800000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.1839999999999999,"5min":67.799000000000007,"15min":67.799000000000007},"last":1.208}]},{"osd":3,"last update":"Wed Sep 11 08:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":1.302,"15min":1.5569999999999999},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":2.1259999999999999,"5min":16.353999999999999,"15min":34.75},"last":2.4929999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":1.2350000000000001,"15min":1.502},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":2.1619999999999999,"5min":16.138999999999999,"15min":32.802},"last":1.157}]},{"osd":4,"last update":"Wed Sep 11 08:53:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.502,"15min":1.238},"min":{"1min":0.38,"5min":0.28699999999999998,"15min":0.255},"max":{"1min":6.0659999999999998,"5min":52.113999999999997,"15min":52.113999999999997},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.4179999999999999,"15min":1.177},"min":{"1min":0.42199999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":5.5780000000000003,"5min":52.097999999999999,"15min":52.097999999999999},"last":1.1839999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:53:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":1.4350000000000001,"15min":1.1599999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.96199999999999997,"5min":52.145000000000003,"15min":52.145000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":1.3200000000000001,"15min":1.0429999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.28399999999999997},"max":{"1min":3.3119999999999998,"5min":52.216000000000001,"15min":52.216000000000001},"last":2.476}]},{"osd":6,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":1.417,"15min":1.262},"min":{"1min":0.34000000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.8640000000000001,"5min":52.235999999999997,"15min":52.235999999999997},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.323,"5min":1.516,"15min":1.1930000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":5.2939999999999996,"5min":52.186,"15min":52.186},"last":0.46500000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:52:42 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.802,"15min":1.21},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.30399999999999999},"max":{"1min":4.4500000000000002,"5min":84.108999999999995,"15min":84.108999999999995},"last":2.5150000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.9510000000000001,"15min":1.3979999999999999},"min":{"1min":0.40699999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":7.0720000000000001,"5min":85.938000000000002,"15min":85.938000000000002},"last":0.86499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575062,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1809360,"kb_used_data":905672,"kb_used_omap":1,"kb_used_meta":903678,"kb_avail":91935792,"statfs":{"total":95995035648,"available":94142251008,"internally_reserved":0,"allocated":927408128,"data_stored":926996632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":925366616},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:52:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.91700000000000004,"15min":1.464},"min":{"1min":0.17499999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":2.794,"5min":3.9470000000000001,"15min":117.01600000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.94699999999999995,"15min":1.7470000000000001},"min":{"1min":0.20799999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":3.6659999999999999,"5min":8.1549999999999994,"15min":117.62},"last":0.53200000000000003}]},{"osd":2,"last update":"Wed Sep 11 08:53:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.97699999999999998,"15min":1.171},"min":{"1min":0.221,"5min":0.21099999999999999,"15min":0.191},"max":{"1min":1.819,"5min":8.1329999999999991,"15min":20.628},"last":2.1019999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":1.0880000000000001,"15min":1.1819999999999999},"min":{"1min":0.22500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":2.032,"5min":9.7200000000000006,"15min":20.148},"last":0.32500000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:53:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":1.1819999999999999,"15min":1.5349999999999999},"min":{"1min":0.255,"5min":0.223,"15min":0.191},"max":{"1min":1.9079999999999999,"5min":5.1859999999999999,"15min":116.98},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":1.0429999999999999,"15min":1.589},"min":{"1min":0.29699999999999999,"5min":0.246,"15min":0.22},"max":{"1min":1.869,"5min":4.4409999999999998,"15min":117.56699999999999},"last":0.25800000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:52:42 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.23,"15min":1.044},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.26200000000000001},"max":{"1min":7.1109999999999998,"5min":7.1109999999999998,"15min":11.19},"last":0.379},{"interface":"front","average":{"1min":1.365,"5min":1.1719999999999999,"15min":0.96599999999999997},"min":{"1min":0.41199999999999998,"5min":0.28399999999999997,"15min":0.251},"max":{"1min":6.952,"5min":6.952,"15min":7.7169999999999996},"last":0.67800000000000005}]},{"osd":5,"last update":"Wed Sep 11 08:53:09 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.3180000000000001,"15min":1.5700000000000001},"min":{"1min":0.315,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.3650000000000002,"5min":8.0999999999999996,"15min":103.337},"last":0.625},{"interface":"front","average":{"1min":0.66100000000000003,"5min":1.3200000000000001,"15min":1.54},"min":{"1min":0.40300000000000002,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":2.0720000000000001,"5min":5.9020000000000001,"15min":103.163},"last":0.40799999999999997}]},{"osd":6,"last update":"Wed Sep 11 08:53:37 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.1539999999999999,"15min":1.028},"min":{"1min":0.317,"5min":0.30499999999999999,"15min":0.29399999999999998},"max":{"1min":2.1440000000000001,"5min":6.2249999999999996,"15min":8.9149999999999991},"last":1.456},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.1819999999999999,"15min":1.1830000000000001},"min":{"1min":0.41099999999999998,"5min":0.315,"15min":0.31},"max":{"1min":2.6520000000000001,"5min":6.2869999999999999,"15min":9.5199999999999996},"last":0.49399999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:52:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.212,"15min":1.087},"min":{"1min":0.371,"5min":0.37,"15min":0.32800000000000001},"max":{"1min":2.5840000000000001,"5min":4.6289999999999996,"15min":11.315},"last":0.433},{"interface":"front","average":{"1min":1.133,"5min":1.3080000000000001,"15min":1.0609999999999999},"min":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.315},"max":{"1min":2.9550000000000001,"5min":10.266,"15min":10.266},"last":0.64600000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738586,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3273540,"kb_used_data":2972028,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":90471612,"statfs":{"total":95995035648,"available":92642930688,"internally_reserved":0,"allocated":3043356672,"data_stored":3042957431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:52:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.7989999999999999,"15min":1.4670000000000001},"min":{"1min":0.247,"5min":0.19,"15min":0.16300000000000001},"max":{"1min":6.9699999999999998,"5min":56.685000000000002,"15min":56.685000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.79,"15min":1.407},"min":{"1min":0.26200000000000001,"5min":0.217,"15min":0.186},"max":{"1min":7.694,"5min":61.012999999999998,"15min":61.012999999999998},"last":0.374}]},{"osd":2,"last update":"Wed Sep 11 08:53:08 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.74399999999999999,"15min":1.143},"min":{"1min":0.253,"5min":0.223,"15min":0.21099999999999999},"max":{"1min":3.2850000000000001,"5min":3.9449999999999998,"15min":19.940000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.875,"5min":0.81200000000000006,"15min":1.2330000000000001},"min":{"1min":0.222,"5min":0.22,"15min":0.20000000000000001},"max":{"1min":3.7730000000000001,"5min":3.9300000000000002,"15min":20.757000000000001},"last":0.59799999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.5549999999999999,"15min":1.55},"min":{"1min":0.35599999999999998,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":7.7709999999999999,"5min":56.095999999999997,"15min":56.095999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":1.5109999999999999,"15min":1.478},"min":{"1min":0.28399999999999997,"5min":0.21199999999999999,"15min":0.20899999999999999},"max":{"1min":0.65900000000000003,"5min":59.651000000000003,"15min":59.651000000000003},"last":0.71299999999999997}]},{"osd":4,"last update":"Wed Sep 11 08:52:38 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.95899999999999996,"15min":1.0760000000000001},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.25},"max":{"1min":2.6659999999999999,"5min":4.6959999999999997,"15min":7.4690000000000003},"last":3.109},{"interface":"front","average":{"1min":1.222,"5min":0.95799999999999996,"15min":1.0509999999999999},"min":{"1min":0.34799999999999998,"5min":0.316,"15min":0.255},"max":{"1min":3.8050000000000002,"5min":3.9670000000000001,"15min":7.4630000000000001},"last":0.50900000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:53:08 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.022,"15min":1.153},"min":{"1min":0.40300000000000002,"5min":0.316,"15min":0.27800000000000002},"max":{"1min":3.8149999999999999,"5min":5.3550000000000004,"15min":7.4009999999999998},"last":2.2789999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":1.1399999999999999,"15min":1.181},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.28299999999999997},"max":{"1min":2.5670000000000002,"5min":6.4420000000000002,"15min":7.423},"last":0.82999999999999996}]},{"osd":6,"last update":"Wed Sep 11 08:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.052,"15min":1.224},"min":{"1min":0.35899999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":3.8460000000000001,"5min":4.1859999999999999,"15min":7.4240000000000004},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.016,"15min":1.2410000000000001},"min":{"1min":0.51000000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.9209999999999998,"5min":4.907,"15min":7.4489999999999998},"last":0.996}]},{"osd":7,"last update":"Wed Sep 11 08:52:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.8069999999999999,"15min":1.409},"min":{"1min":0.41599999999999998,"5min":0.33500000000000002,"15min":0.313},"max":{"1min":3.6030000000000002,"5min":56.119,"15min":56.119},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.591,"5min":1.901,"15min":1.4690000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.307},"max":{"1min":7.29,"5min":56.100000000000001,"15min":56.100000000000001},"last":0.67500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3056730112,"data_stored":3056730112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":927006720,"data_stored":927006720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1821048832,"data_stored":1821048832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1524039680,"data_stored":1524039680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2403336192,"data_stored":2403336192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1515192320,"data_stored":1515192320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2117599232,"data_stored":2117599232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1221328896,"data_stored":1221328896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:53:48.109 INFO:tasks.thrashosds.thrasher:Setting 4.15 to [2, 5, 0, 6, 7, 1] 2024-09-11T08:53:48.109 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '4.15', '2', '5', '0', '6', '7', '1'] 2024-09-11T08:53:48.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pg-upmap-items 4.15 2 5 0 6 7 1 2024-09-11T08:53:48.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:48.111 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:48.111 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:48.155 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:48.212 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:48.257 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:48.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:48.259 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:48.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:48.409 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:48.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:48.410 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:48.483 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:53:48.512 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:48.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:48.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:48.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:48.698 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:48 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:48.698 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:48 smithi092 ceph-mon[29647]: from='client.25318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:53:48.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:48 smithi092 ceph-mon[29647]: pgmap v680: 158 pgs: 158 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.68k op/s 2024-09-11T08:53:48.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:48 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2515007089' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:53:48.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:48 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:53:48.699 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:48.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:48 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:48.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:48 smithi092 ceph-mon[41114]: from='client.25318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:53:48.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:48 smithi092 ceph-mon[41114]: pgmap v680: 158 pgs: 158 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.68k op/s 2024-09-11T08:53:48.700 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:48 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2515007089' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:53:48.700 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:48 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:53:48.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:48.700 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:48.801 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:48.843 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:48.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:48.844 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:48.894 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 44 16 83718 83702 118.852 77.4375 0.00945977 0.00840935 2024-09-11T08:53:48.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:48 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:48 smithi145 ceph-mon[33384]: from='client.25318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:53:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:48 smithi145 ceph-mon[33384]: pgmap v680: 158 pgs: 158 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.68k op/s 2024-09-11T08:53:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:48 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2515007089' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:53:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:48 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:53:48.944 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:48.978 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:48.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:48.980 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:49.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:49.117 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:49.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:49.118 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:49.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:49.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:49.256 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:49.256 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:49.357 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:49.395 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:49.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:49.397 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:49.497 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:49.542 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:49.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:49.543 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:49.643 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:49.682 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:49.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:49.683 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:49.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:49.814 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:49.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:49.816 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:49.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:53:49.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:53:49.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[29647]: osdmap e311: 8 total, 8 up, 8 in 2024-09-11T08:53:49.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[29647]: 4.a scrub starts 2024-09-11T08:53:49.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[29647]: 4.a scrub ok 2024-09-11T08:53:49.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[29647]: 3.1a scrub starts 2024-09-11T08:53:49.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[29647]: 3.1a scrub ok 2024-09-11T08:53:49.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[29647]: osdmap e312: 8 total, 8 up, 8 in 2024-09-11T08:53:49.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:53:49.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:53:49.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:53:49.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:53:49.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[41114]: osdmap e311: 8 total, 8 up, 8 in 2024-09-11T08:53:49.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[41114]: 4.a scrub starts 2024-09-11T08:53:49.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[41114]: 4.a scrub ok 2024-09-11T08:53:49.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[41114]: 3.1a scrub starts 2024-09-11T08:53:49.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[41114]: 3.1a scrub ok 2024-09-11T08:53:49.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[41114]: osdmap e312: 8 total, 8 up, 8 in 2024-09-11T08:53:49.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:53:49.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:49 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:53:49.894 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 45 16 85031 85015 118.034 82.0625 0.0136696 0.00846723 2024-09-11T08:53:49.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:49 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:53:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:49 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:53:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:49 smithi145 ceph-mon[33384]: osdmap e311: 8 total, 8 up, 8 in 2024-09-11T08:53:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:49 smithi145 ceph-mon[33384]: 4.a scrub starts 2024-09-11T08:53:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:49 smithi145 ceph-mon[33384]: 4.a scrub ok 2024-09-11T08:53:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:49 smithi145 ceph-mon[33384]: 3.1a scrub starts 2024-09-11T08:53:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:49 smithi145 ceph-mon[33384]: 3.1a scrub ok 2024-09-11T08:53:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:49 smithi145 ceph-mon[33384]: osdmap e312: 8 total, 8 up, 8 in 2024-09-11T08:53:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:49 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:53:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:49 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:53:49.917 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:49.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:49.961 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:49.961 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:50.062 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:50.098 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:50.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:50.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:50.200 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:50.235 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:50.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:50.237 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:50.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:50.370 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:50.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:50.371 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:50.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:50.505 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:50.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:50.506 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:50.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:50.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:50.641 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:50.742 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:50.773 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:50.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:50.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:50.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:50 smithi092 ceph-mon[29647]: pgmap v683: 158 pgs: 158 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 70 MiB/s wr, 1.11k op/s 2024-09-11T08:53:50.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:50 smithi092 ceph-mon[29647]: osdmap e313: 8 total, 8 up, 8 in 2024-09-11T08:53:50.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:50 smithi092 ceph-mon[41114]: pgmap v683: 158 pgs: 158 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 70 MiB/s wr, 1.11k op/s 2024-09-11T08:53:50.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:50 smithi092 ceph-mon[41114]: osdmap e313: 8 total, 8 up, 8 in 2024-09-11T08:53:50.875 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:50.894 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 46 15 86386 86371 117.311 84.75 0.0115135 0.00851972 2024-09-11T08:53:50.910 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:50.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:50.911 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:50.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:50 smithi145 ceph-mon[33384]: pgmap v683: 158 pgs: 158 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 70 MiB/s wr, 1.11k op/s 2024-09-11T08:53:50.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:50 smithi145 ceph-mon[33384]: osdmap e313: 8 total, 8 up, 8 in 2024-09-11T08:53:51.012 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:51.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:51.052 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:51.053 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:51.154 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:51.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:51.188 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:51.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:51.289 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:51.314 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:51.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:51.315 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:51.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:51.456 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:51.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:51.457 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:51.558 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:51.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:51.597 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:51.597 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:51.698 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:51.736 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:51.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:51.737 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:51.748 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:51 smithi092 ceph-mon[29647]: 4.7 scrub starts 2024-09-11T08:53:51.748 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:51 smithi092 ceph-mon[29647]: 4.7 scrub ok 2024-09-11T08:53:51.748 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:51 smithi092 ceph-mon[41114]: 4.7 scrub starts 2024-09-11T08:53:51.749 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:51 smithi092 ceph-mon[41114]: 4.7 scrub ok 2024-09-11T08:53:51.749 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:53:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:53:51] "GET /metrics HTTP/1.1" 200 224771 "" "Prometheus/2.43.0" 2024-09-11T08:53:51.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:51.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:51.885 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:51.885 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:51.894 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 47 16 87685 87669 116.541 81.125 0.0104761 0.00857594 2024-09-11T08:53:51.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:51 smithi145 ceph-mon[33384]: 4.7 scrub starts 2024-09-11T08:53:51.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:51 smithi145 ceph-mon[33384]: 4.7 scrub ok 2024-09-11T08:53:51.986 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:52.020 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:52.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:52.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:52.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:52.164 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:52.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:52.165 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:52.217 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:53:52.266 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:52.311 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:52.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:52.312 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:52.413 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:52.452 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:52.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:52.453 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:52.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:52.596 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:52.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:52.597 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:52.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:52.732 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:52.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:52.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:52.766 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:52 smithi092 ceph-mon[29647]: pgmap v685: 158 pgs: 158 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 111 MiB/s wr, 1.78k op/s 2024-09-11T08:53:52.766 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:52 smithi092 ceph-mon[29647]: 3.2 scrub starts 2024-09-11T08:53:52.766 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:52 smithi092 ceph-mon[29647]: 3.2 scrub ok 2024-09-11T08:53:52.767 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:52 smithi092 ceph-mon[41114]: pgmap v685: 158 pgs: 158 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 111 MiB/s wr, 1.78k op/s 2024-09-11T08:53:52.767 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:52 smithi092 ceph-mon[41114]: 3.2 scrub starts 2024-09-11T08:53:52.767 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:52 smithi092 ceph-mon[41114]: 3.2 scrub ok 2024-09-11T08:53:52.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:52.875 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:52.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:52.876 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:52.894 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 48 16 88955 88939 115.767 79.375 0.014392 0.00863318 2024-09-11T08:53:52.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:53:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:53:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:53:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:52 smithi145 ceph-mon[33384]: pgmap v685: 158 pgs: 158 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 111 MiB/s wr, 1.78k op/s 2024-09-11T08:53:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:52 smithi145 ceph-mon[33384]: 3.2 scrub starts 2024-09-11T08:53:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:52 smithi145 ceph-mon[33384]: 3.2 scrub ok 2024-09-11T08:53:52.947 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:53:52.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:53.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:53.051 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:53.051 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:53.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:53.184 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:53:53.184 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:53:53.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:53.188 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:53.188 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:53.219 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:53:53.219 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:53:53.248 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:53:53.248 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:53:53.287 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:53:53.287 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:53:53.289 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:53.318 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:53:53.318 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:53:53.326 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:53.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:53.327 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:53.354 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:53:53.354 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:53:53.387 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:53:53.388 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:53:53.417 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:53:53.417 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:53:53.427 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:53.471 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:53.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:53.472 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:53.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:53 smithi092 ceph-mon[29647]: 4.1f scrub starts 2024-09-11T08:53:53.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:53 smithi092 ceph-mon[29647]: 4.1f scrub ok 2024-09-11T08:53:53.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:53 smithi092 ceph-mon[29647]: 6.9 scrub starts 2024-09-11T08:53:53.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:53 smithi092 ceph-mon[29647]: 6.9 scrub ok 2024-09-11T08:53:53.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:53 smithi092 ceph-mon[29647]: 4.b scrub starts 2024-09-11T08:53:53.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:53 smithi092 ceph-mon[29647]: 4.b scrub ok 2024-09-11T08:53:53.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:53 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3557350065' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:53:53.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:53 smithi092 ceph-mon[41114]: 4.1f scrub starts 2024-09-11T08:53:53.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:53 smithi092 ceph-mon[41114]: 4.1f scrub ok 2024-09-11T08:53:53.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:53 smithi092 ceph-mon[41114]: 6.9 scrub starts 2024-09-11T08:53:53.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:53 smithi092 ceph-mon[41114]: 6.9 scrub ok 2024-09-11T08:53:53.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:53 smithi092 ceph-mon[41114]: 4.b scrub starts 2024-09-11T08:53:53.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:53 smithi092 ceph-mon[41114]: 4.b scrub ok 2024-09-11T08:53:53.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:53 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3557350065' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:53:53.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:53.613 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:53.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:53.614 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:53.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:53.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:53.753 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:53.753 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:53.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:53.889 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:53.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:53.891 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:53.894 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 49 16 90135 90119 114.909 73.75 0.00707096 0.00869781 2024-09-11T08:53:53.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:53 smithi145 ceph-mon[33384]: 4.1f scrub starts 2024-09-11T08:53:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:53 smithi145 ceph-mon[33384]: 4.1f scrub ok 2024-09-11T08:53:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:53 smithi145 ceph-mon[33384]: 6.9 scrub starts 2024-09-11T08:53:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:53 smithi145 ceph-mon[33384]: 6.9 scrub ok 2024-09-11T08:53:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:53 smithi145 ceph-mon[33384]: 4.b scrub starts 2024-09-11T08:53:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:53 smithi145 ceph-mon[33384]: 4.b scrub ok 2024-09-11T08:53:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:53 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3557350065' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:53:53.993 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:54.023 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:54.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:54.024 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:54.125 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:54.162 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:54.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:54.163 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:54.263 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:54.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:54.301 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:54.301 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:54.402 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:54.435 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:54.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:54.436 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:54.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:54.583 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:54.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:54.584 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:54.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:54 smithi092 ceph-mon[29647]: pgmap v686: 158 pgs: 158 active+clean; 5.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-11T08:53:54.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:54 smithi092 ceph-mon[29647]: 5.15 scrub starts 2024-09-11T08:53:54.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:54 smithi092 ceph-mon[29647]: 5.15 scrub ok 2024-09-11T08:53:54.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:54 smithi092 ceph-mon[29647]: osdmap e314: 8 total, 8 up, 8 in 2024-09-11T08:53:54.684 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:54 smithi092 ceph-mon[41114]: pgmap v686: 158 pgs: 158 active+clean; 5.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-11T08:53:54.684 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:54 smithi092 ceph-mon[41114]: 5.15 scrub starts 2024-09-11T08:53:54.684 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:54 smithi092 ceph-mon[41114]: 5.15 scrub ok 2024-09-11T08:53:54.684 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:54 smithi092 ceph-mon[41114]: osdmap e314: 8 total, 8 up, 8 in 2024-09-11T08:53:54.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:54.724 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:54.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:54.725 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:54.826 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:54.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:54.861 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:54.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:54.894 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 50 16 91403 91387 114.196 79.25 0.00500744 0.00875208 2024-09-11T08:53:54.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:54 smithi145 ceph-mon[33384]: pgmap v686: 158 pgs: 158 active+clean; 5.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-11T08:53:54.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:54 smithi145 ceph-mon[33384]: 5.15 scrub starts 2024-09-11T08:53:54.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:54 smithi145 ceph-mon[33384]: 5.15 scrub ok 2024-09-11T08:53:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:54 smithi145 ceph-mon[33384]: osdmap e314: 8 total, 8 up, 8 in 2024-09-11T08:53:54.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:55.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:55.010 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:55.010 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:55.018 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:53:55.019 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:53:55.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:55.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:55.152 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:55.252 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:55.283 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:55.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:55.284 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:55.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:53:55.419 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:55.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:55.420 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:53:55.520 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:55.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:55.556 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:55.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:55.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:55.688 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:55.688 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:55.789 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:55.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:55 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:53:55.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:55 smithi092 ceph-mon[29647]: 6.7 scrub starts 2024-09-11T08:53:55.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:55 smithi092 ceph-mon[29647]: 6.7 scrub ok 2024-09-11T08:53:55.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:55 smithi092 ceph-mon[29647]: 5.8 scrub ok 2024-09-11T08:53:55.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:55 smithi092 ceph-mon[29647]: 5.19 scrub starts 2024-09-11T08:53:55.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:55 smithi092 ceph-mon[29647]: 5.19 scrub ok 2024-09-11T08:53:55.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:55 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:53:55.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:55 smithi092 ceph-mon[41114]: 6.7 scrub starts 2024-09-11T08:53:55.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:55 smithi092 ceph-mon[41114]: 6.7 scrub ok 2024-09-11T08:53:55.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:55 smithi092 ceph-mon[41114]: 5.8 scrub ok 2024-09-11T08:53:55.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:55 smithi092 ceph-mon[41114]: 5.19 scrub starts 2024-09-11T08:53:55.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:55 smithi092 ceph-mon[41114]: 5.19 scrub ok 2024-09-11T08:53:55.824 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:55.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:55.825 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:55.895 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 51 16 92694 92678 113.539 80.6875 0.0120323 0.00880286 2024-09-11T08:53:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:55 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:53:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:55 smithi145 ceph-mon[33384]: 6.7 scrub starts 2024-09-11T08:53:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:55 smithi145 ceph-mon[33384]: 6.7 scrub ok 2024-09-11T08:53:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:55 smithi145 ceph-mon[33384]: 5.8 scrub ok 2024-09-11T08:53:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:55 smithi145 ceph-mon[33384]: 5.19 scrub starts 2024-09-11T08:53:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:55 smithi145 ceph-mon[33384]: 5.19 scrub ok 2024-09-11T08:53:55.925 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:55.955 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:55.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:55.956 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:56.057 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:56.093 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:56.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:56.094 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:56.195 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:56.230 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:56.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:56.231 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:56.332 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:56.363 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:56.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:56.364 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:56.465 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:56.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:56.502 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:56.603 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:56.637 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:56.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:56.638 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:56.739 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:56.772 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:56.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:56.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:56.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:56 smithi092 ceph-mon[29647]: pgmap v688: 158 pgs: 158 active+clean; 5.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 812 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T08:53:56.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:56 smithi092 ceph-mon[29647]: 4.8 scrub starts 2024-09-11T08:53:56.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:56 smithi092 ceph-mon[29647]: 4.8 scrub ok 2024-09-11T08:53:56.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:56 smithi092 ceph-mon[41114]: pgmap v688: 158 pgs: 158 active+clean; 5.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 812 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T08:53:56.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:56 smithi092 ceph-mon[41114]: 4.8 scrub starts 2024-09-11T08:53:56.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:56 smithi092 ceph-mon[41114]: 4.8 scrub ok 2024-09-11T08:53:56.874 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:56.895 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 52 16 93872 93856 112.772 73.625 0.00738941 0.00886167 2024-09-11T08:53:56.904 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:56.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:56.905 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:56 smithi145 ceph-mon[33384]: pgmap v688: 158 pgs: 158 active+clean; 5.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 812 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T08:53:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:56 smithi145 ceph-mon[33384]: 4.8 scrub starts 2024-09-11T08:53:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:56 smithi145 ceph-mon[33384]: 4.8 scrub ok 2024-09-11T08:53:57.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:57.046 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:57.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:57.047 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:57.148 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:57.176 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:57.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:57.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:57.278 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:57.313 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:57.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:57.314 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:57.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:53:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:53:57.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:57.453 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:57.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:57.454 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:57.555 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:57.596 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:57.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:57.597 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:57.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:57.732 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:57.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:57.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[29647]: 6.14 scrub starts 2024-09-11T08:53:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[29647]: 6.14 scrub ok 2024-09-11T08:53:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[29647]: 3.d scrub starts 2024-09-11T08:53:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[29647]: 3.d scrub ok 2024-09-11T08:53:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[29647]: 6.1e scrub starts 2024-09-11T08:53:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[29647]: 6.1e scrub ok 2024-09-11T08:53:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/741187928' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.15", "id": [2, 5, 0, 6, 7, 1]}]: dispatch 2024-09-11T08:53:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.15", "id": [2, 5, 0, 6, 7, 1]}]: dispatch 2024-09-11T08:53:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[41114]: 6.14 scrub starts 2024-09-11T08:53:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[41114]: 6.14 scrub ok 2024-09-11T08:53:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[41114]: 3.d scrub starts 2024-09-11T08:53:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[41114]: 3.d scrub ok 2024-09-11T08:53:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[41114]: 6.1e scrub starts 2024-09-11T08:53:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[41114]: 6.1e scrub ok 2024-09-11T08:53:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/741187928' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.15", "id": [2, 5, 0, 6, 7, 1]}]: dispatch 2024-09-11T08:53:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:57 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.15", "id": [2, 5, 0, 6, 7, 1]}]: dispatch 2024-09-11T08:53:57.834 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:57.869 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:57.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:57.870 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:57.895 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 53 15 95009 94994 111.986 71.125 0.0122435 0.00892523 2024-09-11T08:53:57.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:57 smithi145 ceph-mon[33384]: 6.14 scrub starts 2024-09-11T08:53:57.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:57 smithi145 ceph-mon[33384]: 6.14 scrub ok 2024-09-11T08:53:57.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:57 smithi145 ceph-mon[33384]: 3.d scrub starts 2024-09-11T08:53:57.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:57 smithi145 ceph-mon[33384]: 3.d scrub ok 2024-09-11T08:53:57.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:57 smithi145 ceph-mon[33384]: 6.1e scrub starts 2024-09-11T08:53:57.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:57 smithi145 ceph-mon[33384]: 6.1e scrub ok 2024-09-11T08:53:57.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:57 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/741187928' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.15", "id": [2, 5, 0, 6, 7, 1]}]: dispatch 2024-09-11T08:53:57.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:57 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.15", "id": [2, 5, 0, 6, 7, 1]}]: dispatch 2024-09-11T08:53:57.971 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:53:58.003 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:58.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:58.004 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:53:58.034 INFO:teuthology.orchestra.run.smithi092.stderr:set 4.15 pg_upmap_items mapping to [2->5,0->6,7->1] 2024-09-11T08:53:58.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:58.144 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:58.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:58.145 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:58.245 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:58.284 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:58.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:58.285 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:58.386 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:58.422 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:58.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:58.423 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:58.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:58.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:58.567 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:58.568 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:58.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:58.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:58.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:58.803 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:58.814 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:58 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:58.814 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:58 smithi145 ceph-mon[33384]: pgmap v689: 158 pgs: 158 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 99 MiB/s wr, 1.59k op/s 2024-09-11T08:53:58.814 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:58 smithi145 ceph-mon[33384]: 4.3 scrub starts 2024-09-11T08:53:58.814 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:58 smithi145 ceph-mon[33384]: 4.3 scrub ok 2024-09-11T08:53:58.814 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:58 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "4.15", "id": [2, 5, 0, 6, 7, 1]}]': finished 2024-09-11T08:53:58.814 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:58 smithi145 ceph-mon[33384]: osdmap e315: 8 total, 8 up, 8 in 2024-09-11T08:53:58.814 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:58 smithi145 ceph-mon[33384]: 6.5 scrub starts 2024-09-11T08:53:58.814 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:58 smithi145 ceph-mon[33384]: 6.5 scrub ok 2024-09-11T08:53:58.814 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:58 smithi145 ceph-mon[33384]: 5.6 scrub starts 2024-09-11T08:53:58.814 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:58 smithi145 ceph-mon[33384]: 5.6 scrub ok 2024-09-11T08:53:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[29647]: pgmap v689: 158 pgs: 158 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 99 MiB/s wr, 1.59k op/s 2024-09-11T08:53:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[29647]: 4.3 scrub starts 2024-09-11T08:53:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[29647]: 4.3 scrub ok 2024-09-11T08:53:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "4.15", "id": [2, 5, 0, 6, 7, 1]}]': finished 2024-09-11T08:53:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[29647]: osdmap e315: 8 total, 8 up, 8 in 2024-09-11T08:53:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[29647]: 6.5 scrub starts 2024-09-11T08:53:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[29647]: 6.5 scrub ok 2024-09-11T08:53:58.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[29647]: 5.6 scrub starts 2024-09-11T08:53:58.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[29647]: 5.6 scrub ok 2024-09-11T08:53:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:53:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[41114]: pgmap v689: 158 pgs: 158 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 99 MiB/s wr, 1.59k op/s 2024-09-11T08:53:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[41114]: 4.3 scrub starts 2024-09-11T08:53:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[41114]: 4.3 scrub ok 2024-09-11T08:53:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "4.15", "id": [2, 5, 0, 6, 7, 1]}]': finished 2024-09-11T08:53:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[41114]: osdmap e315: 8 total, 8 up, 8 in 2024-09-11T08:53:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[41114]: 6.5 scrub starts 2024-09-11T08:53:58.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[41114]: 6.5 scrub ok 2024-09-11T08:53:58.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[41114]: 5.6 scrub starts 2024-09-11T08:53:58.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:58 smithi092 ceph-mon[41114]: 5.6 scrub ok 2024-09-11T08:53:58.837 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:58.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:58.837 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:53:58.895 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 54 16 96254 96238 111.352 77.75 0.0110681 0.00897595 2024-09-11T08:53:58.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:58.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:58.978 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:58.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:59.079 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:59.111 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:59.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:59.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:59.213 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:53:59.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:59.251 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:53:59.351 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:53:59.387 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:59.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:59.389 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:53:59.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:53:59.527 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:59.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:59.529 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:53:59.630 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:53:59.664 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:59.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:59.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:53:59.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:53:59.800 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:53:59.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:59.801 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:53:59.813 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[29647]: 6.b scrub starts 2024-09-11T08:53:59.813 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[29647]: 6.b scrub ok 2024-09-11T08:53:59.813 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[29647]: 6.1c scrub starts 2024-09-11T08:53:59.813 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[29647]: osdmap e316: 8 total, 8 up, 8 in 2024-09-11T08:53:59.813 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[29647]: 6.1c scrub ok 2024-09-11T08:53:59.814 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[29647]: 6.4 scrub starts 2024-09-11T08:53:59.814 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[29647]: 6.4 scrub ok 2024-09-11T08:53:59.814 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[29647]: 3.1b scrub starts 2024-09-11T08:53:59.814 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[29647]: 3.1b scrub ok 2024-09-11T08:53:59.814 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[41114]: 6.b scrub starts 2024-09-11T08:53:59.814 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[41114]: 6.b scrub ok 2024-09-11T08:53:59.814 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[41114]: 6.1c scrub starts 2024-09-11T08:53:59.814 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[41114]: osdmap e316: 8 total, 8 up, 8 in 2024-09-11T08:53:59.814 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[41114]: 6.1c scrub ok 2024-09-11T08:53:59.814 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[41114]: 6.4 scrub starts 2024-09-11T08:53:59.814 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[41114]: 6.4 scrub ok 2024-09-11T08:53:59.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[41114]: 3.1b scrub starts 2024-09-11T08:53:59.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:53:59 smithi092 ceph-mon[41114]: 3.1b scrub ok 2024-09-11T08:53:59.895 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 55 16 97441 97425 110.676 74.1875 0.00869158 0.00903036 2024-09-11T08:53:59.902 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:53:59.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:59 smithi145 ceph-mon[33384]: 6.b scrub starts 2024-09-11T08:53:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:59 smithi145 ceph-mon[33384]: 6.b scrub ok 2024-09-11T08:53:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:59 smithi145 ceph-mon[33384]: 6.1c scrub starts 2024-09-11T08:53:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:59 smithi145 ceph-mon[33384]: osdmap e316: 8 total, 8 up, 8 in 2024-09-11T08:53:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:59 smithi145 ceph-mon[33384]: 6.1c scrub ok 2024-09-11T08:53:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:59 smithi145 ceph-mon[33384]: 6.4 scrub starts 2024-09-11T08:53:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:59 smithi145 ceph-mon[33384]: 6.4 scrub ok 2024-09-11T08:53:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:59 smithi145 ceph-mon[33384]: 3.1b scrub starts 2024-09-11T08:53:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:53:59 smithi145 ceph-mon[33384]: 3.1b scrub ok 2024-09-11T08:53:59.943 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:53:59.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:53:59.945 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:00.045 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:00.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:00.081 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:00.081 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:00.182 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:00.221 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:00.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:00.223 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:00.323 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:00.360 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:00.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:00.361 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:00.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:00.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:00.503 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:00.504 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:00.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T08:54:00.569 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:00.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:00.661 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:00.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:00.663 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:00.763 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:00.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:00.801 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:00.802 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[29647]: pgmap v692: 158 pgs: 158 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-11T08:54:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[29647]: 5.1b scrub starts 2024-09-11T08:54:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[29647]: 5.1b scrub ok 2024-09-11T08:54:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[29647]: 5.1f scrub starts 2024-09-11T08:54:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[29647]: 5.1f scrub ok 2024-09-11T08:54:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[29647]: 5.1d scrub starts 2024-09-11T08:54:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[29647]: 5.1d scrub ok 2024-09-11T08:54:00.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:54:00.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[41114]: pgmap v692: 158 pgs: 158 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-11T08:54:00.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[41114]: 5.1b scrub starts 2024-09-11T08:54:00.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[41114]: 5.1b scrub ok 2024-09-11T08:54:00.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[41114]: 5.1f scrub starts 2024-09-11T08:54:00.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[41114]: 5.1f scrub ok 2024-09-11T08:54:00.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[41114]: 5.1d scrub starts 2024-09-11T08:54:00.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[41114]: 5.1d scrub ok 2024-09-11T08:54:00.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:54:00.896 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 56 16 98686 98670 110.086 77.8125 0.00914883 0.00907857 2024-09-11T08:54:00.902 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:00.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:00 smithi145 ceph-mon[33384]: pgmap v692: 158 pgs: 158 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-11T08:54:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:00 smithi145 ceph-mon[33384]: 5.1b scrub starts 2024-09-11T08:54:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:00 smithi145 ceph-mon[33384]: 5.1b scrub ok 2024-09-11T08:54:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:00 smithi145 ceph-mon[33384]: 5.1f scrub starts 2024-09-11T08:54:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:00 smithi145 ceph-mon[33384]: 5.1f scrub ok 2024-09-11T08:54:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:00 smithi145 ceph-mon[33384]: 5.1d scrub starts 2024-09-11T08:54:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:00 smithi145 ceph-mon[33384]: 5.1d scrub ok 2024-09-11T08:54:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:54:00.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:00.949 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:00.949 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:01.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:01.093 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:01.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:01.094 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:01.195 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:01.230 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:01.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:01.231 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:01.332 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:01.360 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:01.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:01.361 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:01.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:01.497 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:01.503 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:01.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:01.505 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:01.556 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:54:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:54:01] "GET /metrics HTTP/1.1" 200 224799 "" "Prometheus/2.43.0" 2024-09-11T08:54:01.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:01.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:01.776 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:01.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:01 smithi092 ceph-mon[29647]: 4.5 scrub starts 2024-09-11T08:54:01.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:01 smithi092 ceph-mon[29647]: 4.5 scrub ok 2024-09-11T08:54:01.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:01 smithi092 ceph-mon[29647]: 6.2 scrub starts 2024-09-11T08:54:01.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:01 smithi092 ceph-mon[29647]: 6.2 scrub ok 2024-09-11T08:54:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:01 smithi092 ceph-mon[41114]: 4.5 scrub starts 2024-09-11T08:54:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:01 smithi092 ceph-mon[41114]: 4.5 scrub ok 2024-09-11T08:54:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:01 smithi092 ceph-mon[41114]: 6.2 scrub starts 2024-09-11T08:54:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:01 smithi092 ceph-mon[41114]: 6.2 scrub ok 2024-09-11T08:54:01.855 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:54:01.877 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:01.896 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 57 16 99935 99919 109.525 78.0625 0.004981 0.00912612 2024-09-11T08:54:01.909 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:01.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:01.911 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:01.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:01 smithi145 ceph-mon[33384]: 4.5 scrub starts 2024-09-11T08:54:01.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:01 smithi145 ceph-mon[33384]: 4.5 scrub ok 2024-09-11T08:54:01.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:01 smithi145 ceph-mon[33384]: 6.2 scrub starts 2024-09-11T08:54:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:01 smithi145 ceph-mon[33384]: 6.2 scrub ok 2024-09-11T08:54:02.011 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:02.073 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:02.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:02.074 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:02.175 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:02.204 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:02.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:02.205 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:02.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:02.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:02.352 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:02.352 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:02.453 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:02.488 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:02.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:02.489 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:02.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:02.627 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:02.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:02.628 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:02.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:02.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:02.767 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:02.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:02 smithi092 ceph-mon[29647]: pgmap v693: 158 pgs: 158 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 701 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-11T08:54:02.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:02 smithi092 ceph-mon[29647]: 10.5 scrub starts 2024-09-11T08:54:02.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:02 smithi092 ceph-mon[29647]: 3.10 scrub starts 2024-09-11T08:54:02.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:02 smithi092 ceph-mon[29647]: 3.10 scrub ok 2024-09-11T08:54:02.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:02 smithi092 ceph-mon[41114]: pgmap v693: 158 pgs: 158 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 701 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-11T08:54:02.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:02 smithi092 ceph-mon[41114]: 10.5 scrub starts 2024-09-11T08:54:02.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:02 smithi092 ceph-mon[41114]: 3.10 scrub starts 2024-09-11T08:54:02.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:02 smithi092 ceph-mon[41114]: 3.10 scrub ok 2024-09-11T08:54:02.868 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:02.896 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 58 16 101087 101071 108.878 72 0.00948573 0.00918052 2024-09-11T08:54:02.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:02.905 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:02.905 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:02.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:54:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:54:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:54:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:02 smithi145 ceph-mon[33384]: pgmap v693: 158 pgs: 158 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 701 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-11T08:54:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:02 smithi145 ceph-mon[33384]: 10.5 scrub starts 2024-09-11T08:54:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:02 smithi145 ceph-mon[33384]: 3.10 scrub starts 2024-09-11T08:54:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:02 smithi145 ceph-mon[33384]: 3.10 scrub ok 2024-09-11T08:54:03.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:03.050 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:03.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:03.051 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:03.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:03.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:03.200 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:03.200 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:03.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:03.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:03.351 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:03.351 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:03.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:03.486 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:03.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:03.487 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:03.588 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:03.618 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:03.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:03.619 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:03.720 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:03.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:03.758 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:03.758 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:03.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:03 smithi092 ceph-mon[29647]: 3.17 scrub starts 2024-09-11T08:54:03.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:03 smithi092 ceph-mon[29647]: 3.17 scrub ok 2024-09-11T08:54:03.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:03 smithi092 ceph-mon[29647]: 5.14 scrub starts 2024-09-11T08:54:03.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:03 smithi092 ceph-mon[29647]: 5.14 scrub ok 2024-09-11T08:54:03.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:03 smithi092 ceph-mon[29647]: 3.19 scrub starts 2024-09-11T08:54:03.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:03 smithi092 ceph-mon[29647]: 3.19 scrub ok 2024-09-11T08:54:03.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:03 smithi092 ceph-mon[41114]: 3.17 scrub starts 2024-09-11T08:54:03.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:03 smithi092 ceph-mon[41114]: 3.17 scrub ok 2024-09-11T08:54:03.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:03 smithi092 ceph-mon[41114]: 5.14 scrub starts 2024-09-11T08:54:03.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:03 smithi092 ceph-mon[41114]: 5.14 scrub ok 2024-09-11T08:54:03.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:03 smithi092 ceph-mon[41114]: 3.19 scrub starts 2024-09-11T08:54:03.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:03 smithi092 ceph-mon[41114]: 3.19 scrub ok 2024-09-11T08:54:03.859 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:03.895 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:03.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:03.897 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:03.897 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 59 16 102231 102215 108.244 71.5 0.0148494 0.00923382 2024-09-11T08:54:03.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:03 smithi145 ceph-mon[33384]: 3.17 scrub starts 2024-09-11T08:54:03.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:03 smithi145 ceph-mon[33384]: 3.17 scrub ok 2024-09-11T08:54:03.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:03 smithi145 ceph-mon[33384]: 5.14 scrub starts 2024-09-11T08:54:03.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:03 smithi145 ceph-mon[33384]: 5.14 scrub ok 2024-09-11T08:54:03.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:03 smithi145 ceph-mon[33384]: 3.19 scrub starts 2024-09-11T08:54:03.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:03 smithi145 ceph-mon[33384]: 3.19 scrub ok 2024-09-11T08:54:03.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:04.037 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:04.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:04.038 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:04.139 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:04.172 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:04.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:04.173 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:04.274 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:04.306 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:04.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:04.307 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:04.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:04.419 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:54:04.419 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:54:04.419 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-11T08:54:04.419 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.528314 2024-09-11T08:54:04.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 2 0.5283137664180987 2024-09-11T08:54:04.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:04.446 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:04.446 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:04.547 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:04.581 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:04.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:04.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:04.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:04.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:04.741 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:04.742 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:04.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:04 smithi092 ceph-mon[41114]: pgmap v694: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.54k op/s; 13 B/s, 0 objects/s recovering 2024-09-11T08:54:04.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:04 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2794160796' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:54:04.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:04 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:54:04.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:04 smithi092 ceph-mon[29647]: pgmap v694: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.54k op/s; 13 B/s, 0 objects/s recovering 2024-09-11T08:54:04.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:04 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2794160796' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:54:04.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:04 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:54:04.835 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:54:04.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:04.881 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:04.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:04.882 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:04.897 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:54:04.898342+0000 min lat: 0.00119342 max lat: 0.169837 avg lat: 0.00928948 2024-09-11T08:54:04.897 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:54:04.897 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 60 16 103343 103327 107.599 69.5 0.00737524 0.00928948 2024-09-11T08:54:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:04 smithi145 ceph-mon[33384]: pgmap v694: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.54k op/s; 13 B/s, 0 objects/s recovering 2024-09-11T08:54:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:04 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2794160796' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:54:04.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:04 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:54:04.983 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:05.021 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:05.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:05.022 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:05.037 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:05.123 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:05.159 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:05.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:05.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:05.261 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:05.299 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:05.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:05.300 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:05.401 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:05.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:05.423 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:05.423 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:05.524 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:05.546 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:05.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:05.547 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:05.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:05.669 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:05.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:05.670 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:05.771 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:05.793 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:05.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:05.794 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:05.835 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:54:05.835 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:54:05.835 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:54:05.835 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[29647]: osdmap e317: 8 total, 8 up, 8 in 2024-09-11T08:54:05.835 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[29647]: 3.6 scrub starts 2024-09-11T08:54:05.835 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[29647]: 3.6 scrub ok 2024-09-11T08:54:05.835 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T08:54:05.835 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[29647]: 4.1c scrub ok 2024-09-11T08:54:05.836 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[29647]: 6.3 scrub starts 2024-09-11T08:54:05.836 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[29647]: 6.3 scrub ok 2024-09-11T08:54:05.836 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[29647]: 5.c scrub starts 2024-09-11T08:54:05.836 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[29647]: 5.c scrub ok 2024-09-11T08:54:05.836 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[29647]: 10.5 scrub ok 2024-09-11T08:54:05.836 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:54:05.836 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:54:05.836 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:54:05.836 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[41114]: osdmap e317: 8 total, 8 up, 8 in 2024-09-11T08:54:05.836 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[41114]: 3.6 scrub starts 2024-09-11T08:54:05.836 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[41114]: 3.6 scrub ok 2024-09-11T08:54:05.836 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T08:54:05.837 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[41114]: 4.1c scrub ok 2024-09-11T08:54:05.837 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[41114]: 6.3 scrub starts 2024-09-11T08:54:05.837 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[41114]: 6.3 scrub ok 2024-09-11T08:54:05.837 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[41114]: 5.c scrub starts 2024-09-11T08:54:05.837 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[41114]: 5.c scrub ok 2024-09-11T08:54:05.837 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:05 smithi092 ceph-mon[41114]: 10.5 scrub ok 2024-09-11T08:54:05.895 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:05.897 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 61 16 103960 103944 106.467 38.5625 0.00332367 0.00930656 2024-09-11T08:54:05.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:05 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:54:05.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:05 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:54:05.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:05 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:54:05.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:05 smithi145 ceph-mon[33384]: osdmap e317: 8 total, 8 up, 8 in 2024-09-11T08:54:05.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:05 smithi145 ceph-mon[33384]: 3.6 scrub starts 2024-09-11T08:54:05.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:05 smithi145 ceph-mon[33384]: 3.6 scrub ok 2024-09-11T08:54:05.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:05 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T08:54:05.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:05 smithi145 ceph-mon[33384]: 4.1c scrub ok 2024-09-11T08:54:05.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:05 smithi145 ceph-mon[33384]: 6.3 scrub starts 2024-09-11T08:54:05.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:05 smithi145 ceph-mon[33384]: 6.3 scrub ok 2024-09-11T08:54:05.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:05 smithi145 ceph-mon[33384]: 5.c scrub starts 2024-09-11T08:54:05.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:05 smithi145 ceph-mon[33384]: 5.c scrub ok 2024-09-11T08:54:05.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:05 smithi145 ceph-mon[33384]: 10.5 scrub ok 2024-09-11T08:54:05.917 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:05.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:05.918 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:06.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:06.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:06.061 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:06.061 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:06.162 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:06.197 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:06.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:06.198 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:06.299 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:06.337 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:06.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:06.338 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:06.388 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:54:06.388 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:54:06.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:06.473 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:06.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:06.475 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:06.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:06.615 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:06.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:06.616 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:06.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:06.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:06.754 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:06.754 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:06.855 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:06.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:06.890 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:06.890 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:06.897 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 62 16 105131 105115 105.93 73.1875 0.0118235 0.00943536 2024-09-11T08:54:06.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:06 smithi145 ceph-mon[33384]: pgmap v696: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 656 B/s rd, 49 MiB/s wr, 778 op/s; 14 B/s, 0 objects/s recovering 2024-09-11T08:54:06.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:06 smithi145 ceph-mon[33384]: 4.2 scrub starts 2024-09-11T08:54:06.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:06 smithi145 ceph-mon[33384]: 4.2 scrub ok 2024-09-11T08:54:06.991 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:07.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:07.028 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:07.028 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:07.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:06 smithi092 ceph-mon[29647]: pgmap v696: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 656 B/s rd, 49 MiB/s wr, 778 op/s; 14 B/s, 0 objects/s recovering 2024-09-11T08:54:07.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:06 smithi092 ceph-mon[29647]: 4.2 scrub starts 2024-09-11T08:54:07.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:06 smithi092 ceph-mon[29647]: 4.2 scrub ok 2024-09-11T08:54:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:06 smithi092 ceph-mon[41114]: pgmap v696: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 656 B/s rd, 49 MiB/s wr, 778 op/s; 14 B/s, 0 objects/s recovering 2024-09-11T08:54:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:06 smithi092 ceph-mon[41114]: 4.2 scrub starts 2024-09-11T08:54:07.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:06 smithi092 ceph-mon[41114]: 4.2 scrub ok 2024-09-11T08:54:07.129 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:07.165 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:07.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:07.166 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:07.266 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:07.300 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:07.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:07.301 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:07.402 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:07.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:54:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:54:07.435 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:07.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:07.437 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:07.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:07.580 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:07.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:07.581 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:07.602 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:54:07.682 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:07.747 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:07.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:07.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:07.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[29647]: 5.17 scrub starts 2024-09-11T08:54:07.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[29647]: 5.17 scrub ok 2024-09-11T08:54:07.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[29647]: 10.a scrub starts 2024-09-11T08:54:07.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[29647]: 5.a scrub starts 2024-09-11T08:54:07.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[29647]: 5.a scrub ok 2024-09-11T08:54:07.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:54:07.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:54:07.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[29647]: 4.d deep-scrub starts 2024-09-11T08:54:07.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[29647]: 4.d deep-scrub ok 2024-09-11T08:54:07.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[41114]: 5.17 scrub starts 2024-09-11T08:54:07.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[41114]: 5.17 scrub ok 2024-09-11T08:54:07.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[41114]: 10.a scrub starts 2024-09-11T08:54:07.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[41114]: 5.a scrub starts 2024-09-11T08:54:07.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[41114]: 5.a scrub ok 2024-09-11T08:54:07.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:54:07.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:54:07.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[41114]: 4.d deep-scrub starts 2024-09-11T08:54:07.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:07 smithi092 ceph-mon[41114]: 4.d deep-scrub ok 2024-09-11T08:54:07.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:07.870 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:07.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:07.871 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:07.897 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 63 16 106232 106216 105.341 68.8125 0.00366382 0.00947408 2024-09-11T08:54:07.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:07 smithi145 ceph-mon[33384]: 5.17 scrub starts 2024-09-11T08:54:07.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:07 smithi145 ceph-mon[33384]: 5.17 scrub ok 2024-09-11T08:54:07.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:07 smithi145 ceph-mon[33384]: 10.a scrub starts 2024-09-11T08:54:07.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:07 smithi145 ceph-mon[33384]: 5.a scrub starts 2024-09-11T08:54:07.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:07 smithi145 ceph-mon[33384]: 5.a scrub ok 2024-09-11T08:54:07.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:07 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:54:07.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:07 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:54:07.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:07 smithi145 ceph-mon[33384]: 4.d deep-scrub starts 2024-09-11T08:54:07.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:07 smithi145 ceph-mon[33384]: 4.d deep-scrub ok 2024-09-11T08:54:07.971 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:08.007 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:08.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:08.008 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:08.109 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:08.149 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:08.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:08.150 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:08.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:08.290 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:08.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:08.291 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:08.392 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:08.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:08.414 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:08.414 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:08.515 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:08.553 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:08.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:08.554 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:08.655 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:08.694 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:08.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:08.696 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:08.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:08.838 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:08.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:08.839 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:08.897 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 64 16 107325 107309 104.762 68.3125 0.0166741 0.00954117 2024-09-11T08:54:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:08 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:08 smithi145 ceph-mon[33384]: pgmap v697: 158 pgs: 158 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 80 MiB/s wr, 1.28k op/s; 12 B/s, 0 objects/s recovering 2024-09-11T08:54:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:08 smithi145 ceph-mon[33384]: 6.10 scrub starts 2024-09-11T08:54:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:08 smithi145 ceph-mon[33384]: 6.10 scrub ok 2024-09-11T08:54:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:08 smithi145 ceph-mon[33384]: osdmap e318: 8 total, 8 up, 8 in 2024-09-11T08:54:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:08 smithi145 ceph-mon[33384]: 10.8 scrub starts 2024-09-11T08:54:08.940 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:08.977 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:08.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:08.978 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:09.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:08 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:09.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:08 smithi092 ceph-mon[29647]: pgmap v697: 158 pgs: 158 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 80 MiB/s wr, 1.28k op/s; 12 B/s, 0 objects/s recovering 2024-09-11T08:54:09.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:08 smithi092 ceph-mon[29647]: 6.10 scrub starts 2024-09-11T08:54:09.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:08 smithi092 ceph-mon[29647]: 6.10 scrub ok 2024-09-11T08:54:09.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:08 smithi092 ceph-mon[29647]: osdmap e318: 8 total, 8 up, 8 in 2024-09-11T08:54:09.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:08 smithi092 ceph-mon[29647]: 10.8 scrub starts 2024-09-11T08:54:09.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:08 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:09.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:08 smithi092 ceph-mon[41114]: pgmap v697: 158 pgs: 158 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 80 MiB/s wr, 1.28k op/s; 12 B/s, 0 objects/s recovering 2024-09-11T08:54:09.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:08 smithi092 ceph-mon[41114]: 6.10 scrub starts 2024-09-11T08:54:09.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:08 smithi092 ceph-mon[41114]: 6.10 scrub ok 2024-09-11T08:54:09.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:08 smithi092 ceph-mon[41114]: osdmap e318: 8 total, 8 up, 8 in 2024-09-11T08:54:09.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:08 smithi092 ceph-mon[41114]: 10.8 scrub starts 2024-09-11T08:54:09.079 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:09.109 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:09.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:09.110 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:09.211 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:09.246 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:09.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:09.247 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:09.251 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:09.271 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-11T08:54:09.348 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:09.384 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:09.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:09.385 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:09.486 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:09.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:09.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:09.636 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:09.671 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:09.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:09.672 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:09.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:09.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:09.808 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:09.808 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:09.897 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 65 16 108435 108419 104.218 69.375 0.00808073 0.00959066 2024-09-11T08:54:09.909 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:09.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:09 smithi145 ceph-mon[33384]: 4.15 scrub starts 2024-09-11T08:54:09.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:09 smithi145 ceph-mon[33384]: 4.15 scrub ok 2024-09-11T08:54:09.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:09 smithi145 ceph-mon[33384]: 6.13 scrub starts 2024-09-11T08:54:09.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:09 smithi145 ceph-mon[33384]: 6.13 scrub ok 2024-09-11T08:54:09.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:09 smithi145 ceph-mon[33384]: 10.7 scrub starts 2024-09-11T08:54:09.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:09 smithi145 ceph-mon[33384]: 5.12 scrub starts 2024-09-11T08:54:09.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:09 smithi145 ceph-mon[33384]: 5.12 scrub ok 2024-09-11T08:54:09.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:09 smithi145 ceph-mon[33384]: 5.10 scrub starts 2024-09-11T08:54:09.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:09 smithi145 ceph-mon[33384]: 5.10 scrub ok 2024-09-11T08:54:09.932 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[29647]: 4.15 scrub starts 2024-09-11T08:54:09.932 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[29647]: 4.15 scrub ok 2024-09-11T08:54:09.932 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[29647]: 6.13 scrub starts 2024-09-11T08:54:09.932 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[29647]: 6.13 scrub ok 2024-09-11T08:54:09.932 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[29647]: 10.7 scrub starts 2024-09-11T08:54:09.932 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[29647]: 5.12 scrub starts 2024-09-11T08:54:09.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[29647]: 5.12 scrub ok 2024-09-11T08:54:09.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[29647]: 5.10 scrub starts 2024-09-11T08:54:09.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[29647]: 5.10 scrub ok 2024-09-11T08:54:09.933 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[41114]: 4.15 scrub starts 2024-09-11T08:54:09.933 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[41114]: 4.15 scrub ok 2024-09-11T08:54:09.933 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[41114]: 6.13 scrub starts 2024-09-11T08:54:09.933 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[41114]: 6.13 scrub ok 2024-09-11T08:54:09.933 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[41114]: 10.7 scrub starts 2024-09-11T08:54:09.933 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[41114]: 5.12 scrub starts 2024-09-11T08:54:09.933 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[41114]: 5.12 scrub ok 2024-09-11T08:54:09.933 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[41114]: 5.10 scrub starts 2024-09-11T08:54:09.933 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:09 smithi092 ceph-mon[41114]: 5.10 scrub ok 2024-09-11T08:54:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:09.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:09.958 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:10.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:10.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:10.095 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:10.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:10.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:10.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:10.236 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:10.236 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:10.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:10.371 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:10.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:10.372 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:10.473 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:10.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:10.512 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:10.512 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:10.613 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:10.654 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:10.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:10.656 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:10.757 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:10.794 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:10.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:10.795 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[29647]: 10.a scrub ok 2024-09-11T08:54:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[29647]: pgmap v699: 158 pgs: 158 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.21k op/s; 13 B/s, 0 objects/s recovering 2024-09-11T08:54:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[29647]: 1.1 scrub starts 2024-09-11T08:54:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[29647]: 1.1 scrub ok 2024-09-11T08:54:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[29647]: 4.16 scrub starts 2024-09-11T08:54:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[29647]: 4.16 scrub ok 2024-09-11T08:54:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[29647]: 4.1a scrub starts 2024-09-11T08:54:10.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[29647]: 4.1a scrub ok 2024-09-11T08:54:10.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[41114]: 10.a scrub ok 2024-09-11T08:54:10.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[41114]: pgmap v699: 158 pgs: 158 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.21k op/s; 13 B/s, 0 objects/s recovering 2024-09-11T08:54:10.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[41114]: 1.1 scrub starts 2024-09-11T08:54:10.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[41114]: 1.1 scrub ok 2024-09-11T08:54:10.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[41114]: 4.16 scrub starts 2024-09-11T08:54:10.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[41114]: 4.16 scrub ok 2024-09-11T08:54:10.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[41114]: 4.1a scrub starts 2024-09-11T08:54:10.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:10 smithi092 ceph-mon[41114]: 4.1a scrub ok 2024-09-11T08:54:10.896 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:10.897 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 66 16 109557 109541 103.701 70.125 0.010158 0.00963835 2024-09-11T08:54:10.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:10 smithi145 ceph-mon[33384]: 10.a scrub ok 2024-09-11T08:54:10.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:10 smithi145 ceph-mon[33384]: pgmap v699: 158 pgs: 158 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.21k op/s; 13 B/s, 0 objects/s recovering 2024-09-11T08:54:10.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:10 smithi145 ceph-mon[33384]: 1.1 scrub starts 2024-09-11T08:54:10.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:10 smithi145 ceph-mon[33384]: 1.1 scrub ok 2024-09-11T08:54:10.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:10 smithi145 ceph-mon[33384]: 4.16 scrub starts 2024-09-11T08:54:10.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:10 smithi145 ceph-mon[33384]: 4.16 scrub ok 2024-09-11T08:54:10.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:10 smithi145 ceph-mon[33384]: 4.1a scrub starts 2024-09-11T08:54:10.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:10 smithi145 ceph-mon[33384]: 4.1a scrub ok 2024-09-11T08:54:10.917 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:10.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:10.918 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:11.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:11.048 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:11.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:11.049 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:11.150 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:11.190 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:11.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:11.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:11.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:11.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:11.333 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:11.334 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:11.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:11.466 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:11.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:11.468 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:11.568 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:11.607 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:11.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:11.608 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:11.708 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:11.740 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:11.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:11.741 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:11.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:11 smithi092 ceph-mon[29647]: 3.1d scrub starts 2024-09-11T08:54:11.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:11 smithi092 ceph-mon[29647]: 3.1d scrub ok 2024-09-11T08:54:11.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:11 smithi092 ceph-mon[29647]: 10.d scrub starts 2024-09-11T08:54:11.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:11 smithi092 ceph-mon[29647]: 10.8 scrub ok 2024-09-11T08:54:11.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:11 smithi092 ceph-mon[29647]: 6.18 scrub starts 2024-09-11T08:54:11.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:11 smithi092 ceph-mon[29647]: 6.18 scrub ok 2024-09-11T08:54:11.759 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:11 smithi092 ceph-mon[41114]: 3.1d scrub starts 2024-09-11T08:54:11.759 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:11 smithi092 ceph-mon[41114]: 3.1d scrub ok 2024-09-11T08:54:11.759 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:11 smithi092 ceph-mon[41114]: 10.d scrub starts 2024-09-11T08:54:11.759 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:11 smithi092 ceph-mon[41114]: 10.8 scrub ok 2024-09-11T08:54:11.759 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:11 smithi092 ceph-mon[41114]: 6.18 scrub starts 2024-09-11T08:54:11.759 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:11 smithi092 ceph-mon[41114]: 6.18 scrub ok 2024-09-11T08:54:11.759 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:54:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:54:11] "GET /metrics HTTP/1.1" 200 224799 "" "Prometheus/2.43.0" 2024-09-11T08:54:11.841 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:11.886 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:11.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:11.887 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:11.898 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 67 16 110738 110722 103.255 73.8125 0.00528062 0.00967978 2024-09-11T08:54:11.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:11 smithi145 ceph-mon[33384]: 3.1d scrub starts 2024-09-11T08:54:11.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:11 smithi145 ceph-mon[33384]: 3.1d scrub ok 2024-09-11T08:54:11.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:11 smithi145 ceph-mon[33384]: 10.d scrub starts 2024-09-11T08:54:11.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:11 smithi145 ceph-mon[33384]: 10.8 scrub ok 2024-09-11T08:54:11.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:11 smithi145 ceph-mon[33384]: 6.18 scrub starts 2024-09-11T08:54:11.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:11 smithi145 ceph-mon[33384]: 6.18 scrub ok 2024-09-11T08:54:11.988 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:12.026 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:12.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:12.027 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:12.128 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:12.166 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:12.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:12.167 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:12.268 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:12.307 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:12.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:12.308 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:12.407 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:54:12.407 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:54:12.408 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:12.435 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:54:12.436 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:54:12.442 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:12.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:12.443 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:12.458 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:54:12.458 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:54:12.475 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:54:12.475 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:54:12.538 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:54:12.538 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:54:12.544 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:12.572 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:54:12.573 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:54:12.578 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:12.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:12.580 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:12.602 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:54:12.602 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:54:12.631 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:54:12.632 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:54:12.680 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:12.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:12.718 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:12.719 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[29647]: pgmap v700: 158 pgs: 158 active+clean; 6.5 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 52 MiB/s wr, 827 op/s 2024-09-11T08:54:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[29647]: 10.7 scrub ok 2024-09-11T08:54:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[29647]: 6.0 scrub starts 2024-09-11T08:54:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[29647]: 6.0 scrub ok 2024-09-11T08:54:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[29647]: 6.12 scrub starts 2024-09-11T08:54:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[29647]: 6.12 scrub ok 2024-09-11T08:54:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[29647]: 6.1d scrub starts 2024-09-11T08:54:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[29647]: 6.1d scrub ok 2024-09-11T08:54:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[29647]: 6.15 scrub starts 2024-09-11T08:54:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[29647]: 6.15 scrub ok 2024-09-11T08:54:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3457381107' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:54:12.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[41114]: pgmap v700: 158 pgs: 158 active+clean; 6.5 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 52 MiB/s wr, 827 op/s 2024-09-11T08:54:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[41114]: 10.7 scrub ok 2024-09-11T08:54:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[41114]: 6.0 scrub starts 2024-09-11T08:54:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[41114]: 6.0 scrub ok 2024-09-11T08:54:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[41114]: 6.12 scrub starts 2024-09-11T08:54:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[41114]: 6.12 scrub ok 2024-09-11T08:54:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[41114]: 6.1d scrub starts 2024-09-11T08:54:12.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[41114]: 6.1d scrub ok 2024-09-11T08:54:12.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[41114]: 6.15 scrub starts 2024-09-11T08:54:12.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[41114]: 6.15 scrub ok 2024-09-11T08:54:12.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:12 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3457381107' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:54:12.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:12.855 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:12.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:12.856 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:12.898 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 68 16 111879 111863 102.786 71.3125 0.00823017 0.00972462 2024-09-11T08:54:12.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:54:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:54:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:54:12.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:12 smithi145 ceph-mon[33384]: pgmap v700: 158 pgs: 158 active+clean; 6.5 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 52 MiB/s wr, 827 op/s 2024-09-11T08:54:12.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:12 smithi145 ceph-mon[33384]: 10.7 scrub ok 2024-09-11T08:54:12.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:12 smithi145 ceph-mon[33384]: 6.0 scrub starts 2024-09-11T08:54:12.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:12 smithi145 ceph-mon[33384]: 6.0 scrub ok 2024-09-11T08:54:12.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:12 smithi145 ceph-mon[33384]: 6.12 scrub starts 2024-09-11T08:54:12.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:12 smithi145 ceph-mon[33384]: 6.12 scrub ok 2024-09-11T08:54:12.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:12 smithi145 ceph-mon[33384]: 6.1d scrub starts 2024-09-11T08:54:12.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:12 smithi145 ceph-mon[33384]: 6.1d scrub ok 2024-09-11T08:54:12.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:12 smithi145 ceph-mon[33384]: 6.15 scrub starts 2024-09-11T08:54:12.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:12 smithi145 ceph-mon[33384]: 6.15 scrub ok 2024-09-11T08:54:12.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:12 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3457381107' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:54:12.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:13.008 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:13.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:13.009 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:13.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:13.145 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:13.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:13.146 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:13.209 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:13.210 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:13.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:13.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:13.294 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:13.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:13.431 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:13.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:13.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:13.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:13.558 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:13.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:13.559 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:13.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:13.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:13.682 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:13.682 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:13.783 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:13.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:13.823 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:13.824 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:13.898 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 69 16 112969 112953 102.283 68.125 0.0136825 0.00977221 2024-09-11T08:54:13.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:13 smithi145 ceph-mon[33384]: 4.19 scrub starts 2024-09-11T08:54:13.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:13 smithi145 ceph-mon[33384]: 4.19 scrub ok 2024-09-11T08:54:13.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:13 smithi145 ceph-mon[33384]: 10.b scrub starts 2024-09-11T08:54:13.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:13 smithi145 ceph-mon[33384]: 2.0 deep-scrub starts 2024-09-11T08:54:13.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:13 smithi145 ceph-mon[33384]: 2.0 deep-scrub ok 2024-09-11T08:54:13.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:13.962 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:13.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:13.963 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:14.064 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:13 smithi092 ceph-mon[29647]: 4.19 scrub starts 2024-09-11T08:54:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:13 smithi092 ceph-mon[29647]: 4.19 scrub ok 2024-09-11T08:54:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:13 smithi092 ceph-mon[29647]: 10.b scrub starts 2024-09-11T08:54:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:13 smithi092 ceph-mon[29647]: 2.0 deep-scrub starts 2024-09-11T08:54:14.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:13 smithi092 ceph-mon[29647]: 2.0 deep-scrub ok 2024-09-11T08:54:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:13 smithi092 ceph-mon[41114]: 4.19 scrub starts 2024-09-11T08:54:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:13 smithi092 ceph-mon[41114]: 4.19 scrub ok 2024-09-11T08:54:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:13 smithi092 ceph-mon[41114]: 10.b scrub starts 2024-09-11T08:54:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:13 smithi092 ceph-mon[41114]: 2.0 deep-scrub starts 2024-09-11T08:54:14.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:13 smithi092 ceph-mon[41114]: 2.0 deep-scrub ok 2024-09-11T08:54:14.099 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:14.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:14.100 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:14.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:14.228 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:14.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:14.229 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:14.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:14.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:14.365 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:14.466 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:14.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:14.512 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:14.512 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:14.613 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:14.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:14.654 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:14.654 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:14.755 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:14.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:14.798 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:14.798 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:14.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[29647]: pgmap v701: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 6.7 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 77 MiB/s wr, 1.23k op/s 2024-09-11T08:54:14.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[29647]: 3.18 scrub starts 2024-09-11T08:54:14.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[29647]: 5.9 scrub starts 2024-09-11T08:54:14.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[29647]: 5.9 scrub ok 2024-09-11T08:54:14.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[29647]: 10.d scrub ok 2024-09-11T08:54:14.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[29647]: 3.18 scrub ok 2024-09-11T08:54:14.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[29647]: 6.1a scrub starts 2024-09-11T08:54:14.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[29647]: 6.1a scrub ok 2024-09-11T08:54:14.807 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[41114]: pgmap v701: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 6.7 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 77 MiB/s wr, 1.23k op/s 2024-09-11T08:54:14.807 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[41114]: 3.18 scrub starts 2024-09-11T08:54:14.807 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[41114]: 5.9 scrub starts 2024-09-11T08:54:14.807 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[41114]: 5.9 scrub ok 2024-09-11T08:54:14.807 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[41114]: 10.d scrub ok 2024-09-11T08:54:14.807 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[41114]: 3.18 scrub ok 2024-09-11T08:54:14.807 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[41114]: 6.1a scrub starts 2024-09-11T08:54:14.807 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:14 smithi092 ceph-mon[41114]: 6.1a scrub ok 2024-09-11T08:54:14.898 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 70 16 114131 114115 101.86 72.625 0.0136449 0.00981255 2024-09-11T08:54:14.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:14.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:14 smithi145 ceph-mon[33384]: pgmap v701: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 6.7 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 77 MiB/s wr, 1.23k op/s 2024-09-11T08:54:14.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:14 smithi145 ceph-mon[33384]: 3.18 scrub starts 2024-09-11T08:54:14.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:14 smithi145 ceph-mon[33384]: 5.9 scrub starts 2024-09-11T08:54:14.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:14 smithi145 ceph-mon[33384]: 5.9 scrub ok 2024-09-11T08:54:14.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:14 smithi145 ceph-mon[33384]: 10.d scrub ok 2024-09-11T08:54:14.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:14 smithi145 ceph-mon[33384]: 3.18 scrub ok 2024-09-11T08:54:14.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:14 smithi145 ceph-mon[33384]: 6.1a scrub starts 2024-09-11T08:54:14.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:14 smithi145 ceph-mon[33384]: 6.1a scrub ok 2024-09-11T08:54:14.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:14.936 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:14.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:15.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:15.075 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:15.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:15.076 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:15.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:15.216 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:15.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:15.217 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:15.318 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:15.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:15.351 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:15.351 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:15.452 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:15.490 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:15.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:15.491 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:15.592 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:15.622 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:15.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:15.623 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:15.724 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:15.758 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:15.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:15.759 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:15.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[29647]: 2.2 scrub starts 2024-09-11T08:54:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[29647]: 2.2 scrub ok 2024-09-11T08:54:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[29647]: 4.6 scrub starts 2024-09-11T08:54:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[29647]: 4.6 scrub ok 2024-09-11T08:54:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[29647]: 6.8 scrub starts 2024-09-11T08:54:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[29647]: 6.8 scrub ok 2024-09-11T08:54:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:54:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:54:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[29647]: 3.13 deep-scrub starts 2024-09-11T08:54:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[29647]: 3.13 deep-scrub ok 2024-09-11T08:54:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1989322972' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.5283137664180987}]: dispatch 2024-09-11T08:54:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.5283137664180987}]: dispatch 2024-09-11T08:54:15.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[41114]: 2.2 scrub starts 2024-09-11T08:54:15.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[41114]: 2.2 scrub ok 2024-09-11T08:54:15.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[41114]: 4.6 scrub starts 2024-09-11T08:54:15.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[41114]: 4.6 scrub ok 2024-09-11T08:54:15.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[41114]: 6.8 scrub starts 2024-09-11T08:54:15.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[41114]: 6.8 scrub ok 2024-09-11T08:54:15.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:54:15.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:54:15.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[41114]: 3.13 deep-scrub starts 2024-09-11T08:54:15.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[41114]: 3.13 deep-scrub ok 2024-09-11T08:54:15.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1989322972' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.5283137664180987}]: dispatch 2024-09-11T08:54:15.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:15 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.5283137664180987}]: dispatch 2024-09-11T08:54:15.860 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:15.898 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 71 16 115356 115340 101.503 76.5625 0.0169029 0.00984746 2024-09-11T08:54:15.900 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:15.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:15.901 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:15.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:15 smithi145 ceph-mon[33384]: 2.2 scrub starts 2024-09-11T08:54:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:15 smithi145 ceph-mon[33384]: 2.2 scrub ok 2024-09-11T08:54:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:15 smithi145 ceph-mon[33384]: 4.6 scrub starts 2024-09-11T08:54:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:15 smithi145 ceph-mon[33384]: 4.6 scrub ok 2024-09-11T08:54:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:15 smithi145 ceph-mon[33384]: 6.8 scrub starts 2024-09-11T08:54:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:15 smithi145 ceph-mon[33384]: 6.8 scrub ok 2024-09-11T08:54:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:15 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:54:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:54:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:15 smithi145 ceph-mon[33384]: 3.13 deep-scrub starts 2024-09-11T08:54:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:15 smithi145 ceph-mon[33384]: 3.13 deep-scrub ok 2024-09-11T08:54:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:15 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1989322972' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.5283137664180987}]: dispatch 2024-09-11T08:54:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:15 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.5283137664180987}]: dispatch 2024-09-11T08:54:16.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:16.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:16.046 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:16.147 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:16.176 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:16.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:16.177 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:16.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:54:16.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:16.331 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:16.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:16.333 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:16.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:16.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:16.459 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:16.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:16.587 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:16.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:16.588 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:16.688 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:16.710 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:16.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:16.711 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:16.812 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:16.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:16 smithi092 ceph-mon[41114]: 10.b scrub ok 2024-09-11T08:54:16.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:16 smithi092 ceph-mon[41114]: pgmap v702: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 6.7 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 67 MiB/s wr, 1.08k op/s 2024-09-11T08:54:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:16 smithi092 ceph-mon[41114]: 4.13 scrub starts 2024-09-11T08:54:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:16 smithi092 ceph-mon[41114]: 4.13 scrub ok 2024-09-11T08:54:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:16 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.5283137664180987}]': finished 2024-09-11T08:54:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:16 smithi092 ceph-mon[41114]: osdmap e319: 8 total, 8 up, 8 in 2024-09-11T08:54:16.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:16 smithi092 ceph-mon[29647]: 10.b scrub ok 2024-09-11T08:54:16.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:16 smithi092 ceph-mon[29647]: pgmap v702: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 6.7 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 67 MiB/s wr, 1.08k op/s 2024-09-11T08:54:16.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:16 smithi092 ceph-mon[29647]: 4.13 scrub starts 2024-09-11T08:54:16.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:16 smithi092 ceph-mon[29647]: 4.13 scrub ok 2024-09-11T08:54:16.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:16 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.5283137664180987}]': finished 2024-09-11T08:54:16.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:16 smithi092 ceph-mon[29647]: osdmap e319: 8 total, 8 up, 8 in 2024-09-11T08:54:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:16.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:16.834 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:16.898 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 72 16 115699 115683 100.391 21.4375 0.00324745 0.00985476 2024-09-11T08:54:16.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:16 smithi145 ceph-mon[33384]: 10.b scrub ok 2024-09-11T08:54:16.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:16 smithi145 ceph-mon[33384]: pgmap v702: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 6.7 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 67 MiB/s wr, 1.08k op/s 2024-09-11T08:54:16.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:16 smithi145 ceph-mon[33384]: 4.13 scrub starts 2024-09-11T08:54:16.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:16 smithi145 ceph-mon[33384]: 4.13 scrub ok 2024-09-11T08:54:16.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:16 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.5283137664180987}]': finished 2024-09-11T08:54:16.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:16 smithi145 ceph-mon[33384]: osdmap e319: 8 total, 8 up, 8 in 2024-09-11T08:54:16.935 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:16.957 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:16.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:16.958 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:17.059 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:17.080 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:17.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:17.081 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:17.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:17.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:17.220 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:17.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:17.359 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:17.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:17.361 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:17.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:54:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:54:17.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:17.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:17.494 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:17.494 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:17.595 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:17.622 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:17.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:17.623 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:17.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:17.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:17.778 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:17.778 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:17.879 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:17.882 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:54:17.882 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:54:17.882 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:54:17.882 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:54:17.882 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:54:17.883 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:17 smithi092 ceph-mon[29647]: 5.7 deep-scrub starts 2024-09-11T08:54:17.883 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:17 smithi092 ceph-mon[29647]: 5.7 deep-scrub ok 2024-09-11T08:54:17.883 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:17 smithi092 ceph-mon[29647]: 2.1 scrub starts 2024-09-11T08:54:17.883 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:17 smithi092 ceph-mon[29647]: 2.1 scrub ok 2024-09-11T08:54:17.883 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:17 smithi092 ceph-mon[29647]: osdmap e320: 8 total, 8 up, 8 in 2024-09-11T08:54:17.883 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:17 smithi092 ceph-mon[41114]: 5.7 deep-scrub starts 2024-09-11T08:54:17.883 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:17 smithi092 ceph-mon[41114]: 5.7 deep-scrub ok 2024-09-11T08:54:17.883 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:17 smithi092 ceph-mon[41114]: 2.1 scrub starts 2024-09-11T08:54:17.884 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:17 smithi092 ceph-mon[41114]: 2.1 scrub ok 2024-09-11T08:54:17.884 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:17 smithi092 ceph-mon[41114]: osdmap e320: 8 total, 8 up, 8 in 2024-09-11T08:54:17.899 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 73 16 116660 116644 99.8376 60.0625 0.0105953 0.0100119 2024-09-11T08:54:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:17 smithi145 ceph-mon[33384]: 5.7 deep-scrub starts 2024-09-11T08:54:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:17 smithi145 ceph-mon[33384]: 5.7 deep-scrub ok 2024-09-11T08:54:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:17 smithi145 ceph-mon[33384]: 2.1 scrub starts 2024-09-11T08:54:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:17 smithi145 ceph-mon[33384]: 2.1 scrub ok 2024-09-11T08:54:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:17 smithi145 ceph-mon[33384]: osdmap e320: 8 total, 8 up, 8 in 2024-09-11T08:54:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:17.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:17.933 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:18.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:18.068 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:18.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:18.069 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:18.170 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:18.211 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:18.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:18.212 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:18.219 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.2 primary-affinity to 0.528314 (8346232) 2024-09-11T08:54:18.313 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:18.355 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:18.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:18.356 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:18.457 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:18.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:18.499 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:18.600 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:18.633 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:18.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:18.634 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:18.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:18.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:18.774 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:18.774 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:18.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:18.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[41114]: pgmap v705: 158 pgs: 6 peering, 152 active+clean; 7.1 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 87 MiB/s wr, 1.39k op/s 2024-09-11T08:54:18.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[41114]: 3.1c scrub starts 2024-09-11T08:54:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[41114]: 3.1c scrub ok 2024-09-11T08:54:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[41114]: 5.0 scrub starts 2024-09-11T08:54:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[41114]: 5.0 scrub ok 2024-09-11T08:54:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[41114]: 10.2 scrub starts 2024-09-11T08:54:18.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[41114]: Health check failed: Reduced data availability: 3 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-09-11T08:54:18.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:18.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[29647]: pgmap v705: 158 pgs: 6 peering, 152 active+clean; 7.1 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 87 MiB/s wr, 1.39k op/s 2024-09-11T08:54:18.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[29647]: 3.1c scrub starts 2024-09-11T08:54:18.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[29647]: 3.1c scrub ok 2024-09-11T08:54:18.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[29647]: 5.0 scrub starts 2024-09-11T08:54:18.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[29647]: 5.0 scrub ok 2024-09-11T08:54:18.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[29647]: 10.2 scrub starts 2024-09-11T08:54:18.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:18 smithi092 ceph-mon[29647]: Health check failed: Reduced data availability: 3 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-09-11T08:54:18.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:18.900 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 74 16 117743 117727 99.4019 67.6875 0.0151293 0.0100556 2024-09-11T08:54:18.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:18 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:18.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:18 smithi145 ceph-mon[33384]: pgmap v705: 158 pgs: 6 peering, 152 active+clean; 7.1 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 87 MiB/s wr, 1.39k op/s 2024-09-11T08:54:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:18 smithi145 ceph-mon[33384]: 3.1c scrub starts 2024-09-11T08:54:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:18 smithi145 ceph-mon[33384]: 3.1c scrub ok 2024-09-11T08:54:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:18 smithi145 ceph-mon[33384]: 5.0 scrub starts 2024-09-11T08:54:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:18 smithi145 ceph-mon[33384]: 5.0 scrub ok 2024-09-11T08:54:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:18 smithi145 ceph-mon[33384]: 10.2 scrub starts 2024-09-11T08:54:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:18 smithi145 ceph-mon[33384]: Health check failed: Reduced data availability: 3 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-09-11T08:54:18.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:18.925 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:18.925 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:19.026 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:19.055 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:19.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:19.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:19.157 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:19.190 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:19.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:19.191 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:19.292 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:19.331 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:19.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:19.332 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:19.433 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:19.465 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:19.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:19.466 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:19.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:19.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:19.601 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:19.601 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:19.663 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:54:19.702 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:19.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:19.736 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:19.837 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:19.877 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:19.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:19.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:19.900 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 75 16 118910 118894 99.049 72.9375 0.00883136 0.0100914 2024-09-11T08:54:19.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:19 smithi145 ceph-mon[33384]: 3.1f deep-scrub starts 2024-09-11T08:54:19.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:19 smithi145 ceph-mon[33384]: 3.1f deep-scrub ok 2024-09-11T08:54:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:19 smithi145 ceph-mon[33384]: 3.0 scrub starts 2024-09-11T08:54:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:19 smithi145 ceph-mon[33384]: 3.0 scrub ok 2024-09-11T08:54:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:19 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1848550338' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:54:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:19 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:54:19.979 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:20.013 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:20.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:20.014 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:20.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:19 smithi092 ceph-mon[29647]: 3.1f deep-scrub starts 2024-09-11T08:54:20.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:19 smithi092 ceph-mon[29647]: 3.1f deep-scrub ok 2024-09-11T08:54:20.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:19 smithi092 ceph-mon[29647]: 3.0 scrub starts 2024-09-11T08:54:20.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:19 smithi092 ceph-mon[29647]: 3.0 scrub ok 2024-09-11T08:54:20.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:19 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1848550338' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:54:20.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:19 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:54:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:19 smithi092 ceph-mon[41114]: 3.1f deep-scrub starts 2024-09-11T08:54:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:19 smithi092 ceph-mon[41114]: 3.1f deep-scrub ok 2024-09-11T08:54:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:19 smithi092 ceph-mon[41114]: 3.0 scrub starts 2024-09-11T08:54:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:19 smithi092 ceph-mon[41114]: 3.0 scrub ok 2024-09-11T08:54:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:19 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1848550338' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:54:20.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:19 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:54:20.115 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:20.151 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:20.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:20.153 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:20.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:20.295 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:20.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:20.296 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:20.397 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:20.432 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:20.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:20.433 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:20.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:20.577 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:20.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:20.579 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:20.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:20.716 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:20.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:20.717 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:20.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:20 smithi092 ceph-mon[29647]: pgmap v706: 158 pgs: 6 peering, 152 active+clean; 7.1 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 76 MiB/s wr, 1.21k op/s 2024-09-11T08:54:20.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:20 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:54:20.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:20 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:54:20.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:20 smithi092 ceph-mon[29647]: osdmap e321: 8 total, 8 up, 8 in 2024-09-11T08:54:20.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:20 smithi092 ceph-mon[41114]: pgmap v706: 158 pgs: 6 peering, 152 active+clean; 7.1 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 76 MiB/s wr, 1.21k op/s 2024-09-11T08:54:20.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:20 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:54:20.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:20 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:54:20.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:20 smithi092 ceph-mon[41114]: osdmap e321: 8 total, 8 up, 8 in 2024-09-11T08:54:20.817 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:20.859 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:20.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:20.860 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:20.900 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 76 16 120232 120216 98.8328 82.625 0.00979928 0.0101138 2024-09-11T08:54:20.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:20 smithi145 ceph-mon[33384]: pgmap v706: 158 pgs: 6 peering, 152 active+clean; 7.1 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 76 MiB/s wr, 1.21k op/s 2024-09-11T08:54:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:20 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:54:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:20 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:54:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:20 smithi145 ceph-mon[33384]: osdmap e321: 8 total, 8 up, 8 in 2024-09-11T08:54:20.961 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:20.997 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:20.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:20.998 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:21.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:21.132 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:21.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:21.133 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:21.234 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:21.272 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:21.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:21.273 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:21.374 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:21.410 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:21.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:21.411 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:21.512 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:21.546 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:21.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:21.548 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:21.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:21.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:21.688 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:21.733 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:21 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:54:21.733 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:21 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:54:21.734 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:54:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:54:21] "GET /metrics HTTP/1.1" 200 224823 "" "Prometheus/2.43.0" 2024-09-11T08:54:21.734 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:21 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:54:21.734 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:21 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:54:21.789 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:21.830 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:21.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:21.831 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:21.900 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 77 16 121546 121530 98.6158 82.125 0.00471851 0.010136 2024-09-11T08:54:21.932 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:21.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:21.966 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:21.966 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:22.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:22.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:22.099 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:22.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:21 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:54:22.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:21 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:54:22.200 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:22.232 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:22.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:22.233 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:22.271 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-11T08:54:22.298 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:22.334 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:22.365 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:22.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:22.366 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:22.467 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:22.497 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:22.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:22.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:22.599 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:22.640 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:22.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:22.641 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:22.742 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:22.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:22.775 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:22.775 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:22.814 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:22 smithi092 ceph-mon[29647]: pgmap v708: 158 pgs: 4 peering, 154 active+clean; 7.2 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 682 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-11T08:54:22.814 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:22 smithi092 ceph-mon[29647]: osdmap e322: 8 total, 8 up, 8 in 2024-09-11T08:54:22.814 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:22 smithi092 ceph-mon[41114]: pgmap v708: 158 pgs: 4 peering, 154 active+clean; 7.2 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 682 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-11T08:54:22.814 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:22 smithi092 ceph-mon[41114]: osdmap e322: 8 total, 8 up, 8 in 2024-09-11T08:54:22.877 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:22.900 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 78 16 122803 122787 98.3587 78.5625 0.00595271 0.0101625 2024-09-11T08:54:22.914 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:22.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:22.915 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:23.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:23.052 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:23.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:23.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:23.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:23.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:54:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:54:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:54:23.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:22 smithi145 ceph-mon[33384]: pgmap v708: 158 pgs: 4 peering, 154 active+clean; 7.2 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 682 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-11T08:54:23.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:22 smithi145 ceph-mon[33384]: osdmap e322: 8 total, 8 up, 8 in 2024-09-11T08:54:23.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:23.195 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:23.195 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:23.296 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:23.326 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:23.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:23.327 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:23.428 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:23.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:23.467 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:23.467 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:23.568 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:23.603 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:23.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:23.607 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:23.708 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:23.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:23.746 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:23.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:23.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:54:23.777 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:23.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:23.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:23.891 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:23.891 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:23.901 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 79 16 124088 124072 98.1297 80.3125 0.00967722 0.0101867 2024-09-11T08:54:23.915 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:23 smithi092 ceph-mon[29647]: pgmap v710: 158 pgs: 158 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 823 B/s rd, 53 MiB/s wr, 841 op/s; 10 KiB/s, 0 objects/s recovering 2024-09-11T08:54:23.915 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:23 smithi092 ceph-mon[29647]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-11T08:54:23.915 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:23 smithi092 ceph-mon[41114]: pgmap v710: 158 pgs: 158 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 823 B/s rd, 53 MiB/s wr, 841 op/s; 10 KiB/s, 0 objects/s recovering 2024-09-11T08:54:23.915 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:23 smithi092 ceph-mon[41114]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-11T08:54:23.992 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:24.024 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:24.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:24.025 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:24.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:24.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:23 smithi145 ceph-mon[33384]: pgmap v710: 158 pgs: 158 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 823 B/s rd, 53 MiB/s wr, 841 op/s; 10 KiB/s, 0 objects/s recovering 2024-09-11T08:54:24.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:23 smithi145 ceph-mon[33384]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-11T08:54:24.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:24.167 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:24.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:24.268 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:24.307 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:24.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:24.309 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:24.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:24.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:24.444 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:24.444 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:24.545 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:24.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:24.581 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:24.581 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:24.682 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:24.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:24.719 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:24.719 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:24.821 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:24.853 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:24.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:24.854 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:24.901 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:54:24.902560+0000 min lat: 0.00119342 max lat: 1.03621 avg lat: 0.0102133 2024-09-11T08:54:24.901 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:54:24.901 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 80 16 125326 125310 97.8702 77.375 0.00681358 0.0102133 2024-09-11T08:54:24.955 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:24.987 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:24.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:24.988 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:25.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:25.124 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:25.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:25.125 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:25.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:25.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:25.248 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:25.248 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:25.349 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:25.390 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:25.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:25.391 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:25.481 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:54:25.481 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:54:25.481 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:54:25.481 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:54:25.481 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:54:25.481 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:54:25.481 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:54:25.481 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:54:25.482 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:54:25.482 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:54:25.482 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:54:25.482 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:54:25.492 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:25.495 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:54:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:54:25.526 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:25.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:25.527 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:25.527 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:54:25.528 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:54:25.582 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:54:25.582 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:54:25.583 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_3 2024-09-11T08:54:25.583 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_3 by 10 min 8 2024-09-11T08:54:25.583 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:54:25.628 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:25.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:25.663 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:25.663 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:25.764 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:25.800 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:25.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:25.801 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:25.901 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 81 16 126265 126249 97.3865 58.6875 0.00701663 0.010264 2024-09-11T08:54:25.902 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:25.937 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:25 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:54:25.937 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:25 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:54:25.937 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:25 smithi092 ceph-mon[29647]: osdmap e323: 8 total, 8 up, 8 in 2024-09-11T08:54:25.937 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:25 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4204791171' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:54:25.937 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:25 smithi092 ceph-mon[29647]: pgmap v712: 158 pgs: 158 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 852 B/s rd, 54 MiB/s wr, 871 op/s; 11 KiB/s, 0 objects/s recovering 2024-09-11T08:54:25.937 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:25 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:54:25.937 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:25 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:54:25.938 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:25 smithi092 ceph-mon[41114]: osdmap e323: 8 total, 8 up, 8 in 2024-09-11T08:54:25.938 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:25 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4204791171' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:54:25.938 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:25 smithi092 ceph-mon[41114]: pgmap v712: 158 pgs: 158 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 852 B/s rd, 54 MiB/s wr, 871 op/s; 11 KiB/s, 0 objects/s recovering 2024-09-11T08:54:25.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:25.942 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:25.942 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:26.043 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:26.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:26.080 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:26.081 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:26.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:25 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:54:26.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:25 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:54:26.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:25 smithi145 ceph-mon[33384]: osdmap e323: 8 total, 8 up, 8 in 2024-09-11T08:54:26.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:25 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4204791171' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:54:26.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:25 smithi145 ceph-mon[33384]: pgmap v712: 158 pgs: 158 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 852 B/s rd, 54 MiB/s wr, 871 op/s; 11 KiB/s, 0 objects/s recovering 2024-09-11T08:54:26.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:26.215 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:26.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:26.217 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:26.317 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:26.352 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:26.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:26.353 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:26.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:26.487 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:26.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:26.488 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:26.589 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:26.623 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:26.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:26.624 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:26.725 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:26.780 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:26.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:26.781 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:26.882 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:26.903 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 82 15 127570 127555 97.192 81.625 0.0170462 0.0102845 2024-09-11T08:54:26.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:26.916 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:26.916 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:27.017 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:27.058 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:27.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:27.059 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:27.160 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:27.196 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:27.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:27.198 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:27.198 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:27.299 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:27.336 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:27.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:27.337 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:27.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:54:26 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:54:27.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:27.477 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:27.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:27.478 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:27.579 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:27.614 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:27.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:27.615 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:27.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:27.754 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:27.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:27.755 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:27.775 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:27 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:54:27.775 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:27 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:54:27.775 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:27 smithi092 ceph-mon[29647]: osdmap e324: 8 total, 8 up, 8 in 2024-09-11T08:54:27.775 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:27 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:27.775 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:27 smithi092 ceph-mon[29647]: pgmap v714: 158 pgs: 158 active+clean; 7.7 GiB data, 29 GiB used, 687 GiB / 715 GiB avail; 1.7 KiB/s rd, 101 MiB/s wr, 1.62k op/s; 11 KiB/s, 0 objects/s recovering 2024-09-11T08:54:27.776 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:27 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:54:27.776 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:27 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:54:27.776 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:27 smithi092 ceph-mon[41114]: osdmap e324: 8 total, 8 up, 8 in 2024-09-11T08:54:27.776 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:27 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:27.776 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:27 smithi092 ceph-mon[41114]: pgmap v714: 158 pgs: 158 active+clean; 7.7 GiB data, 29 GiB used, 687 GiB / 715 GiB avail; 1.7 KiB/s rd, 101 MiB/s wr, 1.62k op/s; 11 KiB/s, 0 objects/s recovering 2024-09-11T08:54:27.855 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:27.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:27.894 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:27.895 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:27.903 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 83 15 128858 128843 96.9908 80.5 0.0158546 0.0103061 2024-09-11T08:54:27.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:28.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:28.032 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:28.032 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:28.073 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:54:28.073 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:54:28.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:28.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:27 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:54:28.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:27 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:54:28.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:27 smithi145 ceph-mon[33384]: osdmap e324: 8 total, 8 up, 8 in 2024-09-11T08:54:28.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:27 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:28.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:27 smithi145 ceph-mon[33384]: pgmap v714: 158 pgs: 158 active+clean; 7.7 GiB data, 29 GiB used, 687 GiB / 715 GiB avail; 1.7 KiB/s rd, 101 MiB/s wr, 1.62k op/s; 11 KiB/s, 0 objects/s recovering 2024-09-11T08:54:28.166 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:28.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:28.167 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:28.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:28.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:28.308 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:28.308 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:28.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:28.442 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:28.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:28.443 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:28.543 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:28.575 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:28.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:28.577 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:28.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:28.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:28.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:28.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:28.859 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:28.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:28.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:28.903 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 84 16 130132 130116 96.7833 79.5625 0.0115892 0.0103282 2024-09-11T08:54:28.961 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:28.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:28.998 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:28.998 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:29.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:29.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:29.132 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:29.132 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:29.233 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:29.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:54:29.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:29.266 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:29.266 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:29.367 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:29.415 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:29.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:29.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:29.517 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:29.592 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:29.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:29.593 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:29.694 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:29.729 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:29.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:29.730 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:29.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:29.835 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:29.864 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:29.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:29.866 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:29.904 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 85 15 131388 131373 96.5677 78.5625 0.0119376 0.0103512 2024-09-11T08:54:29.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:29.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:29.999 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:29.999 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:30.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:30.138 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:30.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:30.139 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:30.240 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:30.276 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:30.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:30.277 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:30.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:30 smithi092 ceph-mon[29647]: pgmap v715: 158 pgs: 158 active+clean; 7.7 GiB data, 29 GiB used, 687 GiB / 715 GiB avail; 1.3 KiB/s rd, 56 MiB/s wr, 890 op/s 2024-09-11T08:54:30.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:30 smithi092 ceph-mon[41114]: pgmap v715: 158 pgs: 158 active+clean; 7.7 GiB data, 29 GiB used, 687 GiB / 715 GiB avail; 1.3 KiB/s rd, 56 MiB/s wr, 890 op/s 2024-09-11T08:54:30.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:30.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:30 smithi145 ceph-mon[33384]: pgmap v715: 158 pgs: 158 active+clean; 7.7 GiB data, 29 GiB used, 687 GiB / 715 GiB avail; 1.3 KiB/s rd, 56 MiB/s wr, 890 op/s 2024-09-11T08:54:30.414 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:30.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:30.415 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:30.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-11T08:54:30.516 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:30.554 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:30.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:30.555 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:30.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:30.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:30.697 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:30.697 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:30.798 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:30.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:30.834 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:30.834 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:30.904 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 86 16 132652 132636 96.3626 78.9375 0.014312 0.0103729 2024-09-11T08:54:30.935 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:30.973 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:30.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:30.974 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:31.075 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:31.110 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:31.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:31.111 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:31.212 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:31.251 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:31.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:31.252 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:31.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:31 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:54:31.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:31 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:54:31.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:31 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:54:31.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:31 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:54:31.353 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:31.393 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:31.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:31.394 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:31.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:31 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:54:31.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:31 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:54:31.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:31.540 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:31.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:31.541 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:31.642 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:31.673 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:31.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:31.674 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:31.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:31.797 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:31.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:31.798 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:31.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:54:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:54:31] "GET /metrics HTTP/1.1" 200 224833 "" "Prometheus/2.43.0" 2024-09-11T08:54:31.899 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:31.905 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 87 16 133875 133859 96.1335 76.4375 0.00896103 0.0103979 2024-09-11T08:54:31.937 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:31.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:31.938 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:32.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:32.076 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:32.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:32.077 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:32.178 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:32.199 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:32.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:32.200 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:32.301 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:32.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:32.374 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:32.374 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:32 smithi145 ceph-mon[33384]: pgmap v716: 158 pgs: 158 active+clean; 7.9 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 639 B/s rd, 63 MiB/s wr, 1.00k op/s 2024-09-11T08:54:32.476 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:32.497 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:32.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:32.498 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:32 smithi092 ceph-mon[29647]: pgmap v716: 158 pgs: 158 active+clean; 7.9 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 639 B/s rd, 63 MiB/s wr, 1.00k op/s 2024-09-11T08:54:32.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:32 smithi092 ceph-mon[41114]: pgmap v716: 158 pgs: 158 active+clean; 7.9 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 639 B/s rd, 63 MiB/s wr, 1.00k op/s 2024-09-11T08:54:32.599 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:32.636 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:32.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:32.637 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:32.738 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:32.759 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:32.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:32.760 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:32.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:32.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:32.901 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:32.901 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:32.905 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 88 16 134679 134663 95.6122 50.25 0.0119435 0.0104547 2024-09-11T08:54:32.990 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:33.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:33.049 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:33.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:33.050 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:33.105 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:54:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:54:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:54:33.129 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:54:33.151 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:33.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:33.201 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:33.201 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:33.302 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:33.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:33 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3833717908' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:54:33.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:33 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:54:33.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:33 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3833717908' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:54:33.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:33 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:54:33.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:33.340 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:33.340 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:33.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:33 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3833717908' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:54:33.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:33 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:54:33.441 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:33.479 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:33.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:33.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:33.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:33.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:33.636 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:33.737 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:33.768 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:33.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:33.769 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:33.870 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:33.905 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 89 16 135748 135732 95.2886 66.8125 0.00365983 0.0104899 2024-09-11T08:54:33.911 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:33.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:33.912 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:34.013 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:34.055 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:34.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:34.056 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:34.138 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:34 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:54:34.138 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:34 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:54:34.138 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:34 smithi092 ceph-mon[29647]: osdmap e325: 8 total, 8 up, 8 in 2024-09-11T08:54:34.138 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:34 smithi092 ceph-mon[29647]: pgmap v718: 158 pgs: 158 active+clean; 8.1 GiB data, 30 GiB used, 685 GiB / 715 GiB avail; 1.2 KiB/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-11T08:54:34.139 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:34 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:54:34.139 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:34 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:54:34.139 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:34 smithi092 ceph-mon[41114]: osdmap e325: 8 total, 8 up, 8 in 2024-09-11T08:54:34.139 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:34 smithi092 ceph-mon[41114]: pgmap v718: 158 pgs: 158 active+clean; 8.1 GiB data, 30 GiB used, 685 GiB / 715 GiB avail; 1.2 KiB/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-11T08:54:34.156 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:34.193 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:34.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:34.194 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:34.295 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:34.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:34.328 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:34.329 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:34 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:54:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:34 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:54:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:34 smithi145 ceph-mon[33384]: osdmap e325: 8 total, 8 up, 8 in 2024-09-11T08:54:34.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:34 smithi145 ceph-mon[33384]: pgmap v718: 158 pgs: 158 active+clean; 8.1 GiB data, 30 GiB used, 685 GiB / 715 GiB avail; 1.2 KiB/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-11T08:54:34.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:34.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:34.468 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:34.468 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:34.569 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:34.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:34.602 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:34.603 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:34.703 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:34.738 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:34.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:34.739 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:34.840 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:34.873 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:34.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:34.874 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:34.905 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Total time run: 90.0045 2024-09-11T08:54:34.905 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Total writes made: 136956 2024-09-11T08:54:34.905 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Write size: 65536 2024-09-11T08:54:34.905 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Object size: 65536 2024-09-11T08:54:34.905 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Bandwidth (MB/sec): 95.1036 2024-09-11T08:54:34.905 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Stddev Bandwidth: 38.5805 2024-09-11T08:54:34.906 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Max bandwidth (MB/sec): 254.375 2024-09-11T08:54:34.906 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Min bandwidth (MB/sec): 21.4375 2024-09-11T08:54:34.906 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Average IOPS: 1521 2024-09-11T08:54:34.906 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Stddev IOPS: 617.288 2024-09-11T08:54:34.906 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Max IOPS: 4070 2024-09-11T08:54:34.906 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Min IOPS: 343 2024-09-11T08:54:34.906 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Average Latency(s): 0.0105112 2024-09-11T08:54:34.906 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Stddev Latency(s): 0.0150013 2024-09-11T08:54:34.906 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Max latency(s): 1.03621 2024-09-11T08:54:34.906 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Min latency(s): 0.00119342 2024-09-11T08:54:34.909 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Cleaning up (deleting benchmark objects) 2024-09-11T08:54:34.975 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:35.012 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:35.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:35.018 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:35.118 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:35.153 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:35.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:35.154 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:35.255 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:35.288 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:35.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:35.290 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:35.391 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:35.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:35.429 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:35.429 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:35.530 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:35.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:35.571 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:35.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:35.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:35.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:35.707 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:54:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:54:35.808 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:35.845 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:35.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:35.846 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:35.947 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:35.981 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:35.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:35.982 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:36.082 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:36.121 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:36.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:36.122 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:36.223 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:36.253 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:36.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:36.254 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:36.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:36.395 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:36.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:36.397 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:36.498 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:36.529 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:36.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:36.530 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:36 smithi092 ceph-mon[29647]: pgmap v719: 158 pgs: 158 active+clean; 8.1 GiB data, 30 GiB used, 685 GiB / 715 GiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 831 op/s 2024-09-11T08:54:36.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:36 smithi092 ceph-mon[29647]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:54:36.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:36 smithi092 ceph-mon[41114]: pgmap v719: 158 pgs: 158 active+clean; 8.1 GiB data, 30 GiB used, 685 GiB / 715 GiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 831 op/s 2024-09-11T08:54:36.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:36 smithi092 ceph-mon[41114]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:54:36.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:36.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:36 smithi145 ceph-mon[33384]: pgmap v719: 158 pgs: 158 active+clean; 8.1 GiB data, 30 GiB used, 685 GiB / 715 GiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 831 op/s 2024-09-11T08:54:36.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:36 smithi145 ceph-mon[33384]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:54:36.674 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:36.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:36.675 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:36.776 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:36.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:36.812 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:36.812 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:36.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:36.934 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:36.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:36.935 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:37.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:37.069 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:37.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:37.070 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:37.171 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:37.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:37.207 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:37.207 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:37.308 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:37.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:37.347 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:37.348 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:37.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:54:36 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:54:37.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:37.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:37.481 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:37.582 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:37.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:37.616 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:37.616 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:37.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:37.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:37.750 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:37.750 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:37.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:37.886 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:37.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:37.887 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:37.979 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":719,"stamp":"2024-09-11T08:54:35.338207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8736193576,"num_objects":133490,"num_object_clones":0,"num_object_copies":400470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133490,"num_whiteouts":0,"num_read":1269,"num_read_kb":1040,"num_write":134030,"num_write_kb":8531091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":284865,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84433,"ondisk_log_size":84433,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":31471204,"kb_used_data":25500900,"kb_used_omap":13,"kb_used_meta":5970226,"kb_avail":718490012,"statfs":{"total":767960285184,"available":735733772288,"internally_reserved":0,"allocated":26112921600,"data_stored":26110267060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":6113512128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":467206144,"num_objects":7129,"num_object_clones":0,"num_object_copies":21387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7129,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":7129,"num_write_kb":456256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.587933"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":142,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.736913+0000","last_change":"2024-09-11T08:54:18.331227+0000","last_active":"2024-09-11T08:54:24.736913+0000","last_peered":"2024-09-11T08:54:24.736913+0000","last_clean":"2024-09-11T08:54:24.736913+0000","last_became_active":"2024-09-11T08:52:49.568416+0000","last_became_peered":"2024-09-11T08:52:49.568416+0000","last_unstale":"2024-09-11T08:54:24.736913+0000","last_undegraded":"2024-09-11T08:54:24.736913+0000","last_fullsized":"2024-09-11T08:54:24.736913+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:18.331155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:18.331155+0000","last_clean_scrub_stamp":"2024-09-11T08:54:18.331155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:30.540212+0000","scrub_duration":0.0042788669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"200'19","reported_seq":394,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734754+0000","last_change":"2024-09-11T08:52:30.312273+0000","last_active":"2024-09-11T08:54:24.734754+0000","last_peered":"2024-09-11T08:54:24.734754+0000","last_clean":"2024-09-11T08:54:24.734754+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:54:24.734754+0000","last_undegraded":"2024-09-11T08:54:24.734754+0000","last_fullsized":"2024-09-11T08:54:24.734754+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:52:30.312197+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:52:30.312197+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002912516,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":342,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.750874+0000","last_change":"2024-09-11T08:53:54.853514+0000","last_active":"2024-09-11T08:54:24.750874+0000","last_peered":"2024-09-11T08:54:24.750874+0000","last_clean":"2024-09-11T08:54:24.750874+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:54:24.750874+0000","last_undegraded":"2024-09-11T08:54:24.750874+0000","last_fullsized":"2024-09-11T08:54:24.750874+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:54.853421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:54.853421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:22.460919+0000","scrub_duration":0.0077955339999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":321,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734140+0000","last_change":"2024-09-11T08:54:13.811462+0000","last_active":"2024-09-11T08:54:24.734140+0000","last_peered":"2024-09-11T08:54:24.734140+0000","last_clean":"2024-09-11T08:54:24.734140+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:54:24.734140+0000","last_undegraded":"2024-09-11T08:54:24.734140+0000","last_fullsized":"2024-09-11T08:54:24.734140+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:13.811054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:13.811054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:28.408617+0000","scrub_duration":0.010996769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":340,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742407+0000","last_change":"2024-09-11T08:51:35.370494+0000","last_active":"2024-09-11T08:54:24.742407+0000","last_peered":"2024-09-11T08:54:24.742407+0000","last_clean":"2024-09-11T08:54:24.742407+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:54:24.742407+0000","last_undegraded":"2024-09-11T08:54:24.742407+0000","last_fullsized":"2024-09-11T08:54:24.742407+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:35.370438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:51:35.370438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053884329999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":346,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.743603+0000","last_change":"2024-09-11T08:51:28.347060+0000","last_active":"2024-09-11T08:54:24.743603+0000","last_peered":"2024-09-11T08:54:24.743603+0000","last_clean":"2024-09-11T08:54:24.743603+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:54:24.743603+0000","last_undegraded":"2024-09-11T08:54:24.743603+0000","last_fullsized":"2024-09-11T08:54:24.743603+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:28.346979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:51:28.346979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020416560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"200'34","reported_seq":436,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742331+0000","last_change":"2024-09-11T08:54:12.545864+0000","last_active":"2024-09-11T08:54:24.742331+0000","last_peered":"2024-09-11T08:54:24.742331+0000","last_clean":"2024-09-11T08:54:24.742331+0000","last_became_active":"2024-09-11T08:52:18.585600+0000","last_became_peered":"2024-09-11T08:52:18.585600+0000","last_unstale":"2024-09-11T08:54:24.742331+0000","last_undegraded":"2024-09-11T08:54:24.742331+0000","last_fullsized":"2024-09-11T08:54:24.742331+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:54:12.545773+0000","last_deep_scrub":"200'34","last_deep_scrub_stamp":"2024-09-11T08:51:31.358660+0000","last_clean_scrub_stamp":"2024-09-11T08:54:12.545773+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:18.086513+0000","scrub_duration":0.0070952350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":313,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734685+0000","last_change":"2024-09-11T08:51:19.004758+0000","last_active":"2024-09-11T08:54:24.734685+0000","last_peered":"2024-09-11T08:54:24.734685+0000","last_clean":"2024-09-11T08:54:24.734685+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:54:24.734685+0000","last_undegraded":"2024-09-11T08:54:24.734685+0000","last_fullsized":"2024-09-11T08:54:24.734685+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:19.004676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:51:19.004676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006451526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":362,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.736115+0000","last_change":"2024-09-11T08:54:10.466581+0000","last_active":"2024-09-11T08:54:24.736115+0000","last_peered":"2024-09-11T08:54:24.736115+0000","last_clean":"2024-09-11T08:54:24.736115+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:54:24.736115+0000","last_undegraded":"2024-09-11T08:54:24.736115+0000","last_fullsized":"2024-09-11T08:54:24.736115+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:10.466478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:54:10.466478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:36.076266+0000","scrub_duration":0.0034798429999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"200'19","reported_seq":404,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734402+0000","last_change":"2024-09-11T08:54:09.865650+0000","last_active":"2024-09-11T08:54:24.734402+0000","last_peered":"2024-09-11T08:54:24.734402+0000","last_clean":"2024-09-11T08:54:24.734402+0000","last_became_active":"2024-09-11T08:52:18.577411+0000","last_became_peered":"2024-09-11T08:52:18.577411+0000","last_unstale":"2024-09-11T08:54:24.734402+0000","last_undegraded":"2024-09-11T08:54:24.734402+0000","last_fullsized":"2024-09-11T08:54:24.734402+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:54:09.865565+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.865565+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:31.834039+0000","scrub_duration":0.0044543600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":358,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.739968+0000","last_change":"2024-09-11T08:53:59.484790+0000","last_active":"2024-09-11T08:54:24.739968+0000","last_peered":"2024-09-11T08:54:24.739968+0000","last_clean":"2024-09-11T08:54:24.739968+0000","last_became_active":"2024-09-11T08:52:18.573461+0000","last_became_peered":"2024-09-11T08:52:18.573461+0000","last_unstale":"2024-09-11T08:54:24.739968+0000","last_undegraded":"2024-09-11T08:54:24.739968+0000","last_fullsized":"2024-09-11T08:54:24.739968+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:59.484561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:59.484561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:05.181267+0000","scrub_duration":0.0060038859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":142,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.752041+0000","last_change":"2024-09-11T08:54:10.819941+0000","last_active":"2024-09-11T08:54:24.752041+0000","last_peered":"2024-09-11T08:54:24.752041+0000","last_clean":"2024-09-11T08:54:24.752041+0000","last_became_active":"2024-09-11T08:52:49.568147+0000","last_became_peered":"2024-09-11T08:52:49.568147+0000","last_unstale":"2024-09-11T08:54:24.752041+0000","last_undegraded":"2024-09-11T08:54:24.752041+0000","last_fullsized":"2024-09-11T08:54:24.752041+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:10.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:33.520462+0000","last_clean_scrub_stamp":"2024-09-11T08:54:10.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:32.684708+0000","scrub_duration":0.002056146,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":342,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.736392+0000","last_change":"2024-09-11T08:54:17.338878+0000","last_active":"2024-09-11T08:54:24.736392+0000","last_peered":"2024-09-11T08:54:24.736392+0000","last_clean":"2024-09-11T08:54:24.736392+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:54:24.736392+0000","last_undegraded":"2024-09-11T08:54:24.736392+0000","last_fullsized":"2024-09-11T08:54:24.736392+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:17.338793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:30.465003+0000","last_clean_scrub_stamp":"2024-09-11T08:54:17.338793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:36.334048+0000","scrub_duration":0.0028270790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"200'7","reported_seq":331,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734410+0000","last_change":"2024-09-11T08:52:11.155994+0000","last_active":"2024-09-11T08:54:24.734410+0000","last_peered":"2024-09-11T08:54:24.734410+0000","last_clean":"2024-09-11T08:54:24.734410+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:54:24.734410+0000","last_undegraded":"2024-09-11T08:54:24.734410+0000","last_fullsized":"2024-09-11T08:54:24.734410+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'7","last_scrub_stamp":"2024-09-11T08:52:11.155898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:52:11.155898+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002524072,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":341,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.744945+0000","last_change":"2024-09-11T08:52:52.458278+0000","last_active":"2024-09-11T08:54:24.744945+0000","last_peered":"2024-09-11T08:54:24.744945+0000","last_clean":"2024-09-11T08:54:24.744945+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:54:24.744945+0000","last_undegraded":"2024-09-11T08:54:24.744945+0000","last_fullsized":"2024-09-11T08:54:24.744945+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:52.458219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:18.199651+0000","last_clean_scrub_stamp":"2024-09-11T08:52:52.458219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013148910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":326,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734691+0000","last_change":"2024-09-11T08:52:50.730859+0000","last_active":"2024-09-11T08:54:24.734691+0000","last_peered":"2024-09-11T08:54:24.734691+0000","last_clean":"2024-09-11T08:54:24.734691+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:54:24.734691+0000","last_undegraded":"2024-09-11T08:54:24.734691+0000","last_fullsized":"2024-09-11T08:54:24.734691+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:50.730804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:52:50.730804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001426482,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":341,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.739322+0000","last_change":"2024-09-11T08:53:56.854824+0000","last_active":"2024-09-11T08:54:24.739322+0000","last_peered":"2024-09-11T08:54:24.739322+0000","last_clean":"2024-09-11T08:54:24.739322+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:54:24.739322+0000","last_undegraded":"2024-09-11T08:54:24.739322+0000","last_fullsized":"2024-09-11T08:54:24.739322+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:56.854710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:56.854710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:03.522639+0000","scrub_duration":0.0061159049999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":142,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734553+0000","last_change":"2024-09-11T08:53:58.835251+0000","last_active":"2024-09-11T08:54:24.734553+0000","last_peered":"2024-09-11T08:54:24.734553+0000","last_clean":"2024-09-11T08:54:24.734553+0000","last_became_active":"2024-09-11T08:52:49.566575+0000","last_became_peered":"2024-09-11T08:52:49.566575+0000","last_unstale":"2024-09-11T08:54:24.734553+0000","last_undegraded":"2024-09-11T08:54:24.734553+0000","last_fullsized":"2024-09-11T08:54:24.734553+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:58.835161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:58.835161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:03.968370+0000","scrub_duration":0.0065024590000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"200'34","reported_seq":281,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.751587+0000","last_change":"2024-09-11T08:54:17.121799+0000","last_active":"2024-09-11T08:54:24.751587+0000","last_peered":"2024-09-11T08:54:24.751587+0000","last_clean":"2024-09-11T08:54:24.751587+0000","last_became_active":"2024-09-11T08:54:17.121459+0000","last_became_peered":"2024-09-11T08:54:17.121459+0000","last_unstale":"2024-09-11T08:54:24.751587+0000","last_undegraded":"2024-09-11T08:54:24.751587+0000","last_fullsized":"2024-09-11T08:54:24.751587+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:54:04.639215+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:54:04.639215+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:21.573487+0000","scrub_duration":0.011484708999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":362,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.751575+0000","last_change":"2024-09-11T08:53:59.798776+0000","last_active":"2024-09-11T08:54:24.751575+0000","last_peered":"2024-09-11T08:54:24.751575+0000","last_clean":"2024-09-11T08:54:24.751575+0000","last_became_active":"2024-09-11T08:52:18.579026+0000","last_became_peered":"2024-09-11T08:52:18.579026+0000","last_unstale":"2024-09-11T08:54:24.751575+0000","last_undegraded":"2024-09-11T08:54:24.751575+0000","last_fullsized":"2024-09-11T08:54:24.751575+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:59.798705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:59.798705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:22.435474+0000","scrub_duration":0.004245665,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":349,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742528+0000","last_change":"2024-09-11T08:53:14.430444+0000","last_active":"2024-09-11T08:54:24.742528+0000","last_peered":"2024-09-11T08:54:24.742528+0000","last_clean":"2024-09-11T08:54:24.742528+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:54:24.742528+0000","last_undegraded":"2024-09-11T08:54:24.742528+0000","last_fullsized":"2024-09-11T08:54:24.742528+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:53:14.430366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:14.430366+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024769649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":327,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734524+0000","last_change":"2024-09-11T08:53:48.866804+0000","last_active":"2024-09-11T08:54:24.734524+0000","last_peered":"2024-09-11T08:54:24.734524+0000","last_clean":"2024-09-11T08:54:24.734524+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:54:24.734524+0000","last_undegraded":"2024-09-11T08:54:24.734524+0000","last_fullsized":"2024-09-11T08:54:24.734524+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:48.866615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:48.866615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:16.879761+0000","scrub_duration":0.0075952449999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"200'18","reported_seq":378,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742267+0000","last_change":"2024-09-11T08:53:07.440507+0000","last_active":"2024-09-11T08:54:24.742267+0000","last_peered":"2024-09-11T08:54:24.742267+0000","last_clean":"2024-09-11T08:54:24.742267+0000","last_became_active":"2024-09-11T08:49:16.943886+0000","last_became_peered":"2024-09-11T08:49:16.943886+0000","last_unstale":"2024-09-11T08:54:24.742267+0000","last_undegraded":"2024-09-11T08:54:24.742267+0000","last_fullsized":"2024-09-11T08:54:24.742267+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'18","last_scrub_stamp":"2024-09-11T08:53:07.440404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:07.440404+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:28.528108+0000","scrub_duration":0.0041402180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":323,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.738317+0000","last_change":"2024-09-11T08:53:12.948801+0000","last_active":"2024-09-11T08:54:24.738317+0000","last_peered":"2024-09-11T08:54:24.738317+0000","last_clean":"2024-09-11T08:54:24.738317+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:54:24.738317+0000","last_undegraded":"2024-09-11T08:54:24.738317+0000","last_fullsized":"2024-09-11T08:54:24.738317+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:12.948659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:12.948659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039731519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":69,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.736246+0000","last_change":"2024-09-11T08:53:58.534161+0000","last_active":"2024-09-11T08:54:24.736246+0000","last_peered":"2024-09-11T08:54:24.736246+0000","last_clean":"2024-09-11T08:54:24.736246+0000","last_became_active":"2024-09-11T08:52:38.483603+0000","last_became_peered":"2024-09-11T08:52:38.483603+0000","last_unstale":"2024-09-11T08:54:24.736246+0000","last_undegraded":"2024-09-11T08:54:24.736246+0000","last_fullsized":"2024-09-11T08:54:24.736246+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:53:58.534079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:58.534079+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:24.012675+0000","scrub_duration":0.0070998140000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":346,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.751005+0000","last_change":"2024-09-11T08:54:02.804185+0000","last_active":"2024-09-11T08:54:24.751005+0000","last_peered":"2024-09-11T08:54:24.751005+0000","last_clean":"2024-09-11T08:54:24.751005+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:54:24.751005+0000","last_undegraded":"2024-09-11T08:54:24.751005+0000","last_fullsized":"2024-09-11T08:54:24.751005+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:02.804091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:54:02.804091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:22.145193+0000","scrub_duration":0.0066693589999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"280'25","reported_seq":75,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.743385+0000","last_change":"2024-09-11T08:52:49.568462+0000","last_active":"2024-09-11T08:54:24.743385+0000","last_peered":"2024-09-11T08:54:24.743385+0000","last_clean":"2024-09-11T08:54:24.743385+0000","last_became_active":"2024-09-11T08:52:49.568074+0000","last_became_peered":"2024-09-11T08:52:49.568074+0000","last_unstale":"2024-09-11T08:54:24.743385+0000","last_undegraded":"2024-09-11T08:54:24.743385+0000","last_fullsized":"2024-09-11T08:54:24.743385+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"280'25","last_scrub_stamp":"2024-09-11T08:52:24.584533+0000","last_deep_scrub":"280'25","last_deep_scrub_stamp":"2024-09-11T08:52:24.584533+0000","last_clean_scrub_stamp":"2024-09-11T08:52:24.584533+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050623580000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":552,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:26.760252+0000","last_change":"2024-09-11T08:53:59.582420+0000","last_active":"2024-09-11T08:54:26.760252+0000","last_peered":"2024-09-11T08:54:26.760252+0000","last_clean":"2024-09-11T08:54:26.760252+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:54:26.760252+0000","last_undegraded":"2024-09-11T08:54:26.760252+0000","last_fullsized":"2024-09-11T08:54:26.760252+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:53:59.582332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:59.582332+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:06.952812+0000","scrub_duration":0.0076609360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"324'8301","reported_seq":8340,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:31.246325+0000","last_change":"2024-09-11T08:52:56.415129+0000","last_active":"2024-09-11T08:54:31.246325+0000","last_peered":"2024-09-11T08:54:31.246325+0000","last_clean":"2024-09-11T08:54:31.246325+0000","last_became_active":"2024-09-11T08:52:56.414844+0000","last_became_peered":"2024-09-11T08:52:56.414844+0000","last_unstale":"2024-09-11T08:54:31.246325+0000","last_undegraded":"2024-09-11T08:54:31.246325+0000","last_fullsized":"2024-09-11T08:54:31.246325+0000","mapping_epoch":293,"log_start":"307'3900","ondisk_log_start":"307'3900","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4401,"log_dups_size":0,"ondisk_log_size":4401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":544014336,"num_objects":8301,"num_object_clones":0,"num_object_copies":24903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8301,"num_write_kb":531264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"199'81","reported_seq":93,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735769+0000","last_change":"2024-09-11T08:53:16.826016+0000","last_active":"2024-09-11T08:54:24.735769+0000","last_peered":"2024-09-11T08:54:24.735769+0000","last_clean":"2024-09-11T08:54:24.735769+0000","last_became_active":"2024-09-11T08:51:03.023615+0000","last_became_peered":"2024-09-11T08:51:03.023615+0000","last_unstale":"2024-09-11T08:54:24.735769+0000","last_undegraded":"2024-09-11T08:54:24.735769+0000","last_fullsized":"2024-09-11T08:54:24.735769+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:16.825859+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:16.825859+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:32.758862+0000","scrub_duration":0.0087576560000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"200'34","reported_seq":384,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.752491+0000","last_change":"2024-09-11T08:53:03.680968+0000","last_active":"2024-09-11T08:54:24.752491+0000","last_peered":"2024-09-11T08:54:24.752491+0000","last_clean":"2024-09-11T08:54:24.752491+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:54:24.752491+0000","last_undegraded":"2024-09-11T08:54:24.752491+0000","last_fullsized":"2024-09-11T08:54:24.752491+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:53:03.680906+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:53:03.680906+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:26.793062+0000","scrub_duration":0.0038860740000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":326,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742413+0000","last_change":"2024-09-11T08:53:02.487714+0000","last_active":"2024-09-11T08:54:24.742413+0000","last_peered":"2024-09-11T08:54:24.742413+0000","last_clean":"2024-09-11T08:54:24.742413+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:54:24.742413+0000","last_undegraded":"2024-09-11T08:54:24.742413+0000","last_fullsized":"2024-09-11T08:54:24.742413+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:02.487657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:02.487657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011084630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"277'11","reported_seq":550,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:26.767783+0000","last_change":"2024-09-11T08:52:56.987065+0000","last_active":"2024-09-11T08:54:26.767783+0000","last_peered":"2024-09-11T08:54:26.767783+0000","last_clean":"2024-09-11T08:54:26.767783+0000","last_became_active":"2024-09-11T08:52:18.578328+0000","last_became_peered":"2024-09-11T08:52:18.578328+0000","last_unstale":"2024-09-11T08:54:26.767783+0000","last_undegraded":"2024-09-11T08:54:26.767783+0000","last_fullsized":"2024-09-11T08:54:26.767783+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'11","last_scrub_stamp":"2024-09-11T08:52:56.987010+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:52:56.987010+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033882560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":343,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735763+0000","last_change":"2024-09-11T08:52:57.679482+0000","last_active":"2024-09-11T08:54:24.735763+0000","last_peered":"2024-09-11T08:54:24.735763+0000","last_clean":"2024-09-11T08:54:24.735763+0000","last_became_active":"2024-09-11T08:52:18.576898+0000","last_became_peered":"2024-09-11T08:52:18.576898+0000","last_unstale":"2024-09-11T08:54:24.735763+0000","last_undegraded":"2024-09-11T08:54:24.735763+0000","last_fullsized":"2024-09-11T08:54:24.735763+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:57.679430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:57.679430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013664560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"324'8297","reported_seq":8336,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:31.178778+0000","last_change":"2024-09-11T08:52:55.610689+0000","last_active":"2024-09-11T08:54:31.178778+0000","last_peered":"2024-09-11T08:54:31.178778+0000","last_clean":"2024-09-11T08:54:31.178778+0000","last_became_active":"2024-09-11T08:52:55.610383+0000","last_became_peered":"2024-09-11T08:52:55.610383+0000","last_unstale":"2024-09-11T08:54:31.178778+0000","last_undegraded":"2024-09-11T08:54:31.178778+0000","last_fullsized":"2024-09-11T08:54:31.178778+0000","mapping_epoch":293,"log_start":"308'3900","ondisk_log_start":"308'3900","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4397,"log_dups_size":0,"ondisk_log_size":4397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543752192,"num_objects":8297,"num_object_clones":0,"num_object_copies":24891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8297,"num_write_kb":531008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"199'81","reported_seq":93,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735371+0000","last_change":"2024-09-11T08:53:00.953735+0000","last_active":"2024-09-11T08:54:24.735371+0000","last_peered":"2024-09-11T08:54:24.735371+0000","last_clean":"2024-09-11T08:54:24.735371+0000","last_became_active":"2024-09-11T08:51:03.023365+0000","last_became_peered":"2024-09-11T08:51:03.023365+0000","last_unstale":"2024-09-11T08:54:24.735371+0000","last_undegraded":"2024-09-11T08:54:24.735371+0000","last_fullsized":"2024-09-11T08:54:24.735371+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:00.953685+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:00.953685+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001183323,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"200'38","reported_seq":438,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742199+0000","last_change":"2024-09-11T08:52:18.611214+0000","last_active":"2024-09-11T08:54:24.742199+0000","last_peered":"2024-09-11T08:54:24.742199+0000","last_clean":"2024-09-11T08:54:24.742199+0000","last_became_active":"2024-09-11T08:52:18.595013+0000","last_became_peered":"2024-09-11T08:52:18.595013+0000","last_unstale":"2024-09-11T08:54:24.742199+0000","last_undegraded":"2024-09-11T08:54:24.742199+0000","last_fullsized":"2024-09-11T08:54:24.742199+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'38","last_scrub_stamp":"2024-09-11T08:52:14.460428+0000","last_deep_scrub":"199'26","last_deep_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_clean_scrub_stamp":"2024-09-11T08:52:14.460428+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012981754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":349,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.743019+0000","last_change":"2024-09-11T08:52:18.587201+0000","last_active":"2024-09-11T08:54:24.743019+0000","last_peered":"2024-09-11T08:54:24.743019+0000","last_clean":"2024-09-11T08:54:24.743019+0000","last_became_active":"2024-09-11T08:52:18.586084+0000","last_became_peered":"2024-09-11T08:52:18.586084+0000","last_unstale":"2024-09-11T08:54:24.743019+0000","last_undegraded":"2024-09-11T08:54:24.743019+0000","last_fullsized":"2024-09-11T08:54:24.743019+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:13.467288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:13.467288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046940990000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":323,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735018+0000","last_change":"2024-09-11T08:52:59.933645+0000","last_active":"2024-09-11T08:54:24.735018+0000","last_peered":"2024-09-11T08:54:24.735018+0000","last_clean":"2024-09-11T08:54:24.735018+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:54:24.735018+0000","last_undegraded":"2024-09-11T08:54:24.735018+0000","last_fullsized":"2024-09-11T08:54:24.735018+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:59.933599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:29.687577+0000","last_clean_scrub_stamp":"2024-09-11T08:52:59.933599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:28.572636+0000","scrub_duration":0.001022233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":321,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734322+0000","last_change":"2024-09-11T08:54:00.820805+0000","last_active":"2024-09-11T08:54:24.734322+0000","last_peered":"2024-09-11T08:54:24.734322+0000","last_clean":"2024-09-11T08:54:24.734322+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:54:24.734322+0000","last_undegraded":"2024-09-11T08:54:24.734322+0000","last_fullsized":"2024-09-11T08:54:24.734322+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:00.820729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:00.820729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:03.316764+0000","scrub_duration":0.0084568009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"324'8389","reported_seq":8427,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:31.325871+0000","last_change":"2024-09-11T08:52:55.612830+0000","last_active":"2024-09-11T08:54:31.325871+0000","last_peered":"2024-09-11T08:54:31.325871+0000","last_clean":"2024-09-11T08:54:31.325871+0000","last_became_active":"2024-09-11T08:52:55.612351+0000","last_became_peered":"2024-09-11T08:52:55.612351+0000","last_unstale":"2024-09-11T08:54:31.325871+0000","last_undegraded":"2024-09-11T08:54:31.325871+0000","last_fullsized":"2024-09-11T08:54:31.325871+0000","mapping_epoch":293,"log_start":"305'2900","ondisk_log_start":"305'2900","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":5489,"log_dups_size":0,"ondisk_log_size":5489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":549781504,"num_objects":8389,"num_object_clones":0,"num_object_copies":25167,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8389,"num_write_kb":536896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"199'81","reported_seq":98,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742936+0000","last_change":"2024-09-11T08:53:08.418382+0000","last_active":"2024-09-11T08:54:24.742936+0000","last_peered":"2024-09-11T08:54:24.742936+0000","last_clean":"2024-09-11T08:54:24.742936+0000","last_became_active":"2024-09-11T08:51:03.009983+0000","last_became_peered":"2024-09-11T08:51:03.009983+0000","last_unstale":"2024-09-11T08:54:24.742936+0000","last_undegraded":"2024-09-11T08:54:24.742936+0000","last_fullsized":"2024-09-11T08:54:24.742936+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:08.418288+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:08.418288+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003188832,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"200'35","reported_seq":367,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735278+0000","last_change":"2024-09-11T08:52:25.186590+0000","last_active":"2024-09-11T08:54:24.735278+0000","last_peered":"2024-09-11T08:54:24.735278+0000","last_clean":"2024-09-11T08:54:24.735278+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:54:24.735278+0000","last_undegraded":"2024-09-11T08:54:24.735278+0000","last_fullsized":"2024-09-11T08:54:24.735278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'35","last_scrub_stamp":"2024-09-11T08:52:25.186528+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:52:25.186528+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018614930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":76,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.741333+0000","last_change":"2024-09-11T08:54:04.611817+0000","last_active":"2024-09-11T08:54:24.741333+0000","last_peered":"2024-09-11T08:54:24.741333+0000","last_clean":"2024-09-11T08:54:24.741333+0000","last_became_active":"2024-09-11T08:52:18.586549+0000","last_became_peered":"2024-09-11T08:52:18.586549+0000","last_unstale":"2024-09-11T08:54:24.741333+0000","last_undegraded":"2024-09-11T08:54:24.741333+0000","last_fullsized":"2024-09-11T08:54:24.741333+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:04.611505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:28.487573+0000","last_clean_scrub_stamp":"2024-09-11T08:54:04.611505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:30.421697+0000","scrub_duration":0.0061628730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":523,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:26.768567+0000","last_change":"2024-09-11T08:54:17.570568+0000","last_active":"2024-09-11T08:54:26.768567+0000","last_peered":"2024-09-11T08:54:26.768567+0000","last_clean":"2024-09-11T08:54:26.768567+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:54:26.768567+0000","last_undegraded":"2024-09-11T08:54:26.768567+0000","last_fullsized":"2024-09-11T08:54:26.768567+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:54:17.570479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:17.570479+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:18.330338+0000","scrub_duration":0.0077913949999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":234,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.744880+0000","last_change":"2024-09-11T08:54:04.676970+0000","last_active":"2024-09-11T08:54:24.744880+0000","last_peered":"2024-09-11T08:54:24.744880+0000","last_clean":"2024-09-11T08:54:24.744880+0000","last_became_active":"2024-09-11T08:52:38.488567+0000","last_became_peered":"2024-09-11T08:52:38.488567+0000","last_unstale":"2024-09-11T08:54:24.744880+0000","last_undegraded":"2024-09-11T08:54:24.744880+0000","last_fullsized":"2024-09-11T08:54:24.744880+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:04.676760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:04.676760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:26.906059+0000","scrub_duration":0.0044944620000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"324'8364","reported_seq":8403,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:31.243652+0000","last_change":"2024-09-11T08:52:56.414997+0000","last_active":"2024-09-11T08:54:31.243652+0000","last_peered":"2024-09-11T08:54:31.243652+0000","last_clean":"2024-09-11T08:54:31.243652+0000","last_became_active":"2024-09-11T08:52:56.414726+0000","last_became_peered":"2024-09-11T08:52:56.414726+0000","last_unstale":"2024-09-11T08:54:31.243652+0000","last_undegraded":"2024-09-11T08:54:31.243652+0000","last_fullsized":"2024-09-11T08:54:31.243652+0000","mapping_epoch":293,"log_start":"308'4000","ondisk_log_start":"308'4000","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4364,"log_dups_size":0,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":548143104,"num_objects":8364,"num_object_clones":0,"num_object_copies":25092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8364,"num_write_kb":535296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"199'81","reported_seq":344,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.740940+0000","last_change":"2024-09-11T08:53:13.350933+0000","last_active":"2024-09-11T08:54:24.740940+0000","last_peered":"2024-09-11T08:54:24.740940+0000","last_clean":"2024-09-11T08:54:24.740940+0000","last_became_active":"2024-09-11T08:51:03.003800+0000","last_became_peered":"2024-09-11T08:51:03.003800+0000","last_unstale":"2024-09-11T08:54:24.740940+0000","last_undegraded":"2024-09-11T08:54:24.740940+0000","last_fullsized":"2024-09-11T08:54:24.740940+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:13.350843+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:13.350843+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:33.425416+0000","scrub_duration":0.0032054900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"200'25","reported_seq":379,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.751396+0000","last_change":"2024-09-11T08:54:05.976193+0000","last_active":"2024-09-11T08:54:24.751396+0000","last_peered":"2024-09-11T08:54:24.751396+0000","last_clean":"2024-09-11T08:54:24.751396+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:54:24.751396+0000","last_undegraded":"2024-09-11T08:54:24.751396+0000","last_fullsized":"2024-09-11T08:54:24.751396+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:54:05.976089+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_clean_scrub_stamp":"2024-09-11T08:54:05.976089+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:34.076324+0000","scrub_duration":0.20122382899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":332,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.736182+0000","last_change":"2024-09-11T08:52:55.701978+0000","last_active":"2024-09-11T08:54:24.736182+0000","last_peered":"2024-09-11T08:54:24.736182+0000","last_clean":"2024-09-11T08:54:24.736182+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:54:24.736182+0000","last_undegraded":"2024-09-11T08:54:24.736182+0000","last_fullsized":"2024-09-11T08:54:24.736182+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:55.701902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:55.701902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016250119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"292'11","reported_seq":221,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:26.756154+0000","last_change":"2024-09-11T08:53:01.380925+0000","last_active":"2024-09-11T08:54:26.756154+0000","last_peered":"2024-09-11T08:54:26.756154+0000","last_clean":"2024-09-11T08:54:26.756154+0000","last_became_active":"2024-09-11T08:52:49.564770+0000","last_became_peered":"2024-09-11T08:52:49.564770+0000","last_unstale":"2024-09-11T08:54:26.756154+0000","last_undegraded":"2024-09-11T08:54:26.756154+0000","last_fullsized":"2024-09-11T08:54:26.756154+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"292'11","last_scrub_stamp":"2024-09-11T08:53:01.380869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:01.380869+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:27.934762+0000","scrub_duration":0.0014786319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":142,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.741949+0000","last_change":"2024-09-11T08:54:11.523221+0000","last_active":"2024-09-11T08:54:24.741949+0000","last_peered":"2024-09-11T08:54:24.741949+0000","last_clean":"2024-09-11T08:54:24.741949+0000","last_became_active":"2024-09-11T08:52:49.571683+0000","last_became_peered":"2024-09-11T08:52:49.571683+0000","last_unstale":"2024-09-11T08:54:24.741949+0000","last_undegraded":"2024-09-11T08:54:24.741949+0000","last_fullsized":"2024-09-11T08:54:24.741949+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:11.523143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:54:11.523143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:14.335044+0000","scrub_duration":0.010580648999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.d","version":"324'8318","reported_seq":8643,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:31.304041+0000","last_change":"2024-09-11T08:54:13.634665+0000","last_active":"2024-09-11T08:54:31.304041+0000","last_peered":"2024-09-11T08:54:31.304041+0000","last_clean":"2024-09-11T08:54:31.304041+0000","last_became_active":"2024-09-11T08:52:56.414830+0000","last_became_peered":"2024-09-11T08:52:56.414830+0000","last_unstale":"2024-09-11T08:54:31.304041+0000","last_undegraded":"2024-09-11T08:54:31.304041+0000","last_fullsized":"2024-09-11T08:54:31.304041+0000","mapping_epoch":293,"log_start":"305'2800","ondisk_log_start":"305'2800","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'7046","last_scrub_stamp":"2024-09-11T08:54:13.634607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:13.634607+0000","objects_scrubbed":6942,"log_size":5518,"log_dups_size":0,"ondisk_log_size":5518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:39.511951+0000","scrub_duration":3.0316847519999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":545128448,"num_objects":8318,"num_object_clones":0,"num_object_copies":24954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8318,"num_write_kb":532352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"199'81","reported_seq":354,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.741011+0000","last_change":"2024-09-11T08:53:21.460030+0000","last_active":"2024-09-11T08:54:24.741011+0000","last_peered":"2024-09-11T08:54:24.741011+0000","last_clean":"2024-09-11T08:54:24.741011+0000","last_became_active":"2024-09-11T08:52:18.584896+0000","last_became_peered":"2024-09-11T08:52:18.584896+0000","last_unstale":"2024-09-11T08:54:24.741011+0000","last_undegraded":"2024-09-11T08:54:24.741011+0000","last_fullsized":"2024-09-11T08:54:24.741011+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:21.459982+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:21.459982+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:39.076812+0000","scrub_duration":0.0023340290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"200'29","reported_seq":390,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.740104+0000","last_change":"2024-09-11T08:53:57.508836+0000","last_active":"2024-09-11T08:54:24.740104+0000","last_peered":"2024-09-11T08:54:24.740104+0000","last_clean":"2024-09-11T08:54:24.740104+0000","last_became_active":"2024-09-11T08:52:49.563364+0000","last_became_peered":"2024-09-11T08:52:49.563364+0000","last_unstale":"2024-09-11T08:54:24.740104+0000","last_undegraded":"2024-09-11T08:54:24.740104+0000","last_fullsized":"2024-09-11T08:54:24.740104+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'29","last_scrub_stamp":"2024-09-11T08:53:57.508749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:57.508749+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:12.897702+0000","scrub_duration":0.0044953129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":346,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.751782+0000","last_change":"2024-09-11T08:53:13.677539+0000","last_active":"2024-09-11T08:54:24.751782+0000","last_peered":"2024-09-11T08:54:24.751782+0000","last_clean":"2024-09-11T08:54:24.751782+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:54:24.751782+0000","last_undegraded":"2024-09-11T08:54:24.751782+0000","last_fullsized":"2024-09-11T08:54:24.751782+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:13.677387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:13.677387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029413159999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":371,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.740656+0000","last_change":"2024-09-11T08:53:09.384145+0000","last_active":"2024-09-11T08:54:24.740656+0000","last_peered":"2024-09-11T08:54:24.740656+0000","last_clean":"2024-09-11T08:54:24.740656+0000","last_became_active":"2024-09-11T08:52:18.573235+0000","last_became_peered":"2024-09-11T08:52:18.573235+0000","last_unstale":"2024-09-11T08:54:24.740656+0000","last_undegraded":"2024-09-11T08:54:24.740656+0000","last_fullsized":"2024-09-11T08:54:24.740656+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:09.384067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_clean_scrub_stamp":"2024-09-11T08:53:09.384067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:28.457187+0000","scrub_duration":0.0035749929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":360,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.750799+0000","last_change":"2024-09-11T08:53:18.654523+0000","last_active":"2024-09-11T08:54:24.750799+0000","last_peered":"2024-09-11T08:54:24.750799+0000","last_clean":"2024-09-11T08:54:24.750799+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:54:24.750799+0000","last_undegraded":"2024-09-11T08:54:24.750799+0000","last_fullsized":"2024-09-11T08:54:24.750799+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:18.654437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:53:18.654437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016784286999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"324'8255","reported_seq":8566,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:31.294569+0000","last_change":"2024-09-11T08:54:09.063052+0000","last_active":"2024-09-11T08:54:31.294569+0000","last_peered":"2024-09-11T08:54:31.294569+0000","last_clean":"2024-09-11T08:54:31.294569+0000","last_became_active":"2024-09-11T08:52:56.414809+0000","last_became_peered":"2024-09-11T08:52:56.414809+0000","last_unstale":"2024-09-11T08:54:31.294569+0000","last_undegraded":"2024-09-11T08:54:31.294569+0000","last_fullsized":"2024-09-11T08:54:31.294569+0000","mapping_epoch":293,"log_start":"305'2800","ondisk_log_start":"305'2800","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'6646","last_scrub_stamp":"2024-09-11T08:54:09.062953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.062953+0000","objects_scrubbed":6570,"log_size":5455,"log_dups_size":0,"ondisk_log_size":5455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:32.019378+0000","scrub_duration":2.4302389890000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540999680,"num_objects":8255,"num_object_clones":0,"num_object_copies":24765,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8255,"num_write_kb":528320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"199'81","reported_seq":103,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734470+0000","last_change":"2024-09-11T08:54:09.431646+0000","last_active":"2024-09-11T08:54:24.734470+0000","last_peered":"2024-09-11T08:54:24.734470+0000","last_clean":"2024-09-11T08:54:24.734470+0000","last_became_active":"2024-09-11T08:52:18.574468+0000","last_became_peered":"2024-09-11T08:52:18.574468+0000","last_unstale":"2024-09-11T08:54:24.734470+0000","last_undegraded":"2024-09-11T08:54:24.734470+0000","last_fullsized":"2024-09-11T08:54:24.734470+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:54:09.431556+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.431556+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:33.839540+0000","scrub_duration":0.0066392939999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"200'50","reported_seq":610,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:26.765502+0000","last_change":"2024-09-11T08:53:09.745429+0000","last_active":"2024-09-11T08:54:26.765502+0000","last_peered":"2024-09-11T08:54:26.765502+0000","last_clean":"2024-09-11T08:54:26.765502+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:54:26.765502+0000","last_undegraded":"2024-09-11T08:54:26.765502+0000","last_fullsized":"2024-09-11T08:54:26.765502+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'50","last_scrub_stamp":"2024-09-11T08:53:09.745322+0000","last_deep_scrub":"200'50","last_deep_scrub_stamp":"2024-09-11T08:53:09.745322+0000","last_clean_scrub_stamp":"2024-09-11T08:53:09.745322+0000","objects_scrubbed":10,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095378719999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":347,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734699+0000","last_change":"2024-09-11T08:52:18.579773+0000","last_active":"2024-09-11T08:54:24.734699+0000","last_peered":"2024-09-11T08:54:24.734699+0000","last_clean":"2024-09-11T08:54:24.734699+0000","last_became_active":"2024-09-11T08:52:18.579509+0000","last_became_peered":"2024-09-11T08:52:18.579509+0000","last_unstale":"2024-09-11T08:54:24.734699+0000","last_undegraded":"2024-09-11T08:54:24.734699+0000","last_fullsized":"2024-09-11T08:54:24.734699+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:33.010134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_clean_scrub_stamp":"2024-09-11T08:51:33.010134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033959530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":377,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.736345+0000","last_change":"2024-09-11T08:54:14.424233+0000","last_active":"2024-09-11T08:54:24.736345+0000","last_peered":"2024-09-11T08:54:24.736345+0000","last_clean":"2024-09-11T08:54:24.736345+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:54:24.736345+0000","last_undegraded":"2024-09-11T08:54:24.736345+0000","last_fullsized":"2024-09-11T08:54:24.736345+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:54:14.423685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:54:14.423685+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:25.489159+0000","scrub_duration":0.012346487999999999,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":142,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.751447+0000","last_change":"2024-09-11T08:53:20.721615+0000","last_active":"2024-09-11T08:54:24.751447+0000","last_peered":"2024-09-11T08:54:24.751447+0000","last_clean":"2024-09-11T08:54:24.751447+0000","last_became_active":"2024-09-11T08:52:49.569137+0000","last_became_peered":"2024-09-11T08:52:49.569137+0000","last_unstale":"2024-09-11T08:54:24.751447+0000","last_undegraded":"2024-09-11T08:54:24.751447+0000","last_fullsized":"2024-09-11T08:54:24.751447+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:20.721530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:20.721530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:27.624638+0000","scrub_duration":0.0026422770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":320,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.743309+0000","last_change":"2024-09-11T08:53:11.420230+0000","last_active":"2024-09-11T08:54:24.743309+0000","last_peered":"2024-09-11T08:54:24.743309+0000","last_clean":"2024-09-11T08:54:24.743309+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:54:24.743309+0000","last_undegraded":"2024-09-11T08:54:24.743309+0000","last_fullsized":"2024-09-11T08:54:24.743309+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:11.420182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:11.420182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:27.596432+0000","scrub_duration":0.0034052280000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"324'8385","reported_seq":8424,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:31.572696+0000","last_change":"2024-09-11T08:52:56.413844+0000","last_active":"2024-09-11T08:54:31.572696+0000","last_peered":"2024-09-11T08:54:31.572696+0000","last_clean":"2024-09-11T08:54:31.572696+0000","last_became_active":"2024-09-11T08:52:56.413594+0000","last_became_peered":"2024-09-11T08:52:56.413594+0000","last_unstale":"2024-09-11T08:54:31.572696+0000","last_undegraded":"2024-09-11T08:54:31.572696+0000","last_fullsized":"2024-09-11T08:54:31.572696+0000","mapping_epoch":293,"log_start":"306'3200","ondisk_log_start":"306'3200","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":5185,"log_dups_size":0,"ondisk_log_size":5185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":549519360,"num_objects":8385,"num_object_clones":0,"num_object_copies":25155,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8385,"num_write_kb":536640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"199'81","reported_seq":372,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734002+0000","last_change":"2024-09-11T08:52:49.570289+0000","last_active":"2024-09-11T08:54:24.734002+0000","last_peered":"2024-09-11T08:54:24.734002+0000","last_clean":"2024-09-11T08:54:24.734002+0000","last_became_active":"2024-09-11T08:52:49.569934+0000","last_became_peered":"2024-09-11T08:52:49.569934+0000","last_unstale":"2024-09-11T08:54:24.734002+0000","last_undegraded":"2024-09-11T08:54:24.734002+0000","last_fullsized":"2024-09-11T08:54:24.734002+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"200'17","reported_seq":421,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.752564+0000","last_change":"2024-09-11T08:53:50.749880+0000","last_active":"2024-09-11T08:54:24.752564+0000","last_peered":"2024-09-11T08:54:24.752564+0000","last_clean":"2024-09-11T08:54:24.752564+0000","last_became_active":"2024-09-11T08:52:18.580008+0000","last_became_peered":"2024-09-11T08:52:18.580008+0000","last_unstale":"2024-09-11T08:54:24.752564+0000","last_undegraded":"2024-09-11T08:54:24.752564+0000","last_fullsized":"2024-09-11T08:54:24.752564+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'17","last_scrub_stamp":"2024-09-11T08:53:50.749772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:50.749772+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:14.966848+0000","scrub_duration":0.0041242989999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":372,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.751141+0000","last_change":"2024-09-11T08:54:18.762957+0000","last_active":"2024-09-11T08:54:24.751141+0000","last_peered":"2024-09-11T08:54:24.751141+0000","last_clean":"2024-09-11T08:54:24.751141+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:54:24.751141+0000","last_undegraded":"2024-09-11T08:54:24.751141+0000","last_fullsized":"2024-09-11T08:54:24.751141+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:18.762880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:54:18.762880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:38.949705+0000","scrub_duration":0.001880451,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"263'2","reported_seq":417,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.744562+0000","last_change":"2024-09-11T08:54:16.866098+0000","last_active":"2024-09-11T08:54:24.744562+0000","last_peered":"2024-09-11T08:54:24.744562+0000","last_clean":"2024-09-11T08:54:24.744562+0000","last_became_active":"2024-09-11T08:52:18.586545+0000","last_became_peered":"2024-09-11T08:52:18.586545+0000","last_unstale":"2024-09-11T08:54:24.744562+0000","last_undegraded":"2024-09-11T08:54:24.744562+0000","last_fullsized":"2024-09-11T08:54:24.744562+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'2","last_scrub_stamp":"2024-09-11T08:54:16.866040+0000","last_deep_scrub":"263'2","last_deep_scrub_stamp":"2024-09-11T08:53:08.772979+0000","last_clean_scrub_stamp":"2024-09-11T08:54:16.866040+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:37.122737+0000","scrub_duration":0.0014173129999999999,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":367,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.746007+0000","last_change":"2024-09-11T08:53:57.574981+0000","last_active":"2024-09-11T08:54:24.746007+0000","last_peered":"2024-09-11T08:54:24.746007+0000","last_clean":"2024-09-11T08:54:24.746007+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:54:24.746007+0000","last_undegraded":"2024-09-11T08:54:24.746007+0000","last_fullsized":"2024-09-11T08:54:24.746007+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:57.574778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_clean_scrub_stamp":"2024-09-11T08:53:57.574778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:06.393916+0000","scrub_duration":0.0050739540000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":224,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.740851+0000","last_change":"2024-09-11T08:53:57.539830+0000","last_active":"2024-09-11T08:54:24.740851+0000","last_peered":"2024-09-11T08:54:24.740851+0000","last_clean":"2024-09-11T08:54:24.740851+0000","last_became_active":"2024-09-11T08:52:38.484800+0000","last_became_peered":"2024-09-11T08:52:38.484800+0000","last_unstale":"2024-09-11T08:54:24.740851+0000","last_undegraded":"2024-09-11T08:54:24.740851+0000","last_fullsized":"2024-09-11T08:54:24.740851+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:57.539760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:57.539760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:15.603679+0000","scrub_duration":0.0039035570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"324'8326","reported_seq":8646,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:31.007095+0000","last_change":"2024-09-11T08:54:10.753182+0000","last_active":"2024-09-11T08:54:31.007095+0000","last_peered":"2024-09-11T08:54:31.007095+0000","last_clean":"2024-09-11T08:54:31.007095+0000","last_became_active":"2024-09-11T08:52:56.413725+0000","last_became_peered":"2024-09-11T08:52:56.413725+0000","last_unstale":"2024-09-11T08:54:31.007095+0000","last_undegraded":"2024-09-11T08:54:31.007095+0000","last_fullsized":"2024-09-11T08:54:31.007095+0000","mapping_epoch":293,"log_start":"302'2300","ondisk_log_start":"302'2300","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'6916","last_scrub_stamp":"2024-09-11T08:54:10.753112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:10.753112+0000","objects_scrubbed":6818,"log_size":6026,"log_dups_size":0,"ondisk_log_size":6026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:37.548086+0000","scrub_duration":3.0166241519999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":545652736,"num_objects":8326,"num_object_clones":0,"num_object_copies":24978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8326,"num_write_kb":532864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"199'81","reported_seq":356,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.756183+0000","last_change":"2024-09-11T08:53:01.761090+0000","last_active":"2024-09-11T08:54:24.756183+0000","last_peered":"2024-09-11T08:54:24.756183+0000","last_clean":"2024-09-11T08:54:24.756183+0000","last_became_active":"2024-09-11T08:51:03.027281+0000","last_became_peered":"2024-09-11T08:51:03.027281+0000","last_unstale":"2024-09-11T08:54:24.756183+0000","last_undegraded":"2024-09-11T08:54:24.756183+0000","last_fullsized":"2024-09-11T08:54:24.756183+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:01.760951+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:01.760951+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017313369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"200'24","reported_seq":90,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.740889+0000","last_change":"2024-09-11T08:54:14.506978+0000","last_active":"2024-09-11T08:54:24.740889+0000","last_peered":"2024-09-11T08:54:24.740889+0000","last_clean":"2024-09-11T08:54:24.740889+0000","last_became_active":"2024-09-11T08:52:18.580950+0000","last_became_peered":"2024-09-11T08:52:18.580950+0000","last_unstale":"2024-09-11T08:54:24.740889+0000","last_undegraded":"2024-09-11T08:54:24.740889+0000","last_fullsized":"2024-09-11T08:54:24.740889+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'24","last_scrub_stamp":"2024-09-11T08:54:14.506895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:14.506895+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:33.854703+0000","scrub_duration":0.0076186200000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":142,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735088+0000","last_change":"2024-09-11T08:53:18.845981+0000","last_active":"2024-09-11T08:54:24.735088+0000","last_peered":"2024-09-11T08:54:24.735088+0000","last_clean":"2024-09-11T08:54:24.735088+0000","last_became_active":"2024-09-11T08:52:49.566437+0000","last_became_peered":"2024-09-11T08:52:49.566437+0000","last_unstale":"2024-09-11T08:54:24.735088+0000","last_undegraded":"2024-09-11T08:54:24.735088+0000","last_fullsized":"2024-09-11T08:54:24.735088+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:18.845896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:18.845896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:35.649215+0000","scrub_duration":0.0053148020000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"318'6","reported_seq":212,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:30.777920+0000","last_change":"2024-09-11T08:54:12.873744+0000","last_active":"2024-09-11T08:54:30.777920+0000","last_peered":"2024-09-11T08:54:30.777920+0000","last_clean":"2024-09-11T08:54:30.777920+0000","last_became_active":"2024-09-11T08:52:38.484664+0000","last_became_peered":"2024-09-11T08:52:38.484664+0000","last_unstale":"2024-09-11T08:54:30.777920+0000","last_undegraded":"2024-09-11T08:54:30.777920+0000","last_fullsized":"2024-09-11T08:54:30.777920+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'6","last_scrub_stamp":"2024-09-11T08:54:12.873653+0000","last_deep_scrub":"318'6","last_deep_scrub_stamp":"2024-09-11T08:54:12.873653+0000","last_clean_scrub_stamp":"2024-09-11T08:54:12.873653+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:21.779837+0000","scrub_duration":0.01500046,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":358,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.736850+0000","last_change":"2024-09-11T08:54:16.367927+0000","last_active":"2024-09-11T08:54:24.736850+0000","last_peered":"2024-09-11T08:54:24.736850+0000","last_clean":"2024-09-11T08:54:24.736850+0000","last_became_active":"2024-09-11T08:52:18.578530+0000","last_became_peered":"2024-09-11T08:52:18.578530+0000","last_unstale":"2024-09-11T08:54:24.736850+0000","last_undegraded":"2024-09-11T08:54:24.736850+0000","last_fullsized":"2024-09-11T08:54:24.736850+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:16.367873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:16.367873+0000","last_clean_scrub_stamp":"2024-09-11T08:54:16.367873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:41.013346+0000","scrub_duration":0.0013677540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":340,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.751881+0000","last_change":"2024-09-11T08:53:58.816756+0000","last_active":"2024-09-11T08:54:24.751881+0000","last_peered":"2024-09-11T08:54:24.751881+0000","last_clean":"2024-09-11T08:54:24.751881+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:54:24.751881+0000","last_undegraded":"2024-09-11T08:54:24.751881+0000","last_fullsized":"2024-09-11T08:54:24.751881+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:58.816631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:53:58.816631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:12.634412+0000","scrub_duration":0.0076590699999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"324'8321","reported_seq":8653,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:31.294434+0000","last_change":"2024-09-11T08:54:15.231631+0000","last_active":"2024-09-11T08:54:31.294434+0000","last_peered":"2024-09-11T08:54:31.294434+0000","last_clean":"2024-09-11T08:54:31.294434+0000","last_became_active":"2024-09-11T08:52:56.413344+0000","last_became_peered":"2024-09-11T08:52:56.413344+0000","last_unstale":"2024-09-11T08:54:31.294434+0000","last_undegraded":"2024-09-11T08:54:31.294434+0000","last_fullsized":"2024-09-11T08:54:31.294434+0000","mapping_epoch":293,"log_start":"305'2800","ondisk_log_start":"305'2800","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'7183","last_scrub_stamp":"2024-09-11T08:54:15.231547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:15.231547+0000","objects_scrubbed":7103,"log_size":5521,"log_dups_size":0,"ondisk_log_size":5521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:26.393977+0000","scrub_duration":2.5900008689999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":545325056,"num_objects":8321,"num_object_clones":0,"num_object_copies":24963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8321,"num_write_kb":532544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"201'27","reported_seq":250,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.741406+0000","last_change":"2024-09-11T08:54:00.612065+0000","last_active":"2024-09-11T08:54:24.741406+0000","last_peered":"2024-09-11T08:54:24.741406+0000","last_clean":"2024-09-11T08:54:24.741406+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:54:24.741406+0000","last_undegraded":"2024-09-11T08:54:24.741406+0000","last_fullsized":"2024-09-11T08:54:24.741406+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'27","last_scrub_stamp":"2024-09-11T08:54:00.611977+0000","last_deep_scrub":"201'27","last_deep_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_clean_scrub_stamp":"2024-09-11T08:54:00.611977+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:13.191188+0000","scrub_duration":0.0021148109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":335,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.743109+0000","last_change":"2024-09-11T08:53:51.352190+0000","last_active":"2024-09-11T08:54:24.743109+0000","last_peered":"2024-09-11T08:54:24.743109+0000","last_clean":"2024-09-11T08:54:24.743109+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:54:24.743109+0000","last_undegraded":"2024-09-11T08:54:24.743109+0000","last_fullsized":"2024-09-11T08:54:24.743109+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:53:51.352089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:51.352089+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:53.805668+0000","scrub_duration":0.0032468750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"199'81","reported_seq":294,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.741911+0000","last_change":"2024-09-11T08:53:15.431689+0000","last_active":"2024-09-11T08:54:24.741911+0000","last_peered":"2024-09-11T08:54:24.741911+0000","last_clean":"2024-09-11T08:54:24.741911+0000","last_became_active":"2024-09-11T08:52:49.565126+0000","last_became_peered":"2024-09-11T08:52:49.565126+0000","last_unstale":"2024-09-11T08:54:24.741911+0000","last_undegraded":"2024-09-11T08:54:24.741911+0000","last_fullsized":"2024-09-11T08:54:24.741911+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:15.431514+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:15.431514+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:33.145059+0000","scrub_duration":0.0073442550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":59,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.744225+0000","last_change":"2024-09-11T08:52:53.870726+0000","last_active":"2024-09-11T08:54:24.744225+0000","last_peered":"2024-09-11T08:54:24.744225+0000","last_clean":"2024-09-11T08:54:24.744225+0000","last_became_active":"2024-09-11T08:52:38.485446+0000","last_became_peered":"2024-09-11T08:52:38.485446+0000","last_unstale":"2024-09-11T08:54:24.744225+0000","last_undegraded":"2024-09-11T08:54:24.744225+0000","last_fullsized":"2024-09-11T08:54:24.744225+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:53.870644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:53.870644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001623457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":336,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.736729+0000","last_change":"2024-09-11T08:53:54.582425+0000","last_active":"2024-09-11T08:54:24.736729+0000","last_peered":"2024-09-11T08:54:24.736729+0000","last_clean":"2024-09-11T08:54:24.736729+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:54:24.736729+0000","last_undegraded":"2024-09-11T08:54:24.736729+0000","last_fullsized":"2024-09-11T08:54:24.736729+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:54.582338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:54.582338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:18.198186+0000","scrub_duration":0.004605914,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"324'8374","reported_seq":8412,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:31.351431+0000","last_change":"2024-09-11T08:52:55.612239+0000","last_active":"2024-09-11T08:54:31.351431+0000","last_peered":"2024-09-11T08:54:31.351431+0000","last_clean":"2024-09-11T08:54:31.351431+0000","last_became_active":"2024-09-11T08:52:55.611835+0000","last_became_peered":"2024-09-11T08:52:55.611835+0000","last_unstale":"2024-09-11T08:54:31.351431+0000","last_undegraded":"2024-09-11T08:54:31.351431+0000","last_fullsized":"2024-09-11T08:54:31.351431+0000","mapping_epoch":293,"log_start":"305'2900","ondisk_log_start":"305'2900","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":5474,"log_dups_size":0,"ondisk_log_size":5474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":548798464,"num_objects":8374,"num_object_clones":0,"num_object_copies":25122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8374,"num_write_kb":535936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"200'19","reported_seq":415,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.739489+0000","last_change":"2024-09-11T08:51:21.979895+0000","last_active":"2024-09-11T08:54:24.739489+0000","last_peered":"2024-09-11T08:54:24.739489+0000","last_clean":"2024-09-11T08:54:24.739489+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:54:24.739489+0000","last_undegraded":"2024-09-11T08:54:24.739489+0000","last_fullsized":"2024-09-11T08:54:24.739489+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:21.979796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:21.979796+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045321210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":317,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735561+0000","last_change":"2024-09-11T08:51:38.083246+0000","last_active":"2024-09-11T08:54:24.735561+0000","last_peered":"2024-09-11T08:54:24.735561+0000","last_clean":"2024-09-11T08:54:24.735561+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:54:24.735561+0000","last_undegraded":"2024-09-11T08:54:24.735561+0000","last_fullsized":"2024-09-11T08:54:24.735561+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:38.083168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:51:38.083168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029716930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":335,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.736611+0000","last_change":"2024-09-11T08:52:56.675410+0000","last_active":"2024-09-11T08:54:24.736611+0000","last_peered":"2024-09-11T08:54:24.736611+0000","last_clean":"2024-09-11T08:54:24.736611+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:54:24.736611+0000","last_undegraded":"2024-09-11T08:54:24.736611+0000","last_fullsized":"2024-09-11T08:54:24.736611+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:56.675326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:52:56.675326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033910770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":340,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.743003+0000","last_change":"2024-09-11T08:53:15.469630+0000","last_active":"2024-09-11T08:54:24.743003+0000","last_peered":"2024-09-11T08:54:24.743003+0000","last_clean":"2024-09-11T08:54:24.743003+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:54:24.743003+0000","last_undegraded":"2024-09-11T08:54:24.743003+0000","last_fullsized":"2024-09-11T08:54:24.743003+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:15.469262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:15.469262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:34.242644+0000","scrub_duration":0.0071803099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"324'8384","reported_seq":8435,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:31.571503+0000","last_change":"2024-09-11T08:54:17.163983+0000","last_active":"2024-09-11T08:54:31.571503+0000","last_peered":"2024-09-11T08:54:31.571503+0000","last_clean":"2024-09-11T08:54:31.571503+0000","last_became_active":"2024-09-11T08:54:17.120339+0000","last_became_peered":"2024-09-11T08:54:17.120339+0000","last_unstale":"2024-09-11T08:54:31.571503+0000","last_undegraded":"2024-09-11T08:54:31.571503+0000","last_fullsized":"2024-09-11T08:54:31.571503+0000","mapping_epoch":319,"log_start":"306'3173","ondisk_log_start":"306'3173","created":293,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":549453824,"num_objects":8384,"num_object_clones":0,"num_object_copies":25152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8384,"num_write_kb":536576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.8","version":"199'81","reported_seq":100,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734352+0000","last_change":"2024-09-11T08:53:14.810007+0000","last_active":"2024-09-11T08:54:24.734352+0000","last_peered":"2024-09-11T08:54:24.734352+0000","last_clean":"2024-09-11T08:54:24.734352+0000","last_became_active":"2024-09-11T08:51:03.020941+0000","last_became_peered":"2024-09-11T08:51:03.020941+0000","last_unstale":"2024-09-11T08:54:24.734352+0000","last_undegraded":"2024-09-11T08:54:24.734352+0000","last_fullsized":"2024-09-11T08:54:24.734352+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:14.809894+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:53:14.809894+0000","last_clean_scrub_stamp":"2024-09-11T08:53:14.809894+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0065883419999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"317'39","reported_seq":392,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735236+0000","last_change":"2024-09-11T08:54:06.928521+0000","last_active":"2024-09-11T08:54:24.735236+0000","last_peered":"2024-09-11T08:54:24.735236+0000","last_clean":"2024-09-11T08:54:24.735236+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:54:24.735236+0000","last_undegraded":"2024-09-11T08:54:24.735236+0000","last_fullsized":"2024-09-11T08:54:24.735236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"317'39","last_scrub_stamp":"2024-09-11T08:54:06.928266+0000","last_deep_scrub":"317'39","last_deep_scrub_stamp":"2024-09-11T08:54:06.928266+0000","last_clean_scrub_stamp":"2024-09-11T08:54:06.928266+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:15.212159+0000","scrub_duration":0.016459872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":365,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.740923+0000","last_change":"2024-09-11T08:53:10.339152+0000","last_active":"2024-09-11T08:54:24.740923+0000","last_peered":"2024-09-11T08:54:24.740923+0000","last_clean":"2024-09-11T08:54:24.740923+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:54:24.740923+0000","last_undegraded":"2024-09-11T08:54:24.740923+0000","last_fullsized":"2024-09-11T08:54:24.740923+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:10.339096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:25.273134+0000","last_clean_scrub_stamp":"2024-09-11T08:53:10.339096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:31.094115+0000","scrub_duration":0.0027644290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":362,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.751516+0000","last_change":"2024-09-11T08:54:04.809120+0000","last_active":"2024-09-11T08:54:24.751516+0000","last_peered":"2024-09-11T08:54:24.751516+0000","last_clean":"2024-09-11T08:54:24.751516+0000","last_became_active":"2024-09-11T08:52:18.577201+0000","last_became_peered":"2024-09-11T08:52:18.577201+0000","last_unstale":"2024-09-11T08:54:24.751516+0000","last_undegraded":"2024-09-11T08:54:24.751516+0000","last_fullsized":"2024-09-11T08:54:24.751516+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:04.809032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:04.809032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:34.035088+0000","scrub_duration":0.005911347,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":17,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742417+0000","last_change":"2024-09-11T08:54:17.121779+0000","last_active":"2024-09-11T08:54:24.742417+0000","last_peered":"2024-09-11T08:54:24.742417+0000","last_clean":"2024-09-11T08:54:24.742417+0000","last_became_active":"2024-09-11T08:54:17.121372+0000","last_became_peered":"2024-09-11T08:54:17.121372+0000","last_unstale":"2024-09-11T08:54:24.742417+0000","last_undegraded":"2024-09-11T08:54:24.742417+0000","last_fullsized":"2024-09-11T08:54:24.742417+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:07.808138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:07.808138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"324'8306","reported_seq":8530,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:31.813968+0000","last_change":"2024-09-11T08:54:19.747562+0000","last_active":"2024-09-11T08:54:31.813968+0000","last_peered":"2024-09-11T08:54:31.813968+0000","last_clean":"2024-09-11T08:54:31.813968+0000","last_became_active":"2024-09-11T08:52:56.414765+0000","last_became_peered":"2024-09-11T08:52:56.414765+0000","last_unstale":"2024-09-11T08:54:31.813968+0000","last_undegraded":"2024-09-11T08:54:31.813968+0000","last_fullsized":"2024-09-11T08:54:31.813968+0000","mapping_epoch":293,"log_start":"306'3036","ondisk_log_start":"306'3036","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":4461,"log_size":5270,"log_dups_size":0,"ondisk_log_size":5270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":544342016,"num_objects":8306,"num_object_clones":0,"num_object_copies":24918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8306,"num_write_kb":531584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"199'81","reported_seq":376,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742487+0000","last_change":"2024-09-11T08:52:38.482411+0000","last_active":"2024-09-11T08:54:24.742487+0000","last_peered":"2024-09-11T08:54:24.742487+0000","last_clean":"2024-09-11T08:54:24.742487+0000","last_became_active":"2024-09-11T08:52:38.481475+0000","last_became_peered":"2024-09-11T08:52:38.481475+0000","last_unstale":"2024-09-11T08:54:24.742487+0000","last_undegraded":"2024-09-11T08:54:24.742487+0000","last_fullsized":"2024-09-11T08:54:24.742487+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:48:46.028660+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"200'20","reported_seq":395,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.738392+0000","last_change":"2024-09-11T08:52:18.090235+0000","last_active":"2024-09-11T08:54:24.738392+0000","last_peered":"2024-09-11T08:54:24.738392+0000","last_clean":"2024-09-11T08:54:24.738392+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:54:24.738392+0000","last_undegraded":"2024-09-11T08:54:24.738392+0000","last_fullsized":"2024-09-11T08:54:24.738392+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'20","last_scrub_stamp":"2024-09-11T08:52:18.090140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:52:18.090140+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018601010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":346,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742786+0000","last_change":"2024-09-11T08:53:19.389486+0000","last_active":"2024-09-11T08:54:24.742786+0000","last_peered":"2024-09-11T08:54:24.742786+0000","last_clean":"2024-09-11T08:54:24.742786+0000","last_became_active":"2024-09-11T08:52:18.586343+0000","last_became_peered":"2024-09-11T08:52:18.586343+0000","last_unstale":"2024-09-11T08:54:24.742786+0000","last_undegraded":"2024-09-11T08:54:24.742786+0000","last_fullsized":"2024-09-11T08:54:24.742786+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:19.389411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:19.389411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058663470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"299'9","reported_seq":563,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:26.757939+0000","last_change":"2024-09-11T08:53:05.835268+0000","last_active":"2024-09-11T08:54:26.757939+0000","last_peered":"2024-09-11T08:54:26.757939+0000","last_clean":"2024-09-11T08:54:26.757939+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:54:26.757939+0000","last_undegraded":"2024-09-11T08:54:26.757939+0000","last_fullsized":"2024-09-11T08:54:26.757939+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'9","last_scrub_stamp":"2024-09-11T08:53:05.835177+0000","last_deep_scrub":"299'9","last_deep_scrub_stamp":"2024-09-11T08:53:05.835177+0000","last_clean_scrub_stamp":"2024-09-11T08:53:05.835177+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089053550000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":321,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735132+0000","last_change":"2024-09-11T08:53:07.008184+0000","last_active":"2024-09-11T08:54:24.735132+0000","last_peered":"2024-09-11T08:54:24.735132+0000","last_clean":"2024-09-11T08:54:24.735132+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:54:24.735132+0000","last_undegraded":"2024-09-11T08:54:24.735132+0000","last_fullsized":"2024-09-11T08:54:24.735132+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:07.008088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:53:07.008088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041216330000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"324'8577","reported_seq":8893,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:32.036376+0000","last_change":"2024-09-11T08:54:17.137423+0000","last_active":"2024-09-11T08:54:32.036376+0000","last_peered":"2024-09-11T08:54:32.036376+0000","last_clean":"2024-09-11T08:54:32.036376+0000","last_became_active":"2024-09-11T08:54:17.121397+0000","last_became_peered":"2024-09-11T08:54:17.121397+0000","last_unstale":"2024-09-11T08:54:32.036376+0000","last_undegraded":"2024-09-11T08:54:32.036376+0000","last_fullsized":"2024-09-11T08:54:32.036376+0000","mapping_epoch":319,"log_start":"306'3403","ondisk_log_start":"306'3403","created":293,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"317'6616","last_scrub_stamp":"2024-09-11T08:54:04.991011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:04.991011+0000","objects_scrubbed":6499,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:21.176143+0000","scrub_duration":3.4878316329999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":562102272,"num_objects":8577,"num_object_clones":0,"num_object_copies":25731,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8577,"num_write_kb":548928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"200'15","reported_seq":86,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.745260+0000","last_change":"2024-09-11T08:53:52.645375+0000","last_active":"2024-09-11T08:54:24.745260+0000","last_peered":"2024-09-11T08:54:24.745260+0000","last_clean":"2024-09-11T08:54:24.745260+0000","last_became_active":"2024-09-11T08:52:18.588329+0000","last_became_peered":"2024-09-11T08:52:18.588329+0000","last_unstale":"2024-09-11T08:54:24.745260+0000","last_undegraded":"2024-09-11T08:54:24.745260+0000","last_fullsized":"2024-09-11T08:54:24.745260+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:53:52.645266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:52.645266+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:14.532828+0000","scrub_duration":0.00332431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":339,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.736502+0000","last_change":"2024-09-11T08:53:13.771378+0000","last_active":"2024-09-11T08:54:24.736502+0000","last_peered":"2024-09-11T08:54:24.736502+0000","last_clean":"2024-09-11T08:54:24.736502+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:54:24.736502+0000","last_undegraded":"2024-09-11T08:54:24.736502+0000","last_fullsized":"2024-09-11T08:54:24.736502+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:13.771300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:53:13.771300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:35.854425+0000","scrub_duration":0.0068129849999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":17,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734204+0000","last_change":"2024-09-11T08:54:17.120062+0000","last_active":"2024-09-11T08:54:24.734204+0000","last_peered":"2024-09-11T08:54:24.734204+0000","last_clean":"2024-09-11T08:54:24.734204+0000","last_became_active":"2024-09-11T08:54:17.119864+0000","last_became_peered":"2024-09-11T08:54:17.119864+0000","last_unstale":"2024-09-11T08:54:24.734204+0000","last_undegraded":"2024-09-11T08:54:24.734204+0000","last_fullsized":"2024-09-11T08:54:24.734204+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:06.683645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:06.683645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:17.562942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":59,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.744293+0000","last_change":"2024-09-11T08:53:52.601909+0000","last_active":"2024-09-11T08:54:24.744293+0000","last_peered":"2024-09-11T08:54:24.744293+0000","last_clean":"2024-09-11T08:54:24.744293+0000","last_became_active":"2024-09-11T08:52:49.569749+0000","last_became_peered":"2024-09-11T08:52:49.569749+0000","last_unstale":"2024-09-11T08:54:24.744293+0000","last_undegraded":"2024-09-11T08:54:24.744293+0000","last_fullsized":"2024-09-11T08:54:24.744293+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:52.601794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:52.601794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:03.614905+0000","scrub_duration":0.0053285770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"324'8231","reported_seq":8270,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:31.198626+0000","last_change":"2024-09-11T08:52:56.413137+0000","last_active":"2024-09-11T08:54:31.198626+0000","last_peered":"2024-09-11T08:54:31.198626+0000","last_clean":"2024-09-11T08:54:31.198626+0000","last_became_active":"2024-09-11T08:52:56.412780+0000","last_became_peered":"2024-09-11T08:52:56.412780+0000","last_unstale":"2024-09-11T08:54:31.198626+0000","last_undegraded":"2024-09-11T08:54:31.198626+0000","last_fullsized":"2024-09-11T08:54:31.198626+0000","mapping_epoch":293,"log_start":"308'3800","ondisk_log_start":"308'3800","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4431,"log_dups_size":0,"ondisk_log_size":4431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539426816,"num_objects":8231,"num_object_clones":0,"num_object_copies":24693,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8231,"num_write_kb":526784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"200'35","reported_seq":252,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.740735+0000","last_change":"2024-09-11T08:53:48.594039+0000","last_active":"2024-09-11T08:54:24.740735+0000","last_peered":"2024-09-11T08:54:24.740735+0000","last_clean":"2024-09-11T08:54:24.740735+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:54:24.740735+0000","last_undegraded":"2024-09-11T08:54:24.740735+0000","last_fullsized":"2024-09-11T08:54:24.740735+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'35","last_scrub_stamp":"2024-09-11T08:53:48.593945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:48.593945+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:53.869685+0000","scrub_duration":0.0058120400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":60,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735962+0000","last_change":"2024-09-11T08:53:56.555571+0000","last_active":"2024-09-11T08:54:24.735962+0000","last_peered":"2024-09-11T08:54:24.735962+0000","last_clean":"2024-09-11T08:54:24.735962+0000","last_became_active":"2024-09-11T08:52:38.484857+0000","last_became_peered":"2024-09-11T08:52:38.484857+0000","last_unstale":"2024-09-11T08:54:24.735962+0000","last_undegraded":"2024-09-11T08:54:24.735962+0000","last_fullsized":"2024-09-11T08:54:24.735962+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:56.555486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:56.555486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:14.819930+0000","scrub_duration":0.0053230159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":384,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.744794+0000","last_change":"2024-09-11T08:53:01.907502+0000","last_active":"2024-09-11T08:54:24.744794+0000","last_peered":"2024-09-11T08:54:24.744794+0000","last_clean":"2024-09-11T08:54:24.744794+0000","last_became_active":"2024-09-11T08:52:18.584748+0000","last_became_peered":"2024-09-11T08:52:18.584748+0000","last_unstale":"2024-09-11T08:54:24.744794+0000","last_undegraded":"2024-09-11T08:54:24.744794+0000","last_fullsized":"2024-09-11T08:54:24.744794+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:01.907441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:35.022256+0000","last_clean_scrub_stamp":"2024-09-11T08:53:01.907441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:28.038717+0000","scrub_duration":0.001627903,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742273+0000","last_change":"2024-09-11T08:54:17.155880+0000","last_active":"2024-09-11T08:54:24.742273+0000","last_peered":"2024-09-11T08:54:24.742273+0000","last_clean":"2024-09-11T08:54:24.742273+0000","last_became_active":"2024-09-11T08:54:17.155625+0000","last_became_peered":"2024-09-11T08:54:17.155625+0000","last_unstale":"2024-09-11T08:54:24.742273+0000","last_undegraded":"2024-09-11T08:54:24.742273+0000","last_fullsized":"2024-09-11T08:54:24.742273+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:14.828985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:14.828985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:43.840148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.7","version":"324'8299","reported_seq":8616,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:31.323888+0000","last_change":"2024-09-11T08:54:11.461673+0000","last_active":"2024-09-11T08:54:31.323888+0000","last_peered":"2024-09-11T08:54:31.323888+0000","last_clean":"2024-09-11T08:54:31.323888+0000","last_became_active":"2024-09-11T08:52:55.613121+0000","last_became_peered":"2024-09-11T08:52:55.613121+0000","last_unstale":"2024-09-11T08:54:31.323888+0000","last_undegraded":"2024-09-11T08:54:31.323888+0000","last_fullsized":"2024-09-11T08:54:31.323888+0000","mapping_epoch":293,"log_start":"305'2800","ondisk_log_start":"305'2800","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'6823","last_scrub_stamp":"2024-09-11T08:54:11.461591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:11.461591+0000","objects_scrubbed":6729,"log_size":5499,"log_dups_size":0,"ondisk_log_size":5499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:13.382893+0000","scrub_duration":2.7728704209999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543883264,"num_objects":8299,"num_object_clones":0,"num_object_copies":24897,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8299,"num_write_kb":531136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"302'23","reported_seq":362,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735160+0000","last_change":"2024-09-11T08:53:16.897477+0000","last_active":"2024-09-11T08:54:24.735160+0000","last_peered":"2024-09-11T08:54:24.735160+0000","last_clean":"2024-09-11T08:54:24.735160+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:54:24.735160+0000","last_undegraded":"2024-09-11T08:54:24.735160+0000","last_fullsized":"2024-09-11T08:54:24.735160+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'23","last_scrub_stamp":"2024-09-11T08:53:16.896643+0000","last_deep_scrub":"302'23","last_deep_scrub_stamp":"2024-09-11T08:53:16.896643+0000","last_clean_scrub_stamp":"2024-09-11T08:53:16.896643+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:45.449138+0000","scrub_duration":0.0090878710000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":50,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.739658+0000","last_change":"2024-09-11T08:52:38.488564+0000","last_active":"2024-09-11T08:54:24.739658+0000","last_peered":"2024-09-11T08:54:24.739658+0000","last_clean":"2024-09-11T08:54:24.739658+0000","last_became_active":"2024-09-11T08:52:38.487881+0000","last_became_peered":"2024-09-11T08:52:38.487881+0000","last_unstale":"2024-09-11T08:54:24.739658+0000","last_undegraded":"2024-09-11T08:54:24.739658+0000","last_fullsized":"2024-09-11T08:54:24.739658+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:24.331966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:24.331966+0000","last_clean_scrub_stamp":"2024-09-11T08:51:24.331966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":326,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.751860+0000","last_change":"2024-09-11T08:54:17.122433+0000","last_active":"2024-09-11T08:54:24.751860+0000","last_peered":"2024-09-11T08:54:24.751860+0000","last_clean":"2024-09-11T08:54:24.751860+0000","last_became_active":"2024-09-11T08:54:17.122020+0000","last_became_peered":"2024-09-11T08:54:17.122020+0000","last_unstale":"2024-09-11T08:54:24.751860+0000","last_undegraded":"2024-09-11T08:54:24.751860+0000","last_fullsized":"2024-09-11T08:54:24.751860+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:54.587365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:54.587365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:08.870178+0000","scrub_duration":0.0080726240000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":320,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742483+0000","last_change":"2024-09-11T08:53:58.389027+0000","last_active":"2024-09-11T08:54:24.742483+0000","last_peered":"2024-09-11T08:54:24.742483+0000","last_clean":"2024-09-11T08:54:24.742483+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:54:24.742483+0000","last_undegraded":"2024-09-11T08:54:24.742483+0000","last_fullsized":"2024-09-11T08:54:24.742483+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:58.388796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:53:58.388796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:11.015376+0000","scrub_duration":0.004542244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"324'8166","reported_seq":8205,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:31.546573+0000","last_change":"2024-09-11T08:52:56.413103+0000","last_active":"2024-09-11T08:54:31.546573+0000","last_peered":"2024-09-11T08:54:31.546573+0000","last_clean":"2024-09-11T08:54:31.546573+0000","last_became_active":"2024-09-11T08:52:56.412787+0000","last_became_peered":"2024-09-11T08:52:56.412787+0000","last_unstale":"2024-09-11T08:54:31.546573+0000","last_undegraded":"2024-09-11T08:54:31.546573+0000","last_fullsized":"2024-09-11T08:54:31.546573+0000","mapping_epoch":293,"log_start":"305'2900","ondisk_log_start":"305'2900","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":5266,"log_dups_size":0,"ondisk_log_size":5266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":535166976,"num_objects":8166,"num_object_clones":0,"num_object_copies":24498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8166,"num_write_kb":522624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"200'34","reported_seq":442,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734503+0000","last_change":"2024-09-11T08:53:55.561925+0000","last_active":"2024-09-11T08:54:24.734503+0000","last_peered":"2024-09-11T08:54:24.734503+0000","last_clean":"2024-09-11T08:54:24.734503+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:54:24.734503+0000","last_undegraded":"2024-09-11T08:54:24.734503+0000","last_fullsized":"2024-09-11T08:54:24.734503+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:53:55.561835+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:53:55.561835+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:18.099620+0000","scrub_duration":0.0037196429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":69,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735585+0000","last_change":"2024-09-11T08:52:54.988951+0000","last_active":"2024-09-11T08:54:24.735585+0000","last_peered":"2024-09-11T08:54:24.735585+0000","last_clean":"2024-09-11T08:54:24.735585+0000","last_became_active":"2024-09-11T08:52:38.484199+0000","last_became_peered":"2024-09-11T08:52:38.484199+0000","last_unstale":"2024-09-11T08:54:24.735585+0000","last_undegraded":"2024-09-11T08:54:24.735585+0000","last_fullsized":"2024-09-11T08:54:24.735585+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:52:54.988868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.988868+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019261269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"287'11","reported_seq":359,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:26.753922+0000","last_change":"2024-09-11T08:54:13.613549+0000","last_active":"2024-09-11T08:54:26.753922+0000","last_peered":"2024-09-11T08:54:26.753922+0000","last_clean":"2024-09-11T08:54:26.753922+0000","last_became_active":"2024-09-11T08:52:38.488856+0000","last_became_peered":"2024-09-11T08:52:38.488856+0000","last_unstale":"2024-09-11T08:54:26.753922+0000","last_undegraded":"2024-09-11T08:54:26.753922+0000","last_fullsized":"2024-09-11T08:54:26.753922+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"287'11","last_scrub_stamp":"2024-09-11T08:54:13.613459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:13.613459+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:42.670483+0000","scrub_duration":0.0036276939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":346,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735980+0000","last_change":"2024-09-11T08:52:26.813313+0000","last_active":"2024-09-11T08:54:24.735980+0000","last_peered":"2024-09-11T08:54:24.735980+0000","last_clean":"2024-09-11T08:54:24.735980+0000","last_became_active":"2024-09-11T08:52:18.573947+0000","last_became_peered":"2024-09-11T08:52:18.573947+0000","last_unstale":"2024-09-11T08:54:24.735980+0000","last_undegraded":"2024-09-11T08:54:24.735980+0000","last_fullsized":"2024-09-11T08:54:24.735980+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:26.813220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:26.813220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001765602,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":395,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.741841+0000","last_change":"2024-09-11T08:54:01.629327+0000","last_active":"2024-09-11T08:54:24.741841+0000","last_peered":"2024-09-11T08:54:24.741841+0000","last_clean":"2024-09-11T08:54:24.741841+0000","last_became_active":"2024-09-11T08:52:18.584768+0000","last_became_peered":"2024-09-11T08:52:18.584768+0000","last_unstale":"2024-09-11T08:54:24.741841+0000","last_undegraded":"2024-09-11T08:54:24.741841+0000","last_fullsized":"2024-09-11T08:54:24.741841+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:01.629242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:54:01.629242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:29.803832+0000","scrub_duration":0.003718659,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"200'12","reported_seq":360,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742875+0000","last_change":"2024-09-11T08:52:58.504342+0000","last_active":"2024-09-11T08:54:24.742875+0000","last_peered":"2024-09-11T08:54:24.742875+0000","last_clean":"2024-09-11T08:54:24.742875+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:54:24.742875+0000","last_undegraded":"2024-09-11T08:54:24.742875+0000","last_fullsized":"2024-09-11T08:54:24.742875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'12","last_scrub_stamp":"2024-09-11T08:52:58.504277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:52:58.504277+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016055349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":332,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.736876+0000","last_change":"2024-09-11T08:53:20.732004+0000","last_active":"2024-09-11T08:54:24.736876+0000","last_peered":"2024-09-11T08:54:24.736876+0000","last_clean":"2024-09-11T08:54:24.736876+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:54:24.736876+0000","last_undegraded":"2024-09-11T08:54:24.736876+0000","last_fullsized":"2024-09-11T08:54:24.736876+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:20.731788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:20.731788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:36.734097+0000","scrub_duration":0.0022864600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":359,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.745134+0000","last_change":"2024-09-11T08:54:12.234900+0000","last_active":"2024-09-11T08:54:24.745134+0000","last_peered":"2024-09-11T08:54:24.745134+0000","last_clean":"2024-09-11T08:54:24.745134+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:54:24.745134+0000","last_undegraded":"2024-09-11T08:54:24.745134+0000","last_fullsized":"2024-09-11T08:54:24.745134+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:12.234807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:12.234807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:40.923144+0000","scrub_duration":0.0030160999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"200'15","reported_seq":366,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.743467+0000","last_change":"2024-09-11T08:54:09.548343+0000","last_active":"2024-09-11T08:54:24.743467+0000","last_peered":"2024-09-11T08:54:24.743467+0000","last_clean":"2024-09-11T08:54:24.743467+0000","last_became_active":"2024-09-11T08:52:49.569850+0000","last_became_peered":"2024-09-11T08:52:49.569850+0000","last_unstale":"2024-09-11T08:54:24.743467+0000","last_undegraded":"2024-09-11T08:54:24.743467+0000","last_fullsized":"2024-09-11T08:54:24.743467+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:54:09.548256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.548256+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:22.773051+0000","scrub_duration":0.0072201399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":50,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735493+0000","last_change":"2024-09-11T08:52:38.486022+0000","last_active":"2024-09-11T08:54:24.735493+0000","last_peered":"2024-09-11T08:54:24.735493+0000","last_clean":"2024-09-11T08:54:24.735493+0000","last_became_active":"2024-09-11T08:52:38.485708+0000","last_became_peered":"2024-09-11T08:52:38.485708+0000","last_unstale":"2024-09-11T08:54:24.735493+0000","last_undegraded":"2024-09-11T08:54:24.735493+0000","last_fullsized":"2024-09-11T08:54:24.735493+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:51:20.368038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:51:20.368038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":322,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742658+0000","last_change":"2024-09-11T08:54:06.496717+0000","last_active":"2024-09-11T08:54:24.742658+0000","last_peered":"2024-09-11T08:54:24.742658+0000","last_clean":"2024-09-11T08:54:24.742658+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:54:24.742658+0000","last_undegraded":"2024-09-11T08:54:24.742658+0000","last_fullsized":"2024-09-11T08:54:24.742658+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:06.496497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:06.496497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:32.871720+0000","scrub_duration":0.0072082789999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":360,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.744152+0000","last_change":"2024-09-11T08:53:56.542716+0000","last_active":"2024-09-11T08:54:24.744152+0000","last_peered":"2024-09-11T08:54:24.744152+0000","last_clean":"2024-09-11T08:54:24.744152+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:54:24.744152+0000","last_undegraded":"2024-09-11T08:54:24.744152+0000","last_fullsized":"2024-09-11T08:54:24.744152+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:56.542653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_clean_scrub_stamp":"2024-09-11T08:53:56.542653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:15.861997+0000","scrub_duration":0.010395433000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"200'15","reported_seq":385,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.736041+0000","last_change":"2024-09-11T08:54:08.406799+0000","last_active":"2024-09-11T08:54:24.736041+0000","last_peered":"2024-09-11T08:54:24.736041+0000","last_clean":"2024-09-11T08:54:24.736041+0000","last_became_active":"2024-09-11T08:53:58.555987+0000","last_became_peered":"2024-09-11T08:53:58.555987+0000","last_unstale":"2024-09-11T08:54:24.736041+0000","last_undegraded":"2024-09-11T08:54:24.736041+0000","last_fullsized":"2024-09-11T08:54:24.736041+0000","mapping_epoch":315,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:54:08.406695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:08.406695+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:31.897226+0000","scrub_duration":0.001945411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":50,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.743031+0000","last_change":"2024-09-11T08:52:49.567663+0000","last_active":"2024-09-11T08:54:24.743031+0000","last_peered":"2024-09-11T08:54:24.743031+0000","last_clean":"2024-09-11T08:54:24.743031+0000","last_became_active":"2024-09-11T08:52:49.566759+0000","last_became_peered":"2024-09-11T08:52:49.566759+0000","last_unstale":"2024-09-11T08:54:24.743031+0000","last_undegraded":"2024-09-11T08:54:24.743031+0000","last_fullsized":"2024-09-11T08:54:24.743031+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:16.609826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:16.609826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":520,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:26.770492+0000","last_change":"2024-09-11T08:54:02.455288+0000","last_active":"2024-09-11T08:54:26.770492+0000","last_peered":"2024-09-11T08:54:26.770492+0000","last_clean":"2024-09-11T08:54:26.770492+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:54:26.770492+0000","last_undegraded":"2024-09-11T08:54:26.770492+0000","last_fullsized":"2024-09-11T08:54:26.770492+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:54:02.454907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:02.454907+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:26.859984+0000","scrub_duration":0.0078828809999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":314,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735255+0000","last_change":"2024-09-11T08:52:27.235517+0000","last_active":"2024-09-11T08:54:24.735255+0000","last_peered":"2024-09-11T08:54:24.735255+0000","last_clean":"2024-09-11T08:54:24.735255+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:54:24.735255+0000","last_undegraded":"2024-09-11T08:54:24.735255+0000","last_fullsized":"2024-09-11T08:54:24.735255+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:27.235448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:27.235448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017172019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"200'25","reported_seq":359,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.743336+0000","last_change":"2024-09-11T08:52:53.432742+0000","last_active":"2024-09-11T08:54:24.743336+0000","last_peered":"2024-09-11T08:54:24.743336+0000","last_clean":"2024-09-11T08:54:24.743336+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:54:24.743336+0000","last_undegraded":"2024-09-11T08:54:24.743336+0000","last_fullsized":"2024-09-11T08:54:24.743336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:52:53.432685+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:52:53.432685+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001930122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":355,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.745368+0000","last_change":"2024-09-11T08:54:15.235583+0000","last_active":"2024-09-11T08:54:24.745368+0000","last_peered":"2024-09-11T08:54:24.745368+0000","last_clean":"2024-09-11T08:54:24.745368+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:54:24.745368+0000","last_undegraded":"2024-09-11T08:54:24.745368+0000","last_fullsized":"2024-09-11T08:54:24.745368+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:15.235519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:15.235519+0000","last_clean_scrub_stamp":"2024-09-11T08:54:15.235519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:40.532313+0000","scrub_duration":0.0033574619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"277'10","reported_seq":558,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T08:54:26.751182+0000","last_change":"2024-09-11T08:53:53.566688+0000","last_active":"2024-09-11T08:54:26.751182+0000","last_peered":"2024-09-11T08:54:26.751182+0000","last_clean":"2024-09-11T08:54:26.751182+0000","last_became_active":"2024-09-11T08:52:18.585914+0000","last_became_peered":"2024-09-11T08:52:18.585914+0000","last_unstale":"2024-09-11T08:54:26.751182+0000","last_undegraded":"2024-09-11T08:54:26.751182+0000","last_fullsized":"2024-09-11T08:54:26.751182+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'10","last_scrub_stamp":"2024-09-11T08:53:53.566589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:53.566589+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:02.630060+0000","scrub_duration":0.0096769479999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":60,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.743541+0000","last_change":"2024-09-11T08:53:01.446031+0000","last_active":"2024-09-11T08:54:24.743541+0000","last_peered":"2024-09-11T08:54:24.743541+0000","last_clean":"2024-09-11T08:54:24.743541+0000","last_became_active":"2024-09-11T08:52:49.567693+0000","last_became_peered":"2024-09-11T08:52:49.567693+0000","last_unstale":"2024-09-11T08:54:24.743541+0000","last_undegraded":"2024-09-11T08:54:24.743541+0000","last_fullsized":"2024-09-11T08:54:24.743541+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:01.445935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:01.445935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019176729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"200'19","reported_seq":378,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734487+0000","last_change":"2024-09-11T08:54:15.800180+0000","last_active":"2024-09-11T08:54:24.734487+0000","last_peered":"2024-09-11T08:54:24.734487+0000","last_clean":"2024-09-11T08:54:24.734487+0000","last_became_active":"2024-09-11T08:49:16.937185+0000","last_became_peered":"2024-09-11T08:49:16.937185+0000","last_unstale":"2024-09-11T08:54:24.734487+0000","last_undegraded":"2024-09-11T08:54:24.734487+0000","last_fullsized":"2024-09-11T08:54:24.734487+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:54:15.800092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:15.800092+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:44.218489+0000","scrub_duration":0.014286116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":327,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.738461+0000","last_change":"2024-09-11T08:53:21.891151+0000","last_active":"2024-09-11T08:54:24.738461+0000","last_peered":"2024-09-11T08:54:24.738461+0000","last_clean":"2024-09-11T08:54:24.738461+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:54:24.738461+0000","last_undegraded":"2024-09-11T08:54:24.738461+0000","last_fullsized":"2024-09-11T08:54:24.738461+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:21.891073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:21.891073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:51.054430+0000","scrub_duration":0.0024938569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":342,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.751521+0000","last_change":"2024-09-11T08:54:08.814739+0000","last_active":"2024-09-11T08:54:24.751521+0000","last_peered":"2024-09-11T08:54:24.751521+0000","last_clean":"2024-09-11T08:54:24.751521+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:54:24.751521+0000","last_undegraded":"2024-09-11T08:54:24.751521+0000","last_fullsized":"2024-09-11T08:54:24.751521+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:08.814646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:08.814646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:15.264001+0000","scrub_duration":0.0047888419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":340,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742604+0000","last_change":"2024-09-11T08:53:21.391938+0000","last_active":"2024-09-11T08:54:24.742604+0000","last_peered":"2024-09-11T08:54:24.742604+0000","last_clean":"2024-09-11T08:54:24.742604+0000","last_became_active":"2024-09-11T08:52:18.585089+0000","last_became_peered":"2024-09-11T08:52:18.585089+0000","last_unstale":"2024-09-11T08:54:24.742604+0000","last_undegraded":"2024-09-11T08:54:24.742604+0000","last_fullsized":"2024-09-11T08:54:24.742604+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:21.391849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:53:21.391849+0000","last_clean_scrub_stamp":"2024-09-11T08:53:21.391849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:40.478905+0000","scrub_duration":0.003344732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"200'15","reported_seq":372,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.735851+0000","last_change":"2024-09-11T08:53:19.736495+0000","last_active":"2024-09-11T08:54:24.735851+0000","last_peered":"2024-09-11T08:54:24.735851+0000","last_clean":"2024-09-11T08:54:24.735851+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:54:24.735851+0000","last_undegraded":"2024-09-11T08:54:24.735851+0000","last_fullsized":"2024-09-11T08:54:24.735851+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:53:19.736363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:19.736363+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:27.755432+0000","scrub_duration":0.0040364540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":60,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742271+0000","last_change":"2024-09-11T08:53:16.431184+0000","last_active":"2024-09-11T08:54:24.742271+0000","last_peered":"2024-09-11T08:54:24.742271+0000","last_clean":"2024-09-11T08:54:24.742271+0000","last_became_active":"2024-09-11T08:52:38.485748+0000","last_became_peered":"2024-09-11T08:52:38.485748+0000","last_unstale":"2024-09-11T08:54:24.742271+0000","last_undegraded":"2024-09-11T08:54:24.742271+0000","last_fullsized":"2024-09-11T08:54:24.742271+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:16.430996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:21.208684+0000","last_clean_scrub_stamp":"2024-09-11T08:53:16.430996+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051148779999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":342,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742599+0000","last_change":"2024-09-11T08:52:54.441725+0000","last_active":"2024-09-11T08:54:24.742599+0000","last_peered":"2024-09-11T08:54:24.742599+0000","last_clean":"2024-09-11T08:54:24.742599+0000","last_became_active":"2024-09-11T08:52:18.585691+0000","last_became_peered":"2024-09-11T08:52:18.585691+0000","last_unstale":"2024-09-11T08:54:24.742599+0000","last_undegraded":"2024-09-11T08:54:24.742599+0000","last_fullsized":"2024-09-11T08:54:24.742599+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.441666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.441666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012216760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":142,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734430+0000","last_change":"2024-09-11T08:54:07.453666+0000","last_active":"2024-09-11T08:54:24.734430+0000","last_peered":"2024-09-11T08:54:24.734430+0000","last_clean":"2024-09-11T08:54:24.734430+0000","last_became_active":"2024-09-11T08:52:49.566694+0000","last_became_peered":"2024-09-11T08:52:49.566694+0000","last_unstale":"2024-09-11T08:54:24.734430+0000","last_undegraded":"2024-09-11T08:54:24.734430+0000","last_fullsized":"2024-09-11T08:54:24.734430+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:07.453531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:24.401480+0000","last_clean_scrub_stamp":"2024-09-11T08:54:07.453531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:24.530018+0000","scrub_duration":0.0048569140000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"200'19","reported_seq":345,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742409+0000","last_change":"2024-09-11T08:51:21.423287+0000","last_active":"2024-09-11T08:54:24.742409+0000","last_peered":"2024-09-11T08:54:24.742409+0000","last_clean":"2024-09-11T08:54:24.742409+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:54:24.742409+0000","last_undegraded":"2024-09-11T08:54:24.742409+0000","last_fullsized":"2024-09-11T08:54:24.742409+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:51:21.423213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:51:21.423213+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086807859999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":336,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.736941+0000","last_change":"2024-09-11T08:53:04.734505+0000","last_active":"2024-09-11T08:54:24.736941+0000","last_peered":"2024-09-11T08:54:24.736941+0000","last_clean":"2024-09-11T08:54:24.736941+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:54:24.736941+0000","last_undegraded":"2024-09-11T08:54:24.736941+0000","last_fullsized":"2024-09-11T08:54:24.736941+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:04.734452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:04.734452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001307342,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":354,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.743873+0000","last_change":"2024-09-11T08:54:09.162741+0000","last_active":"2024-09-11T08:54:24.743873+0000","last_peered":"2024-09-11T08:54:24.743873+0000","last_clean":"2024-09-11T08:54:24.743873+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:54:24.743873+0000","last_undegraded":"2024-09-11T08:54:24.743873+0000","last_fullsized":"2024-09-11T08:54:24.743873+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:09.162547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.162547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:27.715851+0000","scrub_duration":0.0020448530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":360,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742720+0000","last_change":"2024-09-11T08:54:08.525090+0000","last_active":"2024-09-11T08:54:24.742720+0000","last_peered":"2024-09-11T08:54:24.742720+0000","last_clean":"2024-09-11T08:54:24.742720+0000","last_became_active":"2024-09-11T08:52:18.585315+0000","last_became_peered":"2024-09-11T08:52:18.585315+0000","last_unstale":"2024-09-11T08:54:24.742720+0000","last_undegraded":"2024-09-11T08:54:24.742720+0000","last_fullsized":"2024-09-11T08:54:24.742720+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:08.524974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:25.255332+0000","last_clean_scrub_stamp":"2024-09-11T08:54:08.524974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:26.888865+0000","scrub_duration":0.0080002040000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"200'10","reported_seq":373,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742118+0000","last_change":"2024-09-11T08:52:11.431872+0000","last_active":"2024-09-11T08:54:24.742118+0000","last_peered":"2024-09-11T08:54:24.742118+0000","last_clean":"2024-09-11T08:54:24.742118+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:54:24.742118+0000","last_undegraded":"2024-09-11T08:54:24.742118+0000","last_fullsized":"2024-09-11T08:54:24.742118+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'10","last_scrub_stamp":"2024-09-11T08:52:11.431805+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:52:11.431805+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013697399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":154,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.736989+0000","last_change":"2024-09-11T08:54:02.427838+0000","last_active":"2024-09-11T08:54:24.736989+0000","last_peered":"2024-09-11T08:54:24.736989+0000","last_clean":"2024-09-11T08:54:24.736989+0000","last_became_active":"2024-09-11T08:52:49.569954+0000","last_became_peered":"2024-09-11T08:52:49.569954+0000","last_unstale":"2024-09-11T08:54:24.736989+0000","last_undegraded":"2024-09-11T08:54:24.736989+0000","last_fullsized":"2024-09-11T08:54:24.736989+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:54:02.427745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:54:02.427745+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:15.894988+0000","scrub_duration":0.0045687649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":205,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.740110+0000","last_change":"2024-09-11T08:53:14.382649+0000","last_active":"2024-09-11T08:54:24.740110+0000","last_peered":"2024-09-11T08:54:24.740110+0000","last_clean":"2024-09-11T08:54:24.740110+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:54:24.740110+0000","last_undegraded":"2024-09-11T08:54:24.740110+0000","last_fullsized":"2024-09-11T08:54:24.740110+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:14.382556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:14.382556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028503370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.734029+0000","last_change":"2024-09-11T08:54:17.155339+0000","last_active":"2024-09-11T08:54:24.734029+0000","last_peered":"2024-09-11T08:54:24.734029+0000","last_clean":"2024-09-11T08:54:24.734029+0000","last_became_active":"2024-09-11T08:54:17.154933+0000","last_became_peered":"2024-09-11T08:54:17.154933+0000","last_unstale":"2024-09-11T08:54:24.734029+0000","last_undegraded":"2024-09-11T08:54:24.734029+0000","last_fullsized":"2024-09-11T08:54:24.734029+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:11.694784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:30.788303+0000","last_clean_scrub_stamp":"2024-09-11T08:54:11.694784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:16.882580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":350,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.751372+0000","last_change":"2024-09-11T08:54:11.861507+0000","last_active":"2024-09-11T08:54:24.751372+0000","last_peered":"2024-09-11T08:54:24.751372+0000","last_clean":"2024-09-11T08:54:24.751372+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:54:24.751372+0000","last_undegraded":"2024-09-11T08:54:24.751372+0000","last_fullsized":"2024-09-11T08:54:24.751372+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:11.861416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:11.861416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:35.847712+0000","scrub_duration":0.0055713560000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":356,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.742341+0000","last_change":"2024-09-11T08:54:13.701850+0000","last_active":"2024-09-11T08:54:24.742341+0000","last_peered":"2024-09-11T08:54:24.742341+0000","last_clean":"2024-09-11T08:54:24.742341+0000","last_became_active":"2024-09-11T08:52:18.595146+0000","last_became_peered":"2024-09-11T08:52:18.595146+0000","last_unstale":"2024-09-11T08:54:24.742341+0000","last_undegraded":"2024-09-11T08:54:24.742341+0000","last_fullsized":"2024-09-11T08:54:24.742341+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:13.701753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:32.393197+0000","last_clean_scrub_stamp":"2024-09-11T08:54:13.701753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:32.313595+0000","scrub_duration":0.113734606,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"200'19","reported_seq":229,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.741080+0000","last_change":"2024-09-11T08:53:52.564728+0000","last_active":"2024-09-11T08:54:24.741080+0000","last_peered":"2024-09-11T08:54:24.741080+0000","last_clean":"2024-09-11T08:54:24.741080+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:54:24.741080+0000","last_undegraded":"2024-09-11T08:54:24.741080+0000","last_fullsized":"2024-09-11T08:54:24.741080+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:53:52.564594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:52.564594+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:09.458091+0000","scrub_duration":0.0020268980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":141,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T08:54:24.744431+0000","last_change":"2024-09-11T08:52:53.465187+0000","last_active":"2024-09-11T08:54:24.744431+0000","last_peered":"2024-09-11T08:54:24.744431+0000","last_clean":"2024-09-11T08:54:24.744431+0000","last_became_active":"2024-09-11T08:52:49.568537+0000","last_became_peered":"2024-09-11T08:52:49.568537+0000","last_unstale":"2024-09-11T08:54:24.744431+0000","last_undegraded":"2024-09-11T08:54:24.744431+0000","last_fullsized":"2024-09-11T08:54:24.744431+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:53.465128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:53.465128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00141025,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":8735490048,"num_objects":133293,"num_object_clones":0,"num_object_copies":399879,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":133293,"num_write_kb":8530752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":26218201088,"data_stored":26218201088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82681,"ondisk_log_size":82681,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1118,"num_read_kb":875,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":4224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":769,"ondisk_log_size":769,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":697708,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":42,"num_write":21,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":214679,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299588,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3026404,"kb_used_data":2129180,"kb_used_omap":1,"kb_used_meta":897214,"kb_avail":90718748,"statfs":{"total":95995035648,"available":92895997952,"internally_reserved":0,"allocated":2180280320,"data_stored":2179956814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":918747480},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.80100000000000005,"15min":0.94499999999999995},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.601,"5min":4.8730000000000002,"15min":32.171999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.84299999999999997,"15min":1.044},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.21199999999999999},"max":{"1min":1.276,"5min":4.0069999999999997,"15min":32.149999999999999},"last":1.194}]},{"osd":1,"last update":"Wed Sep 11 08:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.92900000000000005,"15min":1.123},"min":{"1min":0.26400000000000001,"5min":0.23799999999999999,"15min":0.224},"max":{"1min":1.5449999999999999,"5min":4.8970000000000002,"15min":33.692999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.91200000000000003,"15min":1.0349999999999999},"min":{"1min":0.307,"5min":0.249,"15min":0.23300000000000001},"max":{"1min":1.2090000000000001,"5min":3.9780000000000002,"15min":35.591000000000001},"last":1.23}]},{"osd":2,"last update":"Wed Sep 11 08:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.022,"15min":0.95999999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.249},"max":{"1min":2.4049999999999998,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.97799999999999998,"15min":0.91800000000000004},"min":{"1min":0.38,"5min":0.27900000000000003,"15min":0.25800000000000001},"max":{"1min":2.7509999999999999,"5min":7.5259999999999998,"15min":7.5259999999999998},"last":0.58299999999999996}]},{"osd":3,"last update":"Wed Sep 11 08:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":1.6950000000000001,"15min":1.212},"min":{"1min":0.316,"5min":0.28699999999999998,"15min":0.25},"max":{"1min":2.3759999999999999,"5min":56.265999999999998,"15min":56.265999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":1.619,"15min":1.2649999999999999},"min":{"1min":0.36599999999999999,"5min":0.34499999999999997,"15min":0.27400000000000002},"max":{"1min":1.5720000000000001,"5min":57.207000000000001,"15min":57.207000000000001},"last":0.77000000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":1.018,"15min":0.68100000000000005},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23400000000000001},"max":{"1min":2.5089999999999999,"5min":7.1520000000000001,"15min":7.1520000000000001},"last":1.2889999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.085,"15min":0.74299999999999999},"min":{"1min":0.35199999999999998,"5min":0.27900000000000003,"15min":0.216},"max":{"1min":2.6110000000000002,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":0.35699999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.98799999999999999,"15min":0.73499999999999999},"min":{"1min":0.33300000000000002,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":2.9060000000000001,"5min":5.0110000000000001,"15min":5.1059999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":1.1279999999999999,"15min":0.79700000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.26700000000000002},"max":{"1min":2.956,"5min":6.3650000000000002,"15min":6.3650000000000002},"last":0.40000000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:53:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":1.052,"15min":0.77600000000000002},"min":{"1min":0.40300000000000002,"5min":0.34300000000000003,"15min":0.315},"max":{"1min":2.3069999999999999,"5min":4.5449999999999999,"15min":4.5449999999999999},"last":1.27},{"interface":"front","average":{"1min":0.72399999999999998,"5min":1.1339999999999999,"15min":0.81299999999999994},"min":{"1min":0.34599999999999997,"5min":0.33300000000000002,"15min":0.27900000000000003},"max":{"1min":2.4329999999999998,"5min":5.0620000000000003,"15min":5.0620000000000003},"last":1.6180000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528521,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4302172,"kb_used_data":3712276,"kb_used_omap":1,"kb_used_meta":589886,"kb_avail":89442980,"statfs":{"total":95995035648,"available":91589611520,"internally_reserved":0,"allocated":3801370624,"data_stored":3801055256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":604043608},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":6.7320000000000002,"5min":2.2719999999999998,"15min":1.5680000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.217},"max":{"1min":121.208,"5min":121.208,"15min":121.208},"last":0.53800000000000003},{"interface":"front","average":{"1min":6.8529999999999998,"5min":2.4550000000000001,"15min":1.5669999999999999},"min":{"1min":0.44800000000000001,"5min":0.24399999999999999,"15min":0.23300000000000001},"max":{"1min":121.05200000000001,"5min":121.05200000000001,"15min":121.05200000000001},"last":0.32900000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":1.3220000000000001,"15min":1.173},"min":{"1min":0.247,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.4159999999999999,"5min":27.984000000000002,"15min":27.984000000000002},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":1.327,"15min":1.1299999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.253},"max":{"1min":2.1520000000000001,"5min":28.106000000000002,"15min":28.106000000000002},"last":0.59599999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.72199999999999998,"15min":0.82699999999999996},"min":{"1min":0.32200000000000001,"5min":0.27500000000000002,"15min":0.23200000000000001},"max":{"1min":1.4570000000000001,"5min":3.899,"15min":14.313000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.82899999999999996,"15min":0.876},"min":{"1min":0.42499999999999999,"5min":0.27400000000000002,"15min":0.251},"max":{"1min":1.909,"5min":4.9020000000000001,"15min":6.5979999999999999},"last":2.7509999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":1.357,"15min":1.206},"min":{"1min":0.27200000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.4550000000000001,"5min":27.795000000000002,"15min":27.795000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":1.3959999999999999,"15min":1.194},"min":{"1min":0.309,"5min":0.30499999999999999,"15min":0.224},"max":{"1min":1.9730000000000001,"5min":27.841999999999999,"15min":27.841999999999999},"last":0.66200000000000003}]},{"osd":4,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.325,"15min":0.78100000000000003},"min":{"1min":0.27400000000000002,"5min":0.24299999999999999,"15min":0.20799999999999999},"max":{"1min":5.8689999999999998,"5min":40.927999999999997,"15min":40.927999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":1.337,"15min":0.78800000000000003},"min":{"1min":0.308,"5min":0.27700000000000002,"15min":0.26400000000000001},"max":{"1min":2.5209999999999999,"5min":40.706000000000003,"15min":40.706000000000003},"last":0.29599999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.5029999999999999,"15min":0.91100000000000003},"min":{"1min":0.311,"5min":0.27600000000000002,"15min":0.23899999999999999},"max":{"1min":3.8540000000000001,"5min":40.847000000000001,"15min":40.847000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.3759999999999999,"15min":0.85799999999999998},"min":{"1min":0.29899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.758,"5min":41.401000000000003,"15min":41.401000000000003},"last":0.46800000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:53:53 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":1.635,"15min":0.95599999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.247},"max":{"1min":1.427,"5min":40.426000000000002,"15min":40.426000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":1.657,"15min":0.95399999999999996},"min":{"1min":0.372,"5min":0.35199999999999998,"15min":0.27700000000000002},"max":{"1min":1.484,"5min":45.401000000000003,"15min":45.401000000000003},"last":0.623}]}]},{"osd":5,"up_from":38,"seq":163208757453,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3541552,"kb_used_data":2651752,"kb_used_omap":1,"kb_used_meta":889790,"kb_avail":90203600,"statfs":{"total":95995035648,"available":92368486400,"internally_reserved":0,"allocated":2715394048,"data_stored":2715059651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":911145304},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.145,"15min":0.96299999999999997},"min":{"1min":0.222,"5min":0.221,"15min":0.20599999999999999},"max":{"1min":3.7040000000000002,"5min":8.7349999999999994,"15min":8.7349999999999994},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.96799999999999997,"15min":1.0529999999999999},"min":{"1min":0.26800000000000002,"5min":0.23899999999999999,"15min":0.23100000000000001},"max":{"1min":2.1120000000000001,"5min":5.2560000000000002,"15min":8.0410000000000004},"last":0.45800000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.0640000000000001,"15min":1.046},"min":{"1min":0.252,"5min":0.216,"15min":0.216},"max":{"1min":3.1539999999999999,"5min":5.3280000000000003,"15min":8},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":1.1679999999999999,"15min":1.038},"min":{"1min":0.26900000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":2.4260000000000002,"5min":12.173,"15min":12.173},"last":0.41999999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.147,"15min":1.0529999999999999},"min":{"1min":0.27600000000000002,"5min":0.253,"15min":0.23799999999999999},"max":{"1min":3.984,"5min":8.7789999999999999,"15min":8.7789999999999999},"last":0.436},{"interface":"front","average":{"1min":1.375,"5min":1.093,"15min":0.99099999999999999},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":4.7069999999999999,"5min":5.2329999999999997,"15min":8.0640000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":2.1280000000000001,"15min":1.337},"min":{"1min":0.314,"5min":0.30099999999999999,"15min":0.26900000000000002},"max":{"1min":5.9779999999999998,"5min":85.299999999999997,"15min":85.299999999999997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":2.0579999999999998,"15min":1.4490000000000001},"min":{"1min":0.33000000000000002,"5min":0.29099999999999998,"15min":0.27400000000000002},"max":{"1min":2.5750000000000002,"5min":85.941000000000003,"15min":85.941000000000003},"last":0.69399999999999995}]},{"osd":4,"last update":"Wed Sep 11 08:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":1.131,"15min":0.754},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.20699999999999999},"max":{"1min":1.778,"5min":16.021999999999998,"15min":16.021999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.81899999999999995,"5min":1.0700000000000001,"15min":0.71299999999999997},"min":{"1min":0.29499999999999998,"5min":0.28999999999999998,"15min":0.20499999999999999},"max":{"1min":3.6850000000000001,"5min":9.6140000000000008,"15min":9.6140000000000008},"last":0.73699999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:53:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":1.1259999999999999,"15min":0.76500000000000001},"min":{"1min":0.33800000000000002,"5min":0.30399999999999999,"15min":0.23599999999999999},"max":{"1min":1.855,"5min":8.8059999999999992,"15min":8.8059999999999992},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":1.0469999999999999,"15min":0.72799999999999998},"min":{"1min":0.27400000000000002,"5min":0.27300000000000002,"15min":0.251},"max":{"1min":3.2170000000000001,"5min":5.3090000000000002,"15min":8.1920000000000002},"last":0.51200000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:53:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":1.371,"15min":1.115},"min":{"1min":0.33300000000000002,"5min":0.30099999999999999,"15min":0.27800000000000002},"max":{"1min":1.804,"5min":16.062999999999999,"15min":88.513000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.624,"5min":1.304,"15min":1.105},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.29599999999999999},"max":{"1min":1.8740000000000001,"5min":9.7089999999999996,"15min":86.963999999999999},"last":0.52800000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986385,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4819560,"kb_used_data":4221344,"kb_used_omap":1,"kb_used_meta":598206,"kb_avail":88925592,"statfs":{"total":95995035648,"available":91059806208,"internally_reserved":0,"allocated":4322656256,"data_stored":4322328226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":612563288},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.81000000000000005,"15min":1.6539999999999999},"min":{"1min":0.247,"5min":0.22600000000000001,"15min":0.20399999999999999},"max":{"1min":1.8839999999999999,"5min":5.2400000000000002,"15min":89.510000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":1.1930000000000001,"5min":0.86499999999999999,"15min":1.7609999999999999},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.22600000000000001},"max":{"1min":4.048,"5min":5.3220000000000001,"15min":89.765000000000001},"last":1.7649999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":1.7490000000000001,"15min":1.4770000000000001},"min":{"1min":0.28299999999999997,"5min":0.27400000000000002,"15min":0.24399999999999999},"max":{"1min":3.9279999999999999,"5min":48.976999999999997,"15min":48.976999999999997},"last":1.25},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.627,"15min":1.4199999999999999},"min":{"1min":0.28699999999999998,"5min":0.23499999999999999,"15min":0.22800000000000001},"max":{"1min":1.968,"5min":45.787999999999997,"15min":45.787999999999997},"last":0.44500000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":3.8650000000000002,"15min":1.901},"min":{"1min":0.28499999999999998,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.9419999999999999,"5min":255.88900000000001,"15min":255.88900000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":3.7309999999999999,"15min":1.8680000000000001},"min":{"1min":0.36199999999999999,"5min":0.29199999999999998,"15min":0.25800000000000001},"max":{"1min":1.754,"5min":255.41200000000001,"15min":255.41200000000001},"last":3.742}]},{"osd":3,"last update":"Wed Sep 11 08:53:50 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.5920000000000001,"15min":2.004},"min":{"1min":0.33100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":3.9449999999999998,"5min":44.052,"15min":89.881},"last":3.7810000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":1.4690000000000001,"15min":1.9299999999999999},"min":{"1min":0.36399999999999999,"5min":0.318,"15min":0.25900000000000001},"max":{"1min":1.9950000000000001,"5min":43.646000000000001,"15min":89.715000000000003},"last":1.2849999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:54:11 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.056,"15min":0.74399999999999999},"min":{"1min":0.34599999999999997,"5min":0.25900000000000001,"15min":0.23999999999999999},"max":{"1min":3.3650000000000002,"5min":11.045999999999999,"15min":11.045999999999999},"last":1.3109999999999999},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.103,"15min":0.73699999999999999},"min":{"1min":0.27000000000000002,"5min":0.254,"15min":0.24199999999999999},"max":{"1min":2.6190000000000002,"5min":10.526999999999999,"15min":10.526999999999999},"last":0.36899999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:53:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.88100000000000001,"15min":0.63},"min":{"1min":0.31900000000000001,"5min":0.27300000000000002,"15min":0.22900000000000001},"max":{"1min":1.6399999999999999,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":3.8100000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.85399999999999998,"15min":0.67900000000000005},"min":{"1min":0.32700000000000001,"5min":0.27800000000000002,"15min":0.221},"max":{"1min":2.6299999999999999,"5min":6.2619999999999996,"15min":6.2619999999999996},"last":2.5590000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:53:50 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.91700000000000004,"15min":0.68100000000000005},"min":{"1min":0.35299999999999998,"5min":0.253,"15min":0.252},"max":{"1min":2.2069999999999999,"5min":4.8079999999999998,"15min":4.8079999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":1.034,"15min":0.76400000000000001},"min":{"1min":0.34599999999999997,"5min":0.32300000000000001,"15min":0.28799999999999998},"max":{"1min":3.0659999999999998,"5min":5.0199999999999996,"15min":5.1970000000000001},"last":1.7330000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149911,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3567492,"kb_used_data":2659964,"kb_used_omap":1,"kb_used_meta":907518,"kb_avail":90177660,"statfs":{"total":95995035648,"available":92341923840,"internally_reserved":0,"allocated":2723803136,"data_stored":2723448559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":929298776},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.66400000000000003,"15min":1.1100000000000001},"min":{"1min":0.26400000000000001,"5min":0.20100000000000001,"15min":0.16500000000000001},"max":{"1min":2.8639999999999999,"5min":3.0310000000000001,"15min":24.280000000000001},"last":1.278},{"interface":"front","average":{"1min":1.125,"5min":0.79200000000000004,"15min":1.3340000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.17000000000000001},"max":{"1min":6.0780000000000003,"5min":6.0780000000000003,"15min":22.225000000000001},"last":3.0950000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.98299999999999998,"15min":1.7},"min":{"1min":0.25700000000000001,"5min":0.20100000000000001,"15min":0.19800000000000001},"max":{"1min":2.2549999999999999,"5min":7.4909999999999997,"15min":115.601},"last":4.7409999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.879,"15min":1.581},"min":{"1min":0.22900000000000001,"5min":0.20499999999999999,"15min":0.17999999999999999},"max":{"1min":1.4450000000000001,"5min":6.6639999999999997,"15min":115.474},"last":0.81699999999999995}]},{"osd":2,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":1.694,"15min":1.4850000000000001},"min":{"1min":0.26000000000000001,"5min":0.24299999999999999,"15min":0.23200000000000001},"max":{"1min":2.625,"5min":86.766000000000005,"15min":86.766000000000005},"last":2.9359999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":1.754,"15min":1.4550000000000001},"min":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.20799999999999999},"max":{"1min":3.5760000000000001,"5min":85.367999999999995,"15min":85.367999999999995},"last":0.90400000000000003}]},{"osd":4,"last update":"Wed Sep 11 08:53:43 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":0.99199999999999999,"15min":1.2070000000000001},"min":{"1min":0.41799999999999998,"5min":0.315,"15min":0.26700000000000002},"max":{"1min":2.4740000000000002,"5min":4.843,"15min":8.9969999999999999},"last":2.9009999999999998},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.90700000000000003,"15min":1.022},"min":{"1min":0.373,"5min":0.29099999999999998,"15min":0.27700000000000002},"max":{"1min":2.7690000000000001,"5min":2.8559999999999999,"15min":7.234},"last":1.159}]},{"osd":5,"last update":"Wed Sep 11 08:54:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.0549999999999999,"15min":1.2070000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.28299999999999997},"max":{"1min":3.1720000000000002,"5min":3.391,"15min":11.526999999999999},"last":1.036},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.077,"15min":1.1060000000000001},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.29399999999999998},"max":{"1min":2.3199999999999998,"5min":4.8179999999999996,"15min":7.1689999999999996},"last":1.1890000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":1.4239999999999999,"15min":1.3560000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.6970000000000001,"5min":33.106999999999999,"15min":33.106999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":1.4019999999999999,"15min":1.335},"min":{"1min":0.371,"5min":0.36499999999999999,"15min":0.311},"max":{"1min":3.2730000000000001,"5min":33.076000000000001,"15min":33.076000000000001},"last":3.0659999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:53:55 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.647,"15min":1.4630000000000001},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.35299999999999998},"max":{"1min":3.161,"5min":35.073,"15min":35.073},"last":2.9580000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":1.6200000000000001,"15min":1.3180000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.34699999999999998},"max":{"1min":2.5150000000000001,"5min":34.844999999999999,"15min":34.844999999999999},"last":1.7809999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411547,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3796192,"kb_used_data":3206296,"kb_used_omap":1,"kb_used_meta":589886,"kb_avail":89948960,"statfs":{"total":95995035648,"available":92107735040,"internally_reserved":0,"allocated":3283247104,"data_stored":3282927741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":604043608},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:54:08 2024","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":0.69599999999999995,"15min":1.6060000000000001},"min":{"1min":0.20599999999999999,"5min":0.191,"15min":0.159},"max":{"1min":3.5830000000000002,"5min":3.5830000000000002,"15min":34.713999999999999},"last":1.702},{"interface":"front","average":{"1min":1.9470000000000001,"5min":0.745,"15min":1.718},"min":{"1min":0.191,"5min":0.191,"15min":0.17399999999999999},"max":{"1min":5.9489999999999998,"5min":5.9489999999999998,"15min":34.399000000000001},"last":0.50900000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:54:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":0.94999999999999996,"15min":1.718},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":5.9850000000000003,"5min":11.552,"15min":72.867000000000004},"last":1.667},{"interface":"front","average":{"1min":1.3600000000000001,"5min":0.98199999999999998,"15min":1.827},"min":{"1min":0.24099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.8570000000000002,"5min":11.616,"15min":67.799000000000007},"last":0.26800000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:54:16 2024","interfaces":[{"interface":"back","average":{"1min":1.774,"5min":1.1160000000000001,"15min":1.6459999999999999},"min":{"1min":0.24099999999999999,"5min":0.186,"15min":0.186},"max":{"1min":6.0369999999999999,"5min":11.878,"15min":34.75},"last":0.20999999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.056,"15min":1.5620000000000001},"min":{"1min":0.22800000000000001,"5min":0.216,"15min":0.216},"max":{"1min":5.3319999999999999,"5min":9.2759999999999998,"15min":32.802},"last":1.4410000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:53:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.502,"15min":1.238},"min":{"1min":0.38,"5min":0.28699999999999998,"15min":0.255},"max":{"1min":6.0659999999999998,"5min":52.113999999999997,"15min":52.113999999999997},"last":1.895},{"interface":"front","average":{"1min":1.284,"5min":1.4179999999999999,"15min":1.177},"min":{"1min":0.42199999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":5.5780000000000003,"5min":52.097999999999999,"15min":52.097999999999999},"last":0.66100000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:54:08 2024","interfaces":[{"interface":"back","average":{"1min":2.1549999999999998,"5min":1.631,"15min":1.266},"min":{"1min":0.42099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":5.8860000000000001,"5min":52.145000000000003,"15min":52.145000000000003},"last":2.5539999999999998},{"interface":"front","average":{"1min":1.105,"5min":1.3620000000000001,"15min":1.079},"min":{"1min":0.33300000000000002,"5min":0.29799999999999999,"15min":0.28399999999999997},"max":{"1min":3.2789999999999999,"5min":52.216000000000001,"15min":52.216000000000001},"last":2.1869999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:53:32 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":1.417,"15min":1.262},"min":{"1min":0.34000000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.8640000000000001,"5min":52.235999999999997,"15min":52.235999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":1.323,"5min":1.516,"15min":1.1930000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":5.2939999999999996,"5min":52.186,"15min":52.186},"last":1.6839999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:53:45 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.7629999999999999,"15min":1.2390000000000001},"min":{"1min":0.39100000000000001,"5min":0.34399999999999997,"15min":0.30399999999999999},"max":{"1min":2.9630000000000001,"5min":84.108999999999995,"15min":84.108999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.149,"5min":1.929,"15min":1.4339999999999999},"min":{"1min":0.46200000000000002,"5min":0.39500000000000002,"15min":0.30299999999999999},"max":{"1min":5.9199999999999999,"5min":85.938000000000002,"15min":85.938000000000002},"last":0.36799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575072,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2487100,"kb_used_data":1591732,"kb_used_omap":1,"kb_used_meta":895358,"kb_avail":91258052,"statfs":{"total":95995035648,"available":93448245248,"internally_reserved":0,"allocated":1629933568,"data_stored":1629585798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":916846936},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:53:46 2024","interfaces":[{"interface":"back","average":{"1min":3.7490000000000001,"5min":1.3149999999999999,"15min":1.6890000000000001},"min":{"1min":0.246,"5min":0.17499999999999999,"15min":0.16600000000000001},"max":{"1min":56.613,"5min":56.613,"15min":117.01600000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":4.3209999999999997,"5min":1.4099999999999999,"15min":2.0089999999999999},"min":{"1min":0.23899999999999999,"5min":0.20799999999999999,"15min":0.17499999999999999},"max":{"1min":55.533999999999999,"5min":55.533999999999999,"15min":117.62},"last":0.223}]},{"osd":2,"last update":"Wed Sep 11 08:54:11 2024","interfaces":[{"interface":"back","average":{"1min":4.3979999999999997,"5min":1.4630000000000001,"15min":1.4370000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":61.027999999999999,"5min":61.027999999999999,"15min":61.027999999999999},"last":1.659},{"interface":"front","average":{"1min":3.923,"5min":1.407,"15min":1.4179999999999999},"min":{"1min":0.22,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":56.130000000000003,"5min":56.130000000000003,"15min":56.130000000000003},"last":0.36199999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:54:25 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0609999999999999,"15min":1.579},"min":{"1min":0.23300000000000001,"5min":0.223,"15min":0.191},"max":{"1min":3.0680000000000001,"5min":5.1859999999999999,"15min":116.98},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.92200000000000004,"15min":1.6200000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.22},"max":{"1min":5.9029999999999996,"5min":5.9029999999999996,"15min":117.56699999999999},"last":1.7569999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:53:46 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.224,"15min":1.101},"min":{"1min":0.35399999999999998,"5min":0.28699999999999998,"15min":0.26200000000000001},"max":{"1min":7.9450000000000003,"5min":7.9450000000000003,"15min":11.19},"last":2.198},{"interface":"front","average":{"1min":0.89000000000000001,"5min":1.0409999999999999,"15min":0.98699999999999999},"min":{"1min":0.29899999999999999,"5min":0.28399999999999997,"15min":0.251},"max":{"1min":2.5830000000000002,"5min":6.952,"15min":7.7169999999999996},"last":0.71599999999999997}]},{"osd":5,"last update":"Wed Sep 11 08:54:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.206,"15min":1.6040000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.27600000000000002},"max":{"1min":3.1440000000000001,"5min":5.9880000000000004,"15min":103.337},"last":0.59999999999999998},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.333,"15min":1.6000000000000001},"min":{"1min":0.38300000000000001,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":8.3810000000000002,"5min":8.3810000000000002,"15min":103.163},"last":2.1629999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:53:37 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.1539999999999999,"15min":1.028},"min":{"1min":0.317,"5min":0.30499999999999999,"15min":0.29399999999999998},"max":{"1min":2.1440000000000001,"5min":6.2249999999999996,"15min":8.9149999999999991},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.1819999999999999,"15min":1.1830000000000001},"min":{"1min":0.41099999999999998,"5min":0.315,"15min":0.31},"max":{"1min":2.6520000000000001,"5min":6.2869999999999999,"15min":9.5199999999999996},"last":1.0649999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:53:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1739999999999999,"15min":1.127},"min":{"1min":0.41099999999999998,"5min":0.37,"15min":0.32800000000000001},"max":{"1min":8.3390000000000004,"5min":8.3390000000000004,"15min":11.315},"last":2.125},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.1299999999999999,"15min":1.079},"min":{"1min":0.373,"5min":0.373,"15min":0.315},"max":{"1min":2.206,"5min":5.9089999999999998,"15min":10.266},"last":2.2349999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738596,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5930732,"kb_used_data":5328356,"kb_used_omap":1,"kb_used_meta":602366,"kb_avail":87814420,"statfs":{"total":95995035648,"available":89921966080,"internally_reserved":0,"allocated":5456236544,"data_stored":5455905015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":616823128},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:53:42 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":1.5920000000000001,"15min":1.4790000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.16300000000000001},"max":{"1min":2.234,"5min":56.685000000000002,"15min":56.685000000000002},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.6970000000000001,"15min":1.444},"min":{"1min":0.221,"5min":0.217,"15min":0.186},"max":{"1min":2.4830000000000001,"5min":61.012999999999998,"15min":61.012999999999998},"last":2.0529999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:54:10 2024","interfaces":[{"interface":"back","average":{"1min":1.7250000000000001,"5min":0.85299999999999998,"15min":1.2310000000000001},"min":{"1min":0.27800000000000002,"5min":0.223,"15min":0.21099999999999999},"max":{"1min":8.1059999999999999,"5min":8.1059999999999999,"15min":19.940000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":0.89300000000000002,"15min":1.302},"min":{"1min":0.34000000000000002,"5min":0.22,"15min":0.20000000000000001},"max":{"1min":5.4279999999999999,"5min":5.4279999999999999,"15min":20.757000000000001},"last":0.81599999999999995}]},{"osd":3,"last update":"Wed Sep 11 08:54:31 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.657,"15min":1.6200000000000001},"min":{"1min":0.33300000000000002,"5min":0.218,"15min":0.19800000000000001},"max":{"1min":4.0039999999999996,"5min":56.095999999999997,"15min":56.095999999999997},"last":1.607},{"interface":"front","average":{"1min":1.53,"5min":1.5860000000000001,"15min":1.554},"min":{"1min":0.26400000000000001,"5min":0.21199999999999999,"15min":0.20899999999999999},"max":{"1min":5.4770000000000003,"5min":59.651000000000003,"15min":59.651000000000003},"last":1.4590000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:53:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":0.85999999999999999,"15min":1.119},"min":{"1min":0.504,"5min":0.30099999999999999,"15min":0.25},"max":{"1min":3.109,"5min":3.109,"15min":7.4690000000000003},"last":5.117},{"interface":"front","average":{"1min":1.3480000000000001,"5min":0.93400000000000005,"15min":1.1060000000000001},"min":{"1min":0.44700000000000001,"5min":0.316,"15min":0.255},"max":{"1min":3.0019999999999998,"5min":3.8050000000000002,"15min":7.4630000000000001},"last":1.3600000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:54:10 2024","interfaces":[{"interface":"back","average":{"1min":1.659,"5min":1.1160000000000001,"15min":1.228},"min":{"1min":0.436,"5min":0.316,"15min":0.27800000000000002},"max":{"1min":5.3499999999999996,"5min":5.3550000000000004,"15min":7.4009999999999998},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.764,"5min":1.1759999999999999,"15min":1.2609999999999999},"min":{"1min":0.42499999999999999,"5min":0.34300000000000003,"15min":0.28299999999999997},"max":{"1min":7.6689999999999996,"5min":7.6689999999999996,"15min":7.6689999999999996},"last":5.0819999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:54:31 2024","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.171,"15min":1.2989999999999999},"min":{"1min":0.41999999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":5.399,"5min":5.399,"15min":7.4240000000000004},"last":2.9609999999999999},{"interface":"front","average":{"1min":1.754,"5min":1.1559999999999999,"15min":1.319},"min":{"1min":0.45700000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.6900000000000004,"5min":5.6900000000000004,"15min":7.4489999999999998},"last":1.3109999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:53:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.774,"15min":1.462},"min":{"1min":0.47699999999999998,"5min":0.33500000000000002,"15min":0.313},"max":{"1min":3.0659999999999998,"5min":56.119,"15min":56.119},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.9590000000000001,"15min":1.54},"min":{"1min":0.502,"5min":0.36399999999999999,"15min":0.307},"max":{"1min":5.5019999999999998,"5min":56.100000000000001,"15min":56.100000000000001},"last":1.1080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5481562112,"data_stored":5481562112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1643184128,"data_stored":1643184128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3286106112,"data_stored":3286106112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2731999232,"data_stored":2731999232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4342677504,"data_stored":4342677504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2721251328,"data_stored":2721251328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3818651648,"data_stored":3818651648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2192769024,"data_stored":2192769024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:54:37.988 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-11T08:54:37.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:38.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:38.020 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:38.020 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:38.121 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:38.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:38.154 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:38.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:38.255 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:38.300 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:38.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:38.302 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:38.403 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:38.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:38.447 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:38.447 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:38.548 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:38.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:38.590 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:38.590 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:38.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:38 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:38 smithi145 ceph-mon[33384]: pgmap v720: 158 pgs: 158 active+clean; 8.2 GiB data, 31 GiB used, 685 GiB / 715 GiB avail; 1.1 KiB/s rd, 63 MiB/s wr, 1.21k op/s 2024-09-11T08:54:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:38 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:54:38.691 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:38.697 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:38 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:38.697 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:38 smithi092 ceph-mon[29647]: pgmap v720: 158 pgs: 158 active+clean; 8.2 GiB data, 31 GiB used, 685 GiB / 715 GiB avail; 1.1 KiB/s rd, 63 MiB/s wr, 1.21k op/s 2024-09-11T08:54:38.697 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:38 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:54:38.698 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:38 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:38.698 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:38 smithi092 ceph-mon[41114]: pgmap v720: 158 pgs: 158 active+clean; 8.2 GiB data, 31 GiB used, 685 GiB / 715 GiB avail; 1.1 KiB/s rd, 63 MiB/s wr, 1.21k op/s 2024-09-11T08:54:38.698 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:38 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:54:38.723 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:38.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:38.724 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:38.743 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:54:38.743 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:54:38.826 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:38.871 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:38.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:38.872 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:38.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:39.018 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:39.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:39.020 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:39.120 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:39.149 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:39.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:39.150 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:39.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:39.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:39.288 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:39.288 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:39.390 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:39.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:39.428 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:39.428 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:39.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:39.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:39.563 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:39.563 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:39.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:39 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:54:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:39 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:54:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:39 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:54:39.664 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:39.703 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:39.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:39.704 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:39.775 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:39 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:54:39.776 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:39 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:54:39.776 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:39 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:54:39.776 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:39 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:54:39.776 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:39 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:54:39.776 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:39 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:54:39.805 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:39.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-11T08:54:39.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:39.839 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:39.839 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:39.939 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:39.970 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:39.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:39.971 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:40.065 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:54:40.065 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:54:40.072 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:40.101 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:54:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:54:40.132 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:54:40.132 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:54:40.144 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:40.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:40.145 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:40.165 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:54:40.165 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:54:40.195 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:54:40.195 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:54:40.231 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:54:40.231 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:54:40.246 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:40.274 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:40.275 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:40.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:40.276 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:40.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:40.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:40.417 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:40.417 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:40.471 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:54:40.471 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:54:40.473 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:54:40.501 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:54:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:54:40.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:40.582 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:40.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:40.583 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:40.653 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:40 smithi092 ceph-mon[29647]: pgmap v721: 158 pgs: 158 active+clean; 8.2 GiB data, 31 GiB used, 685 GiB / 715 GiB avail; 1.1 KiB/s rd, 63 MiB/s wr, 1.21k op/s 2024-09-11T08:54:40.653 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:40 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2905556195' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:54:40.653 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:40 smithi092 ceph-mon[41114]: pgmap v721: 158 pgs: 158 active+clean; 8.2 GiB data, 31 GiB used, 685 GiB / 715 GiB avail; 1.1 KiB/s rd, 63 MiB/s wr, 1.21k op/s 2024-09-11T08:54:40.653 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:40 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2905556195' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:54:40.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:40 smithi145 ceph-mon[33384]: pgmap v721: 158 pgs: 158 active+clean; 8.2 GiB data, 31 GiB used, 685 GiB / 715 GiB avail; 1.1 KiB/s rd, 63 MiB/s wr, 1.21k op/s 2024-09-11T08:54:40.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:40 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2905556195' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:54:40.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:40.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:40.735 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:40.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:40.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:40.871 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:40.972 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:41.007 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:41.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:41.008 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:41.108 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:41.139 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:41.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:41.140 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:41.241 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:41.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:41.277 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:41.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:41.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:41.410 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:41.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:41.411 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:41.512 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:41.544 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:41.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:41.545 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:41.646 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:41.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:41.674 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:41.675 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:41.701 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:54:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:54:41] "GET /metrics HTTP/1.1" 200 224833 "" "Prometheus/2.43.0" 2024-09-11T08:54:41.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:41.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:41.823 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:41.823 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:41.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:41.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:41.975 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:41.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:42.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:42.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:42.118 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:42.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:42.260 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:42.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:42.261 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:42.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:42.399 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:42.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:42.400 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:42.501 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:42.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:42.529 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:42.530 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:42.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:42.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:42.661 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:42.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:42 smithi092 ceph-mon[29647]: pgmap v722: 158 pgs: 158 active+clean; 8.1 GiB data, 30 GiB used, 685 GiB / 715 GiB avail; 1.1 KiB/s rd, 50 MiB/s wr, 1.21k op/s 2024-09-11T08:54:42.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:42 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:54:42.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:42 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:54:42.679 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:42 smithi092 ceph-mon[41114]: pgmap v722: 158 pgs: 158 active+clean; 8.1 GiB data, 30 GiB used, 685 GiB / 715 GiB avail; 1.1 KiB/s rd, 50 MiB/s wr, 1.21k op/s 2024-09-11T08:54:42.679 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:42 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:54:42.679 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:42 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:54:42.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:42 smithi145 ceph-mon[33384]: pgmap v722: 158 pgs: 158 active+clean; 8.1 GiB data, 30 GiB used, 685 GiB / 715 GiB avail; 1.1 KiB/s rd, 50 MiB/s wr, 1.21k op/s 2024-09-11T08:54:42.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:42 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:54:42.722 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:42 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:54:42.762 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:42.799 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:42.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:42.800 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:42.901 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:42.935 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:42.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:42.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:42.989 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:54:42.989 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:54:42.989 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-11T08:54:42.989 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T08:54:42.990 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T08:54:42.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 config set filestore_inject_stall 3 2024-09-11T08:54:43.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:43.072 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:43.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:43.090 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:43.091 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:43.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:54:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:54:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:54:43.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:43.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:43.228 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:43.228 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:43.329 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:43.365 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:43.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:43.366 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:43.467 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:43.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:43.506 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:43.507 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:43.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:43.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:43.651 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:43.651 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:43.726 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:43 smithi092 ceph-mon[29647]: osdmap e326: 8 total, 8 up, 8 in 2024-09-11T08:54:43.726 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:43 smithi092 ceph-mon[41114]: osdmap e326: 8 total, 8 up, 8 in 2024-09-11T08:54:43.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:43.759 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:54:43.759 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:54:43.759 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:54:43.759 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:54:43.759 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:54:43.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:43.798 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:43.899 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:43.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:43 smithi145 ceph-mon[33384]: osdmap e326: 8 total, 8 up, 8 in 2024-09-11T08:54:43.930 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:43.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:43.931 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:44.032 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:44.063 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:44.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:44.064 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:44.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:44.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:44.198 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:44.198 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:44.299 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:44.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:44.336 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:44.336 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:44.437 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:44.470 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:44.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:44.472 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:44.572 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:44.608 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:44.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:44.609 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:44.710 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:44.746 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:44.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:44.747 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:44.800 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:44 smithi092 ceph-mon[29647]: pgmap v724: 158 pgs: 158 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.1 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-09-11T08:54:44.801 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:44 smithi092 ceph-mon[41114]: pgmap v724: 158 pgs: 158 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.1 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-09-11T08:54:44.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-11T08:54:44.847 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:44.880 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:44.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:44.881 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:44.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:44 smithi145 ceph-mon[33384]: pgmap v724: 158 pgs: 158 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.1 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-09-11T08:54:44.981 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:45.020 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:45.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:45.021 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:45.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:54:45.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:45.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:45.182 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:45.283 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:45.312 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:45.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:45.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:45.414 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:45.461 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:45.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:45.462 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:45.563 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:45.597 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:45.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:45.598 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:45.699 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:45.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:45.736 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:45.737 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:45.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:54:45.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:54:45.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:45.873 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:45.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:45.874 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:45.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:54:45.975 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:46.009 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:46.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:46.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:46.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:46.142 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:46.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:46.143 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:46.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:46.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:46.279 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:46.379 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:46.412 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:46.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:46.413 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:46.513 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:46.547 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:46.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:46.548 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:46.649 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:46.680 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:46.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:46.681 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:46.782 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:46.803 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:46.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:46.804 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:46.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:46 smithi092 ceph-mon[29647]: pgmap v725: 158 pgs: 158 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.1 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-09-11T08:54:46.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:46 smithi092 ceph-mon[41114]: pgmap v725: 158 pgs: 158 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.1 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-09-11T08:54:46.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:46.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:46 smithi145 ceph-mon[33384]: pgmap v725: 158 pgs: 158 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.1 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-09-11T08:54:46.949 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:46.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:46.950 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:47.051 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:47.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:47.091 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:47.091 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:47.192 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:47.224 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:47.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:47.225 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:47.326 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:47.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:47.359 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:47.359 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:47.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:54:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:54:47.460 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:47.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:47.501 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:47.501 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:47.602 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:47.624 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:47.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:47.625 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:47.725 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:47.762 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:47.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:47.763 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:47.864 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:47.895 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:47.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:47.897 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:47.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:48.031 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:48.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:48.032 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:48.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:48.165 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:48.199 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:48.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:48.201 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:48.301 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:48.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:48.340 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:48.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:48.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:48.474 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:48.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:48.475 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:48.576 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:48.611 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:48.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:48.612 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:48.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:48.723 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:48 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:48.723 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:48 smithi092 ceph-mon[41114]: pgmap v726: 158 pgs: 158 active+clean; 7.5 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-11T08:54:48.724 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:48 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:48.724 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:48 smithi092 ceph-mon[29647]: pgmap v726: 158 pgs: 158 active+clean; 7.5 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-11T08:54:48.752 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:48.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:48.753 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:48.854 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:48.892 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:48.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:48.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:48.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:48 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:48.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:48 smithi145 ceph-mon[33384]: pgmap v726: 158 pgs: 158 active+clean; 7.5 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-11T08:54:48.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:49.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:49.033 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:49.033 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:49.134 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:49.173 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:49.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:49.175 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:49.275 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:49.305 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:49.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:49.306 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:49.407 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:49.441 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:49.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:49.442 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:49.472 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:49.544 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:49.578 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:49.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:49.579 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:49.680 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:49.711 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:49.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:49.712 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:49.813 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:49.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:49.842 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:49.843 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:49.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:49.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:49.980 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:50.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:50.114 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:50.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:50.115 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:50.216 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:50.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:50.253 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:50.253 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:50.327 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:50 smithi092 ceph-mon[29647]: pgmap v727: 158 pgs: 158 active+clean; 7.5 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-11T08:54:50.328 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:50 smithi092 ceph-mon[41114]: pgmap v727: 158 pgs: 158 active+clean; 7.5 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-11T08:54:50.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:50.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:50.391 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:50.392 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:50.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:50 smithi145 ceph-mon[33384]: pgmap v727: 158 pgs: 158 active+clean; 7.5 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-11T08:54:50.492 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:50.531 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:50.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:50.532 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:50.633 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:50.634 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:50.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:50.668 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:50.668 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:50.769 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:50.800 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:50.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:50.801 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:50.902 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:50.930 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:50.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:50.931 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:51.032 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:51.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:51.067 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:51.068 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:51.088 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:54:51.168 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:51.204 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:51.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:51.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:51.306 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:51.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:51 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3541441932' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:54:51.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:51 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:54:51.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:51 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3541441932' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:54:51.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:51 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:54:51.342 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:51.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:51.343 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:51.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:51 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3541441932' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:54:51.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:51 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:54:51.443 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:51.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:51.480 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:51.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:51.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:51.625 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:51.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:51.626 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:51.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:51.760 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:51.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:51.761 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:51.788 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:54:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:54:51] "GET /metrics HTTP/1.1" 200 224781 "" "Prometheus/2.43.0" 2024-09-11T08:54:51.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:51.893 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:51.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:51.894 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:51.995 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:52.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:52.033 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:52.033 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:52.134 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:52.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:52.168 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:52.169 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:52.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:52.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:52.306 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:52.306 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:52.408 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:52.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:52 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:54:52.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:52 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:54:52.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:52 smithi145 ceph-mon[33384]: osdmap e327: 8 total, 8 up, 8 in 2024-09-11T08:54:52.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:52 smithi145 ceph-mon[33384]: pgmap v729: 158 pgs: 158 active+clean; 7.3 GiB data, 28 GiB used, 688 GiB / 715 GiB avail; 458 B/s rd, 0 B/s wr, 940 op/s 2024-09-11T08:54:52.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:52 smithi145 ceph-mon[33384]: 1.9 scrub starts 2024-09-11T08:54:52.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:52 smithi145 ceph-mon[33384]: 1.9 scrub ok 2024-09-11T08:54:52.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:52.440 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:52.440 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:52.541 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:52.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:52 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:54:52.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:52 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:54:52.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:52 smithi092 ceph-mon[29647]: osdmap e327: 8 total, 8 up, 8 in 2024-09-11T08:54:52.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:52 smithi092 ceph-mon[29647]: pgmap v729: 158 pgs: 158 active+clean; 7.3 GiB data, 28 GiB used, 688 GiB / 715 GiB avail; 458 B/s rd, 0 B/s wr, 940 op/s 2024-09-11T08:54:52.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:52 smithi092 ceph-mon[29647]: 1.9 scrub starts 2024-09-11T08:54:52.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:52 smithi092 ceph-mon[29647]: 1.9 scrub ok 2024-09-11T08:54:52.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:52 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:54:52.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:52 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:54:52.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:52 smithi092 ceph-mon[41114]: osdmap e327: 8 total, 8 up, 8 in 2024-09-11T08:54:52.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:52 smithi092 ceph-mon[41114]: pgmap v729: 158 pgs: 158 active+clean; 7.3 GiB data, 28 GiB used, 688 GiB / 715 GiB avail; 458 B/s rd, 0 B/s wr, 940 op/s 2024-09-11T08:54:52.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:52 smithi092 ceph-mon[41114]: 1.9 scrub starts 2024-09-11T08:54:52.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:52 smithi092 ceph-mon[41114]: 1.9 scrub ok 2024-09-11T08:54:52.580 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:52.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:52.581 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:52.682 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:52.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:52.717 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:52.717 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:52.818 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:52.848 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:52.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:52.849 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:52.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:52.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:52.991 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:52.991 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:53.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:53.128 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:54:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:54:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:54:53.142 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:53.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:53.143 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:53.244 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:53.280 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:53.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:53.281 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:53.382 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:53.392 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:53 smithi092 ceph-mon[29647]: 10.2 scrub starts 2024-09-11T08:54:53.392 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:53 smithi092 ceph-mon[29647]: 10.f scrub starts 2024-09-11T08:54:53.392 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:53 smithi092 ceph-mon[29647]: 3.1e scrub starts 2024-09-11T08:54:53.392 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:53 smithi092 ceph-mon[29647]: 3.1e scrub ok 2024-09-11T08:54:53.393 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:53 smithi092 ceph-mon[41114]: 10.2 scrub starts 2024-09-11T08:54:53.393 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:53 smithi092 ceph-mon[41114]: 10.f scrub starts 2024-09-11T08:54:53.393 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:53 smithi092 ceph-mon[41114]: 3.1e scrub starts 2024-09-11T08:54:53.393 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:53 smithi092 ceph-mon[41114]: 3.1e scrub ok 2024-09-11T08:54:53.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:53 smithi145 ceph-mon[33384]: 10.2 scrub starts 2024-09-11T08:54:53.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:53 smithi145 ceph-mon[33384]: 10.f scrub starts 2024-09-11T08:54:53.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:53 smithi145 ceph-mon[33384]: 3.1e scrub starts 2024-09-11T08:54:53.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:53 smithi145 ceph-mon[33384]: 3.1e scrub ok 2024-09-11T08:54:53.423 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:53.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:53.424 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:53.436 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:54:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "" 2024-09-11T08:54:53.436 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:54:53.525 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:53.565 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:53.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:53.566 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:53.667 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:53.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:53.706 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:53.806 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:53.842 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:53.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:53.843 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:53.944 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:53.982 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:53.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:53.983 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:54.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:54.124 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:54.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:54.125 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:54.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:54.258 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:54.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:54.259 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:54.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:54.398 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:54.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:54.399 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:54.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:54 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:54:54.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:54 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:54:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:54 smithi145 ceph-mon[33384]: osdmap e328: 8 total, 8 up, 8 in 2024-09-11T08:54:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:54 smithi145 ceph-mon[33384]: 3.4 scrub starts 2024-09-11T08:54:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:54 smithi145 ceph-mon[33384]: 3.4 scrub ok 2024-09-11T08:54:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:54 smithi145 ceph-mon[33384]: pgmap v731: 158 pgs: 158 active+clean; 7.1 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-11T08:54:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:54 smithi145 ceph-mon[33384]: 10.0 scrub starts 2024-09-11T08:54:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:54 smithi145 ceph-mon[33384]: 4.11 scrub starts 2024-09-11T08:54:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:54 smithi145 ceph-mon[33384]: 4.11 scrub ok 2024-09-11T08:54:54.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:54:54.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:54:54.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[29647]: osdmap e328: 8 total, 8 up, 8 in 2024-09-11T08:54:54.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[29647]: 3.4 scrub starts 2024-09-11T08:54:54.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[29647]: 3.4 scrub ok 2024-09-11T08:54:54.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[29647]: pgmap v731: 158 pgs: 158 active+clean; 7.1 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-11T08:54:54.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[29647]: 10.0 scrub starts 2024-09-11T08:54:54.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[29647]: 4.11 scrub starts 2024-09-11T08:54:54.444 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[29647]: 4.11 scrub ok 2024-09-11T08:54:54.444 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:54:54.444 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:54:54.444 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[41114]: osdmap e328: 8 total, 8 up, 8 in 2024-09-11T08:54:54.444 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[41114]: 3.4 scrub starts 2024-09-11T08:54:54.444 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[41114]: 3.4 scrub ok 2024-09-11T08:54:54.444 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[41114]: pgmap v731: 158 pgs: 158 active+clean; 7.1 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-11T08:54:54.444 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[41114]: 10.0 scrub starts 2024-09-11T08:54:54.444 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[41114]: 4.11 scrub starts 2024-09-11T08:54:54.444 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:54 smithi092 ceph-mon[41114]: 4.11 scrub ok 2024-09-11T08:54:54.496 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:54:54.496 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-11T08:54:54.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-11T08:54:54.496 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:54:54.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.496 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120753 9.1 9:80027676:::benchmark_data_smithi092_145083_object120752:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e234)", 2024-09-11T08:54:54.497 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.482645+0000", 2024-09-11T08:54:54.497 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 269.01026947999998, 2024-09-11T08:54:54.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.202329761, 2024-09-11T08:54:54.497 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.497 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:54:54.497 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:54:54.497 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:54:54.497 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:54:54.497 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120753 2024-09-11T08:54:54.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.497 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482645+0000", 2024-09-11T08:54:54.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482645+0000", 2024-09-11T08:54:54.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482644+0000", 2024-09-11T08:54:54.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:54:54.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482670+0000", 2024-09-11T08:54:54.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6781999999999999e-05 2024-09-11T08:54:54.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482682+0000", 2024-09-11T08:54:54.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2051e-05 2024-09-11T08:54:54.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482688+0000", 2024-09-11T08:54:54.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0870000000000003e-06 2024-09-11T08:54:54.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.487390+0000", 2024-09-11T08:54:54.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047011450000000003 2024-09-11T08:54:54.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:54:54.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.487514+0000", 2024-09-11T08:54:54.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000124664 2024-09-11T08:54:54.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6,7", 2024-09-11T08:54:54.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.487660+0000", 2024-09-11T08:54:54.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014545500000000001 2024-09-11T08:54:54.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.496777+0000", 2024-09-11T08:54:54.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0091171150000000003 2024-09-11T08:54:54.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:54:54.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.497628+0000", 2024-09-11T08:54:54.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00085127100000000002 2024-09-11T08:54:54.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684968+0000", 2024-09-11T08:54:54.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.187339704 2024-09-11T08:54:54.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684975+0000", 2024-09-11T08:54:54.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-11T08:54:54.503 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.503 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.503 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.503 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120754 9.1 9:8766ca32:::benchmark_data_smithi092_145083_object120753:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e234)", 2024-09-11T08:54:54.503 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.483486+0000", 2024-09-11T08:54:54.503 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 269.00942863900002, 2024-09-11T08:54:54.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20167465100000001, 2024-09-11T08:54:54.504 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.504 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:54:54.504 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:54:54.504 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:54:54.504 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:54:54.504 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120754 2024-09-11T08:54:54.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.504 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.483486+0000", 2024-09-11T08:54:54.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.483486+0000", 2024-09-11T08:54:54.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.483485+0000", 2024-09-11T08:54:54.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:54:54.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.483509+0000", 2024-09-11T08:54:54.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3549000000000001e-05 2024-09-11T08:54:54.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.483521+0000", 2024-09-11T08:54:54.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2855e-05 2024-09-11T08:54:54.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.483527+0000", 2024-09-11T08:54:54.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8300000000000001e-06 2024-09-11T08:54:54.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.492869+0000", 2024-09-11T08:54:54.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093416190000000007 2024-09-11T08:54:54.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:54:54.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.493003+0000", 2024-09-11T08:54:54.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013431599999999999 2024-09-11T08:54:54.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6,7", 2024-09-11T08:54:54.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.493150+0000", 2024-09-11T08:54:54.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014698099999999999 2024-09-11T08:54:54.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:54:54.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.497632+0000", 2024-09-11T08:54:54.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044822689999999997 2024-09-11T08:54:54.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685155+0000", 2024-09-11T08:54:54.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18752247599999999 2024-09-11T08:54:54.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685161+0000", 2024-09-11T08:54:54.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.908e-06 2024-09-11T08:54:54.508 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.509 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.509 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.509 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.15366.0:120754 9.1 e234/206)", 2024-09-11T08:54:54.509 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.496702+0000", 2024-09-11T08:54:54.509 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 268.99621231399999, 2024-09-11T08:54:54.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18818669199999999, 2024-09-11T08:54:54.509 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.509 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:54:54.509 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.496702+0000", 2024-09-11T08:54:54.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.496702+0000", 2024-09-11T08:54:54.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.496702+0000", 2024-09-11T08:54:54.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:54:54.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.496705+0000", 2024-09-11T08:54:54.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.518e-06 2024-09-11T08:54:54.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.496716+0000", 2024-09-11T08:54:54.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0454e-05 2024-09-11T08:54:54.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.496720+0000", 2024-09-11T08:54:54.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1779999999999996e-06 2024-09-11T08:54:54.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684876+0000", 2024-09-11T08:54:54.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18815546899999999 2024-09-11T08:54:54.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684889+0000", 2024-09-11T08:54:54.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3625000000000001e-05 2024-09-11T08:54:54.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.512 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.513 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120753 9.1 9:80027676:::benchmark_data_smithi092_145083_object120752:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:54:54.513 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.504067+0000", 2024-09-11T08:54:54.513 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 268.98884731200002, 2024-09-11T08:54:54.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0247133180000001, 2024-09-11T08:54:54.513 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.513 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:54:54.513 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:54:54.513 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:54:54.513 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:54:54.513 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120753 2024-09-11T08:54:54.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.513 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504067+0000", 2024-09-11T08:54:54.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504067+0000", 2024-09-11T08:54:54.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504066+0000", 2024-09-11T08:54:54.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:54:54.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504100+0000", 2024-09-11T08:54:54.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4513e-05 2024-09-11T08:54:54.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504114+0000", 2024-09-11T08:54:54.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305999999999999e-05 2024-09-11T08:54:54.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504120+0000", 2024-09-11T08:54:54.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2450000000000003e-06 2024-09-11T08:54:54.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685178+0000", 2024-09-11T08:54:54.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.181057575 2024-09-11T08:54:54.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:54:54.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685184+0000", 2024-09-11T08:54:54.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9600000000000003e-06 2024-09-11T08:54:54.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.528703+0000", 2024-09-11T08:54:54.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84351812699999995 2024-09-11T08:54:54.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.528781+0000", 2024-09-11T08:54:54.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8183999999999995e-05 2024-09-11T08:54:54.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.517 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.517 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120754 9.1 9:8766ca32:::benchmark_data_smithi092_145083_object120753:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:54:54.518 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.504271+0000", 2024-09-11T08:54:54.518 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 268.98864372700001, 2024-09-11T08:54:54.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0247913390000001, 2024-09-11T08:54:54.518 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.518 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:54:54.518 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:54:54.518 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:54:54.518 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:54:54.518 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120754 2024-09-11T08:54:54.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.518 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504271+0000", 2024-09-11T08:54:54.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504271+0000", 2024-09-11T08:54:54.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504270+0000", 2024-09-11T08:54:54.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:54:54.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504292+0000", 2024-09-11T08:54:54.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1151000000000001e-05 2024-09-11T08:54:54.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504299+0000", 2024-09-11T08:54:54.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9079999999999995e-06 2024-09-11T08:54:54.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504304+0000", 2024-09-11T08:54:54.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5109999999999997e-06 2024-09-11T08:54:54.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685200+0000", 2024-09-11T08:54:54.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18089587500000001 2024-09-11T08:54:54.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:54:54.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685205+0000", 2024-09-11T08:54:54.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9439999999999996e-06 2024-09-11T08:54:54.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529010+0000", 2024-09-11T08:54:54.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84380478599999997 2024-09-11T08:54:54.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529062+0000", 2024-09-11T08:54:54.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2797999999999998e-05 2024-09-11T08:54:54.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.522 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.523 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.15366.0:120753 9.1 e234/206)", 2024-09-11T08:54:54.523 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.507259+0000", 2024-09-11T08:54:54.523 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 268.98565574100002, 2024-09-11T08:54:54.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.17765646299999999, 2024-09-11T08:54:54.523 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.523 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:54:54.523 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507259+0000", 2024-09-11T08:54:54.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507259+0000", 2024-09-11T08:54:54.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507259+0000", 2024-09-11T08:54:54.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:54:54.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507262+0000", 2024-09-11T08:54:54.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7000000000000002e-06 2024-09-11T08:54:54.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507274+0000", 2024-09-11T08:54:54.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1742999999999999e-05 2024-09-11T08:54:54.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507281+0000", 2024-09-11T08:54:54.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5420000000000002e-06 2024-09-11T08:54:54.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:54.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684905+0000", 2024-09-11T08:54:54.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.177624169 2024-09-11T08:54:54.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684916+0000", 2024-09-11T08:54:54.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0679000000000001e-05 2024-09-11T08:54:54.528 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.528 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.528 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.528 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.15366.0:120754 9.1 e234/206)", 2024-09-11T08:54:54.528 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.507355+0000", 2024-09-11T08:54:54.528 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 268.98555998699999, 2024-09-11T08:54:54.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.17758486300000001, 2024-09-11T08:54:54.528 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.528 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:54:54.528 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507355+0000", 2024-09-11T08:54:54.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507355+0000", 2024-09-11T08:54:54.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507354+0000", 2024-09-11T08:54:54.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:54:54.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507358+0000", 2024-09-11T08:54:54.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7610000000000001e-06 2024-09-11T08:54:54.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507365+0000", 2024-09-11T08:54:54.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4860000000000002e-06 2024-09-11T08:54:54.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507369+0000", 2024-09-11T08:54:54.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9779999999999999e-06 2024-09-11T08:54:54.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684929+0000", 2024-09-11T08:54:54.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.17756022599999999 2024-09-11T08:54:54.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684940+0000", 2024-09-11T08:54:54.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0879e-05 2024-09-11T08:54:54.531 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.531 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.531 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.532 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120773 9.1 9:84313f40:::benchmark_data_smithi092_145083_object120772:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:54:54.532 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.524403+0000", 2024-09-11T08:54:54.532 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 268.96851171499998, 2024-09-11T08:54:54.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0196523550000001, 2024-09-11T08:54:54.532 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.532 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:54:54.532 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:54:54.532 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:54:54.532 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:54:54.532 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120773 2024-09-11T08:54:54.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.533 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524403+0000", 2024-09-11T08:54:54.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524403+0000", 2024-09-11T08:54:54.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524402+0000", 2024-09-11T08:54:54.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:54:54.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524428+0000", 2024-09-11T08:54:54.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6698000000000001e-05 2024-09-11T08:54:54.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524442+0000", 2024-09-11T08:54:54.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3806e-05 2024-09-11T08:54:54.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524448+0000", 2024-09-11T08:54:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3109999999999996e-06 2024-09-11T08:54:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685220+0000", 2024-09-11T08:54:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16077285399999999 2024-09-11T08:54:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:54:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685225+0000", 2024-09-11T08:54:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8810000000000004e-06 2024-09-11T08:54:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529083+0000", 2024-09-11T08:54:54.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84385757900000002 2024-09-11T08:54:54.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:54:54.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529204+0000", 2024-09-11T08:54:54.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121352 2024-09-11T08:54:54.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6,7", 2024-09-11T08:54:54.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529650+0000", 2024-09-11T08:54:54.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044596399999999997 2024-09-11T08:54:54.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:54:54.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.533815+0000", 2024-09-11T08:54:54.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041648370000000002 2024-09-11T08:54:54.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.536911+0000", 2024-09-11T08:54:54.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030961280000000001 2024-09-11T08:54:54.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.543805+0000", 2024-09-11T08:54:54.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006893939 2024-09-11T08:54:54.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:54:54.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.544015+0000", 2024-09-11T08:54:54.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020939999999999999 2024-09-11T08:54:54.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.544055+0000", 2024-09-11T08:54:54.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0908000000000002e-05 2024-09-11T08:54:54.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.538 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:122641 9.19 9:9f7f102d:::benchmark_data_smithi092_145083_object122640:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:54:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:27.538908+0000", 2024-09-11T08:54:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 266.95400631699999, 2024-09-11T08:54:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.439093481, 2024-09-11T08:54:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:54:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:54:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:54:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:54:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122641 2024-09-11T08:54:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538908+0000", 2024-09-11T08:54:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538908+0000", 2024-09-11T08:54:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538907+0000", 2024-09-11T08:54:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:54:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538933+0000", 2024-09-11T08:54:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5627e-05 2024-09-11T08:54:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538945+0000", 2024-09-11T08:54:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2721000000000001e-05 2024-09-11T08:54:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538951+0000", 2024-09-11T08:54:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5849999999999999e-06 2024-09-11T08:54:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.539347+0000", 2024-09-11T08:54:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000395936 2024-09-11T08:54:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:54:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.539355+0000", 2024-09-11T08:54:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6629999999999999e-06 2024-09-11T08:54:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.534990+0000", 2024-09-11T08:54:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99563493400000003 2024-09-11T08:54:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:54:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535002+0000", 2024-09-11T08:54:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1333e-05 2024-09-11T08:54:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.963852+0000", 2024-09-11T08:54:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.42885004700000001 2024-09-11T08:54:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:54:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.964026+0000", 2024-09-11T08:54:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017450399999999999 2024-09-11T08:54:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6,7", 2024-09-11T08:54:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.964280+0000", 2024-09-11T08:54:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025399599999999999 2024-09-11T08:54:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:54:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.965296+0000", 2024-09-11T08:54:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010152290000000001 2024-09-11T08:54:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.969974+0000", 2024-09-11T08:54:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046781729999999999 2024-09-11T08:54:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.971213+0000", 2024-09-11T08:54:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001239431 2024-09-11T08:54:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.977816+0000", 2024-09-11T08:54:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066024580000000003 2024-09-11T08:54:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:54:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.977893+0000", 2024-09-11T08:54:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7600000000000002e-05 2024-09-11T08:54:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.978002+0000", 2024-09-11T08:54:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010877199999999999 2024-09-11T08:54:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:122656 9.19 9:9b532791:::benchmark_data_smithi092_145083_object122655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:54:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:27.549187+0000", 2024-09-11T08:54:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 266.94372752599998, 2024-09-11T08:54:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4294275089999999, 2024-09-11T08:54:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:54:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:54:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:54:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:54:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122656 2024-09-11T08:54:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549187+0000", 2024-09-11T08:54:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549187+0000", 2024-09-11T08:54:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549186+0000", 2024-09-11T08:54:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:54:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549208+0000", 2024-09-11T08:54:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2378000000000002e-05 2024-09-11T08:54:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549220+0000", 2024-09-11T08:54:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1711e-05 2024-09-11T08:54:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549226+0000", 2024-09-11T08:54:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7810000000000002e-06 2024-09-11T08:54:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549320+0000", 2024-09-11T08:54:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3955000000000004e-05 2024-09-11T08:54:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:54:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549329+0000", 2024-09-11T08:54:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.952e-06 2024-09-11T08:54:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535136+0000", 2024-09-11T08:54:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98580697900000003 2024-09-11T08:54:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:54:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535143+0000", 2024-09-11T08:54:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4440000000000002e-06 2024-09-11T08:54:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.965346+0000", 2024-09-11T08:54:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43020250500000001 2024-09-11T08:54:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:54:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.965438+0000", 2024-09-11T08:54:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2334000000000001e-05 2024-09-11T08:54:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6,7", 2024-09-11T08:54:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.965597+0000", 2024-09-11T08:54:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015942300000000001 2024-09-11T08:54:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:54:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.966745+0000", 2024-09-11T08:54:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001147482 2024-09-11T08:54:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.971264+0000", 2024-09-11T08:54:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045190960000000002 2024-09-11T08:54:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.978072+0000", 2024-09-11T08:54:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068077540000000001 2024-09-11T08:54:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.978396+0000", 2024-09-11T08:54:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000324247 2024-09-11T08:54:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:54:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.978508+0000", 2024-09-11T08:54:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112379 2024-09-11T08:54:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.978615+0000", 2024-09-11T08:54:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010634800000000001 2024-09-11T08:54:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:165732 9.2 9:47194da5:::benchmark_data_smithi092_145083_object22334:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-11T08:54:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:51:02.891989+0000", 2024-09-11T08:54:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 231.60092625600001, 2024-09-11T08:54:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.13579332399999999, 2024-09-11T08:54:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:54:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:54:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:54:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:54:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 165732 2024-09-11T08:54:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.891989+0000", 2024-09-11T08:54:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.891989+0000", 2024-09-11T08:54:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.891988+0000", 2024-09-11T08:54:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:54:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.891993+0000", 2024-09-11T08:54:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.48e-06 2024-09-11T08:54:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.996787+0000", 2024-09-11T08:54:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104793624 2024-09-11T08:54:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.996795+0000", 2024-09-11T08:54:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7520000000000002e-06 2024-09-11T08:54:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.997074+0000", 2024-09-11T08:54:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027848 2024-09-11T08:54:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:54:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.997387+0000", 2024-09-11T08:54:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031345599999999999 2024-09-11T08:54:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:54:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:02.998385+0000", 2024-09-11T08:54:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00099731600000000004 2024-09-11T08:54:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:54:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.000687+0000", 2024-09-11T08:54:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023024920000000002 2024-09-11T08:54:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.024796+0000", 2024-09-11T08:54:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024108950000000001 2024-09-11T08:54:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.027696+0000", 2024-09-11T08:54:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002899413 2024-09-11T08:54:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:54:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.027745+0000", 2024-09-11T08:54:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8912000000000002e-05 2024-09-11T08:54:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:51:03.027782+0000", 2024-09-11T08:54:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7345000000000002e-05 2024-09-11T08:54:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272348 9.8 9:15db9131:::benchmark_data_smithi092_145083_object128950:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:54:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.641892+0000", 2024-09-11T08:54:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 149.85102305500001, 2024-09-11T08:54:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98679250600000001, 2024-09-11T08:54:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:54:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:54:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:54:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:54:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272348 2024-09-11T08:54:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641892+0000", 2024-09-11T08:54:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641892+0000", 2024-09-11T08:54:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641890+0000", 2024-09-11T08:54:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:54:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641898+0000", 2024-09-11T08:54:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7849999999999992e-06 2024-09-11T08:54:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641915+0000", 2024-09-11T08:54:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7677e-05 2024-09-11T08:54:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641923+0000", 2024-09-11T08:54:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6199999999999999e-06 2024-09-11T08:54:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641981+0000", 2024-09-11T08:54:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7901e-05 2024-09-11T08:54:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:54:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641989+0000", 2024-09-11T08:54:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3040000000000005e-06 2024-09-11T08:54:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.624095+0000", 2024-09-11T08:54:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.982105699 2024-09-11T08:54:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:54:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.624185+0000", 2024-09-11T08:54:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0476999999999994e-05 2024-09-11T08:54:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:54:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.624299+0000", 2024-09-11T08:54:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113366 2024-09-11T08:54:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628589+0000", 2024-09-11T08:54:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004290622 2024-09-11T08:54:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628628+0000", 2024-09-11T08:54:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8549e-05 2024-09-11T08:54:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:54:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628633+0000", 2024-09-11T08:54:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1059999999999999e-06 2024-09-11T08:54:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:54:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628662+0000", 2024-09-11T08:54:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8758000000000001e-05 2024-09-11T08:54:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628684+0000", 2024-09-11T08:54:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2416999999999999e-05 2024-09-11T08:54:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272383 9.8 9:12173f2a:::benchmark_data_smithi092_145083_object128985:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:54:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.665664+0000", 2024-09-11T08:54:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 149.827251163, 2024-09-11T08:54:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.96324268999999996, 2024-09-11T08:54:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:54:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:54:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:54:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:54:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272383 2024-09-11T08:54:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665664+0000", 2024-09-11T08:54:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665664+0000", 2024-09-11T08:54:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665663+0000", 2024-09-11T08:54:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:54:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665668+0000", 2024-09-11T08:54:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9049999999999996e-06 2024-09-11T08:54:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665678+0000", 2024-09-11T08:54:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0057e-05 2024-09-11T08:54:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665683+0000", 2024-09-11T08:54:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2270000000000001e-06 2024-09-11T08:54:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665724+0000", 2024-09-11T08:54:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000003e-05 2024-09-11T08:54:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:54:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665731+0000", 2024-09-11T08:54:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6030000000000001e-06 2024-09-11T08:54:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.626022+0000", 2024-09-11T08:54:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.96029152699999998 2024-09-11T08:54:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:54:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.626136+0000", 2024-09-11T08:54:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114009 2024-09-11T08:54:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:54:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.626246+0000", 2024-09-11T08:54:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010950000000000001 2024-09-11T08:54:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:54:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628734+0000", 2024-09-11T08:54:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024882149999999998 2024-09-11T08:54:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628762+0000", 2024-09-11T08:54:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8354e-05 2024-09-11T08:54:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628852+0000", 2024-09-11T08:54:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9708000000000002e-05 2024-09-11T08:54:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:54:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628888+0000", 2024-09-11T08:54:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6128999999999999e-05 2024-09-11T08:54:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628906+0000", 2024-09-11T08:54:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8218000000000001e-05 2024-09-11T08:54:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272428 9.8 9:14391abd:::benchmark_data_smithi092_145083_object129030:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:54:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.737328+0000", 2024-09-11T08:54:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 149.755586631, 2024-09-11T08:54:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.89314801799999999, 2024-09-11T08:54:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:54:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:54:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:54:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:54:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272428 2024-09-11T08:54:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737328+0000", 2024-09-11T08:54:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737328+0000", 2024-09-11T08:54:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737327+0000", 2024-09-11T08:54:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:54:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737333+0000", 2024-09-11T08:54:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6720000000000001e-06 2024-09-11T08:54:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737345+0000", 2024-09-11T08:54:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2007999999999999e-05 2024-09-11T08:54:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737353+0000", 2024-09-11T08:54:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0930000000000003e-06 2024-09-11T08:54:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737385+0000", 2024-09-11T08:54:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2394999999999997e-05 2024-09-11T08:54:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:54:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737391+0000", 2024-09-11T08:54:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1800000000000004e-06 2024-09-11T08:54:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.627276+0000", 2024-09-11T08:54:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.88988573100000001 2024-09-11T08:54:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:54:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.627371+0000", 2024-09-11T08:54:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4772000000000007e-05 2024-09-11T08:54:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:54:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.627432+0000", 2024-09-11T08:54:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1106999999999996e-05 2024-09-11T08:54:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628813+0000", 2024-09-11T08:54:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001381195 2024-09-11T08:54:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:54:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628822+0000", 2024-09-11T08:54:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5169999999999999e-06 2024-09-11T08:54:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630397+0000", 2024-09-11T08:54:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001574717 2024-09-11T08:54:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:54:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630443+0000", 2024-09-11T08:54:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6513e-05 2024-09-11T08:54:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630476+0000", 2024-09-11T08:54:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3039e-05 2024-09-11T08:54:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272436 9.8 9:16049abc:::benchmark_data_smithi092_145083_object129038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:54:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.749067+0000", 2024-09-11T08:54:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 149.74384796699999, 2024-09-11T08:54:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.88488331399999998, 2024-09-11T08:54:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:54:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:54:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:54:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:54:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272436 2024-09-11T08:54:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749067+0000", 2024-09-11T08:54:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749067+0000", 2024-09-11T08:54:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749065+0000", 2024-09-11T08:54:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:54:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749073+0000", 2024-09-11T08:54:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9009999999999994e-06 2024-09-11T08:54:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749094+0000", 2024-09-11T08:54:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1129999999999999e-05 2024-09-11T08:54:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749101+0000", 2024-09-11T08:54:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3699999999999997e-06 2024-09-11T08:54:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749220+0000", 2024-09-11T08:54:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118566 2024-09-11T08:54:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:54:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749229+0000", 2024-09-11T08:54:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1370000000000008e-06 2024-09-11T08:54:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628938+0000", 2024-09-11T08:54:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.87970911100000004 2024-09-11T08:54:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:54:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.629008+0000", 2024-09-11T08:54:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9467000000000004e-05 2024-09-11T08:54:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:54:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.629069+0000", 2024-09-11T08:54:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1353999999999996e-05 2024-09-11T08:54:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:54:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633156+0000", 2024-09-11T08:54:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040865729999999996 2024-09-11T08:54:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633192+0000", 2024-09-11T08:54:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5939999999999998e-05 2024-09-11T08:54:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633837+0000", 2024-09-11T08:54:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00064550800000000004 2024-09-11T08:54:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:54:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633902+0000", 2024-09-11T08:54:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5283000000000005e-05 2024-09-11T08:54:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633950+0000", 2024-09-11T08:54:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7713999999999998e-05 2024-09-11T08:54:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272445 9.8 9:14cf6d17:::benchmark_data_smithi092_145083_object129047:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:54:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.755404+0000", 2024-09-11T08:54:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 149.73751128500001, 2024-09-11T08:54:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.87888003299999995, 2024-09-11T08:54:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:54:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:54:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:54:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:54:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272445 2024-09-11T08:54:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755404+0000", 2024-09-11T08:54:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755404+0000", 2024-09-11T08:54:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755402+0000", 2024-09-11T08:54:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:54:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755410+0000", 2024-09-11T08:54:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8029999999999997e-06 2024-09-11T08:54:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755424+0000", 2024-09-11T08:54:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3912e-05 2024-09-11T08:54:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755429+0000", 2024-09-11T08:54:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7339999999999996e-06 2024-09-11T08:54:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755475+0000", 2024-09-11T08:54:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5982999999999997e-05 2024-09-11T08:54:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:54:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755485+0000", 2024-09-11T08:54:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3170000000000008e-06 2024-09-11T08:54:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630509+0000", 2024-09-11T08:54:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.87502458000000005 2024-09-11T08:54:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:54:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630612+0000", 2024-09-11T08:54:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102422 2024-09-11T08:54:54.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:54:54.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630716+0000", 2024-09-11T08:54:54.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104014 2024-09-11T08:54:54.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:54:54.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633274+0000", 2024-09-11T08:54:54.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025580450000000001 2024-09-11T08:54:54.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.634132+0000", 2024-09-11T08:54:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00085821300000000003 2024-09-11T08:54:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.634183+0000", 2024-09-11T08:54:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1190000000000003e-05 2024-09-11T08:54:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:54:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.634236+0000", 2024-09-11T08:54:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2466000000000002e-05 2024-09-11T08:54:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.634284+0000", 2024-09-11T08:54:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7987e-05 2024-09-11T08:54:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:280459 9.9 9:928296ec:::benchmark_data_smithi092_145083_object137061:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e284)", 2024-09-11T08:54:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:32.131871+0000", 2024-09-11T08:54:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 142.36104363499999, 2024-09-11T08:54:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15378178100000001, 2024-09-11T08:54:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:54:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:54:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:54:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:54:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 280459 2024-09-11T08:54:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:32.131871+0000", 2024-09-11T08:54:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:32.131871+0000", 2024-09-11T08:54:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:32.131870+0000", 2024-09-11T08:54:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:54:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:32.131876+0000", 2024-09-11T08:54:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2920000000000003e-06 2024-09-11T08:54:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:32.131887+0000", 2024-09-11T08:54:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1429e-05 2024-09-11T08:54:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:32.131893+0000", 2024-09-11T08:54:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.66e-06 2024-09-11T08:54:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:32.131925+0000", 2024-09-11T08:54:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2555999999999998e-05 2024-09-11T08:54:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:54:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:32.132028+0000", 2024-09-11T08:54:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102445 2024-09-11T08:54:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5,7", 2024-09-11T08:54:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:32.132124+0000", 2024-09-11T08:54:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6451000000000002e-05 2024-09-11T08:54:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:54:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:32.134244+0000", 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021203049999999998 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:32.134997+0000", 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00075248299999999997 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:32.135217+0000", 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000220039 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:32.285479+0000", 2024-09-11T08:54:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15026218899999999 2024-09-11T08:54:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:54:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:32.285534+0000", 2024-09-11T08:54:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5087000000000003e-05 2024-09-11T08:54:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:32.285653+0000", 2024-09-11T08:54:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118683 2024-09-11T08:54:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15621.0:23113 10.3 e300/293 10:c4b16d09:::benchmark_data_smithi092_173109_object23112:head v 300'1415, mlcod=300'1413)", 2024-09-11T08:54:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:53:12.260506+0000", 2024-09-11T08:54:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 102.232408921, 2024-09-11T08:54:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.163533123, 2024-09-11T08:54:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:54:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.260506+0000", 2024-09-11T08:54:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.260506+0000", 2024-09-11T08:54:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.260505+0000", 2024-09-11T08:54:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:54:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.260526+0000", 2024-09-11T08:54:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1168000000000001e-05 2024-09-11T08:54:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.260537+0000", 2024-09-11T08:54:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0438000000000001e-05 2024-09-11T08:54:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.260542+0000", 2024-09-11T08:54:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2900000000000002e-06 2024-09-11T08:54:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.260607+0000", 2024-09-11T08:54:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4529000000000005e-05 2024-09-11T08:54:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:54:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.260625+0000", 2024-09-11T08:54:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8277000000000001e-05 2024-09-11T08:54:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:54:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.261996+0000", 2024-09-11T08:54:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001371316 2024-09-11T08:54:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.424039+0000", 2024-09-11T08:54:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.162042731 2024-09-11T08:54:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15621.0:23121 10.3 e300/293 10:c987efe1:::benchmark_data_smithi092_173109_object23120:head v 300'1416, mlcod=300'1413)", 2024-09-11T08:54:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:53:12.262958+0000", 2024-09-11T08:54:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 102.229957239, 2024-09-11T08:54:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16306765500000001, 2024-09-11T08:54:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:54:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.262958+0000", 2024-09-11T08:54:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.262958+0000", 2024-09-11T08:54:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.262957+0000", 2024-09-11T08:54:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:54:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.262981+0000", 2024-09-11T08:54:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4065000000000001e-05 2024-09-11T08:54:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.423944+0000", 2024-09-11T08:54:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16096374399999999 2024-09-11T08:54:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.423955+0000", 2024-09-11T08:54:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0254999999999999e-05 2024-09-11T08:54:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.424326+0000", 2024-09-11T08:54:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037135899999999998 2024-09-11T08:54:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:54:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.424359+0000", 2024-09-11T08:54:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3145999999999997e-05 2024-09-11T08:54:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:54:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.425967+0000", 2024-09-11T08:54:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016078609999999999 2024-09-11T08:54:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.426025+0000", 2024-09-11T08:54:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8229000000000001e-05 2024-09-11T08:54:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15621.0:23123 10.f e300/293 10:f5ec2e21:::benchmark_data_smithi092_173109_object23122:head v 300'1482, mlcod=300'1480)", 2024-09-11T08:54:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:53:12.264094+0000", 2024-09-11T08:54:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 102.228821153, 2024-09-11T08:54:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.162181943, 2024-09-11T08:54:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:54:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:54:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:54:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:54:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.264094+0000", 2024-09-11T08:54:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:54:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.264094+0000", 2024-09-11T08:54:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:54:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:54:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.264093+0000", 2024-09-11T08:54:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:54:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:54:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.264192+0000", 2024-09-11T08:54:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9038999999999998e-05 2024-09-11T08:54:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:54:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.423974+0000", 2024-09-11T08:54:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15978205200000001 2024-09-11T08:54:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:54:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.423978+0000", 2024-09-11T08:54:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6079999999999998e-06 2024-09-11T08:54:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:54:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.424369+0000", 2024-09-11T08:54:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00039022200000000002 2024-09-11T08:54:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:54:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.424387+0000", 2024-09-11T08:54:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8046999999999998e-05 2024-09-11T08:54:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:54:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.426213+0000", 2024-09-11T08:54:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018261060000000001 2024-09-11T08:54:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:54:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:54:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:54:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.426276+0000", 2024-09-11T08:54:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2753e-05 2024-09-11T08:54:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:54:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:54:54.618 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:54:54.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:54.619 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:54.619 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:54.720 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:54.742 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:54.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:54.743 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:54.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:54.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:54.884 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:54.884 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:54.985 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:55.020 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:55.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:55.021 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:55.122 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:55.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:55.159 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:55.159 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:55.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:54:55.260 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:55.295 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:55.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:55.297 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:55.397 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:55.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:55 smithi145 ceph-mon[33384]: 3.11 scrub starts 2024-09-11T08:54:55.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:55 smithi145 ceph-mon[33384]: 3.11 scrub ok 2024-09-11T08:54:55.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:55 smithi145 ceph-mon[33384]: 5.1a scrub starts 2024-09-11T08:54:55.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:55 smithi145 ceph-mon[33384]: 5.1a scrub ok 2024-09-11T08:54:55.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:55 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:54:55.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:55 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:54:55.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:55 smithi145 ceph-mon[33384]: 1.3 scrub starts 2024-09-11T08:54:55.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:55 smithi145 ceph-mon[33384]: 1.3 scrub ok 2024-09-11T08:54:55.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:55 smithi145 ceph-mon[33384]: 10.2 scrub ok 2024-09-11T08:54:55.431 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:55.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:55.432 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:55.533 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:55.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[29647]: 3.11 scrub starts 2024-09-11T08:54:55.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[29647]: 3.11 scrub ok 2024-09-11T08:54:55.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[29647]: 5.1a scrub starts 2024-09-11T08:54:55.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[29647]: 5.1a scrub ok 2024-09-11T08:54:55.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:54:55.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:54:55.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[29647]: 1.3 scrub starts 2024-09-11T08:54:55.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[29647]: 1.3 scrub ok 2024-09-11T08:54:55.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[29647]: 10.2 scrub ok 2024-09-11T08:54:55.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[41114]: 3.11 scrub starts 2024-09-11T08:54:55.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[41114]: 3.11 scrub ok 2024-09-11T08:54:55.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[41114]: 5.1a scrub starts 2024-09-11T08:54:55.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[41114]: 5.1a scrub ok 2024-09-11T08:54:55.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:54:55.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:54:55.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[41114]: 1.3 scrub starts 2024-09-11T08:54:55.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[41114]: 1.3 scrub ok 2024-09-11T08:54:55.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:55 smithi092 ceph-mon[41114]: 10.2 scrub ok 2024-09-11T08:54:55.569 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:55.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:55.570 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:55.671 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:55.707 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:55.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:55.709 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:55.810 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:55.840 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:55.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:55.841 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:55.941 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:55.977 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:55.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:55.979 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:56.080 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:56.119 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:56.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:56.121 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:56.221 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:56.244 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:56.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:56.246 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:56.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:56.383 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:56.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:56.384 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:56.485 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:56.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:56.522 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:56.523 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:56.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:56 smithi092 ceph-mon[29647]: 5.1e scrub starts 2024-09-11T08:54:56.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:56 smithi092 ceph-mon[29647]: 5.1e scrub ok 2024-09-11T08:54:56.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:56 smithi092 ceph-mon[29647]: osdmap e329: 8 total, 8 up, 8 in 2024-09-11T08:54:56.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:56 smithi092 ceph-mon[29647]: pgmap v733: 158 pgs: 158 active+clean; 7.1 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T08:54:56.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:56 smithi092 ceph-mon[29647]: 5.11 scrub starts 2024-09-11T08:54:56.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:56 smithi092 ceph-mon[29647]: 5.11 scrub ok 2024-09-11T08:54:56.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:56 smithi092 ceph-mon[41114]: 5.1e scrub starts 2024-09-11T08:54:56.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:56 smithi092 ceph-mon[41114]: 5.1e scrub ok 2024-09-11T08:54:56.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:56 smithi092 ceph-mon[41114]: osdmap e329: 8 total, 8 up, 8 in 2024-09-11T08:54:56.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:56 smithi092 ceph-mon[41114]: pgmap v733: 158 pgs: 158 active+clean; 7.1 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T08:54:56.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:56 smithi092 ceph-mon[41114]: 5.11 scrub starts 2024-09-11T08:54:56.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:56 smithi092 ceph-mon[41114]: 5.11 scrub ok 2024-09-11T08:54:56.624 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:56.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:56.660 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:56.660 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:56.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:56 smithi145 ceph-mon[33384]: 5.1e scrub starts 2024-09-11T08:54:56.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:56 smithi145 ceph-mon[33384]: 5.1e scrub ok 2024-09-11T08:54:56.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:56 smithi145 ceph-mon[33384]: osdmap e329: 8 total, 8 up, 8 in 2024-09-11T08:54:56.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:56 smithi145 ceph-mon[33384]: pgmap v733: 158 pgs: 158 active+clean; 7.1 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T08:54:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:56 smithi145 ceph-mon[33384]: 5.11 scrub starts 2024-09-11T08:54:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:56 smithi145 ceph-mon[33384]: 5.11 scrub ok 2024-09-11T08:54:56.761 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:56.800 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:56.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:56.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:56.902 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:56.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:56.939 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:56.940 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:57.040 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:57.081 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:57.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:57.082 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:57.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:57.221 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:57.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:57.222 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:57.323 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:57.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:57.358 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:57.359 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:57 smithi145 ceph-mon[33384]: 5.18 scrub starts 2024-09-11T08:54:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:57 smithi145 ceph-mon[33384]: 5.18 scrub ok 2024-09-11T08:54:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:57 smithi145 ceph-mon[33384]: 10.0 scrub ok 2024-09-11T08:54:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:57 smithi145 ceph-mon[33384]: 10.f scrub ok 2024-09-11T08:54:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:57 smithi145 ceph-mon[33384]: 6.a scrub starts 2024-09-11T08:54:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:57 smithi145 ceph-mon[33384]: 6.a scrub ok 2024-09-11T08:54:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:57 smithi145 ceph-mon[33384]: 5.3 scrub starts 2024-09-11T08:54:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:57 smithi145 ceph-mon[33384]: 5.3 scrub ok 2024-09-11T08:54:57.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:54:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:54:57.459 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:57.493 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:57.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:57.494 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:57.532 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[29647]: 5.18 scrub starts 2024-09-11T08:54:57.532 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[29647]: 5.18 scrub ok 2024-09-11T08:54:57.532 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[29647]: 10.0 scrub ok 2024-09-11T08:54:57.532 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[29647]: 10.f scrub ok 2024-09-11T08:54:57.532 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[29647]: 6.a scrub starts 2024-09-11T08:54:57.532 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[29647]: 6.a scrub ok 2024-09-11T08:54:57.533 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[29647]: 5.3 scrub starts 2024-09-11T08:54:57.533 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[29647]: 5.3 scrub ok 2024-09-11T08:54:57.533 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[41114]: 5.18 scrub starts 2024-09-11T08:54:57.533 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[41114]: 5.18 scrub ok 2024-09-11T08:54:57.533 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[41114]: 10.0 scrub ok 2024-09-11T08:54:57.533 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[41114]: 10.f scrub ok 2024-09-11T08:54:57.533 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[41114]: 6.a scrub starts 2024-09-11T08:54:57.533 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[41114]: 6.a scrub ok 2024-09-11T08:54:57.533 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[41114]: 5.3 scrub starts 2024-09-11T08:54:57.534 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:57 smithi092 ceph-mon[41114]: 5.3 scrub ok 2024-09-11T08:54:57.595 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:57.628 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:57.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:57.629 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:57.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:57.761 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:57.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:57.762 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:57.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:54:57.897 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:57.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:57.898 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:54:57.999 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:58.032 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:58.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:58.033 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:58.133 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:58.173 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:58.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:58.175 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:58.275 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:58.308 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:58.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:58.309 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:58.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:58 smithi092 ceph-mon[29647]: 3.3 scrub starts 2024-09-11T08:54:58.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:58 smithi092 ceph-mon[29647]: 3.3 scrub ok 2024-09-11T08:54:58.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:58 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:58.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:58 smithi092 ceph-mon[29647]: pgmap v734: 158 pgs: 158 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 0 B/s wr, 1.89k op/s 2024-09-11T08:54:58.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:58 smithi092 ceph-mon[41114]: 3.3 scrub starts 2024-09-11T08:54:58.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:58 smithi092 ceph-mon[41114]: 3.3 scrub ok 2024-09-11T08:54:58.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:58 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:58.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:58 smithi092 ceph-mon[41114]: pgmap v734: 158 pgs: 158 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 0 B/s wr, 1.89k op/s 2024-09-11T08:54:58.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:58.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:58.447 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:58.447 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:58.548 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:58.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:58 smithi145 ceph-mon[33384]: 3.3 scrub starts 2024-09-11T08:54:58.564 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:58 smithi145 ceph-mon[33384]: 3.3 scrub ok 2024-09-11T08:54:58.564 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:58 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:54:58.564 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:58 smithi145 ceph-mon[33384]: pgmap v734: 158 pgs: 158 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 0 B/s wr, 1.89k op/s 2024-09-11T08:54:58.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:58.582 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:58.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:58.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:54:58.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:58.715 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:58.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:54:58.816 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:58.846 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:58.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:58.847 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:58.903 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-11T08:54:58.948 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:54:58.991 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:58.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:58.992 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:54:59.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:59.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:59.127 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:59.127 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:59.228 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:54:59.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:59.295 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:59.295 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:54:59.379 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:54:59.396 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:54:59.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:59 smithi145 ceph-mon[33384]: 6.1b scrub starts 2024-09-11T08:54:59.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:54:59 smithi145 ceph-mon[33384]: 6.1b scrub ok 2024-09-11T08:54:59.456 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:59.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:59.457 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:54:59.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:59 smithi092 ceph-mon[29647]: 6.1b scrub starts 2024-09-11T08:54:59.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:54:59 smithi092 ceph-mon[29647]: 6.1b scrub ok 2024-09-11T08:54:59.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:59 smithi092 ceph-mon[41114]: 6.1b scrub starts 2024-09-11T08:54:59.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:54:59 smithi092 ceph-mon[41114]: 6.1b scrub ok 2024-09-11T08:54:59.549 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:54:59.549 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:54:59.558 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:54:59.578 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:54:59.578 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:54:59.596 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:59.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:59.597 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:54:59.622 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:54:59.622 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:54:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:54:59.654 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:54:59.654 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:54:59.682 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:54:59.683 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:54:59.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:54:59.715 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:54:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:54:59.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:59.736 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:54:59.736 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:54:59.754 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:54:59.754 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:54:59.783 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:54:59.783 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:54:59.837 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:54:59.883 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:54:59.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:54:59.884 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:54:59.985 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:00.030 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:00.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:00.031 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:00.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:00.168 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:00.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:00.169 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:00.270 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:00.306 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:00.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:00.307 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:00.408 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:00.416 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:00 smithi145 ceph-mon[33384]: 6.1 scrub starts 2024-09-11T08:55:00.416 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:00 smithi145 ceph-mon[33384]: 6.1 scrub ok 2024-09-11T08:55:00.416 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:00 smithi145 ceph-mon[33384]: pgmap v735: 158 pgs: 158 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 0 B/s wr, 1.29k op/s 2024-09-11T08:55:00.416 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:00 smithi145 ceph-mon[33384]: 5.2 scrub starts 2024-09-11T08:55:00.416 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:00 smithi145 ceph-mon[33384]: 5.2 scrub ok 2024-09-11T08:55:00.416 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:00 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1181285311' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:00.416 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:55:00.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:00.449 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:00.449 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:00.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:00.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:00 smithi092 ceph-mon[29647]: 6.1 scrub starts 2024-09-11T08:55:00.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:00 smithi092 ceph-mon[29647]: 6.1 scrub ok 2024-09-11T08:55:00.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:00 smithi092 ceph-mon[29647]: pgmap v735: 158 pgs: 158 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 0 B/s wr, 1.29k op/s 2024-09-11T08:55:00.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:00 smithi092 ceph-mon[29647]: 5.2 scrub starts 2024-09-11T08:55:00.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:00 smithi092 ceph-mon[29647]: 5.2 scrub ok 2024-09-11T08:55:00.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:00 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1181285311' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:00.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:55:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:00 smithi092 ceph-mon[41114]: 6.1 scrub starts 2024-09-11T08:55:00.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:00 smithi092 ceph-mon[41114]: 6.1 scrub ok 2024-09-11T08:55:00.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:00 smithi092 ceph-mon[41114]: pgmap v735: 158 pgs: 158 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 0 B/s wr, 1.29k op/s 2024-09-11T08:55:00.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:00 smithi092 ceph-mon[41114]: 5.2 scrub starts 2024-09-11T08:55:00.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:00 smithi092 ceph-mon[41114]: 5.2 scrub ok 2024-09-11T08:55:00.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:00 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1181285311' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:00.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:55:00.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:00.580 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:00.580 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:00.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:00.711 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:00.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:00.712 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:00.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:00.851 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:00.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:00.852 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:00.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:00.991 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:00.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:00.993 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:01.094 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:01.131 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:01.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:01.133 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:01.233 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:01.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:01.284 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:01.284 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:01.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:01 smithi092 ceph-mon[29647]: 4.4 scrub starts 2024-09-11T08:55:01.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:01 smithi092 ceph-mon[29647]: 4.4 scrub ok 2024-09-11T08:55:01.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:01 smithi092 ceph-mon[29647]: 4.10 scrub starts 2024-09-11T08:55:01.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:01 smithi092 ceph-mon[29647]: 4.10 scrub ok 2024-09-11T08:55:01.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:01 smithi092 ceph-mon[41114]: 4.4 scrub starts 2024-09-11T08:55:01.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:01 smithi092 ceph-mon[41114]: 4.4 scrub ok 2024-09-11T08:55:01.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:01 smithi092 ceph-mon[41114]: 4.10 scrub starts 2024-09-11T08:55:01.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:01 smithi092 ceph-mon[41114]: 4.10 scrub ok 2024-09-11T08:55:01.384 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:01.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:01 smithi145 ceph-mon[33384]: 4.4 scrub starts 2024-09-11T08:55:01.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:01 smithi145 ceph-mon[33384]: 4.4 scrub ok 2024-09-11T08:55:01.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:01 smithi145 ceph-mon[33384]: 4.10 scrub starts 2024-09-11T08:55:01.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:01 smithi145 ceph-mon[33384]: 4.10 scrub ok 2024-09-11T08:55:01.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:01.413 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:01.413 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:01.514 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:01.555 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:01.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:01.556 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:01.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:01.687 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:01.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:01.688 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:01.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:01.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:55:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:55:01] "GET /metrics HTTP/1.1" 200 224830 "" "Prometheus/2.43.0" 2024-09-11T08:55:01.818 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:01.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:01.819 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:01.871 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:55:01.871 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:55:01.920 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:01.959 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:01.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:01.960 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:02.061 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:02.092 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:02.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:02.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:02.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:02.236 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:02.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:02.237 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:02.338 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:02.373 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:02.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:02.374 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:02.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:02.492 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:02 smithi092 ceph-mon[29647]: 1.2 scrub starts 2024-09-11T08:55:02.492 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:02 smithi092 ceph-mon[29647]: 1.2 scrub ok 2024-09-11T08:55:02.492 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:02 smithi092 ceph-mon[29647]: pgmap v736: 158 pgs: 158 active+clean; 6.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 0 B/s wr, 988 op/s 2024-09-11T08:55:02.492 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:02 smithi092 ceph-mon[41114]: 1.2 scrub starts 2024-09-11T08:55:02.492 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:02 smithi092 ceph-mon[41114]: 1.2 scrub ok 2024-09-11T08:55:02.493 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:02 smithi092 ceph-mon[41114]: pgmap v736: 158 pgs: 158 active+clean; 6.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 0 B/s wr, 988 op/s 2024-09-11T08:55:02.515 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:02.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:02.516 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:02.575 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:02 smithi145 ceph-mon[33384]: 1.2 scrub starts 2024-09-11T08:55:02.575 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:02 smithi145 ceph-mon[33384]: 1.2 scrub ok 2024-09-11T08:55:02.575 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:02 smithi145 ceph-mon[33384]: pgmap v736: 158 pgs: 158 active+clean; 6.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 0 B/s wr, 988 op/s 2024-09-11T08:55:02.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:02.628 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-11T08:55:02.650 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:02.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:02.651 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:02.663 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:55:02.752 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:02.791 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:02.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:02.792 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:02.893 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:02.912 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:55:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:55:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:55:02.933 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:02.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:02.934 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:03.035 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:03.063 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:03.080 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:55:03.083 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:55:03.083 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:55:03.083 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-11T08:55:03.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-11T08:55:03.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:03.105 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:03.105 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:03.206 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:03.218 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:55:03.246 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:03.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:03.247 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:03.348 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:03.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:03.382 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:03.382 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:03.397 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:03 smithi092 ceph-mon[29647]: 4.e scrub starts 2024-09-11T08:55:03.397 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:03 smithi092 ceph-mon[29647]: 4.e scrub ok 2024-09-11T08:55:03.397 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:03 smithi092 ceph-mon[29647]: 5.4 scrub starts 2024-09-11T08:55:03.397 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:03 smithi092 ceph-mon[29647]: 5.4 scrub ok 2024-09-11T08:55:03.397 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:03 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1133444964' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:55:03.397 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:03 smithi092 ceph-mon[41114]: 4.e scrub starts 2024-09-11T08:55:03.398 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:03 smithi092 ceph-mon[41114]: 4.e scrub ok 2024-09-11T08:55:03.398 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:03 smithi092 ceph-mon[41114]: 5.4 scrub starts 2024-09-11T08:55:03.398 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:03 smithi092 ceph-mon[41114]: 5.4 scrub ok 2024-09-11T08:55:03.398 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:03 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1133444964' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:55:03.467 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:03 smithi145 ceph-mon[33384]: 4.e scrub starts 2024-09-11T08:55:03.467 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:03 smithi145 ceph-mon[33384]: 4.e scrub ok 2024-09-11T08:55:03.467 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:03 smithi145 ceph-mon[33384]: 5.4 scrub starts 2024-09-11T08:55:03.467 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:03 smithi145 ceph-mon[33384]: 5.4 scrub ok 2024-09-11T08:55:03.467 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:03 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1133444964' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:55:03.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:03.514 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:03.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:03.515 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:03.617 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:03.650 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:03.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:03.651 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:03.751 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:03.783 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:03.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:03.785 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:03.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:03.918 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:03.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:03.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:04.019 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:04.058 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:04.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:04.059 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:04.160 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:04.163 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 08:55:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=sqlstore.transactions t=2024-09-11T08:55:03.885192564Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:55:04.192 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:04.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:04.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:04.293 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:04.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:04.328 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:04.328 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:04.429 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:04.465 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:04.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:04.466 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:04.554 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:04 smithi145 ceph-mon[33384]: 3.e scrub starts 2024-09-11T08:55:04.554 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:04 smithi145 ceph-mon[33384]: 3.e scrub ok 2024-09-11T08:55:04.554 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:04 smithi145 ceph-mon[33384]: 4.f scrub starts 2024-09-11T08:55:04.554 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:04 smithi145 ceph-mon[33384]: 4.f scrub ok 2024-09-11T08:55:04.554 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:04 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:55:04.554 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:04 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:55:04.554 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:04 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1133444964' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:55:04.554 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:04 smithi145 ceph-mon[33384]: osdmap e330: 8 total, 8 up, 8 in 2024-09-11T08:55:04.554 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:04 smithi145 ceph-mon[33384]: pgmap v738: 158 pgs: 158 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 0 B/s wr, 1.54k op/s 2024-09-11T08:55:04.555 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:04 smithi145 ceph-mon[33384]: 3.12 deep-scrub starts 2024-09-11T08:55:04.555 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:04 smithi145 ceph-mon[33384]: 3.12 deep-scrub ok 2024-09-11T08:55:04.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[29647]: 3.e scrub starts 2024-09-11T08:55:04.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[29647]: 3.e scrub ok 2024-09-11T08:55:04.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[29647]: 4.f scrub starts 2024-09-11T08:55:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[29647]: 4.f scrub ok 2024-09-11T08:55:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:55:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:55:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1133444964' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:55:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[29647]: osdmap e330: 8 total, 8 up, 8 in 2024-09-11T08:55:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[29647]: pgmap v738: 158 pgs: 158 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 0 B/s wr, 1.54k op/s 2024-09-11T08:55:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[29647]: 3.12 deep-scrub starts 2024-09-11T08:55:04.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[29647]: 3.12 deep-scrub ok 2024-09-11T08:55:04.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[41114]: 3.e scrub starts 2024-09-11T08:55:04.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[41114]: 3.e scrub ok 2024-09-11T08:55:04.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[41114]: 4.f scrub starts 2024-09-11T08:55:04.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[41114]: 4.f scrub ok 2024-09-11T08:55:04.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:55:04.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:55:04.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1133444964' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:55:04.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[41114]: osdmap e330: 8 total, 8 up, 8 in 2024-09-11T08:55:04.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[41114]: pgmap v738: 158 pgs: 158 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 0 B/s wr, 1.54k op/s 2024-09-11T08:55:04.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[41114]: 3.12 deep-scrub starts 2024-09-11T08:55:04.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:04 smithi092 ceph-mon[41114]: 3.12 deep-scrub ok 2024-09-11T08:55:04.567 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:04.615 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:04.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:04.616 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:04.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:04.748 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:04.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:04.749 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:04.850 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:04.889 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:04.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:04.890 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:04.991 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:05.024 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:05.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:05.025 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:05.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:05.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:05.160 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:05.160 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:05.261 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:05.295 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:05.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:05.297 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:05.397 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:05.431 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:05 smithi145 ceph-mon[33384]: 6.d scrub starts 2024-09-11T08:55:05.431 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:05 smithi145 ceph-mon[33384]: 6.d scrub ok 2024-09-11T08:55:05.431 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:05 smithi145 ceph-mon[33384]: 3.7 scrub starts 2024-09-11T08:55:05.431 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:05 smithi145 ceph-mon[33384]: 3.7 scrub ok 2024-09-11T08:55:05.435 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:05.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:05.436 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:05.470 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:55:05.470 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:55:05.538 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:05.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:05 smithi092 ceph-mon[29647]: 6.d scrub starts 2024-09-11T08:55:05.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:05 smithi092 ceph-mon[29647]: 6.d scrub ok 2024-09-11T08:55:05.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:05 smithi092 ceph-mon[29647]: 3.7 scrub starts 2024-09-11T08:55:05.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:05 smithi092 ceph-mon[29647]: 3.7 scrub ok 2024-09-11T08:55:05.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:05 smithi092 ceph-mon[41114]: 6.d scrub starts 2024-09-11T08:55:05.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:05 smithi092 ceph-mon[41114]: 6.d scrub ok 2024-09-11T08:55:05.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:05 smithi092 ceph-mon[41114]: 3.7 scrub starts 2024-09-11T08:55:05.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:05 smithi092 ceph-mon[41114]: 3.7 scrub ok 2024-09-11T08:55:05.572 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:05.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:05.573 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:05.674 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:05.699 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:05.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:05.701 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:05.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:05.842 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:05.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:05.844 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:05.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:05.986 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:05.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:05.988 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:06.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:06.122 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:06.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:06.124 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:06.225 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:06.258 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:06.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:06.259 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:06.272 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:06 smithi092 ceph-mon[29647]: 10.3 deep-scrub starts 2024-09-11T08:55:06.273 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:06 smithi092 ceph-mon[29647]: pgmap v739: 158 pgs: 158 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 0 B/s wr, 1.27k op/s 2024-09-11T08:55:06.273 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:06 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T08:55:06.273 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:06 smithi092 ceph-mon[29647]: 5.d scrub ok 2024-09-11T08:55:06.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:06 smithi092 ceph-mon[41114]: 10.3 deep-scrub starts 2024-09-11T08:55:06.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:06 smithi092 ceph-mon[41114]: pgmap v739: 158 pgs: 158 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 0 B/s wr, 1.27k op/s 2024-09-11T08:55:06.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:06 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T08:55:06.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:06 smithi092 ceph-mon[41114]: 5.d scrub ok 2024-09-11T08:55:06.282 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:06.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:06.361 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:55:06.362 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:55:06.396 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:06.397 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:55:06.397 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:55:06.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:06.398 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:06.420 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:55:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:55:06.435 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:55:06.435 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:55:06.492 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:55:06.492 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:55:06.500 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:06.521 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:55:06.521 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:55:06.535 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:06.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:06.536 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:06.553 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:55:06.553 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:55:06.584 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:55:06.584 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:55:06.637 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:06.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:06 smithi145 ceph-mon[33384]: 10.3 deep-scrub starts 2024-09-11T08:55:06.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:06 smithi145 ceph-mon[33384]: pgmap v739: 158 pgs: 158 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 0 B/s wr, 1.27k op/s 2024-09-11T08:55:06.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:06 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T08:55:06.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:06 smithi145 ceph-mon[33384]: 5.d scrub ok 2024-09-11T08:55:06.671 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:55:06.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:06.674 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:06.674 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:06.775 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:06.811 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:06.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:06.812 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:06.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:06.966 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:06.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:06.967 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:07.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:07.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:07.103 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:07.103 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:07.204 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:07.244 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:07.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:07.245 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:07.345 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:07.384 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:07.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:07.385 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:07.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:55:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:55:07.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:07 smithi145 ceph-mon[33384]: 5.f scrub starts 2024-09-11T08:55:07.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:07 smithi145 ceph-mon[33384]: 5.f scrub ok 2024-09-11T08:55:07.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:07 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2396253782' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:07.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:07 smithi145 ceph-mon[33384]: 4.0 scrub starts 2024-09-11T08:55:07.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:07 smithi145 ceph-mon[33384]: 4.0 scrub ok 2024-09-11T08:55:07.485 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:07.519 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:07.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:07.520 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:07.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:07 smithi092 ceph-mon[29647]: 5.f scrub starts 2024-09-11T08:55:07.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:07 smithi092 ceph-mon[29647]: 5.f scrub ok 2024-09-11T08:55:07.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:07 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2396253782' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:07.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:07 smithi092 ceph-mon[29647]: 4.0 scrub starts 2024-09-11T08:55:07.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:07 smithi092 ceph-mon[29647]: 4.0 scrub ok 2024-09-11T08:55:07.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:07 smithi092 ceph-mon[41114]: 5.f scrub starts 2024-09-11T08:55:07.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:07 smithi092 ceph-mon[41114]: 5.f scrub ok 2024-09-11T08:55:07.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:07 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2396253782' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:07.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:07 smithi092 ceph-mon[41114]: 4.0 scrub starts 2024-09-11T08:55:07.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:07 smithi092 ceph-mon[41114]: 4.0 scrub ok 2024-09-11T08:55:07.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:07.655 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:07.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:07.656 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:07.757 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:07.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:07.792 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:07.792 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:07.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:07.930 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:07.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:07.932 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:08.032 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:08.063 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:08.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:08.064 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:08.165 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:08.196 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:08.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:08.197 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:08.298 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:08.326 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:08.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:08.327 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:08.428 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:08.458 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:08.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:08.459 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:08.536 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:08.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:08.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:08 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:08.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:08 smithi092 ceph-mon[29647]: 10.6 deep-scrub starts 2024-09-11T08:55:08.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:08 smithi092 ceph-mon[29647]: pgmap v740: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 0 B/s wr, 1.25k op/s 2024-09-11T08:55:08.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:08 smithi092 ceph-mon[29647]: 3.a deep-scrub starts 2024-09-11T08:55:08.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:08 smithi092 ceph-mon[29647]: 3.a deep-scrub ok 2024-09-11T08:55:08.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:08 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:08.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:08 smithi092 ceph-mon[41114]: 10.6 deep-scrub starts 2024-09-11T08:55:08.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:08 smithi092 ceph-mon[41114]: pgmap v740: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 0 B/s wr, 1.25k op/s 2024-09-11T08:55:08.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:08 smithi092 ceph-mon[41114]: 3.a deep-scrub starts 2024-09-11T08:55:08.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:08 smithi092 ceph-mon[41114]: 3.a deep-scrub ok 2024-09-11T08:55:08.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:08.609 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:08.610 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:08.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:08 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:08.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:08 smithi145 ceph-mon[33384]: 10.6 deep-scrub starts 2024-09-11T08:55:08.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:08 smithi145 ceph-mon[33384]: pgmap v740: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 0 B/s wr, 1.25k op/s 2024-09-11T08:55:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:08 smithi145 ceph-mon[33384]: 3.a deep-scrub starts 2024-09-11T08:55:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:08 smithi145 ceph-mon[33384]: 3.a deep-scrub ok 2024-09-11T08:55:08.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:08.732 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:08.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:08.733 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:08.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:08.877 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:08.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:08.878 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:08.979 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:09.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:09.011 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:09.011 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:09.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:09.138 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:09.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:09.139 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:09.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:09.274 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:09.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:09.276 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:09.376 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:09.397 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-11T08:55:09.422 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:09.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:09.424 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:09.525 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:09.553 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:09.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:09.554 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:09.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:09 smithi092 ceph-mon[29647]: 5.5 scrub starts 2024-09-11T08:55:09.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:09 smithi092 ceph-mon[29647]: 5.5 scrub ok 2024-09-11T08:55:09.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:09 smithi092 ceph-mon[29647]: 1.0 deep-scrub starts 2024-09-11T08:55:09.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:09 smithi092 ceph-mon[29647]: 1.0 deep-scrub ok 2024-09-11T08:55:09.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:09 smithi092 ceph-mon[29647]: 5.b scrub starts 2024-09-11T08:55:09.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:09 smithi092 ceph-mon[29647]: 5.b scrub ok 2024-09-11T08:55:09.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:09 smithi092 ceph-mon[41114]: 5.5 scrub starts 2024-09-11T08:55:09.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:09 smithi092 ceph-mon[41114]: 5.5 scrub ok 2024-09-11T08:55:09.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:09 smithi092 ceph-mon[41114]: 1.0 deep-scrub starts 2024-09-11T08:55:09.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:09 smithi092 ceph-mon[41114]: 1.0 deep-scrub ok 2024-09-11T08:55:09.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:09 smithi092 ceph-mon[41114]: 5.b scrub starts 2024-09-11T08:55:09.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:09 smithi092 ceph-mon[41114]: 5.b scrub ok 2024-09-11T08:55:09.655 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:09.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:09 smithi145 ceph-mon[33384]: 5.5 scrub starts 2024-09-11T08:55:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:09 smithi145 ceph-mon[33384]: 5.5 scrub ok 2024-09-11T08:55:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:09 smithi145 ceph-mon[33384]: 1.0 deep-scrub starts 2024-09-11T08:55:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:09 smithi145 ceph-mon[33384]: 1.0 deep-scrub ok 2024-09-11T08:55:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:09 smithi145 ceph-mon[33384]: 5.b scrub starts 2024-09-11T08:55:09.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:09 smithi145 ceph-mon[33384]: 5.b scrub ok 2024-09-11T08:55:09.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:09.692 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:09.692 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:09.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:09.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:09.823 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:09.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:55:09.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:10.003 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:10.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:10.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:10.079 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:55:10.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:10.137 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:10.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:10.138 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:10.239 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:10.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:10.265 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:10.366 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:10.390 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:10.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:10.391 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:10.492 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:10.514 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:10.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:10.515 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:10.560 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:10 smithi145 ceph-mon[33384]: pgmap v741: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 0 B/s wr, 1.25k op/s 2024-09-11T08:55:10.560 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:10 smithi145 ceph-mon[33384]: 4.1e scrub starts 2024-09-11T08:55:10.560 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:10 smithi145 ceph-mon[33384]: 4.1e scrub ok 2024-09-11T08:55:10.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:10 smithi092 ceph-mon[29647]: pgmap v741: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 0 B/s wr, 1.25k op/s 2024-09-11T08:55:10.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:10 smithi092 ceph-mon[29647]: 4.1e scrub starts 2024-09-11T08:55:10.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:10 smithi092 ceph-mon[29647]: 4.1e scrub ok 2024-09-11T08:55:10.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:10 smithi092 ceph-mon[41114]: pgmap v741: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 0 B/s wr, 1.25k op/s 2024-09-11T08:55:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:10 smithi092 ceph-mon[41114]: 4.1e scrub starts 2024-09-11T08:55:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:10 smithi092 ceph-mon[41114]: 4.1e scrub ok 2024-09-11T08:55:10.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:10.655 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:10.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:10.656 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:10.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:10.790 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:10.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:10.791 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:10.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:10.926 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:10.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:10.927 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:11.028 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:11.060 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:11.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:11.062 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:11.163 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:11.199 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:11.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:11.201 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:11.302 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:11.351 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:11.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:11.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:11.375 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:11 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:55:11.375 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:11 smithi145 ceph-mon[33384]: 5.8 scrub ok 2024-09-11T08:55:11.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:11.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:11.488 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:11.488 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:11.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:11 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:55:11.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:11 smithi092 ceph-mon[29647]: 5.8 scrub ok 2024-09-11T08:55:11.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:11 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:55:11.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:11 smithi092 ceph-mon[41114]: 5.8 scrub ok 2024-09-11T08:55:11.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:55:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:55:11] "GET /metrics HTTP/1.1" 200 224830 "" "Prometheus/2.43.0" 2024-09-11T08:55:11.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:11.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:11.639 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:11.640 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:11.740 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:11.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:11.785 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:11.785 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:11.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:11.922 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:11.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:11.923 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:12.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:12.059 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:55:12.059 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T08:55:12.059 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T08:55:12.059 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:55:12.059 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.059 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122629 9.16 9.b9988b96 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:55:12.059 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.542528+0000", 2024-09-11T08:55:12.059 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 284.51381247400002, 2024-09-11T08:55:12.059 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99068280600000003, 2024-09-11T08:55:12.059 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.059 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:55:12.060 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.060 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:12.060 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:12.060 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122629 2024-09-11T08:55:12.060 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.060 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.060 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.060 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.060 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542528+0000", 2024-09-11T08:55:12.060 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.060 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.060 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.060 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.061 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542528+0000", 2024-09-11T08:55:12.061 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.061 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.061 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.061 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.061 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542526+0000", 2024-09-11T08:55:12.061 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:55:12.061 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.061 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.061 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.061 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542748+0000", 2024-09-11T08:55:12.061 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000221917 2024-09-11T08:55:12.062 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.062 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.062 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.062 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542773+0000", 2024-09-11T08:55:12.062 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4735e-05 2024-09-11T08:55:12.062 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.062 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.062 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.062 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542780+0000", 2024-09-11T08:55:12.062 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1239999999999999e-06 2024-09-11T08:55:12.062 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.062 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.062 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.063 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542829+0000", 2024-09-11T08:55:12.063 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8856000000000001e-05 2024-09-11T08:55:12.063 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.063 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.063 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:12.063 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542835+0000", 2024-09-11T08:55:12.063 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3509999999999998e-06 2024-09-11T08:55:12.063 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.063 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.063 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.063 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533203+0000", 2024-09-11T08:55:12.063 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99036770799999996 2024-09-11T08:55:12.063 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.063 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.063 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.064 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533210+0000", 2024-09-11T08:55:12.064 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7600000000000002e-06 2024-09-11T08:55:12.064 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.064 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.064 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.064 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.064 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.064 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122630 9.16 9.b4953096 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:55:12.064 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.542807+0000", 2024-09-11T08:55:12.064 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 284.51353353799999, 2024-09-11T08:55:12.064 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.990428643, 2024-09-11T08:55:12.064 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.064 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:55:12.064 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.065 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:12.065 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:12.065 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122630 2024-09-11T08:55:12.065 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.065 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.065 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.065 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.065 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542807+0000", 2024-09-11T08:55:12.065 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.065 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.065 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.065 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.065 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542807+0000", 2024-09-11T08:55:12.065 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.066 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.066 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.066 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.066 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542793+0000", 2024-09-11T08:55:12.066 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999866 2024-09-11T08:55:12.066 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.066 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.066 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.066 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542848+0000", 2024-09-11T08:55:12.066 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4406000000000001e-05 2024-09-11T08:55:12.066 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.066 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.066 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.066 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542856+0000", 2024-09-11T08:55:12.066 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.4090000000000003e-06 2024-09-11T08:55:12.067 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.067 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.067 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.067 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542861+0000", 2024-09-11T08:55:12.067 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0429999999999998e-06 2024-09-11T08:55:12.067 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.067 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.067 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.067 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542889+0000", 2024-09-11T08:55:12.067 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7464999999999999e-05 2024-09-11T08:55:12.067 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.067 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.067 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:12.067 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542901+0000", 2024-09-11T08:55:12.068 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2782999999999999e-05 2024-09-11T08:55:12.068 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.068 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.068 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.068 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533229+0000", 2024-09-11T08:55:12.068 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99032747099999996 2024-09-11T08:55:12.068 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.068 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.068 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.068 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533235+0000", 2024-09-11T08:55:12.068 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2720000000000003e-06 2024-09-11T08:55:12.068 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.068 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.069 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.069 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.069 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.069 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122638 9.16 9.47eb3dd6 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:55:12.069 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.542872+0000", 2024-09-11T08:55:12.069 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 284.51346856700002, 2024-09-11T08:55:12.069 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99038658199999996, 2024-09-11T08:55:12.069 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.069 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:55:12.069 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.069 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:12.069 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:12.070 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122638 2024-09-11T08:55:12.070 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.070 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.070 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.070 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.070 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542872+0000", 2024-09-11T08:55:12.070 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.070 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.070 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.070 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.070 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542872+0000", 2024-09-11T08:55:12.070 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.070 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.071 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.071 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.071 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542871+0000", 2024-09-11T08:55:12.071 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:55:12.071 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.071 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.071 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.071 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542904+0000", 2024-09-11T08:55:12.071 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2978999999999997e-05 2024-09-11T08:55:12.071 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.071 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.071 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.071 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542912+0000", 2024-09-11T08:55:12.072 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.763e-06 2024-09-11T08:55:12.072 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.072 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.072 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.072 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542916+0000", 2024-09-11T08:55:12.072 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7520000000000004e-06 2024-09-11T08:55:12.072 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.072 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.072 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.072 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542942+0000", 2024-09-11T08:55:12.072 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5474999999999999e-05 2024-09-11T08:55:12.072 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.073 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.073 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:12.073 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.542946+0000", 2024-09-11T08:55:12.073 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7919999999999999e-06 2024-09-11T08:55:12.073 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.073 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.073 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.073 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533252+0000", 2024-09-11T08:55:12.073 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.990306142 2024-09-11T08:55:12.073 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.073 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.073 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.074 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533258+0000", 2024-09-11T08:55:12.074 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2199999999999997e-06 2024-09-11T08:55:12.074 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.074 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.074 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.074 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.075 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.075 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122650 9.16 9.a38ecfb6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:55:12.075 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.543066+0000", 2024-09-11T08:55:12.075 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 284.51327398000001, 2024-09-11T08:55:12.075 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99021418400000005, 2024-09-11T08:55:12.075 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.075 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:55:12.075 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.075 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:12.075 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:12.075 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122650 2024-09-11T08:55:12.075 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.076 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.076 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.076 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.076 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.543066+0000", 2024-09-11T08:55:12.076 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.076 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.076 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.076 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.076 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.543066+0000", 2024-09-11T08:55:12.076 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.076 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.076 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.076 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.077 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.543065+0000", 2024-09-11T08:55:12.077 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:12.077 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.077 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.077 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.077 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.543094+0000", 2024-09-11T08:55:12.077 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.8459e-05 2024-09-11T08:55:12.077 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.077 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.077 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.077 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.543107+0000", 2024-09-11T08:55:12.077 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2957e-05 2024-09-11T08:55:12.077 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.078 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.078 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.078 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.543112+0000", 2024-09-11T08:55:12.078 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.2499999999999997e-06 2024-09-11T08:55:12.078 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.078 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.078 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.078 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.543142+0000", 2024-09-11T08:55:12.078 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9937000000000002e-05 2024-09-11T08:55:12.078 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.078 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.078 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:12.079 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.543146+0000", 2024-09-11T08:55:12.079 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.279e-06 2024-09-11T08:55:12.079 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.079 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.079 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.079 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533274+0000", 2024-09-11T08:55:12.079 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99012810799999995 2024-09-11T08:55:12.079 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.079 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.079 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.079 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533280+0000", 2024-09-11T08:55:12.079 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0719999999999996e-06 2024-09-11T08:55:12.079 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.080 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.080 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.080 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.080 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.080 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122665 9.16 9.f6768356 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:55:12.080 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.553424+0000", 2024-09-11T08:55:12.080 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 284.50291657499997, 2024-09-11T08:55:12.080 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97987885100000005, 2024-09-11T08:55:12.080 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.080 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:55:12.080 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.080 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:12.080 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:12.081 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122665 2024-09-11T08:55:12.081 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.081 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.081 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.081 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.081 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.553424+0000", 2024-09-11T08:55:12.081 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.081 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.081 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.081 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.081 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.553424+0000", 2024-09-11T08:55:12.081 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.082 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.082 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.082 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.082 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.553422+0000", 2024-09-11T08:55:12.082 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:55:12.082 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.082 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.082 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.082 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.553473+0000", 2024-09-11T08:55:12.082 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1595999999999999e-05 2024-09-11T08:55:12.082 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.082 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.082 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.083 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.553488+0000", 2024-09-11T08:55:12.083 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.473e-05 2024-09-11T08:55:12.083 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.083 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.083 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.083 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.553493+0000", 2024-09-11T08:55:12.083 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1329999999999998e-06 2024-09-11T08:55:12.083 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.083 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.083 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.083 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.553529+0000", 2024-09-11T08:55:12.083 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5831999999999998e-05 2024-09-11T08:55:12.083 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.083 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.084 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:12.084 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.553535+0000", 2024-09-11T08:55:12.084 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.395e-06 2024-09-11T08:55:12.084 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.084 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.084 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.084 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533296+0000", 2024-09-11T08:55:12.084 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97976175600000004 2024-09-11T08:55:12.084 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.084 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.084 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.084 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533302+0000", 2024-09-11T08:55:12.084 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0900000000000001e-06 2024-09-11T08:55:12.085 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.085 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.085 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.085 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.085 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.085 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122672 9.10 9.2df87d70 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:55:12.085 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.558554+0000", 2024-09-11T08:55:12.085 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 284.49778643299999, 2024-09-11T08:55:12.085 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97324229600000001, 2024-09-11T08:55:12.085 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.085 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:55:12.085 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.086 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:12.086 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:12.086 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122672 2024-09-11T08:55:12.086 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.086 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.086 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.086 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.086 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558554+0000", 2024-09-11T08:55:12.086 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.086 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.086 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.086 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.086 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558554+0000", 2024-09-11T08:55:12.087 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.087 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.087 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.087 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.087 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558552+0000", 2024-09-11T08:55:12.087 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:55:12.087 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.087 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.087 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.087 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558657+0000", 2024-09-11T08:55:12.087 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000104595 2024-09-11T08:55:12.087 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.087 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.088 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.088 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558667+0000", 2024-09-11T08:55:12.088 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.7389999999999994e-06 2024-09-11T08:55:12.088 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.088 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.088 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.088 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558671+0000", 2024-09-11T08:55:12.088 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3359999999999997e-06 2024-09-11T08:55:12.088 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.088 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.088 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.088 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558714+0000", 2024-09-11T08:55:12.088 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3123e-05 2024-09-11T08:55:12.088 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.089 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.089 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:12.089 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558720+0000", 2024-09-11T08:55:12.089 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6489999999999996e-06 2024-09-11T08:55:12.089 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.089 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.089 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.089 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.531782+0000", 2024-09-11T08:55:12.089 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97306207 2024-09-11T08:55:12.089 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.089 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.089 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.089 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.531796+0000", 2024-09-11T08:55:12.089 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4103e-05 2024-09-11T08:55:12.090 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.090 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.090 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.090 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.090 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.090 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122673 9.16 9.41226b16 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:55:12.090 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.558855+0000", 2024-09-11T08:55:12.090 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 284.497485138, 2024-09-11T08:55:12.090 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97446970099999997, 2024-09-11T08:55:12.090 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.090 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:55:12.090 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.090 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:12.091 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:12.091 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122673 2024-09-11T08:55:12.091 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.091 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.091 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.091 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.091 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558855+0000", 2024-09-11T08:55:12.091 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.091 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.091 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.091 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.091 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558855+0000", 2024-09-11T08:55:12.092 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.092 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.092 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.092 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.092 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558854+0000", 2024-09-11T08:55:12.092 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:12.092 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.092 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.092 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.092 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558925+0000", 2024-09-11T08:55:12.092 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.0386e-05 2024-09-11T08:55:12.092 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.093 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.093 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.093 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558937+0000", 2024-09-11T08:55:12.093 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2751999999999999e-05 2024-09-11T08:55:12.093 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.093 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.093 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.093 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558942+0000", 2024-09-11T08:55:12.093 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3950000000000003e-06 2024-09-11T08:55:12.094 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.094 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.094 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.094 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558985+0000", 2024-09-11T08:55:12.094 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3099999999999997e-05 2024-09-11T08:55:12.094 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.094 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.094 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:12.094 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558989+0000", 2024-09-11T08:55:12.094 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2300000000000002e-06 2024-09-11T08:55:12.094 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.095 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.095 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.095 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533319+0000", 2024-09-11T08:55:12.095 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97432950600000001 2024-09-11T08:55:12.095 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.095 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.095 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.095 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533325+0000", 2024-09-11T08:55:12.095 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1299999999999998e-06 2024-09-11T08:55:12.095 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.096 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.096 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.096 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.096 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.096 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122675 9.16 9.a08f9936 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:55:12.096 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.559217+0000", 2024-09-11T08:55:12.096 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 284.49712295500001, 2024-09-11T08:55:12.096 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97412840599999995, 2024-09-11T08:55:12.096 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.096 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:55:12.096 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.096 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:12.096 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:12.097 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122675 2024-09-11T08:55:12.097 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.097 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.097 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.097 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.097 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.559217+0000", 2024-09-11T08:55:12.097 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.097 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.097 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.097 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.097 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.559217+0000", 2024-09-11T08:55:12.097 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.097 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.098 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.098 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.098 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.559216+0000", 2024-09-11T08:55:12.098 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:12.098 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.098 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.098 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.098 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.559278+0000", 2024-09-11T08:55:12.098 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2120999999999997e-05 2024-09-11T08:55:12.098 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.098 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.098 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.098 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.559291+0000", 2024-09-11T08:55:12.099 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2879e-05 2024-09-11T08:55:12.099 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.099 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.099 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.099 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.559296+0000", 2024-09-11T08:55:12.099 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5290000000000002e-06 2024-09-11T08:55:12.099 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.099 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.099 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.099 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.559331+0000", 2024-09-11T08:55:12.099 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5077000000000003e-05 2024-09-11T08:55:12.099 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.099 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.099 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:12.099 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.559336+0000", 2024-09-11T08:55:12.100 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.2909999999999998e-06 2024-09-11T08:55:12.100 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.100 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.100 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.100 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533340+0000", 2024-09-11T08:55:12.100 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97400369600000003 2024-09-11T08:55:12.100 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.100 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.100 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.100 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533346+0000", 2024-09-11T08:55:12.100 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7629999999999997e-06 2024-09-11T08:55:12.100 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.100 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.101 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.101 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.101 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.101 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122678 9.16 9.4bcb9396 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:55:12.101 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.562566+0000", 2024-09-11T08:55:12.101 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 284.49377391000002, 2024-09-11T08:55:12.101 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97080314300000004, 2024-09-11T08:55:12.101 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.101 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:55:12.101 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.101 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:12.101 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:12.101 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122678 2024-09-11T08:55:12.101 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.102 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.102 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.102 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.102 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562566+0000", 2024-09-11T08:55:12.102 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.102 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.102 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.102 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.102 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562566+0000", 2024-09-11T08:55:12.102 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.102 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.102 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.102 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.102 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562565+0000", 2024-09-11T08:55:12.103 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:55:12.103 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.103 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.103 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.103 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562648+0000", 2024-09-11T08:55:12.103 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2812999999999995e-05 2024-09-11T08:55:12.103 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.103 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.103 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.103 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562660+0000", 2024-09-11T08:55:12.103 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2044e-05 2024-09-11T08:55:12.103 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.103 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.103 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.104 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562665+0000", 2024-09-11T08:55:12.104 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5099999999999998e-06 2024-09-11T08:55:12.104 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.104 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.104 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.104 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562699+0000", 2024-09-11T08:55:12.104 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3569999999999999e-05 2024-09-11T08:55:12.104 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.104 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.104 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:12.104 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.562704+0000", 2024-09-11T08:55:12.104 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6149999999999999e-06 2024-09-11T08:55:12.104 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.104 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.105 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.105 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533363+0000", 2024-09-11T08:55:12.105 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97065928800000001 2024-09-11T08:55:12.105 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.105 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:12.106 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.106 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.533369+0000", 2024-09-11T08:55:12.106 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5250000000000002e-06 2024-09-11T08:55:12.107 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.107 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.107 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.107 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.107 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.107 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272398 9.10 9:0afe1f3f:::benchmark_data_smithi092_145083_object129000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:55:12.107 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:24.678144+0000", 2024-09-11T08:55:12.107 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 167.378195707, 2024-09-11T08:55:12.107 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.95568126099999995, 2024-09-11T08:55:12.107 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.107 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:12.107 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.107 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:12.108 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:12.108 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272398 2024-09-11T08:55:12.108 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.108 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.108 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.108 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.108 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678144+0000", 2024-09-11T08:55:12.108 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.108 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.108 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.108 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.108 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678144+0000", 2024-09-11T08:55:12.108 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.108 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.109 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.109 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.109 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678143+0000", 2024-09-11T08:55:12.109 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:55:12.109 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.109 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.109 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.109 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678149+0000", 2024-09-11T08:55:12.109 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2489999999999998e-06 2024-09-11T08:55:12.109 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.109 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.109 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.109 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678161+0000", 2024-09-11T08:55:12.110 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1782e-05 2024-09-11T08:55:12.110 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.110 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.110 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.110 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678167+0000", 2024-09-11T08:55:12.110 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8449999999999999e-06 2024-09-11T08:55:12.110 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.110 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.110 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.110 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678218+0000", 2024-09-11T08:55:12.110 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1109000000000001e-05 2024-09-11T08:55:12.110 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.110 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.110 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:12.110 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678225+0000", 2024-09-11T08:55:12.111 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6890000000000004e-06 2024-09-11T08:55:12.111 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.111 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.111 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.111 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.627423+0000", 2024-09-11T08:55:12.111 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.94919719899999999 2024-09-11T08:55:12.111 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.111 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.111 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:12.111 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.627524+0000", 2024-09-11T08:55:12.111 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000101258 2024-09-11T08:55:12.111 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.112 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.112 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,7", 2024-09-11T08:55:12.112 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.627606+0000", 2024-09-11T08:55:12.112 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2095000000000006e-05 2024-09-11T08:55:12.112 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.112 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.112 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.112 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.633662+0000", 2024-09-11T08:55:12.112 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0060555619999999996 2024-09-11T08:55:12.112 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.112 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.112 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.112 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.633731+0000", 2024-09-11T08:55:12.113 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9865999999999998e-05 2024-09-11T08:55:12.113 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.113 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.113 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:12.113 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.633739+0000", 2024-09-11T08:55:12.113 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.379e-06 2024-09-11T08:55:12.113 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.113 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.113 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:12.113 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.633783+0000", 2024-09-11T08:55:12.113 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3732000000000001e-05 2024-09-11T08:55:12.113 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.113 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.113 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.114 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.633826+0000", 2024-09-11T08:55:12.114 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3099999999999997e-05 2024-09-11T08:55:12.114 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.114 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.114 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.114 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.114 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.114 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272433 9.10 9:0a114e2d:::benchmark_data_smithi092_145083_object129035:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:55:12.114 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:24.742605+0000", 2024-09-11T08:55:12.114 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 167.313735221, 2024-09-11T08:55:12.114 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89161613399999995, 2024-09-11T08:55:12.114 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.114 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:12.114 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.115 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:12.115 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:12.115 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272433 2024-09-11T08:55:12.115 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.115 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.115 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.115 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.115 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742605+0000", 2024-09-11T08:55:12.115 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.115 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.115 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.115 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.115 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742605+0000", 2024-09-11T08:55:12.116 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.116 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.116 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.116 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.116 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742604+0000", 2024-09-11T08:55:12.116 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:55:12.116 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.116 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.116 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.116 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742610+0000", 2024-09-11T08:55:12.116 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2670000000000003e-06 2024-09-11T08:55:12.116 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.116 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.117 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.117 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742622+0000", 2024-09-11T08:55:12.117 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2029e-05 2024-09-11T08:55:12.117 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.117 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.117 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.117 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742627+0000", 2024-09-11T08:55:12.117 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8099999999999997e-06 2024-09-11T08:55:12.117 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.117 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.117 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.117 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742665+0000", 2024-09-11T08:55:12.117 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8421999999999999e-05 2024-09-11T08:55:12.117 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742671+0000", 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4890000000000003e-06 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.629333+0000", 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.88666230800000001 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.629439+0000", 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000106183 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.118 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.119 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,7", 2024-09-11T08:55:12.119 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.629559+0000", 2024-09-11T08:55:12.119 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000120099 2024-09-11T08:55:12.119 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.119 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.119 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:12.119 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.633833+0000", 2024-09-11T08:55:12.119 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0042741630000000001 2024-09-11T08:55:12.119 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.119 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.119 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.119 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.634007+0000", 2024-09-11T08:55:12.119 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00017368400000000001 2024-09-11T08:55:12.120 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.120 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.120 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.120 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.634126+0000", 2024-09-11T08:55:12.120 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000118921 2024-09-11T08:55:12.120 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.120 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.120 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:12.120 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.634185+0000", 2024-09-11T08:55:12.120 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8594999999999997e-05 2024-09-11T08:55:12.120 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.120 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.120 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.121 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.634221+0000", 2024-09-11T08:55:12.121 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6502999999999999e-05 2024-09-11T08:55:12.121 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.121 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.121 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.121 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.121 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.121 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:163328 10.1 10:8b5e1708:::benchmark_data_smithi092_173109_object26369:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-11T08:55:12.121 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:55.718214+0000", 2024-09-11T08:55:12.121 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 16.338126495000001, 2024-09-11T08:55:12.121 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10076722, 2024-09-11T08:55:12.121 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.122 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:12.122 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.122 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:55:12.122 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:55:12.122 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 163328 2024-09-11T08:55:12.122 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.122 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.122 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.122 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.122 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.718214+0000", 2024-09-11T08:55:12.122 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.122 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.122 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.122 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.123 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.718214+0000", 2024-09-11T08:55:12.123 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.123 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.123 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.123 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.123 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.718212+0000", 2024-09-11T08:55:12.123 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:55:12.123 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.123 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.123 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.123 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.718218+0000", 2024-09-11T08:55:12.123 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2859999999999996e-06 2024-09-11T08:55:12.123 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.124 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.124 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.124 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.718235+0000", 2024-09-11T08:55:12.124 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6103000000000002e-05 2024-09-11T08:55:12.124 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.124 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.124 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.124 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.718240+0000", 2024-09-11T08:55:12.124 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7570000000000001e-06 2024-09-11T08:55:12.124 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.124 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.124 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.124 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.718278+0000", 2024-09-11T08:55:12.124 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.756e-05 2024-09-11T08:55:12.125 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.125 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.125 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:12.125 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.718385+0000", 2024-09-11T08:55:12.125 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010717699999999999 2024-09-11T08:55:12.125 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.125 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.125 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,7", 2024-09-11T08:55:12.125 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.718487+0000", 2024-09-11T08:55:12.125 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000102378 2024-09-11T08:55:12.125 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.125 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.125 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:12.125 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.721349+0000", 2024-09-11T08:55:12.126 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0028619370000000002 2024-09-11T08:55:12.126 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.126 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.126 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.126 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.727597+0000", 2024-09-11T08:55:12.126 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0062478189999999999 2024-09-11T08:55:12.126 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.126 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.126 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.126 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.818888+0000", 2024-09-11T08:55:12.126 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.091291307000000002 2024-09-11T08:55:12.126 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.126 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.126 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.818936+0000", 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7939999999999998e-05 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.818981+0000", 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4422000000000003e-05 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:163332 10.4 10:261544b4:::benchmark_data_smithi092_173109_object26373:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:55.722574+0000", 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 16.333766141000002, 2024-09-11T08:55:12.127 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.097807794000000003, 2024-09-11T08:55:12.128 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.128 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:12.128 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.128 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:55:12.128 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:55:12.128 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 163332 2024-09-11T08:55:12.128 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.128 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.128 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.128 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.128 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.722574+0000", 2024-09-11T08:55:12.128 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.128 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.129 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.129 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.129 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.722574+0000", 2024-09-11T08:55:12.129 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.129 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.129 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.129 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.129 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.722573+0000", 2024-09-11T08:55:12.129 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:55:12.129 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.129 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.129 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.129 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.722576+0000", 2024-09-11T08:55:12.129 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9950000000000001e-06 2024-09-11T08:55:12.129 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.130 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.130 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.130 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.722584+0000", 2024-09-11T08:55:12.130 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6990000000000003e-06 2024-09-11T08:55:12.130 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.130 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.130 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.130 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.722588+0000", 2024-09-11T08:55:12.130 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2719999999999999e-06 2024-09-11T08:55:12.130 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.130 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.130 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.130 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.722708+0000", 2024-09-11T08:55:12.131 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000119726 2024-09-11T08:55:12.131 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.131 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.131 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:12.131 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.722811+0000", 2024-09-11T08:55:12.131 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000102488 2024-09-11T08:55:12.131 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.131 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.131 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,3", 2024-09-11T08:55:12.131 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.722909+0000", 2024-09-11T08:55:12.131 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.8270000000000006e-05 2024-09-11T08:55:12.131 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.131 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.131 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.131 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.727318+0000", 2024-09-11T08:55:12.132 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0044087930000000003 2024-09-11T08:55:12.132 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.132 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.132 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:12.132 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.727324+0000", 2024-09-11T08:55:12.132 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8220000000000003e-06 2024-09-11T08:55:12.132 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.132 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.132 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.132 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.820307+0000", 2024-09-11T08:55:12.132 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.092983107999999995 2024-09-11T08:55:12.132 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.132 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.133 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:12.133 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.820348+0000", 2024-09-11T08:55:12.133 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1357999999999999e-05 2024-09-11T08:55:12.133 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.133 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.133 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.133 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:55.820382+0000", 2024-09-11T08:55:12.133 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3748999999999998e-05 2024-09-11T08:55:12.133 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.133 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.133 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.133 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.133 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.134 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:163865 10.c 10:3b995a9a:::benchmark_data_smithi092_173109_object26906:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-11T08:55:12.134 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:56.209520+0000", 2024-09-11T08:55:12.134 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 15.846819964, 2024-09-11T08:55:12.134 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.123454429, 2024-09-11T08:55:12.134 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.134 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:12.134 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.134 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:55:12.134 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:55:12.134 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 163865 2024-09-11T08:55:12.134 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.134 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.134 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.135 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.135 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.209520+0000", 2024-09-11T08:55:12.135 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.135 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.135 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.135 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.135 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.209520+0000", 2024-09-11T08:55:12.135 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.135 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.135 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.135 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.135 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.209519+0000", 2024-09-11T08:55:12.135 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:55:12.135 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.136 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.136 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.136 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.209526+0000", 2024-09-11T08:55:12.136 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1899999999999998e-06 2024-09-11T08:55:12.136 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.136 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.136 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.136 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.209541+0000", 2024-09-11T08:55:12.136 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5472e-05 2024-09-11T08:55:12.136 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.136 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.136 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.136 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.209547+0000", 2024-09-11T08:55:12.137 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8270000000000003e-06 2024-09-11T08:55:12.137 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.137 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.137 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.137 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.209782+0000", 2024-09-11T08:55:12.137 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00023512000000000001 2024-09-11T08:55:12.137 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.137 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.137 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:12.137 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.209880+0000", 2024-09-11T08:55:12.137 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.7272e-05 2024-09-11T08:55:12.137 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.137 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.137 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,7", 2024-09-11T08:55:12.138 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.209992+0000", 2024-09-11T08:55:12.138 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000112625 2024-09-11T08:55:12.138 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.138 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.138 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:12.138 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.216388+0000", 2024-09-11T08:55:12.138 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0063956300000000002 2024-09-11T08:55:12.138 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.138 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.138 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.138 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.217632+0000", 2024-09-11T08:55:12.138 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001244111 2024-09-11T08:55:12.138 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.139 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.139 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.139 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.332878+0000", 2024-09-11T08:55:12.139 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.115245852 2024-09-11T08:55:12.139 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.139 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.139 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:12.139 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.332924+0000", 2024-09-11T08:55:12.139 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5658999999999998e-05 2024-09-11T08:55:12.139 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.139 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.139 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.139 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.332975+0000", 2024-09-11T08:55:12.139 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1066999999999997e-05 2024-09-11T08:55:12.140 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.140 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.140 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.140 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.140 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.140 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:163867 10.c 10:359c7a69:::benchmark_data_smithi092_173109_object26908:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-11T08:55:12.140 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:56.210530+0000", 2024-09-11T08:55:12.140 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 15.845810051000001, 2024-09-11T08:55:12.140 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.12253868800000001, 2024-09-11T08:55:12.140 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.140 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:12.140 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.140 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:55:12.140 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:55:12.141 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 163867 2024-09-11T08:55:12.141 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.141 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.141 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.141 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.141 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.210530+0000", 2024-09-11T08:55:12.141 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.141 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.141 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.141 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.141 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.210530+0000", 2024-09-11T08:55:12.141 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.141 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.142 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.142 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.142 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.210529+0000", 2024-09-11T08:55:12.142 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:12.142 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.142 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.142 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.142 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.210534+0000", 2024-09-11T08:55:12.142 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1569999999999999e-06 2024-09-11T08:55:12.142 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.142 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.142 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.142 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.210549+0000", 2024-09-11T08:55:12.142 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4799e-05 2024-09-11T08:55:12.143 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.143 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.143 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.143 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.210555+0000", 2024-09-11T08:55:12.143 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0979999999999999e-06 2024-09-11T08:55:12.143 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.143 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.143 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.143 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.213647+0000", 2024-09-11T08:55:12.143 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0030919989999999998 2024-09-11T08:55:12.143 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.143 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.143 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:12.144 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.213767+0000", 2024-09-11T08:55:12.144 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000120143 2024-09-11T08:55:12.144 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.144 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.144 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,7", 2024-09-11T08:55:12.144 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.213868+0000", 2024-09-11T08:55:12.144 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000100333 2024-09-11T08:55:12.144 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.144 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.144 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:12.144 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.216394+0000", 2024-09-11T08:55:12.144 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0025260970000000002 2024-09-11T08:55:12.144 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.145 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.145 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.145 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.217656+0000", 2024-09-11T08:55:12.145 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001262319 2024-09-11T08:55:12.145 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.145 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.145 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.145 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.333012+0000", 2024-09-11T08:55:12.145 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.115356312 2024-09-11T08:55:12.145 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.145 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.145 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:12.145 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.333043+0000", 2024-09-11T08:55:12.146 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0951000000000002e-05 2024-09-11T08:55:12.146 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.146 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.146 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.146 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.333069+0000", 2024-09-11T08:55:12.146 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5369999999999999e-05 2024-09-11T08:55:12.146 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.146 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.146 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.146 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.146 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.146 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:163870 10.1 10:8d9d30a2:::benchmark_data_smithi092_173109_object26911:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-11T08:55:12.146 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:56.214646+0000", 2024-09-11T08:55:12.146 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 15.841694249, 2024-09-11T08:55:12.147 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.121111441, 2024-09-11T08:55:12.147 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.147 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:12.147 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.147 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:55:12.147 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:55:12.147 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 163870 2024-09-11T08:55:12.147 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.147 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.147 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.147 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.147 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.214646+0000", 2024-09-11T08:55:12.147 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.148 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.148 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.148 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.148 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.214646+0000", 2024-09-11T08:55:12.148 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.148 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.148 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.148 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.148 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.214645+0000", 2024-09-11T08:55:12.148 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:12.148 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.149 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:12.149 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:12.150 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.150 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.150 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.214648+0000", 2024-09-11T08:55:12.150 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6239999999999999e-06 2024-09-11T08:55:12.150 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.150 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.150 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.150 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.214656+0000", 2024-09-11T08:55:12.150 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.3329999999999999e-06 2024-09-11T08:55:12.150 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.150 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.150 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.151 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.214659+0000", 2024-09-11T08:55:12.151 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6650000000000001e-06 2024-09-11T08:55:12.151 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.151 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.151 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.151 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.214955+0000", 2024-09-11T08:55:12.151 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00029538600000000002 2024-09-11T08:55:12.151 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.151 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.151 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:12.151 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.215059+0000", 2024-09-11T08:55:12.151 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010388299999999999 2024-09-11T08:55:12.151 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.152 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.152 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,7", 2024-09-11T08:55:12.152 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.215155+0000", 2024-09-11T08:55:12.152 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.5971000000000007e-05 2024-09-11T08:55:12.152 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.152 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.152 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:12.152 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220342+0000", 2024-09-11T08:55:12.152 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0051875610000000003 2024-09-11T08:55:12.152 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.152 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.152 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.152 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.225514+0000", 2024-09-11T08:55:12.153 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0051718859999999997 2024-09-11T08:55:12.153 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.153 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.153 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.153 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.335679+0000", 2024-09-11T08:55:12.153 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.110165187 2024-09-11T08:55:12.153 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.153 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.153 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:12.153 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.335722+0000", 2024-09-11T08:55:12.153 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2648999999999998e-05 2024-09-11T08:55:12.153 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.153 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.153 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.153 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.335757+0000", 2024-09-11T08:55:12.154 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5417000000000003e-05 2024-09-11T08:55:12.154 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.154 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.154 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.154 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.154 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.154 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:163876 10.c 10:397f99ed:::benchmark_data_smithi092_173109_object26917:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-11T08:55:12.154 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:56.220251+0000", 2024-09-11T08:55:12.154 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 15.836089586, 2024-09-11T08:55:12.154 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.11293073200000001, 2024-09-11T08:55:12.154 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.154 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:12.154 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.154 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:55:12.155 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:55:12.155 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 163876 2024-09-11T08:55:12.155 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.155 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.155 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.155 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.155 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220251+0000", 2024-09-11T08:55:12.155 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.155 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.155 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.155 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.155 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220251+0000", 2024-09-11T08:55:12.155 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.156 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.156 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.156 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.156 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220248+0000", 2024-09-11T08:55:12.156 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:55:12.156 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.156 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.156 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.156 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220256+0000", 2024-09-11T08:55:12.156 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.5170000000000002e-06 2024-09-11T08:55:12.156 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.156 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.156 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.157 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220273+0000", 2024-09-11T08:55:12.157 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7342000000000001e-05 2024-09-11T08:55:12.157 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.157 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.157 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.157 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220279+0000", 2024-09-11T08:55:12.157 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6629999999999998e-06 2024-09-11T08:55:12.157 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.157 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.157 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.157 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.221073+0000", 2024-09-11T08:55:12.157 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00079404300000000003 2024-09-11T08:55:12.157 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.157 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.158 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:12.158 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.221179+0000", 2024-09-11T08:55:12.158 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000106277 2024-09-11T08:55:12.158 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.158 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.158 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,7", 2024-09-11T08:55:12.158 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.221304+0000", 2024-09-11T08:55:12.158 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000124308 2024-09-11T08:55:12.158 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.158 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.158 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:12.158 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.224371+0000", 2024-09-11T08:55:12.158 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0030673129999999999 2024-09-11T08:55:12.158 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.159 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.159 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.159 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.224721+0000", 2024-09-11T08:55:12.159 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00035018099999999997 2024-09-11T08:55:12.159 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.159 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.159 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.159 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.333108+0000", 2024-09-11T08:55:12.159 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10838692900000001 2024-09-11T08:55:12.159 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.159 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.159 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:12.159 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.333152+0000", 2024-09-11T08:55:12.159 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3602e-05 2024-09-11T08:55:12.160 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.160 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.160 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.160 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.333181+0000", 2024-09-11T08:55:12.160 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9575999999999999e-05 2024-09-11T08:55:12.160 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.160 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.160 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.160 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.160 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.160 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:163878 10.1 10:8300a444:::benchmark_data_smithi092_173109_object26919:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-11T08:55:12.160 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:56.220326+0000", 2024-09-11T08:55:12.160 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 15.836014295, 2024-09-11T08:55:12.160 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.11557036, 2024-09-11T08:55:12.161 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.161 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:12.161 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.161 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:55:12.161 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:55:12.161 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 163878 2024-09-11T08:55:12.161 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.161 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.161 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.161 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.161 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220326+0000", 2024-09-11T08:55:12.161 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.161 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.161 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.162 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.162 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220326+0000", 2024-09-11T08:55:12.162 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.162 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.162 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.162 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.162 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220325+0000", 2024-09-11T08:55:12.162 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:55:12.162 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.162 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.162 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.162 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220329+0000", 2024-09-11T08:55:12.162 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0559999999999998e-06 2024-09-11T08:55:12.162 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.162 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.163 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.163 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220343+0000", 2024-09-11T08:55:12.163 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3429e-05 2024-09-11T08:55:12.163 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.163 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.163 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.163 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220347+0000", 2024-09-11T08:55:12.163 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3200000000000001e-06 2024-09-11T08:55:12.163 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.163 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.164 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.164 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220386+0000", 2024-09-11T08:55:12.164 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8668999999999999e-05 2024-09-11T08:55:12.164 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.164 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.164 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:12.164 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220504+0000", 2024-09-11T08:55:12.164 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011789 2024-09-11T08:55:12.164 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.164 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.165 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,7", 2024-09-11T08:55:12.165 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.220621+0000", 2024-09-11T08:55:12.165 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011785099999999999 2024-09-11T08:55:12.165 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.165 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.165 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.165 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.225541+0000", 2024-09-11T08:55:12.165 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0049196259999999999 2024-09-11T08:55:12.165 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.165 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.165 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:12.165 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.227003+0000", 2024-09-11T08:55:12.165 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001461614 2024-09-11T08:55:12.166 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.166 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.166 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.166 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.335833+0000", 2024-09-11T08:55:12.166 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10883011200000001 2024-09-11T08:55:12.166 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.166 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.166 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:12.166 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.335864+0000", 2024-09-11T08:55:12.166 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1454000000000001e-05 2024-09-11T08:55:12.166 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.166 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.166 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.167 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.335896+0000", 2024-09-11T08:55:12.167 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1928999999999999e-05 2024-09-11T08:55:12.167 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.167 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.167 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.167 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.167 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.167 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:163887 10.1 10:8b7fc521:::benchmark_data_smithi092_173109_object26928:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-11T08:55:12.167 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:56.237556+0000", 2024-09-11T08:55:12.167 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 15.818783637999999, 2024-09-11T08:55:12.167 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.10141563300000001, 2024-09-11T08:55:12.167 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.167 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:12.167 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.168 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:55:12.168 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:55:12.168 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 163887 2024-09-11T08:55:12.168 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.168 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.168 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.168 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.168 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.237556+0000", 2024-09-11T08:55:12.168 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.168 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.168 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.168 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.168 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.237556+0000", 2024-09-11T08:55:12.169 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.169 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.169 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.169 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.169 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.237555+0000", 2024-09-11T08:55:12.169 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:55:12.169 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.169 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.169 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.169 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.237563+0000", 2024-09-11T08:55:12.169 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7500000000000003e-06 2024-09-11T08:55:12.169 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.169 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.169 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.170 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.237580+0000", 2024-09-11T08:55:12.170 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6759e-05 2024-09-11T08:55:12.170 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.170 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.170 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.170 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.237585+0000", 2024-09-11T08:55:12.170 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6110000000000001e-06 2024-09-11T08:55:12.170 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.170 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.170 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.170 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.237636+0000", 2024-09-11T08:55:12.170 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1044999999999997e-05 2024-09-11T08:55:12.170 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.171 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.171 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:12.171 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.237754+0000", 2024-09-11T08:55:12.171 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011782600000000001 2024-09-11T08:55:12.171 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.171 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.171 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,7", 2024-09-11T08:55:12.171 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.237876+0000", 2024-09-11T08:55:12.171 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000121526 2024-09-11T08:55:12.171 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.171 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.171 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:12.171 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.239904+0000", 2024-09-11T08:55:12.172 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.002028354 2024-09-11T08:55:12.172 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.172 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.172 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.172 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.240194+0000", 2024-09-11T08:55:12.172 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00029023200000000002 2024-09-11T08:55:12.172 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.172 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.172 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.172 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.338855+0000", 2024-09-11T08:55:12.172 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.098660591000000006 2024-09-11T08:55:12.172 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.172 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.172 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:12.173 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.338933+0000", 2024-09-11T08:55:12.173 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.8281000000000005e-05 2024-09-11T08:55:12.173 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.173 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.173 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.173 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:56.338972+0000", 2024-09-11T08:55:12.173 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9041000000000001e-05 2024-09-11T08:55:12.173 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.173 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.173 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.173 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.173 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.173 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:180218 10.e 10:730e92fe:::benchmark_data_smithi092_173109_object43259:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-11T08:55:12.174 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:55:10.343242+0000", 2024-09-11T08:55:12.174 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 1.7130980390000001, 2024-09-11T08:55:12.174 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.232542421, 2024-09-11T08:55:12.174 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:12.174 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:12.174 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:12.174 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:55:12.174 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:55:12.174 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 180218 2024-09-11T08:55:12.174 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.174 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:12.174 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.174 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:12.174 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343242+0000", 2024-09-11T08:55:12.175 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.175 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.175 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.175 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:12.175 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343242+0000", 2024-09-11T08:55:12.175 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:12.175 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.175 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.175 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:12.175 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343241+0000", 2024-09-11T08:55:12.175 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:55:12.175 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.175 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.176 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:12.176 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343245+0000", 2024-09-11T08:55:12.176 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6430000000000001e-06 2024-09-11T08:55:12.176 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.176 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.176 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:12.176 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343255+0000", 2024-09-11T08:55:12.176 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.8700000000000004e-06 2024-09-11T08:55:12.176 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.176 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.176 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:12.176 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343260+0000", 2024-09-11T08:55:12.176 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4909999999999996e-06 2024-09-11T08:55:12.176 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.177 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.177 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:12.177 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343301+0000", 2024-09-11T08:55:12.177 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0436999999999999e-05 2024-09-11T08:55:12.177 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.177 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.177 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:12.177 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343411+0000", 2024-09-11T08:55:12.177 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011047799999999999 2024-09-11T08:55:12.177 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.177 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.177 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,6", 2024-09-11T08:55:12.177 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343523+0000", 2024-09-11T08:55:12.178 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000111548 2024-09-11T08:55:12.178 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.178 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.178 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.178 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.350779+0000", 2024-09-11T08:55:12.178 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0072566460000000003 2024-09-11T08:55:12.178 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.178 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.178 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:12.178 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.350932+0000", 2024-09-11T08:55:12.178 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000152578 2024-09-11T08:55:12.178 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.178 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.178 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:12.179 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.575671+0000", 2024-09-11T08:55:12.179 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.22473905299999999 2024-09-11T08:55:12.179 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.179 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.179 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:12.179 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.575737+0000", 2024-09-11T08:55:12.179 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6239000000000001e-05 2024-09-11T08:55:12.179 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:12.179 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:12.179 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:12.179 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.575784+0000", 2024-09-11T08:55:12.179 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7122e-05 2024-09-11T08:55:12.179 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.179 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.179 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.180 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:12.180 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:12.180 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:55:12.249 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:12.282 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:12.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:12.283 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:12.383 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:12.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:12.415 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:12.415 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:12.516 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:12.543 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:12.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:12.544 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:12.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:12 smithi092 ceph-mon[29647]: pgmap v742: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 5.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-11T08:55:12.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:12 smithi092 ceph-mon[41114]: pgmap v742: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 5.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-11T08:55:12.569 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:12 smithi145 ceph-mon[33384]: pgmap v742: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 5.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-11T08:55:12.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:12.664 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:55:12.665 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:55:12.665 INFO:teuthology.orchestra.run.smithi092.stdout: "epoch": 330, 2024-09-11T08:55:12.665 INFO:teuthology.orchestra.run.smithi092.stdout: "fsid": "211625d4-7018-11ef-bcea-c7b262605968", 2024-09-11T08:55:12.665 INFO:teuthology.orchestra.run.smithi092.stdout: "created": "2024-09-11T08:33:32.174444+0000", 2024-09-11T08:55:12.666 INFO:teuthology.orchestra.run.smithi092.stdout: "modified": "2024-09-11T08:55:03.201404+0000", 2024-09-11T08:55:12.666 INFO:teuthology.orchestra.run.smithi092.stdout: "last_up_change": "2024-09-11T08:38:23.097480+0000", 2024-09-11T08:55:12.666 INFO:teuthology.orchestra.run.smithi092.stdout: "last_in_change": "2024-09-11T08:52:17.544985+0000", 2024-09-11T08:55:12.666 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T08:55:12.666 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_num": 5799936, 2024-09-11T08:55:12.666 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_set": [ 2024-09-11T08:55:12.666 INFO:teuthology.orchestra.run.smithi092.stdout: "pglog_hardlimit", 2024-09-11T08:55:12.666 INFO:teuthology.orchestra.run.smithi092.stdout: "purged_snapdirs", 2024-09-11T08:55:12.666 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_deletes", 2024-09-11T08:55:12.666 INFO:teuthology.orchestra.run.smithi092.stdout: "sortbitwise" 2024-09-11T08:55:12.666 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:55:12.666 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_version": 18, 2024-09-11T08:55:12.667 INFO:teuthology.orchestra.run.smithi092.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T08:55:12.667 INFO:teuthology.orchestra.run.smithi092.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T08:55:12.667 INFO:teuthology.orchestra.run.smithi092.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T08:55:12.667 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_snapshot": "", 2024-09-11T08:55:12.667 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_max": 10, 2024-09-11T08:55:12.667 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd": 8, 2024-09-11T08:55:12.667 INFO:teuthology.orchestra.run.smithi092.stdout: "require_min_compat_client": "luminous", 2024-09-11T08:55:12.667 INFO:teuthology.orchestra.run.smithi092.stdout: "min_compat_client": "luminous", 2024-09-11T08:55:12.667 INFO:teuthology.orchestra.run.smithi092.stdout: "require_osd_release": "quincy", 2024-09-11T08:55:12.667 INFO:teuthology.orchestra.run.smithi092.stdout: "pools": [ 2024-09-11T08:55:12.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.667 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 1, 2024-09-11T08:55:12.667 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": ".mgr", 2024-09-11T08:55:12.667 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:36:29.606460+0000", 2024-09-11T08:55:12.668 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:55:12.668 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:55:12.668 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:55:12.668 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:55:12.668 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:55:12.668 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:55:12.668 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:55:12.668 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:55:12.668 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:55:12.668 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:55:12.668 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:55:12.668 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:55:12.668 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 11, 2024-09-11T08:55:12.668 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 11, 2024-09-11T08:55:12.669 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 11, 2024-09-11T08:55:12.669 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 11, 2024-09-11T08:55:12.669 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 11, 2024-09-11T08:55:12.669 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:55:12.669 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:55:12.669 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:55:12.669 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:55:12.669 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:55:12.669 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:55:12.669 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:55:12.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.669 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "250", 2024-09-11T08:55:12.669 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:55:12.670 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:55:12.670 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "248", 2024-09-11T08:55:12.670 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:55:12.670 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:55:12.670 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:55:12.670 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:55:12.670 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:55:12.670 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:55:12.670 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:55:12.670 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:55:12.670 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:55:12.670 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:55:12.670 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:55:12.670 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:55:12.671 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:55:12.671 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:55:12.671 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:55:12.671 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:55:12.671 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:55:12.671 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:55:12.671 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:55:12.671 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:55:12.671 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:55:12.671 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:55:12.671 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:55:12.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.672 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:55:12.672 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:55:12.672 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:55:12.672 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:55:12.672 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:55:12.672 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:55:12.672 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:55:12.672 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:55:12.672 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:55:12.672 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:55:12.672 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:55:12.672 INFO:teuthology.orchestra.run.smithi092.stdout: "options": { 2024-09-11T08:55:12.672 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_max": 32, 2024-09-11T08:55:12.673 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_min": 1 2024-09-11T08:55:12.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.673 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:55:12.673 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": {} 2024-09-11T08:55:12.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.673 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 2, 2024-09-11T08:55:12.673 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "datapool", 2024-09-11T08:55:12.673 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:37.674272+0000", 2024-09-11T08:55:12.673 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 8193, 2024-09-11T08:55:12.673 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T08:55:12.673 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:55:12.674 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:55:12.674 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:55:12.674 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:55:12.674 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:55:12.674 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:55:12.674 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:55:12.674 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:55:12.674 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:55:12.674 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:55:12.674 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 3, 2024-09-11T08:55:12.674 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 3, 2024-09-11T08:55:12.674 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 3, 2024-09-11T08:55:12.674 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 3, 2024-09-11T08:55:12.674 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 3, 2024-09-11T08:55:12.675 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:55:12.675 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:55:12.675 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:55:12.675 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:55:12.675 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:55:12.675 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:55:12.675 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:55:12.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.675 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "60", 2024-09-11T08:55:12.675 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:55:12.675 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:55:12.675 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:55:12.676 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:55:12.676 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:55:12.676 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 3, 2024-09-11T08:55:12.676 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 60, 2024-09-11T08:55:12.676 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:55:12.676 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:55:12.676 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:55:12.676 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:55:12.676 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:55:12.676 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:55:12.677 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:55:12.677 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:55:12.677 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:55:12.677 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:55:12.677 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:55:12.677 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:55:12.677 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:55:12.677 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:55:12.677 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:55:12.677 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:55:12.677 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:55:12.677 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:55:12.677 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:55:12.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.678 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:55:12.678 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:55:12.678 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:55:12.678 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:55:12.678 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:55:12.678 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:55:12.678 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:55:12.678 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:55:12.678 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:55:12.678 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:55:12.678 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:55:12.678 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:55:12.679 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:55:12.679 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd": {} 2024-09-11T08:55:12.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.679 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 3, 2024-09-11T08:55:12.679 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": ".rgw.root", 2024-09-11T08:55:12.679 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:37.912906+0000", 2024-09-11T08:55:12.679 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:55:12.679 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:55:12.679 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:55:12.679 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:55:12.679 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:55:12.679 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:55:12.679 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:55:12.680 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:55:12.680 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:55:12.680 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:55:12.680 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:55:12.680 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:55:12.680 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:55:12.680 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:55:12.680 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:55:12.680 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:55:12.680 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:55:12.680 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:55:12.680 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:55:12.680 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:55:12.681 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:55:12.681 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:55:12.681 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:55:12.681 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:55:12.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.681 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "57", 2024-09-11T08:55:12.681 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:55:12.681 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:55:12.681 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:55:12.681 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:55:12.681 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:55:12.681 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:55:12.681 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:55:12.681 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:55:12.682 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:55:12.682 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:55:12.682 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:55:12.682 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:55:12.682 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:55:12.682 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:55:12.682 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:55:12.682 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:55:12.682 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:55:12.682 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:55:12.682 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:55:12.682 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:55:12.682 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:55:12.682 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:55:12.683 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:55:12.683 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:55:12.683 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:55:12.683 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:55:12.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.683 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:55:12.684 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:55:12.684 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:55:12.684 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:55:12.684 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:55:12.684 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:55:12.684 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:55:12.684 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:55:12.684 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:55:12.684 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:55:12.684 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:55:12.684 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:55:12.684 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:55:12.685 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:55:12.685 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.685 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 4, 2024-09-11T08:55:12.685 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "default.rgw.log", 2024-09-11T08:55:12.685 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:39.234173+0000", 2024-09-11T08:55:12.685 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:55:12.685 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:55:12.685 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:55:12.685 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:55:12.685 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:55:12.685 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:55:12.685 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:55:12.686 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:55:12.686 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:55:12.686 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:55:12.686 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:55:12.686 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:55:12.686 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:55:12.686 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:55:12.686 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:55:12.686 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:55:12.686 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:55:12.686 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:55:12.686 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:55:12.686 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:55:12.686 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:55:12.687 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:55:12.687 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:55:12.687 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:55:12.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.687 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "59", 2024-09-11T08:55:12.687 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:55:12.687 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:55:12.687 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:55:12.687 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:55:12.687 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:55:12.687 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:55:12.687 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:55:12.687 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:55:12.688 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:55:12.688 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:55:12.688 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:55:12.688 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:55:12.688 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:55:12.688 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:55:12.688 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:55:12.688 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:55:12.688 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:55:12.688 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:55:12.688 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:55:12.688 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:55:12.689 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:55:12.689 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:55:12.689 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:55:12.689 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:55:12.689 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:55:12.689 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:55:12.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.689 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:55:12.689 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:55:12.689 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:55:12.689 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:55:12.689 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:55:12.689 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:55:12.689 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:55:12.690 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:55:12.690 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:55:12.690 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:55:12.690 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:55:12.690 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:55:12.690 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:55:12.690 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:55:12.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.690 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 5, 2024-09-11T08:55:12.690 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "default.rgw.control", 2024-09-11T08:55:12.690 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:41.216386+0000", 2024-09-11T08:55:12.691 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:55:12.691 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:55:12.691 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:55:12.691 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:55:12.691 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:55:12.691 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:55:12.691 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:55:12.691 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:55:12.691 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:55:12.691 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:55:12.691 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:55:12.691 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:55:12.691 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:55:12.692 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:55:12.692 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:55:12.692 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:55:12.692 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:55:12.692 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:55:12.692 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:55:12.692 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:55:12.692 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:55:12.692 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:55:12.692 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:55:12.692 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:55:12.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.692 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "61", 2024-09-11T08:55:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:55:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:55:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:55:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:55:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:55:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:55:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:55:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:55:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:55:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:55:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:55:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:55:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:55:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:55:12.694 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:55:12.694 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:55:12.694 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:55:12.694 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:55:12.694 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:55:12.694 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:55:12.694 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:55:12.694 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:55:12.694 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:55:12.694 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:55:12.694 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:55:12.694 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:55:12.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.695 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:55:12.695 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:55:12.695 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:55:12.695 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:55:12.695 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:55:12.695 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:55:12.695 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:55:12.695 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:55:12.695 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:55:12.695 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:55:12.695 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:55:12.695 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:55:12.695 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:55:12.695 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:55:12.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.696 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 6, 2024-09-11T08:55:12.696 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "default.rgw.meta", 2024-09-11T08:55:12.696 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:43.285497+0000", 2024-09-11T08:55:12.696 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:55:12.696 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:55:12.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:55:12.696 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:55:12.696 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:55:12.696 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:55:12.696 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:55:12.696 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:55:12.697 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:55:12.697 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:55:12.697 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:55:12.697 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:55:12.697 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T08:55:12.697 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T08:55:12.697 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T08:55:12.697 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T08:55:12.697 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T08:55:12.697 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:55:12.697 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:55:12.697 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:55:12.697 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:55:12.698 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:55:12.698 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:55:12.698 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:55:12.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.698 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "63", 2024-09-11T08:55:12.698 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:55:12.698 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:55:12.698 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:55:12.698 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:55:12.698 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:55:12.698 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:55:12.698 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:55:12.698 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:55:12.698 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:55:12.699 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:55:12.699 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:55:12.699 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:55:12.699 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:55:12.699 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:55:12.699 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:55:12.699 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:55:12.699 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:55:12.699 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:55:12.699 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:55:12.700 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:55:12.700 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:55:12.700 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:55:12.700 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:55:12.700 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:55:12.700 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:55:12.700 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:55:12.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.700 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:55:12.700 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:55:12.700 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:55:12.700 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:55:12.700 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:55:12.701 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:55:12.701 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:55:12.701 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:55:12.701 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:55:12.701 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:55:12.701 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:55:12.701 INFO:teuthology.orchestra.run.smithi092.stdout: "options": { 2024-09-11T08:55:12.701 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_bias": 4 2024-09-11T08:55:12.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.701 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:55:12.701 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T08:55:12.701 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.702 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 10, 2024-09-11T08:55:12.702 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "unique_pool_3", 2024-09-11T08:55:12.702 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:52:53.990481+0000", 2024-09-11T08:55:12.702 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T08:55:12.702 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T08:55:12.702 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T08:55:12.702 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T08:55:12.702 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T08:55:12.702 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T08:55:12.702 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T08:55:12.702 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T08:55:12.702 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T08:55:12.702 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T08:55:12.703 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T08:55:12.703 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T08:55:12.703 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 16, 2024-09-11T08:55:12.703 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 16, 2024-09-11T08:55:12.703 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 16, 2024-09-11T08:55:12.703 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 16, 2024-09-11T08:55:12.703 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 16, 2024-09-11T08:55:12.703 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T08:55:12.703 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T08:55:12.703 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T08:55:12.703 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T08:55:12.703 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T08:55:12.703 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T08:55:12.703 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T08:55:12.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.704 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "297", 2024-09-11T08:55:12.704 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T08:55:12.704 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T08:55:12.704 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T08:55:12.704 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T08:55:12.704 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T08:55:12.704 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T08:55:12.704 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T08:55:12.704 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T08:55:12.704 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T08:55:12.704 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T08:55:12.704 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T08:55:12.704 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T08:55:12.705 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T08:55:12.705 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T08:55:12.705 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T08:55:12.705 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T08:55:12.705 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T08:55:12.705 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T08:55:12.705 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T08:55:12.705 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T08:55:12.705 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T08:55:12.705 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T08:55:12.705 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T08:55:12.705 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T08:55:12.705 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T08:55:12.706 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T08:55:12.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.706 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T08:55:12.706 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T08:55:12.706 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T08:55:12.706 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T08:55:12.706 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T08:55:12.706 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T08:55:12.706 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T08:55:12.706 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T08:55:12.706 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T08:55:12.706 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T08:55:12.706 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T08:55:12.706 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T08:55:12.707 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T08:55:12.707 INFO:teuthology.orchestra.run.smithi092.stdout: "rados": {} 2024-09-11T08:55:12.707 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.707 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.707 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:55:12.707 INFO:teuthology.orchestra.run.smithi092.stdout: "osds": [ 2024-09-11T08:55:12.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.707 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-09-11T08:55:12.707 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "9d8a3811-e801-4b6a-9a63-83936283cb7d", 2024-09-11T08:55:12.707 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:55:12.707 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:55:12.707 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:55:12.707 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 0.1711883544921875, 2024-09-11T08:55:12.707 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:55:12.708 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:55:12.708 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 8, 2024-09-11T08:55:12.708 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 319, 2024-09-11T08:55:12.708 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:55:12.708 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:55:12.708 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:55:12.708 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.708 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.708 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6802", 2024-09-11T08:55:12.708 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:55:12.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.709 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.709 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6803", 2024-09-11T08:55:12.709 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:55:12.709 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.709 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.709 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:55:12.709 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.709 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.709 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6804", 2024-09-11T08:55:12.709 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:55:12.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.710 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.710 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6805", 2024-09-11T08:55:12.710 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:55:12.710 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.710 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.710 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:55:12.710 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.710 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.710 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6808", 2024-09-11T08:55:12.710 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:55:12.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.711 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6809", 2024-09-11T08:55:12.711 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:55:12.711 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.711 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.711 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:55:12.711 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.711 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6806", 2024-09-11T08:55:12.711 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:55:12.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.712 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.712 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6807", 2024-09-11T08:55:12.712 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T08:55:12.712 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.712 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.712 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6803/2040613580", 2024-09-11T08:55:12.712 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6805/2040613580", 2024-09-11T08:55:12.712 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6809/2040613580", 2024-09-11T08:55:12.712 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6807/2040613580", 2024-09-11T08:55:12.712 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:55:12.712 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:55:12.712 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:55:12.713 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.713 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-09-11T08:55:12.713 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "e7a18e45-0137-45ef-8b85-e458cf3737ac", 2024-09-11T08:55:12.713 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:55:12.713 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:55:12.713 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:55:12.713 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:55:12.713 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:55:12.713 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:55:12.713 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 13, 2024-09-11T08:55:12.713 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 319, 2024-09-11T08:55:12.714 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:55:12.714 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:55:12.714 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:55:12.714 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.714 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.714 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6810", 2024-09-11T08:55:12.714 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:55:12.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.714 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.715 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6811", 2024-09-11T08:55:12.715 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:55:12.715 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.715 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.715 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:55:12.715 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.715 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.715 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6812", 2024-09-11T08:55:12.715 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:55:12.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.715 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.716 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6813", 2024-09-11T08:55:12.716 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:55:12.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.716 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.716 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:55:12.716 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.716 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.716 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6816", 2024-09-11T08:55:12.716 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:55:12.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.716 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.717 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6817", 2024-09-11T08:55:12.717 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:55:12.717 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.717 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.717 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:55:12.717 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.717 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.717 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6814", 2024-09-11T08:55:12.717 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:55:12.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.717 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.718 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6815", 2024-09-11T08:55:12.718 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T08:55:12.718 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.718 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.718 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6811/544779328", 2024-09-11T08:55:12.718 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6813/544779328", 2024-09-11T08:55:12.718 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6817/544779328", 2024-09-11T08:55:12.718 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6815/544779328", 2024-09-11T08:55:12.718 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:55:12.718 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:55:12.718 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:55:12.718 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.719 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-09-11T08:55:12.719 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "017282e2-8053-4fa7-affc-a05408fa481e", 2024-09-11T08:55:12.719 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:55:12.719 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:55:12.719 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:55:12.719 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 0.5283050537109375, 2024-09-11T08:55:12.719 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:55:12.719 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:55:12.719 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 18, 2024-09-11T08:55:12.719 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 293, 2024-09-11T08:55:12.719 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:55:12.719 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:55:12.719 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:55:12.719 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:12.721 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:12.721 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:12.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.722 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.722 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6818", 2024-09-11T08:55:12.722 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:55:12.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.722 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.722 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6819", 2024-09-11T08:55:12.722 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:55:12.722 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.722 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.722 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:55:12.722 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.723 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.723 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6820", 2024-09-11T08:55:12.723 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:55:12.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.723 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.723 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6821", 2024-09-11T08:55:12.723 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:55:12.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.723 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.723 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:55:12.723 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.724 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6824", 2024-09-11T08:55:12.724 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:55:12.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.724 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6825", 2024-09-11T08:55:12.724 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:55:12.724 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.724 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.724 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:55:12.724 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.725 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.725 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6822", 2024-09-11T08:55:12.725 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:55:12.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.725 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.725 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6823", 2024-09-11T08:55:12.725 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T08:55:12.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.725 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.725 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6819/933736777", 2024-09-11T08:55:12.725 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6821/933736777", 2024-09-11T08:55:12.726 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6825/933736777", 2024-09-11T08:55:12.726 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6823/933736777", 2024-09-11T08:55:12.726 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:55:12.726 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:55:12.726 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:55:12.726 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.726 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-09-11T08:55:12.726 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "58e5a8a4-9e58-40ad-9b40-90cb5ff5414b", 2024-09-11T08:55:12.726 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:55:12.726 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:55:12.726 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:55:12.726 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:55:12.727 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:55:12.727 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:55:12.727 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 26, 2024-09-11T08:55:12.727 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 319, 2024-09-11T08:55:12.727 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:55:12.727 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:55:12.727 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:55:12.727 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.727 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.727 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6826", 2024-09-11T08:55:12.727 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:55:12.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.728 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.728 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6827", 2024-09-11T08:55:12.728 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:55:12.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.728 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.728 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:55:12.728 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.728 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.728 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6828", 2024-09-11T08:55:12.728 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:55:12.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.729 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.729 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6829", 2024-09-11T08:55:12.729 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:55:12.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.729 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.729 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:55:12.729 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.729 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.729 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6832", 2024-09-11T08:55:12.729 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:55:12.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.730 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.730 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6833", 2024-09-11T08:55:12.730 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:55:12.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.730 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.730 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:55:12.730 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.730 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.730 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6830", 2024-09-11T08:55:12.730 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:55:12.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.731 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6831", 2024-09-11T08:55:12.731 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T08:55:12.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.731 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.731 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6827/878892459", 2024-09-11T08:55:12.731 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6829/878892459", 2024-09-11T08:55:12.731 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6833/878892459", 2024-09-11T08:55:12.731 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6831/878892459", 2024-09-11T08:55:12.731 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:55:12.731 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:55:12.731 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:55:12.731 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.732 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-09-11T08:55:12.732 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "e144e102-963b-48b7-92a9-9391706164bf", 2024-09-11T08:55:12.732 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:55:12.732 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:55:12.732 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:55:12.732 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:55:12.732 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:55:12.732 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:55:12.732 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 31, 2024-09-11T08:55:12.732 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 319, 2024-09-11T08:55:12.732 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:55:12.732 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:55:12.733 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:55:12.733 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.733 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.733 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6800", 2024-09-11T08:55:12.733 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:55:12.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.733 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.733 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6801", 2024-09-11T08:55:12.733 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:55:12.733 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.733 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.734 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:55:12.734 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.734 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.734 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6802", 2024-09-11T08:55:12.734 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:55:12.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.734 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.734 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6803", 2024-09-11T08:55:12.734 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:55:12.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.734 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.735 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:55:12.735 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.735 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6806", 2024-09-11T08:55:12.735 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:55:12.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.735 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6807", 2024-09-11T08:55:12.735 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:55:12.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.735 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.736 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:55:12.736 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.736 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.736 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6804", 2024-09-11T08:55:12.736 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:55:12.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.736 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.736 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6805", 2024-09-11T08:55:12.736 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T08:55:12.736 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.736 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.737 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6801/2792751669", 2024-09-11T08:55:12.737 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6803/2792751669", 2024-09-11T08:55:12.737 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6807/2792751669", 2024-09-11T08:55:12.737 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6805/2792751669", 2024-09-11T08:55:12.737 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:55:12.737 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:55:12.737 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:55:12.737 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.737 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-09-11T08:55:12.737 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "811b0dc9-b7a7-439a-86c1-5dc9d0d54e16", 2024-09-11T08:55:12.737 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:55:12.737 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:55:12.738 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:55:12.738 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:55:12.738 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:55:12.738 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:55:12.738 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 38, 2024-09-11T08:55:12.738 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 319, 2024-09-11T08:55:12.738 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:55:12.738 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:55:12.738 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:55:12.738 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.738 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.738 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6808", 2024-09-11T08:55:12.738 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:55:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6809", 2024-09-11T08:55:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:55:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:55:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.739 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6810", 2024-09-11T08:55:12.740 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:55:12.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.740 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6811", 2024-09-11T08:55:12.740 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:55:12.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.740 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.740 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:55:12.740 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.741 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.741 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6814", 2024-09-11T08:55:12.741 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:55:12.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.741 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.741 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6815", 2024-09-11T08:55:12.741 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:55:12.741 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.741 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.741 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:55:12.741 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.742 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.742 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6812", 2024-09-11T08:55:12.742 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:55:12.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.742 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.742 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6813", 2024-09-11T08:55:12.742 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T08:55:12.742 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.742 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.742 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6809/3569637188", 2024-09-11T08:55:12.742 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6811/3569637188", 2024-09-11T08:55:12.743 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6815/3569637188", 2024-09-11T08:55:12.743 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6813/3569637188", 2024-09-11T08:55:12.743 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:55:12.743 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:55:12.743 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:55:12.743 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.743 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-09-11T08:55:12.743 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "bd162f52-097c-4c5d-9d37-1289779a196b", 2024-09-11T08:55:12.743 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:55:12.743 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:55:12.743 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:55:12.743 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T08:55:12.744 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:55:12.744 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:55:12.744 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 45, 2024-09-11T08:55:12.744 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 293, 2024-09-11T08:55:12.744 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:55:12.744 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:55:12.744 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:55:12.744 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.744 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6816", 2024-09-11T08:55:12.744 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:55:12.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.745 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.745 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6817", 2024-09-11T08:55:12.745 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:55:12.745 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.745 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.745 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:55:12.745 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.746 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.746 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6818", 2024-09-11T08:55:12.746 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:55:12.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.746 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.746 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6819", 2024-09-11T08:55:12.746 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:55:12.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.746 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.746 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:55:12.747 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.747 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.747 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6822", 2024-09-11T08:55:12.747 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:55:12.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.747 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.747 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6823", 2024-09-11T08:55:12.747 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:55:12.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.747 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.747 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:55:12.748 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.748 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6820", 2024-09-11T08:55:12.748 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:55:12.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.748 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6821", 2024-09-11T08:55:12.748 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T08:55:12.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.748 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.748 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6817/2615047110", 2024-09-11T08:55:12.749 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6819/2615047110", 2024-09-11T08:55:12.749 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6823/2615047110", 2024-09-11T08:55:12.749 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6821/2615047110", 2024-09-11T08:55:12.749 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:55:12.749 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:55:12.749 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:55:12.749 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.749 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-09-11T08:55:12.749 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "4e4021bf-78c7-4bc6-b222-3494f724ddec", 2024-09-11T08:55:12.749 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T08:55:12.749 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T08:55:12.749 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T08:55:12.750 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 0.2356109619140625, 2024-09-11T08:55:12.750 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T08:55:12.750 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T08:55:12.750 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 52, 2024-09-11T08:55:12.750 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 290, 2024-09-11T08:55:12.750 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T08:55:12.750 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T08:55:12.750 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T08:55:12.750 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.750 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.750 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6824", 2024-09-11T08:55:12.750 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:55:12.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.751 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.751 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6825", 2024-09-11T08:55:12.751 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:55:12.751 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.751 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.751 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T08:55:12.751 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.751 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.751 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6826", 2024-09-11T08:55:12.751 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:55:12.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.752 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.752 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6827", 2024-09-11T08:55:12.752 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:55:12.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.752 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.752 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T08:55:12.752 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.752 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.752 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6830", 2024-09-11T08:55:12.752 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:55:12.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.753 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6831", 2024-09-11T08:55:12.753 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:55:12.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.753 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.753 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T08:55:12.753 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T08:55:12.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T08:55:12.753 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6828", 2024-09-11T08:55:12.753 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:55:12.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.754 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T08:55:12.754 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6829", 2024-09-11T08:55:12.754 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T08:55:12.754 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.754 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.754 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6825/3195898530", 2024-09-11T08:55:12.754 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6827/3195898530", 2024-09-11T08:55:12.754 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6831/3195898530", 2024-09-11T08:55:12.754 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6829/3195898530", 2024-09-11T08:55:12.754 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T08:55:12.754 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T08:55:12.755 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T08:55:12.755 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.755 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.755 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:55:12.755 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_xinfo": [ 2024-09-11T08:55:12.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.755 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-09-11T08:55:12.755 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:55:12.755 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:55:12.755 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:55:12.755 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:55:12.755 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:55:12.755 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:54:26.062746+0000", 2024-09-11T08:55:12.755 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:55:12.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.756 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-09-11T08:55:12.756 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:55:12.756 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:55:12.756 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:55:12.756 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:55:12.756 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:55:12.756 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:54:06.883496+0000", 2024-09-11T08:55:12.756 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:55:12.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.756 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-09-11T08:55:12.757 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:55:12.757 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:55:12.757 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:55:12.757 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:55:12.757 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:55:12.757 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:54:24.142709+0000", 2024-09-11T08:55:12.757 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:55:12.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.757 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-09-11T08:55:12.757 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:55:12.757 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:55:12.757 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:55:12.758 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:55:12.758 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:55:12.758 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:54:41.689307+0000", 2024-09-11T08:55:12.758 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:55:12.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.758 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-09-11T08:55:12.758 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:55:12.758 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:55:12.758 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:55:12.758 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:55:12.759 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:55:12.759 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:53:49.230911+0000", 2024-09-11T08:55:12.759 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:55:12.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.759 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-09-11T08:55:12.759 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:55:12.759 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:55:12.759 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:55:12.759 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:55:12.760 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:55:12.760 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:54:54.134967+0000", 2024-09-11T08:55:12.760 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:55:12.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.760 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-09-11T08:55:12.760 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:55:12.760 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:55:12.760 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:55:12.760 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:55:12.760 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:55:12.760 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:54:52.888831+0000", 2024-09-11T08:55:12.760 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:55:12.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.761 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-09-11T08:55:12.761 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T08:55:12.761 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T08:55:12.761 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T08:55:12.761 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T08:55:12.761 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T08:55:12.761 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:54:20.722058+0000", 2024-09-11T08:55:12.761 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T08:55:12.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.761 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:55:12.761 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap": [], 2024-09-11T08:55:12.761 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap_items": [ 2024-09-11T08:55:12.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.762 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "3.5", 2024-09-11T08:55:12.762 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:55:12.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.762 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 5, 2024-09-11T08:55:12.762 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T08:55:12.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.762 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.762 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.e", 2024-09-11T08:55:12.762 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:55:12.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.762 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:55:12.762 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T08:55:12.763 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.763 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.763 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.12", 2024-09-11T08:55:12.763 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:55:12.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.763 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:55:12.763 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 5 2024-09-11T08:55:12.763 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.763 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.763 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.13", 2024-09-11T08:55:12.763 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:55:12.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.764 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:55:12.764 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 5 2024-09-11T08:55:12.764 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.764 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.764 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.15", 2024-09-11T08:55:12.764 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:55:12.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.764 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 7, 2024-09-11T08:55:12.764 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 1 2024-09-11T08:55:12.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.765 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.765 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.17", 2024-09-11T08:55:12.765 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:55:12.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.765 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:55:12.765 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T08:55:12.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.765 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.765 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.1d", 2024-09-11T08:55:12.766 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T08:55:12.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:55:12.766 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T08:55:12.766 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T08:55:12.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.766 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:55:12.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.766 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T08:55:12.766 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_temp": [], 2024-09-11T08:55:12.766 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_temp": [], 2024-09-11T08:55:12.766 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklist": { 2024-09-11T08:55:12.766 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6800/1861854063": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:55:12.766 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/4113910808": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:55:12.766 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/3501902574": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:55:12.767 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/1583695466": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:55:12.767 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/736380628": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:55:12.767 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6800/4060453592": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:55:12.767 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/2234574262": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:55:12.767 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6800/2480574848": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:55:12.767 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/1540079149": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:55:12.767 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/2680744738": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:55:12.767 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6801/4060453592": "2024-09-12T08:33:54.195816+0000", 2024-09-11T08:55:12.767 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6801/1861854063": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:55:12.767 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6801/2480574848": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:55:12.767 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/12697418": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:55:12.767 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/4280958069": "2024-09-12T08:34:17.306389+0000", 2024-09-11T08:55:12.767 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/2686720854": "2024-09-12T08:39:14.984442+0000", 2024-09-11T08:55:12.767 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/969221731": "2024-09-12T08:39:14.984442+0000" 2024-09-11T08:55:12.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.768 INFO:teuthology.orchestra.run.smithi092.stdout: "range_blocklist": {}, 2024-09-11T08:55:12.768 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profiles": { 2024-09-11T08:55:12.768 INFO:teuthology.orchestra.run.smithi092.stdout: "default": { 2024-09-11T08:55:12.768 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-failure-domain": "osd", 2024-09-11T08:55:12.768 INFO:teuthology.orchestra.run.smithi092.stdout: "k": "2", 2024-09-11T08:55:12.768 INFO:teuthology.orchestra.run.smithi092.stdout: "m": "1", 2024-09-11T08:55:12.768 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin": "jerasure", 2024-09-11T08:55:12.768 INFO:teuthology.orchestra.run.smithi092.stdout: "technique": "reed_sol_van" 2024-09-11T08:55:12.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:55:12.768 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps_queue": [], 2024-09-11T08:55:12.768 INFO:teuthology.orchestra.run.smithi092.stdout: "new_removed_snaps": [], 2024-09-11T08:55:12.768 INFO:teuthology.orchestra.run.smithi092.stdout: "new_purged_snaps": [], 2024-09-11T08:55:12.768 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_node_flags": {}, 2024-09-11T08:55:12.768 INFO:teuthology.orchestra.run.smithi092.stdout: "device_class_flags": {}, 2024-09-11T08:55:12.769 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode": { 2024-09-11T08:55:12.769 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_enabled": false, 2024-09-11T08:55:12.769 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_bucket_count": 0, 2024-09-11T08:55:12.769 INFO:teuthology.orchestra.run.smithi092.stdout: "degraded_stretch_mode": 0, 2024-09-11T08:55:12.769 INFO:teuthology.orchestra.run.smithi092.stdout: "recovering_stretch_mode": 0, 2024-09-11T08:55:12.769 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_bucket": 0 2024-09-11T08:55:12.769 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:55:12.769 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:55:12.822 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:12.852 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:12.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:12.853 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:12.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:55:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:55:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:55:12.954 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:12.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:12.989 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:12.989 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:13.074 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-11T08:55:13.090 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:13.165 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:13.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:13.166 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:13.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:13.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:13.302 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:13.302 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:13.403 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:13.412 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:13.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:13.453 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:13.453 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:13.499 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:55:13.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:13 smithi092 ceph-mon[29647]: 3.9 scrub starts 2024-09-11T08:55:13.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:13 smithi092 ceph-mon[29647]: 3.9 scrub ok 2024-09-11T08:55:13.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:13 smithi092 ceph-mon[29647]: 4.17 scrub starts 2024-09-11T08:55:13.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:13 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/213694462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T08:55:13.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:13 smithi092 ceph-mon[29647]: 4.17 scrub ok 2024-09-11T08:55:13.519 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:13 smithi092 ceph-mon[41114]: 3.9 scrub starts 2024-09-11T08:55:13.519 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:13 smithi092 ceph-mon[41114]: 3.9 scrub ok 2024-09-11T08:55:13.519 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:13 smithi092 ceph-mon[41114]: 4.17 scrub starts 2024-09-11T08:55:13.519 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:13 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/213694462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T08:55:13.519 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:13 smithi092 ceph-mon[41114]: 4.17 scrub ok 2024-09-11T08:55:13.554 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:13.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:13.597 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:13.598 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:13.632 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:13 smithi145 ceph-mon[33384]: 3.9 scrub starts 2024-09-11T08:55:13.632 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:13 smithi145 ceph-mon[33384]: 3.9 scrub ok 2024-09-11T08:55:13.632 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:13 smithi145 ceph-mon[33384]: 4.17 scrub starts 2024-09-11T08:55:13.632 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:13 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/213694462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T08:55:13.632 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:13 smithi145 ceph-mon[33384]: 4.17 scrub ok 2024-09-11T08:55:13.699 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:13.738 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:13.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:13.739 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:13.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:13.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:13.870 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:13.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:14.000 INFO:tasks.thrashosds.thrasher:j is {'epoch': 330, 'fsid': '211625d4-7018-11ef-bcea-c7b262605968', 'created': '2024-09-11T08:33:32.174444+0000', 'modified': '2024-09-11T08:55:03.201404+0000', 'last_up_change': '2024-09-11T08:38:23.097480+0000', 'last_in_change': '2024-09-11T08:52:17.544985+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 10, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T08:36:29.606460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '250', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '248', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-11T08:38:37.674272+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-11T08:38:37.912906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-11T08:38:39.234173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-11T08:38:41.216386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-11T08:38:43.285497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 10, 'pool_name': 'unique_pool_3', 'create_time': '2024-09-11T08:52:53.990481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '297', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '9d8a3811-e801-4b6a-9a63-83936283cb7d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1711883544921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 319, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 2040613580}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 2040613580}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 2040613580}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 2040613580}]}, 'public_addr': '172.21.15.92:6803/2040613580', 'cluster_addr': '172.21.15.92:6805/2040613580', 'heartbeat_back_addr': '172.21.15.92:6809/2040613580', 'heartbeat_front_addr': '172.21.15.92:6807/2040613580', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e7a18e45-0137-45ef-8b85-e458cf3737ac', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 319, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 544779328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 544779328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 544779328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 544779328}]}, 'public_addr': '172.21.15.92:6811/544779328', 'cluster_addr': '172.21.15.92:6813/544779328', 'heartbeat_back_addr': '172.21.15.92:6817/544779328', 'heartbeat_front_addr': '172.21.15.92:6815/544779328', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '017282e2-8053-4fa7-affc-a05408fa481e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5283050537109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 933736777}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 933736777}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 933736777}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 933736777}]}, 'public_addr': '172.21.15.92:6819/933736777', 'cluster_addr': '172.21.15.92:6821/933736777', 'heartbeat_back_addr': '172.21.15.92:6825/933736777', 'heartbeat_front_addr': '172.21.15.92:6823/933736777', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '58e5a8a4-9e58-40ad-9b40-90cb5ff5414b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 319, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 878892459}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 878892459}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 878892459}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 878892459}]}, 'public_addr': '172.21.15.92:6827/878892459', 'cluster_addr': '172.21.15.92:6829/878892459', 'heartbeat_back_addr': '172.21.15.92:6833/878892459', 'heartbeat_front_addr': '172.21.15.92:6831/878892459', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e144e102-963b-48b7-92a9-9391706164bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 319, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6800', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6801', 'nonce': 2792751669}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6802', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6803', 'nonce': 2792751669}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6806', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6807', 'nonce': 2792751669}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6804', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6805', 'nonce': 2792751669}]}, 'public_addr': '172.21.15.145:6801/2792751669', 'cluster_addr': '172.21.15.145:6803/2792751669', 'heartbeat_back_addr': '172.21.15.145:6807/2792751669', 'heartbeat_front_addr': '172.21.15.145:6805/2792751669', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '811b0dc9-b7a7-439a-86c1-5dc9d0d54e16', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 38, 'up_thru': 319, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6808', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6809', 'nonce': 3569637188}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6810', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6811', 'nonce': 3569637188}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6814', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6815', 'nonce': 3569637188}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6812', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6813', 'nonce': 3569637188}]}, 'public_addr': '172.21.15.145:6809/3569637188', 'cluster_addr': '172.21.15.145:6811/3569637188', 'heartbeat_back_addr': '172.21.15.145:6815/3569637188', 'heartbeat_front_addr': '172.21.15.145:6813/3569637188', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'bd162f52-097c-4c5d-9d37-1289779a196b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6816', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6817', 'nonce': 2615047110}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6818', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6819', 'nonce': 2615047110}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6822', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6823', 'nonce': 2615047110}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6820', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6821', 'nonce': 2615047110}]}, 'public_addr': '172.21.15.145:6817/2615047110', 'cluster_addr': '172.21.15.145:6819/2615047110', 'heartbeat_back_addr': '172.21.15.145:6823/2615047110', 'heartbeat_front_addr': '172.21.15.145:6821/2615047110', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '4e4021bf-78c7-4bc6-b222-3494f724ddec', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.2356109619140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 52, 'up_thru': 290, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6824', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6825', 'nonce': 3195898530}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6826', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6827', 'nonce': 3195898530}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6830', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6831', 'nonce': 3195898530}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6828', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6829', 'nonce': 3195898530}]}, 'public_addr': '172.21.15.145:6825/3195898530', 'cluster_addr': '172.21.15.145:6827/3195898530', 'heartbeat_back_addr': '172.21.15.145:6831/3195898530', 'heartbeat_front_addr': '172.21.15.145:6829/3195898530', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:54:26.062746+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:54:06.883496+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:54:24.142709+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:54:41.689307+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:53:49.230911+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:54:54.134967+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:54:52.888831+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:54:20.722058+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.5', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '4.e', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.12', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.15', 'mappings': [{'from': 7, 'to': 1}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.1d', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.92:6800/1861854063': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/4113910808': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:0/3501902574': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/1583695466': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:0/736380628': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:6800/4060453592': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:0/2234574262': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:6800/2480574848': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/1540079149': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/2680744738': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:6801/4060453592': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:6801/1861854063': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:6801/2480574848': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/12697418': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/4280958069': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/2686720854': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/969221731': '2024-09-12T08:39:14.984442+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T08:55:14.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:55:14.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:14.024 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:14.024 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:14.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:14.160 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:14.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:14.161 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:14.262 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:14.311 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:55:14.328 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:14.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:14.329 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:14.430 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:14.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:14.471 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:14.471 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:14.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[29647]: 6.4 scrub starts 2024-09-11T08:55:14.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[29647]: 6.4 scrub ok 2024-09-11T08:55:14.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1397062267' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:55:14.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[29647]: pgmap v743: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1009 B/s rd, 0 B/s wr, 1.02k op/s 2024-09-11T08:55:14.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[29647]: 1.7 scrub starts 2024-09-11T08:55:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[29647]: 1.7 scrub ok 2024-09-11T08:55:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[29647]: 6.19 scrub starts 2024-09-11T08:55:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[29647]: 6.19 scrub ok 2024-09-11T08:55:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[29647]: 6.9 deep-scrub starts 2024-09-11T08:55:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[29647]: 6.9 deep-scrub ok 2024-09-11T08:55:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[41114]: 6.4 scrub starts 2024-09-11T08:55:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[41114]: 6.4 scrub ok 2024-09-11T08:55:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1397062267' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:55:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[41114]: pgmap v743: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1009 B/s rd, 0 B/s wr, 1.02k op/s 2024-09-11T08:55:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[41114]: 1.7 scrub starts 2024-09-11T08:55:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[41114]: 1.7 scrub ok 2024-09-11T08:55:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[41114]: 6.19 scrub starts 2024-09-11T08:55:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[41114]: 6.19 scrub ok 2024-09-11T08:55:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[41114]: 6.9 deep-scrub starts 2024-09-11T08:55:14.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:14 smithi092 ceph-mon[41114]: 6.9 deep-scrub ok 2024-09-11T08:55:14.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:14.602 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:14.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:14.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:14.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:14 smithi145 ceph-mon[33384]: 6.4 scrub starts 2024-09-11T08:55:14.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:14 smithi145 ceph-mon[33384]: 6.4 scrub ok 2024-09-11T08:55:14.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:14 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1397062267' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:55:14.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:14 smithi145 ceph-mon[33384]: pgmap v743: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1009 B/s rd, 0 B/s wr, 1.02k op/s 2024-09-11T08:55:14.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:14 smithi145 ceph-mon[33384]: 1.7 scrub starts 2024-09-11T08:55:14.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:14 smithi145 ceph-mon[33384]: 1.7 scrub ok 2024-09-11T08:55:14.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:14 smithi145 ceph-mon[33384]: 6.19 scrub starts 2024-09-11T08:55:14.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:14 smithi145 ceph-mon[33384]: 6.19 scrub ok 2024-09-11T08:55:14.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:14 smithi145 ceph-mon[33384]: 6.9 deep-scrub starts 2024-09-11T08:55:14.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:14 smithi145 ceph-mon[33384]: 6.9 deep-scrub ok 2024-09-11T08:55:14.704 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:14.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:14.742 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:14.742 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:14.843 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:14.874 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:14.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:14.875 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:14.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:15.007 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:15.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:15.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:15.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:15.143 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:15.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:15.144 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:15.245 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:15.276 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:15.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:15.277 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:15.377 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:15.407 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:15.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:15.409 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:15.509 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:15.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:15.539 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:15.539 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:15.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:15 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:55:15.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:15 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1397062267' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:55:15.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:15 smithi092 ceph-mon[41114]: osdmap e331: 8 total, 8 up, 8 in 2024-09-11T08:55:15.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:55:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:15 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:55:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:15 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1397062267' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:55:15.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:15 smithi092 ceph-mon[29647]: osdmap e331: 8 total, 8 up, 8 in 2024-09-11T08:55:15.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:55:15.640 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:15.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:15 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:55:15.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:15 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1397062267' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:55:15.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:15 smithi145 ceph-mon[33384]: osdmap e331: 8 total, 8 up, 8 in 2024-09-11T08:55:15.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:55:15.673 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:15.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:15.675 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:15.776 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:15.810 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:15.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:15.811 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:15.912 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:15.950 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:15.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:15.951 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:16.052 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:16.087 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:16.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:16.088 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:16.185 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:16.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:16.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:16.256 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:16.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:16.398 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:16.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:16.400 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:16.496 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:16 smithi092 ceph-mon[29647]: pgmap v745: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T08:55:16.496 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:16 smithi092 ceph-mon[41114]: pgmap v745: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T08:55:16.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:16.537 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:16.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:16.538 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:16.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:16.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:16 smithi145 ceph-mon[33384]: pgmap v745: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T08:55:16.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:16.680 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:16.680 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:16.781 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:16.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:16.814 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:16.814 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:16.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:16.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:16.937 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:16.937 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:17.038 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:17.082 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:17.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:17.083 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:17.125 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:55:17.125 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:55:17.155 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:55:17.155 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:55:17.181 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:55:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:55:17.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:17.209 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:55:17.210 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:55:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:17.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:17.223 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:17.253 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:55:17.253 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:55:17.292 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:55:17.292 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:55:17.310 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:55:17 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:55:17.325 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:17.330 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:55:17.330 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:55:17.357 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:17.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:17.358 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:17.361 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:55:17.362 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:55:17.362 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:17 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1633762060' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:17.363 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:17 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1633762060' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:17.459 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:17.492 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:17.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:17.493 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:17.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:17.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:17.628 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:17.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:55:17.639 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:17 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1633762060' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:17.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:17.752 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:17.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:17.753 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:17.854 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:17.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:17.883 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:17.884 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:17.984 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:18.007 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:18.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:18.008 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:18.109 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:18.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:18.147 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:18.148 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:18.249 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:18.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:18.283 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:18.283 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:18.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:18.417 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:18.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:18.418 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:18.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:18.551 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:18.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:18.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:18.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:18 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:18.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:18 smithi092 ceph-mon[29647]: pgmap v746: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 887 op/s 2024-09-11T08:55:18.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:18 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:18.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:18 smithi092 ceph-mon[41114]: pgmap v746: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 887 op/s 2024-09-11T08:55:18.618 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:18 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:18.618 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:18 smithi145 ceph-mon[33384]: pgmap v746: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 887 op/s 2024-09-11T08:55:18.652 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:18.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:18.680 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:18.680 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:18.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:18.824 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:18.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:18.825 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:18.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:18.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:18.955 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:18.956 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:19.057 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:19.086 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:19.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:19.087 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:19.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:19.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:19.216 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:19.216 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:19.318 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:19.337 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:19.351 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:19.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:19.352 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:19.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:19.483 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:19.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:19.484 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:19.563 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:19 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:55:19.563 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:19 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:55:19.563 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:19 smithi092 ceph-mon[41114]: osdmap e332: 8 total, 8 up, 8 in 2024-09-11T08:55:19.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:19 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:55:19.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:19 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:55:19.564 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:19 smithi092 ceph-mon[29647]: osdmap e332: 8 total, 8 up, 8 in 2024-09-11T08:55:19.585 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:19.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:19.616 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:19.616 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:19.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:19 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:55:19.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:19 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:55:19.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:19 smithi145 ceph-mon[33384]: osdmap e332: 8 total, 8 up, 8 in 2024-09-11T08:55:19.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:19.752 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:19.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:19.753 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:19.854 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:19.890 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:19.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:19.891 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:19.992 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:20.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:20.026 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:20.026 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:20.127 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:20.161 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:20.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:20.162 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:20.263 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:20.297 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:20.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:20.298 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:20.399 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:20.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:20 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:55:20.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:20 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:55:20.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:20 smithi145 ceph-mon[33384]: osdmap e333: 8 total, 8 up, 8 in 2024-09-11T08:55:20.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:20 smithi145 ceph-mon[33384]: pgmap v749: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 742 op/s 2024-09-11T08:55:20.426 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:20.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:20.427 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:55:20.502 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:20 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:55:20.502 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:20 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:55:20.502 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:20 smithi092 ceph-mon[29647]: osdmap e333: 8 total, 8 up, 8 in 2024-09-11T08:55:20.503 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:20 smithi092 ceph-mon[29647]: pgmap v749: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 742 op/s 2024-09-11T08:55:20.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:20 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:55:20.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:20 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:55:20.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:20 smithi092 ceph-mon[41114]: osdmap e333: 8 total, 8 up, 8 in 2024-09-11T08:55:20.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:20 smithi092 ceph-mon[41114]: pgmap v749: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 742 op/s 2024-09-11T08:55:20.511 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:55:20.515 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:55:20.515 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:55:20.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:20.562 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:20.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:20.563 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:20.664 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:20.697 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:20.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:20.698 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:20.723 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:55:20.799 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:20.836 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:20.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:20.837 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:20.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:20.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:20.971 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:20.971 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:21.072 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:21.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:21.109 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:21.109 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:21.210 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:21.213 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-11T08:55:21.239 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:21.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:21.240 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:21.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:21.392 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":749,"stamp":"2024-09-11T08:55:19.359821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5794830580,"num_objects":88610,"num_object_clones":0,"num_object_copies":265830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88610,"num_whiteouts":0,"num_read":1291,"num_read_kb":1043,"num_write":186216,"num_write_kb":8765228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":70186,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84684,"ondisk_log_size":84684,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":23039520,"kb_used_data":17060816,"kb_used_omap":13,"kb_used_meta":5978546,"kb_avail":726921696,"statfs":{"total":767960285184,"available":744367816704,"internally_reserved":0,"allocated":17470275584,"data_stored":17467703788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":6122031808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,5,2,13,15,15,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-291700736,"num_objects":-4451,"num_object_clones":0,"num_object_copies":-13353,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4451,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4451,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002123"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":149,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.227427+0000","last_change":"2024-09-11T08:54:18.331227+0000","last_active":"2024-09-11T08:55:03.227427+0000","last_peered":"2024-09-11T08:55:03.227427+0000","last_clean":"2024-09-11T08:55:03.227427+0000","last_became_active":"2024-09-11T08:52:49.568416+0000","last_became_peered":"2024-09-11T08:52:49.568416+0000","last_unstale":"2024-09-11T08:55:03.227427+0000","last_undegraded":"2024-09-11T08:55:03.227427+0000","last_fullsized":"2024-09-11T08:55:03.227427+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:18.331155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:18.331155+0000","last_clean_scrub_stamp":"2024-09-11T08:54:18.331155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:30.540212+0000","scrub_duration":0.0042788669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"200'19","reported_seq":401,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.224478+0000","last_change":"2024-09-11T08:52:30.312273+0000","last_active":"2024-09-11T08:55:03.224478+0000","last_peered":"2024-09-11T08:55:03.224478+0000","last_clean":"2024-09-11T08:55:03.224478+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:55:03.224478+0000","last_undegraded":"2024-09-11T08:55:03.224478+0000","last_fullsized":"2024-09-11T08:55:03.224478+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:52:30.312197+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:52:30.312197+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002912516,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":349,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.232671+0000","last_change":"2024-09-11T08:53:54.853514+0000","last_active":"2024-09-11T08:55:03.232671+0000","last_peered":"2024-09-11T08:55:03.232671+0000","last_clean":"2024-09-11T08:55:03.232671+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:55:03.232671+0000","last_undegraded":"2024-09-11T08:55:03.232671+0000","last_fullsized":"2024-09-11T08:55:03.232671+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:54.853421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:54.853421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:22.460919+0000","scrub_duration":0.0077955339999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":328,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.224113+0000","last_change":"2024-09-11T08:54:13.811462+0000","last_active":"2024-09-11T08:55:03.224113+0000","last_peered":"2024-09-11T08:55:03.224113+0000","last_clean":"2024-09-11T08:55:03.224113+0000","last_became_active":"2024-09-11T08:38:44.991822+0000","last_became_peered":"2024-09-11T08:38:44.991822+0000","last_unstale":"2024-09-11T08:55:03.224113+0000","last_undegraded":"2024-09-11T08:55:03.224113+0000","last_fullsized":"2024-09-11T08:55:03.224113+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:13.811054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:13.811054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:28.408617+0000","scrub_duration":0.010996769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":357,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.244965+0000","last_change":"2024-09-11T08:54:58.730290+0000","last_active":"2024-09-11T08:55:03.244965+0000","last_peered":"2024-09-11T08:55:03.244965+0000","last_clean":"2024-09-11T08:55:03.244965+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:55:03.244965+0000","last_undegraded":"2024-09-11T08:55:03.244965+0000","last_fullsized":"2024-09-11T08:55:03.244965+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:58.730211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:58.730211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:08.428169+0000","scrub_duration":0.011911339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":363,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.245358+0000","last_change":"2024-09-11T08:54:52.562148+0000","last_active":"2024-09-11T08:55:03.245358+0000","last_peered":"2024-09-11T08:55:03.245358+0000","last_clean":"2024-09-11T08:55:03.245358+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:55:03.245358+0000","last_undegraded":"2024-09-11T08:55:03.245358+0000","last_fullsized":"2024-09-11T08:55:03.245358+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:52.562065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:54:52.562065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:18.619494+0000","scrub_duration":0.0074997070000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"200'34","reported_seq":443,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.244543+0000","last_change":"2024-09-11T08:54:12.545864+0000","last_active":"2024-09-11T08:55:03.244543+0000","last_peered":"2024-09-11T08:55:03.244543+0000","last_clean":"2024-09-11T08:55:03.244543+0000","last_became_active":"2024-09-11T08:52:18.585600+0000","last_became_peered":"2024-09-11T08:52:18.585600+0000","last_unstale":"2024-09-11T08:55:03.244543+0000","last_undegraded":"2024-09-11T08:55:03.244543+0000","last_fullsized":"2024-09-11T08:55:03.244543+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:54:12.545773+0000","last_deep_scrub":"200'34","last_deep_scrub_stamp":"2024-09-11T08:51:31.358660+0000","last_clean_scrub_stamp":"2024-09-11T08:54:12.545773+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:18.086513+0000","scrub_duration":0.0070952350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":330,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.224404+0000","last_change":"2024-09-11T08:54:55.729780+0000","last_active":"2024-09-11T08:55:03.224404+0000","last_peered":"2024-09-11T08:55:03.224404+0000","last_clean":"2024-09-11T08:55:03.224404+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:55:03.224404+0000","last_undegraded":"2024-09-11T08:55:03.224404+0000","last_fullsized":"2024-09-11T08:55:03.224404+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:55.729679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:54:55.729679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:57.001810+0000","scrub_duration":0.002263007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":369,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.226848+0000","last_change":"2024-09-11T08:54:10.466581+0000","last_active":"2024-09-11T08:55:03.226848+0000","last_peered":"2024-09-11T08:55:03.226848+0000","last_clean":"2024-09-11T08:55:03.226848+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:55:03.226848+0000","last_undegraded":"2024-09-11T08:55:03.226848+0000","last_fullsized":"2024-09-11T08:55:03.226848+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:10.466478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:54:10.466478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:36.076266+0000","scrub_duration":0.0034798429999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"200'19","reported_seq":411,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.224812+0000","last_change":"2024-09-11T08:54:09.865650+0000","last_active":"2024-09-11T08:55:03.224812+0000","last_peered":"2024-09-11T08:55:03.224812+0000","last_clean":"2024-09-11T08:55:03.224812+0000","last_became_active":"2024-09-11T08:52:18.577411+0000","last_became_peered":"2024-09-11T08:52:18.577411+0000","last_unstale":"2024-09-11T08:55:03.224812+0000","last_undegraded":"2024-09-11T08:55:03.224812+0000","last_fullsized":"2024-09-11T08:55:03.224812+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:54:09.865565+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.865565+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:31.834039+0000","scrub_duration":0.0044543600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":365,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.226501+0000","last_change":"2024-09-11T08:53:59.484790+0000","last_active":"2024-09-11T08:55:03.226501+0000","last_peered":"2024-09-11T08:55:03.226501+0000","last_clean":"2024-09-11T08:55:03.226501+0000","last_became_active":"2024-09-11T08:52:18.573461+0000","last_became_peered":"2024-09-11T08:52:18.573461+0000","last_unstale":"2024-09-11T08:55:03.226501+0000","last_undegraded":"2024-09-11T08:55:03.226501+0000","last_fullsized":"2024-09-11T08:55:03.226501+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:59.484561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:59.484561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060038859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":149,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.236310+0000","last_change":"2024-09-11T08:54:10.819941+0000","last_active":"2024-09-11T08:55:03.236310+0000","last_peered":"2024-09-11T08:55:03.236310+0000","last_clean":"2024-09-11T08:55:03.236310+0000","last_became_active":"2024-09-11T08:52:49.568147+0000","last_became_peered":"2024-09-11T08:52:49.568147+0000","last_unstale":"2024-09-11T08:55:03.236310+0000","last_undegraded":"2024-09-11T08:55:03.236310+0000","last_fullsized":"2024-09-11T08:55:03.236310+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:10.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:33.520462+0000","last_clean_scrub_stamp":"2024-09-11T08:54:10.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:32.684708+0000","scrub_duration":0.002056146,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":349,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.226631+0000","last_change":"2024-09-11T08:54:17.338878+0000","last_active":"2024-09-11T08:55:03.226631+0000","last_peered":"2024-09-11T08:55:03.226631+0000","last_clean":"2024-09-11T08:55:03.226631+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:55:03.226631+0000","last_undegraded":"2024-09-11T08:55:03.226631+0000","last_fullsized":"2024-09-11T08:55:03.226631+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:17.338793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:30.465003+0000","last_clean_scrub_stamp":"2024-09-11T08:54:17.338793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:36.334048+0000","scrub_duration":0.0028270790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"200'7","reported_seq":338,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223247+0000","last_change":"2024-09-11T08:52:11.155994+0000","last_active":"2024-09-11T08:55:03.223247+0000","last_peered":"2024-09-11T08:55:03.223247+0000","last_clean":"2024-09-11T08:55:03.223247+0000","last_became_active":"2024-09-11T08:38:40.973732+0000","last_became_peered":"2024-09-11T08:38:40.973732+0000","last_unstale":"2024-09-11T08:55:03.223247+0000","last_undegraded":"2024-09-11T08:55:03.223247+0000","last_fullsized":"2024-09-11T08:55:03.223247+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'7","last_scrub_stamp":"2024-09-11T08:52:11.155898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:52:11.155898+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002524072,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":358,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.218537+0000","last_change":"2024-09-11T08:54:53.764137+0000","last_active":"2024-09-11T08:55:03.218537+0000","last_peered":"2024-09-11T08:55:03.218537+0000","last_clean":"2024-09-11T08:55:03.218537+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:55:03.218537+0000","last_undegraded":"2024-09-11T08:55:03.218537+0000","last_fullsized":"2024-09-11T08:55:03.218537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:53.764049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:18.199651+0000","last_clean_scrub_stamp":"2024-09-11T08:54:53.764049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:02.539775+0000","scrub_duration":0.0073424750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":343,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:13.401898+0000","last_change":"2024-09-11T08:55:13.401842+0000","last_active":"2024-09-11T08:55:13.401898+0000","last_peered":"2024-09-11T08:55:13.401898+0000","last_clean":"2024-09-11T08:55:13.401898+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:55:13.401898+0000","last_undegraded":"2024-09-11T08:55:13.401898+0000","last_fullsized":"2024-09-11T08:55:13.401898+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:13.401759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:55:13.401759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:26.832209+0000","scrub_duration":0.0070886450000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":348,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.224526+0000","last_change":"2024-09-11T08:53:56.854824+0000","last_active":"2024-09-11T08:55:03.224526+0000","last_peered":"2024-09-11T08:55:03.224526+0000","last_clean":"2024-09-11T08:55:03.224526+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:55:03.224526+0000","last_undegraded":"2024-09-11T08:55:03.224526+0000","last_fullsized":"2024-09-11T08:55:03.224526+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:56.854710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:56.854710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061159049999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":149,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223319+0000","last_change":"2024-09-11T08:53:58.835251+0000","last_active":"2024-09-11T08:55:03.223319+0000","last_peered":"2024-09-11T08:55:03.223319+0000","last_clean":"2024-09-11T08:55:03.223319+0000","last_became_active":"2024-09-11T08:52:49.566575+0000","last_became_peered":"2024-09-11T08:52:49.566575+0000","last_unstale":"2024-09-11T08:55:03.223319+0000","last_undegraded":"2024-09-11T08:55:03.223319+0000","last_fullsized":"2024-09-11T08:55:03.223319+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:58.835161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:58.835161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0065024590000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"200'34","reported_seq":288,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.233699+0000","last_change":"2024-09-11T08:54:17.121799+0000","last_active":"2024-09-11T08:55:03.233699+0000","last_peered":"2024-09-11T08:55:03.233699+0000","last_clean":"2024-09-11T08:55:03.233699+0000","last_became_active":"2024-09-11T08:54:17.121459+0000","last_became_peered":"2024-09-11T08:54:17.121459+0000","last_unstale":"2024-09-11T08:55:03.233699+0000","last_undegraded":"2024-09-11T08:55:03.233699+0000","last_fullsized":"2024-09-11T08:55:03.233699+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:54:04.639215+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:54:04.639215+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:21.573487+0000","scrub_duration":0.011484708999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":369,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.233223+0000","last_change":"2024-09-11T08:53:59.798776+0000","last_active":"2024-09-11T08:55:03.233223+0000","last_peered":"2024-09-11T08:55:03.233223+0000","last_clean":"2024-09-11T08:55:03.233223+0000","last_became_active":"2024-09-11T08:52:18.579026+0000","last_became_peered":"2024-09-11T08:52:18.579026+0000","last_unstale":"2024-09-11T08:55:03.233223+0000","last_undegraded":"2024-09-11T08:55:03.233223+0000","last_fullsized":"2024-09-11T08:55:03.233223+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:59.798705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:59.798705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:22.435474+0000","scrub_duration":0.004245665,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":356,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.250981+0000","last_change":"2024-09-11T08:53:14.430444+0000","last_active":"2024-09-11T08:55:03.250981+0000","last_peered":"2024-09-11T08:55:03.250981+0000","last_clean":"2024-09-11T08:55:03.250981+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:55:03.250981+0000","last_undegraded":"2024-09-11T08:55:03.250981+0000","last_fullsized":"2024-09-11T08:55:03.250981+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:53:14.430366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:14.430366+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024769649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":334,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.224469+0000","last_change":"2024-09-11T08:53:48.866804+0000","last_active":"2024-09-11T08:55:03.224469+0000","last_peered":"2024-09-11T08:55:03.224469+0000","last_clean":"2024-09-11T08:55:03.224469+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:55:03.224469+0000","last_undegraded":"2024-09-11T08:55:03.224469+0000","last_fullsized":"2024-09-11T08:55:03.224469+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:48.866615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:48.866615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:16.879761+0000","scrub_duration":0.0075952449999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"200'18","reported_seq":385,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.247284+0000","last_change":"2024-09-11T08:53:07.440507+0000","last_active":"2024-09-11T08:55:03.247284+0000","last_peered":"2024-09-11T08:55:03.247284+0000","last_clean":"2024-09-11T08:55:03.247284+0000","last_became_active":"2024-09-11T08:49:16.943886+0000","last_became_peered":"2024-09-11T08:49:16.943886+0000","last_unstale":"2024-09-11T08:55:03.247284+0000","last_undegraded":"2024-09-11T08:55:03.247284+0000","last_fullsized":"2024-09-11T08:55:03.247284+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'18","last_scrub_stamp":"2024-09-11T08:53:07.440404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:07.440404+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041402180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":330,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223540+0000","last_change":"2024-09-11T08:53:12.948801+0000","last_active":"2024-09-11T08:55:03.223540+0000","last_peered":"2024-09-11T08:55:03.223540+0000","last_clean":"2024-09-11T08:55:03.223540+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:55:03.223540+0000","last_undegraded":"2024-09-11T08:55:03.223540+0000","last_fullsized":"2024-09-11T08:55:03.223540+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:12.948659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:12.948659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039731519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":76,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.226372+0000","last_change":"2024-09-11T08:53:58.534161+0000","last_active":"2024-09-11T08:55:03.226372+0000","last_peered":"2024-09-11T08:55:03.226372+0000","last_clean":"2024-09-11T08:55:03.226372+0000","last_became_active":"2024-09-11T08:52:38.483603+0000","last_became_peered":"2024-09-11T08:52:38.483603+0000","last_unstale":"2024-09-11T08:55:03.226372+0000","last_undegraded":"2024-09-11T08:55:03.226372+0000","last_fullsized":"2024-09-11T08:55:03.226372+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:53:58.534079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:58.534079+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:24.012675+0000","scrub_duration":0.0070998140000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":353,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.232798+0000","last_change":"2024-09-11T08:54:02.804185+0000","last_active":"2024-09-11T08:55:03.232798+0000","last_peered":"2024-09-11T08:55:03.232798+0000","last_clean":"2024-09-11T08:55:03.232798+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:55:03.232798+0000","last_undegraded":"2024-09-11T08:55:03.232798+0000","last_fullsized":"2024-09-11T08:55:03.232798+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:02.804091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:54:02.804091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:22.145193+0000","scrub_duration":0.0066693589999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"280'25","reported_seq":92,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:09.606836+0000","last_change":"2024-09-11T08:55:09.606768+0000","last_active":"2024-09-11T08:55:09.606836+0000","last_peered":"2024-09-11T08:55:09.606836+0000","last_clean":"2024-09-11T08:55:09.606836+0000","last_became_active":"2024-09-11T08:52:49.568074+0000","last_became_peered":"2024-09-11T08:52:49.568074+0000","last_unstale":"2024-09-11T08:55:09.606836+0000","last_undegraded":"2024-09-11T08:55:09.606836+0000","last_fullsized":"2024-09-11T08:55:09.606836+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"280'25","last_scrub_stamp":"2024-09-11T08:55:09.606687+0000","last_deep_scrub":"280'25","last_deep_scrub_stamp":"2024-09-11T08:52:24.584533+0000","last_clean_scrub_stamp":"2024-09-11T08:55:09.606687+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:33.333005+0000","scrub_duration":0.005718791,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":568,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:12.861683+0000","last_change":"2024-09-11T08:53:59.582420+0000","last_active":"2024-09-11T08:55:12.861683+0000","last_peered":"2024-09-11T08:55:12.861683+0000","last_clean":"2024-09-11T08:55:12.861683+0000","last_became_active":"2024-09-11T08:38:42.985578+0000","last_became_peered":"2024-09-11T08:38:42.985578+0000","last_unstale":"2024-09-11T08:55:12.861683+0000","last_undegraded":"2024-09-11T08:55:12.861683+0000","last_fullsized":"2024-09-11T08:55:12.861683+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:53:59.582332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:59.582332+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076609360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"331'11602","reported_seq":11648,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-09-11T08:55:16.265356+0000","last_change":"2024-09-11T08:52:56.415129+0000","last_active":"2024-09-11T08:55:16.265356+0000","last_peered":"2024-09-11T08:55:16.265356+0000","last_clean":"2024-09-11T08:55:16.265356+0000","last_became_active":"2024-09-11T08:52:56.414844+0000","last_became_peered":"2024-09-11T08:52:56.414844+0000","last_unstale":"2024-09-11T08:55:16.265356+0000","last_undegraded":"2024-09-11T08:55:16.265356+0000","last_fullsized":"2024-09-11T08:55:16.265356+0000","mapping_epoch":293,"log_start":"318'7200","ondisk_log_start":"318'7200","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4402,"log_dups_size":0,"ondisk_log_size":4402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":355860480,"num_objects":5430,"num_object_clones":0,"num_object_copies":16290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11602,"num_write_kb":545024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"199'81","reported_seq":100,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.226213+0000","last_change":"2024-09-11T08:53:16.826016+0000","last_active":"2024-09-11T08:55:03.226213+0000","last_peered":"2024-09-11T08:55:03.226213+0000","last_clean":"2024-09-11T08:55:03.226213+0000","last_became_active":"2024-09-11T08:51:03.023615+0000","last_became_peered":"2024-09-11T08:51:03.023615+0000","last_unstale":"2024-09-11T08:55:03.226213+0000","last_undegraded":"2024-09-11T08:55:03.226213+0000","last_fullsized":"2024-09-11T08:55:03.226213+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:16.825859+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:16.825859+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0087576560000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"200'34","reported_seq":401,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.232839+0000","last_change":"2024-09-11T08:55:03.157598+0000","last_active":"2024-09-11T08:55:03.232839+0000","last_peered":"2024-09-11T08:55:03.232839+0000","last_clean":"2024-09-11T08:55:03.232839+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:55:03.232839+0000","last_undegraded":"2024-09-11T08:55:03.232839+0000","last_fullsized":"2024-09-11T08:55:03.232839+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:55:03.157511+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:55:03.157511+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:07.933208+0000","scrub_duration":0.0047993619999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":333,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.245794+0000","last_change":"2024-09-11T08:53:02.487714+0000","last_active":"2024-09-11T08:55:03.245794+0000","last_peered":"2024-09-11T08:55:03.245794+0000","last_clean":"2024-09-11T08:55:03.245794+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:55:03.245794+0000","last_undegraded":"2024-09-11T08:55:03.245794+0000","last_fullsized":"2024-09-11T08:55:03.245794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:02.487657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:02.487657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011084630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"277'11","reported_seq":566,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:12.864108+0000","last_change":"2024-09-11T08:52:56.987065+0000","last_active":"2024-09-11T08:55:12.864108+0000","last_peered":"2024-09-11T08:55:12.864108+0000","last_clean":"2024-09-11T08:55:12.864108+0000","last_became_active":"2024-09-11T08:52:18.578328+0000","last_became_peered":"2024-09-11T08:52:18.578328+0000","last_unstale":"2024-09-11T08:55:12.864108+0000","last_undegraded":"2024-09-11T08:55:12.864108+0000","last_fullsized":"2024-09-11T08:55:12.864108+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'11","last_scrub_stamp":"2024-09-11T08:52:56.987010+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:52:56.987010+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033882560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":360,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:04.263347+0000","last_change":"2024-09-11T08:55:04.263289+0000","last_active":"2024-09-11T08:55:04.263347+0000","last_peered":"2024-09-11T08:55:04.263347+0000","last_clean":"2024-09-11T08:55:04.263347+0000","last_became_active":"2024-09-11T08:52:18.576898+0000","last_became_peered":"2024-09-11T08:52:18.576898+0000","last_unstale":"2024-09-11T08:55:04.263347+0000","last_undegraded":"2024-09-11T08:55:04.263347+0000","last_fullsized":"2024-09-11T08:55:04.263347+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:04.263219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:55:04.263219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:22.403612+0000","scrub_duration":0.0034071710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"331'11447","reported_seq":11493,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-09-11T08:55:16.252093+0000","last_change":"2024-09-11T08:52:55.610689+0000","last_active":"2024-09-11T08:55:16.252093+0000","last_peered":"2024-09-11T08:55:16.252093+0000","last_clean":"2024-09-11T08:55:16.252093+0000","last_became_active":"2024-09-11T08:52:55.610383+0000","last_became_peered":"2024-09-11T08:52:55.610383+0000","last_unstale":"2024-09-11T08:55:16.252093+0000","last_undegraded":"2024-09-11T08:55:16.252093+0000","last_fullsized":"2024-09-11T08:55:16.252093+0000","mapping_epoch":293,"log_start":"318'7000","ondisk_log_start":"318'7000","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4447,"log_dups_size":0,"ondisk_log_size":4447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":367984640,"num_objects":5615,"num_object_clones":0,"num_object_copies":16845,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11447,"num_write_kb":545984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"199'81","reported_seq":100,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.225821+0000","last_change":"2024-09-11T08:53:00.953735+0000","last_active":"2024-09-11T08:55:03.225821+0000","last_peered":"2024-09-11T08:55:03.225821+0000","last_clean":"2024-09-11T08:55:03.225821+0000","last_became_active":"2024-09-11T08:51:03.023365+0000","last_became_peered":"2024-09-11T08:51:03.023365+0000","last_unstale":"2024-09-11T08:55:03.225821+0000","last_undegraded":"2024-09-11T08:55:03.225821+0000","last_fullsized":"2024-09-11T08:55:03.225821+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:00.953685+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:00.953685+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001183323,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"200'38","reported_seq":455,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:06.625715+0000","last_change":"2024-09-11T08:55:06.625605+0000","last_active":"2024-09-11T08:55:06.625715+0000","last_peered":"2024-09-11T08:55:06.625715+0000","last_clean":"2024-09-11T08:55:06.625715+0000","last_became_active":"2024-09-11T08:52:18.595013+0000","last_became_peered":"2024-09-11T08:52:18.595013+0000","last_unstale":"2024-09-11T08:55:06.625715+0000","last_undegraded":"2024-09-11T08:55:06.625715+0000","last_fullsized":"2024-09-11T08:55:06.625715+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'38","last_scrub_stamp":"2024-09-11T08:55:06.625504+0000","last_deep_scrub":"199'26","last_deep_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_clean_scrub_stamp":"2024-09-11T08:55:06.625504+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:17.015205+0000","scrub_duration":0.010430116999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":366,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:04.639016+0000","last_change":"2024-09-11T08:55:04.638886+0000","last_active":"2024-09-11T08:55:04.639016+0000","last_peered":"2024-09-11T08:55:04.639016+0000","last_clean":"2024-09-11T08:55:04.639016+0000","last_became_active":"2024-09-11T08:52:18.586084+0000","last_became_peered":"2024-09-11T08:52:18.586084+0000","last_unstale":"2024-09-11T08:55:04.639016+0000","last_undegraded":"2024-09-11T08:55:04.639016+0000","last_fullsized":"2024-09-11T08:55:04.639016+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:04.638693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:55:04.638693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:29.409516+0000","scrub_duration":0.0070800059999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":330,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223353+0000","last_change":"2024-09-11T08:52:59.933645+0000","last_active":"2024-09-11T08:55:03.223353+0000","last_peered":"2024-09-11T08:55:03.223353+0000","last_clean":"2024-09-11T08:55:03.223353+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:55:03.223353+0000","last_undegraded":"2024-09-11T08:55:03.223353+0000","last_fullsized":"2024-09-11T08:55:03.223353+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:59.933599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:29.687577+0000","last_clean_scrub_stamp":"2024-09-11T08:52:59.933599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001022233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":328,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.224306+0000","last_change":"2024-09-11T08:54:00.820805+0000","last_active":"2024-09-11T08:55:03.224306+0000","last_peered":"2024-09-11T08:55:03.224306+0000","last_clean":"2024-09-11T08:55:03.224306+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:55:03.224306+0000","last_undegraded":"2024-09-11T08:55:03.224306+0000","last_fullsized":"2024-09-11T08:55:03.224306+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:00.820729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:00.820729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0084568009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"331'11702","reported_seq":12039,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-09-11T08:55:16.408839+0000","last_change":"2024-09-11T08:54:56.145000+0000","last_active":"2024-09-11T08:55:16.408839+0000","last_peered":"2024-09-11T08:55:16.408839+0000","last_clean":"2024-09-11T08:55:16.408839+0000","last_became_active":"2024-09-11T08:52:55.612351+0000","last_became_peered":"2024-09-11T08:52:55.612351+0000","last_unstale":"2024-09-11T08:55:16.408839+0000","last_undegraded":"2024-09-11T08:55:16.408839+0000","last_fullsized":"2024-09-11T08:55:16.408839+0000","mapping_epoch":293,"log_start":"316'6200","ondisk_log_start":"316'6200","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"329'10355","last_scrub_stamp":"2024-09-11T08:54:56.144911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:56.144911+0000","objects_scrubbed":7055,"log_size":5502,"log_dups_size":0,"ondisk_log_size":5502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:17.293392+0000","scrub_duration":3.6035077170000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364380160,"num_objects":5560,"num_object_clones":0,"num_object_copies":16680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11702,"num_write_kb":552384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"199'81","reported_seq":105,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.246028+0000","last_change":"2024-09-11T08:53:08.418382+0000","last_active":"2024-09-11T08:55:03.246028+0000","last_peered":"2024-09-11T08:55:03.246028+0000","last_clean":"2024-09-11T08:55:03.246028+0000","last_became_active":"2024-09-11T08:51:03.009983+0000","last_became_peered":"2024-09-11T08:51:03.009983+0000","last_unstale":"2024-09-11T08:55:03.246028+0000","last_undegraded":"2024-09-11T08:55:03.246028+0000","last_fullsized":"2024-09-11T08:55:03.246028+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:08.418288+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:08.418288+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003188832,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"200'35","reported_seq":374,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223582+0000","last_change":"2024-09-11T08:52:25.186590+0000","last_active":"2024-09-11T08:55:03.223582+0000","last_peered":"2024-09-11T08:55:03.223582+0000","last_clean":"2024-09-11T08:55:03.223582+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:55:03.223582+0000","last_undegraded":"2024-09-11T08:55:03.223582+0000","last_fullsized":"2024-09-11T08:55:03.223582+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'35","last_scrub_stamp":"2024-09-11T08:52:25.186528+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:52:25.186528+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018614930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":83,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.234957+0000","last_change":"2024-09-11T08:54:04.611817+0000","last_active":"2024-09-11T08:55:03.234957+0000","last_peered":"2024-09-11T08:55:03.234957+0000","last_clean":"2024-09-11T08:55:03.234957+0000","last_became_active":"2024-09-11T08:52:18.586549+0000","last_became_peered":"2024-09-11T08:52:18.586549+0000","last_unstale":"2024-09-11T08:55:03.234957+0000","last_undegraded":"2024-09-11T08:55:03.234957+0000","last_fullsized":"2024-09-11T08:55:03.234957+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:04.611505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:28.487573+0000","last_clean_scrub_stamp":"2024-09-11T08:54:04.611505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:30.421697+0000","scrub_duration":0.0061628730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":539,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:12.861583+0000","last_change":"2024-09-11T08:54:17.570568+0000","last_active":"2024-09-11T08:55:12.861583+0000","last_peered":"2024-09-11T08:55:12.861583+0000","last_clean":"2024-09-11T08:55:12.861583+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:55:12.861583+0000","last_undegraded":"2024-09-11T08:55:12.861583+0000","last_fullsized":"2024-09-11T08:55:12.861583+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:54:17.570479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:17.570479+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:18.330338+0000","scrub_duration":0.0077913949999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":241,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.226207+0000","last_change":"2024-09-11T08:54:04.676970+0000","last_active":"2024-09-11T08:55:03.226207+0000","last_peered":"2024-09-11T08:55:03.226207+0000","last_clean":"2024-09-11T08:55:03.226207+0000","last_became_active":"2024-09-11T08:52:38.488567+0000","last_became_peered":"2024-09-11T08:52:38.488567+0000","last_unstale":"2024-09-11T08:55:03.226207+0000","last_undegraded":"2024-09-11T08:55:03.226207+0000","last_fullsized":"2024-09-11T08:55:03.226207+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:04.676760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:04.676760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:26.906059+0000","scrub_duration":0.0044944620000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"331'11643","reported_seq":11689,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-09-11T08:55:16.294167+0000","last_change":"2024-09-11T08:52:56.414997+0000","last_active":"2024-09-11T08:55:16.294167+0000","last_peered":"2024-09-11T08:55:16.294167+0000","last_clean":"2024-09-11T08:55:16.294167+0000","last_became_active":"2024-09-11T08:52:56.414726+0000","last_became_peered":"2024-09-11T08:52:56.414726+0000","last_unstale":"2024-09-11T08:55:16.294167+0000","last_undegraded":"2024-09-11T08:55:16.294167+0000","last_fullsized":"2024-09-11T08:55:16.294167+0000","mapping_epoch":293,"log_start":"318'7200","ondisk_log_start":"318'7200","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4443,"log_dups_size":0,"ondisk_log_size":4443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":361299968,"num_objects":5513,"num_object_clones":0,"num_object_copies":16539,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11643,"num_write_kb":548992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"199'81","reported_seq":361,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:13.378184+0000","last_change":"2024-09-11T08:55:13.378021+0000","last_active":"2024-09-11T08:55:13.378184+0000","last_peered":"2024-09-11T08:55:13.378184+0000","last_clean":"2024-09-11T08:55:13.378184+0000","last_became_active":"2024-09-11T08:51:03.003800+0000","last_became_peered":"2024-09-11T08:51:03.003800+0000","last_unstale":"2024-09-11T08:55:13.378184+0000","last_undegraded":"2024-09-11T08:55:13.378184+0000","last_fullsized":"2024-09-11T08:55:13.378184+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:55:13.377713+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:55:13.377713+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:31.774678+0000","scrub_duration":0.0022056850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"200'25","reported_seq":386,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.237115+0000","last_change":"2024-09-11T08:54:05.976193+0000","last_active":"2024-09-11T08:55:03.237115+0000","last_peered":"2024-09-11T08:55:03.237115+0000","last_clean":"2024-09-11T08:55:03.237115+0000","last_became_active":"2024-09-11T08:38:40.976781+0000","last_became_peered":"2024-09-11T08:38:40.976781+0000","last_unstale":"2024-09-11T08:55:03.237115+0000","last_undegraded":"2024-09-11T08:55:03.237115+0000","last_fullsized":"2024-09-11T08:55:03.237115+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:54:05.976089+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_clean_scrub_stamp":"2024-09-11T08:54:05.976089+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:34.076324+0000","scrub_duration":0.20122382899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":339,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.226907+0000","last_change":"2024-09-11T08:52:55.701978+0000","last_active":"2024-09-11T08:55:03.226907+0000","last_peered":"2024-09-11T08:55:03.226907+0000","last_clean":"2024-09-11T08:55:03.226907+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:55:03.226907+0000","last_undegraded":"2024-09-11T08:55:03.226907+0000","last_fullsized":"2024-09-11T08:55:03.226907+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:55.701902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:55.701902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016250119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"292'11","reported_seq":247,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:12.861414+0000","last_change":"2024-09-11T08:54:56.468209+0000","last_active":"2024-09-11T08:55:12.861414+0000","last_peered":"2024-09-11T08:55:12.861414+0000","last_clean":"2024-09-11T08:55:12.861414+0000","last_became_active":"2024-09-11T08:52:49.564770+0000","last_became_peered":"2024-09-11T08:52:49.564770+0000","last_unstale":"2024-09-11T08:55:12.861414+0000","last_undegraded":"2024-09-11T08:55:12.861414+0000","last_fullsized":"2024-09-11T08:55:12.861414+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"292'11","last_scrub_stamp":"2024-09-11T08:54:56.468128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:56.468128+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:09.904664+0000","scrub_duration":0.0049519330000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":149,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.244402+0000","last_change":"2024-09-11T08:54:11.523221+0000","last_active":"2024-09-11T08:55:03.244402+0000","last_peered":"2024-09-11T08:55:03.244402+0000","last_clean":"2024-09-11T08:55:03.244402+0000","last_became_active":"2024-09-11T08:52:49.571683+0000","last_became_peered":"2024-09-11T08:52:49.571683+0000","last_unstale":"2024-09-11T08:55:03.244402+0000","last_undegraded":"2024-09-11T08:55:03.244402+0000","last_fullsized":"2024-09-11T08:55:03.244402+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:11.523143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:54:11.523143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:14.335044+0000","scrub_duration":0.010580648999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.d","version":"331'11457","reported_seq":11790,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-09-11T08:55:16.379522+0000","last_change":"2024-09-11T08:54:13.634665+0000","last_active":"2024-09-11T08:55:16.379522+0000","last_peered":"2024-09-11T08:55:16.379522+0000","last_clean":"2024-09-11T08:55:16.379522+0000","last_became_active":"2024-09-11T08:52:56.414830+0000","last_became_peered":"2024-09-11T08:52:56.414830+0000","last_unstale":"2024-09-11T08:55:16.379522+0000","last_undegraded":"2024-09-11T08:55:16.379522+0000","last_fullsized":"2024-09-11T08:55:16.379522+0000","mapping_epoch":293,"log_start":"316'6000","ondisk_log_start":"316'6000","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'7046","last_scrub_stamp":"2024-09-11T08:54:13.634607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:13.634607+0000","objects_scrubbed":6942,"log_size":5457,"log_dups_size":0,"ondisk_log_size":5457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:39.511951+0000","scrub_duration":3.0316847519999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":367001624,"num_objects":5601,"num_object_clones":0,"num_object_copies":16803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5601,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11457,"num_write_kb":545793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"199'81","reported_seq":361,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.228600+0000","last_change":"2024-09-11T08:53:21.460030+0000","last_active":"2024-09-11T08:55:03.228600+0000","last_peered":"2024-09-11T08:55:03.228600+0000","last_clean":"2024-09-11T08:55:03.228600+0000","last_became_active":"2024-09-11T08:52:18.584896+0000","last_became_peered":"2024-09-11T08:52:18.584896+0000","last_unstale":"2024-09-11T08:55:03.228600+0000","last_undegraded":"2024-09-11T08:55:03.228600+0000","last_fullsized":"2024-09-11T08:55:03.228600+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:21.459982+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:53:21.459982+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023340290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"200'29","reported_seq":397,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.226626+0000","last_change":"2024-09-11T08:53:57.508836+0000","last_active":"2024-09-11T08:55:03.226626+0000","last_peered":"2024-09-11T08:55:03.226626+0000","last_clean":"2024-09-11T08:55:03.226626+0000","last_became_active":"2024-09-11T08:52:49.563364+0000","last_became_peered":"2024-09-11T08:52:49.563364+0000","last_unstale":"2024-09-11T08:55:03.226626+0000","last_undegraded":"2024-09-11T08:55:03.226626+0000","last_fullsized":"2024-09-11T08:55:03.226626+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'29","last_scrub_stamp":"2024-09-11T08:53:57.508749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:57.508749+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044953129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":363,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.233961+0000","last_change":"2024-09-11T08:54:53.192428+0000","last_active":"2024-09-11T08:55:03.233961+0000","last_peered":"2024-09-11T08:55:03.233961+0000","last_clean":"2024-09-11T08:55:03.233961+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:55:03.233961+0000","last_undegraded":"2024-09-11T08:55:03.233961+0000","last_fullsized":"2024-09-11T08:55:03.233961+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:53.192352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:54:53.192352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:18.348281+0000","scrub_duration":0.0058229110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":388,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.225873+0000","last_change":"2024-09-11T08:54:59.511967+0000","last_active":"2024-09-11T08:55:03.225873+0000","last_peered":"2024-09-11T08:55:03.225873+0000","last_clean":"2024-09-11T08:55:03.225873+0000","last_became_active":"2024-09-11T08:52:18.573235+0000","last_became_peered":"2024-09-11T08:52:18.573235+0000","last_unstale":"2024-09-11T08:55:03.225873+0000","last_undegraded":"2024-09-11T08:55:03.225873+0000","last_fullsized":"2024-09-11T08:55:03.225873+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:59.511833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_clean_scrub_stamp":"2024-09-11T08:54:59.511833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:02.647936+0000","scrub_duration":0.012704131,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":377,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.232587+0000","last_change":"2024-09-11T08:54:59.195553+0000","last_active":"2024-09-11T08:55:03.232587+0000","last_peered":"2024-09-11T08:55:03.232587+0000","last_clean":"2024-09-11T08:55:03.232587+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:55:03.232587+0000","last_undegraded":"2024-09-11T08:55:03.232587+0000","last_fullsized":"2024-09-11T08:55:03.232587+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:59.195468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:54:59.195468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:15.660068+0000","scrub_duration":0.0016653340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"331'11529","reported_seq":11847,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-09-11T08:55:16.390733+0000","last_change":"2024-09-11T08:54:09.063052+0000","last_active":"2024-09-11T08:55:16.390733+0000","last_peered":"2024-09-11T08:55:16.390733+0000","last_clean":"2024-09-11T08:55:16.390733+0000","last_became_active":"2024-09-11T08:52:56.414809+0000","last_became_peered":"2024-09-11T08:52:56.414809+0000","last_unstale":"2024-09-11T08:55:16.390733+0000","last_undegraded":"2024-09-11T08:55:16.390733+0000","last_fullsized":"2024-09-11T08:55:16.390733+0000","mapping_epoch":293,"log_start":"316'6000","ondisk_log_start":"316'6000","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'6646","last_scrub_stamp":"2024-09-11T08:54:09.062953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.062953+0000","objects_scrubbed":6570,"log_size":5529,"log_dups_size":0,"ondisk_log_size":5529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:32.019378+0000","scrub_duration":2.4302389890000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359071744,"num_objects":5479,"num_object_clones":0,"num_object_copies":16437,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11529,"num_write_kb":544256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"199'81","reported_seq":110,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.225637+0000","last_change":"2024-09-11T08:54:09.431646+0000","last_active":"2024-09-11T08:55:03.225637+0000","last_peered":"2024-09-11T08:55:03.225637+0000","last_clean":"2024-09-11T08:55:03.225637+0000","last_became_active":"2024-09-11T08:52:18.574468+0000","last_became_peered":"2024-09-11T08:52:18.574468+0000","last_unstale":"2024-09-11T08:55:03.225637+0000","last_undegraded":"2024-09-11T08:55:03.225637+0000","last_fullsized":"2024-09-11T08:55:03.225637+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:54:09.431556+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.431556+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:33.839540+0000","scrub_duration":0.0066392939999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"200'50","reported_seq":636,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:12.861406+0000","last_change":"2024-09-11T08:55:00.215955+0000","last_active":"2024-09-11T08:55:12.861406+0000","last_peered":"2024-09-11T08:55:12.861406+0000","last_clean":"2024-09-11T08:55:12.861406+0000","last_became_active":"2024-09-11T08:38:40.979367+0000","last_became_peered":"2024-09-11T08:38:40.979367+0000","last_unstale":"2024-09-11T08:55:12.861406+0000","last_undegraded":"2024-09-11T08:55:12.861406+0000","last_fullsized":"2024-09-11T08:55:12.861406+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'50","last_scrub_stamp":"2024-09-11T08:55:00.215860+0000","last_deep_scrub":"200'50","last_deep_scrub_stamp":"2024-09-11T08:53:09.745322+0000","last_clean_scrub_stamp":"2024-09-11T08:55:00.215860+0000","objects_scrubbed":10,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:06.023884+0000","scrub_duration":0.0052089689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":364,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223402+0000","last_change":"2024-09-11T08:54:56.719900+0000","last_active":"2024-09-11T08:55:03.223402+0000","last_peered":"2024-09-11T08:55:03.223402+0000","last_clean":"2024-09-11T08:55:03.223402+0000","last_became_active":"2024-09-11T08:52:18.579509+0000","last_became_peered":"2024-09-11T08:52:18.579509+0000","last_unstale":"2024-09-11T08:55:03.223402+0000","last_undegraded":"2024-09-11T08:55:03.223402+0000","last_fullsized":"2024-09-11T08:55:03.223402+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:56.719827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_clean_scrub_stamp":"2024-09-11T08:54:56.719827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:02.575226+0000","scrub_duration":0.003589297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":384,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.226909+0000","last_change":"2024-09-11T08:54:14.424233+0000","last_active":"2024-09-11T08:55:03.226909+0000","last_peered":"2024-09-11T08:55:03.226909+0000","last_clean":"2024-09-11T08:55:03.226909+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:55:03.226909+0000","last_undegraded":"2024-09-11T08:55:03.226909+0000","last_fullsized":"2024-09-11T08:55:03.226909+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:54:14.423685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:54:14.423685+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:25.489159+0000","scrub_duration":0.012346487999999999,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":159,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:08.121883+0000","last_change":"2024-09-11T08:55:08.121800+0000","last_active":"2024-09-11T08:55:08.121883+0000","last_peered":"2024-09-11T08:55:08.121883+0000","last_clean":"2024-09-11T08:55:08.121883+0000","last_became_active":"2024-09-11T08:52:49.569137+0000","last_became_peered":"2024-09-11T08:52:49.569137+0000","last_unstale":"2024-09-11T08:55:08.121883+0000","last_undegraded":"2024-09-11T08:55:08.121883+0000","last_fullsized":"2024-09-11T08:55:08.121883+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:08.121720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:55:08.121720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:34.559575+0000","scrub_duration":0.0038461900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":327,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.245078+0000","last_change":"2024-09-11T08:53:11.420230+0000","last_active":"2024-09-11T08:55:03.245078+0000","last_peered":"2024-09-11T08:55:03.245078+0000","last_clean":"2024-09-11T08:55:03.245078+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:55:03.245078+0000","last_undegraded":"2024-09-11T08:55:03.245078+0000","last_fullsized":"2024-09-11T08:55:03.245078+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:11.420182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:11.420182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034052280000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"331'11678","reported_seq":11724,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-09-11T08:55:16.674001+0000","last_change":"2024-09-11T08:52:56.413844+0000","last_active":"2024-09-11T08:55:16.674001+0000","last_peered":"2024-09-11T08:55:16.674001+0000","last_clean":"2024-09-11T08:55:16.674001+0000","last_became_active":"2024-09-11T08:52:56.413594+0000","last_became_peered":"2024-09-11T08:52:56.413594+0000","last_unstale":"2024-09-11T08:55:16.674001+0000","last_undegraded":"2024-09-11T08:55:16.674001+0000","last_fullsized":"2024-09-11T08:55:16.674001+0000","mapping_epoch":293,"log_start":"316'6500","ondisk_log_start":"316'6500","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":5178,"log_dups_size":0,"ondisk_log_size":5178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363200512,"num_objects":5542,"num_object_clones":0,"num_object_copies":16626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11678,"num_write_kb":551040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"199'81","reported_seq":389,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223989+0000","last_change":"2024-09-11T08:55:00.708661+0000","last_active":"2024-09-11T08:55:03.223989+0000","last_peered":"2024-09-11T08:55:03.223989+0000","last_clean":"2024-09-11T08:55:03.223989+0000","last_became_active":"2024-09-11T08:52:49.569934+0000","last_became_peered":"2024-09-11T08:52:49.569934+0000","last_unstale":"2024-09-11T08:55:03.223989+0000","last_undegraded":"2024-09-11T08:55:03.223989+0000","last_fullsized":"2024-09-11T08:55:03.223989+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:55:00.708581+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:55:00.708581+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:01.937356+0000","scrub_duration":0.0053390399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"200'17","reported_seq":428,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.232922+0000","last_change":"2024-09-11T08:53:50.749880+0000","last_active":"2024-09-11T08:55:03.232922+0000","last_peered":"2024-09-11T08:55:03.232922+0000","last_clean":"2024-09-11T08:55:03.232922+0000","last_became_active":"2024-09-11T08:52:18.580008+0000","last_became_peered":"2024-09-11T08:52:18.580008+0000","last_unstale":"2024-09-11T08:55:03.232922+0000","last_undegraded":"2024-09-11T08:55:03.232922+0000","last_fullsized":"2024-09-11T08:55:03.232922+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'17","last_scrub_stamp":"2024-09-11T08:53:50.749772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:50.749772+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:14.966848+0000","scrub_duration":0.0041242989999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":379,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.234584+0000","last_change":"2024-09-11T08:54:18.762957+0000","last_active":"2024-09-11T08:55:03.234584+0000","last_peered":"2024-09-11T08:55:03.234584+0000","last_clean":"2024-09-11T08:55:03.234584+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:55:03.234584+0000","last_undegraded":"2024-09-11T08:55:03.234584+0000","last_fullsized":"2024-09-11T08:55:03.234584+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:18.762880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:54:18.762880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:38.949705+0000","scrub_duration":0.001880451,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"263'2","reported_seq":424,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223186+0000","last_change":"2024-09-11T08:54:16.866098+0000","last_active":"2024-09-11T08:55:03.223186+0000","last_peered":"2024-09-11T08:55:03.223186+0000","last_clean":"2024-09-11T08:55:03.223186+0000","last_became_active":"2024-09-11T08:52:18.586545+0000","last_became_peered":"2024-09-11T08:52:18.586545+0000","last_unstale":"2024-09-11T08:55:03.223186+0000","last_undegraded":"2024-09-11T08:55:03.223186+0000","last_fullsized":"2024-09-11T08:55:03.223186+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'2","last_scrub_stamp":"2024-09-11T08:54:16.866040+0000","last_deep_scrub":"263'2","last_deep_scrub_stamp":"2024-09-11T08:53:08.772979+0000","last_clean_scrub_stamp":"2024-09-11T08:54:16.866040+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:37.122737+0000","scrub_duration":0.0014173129999999999,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":374,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223106+0000","last_change":"2024-09-11T08:53:57.574981+0000","last_active":"2024-09-11T08:55:03.223106+0000","last_peered":"2024-09-11T08:55:03.223106+0000","last_clean":"2024-09-11T08:55:03.223106+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:55:03.223106+0000","last_undegraded":"2024-09-11T08:55:03.223106+0000","last_fullsized":"2024-09-11T08:55:03.223106+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:57.574778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_clean_scrub_stamp":"2024-09-11T08:53:57.574778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050739540000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":231,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.227540+0000","last_change":"2024-09-11T08:53:57.539830+0000","last_active":"2024-09-11T08:55:03.227540+0000","last_peered":"2024-09-11T08:55:03.227540+0000","last_clean":"2024-09-11T08:55:03.227540+0000","last_became_active":"2024-09-11T08:52:38.484800+0000","last_became_peered":"2024-09-11T08:52:38.484800+0000","last_unstale":"2024-09-11T08:55:03.227540+0000","last_undegraded":"2024-09-11T08:55:03.227540+0000","last_fullsized":"2024-09-11T08:55:03.227540+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:57.539760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:57.539760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:15.603679+0000","scrub_duration":0.0039035570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"331'11602","reported_seq":11930,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-09-11T08:55:16.044084+0000","last_change":"2024-09-11T08:54:10.753182+0000","last_active":"2024-09-11T08:55:16.044084+0000","last_peered":"2024-09-11T08:55:16.044084+0000","last_clean":"2024-09-11T08:55:16.044084+0000","last_became_active":"2024-09-11T08:52:56.413725+0000","last_became_peered":"2024-09-11T08:52:56.413725+0000","last_unstale":"2024-09-11T08:55:16.044084+0000","last_undegraded":"2024-09-11T08:55:16.044084+0000","last_fullsized":"2024-09-11T08:55:16.044084+0000","mapping_epoch":293,"log_start":"313'5600","ondisk_log_start":"313'5600","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'6916","last_scrub_stamp":"2024-09-11T08:54:10.753112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:10.753112+0000","objects_scrubbed":6818,"log_size":6002,"log_dups_size":0,"ondisk_log_size":6002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:37.548086+0000","scrub_duration":3.0166241519999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":366870528,"num_objects":5598,"num_object_clones":0,"num_object_copies":16794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11602,"num_write_kb":550400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"199'81","reported_seq":373,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.233740+0000","last_change":"2024-09-11T08:54:54.193424+0000","last_active":"2024-09-11T08:55:03.233740+0000","last_peered":"2024-09-11T08:55:03.233740+0000","last_clean":"2024-09-11T08:55:03.233740+0000","last_became_active":"2024-09-11T08:51:03.027281+0000","last_became_peered":"2024-09-11T08:51:03.027281+0000","last_unstale":"2024-09-11T08:55:03.233740+0000","last_undegraded":"2024-09-11T08:55:03.233740+0000","last_fullsized":"2024-09-11T08:55:03.233740+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:54:54.193339+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:54:54.193339+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:10.321174+0000","scrub_duration":0.006055899,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"200'24","reported_seq":97,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.234623+0000","last_change":"2024-09-11T08:54:14.506978+0000","last_active":"2024-09-11T08:55:03.234623+0000","last_peered":"2024-09-11T08:55:03.234623+0000","last_clean":"2024-09-11T08:55:03.234623+0000","last_became_active":"2024-09-11T08:52:18.580950+0000","last_became_peered":"2024-09-11T08:52:18.580950+0000","last_unstale":"2024-09-11T08:55:03.234623+0000","last_undegraded":"2024-09-11T08:55:03.234623+0000","last_fullsized":"2024-09-11T08:55:03.234623+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'24","last_scrub_stamp":"2024-09-11T08:54:14.506895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:14.506895+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:33.854703+0000","scrub_duration":0.0076186200000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":149,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223418+0000","last_change":"2024-09-11T08:53:18.845981+0000","last_active":"2024-09-11T08:55:03.223418+0000","last_peered":"2024-09-11T08:55:03.223418+0000","last_clean":"2024-09-11T08:55:03.223418+0000","last_became_active":"2024-09-11T08:52:49.566437+0000","last_became_peered":"2024-09-11T08:52:49.566437+0000","last_unstale":"2024-09-11T08:55:03.223418+0000","last_undegraded":"2024-09-11T08:55:03.223418+0000","last_fullsized":"2024-09-11T08:55:03.223418+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:18.845896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:18.845896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053148020000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"318'6","reported_seq":263,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-09-11T08:55:14.926600+0000","last_change":"2024-09-11T08:54:12.873744+0000","last_active":"2024-09-11T08:55:14.926600+0000","last_peered":"2024-09-11T08:55:14.926600+0000","last_clean":"2024-09-11T08:55:14.926600+0000","last_became_active":"2024-09-11T08:52:38.484664+0000","last_became_peered":"2024-09-11T08:52:38.484664+0000","last_unstale":"2024-09-11T08:55:14.926600+0000","last_undegraded":"2024-09-11T08:55:14.926600+0000","last_fullsized":"2024-09-11T08:55:14.926600+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'6","last_scrub_stamp":"2024-09-11T08:54:12.873653+0000","last_deep_scrub":"318'6","last_deep_scrub_stamp":"2024-09-11T08:54:12.873653+0000","last_clean_scrub_stamp":"2024-09-11T08:54:12.873653+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:21.779837+0000","scrub_duration":0.01500046,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":365,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.227338+0000","last_change":"2024-09-11T08:54:16.367927+0000","last_active":"2024-09-11T08:55:03.227338+0000","last_peered":"2024-09-11T08:55:03.227338+0000","last_clean":"2024-09-11T08:55:03.227338+0000","last_became_active":"2024-09-11T08:52:18.578530+0000","last_became_peered":"2024-09-11T08:52:18.578530+0000","last_unstale":"2024-09-11T08:55:03.227338+0000","last_undegraded":"2024-09-11T08:55:03.227338+0000","last_fullsized":"2024-09-11T08:55:03.227338+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:16.367873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:16.367873+0000","last_clean_scrub_stamp":"2024-09-11T08:54:16.367873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:41.013346+0000","scrub_duration":0.0013677540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":357,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:13.179806+0000","last_change":"2024-09-11T08:55:13.179749+0000","last_active":"2024-09-11T08:55:13.179806+0000","last_peered":"2024-09-11T08:55:13.179806+0000","last_clean":"2024-09-11T08:55:13.179806+0000","last_became_active":"2024-09-11T08:38:44.993974+0000","last_became_peered":"2024-09-11T08:38:44.993974+0000","last_unstale":"2024-09-11T08:55:13.179806+0000","last_undegraded":"2024-09-11T08:55:13.179806+0000","last_fullsized":"2024-09-11T08:55:13.179806+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:13.179675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:55:13.179675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:29.701040+0000","scrub_duration":0.00222916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"331'11554","reported_seq":11893,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-09-11T08:55:16.398270+0000","last_change":"2024-09-11T08:54:15.231631+0000","last_active":"2024-09-11T08:55:16.398270+0000","last_peered":"2024-09-11T08:55:16.398270+0000","last_clean":"2024-09-11T08:55:16.398270+0000","last_became_active":"2024-09-11T08:52:56.413344+0000","last_became_peered":"2024-09-11T08:52:56.413344+0000","last_unstale":"2024-09-11T08:55:16.398270+0000","last_undegraded":"2024-09-11T08:55:16.398270+0000","last_fullsized":"2024-09-11T08:55:16.398270+0000","mapping_epoch":293,"log_start":"315'6000","ondisk_log_start":"315'6000","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'7183","last_scrub_stamp":"2024-09-11T08:54:15.231547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:15.231547+0000","objects_scrubbed":7103,"log_size":5554,"log_dups_size":0,"ondisk_log_size":5554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:26.393977+0000","scrub_duration":2.5900008689999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":361889792,"num_objects":5522,"num_object_clones":0,"num_object_copies":16566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11554,"num_write_kb":546432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"201'27","reported_seq":257,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.227628+0000","last_change":"2024-09-11T08:54:00.612065+0000","last_active":"2024-09-11T08:55:03.227628+0000","last_peered":"2024-09-11T08:55:03.227628+0000","last_clean":"2024-09-11T08:55:03.227628+0000","last_became_active":"2024-09-11T08:46:01.981242+0000","last_became_peered":"2024-09-11T08:46:01.981242+0000","last_unstale":"2024-09-11T08:55:03.227628+0000","last_undegraded":"2024-09-11T08:55:03.227628+0000","last_fullsized":"2024-09-11T08:55:03.227628+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'27","last_scrub_stamp":"2024-09-11T08:54:00.611977+0000","last_deep_scrub":"201'27","last_deep_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_clean_scrub_stamp":"2024-09-11T08:54:00.611977+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021148109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":342,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.244896+0000","last_change":"2024-09-11T08:53:51.352190+0000","last_active":"2024-09-11T08:55:03.244896+0000","last_peered":"2024-09-11T08:55:03.244896+0000","last_clean":"2024-09-11T08:55:03.244896+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:55:03.244896+0000","last_undegraded":"2024-09-11T08:55:03.244896+0000","last_fullsized":"2024-09-11T08:55:03.244896+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:53:51.352089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:51.352089+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032468750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"199'81","reported_seq":311,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:08.518825+0000","last_change":"2024-09-11T08:55:08.518744+0000","last_active":"2024-09-11T08:55:08.518825+0000","last_peered":"2024-09-11T08:55:08.518825+0000","last_clean":"2024-09-11T08:55:08.518825+0000","last_became_active":"2024-09-11T08:52:49.565126+0000","last_became_peered":"2024-09-11T08:52:49.565126+0000","last_unstale":"2024-09-11T08:55:08.518825+0000","last_undegraded":"2024-09-11T08:55:08.518825+0000","last_fullsized":"2024-09-11T08:55:08.518825+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:55:08.518651+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:55:08.518651+0000","last_clean_scrub_stamp":"2024-09-11T08:55:08.518651+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:19.491624+0000","scrub_duration":0.0055737219999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":76,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.222193+0000","last_change":"2024-09-11T08:55:02.763310+0000","last_active":"2024-09-11T08:55:03.222193+0000","last_peered":"2024-09-11T08:55:03.222193+0000","last_clean":"2024-09-11T08:55:03.222193+0000","last_became_active":"2024-09-11T08:52:38.485446+0000","last_became_peered":"2024-09-11T08:52:38.485446+0000","last_unstale":"2024-09-11T08:55:03.222193+0000","last_undegraded":"2024-09-11T08:55:03.222193+0000","last_fullsized":"2024-09-11T08:55:03.222193+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:02.763222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:55:02.763222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:28.264055+0000","scrub_duration":0.0040819920000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":343,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.227221+0000","last_change":"2024-09-11T08:53:54.582425+0000","last_active":"2024-09-11T08:55:03.227221+0000","last_peered":"2024-09-11T08:55:03.227221+0000","last_clean":"2024-09-11T08:55:03.227221+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:55:03.227221+0000","last_undegraded":"2024-09-11T08:55:03.227221+0000","last_fullsized":"2024-09-11T08:55:03.227221+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:54.582338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:54.582338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:18.198186+0000","scrub_duration":0.004605914,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"331'11751","reported_seq":12083,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-09-11T08:55:16.407482+0000","last_change":"2024-09-11T08:54:56.045835+0000","last_active":"2024-09-11T08:55:16.407482+0000","last_peered":"2024-09-11T08:55:16.407482+0000","last_clean":"2024-09-11T08:55:16.407482+0000","last_became_active":"2024-09-11T08:52:55.611835+0000","last_became_peered":"2024-09-11T08:52:55.611835+0000","last_unstale":"2024-09-11T08:55:16.407482+0000","last_undegraded":"2024-09-11T08:55:16.407482+0000","last_fullsized":"2024-09-11T08:55:16.407482+0000","mapping_epoch":293,"log_start":"316'6200","ondisk_log_start":"316'6200","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"329'10345","last_scrub_stamp":"2024-09-11T08:54:56.045754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:56.045754+0000","objects_scrubbed":6935,"log_size":5551,"log_dups_size":0,"ondisk_log_size":5551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:02.760808+0000","scrub_duration":2.5360603799999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356843520,"num_objects":5445,"num_object_clones":0,"num_object_copies":16335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11751,"num_write_kb":550272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"200'19","reported_seq":432,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.224627+0000","last_change":"2024-09-11T08:55:01.736021+0000","last_active":"2024-09-11T08:55:03.224627+0000","last_peered":"2024-09-11T08:55:03.224627+0000","last_clean":"2024-09-11T08:55:03.224627+0000","last_became_active":"2024-09-11T08:46:01.980873+0000","last_became_peered":"2024-09-11T08:46:01.980873+0000","last_unstale":"2024-09-11T08:55:03.224627+0000","last_undegraded":"2024-09-11T08:55:03.224627+0000","last_fullsized":"2024-09-11T08:55:03.224627+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:55:01.735933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:01.735933+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:20.462768+0000","scrub_duration":0.017616557000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":334,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:11.804708+0000","last_change":"2024-09-11T08:55:11.804648+0000","last_active":"2024-09-11T08:55:11.804708+0000","last_peered":"2024-09-11T08:55:11.804708+0000","last_clean":"2024-09-11T08:55:11.804708+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:55:11.804708+0000","last_undegraded":"2024-09-11T08:55:11.804708+0000","last_fullsized":"2024-09-11T08:55:11.804708+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:11.804570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:55:11.804570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:38.050044+0000","scrub_duration":0.010636980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":352,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:06.261997+0000","last_change":"2024-09-11T08:55:06.261924+0000","last_active":"2024-09-11T08:55:06.261997+0000","last_peered":"2024-09-11T08:55:06.261997+0000","last_clean":"2024-09-11T08:55:06.261997+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:55:06.261997+0000","last_undegraded":"2024-09-11T08:55:06.261997+0000","last_fullsized":"2024-09-11T08:55:06.261997+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:06.261834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:55:06.261834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:33.751138+0000","scrub_duration":0.0015153250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":347,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.246104+0000","last_change":"2024-09-11T08:53:15.469630+0000","last_active":"2024-09-11T08:55:03.246104+0000","last_peered":"2024-09-11T08:55:03.246104+0000","last_clean":"2024-09-11T08:55:03.246104+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:55:03.246104+0000","last_undegraded":"2024-09-11T08:55:03.246104+0000","last_fullsized":"2024-09-11T08:55:03.246104+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:15.469262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:15.469262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071803099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"331'13746","reported_seq":13893,"reported_epoch":331,"state":"active+clean+scrubbing+deep","last_fresh":"2024-09-11T08:55:16.663769+0000","last_change":"2024-09-11T08:55:05.293296+0000","last_active":"2024-09-11T08:55:16.663769+0000","last_peered":"2024-09-11T08:55:16.663769+0000","last_clean":"2024-09-11T08:55:16.663769+0000","last_became_active":"2024-09-11T08:54:17.120339+0000","last_became_peered":"2024-09-11T08:54:17.120339+0000","last_unstale":"2024-09-11T08:55:16.663769+0000","last_undegraded":"2024-09-11T08:55:16.663769+0000","last_fullsized":"2024-09-11T08:55:16.663769+0000","mapping_epoch":319,"log_start":"324'8475","ondisk_log_start":"324'8475","created":293,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":2124,"log_size":5271,"log_dups_size":0,"ondisk_log_size":5271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 11s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369950720,"num_objects":5645,"num_object_clones":0,"num_object_copies":16935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11647,"num_write_kb":553344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.8","version":"199'81","reported_seq":107,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.225549+0000","last_change":"2024-09-11T08:53:14.810007+0000","last_active":"2024-09-11T08:55:03.225549+0000","last_peered":"2024-09-11T08:55:03.225549+0000","last_clean":"2024-09-11T08:55:03.225549+0000","last_became_active":"2024-09-11T08:51:03.020941+0000","last_became_peered":"2024-09-11T08:51:03.020941+0000","last_unstale":"2024-09-11T08:55:03.225549+0000","last_undegraded":"2024-09-11T08:55:03.225549+0000","last_fullsized":"2024-09-11T08:55:03.225549+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:53:14.809894+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:53:14.809894+0000","last_clean_scrub_stamp":"2024-09-11T08:53:14.809894+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0065883419999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"317'39","reported_seq":399,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.225693+0000","last_change":"2024-09-11T08:54:06.928521+0000","last_active":"2024-09-11T08:55:03.225693+0000","last_peered":"2024-09-11T08:55:03.225693+0000","last_clean":"2024-09-11T08:55:03.225693+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:55:03.225693+0000","last_undegraded":"2024-09-11T08:55:03.225693+0000","last_fullsized":"2024-09-11T08:55:03.225693+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"317'39","last_scrub_stamp":"2024-09-11T08:54:06.928266+0000","last_deep_scrub":"317'39","last_deep_scrub_stamp":"2024-09-11T08:54:06.928266+0000","last_clean_scrub_stamp":"2024-09-11T08:54:06.928266+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:15.212159+0000","scrub_duration":0.016459872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":382,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:07.504203+0000","last_change":"2024-09-11T08:55:07.504114+0000","last_active":"2024-09-11T08:55:07.504203+0000","last_peered":"2024-09-11T08:55:07.504203+0000","last_clean":"2024-09-11T08:55:07.504203+0000","last_became_active":"2024-09-11T08:38:38.948858+0000","last_became_peered":"2024-09-11T08:38:38.948858+0000","last_unstale":"2024-09-11T08:55:07.504203+0000","last_undegraded":"2024-09-11T08:55:07.504203+0000","last_fullsized":"2024-09-11T08:55:07.504203+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:07.504031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:07.504031+0000","last_clean_scrub_stamp":"2024-09-11T08:55:07.504031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:25.435453+0000","scrub_duration":0.0037070129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":369,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.233624+0000","last_change":"2024-09-11T08:54:04.809120+0000","last_active":"2024-09-11T08:55:03.233624+0000","last_peered":"2024-09-11T08:55:03.233624+0000","last_clean":"2024-09-11T08:55:03.233624+0000","last_became_active":"2024-09-11T08:52:18.577201+0000","last_became_peered":"2024-09-11T08:52:18.577201+0000","last_unstale":"2024-09-11T08:55:03.233624+0000","last_undegraded":"2024-09-11T08:55:03.233624+0000","last_fullsized":"2024-09-11T08:55:03.233624+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:04.809032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:04.809032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:34.035088+0000","scrub_duration":0.005911347,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":24,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.249855+0000","last_change":"2024-09-11T08:54:17.121779+0000","last_active":"2024-09-11T08:55:03.249855+0000","last_peered":"2024-09-11T08:55:03.249855+0000","last_clean":"2024-09-11T08:55:03.249855+0000","last_became_active":"2024-09-11T08:54:17.121372+0000","last_became_peered":"2024-09-11T08:54:17.121372+0000","last_unstale":"2024-09-11T08:55:03.249855+0000","last_undegraded":"2024-09-11T08:55:03.249855+0000","last_fullsized":"2024-09-11T08:55:03.249855+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:07.808138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:07.808138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"331'11565","reported_seq":12086,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-09-11T08:55:16.827080+0000","last_change":"2024-09-11T08:54:54.973653+0000","last_active":"2024-09-11T08:55:16.827080+0000","last_peered":"2024-09-11T08:55:16.827080+0000","last_clean":"2024-09-11T08:55:16.827080+0000","last_became_active":"2024-09-11T08:52:56.414765+0000","last_became_peered":"2024-09-11T08:52:56.414765+0000","last_unstale":"2024-09-11T08:55:16.827080+0000","last_undegraded":"2024-09-11T08:55:16.827080+0000","last_fullsized":"2024-09-11T08:55:16.827080+0000","mapping_epoch":293,"log_start":"316'6236","ondisk_log_start":"316'6236","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'10137","last_scrub_stamp":"2024-09-11T08:54:54.973568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:54.973568+0000","objects_scrubbed":6983,"log_size":5329,"log_dups_size":0,"ondisk_log_size":5329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:24.494920+0000","scrub_duration":2.7437013509999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":357105664,"num_objects":5449,"num_object_clones":0,"num_object_copies":16347,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11565,"num_write_kb":544448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"199'81","reported_seq":393,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.244634+0000","last_change":"2024-09-11T08:54:51.543848+0000","last_active":"2024-09-11T08:55:03.244634+0000","last_peered":"2024-09-11T08:55:03.244634+0000","last_clean":"2024-09-11T08:55:03.244634+0000","last_became_active":"2024-09-11T08:52:38.481475+0000","last_became_peered":"2024-09-11T08:52:38.481475+0000","last_unstale":"2024-09-11T08:55:03.244634+0000","last_undegraded":"2024-09-11T08:55:03.244634+0000","last_fullsized":"2024-09-11T08:55:03.244634+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:54:51.543754+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:54:51.543754+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:11.363485+0000","scrub_duration":0.0034342589999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"200'20","reported_seq":402,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223615+0000","last_change":"2024-09-11T08:52:18.090235+0000","last_active":"2024-09-11T08:55:03.223615+0000","last_peered":"2024-09-11T08:55:03.223615+0000","last_clean":"2024-09-11T08:55:03.223615+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:55:03.223615+0000","last_undegraded":"2024-09-11T08:55:03.223615+0000","last_fullsized":"2024-09-11T08:55:03.223615+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'20","last_scrub_stamp":"2024-09-11T08:52:18.090140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:52:18.090140+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018601010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":353,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.245395+0000","last_change":"2024-09-11T08:53:19.389486+0000","last_active":"2024-09-11T08:55:03.245395+0000","last_peered":"2024-09-11T08:55:03.245395+0000","last_clean":"2024-09-11T08:55:03.245395+0000","last_became_active":"2024-09-11T08:52:18.586343+0000","last_became_peered":"2024-09-11T08:52:18.586343+0000","last_unstale":"2024-09-11T08:55:03.245395+0000","last_undegraded":"2024-09-11T08:55:03.245395+0000","last_fullsized":"2024-09-11T08:55:03.245395+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:19.389411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:19.389411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058663470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"299'9","reported_seq":589,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:12.861499+0000","last_change":"2024-09-11T08:55:05.776419+0000","last_active":"2024-09-11T08:55:12.861499+0000","last_peered":"2024-09-11T08:55:12.861499+0000","last_clean":"2024-09-11T08:55:12.861499+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:55:12.861499+0000","last_undegraded":"2024-09-11T08:55:12.861499+0000","last_fullsized":"2024-09-11T08:55:12.861499+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'9","last_scrub_stamp":"2024-09-11T08:55:05.776257+0000","last_deep_scrub":"299'9","last_deep_scrub_stamp":"2024-09-11T08:53:05.835177+0000","last_clean_scrub_stamp":"2024-09-11T08:55:05.776257+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:22.506912+0000","scrub_duration":0.0048283349999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":328,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.224523+0000","last_change":"2024-09-11T08:53:07.008184+0000","last_active":"2024-09-11T08:55:03.224523+0000","last_peered":"2024-09-11T08:55:03.224523+0000","last_clean":"2024-09-11T08:55:03.224523+0000","last_became_active":"2024-09-11T08:38:44.993479+0000","last_became_peered":"2024-09-11T08:38:44.993479+0000","last_unstale":"2024-09-11T08:55:03.224523+0000","last_undegraded":"2024-09-11T08:55:03.224523+0000","last_fullsized":"2024-09-11T08:55:03.224523+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:07.008088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:53:07.008088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041216330000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"331'11930","reported_seq":12253,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-09-11T08:55:17.156722+0000","last_change":"2024-09-11T08:54:17.137423+0000","last_active":"2024-09-11T08:55:17.156722+0000","last_peered":"2024-09-11T08:55:17.156722+0000","last_clean":"2024-09-11T08:55:17.156722+0000","last_became_active":"2024-09-11T08:54:17.121397+0000","last_became_peered":"2024-09-11T08:54:17.121397+0000","last_unstale":"2024-09-11T08:55:17.156722+0000","last_undegraded":"2024-09-11T08:55:17.156722+0000","last_fullsized":"2024-09-11T08:55:17.156722+0000","mapping_epoch":319,"log_start":"318'6803","ondisk_log_start":"318'6803","created":293,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"317'6616","last_scrub_stamp":"2024-09-11T08:54:04.991011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:04.991011+0000","objects_scrubbed":6499,"log_size":5127,"log_dups_size":0,"ondisk_log_size":5127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:21.176143+0000","scrub_duration":3.4878316329999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":367001600,"num_objects":5600,"num_object_clones":0,"num_object_copies":16800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11930,"num_write_kb":560960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"200'15","reported_seq":93,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.229902+0000","last_change":"2024-09-11T08:53:52.645375+0000","last_active":"2024-09-11T08:55:03.229902+0000","last_peered":"2024-09-11T08:55:03.229902+0000","last_clean":"2024-09-11T08:55:03.229902+0000","last_became_active":"2024-09-11T08:52:18.588329+0000","last_became_peered":"2024-09-11T08:52:18.588329+0000","last_unstale":"2024-09-11T08:55:03.229902+0000","last_undegraded":"2024-09-11T08:55:03.229902+0000","last_fullsized":"2024-09-11T08:55:03.229902+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:53:52.645266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:52.645266+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:14.532828+0000","scrub_duration":0.00332431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":346,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.227050+0000","last_change":"2024-09-11T08:53:13.771378+0000","last_active":"2024-09-11T08:55:03.227050+0000","last_peered":"2024-09-11T08:55:03.227050+0000","last_clean":"2024-09-11T08:55:03.227050+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:55:03.227050+0000","last_undegraded":"2024-09-11T08:55:03.227050+0000","last_fullsized":"2024-09-11T08:55:03.227050+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:13.771300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:53:13.771300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068129849999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":24,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.224153+0000","last_change":"2024-09-11T08:54:17.120062+0000","last_active":"2024-09-11T08:55:03.224153+0000","last_peered":"2024-09-11T08:55:03.224153+0000","last_clean":"2024-09-11T08:55:03.224153+0000","last_became_active":"2024-09-11T08:54:17.119864+0000","last_became_peered":"2024-09-11T08:54:17.119864+0000","last_unstale":"2024-09-11T08:55:03.224153+0000","last_undegraded":"2024-09-11T08:55:03.224153+0000","last_fullsized":"2024-09-11T08:55:03.224153+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:06.683645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:06.683645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:17.562942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":76,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:13.770200+0000","last_change":"2024-09-11T08:55:13.770134+0000","last_active":"2024-09-11T08:55:13.770200+0000","last_peered":"2024-09-11T08:55:13.770200+0000","last_clean":"2024-09-11T08:55:13.770200+0000","last_became_active":"2024-09-11T08:52:49.569749+0000","last_became_peered":"2024-09-11T08:52:49.569749+0000","last_unstale":"2024-09-11T08:55:13.770200+0000","last_undegraded":"2024-09-11T08:55:13.770200+0000","last_fullsized":"2024-09-11T08:55:13.770200+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:13.770061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:13.770061+0000","last_clean_scrub_stamp":"2024-09-11T08:55:13.770061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:23.903893+0000","scrub_duration":0.0014002400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"331'11400","reported_seq":11446,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-09-11T08:55:16.236747+0000","last_change":"2024-09-11T08:52:56.413137+0000","last_active":"2024-09-11T08:55:16.236747+0000","last_peered":"2024-09-11T08:55:16.236747+0000","last_clean":"2024-09-11T08:55:16.236747+0000","last_became_active":"2024-09-11T08:52:56.412780+0000","last_became_peered":"2024-09-11T08:52:56.412780+0000","last_unstale":"2024-09-11T08:55:16.236747+0000","last_undegraded":"2024-09-11T08:55:16.236747+0000","last_fullsized":"2024-09-11T08:55:16.236747+0000","mapping_epoch":293,"log_start":"318'7000","ondisk_log_start":"318'7000","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4400,"log_dups_size":0,"ondisk_log_size":4400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364380160,"num_objects":5560,"num_object_clones":0,"num_object_copies":16680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11400,"num_write_kb":542720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"200'35","reported_seq":259,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.226150+0000","last_change":"2024-09-11T08:53:48.594039+0000","last_active":"2024-09-11T08:55:03.226150+0000","last_peered":"2024-09-11T08:55:03.226150+0000","last_clean":"2024-09-11T08:55:03.226150+0000","last_became_active":"2024-09-11T08:46:01.981339+0000","last_became_peered":"2024-09-11T08:46:01.981339+0000","last_unstale":"2024-09-11T08:55:03.226150+0000","last_undegraded":"2024-09-11T08:55:03.226150+0000","last_fullsized":"2024-09-11T08:55:03.226150+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'35","last_scrub_stamp":"2024-09-11T08:53:48.593945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:48.593945+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058120400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":67,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.226777+0000","last_change":"2024-09-11T08:53:56.555571+0000","last_active":"2024-09-11T08:55:03.226777+0000","last_peered":"2024-09-11T08:55:03.226777+0000","last_clean":"2024-09-11T08:55:03.226777+0000","last_became_active":"2024-09-11T08:52:38.484857+0000","last_became_peered":"2024-09-11T08:52:38.484857+0000","last_unstale":"2024-09-11T08:55:03.226777+0000","last_undegraded":"2024-09-11T08:55:03.226777+0000","last_fullsized":"2024-09-11T08:55:03.226777+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:56.555486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:56.555486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:14.819930+0000","scrub_duration":0.0053230159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":401,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:08.763939+0000","last_change":"2024-09-11T08:55:08.763871+0000","last_active":"2024-09-11T08:55:08.763939+0000","last_peered":"2024-09-11T08:55:08.763939+0000","last_clean":"2024-09-11T08:55:08.763939+0000","last_became_active":"2024-09-11T08:52:18.584748+0000","last_became_peered":"2024-09-11T08:52:18.584748+0000","last_unstale":"2024-09-11T08:55:08.763939+0000","last_undegraded":"2024-09-11T08:55:08.763939+0000","last_fullsized":"2024-09-11T08:55:08.763939+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:08.763752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:35.022256+0000","last_clean_scrub_stamp":"2024-09-11T08:55:08.763752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:36.363939+0000","scrub_duration":0.001980589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":28,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.245667+0000","last_change":"2024-09-11T08:54:17.155880+0000","last_active":"2024-09-11T08:55:03.245667+0000","last_peered":"2024-09-11T08:55:03.245667+0000","last_clean":"2024-09-11T08:55:03.245667+0000","last_became_active":"2024-09-11T08:54:17.155625+0000","last_became_peered":"2024-09-11T08:54:17.155625+0000","last_unstale":"2024-09-11T08:55:03.245667+0000","last_undegraded":"2024-09-11T08:55:03.245667+0000","last_fullsized":"2024-09-11T08:55:03.245667+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:14.828985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:14.828985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:43.840148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.7","version":"331'11591","reported_seq":11915,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-09-11T08:55:16.408940+0000","last_change":"2024-09-11T08:54:11.461673+0000","last_active":"2024-09-11T08:55:16.408940+0000","last_peered":"2024-09-11T08:55:16.408940+0000","last_clean":"2024-09-11T08:55:16.408940+0000","last_became_active":"2024-09-11T08:52:55.613121+0000","last_became_peered":"2024-09-11T08:52:55.613121+0000","last_unstale":"2024-09-11T08:55:16.408940+0000","last_undegraded":"2024-09-11T08:55:16.408940+0000","last_fullsized":"2024-09-11T08:55:16.408940+0000","mapping_epoch":293,"log_start":"316'6100","ondisk_log_start":"316'6100","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'6823","last_scrub_stamp":"2024-09-11T08:54:11.461591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:11.461591+0000","objects_scrubbed":6729,"log_size":5491,"log_dups_size":0,"ondisk_log_size":5491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2.7728704209999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359989248,"num_objects":5493,"num_object_clones":0,"num_object_copies":16479,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11591,"num_write_kb":546688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"302'23","reported_seq":369,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223499+0000","last_change":"2024-09-11T08:53:16.897477+0000","last_active":"2024-09-11T08:55:03.223499+0000","last_peered":"2024-09-11T08:55:03.223499+0000","last_clean":"2024-09-11T08:55:03.223499+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:55:03.223499+0000","last_undegraded":"2024-09-11T08:55:03.223499+0000","last_fullsized":"2024-09-11T08:55:03.223499+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'23","last_scrub_stamp":"2024-09-11T08:53:16.896643+0000","last_deep_scrub":"302'23","last_deep_scrub_stamp":"2024-09-11T08:53:16.896643+0000","last_clean_scrub_stamp":"2024-09-11T08:53:16.896643+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090878710000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":67,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.224752+0000","last_change":"2024-09-11T08:55:02.779440+0000","last_active":"2024-09-11T08:55:03.224752+0000","last_peered":"2024-09-11T08:55:03.224752+0000","last_clean":"2024-09-11T08:55:03.224752+0000","last_became_active":"2024-09-11T08:52:38.487881+0000","last_became_peered":"2024-09-11T08:52:38.487881+0000","last_unstale":"2024-09-11T08:55:03.224752+0000","last_undegraded":"2024-09-11T08:55:03.224752+0000","last_fullsized":"2024-09-11T08:55:03.224752+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:02.779144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:24.331966+0000","last_clean_scrub_stamp":"2024-09-11T08:55:02.779144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:12.348550+0000","scrub_duration":0.026798161000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":343,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:10.138453+0000","last_change":"2024-09-11T08:55:10.138179+0000","last_active":"2024-09-11T08:55:10.138453+0000","last_peered":"2024-09-11T08:55:10.138453+0000","last_clean":"2024-09-11T08:55:10.138453+0000","last_became_active":"2024-09-11T08:54:17.122020+0000","last_became_peered":"2024-09-11T08:54:17.122020+0000","last_unstale":"2024-09-11T08:55:10.138453+0000","last_undegraded":"2024-09-11T08:55:10.138453+0000","last_fullsized":"2024-09-11T08:55:10.138453+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:10.137715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:55:10.137715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:13.655190+0000","scrub_duration":0.013124693999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":327,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.245040+0000","last_change":"2024-09-11T08:53:58.389027+0000","last_active":"2024-09-11T08:55:03.245040+0000","last_peered":"2024-09-11T08:55:03.245040+0000","last_clean":"2024-09-11T08:55:03.245040+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:55:03.245040+0000","last_undegraded":"2024-09-11T08:55:03.245040+0000","last_fullsized":"2024-09-11T08:55:03.245040+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:58.388796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:53:58.388796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004542244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"331'13150","reported_seq":13262,"reported_epoch":331,"state":"active+clean+scrubbing+deep","last_fresh":"2024-09-11T08:55:16.707042+0000","last_change":"2024-09-11T08:55:07.271047+0000","last_active":"2024-09-11T08:55:16.707042+0000","last_peered":"2024-09-11T08:55:16.707042+0000","last_clean":"2024-09-11T08:55:16.707042+0000","last_became_active":"2024-09-11T08:52:56.412787+0000","last_became_peered":"2024-09-11T08:52:56.412787+0000","last_unstale":"2024-09-11T08:55:16.707042+0000","last_undegraded":"2024-09-11T08:55:16.707042+0000","last_fullsized":"2024-09-11T08:55:16.707042+0000","mapping_epoch":293,"log_start":"324'7901","ondisk_log_start":"324'7901","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":1748,"log_size":5249,"log_dups_size":0,"ondisk_log_size":5249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 9s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":351404032,"num_objects":5362,"num_object_clones":0,"num_object_copies":16086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11402,"num_write_kb":536448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"200'34","reported_seq":449,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.225715+0000","last_change":"2024-09-11T08:53:55.561925+0000","last_active":"2024-09-11T08:55:03.225715+0000","last_peered":"2024-09-11T08:55:03.225715+0000","last_clean":"2024-09-11T08:55:03.225715+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:55:03.225715+0000","last_undegraded":"2024-09-11T08:55:03.225715+0000","last_fullsized":"2024-09-11T08:55:03.225715+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:53:55.561835+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:53:55.561835+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:18.099620+0000","scrub_duration":0.0037196429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":76,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223976+0000","last_change":"2024-09-11T08:52:54.988951+0000","last_active":"2024-09-11T08:55:03.223976+0000","last_peered":"2024-09-11T08:55:03.223976+0000","last_clean":"2024-09-11T08:55:03.223976+0000","last_became_active":"2024-09-11T08:52:38.484199+0000","last_became_peered":"2024-09-11T08:52:38.484199+0000","last_unstale":"2024-09-11T08:55:03.223976+0000","last_undegraded":"2024-09-11T08:55:03.223976+0000","last_fullsized":"2024-09-11T08:55:03.223976+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:52:54.988868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.988868+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019261269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"287'11","reported_seq":375,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:12.861434+0000","last_change":"2024-09-11T08:54:13.613549+0000","last_active":"2024-09-11T08:55:12.861434+0000","last_peered":"2024-09-11T08:55:12.861434+0000","last_clean":"2024-09-11T08:55:12.861434+0000","last_became_active":"2024-09-11T08:52:38.488856+0000","last_became_peered":"2024-09-11T08:52:38.488856+0000","last_unstale":"2024-09-11T08:55:12.861434+0000","last_undegraded":"2024-09-11T08:55:12.861434+0000","last_fullsized":"2024-09-11T08:55:12.861434+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"287'11","last_scrub_stamp":"2024-09-11T08:54:13.613459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:13.613459+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:42.670483+0000","scrub_duration":0.0036276939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":363,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.226520+0000","last_change":"2024-09-11T08:54:56.341981+0000","last_active":"2024-09-11T08:55:03.226520+0000","last_peered":"2024-09-11T08:55:03.226520+0000","last_clean":"2024-09-11T08:55:03.226520+0000","last_became_active":"2024-09-11T08:52:18.573947+0000","last_became_peered":"2024-09-11T08:52:18.573947+0000","last_unstale":"2024-09-11T08:55:03.226520+0000","last_undegraded":"2024-09-11T08:55:03.226520+0000","last_fullsized":"2024-09-11T08:55:03.226520+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:56.341896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:56.341896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:01.133096+0000","scrub_duration":0.13742462999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":402,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.225444+0000","last_change":"2024-09-11T08:54:01.629327+0000","last_active":"2024-09-11T08:55:03.225444+0000","last_peered":"2024-09-11T08:55:03.225444+0000","last_clean":"2024-09-11T08:55:03.225444+0000","last_became_active":"2024-09-11T08:52:18.584768+0000","last_became_peered":"2024-09-11T08:52:18.584768+0000","last_unstale":"2024-09-11T08:55:03.225444+0000","last_undegraded":"2024-09-11T08:55:03.225444+0000","last_fullsized":"2024-09-11T08:55:03.225444+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:01.629242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:54:01.629242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:29.803832+0000","scrub_duration":0.003718659,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"200'12","reported_seq":377,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:12.679149+0000","last_change":"2024-09-11T08:55:12.678723+0000","last_active":"2024-09-11T08:55:12.679149+0000","last_peered":"2024-09-11T08:55:12.679149+0000","last_clean":"2024-09-11T08:55:12.679149+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:55:12.679149+0000","last_undegraded":"2024-09-11T08:55:12.679149+0000","last_fullsized":"2024-09-11T08:55:12.679149+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'12","last_scrub_stamp":"2024-09-11T08:55:12.678247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:12.678247+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:39.893337+0000","scrub_duration":0.020917618999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":339,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.227086+0000","last_change":"2024-09-11T08:53:20.732004+0000","last_active":"2024-09-11T08:55:03.227086+0000","last_peered":"2024-09-11T08:55:03.227086+0000","last_clean":"2024-09-11T08:55:03.227086+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:55:03.227086+0000","last_undegraded":"2024-09-11T08:55:03.227086+0000","last_fullsized":"2024-09-11T08:55:03.227086+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:20.731788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:20.731788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022864600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":366,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.218555+0000","last_change":"2024-09-11T08:54:12.234900+0000","last_active":"2024-09-11T08:55:03.218555+0000","last_peered":"2024-09-11T08:55:03.218555+0000","last_clean":"2024-09-11T08:55:03.218555+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:55:03.218555+0000","last_undegraded":"2024-09-11T08:55:03.218555+0000","last_fullsized":"2024-09-11T08:55:03.218555+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:12.234807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:12.234807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:40.923144+0000","scrub_duration":0.0030160999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"200'15","reported_seq":373,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.245218+0000","last_change":"2024-09-11T08:54:09.548343+0000","last_active":"2024-09-11T08:55:03.245218+0000","last_peered":"2024-09-11T08:55:03.245218+0000","last_clean":"2024-09-11T08:55:03.245218+0000","last_became_active":"2024-09-11T08:52:49.569850+0000","last_became_peered":"2024-09-11T08:52:49.569850+0000","last_unstale":"2024-09-11T08:55:03.245218+0000","last_undegraded":"2024-09-11T08:55:03.245218+0000","last_fullsized":"2024-09-11T08:55:03.245218+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:54:09.548256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.548256+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:22.773051+0000","scrub_duration":0.0072201399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":67,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223582+0000","last_change":"2024-09-11T08:54:53.712104+0000","last_active":"2024-09-11T08:55:03.223582+0000","last_peered":"2024-09-11T08:55:03.223582+0000","last_clean":"2024-09-11T08:55:03.223582+0000","last_became_active":"2024-09-11T08:52:38.485708+0000","last_became_peered":"2024-09-11T08:52:38.485708+0000","last_unstale":"2024-09-11T08:55:03.223582+0000","last_undegraded":"2024-09-11T08:55:03.223582+0000","last_fullsized":"2024-09-11T08:55:03.223582+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:53.709804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:54:53.709804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:22.058680+0000","scrub_duration":0.0042052879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":329,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.250862+0000","last_change":"2024-09-11T08:54:06.496717+0000","last_active":"2024-09-11T08:55:03.250862+0000","last_peered":"2024-09-11T08:55:03.250862+0000","last_clean":"2024-09-11T08:55:03.250862+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:55:03.250862+0000","last_undegraded":"2024-09-11T08:55:03.250862+0000","last_fullsized":"2024-09-11T08:55:03.250862+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:06.496497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:06.496497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:32.871720+0000","scrub_duration":0.0072082789999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":367,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.221692+0000","last_change":"2024-09-11T08:53:56.542716+0000","last_active":"2024-09-11T08:55:03.221692+0000","last_peered":"2024-09-11T08:55:03.221692+0000","last_clean":"2024-09-11T08:55:03.221692+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:55:03.221692+0000","last_undegraded":"2024-09-11T08:55:03.221692+0000","last_fullsized":"2024-09-11T08:55:03.221692+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:56.542653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_clean_scrub_stamp":"2024-09-11T08:53:56.542653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:15.861997+0000","scrub_duration":0.010395433000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"200'15","reported_seq":392,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.226823+0000","last_change":"2024-09-11T08:54:08.406799+0000","last_active":"2024-09-11T08:55:03.226823+0000","last_peered":"2024-09-11T08:55:03.226823+0000","last_clean":"2024-09-11T08:55:03.226823+0000","last_became_active":"2024-09-11T08:53:58.555987+0000","last_became_peered":"2024-09-11T08:53:58.555987+0000","last_unstale":"2024-09-11T08:55:03.226823+0000","last_undegraded":"2024-09-11T08:55:03.226823+0000","last_fullsized":"2024-09-11T08:55:03.226823+0000","mapping_epoch":315,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:54:08.406695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:08.406695+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:31.897226+0000","scrub_duration":0.001945411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":67,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.641141+0000","last_change":"2024-09-11T08:55:03.641087+0000","last_active":"2024-09-11T08:55:03.641141+0000","last_peered":"2024-09-11T08:55:03.641141+0000","last_clean":"2024-09-11T08:55:03.641141+0000","last_became_active":"2024-09-11T08:52:49.566759+0000","last_became_peered":"2024-09-11T08:52:49.566759+0000","last_unstale":"2024-09-11T08:55:03.641141+0000","last_undegraded":"2024-09-11T08:55:03.641141+0000","last_fullsized":"2024-09-11T08:55:03.641141+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:03.641009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:03.641009+0000","last_clean_scrub_stamp":"2024-09-11T08:55:03.641009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:24.824655+0000","scrub_duration":0.0057082440000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":536,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:12.861623+0000","last_change":"2024-09-11T08:54:02.455288+0000","last_active":"2024-09-11T08:55:12.861623+0000","last_peered":"2024-09-11T08:55:12.861623+0000","last_clean":"2024-09-11T08:55:12.861623+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:55:12.861623+0000","last_undegraded":"2024-09-11T08:55:12.861623+0000","last_fullsized":"2024-09-11T08:55:12.861623+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:54:02.454907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:02.454907+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:26.859984+0000","scrub_duration":0.0078828809999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":321,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223696+0000","last_change":"2024-09-11T08:52:27.235517+0000","last_active":"2024-09-11T08:55:03.223696+0000","last_peered":"2024-09-11T08:55:03.223696+0000","last_clean":"2024-09-11T08:55:03.223696+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:55:03.223696+0000","last_undegraded":"2024-09-11T08:55:03.223696+0000","last_fullsized":"2024-09-11T08:55:03.223696+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:27.235448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:52:27.235448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017172019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"200'25","reported_seq":366,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.246446+0000","last_change":"2024-09-11T08:52:53.432742+0000","last_active":"2024-09-11T08:55:03.246446+0000","last_peered":"2024-09-11T08:55:03.246446+0000","last_clean":"2024-09-11T08:55:03.246446+0000","last_became_active":"2024-09-11T08:38:41.040155+0000","last_became_peered":"2024-09-11T08:38:41.040155+0000","last_unstale":"2024-09-11T08:55:03.246446+0000","last_undegraded":"2024-09-11T08:55:03.246446+0000","last_fullsized":"2024-09-11T08:55:03.246446+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:52:53.432685+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:52:53.432685+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001930122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":362,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.218370+0000","last_change":"2024-09-11T08:54:15.235583+0000","last_active":"2024-09-11T08:55:03.218370+0000","last_peered":"2024-09-11T08:55:03.218370+0000","last_clean":"2024-09-11T08:55:03.218370+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:55:03.218370+0000","last_undegraded":"2024-09-11T08:55:03.218370+0000","last_fullsized":"2024-09-11T08:55:03.218370+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:15.235519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:15.235519+0000","last_clean_scrub_stamp":"2024-09-11T08:54:15.235519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:40.532313+0000","scrub_duration":0.0033574619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"277'10","reported_seq":574,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:12.863448+0000","last_change":"2024-09-11T08:53:53.566688+0000","last_active":"2024-09-11T08:55:12.863448+0000","last_peered":"2024-09-11T08:55:12.863448+0000","last_clean":"2024-09-11T08:55:12.863448+0000","last_became_active":"2024-09-11T08:52:18.585914+0000","last_became_peered":"2024-09-11T08:52:18.585914+0000","last_unstale":"2024-09-11T08:55:12.863448+0000","last_undegraded":"2024-09-11T08:55:12.863448+0000","last_fullsized":"2024-09-11T08:55:12.863448+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'10","last_scrub_stamp":"2024-09-11T08:53:53.566589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:53.566589+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0096769479999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":67,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.245289+0000","last_change":"2024-09-11T08:53:01.446031+0000","last_active":"2024-09-11T08:55:03.245289+0000","last_peered":"2024-09-11T08:55:03.245289+0000","last_clean":"2024-09-11T08:55:03.245289+0000","last_became_active":"2024-09-11T08:52:49.567693+0000","last_became_peered":"2024-09-11T08:52:49.567693+0000","last_unstale":"2024-09-11T08:55:03.245289+0000","last_undegraded":"2024-09-11T08:55:03.245289+0000","last_fullsized":"2024-09-11T08:55:03.245289+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:01.445935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:01.445935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019176729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"200'19","reported_seq":385,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223274+0000","last_change":"2024-09-11T08:54:15.800180+0000","last_active":"2024-09-11T08:55:03.223274+0000","last_peered":"2024-09-11T08:55:03.223274+0000","last_clean":"2024-09-11T08:55:03.223274+0000","last_became_active":"2024-09-11T08:49:16.937185+0000","last_became_peered":"2024-09-11T08:49:16.937185+0000","last_unstale":"2024-09-11T08:55:03.223274+0000","last_undegraded":"2024-09-11T08:55:03.223274+0000","last_fullsized":"2024-09-11T08:55:03.223274+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:54:15.800092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:15.800092+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:44.218489+0000","scrub_duration":0.014286116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":334,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.223686+0000","last_change":"2024-09-11T08:53:21.891151+0000","last_active":"2024-09-11T08:55:03.223686+0000","last_peered":"2024-09-11T08:55:03.223686+0000","last_clean":"2024-09-11T08:55:03.223686+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:55:03.223686+0000","last_undegraded":"2024-09-11T08:55:03.223686+0000","last_fullsized":"2024-09-11T08:55:03.223686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:21.891073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:21.891073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024938569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":349,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.237228+0000","last_change":"2024-09-11T08:54:08.814739+0000","last_active":"2024-09-11T08:55:03.237228+0000","last_peered":"2024-09-11T08:55:03.237228+0000","last_clean":"2024-09-11T08:55:03.237228+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:55:03.237228+0000","last_undegraded":"2024-09-11T08:55:03.237228+0000","last_fullsized":"2024-09-11T08:55:03.237228+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:08.814646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:08.814646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:15.264001+0000","scrub_duration":0.0047888419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":347,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.244772+0000","last_change":"2024-09-11T08:53:21.391938+0000","last_active":"2024-09-11T08:55:03.244772+0000","last_peered":"2024-09-11T08:55:03.244772+0000","last_clean":"2024-09-11T08:55:03.244772+0000","last_became_active":"2024-09-11T08:52:18.585089+0000","last_became_peered":"2024-09-11T08:52:18.585089+0000","last_unstale":"2024-09-11T08:55:03.244772+0000","last_undegraded":"2024-09-11T08:55:03.244772+0000","last_fullsized":"2024-09-11T08:55:03.244772+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:21.391849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:53:21.391849+0000","last_clean_scrub_stamp":"2024-09-11T08:53:21.391849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003344732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"200'15","reported_seq":379,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.227325+0000","last_change":"2024-09-11T08:53:19.736495+0000","last_active":"2024-09-11T08:55:03.227325+0000","last_peered":"2024-09-11T08:55:03.227325+0000","last_clean":"2024-09-11T08:55:03.227325+0000","last_became_active":"2024-09-11T08:46:01.977097+0000","last_became_peered":"2024-09-11T08:46:01.977097+0000","last_unstale":"2024-09-11T08:55:03.227325+0000","last_undegraded":"2024-09-11T08:55:03.227325+0000","last_fullsized":"2024-09-11T08:55:03.227325+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:53:19.736363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:19.736363+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040364540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":67,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.247347+0000","last_change":"2024-09-11T08:53:16.431184+0000","last_active":"2024-09-11T08:55:03.247347+0000","last_peered":"2024-09-11T08:55:03.247347+0000","last_clean":"2024-09-11T08:55:03.247347+0000","last_became_active":"2024-09-11T08:52:38.485748+0000","last_became_peered":"2024-09-11T08:52:38.485748+0000","last_unstale":"2024-09-11T08:55:03.247347+0000","last_undegraded":"2024-09-11T08:55:03.247347+0000","last_fullsized":"2024-09-11T08:55:03.247347+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:16.430996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:21.208684+0000","last_clean_scrub_stamp":"2024-09-11T08:53:16.430996+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051148779999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":349,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.245178+0000","last_change":"2024-09-11T08:52:54.441725+0000","last_active":"2024-09-11T08:55:03.245178+0000","last_peered":"2024-09-11T08:55:03.245178+0000","last_clean":"2024-09-11T08:55:03.245178+0000","last_became_active":"2024-09-11T08:52:18.585691+0000","last_became_peered":"2024-09-11T08:52:18.585691+0000","last_unstale":"2024-09-11T08:55:03.245178+0000","last_undegraded":"2024-09-11T08:55:03.245178+0000","last_fullsized":"2024-09-11T08:55:03.245178+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.441666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.441666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012216760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":149,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.225634+0000","last_change":"2024-09-11T08:54:07.453666+0000","last_active":"2024-09-11T08:55:03.225634+0000","last_peered":"2024-09-11T08:55:03.225634+0000","last_clean":"2024-09-11T08:55:03.225634+0000","last_became_active":"2024-09-11T08:52:49.566694+0000","last_became_peered":"2024-09-11T08:52:49.566694+0000","last_unstale":"2024-09-11T08:55:03.225634+0000","last_undegraded":"2024-09-11T08:55:03.225634+0000","last_fullsized":"2024-09-11T08:55:03.225634+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:07.453531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:24.401480+0000","last_clean_scrub_stamp":"2024-09-11T08:54:07.453531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:24.530018+0000","scrub_duration":0.0048569140000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"200'19","reported_seq":362,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.244538+0000","last_change":"2024-09-11T08:54:53.565356+0000","last_active":"2024-09-11T08:55:03.244538+0000","last_peered":"2024-09-11T08:55:03.244538+0000","last_clean":"2024-09-11T08:55:03.244538+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:55:03.244538+0000","last_undegraded":"2024-09-11T08:55:03.244538+0000","last_fullsized":"2024-09-11T08:55:03.244538+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:54:53.565275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:53.565275+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:11.498704+0000","scrub_duration":0.0061943880000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":343,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.227147+0000","last_change":"2024-09-11T08:53:04.734505+0000","last_active":"2024-09-11T08:55:03.227147+0000","last_peered":"2024-09-11T08:55:03.227147+0000","last_clean":"2024-09-11T08:55:03.227147+0000","last_became_active":"2024-09-11T08:38:38.950839+0000","last_became_peered":"2024-09-11T08:38:38.950839+0000","last_unstale":"2024-09-11T08:55:03.227147+0000","last_undegraded":"2024-09-11T08:55:03.227147+0000","last_fullsized":"2024-09-11T08:55:03.227147+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:04.734452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:04.734452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001307342,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":361,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.217986+0000","last_change":"2024-09-11T08:54:09.162741+0000","last_active":"2024-09-11T08:55:03.217986+0000","last_peered":"2024-09-11T08:55:03.217986+0000","last_clean":"2024-09-11T08:55:03.217986+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:55:03.217986+0000","last_undegraded":"2024-09-11T08:55:03.217986+0000","last_fullsized":"2024-09-11T08:55:03.217986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:09.162547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.162547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:27.715851+0000","scrub_duration":0.0020448530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":367,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.245323+0000","last_change":"2024-09-11T08:54:08.525090+0000","last_active":"2024-09-11T08:55:03.245323+0000","last_peered":"2024-09-11T08:55:03.245323+0000","last_clean":"2024-09-11T08:55:03.245323+0000","last_became_active":"2024-09-11T08:52:18.585315+0000","last_became_peered":"2024-09-11T08:52:18.585315+0000","last_unstale":"2024-09-11T08:55:03.245323+0000","last_undegraded":"2024-09-11T08:55:03.245323+0000","last_fullsized":"2024-09-11T08:55:03.245323+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:08.524974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:25.255332+0000","last_clean_scrub_stamp":"2024-09-11T08:54:08.524974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:26.888865+0000","scrub_duration":0.0080002040000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"200'10","reported_seq":390,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.245518+0000","last_change":"2024-09-11T08:55:00.665641+0000","last_active":"2024-09-11T08:55:03.245518+0000","last_peered":"2024-09-11T08:55:03.245518+0000","last_clean":"2024-09-11T08:55:03.245518+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:55:03.245518+0000","last_undegraded":"2024-09-11T08:55:03.245518+0000","last_fullsized":"2024-09-11T08:55:03.245518+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'10","last_scrub_stamp":"2024-09-11T08:55:00.665557+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:55:00.665557+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:28.364054+0000","scrub_duration":0.0031074079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":161,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.227434+0000","last_change":"2024-09-11T08:54:02.427838+0000","last_active":"2024-09-11T08:55:03.227434+0000","last_peered":"2024-09-11T08:55:03.227434+0000","last_clean":"2024-09-11T08:55:03.227434+0000","last_became_active":"2024-09-11T08:52:49.569954+0000","last_became_peered":"2024-09-11T08:52:49.569954+0000","last_unstale":"2024-09-11T08:55:03.227434+0000","last_undegraded":"2024-09-11T08:55:03.227434+0000","last_fullsized":"2024-09-11T08:55:03.227434+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:54:02.427745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:54:02.427745+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:15.894988+0000","scrub_duration":0.0045687649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":222,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.225535+0000","last_change":"2024-09-11T08:54:55.481853+0000","last_active":"2024-09-11T08:55:03.225535+0000","last_peered":"2024-09-11T08:55:03.225535+0000","last_clean":"2024-09-11T08:55:03.225535+0000","last_became_active":"2024-09-11T08:46:01.978484+0000","last_became_peered":"2024-09-11T08:46:01.978484+0000","last_unstale":"2024-09-11T08:55:03.225535+0000","last_undegraded":"2024-09-11T08:55:03.225535+0000","last_fullsized":"2024-09-11T08:55:03.225535+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:55.481627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:55.481627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:14.975320+0000","scrub_duration":0.001890325,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":28,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.224042+0000","last_change":"2024-09-11T08:54:17.155339+0000","last_active":"2024-09-11T08:55:03.224042+0000","last_peered":"2024-09-11T08:55:03.224042+0000","last_clean":"2024-09-11T08:55:03.224042+0000","last_became_active":"2024-09-11T08:54:17.154933+0000","last_became_peered":"2024-09-11T08:54:17.154933+0000","last_unstale":"2024-09-11T08:55:03.224042+0000","last_undegraded":"2024-09-11T08:55:03.224042+0000","last_fullsized":"2024-09-11T08:55:03.224042+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:11.694784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:30.788303+0000","last_clean_scrub_stamp":"2024-09-11T08:54:11.694784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:16.882580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":357,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.232977+0000","last_change":"2024-09-11T08:54:11.861507+0000","last_active":"2024-09-11T08:55:03.232977+0000","last_peered":"2024-09-11T08:55:03.232977+0000","last_clean":"2024-09-11T08:55:03.232977+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:55:03.232977+0000","last_undegraded":"2024-09-11T08:55:03.232977+0000","last_fullsized":"2024-09-11T08:55:03.232977+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:11.861416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:11.861416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:35.847712+0000","scrub_duration":0.0055713560000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":363,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.245729+0000","last_change":"2024-09-11T08:54:13.701850+0000","last_active":"2024-09-11T08:55:03.245729+0000","last_peered":"2024-09-11T08:55:03.245729+0000","last_clean":"2024-09-11T08:55:03.245729+0000","last_became_active":"2024-09-11T08:52:18.595146+0000","last_became_peered":"2024-09-11T08:52:18.595146+0000","last_unstale":"2024-09-11T08:55:03.245729+0000","last_undegraded":"2024-09-11T08:55:03.245729+0000","last_fullsized":"2024-09-11T08:55:03.245729+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:13.701753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:32.393197+0000","last_clean_scrub_stamp":"2024-09-11T08:54:13.701753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:32.313595+0000","scrub_duration":0.113734606,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"200'19","reported_seq":236,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.226917+0000","last_change":"2024-09-11T08:53:52.564728+0000","last_active":"2024-09-11T08:55:03.226917+0000","last_peered":"2024-09-11T08:55:03.226917+0000","last_clean":"2024-09-11T08:55:03.226917+0000","last_became_active":"2024-09-11T08:46:01.985966+0000","last_became_peered":"2024-09-11T08:46:01.985966+0000","last_unstale":"2024-09-11T08:55:03.226917+0000","last_undegraded":"2024-09-11T08:55:03.226917+0000","last_fullsized":"2024-09-11T08:55:03.226917+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:53:52.564594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:52.564594+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020268980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":158,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-11T08:55:03.218976+0000","last_change":"2024-09-11T08:54:54.799438+0000","last_active":"2024-09-11T08:55:03.218976+0000","last_peered":"2024-09-11T08:55:03.218976+0000","last_clean":"2024-09-11T08:55:03.218976+0000","last_became_active":"2024-09-11T08:52:49.568537+0000","last_became_peered":"2024-09-11T08:52:49.568537+0000","last_unstale":"2024-09-11T08:55:03.218976+0000","last_undegraded":"2024-09-11T08:55:03.218976+0000","last_fullsized":"2024-09-11T08:55:03.218976+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:54.799254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:54.799254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:12.982558+0000","scrub_duration":0.013107982000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":5794234392,"num_objects":88414,"num_object_clones":0,"num_object_copies":265242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88414,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185500,"num_write_kb":8765185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17373605888,"data_stored":17373593672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82932,"ondisk_log_size":82932,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1118,"num_read_kb":875,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":4224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":769,"ondisk_log_size":769,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299597,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2322692,"kb_used_data":1425464,"kb_used_omap":1,"kb_used_meta":897214,"kb_avail":91422460,"statfs":{"total":95995035648,"available":93616599040,"internally_reserved":0,"allocated":1459675136,"data_stored":1459363500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":918747480},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:54:57 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":0.86899999999999999,"15min":0.98399999999999999},"min":{"1min":0.24099999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":5.0449999999999999,"5min":5.0449999999999999,"15min":32.171999999999997},"last":1.5980000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.79800000000000004,"15min":1.071},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.21199999999999999},"max":{"1min":2.2829999999999999,"5min":4.0069999999999997,"15min":32.149999999999999},"last":0.372}]},{"osd":1,"last update":"Wed Sep 11 08:54:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.91500000000000004,"15min":1.137},"min":{"1min":0.32100000000000001,"5min":0.23799999999999999,"15min":0.224},"max":{"1min":2.5609999999999999,"5min":4.8970000000000002,"15min":33.692999999999998},"last":1.4690000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.81899999999999995,"15min":1.0469999999999999},"min":{"1min":0.28699999999999998,"5min":0.249,"15min":0.23300000000000001},"max":{"1min":2.1619999999999999,"5min":3.9780000000000002,"15min":35.591000000000001},"last":0.33900000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:54:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.0980000000000001,"15min":1.006},"min":{"1min":0.32500000000000001,"5min":0.30499999999999999,"15min":0.249},"max":{"1min":3.0339999999999998,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.91100000000000003,"15min":0.94399999999999995},"min":{"1min":0.29699999999999999,"5min":0.27900000000000003,"15min":0.25800000000000001},"max":{"1min":2.5350000000000001,"5min":4.9189999999999996,"15min":7.5259999999999998},"last":1.496}]},{"osd":3,"last update":"Wed Sep 11 08:54:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.74,"15min":1.2549999999999999},"min":{"1min":0.372,"5min":0.28699999999999998,"15min":0.25},"max":{"1min":3.0910000000000002,"5min":56.265999999999998,"15min":56.265999999999998},"last":0.311},{"interface":"front","average":{"1min":1.298,"5min":1.71,"15min":1.3160000000000001},"min":{"1min":0.45400000000000001,"5min":0.34499999999999997,"15min":0.27400000000000002},"max":{"1min":3.4239999999999999,"5min":57.207000000000001,"15min":57.207000000000001},"last":1.5669999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:54:57 2024","interfaces":[{"interface":"back","average":{"1min":10.523,"5min":2.8719999999999999,"15min":1.355},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23400000000000001},"max":{"1min":163.60499999999999,"5min":163.60499999999999,"15min":163.60499999999999},"last":0.442},{"interface":"front","average":{"1min":10.874000000000001,"5min":3.0819999999999999,"15min":1.4390000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":163.88300000000001,"5min":163.88300000000001,"15min":163.88300000000001},"last":0.629}]},{"osd":5,"last update":"Wed Sep 11 08:54:57 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.0720000000000001,"15min":0.79200000000000004},"min":{"1min":0.308,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":5.1349999999999998,"5min":5.1349999999999998,"15min":5.1349999999999998},"last":1.9650000000000001},{"interface":"front","average":{"1min":1.48,"5min":1.153,"15min":0.86599999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":6.492,"5min":6.492,"15min":6.492},"last":0.39800000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:54:57 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.1359999999999999,"15min":0.84299999999999997},"min":{"1min":0.41799999999999998,"5min":0.34300000000000003,"15min":0.315},"max":{"1min":3.4769999999999999,"5min":4.5449999999999999,"15min":4.5449999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.1459999999999999,"15min":0.85899999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.27900000000000003},"max":{"1min":3.0649999999999999,"5min":5.0620000000000003,"15min":5.0620000000000003},"last":1.425}]}]},{"osd":6,"up_from":45,"seq":193273528530,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3074940,"kb_used_data":2480820,"kb_used_omap":1,"kb_used_meta":594046,"kb_avail":90670212,"statfs":{"total":95995035648,"available":92846297088,"internally_reserved":0,"allocated":2540359680,"data_stored":2540052110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":608303448},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:54:36 2024","interfaces":[{"interface":"back","average":{"1min":2.7679999999999998,"5min":2.3519999999999999,"15min":1.7170000000000001},"min":{"1min":0.28799999999999998,"5min":0.23400000000000001,"15min":0.217},"max":{"1min":31.661999999999999,"5min":121.208,"15min":121.208},"last":2.1509999999999998},{"interface":"front","average":{"1min":3.7309999999999999,"5min":2.653,"15min":1.78},"min":{"1min":0.32600000000000001,"5min":0.24399999999999999,"15min":0.23300000000000001},"max":{"1min":32.015000000000001,"5min":121.05200000000001,"15min":121.05200000000001},"last":5.0090000000000003}]},{"osd":1,"last update":"Wed Sep 11 08:54:36 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.1279999999999999,"15min":1.226},"min":{"1min":0.26200000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":3.6349999999999998,"5min":15.919,"15min":27.984000000000002},"last":5.8689999999999998},{"interface":"front","average":{"1min":1.26,"5min":1.145,"15min":1.1799999999999999},"min":{"1min":0.313,"5min":0.254,"15min":0.253},"max":{"1min":3.411,"5min":15.629,"15min":28.106000000000002},"last":0.95999999999999996}]},{"osd":2,"last update":"Wed Sep 11 08:54:36 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":0.88800000000000001,"15min":0.88200000000000001},"min":{"1min":0.378,"5min":0.27500000000000002,"15min":0.23200000000000001},"max":{"1min":6.9160000000000004,"5min":6.9160000000000004,"15min":14.313000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":3.3399999999999999,"5min":1.3680000000000001,"15min":1.0649999999999999},"min":{"1min":0.377,"5min":0.27400000000000002,"15min":0.251},"max":{"1min":31.260999999999999,"5min":31.260999999999999,"15min":31.260999999999999},"last":4.9370000000000003}]},{"osd":3,"last update":"Wed Sep 11 08:54:36 2024","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.2390000000000001,"15min":1.278},"min":{"1min":0.31,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":4.883,"5min":15.798999999999999,"15min":27.795000000000002},"last":5.8470000000000004},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.198,"15min":1.252},"min":{"1min":0.379,"5min":0.30499999999999999,"15min":0.224},"max":{"1min":3.0590000000000002,"5min":15.599,"15min":27.841999999999999},"last":0.97999999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:54:36 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.544,"15min":0.85999999999999999},"min":{"1min":0.35299999999999998,"5min":0.24299999999999999,"15min":0.20799999999999999},"max":{"1min":4.516,"5min":40.927999999999997,"15min":40.927999999999997},"last":5.085},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.518,"15min":0.84899999999999998},"min":{"1min":0.28199999999999997,"5min":0.27700000000000002,"15min":0.26400000000000001},"max":{"1min":4.0750000000000002,"5min":40.706000000000003,"15min":40.706000000000003},"last":5.891}]},{"osd":5,"last update":"Wed Sep 11 08:54:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.6990000000000001,"15min":0.99299999999999999},"min":{"1min":0.40899999999999997,"5min":0.27600000000000002,"15min":0.23899999999999999},"max":{"1min":5.2069999999999999,"5min":40.847000000000001,"15min":40.847000000000001},"last":5.117},{"interface":"front","average":{"1min":1.135,"5min":1.504,"15min":0.90500000000000003},"min":{"1min":0.34200000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":4.4189999999999996,"5min":41.401000000000003,"15min":41.401000000000003},"last":5.8159999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:54:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.756,"15min":1.0189999999999999},"min":{"1min":0.376,"5min":0.32300000000000001,"15min":0.247},"max":{"1min":3.7170000000000001,"5min":40.426000000000002,"15min":40.426000000000002},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.143,"5min":1.742,"15min":0.998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.27700000000000002},"max":{"1min":3.0840000000000001,"5min":45.401000000000003,"15min":45.401000000000003},"last":0.50600000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757462,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2662604,"kb_used_data":1764484,"kb_used_omap":1,"kb_used_meta":898110,"kb_avail":91082548,"statfs":{"total":95995035648,"available":93268529152,"internally_reserved":0,"allocated":1806831616,"data_stored":1806509089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":919664984},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,12,14,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:55:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.073,"15min":0.98199999999999998},"min":{"1min":0.26300000000000001,"5min":0.221,"15min":0.20599999999999999},"max":{"1min":2.2589999999999999,"5min":8.7349999999999994,"15min":8.7349999999999994},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92900000000000005,"15min":1.0780000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23100000000000001},"max":{"1min":1.966,"5min":4.2469999999999999,"15min":8.0410000000000004},"last":1.127}]},{"osd":1,"last update":"Wed Sep 11 08:55:16 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.006,"15min":1.071},"min":{"1min":0.26500000000000001,"5min":0.216,"15min":0.216},"max":{"1min":1.8759999999999999,"5min":5.2060000000000004,"15min":8},"last":1.165},{"interface":"front","average":{"1min":0.752,"5min":1.0469999999999999,"15min":1.052},"min":{"1min":0.252,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":2.726,"5min":12.173,"15min":12.173},"last":0.48199999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:55:16 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.0740000000000001,"15min":1.075},"min":{"1min":0.309,"5min":0.253,"15min":0.23799999999999999},"max":{"1min":2.2290000000000001,"5min":8.7789999999999999,"15min":8.7789999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.046,"15min":1.0209999999999999},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.3980000000000001,"5min":4.7069999999999999,"15min":8.0640000000000001},"last":1.1970000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:55:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":2.016,"15min":1.355},"min":{"1min":0.34999999999999998,"5min":0.30099999999999999,"15min":0.26900000000000002},"max":{"1min":2.2000000000000002,"5min":85.299999999999997,"15min":85.299999999999997},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":1.901,"15min":1.4570000000000001},"min":{"1min":0.29599999999999999,"5min":0.29099999999999998,"15min":0.27400000000000002},"max":{"1min":1.9890000000000001,"5min":85.941000000000003,"15min":85.941000000000003},"last":1.252}]},{"osd":4,"last update":"Wed Sep 11 08:55:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.052,"15min":0.79200000000000004},"min":{"1min":0.28299999999999997,"5min":0.23300000000000001,"15min":0.20699999999999999},"max":{"1min":3.5960000000000001,"5min":16.021999999999998,"15min":16.021999999999998},"last":3.5960000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.95299999999999996,"15min":0.73899999999999999},"min":{"1min":0.373,"5min":0.28999999999999998,"15min":0.20499999999999999},"max":{"1min":1.944,"5min":9.6140000000000008,"15min":9.6140000000000008},"last":0.65400000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:54:37 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.1599999999999999,"15min":0.80100000000000005},"min":{"1min":0.38700000000000001,"5min":0.30399999999999999,"15min":0.23599999999999999},"max":{"1min":3.726,"5min":8.8059999999999992,"15min":8.8059999999999992},"last":1.026},{"interface":"front","average":{"1min":0.78100000000000003,"5min":1.0229999999999999,"15min":0.753},"min":{"1min":0.34999999999999998,"5min":0.27300000000000002,"15min":0.251},"max":{"1min":2.1360000000000001,"5min":5.3090000000000002,"15min":8.1920000000000002},"last":1.2250000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:54:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":1.2,"15min":1.121},"min":{"1min":0.30399999999999999,"5min":0.30099999999999999,"15min":0.27800000000000002},"max":{"1min":1.506,"5min":16.062999999999999,"15min":88.513000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":1.119,"15min":1.1180000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.29599999999999999},"max":{"1min":3.5920000000000001,"5min":8.8300000000000001,"15min":86.963999999999999},"last":0.40300000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986394,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3434508,"kb_used_data":2832128,"kb_used_omap":1,"kb_used_meta":602366,"kb_avail":90310644,"statfs":{"total":95995035648,"available":92478099456,"internally_reserved":0,"allocated":2900099072,"data_stored":2899778840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":616823128},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86299999999999999,"15min":1.6759999999999999},"min":{"1min":0.312,"5min":0.23899999999999999,"15min":0.20399999999999999},"max":{"1min":3.5230000000000001,"5min":5.2400000000000002,"15min":89.510000000000005},"last":2.629},{"interface":"front","average":{"1min":1.298,"5min":1.0069999999999999,"15min":1.8140000000000001},"min":{"1min":0.26700000000000002,"5min":0.248,"15min":0.22600000000000001},"max":{"1min":4.0579999999999998,"5min":5.3220000000000001,"15min":89.765000000000001},"last":2.593}]},{"osd":1,"last update":"Wed Sep 11 08:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":1.161,"15min":1.5},"min":{"1min":0.253,"5min":0.253,"15min":0.24399999999999999},"max":{"1min":2.7629999999999999,"5min":18.065999999999999,"15min":48.976999999999997},"last":3.6059999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.1100000000000001,"15min":1.4390000000000001},"min":{"1min":0.32300000000000001,"5min":0.23499999999999999,"15min":0.22800000000000001},"max":{"1min":4.532,"5min":18.713999999999999,"15min":45.787999999999997},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Sep 11 08:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":3.8679999999999999,"15min":1.919},"min":{"1min":0.377,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":2.0019999999999998,"5min":255.88900000000001,"15min":255.88900000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":1.556,"5min":3.944,"15min":1.9379999999999999},"min":{"1min":0.34799999999999998,"5min":0.29599999999999999,"15min":0.25800000000000001},"max":{"1min":4.5819999999999999,"5min":255.41200000000001,"15min":255.41200000000001},"last":2.7810000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:54:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.2509999999999999,"15min":2.0680000000000001},"min":{"1min":0.29499999999999998,"5min":0.252,"15min":0.252},"max":{"1min":3.7810000000000001,"5min":16.663,"15min":89.881},"last":2.895},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.034,"15min":1.96},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.25900000000000001},"max":{"1min":2.8620000000000001,"5min":16.795999999999999,"15min":89.715000000000003},"last":3.6349999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:55:14 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0620000000000001,"15min":0.78100000000000003},"min":{"1min":0.27800000000000002,"5min":0.26400000000000001,"15min":0.23999999999999999},"max":{"1min":3.6779999999999999,"5min":11.045999999999999,"15min":11.045999999999999},"last":3.6779999999999999},{"interface":"front","average":{"1min":1.006,"5min":1.1559999999999999,"15min":0.77100000000000002},"min":{"1min":0.35799999999999998,"5min":0.254,"15min":0.24199999999999999},"max":{"1min":4.7869999999999999,"5min":10.526999999999999,"15min":10.526999999999999},"last":0.55100000000000005}]},{"osd":6,"last update":"Wed Sep 11 08:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.0129999999999999,"15min":0.70099999999999996},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.22900000000000001},"max":{"1min":5.3460000000000001,"5min":5.3460000000000001,"15min":5.3460000000000001},"last":3.766},{"interface":"front","average":{"1min":1.1060000000000001,"5min":0.93200000000000005,"15min":0.72199999999999998},"min":{"1min":0.33700000000000002,"5min":0.27800000000000002,"15min":0.221},"max":{"1min":2.794,"5min":6.2619999999999996,"15min":6.2619999999999996},"last":3.6579999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.97399999999999998,"15min":0.70899999999999996},"min":{"1min":0.35399999999999998,"5min":0.253,"15min":0.252},"max":{"1min":3.2650000000000001,"5min":4.8079999999999998,"15min":4.8079999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":1.393,"5min":1.1759999999999999,"15min":0.82599999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.28799999999999998},"max":{"1min":5.2539999999999996,"5min":5.2539999999999996,"15min":5.2539999999999996},"last":0.76500000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149920,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2709988,"kb_used_data":1802456,"kb_used_omap":1,"kb_used_meta":907518,"kb_avail":91035164,"statfs":{"total":95995035648,"available":93220007936,"internally_reserved":0,"allocated":1845714944,"data_stored":1845372237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":929298776},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:54:35 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.76600000000000001,"15min":1.1579999999999999},"min":{"1min":0.27600000000000002,"5min":0.20100000000000001,"15min":0.16500000000000001},"max":{"1min":4.9740000000000002,"5min":4.9740000000000002,"15min":24.280000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":0.94699999999999995,"15min":1.4019999999999999},"min":{"1min":0.21199999999999999,"5min":0.19900000000000001,"15min":0.17000000000000001},"max":{"1min":7.5599999999999996,"5min":7.5599999999999996,"15min":22.225000000000001},"last":0.23999999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:54:35 2024","interfaces":[{"interface":"back","average":{"1min":5.9820000000000002,"5min":2.0110000000000001,"15min":2.0720000000000001},"min":{"1min":0.218,"5min":0.20100000000000001,"15min":0.19800000000000001},"max":{"1min":84.284999999999997,"5min":84.284999999999997,"15min":115.601},"last":1.391},{"interface":"front","average":{"1min":5.9790000000000001,"5min":1.8759999999999999,"15min":1.95},"min":{"1min":0.249,"5min":0.20499999999999999,"15min":0.17999999999999999},"max":{"1min":85.832999999999998,"5min":85.832999999999998,"15min":115.474},"last":0.79000000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:54:35 2024","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.8169999999999999,"15min":1.552},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":5.4950000000000001,"5min":86.766000000000005,"15min":86.766000000000005},"last":0.22500000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.849,"15min":1.498},"min":{"1min":0.32100000000000001,"5min":0.23400000000000001,"15min":0.20799999999999999},"max":{"1min":5.6820000000000004,"5min":85.367999999999995,"15min":85.367999999999995},"last":1.3380000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.9339999999999999,"5min":1.2450000000000001,"15min":1.298},"min":{"1min":0.39000000000000001,"5min":0.315,"15min":0.26700000000000002},"max":{"1min":5.9039999999999999,"5min":5.9039999999999999,"15min":8.9969999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.044,"15min":1.0800000000000001},"min":{"1min":0.47799999999999998,"5min":0.29099999999999998,"15min":0.27700000000000002},"max":{"1min":5.617,"5min":5.617,"15min":7.234},"last":1.216}]},{"osd":5,"last update":"Wed Sep 11 08:55:05 2024","interfaces":[{"interface":"back","average":{"1min":1.7470000000000001,"5min":1.216,"15min":1.284},"min":{"1min":0.38600000000000001,"5min":0.30599999999999999,"15min":0.28299999999999997},"max":{"1min":5.9470000000000001,"5min":5.9470000000000001,"15min":11.526999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.7130000000000001,"5min":1.1759999999999999,"15min":1.1819999999999999},"min":{"1min":0.38,"5min":0.379,"15min":0.29399999999999998},"max":{"1min":5.5190000000000001,"5min":5.5190000000000001,"15min":7.1689999999999996},"last":1.282}]},{"osd":6,"last update":"Wed Sep 11 08:54:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.5529999999999999,"15min":1.4259999999999999},"min":{"1min":0.33800000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.6079999999999997,"5min":33.106999999999999,"15min":33.106999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.5329999999999999,"15min":1.395},"min":{"1min":0.316,"5min":0.316,"15min":0.311},"max":{"1min":7.4809999999999999,"5min":33.076000000000001,"15min":33.076000000000001},"last":1.355}]},{"osd":7,"last update":"Wed Sep 11 08:54:56 2024","interfaces":[{"interface":"back","average":{"1min":2.048,"5min":1.891,"15min":1.5580000000000001},"min":{"1min":0.53600000000000003,"5min":0.39100000000000001,"15min":0.35299999999999998},"max":{"1min":6.0190000000000001,"5min":35.073,"15min":35.073},"last":0.34399999999999997},{"interface":"front","average":{"1min":1.758,"5min":1.708,"15min":1.3959999999999999},"min":{"1min":0.46000000000000002,"5min":0.42799999999999999,"15min":0.34699999999999998},"max":{"1min":5.6529999999999996,"5min":34.844999999999999,"15min":34.844999999999999},"last":1.2529999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411556,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2753152,"kb_used_data":2167412,"kb_used_omap":1,"kb_used_meta":585726,"kb_avail":90992000,"statfs":{"total":95995035648,"available":93175808000,"internally_reserved":0,"allocated":2219429888,"data_stored":2219122395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":599783768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:55:12 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":0.82099999999999995,"15min":1.6619999999999999},"min":{"1min":0.189,"5min":0.189,"15min":0.159},"max":{"1min":7.3380000000000001,"5min":7.3380000000000001,"15min":34.713999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":1.484,"5min":0.96499999999999997,"15min":1.79},"min":{"1min":0.28000000000000003,"5min":0.191,"15min":0.17399999999999999},"max":{"1min":9.0630000000000006,"5min":9.0630000000000006,"15min":34.399000000000001},"last":0.70099999999999996}]},{"osd":1,"last update":"Wed Sep 11 08:55:12 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.91400000000000003,"15min":1.742},"min":{"1min":0.193,"5min":0.189,"15min":0.189},"max":{"1min":2.1499999999999999,"5min":11.552,"15min":72.867000000000004},"last":0.20100000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.0389999999999999,"15min":1.8620000000000001},"min":{"1min":0.26800000000000002,"5min":0.22500000000000001,"15min":0.20399999999999999},"max":{"1min":3.9660000000000002,"5min":11.616,"15min":67.799000000000007},"last":0.35099999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:54:16 2024","interfaces":[{"interface":"back","average":{"1min":1.774,"5min":1.1160000000000001,"15min":1.6459999999999999},"min":{"1min":0.24099999999999999,"5min":0.186,"15min":0.186},"max":{"1min":6.0369999999999999,"5min":11.878,"15min":34.75},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.389,"5min":1.056,"15min":1.5620000000000001},"min":{"1min":0.22800000000000001,"5min":0.216,"15min":0.216},"max":{"1min":5.3319999999999999,"5min":9.2759999999999998,"15min":32.802},"last":0.38300000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:54:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.6499999999999999,"15min":1.2989999999999999},"min":{"1min":0.33600000000000002,"5min":0.28699999999999998,"15min":0.255},"max":{"1min":3.7480000000000002,"5min":52.113999999999997,"15min":52.113999999999997},"last":0.754},{"interface":"front","average":{"1min":1.8779999999999999,"5min":1.6759999999999999,"15min":1.2629999999999999},"min":{"1min":0.38500000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":5.3490000000000002,"5min":52.097999999999999,"15min":52.097999999999999},"last":0.60599999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:55:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.659,"15min":1.3080000000000001},"min":{"1min":0.35399999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.0510000000000002,"5min":52.145000000000003,"15min":52.145000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.472,"15min":1.1339999999999999},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.28399999999999997},"max":{"1min":4.8109999999999999,"5min":52.216000000000001,"15min":52.216000000000001},"last":0.68200000000000005}]},{"osd":6,"last update":"Wed Sep 11 08:54:33 2024","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.5980000000000001,"15min":1.331},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.28699999999999998},"max":{"1min":6.0940000000000003,"5min":52.235999999999997,"15min":52.235999999999997},"last":3.3999999999999999},{"interface":"front","average":{"1min":2.032,"5min":1.7470000000000001,"15min":1.2909999999999999},"min":{"1min":0.46500000000000002,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":5.6440000000000001,"5min":52.186,"15min":52.186},"last":0.64000000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:54:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.891,"15min":1.2849999999999999},"min":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.30399999999999999},"max":{"1min":4.8570000000000002,"5min":84.108999999999995,"15min":84.108999999999995},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.8140000000000001,"5min":2.0539999999999998,"15min":1.5149999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.30299999999999999},"max":{"1min":6.2050000000000001,"5min":85.938000000000002,"15min":85.938000000000002},"last":0.72699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575081,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1958040,"kb_used_data":1054352,"kb_used_omap":1,"kb_used_meta":903678,"kb_avail":91787112,"statfs":{"total":95995035648,"available":93990002688,"internally_reserved":0,"allocated":1079656448,"data_stored":1079320548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":925366616},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.821,"5min":1.5429999999999999,"15min":1.788},"min":{"1min":0.25600000000000001,"5min":0.17499999999999999,"15min":0.16600000000000001},"max":{"1min":18.314,"5min":56.613,"15min":117.01600000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.9730000000000001,"5min":1.659,"15min":2.1160000000000001},"min":{"1min":0.223,"5min":0.20799999999999999,"15min":0.17499999999999999},"max":{"1min":20.295999999999999,"5min":55.533999999999999,"15min":117.62},"last":4.4800000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:55:12 2024","interfaces":[{"interface":"back","average":{"1min":9.3979999999999997,"5min":3.2040000000000002,"15min":2.04},"min":{"1min":0.20499999999999999,"5min":0.19,"15min":0.19},"max":{"1min":169.607,"5min":169.607,"15min":169.607},"last":3.8540000000000001},{"interface":"front","average":{"1min":8.8230000000000004,"5min":2.9990000000000001,"15min":1.982},"min":{"1min":0.245,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":169.762,"5min":169.762,"15min":169.762},"last":3.3250000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:54:25 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0609999999999999,"15min":1.579},"min":{"1min":0.23300000000000001,"5min":0.223,"15min":0.191},"max":{"1min":3.0680000000000001,"5min":5.1859999999999999,"15min":116.98},"last":3.774},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.92200000000000004,"15min":1.6200000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.22},"max":{"1min":5.9029999999999996,"5min":5.9029999999999996,"15min":117.56699999999999},"last":2.1579999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.2190000000000001,"15min":1.1279999999999999},"min":{"1min":0.33600000000000002,"5min":0.28699999999999998,"15min":0.26200000000000001},"max":{"1min":2.6640000000000001,"5min":7.9450000000000003,"15min":11.19},"last":0.50700000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.101,"15min":1.0189999999999999},"min":{"1min":0.28699999999999998,"5min":0.28399999999999997,"15min":0.251},"max":{"1min":3.5819999999999999,"5min":6.952,"15min":7.7169999999999996},"last":0.56499999999999995}]},{"osd":5,"last update":"Wed Sep 11 08:55:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.1930000000000001,"15min":1.635},"min":{"1min":0.33200000000000002,"5min":0.314,"15min":0.27600000000000002},"max":{"1min":3.7690000000000001,"5min":5.9880000000000004,"15min":103.337},"last":2.218},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.306,"15min":1.643},"min":{"1min":0.374,"5min":0.33100000000000002,"15min":0.27400000000000002},"max":{"1min":7.3220000000000001,"5min":8.3810000000000002,"15min":103.163},"last":0.47199999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:54:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.25,"15min":1.0720000000000001},"min":{"1min":0.317,"5min":0.30499999999999999,"15min":0.29399999999999998},"max":{"1min":3.6309999999999998,"5min":6.2249999999999996,"15min":8.9149999999999991},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.004,"5min":1.208,"15min":1.2090000000000001},"min":{"1min":0.33300000000000002,"5min":0.315,"15min":0.31},"max":{"1min":4.367,"5min":6.2869999999999999,"15min":9.5199999999999996},"last":2.1920000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.1910000000000001,"15min":1.155},"min":{"1min":0.39500000000000002,"5min":0.37,"15min":0.32800000000000001},"max":{"1min":3.4540000000000002,"5min":8.3390000000000004,"15min":11.315},"last":0.53400000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.198,"15min":1.1100000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.315},"max":{"1min":3.3769999999999998,"5min":5.9089999999999998,"15min":10.266},"last":0.58999999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738605,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4123596,"kb_used_data":3533700,"kb_used_omap":1,"kb_used_meta":589886,"kb_avail":89621556,"statfs":{"total":95995035648,"available":91772473344,"internally_reserved":0,"allocated":3618508800,"data_stored":3618185069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":604043608},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:54:46 2024","interfaces":[{"interface":"back","average":{"1min":1.7549999999999999,"5min":1.74,"15min":1.5720000000000001},"min":{"1min":0.34799999999999998,"5min":0.21099999999999999,"15min":0.16300000000000001},"max":{"1min":6.2800000000000002,"5min":56.685000000000002,"15min":56.685000000000002},"last":0.24399999999999999},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.899,"15min":1.5309999999999999},"min":{"1min":0.188,"5min":0.188,"15min":0.186},"max":{"1min":5.032,"5min":61.012999999999998,"15min":61.012999999999998},"last":0.192}]},{"osd":2,"last update":"Wed Sep 11 08:55:13 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.0129999999999999,"15min":1.3009999999999999},"min":{"1min":0.22700000000000001,"5min":0.223,"15min":0.21099999999999999},"max":{"1min":5.7000000000000002,"5min":8.1059999999999999,"15min":19.940000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.044,"15min":1.3700000000000001},"min":{"1min":0.30099999999999999,"5min":0.22,"15min":0.20000000000000001},"max":{"1min":4.8639999999999999,"5min":5.4279999999999999,"15min":20.757000000000001},"last":0.30099999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:54:31 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.657,"15min":1.6200000000000001},"min":{"1min":0.33300000000000002,"5min":0.218,"15min":0.19800000000000001},"max":{"1min":4.0039999999999996,"5min":56.095999999999997,"15min":56.095999999999997},"last":0.378},{"interface":"front","average":{"1min":1.53,"5min":1.5860000000000001,"15min":1.554},"min":{"1min":0.26400000000000001,"5min":0.21199999999999999,"15min":0.20899999999999999},"max":{"1min":5.4770000000000003,"5min":59.651000000000003,"15min":59.651000000000003},"last":0.41799999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:54:46 2024","interfaces":[{"interface":"back","average":{"1min":9.5510000000000002,"5min":2.6040000000000001,"15min":1.7190000000000001},"min":{"1min":0.51100000000000001,"5min":0.30099999999999999,"15min":0.25},"max":{"1min":168.72200000000001,"5min":168.72200000000001,"15min":168.72200000000001},"last":2.8610000000000002},{"interface":"front","average":{"1min":9.5050000000000008,"5min":2.7010000000000001,"15min":1.704},"min":{"1min":0.46999999999999997,"5min":0.316,"15min":0.255},"max":{"1min":168.88999999999999,"5min":168.88999999999999,"15min":168.88999999999999},"last":1.1100000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:55:13 2024","interfaces":[{"interface":"back","average":{"1min":1.681,"5min":1.296,"15min":1.3029999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":7.7830000000000004,"5min":7.7830000000000004,"15min":7.7830000000000004},"last":2.9159999999999999},{"interface":"front","average":{"1min":1.7430000000000001,"5min":1.296,"15min":1.3380000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.28299999999999997},"max":{"1min":9.5239999999999991,"5min":9.5239999999999991,"15min":9.5239999999999991},"last":0.32800000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:54:31 2024","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.171,"15min":1.2989999999999999},"min":{"1min":0.41999999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":5.399,"5min":5.399,"15min":7.4240000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":1.754,"5min":1.1559999999999999,"15min":1.319},"min":{"1min":0.45700000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.6900000000000004,"5min":5.6900000000000004,"15min":7.4489999999999998},"last":0.85199999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:55:02 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.2230000000000001,"15min":1.522},"min":{"1min":0.48599999999999999,"5min":0.33500000000000002,"15min":0.313},"max":{"1min":5.0739999999999998,"5min":5.0739999999999998,"15min":56.119},"last":0.437},{"interface":"front","average":{"1min":2.327,"5min":1.5549999999999999,"15min":1.6579999999999999},"min":{"1min":0.38100000000000001,"5min":0.36399999999999999,"15min":0.307},"max":{"1min":9.9459999999999997,"5min":9.9459999999999997,"15min":56.100000000000001},"last":2.6640000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3589083136,"data_stored":3589079064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1072627712,"data_stored":1072627712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2209021952,"data_stored":2209021952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1834942464,"data_stored":1834942464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2885292032,"data_stored":2885287960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1801453568,"data_stored":1801453568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2536640512,"data_stored":2536636440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1444544512,"data_stored":1444544512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:55:21.400 INFO:tasks.thrashosds.thrasher:Setting 3.1 to [5, 4, 2] 2024-09-11T08:55:21.400 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1', '5', '4', '2'] 2024-09-11T08:55:21.400 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pg-upmap 3.1 5 4 2 2024-09-11T08:55:21.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:21.407 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:21.407 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:21.508 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:21.550 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:21.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:21.551 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:21.630 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:55:21.652 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:21.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:21 smithi145 ceph-mon[33384]: from='client.25381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:55:21.707 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:21.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:21.708 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:21.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:21.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:21 smithi092 ceph-mon[41114]: from='client.25381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:55:21.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:21 smithi092 ceph-mon[29647]: from='client.25381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:55:21.816 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:55:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:55:21] "GET /metrics HTTP/1.1" 200 224808 "" "Prometheus/2.43.0" 2024-09-11T08:55:21.847 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:21.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:21.848 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:21.937 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:21.949 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:21.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:21.980 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:21.981 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:22.082 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:22.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:22.123 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:22.124 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:22.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:22.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:22.284 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:22.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:22.422 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:22.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:22.424 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:22.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:22.557 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:22.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:22.558 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:22.607 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:22 smithi145 ceph-mon[33384]: pgmap v750: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.3 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 725 B/s rd, 0 B/s wr, 829 op/s 2024-09-11T08:55:22.610 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:22 smithi092 ceph-mon[29647]: pgmap v750: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.3 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 725 B/s rd, 0 B/s wr, 829 op/s 2024-09-11T08:55:22.610 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:22 smithi092 ceph-mon[41114]: pgmap v750: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.3 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 725 B/s rd, 0 B/s wr, 829 op/s 2024-09-11T08:55:22.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:22.696 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:22.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:22.698 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:22.728 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:22.800 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:22.828 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:22.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:22.829 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:22.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:55:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:55:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:55:22.930 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:22.967 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:22.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:22.968 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:23.069 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:23.105 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:23.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:23.106 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:23.207 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:23.243 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:23.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:23.244 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:23.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:23.362 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:55:23.380 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:23.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:23.382 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:23.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:23.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:23 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4224789289' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:55:23.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:23 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:55:23.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:23 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4224789289' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:55:23.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:23 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:55:23.595 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:23.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:23.596 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:23.608 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:55:23.608 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:55:23.608 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:23 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4224789289' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:55:23.609 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:23 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:55:23.697 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:23.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:23.730 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:23.730 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:23.831 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:23.864 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:23.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:23.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:23.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:24.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:24.006 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:24.006 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:24.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:24.138 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:24.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:24.139 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:24.240 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:24.270 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:24.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:24.272 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:24.372 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:24.400 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:24.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:24.402 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:24.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:24.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:24.541 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:24.541 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:24.642 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:24 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:55:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:24 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:55:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:24 smithi145 ceph-mon[33384]: pgmap v752: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 730 op/s 2024-09-11T08:55:24.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:24 smithi145 ceph-mon[33384]: osdmap e334: 8 total, 8 up, 8 in 2024-09-11T08:55:24.675 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:24.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:24.676 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:24.776 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:24.804 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:24.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:24.805 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:24.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:24 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:55:24.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:24 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:55:24.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:24 smithi092 ceph-mon[29647]: pgmap v752: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 730 op/s 2024-09-11T08:55:24.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:24 smithi092 ceph-mon[29647]: osdmap e334: 8 total, 8 up, 8 in 2024-09-11T08:55:24.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:24 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:55:24.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:24 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:55:24.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:24 smithi092 ceph-mon[41114]: pgmap v752: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 730 op/s 2024-09-11T08:55:24.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:24 smithi092 ceph-mon[41114]: osdmap e334: 8 total, 8 up, 8 in 2024-09-11T08:55:24.906 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:24.927 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:24.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:24.928 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:25.029 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:25.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:25.066 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:25.066 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:25.167 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:25.204 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:25.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:25.205 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:25.306 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:25.327 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:25.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:25.328 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:25.429 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:25.465 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:25.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:25.467 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:25.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:25.605 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:25.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:25.606 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:25.707 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:25.747 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:25.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:25.748 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:25.849 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:25.889 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:25.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:25.890 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:25.919 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-11T08:55:25.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:26.024 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:26.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:26.025 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:26.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:26.154 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:26.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:26.156 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:26.256 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:26.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:26.306 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:26.307 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:26.375 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:55:26.407 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:26.458 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:26.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:26.460 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:26.560 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:26.590 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:26.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:26.592 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:26.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:26 smithi092 ceph-mon[29647]: pgmap v753: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 727 B/s rd, 0 B/s wr, 623 op/s 2024-09-11T08:55:26.678 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:26 smithi092 ceph-mon[41114]: pgmap v753: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 727 B/s rd, 0 B/s wr, 623 op/s 2024-09-11T08:55:26.693 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:26.699 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:26 smithi145 ceph-mon[33384]: pgmap v753: 158 pgs: 2 active+clean+scrubbing+deep, 156 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 727 B/s rd, 0 B/s wr, 623 op/s 2024-09-11T08:55:26.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:26.737 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:26.737 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:26.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:26.872 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:26.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:26.873 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:26.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:27.009 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:27.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:27.013 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:27.114 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:27.148 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:27.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:27.149 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:27.249 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:27.286 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:27.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:27.287 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:27.387 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:27.388 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:55:27 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:55:27.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:27.427 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:27.427 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:27.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:27.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:27.571 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:27.571 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:27.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:27.704 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:27.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:27.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:27.806 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:27.859 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:27.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:27.860 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:27.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:27.995 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:27.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:27.996 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:28.097 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:28.138 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:28.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:28.139 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:28.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:28.275 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:28.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:28.276 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:28.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:28.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:28.410 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:28.411 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:28.418 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:55:28.418 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:55:28.491 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:55:28.512 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:28.546 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:28.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:28.547 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:28.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:28.678 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:28.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:28.679 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:28.707 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:55:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:55:28.708 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:28 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:28.708 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:28 smithi092 ceph-mon[29647]: pgmap v754: 158 pgs: 158 active+clean; 4.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:55:28.709 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:28 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:28.709 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:28 smithi092 ceph-mon[41114]: pgmap v754: 158 pgs: 158 active+clean; 4.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:55:28.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:55:28.743 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:55:28.777 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:55:28.777 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:55:28.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:28.810 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:55:28.811 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:55:28.823 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:28 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:28.823 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:28 smithi145 ceph-mon[33384]: pgmap v754: 158 pgs: 158 active+clean; 4.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:55:28.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:28.833 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:28.833 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:28.841 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:55:28.841 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:55:28.872 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:55:28.873 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:55:28.914 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:55:28.914 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:55:28.934 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:28.951 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:55:28.951 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:55:28.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:28.966 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:28.966 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:29.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:29.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:29.111 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:29.111 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:29.212 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:29.251 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-11T08:55:29.254 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:29.267 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:29.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:29.271 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:29.372 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:29.418 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:29.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:29.419 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:29.520 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:29.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:29.557 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:29.557 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:29.606 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:29 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1023094489' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:29.606 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:29 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4048586138' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.1", "id": [5, 4, 2]}]: dispatch 2024-09-11T08:55:29.658 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:29.707 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:29.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:29.709 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:29.715 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:55:29.809 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:29.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:29 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1023094489' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:29.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:29 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4048586138' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.1", "id": [5, 4, 2]}]: dispatch 2024-09-11T08:55:29.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:29 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1023094489' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:29.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:29 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4048586138' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.1", "id": [5, 4, 2]}]: dispatch 2024-09-11T08:55:29.843 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:29.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:29.845 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:29.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:29.985 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:29.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:29.986 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:30.063 INFO:teuthology.orchestra.run.smithi092.stderr:set 3.1 pg_upmap mapping to [5,4,2] 2024-09-11T08:55:30.087 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:30.121 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:30.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:30.122 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:30.223 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:30.255 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:30.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:30.257 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:30.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:30.396 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:30.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:30.397 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:30.498 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:30.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:30.533 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:30.533 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:30.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:30.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:30.667 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:30.667 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:30.719 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:30 smithi145 ceph-mon[33384]: pgmap v755: 158 pgs: 158 active+clean; 4.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.00k op/s 2024-09-11T08:55:30.719 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:30 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4048586138' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.1", "id": [5, 4, 2]}]': finished 2024-09-11T08:55:30.719 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:30 smithi145 ceph-mon[33384]: osdmap e335: 8 total, 8 up, 8 in 2024-09-11T08:55:30.719 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:55:30.768 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:30.800 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:30.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:30.801 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:30 smithi092 ceph-mon[29647]: pgmap v755: 158 pgs: 158 active+clean; 4.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.00k op/s 2024-09-11T08:55:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:30 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4048586138' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.1", "id": [5, 4, 2]}]': finished 2024-09-11T08:55:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:30 smithi092 ceph-mon[29647]: osdmap e335: 8 total, 8 up, 8 in 2024-09-11T08:55:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:55:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:30 smithi092 ceph-mon[41114]: pgmap v755: 158 pgs: 158 active+clean; 4.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.00k op/s 2024-09-11T08:55:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:30 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4048586138' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.1", "id": [5, 4, 2]}]': finished 2024-09-11T08:55:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:30 smithi092 ceph-mon[41114]: osdmap e335: 8 total, 8 up, 8 in 2024-09-11T08:55:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:55:30.903 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:30.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:30.935 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:30.935 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:31.036 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:31.066 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:31.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:31.067 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:31.168 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:31.202 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:31.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:31.204 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:31.304 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:31.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:31.342 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:31.342 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:31.443 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:31.486 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:31.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:31.487 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:31.588 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:31.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:31.635 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:31.635 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:31.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:31.767 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:31.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:31.768 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:31.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:55:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:55:31] "GET /metrics HTTP/1.1" 200 224817 "" "Prometheus/2.43.0" 2024-09-11T08:55:31.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:31.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:55:31.901 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:31.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:31.902 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:32.003 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:32.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:32.078 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:32.079 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:32.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:32.186 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:32 smithi092 ceph-mon[29647]: osdmap e336: 8 total, 8 up, 8 in 2024-09-11T08:55:32.187 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:32 smithi092 ceph-mon[29647]: pgmap v758: 158 pgs: 158 active+clean; 4.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 943 op/s 2024-09-11T08:55:32.187 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:32 smithi092 ceph-mon[41114]: osdmap e336: 8 total, 8 up, 8 in 2024-09-11T08:55:32.187 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:32 smithi092 ceph-mon[41114]: pgmap v758: 158 pgs: 158 active+clean; 4.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 943 op/s 2024-09-11T08:55:32.229 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:32.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:32.233 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:32.333 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:32.362 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:32.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:32.363 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:32.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:32 smithi145 ceph-mon[33384]: osdmap e336: 8 total, 8 up, 8 in 2024-09-11T08:55:32.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:32 smithi145 ceph-mon[33384]: pgmap v758: 158 pgs: 158 active+clean; 4.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 943 op/s 2024-09-11T08:55:32.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:32.504 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:32.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:32.505 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:32.606 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:32.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:32.648 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:32.743 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:32.750 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:32.777 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:32.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:32.778 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:32.879 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:32.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:55:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:55:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:55:32.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:32.923 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:32.923 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:33.024 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:33.059 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:33.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:33.060 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:33.101 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:55:33.161 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:33.196 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:33.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:33.197 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:33.215 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:33 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/792088885' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:55:33.215 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:33 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:55:33.216 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:33 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/792088885' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:55:33.216 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:33 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:55:33.298 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:33.329 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:33.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:33.330 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:33.403 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:33 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/792088885' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:55:33.403 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:33 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:55:33.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:33.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:33.464 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:33.565 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:33.603 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:33.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:33.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:33.705 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:33.745 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:33.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:33.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:33.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:33.872 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:33.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:33.873 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:33.974 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:34.012 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:34.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:34.014 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:34.114 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:34.159 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:34.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:34.160 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:34.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:34.291 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:34 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:55:34.291 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:34 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:55:34.291 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:34 smithi145 ceph-mon[33384]: osdmap e337: 8 total, 8 up, 8 in 2024-09-11T08:55:34.291 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:34 smithi145 ceph-mon[33384]: pgmap v760: 158 pgs: 158 active+clean; 4.4 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-11T08:55:34.292 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:34 smithi145 ceph-mon[33384]: 1.8 scrub starts 2024-09-11T08:55:34.292 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:34 smithi145 ceph-mon[33384]: 1.8 scrub ok 2024-09-11T08:55:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:34.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:34.299 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:34.400 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:34.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:34.434 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:34.435 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:34.449 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:55:34.449 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:55:34.449 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.449 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:205365 10.7 10:e4a79d32:::benchmark_data_smithi092_173109_object68406:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e337)", 2024-09-11T08:55:34.449 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:55:34.436062+0000", 2024-09-11T08:55:34.449 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.012300393999999999, 2024-09-11T08:55:34.449 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.012324224, 2024-09-11T08:55:34.449 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:34.450 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:55:34.450 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:34.450 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:55:34.450 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:55:34.450 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 205365 2024-09-11T08:55:34.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.450 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:34.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:34.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.436062+0000", 2024-09-11T08:55:34.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:34.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:34.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.436062+0000", 2024-09-11T08:55:34.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:34.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:34.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.436061+0000", 2024-09-11T08:55:34.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:55:34.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:34.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.436066+0000", 2024-09-11T08:55:34.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6729999999999997e-06 2024-09-11T08:55:34.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:34.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.436081+0000", 2024-09-11T08:55:34.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4017e-05 2024-09-11T08:55:34.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:34.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.436088+0000", 2024-09-11T08:55:34.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.593e-06 2024-09-11T08:55:34.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:34.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.442627+0000", 2024-09-11T08:55:34.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0065390589999999998 2024-09-11T08:55:34.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:34.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.442737+0000", 2024-09-11T08:55:34.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000109817 2024-09-11T08:55:34.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,5", 2024-09-11T08:55:34.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.442850+0000", 2024-09-11T08:55:34.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000113422 2024-09-11T08:55:34.453 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:34.453 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:34.454 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:34.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.454 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:205369 10.7 10:e89b328d:::benchmark_data_smithi092_173109_object68410:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e337)", 2024-09-11T08:55:34.454 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:55:34.439327+0000", 2024-09-11T08:55:34.454 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.0090354140000000003, 2024-09-11T08:55:34.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0091059639999999994, 2024-09-11T08:55:34.454 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:34.454 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "waiting for sub ops", 2024-09-11T08:55:34.454 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:34.454 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:55:34.455 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:55:34.455 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 205369 2024-09-11T08:55:34.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.455 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:34.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.455 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:34.455 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.439327+0000", 2024-09-11T08:55:34.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:34.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.456 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:34.456 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.439327+0000", 2024-09-11T08:55:34.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:34.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.456 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.456 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:34.456 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.439327+0000", 2024-09-11T08:55:34.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:55:34.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.456 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.456 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:34.456 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.439330+0000", 2024-09-11T08:55:34.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6119999999999999e-06 2024-09-11T08:55:34.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:34.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.439337+0000", 2024-09-11T08:55:34.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4910000000000002e-06 2024-09-11T08:55:34.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:34.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.439341+0000", 2024-09-11T08:55:34.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2660000000000003e-06 2024-09-11T08:55:34.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:34.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.446086+0000", 2024-09-11T08:55:34.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0067448639999999997 2024-09-11T08:55:34.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:34.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.446196+0000", 2024-09-11T08:55:34.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000110064 2024-09-11T08:55:34.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,5", 2024-09-11T08:55:34.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.446310+0000", 2024-09-11T08:55:34.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000114233 2024-09-11T08:55:34.459 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:34.459 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:34.459 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:34.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.459 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop_reply(client.15621.0:205365 10.7 e337/293)", 2024-09-11T08:55:34.459 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:55:34.445826+0000", 2024-09-11T08:55:34.459 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.0025370119999999999, 2024-09-11T08:55:34.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0026444179999999999, 2024-09-11T08:55:34.460 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:34.460 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "queued for pg", 2024-09-11T08:55:34.460 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:34.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:34.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.445826+0000", 2024-09-11T08:55:34.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:34.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:34.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.445826+0000", 2024-09-11T08:55:34.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:34.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:34.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.445825+0000", 2024-09-11T08:55:34.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:34.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:34.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.445829+0000", 2024-09-11T08:55:34.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6190000000000002e-06 2024-09-11T08:55:34.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.461 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:34.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.445841+0000", 2024-09-11T08:55:34.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1508000000000001e-05 2024-09-11T08:55:34.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:34.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.445846+0000", 2024-09-11T08:55:34.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.48e-06 2024-09-11T08:55:34.462 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:34.462 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:34.462 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:34.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.462 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop_reply(client.15621.0:205365 10.7 e337/293)", 2024-09-11T08:55:34.462 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:55:34.446685+0000", 2024-09-11T08:55:34.462 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.0016779049999999999, 2024-09-11T08:55:34.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001808481, 2024-09-11T08:55:34.463 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:34.463 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "queued for pg", 2024-09-11T08:55:34.463 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:34.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:34.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.446685+0000", 2024-09-11T08:55:34.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:34.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:34.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.446685+0000", 2024-09-11T08:55:34.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:34.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:34.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.446684+0000", 2024-09-11T08:55:34.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:55:34.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:34.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.446688+0000", 2024-09-11T08:55:34.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9689999999999996e-06 2024-09-11T08:55:34.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:34.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.446698+0000", 2024-09-11T08:55:34.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.8409999999999995e-06 2024-09-11T08:55:34.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:34.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:34.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:34.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:34.446703+0000", 2024-09-11T08:55:34.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4519999999999999e-06 2024-09-11T08:55:34.465 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:34.465 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:34.465 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:34.466 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:34.466 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2024-09-11T08:55:34.466 INFO:teuthology.orchestra.run.smithi145.stdout: "num_ops": 4 2024-09-11T08:55:34.466 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:55:34.478 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:34 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:55:34.478 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:34 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:55:34.478 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:34 smithi092 ceph-mon[29647]: osdmap e337: 8 total, 8 up, 8 in 2024-09-11T08:55:34.478 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:34 smithi092 ceph-mon[29647]: pgmap v760: 158 pgs: 158 active+clean; 4.4 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-11T08:55:34.478 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:34 smithi092 ceph-mon[29647]: 1.8 scrub starts 2024-09-11T08:55:34.478 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:34 smithi092 ceph-mon[29647]: 1.8 scrub ok 2024-09-11T08:55:34.479 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:34 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:55:34.479 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:34 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:55:34.479 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:34 smithi092 ceph-mon[41114]: osdmap e337: 8 total, 8 up, 8 in 2024-09-11T08:55:34.479 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:34 smithi092 ceph-mon[41114]: pgmap v760: 158 pgs: 158 active+clean; 4.4 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-11T08:55:34.479 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:34 smithi092 ceph-mon[41114]: 1.8 scrub starts 2024-09-11T08:55:34.479 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:34 smithi092 ceph-mon[41114]: 1.8 scrub ok 2024-09-11T08:55:34.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:34.567 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:34.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:34.568 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:34.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:34.696 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:34.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:34.697 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:34.798 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:34.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:34.840 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:34.840 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:34.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:34.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:34.973 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:34.974 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:35.074 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:35.117 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:35.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:35.118 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:35.173 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-11T08:55:35.219 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:35.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:35.278 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:35.278 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:35.379 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:35.399 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[29647]: 4.7 scrub starts 2024-09-11T08:55:35.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[29647]: 4.7 scrub ok 2024-09-11T08:55:35.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[29647]: 5.10 scrub starts 2024-09-11T08:55:35.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[29647]: 5.10 scrub ok 2024-09-11T08:55:35.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[29647]: 5.6 scrub starts 2024-09-11T08:55:35.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[29647]: 5.6 scrub ok 2024-09-11T08:55:35.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[29647]: 5.12 scrub starts 2024-09-11T08:55:35.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[29647]: 5.12 scrub ok 2024-09-11T08:55:35.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[29647]: 4.b scrub starts 2024-09-11T08:55:35.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[29647]: 4.b scrub ok 2024-09-11T08:55:35.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[29647]: 10.9 scrub starts 2024-09-11T08:55:35.400 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[41114]: 4.7 scrub starts 2024-09-11T08:55:35.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[41114]: 4.7 scrub ok 2024-09-11T08:55:35.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[41114]: 5.10 scrub starts 2024-09-11T08:55:35.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[41114]: 5.10 scrub ok 2024-09-11T08:55:35.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[41114]: 5.6 scrub starts 2024-09-11T08:55:35.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[41114]: 5.6 scrub ok 2024-09-11T08:55:35.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[41114]: 5.12 scrub starts 2024-09-11T08:55:35.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[41114]: 5.12 scrub ok 2024-09-11T08:55:35.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[41114]: 4.b scrub starts 2024-09-11T08:55:35.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[41114]: 4.b scrub ok 2024-09-11T08:55:35.401 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:35 smithi092 ceph-mon[41114]: 10.9 scrub starts 2024-09-11T08:55:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:35 smithi145 ceph-mon[33384]: 4.7 scrub starts 2024-09-11T08:55:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:35 smithi145 ceph-mon[33384]: 4.7 scrub ok 2024-09-11T08:55:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:35 smithi145 ceph-mon[33384]: 5.10 scrub starts 2024-09-11T08:55:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:35 smithi145 ceph-mon[33384]: 5.10 scrub ok 2024-09-11T08:55:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:35 smithi145 ceph-mon[33384]: 5.6 scrub starts 2024-09-11T08:55:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:35 smithi145 ceph-mon[33384]: 5.6 scrub ok 2024-09-11T08:55:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:35 smithi145 ceph-mon[33384]: 5.12 scrub starts 2024-09-11T08:55:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:35 smithi145 ceph-mon[33384]: 5.12 scrub ok 2024-09-11T08:55:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:35 smithi145 ceph-mon[33384]: 4.b scrub starts 2024-09-11T08:55:35.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:35 smithi145 ceph-mon[33384]: 4.b scrub ok 2024-09-11T08:55:35.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:35 smithi145 ceph-mon[33384]: 10.9 scrub starts 2024-09-11T08:55:35.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:35.422 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:35.422 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:35.428 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:55:35.429 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:55:35.464 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:55:35.464 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:55:35.492 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:55:35.492 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:55:35.521 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:55:35.521 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:55:35.522 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:35.552 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:55:35.553 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:55:35.565 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:35.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:35.566 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:35.576 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:55:35.576 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:55:35.612 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:55:35.612 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:55:35.639 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:55:35.642 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:55:35.642 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:55:35.667 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:35.709 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:35.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:35.711 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:35.811 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:35.851 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:35.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:35.852 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:35.952 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:35.999 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:36.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:36.000 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:36.069 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:55:36.069 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:55:36.069 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-11T08:55:36.069 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T08:55:36.069 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T08:55:36.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 config set filestore_inject_stall 3 2024-09-11T08:55:36.102 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:36.134 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:36.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:36.135 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:36.236 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:36.269 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:36.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:36.270 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:36.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:36 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:55:36.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:36 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:55:36.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:36 smithi092 ceph-mon[29647]: osdmap e338: 8 total, 8 up, 8 in 2024-09-11T08:55:36.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:36 smithi092 ceph-mon[29647]: 1.6 scrub starts 2024-09-11T08:55:36.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:36 smithi092 ceph-mon[29647]: 1.6 scrub ok 2024-09-11T08:55:36.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:36 smithi092 ceph-mon[29647]: pgmap v762: 158 pgs: 158 active+clean; 4.4 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 966 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-11T08:55:36.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:36 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1901635193' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:36.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:36 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:55:36.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:36 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:55:36.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:36 smithi092 ceph-mon[41114]: osdmap e338: 8 total, 8 up, 8 in 2024-09-11T08:55:36.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:36 smithi092 ceph-mon[41114]: 1.6 scrub starts 2024-09-11T08:55:36.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:36 smithi092 ceph-mon[41114]: 1.6 scrub ok 2024-09-11T08:55:36.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:36 smithi092 ceph-mon[41114]: pgmap v762: 158 pgs: 158 active+clean; 4.4 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 966 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-11T08:55:36.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:36 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1901635193' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:36.371 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:36.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:36 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:55:36.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:36 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:55:36.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:36 smithi145 ceph-mon[33384]: osdmap e338: 8 total, 8 up, 8 in 2024-09-11T08:55:36.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:36 smithi145 ceph-mon[33384]: 1.6 scrub starts 2024-09-11T08:55:36.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:36 smithi145 ceph-mon[33384]: 1.6 scrub ok 2024-09-11T08:55:36.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:36 smithi145 ceph-mon[33384]: pgmap v762: 158 pgs: 158 active+clean; 4.4 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 966 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-11T08:55:36.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:36 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1901635193' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:36.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:36.414 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:36.414 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:36.515 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:36.559 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:36.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:36.560 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:36.573 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:36.661 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:36.692 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:36.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:36.693 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:36.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:36.827 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:36.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:36.827 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:36.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:55:36.929 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:36.958 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:36.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:36.959 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:37.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:37.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:37.114 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:37.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:37.215 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:37.221 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:37 smithi145 ceph-mon[33384]: 4.1b scrub starts 2024-09-11T08:55:37.222 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:37 smithi145 ceph-mon[33384]: 4.1b scrub ok 2024-09-11T08:55:37.222 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:37 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:55:37.222 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:37 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:55:37.222 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:37 smithi145 ceph-mon[33384]: 10.9 scrub ok 2024-09-11T08:55:37.222 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:37 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T08:55:37.222 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:37 smithi145 ceph-mon[33384]: 4.1c scrub ok 2024-09-11T08:55:37.222 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:55:37 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:55:37.253 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:37.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:37.254 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:37.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:37 smithi092 ceph-mon[29647]: 4.1b scrub starts 2024-09-11T08:55:37.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:37 smithi092 ceph-mon[29647]: 4.1b scrub ok 2024-09-11T08:55:37.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:37 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:55:37.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:37 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:55:37.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:37 smithi092 ceph-mon[29647]: 10.9 scrub ok 2024-09-11T08:55:37.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:37 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T08:55:37.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:37 smithi092 ceph-mon[29647]: 4.1c scrub ok 2024-09-11T08:55:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:37 smithi092 ceph-mon[41114]: 4.1b scrub starts 2024-09-11T08:55:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:37 smithi092 ceph-mon[41114]: 4.1b scrub ok 2024-09-11T08:55:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:37 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:55:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:37 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:55:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:37 smithi092 ceph-mon[41114]: 10.9 scrub ok 2024-09-11T08:55:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:37 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T08:55:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:37 smithi092 ceph-mon[41114]: 4.1c scrub ok 2024-09-11T08:55:37.355 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:37.401 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:37.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:37.402 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:37.502 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:37.541 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:37.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:37.542 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:37.643 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:37.671 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:37.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:37.672 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:37.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:37.807 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:37.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:37.808 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:37.909 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:37.945 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:37.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:37.946 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:38.043 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:38.047 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:38.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:38.079 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:38.079 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:38.180 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:38.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:38.202 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:38.202 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:38.303 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:38.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:38 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:38.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:38 smithi092 ceph-mon[29647]: osdmap e339: 8 total, 8 up, 8 in 2024-09-11T08:55:38.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:38 smithi092 ceph-mon[29647]: pgmap v764: 158 pgs: 158 active+clean; 4.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.75k op/s 2024-09-11T08:55:38.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:38 smithi092 ceph-mon[29647]: 4.14 scrub starts 2024-09-11T08:55:38.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:38 smithi092 ceph-mon[29647]: 4.14 scrub ok 2024-09-11T08:55:38.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:38 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:38.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:38 smithi092 ceph-mon[41114]: osdmap e339: 8 total, 8 up, 8 in 2024-09-11T08:55:38.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:38 smithi092 ceph-mon[41114]: pgmap v764: 158 pgs: 158 active+clean; 4.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.75k op/s 2024-09-11T08:55:38.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:38 smithi092 ceph-mon[41114]: 4.14 scrub starts 2024-09-11T08:55:38.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:38 smithi092 ceph-mon[41114]: 4.14 scrub ok 2024-09-11T08:55:38.336 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:38.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:38.337 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:38.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:38 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:38.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:38 smithi145 ceph-mon[33384]: osdmap e339: 8 total, 8 up, 8 in 2024-09-11T08:55:38.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:38 smithi145 ceph-mon[33384]: pgmap v764: 158 pgs: 158 active+clean; 4.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.75k op/s 2024-09-11T08:55:38.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:38 smithi145 ceph-mon[33384]: 4.14 scrub starts 2024-09-11T08:55:38.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:38 smithi145 ceph-mon[33384]: 4.14 scrub ok 2024-09-11T08:55:38.437 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:38.451 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:55:38.451 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:55:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:38.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:38.480 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:38.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:55:38.581 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:38.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:38.617 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:38.617 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:38.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:38.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:38.764 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:38.764 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:38.865 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:38.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:38.887 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:38.887 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:38.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:39.024 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:39.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:39.025 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:39.126 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:39.163 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:39.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:39.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:39.265 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:39.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:39.293 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:39.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:39.394 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:39.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:39 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:55:39.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:39 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:55:39.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:39.430 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:39.430 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:39.531 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:39 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:55:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:39 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:55:39.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:39 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:55:39.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:39 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:55:39.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:39.570 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:39.570 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:39.671 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:39.714 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:39.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:39.715 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:39.816 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:39.841 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:39.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:39.842 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:39.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:39.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:39.978 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:40.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:40.160 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:40.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:40.161 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:40.262 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:40.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:40.287 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:40.287 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:40 smithi092 ceph-mon[29647]: osdmap e340: 8 total, 8 up, 8 in 2024-09-11T08:55:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:40 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:55:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:40 smithi092 ceph-mon[29647]: pgmap v766: 158 pgs: 158 active+clean; 4.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-11T08:55:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:40 smithi092 ceph-mon[29647]: 3.8 scrub starts 2024-09-11T08:55:40.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:40 smithi092 ceph-mon[29647]: 3.8 scrub ok 2024-09-11T08:55:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:40 smithi092 ceph-mon[41114]: osdmap e340: 8 total, 8 up, 8 in 2024-09-11T08:55:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:40 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:55:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:40 smithi092 ceph-mon[41114]: pgmap v766: 158 pgs: 158 active+clean; 4.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-11T08:55:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:40 smithi092 ceph-mon[41114]: 3.8 scrub starts 2024-09-11T08:55:40.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:40 smithi092 ceph-mon[41114]: 3.8 scrub ok 2024-09-11T08:55:40.388 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:40.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:40 smithi145 ceph-mon[33384]: osdmap e340: 8 total, 8 up, 8 in 2024-09-11T08:55:40.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:40 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:55:40.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:40 smithi145 ceph-mon[33384]: pgmap v766: 158 pgs: 158 active+clean; 4.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-11T08:55:40.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:40 smithi145 ceph-mon[33384]: 3.8 scrub starts 2024-09-11T08:55:40.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:40 smithi145 ceph-mon[33384]: 3.8 scrub ok 2024-09-11T08:55:40.428 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:40.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:40.429 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:40.483 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:55:40.484 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "" 2024-09-11T08:55:40.484 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:55:40.530 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:40.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:40.578 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:40.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:40.679 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:40.720 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:40.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:40.721 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:40.821 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:40.862 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:40.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:40.864 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:40.935 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-11T08:55:40.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:40.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:40.996 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:40.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:41.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:41.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:41.131 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:41.232 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:41.297 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:41.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:41.299 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:41.369 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:55:41.400 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:41.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:41.442 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:41.442 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:41.505 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:41 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:55:41.505 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:41 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:55:41.505 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:41 smithi145 ceph-mon[33384]: 4.a scrub starts 2024-09-11T08:55:41.505 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:41 smithi145 ceph-mon[33384]: 4.a scrub ok 2024-09-11T08:55:41.505 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:41 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:55:41.506 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:41 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:55:41.506 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:41 smithi145 ceph-mon[33384]: 3.16 scrub starts 2024-09-11T08:55:41.506 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:41 smithi145 ceph-mon[33384]: 3.16 scrub ok 2024-09-11T08:55:41.543 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:41.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:55:41.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:55:41.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[29647]: 4.a scrub starts 2024-09-11T08:55:41.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[29647]: 4.a scrub ok 2024-09-11T08:55:41.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:55:41.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:55:41.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[29647]: 3.16 scrub starts 2024-09-11T08:55:41.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[29647]: 3.16 scrub ok 2024-09-11T08:55:41.567 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:55:41] "GET /metrics HTTP/1.1" 200 224817 "" "Prometheus/2.43.0" 2024-09-11T08:55:41.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:55:41.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:55:41.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[41114]: 4.a scrub starts 2024-09-11T08:55:41.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[41114]: 4.a scrub ok 2024-09-11T08:55:41.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:55:41.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:55:41.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[41114]: 3.16 scrub starts 2024-09-11T08:55:41.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:41 smithi092 ceph-mon[41114]: 3.16 scrub ok 2024-09-11T08:55:41.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:41.575 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:41.575 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:41.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:41.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:41.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:41.810 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:41.835 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:41.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:41.836 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:41.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:41.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:41.973 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:41.973 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:42.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:42.118 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:42.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:42.119 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:42.220 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:42.270 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:42.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:42.271 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:42.372 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:42.407 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:42.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:42.408 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:42.509 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:42.533 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:42.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:42.534 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:42.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:42 smithi092 ceph-mon[29647]: pgmap v767: 158 pgs: 158 active+clean; 3.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 816 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:55:42.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:42 smithi092 ceph-mon[29647]: 5.1f scrub starts 2024-09-11T08:55:42.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:42 smithi092 ceph-mon[29647]: 5.1f scrub ok 2024-09-11T08:55:42.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:42 smithi092 ceph-mon[29647]: 10.5 scrub starts 2024-09-11T08:55:42.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:42 smithi092 ceph-mon[29647]: 6.14 scrub starts 2024-09-11T08:55:42.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:42 smithi092 ceph-mon[29647]: 6.14 scrub ok 2024-09-11T08:55:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:42 smithi092 ceph-mon[41114]: pgmap v767: 158 pgs: 158 active+clean; 3.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 816 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:55:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:42 smithi092 ceph-mon[41114]: 5.1f scrub starts 2024-09-11T08:55:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:42 smithi092 ceph-mon[41114]: 5.1f scrub ok 2024-09-11T08:55:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:42 smithi092 ceph-mon[41114]: 10.5 scrub starts 2024-09-11T08:55:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:42 smithi092 ceph-mon[41114]: 6.14 scrub starts 2024-09-11T08:55:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:42 smithi092 ceph-mon[41114]: 6.14 scrub ok 2024-09-11T08:55:42.634 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:42.656 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:42.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:42.657 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:42.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:42 smithi145 ceph-mon[33384]: pgmap v767: 158 pgs: 158 active+clean; 3.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 816 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T08:55:42.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:42 smithi145 ceph-mon[33384]: 5.1f scrub starts 2024-09-11T08:55:42.658 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:42 smithi145 ceph-mon[33384]: 5.1f scrub ok 2024-09-11T08:55:42.658 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:42 smithi145 ceph-mon[33384]: 10.5 scrub starts 2024-09-11T08:55:42.658 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:42 smithi145 ceph-mon[33384]: 6.14 scrub starts 2024-09-11T08:55:42.658 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:42 smithi145 ceph-mon[33384]: 6.14 scrub ok 2024-09-11T08:55:42.758 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:42.801 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:42.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:42.802 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:42.903 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:42.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:55:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:55:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:55:42.932 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:42.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:42.933 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:43.034 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:43.077 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:43.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:43.079 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:43.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:43.219 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:43.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:43.220 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:43.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:43 smithi092 ceph-mon[29647]: 3.6 scrub starts 2024-09-11T08:55:43.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:43 smithi092 ceph-mon[29647]: 3.6 scrub ok 2024-09-11T08:55:43.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:43 smithi092 ceph-mon[29647]: 2.0 scrub starts 2024-09-11T08:55:43.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:43 smithi092 ceph-mon[29647]: 2.0 scrub ok 2024-09-11T08:55:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:43 smithi092 ceph-mon[29647]: 5.13 scrub starts 2024-09-11T08:55:43.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:43 smithi092 ceph-mon[29647]: 5.13 scrub ok 2024-09-11T08:55:43.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:43 smithi092 ceph-mon[41114]: 3.6 scrub starts 2024-09-11T08:55:43.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:43 smithi092 ceph-mon[41114]: 3.6 scrub ok 2024-09-11T08:55:43.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:43 smithi092 ceph-mon[41114]: 2.0 scrub starts 2024-09-11T08:55:43.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:43 smithi092 ceph-mon[41114]: 2.0 scrub ok 2024-09-11T08:55:43.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:43 smithi092 ceph-mon[41114]: 5.13 scrub starts 2024-09-11T08:55:43.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:43 smithi092 ceph-mon[41114]: 5.13 scrub ok 2024-09-11T08:55:43.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:43.333 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:43 smithi145 ceph-mon[33384]: 3.6 scrub starts 2024-09-11T08:55:43.333 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:43 smithi145 ceph-mon[33384]: 3.6 scrub ok 2024-09-11T08:55:43.333 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:43 smithi145 ceph-mon[33384]: 2.0 scrub starts 2024-09-11T08:55:43.333 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:43 smithi145 ceph-mon[33384]: 2.0 scrub ok 2024-09-11T08:55:43.333 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:43 smithi145 ceph-mon[33384]: 5.13 scrub starts 2024-09-11T08:55:43.333 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:43 smithi145 ceph-mon[33384]: 5.13 scrub ok 2024-09-11T08:55:43.353 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:43.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:43.354 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:43.455 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:43.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:43.497 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:43.497 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:43.598 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:43.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:43.630 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:43.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:43.771 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:43.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:43.772 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:43.873 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:43.902 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:43.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:43.903 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:44.004 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:44.037 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:44.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:44.038 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:44.097 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:55:44.097 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:55:44.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:44.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:44.174 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:44.174 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:44.275 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:44.317 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:44.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:44.318 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:44.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:44 smithi145 ceph-mon[33384]: 10.5 scrub ok 2024-09-11T08:55:44.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:44 smithi145 ceph-mon[33384]: pgmap v768: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 3.7 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-11T08:55:44.418 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:44.450 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:44.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:44.452 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:44.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:44.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:44 smithi092 ceph-mon[41114]: 10.5 scrub ok 2024-09-11T08:55:44.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:44 smithi092 ceph-mon[41114]: pgmap v768: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 3.7 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-11T08:55:44.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:44 smithi092 ceph-mon[29647]: 10.5 scrub ok 2024-09-11T08:55:44.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:44 smithi092 ceph-mon[29647]: pgmap v768: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 3.7 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-11T08:55:44.586 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:44.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:44.588 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:44.689 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:44.714 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:44.770 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:44.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:44.771 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:44.872 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:44.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:44.916 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:44.916 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:45.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:45.058 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:45.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:45.059 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:45.160 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:45.183 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T08:55:45.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:45.198 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:45.198 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:45.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:45.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:45.341 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:45.341 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:45 smithi145 ceph-mon[33384]: 3.19 scrub starts 2024-09-11T08:55:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:45 smithi145 ceph-mon[33384]: 3.19 scrub ok 2024-09-11T08:55:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:55:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:55:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:45 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:55:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:55:45.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:45.474 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:45.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:45.476 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:45.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:45 smithi092 ceph-mon[29647]: 3.19 scrub starts 2024-09-11T08:55:45.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:45 smithi092 ceph-mon[29647]: 3.19 scrub ok 2024-09-11T08:55:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:55:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:55:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:45 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:55:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:55:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:45 smithi092 ceph-mon[41114]: 3.19 scrub starts 2024-09-11T08:55:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:45 smithi092 ceph-mon[41114]: 3.19 scrub ok 2024-09-11T08:55:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:55:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:55:45.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:45 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:55:45.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:55:45.577 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:45.605 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:55:45.608 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:45.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:45.609 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:45.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:45.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:45.749 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:45.850 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:45.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:45.884 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:45.985 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:46.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:46.020 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:46.020 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:46.122 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:46.155 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:46.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:46.156 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:46.257 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:46.287 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:55:46.291 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:46.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:46.292 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:46.393 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:46.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:46 smithi145 ceph-mon[33384]: 3.13 scrub starts 2024-09-11T08:55:46.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:46 smithi145 ceph-mon[33384]: 3.13 scrub ok 2024-09-11T08:55:46.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:46 smithi145 ceph-mon[33384]: pgmap v769: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 3.7 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 621 B/s rd, 0 B/s wr, 733 op/s 2024-09-11T08:55:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:46 smithi145 ceph-mon[33384]: 2.1 scrub starts 2024-09-11T08:55:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:46 smithi145 ceph-mon[33384]: 2.1 scrub ok 2024-09-11T08:55:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:46 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1189243258' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:55:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:46 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:55:46.427 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:46.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:46.428 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:46.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:46 smithi092 ceph-mon[29647]: 3.13 scrub starts 2024-09-11T08:55:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:46 smithi092 ceph-mon[29647]: 3.13 scrub ok 2024-09-11T08:55:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:46 smithi092 ceph-mon[29647]: pgmap v769: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 3.7 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 621 B/s rd, 0 B/s wr, 733 op/s 2024-09-11T08:55:46.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:46 smithi092 ceph-mon[29647]: 2.1 scrub starts 2024-09-11T08:55:46.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:46 smithi092 ceph-mon[29647]: 2.1 scrub ok 2024-09-11T08:55:46.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:46 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1189243258' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:55:46.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:46 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:55:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:46 smithi092 ceph-mon[41114]: 3.13 scrub starts 2024-09-11T08:55:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:46 smithi092 ceph-mon[41114]: 3.13 scrub ok 2024-09-11T08:55:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:46 smithi092 ceph-mon[41114]: pgmap v769: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 3.7 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 621 B/s rd, 0 B/s wr, 733 op/s 2024-09-11T08:55:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:46 smithi092 ceph-mon[41114]: 2.1 scrub starts 2024-09-11T08:55:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:46 smithi092 ceph-mon[41114]: 2.1 scrub ok 2024-09-11T08:55:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:46 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1189243258' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:55:46.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:46 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:55:46.567 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:46.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:46.569 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:46.669 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:46.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:46.706 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:46.707 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:46.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:46.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:46.842 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:46.943 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:46.989 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:46.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:46.990 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:47.091 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:47.129 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:47.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:47.130 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:47.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:47.268 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:55:47 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:55:47.274 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:47.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:47.275 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:47.376 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:47.408 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:47.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:47.409 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:47.448 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:55:47.448 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:55:47.475 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:55:47.475 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:55:47.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:47.510 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:55:47.511 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:55:47.541 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:47 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:55:47.541 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:47 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:55:47.541 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:47 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:55:47.541 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:47 smithi092 ceph-mon[29647]: osdmap e341: 8 total, 8 up, 8 in 2024-09-11T08:55:47.541 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:47 smithi092 ceph-mon[29647]: 6.16 scrub starts 2024-09-11T08:55:47.542 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:47 smithi092 ceph-mon[29647]: 6.16 scrub ok 2024-09-11T08:55:47.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:47 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:55:47.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:47 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:55:47.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:47 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:55:47.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:47 smithi092 ceph-mon[41114]: osdmap e341: 8 total, 8 up, 8 in 2024-09-11T08:55:47.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:47 smithi092 ceph-mon[41114]: 6.16 scrub starts 2024-09-11T08:55:47.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:47 smithi092 ceph-mon[41114]: 6.16 scrub ok 2024-09-11T08:55:47.543 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:55:47.543 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:55:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:47.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:47.546 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:47.577 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:55:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:55:47.583 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:47 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:55:47.584 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:47 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:55:47.584 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:47 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:55:47.584 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:47 smithi145 ceph-mon[33384]: osdmap e341: 8 total, 8 up, 8 in 2024-09-11T08:55:47.584 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:47 smithi145 ceph-mon[33384]: 6.16 scrub starts 2024-09-11T08:55:47.584 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:47 smithi145 ceph-mon[33384]: 6.16 scrub ok 2024-09-11T08:55:47.609 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:55:47.609 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:55:47.646 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:47.647 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:55:47.647 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:55:47.675 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:55:47.675 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:55:47.682 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:47.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:47.684 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:47.742 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:55:47.742 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:55:47.742 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.743 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15621.0:221097 10.1 e341/293 10:82dad571:::benchmark_data_smithi092_173109_object84138:head v 341'13845, mlcod=341'13843)", 2024-09-11T08:55:47.743 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:55:47.734752+0000", 2024-09-11T08:55:47.743 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.0071084110000000002, 2024-09-11T08:55:47.743 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.007135799, 2024-09-11T08:55:47.743 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:47.743 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T08:55:47.743 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:47.743 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.743 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:47.744 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.734752+0000", 2024-09-11T08:55:47.744 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:47.744 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.745 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.745 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:47.745 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.734752+0000", 2024-09-11T08:55:47.745 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:47.745 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.745 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.745 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:47.745 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.734751+0000", 2024-09-11T08:55:47.745 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.734759+0000", 2024-09-11T08:55:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2490000000000004e-06 2024-09-11T08:55:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.734771+0000", 2024-09-11T08:55:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2605999999999999e-05 2024-09-11T08:55:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.734777+0000", 2024-09-11T08:55:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7509999999999997e-06 2024-09-11T08:55:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.734909+0000", 2024-09-11T08:55:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013233799999999999 2024-09-11T08:55:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.734965+0000", 2024-09-11T08:55:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6227000000000002e-05 2024-09-11T08:55:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15621.0:221100 10.9 e341/293 10:92c502a7:::benchmark_data_smithi092_173109_object84141:head v 341'13973, mlcod=341'13971)", 2024-09-11T08:55:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:55:47.738175+0000", 2024-09-11T08:55:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.003685481, 2024-09-11T08:55:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0037744139999999998, 2024-09-11T08:55:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T08:55:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.738175+0000", 2024-09-11T08:55:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:47.749 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.749 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.749 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:47.749 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.738175+0000", 2024-09-11T08:55:47.749 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:47.749 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.749 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.749 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:47.749 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.738174+0000", 2024-09-11T08:55:47.749 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:47.750 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.750 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.750 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:47.750 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.738183+0000", 2024-09-11T08:55:47.750 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.3809999999999999e-06 2024-09-11T08:55:47.750 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.750 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.738200+0000", 2024-09-11T08:55:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7435000000000002e-05 2024-09-11T08:55:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.738207+0000", 2024-09-11T08:55:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8040000000000004e-06 2024-09-11T08:55:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.738424+0000", 2024-09-11T08:55:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00021703299999999999 2024-09-11T08:55:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.738473+0000", 2024-09-11T08:55:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8690000000000003e-05 2024-09-11T08:55:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15621.0:221107 10.8 e341/293 10:107d0fec:::benchmark_data_smithi092_173109_object84148:head v 341'13894, mlcod=341'13892)", 2024-09-11T08:55:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:55:47.740321+0000", 2024-09-11T08:55:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.0015394619999999999, 2024-09-11T08:55:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0016918969999999999, 2024-09-11T08:55:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T08:55:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.740321+0000", 2024-09-11T08:55:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:47.754 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.740321+0000", 2024-09-11T08:55:47.754 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:47.754 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.754 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.754 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:47.754 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.740320+0000", 2024-09-11T08:55:47.754 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:47.754 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.754 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.754 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.740328+0000", 2024-09-11T08:55:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.3150000000000003e-06 2024-09-11T08:55:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.740344+0000", 2024-09-11T08:55:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6450999999999999e-05 2024-09-11T08:55:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.740350+0000", 2024-09-11T08:55:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7950000000000004e-06 2024-09-11T08:55:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.740511+0000", 2024-09-11T08:55:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00016129999999999999 2024-09-11T08:55:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.740572+0000", 2024-09-11T08:55:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1169999999999999e-05 2024-09-11T08:55:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15621.0:221101 10.9 e341/293)", 2024-09-11T08:55:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:55:47.740568+0000", 2024-09-11T08:55:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.001292809, 2024-09-11T08:55:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001495103, 2024-09-11T08:55:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "queued for pg", 2024-09-11T08:55:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.740568+0000", 2024-09-11T08:55:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.740568+0000", 2024-09-11T08:55:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.740567+0000", 2024-09-11T08:55:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:55:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.740574+0000", 2024-09-11T08:55:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.443e-06 2024-09-11T08:55:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.740585+0000", 2024-09-11T08:55:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1257000000000001e-05 2024-09-11T08:55:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.740590+0000", 2024-09-11T08:55:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9130000000000002e-06 2024-09-11T08:55:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:47.760 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:47.760 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.760 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.760 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15621.0:221105 10.9 e341/293)", 2024-09-11T08:55:47.760 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:55:47.741963+0000", 2024-09-11T08:55:47.760 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 4294967295.9998975, 2024-09-11T08:55:47.760 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014116700000000001, 2024-09-11T08:55:47.760 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "queued for pg", 2024-09-11T08:55:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.741963+0000", 2024-09-11T08:55:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.741963+0000", 2024-09-11T08:55:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.741962+0000", 2024-09-11T08:55:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.741969+0000", 2024-09-11T08:55:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3160000000000002e-06 2024-09-11T08:55:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.741984+0000", 2024-09-11T08:55:47.763 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5160999999999999e-05 2024-09-11T08:55:47.763 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:47.763 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:47.763 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:47.763 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:47.741989+0000", 2024-09-11T08:55:47.763 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.288e-06 2024-09-11T08:55:47.764 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:47.764 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:47.764 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:47.764 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:47.764 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2024-09-11T08:55:47.764 INFO:teuthology.orchestra.run.smithi145.stdout: "num_ops": 5 2024-09-11T08:55:47.764 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:55:47.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:47.813 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:47.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:47.815 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:47.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:47.948 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:47.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:47.949 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:48.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:48.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:48.092 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:48.193 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:48.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:48.227 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:48.227 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:48.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:48.364 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:48.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:48.365 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:48.466 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:48.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:48.502 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:48.502 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:48.559 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:48 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:48.559 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:48 smithi145 ceph-mon[33384]: pgmap v771: 158 pgs: 158 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.41k op/s 2024-09-11T08:55:48.559 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:48 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2390389687' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:48.559 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:48 smithi145 ceph-mon[33384]: 4.6 scrub starts 2024-09-11T08:55:48.559 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:48 smithi145 ceph-mon[33384]: 4.6 scrub ok 2024-09-11T08:55:48.559 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:48 smithi145 ceph-mon[33384]: 10.6 scrub starts 2024-09-11T08:55:48.559 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:48 smithi145 ceph-mon[33384]: 10.8 scrub starts 2024-09-11T08:55:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:48 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:48 smithi092 ceph-mon[29647]: pgmap v771: 158 pgs: 158 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.41k op/s 2024-09-11T08:55:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:48 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2390389687' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:48 smithi092 ceph-mon[29647]: 4.6 scrub starts 2024-09-11T08:55:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:48 smithi092 ceph-mon[29647]: 4.6 scrub ok 2024-09-11T08:55:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:48 smithi092 ceph-mon[29647]: 10.6 scrub starts 2024-09-11T08:55:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:48 smithi092 ceph-mon[29647]: 10.8 scrub starts 2024-09-11T08:55:48.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:48 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:48.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:48 smithi092 ceph-mon[41114]: pgmap v771: 158 pgs: 158 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.41k op/s 2024-09-11T08:55:48.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:48 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2390389687' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:48.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:48 smithi092 ceph-mon[41114]: 4.6 scrub starts 2024-09-11T08:55:48.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:48 smithi092 ceph-mon[41114]: 4.6 scrub ok 2024-09-11T08:55:48.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:48 smithi092 ceph-mon[41114]: 10.6 scrub starts 2024-09-11T08:55:48.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:48 smithi092 ceph-mon[41114]: 10.8 scrub starts 2024-09-11T08:55:48.601 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-11T08:55:48.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:48.632 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:48.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:48.633 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:48.734 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:48.779 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:48.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:48.780 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:48.881 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:48.913 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:48.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:48.914 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:49.015 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:49.048 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:55:49.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:49.063 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:49.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:49.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:49.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:49.196 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:49.197 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:49.297 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:49.331 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:49.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:49.333 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:49.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:49.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:49.471 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:49.471 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:49.538 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:49 smithi145 ceph-mon[33384]: 4.1f scrub starts 2024-09-11T08:55:49.538 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:49 smithi145 ceph-mon[33384]: 4.1f scrub ok 2024-09-11T08:55:49.538 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:49 smithi145 ceph-mon[33384]: 5.1d scrub starts 2024-09-11T08:55:49.538 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:49 smithi145 ceph-mon[33384]: 5.1d scrub ok 2024-09-11T08:55:49.538 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:49 smithi145 ceph-mon[33384]: 10.6 scrub ok 2024-09-11T08:55:49.538 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:49 smithi145 ceph-mon[33384]: 10.8 scrub ok 2024-09-11T08:55:49.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:49 smithi092 ceph-mon[29647]: 4.1f scrub starts 2024-09-11T08:55:49.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:49 smithi092 ceph-mon[29647]: 4.1f scrub ok 2024-09-11T08:55:49.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:49 smithi092 ceph-mon[29647]: 5.1d scrub starts 2024-09-11T08:55:49.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:49 smithi092 ceph-mon[29647]: 5.1d scrub ok 2024-09-11T08:55:49.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:49 smithi092 ceph-mon[29647]: 10.6 scrub ok 2024-09-11T08:55:49.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:49 smithi092 ceph-mon[29647]: 10.8 scrub ok 2024-09-11T08:55:49.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:49 smithi092 ceph-mon[41114]: 4.1f scrub starts 2024-09-11T08:55:49.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:49 smithi092 ceph-mon[41114]: 4.1f scrub ok 2024-09-11T08:55:49.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:49 smithi092 ceph-mon[41114]: 5.1d scrub starts 2024-09-11T08:55:49.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:49 smithi092 ceph-mon[41114]: 5.1d scrub ok 2024-09-11T08:55:49.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:49 smithi092 ceph-mon[41114]: 10.6 scrub ok 2024-09-11T08:55:49.568 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:49 smithi092 ceph-mon[41114]: 10.8 scrub ok 2024-09-11T08:55:49.572 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:49.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:49.611 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:49.612 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:49.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:49.743 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:49.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:49.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:49.757 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:55:49.758 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:55:49.758 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-11T08:55:49.758 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T08:55:49.758 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T08:55:49.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 config set filestore_inject_stall 3 2024-09-11T08:55:49.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:49.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:49.888 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:49.889 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:49.989 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:50.027 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:50.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:50.028 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:50.129 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:50.167 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:50.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:50.168 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:50.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:55:50.206 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:50.269 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:50.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:50.304 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:50.305 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:50.405 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:50.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:50.447 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:50.447 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:50.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:50 smithi092 ceph-mon[41114]: pgmap v772: 158 pgs: 158 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-11T08:55:50.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:50 smithi092 ceph-mon[41114]: 3.5 scrub starts 2024-09-11T08:55:50.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:50 smithi092 ceph-mon[41114]: 3.5 scrub ok 2024-09-11T08:55:50.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:50 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T08:55:50.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:50 smithi092 ceph-mon[41114]: 6.f scrub ok 2024-09-11T08:55:50.514 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:50 smithi092 ceph-mon[29647]: pgmap v772: 158 pgs: 158 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-11T08:55:50.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:50 smithi092 ceph-mon[29647]: 3.5 scrub starts 2024-09-11T08:55:50.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:50 smithi092 ceph-mon[29647]: 3.5 scrub ok 2024-09-11T08:55:50.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:50 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T08:55:50.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:50 smithi092 ceph-mon[29647]: 6.f scrub ok 2024-09-11T08:55:50.549 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:50.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:50 smithi145 ceph-mon[33384]: pgmap v772: 158 pgs: 158 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-11T08:55:50.564 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:50 smithi145 ceph-mon[33384]: 3.5 scrub starts 2024-09-11T08:55:50.564 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:50 smithi145 ceph-mon[33384]: 3.5 scrub ok 2024-09-11T08:55:50.564 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:50 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T08:55:50.564 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:50 smithi145 ceph-mon[33384]: 6.f scrub ok 2024-09-11T08:55:50.584 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:50.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:50.585 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:50.686 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:50.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:55:50.725 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:50.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:50.727 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:50.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:50.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:50.881 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:50.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:50.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:51.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:51.041 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:51.041 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:51.095 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:51.142 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:51.174 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:51.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:51.175 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:51.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:51.315 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:51.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:51.316 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:51.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:51.449 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:51.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:51.450 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:51.550 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:51.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:51 smithi092 ceph-mon[29647]: 4.5 scrub starts 2024-09-11T08:55:51.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:51 smithi092 ceph-mon[29647]: 4.5 scrub ok 2024-09-11T08:55:51.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:51 smithi092 ceph-mon[29647]: 10.3 deep-scrub starts 2024-09-11T08:55:51.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:51 smithi092 ceph-mon[29647]: 6.1f scrub starts 2024-09-11T08:55:51.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:51 smithi092 ceph-mon[29647]: 6.1f scrub ok 2024-09-11T08:55:51.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:51 smithi092 ceph-mon[41114]: 4.5 scrub starts 2024-09-11T08:55:51.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:51 smithi092 ceph-mon[41114]: 4.5 scrub ok 2024-09-11T08:55:51.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:51 smithi092 ceph-mon[41114]: 10.3 deep-scrub starts 2024-09-11T08:55:51.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:51 smithi092 ceph-mon[41114]: 6.1f scrub starts 2024-09-11T08:55:51.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:51 smithi092 ceph-mon[41114]: 6.1f scrub ok 2024-09-11T08:55:51.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:55:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:55:51] "GET /metrics HTTP/1.1" 200 224831 "" "Prometheus/2.43.0" 2024-09-11T08:55:51.581 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:51.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:51.582 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:51.607 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:51 smithi145 ceph-mon[33384]: 4.5 scrub starts 2024-09-11T08:55:51.607 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:51 smithi145 ceph-mon[33384]: 4.5 scrub ok 2024-09-11T08:55:51.608 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:51 smithi145 ceph-mon[33384]: 10.3 deep-scrub starts 2024-09-11T08:55:51.608 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:51 smithi145 ceph-mon[33384]: 6.1f scrub starts 2024-09-11T08:55:51.608 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:51 smithi145 ceph-mon[33384]: 6.1f scrub ok 2024-09-11T08:55:51.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:51.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:51.747 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:51.747 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:51.752 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:55:51.752 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T08:55:51.752 INFO:teuthology.orchestra.run.smithi145.stdout: "complaint_time": 30, 2024-09-11T08:55:51.752 INFO:teuthology.orchestra.run.smithi145.stdout: "num_blocked_ops": 0 2024-09-11T08:55:51.752 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:55:51.768 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:51.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:51.877 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:51.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:51.878 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:51.979 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:52.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:52.012 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:52.012 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:52.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:52.150 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:52.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:52.152 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:52.252 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:52.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:52.286 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:52.286 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:52.387 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:52.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:52 smithi145 ceph-mon[33384]: 6.17 scrub starts 2024-09-11T08:55:52.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:52 smithi145 ceph-mon[33384]: 6.17 scrub ok 2024-09-11T08:55:52.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:52 smithi145 ceph-mon[33384]: pgmap v773: 158 pgs: 158 active+clean; 3.2 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.19k op/s 2024-09-11T08:55:52.418 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:52.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:52.419 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:52.501 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-11T08:55:52.520 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:52.553 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:52.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:52.554 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:52.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:52 smithi092 ceph-mon[29647]: 6.17 scrub starts 2024-09-11T08:55:52.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:52 smithi092 ceph-mon[29647]: 6.17 scrub ok 2024-09-11T08:55:52.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:52 smithi092 ceph-mon[29647]: pgmap v773: 158 pgs: 158 active+clean; 3.2 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.19k op/s 2024-09-11T08:55:52.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:52 smithi092 ceph-mon[41114]: 6.17 scrub starts 2024-09-11T08:55:52.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:52 smithi092 ceph-mon[41114]: 6.17 scrub ok 2024-09-11T08:55:52.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:52 smithi092 ceph-mon[41114]: pgmap v773: 158 pgs: 158 active+clean; 3.2 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.19k op/s 2024-09-11T08:55:52.655 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:52.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:52.691 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:52.691 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:52.792 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:52.822 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:52.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:52.823 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:52.924 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:52.947 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:55:52.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:52.968 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:52.968 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:53.069 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:53.091 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:55:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:55:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:55:53.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:53.104 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:53.104 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:53.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:53.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:53.241 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:53.241 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:53.342 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:53.374 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:53.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:53.375 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:53.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:53 smithi145 ceph-mon[33384]: 10.7 scrub starts 2024-09-11T08:55:53.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:53 smithi145 ceph-mon[33384]: 5.19 scrub starts 2024-09-11T08:55:53.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:53 smithi145 ceph-mon[33384]: 5.19 scrub ok 2024-09-11T08:55:53.476 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:53.496 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:53 smithi092 ceph-mon[29647]: 10.7 scrub starts 2024-09-11T08:55:53.496 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:53 smithi092 ceph-mon[29647]: 5.19 scrub starts 2024-09-11T08:55:53.496 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:53 smithi092 ceph-mon[29647]: 5.19 scrub ok 2024-09-11T08:55:53.496 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:53 smithi092 ceph-mon[41114]: 10.7 scrub starts 2024-09-11T08:55:53.497 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:53 smithi092 ceph-mon[41114]: 5.19 scrub starts 2024-09-11T08:55:53.497 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:53 smithi092 ceph-mon[41114]: 5.19 scrub ok 2024-09-11T08:55:53.522 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:53.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:53.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:53.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:53.655 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:53.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:53.656 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:53.757 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:53.794 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:53.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:53.795 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:53.896 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:53.929 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:53.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:53.930 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:54.031 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:54.065 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:54.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:54.066 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:54.167 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:54.198 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:54.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:54.199 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:54.244 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:55:54.244 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "" 2024-09-11T08:55:54.244 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:55:54.300 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:54.331 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:54.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:54.332 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:54.367 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[41114]: 4.c scrub starts 2024-09-11T08:55:54.367 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[41114]: 4.c scrub ok 2024-09-11T08:55:54.367 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:55:54.367 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:55:54.367 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[41114]: osdmap e342: 8 total, 8 up, 8 in 2024-09-11T08:55:54.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[41114]: pgmap v775: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 2.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.52k op/s 2024-09-11T08:55:54.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[41114]: 6.5 scrub starts 2024-09-11T08:55:54.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[41114]: 6.5 scrub ok 2024-09-11T08:55:54.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[41114]: 10.7 scrub ok 2024-09-11T08:55:54.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[41114]: 1.4 scrub starts 2024-09-11T08:55:54.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[41114]: 1.4 scrub ok 2024-09-11T08:55:54.368 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[29647]: 4.c scrub starts 2024-09-11T08:55:54.369 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[29647]: 4.c scrub ok 2024-09-11T08:55:54.369 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:55:54.369 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:55:54.369 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[29647]: osdmap e342: 8 total, 8 up, 8 in 2024-09-11T08:55:54.369 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[29647]: pgmap v775: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 2.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.52k op/s 2024-09-11T08:55:54.369 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[29647]: 6.5 scrub starts 2024-09-11T08:55:54.369 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[29647]: 6.5 scrub ok 2024-09-11T08:55:54.369 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[29647]: 10.7 scrub ok 2024-09-11T08:55:54.369 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[29647]: 1.4 scrub starts 2024-09-11T08:55:54.369 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:54 smithi092 ceph-mon[29647]: 1.4 scrub ok 2024-09-11T08:55:54.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:54.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:54.471 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:54.471 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:54.572 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:54.614 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:54.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:54.615 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:54 smithi145 ceph-mon[33384]: 4.c scrub starts 2024-09-11T08:55:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:54 smithi145 ceph-mon[33384]: 4.c scrub ok 2024-09-11T08:55:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:54 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:55:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:54 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:55:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:54 smithi145 ceph-mon[33384]: osdmap e342: 8 total, 8 up, 8 in 2024-09-11T08:55:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:54 smithi145 ceph-mon[33384]: pgmap v775: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 2.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.52k op/s 2024-09-11T08:55:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:54 smithi145 ceph-mon[33384]: 6.5 scrub starts 2024-09-11T08:55:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:54 smithi145 ceph-mon[33384]: 6.5 scrub ok 2024-09-11T08:55:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:54 smithi145 ceph-mon[33384]: 10.7 scrub ok 2024-09-11T08:55:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:54 smithi145 ceph-mon[33384]: 1.4 scrub starts 2024-09-11T08:55:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:54 smithi145 ceph-mon[33384]: 1.4 scrub ok 2024-09-11T08:55:54.716 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:54.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:54.756 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:54.756 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:54.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:54.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:54.895 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:54.896 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:54.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:55.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:55.035 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:55.035 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:55.136 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:55.171 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:55.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:55.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:55.274 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:55.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:55.317 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:55.317 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:55.332 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:55 smithi145 ceph-mon[33384]: 4.1 scrub starts 2024-09-11T08:55:55.332 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:55 smithi145 ceph-mon[33384]: 4.1 scrub ok 2024-09-11T08:55:55.332 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:55 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:55:55.406 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:55:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T08:55:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T08:55:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:55:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120775 9.19 9:9f3327b4:::benchmark_data_smithi092_145083_object120774:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:55:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.527015+0000", 2024-09-11T08:55:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 329.876353806, 2024-09-11T08:55:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0204559310000001, 2024-09-11T08:55:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120775 2024-09-11T08:55:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527015+0000", 2024-09-11T08:55:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527015+0000", 2024-09-11T08:55:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527008+0000", 2024-09-11T08:55:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999933 2024-09-11T08:55:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527071+0000", 2024-09-11T08:55:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2667000000000002e-05 2024-09-11T08:55:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527089+0000", 2024-09-11T08:55:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.8498000000000001e-05 2024-09-11T08:55:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527099+0000", 2024-09-11T08:55:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0016e-05 2024-09-11T08:55:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.676418+0000", 2024-09-11T08:55:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.14931815200000001 2024-09-11T08:55:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.676427+0000", 2024-09-11T08:55:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.8320000000000001e-06 2024-09-11T08:55:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538222+0000", 2024-09-11T08:55:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.86179418299999999 2024-09-11T08:55:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538346+0000", 2024-09-11T08:55:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012482300000000001 2024-09-11T08:55:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,5", 2024-09-11T08:55:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538574+0000", 2024-09-11T08:55:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00022776599999999999 2024-09-11T08:55:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.539918+0000", 2024-09-11T08:55:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001344065 2024-09-11T08:55:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.539927+0000", 2024-09-11T08:55:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.4710000000000007e-06 2024-09-11T08:55:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547401+0000", 2024-09-11T08:55:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.007474217 2024-09-11T08:55:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547440+0000", 2024-09-11T08:55:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9490999999999998e-05 2024-09-11T08:55:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547471+0000", 2024-09-11T08:55:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0651000000000002e-05 2024-09-11T08:55:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120776 9.13 9:cecf0514:::benchmark_data_smithi092_145083_object120775:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:55:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.655311+0000", 2024-09-11T08:55:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 329.748057464, 2024-09-11T08:55:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89210484800000001, 2024-09-11T08:55:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120776 2024-09-11T08:55:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655311+0000", 2024-09-11T08:55:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655311+0000", 2024-09-11T08:55:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655309+0000", 2024-09-11T08:55:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:55:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655369+0000", 2024-09-11T08:55:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0052e-05 2024-09-11T08:55:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655395+0000", 2024-09-11T08:55:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5205999999999999e-05 2024-09-11T08:55:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655405+0000", 2024-09-11T08:55:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0399e-05 2024-09-11T08:55:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.417 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.417 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.417 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.659087+0000", 2024-09-11T08:55:55.417 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0036818240000000002 2024-09-11T08:55:55.417 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.417 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.417 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.417 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.659098+0000", 2024-09-11T08:55:55.417 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1600000000000001e-05 2024-09-11T08:55:55.417 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.417 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.417 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.417 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.537736+0000", 2024-09-11T08:55:55.417 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.87863754699999996 2024-09-11T08:55:55.418 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.418 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.418 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.418 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.537834+0000", 2024-09-11T08:55:55.418 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.7986999999999995e-05 2024-09-11T08:55:55.418 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.418 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.418 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:55:55.418 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538018+0000", 2024-09-11T08:55:55.418 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00018384599999999999 2024-09-11T08:55:55.418 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.418 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.418 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.419 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538962+0000", 2024-09-11T08:55:55.419 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00094433400000000002 2024-09-11T08:55:55.419 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.419 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.419 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.419 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.540658+0000", 2024-09-11T08:55:55.419 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0016956969999999999 2024-09-11T08:55:55.419 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.419 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.419 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.419 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547326+0000", 2024-09-11T08:55:55.419 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0066684819999999999 2024-09-11T08:55:55.419 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.419 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.420 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.420 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547381+0000", 2024-09-11T08:55:55.420 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4537999999999999e-05 2024-09-11T08:55:55.420 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:55.421 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.421 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.421 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.421 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547416+0000", 2024-09-11T08:55:55.421 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5389000000000003e-05 2024-09-11T08:55:55.422 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.422 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.422 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.422 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.422 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.422 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120777 9.3 9:c63642ab:::benchmark_data_smithi092_145083_object120776:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:55:55.422 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.681266+0000", 2024-09-11T08:55:55.422 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 329.72210305800002, 2024-09-11T08:55:55.422 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.85751633400000005, 2024-09-11T08:55:55.422 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.422 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.422 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.422 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.423 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.423 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120777 2024-09-11T08:55:55.423 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.423 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.423 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.423 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.423 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681266+0000", 2024-09-11T08:55:55.423 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.423 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.423 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.423 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.423 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681266+0000", 2024-09-11T08:55:55.423 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.423 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.424 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.424 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.424 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681264+0000", 2024-09-11T08:55:55.424 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:55:55.424 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.424 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.424 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.424 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681297+0000", 2024-09-11T08:55:55.424 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2461e-05 2024-09-11T08:55:55.424 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.424 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.424 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.424 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681309+0000", 2024-09-11T08:55:55.425 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2519e-05 2024-09-11T08:55:55.425 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.425 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.425 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.425 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681314+0000", 2024-09-11T08:55:55.425 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1279999999999999e-06 2024-09-11T08:55:55.425 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.425 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.425 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.425 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681360+0000", 2024-09-11T08:55:55.425 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5438000000000001e-05 2024-09-11T08:55:55.425 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.425 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.426 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.426 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681368+0000", 2024-09-11T08:55:55.426 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2949999999999994e-06 2024-09-11T08:55:55.426 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.426 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.426 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.426 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.525926+0000", 2024-09-11T08:55:55.426 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.84455778599999998 2024-09-11T08:55:55.426 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.426 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.426 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.426 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.526080+0000", 2024-09-11T08:55:55.426 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015359799999999999 2024-09-11T08:55:55.427 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.427 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.427 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:55:55.427 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.526368+0000", 2024-09-11T08:55:55.427 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000288027 2024-09-11T08:55:55.427 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.427 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.427 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.427 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.527948+0000", 2024-09-11T08:55:55.427 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0015808269999999999 2024-09-11T08:55:55.427 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.427 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.427 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.427 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.535215+0000", 2024-09-11T08:55:55.428 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0072669259999999999 2024-09-11T08:55:55.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.428 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.428 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538703+0000", 2024-09-11T08:55:55.428 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0034881640000000002 2024-09-11T08:55:55.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.428 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.428 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538748+0000", 2024-09-11T08:55:55.428 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4304999999999997e-05 2024-09-11T08:55:55.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.428 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.429 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.429 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538782+0000", 2024-09-11T08:55:55.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4400000000000003e-05 2024-09-11T08:55:55.429 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.429 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.429 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.429 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.429 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.429 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122628 9.13 9:cbe7bc43:::benchmark_data_smithi092_145083_object122627:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236)", 2024-09-11T08:55:55.429 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.520279+0000", 2024-09-11T08:55:55.429 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 327.883089794, 2024-09-11T08:55:55.429 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.435847541, 2024-09-11T08:55:55.429 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.430 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:55:55.430 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.430 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.430 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.430 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122628 2024-09-11T08:55:55.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.430 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.430 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.430 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.430 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520279+0000", 2024-09-11T08:55:55.430 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.430 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.430 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520279+0000", 2024-09-11T08:55:55.431 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.431 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.431 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520278+0000", 2024-09-11T08:55:55.431 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:55:55.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.431 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.431 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.431 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520339+0000", 2024-09-11T08:55:55.431 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1049000000000005e-05 2024-09-11T08:55:55.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.431 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.432 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.432 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520349+0000", 2024-09-11T08:55:55.432 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.6190000000000006e-06 2024-09-11T08:55:55.432 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.432 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.432 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.432 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520353+0000", 2024-09-11T08:55:55.432 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3220000000000003e-06 2024-09-11T08:55:55.432 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.432 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.432 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.432 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520397+0000", 2024-09-11T08:55:55.432 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3739000000000003e-05 2024-09-11T08:55:55.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.433 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.433 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.433 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520511+0000", 2024-09-11T08:55:55.433 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011360499999999999 2024-09-11T08:55:55.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.433 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.433 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:55:55.433 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520760+0000", 2024-09-11T08:55:55.433 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00024918800000000001 2024-09-11T08:55:55.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.433 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.433 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.433 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.522187+0000", 2024-09-11T08:55:55.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0014270960000000001 2024-09-11T08:55:55.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.434 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.434 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.434 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.527785+0000", 2024-09-11T08:55:55.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0055977650000000002 2024-09-11T08:55:55.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.434 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.434 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.434 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.536750+0000", 2024-09-11T08:55:55.434 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0089646959999999994 2024-09-11T08:55:55.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.434 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.434 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.536758+0000", 2024-09-11T08:55:55.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.1410000000000005e-06 2024-09-11T08:55:55.435 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.435 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.435 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537393+0000", 2024-09-11T08:55:55.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.000635798 2024-09-11T08:55:55.435 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.435 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.435 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.435 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537404+0000", 2024-09-11T08:55:55.435 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0471e-05 2024-09-11T08:55:55.435 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.436 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.436 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.436 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.956039+0000", 2024-09-11T08:55:55.436 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.41863534800000002 2024-09-11T08:55:55.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.436 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.436 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.436 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.956127+0000", 2024-09-11T08:55:55.436 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.7373999999999994e-05 2024-09-11T08:55:55.436 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.436 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.436 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.437 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.437 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122632 9.13 9:cb01d006:::benchmark_data_smithi092_145083_object122631:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236)", 2024-09-11T08:55:55.437 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.523542+0000", 2024-09-11T08:55:55.437 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 327.87982664800001, 2024-09-11T08:55:55.437 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.43267765, 2024-09-11T08:55:55.437 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.437 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:55:55.437 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.437 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.437 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.437 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122632 2024-09-11T08:55:55.437 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.437 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.437 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.438 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.438 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523542+0000", 2024-09-11T08:55:55.438 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.438 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.438 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.438 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523542+0000", 2024-09-11T08:55:55.438 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.438 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.438 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.438 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.438 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523541+0000", 2024-09-11T08:55:55.438 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:55.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.439 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.439 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523573+0000", 2024-09-11T08:55:55.439 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2162000000000001e-05 2024-09-11T08:55:55.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.439 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.439 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523585+0000", 2024-09-11T08:55:55.439 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1269e-05 2024-09-11T08:55:55.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.439 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.439 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.439 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523590+0000", 2024-09-11T08:55:55.440 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5720000000000002e-06 2024-09-11T08:55:55.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.440 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.440 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523636+0000", 2024-09-11T08:55:55.440 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6233999999999998e-05 2024-09-11T08:55:55.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.440 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.440 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523944+0000", 2024-09-11T08:55:55.440 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00030767799999999998 2024-09-11T08:55:55.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.440 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.440 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:55:55.441 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.524202+0000", 2024-09-11T08:55:55.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00025762300000000001 2024-09-11T08:55:55.441 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.441 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.441 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.441 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.528196+0000", 2024-09-11T08:55:55.441 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0039945980000000002 2024-09-11T08:55:55.441 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.441 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.441 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.442 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.530011+0000", 2024-09-11T08:55:55.442 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001814525 2024-09-11T08:55:55.442 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.442 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.442 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.442 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.537490+0000", 2024-09-11T08:55:55.442 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0074790059999999998 2024-09-11T08:55:55.442 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.442 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.442 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.442 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.537523+0000", 2024-09-11T08:55:55.442 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3389000000000002e-05 2024-09-11T08:55:55.442 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.443 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.443 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.443 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537576+0000", 2024-09-11T08:55:55.443 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0000530000000001 2024-09-11T08:55:55.443 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.443 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.443 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.443 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537583+0000", 2024-09-11T08:55:55.443 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9650000000000002e-06 2024-09-11T08:55:55.443 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.443 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.443 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.443 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.956160+0000", 2024-09-11T08:55:55.443 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.41857655500000002 2024-09-11T08:55:55.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.444 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.956220+0000", 2024-09-11T08:55:55.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0102000000000002e-05 2024-09-11T08:55:55.444 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.444 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.444 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.444 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122660 9.17 9:e8b491cd:::benchmark_data_smithi092_145083_object122659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:55:55.444 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.551000+0000", 2024-09-11T08:55:55.444 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 327.85236906, 2024-09-11T08:55:55.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4263746720000001, 2024-09-11T08:55:55.445 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.445 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.445 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.445 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.445 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.445 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122660 2024-09-11T08:55:55.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.445 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.445 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.445 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551000+0000", 2024-09-11T08:55:55.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551000+0000", 2024-09-11T08:55:55.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.446 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.550999+0000", 2024-09-11T08:55:55.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:55.446 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551060+0000", 2024-09-11T08:55:55.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1168999999999997e-05 2024-09-11T08:55:55.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551072+0000", 2024-09-11T08:55:55.447 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2099000000000001e-05 2024-09-11T08:55:55.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551078+0000", 2024-09-11T08:55:55.447 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5629999999999999e-06 2024-09-11T08:55:55.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551141+0000", 2024-09-11T08:55:55.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3082000000000004e-05 2024-09-11T08:55:55.448 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.448 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.448 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.448 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551150+0000", 2024-09-11T08:55:55.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8459999999999997e-06 2024-09-11T08:55:55.448 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.448 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.448 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.448 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537577+0000", 2024-09-11T08:55:55.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.98642732899999996 2024-09-11T08:55:55.448 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.448 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.449 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.449 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537585+0000", 2024-09-11T08:55:55.449 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2230000000000007e-06 2024-09-11T08:55:55.449 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.449 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.449 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.449 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.961506+0000", 2024-09-11T08:55:55.449 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.42392071100000001 2024-09-11T08:55:55.449 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.449 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.449 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.449 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.961706+0000", 2024-09-11T08:55:55.449 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00020044799999999999 2024-09-11T08:55:55.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2,3,5", 2024-09-11T08:55:55.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.962037+0000", 2024-09-11T08:55:55.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00033083199999999998 2024-09-11T08:55:55.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.964043+0000", 2024-09-11T08:55:55.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0020053179999999999 2024-09-11T08:55:55.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.965851+0000", 2024-09-11T08:55:55.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0018087610000000001 2024-09-11T08:55:55.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.967059+0000", 2024-09-11T08:55:55.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0012074080000000001 2024-09-11T08:55:55.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.967101+0000", 2024-09-11T08:55:55.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2601000000000001e-05 2024-09-11T08:55:55.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.977211+0000", 2024-09-11T08:55:55.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.010109315000000001 2024-09-11T08:55:55.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.977264+0000", 2024-09-11T08:55:55.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3288999999999998e-05 2024-09-11T08:55:55.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.977375+0000", 2024-09-11T08:55:55.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000110554 2024-09-11T08:55:55.453 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.453 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.453 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.453 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122671 9.15 9:ab433584:::benchmark_data_smithi092_145083_object122670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:55:55.453 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.558491+0000", 2024-09-11T08:55:55.453 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 327.84487824199999, 2024-09-11T08:55:55.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3936105059999999, 2024-09-11T08:55:55.453 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.453 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.453 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.453 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.454 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.454 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122671 2024-09-11T08:55:55.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.454 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558491+0000", 2024-09-11T08:55:55.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558491+0000", 2024-09-11T08:55:55.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.455 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.455 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558490+0000", 2024-09-11T08:55:55.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:55.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.455 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.455 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558704+0000", 2024-09-11T08:55:55.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00021395600000000001 2024-09-11T08:55:55.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.455 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.455 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558716+0000", 2024-09-11T08:55:55.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.278e-05 2024-09-11T08:55:55.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.456 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.456 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.456 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558722+0000", 2024-09-11T08:55:55.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4999999999999999e-06 2024-09-11T08:55:55.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.456 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.456 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.456 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558765+0000", 2024-09-11T08:55:55.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2815999999999998e-05 2024-09-11T08:55:55.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.456 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.456 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558770+0000", 2024-09-11T08:55:55.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1660000000000002e-06 2024-09-11T08:55:55.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537512+0000", 2024-09-11T08:55:55.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97874157399999995 2024-09-11T08:55:55.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537520+0000", 2024-09-11T08:55:55.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.5860000000000005e-06 2024-09-11T08:55:55.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.932889+0000", 2024-09-11T08:55:55.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.39536916900000002 2024-09-11T08:55:55.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.933050+0000", 2024-09-11T08:55:55.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00016028799999999999 2024-09-11T08:55:55.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3,5", 2024-09-11T08:55:55.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.933393+0000", 2024-09-11T08:55:55.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00034307499999999998 2024-09-11T08:55:55.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.937335+0000", 2024-09-11T08:55:55.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0039422570000000002 2024-09-11T08:55:55.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.939190+0000", 2024-09-11T08:55:55.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0018552729999999999 2024-09-11T08:55:55.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.942079+0000", 2024-09-11T08:55:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.002888767 2024-09-11T08:55:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.951900+0000", 2024-09-11T08:55:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0098207899999999994 2024-09-11T08:55:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.951972+0000", 2024-09-11T08:55:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2611e-05 2024-09-11T08:55:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.461 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.952101+0000", 2024-09-11T08:55:55.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012884499999999999 2024-09-11T08:55:55.461 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.461 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.461 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.461 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:161920 9.15 9:afdf2969:::benchmark_data_smithi092_145083_object18522:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:55:55.461 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:58.943979+0000", 2024-09-11T08:55:55.461 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 296.45939024900002, 2024-09-11T08:55:55.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.001253038, 2024-09-11T08:55:55.461 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.461 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.461 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.462 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.462 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.462 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 161920 2024-09-11T08:55:55.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.462 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.943979+0000", 2024-09-11T08:55:55.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.943979+0000", 2024-09-11T08:55:55.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.943978+0000", 2024-09-11T08:55:55.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:55.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.943982+0000", 2024-09-11T08:55:55.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5090000000000004e-06 2024-09-11T08:55:55.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.944019+0000", 2024-09-11T08:55:55.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6127999999999997e-05 2024-09-11T08:55:55.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.944024+0000", 2024-09-11T08:55:55.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9320000000000001e-06 2024-09-11T08:55:55.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.945966+0000", 2024-09-11T08:55:55.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001941047 2024-09-11T08:55:55.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.466 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.945973+0000", 2024-09-11T08:55:55.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.267e-06 2024-09-11T08:55:55.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.466 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.924511+0000", 2024-09-11T08:55:55.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97853800700000004 2024-09-11T08:55:55.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.924629+0000", 2024-09-11T08:55:55.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000117945 2024-09-11T08:55:55.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.467 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:55:55.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.924992+0000", 2024-09-11T08:55:55.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00036293600000000001 2024-09-11T08:55:55.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.467 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.467 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.467 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.931573+0000", 2024-09-11T08:55:55.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0065809639999999999 2024-09-11T08:55:55.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945131+0000", 2024-09-11T08:55:55.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013558374999999999 2024-09-11T08:55:55.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945175+0000", 2024-09-11T08:55:55.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3695000000000001e-05 2024-09-11T08:55:55.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.469 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945208+0000", 2024-09-11T08:55:55.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3596000000000002e-05 2024-09-11T08:55:55.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.469 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.469 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945232+0000", 2024-09-11T08:55:55.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3382000000000001e-05 2024-09-11T08:55:55.469 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.469 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.469 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.469 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:161947 9.15 9:a9036c16:::benchmark_data_smithi092_145083_object18549:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:55:55.469 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:58.962388+0000", 2024-09-11T08:55:55.470 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 296.44098106500002, 2024-09-11T08:55:55.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.98294829399999994, 2024-09-11T08:55:55.470 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.470 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.470 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.470 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.470 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.470 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 161947 2024-09-11T08:55:55.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.470 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962388+0000", 2024-09-11T08:55:55.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962388+0000", 2024-09-11T08:55:55.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962387+0000", 2024-09-11T08:55:55.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:55:55.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962392+0000", 2024-09-11T08:55:55.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8999999999999997e-06 2024-09-11T08:55:55.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962400+0000", 2024-09-11T08:55:55.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6109999999999996e-06 2024-09-11T08:55:55.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962404+0000", 2024-09-11T08:55:55.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5549999999999996e-06 2024-09-11T08:55:55.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962462+0000", 2024-09-11T08:55:55.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7262999999999997e-05 2024-09-11T08:55:55.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962468+0000", 2024-09-11T08:55:55.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3049999999999998e-06 2024-09-11T08:55:55.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.473 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.473 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.926681+0000", 2024-09-11T08:55:55.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.96421365999999997 2024-09-11T08:55:55.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.926819+0000", 2024-09-11T08:55:55.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013747 2024-09-11T08:55:55.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:55:55.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.927081+0000", 2024-09-11T08:55:55.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00026203400000000002 2024-09-11T08:55:55.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.931577+0000", 2024-09-11T08:55:55.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.004495644 2024-09-11T08:55:55.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945265+0000", 2024-09-11T08:55:55.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013688709 2024-09-11T08:55:55.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945295+0000", 2024-09-11T08:55:55.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9221e-05 2024-09-11T08:55:55.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945318+0000", 2024-09-11T08:55:55.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3699000000000001e-05 2024-09-11T08:55:55.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945336+0000", 2024-09-11T08:55:55.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7876999999999998e-05 2024-09-11T08:55:55.476 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.476 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.476 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.476 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:161955 9.15 9:aef2dade:::benchmark_data_smithi092_145083_object18557:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:55:55.477 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:58.968477+0000", 2024-09-11T08:55:55.477 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 296.43489184600003, 2024-09-11T08:55:55.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97706887499999995, 2024-09-11T08:55:55.477 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.477 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.477 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.477 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.477 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.477 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 161955 2024-09-11T08:55:55.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.477 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968477+0000", 2024-09-11T08:55:55.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968477+0000", 2024-09-11T08:55:55.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968477+0000", 2024-09-11T08:55:55.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:55:55.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.479 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.479 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968479+0000", 2024-09-11T08:55:55.479 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5830000000000001e-06 2024-09-11T08:55:55.479 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.479 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.479 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.479 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968485+0000", 2024-09-11T08:55:55.479 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0410000000000003e-06 2024-09-11T08:55:55.479 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.479 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.479 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.479 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968489+0000", 2024-09-11T08:55:55.479 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4520000000000002e-06 2024-09-11T08:55:55.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.480 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.480 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.480 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968632+0000", 2024-09-11T08:55:55.480 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014362199999999999 2024-09-11T08:55:55.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.480 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.480 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.480 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968639+0000", 2024-09-11T08:55:55.480 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8290000000000002e-06 2024-09-11T08:55:55.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.480 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.480 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.928106+0000", 2024-09-11T08:55:55.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.95946673199999999 2024-09-11T08:55:55.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.481 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.928211+0000", 2024-09-11T08:55:55.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000104817 2024-09-11T08:55:55.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.481 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:55:55.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.928417+0000", 2024-09-11T08:55:55.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00020603400000000001 2024-09-11T08:55:55.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.933865+0000", 2024-09-11T08:55:55.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0054485180000000003 2024-09-11T08:55:55.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945414+0000", 2024-09-11T08:55:55.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011548388999999999 2024-09-11T08:55:55.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945462+0000", 2024-09-11T08:55:55.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8201000000000001e-05 2024-09-11T08:55:55.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945511+0000", 2024-09-11T08:55:55.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9518999999999998e-05 2024-09-11T08:55:55.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945546+0000", 2024-09-11T08:55:55.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4551999999999997e-05 2024-09-11T08:55:55.483 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.483 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.483 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.484 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.484 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:161996 9.15 9:a871d7df:::benchmark_data_smithi092_145083_object18598:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:55:55.484 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:59.038626+0000", 2024-09-11T08:55:55.484 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 296.364742724, 2024-09-11T08:55:55.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90711806399999995, 2024-09-11T08:55:55.484 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.484 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.484 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.484 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.484 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.484 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 161996 2024-09-11T08:55:55.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.484 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.485 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.485 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.485 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038626+0000", 2024-09-11T08:55:55.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.485 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.485 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.485 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.485 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038626+0000", 2024-09-11T08:55:55.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.485 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.485 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.485 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.485 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038625+0000", 2024-09-11T08:55:55.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:55.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038633+0000", 2024-09-11T08:55:55.486 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.9629999999999998e-06 2024-09-11T08:55:55.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038645+0000", 2024-09-11T08:55:55.486 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2421e-05 2024-09-11T08:55:55.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038651+0000", 2024-09-11T08:55:55.487 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3399999999999997e-06 2024-09-11T08:55:55.487 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.487 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.487 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.487 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038936+0000", 2024-09-11T08:55:55.487 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00028560199999999998 2024-09-11T08:55:55.487 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.487 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.487 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.487 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038943+0000", 2024-09-11T08:55:55.487 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.7309999999999996e-06 2024-09-11T08:55:55.487 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.487 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.929410+0000", 2024-09-11T08:55:55.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89046684799999998 2024-09-11T08:55:55.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.929473+0000", 2024-09-11T08:55:55.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3482e-05 2024-09-11T08:55:55.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:55:55.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.929657+0000", 2024-09-11T08:55:55.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00018351000000000001 2024-09-11T08:55:55.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.933869+0000", 2024-09-11T08:55:55.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0042121370000000003 2024-09-11T08:55:55.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945597+0000", 2024-09-11T08:55:55.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011727609999999999 2024-09-11T08:55:55.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945665+0000", 2024-09-11T08:55:55.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8650000000000002e-05 2024-09-11T08:55:55.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945708+0000", 2024-09-11T08:55:55.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2342000000000002e-05 2024-09-11T08:55:55.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945744+0000", 2024-09-11T08:55:55.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6569999999999997e-05 2024-09-11T08:55:55.490 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.490 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.490 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.491 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.491 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:161999 9.15 9:acecc340:::benchmark_data_smithi092_145083_object18601:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:55:55.491 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:59.040350+0000", 2024-09-11T08:55:55.491 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 296.36301883900001, 2024-09-11T08:55:55.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90555341600000006, 2024-09-11T08:55:55.491 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.491 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.491 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.491 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.491 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.491 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 161999 2024-09-11T08:55:55.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.491 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.492 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.492 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040350+0000", 2024-09-11T08:55:55.492 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.492 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.492 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.492 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040350+0000", 2024-09-11T08:55:55.492 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.492 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.492 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.492 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040349+0000", 2024-09-11T08:55:55.493 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:55.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.493 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.493 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.493 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040354+0000", 2024-09-11T08:55:55.493 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8829999999999997e-06 2024-09-11T08:55:55.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.493 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.493 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.493 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040366+0000", 2024-09-11T08:55:55.493 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.238e-05 2024-09-11T08:55:55.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.493 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.494 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040372+0000", 2024-09-11T08:55:55.494 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8139999999999997e-06 2024-09-11T08:55:55.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.494 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.494 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040516+0000", 2024-09-11T08:55:55.494 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000143313 2024-09-11T08:55:55.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.494 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.494 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040520+0000", 2024-09-11T08:55:55.494 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4089999999999996e-06 2024-09-11T08:55:55.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.930467+0000", 2024-09-11T08:55:55.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.88994658900000001 2024-09-11T08:55:55.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.930588+0000", 2024-09-11T08:55:55.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012184 2024-09-11T08:55:55.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:55:55.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.930801+0000", 2024-09-11T08:55:55.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000212124 2024-09-11T08:55:55.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.937385+0000", 2024-09-11T08:55:55.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0065848369999999996 2024-09-11T08:55:55.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945800+0000", 2024-09-11T08:55:55.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0084145890000000001 2024-09-11T08:55:55.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.497 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945832+0000", 2024-09-11T08:55:55.497 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2202000000000001e-05 2024-09-11T08:55:55.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.497 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.497 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.497 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945872+0000", 2024-09-11T08:55:55.500 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0256000000000002e-05 2024-09-11T08:55:55.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.500 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.500 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.500 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945903+0000", 2024-09-11T08:55:55.500 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1078000000000003e-05 2024-09-11T08:55:55.500 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.500 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.500 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.501 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.501 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162018 9.15 9:ac3e5f86:::benchmark_data_smithi092_145083_object18620:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:55:55.501 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:59.052029+0000", 2024-09-11T08:55:55.501 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 296.35134010600001, 2024-09-11T08:55:55.501 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89400514099999995, 2024-09-11T08:55:55.501 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.501 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.501 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.501 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.501 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.501 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162018 2024-09-11T08:55:55.501 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.501 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.501 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.502 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.502 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052029+0000", 2024-09-11T08:55:55.502 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.502 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.502 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.502 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052029+0000", 2024-09-11T08:55:55.502 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.502 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.502 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.502 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052027+0000", 2024-09-11T08:55:55.502 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:55:55.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.503 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.503 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.503 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052034+0000", 2024-09-11T08:55:55.503 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3389999999999998e-06 2024-09-11T08:55:55.503 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.503 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.503 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.503 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052046+0000", 2024-09-11T08:55:55.503 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2517999999999999e-05 2024-09-11T08:55:55.503 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.503 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.503 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.503 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052056+0000", 2024-09-11T08:55:55.504 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.023e-05 2024-09-11T08:55:55.504 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.504 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.504 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.504 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052119+0000", 2024-09-11T08:55:55.504 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2984999999999994e-05 2024-09-11T08:55:55.504 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.504 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.504 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.504 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052127+0000", 2024-09-11T08:55:55.504 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.0330000000000008e-06 2024-09-11T08:55:55.504 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.504 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.504 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.505 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.931615+0000", 2024-09-11T08:55:55.505 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.87948807200000001 2024-09-11T08:55:55.505 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.505 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.505 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.505 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.931701+0000", 2024-09-11T08:55:55.505 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.5131999999999997e-05 2024-09-11T08:55:55.505 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.505 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.505 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:55:55.505 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.931887+0000", 2024-09-11T08:55:55.505 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00018613399999999999 2024-09-11T08:55:55.505 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.506 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.506 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.506 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.937393+0000", 2024-09-11T08:55:55.506 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0055058060000000002 2024-09-11T08:55:55.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.506 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.506 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.506 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945937+0000", 2024-09-11T08:55:55.506 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0085445950000000003 2024-09-11T08:55:55.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.506 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.506 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.506 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945969+0000", 2024-09-11T08:55:55.506 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2131000000000001e-05 2024-09-11T08:55:55.507 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.507 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.507 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.507 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946003+0000", 2024-09-11T08:55:55.507 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4224000000000003e-05 2024-09-11T08:55:55.507 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.507 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.507 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.507 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946034+0000", 2024-09-11T08:55:55.507 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0470000000000001e-05 2024-09-11T08:55:55.507 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.507 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.507 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.507 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.508 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.508 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162035 9.15 9:ac737bbe:::benchmark_data_smithi092_145083_object18637:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:55:55.508 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:59.065230+0000", 2024-09-11T08:55:55.508 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 296.338139114, 2024-09-11T08:55:55.508 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.88095609799999997, 2024-09-11T08:55:55.508 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.508 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.508 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.508 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.508 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.508 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162035 2024-09-11T08:55:55.508 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.508 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.509 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.509 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.509 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065230+0000", 2024-09-11T08:55:55.509 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.509 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.509 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.509 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:55 smithi092 ceph-mon[29647]: 4.1 scrub starts 2024-09-11T08:55:55.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:55 smithi092 ceph-mon[29647]: 4.1 scrub ok 2024-09-11T08:55:55.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:55 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:55:55.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:55 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:55:55.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:55 smithi092 ceph-mon[29647]: osdmap e343: 8 total, 8 up, 8 in 2024-09-11T08:55:55.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:55 smithi092 ceph-mon[29647]: 10.b scrub starts 2024-09-11T08:55:55.511 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:55 smithi092 ceph-mon[41114]: 4.1 scrub starts 2024-09-11T08:55:55.512 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:55 smithi092 ceph-mon[41114]: 4.1 scrub ok 2024-09-11T08:55:55.512 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:55 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:55:55.512 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:55 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:55:55.512 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:55 smithi092 ceph-mon[41114]: osdmap e343: 8 total, 8 up, 8 in 2024-09-11T08:55:55.512 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:55 smithi092 ceph-mon[41114]: 10.b scrub starts 2024-09-11T08:55:55.512 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065230+0000", 2024-09-11T08:55:55.512 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.512 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.512 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.512 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.512 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065229+0000", 2024-09-11T08:55:55.513 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:55:55.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.513 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.513 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.513 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065232+0000", 2024-09-11T08:55:55.513 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9340000000000002e-06 2024-09-11T08:55:55.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.513 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.513 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.513 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065239+0000", 2024-09-11T08:55:55.513 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.708e-06 2024-09-11T08:55:55.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.513 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.514 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.514 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065243+0000", 2024-09-11T08:55:55.514 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6430000000000001e-06 2024-09-11T08:55:55.514 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.514 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.514 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.514 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065366+0000", 2024-09-11T08:55:55.514 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000123511 2024-09-11T08:55:55.514 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.514 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.514 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.514 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065372+0000", 2024-09-11T08:55:55.514 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1909999999999997e-06 2024-09-11T08:55:55.514 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.515 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.515 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.515 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.932683+0000", 2024-09-11T08:55:55.515 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.86731089299999997 2024-09-11T08:55:55.515 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.515 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.515 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.515 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.932780+0000", 2024-09-11T08:55:55.515 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.6373999999999996e-05 2024-09-11T08:55:55.515 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.515 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.515 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:55:55.515 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.933044+0000", 2024-09-11T08:55:55.516 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00026401899999999999 2024-09-11T08:55:55.516 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.516 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.516 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.516 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.939749+0000", 2024-09-11T08:55:55.516 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.006705529 2024-09-11T08:55:55.516 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.516 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.516 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.516 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946068+0000", 2024-09-11T08:55:55.516 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0063190540000000002 2024-09-11T08:55:55.516 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.516 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.517 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.517 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946106+0000", 2024-09-11T08:55:55.517 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7839000000000001e-05 2024-09-11T08:55:55.517 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.517 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.517 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.517 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946144+0000", 2024-09-11T08:55:55.517 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8343000000000002e-05 2024-09-11T08:55:55.517 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.517 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.517 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.517 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946186+0000", 2024-09-11T08:55:55.517 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1564999999999999e-05 2024-09-11T08:55:55.517 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.518 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.518 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.518 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.518 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.518 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162098 9.15 9:ab1b879d:::benchmark_data_smithi092_145083_object18700:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:55:55.518 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:59.112673+0000", 2024-09-11T08:55:55.518 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 296.29069610400001, 2024-09-11T08:55:55.518 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.83383003600000005, 2024-09-11T08:55:55.518 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.518 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.518 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.518 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.518 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.519 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162098 2024-09-11T08:55:55.519 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.519 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.519 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.519 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.519 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112673+0000", 2024-09-11T08:55:55.519 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.519 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.519 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.519 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.519 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112673+0000", 2024-09-11T08:55:55.519 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.519 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.519 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.520 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.520 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112672+0000", 2024-09-11T08:55:55.520 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:55.520 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.520 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.520 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.520 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112676+0000", 2024-09-11T08:55:55.520 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4020000000000004e-06 2024-09-11T08:55:55.520 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.520 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.520 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.520 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112685+0000", 2024-09-11T08:55:55.520 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.9390000000000003e-06 2024-09-11T08:55:55.521 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.521 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.521 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.521 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112689+0000", 2024-09-11T08:55:55.521 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5210000000000004e-06 2024-09-11T08:55:55.521 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.521 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.521 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.521 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112725+0000", 2024-09-11T08:55:55.521 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5176000000000003e-05 2024-09-11T08:55:55.521 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.521 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.521 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.521 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112731+0000", 2024-09-11T08:55:55.522 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.088e-06 2024-09-11T08:55:55.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.522 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.522 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.522 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.933898+0000", 2024-09-11T08:55:55.522 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.82116687399999999 2024-09-11T08:55:55.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.522 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.522 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.522 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.933969+0000", 2024-09-11T08:55:55.522 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1109e-05 2024-09-11T08:55:55.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.522 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.522 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:55:55.523 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.934117+0000", 2024-09-11T08:55:55.523 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000148527 2024-09-11T08:55:55.523 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.523 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.523 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.523 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.939757+0000", 2024-09-11T08:55:55.523 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0056393290000000002 2024-09-11T08:55:55.523 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.523 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.523 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.523 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946218+0000", 2024-09-11T08:55:55.523 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0064610709999999997 2024-09-11T08:55:55.523 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.524 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.524 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.524 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946415+0000", 2024-09-11T08:55:55.524 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00019775899999999999 2024-09-11T08:55:55.524 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.524 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.524 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.524 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946453+0000", 2024-09-11T08:55:55.524 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8075999999999999e-05 2024-09-11T08:55:55.524 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.524 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.524 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.524 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946503+0000", 2024-09-11T08:55:55.524 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9345999999999998e-05 2024-09-11T08:55:55.525 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.525 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.525 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.525 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.525 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.525 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162378 9.17 9:eb84acd7:::benchmark_data_smithi092_145083_object18980:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-11T08:55:55.525 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:51:00.001027+0000", 2024-09-11T08:55:55.525 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 295.40234176199999, 2024-09-11T08:55:55.525 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.95130415599999996, 2024-09-11T08:55:55.525 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.525 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.525 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.525 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.526 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.526 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162378 2024-09-11T08:55:55.526 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.526 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.526 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.526 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.526 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001027+0000", 2024-09-11T08:55:55.526 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.526 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.526 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.526 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.526 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001027+0000", 2024-09-11T08:55:55.526 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.527 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.527 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.527 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.527 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001027+0000", 2024-09-11T08:55:55.527 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:55:55.527 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.527 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.527 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.527 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001029+0000", 2024-09-11T08:55:55.527 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0640000000000001e-06 2024-09-11T08:55:55.527 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.527 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.527 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.527 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001035+0000", 2024-09-11T08:55:55.528 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1e-06 2024-09-11T08:55:55.528 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.528 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.528 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.528 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001039+0000", 2024-09-11T08:55:55.528 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7749999999999999e-06 2024-09-11T08:55:55.528 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.528 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.528 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.528 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001115+0000", 2024-09-11T08:55:55.528 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6666000000000001e-05 2024-09-11T08:55:55.528 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.528 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.529 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.529 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001124+0000", 2024-09-11T08:55:55.529 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8990000000000007e-06 2024-09-11T08:55:55.529 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.529 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.529 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.529 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.933510+0000", 2024-09-11T08:55:55.529 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.93238578100000002 2024-09-11T08:55:55.529 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.529 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.529 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.529 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.933633+0000", 2024-09-11T08:55:55.529 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012321599999999999 2024-09-11T08:55:55.529 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.530 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.530 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:55:55.530 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.933908+0000", 2024-09-11T08:55:55.530 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00027423600000000002 2024-09-11T08:55:55.530 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.530 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.530 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.530 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.939240+0000", 2024-09-11T08:55:55.530 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0053320060000000002 2024-09-11T08:55:55.530 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.530 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.530 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.530 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952168+0000", 2024-09-11T08:55:55.531 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.012928439 2024-09-11T08:55:55.531 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.531 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.531 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.531 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952206+0000", 2024-09-11T08:55:55.531 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8479999999999997e-05 2024-09-11T08:55:55.531 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.531 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.531 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.531 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952254+0000", 2024-09-11T08:55:55.531 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7126999999999998e-05 2024-09-11T08:55:55.531 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.531 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.531 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.532 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952331+0000", 2024-09-11T08:55:55.532 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7713999999999995e-05 2024-09-11T08:55:55.532 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.532 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.532 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.532 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.532 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.532 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162412 9.17 9:ee197516:::benchmark_data_smithi092_145083_object19014:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-11T08:55:55.532 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:51:00.024372+0000", 2024-09-11T08:55:55.532 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 295.37899713500002, 2024-09-11T08:55:55.532 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.92838030800000004, 2024-09-11T08:55:55.532 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.532 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.533 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.533 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.533 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.533 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162412 2024-09-11T08:55:55.533 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.533 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.533 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.533 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.533 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024372+0000", 2024-09-11T08:55:55.533 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.533 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.533 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.533 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.534 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024372+0000", 2024-09-11T08:55:55.534 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.534 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.534 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.534 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.534 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024370+0000", 2024-09-11T08:55:55.534 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:55:55.534 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.534 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.534 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.534 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024377+0000", 2024-09-11T08:55:55.534 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1179999999999997e-06 2024-09-11T08:55:55.534 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.534 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.535 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.535 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024388+0000", 2024-09-11T08:55:55.535 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1260999999999999e-05 2024-09-11T08:55:55.535 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.535 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.535 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.535 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024393+0000", 2024-09-11T08:55:55.535 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5600000000000001e-06 2024-09-11T08:55:55.535 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.535 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.535 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.535 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024653+0000", 2024-09-11T08:55:55.535 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00025985299999999999 2024-09-11T08:55:55.535 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.536 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.536 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.536 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024661+0000", 2024-09-11T08:55:55.536 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2760000000000003e-06 2024-09-11T08:55:55.536 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.536 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.536 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.536 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.935055+0000", 2024-09-11T08:55:55.536 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.91039429400000005 2024-09-11T08:55:55.536 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.536 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.536 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.536 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.935124+0000", 2024-09-11T08:55:55.537 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8872000000000001e-05 2024-09-11T08:55:55.537 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.537 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.537 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:55:55.537 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.935310+0000", 2024-09-11T08:55:55.537 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000185823 2024-09-11T08:55:55.537 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.537 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.537 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.537 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.939246+0000", 2024-09-11T08:55:55.537 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0039369030000000003 2024-09-11T08:55:55.537 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.537 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.537 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.538 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952414+0000", 2024-09-11T08:55:55.538 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013167935 2024-09-11T08:55:55.538 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.538 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.538 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.538 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952690+0000", 2024-09-11T08:55:55.538 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00027535 2024-09-11T08:55:55.538 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.538 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.538 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.538 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952724+0000", 2024-09-11T08:55:55.538 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4767000000000001e-05 2024-09-11T08:55:55.538 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.538 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.539 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.539 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952752+0000", 2024-09-11T08:55:55.539 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7591999999999999e-05 2024-09-11T08:55:55.539 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.539 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.539 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.539 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.539 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.539 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162425 9.17 9:ee96b337:::benchmark_data_smithi092_145083_object19027:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-11T08:55:55.539 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:51:00.030921+0000", 2024-09-11T08:55:55.539 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 295.37244811800002, 2024-09-11T08:55:55.539 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.921978466, 2024-09-11T08:55:55.539 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.539 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.540 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.540 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.540 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.540 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162425 2024-09-11T08:55:55.540 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.540 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.540 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.540 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.540 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030921+0000", 2024-09-11T08:55:55.540 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.540 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.540 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.540 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.541 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030921+0000", 2024-09-11T08:55:55.541 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.541 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.541 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.541 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.541 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030920+0000", 2024-09-11T08:55:55.541 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:55:55.541 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.541 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.541 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.541 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030923+0000", 2024-09-11T08:55:55.541 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.362e-06 2024-09-11T08:55:55.542 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.542 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.542 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.542 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030929+0000", 2024-09-11T08:55:55.542 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8769999999999998e-06 2024-09-11T08:55:55.542 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.542 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.542 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.542 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030932+0000", 2024-09-11T08:55:55.542 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.32e-06 2024-09-11T08:55:55.542 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.542 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.542 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.542 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030982+0000", 2024-09-11T08:55:55.543 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9997999999999998e-05 2024-09-11T08:55:55.543 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.543 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.543 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.543 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030989+0000", 2024-09-11T08:55:55.543 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8190000000000003e-06 2024-09-11T08:55:55.543 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.543 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.543 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.543 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.936138+0000", 2024-09-11T08:55:55.543 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90514870999999997 2024-09-11T08:55:55.543 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.543 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.544 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.544 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.936278+0000", 2024-09-11T08:55:55.544 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014009 2024-09-11T08:55:55.544 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.544 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.544 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:55:55.544 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.936509+0000", 2024-09-11T08:55:55.544 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00023174599999999999 2024-09-11T08:55:55.544 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.544 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.544 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.544 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.941634+0000", 2024-09-11T08:55:55.544 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0051242279999999998 2024-09-11T08:55:55.544 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.545 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.545 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.545 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952645+0000", 2024-09-11T08:55:55.545 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011011099 2024-09-11T08:55:55.545 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.545 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.545 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.545 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952820+0000", 2024-09-11T08:55:55.545 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00017567400000000001 2024-09-11T08:55:55.545 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.545 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.545 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.545 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952863+0000", 2024-09-11T08:55:55.546 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3123e-05 2024-09-11T08:55:55.546 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.546 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.546 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.546 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952899+0000", 2024-09-11T08:55:55.546 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.578e-05 2024-09-11T08:55:55.546 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.546 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.546 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.546 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.546 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.546 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162450 9.17 9:e99b70b0:::benchmark_data_smithi092_145083_object19052:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-11T08:55:55.546 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:51:00.045692+0000", 2024-09-11T08:55:55.546 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 295.35767650600002, 2024-09-11T08:55:55.547 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90749497800000001, 2024-09-11T08:55:55.547 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.547 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.547 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.547 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.547 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.547 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162450 2024-09-11T08:55:55.547 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.547 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.547 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.547 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.547 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045692+0000", 2024-09-11T08:55:55.547 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.548 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.548 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.548 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.548 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045692+0000", 2024-09-11T08:55:55.548 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.548 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.548 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.548 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.548 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045692+0000", 2024-09-11T08:55:55.548 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:55:55.548 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.548 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.548 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.548 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045696+0000", 2024-09-11T08:55:55.549 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0729999999999998e-06 2024-09-11T08:55:55.549 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.549 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.549 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.549 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045706+0000", 2024-09-11T08:55:55.549 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.9299999999999998e-06 2024-09-11T08:55:55.549 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.549 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.549 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.549 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045712+0000", 2024-09-11T08:55:55.549 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6829999999999996e-06 2024-09-11T08:55:55.549 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.549 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.549 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.550 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045759+0000", 2024-09-11T08:55:55.550 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7027999999999998e-05 2024-09-11T08:55:55.550 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.550 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.550 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.550 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045766+0000", 2024-09-11T08:55:55.550 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9979999999999997e-06 2024-09-11T08:55:55.550 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.550 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.550 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.550 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.937223+0000", 2024-09-11T08:55:55.550 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89145709200000001 2024-09-11T08:55:55.550 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.550 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.551 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.551 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.937283+0000", 2024-09-11T08:55:55.551 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0676e-05 2024-09-11T08:55:55.551 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.551 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.551 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:55:55.551 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.937427+0000", 2024-09-11T08:55:55.551 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000143642 2024-09-11T08:55:55.551 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.551 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.551 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.551 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.941640+0000", 2024-09-11T08:55:55.551 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0042131299999999998 2024-09-11T08:55:55.551 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.552 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.552 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.552 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952942+0000", 2024-09-11T08:55:55.552 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011301774000000001 2024-09-11T08:55:55.552 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.552 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.552 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.552 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953099+0000", 2024-09-11T08:55:55.552 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015739500000000001 2024-09-11T08:55:55.552 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.552 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.552 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.552 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953141+0000", 2024-09-11T08:55:55.553 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1414e-05 2024-09-11T08:55:55.553 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.553 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.553 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.553 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953187+0000", 2024-09-11T08:55:55.553 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6687999999999998e-05 2024-09-11T08:55:55.553 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.553 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.553 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.553 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.553 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.553 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162513 9.17 9:e993ae52:::benchmark_data_smithi092_145083_object19115:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-11T08:55:55.553 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:51:00.087447+0000", 2024-09-11T08:55:55.554 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 295.31592198099997, 2024-09-11T08:55:55.554 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.86587709199999996, 2024-09-11T08:55:55.554 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:55:55.554 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:55:55.554 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:55:55.554 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:55:55.554 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:55:55.554 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162513 2024-09-11T08:55:55.554 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.554 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:55:55.554 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.554 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:55:55.554 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087447+0000", 2024-09-11T08:55:55.554 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.555 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.555 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.555 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:55:55.555 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087447+0000", 2024-09-11T08:55:55.555 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:55:55.555 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.555 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.555 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:55:55.555 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087446+0000", 2024-09-11T08:55:55.555 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:55:55.555 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.555 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.555 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:55:55.555 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087452+0000", 2024-09-11T08:55:55.556 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6439999999999997e-06 2024-09-11T08:55:55.556 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.556 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.556 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:55:55.556 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087466+0000", 2024-09-11T08:55:55.556 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3966e-05 2024-09-11T08:55:55.556 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.556 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.556 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:55:55.556 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087471+0000", 2024-09-11T08:55:55.556 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6849999999999998e-06 2024-09-11T08:55:55.556 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.556 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.557 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.557 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087515+0000", 2024-09-11T08:55:55.557 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3890000000000002e-05 2024-09-11T08:55:55.557 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.557 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.557 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:55:55.557 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087522+0000", 2024-09-11T08:55:55.557 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5899999999999996e-06 2024-09-11T08:55:55.557 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.557 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.557 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:55:55.557 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.938122+0000", 2024-09-11T08:55:55.557 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.85060005800000005 2024-09-11T08:55:55.557 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.558 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.558 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:55:55.558 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.938185+0000", 2024-09-11T08:55:55.558 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2726999999999998e-05 2024-09-11T08:55:55.558 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.558 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.558 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:55:55.558 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.938349+0000", 2024-09-11T08:55:55.558 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000164535 2024-09-11T08:55:55.558 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.558 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.558 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:55:55.558 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.944372+0000", 2024-09-11T08:55:55.558 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0060229369999999999 2024-09-11T08:55:55.559 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.559 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.559 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.559 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952983+0000", 2024-09-11T08:55:55.559 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0086107600000000003 2024-09-11T08:55:55.559 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.559 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.559 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:55:55.559 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953253+0000", 2024-09-11T08:55:55.559 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00027028600000000001 2024-09-11T08:55:55.559 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.559 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.559 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:55:55.559 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953290+0000", 2024-09-11T08:55:55.560 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6860000000000003e-05 2024-09-11T08:55:55.560 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:55:55.560 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:55:55.560 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:55:55.560 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953324+0000", 2024-09-11T08:55:55.560 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4007999999999997e-05 2024-09-11T08:55:55.560 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.560 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.560 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.560 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:55:55.560 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:55:55.560 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:55:55.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:55.562 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:55 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:55:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:55 smithi145 ceph-mon[33384]: osdmap e343: 8 total, 8 up, 8 in 2024-09-11T08:55:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:55 smithi145 ceph-mon[33384]: 10.b scrub starts 2024-09-11T08:55:55.663 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:55.690 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:55.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:55.692 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:55.792 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:55.833 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:55.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:55.834 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:55.935 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:55.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:55.985 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:55.985 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:56.086 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:56.122 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:56.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:56.123 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:56.224 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:56.256 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:56.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:56.257 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:56.358 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:56.391 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:56.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:56.393 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:56.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:56 smithi092 ceph-mon[29647]: 6.15 scrub starts 2024-09-11T08:55:56.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:56 smithi092 ceph-mon[29647]: 6.15 scrub ok 2024-09-11T08:55:56.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:56 smithi092 ceph-mon[29647]: pgmap v777: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 2.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 782 op/s 2024-09-11T08:55:56.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:56 smithi092 ceph-mon[29647]: 10.b scrub ok 2024-09-11T08:55:56.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:56 smithi092 ceph-mon[29647]: 6.3 scrub starts 2024-09-11T08:55:56.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:56 smithi092 ceph-mon[29647]: 6.3 scrub ok 2024-09-11T08:55:56.448 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:56 smithi092 ceph-mon[41114]: 6.15 scrub starts 2024-09-11T08:55:56.448 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:56 smithi092 ceph-mon[41114]: 6.15 scrub ok 2024-09-11T08:55:56.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:56 smithi092 ceph-mon[41114]: pgmap v777: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 2.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 782 op/s 2024-09-11T08:55:56.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:56 smithi092 ceph-mon[41114]: 10.b scrub ok 2024-09-11T08:55:56.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:56 smithi092 ceph-mon[41114]: 6.3 scrub starts 2024-09-11T08:55:56.449 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:56 smithi092 ceph-mon[41114]: 6.3 scrub ok 2024-09-11T08:55:56.494 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:56.530 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:56.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:56.532 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:56.618 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:55:56.619 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:55:56.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:55:56.648 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:55:56.649 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:55:56.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:56 smithi145 ceph-mon[33384]: 6.15 scrub starts 2024-09-11T08:55:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:56 smithi145 ceph-mon[33384]: 6.15 scrub ok 2024-09-11T08:55:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:56 smithi145 ceph-mon[33384]: pgmap v777: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 2.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 782 op/s 2024-09-11T08:55:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:56 smithi145 ceph-mon[33384]: 10.b scrub ok 2024-09-11T08:55:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:56 smithi145 ceph-mon[33384]: 6.3 scrub starts 2024-09-11T08:55:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:56 smithi145 ceph-mon[33384]: 6.3 scrub ok 2024-09-11T08:55:56.667 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:56.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:56.668 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:55:56.677 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:55:56.677 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:55:56.705 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:55:56.705 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:55:56.731 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:55:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:55:56.765 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:55:56.765 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:55:56.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:56.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-11T08:55:56.795 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:55:56.795 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:55:56.809 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:56.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:56.810 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:56.825 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:55:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:55:56.911 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:56.939 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:56.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:56.940 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:57.041 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:57.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:57.077 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:57.077 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:57.178 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:57.221 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:57.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:57.222 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:57.323 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:57.350 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:55:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:55:57.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:57.361 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:57.361 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:57.371 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:55:57.462 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:57.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:57.497 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:57.497 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:57.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:57 smithi092 ceph-mon[29647]: 4.18 scrub starts 2024-09-11T08:55:57.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:57 smithi092 ceph-mon[29647]: 4.18 scrub ok 2024-09-11T08:55:57.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:57 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1142381157' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:55:57.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:57 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3513147225' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:57.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:57 smithi092 ceph-mon[29647]: 3.b scrub starts 2024-09-11T08:55:57.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:57 smithi092 ceph-mon[29647]: 3.b scrub ok 2024-09-11T08:55:57.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:57 smithi092 ceph-mon[41114]: 4.18 scrub starts 2024-09-11T08:55:57.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:57 smithi092 ceph-mon[41114]: 4.18 scrub ok 2024-09-11T08:55:57.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:57 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1142381157' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:55:57.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:57 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3513147225' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:57.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:57 smithi092 ceph-mon[41114]: 3.b scrub starts 2024-09-11T08:55:57.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:57 smithi092 ceph-mon[41114]: 3.b scrub ok 2024-09-11T08:55:57.598 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:55:57.639 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:57.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:57.640 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:55:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:57 smithi145 ceph-mon[33384]: 4.18 scrub starts 2024-09-11T08:55:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:57 smithi145 ceph-mon[33384]: 4.18 scrub ok 2024-09-11T08:55:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:57 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1142381157' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:55:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:57 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3513147225' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:55:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:57 smithi145 ceph-mon[33384]: 3.b scrub starts 2024-09-11T08:55:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:57 smithi145 ceph-mon[33384]: 3.b scrub ok 2024-09-11T08:55:57.741 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:57.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:57.774 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:57.775 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:57.876 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:57.909 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:57.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:57.910 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:58.011 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:55:58.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:58.049 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:58.050 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:55:58.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:55:58.190 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:55:58.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:58.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:55:58.292 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:58.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:58.327 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:58.327 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:58.402 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:55:58.428 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:55:58.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:58.458 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:58.458 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:55:58.559 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:58.593 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:58.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:58.595 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:58.607 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:58 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:58.607 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:58 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:55:58.607 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:58 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1142381157' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:55:58.608 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:58 smithi145 ceph-mon[33384]: osdmap e344: 8 total, 8 up, 8 in 2024-09-11T08:55:58.608 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:55:58 smithi145 ceph-mon[33384]: pgmap v779: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 2.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.67k op/s 2024-09-11T08:55:58.663 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:58 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:58.663 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:58 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:55:58.663 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:58 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1142381157' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:55:58.663 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:58 smithi092 ceph-mon[29647]: osdmap e344: 8 total, 8 up, 8 in 2024-09-11T08:55:58.663 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:55:58 smithi092 ceph-mon[29647]: pgmap v779: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 2.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.67k op/s 2024-09-11T08:55:58.664 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:58 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:55:58.664 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:58 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:55:58.664 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:58 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1142381157' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:55:58.665 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:58 smithi092 ceph-mon[41114]: osdmap e344: 8 total, 8 up, 8 in 2024-09-11T08:55:58.665 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:55:58 smithi092 ceph-mon[41114]: pgmap v779: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 2.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.67k op/s 2024-09-11T08:55:58.695 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:58.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:58.729 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:58.730 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:55:58.830 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:58.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:58.865 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:58.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:58.966 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:59.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:59.008 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:59.008 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:59.109 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:59.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:59.152 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:59.152 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:59.253 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:59.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:59.389 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:59.389 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:59.490 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:59.521 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:59.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:59.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:59.624 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:55:59.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:59.657 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:59.657 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:55:59.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:55:59.758 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:55:59.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:59.791 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:59.791 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:55:59.892 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:55:59.926 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:55:59.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:55:59.927 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:00.028 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:00.059 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:00.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:00.060 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:00.162 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:00.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:00.196 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:00.196 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:00.297 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:00.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:00 smithi092 ceph-mon[29647]: pgmap v780: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 2.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 847 B/s rd, 0 B/s wr, 958 op/s 2024-09-11T08:56:00.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:00 smithi092 ceph-mon[41114]: pgmap v780: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 2.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 847 B/s rd, 0 B/s wr, 958 op/s 2024-09-11T08:56:00.341 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:00.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:00.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:00.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:00 smithi145 ceph-mon[33384]: pgmap v780: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 2.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 847 B/s rd, 0 B/s wr, 958 op/s 2024-09-11T08:56:00.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:56:00.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:00.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:00.519 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:00.563 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:56:00.564 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:56:00.564 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-11T08:56:00.564 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.000000 2024-09-11T08:56:00.564 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 6 0 2024-09-11T08:56:00.620 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:00.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:00.657 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:00.657 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:00.758 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:00.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:00.794 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:00.794 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:00.895 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:00.930 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:00.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:00.931 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:01.032 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:01.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:01.066 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:01.066 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:01.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:01 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:56:01.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:01 smithi145 ceph-mon[33384]: 10.3 deep-scrub ok 2024-09-11T08:56:01.167 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:01.200 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:01.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:01.201 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:01.302 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:01.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:01.333 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:01.333 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:01.419 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:01 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:56:01.419 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:01 smithi092 ceph-mon[29647]: 10.3 deep-scrub ok 2024-09-11T08:56:01.419 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:01 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:56:01.419 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:01 smithi092 ceph-mon[41114]: 10.3 deep-scrub ok 2024-09-11T08:56:01.434 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:01.459 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:01.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:01.460 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:01.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:01.596 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:01.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:01.597 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:01.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:01.736 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:01.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:01.738 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:01.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:56:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:56:01] "GET /metrics HTTP/1.1" 200 224852 "" "Prometheus/2.43.0" 2024-09-11T08:56:01.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:01.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:01.881 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:01.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:01.982 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:02.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:02.017 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:02.017 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:02.052 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:02.090 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:56:02.090 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:56:02.118 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:02.150 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:02.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:02.151 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:02.219 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:02 smithi092 ceph-mon[29647]: pgmap v781: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 2.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 767 op/s 2024-09-11T08:56:02.219 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:02 smithi092 ceph-mon[41114]: pgmap v781: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 2.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 767 op/s 2024-09-11T08:56:02.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:02.288 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:02.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:02.289 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:02.354 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:02.390 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:02.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:02 smithi145 ceph-mon[33384]: pgmap v781: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 2.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 767 op/s 2024-09-11T08:56:02.429 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:02.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:02.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:02.532 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:02.570 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:02.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:02.571 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:02.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:02.705 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:02.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:02.706 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:02.807 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:02.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:02.841 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:02.942 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:02.973 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:02.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:02.974 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:03.075 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:03.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:03.109 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:03.109 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:03.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:56:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:56:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:56:03.210 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:03.216 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-11T08:56:03.242 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:03.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:03.243 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:03.344 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:03.374 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:03.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:03.375 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:03.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:03.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:03.518 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:03.518 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:03.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:03.653 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:03.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:03.654 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:03.755 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:03.791 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:03.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:03.792 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:03.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:03.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:03.932 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:03.932 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:04.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:04.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:04.074 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:04.074 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:04.175 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:04.207 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:04.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:04.208 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:04.309 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:04.344 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:04.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:04.345 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:04.446 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:04.478 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:04.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:04.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:04.580 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:04.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:04.619 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:04.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:04.720 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:04.753 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:04.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:04.754 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:04.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:04 smithi092 ceph-mon[29647]: pgmap v782: 158 pgs: 158 active+clean; 2.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.28k op/s 2024-09-11T08:56:04.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:04 smithi092 ceph-mon[41114]: pgmap v782: 158 pgs: 158 active+clean; 2.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.28k op/s 2024-09-11T08:56:04.855 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:04.884 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:04.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:04.885 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:04.890 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:04.903 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:04 smithi145 ceph-mon[33384]: pgmap v782: 158 pgs: 158 active+clean; 2.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.28k op/s 2024-09-11T08:56:04.987 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:05.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:05.009 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:05.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:05.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:05.149 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:05.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:05.150 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:05.251 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:05.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:05.285 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:05.285 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:05.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:05.430 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:05.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:05.432 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:05.532 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:05.575 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:05.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:05.576 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:05.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:05.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:05.718 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:05.818 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:05.859 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:05.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:05.861 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:05.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:06.000 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:06.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:06.001 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:06.102 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:06.133 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:06.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:06.134 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:06.235 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:06.267 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:06.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:06.268 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:06.369 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:06.396 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:06.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:06.397 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:06.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:06 smithi092 ceph-mon[29647]: pgmap v783: 158 pgs: 158 active+clean; 2.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-11T08:56:06.460 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:06 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2644493013' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:56:06.460 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:06 smithi092 ceph-mon[41114]: pgmap v783: 158 pgs: 158 active+clean; 2.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-11T08:56:06.460 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:06 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2644493013' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:56:06.497 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:06.533 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:06.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:06.534 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:06.635 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:06.670 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:06.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:06.671 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:06.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:06.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:06.813 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:06.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:06 smithi145 ceph-mon[33384]: pgmap v783: 158 pgs: 158 active+clean; 2.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-11T08:56:06.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:06 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2644493013' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:56:06.913 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:06.935 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:06.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:06.936 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:07.037 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:07.070 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:07.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:07.071 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:07.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:07.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:07.195 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:07.195 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:07.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:07.329 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:07.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:07.330 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:07.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:56:07 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:56:07.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:07.456 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.6 primary-affinity to 0 (802) 2024-09-11T08:56:07.469 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:07.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:07.470 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:07.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:07.592 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:07.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:07.593 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:07.694 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:07.716 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:07.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:07.717 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:07.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:07 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3434613607' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-11T08:56:07.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:07 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3434613607' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-11T08:56:07.818 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:07.840 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:07.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:07.841 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:07.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:07 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3434613607' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-11T08:56:07.942 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:07.966 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:07.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:07.967 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:08.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:08.091 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:08.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:08.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:08.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:08.216 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:08.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:08.218 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:08.319 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:08.344 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:08.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:08.345 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:08.446 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:08.475 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:08.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:08.476 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:08.577 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:08.613 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:08.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:08.615 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:08.716 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:08.751 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:08.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:08.752 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:08.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:08 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:08.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:08 smithi092 ceph-mon[29647]: pgmap v784: 158 pgs: 158 active+clean; 1.8 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T08:56:08.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:08 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3434613607' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]': finished 2024-09-11T08:56:08.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:08 smithi092 ceph-mon[29647]: osdmap e345: 8 total, 8 up, 8 in 2024-09-11T08:56:08.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:08 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:08.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:08 smithi092 ceph-mon[41114]: pgmap v784: 158 pgs: 158 active+clean; 1.8 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T08:56:08.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:08 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3434613607' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]': finished 2024-09-11T08:56:08.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:08 smithi092 ceph-mon[41114]: osdmap e345: 8 total, 8 up, 8 in 2024-09-11T08:56:08.853 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:08.890 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:08.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:08.891 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:08 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:08 smithi145 ceph-mon[33384]: pgmap v784: 158 pgs: 158 active+clean; 1.8 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T08:56:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:08 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3434613607' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]': finished 2024-09-11T08:56:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:08 smithi145 ceph-mon[33384]: osdmap e345: 8 total, 8 up, 8 in 2024-09-11T08:56:08.992 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:09.027 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:09.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:09.028 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:09.129 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:09.160 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:09.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:09.161 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:09.262 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:09.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:09.298 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:09.298 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:09.399 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:09.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:09.435 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:09.435 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:09.536 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:09.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:09 smithi092 ceph-mon[29647]: osdmap e346: 8 total, 8 up, 8 in 2024-09-11T08:56:09.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:09 smithi092 ceph-mon[41114]: osdmap e346: 8 total, 8 up, 8 in 2024-09-11T08:56:09.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:09.573 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:09.573 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:09.674 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:09.715 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:09.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:09.716 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:09.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:09.850 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:09.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:09.851 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:09.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:09 smithi145 ceph-mon[33384]: osdmap e346: 8 total, 8 up, 8 in 2024-09-11T08:56:09.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:09.987 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:09.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:09.988 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:10.089 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:10.119 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:10.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:10.121 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:10.187 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:56:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:56:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:56:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:56:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:56:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:56:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:56:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:56:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:56:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:56:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:56:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:56:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:56:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:56:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:56:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:56:10.221 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:10.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:10.254 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:10.254 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:10.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:10.392 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:10.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:10.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:10.494 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:10.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:10.526 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:10.526 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:10.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:10.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:10.666 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:10.666 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:10.751 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:10 smithi092 ceph-mon[29647]: pgmap v787: 158 pgs: 158 active+clean; 1.8 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-11T08:56:10.751 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:10 smithi092 ceph-mon[41114]: pgmap v787: 158 pgs: 158 active+clean; 1.8 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-11T08:56:10.768 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:10.799 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:10.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:10.800 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:10.901 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:10.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:10 smithi145 ceph-mon[33384]: pgmap v787: 158 pgs: 158 active+clean; 1.8 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-11T08:56:10.936 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:10.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:10.937 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:11.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:11.071 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:11.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:11.072 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:11.173 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:11.205 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:11.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:11.206 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:11.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:11.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:11.353 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:11.353 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:11.454 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:11.485 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:11.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:11.486 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:11.587 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:11.620 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:11.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:11.621 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:11.622 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:56:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:56:11] "GET /metrics HTTP/1.1" 200 224852 "" "Prometheus/2.43.0" 2024-09-11T08:56:11.722 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:11.756 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:11.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:11.757 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:11.858 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:11.887 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:11.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:11.889 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:11.989 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:12.021 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:12.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:12.022 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:12.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:12.160 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:12.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:12.161 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:12.262 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:12.293 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:12.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:12.294 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:12.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:12.430 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:12.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:12.431 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:12.532 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:12.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:12.571 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:12.671 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:12.704 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:12.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:12.705 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:12.806 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:12 smithi092 ceph-mon[29647]: pgmap v788: 158 pgs: 158 active+clean; 1.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 5.9 KiB/s rd, 37 KiB/s wr, 933 op/s; 13 KiB/s, 0 objects/s recovering 2024-09-11T08:56:12.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:12 smithi092 ceph-mon[41114]: pgmap v788: 158 pgs: 158 active+clean; 1.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 5.9 KiB/s rd, 37 KiB/s wr, 933 op/s; 13 KiB/s, 0 objects/s recovering 2024-09-11T08:56:12.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:12.840 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:12.840 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:12.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:56:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:56:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:56:12.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:12 smithi145 ceph-mon[33384]: pgmap v788: 158 pgs: 158 active+clean; 1.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 5.9 KiB/s rd, 37 KiB/s wr, 933 op/s; 13 KiB/s, 0 objects/s recovering 2024-09-11T08:56:12.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:12.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:12.984 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:12.984 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:13.086 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:13.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:13.120 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:13.120 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:13.221 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:13.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:13.262 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:13.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:13.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:13.398 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:13.398 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:13.499 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:13.528 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:13.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:13.529 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:13.629 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:13.660 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:13.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:13.661 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:13.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:13.801 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:13.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:13.802 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:13.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:13.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:13.943 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:13.943 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:14.044 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:14.079 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:14.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:14.080 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:14.085 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:56:14.180 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:14.212 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:14.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:14.213 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:14.314 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:14.341 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:14.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:14.342 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:14.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:14.465 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:56:14.465 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:56:14.465 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:56:14.465 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:56:14.465 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:56:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:14.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:14.499 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:14.599 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:14.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:14.631 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:14.631 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:14.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:14.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:14.769 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:14.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:14 smithi092 ceph-mon[41114]: pgmap v789: 158 pgs: 158 active+clean; 1.5 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 8.9 KiB/s rd, 55 KiB/s wr, 1.45k op/s; 26 KiB/s, 0 objects/s recovering 2024-09-11T08:56:14.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:14 smithi092 ceph-mon[29647]: pgmap v789: 158 pgs: 158 active+clean; 1.5 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 8.9 KiB/s rd, 55 KiB/s wr, 1.45k op/s; 26 KiB/s, 0 objects/s recovering 2024-09-11T08:56:14.870 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:14.904 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:14.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:14.905 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:14.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:14 smithi145 ceph-mon[33384]: pgmap v789: 158 pgs: 158 active+clean; 1.5 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 8.9 KiB/s rd, 55 KiB/s wr, 1.45k op/s; 26 KiB/s, 0 objects/s recovering 2024-09-11T08:56:15.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:15.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:15.046 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:15.046 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:15.147 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:15.191 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:15.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:15.192 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:15.293 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:15.328 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:15.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:15.329 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:15.430 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:15.448 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:15.458 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:15.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:15.459 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:15.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-11T08:56:15.497 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:56:15.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:15.608 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:15.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:15.609 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:15.710 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:15.746 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:15.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:15.747 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:15.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:15 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2256885262' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:56:15.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:15 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:56:15.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:56:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:15 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2256885262' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:56:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:15 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:56:15.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:56:15.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:15.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:15.891 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:15.891 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:15.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:15 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2256885262' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:56:15.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:15 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:56:15.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:56:15.992 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:16.036 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:16.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:16.037 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:16.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:16.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:16.173 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:16.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:16.301 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:16.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:16.302 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:16.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:16.443 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:16.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:16.444 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:16.545 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:16.577 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:16.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:16.578 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:16.679 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:16.719 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:16.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:16.720 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:16.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:16 smithi092 ceph-mon[29647]: pgmap v790: 158 pgs: 158 active+clean; 1.5 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 8.2 KiB/s rd, 55 KiB/s wr, 632 op/s; 26 KiB/s, 0 objects/s recovering 2024-09-11T08:56:16.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:16 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:56:16.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:16 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:56:16.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:16 smithi092 ceph-mon[29647]: osdmap e347: 8 total, 8 up, 8 in 2024-09-11T08:56:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:16 smithi092 ceph-mon[41114]: pgmap v790: 158 pgs: 158 active+clean; 1.5 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 8.2 KiB/s rd, 55 KiB/s wr, 632 op/s; 26 KiB/s, 0 objects/s recovering 2024-09-11T08:56:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:16 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:56:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:16 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:56:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:16 smithi092 ceph-mon[41114]: osdmap e347: 8 total, 8 up, 8 in 2024-09-11T08:56:16.821 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:16.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:16.861 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:16.861 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:16.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:16 smithi145 ceph-mon[33384]: pgmap v790: 158 pgs: 158 active+clean; 1.5 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 8.2 KiB/s rd, 55 KiB/s wr, 632 op/s; 26 KiB/s, 0 objects/s recovering 2024-09-11T08:56:16.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:16 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:56:16.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:16 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:56:16.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:16 smithi145 ceph-mon[33384]: osdmap e347: 8 total, 8 up, 8 in 2024-09-11T08:56:16.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:16.996 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:16.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:16.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:17.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:17.136 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:17.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:17.137 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:17.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:17.274 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:17.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:17.275 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:17.376 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:17.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:17.409 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:17.409 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:17.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:56:17 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:56:17.510 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:17.537 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:17.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:17.538 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:17.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:17.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:17.675 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:17.776 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:17.810 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:17.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:17.811 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:17.912 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:17.933 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:17.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:17.934 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:17.958 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:56:17.958 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:56:17.959 INFO:tasks.thrashosds.thrasher:Reweighting osd 1 to 0.7004276751899566 2024-09-11T08:56:17.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 1 0.7004276751899566 2024-09-11T08:56:18.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:18.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:18.080 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:18.080 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:18.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:18.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:18.255 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:18.255 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:18.356 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:18.383 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:18.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:18.384 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:18.485 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:18.522 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:18.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:18.524 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:18.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:18.670 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:18.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:18.671 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:18.695 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:18.772 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:18.806 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:18.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:18.807 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:18.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:18 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:18.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:18 smithi092 ceph-mon[29647]: pgmap v792: 158 pgs: 158 active+clean; 1.1 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 8.0 KiB/s rd, 50 KiB/s wr, 1.29k op/s; 23 KiB/s, 0 objects/s recovering 2024-09-11T08:56:18.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:18 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:18.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:18 smithi092 ceph-mon[41114]: pgmap v792: 158 pgs: 158 active+clean; 1.1 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 8.0 KiB/s rd, 50 KiB/s wr, 1.29k op/s; 23 KiB/s, 0 objects/s recovering 2024-09-11T08:56:18.908 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:18.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:18 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:18.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:18 smithi145 ceph-mon[33384]: pgmap v792: 158 pgs: 158 active+clean; 1.1 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 8.0 KiB/s rd, 50 KiB/s wr, 1.29k op/s; 23 KiB/s, 0 objects/s recovering 2024-09-11T08:56:18.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:18.942 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:18.942 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:19.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:19.080 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:19.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:19.081 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:19.182 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:19.215 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:19.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:19.217 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:19.312 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:19.317 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:19.347 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:19.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:19.348 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:19.449 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:19.480 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:19.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:19.481 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:19.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:19.625 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:19.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:19.626 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:19.727 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:19.756 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:19.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:19.757 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:19.857 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:19.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:19.890 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:19.890 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:19.991 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:20.024 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:20.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:20.025 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:20.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:20.162 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:20.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:20.163 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:20.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:56:20.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:20.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:20.316 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:20.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:20 smithi145 ceph-mon[33384]: pgmap v793: 158 pgs: 158 active+clean; 1.1 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 7.1 KiB/s rd, 44 KiB/s wr, 1.16k op/s; 21 KiB/s, 0 objects/s recovering 2024-09-11T08:56:20.417 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:20.451 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:20.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:20.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:20.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:20 smithi092 ceph-mon[29647]: pgmap v793: 158 pgs: 158 active+clean; 1.1 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 7.1 KiB/s rd, 44 KiB/s wr, 1.16k op/s; 21 KiB/s, 0 objects/s recovering 2024-09-11T08:56:20.458 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:20 smithi092 ceph-mon[41114]: pgmap v793: 158 pgs: 158 active+clean; 1.1 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 7.1 KiB/s rd, 44 KiB/s wr, 1.16k op/s; 21 KiB/s, 0 objects/s recovering 2024-09-11T08:56:20.553 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:20.587 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:20.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:20.588 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:20.689 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:20.731 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:20.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:20.732 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:20.796 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:56:20.797 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:56:20.826 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:56:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:56:20.833 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:20.852 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:56:20.853 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:56:20.863 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:20.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:20.864 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:20.886 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:56:20.886 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:56:20.913 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:56:20.914 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:56:20.944 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:56:20.944 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:56:20.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:20.979 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:56:20.979 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:56:21.008 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:21.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:21.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:21.013 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:56:21.013 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:56:21.111 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:21.141 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:21.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:21.142 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:21.243 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:21.274 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:21.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:21.275 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:21.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:21 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4248392614' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:56:21.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:21 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4248392614' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:56:21.376 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:21.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:21 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4248392614' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:56:21.413 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:21.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:21.414 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:21.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:21.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:21.548 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:21.548 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:21.649 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:21.684 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:21.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:21.685 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:21.786 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:21.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:56:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:56:21] "GET /metrics HTTP/1.1" 200 224832 "" "Prometheus/2.43.0" 2024-09-11T08:56:21.824 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:21.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:21.825 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:21.926 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:21.956 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:21.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:21.957 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:22.058 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:22.094 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:22.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:22.095 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:22.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:22.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:22.241 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:22.241 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:22.294 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:22 smithi092 ceph-mon[41114]: pgmap v794: 158 pgs: 158 active+clean; 1.1 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.9 KiB/s rd, 15 KiB/s wr, 1.19k op/s; 10 KiB/s, 0 objects/s recovering 2024-09-11T08:56:22.295 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:22 smithi092 ceph-mon[29647]: pgmap v794: 158 pgs: 158 active+clean; 1.1 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.9 KiB/s rd, 15 KiB/s wr, 1.19k op/s; 10 KiB/s, 0 objects/s recovering 2024-09-11T08:56:22.342 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:22.376 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:22.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:22.377 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:22.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:22 smithi145 ceph-mon[33384]: pgmap v794: 158 pgs: 158 active+clean; 1.1 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.9 KiB/s rd, 15 KiB/s wr, 1.19k op/s; 10 KiB/s, 0 objects/s recovering 2024-09-11T08:56:22.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:22.517 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:22.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:22.519 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:22.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:22.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:22.652 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:22.652 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:22.753 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:22.787 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:22.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:22.788 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:22.889 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:22.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:22.926 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:22.926 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:23.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:23.068 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:23.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:23.069 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:23.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:56:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:56:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:56:23.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:23.208 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:23.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:23.212 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:23.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:23.353 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:23.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:23.355 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:23.455 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:23.490 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:23.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:23.491 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:23.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:23.631 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:23.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:23.632 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:23.733 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:23.761 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:23.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:23.762 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:23.863 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:23.889 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:23.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:23.890 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:23.991 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:24.023 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:24.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:24.024 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:24.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:24.160 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:24.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:24.161 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:24.196 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:56:24.196 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:56:24.204 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:24.262 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:24.300 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:24.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:24.301 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:24.402 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:24.459 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:24.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:24.460 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.1 to 0.700428 (b34f) 2024-09-11T08:56:24.562 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:24.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:24.592 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:24.592 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:24.693 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:24.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:24.714 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:24.714 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:24.790 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:24 smithi092 ceph-mon[29647]: pgmap v795: 158 pgs: 158 active+clean; 747 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s 2024-09-11T08:56:24.790 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:24 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4026172278' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 0.7004276751899566}]: dispatch 2024-09-11T08:56:24.790 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:24 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 0.7004276751899566}]: dispatch 2024-09-11T08:56:24.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:24 smithi092 ceph-mon[41114]: pgmap v795: 158 pgs: 158 active+clean; 747 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s 2024-09-11T08:56:24.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:24 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4026172278' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 0.7004276751899566}]: dispatch 2024-09-11T08:56:24.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:24 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 0.7004276751899566}]: dispatch 2024-09-11T08:56:24.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:24.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:24.837 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:24.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:24 smithi145 ceph-mon[33384]: pgmap v795: 158 pgs: 158 active+clean; 747 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s 2024-09-11T08:56:24.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:24 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4026172278' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 0.7004276751899566}]: dispatch 2024-09-11T08:56:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:24 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 0.7004276751899566}]: dispatch 2024-09-11T08:56:24.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:24.958 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:24.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:24.960 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:25.060 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:25.082 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:25.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:25.084 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:25.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:25.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-11T08:56:25.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:25.207 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:25.309 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:25.331 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:25.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:25.333 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:25.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:25.469 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:25.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:25.470 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:25.471 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:25 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 0.7004276751899566}]': finished 2024-09-11T08:56:25.471 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:25 smithi092 ceph-mon[29647]: osdmap e348: 8 total, 8 up, 8 in 2024-09-11T08:56:25.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:25 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:56:25.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:25 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:56:25.541 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:56:25.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:25.608 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:25.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:25.609 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:25.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:25.732 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:25.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:25.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:25.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:25 smithi092 ceph-mon[29647]: osdmap e349: 8 total, 8 up, 8 in 2024-09-11T08:56:25.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:25 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 0.7004276751899566}]': finished 2024-09-11T08:56:25.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:25 smithi092 ceph-mon[41114]: osdmap e348: 8 total, 8 up, 8 in 2024-09-11T08:56:25.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:25 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:56:25.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:25 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:56:25.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:25 smithi092 ceph-mon[41114]: osdmap e349: 8 total, 8 up, 8 in 2024-09-11T08:56:25.834 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:25.858 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:25.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:25.859 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:25.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:25 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 0.7004276751899566}]': finished 2024-09-11T08:56:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:25 smithi145 ceph-mon[33384]: osdmap e348: 8 total, 8 up, 8 in 2024-09-11T08:56:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:25 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:56:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:25 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:56:25.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:25 smithi145 ceph-mon[33384]: osdmap e349: 8 total, 8 up, 8 in 2024-09-11T08:56:25.923 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:25.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:25.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:25.983 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:25.983 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:26.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:26.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:26.107 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:26.208 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:26.233 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:26.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:26.235 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:26.335 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:26.360 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:26.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:26.361 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:26.462 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:26.486 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:26.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:26.487 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:26.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:26.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:26.627 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:26.728 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:26.741 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:26 smithi092 ceph-mon[29647]: pgmap v797: 158 pgs: 158 active+clean; 747 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T08:56:26.741 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:26 smithi092 ceph-mon[29647]: osdmap e350: 8 total, 8 up, 8 in 2024-09-11T08:56:26.741 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:26 smithi092 ceph-mon[41114]: pgmap v797: 158 pgs: 158 active+clean; 747 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T08:56:26.741 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:26 smithi092 ceph-mon[41114]: osdmap e350: 8 total, 8 up, 8 in 2024-09-11T08:56:26.745 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:26.759 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:26.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:26.760 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:26.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:26.905 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:26.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:26.906 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:26.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:26 smithi145 ceph-mon[33384]: pgmap v797: 158 pgs: 158 active+clean; 747 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T08:56:26.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:26 smithi145 ceph-mon[33384]: osdmap e350: 8 total, 8 up, 8 in 2024-09-11T08:56:27.007 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:27.037 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:27.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:27.038 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:27.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:27.173 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:27.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:27.174 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:27.275 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:27.312 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:27.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:27.313 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:27.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:56:27 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:56:27.414 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:27.452 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:27.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:27.453 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:27.554 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:27.585 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:27.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:27.587 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:27.687 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:27.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:27.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:27.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:27 smithi092 ceph-mon[29647]: 10.2 starting backfill to osd.3 from (0'0,0'0] MAX to 347'16489 2024-09-11T08:56:27.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:27 smithi092 ceph-mon[41114]: 10.2 starting backfill to osd.3 from (0'0,0'0] MAX to 347'16489 2024-09-11T08:56:27.833 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:27.861 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:27.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:27.862 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:27.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:27 smithi145 ceph-mon[33384]: 10.2 starting backfill to osd.3 from (0'0,0'0] MAX to 347'16489 2024-09-11T08:56:27.963 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:27.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:27.995 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:27.995 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:28.096 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:28.126 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:28.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:28.127 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:28.228 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:28.257 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:28.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:28.258 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:28.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:28.394 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:28.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:28.395 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:28.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:28.539 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:28.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:28.540 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:28.644 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:28.676 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:28.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:28.679 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:28.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:28 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:28 smithi145 ceph-mon[33384]: pgmap v800: 158 pgs: 1 remapped+peering, 157 active+clean; 480 MiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 1.60k op/s; 124 B/s, 4 objects/s recovering 2024-09-11T08:56:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:28 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/89066776' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:56:28.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:28 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:28.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:28 smithi092 ceph-mon[29647]: pgmap v800: 158 pgs: 1 remapped+peering, 157 active+clean; 480 MiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 1.60k op/s; 124 B/s, 4 objects/s recovering 2024-09-11T08:56:28.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:28 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/89066776' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:56:28.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:28 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:28.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:28 smithi092 ceph-mon[41114]: pgmap v800: 158 pgs: 1 remapped+peering, 157 active+clean; 480 MiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 1.60k op/s; 124 B/s, 4 objects/s recovering 2024-09-11T08:56:28.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:28 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/89066776' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:56:28.821 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:28.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:28.822 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:28.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:28.932 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:56:28.957 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:28.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:28.958 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:29.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:29.124 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:29.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:29.125 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:29.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:29.253 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:29.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:29.254 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:29.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:29.388 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:29.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:29.389 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:29.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:29.525 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:29.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:29.526 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:29.627 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:29.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:29.665 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:29.665 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:29.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:29.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:29.804 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:29.804 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:29.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:29 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:56:29.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:29 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/89066776' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:56:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:29 smithi092 ceph-mon[29647]: osdmap e351: 8 total, 8 up, 8 in 2024-09-11T08:56:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:29 smithi092 ceph-mon[29647]: 3.10 scrub starts 2024-09-11T08:56:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:29 smithi092 ceph-mon[29647]: 3.10 scrub ok 2024-09-11T08:56:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:29 smithi092 ceph-mon[29647]: 4.4 scrub starts 2024-09-11T08:56:29.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:29 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:56:29.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:29 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/89066776' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:56:29.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:29 smithi092 ceph-mon[41114]: osdmap e351: 8 total, 8 up, 8 in 2024-09-11T08:56:29.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:29 smithi092 ceph-mon[41114]: 3.10 scrub starts 2024-09-11T08:56:29.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:29 smithi092 ceph-mon[41114]: 3.10 scrub ok 2024-09-11T08:56:29.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:29 smithi092 ceph-mon[41114]: 4.4 scrub starts 2024-09-11T08:56:29.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:29.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:29 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:56:29.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:29 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/89066776' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:56:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:29 smithi145 ceph-mon[33384]: osdmap e351: 8 total, 8 up, 8 in 2024-09-11T08:56:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:29 smithi145 ceph-mon[33384]: 3.10 scrub starts 2024-09-11T08:56:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:29 smithi145 ceph-mon[33384]: 3.10 scrub ok 2024-09-11T08:56:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:29 smithi145 ceph-mon[33384]: 4.4 scrub starts 2024-09-11T08:56:29.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:29.956 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:29.956 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:30.057 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:30.115 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:30.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:30.116 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:30.216 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:30.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:30.259 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:30.259 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:30.360 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:30.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:30.382 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:30.382 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:30.483 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:30.505 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:30.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:30.506 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:30.607 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:30.628 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:30.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:30.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:30.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:30.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:30.756 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:30 smithi092 ceph-mon[29647]: pgmap v802: 158 pgs: 1 remapped+peering, 157 active+clean; 480 MiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 866 op/s; 151 B/s, 5 objects/s recovering 2024-09-11T08:56:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:30 smithi092 ceph-mon[29647]: 6.18 scrub starts 2024-09-11T08:56:30.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:30 smithi092 ceph-mon[29647]: 6.18 scrub ok 2024-09-11T08:56:30.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:56:30.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:30 smithi092 ceph-mon[29647]: osdmap e352: 8 total, 8 up, 8 in 2024-09-11T08:56:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:30 smithi092 ceph-mon[41114]: pgmap v802: 158 pgs: 1 remapped+peering, 157 active+clean; 480 MiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 866 op/s; 151 B/s, 5 objects/s recovering 2024-09-11T08:56:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:30 smithi092 ceph-mon[41114]: 6.18 scrub starts 2024-09-11T08:56:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:30 smithi092 ceph-mon[41114]: 6.18 scrub ok 2024-09-11T08:56:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:56:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:30 smithi092 ceph-mon[41114]: osdmap e352: 8 total, 8 up, 8 in 2024-09-11T08:56:30.857 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:30.879 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:30.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:30.880 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:30.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:30 smithi145 ceph-mon[33384]: pgmap v802: 158 pgs: 1 remapped+peering, 157 active+clean; 480 MiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 866 op/s; 151 B/s, 5 objects/s recovering 2024-09-11T08:56:30.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:30 smithi145 ceph-mon[33384]: 6.18 scrub starts 2024-09-11T08:56:30.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:30 smithi145 ceph-mon[33384]: 6.18 scrub ok 2024-09-11T08:56:30.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:56:30.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:30 smithi145 ceph-mon[33384]: osdmap e352: 8 total, 8 up, 8 in 2024-09-11T08:56:30.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:31.005 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:31.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:31.006 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:31.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:31.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:31.135 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:31.236 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:31.272 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:31.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:31.273 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:31.374 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:31.409 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:31.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:31.411 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:31.512 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:31.546 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:31.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:31.547 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:31.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:31.675 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:31.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:31.676 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:31.777 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:31 smithi092 ceph-mon[29647]: 5.c scrub starts 2024-09-11T08:56:31.777 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:31 smithi092 ceph-mon[29647]: 5.c scrub ok 2024-09-11T08:56:31.777 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:31 smithi092 ceph-mon[29647]: osdmap e353: 8 total, 8 up, 8 in 2024-09-11T08:56:31.777 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:31 smithi092 ceph-mon[41114]: 5.c scrub starts 2024-09-11T08:56:31.777 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:31 smithi092 ceph-mon[41114]: 5.c scrub ok 2024-09-11T08:56:31.777 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:31 smithi092 ceph-mon[41114]: osdmap e353: 8 total, 8 up, 8 in 2024-09-11T08:56:31.778 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:56:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:56:31] "GET /metrics HTTP/1.1" 200 224858 "" "Prometheus/2.43.0" 2024-09-11T08:56:31.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:31.815 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:31.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:31.816 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:31.847 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:56:31.847 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:56:31.882 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:56:31.882 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:56:31.911 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:56:31.911 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:56:31.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:31 smithi145 ceph-mon[33384]: 5.c scrub starts 2024-09-11T08:56:31.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:31 smithi145 ceph-mon[33384]: 5.c scrub ok 2024-09-11T08:56:31.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:31 smithi145 ceph-mon[33384]: osdmap e353: 8 total, 8 up, 8 in 2024-09-11T08:56:31.917 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:31.943 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:56:31.943 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:56:31.945 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:31.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:31.946 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:31.980 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:56:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:56:32.011 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:56:32.012 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:56:32.044 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:56:32.044 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:56:32.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:32.079 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:56:32.079 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:56:32.088 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:32.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:32.090 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:32.191 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:32.227 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:56:32.228 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:56:32.228 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_3 2024-09-11T08:56:32.228 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:56:32.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:32.230 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:32.230 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:32.332 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:32.375 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:56:32.376 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:56:32.376 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-11T08:56:32.376 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:56:32.401 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:32.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:32.402 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:32.503 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:32.544 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:32.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:32.546 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:32 smithi092 ceph-mon[29647]: pgmap v805: 158 pgs: 1 remapped+peering, 157 active+clean; 420 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 414 B/s rd, 0 B/s wr, 548 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:56:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:32 smithi092 ceph-mon[29647]: 5.e scrub starts 2024-09-11T08:56:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:32 smithi092 ceph-mon[29647]: 5.e scrub ok 2024-09-11T08:56:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:32 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4012280635' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:56:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:32 smithi092 ceph-mon[29647]: 3.15 scrub starts 2024-09-11T08:56:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:32 smithi092 ceph-mon[29647]: 3.15 scrub ok 2024-09-11T08:56:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:32 smithi092 ceph-mon[41114]: pgmap v805: 158 pgs: 1 remapped+peering, 157 active+clean; 420 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 414 B/s rd, 0 B/s wr, 548 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:56:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:32 smithi092 ceph-mon[41114]: 5.e scrub starts 2024-09-11T08:56:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:32 smithi092 ceph-mon[41114]: 5.e scrub ok 2024-09-11T08:56:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:32 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4012280635' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:56:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:32 smithi092 ceph-mon[41114]: 3.15 scrub starts 2024-09-11T08:56:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:32 smithi092 ceph-mon[41114]: 3.15 scrub ok 2024-09-11T08:56:32.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:32.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:32.688 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:32.688 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:32.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:32.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:32.823 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:32.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:56:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:56:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:56:32.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:32 smithi145 ceph-mon[33384]: pgmap v805: 158 pgs: 1 remapped+peering, 157 active+clean; 420 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 414 B/s rd, 0 B/s wr, 548 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:56:32.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:32 smithi145 ceph-mon[33384]: 5.e scrub starts 2024-09-11T08:56:32.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:32 smithi145 ceph-mon[33384]: 5.e scrub ok 2024-09-11T08:56:32.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:32 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4012280635' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:56:32.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:32 smithi145 ceph-mon[33384]: 3.15 scrub starts 2024-09-11T08:56:32.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:32 smithi145 ceph-mon[33384]: 3.15 scrub ok 2024-09-11T08:56:32.924 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:32.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:32.958 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:32.958 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:33.058 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:33.093 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:33.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:33.094 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:33.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:33.233 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:33.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:33.234 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:33.302 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Removed 136956 objects 2024-09-11T08:56:33.303 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Clean up completed and total clean up time :118.393 2024-09-11T08:56:33.336 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:33.357 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:33.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:33.358 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:33.459 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:33.479 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:33.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:33.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:33.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:33.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:56:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:33.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:33.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:33.704 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:33.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:33.726 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:33.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:33.768 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:33 smithi092 ceph-mon[41114]: 3.f scrub starts 2024-09-11T08:56:33.768 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:33 smithi092 ceph-mon[41114]: 3.f scrub ok 2024-09-11T08:56:33.769 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:33 smithi092 ceph-mon[41114]: 6.6 scrub starts 2024-09-11T08:56:33.769 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:33 smithi092 ceph-mon[41114]: 6.6 scrub ok 2024-09-11T08:56:33.769 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:33 smithi092 ceph-mon[41114]: 5.1a scrub starts 2024-09-11T08:56:33.769 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:33 smithi092 ceph-mon[41114]: 5.1a scrub ok 2024-09-11T08:56:33.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:33 smithi092 ceph-mon[29647]: 3.f scrub starts 2024-09-11T08:56:33.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:33 smithi092 ceph-mon[29647]: 3.f scrub ok 2024-09-11T08:56:33.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:33 smithi092 ceph-mon[29647]: 6.6 scrub starts 2024-09-11T08:56:33.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:33 smithi092 ceph-mon[29647]: 6.6 scrub ok 2024-09-11T08:56:33.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:33 smithi092 ceph-mon[29647]: 5.1a scrub starts 2024-09-11T08:56:33.770 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:33 smithi092 ceph-mon[29647]: 5.1a scrub ok 2024-09-11T08:56:33.795 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-11T08:56:33.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:33.865 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:33.865 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:33.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:33.866 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:33.866 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:33 smithi145 ceph-mon[33384]: 3.f scrub starts 2024-09-11T08:56:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:33 smithi145 ceph-mon[33384]: 3.f scrub ok 2024-09-11T08:56:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:33 smithi145 ceph-mon[33384]: 6.6 scrub starts 2024-09-11T08:56:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:33 smithi145 ceph-mon[33384]: 6.6 scrub ok 2024-09-11T08:56:33.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:33 smithi145 ceph-mon[33384]: 5.1a scrub starts 2024-09-11T08:56:33.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:33 smithi145 ceph-mon[33384]: 5.1a scrub ok 2024-09-11T08:56:33.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:33.990 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:33.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:33.991 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:34.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:34.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:34.115 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:34.216 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:34.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:34.238 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:34.238 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:34.339 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:34.361 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:34.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:34.362 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:34.463 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:34.485 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:34.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:34.486 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:34.587 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:34.608 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:34.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:34.609 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:34.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:34.732 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:34.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:34.733 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:34.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[41114]: 4.12 scrub starts 2024-09-11T08:56:34.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[41114]: 4.12 scrub ok 2024-09-11T08:56:34.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[41114]: 1.a scrub starts 2024-09-11T08:56:34.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[41114]: 1.a scrub ok 2024-09-11T08:56:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[41114]: pgmap v806: 158 pgs: 158 active+clean; 145 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 894 op/s; 4.0 MiB/s, 64 objects/s recovering 2024-09-11T08:56:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[41114]: 1.5 scrub starts 2024-09-11T08:56:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[41114]: 1.5 scrub ok 2024-09-11T08:56:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T08:56:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[41114]: 5.d scrub ok 2024-09-11T08:56:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[29647]: 4.12 scrub starts 2024-09-11T08:56:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[29647]: 4.12 scrub ok 2024-09-11T08:56:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[29647]: 1.a scrub starts 2024-09-11T08:56:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[29647]: 1.a scrub ok 2024-09-11T08:56:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[29647]: pgmap v806: 158 pgs: 158 active+clean; 145 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 894 op/s; 4.0 MiB/s, 64 objects/s recovering 2024-09-11T08:56:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[29647]: 1.5 scrub starts 2024-09-11T08:56:34.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[29647]: 1.5 scrub ok 2024-09-11T08:56:34.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T08:56:34.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:34 smithi092 ceph-mon[29647]: 5.d scrub ok 2024-09-11T08:56:34.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:34.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:34.857 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:34.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:34.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:34 smithi145 ceph-mon[33384]: 4.12 scrub starts 2024-09-11T08:56:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:34 smithi145 ceph-mon[33384]: 4.12 scrub ok 2024-09-11T08:56:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:34 smithi145 ceph-mon[33384]: 1.a scrub starts 2024-09-11T08:56:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:34 smithi145 ceph-mon[33384]: 1.a scrub ok 2024-09-11T08:56:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:34 smithi145 ceph-mon[33384]: pgmap v806: 158 pgs: 158 active+clean; 145 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 894 op/s; 4.0 MiB/s, 64 objects/s recovering 2024-09-11T08:56:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:34 smithi145 ceph-mon[33384]: 1.5 scrub starts 2024-09-11T08:56:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:34 smithi145 ceph-mon[33384]: 1.5 scrub ok 2024-09-11T08:56:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:34 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T08:56:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:34 smithi145 ceph-mon[33384]: 5.d scrub ok 2024-09-11T08:56:34.959 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:34.980 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:34.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:34.982 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:35.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:56:35.082 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:35.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:35.104 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:35.104 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:35.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:35.226 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:35.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:35.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:35.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:35.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:35.351 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:35.351 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:35.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:35.473 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:35.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:35.474 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:35.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:35.596 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:35.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:35.597 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:35.698 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:35.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:35.725 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:35.725 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:35.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:35 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:56:35.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:35 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:56:35.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:35 smithi092 ceph-mon[29647]: osdmap e354: 8 total, 8 up, 8 in 2024-09-11T08:56:35.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:35 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:56:35.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:35 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:56:35.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:35 smithi092 ceph-mon[41114]: osdmap e354: 8 total, 8 up, 8 in 2024-09-11T08:56:35.828 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:35.849 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:35.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:35.850 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:35.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:35 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:56:35.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:35 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:56:35.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:35 smithi145 ceph-mon[33384]: osdmap e354: 8 total, 8 up, 8 in 2024-09-11T08:56:35.937 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:35.951 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:35.977 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:35.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:35.978 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:36.079 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:36.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:36.106 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:36.106 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:36.207 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:36.232 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:36.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:36.233 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:36.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:36.359 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:36.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:36.360 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:36.461 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:36.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:36.488 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:36.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:36.613 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:36.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:36.614 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:36.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:36.740 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:36.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:36.741 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[29647]: 3.c scrub starts 2024-09-11T08:56:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[29647]: 3.c scrub ok 2024-09-11T08:56:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[29647]: pgmap v808: 158 pgs: 158 active+clean; 145 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 894 op/s; 4.0 MiB/s, 64 objects/s recovering 2024-09-11T08:56:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[29647]: 6.1d scrub starts 2024-09-11T08:56:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[29647]: 6.1d scrub ok 2024-09-11T08:56:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[29647]: 10.4 scrub starts 2024-09-11T08:56:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[29647]: 10.4 scrub ok 2024-09-11T08:56:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[29647]: 4.1d scrub starts 2024-09-11T08:56:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[29647]: 4.1d scrub ok 2024-09-11T08:56:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[29647]: 1.0 scrub starts 2024-09-11T08:56:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[29647]: 1.0 scrub ok 2024-09-11T08:56:36.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[41114]: 3.c scrub starts 2024-09-11T08:56:36.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[41114]: 3.c scrub ok 2024-09-11T08:56:36.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[41114]: pgmap v808: 158 pgs: 158 active+clean; 145 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 894 op/s; 4.0 MiB/s, 64 objects/s recovering 2024-09-11T08:56:36.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[41114]: 6.1d scrub starts 2024-09-11T08:56:36.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[41114]: 6.1d scrub ok 2024-09-11T08:56:36.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[41114]: 10.4 scrub starts 2024-09-11T08:56:36.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[41114]: 10.4 scrub ok 2024-09-11T08:56:36.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[41114]: 4.1d scrub starts 2024-09-11T08:56:36.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[41114]: 4.1d scrub ok 2024-09-11T08:56:36.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[41114]: 1.0 scrub starts 2024-09-11T08:56:36.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:36 smithi092 ceph-mon[41114]: 1.0 scrub ok 2024-09-11T08:56:36.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:36.867 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:36.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:36.868 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:36.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:36 smithi145 ceph-mon[33384]: 3.c scrub starts 2024-09-11T08:56:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:36 smithi145 ceph-mon[33384]: 3.c scrub ok 2024-09-11T08:56:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:36 smithi145 ceph-mon[33384]: pgmap v808: 158 pgs: 158 active+clean; 145 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 894 op/s; 4.0 MiB/s, 64 objects/s recovering 2024-09-11T08:56:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:36 smithi145 ceph-mon[33384]: 6.1d scrub starts 2024-09-11T08:56:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:36 smithi145 ceph-mon[33384]: 6.1d scrub ok 2024-09-11T08:56:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:36 smithi145 ceph-mon[33384]: 10.4 scrub starts 2024-09-11T08:56:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:36 smithi145 ceph-mon[33384]: 10.4 scrub ok 2024-09-11T08:56:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:36 smithi145 ceph-mon[33384]: 4.1d scrub starts 2024-09-11T08:56:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:36 smithi145 ceph-mon[33384]: 4.1d scrub ok 2024-09-11T08:56:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:36 smithi145 ceph-mon[33384]: 1.0 scrub starts 2024-09-11T08:56:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:36 smithi145 ceph-mon[33384]: 1.0 scrub ok 2024-09-11T08:56:36.969 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:36.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:36.996 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:36.996 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:37.097 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:37.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:37.124 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:37.124 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:37.225 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:37.250 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:37.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:37.251 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:37.352 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:37.376 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:37.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:37.377 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:37.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:56:37 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:56:37.478 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:37.503 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:37.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:37.505 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:37.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:37.630 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:37.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:37.631 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:37.732 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:37.757 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:37.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:37.758 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:37 smithi092 ceph-mon[29647]: 5.16 scrub starts 2024-09-11T08:56:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:37 smithi092 ceph-mon[29647]: 5.16 scrub ok 2024-09-11T08:56:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:37 smithi092 ceph-mon[29647]: 6.e scrub starts 2024-09-11T08:56:37.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:37 smithi092 ceph-mon[29647]: 6.e scrub ok 2024-09-11T08:56:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:37 smithi092 ceph-mon[41114]: 5.16 scrub starts 2024-09-11T08:56:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:37 smithi092 ceph-mon[41114]: 5.16 scrub ok 2024-09-11T08:56:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:37 smithi092 ceph-mon[41114]: 6.e scrub starts 2024-09-11T08:56:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:37 smithi092 ceph-mon[41114]: 6.e scrub ok 2024-09-11T08:56:37.859 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:37.885 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:37.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:37.886 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:37.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:37 smithi145 ceph-mon[33384]: 5.16 scrub starts 2024-09-11T08:56:37.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:37 smithi145 ceph-mon[33384]: 5.16 scrub ok 2024-09-11T08:56:37.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:37 smithi145 ceph-mon[33384]: 6.e scrub starts 2024-09-11T08:56:37.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:37 smithi145 ceph-mon[33384]: 6.e scrub ok 2024-09-11T08:56:37.987 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:38.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:38.014 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:38.014 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:38.115 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:38.141 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:38.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:38.142 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:38.242 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:38.266 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:38.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:38.267 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:38.367 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:38.393 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:38.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:38.395 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:38.495 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:38.521 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:38.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:38.522 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:38.623 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:38.648 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:38.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:38.649 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:38.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:38.775 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:38.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:38.776 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:38.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:38 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:38.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:38 smithi092 ceph-mon[41114]: 3.1 scrub starts 2024-09-11T08:56:38.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:38 smithi092 ceph-mon[41114]: 3.1 scrub ok 2024-09-11T08:56:38.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:38 smithi092 ceph-mon[41114]: pgmap v809: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 1.05k op/s; 3.3 MiB/s, 53 objects/s recovering 2024-09-11T08:56:38.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:38 smithi092 ceph-mon[41114]: 3.0 scrub starts 2024-09-11T08:56:38.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:38 smithi092 ceph-mon[41114]: 3.0 scrub ok 2024-09-11T08:56:38.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:38 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:38.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:38 smithi092 ceph-mon[29647]: 3.1 scrub starts 2024-09-11T08:56:38.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:38 smithi092 ceph-mon[29647]: 3.1 scrub ok 2024-09-11T08:56:38.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:38 smithi092 ceph-mon[29647]: pgmap v809: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 1.05k op/s; 3.3 MiB/s, 53 objects/s recovering 2024-09-11T08:56:38.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:38 smithi092 ceph-mon[29647]: 3.0 scrub starts 2024-09-11T08:56:38.818 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:38 smithi092 ceph-mon[29647]: 3.0 scrub ok 2024-09-11T08:56:38.877 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:38.902 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:38.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:38.903 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:38.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:38 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:38.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:38 smithi145 ceph-mon[33384]: 3.1 scrub starts 2024-09-11T08:56:38.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:38 smithi145 ceph-mon[33384]: 3.1 scrub ok 2024-09-11T08:56:38.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:38 smithi145 ceph-mon[33384]: pgmap v809: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 1.05k op/s; 3.3 MiB/s, 53 objects/s recovering 2024-09-11T08:56:38.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:38 smithi145 ceph-mon[33384]: 3.0 scrub starts 2024-09-11T08:56:38.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:38 smithi145 ceph-mon[33384]: 3.0 scrub ok 2024-09-11T08:56:39.004 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:39.030 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:39.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:39.031 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:39.132 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:39.155 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:39.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:39.157 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:39.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:39.284 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:39.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:39.285 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:39.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:39.411 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:39.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:39.412 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:39.513 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:39.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:39.540 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:39.540 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:39.641 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:39.665 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:39.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:39.667 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:39.767 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:39.791 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:39.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:39.792 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:39.794 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:39 smithi092 ceph-mon[29647]: 5.15 scrub starts 2024-09-11T08:56:39.794 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:39 smithi092 ceph-mon[29647]: 5.15 scrub ok 2024-09-11T08:56:39.794 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:39 smithi092 ceph-mon[29647]: 6.9 scrub starts 2024-09-11T08:56:39.794 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:39 smithi092 ceph-mon[29647]: 6.9 scrub ok 2024-09-11T08:56:39.794 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:39 smithi092 ceph-mon[41114]: 5.15 scrub starts 2024-09-11T08:56:39.794 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:39 smithi092 ceph-mon[41114]: 5.15 scrub ok 2024-09-11T08:56:39.794 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:39 smithi092 ceph-mon[41114]: 6.9 scrub starts 2024-09-11T08:56:39.794 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:39 smithi092 ceph-mon[41114]: 6.9 scrub ok 2024-09-11T08:56:39.807 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:39.893 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:39.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:39 smithi145 ceph-mon[33384]: 5.15 scrub starts 2024-09-11T08:56:39.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:39 smithi145 ceph-mon[33384]: 5.15 scrub ok 2024-09-11T08:56:39.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:39 smithi145 ceph-mon[33384]: 6.9 scrub starts 2024-09-11T08:56:39.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:39 smithi145 ceph-mon[33384]: 6.9 scrub ok 2024-09-11T08:56:39.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:39.919 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:39.919 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:40.020 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:40.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:40.044 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:40.044 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:40.145 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:40.172 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:40.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:40.173 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:40.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:40.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:40.297 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:40.298 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:40.398 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:40.424 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:40.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:40.426 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:40.526 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:40.552 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:40.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:40.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:40.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[29647]: 5.1b scrub starts 2024-09-11T08:56:40.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[29647]: 5.1b scrub ok 2024-09-11T08:56:40.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[29647]: pgmap v810: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 925 op/s; 2.9 MiB/s, 46 objects/s recovering 2024-09-11T08:56:40.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[29647]: 4.f scrub starts 2024-09-11T08:56:40.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[29647]: 4.f scrub ok 2024-09-11T08:56:40.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:56:40.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:56:40.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[29647]: 5.4 scrub starts 2024-09-11T08:56:40.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[29647]: 5.4 scrub ok 2024-09-11T08:56:40.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[29647]: osdmap e355: 8 total, 8 up, 8 in 2024-09-11T08:56:40.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[41114]: 5.1b scrub starts 2024-09-11T08:56:40.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[41114]: 5.1b scrub ok 2024-09-11T08:56:40.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[41114]: pgmap v810: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 925 op/s; 2.9 MiB/s, 46 objects/s recovering 2024-09-11T08:56:40.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[41114]: 4.f scrub starts 2024-09-11T08:56:40.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[41114]: 4.f scrub ok 2024-09-11T08:56:40.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:56:40.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:56:40.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[41114]: 5.4 scrub starts 2024-09-11T08:56:40.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[41114]: 5.4 scrub ok 2024-09-11T08:56:40.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:40 smithi092 ceph-mon[41114]: osdmap e355: 8 total, 8 up, 8 in 2024-09-11T08:56:40.654 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:40.676 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:40.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:40.677 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:40.778 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:40.800 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:40.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:40.801 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:40.901 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:40.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:40 smithi145 ceph-mon[33384]: 5.1b scrub starts 2024-09-11T08:56:40.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:40 smithi145 ceph-mon[33384]: 5.1b scrub ok 2024-09-11T08:56:40.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:40 smithi145 ceph-mon[33384]: pgmap v810: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 925 op/s; 2.9 MiB/s, 46 objects/s recovering 2024-09-11T08:56:40.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:40 smithi145 ceph-mon[33384]: 4.f scrub starts 2024-09-11T08:56:40.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:40 smithi145 ceph-mon[33384]: 4.f scrub ok 2024-09-11T08:56:40.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:40 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:56:40.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:40 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:56:40.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:40 smithi145 ceph-mon[33384]: 5.4 scrub starts 2024-09-11T08:56:40.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:40 smithi145 ceph-mon[33384]: 5.4 scrub ok 2024-09-11T08:56:40.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:40 smithi145 ceph-mon[33384]: osdmap e355: 8 total, 8 up, 8 in 2024-09-11T08:56:40.923 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:40.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:40.924 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:41.025 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:41.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:41.048 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:41.048 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:41.149 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:41.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:41.172 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:41.172 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:41.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:41.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:41.296 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:41.297 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:41.397 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:41.420 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:41.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:41.421 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:41.521 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:41.543 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:41.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:41.544 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:41.550 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:56:41.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[29647]: 1.3 scrub starts 2024-09-11T08:56:41.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[29647]: 1.3 scrub ok 2024-09-11T08:56:41.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[29647]: 5.1e scrub starts 2024-09-11T08:56:41.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[29647]: 5.1e scrub ok 2024-09-11T08:56:41.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[29647]: 10.e scrub starts 2024-09-11T08:56:41.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[29647]: 10.e scrub ok 2024-09-11T08:56:41.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[29647]: 6.c scrub starts 2024-09-11T08:56:41.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[29647]: 6.c scrub ok 2024-09-11T08:56:41.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2981711182' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:56:41.592 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:56:41] "GET /metrics HTTP/1.1" 200 224858 "" "Prometheus/2.43.0" 2024-09-11T08:56:41.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[41114]: 1.3 scrub starts 2024-09-11T08:56:41.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[41114]: 1.3 scrub ok 2024-09-11T08:56:41.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[41114]: 5.1e scrub starts 2024-09-11T08:56:41.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[41114]: 5.1e scrub ok 2024-09-11T08:56:41.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[41114]: 10.e scrub starts 2024-09-11T08:56:41.592 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[41114]: 10.e scrub ok 2024-09-11T08:56:41.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[41114]: 6.c scrub starts 2024-09-11T08:56:41.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[41114]: 6.c scrub ok 2024-09-11T08:56:41.593 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:41 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2981711182' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:56:41.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:41.667 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:41.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:41.668 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:41.769 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:41.790 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:41.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:41.791 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:41.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:41.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:41 smithi145 ceph-mon[33384]: 1.3 scrub starts 2024-09-11T08:56:41.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:41 smithi145 ceph-mon[33384]: 1.3 scrub ok 2024-09-11T08:56:41.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:41 smithi145 ceph-mon[33384]: 5.1e scrub starts 2024-09-11T08:56:41.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:41 smithi145 ceph-mon[33384]: 5.1e scrub ok 2024-09-11T08:56:41.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:41 smithi145 ceph-mon[33384]: 10.e scrub starts 2024-09-11T08:56:41.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:41 smithi145 ceph-mon[33384]: 10.e scrub ok 2024-09-11T08:56:41.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:41 smithi145 ceph-mon[33384]: 6.c scrub starts 2024-09-11T08:56:41.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:41 smithi145 ceph-mon[33384]: 6.c scrub ok 2024-09-11T08:56:41.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:41 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2981711182' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:56:41.913 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:41.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:41.914 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:42.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:42.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:42.037 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:42.138 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:42.159 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:42.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:42.160 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:42.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:42.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:42.283 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:42.283 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:42.384 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:42.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:42.407 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:42.408 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:42.476 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:56:42.479 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:56:42.508 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:42.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:42.531 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:42.531 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:42.632 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:42.653 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:42.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:42.654 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:42.755 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:42.776 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:42.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:42.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:42.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[29647]: pgmap v812: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 286 op/s 2024-09-11T08:56:42.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[29647]: 10.0 scrub starts 2024-09-11T08:56:42.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[29647]: 10.0 scrub ok 2024-09-11T08:56:42.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:56:42.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:56:42.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2981711182' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:56:42.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[29647]: osdmap e356: 8 total, 8 up, 8 in 2024-09-11T08:56:42.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[29647]: 10.c scrub starts 2024-09-11T08:56:42.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[29647]: 10.c scrub ok 2024-09-11T08:56:42.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[41114]: pgmap v812: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 286 op/s 2024-09-11T08:56:42.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[41114]: 10.0 scrub starts 2024-09-11T08:56:42.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[41114]: 10.0 scrub ok 2024-09-11T08:56:42.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:56:42.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:56:42.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2981711182' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:56:42.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[41114]: osdmap e356: 8 total, 8 up, 8 in 2024-09-11T08:56:42.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[41114]: 10.c scrub starts 2024-09-11T08:56:42.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:42 smithi092 ceph-mon[41114]: 10.c scrub ok 2024-09-11T08:56:42.877 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:42.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:42.899 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:42.900 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:42.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:56:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:56:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:56:42.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:42 smithi145 ceph-mon[33384]: pgmap v812: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 286 op/s 2024-09-11T08:56:42.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:42 smithi145 ceph-mon[33384]: 10.0 scrub starts 2024-09-11T08:56:42.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:42 smithi145 ceph-mon[33384]: 10.0 scrub ok 2024-09-11T08:56:42.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:42 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:56:42.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:42 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:56:42.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:42 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2981711182' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:56:42.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:42 smithi145 ceph-mon[33384]: osdmap e356: 8 total, 8 up, 8 in 2024-09-11T08:56:42.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:42 smithi145 ceph-mon[33384]: 10.c scrub starts 2024-09-11T08:56:42.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:42 smithi145 ceph-mon[33384]: 10.c scrub ok 2024-09-11T08:56:43.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:43.021 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:43.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:43.022 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:43.123 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:43.144 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:43.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:43.145 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:43.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:43.267 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:43.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:43.268 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:43.369 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:43.390 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:43.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:43.391 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:43.492 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:43.503 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":813,"stamp":"2024-09-11T08:56:41.548859+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":703528,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1254,"num_read_kb":1026,"num_write":274650,"num_write_kb":8765524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":623,"num_bytes_recovered":25475169,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79517,"ondisk_log_size":79517,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":475,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":6564448,"kb_used_data":9032,"kb_used_omap":13,"kb_used_meta":6555314,"kb_avail":743396768,"statfs":{"total":767960285184,"available":761238290432,"internally_reserved":0,"allocated":9248768,"data_stored":6700962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":6712642240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-150983101,"num_objects":-2304,"num_object_clones":0,"num_object_copies":-6912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2304,"num_whiteouts":0,"num_read":-1,"num_read_kb":-11,"num_write":2294,"num_write_kb":-148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":-107339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.024203"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":172,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156889+0000","last_change":"2024-09-11T08:54:18.331227+0000","last_active":"2024-09-11T08:56:31.156889+0000","last_peered":"2024-09-11T08:56:31.156889+0000","last_clean":"2024-09-11T08:56:31.156889+0000","last_became_active":"2024-09-11T08:52:49.568416+0000","last_became_peered":"2024-09-11T08:52:49.568416+0000","last_unstale":"2024-09-11T08:56:31.156889+0000","last_undegraded":"2024-09-11T08:56:31.156889+0000","last_fullsized":"2024-09-11T08:56:31.156889+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:18.331155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:18.331155+0000","last_clean_scrub_stamp":"2024-09-11T08:54:18.331155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042788669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"200'19","reported_seq":435,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.525688+0000","last_change":"2024-09-11T08:55:55.640259+0000","last_active":"2024-09-11T08:56:34.525688+0000","last_peered":"2024-09-11T08:56:34.525688+0000","last_clean":"2024-09-11T08:56:34.525688+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:56:34.525688+0000","last_undegraded":"2024-09-11T08:56:34.525688+0000","last_fullsized":"2024-09-11T08:56:34.525688+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:55:55.640174+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:55:55.640174+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:57.436630+0000","scrub_duration":0.0045891150000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":382,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.126142+0000","last_change":"2024-09-11T08:55:52.627015+0000","last_active":"2024-09-11T08:56:31.126142+0000","last_peered":"2024-09-11T08:56:31.126142+0000","last_clean":"2024-09-11T08:56:31.126142+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:56:31.126142+0000","last_undegraded":"2024-09-11T08:56:31.126142+0000","last_fullsized":"2024-09-11T08:56:31.126142+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:52.626934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:55:52.626934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:14.766952+0000","scrub_duration":0.00253637,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":362,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.526845+0000","last_change":"2024-09-11T08:56:25.459604+0000","last_active":"2024-09-11T08:56:34.526845+0000","last_peered":"2024-09-11T08:56:34.526845+0000","last_clean":"2024-09-11T08:56:34.526845+0000","last_became_active":"2024-09-11T08:56:25.459294+0000","last_became_peered":"2024-09-11T08:56:25.459294+0000","last_unstale":"2024-09-11T08:56:34.526845+0000","last_undegraded":"2024-09-11T08:56:34.526845+0000","last_fullsized":"2024-09-11T08:56:34.526845+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:13.811054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:13.811054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010996769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":381,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847873+0000","last_change":"2024-09-11T08:54:58.730290+0000","last_active":"2024-09-11T08:56:34.847873+0000","last_peered":"2024-09-11T08:56:34.847873+0000","last_clean":"2024-09-11T08:56:34.847873+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:56:34.847873+0000","last_undegraded":"2024-09-11T08:56:34.847873+0000","last_fullsized":"2024-09-11T08:56:34.847873+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:58.730211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:58.730211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011911339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":387,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848186+0000","last_change":"2024-09-11T08:54:52.562148+0000","last_active":"2024-09-11T08:56:34.848186+0000","last_peered":"2024-09-11T08:56:34.848186+0000","last_clean":"2024-09-11T08:56:34.848186+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:56:34.848186+0000","last_undegraded":"2024-09-11T08:56:34.848186+0000","last_fullsized":"2024-09-11T08:56:34.848186+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:52.562065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:54:52.562065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074997070000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"200'34","reported_seq":467,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847344+0000","last_change":"2024-09-11T08:54:12.545864+0000","last_active":"2024-09-11T08:56:34.847344+0000","last_peered":"2024-09-11T08:56:34.847344+0000","last_clean":"2024-09-11T08:56:34.847344+0000","last_became_active":"2024-09-11T08:52:18.585600+0000","last_became_peered":"2024-09-11T08:52:18.585600+0000","last_unstale":"2024-09-11T08:56:34.847344+0000","last_undegraded":"2024-09-11T08:56:34.847344+0000","last_fullsized":"2024-09-11T08:56:34.847344+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:54:12.545773+0000","last_deep_scrub":"200'34","last_deep_scrub_stamp":"2024-09-11T08:51:31.358660+0000","last_clean_scrub_stamp":"2024-09-11T08:54:12.545773+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070952350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":354,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.525683+0000","last_change":"2024-09-11T08:54:55.729780+0000","last_active":"2024-09-11T08:56:34.525683+0000","last_peered":"2024-09-11T08:56:34.525683+0000","last_clean":"2024-09-11T08:56:34.525683+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:56:34.525683+0000","last_undegraded":"2024-09-11T08:56:34.525683+0000","last_fullsized":"2024-09-11T08:56:34.525683+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:55.729679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:54:55.729679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002263007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":392,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156455+0000","last_change":"2024-09-11T08:54:10.466581+0000","last_active":"2024-09-11T08:56:31.156455+0000","last_peered":"2024-09-11T08:56:31.156455+0000","last_clean":"2024-09-11T08:56:31.156455+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:56:31.156455+0000","last_undegraded":"2024-09-11T08:56:31.156455+0000","last_fullsized":"2024-09-11T08:56:31.156455+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:10.466478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:54:10.466478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034798429999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"200'19","reported_seq":435,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.526906+0000","last_change":"2024-09-11T08:54:09.865650+0000","last_active":"2024-09-11T08:56:34.526906+0000","last_peered":"2024-09-11T08:56:34.526906+0000","last_clean":"2024-09-11T08:56:34.526906+0000","last_became_active":"2024-09-11T08:52:18.577411+0000","last_became_peered":"2024-09-11T08:52:18.577411+0000","last_unstale":"2024-09-11T08:56:34.526906+0000","last_undegraded":"2024-09-11T08:56:34.526906+0000","last_fullsized":"2024-09-11T08:56:34.526906+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:54:09.865565+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.865565+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044543600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":388,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156133+0000","last_change":"2024-09-11T08:53:59.484790+0000","last_active":"2024-09-11T08:56:31.156133+0000","last_peered":"2024-09-11T08:56:31.156133+0000","last_clean":"2024-09-11T08:56:31.156133+0000","last_became_active":"2024-09-11T08:52:18.573461+0000","last_became_peered":"2024-09-11T08:52:18.573461+0000","last_unstale":"2024-09-11T08:56:31.156133+0000","last_undegraded":"2024-09-11T08:56:31.156133+0000","last_fullsized":"2024-09-11T08:56:31.156133+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:59.484561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:59.484561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060038859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":182,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.126560+0000","last_change":"2024-09-11T08:56:29.480301+0000","last_active":"2024-09-11T08:56:31.126560+0000","last_peered":"2024-09-11T08:56:31.126560+0000","last_clean":"2024-09-11T08:56:31.126560+0000","last_became_active":"2024-09-11T08:52:49.568147+0000","last_became_peered":"2024-09-11T08:52:49.568147+0000","last_unstale":"2024-09-11T08:56:31.126560+0000","last_undegraded":"2024-09-11T08:56:31.126560+0000","last_fullsized":"2024-09-11T08:56:31.126560+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:29.480218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:33.520462+0000","last_clean_scrub_stamp":"2024-09-11T08:56:29.480218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:30.884840+0000","scrub_duration":0.0040366990000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":372,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156342+0000","last_change":"2024-09-11T08:54:17.338878+0000","last_active":"2024-09-11T08:56:31.156342+0000","last_peered":"2024-09-11T08:56:31.156342+0000","last_clean":"2024-09-11T08:56:31.156342+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:56:31.156342+0000","last_undegraded":"2024-09-11T08:56:31.156342+0000","last_fullsized":"2024-09-11T08:56:31.156342+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:17.338793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:30.465003+0000","last_clean_scrub_stamp":"2024-09-11T08:54:17.338793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028270790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"200'7","reported_seq":388,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.526844+0000","last_change":"2024-09-11T08:56:25.474365+0000","last_active":"2024-09-11T08:56:34.526844+0000","last_peered":"2024-09-11T08:56:34.526844+0000","last_clean":"2024-09-11T08:56:34.526844+0000","last_became_active":"2024-09-11T08:56:25.465945+0000","last_became_peered":"2024-09-11T08:56:25.465945+0000","last_unstale":"2024-09-11T08:56:34.526844+0000","last_undegraded":"2024-09-11T08:56:34.526844+0000","last_fullsized":"2024-09-11T08:56:34.526844+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'7","last_scrub_stamp":"2024-09-11T08:55:35.725836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:35.725836+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:48.643724+0000","scrub_duration":0.0043511380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":391,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:33.417337+0000","last_change":"2024-09-11T08:56:33.417300+0000","last_active":"2024-09-11T08:56:33.417337+0000","last_peered":"2024-09-11T08:56:33.417337+0000","last_clean":"2024-09-11T08:56:33.417337+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:56:33.417337+0000","last_undegraded":"2024-09-11T08:56:33.417337+0000","last_fullsized":"2024-09-11T08:56:33.417337+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:33.417247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:18.199651+0000","last_clean_scrub_stamp":"2024-09-11T08:56:33.417247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:59.296071+0000","scrub_duration":0.0011470479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":366,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156010+0000","last_change":"2024-09-11T08:55:13.401842+0000","last_active":"2024-09-11T08:56:31.156010+0000","last_peered":"2024-09-11T08:56:31.156010+0000","last_clean":"2024-09-11T08:56:31.156010+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:56:31.156010+0000","last_undegraded":"2024-09-11T08:56:31.156010+0000","last_fullsized":"2024-09-11T08:56:31.156010+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:13.401759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:55:13.401759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070886450000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":372,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.527144+0000","last_change":"2024-09-11T08:53:56.854824+0000","last_active":"2024-09-11T08:56:34.527144+0000","last_peered":"2024-09-11T08:56:34.527144+0000","last_clean":"2024-09-11T08:56:34.527144+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:56:34.527144+0000","last_undegraded":"2024-09-11T08:56:34.527144+0000","last_fullsized":"2024-09-11T08:56:34.527144+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:56.854710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:56.854710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061159049999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":173,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.527031+0000","last_change":"2024-09-11T08:53:58.835251+0000","last_active":"2024-09-11T08:56:34.527031+0000","last_peered":"2024-09-11T08:56:34.527031+0000","last_clean":"2024-09-11T08:56:34.527031+0000","last_became_active":"2024-09-11T08:52:49.566575+0000","last_became_peered":"2024-09-11T08:52:49.566575+0000","last_unstale":"2024-09-11T08:56:34.527031+0000","last_undegraded":"2024-09-11T08:56:34.527031+0000","last_fullsized":"2024-09-11T08:56:34.527031+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:58.835161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:58.835161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0065024590000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"200'34","reported_seq":321,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.126448+0000","last_change":"2024-09-11T08:55:36.496870+0000","last_active":"2024-09-11T08:56:31.126448+0000","last_peered":"2024-09-11T08:56:31.126448+0000","last_clean":"2024-09-11T08:56:31.126448+0000","last_became_active":"2024-09-11T08:54:17.121459+0000","last_became_peered":"2024-09-11T08:54:17.121459+0000","last_unstale":"2024-09-11T08:56:31.126448+0000","last_undegraded":"2024-09-11T08:56:31.126448+0000","last_fullsized":"2024-09-11T08:56:31.126448+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:55:36.496770+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:55:36.496770+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:01.763330+0000","scrub_duration":0.0073101140000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":402,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.125666+0000","last_change":"2024-09-11T08:55:48.651417+0000","last_active":"2024-09-11T08:56:31.125666+0000","last_peered":"2024-09-11T08:56:31.125666+0000","last_clean":"2024-09-11T08:56:31.125666+0000","last_became_active":"2024-09-11T08:52:18.579026+0000","last_became_peered":"2024-09-11T08:52:18.579026+0000","last_unstale":"2024-09-11T08:56:31.125666+0000","last_undegraded":"2024-09-11T08:56:31.125666+0000","last_fullsized":"2024-09-11T08:56:31.125666+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:48.651334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:55:48.651334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:06.491404+0000","scrub_duration":0.0078795440000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":390,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848449+0000","last_change":"2024-09-11T08:55:50.827754+0000","last_active":"2024-09-11T08:56:34.848449+0000","last_peered":"2024-09-11T08:56:34.848449+0000","last_clean":"2024-09-11T08:56:34.848449+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:56:34.848449+0000","last_undegraded":"2024-09-11T08:56:34.848449+0000","last_fullsized":"2024-09-11T08:56:34.848449+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:55:50.827483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:55:50.827483+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:07.878037+0000","scrub_duration":0.0082684079999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":358,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.526996+0000","last_change":"2024-09-11T08:53:48.866804+0000","last_active":"2024-09-11T08:56:34.526996+0000","last_peered":"2024-09-11T08:56:34.526996+0000","last_clean":"2024-09-11T08:56:34.526996+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:56:34.526996+0000","last_undegraded":"2024-09-11T08:56:34.526996+0000","last_fullsized":"2024-09-11T08:56:34.526996+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:48.866615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:48.866615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075952449999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"200'18","reported_seq":419,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:35.815523+0000","last_change":"2024-09-11T08:56:35.815453+0000","last_active":"2024-09-11T08:56:35.815523+0000","last_peered":"2024-09-11T08:56:35.815523+0000","last_clean":"2024-09-11T08:56:35.815523+0000","last_became_active":"2024-09-11T08:49:16.943886+0000","last_became_peered":"2024-09-11T08:49:16.943886+0000","last_unstale":"2024-09-11T08:56:35.815523+0000","last_undegraded":"2024-09-11T08:56:35.815523+0000","last_fullsized":"2024-09-11T08:56:35.815523+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'18","last_scrub_stamp":"2024-09-11T08:56:35.815348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:56:35.815348+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:38.311041+0000","scrub_duration":0.0021465519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":354,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.527202+0000","last_change":"2024-09-11T08:53:12.948801+0000","last_active":"2024-09-11T08:56:34.527202+0000","last_peered":"2024-09-11T08:56:34.527202+0000","last_clean":"2024-09-11T08:56:34.527202+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:56:34.527202+0000","last_undegraded":"2024-09-11T08:56:34.527202+0000","last_fullsized":"2024-09-11T08:56:34.527202+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:12.948659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:12.948659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039731519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":99,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156094+0000","last_change":"2024-09-11T08:53:58.534161+0000","last_active":"2024-09-11T08:56:31.156094+0000","last_peered":"2024-09-11T08:56:31.156094+0000","last_clean":"2024-09-11T08:56:31.156094+0000","last_became_active":"2024-09-11T08:52:38.483603+0000","last_became_peered":"2024-09-11T08:52:38.483603+0000","last_unstale":"2024-09-11T08:56:31.156094+0000","last_undegraded":"2024-09-11T08:56:31.156094+0000","last_fullsized":"2024-09-11T08:56:31.156094+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:53:58.534079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:58.534079+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070998140000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":386,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.126261+0000","last_change":"2024-09-11T08:55:44.633750+0000","last_active":"2024-09-11T08:56:31.126261+0000","last_peered":"2024-09-11T08:56:31.126261+0000","last_clean":"2024-09-11T08:56:31.126261+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:56:31.126261+0000","last_undegraded":"2024-09-11T08:56:31.126261+0000","last_fullsized":"2024-09-11T08:56:31.126261+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:44.633675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:55:44.633675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:00.929748+0000","scrub_duration":0.0018048610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"280'25","reported_seq":116,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848068+0000","last_change":"2024-09-11T08:55:09.606768+0000","last_active":"2024-09-11T08:56:34.848068+0000","last_peered":"2024-09-11T08:56:34.848068+0000","last_clean":"2024-09-11T08:56:34.848068+0000","last_became_active":"2024-09-11T08:52:49.568074+0000","last_became_peered":"2024-09-11T08:52:49.568074+0000","last_unstale":"2024-09-11T08:56:34.848068+0000","last_undegraded":"2024-09-11T08:56:34.848068+0000","last_fullsized":"2024-09-11T08:56:34.848068+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"280'25","last_scrub_stamp":"2024-09-11T08:55:09.606687+0000","last_deep_scrub":"280'25","last_deep_scrub_stamp":"2024-09-11T08:52:24.584533+0000","last_clean_scrub_stamp":"2024-09-11T08:55:09.606687+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005718791,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"346'9","reported_seq":52,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:37.875737+0000","last_change":"2024-09-11T08:56:08.482980+0000","last_active":"2024-09-11T08:56:37.875737+0000","last_peered":"2024-09-11T08:56:37.875737+0000","last_clean":"2024-09-11T08:56:37.875737+0000","last_became_active":"2024-09-11T08:56:08.480106+0000","last_became_peered":"2024-09-11T08:56:08.480106+0000","last_unstale":"2024-09-11T08:56:37.875737+0000","last_undegraded":"2024-09-11T08:56:37.875737+0000","last_fullsized":"2024-09-11T08:56:37.875737+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:55:41.383556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:55:41.383556+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:00.691086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1","version":"353'17032","reported_seq":17102,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.525773+0000","last_change":"2024-09-11T08:52:56.415129+0000","last_active":"2024-09-11T08:56:34.525773+0000","last_peered":"2024-09-11T08:56:34.525773+0000","last_clean":"2024-09-11T08:56:34.525773+0000","last_became_active":"2024-09-11T08:52:56.414844+0000","last_became_peered":"2024-09-11T08:52:56.414844+0000","last_unstale":"2024-09-11T08:56:34.525773+0000","last_undegraded":"2024-09-11T08:56:34.525773+0000","last_fullsized":"2024-09-11T08:56:34.525773+0000","mapping_epoch":293,"log_start":"335'12600","ondisk_log_start":"335'12600","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4432,"log_dups_size":0,"ondisk_log_size":4432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17032,"num_write_kb":545024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"199'81","reported_seq":133,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:33.363076+0000","last_change":"2024-09-11T08:56:33.363021+0000","last_active":"2024-09-11T08:56:33.363076+0000","last_peered":"2024-09-11T08:56:33.363076+0000","last_clean":"2024-09-11T08:56:33.363076+0000","last_became_active":"2024-09-11T08:51:03.023615+0000","last_became_peered":"2024-09-11T08:51:03.023615+0000","last_unstale":"2024-09-11T08:56:33.363076+0000","last_undegraded":"2024-09-11T08:56:33.363076+0000","last_fullsized":"2024-09-11T08:56:33.363076+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:56:33.362954+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:56:33.362954+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:38.473680+0000","scrub_duration":0.001495113,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"200'34","reported_seq":424,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.125315+0000","last_change":"2024-09-11T08:55:03.157598+0000","last_active":"2024-09-11T08:56:31.125315+0000","last_peered":"2024-09-11T08:56:31.125315+0000","last_clean":"2024-09-11T08:56:31.125315+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:56:31.125315+0000","last_undegraded":"2024-09-11T08:56:31.125315+0000","last_fullsized":"2024-09-11T08:56:31.125315+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:55:03.157511+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:55:03.157511+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047993619999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":367,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847768+0000","last_change":"2024-09-11T08:55:39.740071+0000","last_active":"2024-09-11T08:56:34.847768+0000","last_peered":"2024-09-11T08:56:34.847768+0000","last_clean":"2024-09-11T08:56:34.847768+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:56:34.847768+0000","last_undegraded":"2024-09-11T08:56:34.847768+0000","last_fullsized":"2024-09-11T08:56:34.847768+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:39.739981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:55:39.739981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:50.516778+0000","scrub_duration":0.0052641099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"277'11","reported_seq":617,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:37.875739+0000","last_change":"2024-09-11T08:56:31.694340+0000","last_active":"2024-09-11T08:56:37.875739+0000","last_peered":"2024-09-11T08:56:37.875739+0000","last_clean":"2024-09-11T08:56:37.875739+0000","last_became_active":"2024-09-11T08:52:18.578328+0000","last_became_peered":"2024-09-11T08:52:18.578328+0000","last_unstale":"2024-09-11T08:56:37.875739+0000","last_undegraded":"2024-09-11T08:56:37.875739+0000","last_fullsized":"2024-09-11T08:56:37.875739+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'11","last_scrub_stamp":"2024-09-11T08:56:31.694253+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:56:31.694253+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:33.633194+0000","scrub_duration":0.0025274149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":393,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156233+0000","last_change":"2024-09-11T08:56:25.462412+0000","last_active":"2024-09-11T08:56:31.156233+0000","last_peered":"2024-09-11T08:56:31.156233+0000","last_clean":"2024-09-11T08:56:31.156233+0000","last_became_active":"2024-09-11T08:56:25.462088+0000","last_became_peered":"2024-09-11T08:56:25.462088+0000","last_unstale":"2024-09-11T08:56:31.156233+0000","last_undegraded":"2024-09-11T08:56:31.156233+0000","last_fullsized":"2024-09-11T08:56:31.156233+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:04.263219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:55:04.263219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034071710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"353'17062","reported_seq":17142,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-11T08:56:40.705209+0000","last_change":"2024-09-11T08:56:40.705177+0000","last_active":"2024-09-11T08:56:40.705209+0000","last_peered":"2024-09-11T08:56:40.705209+0000","last_clean":"2024-09-11T08:56:40.705209+0000","last_became_active":"2024-09-11T08:52:55.610383+0000","last_became_peered":"2024-09-11T08:52:55.610383+0000","last_unstale":"2024-09-11T08:56:40.705209+0000","last_undegraded":"2024-09-11T08:56:40.705209+0000","last_fullsized":"2024-09-11T08:56:40.705209+0000","mapping_epoch":293,"log_start":"337'12700","ondisk_log_start":"337'12700","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"353'17062","last_scrub_stamp":"2024-09-11T08:56:40.705111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:56:40.705111+0000","objects_scrubbed":0,"log_size":4362,"log_dups_size":0,"ondisk_log_size":4362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:42.818513+0000","scrub_duration":0.0098165260000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17062,"num_write_kb":545984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"199'81","reported_seq":134,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.527748+0000","last_change":"2024-09-11T08:56:33.671358+0000","last_active":"2024-09-11T08:56:34.527748+0000","last_peered":"2024-09-11T08:56:34.527748+0000","last_clean":"2024-09-11T08:56:34.527748+0000","last_became_active":"2024-09-11T08:51:03.023365+0000","last_became_peered":"2024-09-11T08:51:03.023365+0000","last_unstale":"2024-09-11T08:56:34.527748+0000","last_undegraded":"2024-09-11T08:56:34.527748+0000","last_fullsized":"2024-09-11T08:56:34.527748+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:56:33.671306+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:56:33.671306+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:03.642591+0000","scrub_duration":0.0011424930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"200'38","reported_seq":479,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847223+0000","last_change":"2024-09-11T08:55:06.625605+0000","last_active":"2024-09-11T08:56:34.847223+0000","last_peered":"2024-09-11T08:56:34.847223+0000","last_clean":"2024-09-11T08:56:34.847223+0000","last_became_active":"2024-09-11T08:52:18.595013+0000","last_became_peered":"2024-09-11T08:52:18.595013+0000","last_unstale":"2024-09-11T08:56:34.847223+0000","last_undegraded":"2024-09-11T08:56:34.847223+0000","last_fullsized":"2024-09-11T08:56:34.847223+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'38","last_scrub_stamp":"2024-09-11T08:55:06.625504+0000","last_deep_scrub":"199'26","last_deep_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_clean_scrub_stamp":"2024-09-11T08:55:06.625504+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010430116999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":390,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847325+0000","last_change":"2024-09-11T08:55:04.638886+0000","last_active":"2024-09-11T08:56:34.847325+0000","last_peered":"2024-09-11T08:56:34.847325+0000","last_clean":"2024-09-11T08:56:34.847325+0000","last_became_active":"2024-09-11T08:52:18.586084+0000","last_became_peered":"2024-09-11T08:52:18.586084+0000","last_unstale":"2024-09-11T08:56:34.847325+0000","last_undegraded":"2024-09-11T08:56:34.847325+0000","last_fullsized":"2024-09-11T08:56:34.847325+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:04.638693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:55:04.638693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070800059999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":354,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.525797+0000","last_change":"2024-09-11T08:52:59.933645+0000","last_active":"2024-09-11T08:56:34.525797+0000","last_peered":"2024-09-11T08:56:34.525797+0000","last_clean":"2024-09-11T08:56:34.525797+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:56:34.525797+0000","last_undegraded":"2024-09-11T08:56:34.525797+0000","last_fullsized":"2024-09-11T08:56:34.525797+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:59.933599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:29.687577+0000","last_clean_scrub_stamp":"2024-09-11T08:52:59.933599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001022233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":352,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.526822+0000","last_change":"2024-09-11T08:54:00.820805+0000","last_active":"2024-09-11T08:56:34.526822+0000","last_peered":"2024-09-11T08:56:34.526822+0000","last_clean":"2024-09-11T08:56:34.526822+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:56:34.526822+0000","last_undegraded":"2024-09-11T08:56:34.526822+0000","last_fullsized":"2024-09-11T08:56:34.526822+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:00.820729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:00.820729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0084568009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"353'17262","reported_seq":17631,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848456+0000","last_change":"2024-09-11T08:56:08.521607+0000","last_active":"2024-09-11T08:56:34.848456+0000","last_peered":"2024-09-11T08:56:34.848456+0000","last_clean":"2024-09-11T08:56:34.848456+0000","last_became_active":"2024-09-11T08:56:08.480296+0000","last_became_peered":"2024-09-11T08:56:08.480296+0000","last_unstale":"2024-09-11T08:56:34.848456+0000","last_undegraded":"2024-09-11T08:56:34.848456+0000","last_fullsized":"2024-09-11T08:56:34.848456+0000","mapping_epoch":345,"log_start":"337'12976","ondisk_log_start":"337'12976","created":293,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"329'10355","last_scrub_stamp":"2024-09-11T08:54:56.144911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:56.144911+0000","objects_scrubbed":7055,"log_size":4286,"log_dups_size":0,"ondisk_log_size":4286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3.6035077170000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17262,"num_write_kb":552384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"199'81","reported_seq":139,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847757+0000","last_change":"2024-09-11T08:55:53.823820+0000","last_active":"2024-09-11T08:56:34.847757+0000","last_peered":"2024-09-11T08:56:34.847757+0000","last_clean":"2024-09-11T08:56:34.847757+0000","last_became_active":"2024-09-11T08:51:03.009983+0000","last_became_peered":"2024-09-11T08:51:03.009983+0000","last_unstale":"2024-09-11T08:56:34.847757+0000","last_undegraded":"2024-09-11T08:56:34.847757+0000","last_fullsized":"2024-09-11T08:56:34.847757+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:55:53.823609+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:55:53.823609+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:54.416640+0000","scrub_duration":0.0052441400000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"200'35","reported_seq":408,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.526102+0000","last_change":"2024-09-11T08:55:53.637038+0000","last_active":"2024-09-11T08:56:34.526102+0000","last_peered":"2024-09-11T08:56:34.526102+0000","last_clean":"2024-09-11T08:56:34.526102+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:56:34.526102+0000","last_undegraded":"2024-09-11T08:56:34.526102+0000","last_fullsized":"2024-09-11T08:56:34.526102+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'35","last_scrub_stamp":"2024-09-11T08:55:53.636905+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:55:53.636905+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:03.660454+0000","scrub_duration":0.0057013680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":116,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.126722+0000","last_change":"2024-09-11T08:55:42.487232+0000","last_active":"2024-09-11T08:56:31.126722+0000","last_peered":"2024-09-11T08:56:31.126722+0000","last_clean":"2024-09-11T08:56:31.126722+0000","last_became_active":"2024-09-11T08:52:18.586549+0000","last_became_peered":"2024-09-11T08:52:18.586549+0000","last_unstale":"2024-09-11T08:56:31.126722+0000","last_undegraded":"2024-09-11T08:56:31.126722+0000","last_fullsized":"2024-09-11T08:56:31.126722+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:42.487183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:28.487573+0000","last_clean_scrub_stamp":"2024-09-11T08:55:42.487183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:07.788535+0000","scrub_duration":0.0040695080000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":580,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:37.875395+0000","last_change":"2024-09-11T08:54:17.570568+0000","last_active":"2024-09-11T08:56:37.875395+0000","last_peered":"2024-09-11T08:56:37.875395+0000","last_clean":"2024-09-11T08:56:37.875395+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:56:37.875395+0000","last_undegraded":"2024-09-11T08:56:37.875395+0000","last_fullsized":"2024-09-11T08:56:37.875395+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:54:17.570479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:17.570479+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077913949999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":191,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.817354+0000","last_change":"2024-09-11T08:56:08.478613+0000","last_active":"2024-09-11T08:56:34.817354+0000","last_peered":"2024-09-11T08:56:34.817354+0000","last_clean":"2024-09-11T08:56:34.817354+0000","last_became_active":"2024-09-11T08:56:08.478237+0000","last_became_peered":"2024-09-11T08:56:08.478237+0000","last_unstale":"2024-09-11T08:56:34.817354+0000","last_undegraded":"2024-09-11T08:56:34.817354+0000","last_fullsized":"2024-09-11T08:56:34.817354+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:55.551220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:55:55.551220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:20.706293+0000","scrub_duration":0.001492431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"353'17156","reported_seq":17226,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.527129+0000","last_change":"2024-09-11T08:52:56.414997+0000","last_active":"2024-09-11T08:56:34.527129+0000","last_peered":"2024-09-11T08:56:34.527129+0000","last_clean":"2024-09-11T08:56:34.527129+0000","last_became_active":"2024-09-11T08:52:56.414726+0000","last_became_peered":"2024-09-11T08:52:56.414726+0000","last_unstale":"2024-09-11T08:56:34.527129+0000","last_undegraded":"2024-09-11T08:56:34.527129+0000","last_fullsized":"2024-09-11T08:56:34.527129+0000","mapping_epoch":293,"log_start":"337'12800","ondisk_log_start":"337'12800","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:52:54.586221+0000","objects_scrubbed":0,"log_size":4356,"log_dups_size":0,"ondisk_log_size":4356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17156,"num_write_kb":548992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"199'81","reported_seq":265,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848583+0000","last_change":"2024-09-11T08:56:08.521129+0000","last_active":"2024-09-11T08:56:34.848583+0000","last_peered":"2024-09-11T08:56:34.848583+0000","last_clean":"2024-09-11T08:56:34.848583+0000","last_became_active":"2024-09-11T08:56:08.480247+0000","last_became_peered":"2024-09-11T08:56:08.480247+0000","last_unstale":"2024-09-11T08:56:34.848583+0000","last_undegraded":"2024-09-11T08:56:34.848583+0000","last_fullsized":"2024-09-11T08:56:34.848583+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:55:13.377713+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:55:13.377713+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002308867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"200'25","reported_seq":264,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156597+0000","last_change":"2024-09-11T08:56:25.483744+0000","last_active":"2024-09-11T08:56:31.156597+0000","last_peered":"2024-09-11T08:56:31.156597+0000","last_clean":"2024-09-11T08:56:31.156597+0000","last_became_active":"2024-09-11T08:56:25.462200+0000","last_became_peered":"2024-09-11T08:56:25.462200+0000","last_unstale":"2024-09-11T08:56:31.156597+0000","last_undegraded":"2024-09-11T08:56:31.156597+0000","last_fullsized":"2024-09-11T08:56:31.156597+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:54:05.976089+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_clean_scrub_stamp":"2024-09-11T08:54:05.976089+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049584310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":372,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156482+0000","last_change":"2024-09-11T08:55:49.588659+0000","last_active":"2024-09-11T08:56:31.156482+0000","last_peered":"2024-09-11T08:56:31.156482+0000","last_clean":"2024-09-11T08:56:31.156482+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:56:31.156482+0000","last_undegraded":"2024-09-11T08:56:31.156482+0000","last_fullsized":"2024-09-11T08:56:31.156482+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:49.588588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:55:49.588588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:18.312931+0000","scrub_duration":0.0032745880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"346'12","reported_seq":47,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:32.874402+0000","last_change":"2024-09-11T08:56:08.486445+0000","last_active":"2024-09-11T08:56:32.874402+0000","last_peered":"2024-09-11T08:56:32.874402+0000","last_clean":"2024-09-11T08:56:32.874402+0000","last_became_active":"2024-09-11T08:56:08.482468+0000","last_became_peered":"2024-09-11T08:56:08.482468+0000","last_unstale":"2024-09-11T08:56:32.874402+0000","last_undegraded":"2024-09-11T08:56:32.874402+0000","last_fullsized":"2024-09-11T08:56:32.874402+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"292'11","last_scrub_stamp":"2024-09-11T08:54:56.468128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:56.468128+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":173,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847490+0000","last_change":"2024-09-11T08:54:11.523221+0000","last_active":"2024-09-11T08:56:34.847490+0000","last_peered":"2024-09-11T08:56:34.847490+0000","last_clean":"2024-09-11T08:56:34.847490+0000","last_became_active":"2024-09-11T08:52:49.571683+0000","last_became_peered":"2024-09-11T08:52:49.571683+0000","last_unstale":"2024-09-11T08:56:34.847490+0000","last_undegraded":"2024-09-11T08:56:34.847490+0000","last_fullsized":"2024-09-11T08:56:34.847490+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:11.523143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:54:11.523143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010580648999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.d","version":"353'17058","reported_seq":17423,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.526973+0000","last_change":"2024-09-11T08:56:08.545808+0000","last_active":"2024-09-11T08:56:34.526973+0000","last_peered":"2024-09-11T08:56:34.526973+0000","last_clean":"2024-09-11T08:56:34.526973+0000","last_became_active":"2024-09-11T08:56:08.480067+0000","last_became_peered":"2024-09-11T08:56:08.480067+0000","last_unstale":"2024-09-11T08:56:34.526973+0000","last_undegraded":"2024-09-11T08:56:34.526973+0000","last_fullsized":"2024-09-11T08:56:34.526973+0000","mapping_epoch":345,"log_start":"336'12627","ondisk_log_start":"336'12627","created":293,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'7046","last_scrub_stamp":"2024-09-11T08:54:13.634607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:13.634607+0000","objects_scrubbed":6942,"log_size":4431,"log_dups_size":0,"ondisk_log_size":4431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3.0316847519999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17058,"num_write_kb":545793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"199'81","reported_seq":265,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847948+0000","last_change":"2024-09-11T08:56:08.479646+0000","last_active":"2024-09-11T08:56:34.847948+0000","last_peered":"2024-09-11T08:56:34.847948+0000","last_clean":"2024-09-11T08:56:34.847948+0000","last_became_active":"2024-09-11T08:56:08.479256+0000","last_became_peered":"2024-09-11T08:56:08.479256+0000","last_unstale":"2024-09-11T08:56:34.847948+0000","last_undegraded":"2024-09-11T08:56:34.847948+0000","last_fullsized":"2024-09-11T08:56:34.847948+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:55:35.328190+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:55:35.328190+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002308867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"200'29","reported_seq":420,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156319+0000","last_change":"2024-09-11T08:53:57.508836+0000","last_active":"2024-09-11T08:56:31.156319+0000","last_peered":"2024-09-11T08:56:31.156319+0000","last_clean":"2024-09-11T08:56:31.156319+0000","last_became_active":"2024-09-11T08:52:49.563364+0000","last_became_peered":"2024-09-11T08:52:49.563364+0000","last_unstale":"2024-09-11T08:56:31.156319+0000","last_undegraded":"2024-09-11T08:56:31.156319+0000","last_fullsized":"2024-09-11T08:56:31.156319+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'29","last_scrub_stamp":"2024-09-11T08:53:57.508749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:53:57.508749+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044953129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":386,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.126520+0000","last_change":"2024-09-11T08:54:53.192428+0000","last_active":"2024-09-11T08:56:31.126520+0000","last_peered":"2024-09-11T08:56:31.126520+0000","last_clean":"2024-09-11T08:56:31.126520+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:56:31.126520+0000","last_undegraded":"2024-09-11T08:56:31.126520+0000","last_fullsized":"2024-09-11T08:56:31.126520+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:53.192352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:54:53.192352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058229110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":327,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.155948+0000","last_change":"2024-09-11T08:56:08.527490+0000","last_active":"2024-09-11T08:56:31.155948+0000","last_peered":"2024-09-11T08:56:31.155948+0000","last_clean":"2024-09-11T08:56:31.155948+0000","last_became_active":"2024-09-11T08:56:08.527106+0000","last_became_peered":"2024-09-11T08:56:08.527106+0000","last_unstale":"2024-09-11T08:56:31.155948+0000","last_undegraded":"2024-09-11T08:56:31.155948+0000","last_fullsized":"2024-09-11T08:56:31.155948+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:59.511833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_clean_scrub_stamp":"2024-09-11T08:54:59.511833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011823264,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":400,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.126077+0000","last_change":"2024-09-11T08:54:59.195553+0000","last_active":"2024-09-11T08:56:31.126077+0000","last_peered":"2024-09-11T08:56:31.126077+0000","last_clean":"2024-09-11T08:56:31.126077+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:56:31.126077+0000","last_undegraded":"2024-09-11T08:56:31.126077+0000","last_fullsized":"2024-09-11T08:56:31.126077+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:59.195468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:54:59.195468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016653340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"353'17008","reported_seq":17358,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:33.288337+0000","last_change":"2024-09-11T08:56:08.527148+0000","last_active":"2024-09-11T08:56:33.288337+0000","last_peered":"2024-09-11T08:56:33.288337+0000","last_clean":"2024-09-11T08:56:33.288337+0000","last_became_active":"2024-09-11T08:56:08.480572+0000","last_became_peered":"2024-09-11T08:56:08.480572+0000","last_unstale":"2024-09-11T08:56:33.288337+0000","last_undegraded":"2024-09-11T08:56:33.288337+0000","last_fullsized":"2024-09-11T08:56:33.288337+0000","mapping_epoch":345,"log_start":"333'11891","ondisk_log_start":"333'11891","created":293,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'6646","last_scrub_stamp":"2024-09-11T08:54:09.062953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.062953+0000","objects_scrubbed":6570,"log_size":5117,"log_dups_size":0,"ondisk_log_size":5117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2.4302389890000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17008,"num_write_kb":544256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"199'81","reported_seq":133,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.155883+0000","last_change":"2024-09-11T08:54:09.431646+0000","last_active":"2024-09-11T08:56:31.155883+0000","last_peered":"2024-09-11T08:56:31.155883+0000","last_clean":"2024-09-11T08:56:31.155883+0000","last_became_active":"2024-09-11T08:52:18.574468+0000","last_became_peered":"2024-09-11T08:52:18.574468+0000","last_unstale":"2024-09-11T08:56:31.155883+0000","last_undegraded":"2024-09-11T08:56:31.155883+0000","last_fullsized":"2024-09-11T08:56:31.155883+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:54:09.431556+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.431556+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066392939999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"349'51","reported_seq":444,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:37.875628+0000","last_change":"2024-09-11T08:56:28.826027+0000","last_active":"2024-09-11T08:56:37.875628+0000","last_peered":"2024-09-11T08:56:37.875628+0000","last_clean":"2024-09-11T08:56:37.875628+0000","last_became_active":"2024-09-11T08:56:25.466103+0000","last_became_peered":"2024-09-11T08:56:25.466103+0000","last_unstale":"2024-09-11T08:56:37.875628+0000","last_undegraded":"2024-09-11T08:56:37.875628+0000","last_fullsized":"2024-09-11T08:56:37.875628+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'50","last_scrub_stamp":"2024-09-11T08:55:00.215860+0000","last_deep_scrub":"200'50","last_deep_scrub_stamp":"2024-09-11T08:53:09.745322+0000","last_clean_scrub_stamp":"2024-09-11T08:55:00.215860+0000","objects_scrubbed":10,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020004279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":388,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.527337+0000","last_change":"2024-09-11T08:54:56.719900+0000","last_active":"2024-09-11T08:56:34.527337+0000","last_peered":"2024-09-11T08:56:34.527337+0000","last_clean":"2024-09-11T08:56:34.527337+0000","last_became_active":"2024-09-11T08:52:18.579509+0000","last_became_peered":"2024-09-11T08:52:18.579509+0000","last_unstale":"2024-09-11T08:56:34.527337+0000","last_undegraded":"2024-09-11T08:56:34.527337+0000","last_fullsized":"2024-09-11T08:56:34.527337+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:56.719827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_clean_scrub_stamp":"2024-09-11T08:54:56.719827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003589297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":407,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156622+0000","last_change":"2024-09-11T08:54:14.424233+0000","last_active":"2024-09-11T08:56:31.156622+0000","last_peered":"2024-09-11T08:56:31.156622+0000","last_clean":"2024-09-11T08:56:31.156622+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:56:31.156622+0000","last_undegraded":"2024-09-11T08:56:31.156622+0000","last_fullsized":"2024-09-11T08:56:31.156622+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:54:14.423685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:54:14.423685+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012346487999999999,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":182,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.125597+0000","last_change":"2024-09-11T08:55:08.121800+0000","last_active":"2024-09-11T08:56:31.125597+0000","last_peered":"2024-09-11T08:56:31.125597+0000","last_clean":"2024-09-11T08:56:31.125597+0000","last_became_active":"2024-09-11T08:52:49.569137+0000","last_became_peered":"2024-09-11T08:52:49.569137+0000","last_unstale":"2024-09-11T08:56:31.125597+0000","last_undegraded":"2024-09-11T08:56:31.125597+0000","last_fullsized":"2024-09-11T08:56:31.125597+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:08.121720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:55:08.121720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038461900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":361,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848032+0000","last_change":"2024-09-11T08:56:32.836719+0000","last_active":"2024-09-11T08:56:34.848032+0000","last_peered":"2024-09-11T08:56:34.848032+0000","last_clean":"2024-09-11T08:56:34.848032+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:56:34.848032+0000","last_undegraded":"2024-09-11T08:56:34.848032+0000","last_fullsized":"2024-09-11T08:56:34.848032+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:32.836633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:56:32.836633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:43.446681+0000","scrub_duration":0.0050927949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"353'17220","reported_seq":17468,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:33.293928+0000","last_change":"2024-09-11T08:55:36.393911+0000","last_active":"2024-09-11T08:56:33.293928+0000","last_peered":"2024-09-11T08:56:33.293928+0000","last_clean":"2024-09-11T08:56:33.293928+0000","last_became_active":"2024-09-11T08:52:56.413594+0000","last_became_peered":"2024-09-11T08:52:56.413594+0000","last_unstale":"2024-09-11T08:56:33.293928+0000","last_undegraded":"2024-09-11T08:56:33.293928+0000","last_fullsized":"2024-09-11T08:56:33.293928+0000","mapping_epoch":293,"log_start":"334'12068","ondisk_log_start":"334'12068","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"338'13048","last_scrub_stamp":"2024-09-11T08:55:36.393833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:55:36.393833+0000","objects_scrubbed":4232,"log_size":5152,"log_dups_size":0,"ondisk_log_size":5152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:42.628357+0000","scrub_duration":1.5874557139999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17220,"num_write_kb":551040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"199'81","reported_seq":413,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.526051+0000","last_change":"2024-09-11T08:55:00.708661+0000","last_active":"2024-09-11T08:56:34.526051+0000","last_peered":"2024-09-11T08:56:34.526051+0000","last_clean":"2024-09-11T08:56:34.526051+0000","last_became_active":"2024-09-11T08:52:49.569934+0000","last_became_peered":"2024-09-11T08:52:49.569934+0000","last_unstale":"2024-09-11T08:56:34.526051+0000","last_undegraded":"2024-09-11T08:56:34.526051+0000","last_fullsized":"2024-09-11T08:56:34.526051+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:55:00.708581+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:55:00.708581+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053390399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"200'17","reported_seq":262,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156722+0000","last_change":"2024-09-11T08:56:25.468694+0000","last_active":"2024-09-11T08:56:31.156722+0000","last_peered":"2024-09-11T08:56:31.156722+0000","last_clean":"2024-09-11T08:56:31.156722+0000","last_became_active":"2024-09-11T08:56:25.459902+0000","last_became_peered":"2024-09-11T08:56:25.459902+0000","last_unstale":"2024-09-11T08:56:31.156722+0000","last_undegraded":"2024-09-11T08:56:31.156722+0000","last_fullsized":"2024-09-11T08:56:31.156722+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'17","last_scrub_stamp":"2024-09-11T08:55:33.394000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:33.394000+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:02.476458+0000","scrub_duration":0.0023582730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":402,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.124619+0000","last_change":"2024-09-11T08:54:18.762957+0000","last_active":"2024-09-11T08:56:31.124619+0000","last_peered":"2024-09-11T08:56:31.124619+0000","last_clean":"2024-09-11T08:56:31.124619+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:56:31.124619+0000","last_undegraded":"2024-09-11T08:56:31.124619+0000","last_fullsized":"2024-09-11T08:56:31.124619+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:18.762880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:54:18.762880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001880451,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"263'2","reported_seq":458,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.816773+0000","last_change":"2024-09-11T08:55:45.860554+0000","last_active":"2024-09-11T08:56:34.816773+0000","last_peered":"2024-09-11T08:56:34.816773+0000","last_clean":"2024-09-11T08:56:34.816773+0000","last_became_active":"2024-09-11T08:52:18.586545+0000","last_became_peered":"2024-09-11T08:52:18.586545+0000","last_unstale":"2024-09-11T08:56:34.816773+0000","last_undegraded":"2024-09-11T08:56:34.816773+0000","last_fullsized":"2024-09-11T08:56:34.816773+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'2","last_scrub_stamp":"2024-09-11T08:55:45.860496+0000","last_deep_scrub":"263'2","last_deep_scrub_stamp":"2024-09-11T08:53:08.772979+0000","last_clean_scrub_stamp":"2024-09-11T08:55:45.860496+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:53.555036+0000","scrub_duration":0.006812823,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":408,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.817637+0000","last_change":"2024-09-11T08:55:33.820051+0000","last_active":"2024-09-11T08:56:34.817637+0000","last_peered":"2024-09-11T08:56:34.817637+0000","last_clean":"2024-09-11T08:56:34.817637+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:56:34.817637+0000","last_undegraded":"2024-09-11T08:56:34.817637+0000","last_fullsized":"2024-09-11T08:56:34.817637+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:33.820000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_clean_scrub_stamp":"2024-09-11T08:55:33.820000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028405150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":191,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847866+0000","last_change":"2024-09-11T08:56:08.478405+0000","last_active":"2024-09-11T08:56:34.847866+0000","last_peered":"2024-09-11T08:56:34.847866+0000","last_clean":"2024-09-11T08:56:34.847866+0000","last_became_active":"2024-09-11T08:56:08.478027+0000","last_became_peered":"2024-09-11T08:56:08.478027+0000","last_unstale":"2024-09-11T08:56:34.847866+0000","last_undegraded":"2024-09-11T08:56:34.847866+0000","last_fullsized":"2024-09-11T08:56:34.847866+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:53.557746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:55:53.557746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:00.809449+0000","scrub_duration":0.0015184790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"353'17200","reported_seq":17692,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.817038+0000","last_change":"2024-09-11T08:55:49.034480+0000","last_active":"2024-09-11T08:56:34.817038+0000","last_peered":"2024-09-11T08:56:34.817038+0000","last_clean":"2024-09-11T08:56:34.817038+0000","last_became_active":"2024-09-11T08:52:56.413725+0000","last_became_peered":"2024-09-11T08:52:56.413725+0000","last_unstale":"2024-09-11T08:56:34.817038+0000","last_undegraded":"2024-09-11T08:56:34.817038+0000","last_fullsized":"2024-09-11T08:56:34.817038+0000","mapping_epoch":293,"log_start":"331'11547","ondisk_log_start":"331'11547","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"341'14008","last_scrub_stamp":"2024-09-11T08:55:49.034428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:55:49.034428+0000","objects_scrubbed":3241,"log_size":5653,"log_dups_size":0,"ondisk_log_size":5653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:01.376946+0000","scrub_duration":1.2259194879999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17200,"num_write_kb":550400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"199'81","reported_seq":396,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.125703+0000","last_change":"2024-09-11T08:54:54.193424+0000","last_active":"2024-09-11T08:56:31.125703+0000","last_peered":"2024-09-11T08:56:31.125703+0000","last_clean":"2024-09-11T08:56:31.125703+0000","last_became_active":"2024-09-11T08:51:03.027281+0000","last_became_peered":"2024-09-11T08:51:03.027281+0000","last_unstale":"2024-09-11T08:56:31.125703+0000","last_undegraded":"2024-09-11T08:56:31.125703+0000","last_fullsized":"2024-09-11T08:56:31.125703+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:54:54.193339+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:54:54.193339+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006055899,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"200'24","reported_seq":130,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.126534+0000","last_change":"2024-09-11T08:55:47.495396+0000","last_active":"2024-09-11T08:56:31.126534+0000","last_peered":"2024-09-11T08:56:31.126534+0000","last_clean":"2024-09-11T08:56:31.126534+0000","last_became_active":"2024-09-11T08:52:18.580950+0000","last_became_peered":"2024-09-11T08:52:18.580950+0000","last_unstale":"2024-09-11T08:56:31.126534+0000","last_undegraded":"2024-09-11T08:56:31.126534+0000","last_fullsized":"2024-09-11T08:56:31.126534+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'24","last_scrub_stamp":"2024-09-11T08:55:47.495309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:47.495309+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:03.500135+0000","scrub_duration":0.014887852,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.155906+0000","last_change":"2024-09-11T08:55:31.080606+0000","last_active":"2024-09-11T08:56:31.155906+0000","last_peered":"2024-09-11T08:56:31.155906+0000","last_clean":"2024-09-11T08:56:31.155906+0000","last_became_active":"2024-09-11T08:55:31.080178+0000","last_became_peered":"2024-09-11T08:55:31.080178+0000","last_unstale":"2024-09-11T08:56:31.155906+0000","last_undegraded":"2024-09-11T08:56:31.155906+0000","last_fullsized":"2024-09-11T08:56:31.155906+0000","mapping_epoch":335,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":336,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:18.845896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:18.845896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"318'6","reported_seq":36,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:40.499531+0000","last_change":"2024-09-11T08:56:25.470170+0000","last_active":"2024-09-11T08:56:40.499531+0000","last_peered":"2024-09-11T08:56:40.499531+0000","last_clean":"2024-09-11T08:56:40.499531+0000","last_became_active":"2024-09-11T08:56:25.462503+0000","last_became_peered":"2024-09-11T08:56:25.462503+0000","last_unstale":"2024-09-11T08:56:40.499531+0000","last_undegraded":"2024-09-11T08:56:40.499531+0000","last_fullsized":"2024-09-11T08:56:40.499531+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'6","last_scrub_stamp":"2024-09-11T08:55:42.632806+0000","last_deep_scrub":"318'6","last_deep_scrub_stamp":"2024-09-11T08:54:12.873653+0000","last_clean_scrub_stamp":"2024-09-11T08:55:42.632806+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:46.327218+0000","scrub_duration":0.0017088629999999999,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":111,"num_read_kb":106,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":388,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156840+0000","last_change":"2024-09-11T08:54:16.367927+0000","last_active":"2024-09-11T08:56:31.156840+0000","last_peered":"2024-09-11T08:56:31.156840+0000","last_clean":"2024-09-11T08:56:31.156840+0000","last_became_active":"2024-09-11T08:52:18.578530+0000","last_became_peered":"2024-09-11T08:52:18.578530+0000","last_unstale":"2024-09-11T08:56:31.156840+0000","last_undegraded":"2024-09-11T08:56:31.156840+0000","last_fullsized":"2024-09-11T08:56:31.156840+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:16.367873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:16.367873+0000","last_clean_scrub_stamp":"2024-09-11T08:54:16.367873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013677540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":18,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156436+0000","last_change":"2024-09-11T08:56:25.460095+0000","last_active":"2024-09-11T08:56:31.156436+0000","last_peered":"2024-09-11T08:56:31.156436+0000","last_clean":"2024-09-11T08:56:31.156436+0000","last_became_active":"2024-09-11T08:56:25.459701+0000","last_became_peered":"2024-09-11T08:56:25.459701+0000","last_unstale":"2024-09-11T08:56:31.156436+0000","last_undegraded":"2024-09-11T08:56:31.156436+0000","last_fullsized":"2024-09-11T08:56:31.156436+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:13.179675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:55:13.179675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:35.224463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"353'17076","reported_seq":17567,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:33.280547+0000","last_change":"2024-09-11T08:56:08.545644+0000","last_active":"2024-09-11T08:56:33.280547+0000","last_peered":"2024-09-11T08:56:33.280547+0000","last_clean":"2024-09-11T08:56:33.280547+0000","last_became_active":"2024-09-11T08:56:08.479225+0000","last_became_peered":"2024-09-11T08:56:08.479225+0000","last_unstale":"2024-09-11T08:56:33.280547+0000","last_undegraded":"2024-09-11T08:56:33.280547+0000","last_fullsized":"2024-09-11T08:56:33.280547+0000","mapping_epoch":345,"log_start":"330'11264","ondisk_log_start":"330'11264","created":293,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"343'14412","last_scrub_stamp":"2024-09-11T08:55:55.477321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:55:55.477321+0000","objects_scrubbed":2698,"log_size":5812,"log_dups_size":0,"ondisk_log_size":5812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:17.779862+0000","scrub_duration":0.95135739600000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17076,"num_write_kb":546432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"201'27","reported_seq":113,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.527660+0000","last_change":"2024-09-11T08:56:08.547237+0000","last_active":"2024-09-11T08:56:34.527660+0000","last_peered":"2024-09-11T08:56:34.527660+0000","last_clean":"2024-09-11T08:56:34.527660+0000","last_became_active":"2024-09-11T08:56:08.544409+0000","last_became_peered":"2024-09-11T08:56:08.544409+0000","last_unstale":"2024-09-11T08:56:34.527660+0000","last_undegraded":"2024-09-11T08:56:34.527660+0000","last_fullsized":"2024-09-11T08:56:34.527660+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'27","last_scrub_stamp":"2024-09-11T08:55:50.542013+0000","last_deep_scrub":"201'27","last_deep_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_clean_scrub_stamp":"2024-09-11T08:55:50.542013+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:18.501009+0000","scrub_duration":0.0080208199999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":366,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848299+0000","last_change":"2024-09-11T08:53:51.352190+0000","last_active":"2024-09-11T08:56:34.848299+0000","last_peered":"2024-09-11T08:56:34.848299+0000","last_clean":"2024-09-11T08:56:34.848299+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:56:34.848299+0000","last_undegraded":"2024-09-11T08:56:34.848299+0000","last_fullsized":"2024-09-11T08:56:34.848299+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:53:51.352089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:51.352089+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032468750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"199'81","reported_seq":400,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:36.496210+0000","last_change":"2024-09-11T08:56:36.496178+0000","last_active":"2024-09-11T08:56:36.496210+0000","last_peered":"2024-09-11T08:56:36.496210+0000","last_clean":"2024-09-11T08:56:36.496210+0000","last_became_active":"2024-09-11T08:56:08.484662+0000","last_became_peered":"2024-09-11T08:56:08.484662+0000","last_unstale":"2024-09-11T08:56:36.496210+0000","last_undegraded":"2024-09-11T08:56:36.496210+0000","last_fullsized":"2024-09-11T08:56:36.496210+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:56:36.496096+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:55:08.518651+0000","last_clean_scrub_stamp":"2024-09-11T08:56:36.496096+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:39.488379+0000","scrub_duration":0.001718827,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":110,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:39.883401+0000","last_change":"2024-09-11T08:56:39.883358+0000","last_active":"2024-09-11T08:56:39.883401+0000","last_peered":"2024-09-11T08:56:39.883401+0000","last_clean":"2024-09-11T08:56:39.883401+0000","last_became_active":"2024-09-11T08:52:38.485446+0000","last_became_peered":"2024-09-11T08:52:38.485446+0000","last_unstale":"2024-09-11T08:56:39.883401+0000","last_undegraded":"2024-09-11T08:56:39.883401+0000","last_fullsized":"2024-09-11T08:56:39.883401+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:39.883298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:56:39.883298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:01.515267+0000","scrub_duration":0.0018155319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":366,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156773+0000","last_change":"2024-09-11T08:53:54.582425+0000","last_active":"2024-09-11T08:56:31.156773+0000","last_peered":"2024-09-11T08:56:31.156773+0000","last_clean":"2024-09-11T08:56:31.156773+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:56:31.156773+0000","last_undegraded":"2024-09-11T08:56:31.156773+0000","last_fullsized":"2024-09-11T08:56:31.156773+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:54.582338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:53:54.582338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004605914,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"353'17196","reported_seq":17566,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:33.293998+0000","last_change":"2024-09-11T08:56:08.547795+0000","last_active":"2024-09-11T08:56:33.293998+0000","last_peered":"2024-09-11T08:56:33.293998+0000","last_clean":"2024-09-11T08:56:33.293998+0000","last_became_active":"2024-09-11T08:56:08.491471+0000","last_became_peered":"2024-09-11T08:56:08.491471+0000","last_unstale":"2024-09-11T08:56:33.293998+0000","last_undegraded":"2024-09-11T08:56:33.293998+0000","last_fullsized":"2024-09-11T08:56:33.293998+0000","mapping_epoch":345,"log_start":"330'11376","ondisk_log_start":"330'11376","created":293,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"329'10345","last_scrub_stamp":"2024-09-11T08:54:56.045754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:56.045754+0000","objects_scrubbed":6935,"log_size":5820,"log_dups_size":0,"ondisk_log_size":5820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2.5360603799999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17196,"num_write_kb":550272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"200'19","reported_seq":478,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.527321+0000","last_change":"2024-09-11T08:56:25.483701+0000","last_active":"2024-09-11T08:56:34.527321+0000","last_peered":"2024-09-11T08:56:34.527321+0000","last_clean":"2024-09-11T08:56:34.527321+0000","last_became_active":"2024-09-11T08:56:25.465998+0000","last_became_peered":"2024-09-11T08:56:25.465998+0000","last_unstale":"2024-09-11T08:56:34.527321+0000","last_undegraded":"2024-09-11T08:56:34.527321+0000","last_fullsized":"2024-09-11T08:56:34.527321+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:55:01.735933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:01.735933+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017616557000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":358,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.526239+0000","last_change":"2024-09-11T08:55:11.804648+0000","last_active":"2024-09-11T08:56:34.526239+0000","last_peered":"2024-09-11T08:56:34.526239+0000","last_clean":"2024-09-11T08:56:34.526239+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:56:34.526239+0000","last_undegraded":"2024-09-11T08:56:34.526239+0000","last_fullsized":"2024-09-11T08:56:34.526239+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:11.804570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:55:11.804570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010636980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":375,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156698+0000","last_change":"2024-09-11T08:55:06.261924+0000","last_active":"2024-09-11T08:56:31.156698+0000","last_peered":"2024-09-11T08:56:31.156698+0000","last_clean":"2024-09-11T08:56:31.156698+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:56:31.156698+0000","last_undegraded":"2024-09-11T08:56:31.156698+0000","last_fullsized":"2024-09-11T08:56:31.156698+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:06.261834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:55:06.261834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015153250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":381,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-11T08:56:40.886012+0000","last_change":"2024-09-11T08:56:40.885862+0000","last_active":"2024-09-11T08:56:40.886012+0000","last_peered":"2024-09-11T08:56:40.886012+0000","last_clean":"2024-09-11T08:56:40.886012+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:56:40.886012+0000","last_undegraded":"2024-09-11T08:56:40.886012+0000","last_fullsized":"2024-09-11T08:56:40.886012+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:40.885769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:56:40.885769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:10.610689+0000","scrub_duration":0.0025500079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"353'21774","reported_seq":22125,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:33.300615+0000","last_change":"2024-09-11T08:56:00.294292+0000","last_active":"2024-09-11T08:56:33.300615+0000","last_peered":"2024-09-11T08:56:33.300615+0000","last_clean":"2024-09-11T08:56:33.300615+0000","last_became_active":"2024-09-11T08:54:17.120339+0000","last_became_peered":"2024-09-11T08:54:17.120339+0000","last_unstale":"2024-09-11T08:56:33.300615+0000","last_undegraded":"2024-09-11T08:56:33.300615+0000","last_fullsized":"2024-09-11T08:56:33.300615+0000","mapping_epoch":319,"log_start":"340'16618","ondisk_log_start":"340'16618","created":293,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"344'19292","last_scrub_stamp":"2024-09-11T08:56:00.294206+0000","last_deep_scrub":"344'19292","last_deep_scrub_stamp":"2024-09-11T08:56:00.294206+0000","last_clean_scrub_stamp":"2024-09-11T08:56:00.294206+0000","objects_scrubbed":2931,"log_size":5156,"log_dups_size":0,"ondisk_log_size":5156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:13.653742+0000","scrub_duration":9.7480149730000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17292,"num_write_kb":553344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.8","version":"199'81","reported_seq":140,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.155703+0000","last_change":"2024-09-11T08:55:33.781406+0000","last_active":"2024-09-11T08:56:31.155703+0000","last_peered":"2024-09-11T08:56:31.155703+0000","last_clean":"2024-09-11T08:56:31.155703+0000","last_became_active":"2024-09-11T08:51:03.020941+0000","last_became_peered":"2024-09-11T08:51:03.020941+0000","last_unstale":"2024-09-11T08:56:31.155703+0000","last_undegraded":"2024-09-11T08:56:31.155703+0000","last_fullsized":"2024-09-11T08:56:31.155703+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:55:33.781314+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:53:14.809894+0000","last_clean_scrub_stamp":"2024-09-11T08:55:33.781314+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:38.313782+0000","scrub_duration":0.0076612690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"317'39","reported_seq":423,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.527703+0000","last_change":"2024-09-11T08:54:06.928521+0000","last_active":"2024-09-11T08:56:34.527703+0000","last_peered":"2024-09-11T08:56:34.527703+0000","last_clean":"2024-09-11T08:56:34.527703+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:56:34.527703+0000","last_undegraded":"2024-09-11T08:56:34.527703+0000","last_fullsized":"2024-09-11T08:56:34.527703+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"317'39","last_scrub_stamp":"2024-09-11T08:54:06.928266+0000","last_deep_scrub":"317'39","last_deep_scrub_stamp":"2024-09-11T08:54:06.928266+0000","last_clean_scrub_stamp":"2024-09-11T08:54:06.928266+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016459872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.526994+0000","last_change":"2024-09-11T08:56:25.461213+0000","last_active":"2024-09-11T08:56:34.526994+0000","last_peered":"2024-09-11T08:56:34.526994+0000","last_clean":"2024-09-11T08:56:34.526994+0000","last_became_active":"2024-09-11T08:56:25.460770+0000","last_became_peered":"2024-09-11T08:56:25.460770+0000","last_unstale":"2024-09-11T08:56:34.526994+0000","last_undegraded":"2024-09-11T08:56:34.526994+0000","last_fullsized":"2024-09-11T08:56:34.526994+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:07.504031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:07.504031+0000","last_clean_scrub_stamp":"2024-09-11T08:55:07.504031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":402,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.126370+0000","last_change":"2024-09-11T08:56:30.438479+0000","last_active":"2024-09-11T08:56:31.126370+0000","last_peered":"2024-09-11T08:56:31.126370+0000","last_clean":"2024-09-11T08:56:31.126370+0000","last_became_active":"2024-09-11T08:52:18.577201+0000","last_became_peered":"2024-09-11T08:52:18.577201+0000","last_unstale":"2024-09-11T08:56:31.126370+0000","last_undegraded":"2024-09-11T08:56:31.126370+0000","last_fullsized":"2024-09-11T08:56:31.126370+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:30.438424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:56:30.438424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:58.599761+0000","scrub_duration":0.001194999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":58,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848295+0000","last_change":"2024-09-11T08:55:49.845729+0000","last_active":"2024-09-11T08:56:34.848295+0000","last_peered":"2024-09-11T08:56:34.848295+0000","last_clean":"2024-09-11T08:56:34.848295+0000","last_became_active":"2024-09-11T08:54:17.121372+0000","last_became_peered":"2024-09-11T08:54:17.121372+0000","last_unstale":"2024-09-11T08:56:34.848295+0000","last_undegraded":"2024-09-11T08:56:34.848295+0000","last_fullsized":"2024-09-11T08:56:34.848295+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:49.845654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:55:49.845654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:07.421114+0000","scrub_duration":0.018187542000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"353'17014","reported_seq":18358,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.525967+0000","last_change":"2024-09-11T08:56:31.196781+0000","last_active":"2024-09-11T08:56:34.525967+0000","last_peered":"2024-09-11T08:56:34.525967+0000","last_clean":"2024-09-11T08:56:34.525967+0000","last_became_active":"2024-09-11T08:56:31.131313+0000","last_became_peered":"2024-09-11T08:56:31.131313+0000","last_unstale":"2024-09-11T08:56:34.525967+0000","last_undegraded":"2024-09-11T08:56:34.525967+0000","last_fullsized":"2024-09-11T08:56:34.525967+0000","mapping_epoch":352,"log_start":"336'12642","ondisk_log_start":"336'12642","created":293,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'10137","last_scrub_stamp":"2024-09-11T08:54:54.973568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:54:54.973568+0000","objects_scrubbed":6983,"log_size":4372,"log_dups_size":0,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2.7437013509999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":544448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":25296896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.9","version":"199'81","reported_seq":417,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847576+0000","last_change":"2024-09-11T08:54:51.543848+0000","last_active":"2024-09-11T08:56:34.847576+0000","last_peered":"2024-09-11T08:56:34.847576+0000","last_clean":"2024-09-11T08:56:34.847576+0000","last_became_active":"2024-09-11T08:52:38.481475+0000","last_became_peered":"2024-09-11T08:52:38.481475+0000","last_unstale":"2024-09-11T08:56:34.847576+0000","last_undegraded":"2024-09-11T08:56:34.847576+0000","last_fullsized":"2024-09-11T08:56:34.847576+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:54:51.543754+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:54:51.543754+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034342589999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"200'20","reported_seq":436,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.527241+0000","last_change":"2024-09-11T08:55:52.656041+0000","last_active":"2024-09-11T08:56:34.527241+0000","last_peered":"2024-09-11T08:56:34.527241+0000","last_clean":"2024-09-11T08:56:34.527241+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:56:34.527241+0000","last_undegraded":"2024-09-11T08:56:34.527241+0000","last_fullsized":"2024-09-11T08:56:34.527241+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'20","last_scrub_stamp":"2024-09-11T08:55:52.655954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:52.655954+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:10.738713+0000","scrub_duration":0.012388883,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":387,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847241+0000","last_change":"2024-09-11T08:55:56.775203+0000","last_active":"2024-09-11T08:56:34.847241+0000","last_peered":"2024-09-11T08:56:34.847241+0000","last_clean":"2024-09-11T08:56:34.847241+0000","last_became_active":"2024-09-11T08:52:18.586343+0000","last_became_peered":"2024-09-11T08:52:18.586343+0000","last_unstale":"2024-09-11T08:56:34.847241+0000","last_undegraded":"2024-09-11T08:56:34.847241+0000","last_fullsized":"2024-09-11T08:56:34.847241+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:56.775123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:55:56.775123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:02.581734+0000","scrub_duration":0.0096545599999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"299'9","reported_seq":640,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:37.875623+0000","last_change":"2024-09-11T08:56:33.792530+0000","last_active":"2024-09-11T08:56:37.875623+0000","last_peered":"2024-09-11T08:56:37.875623+0000","last_clean":"2024-09-11T08:56:37.875623+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:56:37.875623+0000","last_undegraded":"2024-09-11T08:56:37.875623+0000","last_fullsized":"2024-09-11T08:56:37.875623+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'9","last_scrub_stamp":"2024-09-11T08:56:33.792380+0000","last_deep_scrub":"299'9","last_deep_scrub_stamp":"2024-09-11T08:53:05.835177+0000","last_clean_scrub_stamp":"2024-09-11T08:56:33.792380+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:01.264643+0000","scrub_duration":0.0021331810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":372,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:36.675263+0000","last_change":"2024-09-11T08:56:36.675218+0000","last_active":"2024-09-11T08:56:36.675263+0000","last_peered":"2024-09-11T08:56:36.675263+0000","last_clean":"2024-09-11T08:56:36.675263+0000","last_became_active":"2024-09-11T08:56:25.457566+0000","last_became_peered":"2024-09-11T08:56:25.457566+0000","last_unstale":"2024-09-11T08:56:36.675263+0000","last_undegraded":"2024-09-11T08:56:36.675263+0000","last_fullsized":"2024-09-11T08:56:36.675263+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:36.675151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:56:36.675151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:55.725829+0000","scrub_duration":0.0016343880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"353'17530","reported_seq":18038,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:33.294051+0000","last_change":"2024-09-11T08:55:43.276454+0000","last_active":"2024-09-11T08:56:33.294051+0000","last_peered":"2024-09-11T08:56:33.294051+0000","last_clean":"2024-09-11T08:56:33.294051+0000","last_became_active":"2024-09-11T08:54:17.121397+0000","last_became_peered":"2024-09-11T08:54:17.121397+0000","last_unstale":"2024-09-11T08:56:33.294051+0000","last_undegraded":"2024-09-11T08:56:33.294051+0000","last_fullsized":"2024-09-11T08:56:33.294051+0000","mapping_epoch":319,"log_start":"334'12529","ondisk_log_start":"334'12529","created":293,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"340'13785","last_scrub_stamp":"2024-09-11T08:55:43.276399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:55:43.276399+0000","objects_scrubbed":3801,"log_size":5001,"log_dups_size":0,"ondisk_log_size":5001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:53.503441+0000","scrub_duration":1.765820962,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17530,"num_write_kb":560960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"200'15","reported_seq":126,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.125128+0000","last_change":"2024-09-11T08:55:34.495783+0000","last_active":"2024-09-11T08:56:31.125128+0000","last_peered":"2024-09-11T08:56:31.125128+0000","last_clean":"2024-09-11T08:56:31.125128+0000","last_became_active":"2024-09-11T08:52:18.588329+0000","last_became_peered":"2024-09-11T08:52:18.588329+0000","last_unstale":"2024-09-11T08:56:31.125128+0000","last_undegraded":"2024-09-11T08:56:31.125128+0000","last_fullsized":"2024-09-11T08:56:31.125128+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:55:34.495708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:34.495708+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:59.801407+0000","scrub_duration":0.0049466689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":379,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:35.318081+0000","last_change":"2024-09-11T08:56:35.318047+0000","last_active":"2024-09-11T08:56:35.318081+0000","last_peered":"2024-09-11T08:56:35.318081+0000","last_clean":"2024-09-11T08:56:35.318081+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:56:35.318081+0000","last_undegraded":"2024-09-11T08:56:35.318081+0000","last_fullsized":"2024-09-11T08:56:35.318081+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:35.317983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:56:35.317983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:01.423122+0000","scrub_duration":0.0037140580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":48,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.526685+0000","last_change":"2024-09-11T08:54:17.120062+0000","last_active":"2024-09-11T08:56:34.526685+0000","last_peered":"2024-09-11T08:56:34.526685+0000","last_clean":"2024-09-11T08:56:34.526685+0000","last_became_active":"2024-09-11T08:54:17.119864+0000","last_became_peered":"2024-09-11T08:54:17.119864+0000","last_unstale":"2024-09-11T08:56:34.526685+0000","last_undegraded":"2024-09-11T08:56:34.526685+0000","last_fullsized":"2024-09-11T08:56:34.526685+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:06.683645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:06.683645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":110,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:38.833460+0000","last_change":"2024-09-11T08:56:38.833414+0000","last_active":"2024-09-11T08:56:38.833460+0000","last_peered":"2024-09-11T08:56:38.833460+0000","last_clean":"2024-09-11T08:56:38.833460+0000","last_became_active":"2024-09-11T08:52:49.569749+0000","last_became_peered":"2024-09-11T08:52:49.569749+0000","last_unstale":"2024-09-11T08:56:38.833460+0000","last_undegraded":"2024-09-11T08:56:38.833460+0000","last_fullsized":"2024-09-11T08:56:38.833460+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:38.833352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:13.770061+0000","last_clean_scrub_stamp":"2024-09-11T08:56:38.833352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:43.628241+0000","scrub_duration":0.0018346,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"353'16960","reported_seq":17040,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:35.726674+0000","last_change":"2024-09-11T08:56:35.726617+0000","last_active":"2024-09-11T08:56:35.726674+0000","last_peered":"2024-09-11T08:56:35.726674+0000","last_clean":"2024-09-11T08:56:35.726674+0000","last_became_active":"2024-09-11T08:52:56.412780+0000","last_became_peered":"2024-09-11T08:52:56.412780+0000","last_unstale":"2024-09-11T08:56:35.726674+0000","last_undegraded":"2024-09-11T08:56:35.726674+0000","last_fullsized":"2024-09-11T08:56:35.726674+0000","mapping_epoch":293,"log_start":"337'12600","ondisk_log_start":"337'12600","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"353'16960","last_scrub_stamp":"2024-09-11T08:56:35.726529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:56:35.726529+0000","objects_scrubbed":0,"log_size":4360,"log_dups_size":0,"ondisk_log_size":4360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:04.641465+0000","scrub_duration":0.0095553789999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":542720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"200'35","reported_seq":115,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.131903+0000","last_change":"2024-09-11T08:56:08.478774+0000","last_active":"2024-09-11T08:56:31.131903+0000","last_peered":"2024-09-11T08:56:31.131903+0000","last_clean":"2024-09-11T08:56:31.131903+0000","last_became_active":"2024-09-11T08:56:08.478328+0000","last_became_peered":"2024-09-11T08:56:08.478328+0000","last_unstale":"2024-09-11T08:56:31.131903+0000","last_undegraded":"2024-09-11T08:56:31.131903+0000","last_fullsized":"2024-09-11T08:56:31.131903+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'35","last_scrub_stamp":"2024-09-11T08:55:40.358114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:40.358114+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:06.551836+0000","scrub_duration":0.006189015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":54,"num_read_kb":43,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":90,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156383+0000","last_change":"2024-09-11T08:53:56.555571+0000","last_active":"2024-09-11T08:56:31.156383+0000","last_peered":"2024-09-11T08:56:31.156383+0000","last_clean":"2024-09-11T08:56:31.156383+0000","last_became_active":"2024-09-11T08:52:38.484857+0000","last_became_peered":"2024-09-11T08:52:38.484857+0000","last_unstale":"2024-09-11T08:56:31.156383+0000","last_undegraded":"2024-09-11T08:56:31.156383+0000","last_fullsized":"2024-09-11T08:56:31.156383+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:56.555486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:56.555486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053230159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":425,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.816773+0000","last_change":"2024-09-11T08:55:08.763871+0000","last_active":"2024-09-11T08:56:34.816773+0000","last_peered":"2024-09-11T08:56:34.816773+0000","last_clean":"2024-09-11T08:56:34.816773+0000","last_became_active":"2024-09-11T08:52:18.584748+0000","last_became_peered":"2024-09-11T08:52:18.584748+0000","last_unstale":"2024-09-11T08:56:34.816773+0000","last_undegraded":"2024-09-11T08:56:34.816773+0000","last_fullsized":"2024-09-11T08:56:34.816773+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:08.763752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:35.022256+0000","last_clean_scrub_stamp":"2024-09-11T08:55:08.763752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001980589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":52,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847684+0000","last_change":"2024-09-11T08:54:17.155880+0000","last_active":"2024-09-11T08:56:34.847684+0000","last_peered":"2024-09-11T08:56:34.847684+0000","last_clean":"2024-09-11T08:56:34.847684+0000","last_became_active":"2024-09-11T08:54:17.155625+0000","last_became_peered":"2024-09-11T08:54:17.155625+0000","last_unstale":"2024-09-11T08:56:34.847684+0000","last_undegraded":"2024-09-11T08:56:34.847684+0000","last_fullsized":"2024-09-11T08:56:34.847684+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:14.828985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:14.828985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.7","version":"353'17084","reported_seq":17565,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848363+0000","last_change":"2024-09-11T08:56:08.521501+0000","last_active":"2024-09-11T08:56:34.848363+0000","last_peered":"2024-09-11T08:56:34.848363+0000","last_clean":"2024-09-11T08:56:34.848363+0000","last_became_active":"2024-09-11T08:56:08.480970+0000","last_became_peered":"2024-09-11T08:56:08.480970+0000","last_unstale":"2024-09-11T08:56:34.848363+0000","last_undegraded":"2024-09-11T08:56:34.848363+0000","last_fullsized":"2024-09-11T08:56:34.848363+0000","mapping_epoch":345,"log_start":"337'12772","ondisk_log_start":"337'12772","created":293,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"342'14257","last_scrub_stamp":"2024-09-11T08:55:53.660499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:55:53.660499+0000","objects_scrubbed":2857,"log_size":4312,"log_dups_size":0,"ondisk_log_size":4312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:07.714964+0000","scrub_duration":1.1187108189999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17084,"num_write_kb":546688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"302'23","reported_seq":393,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.525974+0000","last_change":"2024-09-11T08:53:16.897477+0000","last_active":"2024-09-11T08:56:34.525974+0000","last_peered":"2024-09-11T08:56:34.525974+0000","last_clean":"2024-09-11T08:56:34.525974+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:56:34.525974+0000","last_undegraded":"2024-09-11T08:56:34.525974+0000","last_fullsized":"2024-09-11T08:56:34.525974+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'23","last_scrub_stamp":"2024-09-11T08:53:16.896643+0000","last_deep_scrub":"302'23","last_deep_scrub_stamp":"2024-09-11T08:53:16.896643+0000","last_clean_scrub_stamp":"2024-09-11T08:53:16.896643+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090878710000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":91,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.527461+0000","last_change":"2024-09-11T08:55:02.779440+0000","last_active":"2024-09-11T08:56:34.527461+0000","last_peered":"2024-09-11T08:56:34.527461+0000","last_clean":"2024-09-11T08:56:34.527461+0000","last_became_active":"2024-09-11T08:52:38.487881+0000","last_became_peered":"2024-09-11T08:52:38.487881+0000","last_unstale":"2024-09-11T08:56:34.527461+0000","last_undegraded":"2024-09-11T08:56:34.527461+0000","last_fullsized":"2024-09-11T08:56:34.527461+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:02.779144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:24.331966+0000","last_clean_scrub_stamp":"2024-09-11T08:55:02.779144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.026798161000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":366,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.126413+0000","last_change":"2024-09-11T08:55:10.138179+0000","last_active":"2024-09-11T08:56:31.126413+0000","last_peered":"2024-09-11T08:56:31.126413+0000","last_clean":"2024-09-11T08:56:31.126413+0000","last_became_active":"2024-09-11T08:54:17.122020+0000","last_became_peered":"2024-09-11T08:54:17.122020+0000","last_unstale":"2024-09-11T08:56:31.126413+0000","last_undegraded":"2024-09-11T08:56:31.126413+0000","last_fullsized":"2024-09-11T08:56:31.126413+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:10.137715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:55:10.137715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013124693999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":351,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847911+0000","last_change":"2024-09-11T08:53:58.389027+0000","last_active":"2024-09-11T08:56:34.847911+0000","last_peered":"2024-09-11T08:56:34.847911+0000","last_clean":"2024-09-11T08:56:34.847911+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:56:34.847911+0000","last_undegraded":"2024-09-11T08:56:34.847911+0000","last_fullsized":"2024-09-11T08:56:34.847911+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:58.388796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:53:58.388796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004542244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"353'19692","reported_seq":20025,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:33.300592+0000","last_change":"2024-09-11T08:55:48.796772+0000","last_active":"2024-09-11T08:56:33.300592+0000","last_peered":"2024-09-11T08:56:33.300592+0000","last_clean":"2024-09-11T08:56:33.300592+0000","last_became_active":"2024-09-11T08:52:56.412787+0000","last_became_peered":"2024-09-11T08:52:56.412787+0000","last_unstale":"2024-09-11T08:56:33.300592+0000","last_undegraded":"2024-09-11T08:56:33.300592+0000","last_fullsized":"2024-09-11T08:56:33.300592+0000","mapping_epoch":293,"log_start":"333'14553","ondisk_log_start":"333'14553","created":293,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"341'16507","last_scrub_stamp":"2024-09-11T08:55:48.796682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:54.586221+0000","last_clean_scrub_stamp":"2024-09-11T08:55:48.796682+0000","objects_scrubbed":3224,"log_size":5139,"log_dups_size":0,"ondisk_log_size":5139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:02.711082+0000","scrub_duration":1.1901719820000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16764,"num_write_kb":536448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"200'34","reported_seq":472,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.155796+0000","last_change":"2024-09-11T08:53:55.561925+0000","last_active":"2024-09-11T08:56:31.155796+0000","last_peered":"2024-09-11T08:56:31.155796+0000","last_clean":"2024-09-11T08:56:31.155796+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:56:31.155796+0000","last_undegraded":"2024-09-11T08:56:31.155796+0000","last_fullsized":"2024-09-11T08:56:31.155796+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:53:55.561835+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:53:55.561835+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037196429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":110,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.527527+0000","last_change":"2024-09-11T08:56:32.686448+0000","last_active":"2024-09-11T08:56:34.527527+0000","last_peered":"2024-09-11T08:56:34.527527+0000","last_clean":"2024-09-11T08:56:34.527527+0000","last_became_active":"2024-09-11T08:52:38.484199+0000","last_became_peered":"2024-09-11T08:52:38.484199+0000","last_unstale":"2024-09-11T08:56:34.527527+0000","last_undegraded":"2024-09-11T08:56:34.527527+0000","last_fullsized":"2024-09-11T08:56:34.527527+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:56:32.686360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:56:32.686360+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:02.159812+0000","scrub_duration":0.0082837799999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"346'12","reported_seq":49,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:37.875656+0000","last_change":"2024-09-11T08:56:08.480204+0000","last_active":"2024-09-11T08:56:37.875656+0000","last_peered":"2024-09-11T08:56:37.875656+0000","last_clean":"2024-09-11T08:56:37.875656+0000","last_became_active":"2024-09-11T08:56:08.479354+0000","last_became_peered":"2024-09-11T08:56:08.479354+0000","last_unstale":"2024-09-11T08:56:37.875656+0000","last_undegraded":"2024-09-11T08:56:37.875656+0000","last_fullsized":"2024-09-11T08:56:37.875656+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"287'11","last_scrub_stamp":"2024-09-11T08:54:13.613459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:13.613459+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":386,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.155900+0000","last_change":"2024-09-11T08:54:56.341981+0000","last_active":"2024-09-11T08:56:31.155900+0000","last_peered":"2024-09-11T08:56:31.155900+0000","last_clean":"2024-09-11T08:56:31.155900+0000","last_became_active":"2024-09-11T08:52:18.573947+0000","last_became_peered":"2024-09-11T08:52:18.573947+0000","last_unstale":"2024-09-11T08:56:31.155900+0000","last_undegraded":"2024-09-11T08:56:31.155900+0000","last_fullsized":"2024-09-11T08:56:31.155900+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:56.341896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:56.341896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.13742462999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":351,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.125724+0000","last_change":"2024-09-11T08:56:28.649422+0000","last_active":"2024-09-11T08:56:31.125724+0000","last_peered":"2024-09-11T08:56:31.125724+0000","last_clean":"2024-09-11T08:56:31.125724+0000","last_became_active":"2024-09-11T08:56:08.480043+0000","last_became_peered":"2024-09-11T08:56:08.480043+0000","last_unstale":"2024-09-11T08:56:31.125724+0000","last_undegraded":"2024-09-11T08:56:31.125724+0000","last_fullsized":"2024-09-11T08:56:31.125724+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:28.649338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:56:28.649338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:48.748129+0000","scrub_duration":0.0036325789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"200'12","reported_seq":401,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848609+0000","last_change":"2024-09-11T08:55:12.678723+0000","last_active":"2024-09-11T08:56:34.848609+0000","last_peered":"2024-09-11T08:56:34.848609+0000","last_clean":"2024-09-11T08:56:34.848609+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:56:34.848609+0000","last_undegraded":"2024-09-11T08:56:34.848609+0000","last_fullsized":"2024-09-11T08:56:34.848609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'12","last_scrub_stamp":"2024-09-11T08:55:12.678247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:12.678247+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020917618999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":372,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:36.282002+0000","last_change":"2024-09-11T08:56:36.281964+0000","last_active":"2024-09-11T08:56:36.282002+0000","last_peered":"2024-09-11T08:56:36.282002+0000","last_clean":"2024-09-11T08:56:36.282002+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:56:36.282002+0000","last_undegraded":"2024-09-11T08:56:36.282002+0000","last_fullsized":"2024-09-11T08:56:36.282002+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:36.281886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:56:36.281886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:41.829744+0000","scrub_duration":0.0018509690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":399,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.158383+0000","last_change":"2024-09-11T08:55:54.593062+0000","last_active":"2024-09-11T08:56:31.158383+0000","last_peered":"2024-09-11T08:56:31.158383+0000","last_clean":"2024-09-11T08:56:31.158383+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:56:31.158383+0000","last_undegraded":"2024-09-11T08:56:31.158383+0000","last_fullsized":"2024-09-11T08:56:31.158383+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:54.592967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:55:54.592967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:05.013292+0000","scrub_duration":0.0056632269999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"200'15","reported_seq":397,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848109+0000","last_change":"2024-09-11T08:54:09.548343+0000","last_active":"2024-09-11T08:56:34.848109+0000","last_peered":"2024-09-11T08:56:34.848109+0000","last_clean":"2024-09-11T08:56:34.848109+0000","last_became_active":"2024-09-11T08:52:49.569850+0000","last_became_peered":"2024-09-11T08:52:49.569850+0000","last_unstale":"2024-09-11T08:56:34.848109+0000","last_undegraded":"2024-09-11T08:56:34.848109+0000","last_fullsized":"2024-09-11T08:56:34.848109+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:54:09.548256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.548256+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072201399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":91,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.526173+0000","last_change":"2024-09-11T08:54:53.712104+0000","last_active":"2024-09-11T08:56:34.526173+0000","last_peered":"2024-09-11T08:56:34.526173+0000","last_clean":"2024-09-11T08:56:34.526173+0000","last_became_active":"2024-09-11T08:52:38.485708+0000","last_became_peered":"2024-09-11T08:52:38.485708+0000","last_unstale":"2024-09-11T08:56:34.526173+0000","last_undegraded":"2024-09-11T08:56:34.526173+0000","last_fullsized":"2024-09-11T08:56:34.526173+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:53.709804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:54:53.709804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042052879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":353,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848530+0000","last_change":"2024-09-11T08:54:06.496717+0000","last_active":"2024-09-11T08:56:34.848530+0000","last_peered":"2024-09-11T08:56:34.848530+0000","last_clean":"2024-09-11T08:56:34.848530+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:56:34.848530+0000","last_undegraded":"2024-09-11T08:56:34.848530+0000","last_fullsized":"2024-09-11T08:56:34.848530+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:06.496497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:06.496497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072082789999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":401,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.817292+0000","last_change":"2024-09-11T08:55:41.821876+0000","last_active":"2024-09-11T08:56:34.817292+0000","last_peered":"2024-09-11T08:56:34.817292+0000","last_clean":"2024-09-11T08:56:34.817292+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:56:34.817292+0000","last_undegraded":"2024-09-11T08:56:34.817292+0000","last_fullsized":"2024-09-11T08:56:34.817292+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:41.821791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_clean_scrub_stamp":"2024-09-11T08:55:41.821791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:57.865746+0000","scrub_duration":0.0064394760000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"200'15","reported_seq":415,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156406+0000","last_change":"2024-09-11T08:54:08.406799+0000","last_active":"2024-09-11T08:56:31.156406+0000","last_peered":"2024-09-11T08:56:31.156406+0000","last_clean":"2024-09-11T08:56:31.156406+0000","last_became_active":"2024-09-11T08:53:58.555987+0000","last_became_peered":"2024-09-11T08:53:58.555987+0000","last_unstale":"2024-09-11T08:56:31.156406+0000","last_undegraded":"2024-09-11T08:56:31.156406+0000","last_fullsized":"2024-09-11T08:56:31.156406+0000","mapping_epoch":315,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:54:08.406695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:08.406695+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001945411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":91,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848218+0000","last_change":"2024-09-11T08:55:03.641087+0000","last_active":"2024-09-11T08:56:34.848218+0000","last_peered":"2024-09-11T08:56:34.848218+0000","last_clean":"2024-09-11T08:56:34.848218+0000","last_became_active":"2024-09-11T08:52:49.566759+0000","last_became_peered":"2024-09-11T08:52:49.566759+0000","last_unstale":"2024-09-11T08:56:34.848218+0000","last_undegraded":"2024-09-11T08:56:34.848218+0000","last_fullsized":"2024-09-11T08:56:34.848218+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:03.641009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:03.641009+0000","last_clean_scrub_stamp":"2024-09-11T08:55:03.641009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057082440000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":577,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:37.875378+0000","last_change":"2024-09-11T08:54:02.455288+0000","last_active":"2024-09-11T08:56:37.875378+0000","last_peered":"2024-09-11T08:56:37.875378+0000","last_clean":"2024-09-11T08:56:37.875378+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:56:37.875378+0000","last_undegraded":"2024-09-11T08:56:37.875378+0000","last_fullsized":"2024-09-11T08:56:37.875378+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:54:02.454907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:02.454907+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0078828809999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":355,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.527212+0000","last_change":"2024-09-11T08:55:50.590487+0000","last_active":"2024-09-11T08:56:34.527212+0000","last_peered":"2024-09-11T08:56:34.527212+0000","last_clean":"2024-09-11T08:56:34.527212+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:56:34.527212+0000","last_undegraded":"2024-09-11T08:56:34.527212+0000","last_fullsized":"2024-09-11T08:56:34.527212+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:50.590410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:55:50.590410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:17.915626+0000","scrub_duration":0.0044493969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"200'25","reported_seq":422,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848400+0000","last_change":"2024-09-11T08:56:25.484599+0000","last_active":"2024-09-11T08:56:34.848400+0000","last_peered":"2024-09-11T08:56:34.848400+0000","last_clean":"2024-09-11T08:56:34.848400+0000","last_became_active":"2024-09-11T08:56:25.465124+0000","last_became_peered":"2024-09-11T08:56:25.465124+0000","last_unstale":"2024-09-11T08:56:34.848400+0000","last_undegraded":"2024-09-11T08:56:34.848400+0000","last_fullsized":"2024-09-11T08:56:34.848400+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:55:37.744946+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:55:37.744946+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:03.303781+0000","scrub_duration":0.0064657020000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":395,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.158277+0000","last_change":"2024-09-11T08:55:44.505112+0000","last_active":"2024-09-11T08:56:31.158277+0000","last_peered":"2024-09-11T08:56:31.158277+0000","last_clean":"2024-09-11T08:56:31.158277+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:56:31.158277+0000","last_undegraded":"2024-09-11T08:56:31.158277+0000","last_fullsized":"2024-09-11T08:56:31.158277+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:44.505035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:15.235519+0000","last_clean_scrub_stamp":"2024-09-11T08:55:44.505035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:52.657224+0000","scrub_duration":0.001906879,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"277'10","reported_seq":613,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:32.874395+0000","last_change":"2024-09-11T08:53:53.566688+0000","last_active":"2024-09-11T08:56:32.874395+0000","last_peered":"2024-09-11T08:56:32.874395+0000","last_clean":"2024-09-11T08:56:32.874395+0000","last_became_active":"2024-09-11T08:52:18.585914+0000","last_became_peered":"2024-09-11T08:52:18.585914+0000","last_unstale":"2024-09-11T08:56:32.874395+0000","last_undegraded":"2024-09-11T08:56:32.874395+0000","last_fullsized":"2024-09-11T08:56:32.874395+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'10","last_scrub_stamp":"2024-09-11T08:53:53.566589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:53:53.566589+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0096769479999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":101,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848170+0000","last_change":"2024-09-11T08:55:46.880868+0000","last_active":"2024-09-11T08:56:34.848170+0000","last_peered":"2024-09-11T08:56:34.848170+0000","last_clean":"2024-09-11T08:56:34.848170+0000","last_became_active":"2024-09-11T08:52:49.567693+0000","last_became_peered":"2024-09-11T08:52:49.567693+0000","last_unstale":"2024-09-11T08:56:34.848170+0000","last_undegraded":"2024-09-11T08:56:34.848170+0000","last_fullsized":"2024-09-11T08:56:34.848170+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:46.880773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:55:46.880773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:07.804115+0000","scrub_duration":0.0067893529999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"200'19","reported_seq":409,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.526929+0000","last_change":"2024-09-11T08:54:15.800180+0000","last_active":"2024-09-11T08:56:34.526929+0000","last_peered":"2024-09-11T08:56:34.526929+0000","last_clean":"2024-09-11T08:56:34.526929+0000","last_became_active":"2024-09-11T08:49:16.937185+0000","last_became_peered":"2024-09-11T08:49:16.937185+0000","last_unstale":"2024-09-11T08:56:34.526929+0000","last_undegraded":"2024-09-11T08:56:34.526929+0000","last_fullsized":"2024-09-11T08:56:34.526929+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:54:15.800092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:15.800092+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014286116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":358,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.527271+0000","last_change":"2024-09-11T08:53:21.891151+0000","last_active":"2024-09-11T08:56:34.527271+0000","last_peered":"2024-09-11T08:56:34.527271+0000","last_clean":"2024-09-11T08:56:34.527271+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:56:34.527271+0000","last_undegraded":"2024-09-11T08:56:34.527271+0000","last_fullsized":"2024-09-11T08:56:34.527271+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:21.891073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:53:21.891073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024938569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":382,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.131962+0000","last_change":"2024-09-11T08:55:34.422102+0000","last_active":"2024-09-11T08:56:31.131962+0000","last_peered":"2024-09-11T08:56:31.131962+0000","last_clean":"2024-09-11T08:56:31.131962+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:56:31.131962+0000","last_undegraded":"2024-09-11T08:56:31.131962+0000","last_fullsized":"2024-09-11T08:56:31.131962+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:34.422041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:55:34.422041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:52.413084+0000","scrub_duration":0.0071829429999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":371,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847226+0000","last_change":"2024-09-11T08:53:21.391938+0000","last_active":"2024-09-11T08:56:34.847226+0000","last_peered":"2024-09-11T08:56:34.847226+0000","last_clean":"2024-09-11T08:56:34.847226+0000","last_became_active":"2024-09-11T08:52:18.585089+0000","last_became_peered":"2024-09-11T08:52:18.585089+0000","last_unstale":"2024-09-11T08:56:34.847226+0000","last_undegraded":"2024-09-11T08:56:34.847226+0000","last_fullsized":"2024-09-11T08:56:34.847226+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:53:21.391849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:53:21.391849+0000","last_clean_scrub_stamp":"2024-09-11T08:53:21.391849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003344732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"200'15","reported_seq":266,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:35.316319+0000","last_change":"2024-09-11T08:56:33.318042+0000","last_active":"2024-09-11T08:56:35.316319+0000","last_peered":"2024-09-11T08:56:35.316319+0000","last_clean":"2024-09-11T08:56:35.316319+0000","last_became_active":"2024-09-11T08:56:25.462496+0000","last_became_peered":"2024-09-11T08:56:25.462496+0000","last_unstale":"2024-09-11T08:56:35.316319+0000","last_undegraded":"2024-09-11T08:56:35.316319+0000","last_fullsized":"2024-09-11T08:56:35.316319+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:56:33.317985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:56:33.317985+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:54.108040+0000","scrub_duration":0.001689821,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":101,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.848044+0000","last_change":"2024-09-11T08:56:31.849778+0000","last_active":"2024-09-11T08:56:34.848044+0000","last_peered":"2024-09-11T08:56:34.848044+0000","last_clean":"2024-09-11T08:56:34.848044+0000","last_became_active":"2024-09-11T08:52:38.485748+0000","last_became_peered":"2024-09-11T08:52:38.485748+0000","last_unstale":"2024-09-11T08:56:34.848044+0000","last_undegraded":"2024-09-11T08:56:34.848044+0000","last_fullsized":"2024-09-11T08:56:34.848044+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:31.849658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:21.208684+0000","last_clean_scrub_stamp":"2024-09-11T08:56:31.849658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:00.515178+0000","scrub_duration":0.0079402989999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":383,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847778+0000","last_change":"2024-09-11T08:55:42.795101+0000","last_active":"2024-09-11T08:56:34.847778+0000","last_peered":"2024-09-11T08:56:34.847778+0000","last_clean":"2024-09-11T08:56:34.847778+0000","last_became_active":"2024-09-11T08:52:18.585691+0000","last_became_peered":"2024-09-11T08:52:18.585691+0000","last_unstale":"2024-09-11T08:56:34.847778+0000","last_undegraded":"2024-09-11T08:56:34.847778+0000","last_fullsized":"2024-09-11T08:56:34.847778+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:42.795019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:55:42.795019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:08.981586+0000","scrub_duration":0.0020422750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":172,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.155707+0000","last_change":"2024-09-11T08:54:07.453666+0000","last_active":"2024-09-11T08:56:31.155707+0000","last_peered":"2024-09-11T08:56:31.155707+0000","last_clean":"2024-09-11T08:56:31.155707+0000","last_became_active":"2024-09-11T08:52:49.566694+0000","last_became_peered":"2024-09-11T08:52:49.566694+0000","last_unstale":"2024-09-11T08:56:31.155707+0000","last_undegraded":"2024-09-11T08:56:31.155707+0000","last_fullsized":"2024-09-11T08:56:31.155707+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:07.453531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:24.401480+0000","last_clean_scrub_stamp":"2024-09-11T08:54:07.453531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048569140000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"200'19","reported_seq":386,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847554+0000","last_change":"2024-09-11T08:54:53.565356+0000","last_active":"2024-09-11T08:56:34.847554+0000","last_peered":"2024-09-11T08:56:34.847554+0000","last_clean":"2024-09-11T08:56:34.847554+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:56:34.847554+0000","last_undegraded":"2024-09-11T08:56:34.847554+0000","last_fullsized":"2024-09-11T08:56:34.847554+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:54:53.565275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:53.565275+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061943880000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":386,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156634+0000","last_change":"2024-09-11T08:56:25.460703+0000","last_active":"2024-09-11T08:56:31.156634+0000","last_peered":"2024-09-11T08:56:31.156634+0000","last_clean":"2024-09-11T08:56:31.156634+0000","last_became_active":"2024-09-11T08:56:25.460327+0000","last_became_peered":"2024-09-11T08:56:25.460327+0000","last_unstale":"2024-09-11T08:56:31.156634+0000","last_undegraded":"2024-09-11T08:56:31.156634+0000","last_fullsized":"2024-09-11T08:56:31.156634+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:40.622753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:55:40.622753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:41.973719+0000","scrub_duration":0.011839215,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":394,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.157749+0000","last_change":"2024-09-11T08:55:33.562051+0000","last_active":"2024-09-11T08:56:31.157749+0000","last_peered":"2024-09-11T08:56:31.157749+0000","last_clean":"2024-09-11T08:56:31.157749+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:56:31.157749+0000","last_undegraded":"2024-09-11T08:56:31.157749+0000","last_fullsized":"2024-09-11T08:56:31.157749+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:33.561944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:55:33.561944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:45.536252+0000","scrub_duration":0.0057235180000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":391,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847819+0000","last_change":"2024-09-11T08:54:08.525090+0000","last_active":"2024-09-11T08:56:34.847819+0000","last_peered":"2024-09-11T08:56:34.847819+0000","last_clean":"2024-09-11T08:56:34.847819+0000","last_became_active":"2024-09-11T08:52:18.585315+0000","last_became_peered":"2024-09-11T08:52:18.585315+0000","last_unstale":"2024-09-11T08:56:34.847819+0000","last_undegraded":"2024-09-11T08:56:34.847819+0000","last_fullsized":"2024-09-11T08:56:34.847819+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:08.524974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:25.255332+0000","last_clean_scrub_stamp":"2024-09-11T08:54:08.524974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0080002040000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"200'10","reported_seq":414,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847579+0000","last_change":"2024-09-11T08:55:00.665641+0000","last_active":"2024-09-11T08:56:34.847579+0000","last_peered":"2024-09-11T08:56:34.847579+0000","last_clean":"2024-09-11T08:56:34.847579+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:56:34.847579+0000","last_undegraded":"2024-09-11T08:56:34.847579+0000","last_fullsized":"2024-09-11T08:56:34.847579+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'10","last_scrub_stamp":"2024-09-11T08:55:00.665557+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:55:00.665557+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031074079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":184,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156916+0000","last_change":"2024-09-11T08:54:02.427838+0000","last_active":"2024-09-11T08:56:31.156916+0000","last_peered":"2024-09-11T08:56:31.156916+0000","last_clean":"2024-09-11T08:56:31.156916+0000","last_became_active":"2024-09-11T08:52:49.569954+0000","last_became_peered":"2024-09-11T08:52:49.569954+0000","last_unstale":"2024-09-11T08:56:31.156916+0000","last_undegraded":"2024-09-11T08:56:31.156916+0000","last_fullsized":"2024-09-11T08:56:31.156916+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:54:02.427745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:54:02.427745+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045687649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":76,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.525645+0000","last_change":"2024-09-11T08:56:08.486420+0000","last_active":"2024-09-11T08:56:34.525645+0000","last_peered":"2024-09-11T08:56:34.525645+0000","last_clean":"2024-09-11T08:56:34.525645+0000","last_became_active":"2024-09-11T08:56:08.483212+0000","last_became_peered":"2024-09-11T08:56:08.483212+0000","last_unstale":"2024-09-11T08:56:34.525645+0000","last_undegraded":"2024-09-11T08:56:34.525645+0000","last_fullsized":"2024-09-11T08:56:34.525645+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:55.481627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:55.481627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":52,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.526595+0000","last_change":"2024-09-11T08:54:17.155339+0000","last_active":"2024-09-11T08:56:34.526595+0000","last_peered":"2024-09-11T08:56:34.526595+0000","last_clean":"2024-09-11T08:56:34.526595+0000","last_became_active":"2024-09-11T08:54:17.154933+0000","last_became_peered":"2024-09-11T08:54:17.154933+0000","last_unstale":"2024-09-11T08:56:34.526595+0000","last_undegraded":"2024-09-11T08:56:34.526595+0000","last_fullsized":"2024-09-11T08:56:34.526595+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:11.694784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:30.788303+0000","last_clean_scrub_stamp":"2024-09-11T08:54:11.694784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":390,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:35.508868+0000","last_change":"2024-09-11T08:56:35.508830+0000","last_active":"2024-09-11T08:56:35.508868+0000","last_peered":"2024-09-11T08:56:35.508868+0000","last_clean":"2024-09-11T08:56:35.508868+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:56:35.508868+0000","last_undegraded":"2024-09-11T08:56:35.508868+0000","last_fullsized":"2024-09-11T08:56:35.508868+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:35.508746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:56:35.508746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:37.718672+0000","scrub_duration":0.0016595760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":397,"reported_epoch":354,"state":"active+clean","last_fresh":"2024-09-11T08:56:34.847686+0000","last_change":"2024-09-11T08:56:25.458816+0000","last_active":"2024-09-11T08:56:34.847686+0000","last_peered":"2024-09-11T08:56:34.847686+0000","last_clean":"2024-09-11T08:56:34.847686+0000","last_became_active":"2024-09-11T08:56:25.458500+0000","last_became_peered":"2024-09-11T08:56:25.458500+0000","last_unstale":"2024-09-11T08:56:34.847686+0000","last_undegraded":"2024-09-11T08:56:34.847686+0000","last_fullsized":"2024-09-11T08:56:34.847686+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:13.701753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:32.393197+0000","last_clean_scrub_stamp":"2024-09-11T08:54:13.701753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.113734606,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"200'19","reported_seq":95,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.156821+0000","last_change":"2024-09-11T08:56:08.482039+0000","last_active":"2024-09-11T08:56:31.156821+0000","last_peered":"2024-09-11T08:56:31.156821+0000","last_clean":"2024-09-11T08:56:31.156821+0000","last_became_active":"2024-09-11T08:56:08.481037+0000","last_became_peered":"2024-09-11T08:56:08.481037+0000","last_unstale":"2024-09-11T08:56:31.156821+0000","last_undegraded":"2024-09-11T08:56:31.156821+0000","last_fullsized":"2024-09-11T08:56:31.156821+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:55:48.566907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:48.566907+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:00.703462+0000","scrub_duration":0.0096371630000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":181,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-11T08:56:31.158675+0000","last_change":"2024-09-11T08:54:54.799438+0000","last_active":"2024-09-11T08:56:31.158675+0000","last_peered":"2024-09-11T08:56:31.158675+0000","last_clean":"2024-09-11T08:56:31.158675+0000","last_became_active":"2024-09-11T08:52:49.568537+0000","last_became_peered":"2024-09-11T08:52:49.568537+0000","last_unstale":"2024-09-11T08:56:31.158675+0000","last_undegraded":"2024-09-11T08:56:31.158675+0000","last_fullsized":"2024-09-11T08:56:31.158675+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:54.799254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:54.799254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013107982000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":273914,"num_write_kb":8765185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":25362432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77761,"ondisk_log_size":77761,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1112,"num_read_kb":869,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4582,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":770,"ondisk_log_size":770,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":697708,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":42,"num_write":20,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299613,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":890012,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":888894,"kb_avail":92855140,"statfs":{"total":95995035648,"available":95083663360,"internally_reserved":0,"allocated":1130496,"data_stored":809894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":910227800},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:55:58 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":0.89800000000000002,"15min":1.0289999999999999},"min":{"1min":0.23899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.012,"5min":5.0449999999999999,"15min":32.171999999999997},"last":1.268},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.79500000000000004,"15min":1.097},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.21199999999999999},"max":{"1min":3.2839999999999998,"5min":4.0069999999999997,"15min":32.149999999999999},"last":0.48199999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:55:58 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":0.873,"15min":1.177},"min":{"1min":0.251,"5min":0.23799999999999999,"15min":0.224},"max":{"1min":4.3449999999999998,"5min":4.3449999999999998,"15min":33.692999999999998},"last":1.2410000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.70299999999999996,"15min":1.048},"min":{"1min":0.26000000000000001,"5min":0.249,"15min":0.23300000000000001},"max":{"1min":2.2530000000000001,"5min":3.9780000000000002,"15min":35.591000000000001},"last":0.46100000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:55:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.097,"15min":1.0409999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.249},"max":{"1min":4.4889999999999999,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.31900000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.90000000000000002,"15min":0.97899999999999998},"min":{"1min":0.307,"5min":0.27900000000000003,"15min":0.25800000000000001},"max":{"1min":3.0710000000000002,"5min":4.282,"15min":7.5259999999999998},"last":1.4670000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:55:58 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.179,"15min":1.3109999999999999},"min":{"1min":0.28899999999999998,"5min":0.28699999999999998,"15min":0.25},"max":{"1min":8.7739999999999991,"5min":8.7739999999999991,"15min":56.265999999999998},"last":0.38},{"interface":"front","average":{"1min":1.236,"5min":1.054,"15min":1.363},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.27400000000000002},"max":{"1min":5.0140000000000002,"5min":7.0780000000000003,"15min":57.207000000000001},"last":1.3300000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:55:58 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":2.8490000000000002,"15min":1.391},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":3.5870000000000002,"5min":163.60499999999999,"15min":163.60499999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.0529999999999999,"5min":3.0230000000000001,"15min":1.48},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":3.452,"5min":163.88300000000001,"15min":163.88300000000001},"last":0.63}]},{"osd":5,"last update":"Wed Sep 11 08:55:58 2024","interfaces":[{"interface":"back","average":{"1min":3.46,"5min":1.5449999999999999,"15min":0.99099999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.23300000000000001},"max":{"1min":50.128,"5min":50.128,"15min":50.128},"last":1.2130000000000001},{"interface":"front","average":{"1min":3.1659999999999999,"5min":1.5309999999999999,"15min":1.0449999999999999},"min":{"1min":0.33000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":50.241,"5min":50.241,"15min":50.241},"last":0.441}]},{"osd":6,"last update":"Wed Sep 11 08:55:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.141,"15min":0.879},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":3.0259999999999998,"5min":4.5449999999999999,"15min":4.5449999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":1.1060000000000001,"15min":0.88200000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":2.4729999999999999,"5min":5.0620000000000003,"15min":5.0620000000000003},"last":0.51400000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528547,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":893860,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":893054,"kb_avail":92851292,"statfs":{"total":95995035648,"available":95079723008,"internally_reserved":0,"allocated":811008,"data_stored":490422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":914487640},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:56:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":2.4790000000000001,"15min":1.7989999999999999},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":5.577,"5min":121.208,"15min":121.208},"last":0.45000000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":2.8330000000000002,"15min":1.9099999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.23300000000000001},"max":{"1min":3.4220000000000002,"5min":121.05200000000001,"15min":121.05200000000001},"last":0.39400000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:56:39 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1479999999999999,"15min":1.3360000000000001},"min":{"1min":0.36799999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":3.004,"5min":5.8689999999999998,"15min":27.984000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.044,"5min":0.96999999999999997,"15min":1.238},"min":{"1min":0.32600000000000001,"5min":0.254,"15min":0.253},"max":{"1min":2.8439999999999999,"5min":3.411,"15min":28.106000000000002},"last":0.50900000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:56:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.9850000000000001,"15min":1.2949999999999999},"min":{"1min":0.38,"5min":0.23999999999999999,"15min":0.23200000000000001},"max":{"1min":2.8839999999999999,"5min":67.486999999999995,"15min":67.486999999999995},"last":0.38},{"interface":"front","average":{"1min":1.0680000000000001,"5min":2.472,"15min":1.4910000000000001},"min":{"1min":0.28599999999999998,"5min":0.27400000000000002,"15min":0.26600000000000001},"max":{"1min":3.4550000000000001,"5min":67.272000000000006,"15min":67.272000000000006},"last":0.47799999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:56:39 2024","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":1.329,"15min":1.419},"min":{"1min":0.38200000000000001,"5min":0.27200000000000002,"15min":0.26700000000000002},"max":{"1min":5.8070000000000004,"5min":5.8470000000000004,"15min":27.795000000000002},"last":0.435},{"interface":"front","average":{"1min":1.726,"5min":1.135,"15min":1.3540000000000001},"min":{"1min":0.39100000000000001,"5min":0.28199999999999997,"15min":0.224},"max":{"1min":6.8879999999999999,"5min":6.8879999999999999,"15min":27.841999999999999},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed Sep 11 08:56:39 2024","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.8100000000000001,"15min":1.024},"min":{"1min":0.27600000000000002,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":5.4829999999999997,"5min":40.927999999999997,"15min":40.927999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.6719999999999999,"15min":0.96599999999999997},"min":{"1min":0.28599999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.8959999999999999,"5min":40.706000000000003,"15min":40.706000000000003},"last":0.49399999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:56:39 2024","interfaces":[{"interface":"back","average":{"1min":1.823,"5min":1.9419999999999999,"15min":1.177},"min":{"1min":0.315,"5min":0.29899999999999999,"15min":0.23899999999999999},"max":{"1min":19.309999999999999,"5min":40.847000000000001,"15min":40.847000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":2.0249999999999999,"5min":1.8049999999999999,"15min":1.075},"min":{"1min":0.34200000000000003,"5min":0.29899999999999999,"15min":0.27100000000000002},"max":{"1min":19.411000000000001,"5min":41.401000000000003,"15min":41.401000000000003},"last":0.51900000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:55:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.407,"15min":1.056},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.25900000000000001},"max":{"1min":2.9510000000000001,"5min":40.426000000000002,"15min":40.426000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.423,"15min":1.024},"min":{"1min":0.309,"5min":0.309,"15min":0.27700000000000002},"max":{"1min":2.9319999999999999,"5min":45.401000000000003,"15min":45.401000000000003},"last":0.53300000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757478,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":903636,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":902270,"kb_avail":92841516,"statfs":{"total":95995035648,"available":95069712384,"internally_reserved":0,"allocated":1388544,"data_stored":1072995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":923924824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:56:19 2024","interfaces":[{"interface":"back","average":{"1min":2.6339999999999999,"5min":1.3839999999999999,"15min":1.1240000000000001},"min":{"1min":0.30499999999999999,"5min":0.221,"15min":0.20599999999999999},"max":{"1min":10.372,"5min":10.372,"15min":10.372},"last":0.52200000000000002},{"interface":"front","average":{"1min":1.23,"5min":0.98999999999999999,"15min":1.127},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":5.444,"5min":5.444,"15min":8.0410000000000004},"last":0.67600000000000005}]},{"osd":1,"last update":"Wed Sep 11 08:56:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.048,"15min":1.123},"min":{"1min":0.248,"5min":0.216,"15min":0.216},"max":{"1min":5.3799999999999999,"5min":5.3799999999999999,"15min":8},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.0569999999999999,"15min":1.1020000000000001},"min":{"1min":0.28599999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.9159999999999999,"5min":12.173,"15min":12.173},"last":0.56200000000000006}]},{"osd":2,"last update":"Wed Sep 11 08:56:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1419999999999999,"15min":1.121},"min":{"1min":0.30099999999999999,"5min":0.253,"15min":0.23799999999999999},"max":{"1min":7.0199999999999996,"5min":8.7789999999999999,"15min":8.7789999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.738,"5min":1.1839999999999999,"15min":1.1040000000000001},"min":{"1min":0.27200000000000002,"5min":0.26500000000000001,"15min":0.25900000000000001},"max":{"1min":8.0619999999999994,"5min":8.0619999999999994,"15min":8.0640000000000001},"last":0.623}]},{"osd":3,"last update":"Wed Sep 11 08:56:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":2.052,"15min":1.4199999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.26900000000000002},"max":{"1min":6.9589999999999996,"5min":85.299999999999997,"15min":85.299999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.28,"5min":1.855,"15min":1.5069999999999999},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.27400000000000002},"max":{"1min":5.4189999999999996,"5min":85.941000000000003,"15min":85.941000000000003},"last":0.747}]},{"osd":4,"last update":"Wed Sep 11 08:56:19 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.113,"15min":0.85499999999999998},"min":{"1min":0.33100000000000002,"5min":0.23300000000000001,"15min":0.20699999999999999},"max":{"1min":3.8570000000000002,"5min":16.021999999999998,"15min":16.021999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.048,"15min":0.80600000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.20499999999999999},"max":{"1min":6.9909999999999997,"5min":9.6140000000000008,"15min":9.6140000000000008},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:55:40 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.095,"15min":0.83899999999999997},"min":{"1min":0.374,"5min":0.33800000000000002,"15min":0.23599999999999999},"max":{"1min":2.2959999999999998,"5min":8.8059999999999992,"15min":8.8059999999999992},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.181,"15min":0.85299999999999998},"min":{"1min":0.35999999999999999,"5min":0.27400000000000002,"15min":0.251},"max":{"1min":6.0599999999999996,"5min":6.0599999999999996,"15min":8.1920000000000002},"last":0.76800000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:55:53 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.1879999999999999,"15min":1.1759999999999999},"min":{"1min":0.35599999999999998,"5min":0.30099999999999999,"15min":0.27800000000000002},"max":{"1min":6.2969999999999997,"5min":16.062999999999999,"15min":88.513000000000005},"last":0.498},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.097,"15min":1.161},"min":{"1min":0.32700000000000001,"5min":0.315,"15min":0.29599999999999999},"max":{"1min":6.335,"5min":8.8300000000000001,"15min":86.963999999999999},"last":0.70999999999999996}]}]},{"osd":4,"up_from":31,"seq":133143986411,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":586528,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":585726,"kb_avail":93158624,"statfs":{"total":95995035648,"available":95394430976,"internally_reserved":0,"allocated":806912,"data_stored":489690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":599783768},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:55:58 2024","interfaces":[{"interface":"back","average":{"1min":1.9330000000000001,"5min":1.0980000000000001,"15min":1.768},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20399999999999999},"max":{"1min":5.069,"5min":5.2400000000000002,"15min":89.510000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.1699999999999999,"15min":1.8879999999999999},"min":{"1min":0.45600000000000002,"5min":0.25900000000000001,"15min":0.22600000000000001},"max":{"1min":3.8170000000000002,"5min":5.3220000000000001,"15min":89.765000000000001},"last":0.42699999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:55:58 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.169,"15min":1.5469999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.6059999999999999,"5min":18.065999999999999,"15min":48.976999999999997},"last":0.496},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.1899999999999999,"15min":1.4990000000000001},"min":{"1min":0.41699999999999998,"5min":0.23499999999999999,"15min":0.22800000000000001},"max":{"1min":5.0099999999999998,"5min":18.713999999999999,"15min":45.787999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:55:58 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":3.9649999999999999,"15min":1.9690000000000001},"min":{"1min":0.32800000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":5.5030000000000001,"5min":255.88900000000001,"15min":255.88900000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":4.0369999999999999,"15min":1.9930000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.25800000000000001},"max":{"1min":5.8789999999999996,"5min":255.41200000000001,"15min":255.41200000000001},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:55:58 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3440000000000001,"15min":2.1219999999999999},"min":{"1min":0.316,"5min":0.27800000000000002,"15min":0.252},"max":{"1min":3.895,"5min":16.663,"15min":89.881},"last":0.46200000000000002},{"interface":"front","average":{"1min":1.52,"5min":1.177,"15min":2.024},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.25900000000000001},"max":{"1min":3.6349999999999998,"5min":16.795999999999999,"15min":89.715000000000003},"last":0.47099999999999997}]},{"osd":5,"last update":"Wed Sep 11 08:56:15 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":1.022,"15min":0.81100000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.23999999999999999},"max":{"1min":3.7090000000000001,"5min":11.045999999999999,"15min":11.045999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.1399999999999999,"15min":0.80700000000000005},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.24199999999999999},"max":{"1min":3.4929999999999999,"5min":10.526999999999999,"15min":10.526999999999999},"last":0.374}]},{"osd":6,"last update":"Wed Sep 11 08:55:46 2024","interfaces":[{"interface":"back","average":{"1min":6.0519999999999996,"5min":2.0289999999999999,"15min":1.075},"min":{"1min":0.40300000000000002,"5min":0.26800000000000002,"15min":0.22900000000000001},"max":{"1min":72.100999999999999,"5min":72.100999999999999,"15min":72.100999999999999},"last":0.316},{"interface":"front","average":{"1min":6.9160000000000004,"5min":2.1819999999999999,"15min":1.151},"min":{"1min":0.32700000000000001,"5min":0.27800000000000002,"15min":0.221},"max":{"1min":74.527000000000001,"5min":74.527000000000001,"15min":74.527000000000001},"last":0.436}]},{"osd":7,"last update":"Wed Sep 11 08:55:58 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":0.99399999999999999,"15min":0.747},"min":{"1min":0.28000000000000003,"5min":0.253,"15min":0.252},"max":{"1min":4.8280000000000003,"5min":4.8280000000000003,"15min":4.8280000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":1.028,"5min":1.1579999999999999,"15min":0.85899999999999999},"min":{"1min":0.313,"5min":0.29899999999999999,"15min":0.29599999999999999},"max":{"1min":3.0659999999999998,"5min":5.2539999999999996,"15min":5.2539999999999996},"last":0.38700000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149937,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":900596,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":899198,"kb_avail":92844556,"statfs":{"total":95995035648,"available":95072825344,"internally_reserved":0,"allocated":1421312,"data_stored":1073904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":920779096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:56:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1539999999999999,"15min":1.325},"min":{"1min":0.214,"5min":0.20899999999999999,"15min":0.16500000000000001},"max":{"1min":3.9399999999999999,"5min":10.393000000000001,"15min":24.280000000000001},"last":0.38},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.1919999999999999,"15min":1.5449999999999999},"min":{"1min":0.219,"5min":0.19900000000000001,"15min":0.17000000000000001},"max":{"1min":5.2140000000000004,"5min":7.5599999999999996,"15min":22.225000000000001},"last":0.30399999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:56:37 2024","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.9530000000000001,"15min":2.1880000000000002},"min":{"1min":0.24399999999999999,"5min":0.20100000000000001,"15min":0.19800000000000001},"max":{"1min":4.8170000000000002,"5min":84.284999999999997,"15min":115.601},"last":0.34999999999999998},{"interface":"front","average":{"1min":1.6279999999999999,"5min":2.0390000000000001,"15min":2.0920000000000001},"min":{"1min":0.22600000000000001,"5min":0.20499999999999999,"15min":0.17999999999999999},"max":{"1min":6.7080000000000002,"5min":85.832999999999998,"15min":115.474},"last":0.35999999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:56:37 2024","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":2.2149999999999999,"15min":1.7110000000000001},"min":{"1min":0.30099999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":6.117,"5min":86.766000000000005,"15min":86.766000000000005},"last":0.32200000000000001},{"interface":"front","average":{"1min":1.05,"5min":2.0960000000000001,"15min":1.6459999999999999},"min":{"1min":0.32200000000000001,"5min":0.23400000000000001,"15min":0.20799999999999999},"max":{"1min":3.8079999999999998,"5min":85.367999999999995,"15min":85.367999999999995},"last":0.42099999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:55:48 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.3149999999999999,"15min":1.343},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.26700000000000002},"max":{"1min":3.5819999999999999,"5min":5.9039999999999999,"15min":8.9969999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":1.288,"5min":1.137,"15min":1.1299999999999999},"min":{"1min":0.47599999999999998,"5min":0.33300000000000002,"15min":0.27700000000000002},"max":{"1min":3.597,"5min":5.617,"15min":7.234},"last":0.50600000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:56:06 2024","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.2829999999999999,"15min":1.3480000000000001},"min":{"1min":0.39900000000000002,"5min":0.30599999999999999,"15min":0.28299999999999997},"max":{"1min":5.8780000000000001,"5min":5.9470000000000001,"15min":11.526999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.1699999999999999,"15min":1.2090000000000001},"min":{"1min":0.60899999999999999,"5min":0.379,"15min":0.29399999999999998},"max":{"1min":1.734,"5min":5.5190000000000001,"15min":7.1689999999999996},"last":0.53500000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:56:37 2024","interfaces":[{"interface":"back","average":{"1min":1.917,"5min":1.6060000000000001,"15min":1.635},"min":{"1min":0.39900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":6.1959999999999997,"5min":11.007999999999999,"15min":33.106999999999999},"last":0.497},{"interface":"front","average":{"1min":1.6699999999999999,"5min":1.262,"15min":1.51},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":5.819,"5min":7.4809999999999999,"15min":33.076000000000001},"last":0.56499999999999995}]},{"osd":7,"last update":"Wed Sep 11 08:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.4039999999999999,"15min":1.6060000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.7389999999999999,"5min":6.0190000000000001,"15min":35.073},"last":0.46999999999999997},{"interface":"front","average":{"1min":1.101,"5min":1.288,"15min":1.4299999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.34699999999999998},"max":{"1min":2.7410000000000001,"5min":5.6529999999999996,"15min":34.844999999999999},"last":0.57999999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411573,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":590936,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":589886,"kb_avail":93154216,"statfs":{"total":95995035648,"available":95389917184,"internally_reserved":0,"allocated":1060864,"data_stored":752309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":604043608},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:56:12 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.89600000000000002,"15min":1.7070000000000001},"min":{"1min":0.31,"5min":0.189,"15min":0.159},"max":{"1min":3.9089999999999998,"5min":7.3380000000000001,"15min":34.713999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.117,"15min":1.859},"min":{"1min":0.39600000000000002,"5min":0.191,"15min":0.17399999999999999},"max":{"1min":4.641,"5min":9.0630000000000006,"15min":34.399000000000001},"last":0.34200000000000003}]},{"osd":1,"last update":"Wed Sep 11 08:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.872,"15min":1.7809999999999999},"min":{"1min":0.23599999999999999,"5min":0.189,"15min":0.189},"max":{"1min":2.1520000000000001,"5min":5.9850000000000003,"15min":72.867000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":1.1339999999999999,"5min":0.96599999999999997,"15min":1.9139999999999999},"min":{"1min":0.26000000000000001,"5min":0.22500000000000001,"15min":0.20399999999999999},"max":{"1min":4.4109999999999996,"5min":5.7370000000000001,"15min":67.799000000000007},"last":0.39700000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:56:23 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.6739999999999999,"15min":1.9410000000000001},"min":{"1min":0.31,"5min":0.186,"15min":0.186},"max":{"1min":2.1789999999999998,"5min":64.795000000000002,"15min":64.795000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.6879999999999999,"15min":1.877},"min":{"1min":0.29599999999999999,"5min":0.216,"15min":0.216},"max":{"1min":4.4909999999999997,"5min":64.632999999999996,"15min":64.632999999999996},"last":0.376}]},{"osd":4,"last update":"Wed Sep 11 08:55:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.6919999999999999,"15min":1.351},"min":{"1min":0.29399999999999998,"5min":0.28699999999999998,"15min":0.255},"max":{"1min":5.1959999999999997,"5min":52.113999999999997,"15min":52.113999999999997},"last":0.751},{"interface":"front","average":{"1min":1.355,"5min":1.708,"15min":1.3140000000000001},"min":{"1min":0.307,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4690000000000003,"5min":52.097999999999999,"15min":52.097999999999999},"last":0.70399999999999996}]},{"osd":5,"last update":"Wed Sep 11 08:56:12 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3260000000000001,"15min":1.361},"min":{"1min":0.33100000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.4429999999999996,"5min":7.4189999999999996,"15min":52.145000000000003},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.113,"15min":1.1759999999999999},"min":{"1min":0.33400000000000002,"5min":0.29799999999999999,"15min":0.28399999999999997},"max":{"1min":3.9849999999999999,"5min":4.8109999999999999,"15min":52.216000000000001},"last":0.66100000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:56:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.347,"15min":1.4670000000000001},"min":{"1min":0.35699999999999998,"5min":0.30299999999999999,"15min":0.28699999999999998},"max":{"1min":2.2549999999999999,"5min":7.298,"15min":52.235999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.4159999999999999,"15min":1.4019999999999999},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":4.516,"5min":5.6440000000000001,"15min":52.186},"last":0.67300000000000004}]},{"osd":7,"last update":"Wed Sep 11 08:55:46 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":1.1639999999999999,"15min":1.306},"min":{"1min":0.34599999999999997,"5min":0.34399999999999997,"15min":0.30399999999999999},"max":{"1min":2.6000000000000001,"5min":4.8570000000000002,"15min":84.108999999999995},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3680000000000001,"15min":1.5640000000000001},"min":{"1min":0.41399999999999998,"5min":0.36799999999999999,"15min":0.30299999999999999},"max":{"1min":3.8719999999999999,"5min":7.0720000000000001,"15min":85.938000000000002},"last":0.72299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575097,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":901316,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":899518,"kb_avail":92843836,"statfs":{"total":95995035648,"available":95072088064,"internally_reserved":0,"allocated":1826816,"data_stored":1529585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":921106776},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:55:49 2024","interfaces":[{"interface":"back","average":{"1min":2.5609999999999999,"5min":1.911,"15min":1.9370000000000001},"min":{"1min":0.27500000000000002,"5min":0.17499999999999999,"15min":0.16600000000000001},"max":{"1min":18.27,"5min":56.613,"15min":117.01600000000001},"last":0.215},{"interface":"front","average":{"1min":2.3839999999999999,"5min":1.9870000000000001,"15min":2.25},"min":{"1min":0.187,"5min":0.187,"15min":0.17499999999999999},"max":{"1min":18.210999999999999,"5min":55.533999999999999,"15min":117.62},"last":0.41399999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:56:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":3.2530000000000001,"15min":2.0840000000000001},"min":{"1min":0.20100000000000001,"5min":0.19,"15min":0.19},"max":{"1min":3.581,"5min":169.607,"15min":169.607},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":3.0209999999999999,"15min":2.024},"min":{"1min":0.26300000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":3.9870000000000001,"5min":169.762,"15min":169.762},"last":0.24099999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:56:33 2024","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.0760000000000001,"15min":1.714},"min":{"1min":0.26000000000000001,"5min":0.224,"15min":0.191},"max":{"1min":8.1270000000000007,"5min":8.1270000000000007,"15min":116.98},"last":0.32300000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":0.89700000000000002,"15min":1.7110000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.22},"max":{"1min":5.8890000000000002,"5min":5.9029999999999996,"15min":117.56699999999999},"last":0.34799999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.177,"15min":1.153},"min":{"1min":0.28999999999999998,"5min":0.28699999999999998,"15min":0.26200000000000001},"max":{"1min":3.3759999999999999,"5min":7.9450000000000003,"15min":11.19},"last":0.38500000000000001},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.1930000000000001,"15min":1.091},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":3.5670000000000002,"5min":6.952,"15min":7.7169999999999996},"last":0.44900000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:56:13 2024","interfaces":[{"interface":"back","average":{"1min":2.7599999999999998,"5min":1.508,"15min":1.784},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.27600000000000002},"max":{"1min":38.098999999999997,"5min":38.098999999999997,"15min":103.337},"last":0.505},{"interface":"front","average":{"1min":2.6619999999999999,"5min":1.4730000000000001,"15min":1.7849999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.27400000000000002},"max":{"1min":38.130000000000003,"5min":38.130000000000003,"15min":103.163},"last":0.47099999999999997}]},{"osd":6,"last update":"Wed Sep 11 08:55:39 2024","interfaces":[{"interface":"back","average":{"1min":1.8480000000000001,"5min":1.3700000000000001,"15min":1.159},"min":{"1min":0.376,"5min":0.30499999999999999,"15min":0.29399999999999998},"max":{"1min":4.8460000000000001,"5min":6.2249999999999996,"15min":8.9149999999999991},"last":0.39300000000000002},{"interface":"front","average":{"1min":1.238,"5min":1.204,"15min":1.2529999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.31},"max":{"1min":4.931,"5min":6.2869999999999999,"15min":9.5199999999999996},"last":0.47999999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":1.0780000000000001,"15min":1.165},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.1469999999999998,"5min":8.3390000000000004,"15min":11.315},"last":0.40600000000000003},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.1160000000000001,"15min":1.139},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":2.7770000000000001,"5min":5.9089999999999998,"15min":10.266},"last":0.42899999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738621,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":897564,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":896766,"kb_avail":92847588,"statfs":{"total":95995035648,"available":95075930112,"internally_reserved":0,"allocated":802816,"data_stored":482163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":918288728},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:55:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1419999999999999,"15min":1.6240000000000001},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.16300000000000001},"max":{"1min":8.5649999999999995,"5min":8.5649999999999995,"15min":56.685000000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":1.165,"15min":1.5640000000000001},"min":{"1min":0.192,"5min":0.188,"15min":0.186},"max":{"1min":2.5539999999999998,"5min":7.694,"15min":61.012999999999998},"last":0.47099999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:56:15 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.1990000000000001,"15min":1.3779999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.21099999999999999},"max":{"1min":7.1500000000000004,"5min":8.1059999999999999,"15min":19.940000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":1.298,"5min":1.1539999999999999,"15min":1.4350000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20000000000000001},"max":{"1min":4.8650000000000002,"5min":5.4279999999999999,"15min":20.757000000000001},"last":0.36699999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:56:36 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.25,"15min":1.7629999999999999},"min":{"1min":0.22900000000000001,"5min":0.218,"15min":0.19800000000000001},"max":{"1min":4.9059999999999997,"5min":7.7709999999999999,"15min":56.095999999999997},"last":0.44},{"interface":"front","average":{"1min":1.8109999999999999,"5min":1.264,"15min":1.7509999999999999},"min":{"1min":0.23999999999999999,"5min":0.21199999999999999,"15min":0.20899999999999999},"max":{"1min":8.0329999999999995,"5min":8.5419999999999998,"15min":59.651000000000003},"last":0.79900000000000004}]},{"osd":4,"last update":"Wed Sep 11 08:55:50 2024","interfaces":[{"interface":"back","average":{"1min":4.0540000000000003,"5min":3.282,"15min":1.956},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25},"max":{"1min":35.066000000000003,"5min":168.72200000000001,"15min":168.72200000000001},"last":0.871},{"interface":"front","average":{"1min":3.8300000000000001,"5min":3.3330000000000002,"15min":1.925},"min":{"1min":0.41099999999999998,"5min":0.32500000000000001,"15min":0.255},"max":{"1min":34.652000000000001,"5min":168.88999999999999,"15min":168.88999999999999},"last":0.57699999999999996}]},{"osd":5,"last update":"Wed Sep 11 08:56:15 2024","interfaces":[{"interface":"back","average":{"1min":1.79,"5min":1.4570000000000001,"15min":1.3879999999999999},"min":{"1min":0.307,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":4.9370000000000003,"5min":7.7830000000000004,"15min":7.7830000000000004},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.734,"5min":1.427,"15min":1.421},"min":{"1min":0.35999999999999999,"5min":0.32800000000000001,"15min":0.28299999999999997},"max":{"1min":4.9109999999999996,"5min":9.5239999999999991,"15min":9.5239999999999991},"last":0.98599999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:56:36 2024","interfaces":[{"interface":"back","average":{"1min":2.762,"5min":1.784,"15min":1.536},"min":{"1min":0.38900000000000001,"5min":0.33300000000000002,"15min":0.30099999999999999},"max":{"1min":25.027000000000001,"5min":25.027000000000001,"15min":25.027000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":3.383,"5min":1.893,"15min":1.6180000000000001},"min":{"1min":0.499,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":25.076000000000001,"5min":25.076000000000001,"15min":25.076000000000001},"last":1.0069999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:56:05 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.385,"15min":1.595},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.313},"max":{"1min":4.3289999999999997,"5min":5.0739999999999998,"15min":56.119},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.6459999999999999,"15min":1.726},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.307},"max":{"1min":4.9610000000000003,"5min":9.9459999999999997,"15min":56.100000000000001},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:56:43.512 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-11T08:56:43.512 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool set unique_pool_3 pg_num 26 2024-09-11T08:56:43.513 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:43.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:43.515 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:43.616 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:43.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:43.638 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:43.638 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:43.739 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:43.760 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:43.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:43.761 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:43.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:43 smithi092 ceph-mon[29647]: from='client.25670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:56:43.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:43 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:56:43.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:43 smithi092 ceph-mon[29647]: 5.8 scrub ok 2024-09-11T08:56:43.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:43 smithi092 ceph-mon[41114]: from='client.25670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:56:43.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:43 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:56:43.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:43 smithi092 ceph-mon[41114]: 5.8 scrub ok 2024-09-11T08:56:43.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:43.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:43.885 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:43.886 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:43.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:43 smithi145 ceph-mon[33384]: from='client.25670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:56:43.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:43 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:56:43.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:43 smithi145 ceph-mon[33384]: 5.8 scrub ok 2024-09-11T08:56:43.986 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:44.008 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:44.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:44.010 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:44.110 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:44.132 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:44.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:44.133 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:44.234 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:44.256 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:44.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:44.257 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:44.358 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:44.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:44.382 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:44.382 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:44.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:44.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:44.509 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:44.509 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:44.610 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:44.632 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:44.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:44.634 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:44.734 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:44.759 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:44.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:44.760 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:44.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:44 smithi092 ceph-mon[29647]: pgmap v814: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 286 op/s 2024-09-11T08:56:44.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:44 smithi092 ceph-mon[29647]: 6.1 scrub starts 2024-09-11T08:56:44.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:44 smithi092 ceph-mon[29647]: 6.1 scrub ok 2024-09-11T08:56:44.802 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:44 smithi092 ceph-mon[41114]: pgmap v814: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 286 op/s 2024-09-11T08:56:44.802 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:44 smithi092 ceph-mon[41114]: 6.1 scrub starts 2024-09-11T08:56:44.802 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:44 smithi092 ceph-mon[41114]: 6.1 scrub ok 2024-09-11T08:56:44.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:44.885 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:44.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:44.886 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:44.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:44 smithi145 ceph-mon[33384]: pgmap v814: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 286 op/s 2024-09-11T08:56:44.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:44 smithi145 ceph-mon[33384]: 6.1 scrub starts 2024-09-11T08:56:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:44 smithi145 ceph-mon[33384]: 6.1 scrub ok 2024-09-11T08:56:44.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:56:44.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:45.010 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:45.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:45.012 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:45.112 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:45.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:45.135 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:45.135 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:45.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:45.256 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:45.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:45.257 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:45.358 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:45.379 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:45.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:45.380 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:45.441 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:56:45.442 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:56:45.442 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:56:45.442 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:56:45.442 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:56:45.442 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:56:45.442 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:56:45.443 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:56:45.443 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:56:45.443 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:56:45.443 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:56:45.443 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:56:45.443 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:56:45.443 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:56:45.443 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:56:45.444 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:56:45.482 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:45.502 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:45.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:45.503 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:45.604 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:45.625 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:45.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:45.626 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:45.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:45.746 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:45 smithi092 ceph-mon[29647]: 3.4 scrub starts 2024-09-11T08:56:45.746 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:45 smithi092 ceph-mon[29647]: 3.4 scrub ok 2024-09-11T08:56:45.746 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:45 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3103059988' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:56:45.746 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:45 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:56:45.746 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:56:45.746 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:56:45.746 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:45 smithi092 ceph-mon[41114]: 3.4 scrub starts 2024-09-11T08:56:45.747 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:45 smithi092 ceph-mon[41114]: 3.4 scrub ok 2024-09-11T08:56:45.747 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:45 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3103059988' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:56:45.747 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:45 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:56:45.747 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:56:45.747 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:56:45.749 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:45.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:45.750 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:45.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:45.872 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:45.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:45.873 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:45.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:45 smithi145 ceph-mon[33384]: 3.4 scrub starts 2024-09-11T08:56:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:45 smithi145 ceph-mon[33384]: 3.4 scrub ok 2024-09-11T08:56:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:45 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3103059988' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:56:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:45 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:56:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:56:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:56:45.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:45.995 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:45.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:45.996 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:46.097 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:46.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:46.119 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:46.120 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:46.220 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:46.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:46.244 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:46.244 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:46.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:46.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:46.368 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:46.368 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:46.402 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:56:46.402 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:56:46.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:46.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:46.493 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:46.493 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:46.594 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:46.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:46.620 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:46.620 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:46.721 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:46.743 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:46 smithi092 ceph-mon[41114]: pgmap v815: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:56:46.743 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:46 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:56:46.743 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:46 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:56:46.743 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:46 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:56:46.743 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:46 smithi092 ceph-mon[41114]: 5.6 scrub starts 2024-09-11T08:56:46.744 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:46 smithi092 ceph-mon[41114]: 5.6 scrub ok 2024-09-11T08:56:46.744 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:46 smithi092 ceph-mon[29647]: pgmap v815: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:56:46.744 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:46 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:56:46.744 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:46 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:56:46.744 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:46 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:56:46.744 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:46 smithi092 ceph-mon[29647]: 5.6 scrub starts 2024-09-11T08:56:46.744 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:46 smithi092 ceph-mon[29647]: 5.6 scrub ok 2024-09-11T08:56:46.746 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:46.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:46.747 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:46.847 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:46.872 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:46.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:46.873 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:46.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:46 smithi145 ceph-mon[33384]: pgmap v815: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:56:46.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:46 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:56:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:46 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:56:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:46 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:56:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:46 smithi145 ceph-mon[33384]: 5.6 scrub starts 2024-09-11T08:56:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:46 smithi145 ceph-mon[33384]: 5.6 scrub ok 2024-09-11T08:56:46.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:46.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:46.996 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:47.097 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:47.123 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:47.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:47.124 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:47.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:47.226 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:47.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:47.250 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:47.251 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:47.351 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:47.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:47.375 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:47.376 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:47.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:56:47 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:56:47.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:47.498 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:47.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:47.500 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:47.500 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:47.601 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:47.627 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:47.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:47.628 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:47.729 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:47.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:47.755 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:47.755 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:47.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:47 smithi092 ceph-mon[29647]: 5.5 scrub starts 2024-09-11T08:56:47.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:47 smithi092 ceph-mon[29647]: 5.5 scrub ok 2024-09-11T08:56:47.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:47 smithi092 ceph-mon[29647]: 5.b deep-scrub starts 2024-09-11T08:56:47.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:47 smithi092 ceph-mon[29647]: 5.b deep-scrub ok 2024-09-11T08:56:47.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:47 smithi092 ceph-mon[41114]: 5.5 scrub starts 2024-09-11T08:56:47.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:47 smithi092 ceph-mon[41114]: 5.5 scrub ok 2024-09-11T08:56:47.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:47 smithi092 ceph-mon[41114]: 5.b deep-scrub starts 2024-09-11T08:56:47.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:47 smithi092 ceph-mon[41114]: 5.b deep-scrub ok 2024-09-11T08:56:47.855 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:47.878 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:47.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:47.879 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:47.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:47 smithi145 ceph-mon[33384]: 5.5 scrub starts 2024-09-11T08:56:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:47 smithi145 ceph-mon[33384]: 5.5 scrub ok 2024-09-11T08:56:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:47 smithi145 ceph-mon[33384]: 5.b deep-scrub starts 2024-09-11T08:56:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:47 smithi145 ceph-mon[33384]: 5.b deep-scrub ok 2024-09-11T08:56:47.980 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:48.002 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:48.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:48.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:48.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-11T08:56:48.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:48.126 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:48.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:48.127 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:48.228 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:48.254 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:48.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:48.255 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:48.356 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:48.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:48.382 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:48.382 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:48.483 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:48.505 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:48.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:48.507 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:48.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:48.630 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:48.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:48.631 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:48.732 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:48.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:48.758 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:48.759 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:48.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:48 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:48.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:48 smithi092 ceph-mon[29647]: pgmap v816: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:56:48.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:48 smithi092 ceph-mon[29647]: 6.11 scrub starts 2024-09-11T08:56:48.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:48 smithi092 ceph-mon[29647]: 6.11 scrub ok 2024-09-11T08:56:48.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:48 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:48.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:48 smithi092 ceph-mon[41114]: pgmap v816: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:56:48.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:48 smithi092 ceph-mon[41114]: 6.11 scrub starts 2024-09-11T08:56:48.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:48 smithi092 ceph-mon[41114]: 6.11 scrub ok 2024-09-11T08:56:48.860 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:48.883 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:48.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:48.885 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:48.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:48 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:48 smithi145 ceph-mon[33384]: pgmap v816: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:56:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:48 smithi145 ceph-mon[33384]: 6.11 scrub starts 2024-09-11T08:56:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:48 smithi145 ceph-mon[33384]: 6.11 scrub ok 2024-09-11T08:56:48.987 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:49.011 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:49.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:49.012 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:49.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:49.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:49.141 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:49.141 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:49.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:56:49.242 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:49.274 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:49.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:49.275 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:49.375 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:49.400 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:49.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:49.401 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:49.502 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:49.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:49.527 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:49.527 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:49.628 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:49.649 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:49.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:49.651 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:49.751 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:49.776 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:49.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:49.778 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:49.878 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:49.902 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:49.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:49.903 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:50.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:50.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:50.026 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:50.026 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:50.127 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:50.150 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:50.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:50.152 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:50.246 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:50 smithi092 ceph-mon[29647]: 5.10 scrub starts 2024-09-11T08:56:50.246 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:50 smithi092 ceph-mon[29647]: 5.10 scrub ok 2024-09-11T08:56:50.247 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:50 smithi092 ceph-mon[29647]: pgmap v817: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:56:50.247 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:50 smithi092 ceph-mon[41114]: 5.10 scrub starts 2024-09-11T08:56:50.247 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:50 smithi092 ceph-mon[41114]: 5.10 scrub ok 2024-09-11T08:56:50.247 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:50 smithi092 ceph-mon[41114]: pgmap v817: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:56:50.252 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:50.276 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:50.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:50.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:50.325 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:50.378 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:50.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:50.401 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:50.402 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:50.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:50 smithi145 ceph-mon[33384]: 5.10 scrub starts 2024-09-11T08:56:50.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:50 smithi145 ceph-mon[33384]: 5.10 scrub ok 2024-09-11T08:56:50.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:50 smithi145 ceph-mon[33384]: pgmap v817: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:56:50.502 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:50.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:50.526 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:50.527 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:50.627 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:50.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:50.651 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:50.652 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:50.753 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:50.776 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:50.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:50.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:50.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:50.899 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:50.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:50.900 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:51.001 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:51.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:51.026 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:51.026 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:51.127 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:51.149 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:51.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:51.150 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:51.212 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:51 smithi092 ceph-mon[29647]: 4.4 scrub starts 2024-09-11T08:56:51.212 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:51 smithi092 ceph-mon[29647]: 4.4 scrub ok 2024-09-11T08:56:51.212 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:51 smithi092 ceph-mon[29647]: 4.2 scrub starts 2024-09-11T08:56:51.212 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:51 smithi092 ceph-mon[29647]: 4.2 scrub ok 2024-09-11T08:56:51.213 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:51 smithi092 ceph-mon[29647]: 3.2 scrub starts 2024-09-11T08:56:51.213 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:51 smithi092 ceph-mon[29647]: 3.2 scrub ok 2024-09-11T08:56:51.213 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:51 smithi092 ceph-mon[41114]: 4.4 scrub starts 2024-09-11T08:56:51.213 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:51 smithi092 ceph-mon[41114]: 4.4 scrub ok 2024-09-11T08:56:51.213 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:51 smithi092 ceph-mon[41114]: 4.2 scrub starts 2024-09-11T08:56:51.213 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:51 smithi092 ceph-mon[41114]: 4.2 scrub ok 2024-09-11T08:56:51.213 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:51 smithi092 ceph-mon[41114]: 3.2 scrub starts 2024-09-11T08:56:51.213 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:51 smithi092 ceph-mon[41114]: 3.2 scrub ok 2024-09-11T08:56:51.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:51.272 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:51.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:51.273 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:51.294 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:51.374 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:51.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:51.399 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:51.400 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:51.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:51 smithi145 ceph-mon[33384]: 4.4 scrub starts 2024-09-11T08:56:51.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:51 smithi145 ceph-mon[33384]: 4.4 scrub ok 2024-09-11T08:56:51.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:51 smithi145 ceph-mon[33384]: 4.2 scrub starts 2024-09-11T08:56:51.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:51 smithi145 ceph-mon[33384]: 4.2 scrub ok 2024-09-11T08:56:51.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:51 smithi145 ceph-mon[33384]: 3.2 scrub starts 2024-09-11T08:56:51.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:51 smithi145 ceph-mon[33384]: 3.2 scrub ok 2024-09-11T08:56:51.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:51.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:51.525 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:51.525 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:51.531 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:56:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:56:51] "GET /metrics HTTP/1.1" 200 224751 "" "Prometheus/2.43.0" 2024-09-11T08:56:51.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:51.648 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:51.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:51.649 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:51.750 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:51.775 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:51.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:51.776 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:51.877 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:51.907 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:51.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:51.908 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:52.009 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:52.037 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:52.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:52.038 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:52.090 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:56:52.093 INFO:teuthology.orchestra.run.smithi092.stderr:set pool 10 pg_num to 26 2024-09-11T08:56:52.139 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:52.164 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:52.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:52.165 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:52.266 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:52.289 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:52.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:52.290 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:52.390 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:52.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:52 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1590560632' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:56:52.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:52 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/821209391' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"}]: dispatch 2024-09-11T08:56:52.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:52 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"}]: dispatch 2024-09-11T08:56:52.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:52 smithi145 ceph-mon[33384]: pgmap v818: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:56:52.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:52 smithi145 ceph-mon[33384]: 6.b scrub starts 2024-09-11T08:56:52.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:52 smithi145 ceph-mon[33384]: 6.b scrub ok 2024-09-11T08:56:52.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:52.413 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:52.413 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:52.499 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:52 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1590560632' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:56:52.499 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:52 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/821209391' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"}]: dispatch 2024-09-11T08:56:52.499 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:52 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"}]: dispatch 2024-09-11T08:56:52.499 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:52 smithi092 ceph-mon[41114]: pgmap v818: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:56:52.499 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:52 smithi092 ceph-mon[41114]: 6.b scrub starts 2024-09-11T08:56:52.499 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:52 smithi092 ceph-mon[41114]: 6.b scrub ok 2024-09-11T08:56:52.499 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:52 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1590560632' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:56:52.499 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:52 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/821209391' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"}]: dispatch 2024-09-11T08:56:52.500 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:52 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"}]: dispatch 2024-09-11T08:56:52.500 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:52 smithi092 ceph-mon[29647]: pgmap v818: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:56:52.500 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:52 smithi092 ceph-mon[29647]: 6.b scrub starts 2024-09-11T08:56:52.500 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:52 smithi092 ceph-mon[29647]: 6.b scrub ok 2024-09-11T08:56:52.513 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:52.539 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:52.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:52.540 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:52.641 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:52.667 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:52.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:52.668 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:52.768 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:52.793 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:52.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:52.794 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:52.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:52.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:52.922 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:53.023 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:53.049 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:53.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:53.050 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:53.085 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:56:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:56:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:56:53.151 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:53.174 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:53.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:53.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:53.202 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:53 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:56:53.202 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:53 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1590560632' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:56:53.203 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:53 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"}]': finished 2024-09-11T08:56:53.203 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:53 smithi092 ceph-mon[41114]: osdmap e357: 8 total, 8 up, 8 in 2024-09-11T08:56:53.203 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:53 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:56:53.203 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:53 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1590560632' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:56:53.203 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:53 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"}]': finished 2024-09-11T08:56:53.203 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:53 smithi092 ceph-mon[29647]: osdmap e357: 8 total, 8 up, 8 in 2024-09-11T08:56:53.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:53.300 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:53.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:53.301 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:53.402 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:53.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:53 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:56:53.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:53 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1590560632' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:56:53.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:53 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"}]': finished 2024-09-11T08:56:53.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:53 smithi145 ceph-mon[33384]: osdmap e357: 8 total, 8 up, 8 in 2024-09-11T08:56:53.428 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:53.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:53.429 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:53.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:53.537 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_3 2024-09-11T08:56:53.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2024-09-11T08:56:53.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:53.554 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:53.554 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:53.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:53.680 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:53.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:53.681 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:53.782 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:53.807 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:53.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:53.808 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:53.909 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:53.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:53.932 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:53.932 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:54.033 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:54.059 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:54.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:54.060 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:54.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:54.216 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:54.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:54.217 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:54.317 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:54.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:54.353 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:54.354 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:54.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:54 smithi145 ceph-mon[33384]: pgmap v820: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:56:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:54 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-11T08:56:54.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:54 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-11T08:56:54.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:54.476 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:54.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:54.480 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:54.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:54 smithi092 ceph-mon[29647]: pgmap v820: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:56:54.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:54 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-11T08:56:54.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:54 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-11T08:56:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:54 smithi092 ceph-mon[41114]: pgmap v820: 158 pgs: 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:56:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:54 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-11T08:56:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:54 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-11T08:56:54.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:54.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:54.603 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:54.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:54.704 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:54.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:54.727 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:54.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:54.828 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:54.849 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:54.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:54.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:54.929 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:56:54.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:54.994 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:54.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:54.995 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:55.096 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:55.128 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:55.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:55.129 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:55.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:55.251 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:55.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:55.252 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:55.313 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:56:55.313 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:56:55.317 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:56:55.317 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:56:55.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:55 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-11T08:56:55.318 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:55 smithi092 ceph-mon[29647]: osdmap e358: 8 total, 8 up, 8 in 2024-09-11T08:56:55.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:55 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-11T08:56:55.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:55 smithi092 ceph-mon[41114]: osdmap e358: 8 total, 8 up, 8 in 2024-09-11T08:56:55.335 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:56:55.335 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:56:55.353 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:55.355 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:56:55.355 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:56:55.370 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:56:55.371 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:56:55.374 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:55.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:55.375 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:55.386 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:56:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:56:55.402 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:56:55.403 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:56:55.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:55 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-11T08:56:55.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:55 smithi145 ceph-mon[33384]: osdmap e358: 8 total, 8 up, 8 in 2024-09-11T08:56:55.418 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:56:55.418 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:56:55.477 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:55.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:55.499 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:55.499 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:55.600 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:55.622 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:55.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:55.623 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:55.724 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:55.745 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:55.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:55.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:55.847 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:55.868 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:55.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:55.869 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:55.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:55.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:55.993 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:56.094 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:56.115 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:56.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:56.116 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:56.125 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:56 smithi092 ceph-mon[29647]: osdmap e359: 8 total, 8 up, 8 in 2024-09-11T08:56:56.125 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:56 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:56:56.125 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:56 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:56:56.125 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:56 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1413971595' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:56:56.125 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:56 smithi092 ceph-mon[29647]: pgmap v823: 168 pgs: 10 unknown, 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:56:56.125 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:56 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-11T08:56:56.125 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:56 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-11T08:56:56.217 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:56.239 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:56.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:56.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:56.340 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:56.362 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:56.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:56.363 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:56.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:56 smithi145 ceph-mon[33384]: osdmap e359: 8 total, 8 up, 8 in 2024-09-11T08:56:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:56 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:56:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:56 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:56:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:56 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1413971595' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:56:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:56 smithi145 ceph-mon[33384]: pgmap v823: 168 pgs: 10 unknown, 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:56:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:56 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-11T08:56:56.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:56 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-11T08:56:56.412 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:56 smithi092 ceph-mon[41114]: osdmap e359: 8 total, 8 up, 8 in 2024-09-11T08:56:56.412 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:56 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:56:56.412 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:56 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:56:56.413 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:56 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1413971595' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:56:56.413 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:56 smithi092 ceph-mon[41114]: pgmap v823: 168 pgs: 10 unknown, 158 active+clean; 687 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:56:56.413 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:56 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-11T08:56:56.413 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:56 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-11T08:56:56.416 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:56:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-11T08:56:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-11T08:56:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:56:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120766 9.0 9:00840795:::benchmark_data_smithi092_145083_object120765:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:56:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.505554+0000", 2024-09-11T08:56:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 390.90985680900002, 2024-09-11T08:56:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0232278610000001, 2024-09-11T08:56:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:56:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:56:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:56:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120766 2024-09-11T08:56:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.505554+0000", 2024-09-11T08:56:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.505554+0000", 2024-09-11T08:56:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.505552+0000", 2024-09-11T08:56:56.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:56:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.505580+0000", 2024-09-11T08:56:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7098999999999999e-05 2024-09-11T08:56:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.505594+0000", 2024-09-11T08:56:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4822e-05 2024-09-11T08:56:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.505600+0000", 2024-09-11T08:56:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9959999999999999e-06 2024-09-11T08:56:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507033+0000", 2024-09-11T08:56:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014323280000000001 2024-09-11T08:56:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-11T08:56:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.507039+0000", 2024-09-11T08:56:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0979999999999999e-06 2024-09-11T08:56:56.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.652776+0000", 2024-09-11T08:56:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.14573702099999999 2024-09-11T08:56:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:56:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.652787+0000", 2024-09-11T08:56:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1576000000000001e-05 2024-09-11T08:56:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.528710+0000", 2024-09-11T08:56:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.87592291300000003 2024-09-11T08:56:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.528782+0000", 2024-09-11T08:56:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1829000000000006e-05 2024-09-11T08:56:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120779 9.0 9:05180796:::benchmark_data_smithi092_145083_object120778:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:56:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.752494+0000", 2024-09-11T08:56:56.422 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 390.66291722900002, 2024-09-11T08:56:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.79529023499999996, 2024-09-11T08:56:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:56:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:56:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:56:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120779 2024-09-11T08:56:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.752494+0000", 2024-09-11T08:56:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.752494+0000", 2024-09-11T08:56:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.752492+0000", 2024-09-11T08:56:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:56:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.752519+0000", 2024-09-11T08:56:56.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6950000000000001e-05 2024-09-11T08:56:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.752536+0000", 2024-09-11T08:56:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7118000000000001e-05 2024-09-11T08:56:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.752544+0000", 2024-09-11T08:56:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7980000000000006e-06 2024-09-11T08:56:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.752594+0000", 2024-09-11T08:56:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0312000000000002e-05 2024-09-11T08:56:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:56:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.752606+0000", 2024-09-11T08:56:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1717e-05 2024-09-11T08:56:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529063+0000", 2024-09-11T08:56:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.776457438 2024-09-11T08:56:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:56:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529185+0000", 2024-09-11T08:56:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012197300000000001 2024-09-11T08:56:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,6", 2024-09-11T08:56:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529538+0000", 2024-09-11T08:56:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035262999999999999 2024-09-11T08:56:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:56:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.534183+0000", 2024-09-11T08:56:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046452339999999998 2024-09-11T08:56:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.540247+0000", 2024-09-11T08:56:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060644239999999997 2024-09-11T08:56:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.547697+0000", 2024-09-11T08:56:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074494590000000003 2024-09-11T08:56:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:56:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.547751+0000", 2024-09-11T08:56:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3980999999999997e-05 2024-09-11T08:56:56.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.547784+0000", 2024-09-11T08:56:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3293000000000001e-05 2024-09-11T08:56:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:122650 9.16 9:6df371c5:::benchmark_data_smithi092_145083_object122649:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e238)", 2024-09-11T08:56:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:28.534897+0000", 2024-09-11T08:56:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 387.88051398699997, 2024-09-11T08:56:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.445186625, 2024-09-11T08:56:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:56:56.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:56:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:56:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122650 2024-09-11T08:56:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.534897+0000", 2024-09-11T08:56:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.534897+0000", 2024-09-11T08:56:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.534897+0000", 2024-09-11T08:56:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:56:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.534937+0000", 2024-09-11T08:56:56.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0225999999999997e-05 2024-09-11T08:56:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.534943+0000", 2024-09-11T08:56:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4400000000000002e-06 2024-09-11T08:56:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.534947+0000", 2024-09-11T08:56:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9840000000000003e-06 2024-09-11T08:56:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.537785+0000", 2024-09-11T08:56:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028376019999999998 2024-09-11T08:56:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:56:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.537792+0000", 2024-09-11T08:56:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0910000000000004e-06 2024-09-11T08:56:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.967357+0000", 2024-09-11T08:56:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.42956518999999999 2024-09-11T08:56:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:56:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.967440+0000", 2024-09-11T08:56:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2837999999999995e-05 2024-09-11T08:56:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,4,5", 2024-09-11T08:56:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.967626+0000", 2024-09-11T08:56:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018575699999999999 2024-09-11T08:56:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:56:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.971675+0000", 2024-09-11T08:56:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004049647 2024-09-11T08:56:56.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.971901+0000", 2024-09-11T08:56:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022539500000000001 2024-09-11T08:56:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.979922+0000", 2024-09-11T08:56:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080216880000000008 2024-09-11T08:56:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.979958+0000", 2024-09-11T08:56:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5568999999999998e-05 2024-09-11T08:56:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:56:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.979996+0000", 2024-09-11T08:56:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8260000000000003e-05 2024-09-11T08:56:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980084+0000", 2024-09-11T08:56:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7424999999999997e-05 2024-09-11T08:56:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.437 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:122665 9.16 9:6ac16e6f:::benchmark_data_smithi092_145083_object122664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e238)", 2024-09-11T08:56:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:28.534986+0000", 2024-09-11T08:56:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 387.88042522500001, 2024-09-11T08:56:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.44535904799999998, 2024-09-11T08:56:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:56:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:56:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:56:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122665 2024-09-11T08:56:56.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.534986+0000", 2024-09-11T08:56:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.534986+0000", 2024-09-11T08:56:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.534985+0000", 2024-09-11T08:56:56.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:56:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535077+0000", 2024-09-11T08:56:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1791000000000003e-05 2024-09-11T08:56:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535085+0000", 2024-09-11T08:56:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6029999999999999e-06 2024-09-11T08:56:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535089+0000", 2024-09-11T08:56:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1779999999999996e-06 2024-09-11T08:56:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.537812+0000", 2024-09-11T08:56:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027234669999999998 2024-09-11T08:56:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:56:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.537818+0000", 2024-09-11T08:56:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9610000000000002e-06 2024-09-11T08:56:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.968413+0000", 2024-09-11T08:56:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43059455899999999 2024-09-11T08:56:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:56:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.968533+0000", 2024-09-11T08:56:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011961200000000001 2024-09-11T08:56:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,4,5", 2024-09-11T08:56:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.968747+0000", 2024-09-11T08:56:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000214799 2024-09-11T08:56:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:56:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.971683+0000", 2024-09-11T08:56:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029354960000000001 2024-09-11T08:56:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.971953+0000", 2024-09-11T08:56:56.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026998700000000002 2024-09-11T08:56:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980165+0000", 2024-09-11T08:56:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082118139999999996 2024-09-11T08:56:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980195+0000", 2024-09-11T08:56:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0170000000000001e-05 2024-09-11T08:56:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:56:56.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980232+0000", 2024-09-11T08:56:56.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7085999999999997e-05 2024-09-11T08:56:56.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980345+0000", 2024-09-11T08:56:56.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011305 2024-09-11T08:56:56.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.445 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.445 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:122673 9.16 9:68d64482:::benchmark_data_smithi092_145083_object122672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e238)", 2024-09-11T08:56:56.446 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:28.535153+0000", 2024-09-11T08:56:56.446 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 387.88025794399999, 2024-09-11T08:56:56.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.44551755700000001, 2024-09-11T08:56:56.446 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.446 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:56:56.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:56:56.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:56:56.446 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122673 2024-09-11T08:56:56.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.446 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535153+0000", 2024-09-11T08:56:56.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535153+0000", 2024-09-11T08:56:56.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535153+0000", 2024-09-11T08:56:56.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:56:56.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535857+0000", 2024-09-11T08:56:56.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00070474199999999996 2024-09-11T08:56:56.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535868+0000", 2024-09-11T08:56:56.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.048e-05 2024-09-11T08:56:56.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535874+0000", 2024-09-11T08:56:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9900000000000002e-06 2024-09-11T08:56:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.537874+0000", 2024-09-11T08:56:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020001099999999998 2024-09-11T08:56:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:56:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.537880+0000", 2024-09-11T08:56:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6890000000000001e-06 2024-09-11T08:56:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.969734+0000", 2024-09-11T08:56:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43185388499999999 2024-09-11T08:56:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:56:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.969884+0000", 2024-09-11T08:56:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150128 2024-09-11T08:56:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,4,5", 2024-09-11T08:56:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.970118+0000", 2024-09-11T08:56:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023382899999999999 2024-09-11T08:56:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:56:56.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.971912+0000", 2024-09-11T08:56:56.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001794726 2024-09-11T08:56:56.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.979895+0000", 2024-09-11T08:56:56.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079827279999999997 2024-09-11T08:56:56.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980429+0000", 2024-09-11T08:56:56.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00053432999999999996 2024-09-11T08:56:56.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980479+0000", 2024-09-11T08:56:56.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9682999999999998e-05 2024-09-11T08:56:56.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:56:56.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980541+0000", 2024-09-11T08:56:56.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1989000000000006e-05 2024-09-11T08:56:56.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980671+0000", 2024-09-11T08:56:56.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012966400000000001 2024-09-11T08:56:56.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.453 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.453 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:122675 9.16 9:6c99f105:::benchmark_data_smithi092_145083_object122674:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e238)", 2024-09-11T08:56:56.453 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:28.535880+0000", 2024-09-11T08:56:56.453 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 387.87953083999997, 2024-09-11T08:56:56.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.44504194400000002, 2024-09-11T08:56:56.454 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:56:56.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:56:56.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:56:56.454 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122675 2024-09-11T08:56:56.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.454 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535880+0000", 2024-09-11T08:56:56.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535880+0000", 2024-09-11T08:56:56.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535880+0000", 2024-09-11T08:56:56.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:56:56.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535926+0000", 2024-09-11T08:56:56.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5994e-05 2024-09-11T08:56:56.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535933+0000", 2024-09-11T08:56:56.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7000000000000008e-06 2024-09-11T08:56:56.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535938+0000", 2024-09-11T08:56:56.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5859999999999998e-06 2024-09-11T08:56:56.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.537901+0000", 2024-09-11T08:56:56.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001963406 2024-09-11T08:56:56.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:56:56.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.537908+0000", 2024-09-11T08:56:56.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3149999999999997e-06 2024-09-11T08:56:56.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.970686+0000", 2024-09-11T08:56:56.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43277784000000002 2024-09-11T08:56:56.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:56:56.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.970770+0000", 2024-09-11T08:56:56.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4209000000000006e-05 2024-09-11T08:56:56.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,4,5", 2024-09-11T08:56:56.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.970922+0000", 2024-09-11T08:56:56.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015258700000000001 2024-09-11T08:56:56.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:56:56.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.979817+0000", 2024-09-11T08:56:56.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088944909999999992 2024-09-11T08:56:56.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980127+0000", 2024-09-11T08:56:56.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031017799999999998 2024-09-11T08:56:56.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980732+0000", 2024-09-11T08:56:56.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060536200000000002 2024-09-11T08:56:56.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980768+0000", 2024-09-11T08:56:56.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5454000000000003e-05 2024-09-11T08:56:56.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:56:56.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980812+0000", 2024-09-11T08:56:56.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4026000000000002e-05 2024-09-11T08:56:56.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980922+0000", 2024-09-11T08:56:56.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110376 2024-09-11T08:56:56.460 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.460 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.460 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.460 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:122678 9.16 9:69c9d3d2:::benchmark_data_smithi092_145083_object122677:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e238)", 2024-09-11T08:56:56.461 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:28.536006+0000", 2024-09-11T08:56:56.461 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 387.87940500100001, 2024-09-11T08:56:56.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.44519106000000003, 2024-09-11T08:56:56.461 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.461 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:56:56.461 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.461 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:56:56.461 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:56:56.461 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122678 2024-09-11T08:56:56.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.461 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.536006+0000", 2024-09-11T08:56:56.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.536006+0000", 2024-09-11T08:56:56.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.536006+0000", 2024-09-11T08:56:56.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:56:56.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.536052+0000", 2024-09-11T08:56:56.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6402000000000001e-05 2024-09-11T08:56:56.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.536059+0000", 2024-09-11T08:56:56.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3930000000000002e-06 2024-09-11T08:56:56.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.536063+0000", 2024-09-11T08:56:56.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9570000000000004e-06 2024-09-11T08:56:56.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.538009+0000", 2024-09-11T08:56:56.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019457299999999999 2024-09-11T08:56:56.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:56:56.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.538017+0000", 2024-09-11T08:56:56.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8380000000000002e-06 2024-09-11T08:56:56.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.971978+0000", 2024-09-11T08:56:56.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43396122999999998 2024-09-11T08:56:56.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:56:56.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.972085+0000", 2024-09-11T08:56:56.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107101 2024-09-11T08:56:56.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,4,5", 2024-09-11T08:56:56.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.972276+0000", 2024-09-11T08:56:56.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019128 2024-09-11T08:56:56.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:56:56.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.979937+0000", 2024-09-11T08:56:56.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0076602989999999998 2024-09-11T08:56:56.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980408+0000", 2024-09-11T08:56:56.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047103200000000003 2024-09-11T08:56:56.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.980974+0000", 2024-09-11T08:56:56.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00056660399999999996 2024-09-11T08:56:56.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.981040+0000", 2024-09-11T08:56:56.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5878999999999997e-05 2024-09-11T08:56:56.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:56:56.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.981092+0000", 2024-09-11T08:56:56.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1869000000000001e-05 2024-09-11T08:56:56.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.981197+0000", 2024-09-11T08:56:56.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010498899999999999 2024-09-11T08:56:56.468 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.468 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.468 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.468 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:152119 10.2 10:42ef801f:::benchmark_data_smithi092_173109_object15160:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e326)", 2024-09-11T08:56:56.468 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:54:46.700926+0000", 2024-09-11T08:56:56.468 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 129.714484906, 2024-09-11T08:56:56.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.17168703599999999, 2024-09-11T08:56:56.468 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.468 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:56:56.468 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.468 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:56:56.469 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:56:56.469 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 152119 2024-09-11T08:56:56.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.469 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.700926+0000", 2024-09-11T08:56:56.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.700926+0000", 2024-09-11T08:56:56.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.700924+0000", 2024-09-11T08:56:56.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999976 2024-09-11T08:56:56.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.700933+0000", 2024-09-11T08:56:56.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8449999999999992e-06 2024-09-11T08:56:56.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.700949+0000", 2024-09-11T08:56:56.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5979000000000001e-05 2024-09-11T08:56:56.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.700956+0000", 2024-09-11T08:56:56.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1080000000000004e-06 2024-09-11T08:56:56.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.701000+0000", 2024-09-11T08:56:56.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4626000000000003e-05 2024-09-11T08:56:56.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:56:56.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.701122+0000", 2024-09-11T08:56:56.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012190599999999999 2024-09-11T08:56:56.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,4", 2024-09-11T08:56:56.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.701250+0000", 2024-09-11T08:56:56.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012781899999999999 2024-09-11T08:56:56.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:56:56.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872402+0000", 2024-09-11T08:56:56.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.171151944 2024-09-11T08:56:56.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872497+0000", 2024-09-11T08:56:56.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4361e-05 2024-09-11T08:56:56.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872535+0000", 2024-09-11T08:56:56.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8806000000000001e-05 2024-09-11T08:56:56.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:56:56.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872575+0000", 2024-09-11T08:56:56.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9801000000000001e-05 2024-09-11T08:56:56.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872613+0000", 2024-09-11T08:56:56.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8084999999999999e-05 2024-09-11T08:56:56.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.474 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.474 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15621.0:152120 10.b e326/293 10:d8e1f76e:::benchmark_data_smithi092_173109_object15161:head v 326'9499, mlcod=326'9497)", 2024-09-11T08:56:56.474 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:54:46.702234+0000", 2024-09-11T08:56:56.474 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 129.71317707, 2024-09-11T08:56:56.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.170254089, 2024-09-11T08:56:56.474 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.475 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:56:56.475 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.702234+0000", 2024-09-11T08:56:56.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.702234+0000", 2024-09-11T08:56:56.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.702233+0000", 2024-09-11T08:56:56.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:56:56.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.702240+0000", 2024-09-11T08:56:56.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4320000000000002e-06 2024-09-11T08:56:56.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.702252+0000", 2024-09-11T08:56:56.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1493e-05 2024-09-11T08:56:56.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.702257+0000", 2024-09-11T08:56:56.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2669999999999997e-06 2024-09-11T08:56:56.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.702296+0000", 2024-09-11T08:56:56.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8651e-05 2024-09-11T08:56:56.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:56:56.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.702315+0000", 2024-09-11T08:56:56.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8709999999999999e-05 2024-09-11T08:56:56.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:56:56.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872441+0000", 2024-09-11T08:56:56.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.17012603300000001 2024-09-11T08:56:56.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872488+0000", 2024-09-11T08:56:56.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7528999999999998e-05 2024-09-11T08:56:56.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.479 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.479 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.15621.0:152119 10.2 e326/293 ondisk, result = 0)", 2024-09-11T08:56:56.479 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:54:46.705220+0000", 2024-09-11T08:56:56.479 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 129.71019079600001, 2024-09-11T08:56:56.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16739689499999999, 2024-09-11T08:56:56.479 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.479 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:56:56.479 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.705220+0000", 2024-09-11T08:56:56.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.705220+0000", 2024-09-11T08:56:56.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.705219+0000", 2024-09-11T08:56:56.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:56:56.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.705228+0000", 2024-09-11T08:56:56.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1540000000000002e-06 2024-09-11T08:56:56.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872404+0000", 2024-09-11T08:56:56.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16717674099999999 2024-09-11T08:56:56.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872427+0000", 2024-09-11T08:56:56.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2850999999999999e-05 2024-09-11T08:56:56.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872525+0000", 2024-09-11T08:56:56.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8191999999999998e-05 2024-09-11T08:56:56.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:56:56.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872531+0000", 2024-09-11T08:56:56.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7490000000000004e-06 2024-09-11T08:56:56.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872617+0000", 2024-09-11T08:56:56.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6081999999999993e-05 2024-09-11T08:56:56.483 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.483 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.483 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.483 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.15621.0:152119 10.2 e326/293 ondisk, result = 0)", 2024-09-11T08:56:56.483 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:54:46.707296+0000", 2024-09-11T08:56:56.483 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 129.708114728, 2024-09-11T08:56:56.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16533194900000001, 2024-09-11T08:56:56.483 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.483 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:56:56.483 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.707296+0000", 2024-09-11T08:56:56.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.707296+0000", 2024-09-11T08:56:56.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.707295+0000", 2024-09-11T08:56:56.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:56:56.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.707302+0000", 2024-09-11T08:56:56.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5050000000000004e-06 2024-09-11T08:56:56.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872414+0000", 2024-09-11T08:56:56.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.165112488 2024-09-11T08:56:56.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872426+0000", 2024-09-11T08:56:56.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1844e-05 2024-09-11T08:56:56.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872479+0000", 2024-09-11T08:56:56.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3164000000000001e-05 2024-09-11T08:56:56.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:56:56.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872490+0000", 2024-09-11T08:56:56.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0548e-05 2024-09-11T08:56:56.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:46.872628+0000", 2024-09-11T08:56:56.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013842800000000001 2024-09-11T08:56:56.487 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.487 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.487 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.487 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:244892 10.b 10:d0f43bb1:::benchmark_data_smithi092_173109_object107933:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T08:56:56.487 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:56.489 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:07.461766+0000", 2024-09-11T08:56:56.489 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.953645481000002, 2024-09-11T08:56:56.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0670441880000001, 2024-09-11T08:56:56.489 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.489 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:56:56.489 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.489 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:56:56.489 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:56:56.489 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 244892 2024-09-11T08:56:56.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.489 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461766+0000", 2024-09-11T08:56:56.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461766+0000", 2024-09-11T08:56:56.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461764+0000", 2024-09-11T08:56:56.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:56:56.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461769+0000", 2024-09-11T08:56:56.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1080000000000001e-06 2024-09-11T08:56:56.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461782+0000", 2024-09-11T08:56:56.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2706e-05 2024-09-11T08:56:56.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461789+0000", 2024-09-11T08:56:56.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0559999999999999e-06 2024-09-11T08:56:56.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.462023+0000", 2024-09-11T08:56:56.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000234231 2024-09-11T08:56:56.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:56:56.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.462033+0000", 2024-09-11T08:56:56.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2969999999999992e-06 2024-09-11T08:56:56.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.528744+0000", 2024-09-11T08:56:56.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.066711594 2024-09-11T08:56:56.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.528810+0000", 2024-09-11T08:56:56.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5425000000000005e-05 2024-09-11T08:56:56.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.493 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.493 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:244895 10.b 10:d88923c4:::benchmark_data_smithi092_173109_object107936:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T08:56:56.493 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:07.461802+0000", 2024-09-11T08:56:56.494 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.953609026999999, 2024-09-11T08:56:56.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0672735449999999, 2024-09-11T08:56:56.494 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.494 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:56:56.494 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.494 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:56:56.494 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:56:56.494 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 244895 2024-09-11T08:56:56.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.494 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461802+0000", 2024-09-11T08:56:56.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461802+0000", 2024-09-11T08:56:56.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461802+0000", 2024-09-11T08:56:56.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:56:56.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461805+0000", 2024-09-11T08:56:56.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0130000000000002e-06 2024-09-11T08:56:56.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461811+0000", 2024-09-11T08:56:56.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5479999999999998e-06 2024-09-11T08:56:56.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461815+0000", 2024-09-11T08:56:56.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0749999999999999e-06 2024-09-11T08:56:56.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.462051+0000", 2024-09-11T08:56:56.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023617700000000001 2024-09-11T08:56:56.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:56:56.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.462057+0000", 2024-09-11T08:56:56.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2310000000000004e-06 2024-09-11T08:56:56.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.529023+0000", 2024-09-11T08:56:56.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.066966123 2024-09-11T08:56:56.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.529076+0000", 2024-09-11T08:56:56.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2862999999999999e-05 2024-09-11T08:56:56.498 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.498 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.498 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.498 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:244896 10.0 10:0ef8d7ff:::benchmark_data_smithi092_173109_object107937:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T08:56:56.498 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:07.461819+0000", 2024-09-11T08:56:56.498 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.953591891999999, 2024-09-11T08:56:56.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.058419523, 2024-09-11T08:56:56.498 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.498 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:56:56.499 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.499 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:56:56.499 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:56:56.499 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 244896 2024-09-11T08:56:56.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.499 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461819+0000", 2024-09-11T08:56:56.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461819+0000", 2024-09-11T08:56:56.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461819+0000", 2024-09-11T08:56:56.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:56:56.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461821+0000", 2024-09-11T08:56:56.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.379e-06 2024-09-11T08:56:56.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461834+0000", 2024-09-11T08:56:56.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2928000000000001e-05 2024-09-11T08:56:56.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461838+0000", 2024-09-11T08:56:56.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5700000000000001e-06 2024-09-11T08:56:56.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.464155+0000", 2024-09-11T08:56:56.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023176210000000002 2024-09-11T08:56:56.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:56:56.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.464164+0000", 2024-09-11T08:56:56.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0299999999999999e-06 2024-09-11T08:56:56.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.520154+0000", 2024-09-11T08:56:56.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0559898919999999 2024-09-11T08:56:56.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.520239+0000", 2024-09-11T08:56:56.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4451999999999997e-05 2024-09-11T08:56:56.503 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.503 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.503 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.503 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:244898 10.0 10:02383ae0:::benchmark_data_smithi092_173109_object107939:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T08:56:56.504 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:07.461847+0000", 2024-09-11T08:56:56.504 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.953564198999999, 2024-09-11T08:56:56.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0856686630000001, 2024-09-11T08:56:56.504 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.504 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:56:56.504 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.504 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:56:56.504 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:56:56.504 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 244898 2024-09-11T08:56:56.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.504 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461847+0000", 2024-09-11T08:56:56.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461847+0000", 2024-09-11T08:56:56.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461846+0000", 2024-09-11T08:56:56.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:56:56.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461849+0000", 2024-09-11T08:56:56.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8930000000000001e-06 2024-09-11T08:56:56.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461855+0000", 2024-09-11T08:56:56.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6380000000000001e-06 2024-09-11T08:56:56.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461858+0000", 2024-09-11T08:56:56.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4259999999999999e-06 2024-09-11T08:56:56.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.464183+0000", 2024-09-11T08:56:56.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002324591 2024-09-11T08:56:56.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:56:56.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.464189+0000", 2024-09-11T08:56:56.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7060000000000001e-06 2024-09-11T08:56:56.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.520544+0000", 2024-09-11T08:56:56.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0563550020000001 2024-09-11T08:56:56.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-09-11T08:56:56.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.520617+0000", 2024-09-11T08:56:56.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3520000000000001e-05 2024-09-11T08:56:56.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.547438+0000", 2024-09-11T08:56:56.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026820324999999999 2024-09-11T08:56:56.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.547516+0000", 2024-09-11T08:56:56.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7997000000000006e-05 2024-09-11T08:56:56.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.512 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.512 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:244900 10.b 10:d2286edc:::benchmark_data_smithi092_173109_object107941:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T08:56:56.512 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:07.461862+0000", 2024-09-11T08:56:56.513 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.953549258000002, 2024-09-11T08:56:56.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0804710120000001, 2024-09-11T08:56:56.513 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.513 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:56:56.513 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.513 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:56:56.513 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:56:56.513 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 244900 2024-09-11T08:56:56.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.513 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461862+0000", 2024-09-11T08:56:56.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461862+0000", 2024-09-11T08:56:56.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461862+0000", 2024-09-11T08:56:56.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:56:56.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461864+0000", 2024-09-11T08:56:56.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5780000000000001e-06 2024-09-11T08:56:56.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461870+0000", 2024-09-11T08:56:56.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4840000000000003e-06 2024-09-11T08:56:56.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.461873+0000", 2024-09-11T08:56:56.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3170000000000002e-06 2024-09-11T08:56:56.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.462075+0000", 2024-09-11T08:56:56.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020167699999999999 2024-09-11T08:56:56.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:56:56.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.462080+0000", 2024-09-11T08:56:56.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7769999999999999e-06 2024-09-11T08:56:56.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.529094+0000", 2024-09-11T08:56:56.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0670139839999999 2024-09-11T08:56:56.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-09-11T08:56:56.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.529177+0000", 2024-09-11T08:56:56.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2200999999999994e-05 2024-09-11T08:56:56.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.542281+0000", 2024-09-11T08:56:56.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013104154999999999 2024-09-11T08:56:56.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.542333+0000", 2024-09-11T08:56:56.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2162e-05 2024-09-11T08:56:56.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.517 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.518 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.518 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:244911 10.0 10:09bd37d8:::benchmark_data_smithi092_173109_object107952:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T08:56:56.518 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:07.473902+0000", 2024-09-11T08:56:56.518 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.941508792999997, 2024-09-11T08:56:56.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0652263770000001, 2024-09-11T08:56:56.518 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.518 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:56:56.518 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.518 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:56:56.518 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:56:56.518 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 244911 2024-09-11T08:56:56.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.519 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.473902+0000", 2024-09-11T08:56:56.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.473902+0000", 2024-09-11T08:56:56.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.473900+0000", 2024-09-11T08:56:56.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:56:56.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.473908+0000", 2024-09-11T08:56:56.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2599999999999999e-06 2024-09-11T08:56:56.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.473923+0000", 2024-09-11T08:56:56.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5225000000000001e-05 2024-09-11T08:56:56.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.473929+0000", 2024-09-11T08:56:56.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0170000000000002e-06 2024-09-11T08:56:56.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.474070+0000", 2024-09-11T08:56:56.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014119600000000001 2024-09-11T08:56:56.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:56:56.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.474092+0000", 2024-09-11T08:56:56.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2297e-05 2024-09-11T08:56:56.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.520668+0000", 2024-09-11T08:56:56.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.04657584 2024-09-11T08:56:56.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:56:56.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.520797+0000", 2024-09-11T08:56:56.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012863800000000001 2024-09-11T08:56:56.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4,6", 2024-09-11T08:56:56.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.521360+0000", 2024-09-11T08:56:56.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00056280700000000002 2024-09-11T08:56:56.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.538798+0000", 2024-09-11T08:56:56.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017438105999999998 2024-09-11T08:56:56.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.538837+0000", 2024-09-11T08:56:56.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8912999999999998e-05 2024-09-11T08:56:56.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:56:56.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.538845+0000", 2024-09-11T08:56:56.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3790000000000006e-06 2024-09-11T08:56:56.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:56:56.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.538882+0000", 2024-09-11T08:56:56.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6940999999999997e-05 2024-09-11T08:56:56.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.539129+0000", 2024-09-11T08:56:56.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024660200000000001 2024-09-11T08:56:56.524 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.524 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.524 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.525 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:244913 10.b 10:d76135c2:::benchmark_data_smithi092_173109_object107954:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T08:56:56.525 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:07.477743+0000", 2024-09-11T08:56:56.525 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.937667685999998, 2024-09-11T08:56:56.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0641073459999999, 2024-09-11T08:56:56.525 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.525 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:56:56.525 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.525 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:56:56.525 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:56:56.525 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 244913 2024-09-11T08:56:56.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.525 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.477743+0000", 2024-09-11T08:56:56.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.477743+0000", 2024-09-11T08:56:56.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.477742+0000", 2024-09-11T08:56:56.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:56:56.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.477749+0000", 2024-09-11T08:56:56.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4960000000000001e-06 2024-09-11T08:56:56.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.477760+0000", 2024-09-11T08:56:56.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0912e-05 2024-09-11T08:56:56.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.477765+0000", 2024-09-11T08:56:56.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2440000000000001e-06 2024-09-11T08:56:56.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.477813+0000", 2024-09-11T08:56:56.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8217999999999998e-05 2024-09-11T08:56:56.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:56:56.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.477855+0000", 2024-09-11T08:56:56.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2335e-05 2024-09-11T08:56:56.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.529257+0000", 2024-09-11T08:56:56.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0514013010000001 2024-09-11T08:56:56.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:56:56.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.529350+0000", 2024-09-11T08:56:56.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3571000000000002e-05 2024-09-11T08:56:56.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,6", 2024-09-11T08:56:56.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.529819+0000", 2024-09-11T08:56:56.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000468355 2024-09-11T08:56:56.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.540791+0000", 2024-09-11T08:56:56.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010971925 2024-09-11T08:56:56.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.540872+0000", 2024-09-11T08:56:56.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1450000000000001e-05 2024-09-11T08:56:56.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:56:56.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.541716+0000", 2024-09-11T08:56:56.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00084402900000000005 2024-09-11T08:56:56.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:56:56.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.541760+0000", 2024-09-11T08:56:56.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3896999999999997e-05 2024-09-11T08:56:56.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.541851+0000", 2024-09-11T08:56:56.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0853000000000006e-05 2024-09-11T08:56:56.531 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.531 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.531 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.532 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:245440 10.2 10:49481c72:::benchmark_data_smithi092_173109_object108481:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e346)", 2024-09-11T08:56:56.532 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:08.888365+0000", 2024-09-11T08:56:56.532 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.527046425999998, 2024-09-11T08:56:56.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11189455800000001, 2024-09-11T08:56:56.532 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.532 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:56:56.532 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.532 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:56:56.532 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:56:56.532 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 245440 2024-09-11T08:56:56.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.532 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.888365+0000", 2024-09-11T08:56:56.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.888365+0000", 2024-09-11T08:56:56.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.888363+0000", 2024-09-11T08:56:56.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:56:56.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.888370+0000", 2024-09-11T08:56:56.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.477e-06 2024-09-11T08:56:56.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.888381+0000", 2024-09-11T08:56:56.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1853e-05 2024-09-11T08:56:56.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.888387+0000", 2024-09-11T08:56:56.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3789999999999997e-06 2024-09-11T08:56:56.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.888418+0000", 2024-09-11T08:56:56.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0865999999999999e-05 2024-09-11T08:56:56.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:56:56.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.888493+0000", 2024-09-11T08:56:56.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5661000000000006e-05 2024-09-11T08:56:56.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,4", 2024-09-11T08:56:56.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.888567+0000", 2024-09-11T08:56:56.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3994999999999999e-05 2024-09-11T08:56:56.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:56:56.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.890975+0000", 2024-09-11T08:56:56.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024080460000000001 2024-09-11T08:56:56.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.891602+0000", 2024-09-11T08:56:56.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062609799999999995 2024-09-11T08:56:56.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:09.000168+0000", 2024-09-11T08:56:56.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10856674400000001 2024-09-11T08:56:56.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:56:56.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:09.000221+0000", 2024-09-11T08:56:56.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2460999999999998e-05 2024-09-11T08:56:56.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:09.000259+0000", 2024-09-11T08:56:56.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8497000000000001e-05 2024-09-11T08:56:56.537 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.538 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.538 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:245443 10.b 10:dd130e1c:::benchmark_data_smithi092_173109_object108484:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e346)", 2024-09-11T08:56:56.538 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:08.890929+0000", 2024-09-11T08:56:56.538 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.524481862999998, 2024-09-11T08:56:56.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.106367236, 2024-09-11T08:56:56.538 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:56:56.538 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:56:56.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:56:56.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:56:56.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:56:56.538 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 245443 2024-09-11T08:56:56.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.539 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:56:56.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:56:56.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.890929+0000", 2024-09-11T08:56:56.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:56:56.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.890929+0000", 2024-09-11T08:56:56.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:56:56.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:56:56.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.890929+0000", 2024-09-11T08:56:56.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:56:56.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:56:56.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.890932+0000", 2024-09-11T08:56:56.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8970000000000001e-06 2024-09-11T08:56:56.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:56:56.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.890942+0000", 2024-09-11T08:56:56.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4380000000000001e-06 2024-09-11T08:56:56.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:56:56.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.890947+0000", 2024-09-11T08:56:56.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1919999999999996e-06 2024-09-11T08:56:56.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:56:56.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.890986+0000", 2024-09-11T08:56:56.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8909999999999998e-05 2024-09-11T08:56:56.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:56:56.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.891081+0000", 2024-09-11T08:56:56.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5245e-05 2024-09-11T08:56:56.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,6", 2024-09-11T08:56:56.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.891195+0000", 2024-09-11T08:56:56.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114064 2024-09-11T08:56:56.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.993198+0000", 2024-09-11T08:56:56.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102002499 2024-09-11T08:56:56.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:56:56.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.993218+0000", 2024-09-11T08:56:56.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0429e-05 2024-09-11T08:56:56.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:56:56.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.997217+0000", 2024-09-11T08:56:56.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039986409999999998 2024-09-11T08:56:56.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:56:56.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.997263+0000", 2024-09-11T08:56:56.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5877999999999997e-05 2024-09-11T08:56:56.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:56:56.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:56:56.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:56:56.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.997296+0000", 2024-09-11T08:56:56.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3701999999999997e-05 2024-09-11T08:56:56.544 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.544 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.544 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.544 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:56:56.544 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:56:56.544 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:56:56.566 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:56.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:56.567 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:56.667 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:56.689 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:56.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:56.690 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:56.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:56.812 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:56.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:56.813 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:56.851 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:56.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:56.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:56.936 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:56.937 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:57.038 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:57.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:57.061 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:57.061 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:57.162 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:57.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:57.185 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:57.185 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:57.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:57.308 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:57.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:57.309 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:57.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:57 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:56:57.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:57 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:56:57.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:57 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-11T08:56:57.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:57 smithi092 ceph-mon[29647]: osdmap e360: 8 total, 8 up, 8 in 2024-09-11T08:56:57.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:57 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:56:57.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:57 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:56:57.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:57 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-11T08:56:57.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:57 smithi092 ceph-mon[41114]: osdmap e360: 8 total, 8 up, 8 in 2024-09-11T08:56:57.410 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:57.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:57 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:56:57.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:57 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:56:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:57 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-11T08:56:57.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:57 smithi145 ceph-mon[33384]: osdmap e360: 8 total, 8 up, 8 in 2024-09-11T08:56:57.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:56:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:56:57.433 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:57.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:57.434 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:57.535 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:57.557 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:57.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:57.558 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:57.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:57.679 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:57.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:57.680 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:57.781 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:57.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:57.804 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:57.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:57.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:56:57.927 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:57.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:57.928 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:56:58.029 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:58.052 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:58.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:58.053 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:58.154 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:58.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:58.203 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:58.204 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:58.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T08:56:58.304 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:56:58.326 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:58.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:58.327 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:58.411 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:58 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:58.412 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:58 smithi092 ceph-mon[29647]: osdmap e361: 8 total, 8 up, 8 in 2024-09-11T08:56:58.412 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:58 smithi092 ceph-mon[29647]: pgmap v826: 168 pgs: 3 remapped+peering, 7 peering, 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:56:58.412 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:58 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:56:58.412 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:58 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:56:58.413 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:58 smithi092 ceph-mon[29647]: osdmap e362: 8 total, 8 up, 8 in 2024-09-11T08:56:58.413 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:58 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:58.413 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:58 smithi092 ceph-mon[41114]: osdmap e361: 8 total, 8 up, 8 in 2024-09-11T08:56:58.413 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:58 smithi092 ceph-mon[41114]: pgmap v826: 168 pgs: 3 remapped+peering, 7 peering, 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:56:58.413 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:58 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:56:58.414 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:58 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:56:58.414 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:58 smithi092 ceph-mon[41114]: osdmap e362: 8 total, 8 up, 8 in 2024-09-11T08:56:58.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:58 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:56:58.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:58 smithi145 ceph-mon[33384]: osdmap e361: 8 total, 8 up, 8 in 2024-09-11T08:56:58.415 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:58 smithi145 ceph-mon[33384]: pgmap v826: 168 pgs: 3 remapped+peering, 7 peering, 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:56:58.415 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:58 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:56:58.415 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:58 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:56:58.415 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:58 smithi145 ceph-mon[33384]: osdmap e362: 8 total, 8 up, 8 in 2024-09-11T08:56:58.427 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:58.448 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:58.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:58.449 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:58.538 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:56:58.538 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:56:58.549 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:58.571 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:58.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:58.572 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:58.673 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:58.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:58.695 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:58.695 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:58.796 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:58.817 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:58.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:58.818 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:58.919 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:56:58.941 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:58.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:58.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:56:59.043 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:56:59.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:59.065 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:59.065 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:56:59.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:59.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:59.187 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:59.187 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:59.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:56:59.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:59.310 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:59.310 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:56:59.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.15 starting backfill to osd.1 from (0'0,0'0] MAX to 353'17530 2024-09-11T08:56:59.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.11 starting backfill to osd.5 from (0'0,0'0] MAX to 353'17032 2024-09-11T08:56:59.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.10 starting backfill to osd.7 from (0'0,0'0] MAX to 353'17196 2024-09-11T08:56:59.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.14 starting backfill to osd.1 from (0'0,0'0] MAX to 353'16960 2024-09-11T08:56:59.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.12 starting backfill to osd.2 from (0'0,0'0] MAX to 353'17014 2024-09-11T08:56:59.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.17 starting backfill to osd.4 from (0'0,0'0] MAX to 353'17084 2024-09-11T08:56:59.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.16 starting backfill to osd.1 from (0'0,0'0] MAX to 353'19692 2024-09-11T08:56:59.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.19 starting backfill to osd.0 from (0'0,0'0] MAX to 353'17220 2024-09-11T08:56:59.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.13 starting backfill to osd.1 from (0'0,0'0] MAX to 353'21774 2024-09-11T08:56:59.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.18 starting backfill to osd.1 from (0'0,0'0] MAX to 353'17200 2024-09-11T08:56:59.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.15 starting backfill to osd.5 from (0'0,0'0] MAX to 353'17530 2024-09-11T08:56:59.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.16 starting backfill to osd.6 from (0'0,0'0] MAX to 353'19692 2024-09-11T08:56:59.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.12 starting backfill to osd.5 from (0'0,0'0] MAX to 353'17014 2024-09-11T08:56:59.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.14 starting backfill to osd.2 from (0'0,0'0] MAX to 353'16960 2024-09-11T08:56:59.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.13 starting backfill to osd.7 from (0'0,0'0] MAX to 353'21774 2024-09-11T08:56:59.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.19 starting backfill to osd.3 from (0'0,0'0] MAX to 353'17220 2024-09-11T08:56:59.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.18 starting backfill to osd.3 from (0'0,0'0] MAX to 353'17200 2024-09-11T08:56:59.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.12 starting backfill to osd.6 from (0'0,0'0] MAX to 353'17014 2024-09-11T08:56:59.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: 10.19 starting backfill to osd.4 from (0'0,0'0] MAX to 353'17220 2024-09-11T08:56:59.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[29647]: osdmap e363: 8 total, 8 up, 8 in 2024-09-11T08:56:59.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.15 starting backfill to osd.1 from (0'0,0'0] MAX to 353'17530 2024-09-11T08:56:59.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.11 starting backfill to osd.5 from (0'0,0'0] MAX to 353'17032 2024-09-11T08:56:59.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.10 starting backfill to osd.7 from (0'0,0'0] MAX to 353'17196 2024-09-11T08:56:59.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.14 starting backfill to osd.1 from (0'0,0'0] MAX to 353'16960 2024-09-11T08:56:59.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.12 starting backfill to osd.2 from (0'0,0'0] MAX to 353'17014 2024-09-11T08:56:59.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.17 starting backfill to osd.4 from (0'0,0'0] MAX to 353'17084 2024-09-11T08:56:59.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.16 starting backfill to osd.1 from (0'0,0'0] MAX to 353'19692 2024-09-11T08:56:59.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.19 starting backfill to osd.0 from (0'0,0'0] MAX to 353'17220 2024-09-11T08:56:59.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.13 starting backfill to osd.1 from (0'0,0'0] MAX to 353'21774 2024-09-11T08:56:59.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.18 starting backfill to osd.1 from (0'0,0'0] MAX to 353'17200 2024-09-11T08:56:59.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.15 starting backfill to osd.5 from (0'0,0'0] MAX to 353'17530 2024-09-11T08:56:59.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.16 starting backfill to osd.6 from (0'0,0'0] MAX to 353'19692 2024-09-11T08:56:59.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.12 starting backfill to osd.5 from (0'0,0'0] MAX to 353'17014 2024-09-11T08:56:59.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.14 starting backfill to osd.2 from (0'0,0'0] MAX to 353'16960 2024-09-11T08:56:59.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.13 starting backfill to osd.7 from (0'0,0'0] MAX to 353'21774 2024-09-11T08:56:59.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.19 starting backfill to osd.3 from (0'0,0'0] MAX to 353'17220 2024-09-11T08:56:59.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.18 starting backfill to osd.3 from (0'0,0'0] MAX to 353'17200 2024-09-11T08:56:59.318 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.12 starting backfill to osd.6 from (0'0,0'0] MAX to 353'17014 2024-09-11T08:56:59.319 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: 10.19 starting backfill to osd.4 from (0'0,0'0] MAX to 353'17220 2024-09-11T08:56:59.319 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:56:59 smithi092 ceph-mon[41114]: osdmap e363: 8 total, 8 up, 8 in 2024-09-11T08:56:59.411 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:59.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.15 starting backfill to osd.1 from (0'0,0'0] MAX to 353'17530 2024-09-11T08:56:59.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.11 starting backfill to osd.5 from (0'0,0'0] MAX to 353'17032 2024-09-11T08:56:59.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.10 starting backfill to osd.7 from (0'0,0'0] MAX to 353'17196 2024-09-11T08:56:59.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.14 starting backfill to osd.1 from (0'0,0'0] MAX to 353'16960 2024-09-11T08:56:59.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.12 starting backfill to osd.2 from (0'0,0'0] MAX to 353'17014 2024-09-11T08:56:59.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.17 starting backfill to osd.4 from (0'0,0'0] MAX to 353'17084 2024-09-11T08:56:59.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.16 starting backfill to osd.1 from (0'0,0'0] MAX to 353'19692 2024-09-11T08:56:59.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.19 starting backfill to osd.0 from (0'0,0'0] MAX to 353'17220 2024-09-11T08:56:59.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.13 starting backfill to osd.1 from (0'0,0'0] MAX to 353'21774 2024-09-11T08:56:59.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.18 starting backfill to osd.1 from (0'0,0'0] MAX to 353'17200 2024-09-11T08:56:59.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.15 starting backfill to osd.5 from (0'0,0'0] MAX to 353'17530 2024-09-11T08:56:59.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.16 starting backfill to osd.6 from (0'0,0'0] MAX to 353'19692 2024-09-11T08:56:59.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.12 starting backfill to osd.5 from (0'0,0'0] MAX to 353'17014 2024-09-11T08:56:59.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.14 starting backfill to osd.2 from (0'0,0'0] MAX to 353'16960 2024-09-11T08:56:59.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.13 starting backfill to osd.7 from (0'0,0'0] MAX to 353'21774 2024-09-11T08:56:59.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.19 starting backfill to osd.3 from (0'0,0'0] MAX to 353'17220 2024-09-11T08:56:59.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.18 starting backfill to osd.3 from (0'0,0'0] MAX to 353'17200 2024-09-11T08:56:59.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.12 starting backfill to osd.6 from (0'0,0'0] MAX to 353'17014 2024-09-11T08:56:59.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: 10.19 starting backfill to osd.4 from (0'0,0'0] MAX to 353'17220 2024-09-11T08:56:59.414 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:56:59 smithi145 ceph-mon[33384]: osdmap e363: 8 total, 8 up, 8 in 2024-09-11T08:56:59.436 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:59.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:59.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:56:59.538 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:56:59.559 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:59.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:59.560 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:56:59.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:56:59.661 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:56:59.682 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:59.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:59.683 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:56:59.784 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:56:59.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:59.809 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:56:59.809 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:56:59.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:56:59.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:56:59.934 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:56:59.934 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:00.035 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:00.056 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:00.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:00.057 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:00.158 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:00.179 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:00.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:00.180 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:00.281 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:00.302 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:00.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:00.303 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:00.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:00.407 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:00 smithi092 ceph-mon[29647]: pgmap v829: 168 pgs: 3 remapped+peering, 7 peering, 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:57:00.408 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:00 smithi092 ceph-mon[29647]: osdmap e364: 8 total, 8 up, 8 in 2024-09-11T08:57:00.408 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:00 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:57:00.408 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:57:00.408 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:00 smithi092 ceph-mon[41114]: pgmap v829: 168 pgs: 3 remapped+peering, 7 peering, 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:57:00.408 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:00 smithi092 ceph-mon[41114]: osdmap e364: 8 total, 8 up, 8 in 2024-09-11T08:57:00.408 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:00 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:57:00.408 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:57:00.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:00 smithi145 ceph-mon[33384]: pgmap v829: 168 pgs: 3 remapped+peering, 7 peering, 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:57:00.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:00 smithi145 ceph-mon[33384]: osdmap e364: 8 total, 8 up, 8 in 2024-09-11T08:57:00.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:00 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:57:00.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:57:00.428 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:00.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:00.429 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:00.530 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:00.551 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:00.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:00.553 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:00.653 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:00.674 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:00.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:00.675 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:00.776 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:00.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:00.900 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:00.900 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:01.001 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:01.023 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:01.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:01.024 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:01.124 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:01.146 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:01.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:01.148 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:01.248 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:01.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:01.273 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:01.274 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:01.374 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:01.397 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:01.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:01.398 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:01.499 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:01.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:01.523 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:01.523 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:01.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:01 smithi092 ceph-mon[29647]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:57:01.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:57:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:57:01] "GET /metrics HTTP/1.1" 200 224799 "" "Prometheus/2.43.0" 2024-09-11T08:57:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:01 smithi092 ceph-mon[41114]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:57:01.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:01.647 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:01.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:01.648 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:01.749 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:01.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:01.775 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:01.775 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:01.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:01.897 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:01.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:01.898 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:01.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:01 smithi145 ceph-mon[33384]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T08:57:01.999 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:02.025 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:02.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:02.026 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:02.127 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:02.152 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:02.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:02.153 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:02.254 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:02.281 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:02.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:02.282 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:02.383 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:02.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:02.410 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:02.410 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:02.507 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:57:02.509 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'unique_pool_3' removed 2024-09-11T08:57:02.511 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:02.538 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:02.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:02.539 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:02.640 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:02.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:02.663 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:02.664 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:02.764 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:02.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:02.788 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:02.788 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:02.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:02 smithi092 ceph-mon[29647]: pgmap v831: 168 pgs: 3 remapped+peering, 4 peering, 2 active+remapped+backfill_toofull, 159 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 186 MiB/s wr, 5.94k op/s; 2.8 MiB/s, 45 objects/s recovering 2024-09-11T08:57:02.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:02 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2276277181' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:57:02.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:02 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1944531236' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:57:02.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:02 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:57:02.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:02 smithi092 ceph-mon[41114]: pgmap v831: 168 pgs: 3 remapped+peering, 4 peering, 2 active+remapped+backfill_toofull, 159 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 186 MiB/s wr, 5.94k op/s; 2.8 MiB/s, 45 objects/s recovering 2024-09-11T08:57:02.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:02 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2276277181' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:57:02.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:02 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1944531236' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:57:02.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:02 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:57:02.889 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:02.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:57:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:57:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:57:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:02 smithi145 ceph-mon[33384]: pgmap v831: 168 pgs: 3 remapped+peering, 4 peering, 2 active+remapped+backfill_toofull, 159 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 186 MiB/s wr, 5.94k op/s; 2.8 MiB/s, 45 objects/s recovering 2024-09-11T08:57:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:02 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2276277181' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:57:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:02 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1944531236' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:57:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:02 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:57:02.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:02.913 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:02.913 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:03.014 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:03.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:03.042 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:03.042 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:03.143 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:03.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:03.166 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:03.166 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:03.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:03.288 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:03.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:03.289 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:03.390 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:03.412 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:03.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:03.413 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:03.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:03.535 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:03.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:03.536 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:03.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:03 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:57:03.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:03 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2276277181' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:57:03.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:03 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:57:03.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:03 smithi092 ceph-mon[29647]: osdmap e365: 8 total, 8 up, 8 in 2024-09-11T08:57:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:03 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:57:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:03 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2276277181' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:57:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:03 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:57:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:03 smithi092 ceph-mon[41114]: osdmap e365: 8 total, 8 up, 8 in 2024-09-11T08:57:03.637 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:03.658 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:03.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:03.659 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:03.760 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:03.781 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:03.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:03.783 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:03.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:03.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:03.906 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:03.906 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:03.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:03 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:57:03.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:03 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2276277181' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:57:03.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:03 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:57:03.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:03 smithi145 ceph-mon[33384]: osdmap e365: 8 total, 8 up, 8 in 2024-09-11T08:57:04.007 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:04.028 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:04.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:04.030 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:04.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:04.152 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:04.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:04.153 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:04.254 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:04.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:04.276 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:04.277 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:04.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:04.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:04.400 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:04.500 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:04.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:04.523 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:04.523 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:04.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:04.646 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:04.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:04.647 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:04.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:04 smithi092 ceph-mon[29647]: pgmap v833: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 777 B/s rd, 0 op/s 2024-09-11T08:57:04.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:04 smithi092 ceph-mon[29647]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-11T08:57:04.677 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:04 smithi092 ceph-mon[41114]: pgmap v833: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 777 B/s rd, 0 op/s 2024-09-11T08:57:04.678 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:04 smithi092 ceph-mon[41114]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-11T08:57:04.695 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:57:04.695 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:57:04.695 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:57:04.695 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:57:04.707 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:57:04.707 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:57:04.724 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:57:04.725 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:57:04.737 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:57:04.737 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:57:04.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:04.752 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:57:04.753 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:57:04.769 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:57:04.769 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:57:04.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:04.770 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:04.770 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:04.785 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:57:04.785 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:57:04.871 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:04.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:04.894 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:04.895 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:04 smithi145 ceph-mon[33384]: pgmap v833: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 777 B/s rd, 0 op/s 2024-09-11T08:57:04.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:04 smithi145 ceph-mon[33384]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-11T08:57:04.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:05.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:05.018 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:05.018 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:05.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:05.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:05.142 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:05.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:05.243 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:05.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:05.266 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:05.266 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:05.367 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:05.388 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:05.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:05.390 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:05.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:05.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:05.513 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:05.513 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:05.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:05 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2816192938' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:57:05.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:05 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2816192938' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:57:05.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:05.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:05.637 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:05.637 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:05.696 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-11T08:57:05.697 INFO:tasks.radosbench:Beginning radosbench... 2024-09-11T08:57:05.697 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_4 2024-09-11T08:57:05.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool create unique_pool_4 16 2024-09-11T08:57:05.739 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:05.761 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:05.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:05.762 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:05.863 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:05.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:05.885 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:05.885 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:05.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:05 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2816192938' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:57:05.987 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:06.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:06.009 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:06.010 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:06.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:06.132 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:06.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:06.133 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:06.234 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:06.255 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:06.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:06.256 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:06.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:06.379 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:06.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:06.380 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:06.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:06.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:06.503 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:06.503 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:06.604 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:06.624 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:57:06.624 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:57:06.625 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:06.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:06.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:06.727 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:06.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:06.749 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:06.749 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:06.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:06 smithi092 ceph-mon[29647]: pgmap v834: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 648 B/s rd, 0 op/s 2024-09-11T08:57:06.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:06 smithi092 ceph-mon[41114]: pgmap v834: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 648 B/s rd, 0 op/s 2024-09-11T08:57:06.850 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:06.873 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:06.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:06.874 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:06.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:06 smithi145 ceph-mon[33384]: pgmap v834: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 648 B/s rd, 0 op/s 2024-09-11T08:57:06.976 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:06.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:06.998 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:06.998 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:07.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:07.121 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:07.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:07.122 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:07.223 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:07.244 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:07.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:07.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:07.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:07.368 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:07.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:07.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:07.389 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-11T08:57:07.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:57:07 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:57:07.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:07.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:07.493 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:07.493 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:07.594 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:07.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:07.617 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:07.617 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:07.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:07.740 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:07.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:07.741 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:07.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:07.863 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:07.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:07.864 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:07.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:07.986 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:07.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:07.987 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:08.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:08.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:57:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:08.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:08.113 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:08.119 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:08.158 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:08.214 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:08.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:08.239 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:08.239 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:08.340 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:08.362 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:08.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:08.363 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:08.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:08.487 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:08.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:08.488 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:08.589 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:08.613 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:08.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:08.615 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:08.648 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:08 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:08.648 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:08 smithi092 ceph-mon[41114]: pgmap v835: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:57:08.648 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:08 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:08.648 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:08 smithi092 ceph-mon[29647]: pgmap v835: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:57:08.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:08.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:08.738 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:08.839 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:08.864 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:08.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:08.866 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:08 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:08 smithi145 ceph-mon[33384]: pgmap v835: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:57:08.966 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:08.992 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:08.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:08.993 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:09.094 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:09.116 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:09.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:09.117 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:09.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:09.241 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:09.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:09.242 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:09.343 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:09.366 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:09.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:09.367 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:09.468 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:09.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:09.493 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:09.494 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:09.529 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:09.594 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:09.617 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:09.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:09.618 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:09.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:09.741 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:09.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:09.742 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:09.844 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:09.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:09.871 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:09.871 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:09.972 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:09.998 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:09.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:09.999 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:10.100 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:10.109 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:57:10.124 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:10.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:10.125 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:10.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:10.247 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:10.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:10.248 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:10.349 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:10.372 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:10.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:10.373 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:10.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:10 smithi145 ceph-mon[33384]: pgmap v836: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 989 B/s rd, 0 op/s 2024-09-11T08:57:10.472 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:10 smithi092 ceph-mon[41114]: pgmap v836: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 989 B/s rd, 0 op/s 2024-09-11T08:57:10.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:10 smithi092 ceph-mon[29647]: pgmap v836: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 989 B/s rd, 0 op/s 2024-09-11T08:57:10.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:10.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:10.497 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:10.497 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:10.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:10.621 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:10.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:10.622 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:10.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:10.747 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:10.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:10.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:10.849 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:10.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:10.873 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:10.873 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:10.975 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:10.996 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:10.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:10.997 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:11.098 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:11.124 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:11.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:11.125 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:11.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:11.247 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:11.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:11.248 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:11.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:11.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:11.371 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:11.371 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:11.472 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:11.499 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:11.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:11.500 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:11.601 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:11.627 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:11.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:11.628 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:11.729 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:11.743 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:57:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:57:11] "GET /metrics HTTP/1.1" 200 224799 "" "Prometheus/2.43.0" 2024-09-11T08:57:11.754 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:11.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:11.756 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:11.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:11.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:11.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:11.982 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:12.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:12.008 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:12.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:12.109 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:12.131 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:12.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:12.133 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:12.225 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:57:12.225 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:57:12.234 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:12.255 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:12.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:12.256 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:12.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:12.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:12.379 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:12.379 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:12.480 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:12.505 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:12.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:12.506 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:12.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:12.629 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:12.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:12.630 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:12.721 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:12 smithi145 ceph-mon[33384]: pgmap v837: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-11T08:57:12.731 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:12.757 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:12.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:12.758 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:12.789 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:12 smithi092 ceph-mon[29647]: pgmap v837: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-11T08:57:12.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:12 smithi092 ceph-mon[41114]: pgmap v837: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-11T08:57:12.859 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:12.884 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:12.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:12.885 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:12.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:13.007 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:13.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:13.008 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:13.109 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:13.134 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:13.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:13.135 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:13.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:57:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:57:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:57:13.215 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-11T08:57:13.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:13.271 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:13.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:13.272 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:13.373 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:13.398 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:13.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:13.399 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:13.462 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'unique_pool_4' created 2024-09-11T08:57:13.500 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:13.522 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:13.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:13.523 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:13.623 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:13.645 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:13.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:13.646 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:13.747 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:13.773 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:13.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:13.774 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:13.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:13 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1442058852' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-11T08:57:13.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:13 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-11T08:57:13.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:13 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1442058852' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-11T08:57:13.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:13 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-11T08:57:13.875 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:13.901 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:13.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:13.902 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:13.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:13 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1442058852' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-11T08:57:13.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:13 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-11T08:57:14.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:14.026 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:14.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:14.027 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:14.128 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:14.154 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:14.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:14.155 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:14.256 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:14.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:14.278 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:14.279 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:14.379 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:14.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:14.401 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:14.402 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:14.503 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:14.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:14.525 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:14.525 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:14.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:14 smithi092 ceph-mon[41114]: pgmap v838: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-09-11T08:57:14.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:14 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-11T08:57:14.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:14 smithi092 ceph-mon[41114]: osdmap e366: 8 total, 8 up, 8 in 2024-09-11T08:57:14.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:14 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:57:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:14 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:57:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:14 smithi092 ceph-mon[29647]: pgmap v838: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-09-11T08:57:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:14 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-11T08:57:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:14 smithi092 ceph-mon[29647]: osdmap e366: 8 total, 8 up, 8 in 2024-09-11T08:57:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:14 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:57:14.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:14 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:57:14.626 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:14.652 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:14.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:14.653 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:14.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2024-09-11T08:57:14.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:14.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:14.814 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:14.815 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:14.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:14 smithi145 ceph-mon[33384]: pgmap v838: 142 pgs: 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-09-11T08:57:14.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:14 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-11T08:57:14.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:14 smithi145 ceph-mon[33384]: osdmap e366: 8 total, 8 up, 8 in 2024-09-11T08:57:14.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:14 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:57:14.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:14 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:57:14.915 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:14.941 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:14.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:14.942 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:15.043 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:15.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:15.069 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:15.069 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:15.170 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:15.193 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:15.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:15.194 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:15.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:15.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:15.319 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:15.420 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:15.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:15.441 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:15.442 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:15.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:15.564 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:15 smithi092 ceph-mon[29647]: osdmap e367: 8 total, 8 up, 8 in 2024-09-11T08:57:15.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:57:15.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:15.566 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:15.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:15 smithi092 ceph-mon[41114]: osdmap e367: 8 total, 8 up, 8 in 2024-09-11T08:57:15.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:57:15.667 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:15.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:15.691 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:15.691 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:15.792 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:15.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:15.815 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:15.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:15 smithi145 ceph-mon[33384]: osdmap e367: 8 total, 8 up, 8 in 2024-09-11T08:57:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:57:15.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:15.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:15.938 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:15.938 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:16.039 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:16.065 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:16.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:16.066 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:16.167 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:16.192 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:16.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:16.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:16.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:16.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:16.316 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:16.316 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:16.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:16.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:16.440 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:16.440 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:16.541 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:16.567 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:16.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:16.568 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:16.660 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:16.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:16.693 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:16.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:16.694 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:16.795 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:16.796 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:16 smithi092 ceph-mon[29647]: pgmap v841: 158 pgs: 16 unknown, 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:57:16.796 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:16 smithi092 ceph-mon[29647]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:57:16.796 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:16 smithi092 ceph-mon[29647]: osdmap e368: 8 total, 8 up, 8 in 2024-09-11T08:57:16.797 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:16 smithi092 ceph-mon[41114]: pgmap v841: 158 pgs: 16 unknown, 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:57:16.797 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:16 smithi092 ceph-mon[41114]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:57:16.797 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:16 smithi092 ceph-mon[41114]: osdmap e368: 8 total, 8 up, 8 in 2024-09-11T08:57:16.822 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:16.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:16.823 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:16.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:16 smithi145 ceph-mon[33384]: pgmap v841: 158 pgs: 16 unknown, 142 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:57:16.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:16 smithi145 ceph-mon[33384]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:57:16.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:16 smithi145 ceph-mon[33384]: osdmap e368: 8 total, 8 up, 8 in 2024-09-11T08:57:16.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:16.929 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:57:16.930 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:57:16.930 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:57:16.930 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:57:16.930 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:57:16.930 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:57:16.937 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:57:16.937 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:57:16.946 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:16.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:16.947 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:16.954 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:57:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:57:16.970 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:57:16.970 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:57:16.985 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:57:16.985 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:57:17.001 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:57:17.004 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:57:17.048 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:17.073 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:17.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:17.074 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:17.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:17.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:17.197 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:17.225 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:17.298 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:17.323 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:17.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:17.324 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:17.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:57:17 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:57:17.425 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:17.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:17.450 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:17.450 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:17.482 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:57:17.550 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:17.574 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:17.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:17.575 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:17.576 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:17 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1311683918' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:57:17.576 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:17 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:57:17.576 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:17 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2266499031' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:57:17.576 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:17 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1311683918' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:57:17.576 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:17 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:57:17.576 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:17 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2266499031' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:57:17.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:17.698 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:17.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:17.699 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:17.799 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:17.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:17.825 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:17.825 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:17.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:17 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1311683918' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:57:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:17 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:57:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:17 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2266499031' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:57:17.926 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:17.951 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:17.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:17.952 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:18.053 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:18.079 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:18.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:18.080 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:18.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:18.204 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:18.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:18.205 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:18.306 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:18.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:18.333 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:18.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:18.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:18.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:18.457 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:18.457 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:18.558 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:18.583 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:18.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:18.584 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:18.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:18.708 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:18.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:18.709 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:18.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:18 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:18.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:18 smithi092 ceph-mon[29647]: pgmap v843: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:57:18.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:18 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:57:18.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:18 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:57:18.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:18 smithi092 ceph-mon[29647]: osdmap e369: 8 total, 8 up, 8 in 2024-09-11T08:57:18.803 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:18 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:18.803 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:18 smithi092 ceph-mon[41114]: pgmap v843: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:57:18.803 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:18 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:57:18.803 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:18 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:57:18.803 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:18 smithi092 ceph-mon[41114]: osdmap e369: 8 total, 8 up, 8 in 2024-09-11T08:57:18.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:18.831 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:18.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:18.832 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:18.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:18 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:18 smithi145 ceph-mon[33384]: pgmap v843: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:57:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:18 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:57:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:18 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:57:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:18 smithi145 ceph-mon[33384]: osdmap e369: 8 total, 8 up, 8 in 2024-09-11T08:57:18.932 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:18.958 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:18.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:18.959 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:19.060 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:19.083 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:19.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:19.085 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:19.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:19.207 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:19.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:19.209 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:19.310 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:19.336 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:19.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:19.337 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:19.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:19.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:19.461 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:19.461 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:19.562 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:19.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:19 smithi092 ceph-mon[29647]: 5.1c scrub starts 2024-09-11T08:57:19.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:19 smithi092 ceph-mon[29647]: 5.1c scrub ok 2024-09-11T08:57:19.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:19 smithi092 ceph-mon[29647]: 2.1 scrub starts 2024-09-11T08:57:19.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:19 smithi092 ceph-mon[29647]: 2.1 scrub ok 2024-09-11T08:57:19.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:19 smithi092 ceph-mon[29647]: 6.0 scrub starts 2024-09-11T08:57:19.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:19 smithi092 ceph-mon[29647]: 6.0 scrub ok 2024-09-11T08:57:19.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:19 smithi092 ceph-mon[41114]: 5.1c scrub starts 2024-09-11T08:57:19.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:19 smithi092 ceph-mon[41114]: 5.1c scrub ok 2024-09-11T08:57:19.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:19 smithi092 ceph-mon[41114]: 2.1 scrub starts 2024-09-11T08:57:19.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:19 smithi092 ceph-mon[41114]: 2.1 scrub ok 2024-09-11T08:57:19.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:19 smithi092 ceph-mon[41114]: 6.0 scrub starts 2024-09-11T08:57:19.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:19 smithi092 ceph-mon[41114]: 6.0 scrub ok 2024-09-11T08:57:19.588 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:19.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:19.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:19.689 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:19.712 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:19.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:19.713 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:19.813 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:19.840 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:19.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:19.841 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:19.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:19 smithi145 ceph-mon[33384]: 5.1c scrub starts 2024-09-11T08:57:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:19 smithi145 ceph-mon[33384]: 5.1c scrub ok 2024-09-11T08:57:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:19 smithi145 ceph-mon[33384]: 2.1 scrub starts 2024-09-11T08:57:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:19 smithi145 ceph-mon[33384]: 2.1 scrub ok 2024-09-11T08:57:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:19 smithi145 ceph-mon[33384]: 6.0 scrub starts 2024-09-11T08:57:19.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:19 smithi145 ceph-mon[33384]: 6.0 scrub ok 2024-09-11T08:57:19.942 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:19.968 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:19.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:19.969 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:20.030 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:57:20.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:20.095 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:20.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:20.096 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:20.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:20.218 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:20.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:20.219 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:20.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:20.341 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:20.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:20.342 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:20.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:20.465 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:20.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:20.466 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:20.566 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:20.593 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:20.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:20.594 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:20.696 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:20.717 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:20.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:20.718 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:20.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:20 smithi092 ceph-mon[29647]: 5.1 scrub starts 2024-09-11T08:57:20.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:20 smithi092 ceph-mon[29647]: 5.1 scrub ok 2024-09-11T08:57:20.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:20 smithi092 ceph-mon[29647]: 4.3 scrub starts 2024-09-11T08:57:20.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:20 smithi092 ceph-mon[29647]: 4.3 scrub ok 2024-09-11T08:57:20.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:20 smithi092 ceph-mon[29647]: pgmap v845: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-09-11T08:57:20.765 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:20 smithi092 ceph-mon[41114]: 5.1 scrub starts 2024-09-11T08:57:20.766 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:20 smithi092 ceph-mon[41114]: 5.1 scrub ok 2024-09-11T08:57:20.766 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:20 smithi092 ceph-mon[41114]: 4.3 scrub starts 2024-09-11T08:57:20.766 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:20 smithi092 ceph-mon[41114]: 4.3 scrub ok 2024-09-11T08:57:20.766 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:20 smithi092 ceph-mon[41114]: pgmap v845: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-09-11T08:57:20.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:20.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:20.842 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:20.842 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:20.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:20 smithi145 ceph-mon[33384]: 5.1 scrub starts 2024-09-11T08:57:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:20 smithi145 ceph-mon[33384]: 5.1 scrub ok 2024-09-11T08:57:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:20 smithi145 ceph-mon[33384]: 4.3 scrub starts 2024-09-11T08:57:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:20 smithi145 ceph-mon[33384]: 4.3 scrub ok 2024-09-11T08:57:20.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:20 smithi145 ceph-mon[33384]: pgmap v845: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-09-11T08:57:20.942 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:20.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:20.964 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:20.965 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:20.994 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:57:20.994 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-11T08:57:20.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-11T08:57:20.994 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:57:20.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:20.994 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:23113 10.3 10:c4b16d09:::benchmark_data_smithi092_173109_object23112:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e300)", 2024-09-11T08:57:20.994 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:53:12.259973+0000", 2024-09-11T08:57:20.994 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 248.73316375100001, 2024-09-11T08:57:20.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.164834385, 2024-09-11T08:57:20.995 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:20.995 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:20.995 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:57:20.995 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:57:20.995 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:57:20.995 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 23113 2024-09-11T08:57:20.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:20.995 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:20.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:20.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:20.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.259973+0000", 2024-09-11T08:57:20.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:20.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:20.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:20.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:20.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.259973+0000", 2024-09-11T08:57:20.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:20.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:20.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:20.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:20.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.259971+0000", 2024-09-11T08:57:20.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:57:20.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:20.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:20.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:20.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.259993+0000", 2024-09-11T08:57:20.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1376e-05 2024-09-11T08:57:20.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:20.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:20.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:20.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.260003+0000", 2024-09-11T08:57:20.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8349999999999999e-06 2024-09-11T08:57:20.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:20.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:20.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:20.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.260008+0000", 2024-09-11T08:57:20.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0180000000000001e-06 2024-09-11T08:57:20.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:20.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:20.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:20.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.260049+0000", 2024-09-11T08:57:20.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2024-09-11T08:57:20.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:20.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:20.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:20.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.260302+0000", 2024-09-11T08:57:20.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025321600000000001 2024-09-11T08:57:20.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:20.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:20.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3,5", 2024-09-11T08:57:20.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.260425+0000", 2024-09-11T08:57:20.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122454 2024-09-11T08:57:20.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:20.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:20.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:57:20.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.262248+0000", 2024-09-11T08:57:20.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018237629999999999 2024-09-11T08:57:20.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:20.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:20.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:20.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.265556+0000", 2024-09-11T08:57:20.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033081310000000002 2024-09-11T08:57:21.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:21.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.424717+0000", 2024-09-11T08:57:21.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.159161051 2024-09-11T08:57:21.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.424768+0000", 2024-09-11T08:57:21.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0596000000000001e-05 2024-09-11T08:57:21.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.424807+0000", 2024-09-11T08:57:21.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8998000000000002e-05 2024-09-11T08:57:21.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.001 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.001 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:23121 10.3 10:c987efe1:::benchmark_data_smithi092_173109_object23120:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e300)", 2024-09-11T08:57:21.001 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:53:12.262503+0000", 2024-09-11T08:57:21.001 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 248.730633267, 2024-09-11T08:57:21.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16371738399999999, 2024-09-11T08:57:21.002 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.002 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:57:21.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:57:21.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:57:21.002 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 23121 2024-09-11T08:57:21.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.002 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.262503+0000", 2024-09-11T08:57:21.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.262503+0000", 2024-09-11T08:57:21.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.262502+0000", 2024-09-11T08:57:21.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:57:21.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.262523+0000", 2024-09-11T08:57:21.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1211000000000001e-05 2024-09-11T08:57:21.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.262535+0000", 2024-09-11T08:57:21.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1674999999999999e-05 2024-09-11T08:57:21.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.262541+0000", 2024-09-11T08:57:21.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8420000000000001e-06 2024-09-11T08:57:21.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.262579+0000", 2024-09-11T08:57:21.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8197e-05 2024-09-11T08:57:21.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.262685+0000", 2024-09-11T08:57:21.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010631 2024-09-11T08:57:21.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3,5", 2024-09-11T08:57:21.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.262802+0000", 2024-09-11T08:57:21.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116999 2024-09-11T08:57:21.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:57:21.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.263937+0000", 2024-09-11T08:57:21.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011348490000000001 2024-09-11T08:57:21.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:21.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.266148+0000", 2024-09-11T08:57:21.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002211125 2024-09-11T08:57:21.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:21.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.426127+0000", 2024-09-11T08:57:21.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.159978287 2024-09-11T08:57:21.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.426179+0000", 2024-09-11T08:57:21.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1853e-05 2024-09-11T08:57:21.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.426221+0000", 2024-09-11T08:57:21.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1986000000000001e-05 2024-09-11T08:57:21.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.007 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.008 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.008 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:23141 10.5 10:a26f469e:::benchmark_data_smithi092_173109_object23140:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e300)", 2024-09-11T08:57:21.008 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:53:12.268639+0000", 2024-09-11T08:57:21.008 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 248.72449746500001, 2024-09-11T08:57:21.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15798367599999999, 2024-09-11T08:57:21.008 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.008 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.008 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:57:21.008 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:57:21.008 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:57:21.010 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 23141 2024-09-11T08:57:21.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.010 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.268639+0000", 2024-09-11T08:57:21.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.268639+0000", 2024-09-11T08:57:21.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.268637+0000", 2024-09-11T08:57:21.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:57:21.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.268665+0000", 2024-09-11T08:57:21.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7059999999999998e-05 2024-09-11T08:57:21.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.268679+0000", 2024-09-11T08:57:21.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4642e-05 2024-09-11T08:57:21.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.268686+0000", 2024-09-11T08:57:21.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.37e-06 2024-09-11T08:57:21.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.268741+0000", 2024-09-11T08:57:21.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5631000000000003e-05 2024-09-11T08:57:21.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.268875+0000", 2024-09-11T08:57:21.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013393899999999999 2024-09-11T08:57:21.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,3", 2024-09-11T08:57:21.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.268965+0000", 2024-09-11T08:57:21.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0224999999999998e-05 2024-09-11T08:57:21.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:57:21.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.270159+0000", 2024-09-11T08:57:21.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001193424 2024-09-11T08:57:21.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:21.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.270308+0000", 2024-09-11T08:57:21.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014903200000000001 2024-09-11T08:57:21.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:21.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.426541+0000", 2024-09-11T08:57:21.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15623283900000001 2024-09-11T08:57:21.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.426586+0000", 2024-09-11T08:57:21.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5667999999999997e-05 2024-09-11T08:57:21.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.426623+0000", 2024-09-11T08:57:21.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6332999999999999e-05 2024-09-11T08:57:21.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.015 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.016 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:23144 10.5 10:a19ef6fc:::benchmark_data_smithi092_173109_object23143:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e300)", 2024-09-11T08:57:21.016 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:53:12.270682+0000", 2024-09-11T08:57:21.016 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 248.72245466999999, 2024-09-11T08:57:21.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.156656727, 2024-09-11T08:57:21.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:57:21.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:57:21.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:57:21.016 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 23144 2024-09-11T08:57:21.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.017 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.270682+0000", 2024-09-11T08:57:21.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.270682+0000", 2024-09-11T08:57:21.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.270680+0000", 2024-09-11T08:57:21.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:57:21.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.270703+0000", 2024-09-11T08:57:21.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3071e-05 2024-09-11T08:57:21.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.270718+0000", 2024-09-11T08:57:21.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4664000000000001e-05 2024-09-11T08:57:21.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.270724+0000", 2024-09-11T08:57:21.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9290000000000003e-06 2024-09-11T08:57:21.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.270756+0000", 2024-09-11T08:57:21.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1748999999999997e-05 2024-09-11T08:57:21.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.270865+0000", 2024-09-11T08:57:21.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108856 2024-09-11T08:57:21.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,3", 2024-09-11T08:57:21.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.270984+0000", 2024-09-11T08:57:21.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011968600000000001 2024-09-11T08:57:21.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:57:21.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.272256+0000", 2024-09-11T08:57:21.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012716940000000001 2024-09-11T08:57:21.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:21.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.273006+0000", 2024-09-11T08:57:21.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00075040100000000004 2024-09-11T08:57:21.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:21.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.427241+0000", 2024-09-11T08:57:21.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15423453300000001 2024-09-11T08:57:21.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.427304+0000", 2024-09-11T08:57:21.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3286000000000004e-05 2024-09-11T08:57:21.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.427339+0000", 2024-09-11T08:57:21.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4258999999999998e-05 2024-09-11T08:57:21.022 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.022 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.022 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.022 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:23153 10.3 10:c8598d9c:::benchmark_data_smithi092_173109_object23152:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e300)", 2024-09-11T08:57:21.022 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:53:12.273702+0000", 2024-09-11T08:57:21.022 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 248.719434581, 2024-09-11T08:57:21.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15275271000000001, 2024-09-11T08:57:21.022 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.022 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:57:21.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:57:21.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:57:21.023 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 23153 2024-09-11T08:57:21.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.023 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.273702+0000", 2024-09-11T08:57:21.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.273702+0000", 2024-09-11T08:57:21.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.273701+0000", 2024-09-11T08:57:21.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:57:21.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.273719+0000", 2024-09-11T08:57:21.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.836e-05 2024-09-11T08:57:21.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.273732+0000", 2024-09-11T08:57:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3539e-05 2024-09-11T08:57:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.273737+0000", 2024-09-11T08:57:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6639999999999997e-06 2024-09-11T08:57:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.273771+0000", 2024-09-11T08:57:21.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3880000000000001e-05 2024-09-11T08:57:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.273886+0000", 2024-09-11T08:57:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114623 2024-09-11T08:57:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3,5", 2024-09-11T08:57:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.274042+0000", 2024-09-11T08:57:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156095 2024-09-11T08:57:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:57:21.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.275125+0000", 2024-09-11T08:57:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010836509999999999 2024-09-11T08:57:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.275793+0000", 2024-09-11T08:57:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066741299999999995 2024-09-11T08:57:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.426389+0000", 2024-09-11T08:57:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15059610600000001 2024-09-11T08:57:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.426429+0000", 2024-09-11T08:57:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9833000000000003e-05 2024-09-11T08:57:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:12.426455+0000", 2024-09-11T08:57:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5905999999999999e-05 2024-09-11T08:57:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.028 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:37492 10.3 10:c15614bd:::benchmark_data_smithi092_173109_object37491:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e302)", 2024-09-11T08:57:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:53:18.898606+0000", 2024-09-11T08:57:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 242.09453052200001, 2024-09-11T08:57:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16636793499999999, 2024-09-11T08:57:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:57:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:57:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:57:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37492 2024-09-11T08:57:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.898606+0000", 2024-09-11T08:57:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.898606+0000", 2024-09-11T08:57:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.898604+0000", 2024-09-11T08:57:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:57:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.898629+0000", 2024-09-11T08:57:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4547999999999999e-05 2024-09-11T08:57:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.898642+0000", 2024-09-11T08:57:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3623e-05 2024-09-11T08:57:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.898649+0000", 2024-09-11T08:57:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3119999999999999e-06 2024-09-11T08:57:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.898723+0000", 2024-09-11T08:57:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3969999999999998e-05 2024-09-11T08:57:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.898949+0000", 2024-09-11T08:57:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022580899999999999 2024-09-11T08:57:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3,5", 2024-09-11T08:57:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.899080+0000", 2024-09-11T08:57:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000131679 2024-09-11T08:57:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:57:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901035+0000", 2024-09-11T08:57:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019550259999999999 2024-09-11T08:57:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901157+0000", 2024-09-11T08:57:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121808 2024-09-11T08:57:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064876+0000", 2024-09-11T08:57:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16371944799999999 2024-09-11T08:57:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064929+0000", 2024-09-11T08:57:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2876e-05 2024-09-11T08:57:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064974+0000", 2024-09-11T08:57:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4526e-05 2024-09-11T08:57:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:37495 10.8 10:1589e0a2:::benchmark_data_smithi092_173109_object37494:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e302)", 2024-09-11T08:57:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:53:18.899862+0000", 2024-09-11T08:57:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 242.09327474, 2024-09-11T08:57:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.165112126, 2024-09-11T08:57:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:57:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:57:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:57:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37495 2024-09-11T08:57:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.899862+0000", 2024-09-11T08:57:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.899862+0000", 2024-09-11T08:57:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.899861+0000", 2024-09-11T08:57:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:57:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.899884+0000", 2024-09-11T08:57:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3141000000000001e-05 2024-09-11T08:57:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.899894+0000", 2024-09-11T08:57:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.059e-05 2024-09-11T08:57:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.899904+0000", 2024-09-11T08:57:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3589999999999997e-06 2024-09-11T08:57:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.899974+0000", 2024-09-11T08:57:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9870999999999996e-05 2024-09-11T08:57:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.900090+0000", 2024-09-11T08:57:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116585 2024-09-11T08:57:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,7", 2024-09-11T08:57:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.900221+0000", 2024-09-11T08:57:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000131089 2024-09-11T08:57:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:57:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064815+0000", 2024-09-11T08:57:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16459341299999999 2024-09-11T08:57:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064869+0000", 2024-09-11T08:57:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3900000000000002e-05 2024-09-11T08:57:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064895+0000", 2024-09-11T08:57:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6004e-05 2024-09-11T08:57:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064939+0000", 2024-09-11T08:57:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4663000000000003e-05 2024-09-11T08:57:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064974+0000", 2024-09-11T08:57:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4496999999999999e-05 2024-09-11T08:57:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:37500 10.3 10:c00f476a:::benchmark_data_smithi092_173109_object37499:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e302)", 2024-09-11T08:57:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:53:18.901462+0000", 2024-09-11T08:57:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 242.09167433900001, 2024-09-11T08:57:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16379548099999999, 2024-09-11T08:57:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:57:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:57:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:57:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37500 2024-09-11T08:57:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901462+0000", 2024-09-11T08:57:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901462+0000", 2024-09-11T08:57:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901461+0000", 2024-09-11T08:57:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:57:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901480+0000", 2024-09-11T08:57:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9139000000000001e-05 2024-09-11T08:57:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901492+0000", 2024-09-11T08:57:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1647999999999999e-05 2024-09-11T08:57:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901498+0000", 2024-09-11T08:57:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8300000000000001e-06 2024-09-11T08:57:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901536+0000", 2024-09-11T08:57:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8359999999999999e-05 2024-09-11T08:57:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901700+0000", 2024-09-11T08:57:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016369799999999999 2024-09-11T08:57:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3,5", 2024-09-11T08:57:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901899+0000", 2024-09-11T08:57:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019962199999999999 2024-09-11T08:57:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:57:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064992+0000", 2024-09-11T08:57:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.163092922 2024-09-11T08:57:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.065024+0000", 2024-09-11T08:57:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1380000000000001e-05 2024-09-11T08:57:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:57:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.065195+0000", 2024-09-11T08:57:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017097799999999999 2024-09-11T08:57:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.065230+0000", 2024-09-11T08:57:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5048000000000001e-05 2024-09-11T08:57:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.065258+0000", 2024-09-11T08:57:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7973000000000001e-05 2024-09-11T08:57:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.15621.0:37492 10.3 e302/293 ondisk, result = 0)", 2024-09-11T08:57:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:53:18.901962+0000", 2024-09-11T08:57:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 242.091174198, 2024-09-11T08:57:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16302206599999999, 2024-09-11T08:57:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:57:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901962+0000", 2024-09-11T08:57:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901962+0000", 2024-09-11T08:57:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901962+0000", 2024-09-11T08:57:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:57:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901966+0000", 2024-09-11T08:57:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8539999999999996e-06 2024-09-11T08:57:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901977+0000", 2024-09-11T08:57:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0475000000000001e-05 2024-09-11T08:57:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.901983+0000", 2024-09-11T08:57:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6489999999999996e-06 2024-09-11T08:57:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064861+0000", 2024-09-11T08:57:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16287815899999999 2024-09-11T08:57:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064870+0000", 2024-09-11T08:57:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2110000000000004e-06 2024-09-11T08:57:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064984+0000", 2024-09-11T08:57:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011438 2024-09-11T08:57:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.15621.0:37495 10.8 e302/293 ondisk, result = 0)", 2024-09-11T08:57:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:53:18.902855+0000", 2024-09-11T08:57:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 242.09028191199999, 2024-09-11T08:57:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.162015891, 2024-09-11T08:57:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:57:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.902855+0000", 2024-09-11T08:57:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.902855+0000", 2024-09-11T08:57:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.902854+0000", 2024-09-11T08:57:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:57:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.902859+0000", 2024-09-11T08:57:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0239999999999997e-06 2024-09-11T08:57:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064803+0000", 2024-09-11T08:57:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16194476399999999 2024-09-11T08:57:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064811+0000", 2024-09-11T08:57:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7859999999999997e-06 2024-09-11T08:57:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064853+0000", 2024-09-11T08:57:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1844000000000001e-05 2024-09-11T08:57:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064862+0000", 2024-09-11T08:57:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0950000000000002e-06 2024-09-11T08:57:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064870+0000", 2024-09-11T08:57:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2120000000000003e-06 2024-09-11T08:57:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.15621.0:37495 10.8 e302/293 ondisk, result = 0)", 2024-09-11T08:57:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:53:18.902878+0000", 2024-09-11T08:57:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 242.09025811199999, 2024-09-11T08:57:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.162101415, 2024-09-11T08:57:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T08:57:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.902878+0000", 2024-09-11T08:57:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.902878+0000", 2024-09-11T08:57:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.902878+0000", 2024-09-11T08:57:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:57:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.902883+0000", 2024-09-11T08:57:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6189999999999999e-06 2024-09-11T08:57:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064800+0000", 2024-09-11T08:57:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16191657600000001 2024-09-11T08:57:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064812+0000", 2024-09-11T08:57:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1956999999999999e-05 2024-09-11T08:57:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064886+0000", 2024-09-11T08:57:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4055999999999997e-05 2024-09-11T08:57:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064891+0000", 2024-09-11T08:57:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.101e-06 2024-09-11T08:57:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064980+0000", 2024-09-11T08:57:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8906999999999994e-05 2024-09-11T08:57:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15621.0:37498 10.e e302/293 10:77588a8a:::benchmark_data_smithi092_173109_object37497:head v 302'2246, mlcod=302'2244)", 2024-09-11T08:57:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:53:18.903155+0000", 2024-09-11T08:57:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 242.08998194399999, 2024-09-11T08:57:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.167550748, 2024-09-11T08:57:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.903155+0000", 2024-09-11T08:57:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.903155+0000", 2024-09-11T08:57:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.903154+0000", 2024-09-11T08:57:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:57:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:18.903304+0000", 2024-09-11T08:57:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014978699999999999 2024-09-11T08:57:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064807+0000", 2024-09-11T08:57:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16150347900000001 2024-09-11T08:57:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064813+0000", 2024-09-11T08:57:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8159999999999999e-06 2024-09-11T08:57:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064855+0000", 2024-09-11T08:57:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1536999999999999e-05 2024-09-11T08:57:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.064879+0000", 2024-09-11T08:57:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4505999999999999e-05 2024-09-11T08:57:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.070648+0000", 2024-09-11T08:57:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057684520000000003 2024-09-11T08:57:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:53:19.070705+0000", 2024-09-11T08:57:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7726000000000003e-05 2024-09-11T08:57:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:170583 10.3 e330/319 10:c28045be:::benchmark_data_smithi092_173109_object82650:head v 330'12268, mlcod=330'12258)", 2024-09-11T08:57:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:55:10.322295+0000", 2024-09-11T08:57:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 130.67084122399999, 2024-09-11T08:57:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.24968663499999999, 2024-09-11T08:57:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.322295+0000", 2024-09-11T08:57:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.322295+0000", 2024-09-11T08:57:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.322295+0000", 2024-09-11T08:57:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:57:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.322298+0000", 2024-09-11T08:57:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2820000000000001e-06 2024-09-11T08:57:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.322307+0000", 2024-09-11T08:57:21.067 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4319999999999999e-06 2024-09-11T08:57:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.322310+0000", 2024-09-11T08:57:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.326e-06 2024-09-11T08:57:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.326561+0000", 2024-09-11T08:57:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042514670000000001 2024-09-11T08:57:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.326580+0000", 2024-09-11T08:57:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8351000000000001e-05 2024-09-11T08:57:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.571942+0000", 2024-09-11T08:57:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.24536244400000001 2024-09-11T08:57:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.571982+0000", 2024-09-11T08:57:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9604000000000002e-05 2024-09-11T08:57:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:170585 10.3 e330/319 10:c28082bb:::benchmark_data_smithi092_173109_object61121:head v 330'12269, mlcod=330'12258)", 2024-09-11T08:57:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:55:10.328863+0000", 2024-09-11T08:57:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 130.664273471, 2024-09-11T08:57:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.24638299299999999, 2024-09-11T08:57:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.328863+0000", 2024-09-11T08:57:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.328863+0000", 2024-09-11T08:57:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.328862+0000", 2024-09-11T08:57:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:57:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.328872+0000", 2024-09-11T08:57:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0531e-05 2024-09-11T08:57:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.328882+0000", 2024-09-11T08:57:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3219999999999999e-06 2024-09-11T08:57:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.328887+0000", 2024-09-11T08:57:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7599999999999999e-06 2024-09-11T08:57:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.329102+0000", 2024-09-11T08:57:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021458100000000001 2024-09-11T08:57:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.329158+0000", 2024-09-11T08:57:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6162000000000002e-05 2024-09-11T08:57:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.575004+0000", 2024-09-11T08:57:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.245845807 2024-09-11T08:57:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.575246+0000", 2024-09-11T08:57:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024207699999999999 2024-09-11T08:57:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:170586 10.3 e330/319 10:c2809102:::benchmark_data_smithi092_173109_object48376:head v 330'12270, mlcod=330'12258)", 2024-09-11T08:57:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:55:10.328896+0000", 2024-09-11T08:57:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 130.66424042899999, 2024-09-11T08:57:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.24644223100000001, 2024-09-11T08:57:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.328896+0000", 2024-09-11T08:57:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.328896+0000", 2024-09-11T08:57:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.328896+0000", 2024-09-11T08:57:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:57:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.328899+0000", 2024-09-11T08:57:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5719999999999999e-06 2024-09-11T08:57:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.328906+0000", 2024-09-11T08:57:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2269999999999998e-06 2024-09-11T08:57:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.328909+0000", 2024-09-11T08:57:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8029999999999998e-06 2024-09-11T08:57:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.331472+0000", 2024-09-11T08:57:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025624250000000001 2024-09-11T08:57:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.331488+0000", 2024-09-11T08:57:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.626e-05 2024-09-11T08:57:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.575287+0000", 2024-09-11T08:57:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.243798661 2024-09-11T08:57:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.575338+0000", 2024-09-11T08:57:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1581e-05 2024-09-11T08:57:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:170588 10.3 e330/319 10:c281b970:::benchmark_data_smithi092_173109_object58227:head v 330'12271, mlcod=330'12258)", 2024-09-11T08:57:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:55:10.332680+0000", 2024-09-11T08:57:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 130.660456197, 2024-09-11T08:57:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.24537345199999999, 2024-09-11T08:57:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.332680+0000", 2024-09-11T08:57:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.332680+0000", 2024-09-11T08:57:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.332679+0000", 2024-09-11T08:57:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:57:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.332690+0000", 2024-09-11T08:57:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0744000000000001e-05 2024-09-11T08:57:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.332702+0000", 2024-09-11T08:57:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2035000000000001e-05 2024-09-11T08:57:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.332708+0000", 2024-09-11T08:57:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7189999999999998e-06 2024-09-11T08:57:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.572009+0000", 2024-09-11T08:57:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23930094099999999 2024-09-11T08:57:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.572027+0000", 2024-09-11T08:57:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.838e-05 2024-09-11T08:57:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.578023+0000", 2024-09-11T08:57:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059955620000000003 2024-09-11T08:57:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.578054+0000", 2024-09-11T08:57:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1124000000000002e-05 2024-09-11T08:57:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15621.0:180206 10.5 e330/319 10:a20b1f22:::benchmark_data_smithi092_173109_object43247:head v 330'11523, mlcod=330'11521)", 2024-09-11T08:57:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:55:10.333523+0000", 2024-09-11T08:57:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 130.659613229, 2024-09-11T08:57:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.25406857399999999, 2024-09-11T08:57:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.333523+0000", 2024-09-11T08:57:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.333523+0000", 2024-09-11T08:57:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.333522+0000", 2024-09-11T08:57:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:57:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.333531+0000", 2024-09-11T08:57:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1329999999999999e-06 2024-09-11T08:57:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.333543+0000", 2024-09-11T08:57:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2191000000000001e-05 2024-09-11T08:57:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.333549+0000", 2024-09-11T08:57:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8379999999999998e-06 2024-09-11T08:57:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.333617+0000", 2024-09-11T08:57:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8558000000000003e-05 2024-09-11T08:57:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.333637+0000", 2024-09-11T08:57:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0152999999999998e-05 2024-09-11T08:57:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.587353+0000", 2024-09-11T08:57:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.25371543699999999 2024-09-11T08:57:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.587592+0000", 2024-09-11T08:57:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023900000000000001 2024-09-11T08:57:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(osd.5.0:170589 10.3 e330/319 10:c28251d9:::benchmark_data_smithi092_173109_object115886:head v 330'12272, mlcod=330'12258)", 2024-09-11T08:57:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:55:10.335419+0000", 2024-09-11T08:57:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 130.65771735600001, 2024-09-11T08:57:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.25391282999999998, 2024-09-11T08:57:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.335419+0000", 2024-09-11T08:57:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.335419+0000", 2024-09-11T08:57:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.335418+0000", 2024-09-11T08:57:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:57:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.335428+0000", 2024-09-11T08:57:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.73e-06 2024-09-11T08:57:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.571960+0000", 2024-09-11T08:57:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23653206600000001 2024-09-11T08:57:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.571971+0000", 2024-09-11T08:57:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0876000000000001e-05 2024-09-11T08:57:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.575362+0000", 2024-09-11T08:57:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003390924 2024-09-11T08:57:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.575391+0000", 2024-09-11T08:57:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9145e-05 2024-09-11T08:57:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.589287+0000", 2024-09-11T08:57:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01389584 2024-09-11T08:57:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.589332+0000", 2024-09-11T08:57:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5037999999999998e-05 2024-09-11T08:57:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15621.0:180213 10.5 e330/319 10:a310f487:::benchmark_data_smithi092_173109_object43254:head v 330'11524, mlcod=330'11521)", 2024-09-11T08:57:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:55:10.340676+0000", 2024-09-11T08:57:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 130.65246030399999, 2024-09-11T08:57:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.247040759, 2024-09-11T08:57:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.340676+0000", 2024-09-11T08:57:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.340676+0000", 2024-09-11T08:57:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.340675+0000", 2024-09-11T08:57:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:57:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.340685+0000", 2024-09-11T08:57:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9089999999999995e-06 2024-09-11T08:57:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.572536+0000", 2024-09-11T08:57:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23185086599999999 2024-09-11T08:57:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.572542+0000", 2024-09-11T08:57:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3160000000000002e-06 2024-09-11T08:57:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.575171+0000", 2024-09-11T08:57:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026293520000000002 2024-09-11T08:57:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.575191+0000", 2024-09-11T08:57:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0092e-05 2024-09-11T08:57:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.587601+0000", 2024-09-11T08:57:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012409228 2024-09-11T08:57:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.587717+0000", 2024-09-11T08:57:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011636199999999999 2024-09-11T08:57:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15621.0:180218 10.e e330/293 10:730e92fe:::benchmark_data_smithi092_173109_object43259:head v 330'11132, mlcod=330'11130)", 2024-09-11T08:57:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:55:10.343638+0000", 2024-09-11T08:57:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 130.64949874999999, 2024-09-11T08:57:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23190672100000001, 2024-09-11T08:57:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:57:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:57:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:57:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:57:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.343638+0000", 2024-09-11T08:57:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:57:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.343638+0000", 2024-09-11T08:57:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:57:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:57:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.343637+0000", 2024-09-11T08:57:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:57:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:57:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.343646+0000", 2024-09-11T08:57:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0880000000000001e-06 2024-09-11T08:57:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:57:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.572761+0000", 2024-09-11T08:57:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22911537800000001 2024-09-11T08:57:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:57:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.572769+0000", 2024-09-11T08:57:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9030000000000004e-06 2024-09-11T08:57:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:57:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.573015+0000", 2024-09-11T08:57:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024576300000000003 2024-09-11T08:57:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:57:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.573033+0000", 2024-09-11T08:57:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8491e-05 2024-09-11T08:57:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:57:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.575465+0000", 2024-09-11T08:57:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002431651 2024-09-11T08:57:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:57:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:57:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:57:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.575544+0000", 2024-09-11T08:57:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9616999999999993e-05 2024-09-11T08:57:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:57:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:57:21.103 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:57:21.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:21.105 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:21.105 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:21.206 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:21.231 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:21.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:21.232 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:21.333 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:21.359 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:21.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:21.360 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:21.461 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:21.483 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:21.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:21.484 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:21.585 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:21.610 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:21.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:21.611 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:21.711 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:21.735 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:21.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:21.736 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:21.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:21 smithi092 ceph-mon[29647]: 4.9 scrub starts 2024-09-11T08:57:21.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:21 smithi092 ceph-mon[29647]: 4.9 scrub ok 2024-09-11T08:57:21.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:21 smithi092 ceph-mon[29647]: 4.19 scrub starts 2024-09-11T08:57:21.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:21 smithi092 ceph-mon[29647]: 4.19 scrub ok 2024-09-11T08:57:21.816 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:57:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:57:21] "GET /metrics HTTP/1.1" 200 224787 "" "Prometheus/2.43.0" 2024-09-11T08:57:21.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:21 smithi092 ceph-mon[41114]: 4.9 scrub starts 2024-09-11T08:57:21.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:21 smithi092 ceph-mon[41114]: 4.9 scrub ok 2024-09-11T08:57:21.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:21 smithi092 ceph-mon[41114]: 4.19 scrub starts 2024-09-11T08:57:21.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:21 smithi092 ceph-mon[41114]: 4.19 scrub ok 2024-09-11T08:57:21.837 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:21.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:21.861 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:21.861 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:21.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:21 smithi145 ceph-mon[33384]: 4.9 scrub starts 2024-09-11T08:57:21.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:21 smithi145 ceph-mon[33384]: 4.9 scrub ok 2024-09-11T08:57:21.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:21 smithi145 ceph-mon[33384]: 4.19 scrub starts 2024-09-11T08:57:21.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:21 smithi145 ceph-mon[33384]: 4.19 scrub ok 2024-09-11T08:57:21.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:57:21.962 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:21.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:21.986 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:21.986 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:22.087 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:22.110 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:22.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:22.111 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:22.212 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:22.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-11T08:57:22.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:22.236 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:22.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:22.337 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:22.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:22.362 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:22.362 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:22.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:22.485 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:22.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:22.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:22.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:22.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:22.610 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:22.710 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:22.736 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:22.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:22.737 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:22.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[29647]: 3.13 scrub starts 2024-09-11T08:57:22.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[29647]: 3.13 scrub ok 2024-09-11T08:57:22.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[29647]: pgmap v846: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:57:22.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:57:22.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:57:22.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[29647]: osdmap e370: 8 total, 8 up, 8 in 2024-09-11T08:57:22.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[29647]: 6.14 scrub starts 2024-09-11T08:57:22.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[29647]: 6.14 scrub ok 2024-09-11T08:57:22.803 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[41114]: 3.13 scrub starts 2024-09-11T08:57:22.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[41114]: 3.13 scrub ok 2024-09-11T08:57:22.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[41114]: pgmap v846: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:57:22.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:57:22.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:57:22.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[41114]: osdmap e370: 8 total, 8 up, 8 in 2024-09-11T08:57:22.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[41114]: 6.14 scrub starts 2024-09-11T08:57:22.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:22 smithi092 ceph-mon[41114]: 6.14 scrub ok 2024-09-11T08:57:22.810 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:22.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:22.847 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:22.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:22.865 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:22.866 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:22.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:57:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:57:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:57:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:22 smithi145 ceph-mon[33384]: 3.13 scrub starts 2024-09-11T08:57:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:22 smithi145 ceph-mon[33384]: 3.13 scrub ok 2024-09-11T08:57:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:22 smithi145 ceph-mon[33384]: pgmap v846: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:57:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:22 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:57:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:22 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:57:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:22 smithi145 ceph-mon[33384]: osdmap e370: 8 total, 8 up, 8 in 2024-09-11T08:57:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:22 smithi145 ceph-mon[33384]: 6.14 scrub starts 2024-09-11T08:57:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:22 smithi145 ceph-mon[33384]: 6.14 scrub ok 2024-09-11T08:57:22.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:22.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:22.991 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:23.091 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:23.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:23.118 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:23.118 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:23.219 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:23.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:23.246 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:23.246 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:23.347 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:23.373 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:23.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:23.374 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:23.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:23.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:23.500 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:23.500 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:23.601 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:23.627 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:23.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:23.629 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:23.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:23.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:23.753 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:23.753 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:23.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:23 smithi092 ceph-mon[29647]: 6.15 scrub starts 2024-09-11T08:57:23.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:23 smithi092 ceph-mon[29647]: 6.15 scrub ok 2024-09-11T08:57:23.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:23 smithi092 ceph-mon[29647]: 4.b scrub starts 2024-09-11T08:57:23.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:23 smithi092 ceph-mon[29647]: 4.b scrub ok 2024-09-11T08:57:23.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:23 smithi092 ceph-mon[29647]: 5.0 scrub starts 2024-09-11T08:57:23.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:23 smithi092 ceph-mon[29647]: 5.0 scrub ok 2024-09-11T08:57:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:23 smithi092 ceph-mon[41114]: 6.15 scrub starts 2024-09-11T08:57:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:23 smithi092 ceph-mon[41114]: 6.15 scrub ok 2024-09-11T08:57:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:23 smithi092 ceph-mon[41114]: 4.b scrub starts 2024-09-11T08:57:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:23 smithi092 ceph-mon[41114]: 4.b scrub ok 2024-09-11T08:57:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:23 smithi092 ceph-mon[41114]: 5.0 scrub starts 2024-09-11T08:57:23.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:23 smithi092 ceph-mon[41114]: 5.0 scrub ok 2024-09-11T08:57:23.854 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:23.880 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:23.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:23.882 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:23.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:23 smithi145 ceph-mon[33384]: 6.15 scrub starts 2024-09-11T08:57:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:23 smithi145 ceph-mon[33384]: 6.15 scrub ok 2024-09-11T08:57:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:23 smithi145 ceph-mon[33384]: 4.b scrub starts 2024-09-11T08:57:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:23 smithi145 ceph-mon[33384]: 4.b scrub ok 2024-09-11T08:57:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:23 smithi145 ceph-mon[33384]: 5.0 scrub starts 2024-09-11T08:57:23.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:23 smithi145 ceph-mon[33384]: 5.0 scrub ok 2024-09-11T08:57:23.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:24.005 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:24.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:24.006 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:24.107 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:24.133 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:24.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:24.134 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:24.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:24.258 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:24.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:24.259 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:24.360 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:24.386 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:24.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:24.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:24.488 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:24.513 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:24.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:24.514 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:24.538 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:24 smithi092 ceph-mon[29647]: 3.14 scrub starts 2024-09-11T08:57:24.539 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:24 smithi092 ceph-mon[29647]: 3.14 scrub ok 2024-09-11T08:57:24.539 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:24 smithi092 ceph-mon[29647]: pgmap v848: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:57:24.539 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:24 smithi092 ceph-mon[29647]: 6.3 scrub starts 2024-09-11T08:57:24.539 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:24 smithi092 ceph-mon[29647]: 6.3 scrub ok 2024-09-11T08:57:24.539 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:24 smithi092 ceph-mon[29647]: 5.12 scrub starts 2024-09-11T08:57:24.539 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:24 smithi092 ceph-mon[29647]: 5.12 scrub ok 2024-09-11T08:57:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:24 smithi092 ceph-mon[41114]: 3.14 scrub starts 2024-09-11T08:57:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:24 smithi092 ceph-mon[41114]: 3.14 scrub ok 2024-09-11T08:57:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:24 smithi092 ceph-mon[41114]: pgmap v848: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:57:24.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:24 smithi092 ceph-mon[41114]: 6.3 scrub starts 2024-09-11T08:57:24.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:24 smithi092 ceph-mon[41114]: 6.3 scrub ok 2024-09-11T08:57:24.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:24 smithi092 ceph-mon[41114]: 5.12 scrub starts 2024-09-11T08:57:24.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:24 smithi092 ceph-mon[41114]: 5.12 scrub ok 2024-09-11T08:57:24.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:24.637 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:24.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:24.638 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:24.736 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:24.739 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:24.764 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:24.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:24.766 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:24.866 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:24.888 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:24.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:24.889 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:24 smithi145 ceph-mon[33384]: 3.14 scrub starts 2024-09-11T08:57:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:24 smithi145 ceph-mon[33384]: 3.14 scrub ok 2024-09-11T08:57:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:24 smithi145 ceph-mon[33384]: pgmap v848: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:57:24.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:24 smithi145 ceph-mon[33384]: 6.3 scrub starts 2024-09-11T08:57:24.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:24 smithi145 ceph-mon[33384]: 6.3 scrub ok 2024-09-11T08:57:24.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:24 smithi145 ceph-mon[33384]: 5.12 scrub starts 2024-09-11T08:57:24.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:24 smithi145 ceph-mon[33384]: 5.12 scrub ok 2024-09-11T08:57:24.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:25.011 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:25.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:25.013 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:25.114 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:25.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:25.137 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:25.137 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:25.238 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:25.264 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:25.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:25.265 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:25.366 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:25.392 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:25.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:25.393 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:25.494 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:25.515 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:25.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:25.516 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:25.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:25 smithi092 ceph-mon[29647]: 3.d scrub starts 2024-09-11T08:57:25.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:25 smithi092 ceph-mon[29647]: 3.d scrub ok 2024-09-11T08:57:25.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:25 smithi092 ceph-mon[41114]: 3.d scrub starts 2024-09-11T08:57:25.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:25 smithi092 ceph-mon[41114]: 3.d scrub ok 2024-09-11T08:57:25.617 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:25.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:25.645 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:25.645 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:25.746 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:25.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:25.773 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:25.774 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:25.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:25.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:25.899 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:25.900 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:25.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:25 smithi145 ceph-mon[33384]: 3.d scrub starts 2024-09-11T08:57:25.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:25 smithi145 ceph-mon[33384]: 3.d scrub ok 2024-09-11T08:57:26.001 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:26.027 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:26.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:26.028 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:26.128 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:26.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:26.155 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:26.156 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:26.257 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:26.285 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:26.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:26.286 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:26.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:26.412 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:26.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:26.413 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:26.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:26 smithi092 ceph-mon[41114]: 3.17 scrub starts 2024-09-11T08:57:26.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:26 smithi092 ceph-mon[41114]: 3.17 scrub ok 2024-09-11T08:57:26.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:26 smithi092 ceph-mon[41114]: pgmap v849: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:57:26.514 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:26 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1456119340' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:57:26.514 INFO:teuthology.orchestra.run.smithi092.stderr:enabled application 'rados' on pool 'unique_pool_4' 2024-09-11T08:57:26.514 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:26 smithi092 ceph-mon[29647]: 3.17 scrub starts 2024-09-11T08:57:26.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:26 smithi092 ceph-mon[29647]: 3.17 scrub ok 2024-09-11T08:57:26.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:26 smithi092 ceph-mon[29647]: pgmap v849: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:57:26.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:26 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1456119340' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:57:26.515 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:26.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:26.542 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:26.542 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:26.643 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:26.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:26.666 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:26.667 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:26.768 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:26.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:26.795 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:26.795 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:26.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:26 smithi145 ceph-mon[33384]: 3.17 scrub starts 2024-09-11T08:57:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:26 smithi145 ceph-mon[33384]: 3.17 scrub ok 2024-09-11T08:57:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:26 smithi145 ceph-mon[33384]: pgmap v849: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:57:26.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:26 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1456119340' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:57:26.917 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:26.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:26.918 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:27.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:27.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:27.043 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:27.043 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:27.144 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:27.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:27.169 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:27.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:27.270 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:27.292 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:27.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:27.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:27.394 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:27.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:57:27 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:57:27.419 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:27.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:27.420 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:27.521 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:27.546 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:27.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:27.547 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:27.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:27.672 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:27.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:27.673 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:27.774 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:27.798 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:27.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:27.799 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:27.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:27 smithi092 ceph-mon[29647]: 4.8 scrub starts 2024-09-11T08:57:27.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:27 smithi092 ceph-mon[29647]: 4.8 scrub ok 2024-09-11T08:57:27.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:27 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1456119340' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:57:27.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:27 smithi092 ceph-mon[29647]: osdmap e371: 8 total, 8 up, 8 in 2024-09-11T08:57:27.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:27 smithi092 ceph-mon[29647]: 4.6 scrub starts 2024-09-11T08:57:27.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:27 smithi092 ceph-mon[29647]: 4.6 scrub ok 2024-09-11T08:57:27.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:27 smithi092 ceph-mon[41114]: 4.8 scrub starts 2024-09-11T08:57:27.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:27 smithi092 ceph-mon[41114]: 4.8 scrub ok 2024-09-11T08:57:27.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:27 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1456119340' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:57:27.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:27 smithi092 ceph-mon[41114]: osdmap e371: 8 total, 8 up, 8 in 2024-09-11T08:57:27.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:27 smithi092 ceph-mon[41114]: 4.6 scrub starts 2024-09-11T08:57:27.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:27 smithi092 ceph-mon[41114]: 4.6 scrub ok 2024-09-11T08:57:27.900 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:27.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:27 smithi145 ceph-mon[33384]: 4.8 scrub starts 2024-09-11T08:57:27.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:27 smithi145 ceph-mon[33384]: 4.8 scrub ok 2024-09-11T08:57:27.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:27 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1456119340' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:57:27.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:27 smithi145 ceph-mon[33384]: osdmap e371: 8 total, 8 up, 8 in 2024-09-11T08:57:27.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:27 smithi145 ceph-mon[33384]: 4.6 scrub starts 2024-09-11T08:57:27.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:27 smithi145 ceph-mon[33384]: 4.6 scrub ok 2024-09-11T08:57:27.924 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:27.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:27.925 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:28.026 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:28.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:28.051 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:28.051 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:28.152 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:28.176 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:28.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:28.177 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:28.241 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:57:28.242 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:57:28.257 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:57:28.258 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:57:28.273 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:57:28.273 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:57:28.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:28.288 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:57:28.288 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:57:28.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:28.299 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:28.299 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:28.304 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:57:28.304 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:57:28.320 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:57:28.320 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:57:28.335 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:57:28.335 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:57:28.351 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:57:28.352 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:57:28.400 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:28.424 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:28.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:28.425 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:28.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:28.549 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:28.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:28.550 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:28.651 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:28.672 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:28.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:28.673 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:28.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:28.783 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[29647]: 6.7 scrub starts 2024-09-11T08:57:28.783 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[29647]: 6.7 scrub ok 2024-09-11T08:57:28.783 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:28.783 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[29647]: pgmap v851: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:57:28.783 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[29647]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:57:28.783 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[29647]: 6.2 scrub starts 2024-09-11T08:57:28.784 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[29647]: 6.2 scrub ok 2024-09-11T08:57:28.784 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[29647]: 4.16 scrub starts 2024-09-11T08:57:28.784 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[29647]: 4.16 scrub ok 2024-09-11T08:57:28.784 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2703159972' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:57:28.784 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[41114]: 6.7 scrub starts 2024-09-11T08:57:28.784 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[41114]: 6.7 scrub ok 2024-09-11T08:57:28.784 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:28.784 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[41114]: pgmap v851: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:57:28.784 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[41114]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:57:28.784 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[41114]: 6.2 scrub starts 2024-09-11T08:57:28.784 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[41114]: 6.2 scrub ok 2024-09-11T08:57:28.785 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[41114]: 4.16 scrub starts 2024-09-11T08:57:28.785 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[41114]: 4.16 scrub ok 2024-09-11T08:57:28.785 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:28 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2703159972' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:57:28.796 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:28.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:28.797 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:28.813 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:28 smithi145 ceph-mon[33384]: 6.7 scrub starts 2024-09-11T08:57:28.813 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:28 smithi145 ceph-mon[33384]: 6.7 scrub ok 2024-09-11T08:57:28.813 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:28 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:28.813 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:28 smithi145 ceph-mon[33384]: pgmap v851: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:57:28.813 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:28 smithi145 ceph-mon[33384]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:57:28.814 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:28 smithi145 ceph-mon[33384]: 6.2 scrub starts 2024-09-11T08:57:28.814 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:28 smithi145 ceph-mon[33384]: 6.2 scrub ok 2024-09-11T08:57:28.814 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:28 smithi145 ceph-mon[33384]: 4.16 scrub starts 2024-09-11T08:57:28.814 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:28 smithi145 ceph-mon[33384]: 4.16 scrub ok 2024-09-11T08:57:28.814 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:28 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2703159972' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:57:28.897 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:28.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:28.924 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:28.925 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:29.025 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:29.050 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:29.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:29.051 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:29.151 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:29.175 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:29.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:29.176 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:29.277 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:29.282 DEBUG:teuthology.orchestra.run.smithi092:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_4 bench 90 write' 2024-09-11T08:57:29.283 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-11T08:57:29.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:57:29.285 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-11T08:57:29.285 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-11T08:57:29.300 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:29.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:29.301 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:29.330 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:hints = 1 2024-09-11T08:57:29.331 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-11T08:57:29.331 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Object prefix: benchmark_data_smithi092_205252 2024-09-11T08:57:29.331 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:57:29.331 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 0 0 0 0 0 0 - 0 2024-09-11T08:57:29.402 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:29.443 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:29.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:29.444 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:29.536 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:57:29.545 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:29.574 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:29.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:29.576 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:29.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:29.716 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:29.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:29.717 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:29.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:29 smithi092 ceph-mon[41114]: 6.1e scrub starts 2024-09-11T08:57:29.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:29 smithi092 ceph-mon[41114]: 6.1e scrub ok 2024-09-11T08:57:29.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:29 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3735921759' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:57:29.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:29 smithi092 ceph-mon[41114]: 5.14 scrub starts 2024-09-11T08:57:29.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:29 smithi092 ceph-mon[41114]: 5.14 scrub ok 2024-09-11T08:57:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:29 smithi092 ceph-mon[29647]: 6.1e scrub starts 2024-09-11T08:57:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:29 smithi092 ceph-mon[29647]: 6.1e scrub ok 2024-09-11T08:57:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:29 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3735921759' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:57:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:29 smithi092 ceph-mon[29647]: 5.14 scrub starts 2024-09-11T08:57:29.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:29 smithi092 ceph-mon[29647]: 5.14 scrub ok 2024-09-11T08:57:29.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:29.856 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:29.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:29.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:29.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:29 smithi145 ceph-mon[33384]: 6.1e scrub starts 2024-09-11T08:57:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:29 smithi145 ceph-mon[33384]: 6.1e scrub ok 2024-09-11T08:57:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:29 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3735921759' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:57:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:29 smithi145 ceph-mon[33384]: 5.14 scrub starts 2024-09-11T08:57:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:29 smithi145 ceph-mon[33384]: 5.14 scrub ok 2024-09-11T08:57:29.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:29.995 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:29.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:29.996 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:30.097 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:30.127 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:30.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:30.128 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:30.229 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:30.260 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:30.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:30.261 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:30.331 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 1 15 3671 3656 228.488 228.5 0.00496846 0.00436362 2024-09-11T08:57:30.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:30.399 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:30.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:30.400 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:30.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:30.538 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:30.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:30.539 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:30.640 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:30.672 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:30.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:30.673 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:30.774 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:30.805 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:30.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:30.806 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[29647]: pgmap v852: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:57:30.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:57:30.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:57:30.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3735921759' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:57:30.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[29647]: osdmap e372: 8 total, 8 up, 8 in 2024-09-11T08:57:30.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[29647]: 3.19 scrub starts 2024-09-11T08:57:30.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[29647]: 3.19 scrub ok 2024-09-11T08:57:30.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:57:30.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:57:30.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[41114]: pgmap v852: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:57:30.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:57:30.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:57:30.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3735921759' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:57:30.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[41114]: osdmap e372: 8 total, 8 up, 8 in 2024-09-11T08:57:30.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[41114]: 3.19 scrub starts 2024-09-11T08:57:30.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[41114]: 3.19 scrub ok 2024-09-11T08:57:30.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:57:30.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:57:30.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:30.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:30 smithi145 ceph-mon[33384]: pgmap v852: 158 pgs: 158 active+clean; 687 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:57:30.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:30 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:57:30.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:30 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:57:30.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:30 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3735921759' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:57:30.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:30 smithi145 ceph-mon[33384]: osdmap e372: 8 total, 8 up, 8 in 2024-09-11T08:57:30.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:30 smithi145 ceph-mon[33384]: 3.19 scrub starts 2024-09-11T08:57:30.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:30 smithi145 ceph-mon[33384]: 3.19 scrub ok 2024-09-11T08:57:30.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:30 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:57:30.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:57:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:30.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:30.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:31.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:31.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:31.081 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:31.081 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:31.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:57:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:57:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:57:31.182 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:31.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:31.209 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:31.209 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:31.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:31.331 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 2 16 6933 6917 216.141 203.812 0.00449682 0.00461709 2024-09-11T08:57:31.359 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:31.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:31.361 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:31.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:31.501 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:31.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:31.502 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:31.602 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:31.635 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:31.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:31.637 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:31.738 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:31.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:31 smithi092 ceph-mon[29647]: 6.1c scrub starts 2024-09-11T08:57:31.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:31 smithi092 ceph-mon[29647]: 6.1c scrub ok 2024-09-11T08:57:31.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:31 smithi092 ceph-mon[29647]: 3.6 scrub starts 2024-09-11T08:57:31.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:31 smithi092 ceph-mon[29647]: 3.6 scrub ok 2024-09-11T08:57:31.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:31 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T08:57:31.760 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:31 smithi092 ceph-mon[29647]: 4.1c scrub ok 2024-09-11T08:57:31.760 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:57:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:57:31] "GET /metrics HTTP/1.1" 200 224841 "" "Prometheus/2.43.0" 2024-09-11T08:57:31.760 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:31 smithi092 ceph-mon[41114]: 6.1c scrub starts 2024-09-11T08:57:31.760 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:31 smithi092 ceph-mon[41114]: 6.1c scrub ok 2024-09-11T08:57:31.760 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:31 smithi092 ceph-mon[41114]: 3.6 scrub starts 2024-09-11T08:57:31.761 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:31 smithi092 ceph-mon[41114]: 3.6 scrub ok 2024-09-11T08:57:31.761 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:31 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T08:57:31.761 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:31 smithi092 ceph-mon[41114]: 4.1c scrub ok 2024-09-11T08:57:31.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:31.775 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:31.775 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:31.876 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:31.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:31 smithi145 ceph-mon[33384]: 6.1c scrub starts 2024-09-11T08:57:31.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:31 smithi145 ceph-mon[33384]: 6.1c scrub ok 2024-09-11T08:57:31.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:31 smithi145 ceph-mon[33384]: 3.6 scrub starts 2024-09-11T08:57:31.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:31 smithi145 ceph-mon[33384]: 3.6 scrub ok 2024-09-11T08:57:31.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:31 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T08:57:31.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:31 smithi145 ceph-mon[33384]: 4.1c scrub ok 2024-09-11T08:57:31.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:31.915 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:31.915 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:32.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:32.056 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:32.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:32.058 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:32.158 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:32.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-11T08:57:32.192 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:32.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:32.193 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:32.294 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:32.308 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:57:32.325 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:32.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:32.326 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:32.331 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 3 16 10212 10196 212.394 204.938 0.00582663 0.00470058 2024-09-11T08:57:32.427 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:32.494 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:32.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:32.495 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:32.557 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:32.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:32.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:32.649 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:32.649 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:32.750 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:32.773 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:32 smithi092 ceph-mon[29647]: pgmap v854: 158 pgs: 158 active+clean; 180 MiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 22 MiB/s wr, 358 op/s 2024-09-11T08:57:32.773 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:32 smithi092 ceph-mon[29647]: 5.1d deep-scrub starts 2024-09-11T08:57:32.773 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:32 smithi092 ceph-mon[29647]: 5.1d deep-scrub ok 2024-09-11T08:57:32.773 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:32 smithi092 ceph-mon[41114]: pgmap v854: 158 pgs: 158 active+clean; 180 MiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 22 MiB/s wr, 358 op/s 2024-09-11T08:57:32.773 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:32 smithi092 ceph-mon[41114]: 5.1d deep-scrub starts 2024-09-11T08:57:32.773 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:32 smithi092 ceph-mon[41114]: 5.1d deep-scrub ok 2024-09-11T08:57:32.783 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:32.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:32.785 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:32.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:32.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:57:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:57:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:57:32.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:32 smithi145 ceph-mon[33384]: pgmap v854: 158 pgs: 158 active+clean; 180 MiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 22 MiB/s wr, 358 op/s 2024-09-11T08:57:32.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:32 smithi145 ceph-mon[33384]: 5.1d deep-scrub starts 2024-09-11T08:57:32.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:32 smithi145 ceph-mon[33384]: 5.1d deep-scrub ok 2024-09-11T08:57:32.922 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:32.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:32.923 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:33.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:33.069 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:33.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:33.070 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:33.171 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:33.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:33.206 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:33.206 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:33.307 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:33.331 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 4 16 13167 13151 205.464 184.688 0.0066856 0.00486082 2024-09-11T08:57:33.337 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:33.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:33.339 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:33.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:33.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:33.478 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:33.478 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:33.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:33.614 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:33.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:33.615 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:33.699 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:33.717 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:33.739 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:33.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:33.746 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:33.746 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:33.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:33 smithi092 ceph-mon[29647]: 3.1b scrub starts 2024-09-11T08:57:33.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:33 smithi092 ceph-mon[29647]: 3.1b scrub ok 2024-09-11T08:57:33.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:33 smithi092 ceph-mon[29647]: 6.13 scrub starts 2024-09-11T08:57:33.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:33 smithi092 ceph-mon[29647]: 6.13 scrub ok 2024-09-11T08:57:33.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:33 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:57:33.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:33 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:57:33.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:33 smithi092 ceph-mon[41114]: 3.1b scrub starts 2024-09-11T08:57:33.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:33 smithi092 ceph-mon[41114]: 3.1b scrub ok 2024-09-11T08:57:33.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:33 smithi092 ceph-mon[41114]: 6.13 scrub starts 2024-09-11T08:57:33.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:33 smithi092 ceph-mon[41114]: 6.13 scrub ok 2024-09-11T08:57:33.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:33 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:57:33.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:33 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:57:33.846 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:33.876 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:33.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:33.877 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:33.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:33 smithi145 ceph-mon[33384]: 3.1b scrub starts 2024-09-11T08:57:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:33 smithi145 ceph-mon[33384]: 3.1b scrub ok 2024-09-11T08:57:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:33 smithi145 ceph-mon[33384]: 6.13 scrub starts 2024-09-11T08:57:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:33 smithi145 ceph-mon[33384]: 6.13 scrub ok 2024-09-11T08:57:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:33 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:57:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:33 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:57:33.979 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:34.007 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:34.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:34.009 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:34.109 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:34.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:34.151 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:34.252 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:34.283 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:34.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:34.285 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:34.331 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 5 16 16083 16067 200.818 182.25 0.00362638 0.00497432 2024-09-11T08:57:34.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:34.426 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:34.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:34.427 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:34.528 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:34.558 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:34.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:34.559 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:34.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:34.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:34 smithi092 ceph-mon[29647]: pgmap v855: 158 pgs: 158 active+clean; 474 MiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 948 op/s 2024-09-11T08:57:34.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:34 smithi092 ceph-mon[29647]: 4.d scrub starts 2024-09-11T08:57:34.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:34 smithi092 ceph-mon[29647]: 4.d scrub ok 2024-09-11T08:57:34.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:34 smithi092 ceph-mon[29647]: osdmap e373: 8 total, 8 up, 8 in 2024-09-11T08:57:34.685 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:34 smithi092 ceph-mon[29647]: osdmap e374: 8 total, 8 up, 8 in 2024-09-11T08:57:34.685 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:34 smithi092 ceph-mon[41114]: pgmap v855: 158 pgs: 158 active+clean; 474 MiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 948 op/s 2024-09-11T08:57:34.685 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:34 smithi092 ceph-mon[41114]: 4.d scrub starts 2024-09-11T08:57:34.685 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:34 smithi092 ceph-mon[41114]: 4.d scrub ok 2024-09-11T08:57:34.685 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:34 smithi092 ceph-mon[41114]: osdmap e373: 8 total, 8 up, 8 in 2024-09-11T08:57:34.685 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:34 smithi092 ceph-mon[41114]: osdmap e374: 8 total, 8 up, 8 in 2024-09-11T08:57:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:34.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:34.700 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:34.801 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:34.835 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:34.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:34.836 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:34.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:34 smithi145 ceph-mon[33384]: pgmap v855: 158 pgs: 158 active+clean; 474 MiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 948 op/s 2024-09-11T08:57:34.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:34 smithi145 ceph-mon[33384]: 4.d scrub starts 2024-09-11T08:57:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:34 smithi145 ceph-mon[33384]: 4.d scrub ok 2024-09-11T08:57:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:34 smithi145 ceph-mon[33384]: osdmap e373: 8 total, 8 up, 8 in 2024-09-11T08:57:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:34 smithi145 ceph-mon[33384]: osdmap e374: 8 total, 8 up, 8 in 2024-09-11T08:57:34.937 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:34.968 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:34.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:34.969 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:35.070 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:35.106 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:35.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:35.107 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:35.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:35.245 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:35.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:35.246 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:35.331 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 6 15 18948 18933 197.2 179.125 0.00537 0.00506538 2024-09-11T08:57:35.347 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:35.377 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:35.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:35.378 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:35.479 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:35.509 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:35.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:35.511 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:35.611 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:35.643 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:35.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:35.645 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:35.746 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:35.777 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:35.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:35.778 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:35.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[29647]: 6.10 scrub starts 2024-09-11T08:57:35.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[29647]: 6.10 scrub ok 2024-09-11T08:57:35.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[29647]: 3.1a scrub starts 2024-09-11T08:57:35.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[29647]: 3.1a scrub ok 2024-09-11T08:57:35.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[29647]: 5.17 scrub starts 2024-09-11T08:57:35.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[29647]: 5.17 scrub ok 2024-09-11T08:57:35.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[29647]: 4.a deep-scrub starts 2024-09-11T08:57:35.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[29647]: 4.a deep-scrub ok 2024-09-11T08:57:35.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[41114]: 6.10 scrub starts 2024-09-11T08:57:35.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[41114]: 6.10 scrub ok 2024-09-11T08:57:35.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[41114]: 3.1a scrub starts 2024-09-11T08:57:35.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[41114]: 3.1a scrub ok 2024-09-11T08:57:35.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[41114]: 5.17 scrub starts 2024-09-11T08:57:35.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[41114]: 5.17 scrub ok 2024-09-11T08:57:35.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[41114]: 4.a deep-scrub starts 2024-09-11T08:57:35.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:35 smithi092 ceph-mon[41114]: 4.a deep-scrub ok 2024-09-11T08:57:35.879 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:35.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:35 smithi145 ceph-mon[33384]: 6.10 scrub starts 2024-09-11T08:57:35.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:35 smithi145 ceph-mon[33384]: 6.10 scrub ok 2024-09-11T08:57:35.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:35 smithi145 ceph-mon[33384]: 3.1a scrub starts 2024-09-11T08:57:35.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:35 smithi145 ceph-mon[33384]: 3.1a scrub ok 2024-09-11T08:57:35.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:35 smithi145 ceph-mon[33384]: 5.17 scrub starts 2024-09-11T08:57:35.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:35 smithi145 ceph-mon[33384]: 5.17 scrub ok 2024-09-11T08:57:35.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:35 smithi145 ceph-mon[33384]: 4.a deep-scrub starts 2024-09-11T08:57:35.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:35 smithi145 ceph-mon[33384]: 4.a deep-scrub ok 2024-09-11T08:57:35.917 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:35.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:35.918 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:36.018 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:36.057 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:36.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:36.058 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:36.159 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:36.192 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:36.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:36.194 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:36.294 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:36.326 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:36.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:36.327 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:36.331 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 7 16 21700 21684 193.589 171.938 0.0110837 0.00515975 2024-09-11T08:57:36.428 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:36.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:36.454 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:36.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:36.555 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:36.587 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:36.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:36.588 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:36.689 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:36.719 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:36.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:36.720 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:36 smithi092 ceph-mon[29647]: pgmap v858: 158 pgs: 158 active+clean; 474 MiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 852 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-11T08:57:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:36 smithi092 ceph-mon[29647]: 3.18 scrub starts 2024-09-11T08:57:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:36 smithi092 ceph-mon[29647]: 3.18 scrub ok 2024-09-11T08:57:36.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:36 smithi092 ceph-mon[41114]: pgmap v858: 158 pgs: 158 active+clean; 474 MiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 852 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-11T08:57:36.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:36 smithi092 ceph-mon[41114]: 3.18 scrub starts 2024-09-11T08:57:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:36 smithi092 ceph-mon[41114]: 3.18 scrub ok 2024-09-11T08:57:36.821 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:36.842 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:36.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:36.844 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:36.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:36 smithi145 ceph-mon[33384]: pgmap v858: 158 pgs: 158 active+clean; 474 MiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 852 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-11T08:57:36.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:36 smithi145 ceph-mon[33384]: 3.18 scrub starts 2024-09-11T08:57:36.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:36 smithi145 ceph-mon[33384]: 3.18 scrub ok 2024-09-11T08:57:36.944 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:36.978 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:36.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:36.979 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:37.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:37.111 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:37.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:37.113 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:37.213 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:37.252 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:37.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:37.253 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:37.331 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 8 16 24302 24286 189.717 162.625 0.00505868 0.00526547 2024-09-11T08:57:37.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:37.390 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:37.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:37.391 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:37.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:57:37 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:57:37.492 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:37.526 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:37.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:37.527 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:37.628 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:37.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:37.665 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:37.665 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:37.766 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:37.798 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:37.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:37.799 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:37.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:37.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:37 smithi145 ceph-mon[33384]: 5.19 deep-scrub starts 2024-09-11T08:57:37.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:37 smithi145 ceph-mon[33384]: 5.19 deep-scrub ok 2024-09-11T08:57:37.921 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:37.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:37.922 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:38.023 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:38.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:38.062 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:38.063 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:38.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:37 smithi092 ceph-mon[29647]: 5.19 deep-scrub starts 2024-09-11T08:57:38.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:37 smithi092 ceph-mon[29647]: 5.19 deep-scrub ok 2024-09-11T08:57:38.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:37 smithi092 ceph-mon[41114]: 5.19 deep-scrub starts 2024-09-11T08:57:38.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:37 smithi092 ceph-mon[41114]: 5.19 deep-scrub ok 2024-09-11T08:57:38.164 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:38.187 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:38.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:38.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:38.289 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:38.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:38.325 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:38.325 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:38.331 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 9 15 26539 26524 184.178 139.875 0.00728121 0.00542416 2024-09-11T08:57:38.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:38.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:38.463 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:38.463 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:38.564 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:38.601 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:38.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:38.602 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:38.703 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:38.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:38.725 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:38.725 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:38.826 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:38.861 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:38.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:38.862 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:38.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:38 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:38.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:38 smithi145 ceph-mon[33384]: pgmap v859: 158 pgs: 158 active+clean; 1.4 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.3 KiB/s rd, 176 MiB/s wr, 2.82k op/s 2024-09-11T08:57:38.957 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:38 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:38.957 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:38 smithi092 ceph-mon[29647]: pgmap v859: 158 pgs: 158 active+clean; 1.4 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.3 KiB/s rd, 176 MiB/s wr, 2.82k op/s 2024-09-11T08:57:38.958 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:38 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:38.958 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:38 smithi092 ceph-mon[41114]: pgmap v859: 158 pgs: 158 active+clean; 1.4 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.3 KiB/s rd, 176 MiB/s wr, 2.82k op/s 2024-09-11T08:57:38.963 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:38.984 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:38.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:38.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:39.086 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:39.114 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:39.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:39.115 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:39.216 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:39.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:39.248 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:39.248 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:39.332 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 10 16 28420 28404 177.509 117.5 0.0074403 0.0056292 2024-09-11T08:57:39.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:39.385 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:39.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:39.386 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:39.487 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:39.520 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:39.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:39.521 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:39.622 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:39.655 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:39.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:39.656 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:39.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:39.778 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:39.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:39.779 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:39.880 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:39.912 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:39.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:39.913 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:39.960 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T08:57:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T08:57:40.014 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:40.042 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:40.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:40.043 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:40.144 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:40.175 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:40.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:40.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:40.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:40.313 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:40.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:40.314 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:40.332 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 11 16 30406 30390 172.655 124.125 0.00868477 0.00578644 2024-09-11T08:57:40.405 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:40 smithi092 ceph-mon[29647]: 6.18 scrub starts 2024-09-11T08:57:40.406 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:40 smithi092 ceph-mon[29647]: 6.18 scrub ok 2024-09-11T08:57:40.406 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:40 smithi092 ceph-mon[29647]: osdmap e375: 8 total, 8 up, 8 in 2024-09-11T08:57:40.406 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:40 smithi092 ceph-mon[29647]: from='client.25763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:57:40.406 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:40 smithi092 ceph-mon[29647]: pgmap v861: 158 pgs: 158 active+clean; 1.4 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 152 MiB/s wr, 2.44k op/s 2024-09-11T08:57:40.406 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:40 smithi092 ceph-mon[41114]: 6.18 scrub starts 2024-09-11T08:57:40.406 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:40 smithi092 ceph-mon[41114]: 6.18 scrub ok 2024-09-11T08:57:40.406 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:40 smithi092 ceph-mon[41114]: osdmap e375: 8 total, 8 up, 8 in 2024-09-11T08:57:40.406 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:40 smithi092 ceph-mon[41114]: from='client.25763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:57:40.406 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:40 smithi092 ceph-mon[41114]: pgmap v861: 158 pgs: 158 active+clean; 1.4 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 152 MiB/s wr, 2.44k op/s 2024-09-11T08:57:40.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:40 smithi145 ceph-mon[33384]: 6.18 scrub starts 2024-09-11T08:57:40.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:40 smithi145 ceph-mon[33384]: 6.18 scrub ok 2024-09-11T08:57:40.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:40 smithi145 ceph-mon[33384]: osdmap e375: 8 total, 8 up, 8 in 2024-09-11T08:57:40.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:40 smithi145 ceph-mon[33384]: from='client.25763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:57:40.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:40 smithi145 ceph-mon[33384]: pgmap v861: 158 pgs: 158 active+clean; 1.4 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 152 MiB/s wr, 2.44k op/s 2024-09-11T08:57:40.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:40.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:40.451 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:40.451 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:40.552 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:40.585 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:40.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:40.586 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:40.687 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:40.717 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:40.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:40.718 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:40.819 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:40.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:40.846 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:40.846 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:40.947 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:40.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:40.985 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:40.985 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:41.086 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:41.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:41.144 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:41.244 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:41.277 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:41.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:41.278 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:41.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:41 smithi092 ceph-mon[29647]: 6.12 scrub starts 2024-09-11T08:57:41.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:41 smithi092 ceph-mon[29647]: 6.12 scrub ok 2024-09-11T08:57:41.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:41 smithi092 ceph-mon[29647]: 2.2 scrub starts 2024-09-11T08:57:41.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:41 smithi092 ceph-mon[29647]: 2.2 scrub ok 2024-09-11T08:57:41.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:41 smithi092 ceph-mon[29647]: 6.8 scrub starts 2024-09-11T08:57:41.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:41 smithi092 ceph-mon[29647]: 6.8 scrub ok 2024-09-11T08:57:41.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:41 smithi092 ceph-mon[41114]: 6.12 scrub starts 2024-09-11T08:57:41.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:41 smithi092 ceph-mon[41114]: 6.12 scrub ok 2024-09-11T08:57:41.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:41 smithi092 ceph-mon[41114]: 2.2 scrub starts 2024-09-11T08:57:41.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:41 smithi092 ceph-mon[41114]: 2.2 scrub ok 2024-09-11T08:57:41.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:41 smithi092 ceph-mon[41114]: 6.8 scrub starts 2024-09-11T08:57:41.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:41 smithi092 ceph-mon[41114]: 6.8 scrub ok 2024-09-11T08:57:41.332 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 12 16 32413 32397 168.719 125.438 0.0128716 0.00592078 2024-09-11T08:57:41.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:41.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:41 smithi145 ceph-mon[33384]: 6.12 scrub starts 2024-09-11T08:57:41.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:41 smithi145 ceph-mon[33384]: 6.12 scrub ok 2024-09-11T08:57:41.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:41 smithi145 ceph-mon[33384]: 2.2 scrub starts 2024-09-11T08:57:41.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:41 smithi145 ceph-mon[33384]: 2.2 scrub ok 2024-09-11T08:57:41.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:41 smithi145 ceph-mon[33384]: 6.8 scrub starts 2024-09-11T08:57:41.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:41 smithi145 ceph-mon[33384]: 6.8 scrub ok 2024-09-11T08:57:41.418 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:41.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:41.420 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:41.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:41.557 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:41.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:41.558 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:41.659 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:41.679 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:41.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:41.680 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:41.760 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:57:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:57:41] "GET /metrics HTTP/1.1" 200 224841 "" "Prometheus/2.43.0" 2024-09-11T08:57:41.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:41.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:41.819 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:41.920 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:41.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:41.956 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:41.957 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:42.058 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:42.079 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:42.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:42.080 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:42.181 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:42.214 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:42.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:42.215 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:42.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:42.333 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 13 16 33896 33880 162.85 92.6875 0.00396232 0.00613309 2024-09-11T08:57:42.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:42.353 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:42.353 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:42.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:42 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:57:42.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:42 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:57:42.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:42 smithi145 ceph-mon[33384]: 5.a scrub starts 2024-09-11T08:57:42.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:42 smithi145 ceph-mon[33384]: 5.a scrub ok 2024-09-11T08:57:42.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:42 smithi145 ceph-mon[33384]: 3.1f deep-scrub starts 2024-09-11T08:57:42.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:42 smithi145 ceph-mon[33384]: 3.1f deep-scrub ok 2024-09-11T08:57:42.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:42 smithi145 ceph-mon[33384]: osdmap e376: 8 total, 8 up, 8 in 2024-09-11T08:57:42.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:42 smithi145 ceph-mon[33384]: pgmap v863: 158 pgs: 158 active+clean; 1.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.4 KiB/s rd, 166 MiB/s wr, 2.66k op/s 2024-09-11T08:57:42.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:42 smithi145 ceph-mon[33384]: 1.7 scrub starts 2024-09-11T08:57:42.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:42 smithi145 ceph-mon[33384]: 1.7 scrub ok 2024-09-11T08:57:42.454 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:42.470 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:57:42.470 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:57:42.470 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[29647]: 5.a scrub starts 2024-09-11T08:57:42.470 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[29647]: 5.a scrub ok 2024-09-11T08:57:42.470 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[29647]: 3.1f deep-scrub starts 2024-09-11T08:57:42.470 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[29647]: 3.1f deep-scrub ok 2024-09-11T08:57:42.470 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[29647]: osdmap e376: 8 total, 8 up, 8 in 2024-09-11T08:57:42.470 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[29647]: pgmap v863: 158 pgs: 158 active+clean; 1.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.4 KiB/s rd, 166 MiB/s wr, 2.66k op/s 2024-09-11T08:57:42.470 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[29647]: 1.7 scrub starts 2024-09-11T08:57:42.470 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[29647]: 1.7 scrub ok 2024-09-11T08:57:42.471 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:57:42.471 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:57:42.471 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[41114]: 5.a scrub starts 2024-09-11T08:57:42.471 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[41114]: 5.a scrub ok 2024-09-11T08:57:42.471 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[41114]: 3.1f deep-scrub starts 2024-09-11T08:57:42.471 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[41114]: 3.1f deep-scrub ok 2024-09-11T08:57:42.471 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[41114]: osdmap e376: 8 total, 8 up, 8 in 2024-09-11T08:57:42.471 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[41114]: pgmap v863: 158 pgs: 158 active+clean; 1.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.4 KiB/s rd, 166 MiB/s wr, 2.66k op/s 2024-09-11T08:57:42.471 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[41114]: 1.7 scrub starts 2024-09-11T08:57:42.471 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:42 smithi092 ceph-mon[41114]: 1.7 scrub ok 2024-09-11T08:57:42.474 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:42.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:42.475 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:42.576 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:42.597 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:42.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:42.598 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:42.699 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:42.729 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:42.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:42.730 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:42.831 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:42.855 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:42.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:42.856 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:42.957 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:42.978 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:42.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:42.979 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:43.080 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:43.100 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:43.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:43.101 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:43.134 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:57:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:57:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:57:43.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:43.236 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:43.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:43.237 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:43.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:43 smithi092 ceph-mon[29647]: 4.15 scrub starts 2024-09-11T08:57:43.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:43 smithi092 ceph-mon[29647]: 4.15 scrub ok 2024-09-11T08:57:43.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:43 smithi092 ceph-mon[41114]: 4.15 scrub starts 2024-09-11T08:57:43.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:43 smithi092 ceph-mon[41114]: 4.15 scrub ok 2024-09-11T08:57:43.333 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 14 16 35301 35285 157.491 87.8125 0.00875893 0.00634432 2024-09-11T08:57:43.338 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:43.369 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:43.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:43.370 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:43.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:43 smithi145 ceph-mon[33384]: 4.15 scrub starts 2024-09-11T08:57:43.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:43 smithi145 ceph-mon[33384]: 4.15 scrub ok 2024-09-11T08:57:43.471 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:43.500 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:43.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:43.501 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:43.602 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:43.623 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:43.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:43.624 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:43.725 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:43.759 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:43.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:43.760 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:43.861 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:43.884 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:43.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:43.885 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:43.986 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:44.017 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:44.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:44.018 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:44.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:44.155 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:44.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:44.156 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:44.257 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:44.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:44.291 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:44.291 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:44.333 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 15 16 36920 36904 153.737 101.188 0.00279422 0.00649846 2024-09-11T08:57:44.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:44.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:44 smithi145 ceph-mon[33384]: 5.9 scrub starts 2024-09-11T08:57:44.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:44 smithi145 ceph-mon[33384]: 5.9 scrub ok 2024-09-11T08:57:44.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:44 smithi145 ceph-mon[33384]: pgmap v864: 158 pgs: 158 active+clean; 2.0 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 196 MiB/s wr, 3.14k op/s 2024-09-11T08:57:44.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:44 smithi145 ceph-mon[33384]: osdmap e377: 8 total, 8 up, 8 in 2024-09-11T08:57:44.429 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:44.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:44.430 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:44.531 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:44.563 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:44.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:44.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:44.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:44 smithi092 ceph-mon[29647]: 5.9 scrub starts 2024-09-11T08:57:44.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:44 smithi092 ceph-mon[29647]: 5.9 scrub ok 2024-09-11T08:57:44.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:44 smithi092 ceph-mon[29647]: pgmap v864: 158 pgs: 158 active+clean; 2.0 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 196 MiB/s wr, 3.14k op/s 2024-09-11T08:57:44.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:44 smithi092 ceph-mon[29647]: osdmap e377: 8 total, 8 up, 8 in 2024-09-11T08:57:44.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:44 smithi092 ceph-mon[41114]: 5.9 scrub starts 2024-09-11T08:57:44.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:44 smithi092 ceph-mon[41114]: 5.9 scrub ok 2024-09-11T08:57:44.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:44 smithi092 ceph-mon[41114]: pgmap v864: 158 pgs: 158 active+clean; 2.0 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 196 MiB/s wr, 3.14k op/s 2024-09-11T08:57:44.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:44 smithi092 ceph-mon[41114]: osdmap e377: 8 total, 8 up, 8 in 2024-09-11T08:57:44.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:44.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:44.705 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:44.705 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:44.806 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:44.834 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:44.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:44.835 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:44.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:44.979 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:44.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:44.980 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:45.081 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:45.111 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:45.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:45.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:45.213 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:45.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:45.251 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:45.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:45 smithi092 ceph-mon[29647]: 4.1a scrub starts 2024-09-11T08:57:45.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:45 smithi092 ceph-mon[29647]: 4.1a scrub ok 2024-09-11T08:57:45.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:45 smithi092 ceph-mon[29647]: 1.1 scrub starts 2024-09-11T08:57:45.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:45 smithi092 ceph-mon[29647]: 1.1 scrub ok 2024-09-11T08:57:45.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:57:45.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:45 smithi092 ceph-mon[41114]: 4.1a scrub starts 2024-09-11T08:57:45.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:45 smithi092 ceph-mon[41114]: 4.1a scrub ok 2024-09-11T08:57:45.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:45 smithi092 ceph-mon[41114]: 1.1 scrub starts 2024-09-11T08:57:45.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:45 smithi092 ceph-mon[41114]: 1.1 scrub ok 2024-09-11T08:57:45.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:57:45.334 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 16 15 38935 38920 151.997 126 0.00321237 0.00657163 2024-09-11T08:57:45.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:45.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:45.396 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:45.396 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:45.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:45 smithi145 ceph-mon[33384]: 4.1a scrub starts 2024-09-11T08:57:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:45 smithi145 ceph-mon[33384]: 4.1a scrub ok 2024-09-11T08:57:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:45 smithi145 ceph-mon[33384]: 1.1 scrub starts 2024-09-11T08:57:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:45 smithi145 ceph-mon[33384]: 1.1 scrub ok 2024-09-11T08:57:45.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:57:45.496 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:45.525 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:45.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:45.526 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:45.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:45.664 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:45.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:45.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:45.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:45.803 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:45.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:45.804 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:45.824 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:57:45.824 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:57:45.824 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:57:45.824 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:57:45.825 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:57:45.825 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:57:45.825 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:57:45.825 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:57:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:57:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:57:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:57:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:57:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:57:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:57:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:57:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:57:45.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:45.940 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:45.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:45.941 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:46.042 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:46.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:46.087 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:46.087 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:46.188 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:46.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:46.229 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:46.229 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:46.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:46.335 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 17 16 40761 40745 149.76 114.062 0.00456508 0.00667139 2024-09-11T08:57:46.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:46.370 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:46.370 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:46.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:46 smithi145 ceph-mon[33384]: 6.1a scrub starts 2024-09-11T08:57:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:46 smithi145 ceph-mon[33384]: 6.1a scrub ok 2024-09-11T08:57:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:46 smithi145 ceph-mon[33384]: pgmap v866: 158 pgs: 158 active+clean; 2.0 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 810 B/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-11T08:57:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:46 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1045365465' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:57:46.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:46 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:57:46.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:46.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:46.502 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:46 smithi092 ceph-mon[29647]: 6.1a scrub starts 2024-09-11T08:57:46.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:46 smithi092 ceph-mon[29647]: 6.1a scrub ok 2024-09-11T08:57:46.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:46 smithi092 ceph-mon[29647]: pgmap v866: 158 pgs: 158 active+clean; 2.0 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 810 B/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-11T08:57:46.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:46 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1045365465' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:57:46.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:46 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:57:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:46 smithi092 ceph-mon[41114]: 6.1a scrub starts 2024-09-11T08:57:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:46 smithi092 ceph-mon[41114]: 6.1a scrub ok 2024-09-11T08:57:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:46 smithi092 ceph-mon[41114]: pgmap v866: 158 pgs: 158 active+clean; 2.0 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 810 B/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-11T08:57:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:46 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1045365465' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:57:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:46 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:57:46.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:46.623 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:46.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:46.624 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:46.725 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:46.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:46.763 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:46.763 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:46.864 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:46.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:46.902 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:46.902 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:47.003 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:47.038 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:47.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:47.039 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:47.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:47.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:47.189 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:47.190 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:47.290 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:47.322 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:47.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:47.323 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:47.335 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 18 16 42423 42407 147.21 103.875 0.00433967 0.00678835 2024-09-11T08:57:47.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:57:47 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:57:47.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:47 smithi145 ceph-mon[33384]: 6.9 scrub starts 2024-09-11T08:57:47.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:47 smithi145 ceph-mon[33384]: 6.9 scrub ok 2024-09-11T08:57:47.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:47 smithi145 ceph-mon[33384]: 1.6 scrub starts 2024-09-11T08:57:47.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:47 smithi145 ceph-mon[33384]: 1.6 scrub ok 2024-09-11T08:57:47.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:47 smithi145 ceph-mon[33384]: 6.1d scrub starts 2024-09-11T08:57:47.413 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:47 smithi145 ceph-mon[33384]: 6.1d scrub ok 2024-09-11T08:57:47.424 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:47.425 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:47 smithi092 ceph-mon[29647]: 6.9 scrub starts 2024-09-11T08:57:47.425 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:47 smithi092 ceph-mon[29647]: 6.9 scrub ok 2024-09-11T08:57:47.426 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:47 smithi092 ceph-mon[29647]: 1.6 scrub starts 2024-09-11T08:57:47.426 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:47 smithi092 ceph-mon[29647]: 1.6 scrub ok 2024-09-11T08:57:47.426 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:47 smithi092 ceph-mon[29647]: 6.1d scrub starts 2024-09-11T08:57:47.426 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:47 smithi092 ceph-mon[29647]: 6.1d scrub ok 2024-09-11T08:57:47.427 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:47 smithi092 ceph-mon[41114]: 6.9 scrub starts 2024-09-11T08:57:47.427 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:47 smithi092 ceph-mon[41114]: 6.9 scrub ok 2024-09-11T08:57:47.427 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:47 smithi092 ceph-mon[41114]: 1.6 scrub starts 2024-09-11T08:57:47.427 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:47 smithi092 ceph-mon[41114]: 1.6 scrub ok 2024-09-11T08:57:47.427 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:47 smithi092 ceph-mon[41114]: 6.1d scrub starts 2024-09-11T08:57:47.428 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:47 smithi092 ceph-mon[41114]: 6.1d scrub ok 2024-09-11T08:57:47.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:47.466 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:47.466 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:47.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:47.605 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:47.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:47.606 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:47.708 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:47.742 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:47.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:47.743 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:47.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:47.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:47.890 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:47.890 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:47.991 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:48.012 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:57:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:57:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:57:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:57:48.012 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:57:48.019 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:48.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:48.020 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:48.121 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:48.146 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:48.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:48.147 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:48.247 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:48.277 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:48.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:48.278 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:48.335 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 19 16 44249 44233 145.469 114.125 0.00769426 0.00686929 2024-09-11T08:57:48.379 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:48.412 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:48.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:48.413 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:48.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[29647]: 1.0 scrub starts 2024-09-11T08:57:48.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[29647]: 1.0 scrub ok 2024-09-11T08:57:48.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[29647]: 6.1b scrub starts 2024-09-11T08:57:48.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[29647]: 6.1b scrub ok 2024-09-11T08:57:48.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:48.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[29647]: pgmap v867: 158 pgs: 158 active+clean; 2.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 147 MiB/s wr, 2.35k op/s 2024-09-11T08:57:48.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:57:48.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:57:48.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:57:48.460 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[41114]: 1.0 scrub starts 2024-09-11T08:57:48.460 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[41114]: 1.0 scrub ok 2024-09-11T08:57:48.460 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[41114]: 6.1b scrub starts 2024-09-11T08:57:48.460 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[41114]: 6.1b scrub ok 2024-09-11T08:57:48.460 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:48.460 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[41114]: pgmap v867: 158 pgs: 158 active+clean; 2.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 147 MiB/s wr, 2.35k op/s 2024-09-11T08:57:48.461 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:57:48.461 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:57:48.461 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:48 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:57:48.514 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:48.544 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:48.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:48.546 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:48.647 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:48 smithi145 ceph-mon[33384]: 1.0 scrub starts 2024-09-11T08:57:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:48 smithi145 ceph-mon[33384]: 1.0 scrub ok 2024-09-11T08:57:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:48 smithi145 ceph-mon[33384]: 6.1b scrub starts 2024-09-11T08:57:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:48 smithi145 ceph-mon[33384]: 6.1b scrub ok 2024-09-11T08:57:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:48 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:48 smithi145 ceph-mon[33384]: pgmap v867: 158 pgs: 158 active+clean; 2.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 147 MiB/s wr, 2.35k op/s 2024-09-11T08:57:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:48 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:57:48.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:48 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:57:48.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:48 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:57:48.681 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:48.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:48.682 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:48.783 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:48.818 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:48.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:48.819 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:48.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:48.930 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":860,"stamp":"2024-09-11T08:57:39.092713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1455209512,"num_objects":22390,"num_object_clones":0,"num_object_copies":67170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22390,"num_whiteouts":0,"num_read":1308,"num_read_kb":1080,"num_write":22930,"num_write_kb":1420755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":112737,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23959,"ondisk_log_size":23959,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":11205024,"kb_used_data":4015044,"kb_used_omap":13,"kb_used_meta":7189810,"kb_avail":738756192,"statfs":{"total":767960285184,"available":756486340608,"internally_reserved":0,"allocated":4111405056,"data_stored":4108472628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":7362366144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1266941952,"num_objects":19332,"num_object_clones":0,"num_object_copies":57996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19332,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":19332,"num_write_kb":1237248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.691432"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":192,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548723+0000","last_change":"2024-09-11T08:54:18.331227+0000","last_active":"2024-09-11T08:57:33.548723+0000","last_peered":"2024-09-11T08:57:33.548723+0000","last_clean":"2024-09-11T08:57:33.548723+0000","last_became_active":"2024-09-11T08:52:49.568416+0000","last_became_peered":"2024-09-11T08:52:49.568416+0000","last_unstale":"2024-09-11T08:57:33.548723+0000","last_undegraded":"2024-09-11T08:57:33.548723+0000","last_fullsized":"2024-09-11T08:57:33.548723+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:18.331155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:18.331155+0000","last_clean_scrub_stamp":"2024-09-11T08:54:18.331155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042788669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"200'19","reported_seq":454,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546319+0000","last_change":"2024-09-11T08:55:55.640259+0000","last_active":"2024-09-11T08:57:33.546319+0000","last_peered":"2024-09-11T08:57:33.546319+0000","last_clean":"2024-09-11T08:57:33.546319+0000","last_became_active":"2024-09-11T08:46:01.981032+0000","last_became_peered":"2024-09-11T08:46:01.981032+0000","last_unstale":"2024-09-11T08:57:33.546319+0000","last_undegraded":"2024-09-11T08:57:33.546319+0000","last_fullsized":"2024-09-11T08:57:33.546319+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:55:55.640174+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:55:55.640174+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045891150000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":412,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.908085+0000","last_change":"2024-09-11T08:57:36.908045+0000","last_active":"2024-09-11T08:57:36.908085+0000","last_peered":"2024-09-11T08:57:36.908085+0000","last_clean":"2024-09-11T08:57:36.908085+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T08:57:36.908085+0000","last_undegraded":"2024-09-11T08:57:36.908085+0000","last_fullsized":"2024-09-11T08:57:36.908085+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:36.907966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:36.907966+0000","last_clean_scrub_stamp":"2024-09-11T08:57:36.907966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:37.195150+0000","scrub_duration":0.0032222729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":381,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546792+0000","last_change":"2024-09-11T08:56:25.459604+0000","last_active":"2024-09-11T08:57:33.546792+0000","last_peered":"2024-09-11T08:57:33.546792+0000","last_clean":"2024-09-11T08:57:33.546792+0000","last_became_active":"2024-09-11T08:56:25.459294+0000","last_became_peered":"2024-09-11T08:56:25.459294+0000","last_unstale":"2024-09-11T08:57:33.546792+0000","last_undegraded":"2024-09-11T08:57:33.546792+0000","last_fullsized":"2024-09-11T08:57:33.546792+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:13.811054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:13.811054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010996769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":400,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.550276+0000","last_change":"2024-09-11T08:54:58.730290+0000","last_active":"2024-09-11T08:57:33.550276+0000","last_peered":"2024-09-11T08:57:33.550276+0000","last_clean":"2024-09-11T08:57:33.550276+0000","last_became_active":"2024-09-11T08:46:01.983782+0000","last_became_peered":"2024-09-11T08:46:01.983782+0000","last_unstale":"2024-09-11T08:57:33.550276+0000","last_undegraded":"2024-09-11T08:57:33.550276+0000","last_fullsized":"2024-09-11T08:57:33.550276+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:58.730211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:58.730211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011911339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":406,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.553146+0000","last_change":"2024-09-11T08:54:52.562148+0000","last_active":"2024-09-11T08:57:33.553146+0000","last_peered":"2024-09-11T08:57:33.553146+0000","last_clean":"2024-09-11T08:57:33.553146+0000","last_became_active":"2024-09-11T08:46:01.981646+0000","last_became_peered":"2024-09-11T08:46:01.981646+0000","last_unstale":"2024-09-11T08:57:33.553146+0000","last_undegraded":"2024-09-11T08:57:33.553146+0000","last_fullsized":"2024-09-11T08:57:33.553146+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:52.562065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:54:52.562065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074997070000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"200'34","reported_seq":496,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.558009+0000","last_change":"2024-09-11T08:57:20.749529+0000","last_active":"2024-09-11T08:57:33.558009+0000","last_peered":"2024-09-11T08:57:33.558009+0000","last_clean":"2024-09-11T08:57:33.558009+0000","last_became_active":"2024-09-11T08:52:18.585600+0000","last_became_peered":"2024-09-11T08:52:18.585600+0000","last_unstale":"2024-09-11T08:57:33.558009+0000","last_undegraded":"2024-09-11T08:57:33.558009+0000","last_fullsized":"2024-09-11T08:57:33.558009+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:57:20.749472+0000","last_deep_scrub":"200'34","last_deep_scrub_stamp":"2024-09-11T08:51:31.358660+0000","last_clean_scrub_stamp":"2024-09-11T08:57:20.749472+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:49.188458+0000","scrub_duration":0.0017730809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":373,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546083+0000","last_change":"2024-09-11T08:54:55.729780+0000","last_active":"2024-09-11T08:57:33.546083+0000","last_peered":"2024-09-11T08:57:33.546083+0000","last_clean":"2024-09-11T08:57:33.546083+0000","last_became_active":"2024-09-11T08:38:42.983294+0000","last_became_peered":"2024-09-11T08:38:42.983294+0000","last_unstale":"2024-09-11T08:57:33.546083+0000","last_undegraded":"2024-09-11T08:57:33.546083+0000","last_fullsized":"2024-09-11T08:57:33.546083+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:55.729679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:54:55.729679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002263007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":412,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547813+0000","last_change":"2024-09-11T08:54:10.466581+0000","last_active":"2024-09-11T08:57:33.547813+0000","last_peered":"2024-09-11T08:57:33.547813+0000","last_clean":"2024-09-11T08:57:33.547813+0000","last_became_active":"2024-09-11T08:46:01.980179+0000","last_became_peered":"2024-09-11T08:46:01.980179+0000","last_unstale":"2024-09-11T08:57:33.547813+0000","last_undegraded":"2024-09-11T08:57:33.547813+0000","last_fullsized":"2024-09-11T08:57:33.547813+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:10.466478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:54:10.466478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034798429999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"200'19","reported_seq":454,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547010+0000","last_change":"2024-09-11T08:54:09.865650+0000","last_active":"2024-09-11T08:57:33.547010+0000","last_peered":"2024-09-11T08:57:33.547010+0000","last_clean":"2024-09-11T08:57:33.547010+0000","last_became_active":"2024-09-11T08:52:18.577411+0000","last_became_peered":"2024-09-11T08:52:18.577411+0000","last_unstale":"2024-09-11T08:57:33.547010+0000","last_undegraded":"2024-09-11T08:57:33.547010+0000","last_fullsized":"2024-09-11T08:57:33.547010+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:54:09.865565+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.865565+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044543600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":418,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547249+0000","last_change":"2024-09-11T08:56:39.280164+0000","last_active":"2024-09-11T08:57:33.547249+0000","last_peered":"2024-09-11T08:57:33.547249+0000","last_clean":"2024-09-11T08:57:33.547249+0000","last_became_active":"2024-09-11T08:52:18.573461+0000","last_became_peered":"2024-09-11T08:52:18.573461+0000","last_unstale":"2024-09-11T08:57:33.547249+0000","last_undegraded":"2024-09-11T08:57:33.547249+0000","last_fullsized":"2024-09-11T08:57:33.547249+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:39.280109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:56:39.280109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:56.706222+0000","scrub_duration":0.0013476390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":202,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.551906+0000","last_change":"2024-09-11T08:56:29.480301+0000","last_active":"2024-09-11T08:57:33.551906+0000","last_peered":"2024-09-11T08:57:33.551906+0000","last_clean":"2024-09-11T08:57:33.551906+0000","last_became_active":"2024-09-11T08:52:49.568147+0000","last_became_peered":"2024-09-11T08:52:49.568147+0000","last_unstale":"2024-09-11T08:57:33.551906+0000","last_undegraded":"2024-09-11T08:57:33.551906+0000","last_fullsized":"2024-09-11T08:57:33.551906+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:29.480218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:33.520462+0000","last_clean_scrub_stamp":"2024-09-11T08:56:29.480218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040366990000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":392,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548155+0000","last_change":"2024-09-11T08:54:17.338878+0000","last_active":"2024-09-11T08:57:33.548155+0000","last_peered":"2024-09-11T08:57:33.548155+0000","last_clean":"2024-09-11T08:57:33.548155+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T08:57:33.548155+0000","last_undegraded":"2024-09-11T08:57:33.548155+0000","last_fullsized":"2024-09-11T08:57:33.548155+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:17.338793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:30.465003+0000","last_clean_scrub_stamp":"2024-09-11T08:54:17.338793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028270790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"200'7","reported_seq":407,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547236+0000","last_change":"2024-09-11T08:56:25.474365+0000","last_active":"2024-09-11T08:57:33.547236+0000","last_peered":"2024-09-11T08:57:33.547236+0000","last_clean":"2024-09-11T08:57:33.547236+0000","last_became_active":"2024-09-11T08:56:25.465945+0000","last_became_peered":"2024-09-11T08:56:25.465945+0000","last_unstale":"2024-09-11T08:57:33.547236+0000","last_undegraded":"2024-09-11T08:57:33.547236+0000","last_fullsized":"2024-09-11T08:57:33.547236+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'7","last_scrub_stamp":"2024-09-11T08:55:35.725836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:35.725836+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043511380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":411,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.555628+0000","last_change":"2024-09-11T08:56:33.417300+0000","last_active":"2024-09-11T08:57:33.555628+0000","last_peered":"2024-09-11T08:57:33.555628+0000","last_clean":"2024-09-11T08:57:33.555628+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T08:57:33.555628+0000","last_undegraded":"2024-09-11T08:57:33.555628+0000","last_fullsized":"2024-09-11T08:57:33.555628+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:33.417247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:18.199651+0000","last_clean_scrub_stamp":"2024-09-11T08:56:33.417247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:59.296071+0000","scrub_duration":0.0011470479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":386,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546654+0000","last_change":"2024-09-11T08:55:13.401842+0000","last_active":"2024-09-11T08:57:33.546654+0000","last_peered":"2024-09-11T08:57:33.546654+0000","last_clean":"2024-09-11T08:57:33.546654+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T08:57:33.546654+0000","last_undegraded":"2024-09-11T08:57:33.546654+0000","last_fullsized":"2024-09-11T08:57:33.546654+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:13.401759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:55:13.401759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070886450000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":401,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548362+0000","last_change":"2024-09-11T08:57:28.557687+0000","last_active":"2024-09-11T08:57:33.548362+0000","last_peered":"2024-09-11T08:57:33.548362+0000","last_clean":"2024-09-11T08:57:33.548362+0000","last_became_active":"2024-09-11T08:46:01.977861+0000","last_became_peered":"2024-09-11T08:46:01.977861+0000","last_unstale":"2024-09-11T08:57:33.548362+0000","last_undegraded":"2024-09-11T08:57:33.548362+0000","last_fullsized":"2024-09-11T08:57:33.548362+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:28.557623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:57:28.557623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:50.552914+0000","scrub_duration":0.001331053,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":202,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546962+0000","last_change":"2024-09-11T08:57:32.534621+0000","last_active":"2024-09-11T08:57:33.546962+0000","last_peered":"2024-09-11T08:57:33.546962+0000","last_clean":"2024-09-11T08:57:33.546962+0000","last_became_active":"2024-09-11T08:52:49.566575+0000","last_became_peered":"2024-09-11T08:52:49.566575+0000","last_unstale":"2024-09-11T08:57:33.546962+0000","last_undegraded":"2024-09-11T08:57:33.546962+0000","last_fullsized":"2024-09-11T08:57:33.546962+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:32.534529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:57:32.534529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:59.338604+0000","scrub_duration":0.0066860009999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"200'34","reported_seq":351,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.551980+0000","last_change":"2024-09-11T08:57:30.860560+0000","last_active":"2024-09-11T08:57:33.551980+0000","last_peered":"2024-09-11T08:57:33.551980+0000","last_clean":"2024-09-11T08:57:33.551980+0000","last_became_active":"2024-09-11T08:54:17.121459+0000","last_became_peered":"2024-09-11T08:54:17.121459+0000","last_unstale":"2024-09-11T08:57:33.551980+0000","last_undegraded":"2024-09-11T08:57:33.551980+0000","last_fullsized":"2024-09-11T08:57:33.551980+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:57:30.860466+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T08:57:30.860466+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:42.737844+0000","scrub_duration":0.006483768,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":432,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.552672+0000","last_change":"2024-09-11T08:57:31.860994+0000","last_active":"2024-09-11T08:57:33.552672+0000","last_peered":"2024-09-11T08:57:33.552672+0000","last_clean":"2024-09-11T08:57:33.552672+0000","last_became_active":"2024-09-11T08:52:18.579026+0000","last_became_peered":"2024-09-11T08:52:18.579026+0000","last_unstale":"2024-09-11T08:57:33.552672+0000","last_undegraded":"2024-09-11T08:57:33.552672+0000","last_fullsized":"2024-09-11T08:57:33.552672+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:31.860874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:31.860874+0000","last_clean_scrub_stamp":"2024-09-11T08:57:31.860874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:42.153138+0000","scrub_duration":0.0037104849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'1","reported_seq":409,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.553381+0000","last_change":"2024-09-11T08:55:50.827754+0000","last_active":"2024-09-11T08:57:33.553381+0000","last_peered":"2024-09-11T08:57:33.553381+0000","last_clean":"2024-09-11T08:57:33.553381+0000","last_became_active":"2024-09-11T08:46:01.986647+0000","last_became_peered":"2024-09-11T08:46:01.986647+0000","last_unstale":"2024-09-11T08:57:33.553381+0000","last_undegraded":"2024-09-11T08:57:33.553381+0000","last_fullsized":"2024-09-11T08:57:33.553381+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:55:50.827483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:55:50.827483+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082684079999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":387,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:34.558353+0000","last_change":"2024-09-11T08:57:34.558305+0000","last_active":"2024-09-11T08:57:34.558353+0000","last_peered":"2024-09-11T08:57:34.558353+0000","last_clean":"2024-09-11T08:57:34.558353+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T08:57:34.558353+0000","last_undegraded":"2024-09-11T08:57:34.558353+0000","last_fullsized":"2024-09-11T08:57:34.558353+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:34.558209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:57:34.558209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:41.683955+0000","scrub_duration":0.0040844999999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"200'18","reported_seq":438,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.552439+0000","last_change":"2024-09-11T08:56:35.815453+0000","last_active":"2024-09-11T08:57:33.552439+0000","last_peered":"2024-09-11T08:57:33.552439+0000","last_clean":"2024-09-11T08:57:33.552439+0000","last_became_active":"2024-09-11T08:49:16.943886+0000","last_became_peered":"2024-09-11T08:49:16.943886+0000","last_unstale":"2024-09-11T08:57:33.552439+0000","last_undegraded":"2024-09-11T08:57:33.552439+0000","last_fullsized":"2024-09-11T08:57:33.552439+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'18","last_scrub_stamp":"2024-09-11T08:56:35.815348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:56:35.815348+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:38.311041+0000","scrub_duration":0.0021465519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":383,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548049+0000","last_change":"2024-09-11T08:57:17.636602+0000","last_active":"2024-09-11T08:57:33.548049+0000","last_peered":"2024-09-11T08:57:33.548049+0000","last_clean":"2024-09-11T08:57:33.548049+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T08:57:33.548049+0000","last_undegraded":"2024-09-11T08:57:33.548049+0000","last_fullsized":"2024-09-11T08:57:33.548049+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:17.636528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:57:17.636528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:33.412071+0000","scrub_duration":0.0039989719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":129,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547987+0000","last_change":"2024-09-11T08:57:30.044915+0000","last_active":"2024-09-11T08:57:33.547987+0000","last_peered":"2024-09-11T08:57:33.547987+0000","last_clean":"2024-09-11T08:57:33.547987+0000","last_became_active":"2024-09-11T08:52:38.483603+0000","last_became_peered":"2024-09-11T08:52:38.483603+0000","last_unstale":"2024-09-11T08:57:33.547987+0000","last_undegraded":"2024-09-11T08:57:33.547987+0000","last_fullsized":"2024-09-11T08:57:33.547987+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:57:30.044827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:57:30.044827+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:49.364752+0000","scrub_duration":0.0031105490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":416,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.551796+0000","last_change":"2024-09-11T08:57:29.815444+0000","last_active":"2024-09-11T08:57:33.551796+0000","last_peered":"2024-09-11T08:57:33.551796+0000","last_clean":"2024-09-11T08:57:33.551796+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T08:57:33.551796+0000","last_undegraded":"2024-09-11T08:57:33.551796+0000","last_fullsized":"2024-09-11T08:57:33.551796+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:29.815361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T08:57:29.815361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:55.355275+0000","scrub_duration":0.01006806,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"280'25","reported_seq":135,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.552934+0000","last_change":"2024-09-11T08:55:09.606768+0000","last_active":"2024-09-11T08:57:33.552934+0000","last_peered":"2024-09-11T08:57:33.552934+0000","last_clean":"2024-09-11T08:57:33.552934+0000","last_became_active":"2024-09-11T08:52:49.568074+0000","last_became_peered":"2024-09-11T08:52:49.568074+0000","last_unstale":"2024-09-11T08:57:33.552934+0000","last_undegraded":"2024-09-11T08:57:33.552934+0000","last_fullsized":"2024-09-11T08:57:33.552934+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"280'25","last_scrub_stamp":"2024-09-11T08:55:09.606687+0000","last_deep_scrub":"280'25","last_deep_scrub_stamp":"2024-09-11T08:52:24.584533+0000","last_clean_scrub_stamp":"2024-09-11T08:55:09.606687+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005718791,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"346'9","reported_seq":82,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548362+0000","last_change":"2024-09-11T08:56:08.482980+0000","last_active":"2024-09-11T08:57:33.548362+0000","last_peered":"2024-09-11T08:57:33.548362+0000","last_clean":"2024-09-11T08:57:33.548362+0000","last_became_active":"2024-09-11T08:56:08.480106+0000","last_became_peered":"2024-09-11T08:56:08.480106+0000","last_unstale":"2024-09-11T08:57:33.548362+0000","last_undegraded":"2024-09-11T08:57:33.548362+0000","last_fullsized":"2024-09-11T08:57:33.548362+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:55:41.383556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:55:41.383556+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.0","version":"374'1397","reported_seq":1413,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.925782+0000","last_change":"2024-09-11T08:57:14.476990+0000","last_active":"2024-09-11T08:57:36.925782+0000","last_peered":"2024-09-11T08:57:36.925782+0000","last_clean":"2024-09-11T08:57:36.925782+0000","last_became_active":"2024-09-11T08:57:14.476693+0000","last_became_peered":"2024-09-11T08:57:14.476693+0000","last_unstale":"2024-09-11T08:57:36.925782+0000","last_undegraded":"2024-09-11T08:57:36.925782+0000","last_fullsized":"2024-09-11T08:57:36.925782+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1397,"log_dups_size":0,"ondisk_log_size":1397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:30.631365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91553792,"num_objects":1397,"num_object_clones":0,"num_object_copies":4191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1397,"num_write_kb":89408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"199'81","reported_seq":153,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546349+0000","last_change":"2024-09-11T08:56:33.363021+0000","last_active":"2024-09-11T08:57:33.546349+0000","last_peered":"2024-09-11T08:57:33.546349+0000","last_clean":"2024-09-11T08:57:33.546349+0000","last_became_active":"2024-09-11T08:51:03.023615+0000","last_became_peered":"2024-09-11T08:51:03.023615+0000","last_unstale":"2024-09-11T08:57:33.546349+0000","last_undegraded":"2024-09-11T08:57:33.546349+0000","last_fullsized":"2024-09-11T08:57:33.546349+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:56:33.362954+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:56:33.362954+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:38.473680+0000","scrub_duration":0.001495113,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"200'34","reported_seq":454,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.552207+0000","last_change":"2024-09-11T08:56:39.482369+0000","last_active":"2024-09-11T08:57:33.552207+0000","last_peered":"2024-09-11T08:57:33.552207+0000","last_clean":"2024-09-11T08:57:33.552207+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T08:57:33.552207+0000","last_undegraded":"2024-09-11T08:57:33.552207+0000","last_fullsized":"2024-09-11T08:57:33.552207+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:56:39.482296+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:56:39.482296+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:05.827316+0000","scrub_duration":0.0024725670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":386,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.555029+0000","last_change":"2024-09-11T08:55:39.740071+0000","last_active":"2024-09-11T08:57:33.555029+0000","last_peered":"2024-09-11T08:57:33.555029+0000","last_clean":"2024-09-11T08:57:33.555029+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T08:57:33.555029+0000","last_undegraded":"2024-09-11T08:57:33.555029+0000","last_fullsized":"2024-09-11T08:57:33.555029+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:39.739981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:55:39.739981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052641099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"277'11","reported_seq":647,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548629+0000","last_change":"2024-09-11T08:56:31.694340+0000","last_active":"2024-09-11T08:57:33.548629+0000","last_peered":"2024-09-11T08:57:33.548629+0000","last_clean":"2024-09-11T08:57:33.548629+0000","last_became_active":"2024-09-11T08:52:18.578328+0000","last_became_peered":"2024-09-11T08:52:18.578328+0000","last_unstale":"2024-09-11T08:57:33.548629+0000","last_undegraded":"2024-09-11T08:57:33.548629+0000","last_fullsized":"2024-09-11T08:57:33.548629+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'11","last_scrub_stamp":"2024-09-11T08:56:31.694253+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T08:56:31.694253+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025274149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":413,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547548+0000","last_change":"2024-09-11T08:56:25.462412+0000","last_active":"2024-09-11T08:57:33.547548+0000","last_peered":"2024-09-11T08:57:33.547548+0000","last_clean":"2024-09-11T08:57:33.547548+0000","last_became_active":"2024-09-11T08:56:25.462088+0000","last_became_peered":"2024-09-11T08:56:25.462088+0000","last_unstale":"2024-09-11T08:57:33.547548+0000","last_undegraded":"2024-09-11T08:57:33.547548+0000","last_fullsized":"2024-09-11T08:57:33.547548+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:04.263219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:55:04.263219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034071710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"374'1425","reported_seq":1441,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.911958+0000","last_change":"2024-09-11T08:57:14.473925+0000","last_active":"2024-09-11T08:57:36.911958+0000","last_peered":"2024-09-11T08:57:36.911958+0000","last_clean":"2024-09-11T08:57:36.911958+0000","last_became_active":"2024-09-11T08:57:14.473631+0000","last_became_peered":"2024-09-11T08:57:14.473631+0000","last_unstale":"2024-09-11T08:57:36.911958+0000","last_undegraded":"2024-09-11T08:57:36.911958+0000","last_fullsized":"2024-09-11T08:57:36.911958+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:16.128080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93388800,"num_objects":1425,"num_object_clones":0,"num_object_copies":4275,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1425,"num_write_kb":91200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"199'81","reported_seq":153,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548530+0000","last_change":"2024-09-11T08:56:33.671358+0000","last_active":"2024-09-11T08:57:33.548530+0000","last_peered":"2024-09-11T08:57:33.548530+0000","last_clean":"2024-09-11T08:57:33.548530+0000","last_became_active":"2024-09-11T08:51:03.023365+0000","last_became_peered":"2024-09-11T08:51:03.023365+0000","last_unstale":"2024-09-11T08:57:33.548530+0000","last_undegraded":"2024-09-11T08:57:33.548530+0000","last_fullsized":"2024-09-11T08:57:33.548530+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:56:33.671306+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:56:33.671306+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:03.642591+0000","scrub_duration":0.0011424930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"200'38","reported_seq":498,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.550193+0000","last_change":"2024-09-11T08:55:06.625605+0000","last_active":"2024-09-11T08:57:33.550193+0000","last_peered":"2024-09-11T08:57:33.550193+0000","last_clean":"2024-09-11T08:57:33.550193+0000","last_became_active":"2024-09-11T08:52:18.595013+0000","last_became_peered":"2024-09-11T08:52:18.595013+0000","last_unstale":"2024-09-11T08:57:33.550193+0000","last_undegraded":"2024-09-11T08:57:33.550193+0000","last_fullsized":"2024-09-11T08:57:33.550193+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'38","last_scrub_stamp":"2024-09-11T08:55:06.625504+0000","last_deep_scrub":"199'26","last_deep_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_clean_scrub_stamp":"2024-09-11T08:55:06.625504+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010430116999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":409,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.553793+0000","last_change":"2024-09-11T08:55:04.638886+0000","last_active":"2024-09-11T08:57:33.553793+0000","last_peered":"2024-09-11T08:57:33.553793+0000","last_clean":"2024-09-11T08:57:33.553793+0000","last_became_active":"2024-09-11T08:52:18.586084+0000","last_became_peered":"2024-09-11T08:52:18.586084+0000","last_unstale":"2024-09-11T08:57:33.553793+0000","last_undegraded":"2024-09-11T08:57:33.553793+0000","last_fullsized":"2024-09-11T08:57:33.553793+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:04.638693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:55:04.638693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070800059999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":383,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546498+0000","last_change":"2024-09-11T08:57:18.592168+0000","last_active":"2024-09-11T08:57:33.546498+0000","last_peered":"2024-09-11T08:57:33.546498+0000","last_clean":"2024-09-11T08:57:33.546498+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T08:57:33.546498+0000","last_undegraded":"2024-09-11T08:57:33.546498+0000","last_fullsized":"2024-09-11T08:57:33.546498+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:18.592099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:29.687577+0000","last_clean_scrub_stamp":"2024-09-11T08:57:18.592099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:19.166801+0000","scrub_duration":0.001725077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":381,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546921+0000","last_change":"2024-09-11T08:57:27.597138+0000","last_active":"2024-09-11T08:57:33.546921+0000","last_peered":"2024-09-11T08:57:33.546921+0000","last_clean":"2024-09-11T08:57:33.546921+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T08:57:33.546921+0000","last_undegraded":"2024-09-11T08:57:33.546921+0000","last_fullsized":"2024-09-11T08:57:33.546921+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:27.597069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:57:27.597069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:54.291821+0000","scrub_duration":0.001643186,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"374'1386","reported_seq":1402,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.399888+0000","last_change":"2024-09-11T08:57:14.474398+0000","last_active":"2024-09-11T08:57:36.399888+0000","last_peered":"2024-09-11T08:57:36.399888+0000","last_clean":"2024-09-11T08:57:36.399888+0000","last_became_active":"2024-09-11T08:57:14.474062+0000","last_became_peered":"2024-09-11T08:57:14.474062+0000","last_unstale":"2024-09-11T08:57:36.399888+0000","last_undegraded":"2024-09-11T08:57:36.399888+0000","last_fullsized":"2024-09-11T08:57:36.399888+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1386,"log_dups_size":0,"ondisk_log_size":1386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:38.030459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90832896,"num_objects":1386,"num_object_clones":0,"num_object_copies":4158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1386,"num_write_kb":88704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"199'81","reported_seq":158,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.554783+0000","last_change":"2024-09-11T08:55:53.823820+0000","last_active":"2024-09-11T08:57:33.554783+0000","last_peered":"2024-09-11T08:57:33.554783+0000","last_clean":"2024-09-11T08:57:33.554783+0000","last_became_active":"2024-09-11T08:51:03.009983+0000","last_became_peered":"2024-09-11T08:51:03.009983+0000","last_unstale":"2024-09-11T08:57:33.554783+0000","last_undegraded":"2024-09-11T08:57:33.554783+0000","last_fullsized":"2024-09-11T08:57:33.554783+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:55:53.823609+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:55:53.823609+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052441400000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"200'35","reported_seq":427,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546657+0000","last_change":"2024-09-11T08:55:53.637038+0000","last_active":"2024-09-11T08:57:33.546657+0000","last_peered":"2024-09-11T08:57:33.546657+0000","last_clean":"2024-09-11T08:57:33.546657+0000","last_became_active":"2024-09-11T08:38:40.968094+0000","last_became_peered":"2024-09-11T08:38:40.968094+0000","last_unstale":"2024-09-11T08:57:33.546657+0000","last_undegraded":"2024-09-11T08:57:33.546657+0000","last_fullsized":"2024-09-11T08:57:33.546657+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'35","last_scrub_stamp":"2024-09-11T08:55:53.636905+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:55:53.636905+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057013680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":146,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.555679+0000","last_change":"2024-09-11T08:57:30.837703+0000","last_active":"2024-09-11T08:57:33.555679+0000","last_peered":"2024-09-11T08:57:33.555679+0000","last_clean":"2024-09-11T08:57:33.555679+0000","last_became_active":"2024-09-11T08:52:18.586549+0000","last_became_peered":"2024-09-11T08:52:18.586549+0000","last_unstale":"2024-09-11T08:57:33.555679+0000","last_undegraded":"2024-09-11T08:57:33.555679+0000","last_fullsized":"2024-09-11T08:57:33.555679+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:30.837623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:28.487573+0000","last_clean_scrub_stamp":"2024-09-11T08:57:30.837623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:44.002032+0000","scrub_duration":0.0027401750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":620,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.550033+0000","last_change":"2024-09-11T08:57:22.782387+0000","last_active":"2024-09-11T08:57:33.550033+0000","last_peered":"2024-09-11T08:57:33.550033+0000","last_clean":"2024-09-11T08:57:33.550033+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T08:57:33.550033+0000","last_undegraded":"2024-09-11T08:57:33.550033+0000","last_fullsized":"2024-09-11T08:57:33.550033+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:57:22.782332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:57:22.782332+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:49.512985+0000","scrub_duration":0.001789934,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":220,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.553968+0000","last_change":"2024-09-11T08:57:23.554994+0000","last_active":"2024-09-11T08:57:33.553968+0000","last_peered":"2024-09-11T08:57:33.553968+0000","last_clean":"2024-09-11T08:57:33.553968+0000","last_became_active":"2024-09-11T08:56:08.478237+0000","last_became_peered":"2024-09-11T08:56:08.478237+0000","last_unstale":"2024-09-11T08:57:33.553968+0000","last_undegraded":"2024-09-11T08:57:33.553968+0000","last_fullsized":"2024-09-11T08:57:33.553968+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:23.554939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:57:23.554939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:38.345940+0000","scrub_duration":0.0018369529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"374'1380","reported_seq":1396,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.147425+0000","last_change":"2024-09-11T08:57:14.487816+0000","last_active":"2024-09-11T08:57:36.147425+0000","last_peered":"2024-09-11T08:57:36.147425+0000","last_clean":"2024-09-11T08:57:36.147425+0000","last_became_active":"2024-09-11T08:57:14.487324+0000","last_became_peered":"2024-09-11T08:57:14.487324+0000","last_unstale":"2024-09-11T08:57:36.147425+0000","last_undegraded":"2024-09-11T08:57:36.147425+0000","last_fullsized":"2024-09-11T08:57:36.147425+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1380,"log_dups_size":0,"ondisk_log_size":1380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:13.756673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90439680,"num_objects":1380,"num_object_clones":0,"num_object_copies":4140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1380,"num_write_kb":88320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"199'81","reported_seq":284,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.553593+0000","last_change":"2024-09-11T08:56:08.521129+0000","last_active":"2024-09-11T08:57:33.553593+0000","last_peered":"2024-09-11T08:57:33.553593+0000","last_clean":"2024-09-11T08:57:33.553593+0000","last_became_active":"2024-09-11T08:56:08.480247+0000","last_became_peered":"2024-09-11T08:56:08.480247+0000","last_unstale":"2024-09-11T08:57:33.553593+0000","last_undegraded":"2024-09-11T08:57:33.553593+0000","last_fullsized":"2024-09-11T08:57:33.553593+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:55:13.377713+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:55:13.377713+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002308867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"200'25","reported_seq":294,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546405+0000","last_change":"2024-09-11T08:56:50.137432+0000","last_active":"2024-09-11T08:57:33.546405+0000","last_peered":"2024-09-11T08:57:33.546405+0000","last_clean":"2024-09-11T08:57:33.546405+0000","last_became_active":"2024-09-11T08:56:25.462200+0000","last_became_peered":"2024-09-11T08:56:25.462200+0000","last_unstale":"2024-09-11T08:57:33.546405+0000","last_undegraded":"2024-09-11T08:57:33.546405+0000","last_fullsized":"2024-09-11T08:57:33.546405+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:56:50.137368+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_clean_scrub_stamp":"2024-09-11T08:56:50.137368+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:17.969370+0000","scrub_duration":0.0020964199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":392,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547851+0000","last_change":"2024-09-11T08:55:49.588659+0000","last_active":"2024-09-11T08:57:33.547851+0000","last_peered":"2024-09-11T08:57:33.547851+0000","last_clean":"2024-09-11T08:57:33.547851+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T08:57:33.547851+0000","last_undegraded":"2024-09-11T08:57:33.547851+0000","last_fullsized":"2024-09-11T08:57:33.547851+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:49.588588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:55:49.588588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032745880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"346'12","reported_seq":79,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547333+0000","last_change":"2024-09-11T08:56:08.486445+0000","last_active":"2024-09-11T08:57:33.547333+0000","last_peered":"2024-09-11T08:57:33.547333+0000","last_clean":"2024-09-11T08:57:33.547333+0000","last_became_active":"2024-09-11T08:56:08.482468+0000","last_became_peered":"2024-09-11T08:56:08.482468+0000","last_unstale":"2024-09-11T08:57:33.547333+0000","last_undegraded":"2024-09-11T08:57:33.547333+0000","last_fullsized":"2024-09-11T08:57:33.547333+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"292'11","last_scrub_stamp":"2024-09-11T08:54:56.468128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:56.468128+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":202,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.549959+0000","last_change":"2024-09-11T08:57:18.779086+0000","last_active":"2024-09-11T08:57:33.549959+0000","last_peered":"2024-09-11T08:57:33.549959+0000","last_clean":"2024-09-11T08:57:33.549959+0000","last_became_active":"2024-09-11T08:52:49.571683+0000","last_became_peered":"2024-09-11T08:52:49.571683+0000","last_unstale":"2024-09-11T08:57:33.549959+0000","last_undegraded":"2024-09-11T08:57:33.549959+0000","last_fullsized":"2024-09-11T08:57:33.549959+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:18.779033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:57:18.779033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:46.794141+0000","scrub_duration":0.001147438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"374'1404","reported_seq":1420,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.931711+0000","last_change":"2024-09-11T08:57:14.473945+0000","last_active":"2024-09-11T08:57:36.931711+0000","last_peered":"2024-09-11T08:57:36.931711+0000","last_clean":"2024-09-11T08:57:36.931711+0000","last_became_active":"2024-09-11T08:57:14.473651+0000","last_became_peered":"2024-09-11T08:57:14.473651+0000","last_unstale":"2024-09-11T08:57:36.931711+0000","last_undegraded":"2024-09-11T08:57:36.931711+0000","last_fullsized":"2024-09-11T08:57:36.931711+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1404,"log_dups_size":0,"ondisk_log_size":1404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:28.656570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92012544,"num_objects":1404,"num_object_clones":0,"num_object_copies":4212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1404,"num_write_kb":89856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"199'81","reported_seq":284,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.552709+0000","last_change":"2024-09-11T08:56:08.479646+0000","last_active":"2024-09-11T08:57:33.552709+0000","last_peered":"2024-09-11T08:57:33.552709+0000","last_clean":"2024-09-11T08:57:33.552709+0000","last_became_active":"2024-09-11T08:56:08.479256+0000","last_became_peered":"2024-09-11T08:56:08.479256+0000","last_unstale":"2024-09-11T08:57:33.552709+0000","last_undegraded":"2024-09-11T08:57:33.552709+0000","last_fullsized":"2024-09-11T08:57:33.552709+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:55:35.328190+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:55:35.328190+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002308867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"200'29","reported_seq":450,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547369+0000","last_change":"2024-09-11T08:57:19.131847+0000","last_active":"2024-09-11T08:57:33.547369+0000","last_peered":"2024-09-11T08:57:33.547369+0000","last_clean":"2024-09-11T08:57:33.547369+0000","last_became_active":"2024-09-11T08:52:49.563364+0000","last_became_peered":"2024-09-11T08:52:49.563364+0000","last_unstale":"2024-09-11T08:57:33.547369+0000","last_undegraded":"2024-09-11T08:57:33.547369+0000","last_fullsized":"2024-09-11T08:57:33.547369+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'29","last_scrub_stamp":"2024-09-11T08:57:19.131782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:57:19.131782+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:30.785377+0000","scrub_duration":0.0021894190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":416,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.552130+0000","last_change":"2024-09-11T08:56:44.624192+0000","last_active":"2024-09-11T08:57:33.552130+0000","last_peered":"2024-09-11T08:57:33.552130+0000","last_clean":"2024-09-11T08:57:33.552130+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T08:57:33.552130+0000","last_undegraded":"2024-09-11T08:57:33.552130+0000","last_fullsized":"2024-09-11T08:57:33.552130+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:44.624134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:56:44.624134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:45.517224+0000","scrub_duration":0.001236776,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":347,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546288+0000","last_change":"2024-09-11T08:56:08.527490+0000","last_active":"2024-09-11T08:57:33.546288+0000","last_peered":"2024-09-11T08:57:33.546288+0000","last_clean":"2024-09-11T08:57:33.546288+0000","last_became_active":"2024-09-11T08:56:08.527106+0000","last_became_peered":"2024-09-11T08:56:08.527106+0000","last_unstale":"2024-09-11T08:57:33.546288+0000","last_undegraded":"2024-09-11T08:57:33.546288+0000","last_fullsized":"2024-09-11T08:57:33.546288+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:59.511833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_clean_scrub_stamp":"2024-09-11T08:54:59.511833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011823264,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":430,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.551589+0000","last_change":"2024-09-11T08:56:43.590127+0000","last_active":"2024-09-11T08:57:33.551589+0000","last_peered":"2024-09-11T08:57:33.551589+0000","last_clean":"2024-09-11T08:57:33.551589+0000","last_became_active":"2024-09-11T08:46:01.985015+0000","last_became_peered":"2024-09-11T08:46:01.985015+0000","last_unstale":"2024-09-11T08:57:33.551589+0000","last_undegraded":"2024-09-11T08:57:33.551589+0000","last_fullsized":"2024-09-11T08:57:33.551589+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:43.590073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:56:43.590073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:11.851501+0000","scrub_duration":0.0013618040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"374'1332","reported_seq":1348,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.160677+0000","last_change":"2024-09-11T08:57:14.485776+0000","last_active":"2024-09-11T08:57:36.160677+0000","last_peered":"2024-09-11T08:57:36.160677+0000","last_clean":"2024-09-11T08:57:36.160677+0000","last_became_active":"2024-09-11T08:57:14.485494+0000","last_became_peered":"2024-09-11T08:57:14.485494+0000","last_unstale":"2024-09-11T08:57:36.160677+0000","last_undegraded":"2024-09-11T08:57:36.160677+0000","last_fullsized":"2024-09-11T08:57:36.160677+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1332,"log_dups_size":0,"ondisk_log_size":1332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:35.277017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87293952,"num_objects":1332,"num_object_clones":0,"num_object_copies":3996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1332,"num_write_kb":85248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"199'81","reported_seq":153,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546292+0000","last_change":"2024-09-11T08:54:09.431646+0000","last_active":"2024-09-11T08:57:33.546292+0000","last_peered":"2024-09-11T08:57:33.546292+0000","last_clean":"2024-09-11T08:57:33.546292+0000","last_became_active":"2024-09-11T08:52:18.574468+0000","last_became_peered":"2024-09-11T08:52:18.574468+0000","last_unstale":"2024-09-11T08:57:33.546292+0000","last_undegraded":"2024-09-11T08:57:33.546292+0000","last_fullsized":"2024-09-11T08:57:33.546292+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:54:09.431556+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:54:09.431556+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066392939999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"349'51","reported_seq":484,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.554683+0000","last_change":"2024-09-11T08:56:49.793250+0000","last_active":"2024-09-11T08:57:33.554683+0000","last_peered":"2024-09-11T08:57:33.554683+0000","last_clean":"2024-09-11T08:57:33.554683+0000","last_became_active":"2024-09-11T08:56:25.466103+0000","last_became_peered":"2024-09-11T08:56:25.466103+0000","last_unstale":"2024-09-11T08:57:33.554683+0000","last_undegraded":"2024-09-11T08:57:33.554683+0000","last_fullsized":"2024-09-11T08:57:33.554683+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"349'51","last_scrub_stamp":"2024-09-11T08:56:49.793192+0000","last_deep_scrub":"200'50","last_deep_scrub_stamp":"2024-09-11T08:53:09.745322+0000","last_clean_scrub_stamp":"2024-09-11T08:56:49.793192+0000","objects_scrubbed":10,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:50.095725+0000","scrub_duration":0.002354868,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":407,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547472+0000","last_change":"2024-09-11T08:54:56.719900+0000","last_active":"2024-09-11T08:57:33.547472+0000","last_peered":"2024-09-11T08:57:33.547472+0000","last_clean":"2024-09-11T08:57:33.547472+0000","last_became_active":"2024-09-11T08:52:18.579509+0000","last_became_peered":"2024-09-11T08:52:18.579509+0000","last_unstale":"2024-09-11T08:57:33.547472+0000","last_undegraded":"2024-09-11T08:57:33.547472+0000","last_fullsized":"2024-09-11T08:57:33.547472+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:56.719827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_clean_scrub_stamp":"2024-09-11T08:54:56.719827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003589297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":427,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546505+0000","last_change":"2024-09-11T08:54:14.424233+0000","last_active":"2024-09-11T08:57:33.546505+0000","last_peered":"2024-09-11T08:57:33.546505+0000","last_clean":"2024-09-11T08:57:33.546505+0000","last_became_active":"2024-09-11T08:46:01.982326+0000","last_became_peered":"2024-09-11T08:46:01.982326+0000","last_unstale":"2024-09-11T08:57:33.546505+0000","last_undegraded":"2024-09-11T08:57:33.546505+0000","last_fullsized":"2024-09-11T08:57:33.546505+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:54:14.423685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:54:14.423685+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012346487999999999,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":212,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.552608+0000","last_change":"2024-09-11T08:56:46.670619+0000","last_active":"2024-09-11T08:57:33.552608+0000","last_peered":"2024-09-11T08:57:33.552608+0000","last_clean":"2024-09-11T08:57:33.552608+0000","last_became_active":"2024-09-11T08:52:49.569137+0000","last_became_peered":"2024-09-11T08:52:49.569137+0000","last_unstale":"2024-09-11T08:57:33.552608+0000","last_undegraded":"2024-09-11T08:57:33.552608+0000","last_fullsized":"2024-09-11T08:57:33.552608+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:46.670557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:56:46.670557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:02.227574+0000","scrub_duration":0.0015144939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":380,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.552868+0000","last_change":"2024-09-11T08:56:32.836719+0000","last_active":"2024-09-11T08:57:33.552868+0000","last_peered":"2024-09-11T08:57:33.552868+0000","last_clean":"2024-09-11T08:57:33.552868+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T08:57:33.552868+0000","last_undegraded":"2024-09-11T08:57:33.552868+0000","last_fullsized":"2024-09-11T08:57:33.552868+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:32.836633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:56:32.836633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:43.446681+0000","scrub_duration":0.0050927949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"374'1386","reported_seq":1400,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.175999+0000","last_change":"2024-09-11T08:57:14.487382+0000","last_active":"2024-09-11T08:57:36.175999+0000","last_peered":"2024-09-11T08:57:36.175999+0000","last_clean":"2024-09-11T08:57:36.175999+0000","last_became_active":"2024-09-11T08:57:14.487076+0000","last_became_peered":"2024-09-11T08:57:14.487076+0000","last_unstale":"2024-09-11T08:57:36.175999+0000","last_undegraded":"2024-09-11T08:57:36.175999+0000","last_fullsized":"2024-09-11T08:57:36.175999+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1386,"log_dups_size":0,"ondisk_log_size":1386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:18.919525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90832896,"num_objects":1386,"num_object_clones":0,"num_object_copies":4158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1386,"num_write_kb":88704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"199'81","reported_seq":432,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546596+0000","last_change":"2024-09-11T08:55:00.708661+0000","last_active":"2024-09-11T08:57:33.546596+0000","last_peered":"2024-09-11T08:57:33.546596+0000","last_clean":"2024-09-11T08:57:33.546596+0000","last_became_active":"2024-09-11T08:52:49.569934+0000","last_became_peered":"2024-09-11T08:52:49.569934+0000","last_unstale":"2024-09-11T08:57:33.546596+0000","last_undegraded":"2024-09-11T08:57:33.546596+0000","last_fullsized":"2024-09-11T08:57:33.546596+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:55:00.708581+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:55:00.708581+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053390399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"200'17","reported_seq":282,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547847+0000","last_change":"2024-09-11T08:56:25.468694+0000","last_active":"2024-09-11T08:57:33.547847+0000","last_peered":"2024-09-11T08:57:33.547847+0000","last_clean":"2024-09-11T08:57:33.547847+0000","last_became_active":"2024-09-11T08:56:25.459902+0000","last_became_peered":"2024-09-11T08:56:25.459902+0000","last_unstale":"2024-09-11T08:57:33.547847+0000","last_undegraded":"2024-09-11T08:57:33.547847+0000","last_fullsized":"2024-09-11T08:57:33.547847+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'17","last_scrub_stamp":"2024-09-11T08:55:33.394000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:33.394000+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023582730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":432,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.551308+0000","last_change":"2024-09-11T08:56:37.489296+0000","last_active":"2024-09-11T08:57:33.551308+0000","last_peered":"2024-09-11T08:57:33.551308+0000","last_clean":"2024-09-11T08:57:33.551308+0000","last_became_active":"2024-09-11T08:46:01.981561+0000","last_became_peered":"2024-09-11T08:46:01.981561+0000","last_unstale":"2024-09-11T08:57:33.551308+0000","last_undegraded":"2024-09-11T08:57:33.551308+0000","last_fullsized":"2024-09-11T08:57:33.551308+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:37.489227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:56:37.489227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:43.749686+0000","scrub_duration":0.001708876,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"263'2","reported_seq":487,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.553120+0000","last_change":"2024-09-11T08:57:18.539314+0000","last_active":"2024-09-11T08:57:33.553120+0000","last_peered":"2024-09-11T08:57:33.553120+0000","last_clean":"2024-09-11T08:57:33.553120+0000","last_became_active":"2024-09-11T08:52:18.586545+0000","last_became_peered":"2024-09-11T08:52:18.586545+0000","last_unstale":"2024-09-11T08:57:33.553120+0000","last_undegraded":"2024-09-11T08:57:33.553120+0000","last_fullsized":"2024-09-11T08:57:33.553120+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'2","last_scrub_stamp":"2024-09-11T08:57:18.539221+0000","last_deep_scrub":"263'2","last_deep_scrub_stamp":"2024-09-11T08:53:08.772979+0000","last_clean_scrub_stamp":"2024-09-11T08:57:18.539221+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:33.227582+0000","scrub_duration":0.001409722,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":437,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.555014+0000","last_change":"2024-09-11T08:56:45.788535+0000","last_active":"2024-09-11T08:57:33.555014+0000","last_peered":"2024-09-11T08:57:33.555014+0000","last_clean":"2024-09-11T08:57:33.555014+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T08:57:33.555014+0000","last_undegraded":"2024-09-11T08:57:33.555014+0000","last_fullsized":"2024-09-11T08:57:33.555014+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:45.788469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:44.344206+0000","last_clean_scrub_stamp":"2024-09-11T08:56:45.788469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:51.011922+0000","scrub_duration":0.0014813910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":210,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.552312+0000","last_change":"2024-09-11T08:56:08.478405+0000","last_active":"2024-09-11T08:57:33.552312+0000","last_peered":"2024-09-11T08:57:33.552312+0000","last_clean":"2024-09-11T08:57:33.552312+0000","last_became_active":"2024-09-11T08:56:08.478027+0000","last_became_peered":"2024-09-11T08:56:08.478027+0000","last_unstale":"2024-09-11T08:57:33.552312+0000","last_undegraded":"2024-09-11T08:57:33.552312+0000","last_fullsized":"2024-09-11T08:57:33.552312+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:53.557746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:55:53.557746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015184790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"374'1307","reported_seq":1323,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.155515+0000","last_change":"2024-09-11T08:57:14.488520+0000","last_active":"2024-09-11T08:57:36.155515+0000","last_peered":"2024-09-11T08:57:36.155515+0000","last_clean":"2024-09-11T08:57:36.155515+0000","last_became_active":"2024-09-11T08:57:14.488141+0000","last_became_peered":"2024-09-11T08:57:14.488141+0000","last_unstale":"2024-09-11T08:57:36.155515+0000","last_undegraded":"2024-09-11T08:57:36.155515+0000","last_fullsized":"2024-09-11T08:57:36.155515+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1307,"log_dups_size":0,"ondisk_log_size":1307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:31.729614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85655552,"num_objects":1307,"num_object_clones":0,"num_object_copies":3921,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1307,"num_write_kb":83648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"199'81","reported_seq":426,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.552809+0000","last_change":"2024-09-11T08:56:40.470217+0000","last_active":"2024-09-11T08:57:33.552809+0000","last_peered":"2024-09-11T08:57:33.552809+0000","last_clean":"2024-09-11T08:57:33.552809+0000","last_became_active":"2024-09-11T08:51:03.027281+0000","last_became_peered":"2024-09-11T08:51:03.027281+0000","last_unstale":"2024-09-11T08:57:33.552809+0000","last_undegraded":"2024-09-11T08:57:33.552809+0000","last_fullsized":"2024-09-11T08:57:33.552809+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:56:40.470121+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:56:40.470121+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:01.957627+0000","scrub_duration":0.0036931730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"200'24","reported_seq":160,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.555459+0000","last_change":"2024-09-11T08:57:26.809478+0000","last_active":"2024-09-11T08:57:33.555459+0000","last_peered":"2024-09-11T08:57:33.555459+0000","last_clean":"2024-09-11T08:57:33.555459+0000","last_became_active":"2024-09-11T08:52:18.580950+0000","last_became_peered":"2024-09-11T08:52:18.580950+0000","last_unstale":"2024-09-11T08:57:33.555459+0000","last_undegraded":"2024-09-11T08:57:33.555459+0000","last_fullsized":"2024-09-11T08:57:33.555459+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'24","last_scrub_stamp":"2024-09-11T08:57:26.809429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:57:26.809429+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:47.687778+0000","scrub_duration":0.0023858849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":58,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546081+0000","last_change":"2024-09-11T08:56:37.284843+0000","last_active":"2024-09-11T08:57:33.546081+0000","last_peered":"2024-09-11T08:57:33.546081+0000","last_clean":"2024-09-11T08:57:33.546081+0000","last_became_active":"2024-09-11T08:55:31.080178+0000","last_became_peered":"2024-09-11T08:55:31.080178+0000","last_unstale":"2024-09-11T08:57:33.546081+0000","last_undegraded":"2024-09-11T08:57:33.546081+0000","last_fullsized":"2024-09-11T08:57:33.546081+0000","mapping_epoch":335,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":336,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:37.284783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:56:37.284783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:46.295797+0000","scrub_duration":0.001452767,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"318'6","reported_seq":110,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:34.589716+0000","last_change":"2024-09-11T08:56:25.470170+0000","last_active":"2024-09-11T08:57:34.589716+0000","last_peered":"2024-09-11T08:57:34.589716+0000","last_clean":"2024-09-11T08:57:34.589716+0000","last_became_active":"2024-09-11T08:56:25.462503+0000","last_became_peered":"2024-09-11T08:56:25.462503+0000","last_unstale":"2024-09-11T08:57:34.589716+0000","last_undegraded":"2024-09-11T08:57:34.589716+0000","last_fullsized":"2024-09-11T08:57:34.589716+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'6","last_scrub_stamp":"2024-09-11T08:55:42.632806+0000","last_deep_scrub":"318'6","last_deep_scrub_stamp":"2024-09-11T08:54:12.873653+0000","last_clean_scrub_stamp":"2024-09-11T08:55:42.632806+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017088629999999999,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":166,"num_read_kb":161,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":408,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548651+0000","last_change":"2024-09-11T08:54:16.367927+0000","last_active":"2024-09-11T08:57:33.548651+0000","last_peered":"2024-09-11T08:57:33.548651+0000","last_clean":"2024-09-11T08:57:33.548651+0000","last_became_active":"2024-09-11T08:52:18.578530+0000","last_became_peered":"2024-09-11T08:52:18.578530+0000","last_unstale":"2024-09-11T08:57:33.548651+0000","last_undegraded":"2024-09-11T08:57:33.548651+0000","last_fullsized":"2024-09-11T08:57:33.548651+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:16.367873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:16.367873+0000","last_clean_scrub_stamp":"2024-09-11T08:54:16.367873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013677540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":38,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548234+0000","last_change":"2024-09-11T08:56:25.460095+0000","last_active":"2024-09-11T08:57:33.548234+0000","last_peered":"2024-09-11T08:57:33.548234+0000","last_clean":"2024-09-11T08:57:33.548234+0000","last_became_active":"2024-09-11T08:56:25.459701+0000","last_became_peered":"2024-09-11T08:56:25.459701+0000","last_unstale":"2024-09-11T08:57:33.548234+0000","last_undegraded":"2024-09-11T08:57:33.548234+0000","last_fullsized":"2024-09-11T08:57:33.548234+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:13.179675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:55:13.179675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"374'1308","reported_seq":1323,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.171480+0000","last_change":"2024-09-11T08:57:14.487234+0000","last_active":"2024-09-11T08:57:36.171480+0000","last_peered":"2024-09-11T08:57:36.171480+0000","last_clean":"2024-09-11T08:57:36.171480+0000","last_became_active":"2024-09-11T08:57:14.486077+0000","last_became_peered":"2024-09-11T08:57:14.486077+0000","last_unstale":"2024-09-11T08:57:36.171480+0000","last_undegraded":"2024-09-11T08:57:36.171480+0000","last_fullsized":"2024-09-11T08:57:36.171480+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:34.831092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85721088,"num_objects":1308,"num_object_clones":0,"num_object_copies":3924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1308,"num_write_kb":83712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"201'27","reported_seq":132,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548387+0000","last_change":"2024-09-11T08:56:08.547237+0000","last_active":"2024-09-11T08:57:33.548387+0000","last_peered":"2024-09-11T08:57:33.548387+0000","last_clean":"2024-09-11T08:57:33.548387+0000","last_became_active":"2024-09-11T08:56:08.544409+0000","last_became_peered":"2024-09-11T08:56:08.544409+0000","last_unstale":"2024-09-11T08:57:33.548387+0000","last_undegraded":"2024-09-11T08:57:33.548387+0000","last_fullsized":"2024-09-11T08:57:33.548387+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'27","last_scrub_stamp":"2024-09-11T08:55:50.542013+0000","last_deep_scrub":"201'27","last_deep_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_clean_scrub_stamp":"2024-09-11T08:55:50.542013+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0080208199999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":395,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.558977+0000","last_change":"2024-09-11T08:56:50.853820+0000","last_active":"2024-09-11T08:57:33.558977+0000","last_peered":"2024-09-11T08:57:33.558977+0000","last_clean":"2024-09-11T08:57:33.558977+0000","last_became_active":"2024-09-11T08:38:38.947979+0000","last_became_peered":"2024-09-11T08:38:38.947979+0000","last_unstale":"2024-09-11T08:57:33.558977+0000","last_undegraded":"2024-09-11T08:57:33.558977+0000","last_fullsized":"2024-09-11T08:57:33.558977+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:56:50.853747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:56:50.853747+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:01.577500+0000","scrub_duration":0.0018093129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"199'81","reported_seq":420,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.555493+0000","last_change":"2024-09-11T08:56:36.496178+0000","last_active":"2024-09-11T08:57:33.555493+0000","last_peered":"2024-09-11T08:57:33.555493+0000","last_clean":"2024-09-11T08:57:33.555493+0000","last_became_active":"2024-09-11T08:56:08.484662+0000","last_became_peered":"2024-09-11T08:56:08.484662+0000","last_unstale":"2024-09-11T08:57:33.555493+0000","last_undegraded":"2024-09-11T08:57:33.555493+0000","last_fullsized":"2024-09-11T08:57:33.555493+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:56:36.496096+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:55:08.518651+0000","last_clean_scrub_stamp":"2024-09-11T08:56:36.496096+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:39.488379+0000","scrub_duration":0.001718827,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":129,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.554430+0000","last_change":"2024-09-11T08:56:39.883358+0000","last_active":"2024-09-11T08:57:33.554430+0000","last_peered":"2024-09-11T08:57:33.554430+0000","last_clean":"2024-09-11T08:57:33.554430+0000","last_became_active":"2024-09-11T08:52:38.485446+0000","last_became_peered":"2024-09-11T08:52:38.485446+0000","last_unstale":"2024-09-11T08:57:33.554430+0000","last_undegraded":"2024-09-11T08:57:33.554430+0000","last_fullsized":"2024-09-11T08:57:33.554430+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:39.883298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:56:39.883298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:01.515267+0000","scrub_duration":0.0018155319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":396,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548556+0000","last_change":"2024-09-11T08:57:27.101488+0000","last_active":"2024-09-11T08:57:33.548556+0000","last_peered":"2024-09-11T08:57:33.548556+0000","last_clean":"2024-09-11T08:57:33.548556+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T08:57:33.548556+0000","last_undegraded":"2024-09-11T08:57:33.548556+0000","last_fullsized":"2024-09-11T08:57:33.548556+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:27.101421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:57:27.101421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:31.735816+0000","scrub_duration":0.0014808950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"374'1360","reported_seq":1376,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.155584+0000","last_change":"2024-09-11T08:57:14.488269+0000","last_active":"2024-09-11T08:57:36.155584+0000","last_peered":"2024-09-11T08:57:36.155584+0000","last_clean":"2024-09-11T08:57:36.155584+0000","last_became_active":"2024-09-11T08:57:14.487936+0000","last_became_peered":"2024-09-11T08:57:14.487936+0000","last_unstale":"2024-09-11T08:57:36.155584+0000","last_undegraded":"2024-09-11T08:57:36.155584+0000","last_fullsized":"2024-09-11T08:57:36.155584+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1360,"log_dups_size":0,"ondisk_log_size":1360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:32.088106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89128960,"num_objects":1360,"num_object_clones":0,"num_object_copies":4080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1360,"num_write_kb":87040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"200'19","reported_seq":497,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548448+0000","last_change":"2024-09-11T08:56:25.483701+0000","last_active":"2024-09-11T08:57:33.548448+0000","last_peered":"2024-09-11T08:57:33.548448+0000","last_clean":"2024-09-11T08:57:33.548448+0000","last_became_active":"2024-09-11T08:56:25.465998+0000","last_became_peered":"2024-09-11T08:56:25.465998+0000","last_unstale":"2024-09-11T08:57:33.548448+0000","last_undegraded":"2024-09-11T08:57:33.548448+0000","last_fullsized":"2024-09-11T08:57:33.548448+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:55:01.735933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:01.735933+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017616557000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":377,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546728+0000","last_change":"2024-09-11T08:55:11.804648+0000","last_active":"2024-09-11T08:57:33.546728+0000","last_peered":"2024-09-11T08:57:33.546728+0000","last_clean":"2024-09-11T08:57:33.546728+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T08:57:33.546728+0000","last_undegraded":"2024-09-11T08:57:33.546728+0000","last_fullsized":"2024-09-11T08:57:33.546728+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:11.804570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:55:11.804570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010636980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":395,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547760+0000","last_change":"2024-09-11T08:55:06.261924+0000","last_active":"2024-09-11T08:57:33.547760+0000","last_peered":"2024-09-11T08:57:33.547760+0000","last_clean":"2024-09-11T08:57:33.547760+0000","last_became_active":"2024-09-11T08:38:42.982641+0000","last_became_peered":"2024-09-11T08:38:42.982641+0000","last_unstale":"2024-09-11T08:57:33.547760+0000","last_undegraded":"2024-09-11T08:57:33.547760+0000","last_fullsized":"2024-09-11T08:57:33.547760+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:06.261834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:55:06.261834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015153250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":400,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.553932+0000","last_change":"2024-09-11T08:56:40.885862+0000","last_active":"2024-09-11T08:57:33.553932+0000","last_peered":"2024-09-11T08:57:33.553932+0000","last_clean":"2024-09-11T08:57:33.553932+0000","last_became_active":"2024-09-11T08:46:01.988082+0000","last_became_peered":"2024-09-11T08:46:01.988082+0000","last_unstale":"2024-09-11T08:57:33.553932+0000","last_undegraded":"2024-09-11T08:57:33.553932+0000","last_fullsized":"2024-09-11T08:57:33.553932+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:40.885769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:56:40.885769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:10.610689+0000","scrub_duration":0.0025500079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"374'1370","reported_seq":1386,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.171259+0000","last_change":"2024-09-11T08:57:14.487415+0000","last_active":"2024-09-11T08:57:36.171259+0000","last_peered":"2024-09-11T08:57:36.171259+0000","last_clean":"2024-09-11T08:57:36.171259+0000","last_became_active":"2024-09-11T08:57:14.487157+0000","last_became_peered":"2024-09-11T08:57:14.487157+0000","last_unstale":"2024-09-11T08:57:36.171259+0000","last_undegraded":"2024-09-11T08:57:36.171259+0000","last_fullsized":"2024-09-11T08:57:36.171259+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1370,"log_dups_size":0,"ondisk_log_size":1370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:21.867979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89784320,"num_objects":1370,"num_object_clones":0,"num_object_copies":4110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1370,"num_write_kb":87680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"199'81","reported_seq":160,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548426+0000","last_change":"2024-09-11T08:55:33.781406+0000","last_active":"2024-09-11T08:57:33.548426+0000","last_peered":"2024-09-11T08:57:33.548426+0000","last_clean":"2024-09-11T08:57:33.548426+0000","last_became_active":"2024-09-11T08:51:03.020941+0000","last_became_peered":"2024-09-11T08:51:03.020941+0000","last_unstale":"2024-09-11T08:57:33.548426+0000","last_undegraded":"2024-09-11T08:57:33.548426+0000","last_fullsized":"2024-09-11T08:57:33.548426+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:55:33.781314+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:53:14.809894+0000","last_clean_scrub_stamp":"2024-09-11T08:55:33.781314+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076612690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"317'39","reported_seq":452,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548496+0000","last_change":"2024-09-11T08:57:33.510922+0000","last_active":"2024-09-11T08:57:33.548496+0000","last_peered":"2024-09-11T08:57:33.548496+0000","last_clean":"2024-09-11T08:57:33.548496+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T08:57:33.548496+0000","last_undegraded":"2024-09-11T08:57:33.548496+0000","last_fullsized":"2024-09-11T08:57:33.548496+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"317'39","last_scrub_stamp":"2024-09-11T08:57:33.510860+0000","last_deep_scrub":"317'39","last_deep_scrub_stamp":"2024-09-11T08:54:06.928266+0000","last_clean_scrub_stamp":"2024-09-11T08:57:33.510860+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:57.379344+0000","scrub_duration":0.0036910879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":51,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547085+0000","last_change":"2024-09-11T08:56:25.461213+0000","last_active":"2024-09-11T08:57:33.547085+0000","last_peered":"2024-09-11T08:57:33.547085+0000","last_clean":"2024-09-11T08:57:33.547085+0000","last_became_active":"2024-09-11T08:56:25.460770+0000","last_became_peered":"2024-09-11T08:56:25.460770+0000","last_unstale":"2024-09-11T08:57:33.547085+0000","last_undegraded":"2024-09-11T08:57:33.547085+0000","last_fullsized":"2024-09-11T08:57:33.547085+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:07.504031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:07.504031+0000","last_clean_scrub_stamp":"2024-09-11T08:55:07.504031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":422,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.551899+0000","last_change":"2024-09-11T08:56:30.438479+0000","last_active":"2024-09-11T08:57:33.551899+0000","last_peered":"2024-09-11T08:57:33.551899+0000","last_clean":"2024-09-11T08:57:33.551899+0000","last_became_active":"2024-09-11T08:52:18.577201+0000","last_became_peered":"2024-09-11T08:52:18.577201+0000","last_unstale":"2024-09-11T08:57:33.551899+0000","last_undegraded":"2024-09-11T08:57:33.551899+0000","last_fullsized":"2024-09-11T08:57:33.551899+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:30.438424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:56:30.438424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:58.599761+0000","scrub_duration":0.001194999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":77,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.553235+0000","last_change":"2024-09-11T08:55:49.845729+0000","last_active":"2024-09-11T08:57:33.553235+0000","last_peered":"2024-09-11T08:57:33.553235+0000","last_clean":"2024-09-11T08:57:33.553235+0000","last_became_active":"2024-09-11T08:54:17.121372+0000","last_became_peered":"2024-09-11T08:54:17.121372+0000","last_unstale":"2024-09-11T08:57:33.553235+0000","last_undegraded":"2024-09-11T08:57:33.553235+0000","last_fullsized":"2024-09-11T08:57:33.553235+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:49.845654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:55:49.845654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018187542000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3","version":"374'1410","reported_seq":1426,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.880706+0000","last_change":"2024-09-11T08:57:14.487789+0000","last_active":"2024-09-11T08:57:36.880706+0000","last_peered":"2024-09-11T08:57:36.880706+0000","last_clean":"2024-09-11T08:57:36.880706+0000","last_became_active":"2024-09-11T08:57:14.487485+0000","last_became_peered":"2024-09-11T08:57:14.487485+0000","last_unstale":"2024-09-11T08:57:36.880706+0000","last_undegraded":"2024-09-11T08:57:36.880706+0000","last_fullsized":"2024-09-11T08:57:36.880706+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1410,"log_dups_size":0,"ondisk_log_size":1410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:24.896368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92405760,"num_objects":1410,"num_object_clones":0,"num_object_copies":4230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1410,"num_write_kb":90240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"199'81","reported_seq":436,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.558039+0000","last_change":"2024-09-11T08:54:51.543848+0000","last_active":"2024-09-11T08:57:33.558039+0000","last_peered":"2024-09-11T08:57:33.558039+0000","last_clean":"2024-09-11T08:57:33.558039+0000","last_became_active":"2024-09-11T08:52:38.481475+0000","last_became_peered":"2024-09-11T08:52:38.481475+0000","last_unstale":"2024-09-11T08:57:33.558039+0000","last_undegraded":"2024-09-11T08:57:33.558039+0000","last_fullsized":"2024-09-11T08:57:33.558039+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:54:51.543754+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:54:51.543754+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034342589999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"200'20","reported_seq":455,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548132+0000","last_change":"2024-09-11T08:55:52.656041+0000","last_active":"2024-09-11T08:57:33.548132+0000","last_peered":"2024-09-11T08:57:33.548132+0000","last_clean":"2024-09-11T08:57:33.548132+0000","last_became_active":"2024-09-11T08:46:01.988898+0000","last_became_peered":"2024-09-11T08:46:01.988898+0000","last_unstale":"2024-09-11T08:57:33.548132+0000","last_undegraded":"2024-09-11T08:57:33.548132+0000","last_fullsized":"2024-09-11T08:57:33.548132+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'20","last_scrub_stamp":"2024-09-11T08:55:52.655954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:52.655954+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012388883,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":406,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.552090+0000","last_change":"2024-09-11T08:55:56.775203+0000","last_active":"2024-09-11T08:57:33.552090+0000","last_peered":"2024-09-11T08:57:33.552090+0000","last_clean":"2024-09-11T08:57:33.552090+0000","last_became_active":"2024-09-11T08:52:18.586343+0000","last_became_peered":"2024-09-11T08:52:18.586343+0000","last_unstale":"2024-09-11T08:57:33.552090+0000","last_undegraded":"2024-09-11T08:57:33.552090+0000","last_fullsized":"2024-09-11T08:57:33.552090+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:56.775123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:55:56.775123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0096545599999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"299'9","reported_seq":670,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.553393+0000","last_change":"2024-09-11T08:56:33.792530+0000","last_active":"2024-09-11T08:57:33.553393+0000","last_peered":"2024-09-11T08:57:33.553393+0000","last_clean":"2024-09-11T08:57:33.553393+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T08:57:33.553393+0000","last_undegraded":"2024-09-11T08:57:33.553393+0000","last_fullsized":"2024-09-11T08:57:33.553393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'9","last_scrub_stamp":"2024-09-11T08:56:33.792380+0000","last_deep_scrub":"299'9","last_deep_scrub_stamp":"2024-09-11T08:53:05.835177+0000","last_clean_scrub_stamp":"2024-09-11T08:56:33.792380+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:01.264643+0000","scrub_duration":0.0021331810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":391,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548288+0000","last_change":"2024-09-11T08:56:36.675218+0000","last_active":"2024-09-11T08:57:33.548288+0000","last_peered":"2024-09-11T08:57:33.548288+0000","last_clean":"2024-09-11T08:57:33.548288+0000","last_became_active":"2024-09-11T08:56:25.457566+0000","last_became_peered":"2024-09-11T08:56:25.457566+0000","last_unstale":"2024-09-11T08:57:33.548288+0000","last_undegraded":"2024-09-11T08:57:33.548288+0000","last_fullsized":"2024-09-11T08:57:33.548288+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:36.675151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T08:56:36.675151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:55.725829+0000","scrub_duration":0.0016343880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"374'1403","reported_seq":1419,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.551008+0000","last_change":"2024-09-11T08:57:14.487127+0000","last_active":"2024-09-11T08:57:36.551008+0000","last_peered":"2024-09-11T08:57:36.551008+0000","last_clean":"2024-09-11T08:57:36.551008+0000","last_became_active":"2024-09-11T08:57:14.486806+0000","last_became_peered":"2024-09-11T08:57:14.486806+0000","last_unstale":"2024-09-11T08:57:36.551008+0000","last_undegraded":"2024-09-11T08:57:36.551008+0000","last_fullsized":"2024-09-11T08:57:36.551008+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1403,"log_dups_size":0,"ondisk_log_size":1403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:24.416692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91947008,"num_objects":1403,"num_object_clones":0,"num_object_copies":4209,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1403,"num_write_kb":89792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"200'15","reported_seq":156,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.555675+0000","last_change":"2024-09-11T08:57:22.719572+0000","last_active":"2024-09-11T08:57:33.555675+0000","last_peered":"2024-09-11T08:57:33.555675+0000","last_clean":"2024-09-11T08:57:33.555675+0000","last_became_active":"2024-09-11T08:52:18.588329+0000","last_became_peered":"2024-09-11T08:52:18.588329+0000","last_unstale":"2024-09-11T08:57:33.555675+0000","last_undegraded":"2024-09-11T08:57:33.555675+0000","last_fullsized":"2024-09-11T08:57:33.555675+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:57:22.719482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:57:22.719482+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:28.861771+0000","scrub_duration":0.001683714,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":399,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548300+0000","last_change":"2024-09-11T08:56:35.318047+0000","last_active":"2024-09-11T08:57:33.548300+0000","last_peered":"2024-09-11T08:57:33.548300+0000","last_clean":"2024-09-11T08:57:33.548300+0000","last_became_active":"2024-09-11T08:38:38.945370+0000","last_became_peered":"2024-09-11T08:38:38.945370+0000","last_unstale":"2024-09-11T08:57:33.548300+0000","last_undegraded":"2024-09-11T08:57:33.548300+0000","last_fullsized":"2024-09-11T08:57:33.548300+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:35.317983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:56:35.317983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:01.423122+0000","scrub_duration":0.0037140580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":67,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546843+0000","last_change":"2024-09-11T08:54:17.120062+0000","last_active":"2024-09-11T08:57:33.546843+0000","last_peered":"2024-09-11T08:57:33.546843+0000","last_clean":"2024-09-11T08:57:33.546843+0000","last_became_active":"2024-09-11T08:54:17.119864+0000","last_became_peered":"2024-09-11T08:54:17.119864+0000","last_unstale":"2024-09-11T08:57:33.546843+0000","last_undegraded":"2024-09-11T08:57:33.546843+0000","last_fullsized":"2024-09-11T08:57:33.546843+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:06.683645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:06.683645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":129,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.553002+0000","last_change":"2024-09-11T08:56:38.833414+0000","last_active":"2024-09-11T08:57:33.553002+0000","last_peered":"2024-09-11T08:57:33.553002+0000","last_clean":"2024-09-11T08:57:33.553002+0000","last_became_active":"2024-09-11T08:52:49.569749+0000","last_became_peered":"2024-09-11T08:52:49.569749+0000","last_unstale":"2024-09-11T08:57:33.553002+0000","last_undegraded":"2024-09-11T08:57:33.553002+0000","last_fullsized":"2024-09-11T08:57:33.553002+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:38.833352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:13.770061+0000","last_clean_scrub_stamp":"2024-09-11T08:56:38.833352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:43.628241+0000","scrub_duration":0.0018346,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"374'1455","reported_seq":1471,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.900324+0000","last_change":"2024-09-11T08:57:14.488032+0000","last_active":"2024-09-11T08:57:36.900324+0000","last_peered":"2024-09-11T08:57:36.900324+0000","last_clean":"2024-09-11T08:57:36.900324+0000","last_became_active":"2024-09-11T08:57:14.486890+0000","last_became_peered":"2024-09-11T08:57:14.486890+0000","last_unstale":"2024-09-11T08:57:36.900324+0000","last_undegraded":"2024-09-11T08:57:36.900324+0000","last_fullsized":"2024-09-11T08:57:36.900324+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1455,"log_dups_size":0,"ondisk_log_size":1455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:17.037649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95354880,"num_objects":1455,"num_object_clones":0,"num_object_copies":4365,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1455,"num_write_kb":93120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"374'44","reported_seq":154,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:34.897622+0000","last_change":"2024-09-11T08:57:34.897564+0000","last_active":"2024-09-11T08:57:34.897622+0000","last_peered":"2024-09-11T08:57:34.897622+0000","last_clean":"2024-09-11T08:57:34.897622+0000","last_became_active":"2024-09-11T08:56:08.478328+0000","last_became_peered":"2024-09-11T08:56:08.478328+0000","last_unstale":"2024-09-11T08:57:34.897622+0000","last_undegraded":"2024-09-11T08:57:34.897622+0000","last_fullsized":"2024-09-11T08:57:34.897622+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"374'44","last_scrub_stamp":"2024-09-11T08:57:34.896912+0000","last_deep_scrub":"374'44","last_deep_scrub_stamp":"2024-09-11T08:57:34.896912+0000","last_clean_scrub_stamp":"2024-09-11T08:57:34.896912+0000","objects_scrubbed":9,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:36.073987+0000","scrub_duration":0.011841266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":54,"num_read_kb":43,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":120,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547760+0000","last_change":"2024-09-11T08:57:24.028483+0000","last_active":"2024-09-11T08:57:33.547760+0000","last_peered":"2024-09-11T08:57:33.547760+0000","last_clean":"2024-09-11T08:57:33.547760+0000","last_became_active":"2024-09-11T08:52:38.484857+0000","last_became_peered":"2024-09-11T08:52:38.484857+0000","last_unstale":"2024-09-11T08:57:33.547760+0000","last_undegraded":"2024-09-11T08:57:33.547760+0000","last_fullsized":"2024-09-11T08:57:33.547760+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:24.028412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:57:24.028412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:47.943054+0000","scrub_duration":0.001520041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":454,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.554040+0000","last_change":"2024-09-11T08:56:46.744628+0000","last_active":"2024-09-11T08:57:33.554040+0000","last_peered":"2024-09-11T08:57:33.554040+0000","last_clean":"2024-09-11T08:57:33.554040+0000","last_became_active":"2024-09-11T08:52:18.584748+0000","last_became_peered":"2024-09-11T08:52:18.584748+0000","last_unstale":"2024-09-11T08:57:33.554040+0000","last_undegraded":"2024-09-11T08:57:33.554040+0000","last_fullsized":"2024-09-11T08:57:33.554040+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:46.744542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:56:46.744542+0000","last_clean_scrub_stamp":"2024-09-11T08:56:46.744542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:09.731642+0000","scrub_duration":0.001567075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":71,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.550273+0000","last_change":"2024-09-11T08:54:17.155880+0000","last_active":"2024-09-11T08:57:33.550273+0000","last_peered":"2024-09-11T08:57:33.550273+0000","last_clean":"2024-09-11T08:57:33.550273+0000","last_became_active":"2024-09-11T08:54:17.155625+0000","last_became_peered":"2024-09-11T08:54:17.155625+0000","last_unstale":"2024-09-11T08:57:33.550273+0000","last_undegraded":"2024-09-11T08:57:33.550273+0000","last_fullsized":"2024-09-11T08:57:33.550273+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:14.828985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:14.828985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"374'1432","reported_seq":1448,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.926991+0000","last_change":"2024-09-11T08:57:14.485796+0000","last_active":"2024-09-11T08:57:36.926991+0000","last_peered":"2024-09-11T08:57:36.926991+0000","last_clean":"2024-09-11T08:57:36.926991+0000","last_became_active":"2024-09-11T08:57:14.485531+0000","last_became_peered":"2024-09-11T08:57:14.485531+0000","last_unstale":"2024-09-11T08:57:36.926991+0000","last_undegraded":"2024-09-11T08:57:36.926991+0000","last_fullsized":"2024-09-11T08:57:36.926991+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1432,"log_dups_size":0,"ondisk_log_size":1432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:16.908881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93847552,"num_objects":1432,"num_object_clones":0,"num_object_copies":4296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1432,"num_write_kb":91648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"302'23","reported_seq":422,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546584+0000","last_change":"2024-09-11T08:57:19.603379+0000","last_active":"2024-09-11T08:57:33.546584+0000","last_peered":"2024-09-11T08:57:33.546584+0000","last_clean":"2024-09-11T08:57:33.546584+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T08:57:33.546584+0000","last_undegraded":"2024-09-11T08:57:33.546584+0000","last_fullsized":"2024-09-11T08:57:33.546584+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'23","last_scrub_stamp":"2024-09-11T08:57:19.603310+0000","last_deep_scrub":"302'23","last_deep_scrub_stamp":"2024-09-11T08:53:16.896643+0000","last_clean_scrub_stamp":"2024-09-11T08:57:19.603310+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:36.572594+0000","scrub_duration":0.0023914409999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":110,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548698+0000","last_change":"2024-09-11T08:55:02.779440+0000","last_active":"2024-09-11T08:57:33.548698+0000","last_peered":"2024-09-11T08:57:33.548698+0000","last_clean":"2024-09-11T08:57:33.548698+0000","last_became_active":"2024-09-11T08:52:38.487881+0000","last_became_peered":"2024-09-11T08:52:38.487881+0000","last_unstale":"2024-09-11T08:57:33.548698+0000","last_undegraded":"2024-09-11T08:57:33.548698+0000","last_fullsized":"2024-09-11T08:57:33.548698+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:02.779144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:24.331966+0000","last_clean_scrub_stamp":"2024-09-11T08:55:02.779144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.026798161000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":396,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.551685+0000","last_change":"2024-09-11T08:56:42.551883+0000","last_active":"2024-09-11T08:57:33.551685+0000","last_peered":"2024-09-11T08:57:33.551685+0000","last_clean":"2024-09-11T08:57:33.551685+0000","last_became_active":"2024-09-11T08:54:17.122020+0000","last_became_peered":"2024-09-11T08:54:17.122020+0000","last_unstale":"2024-09-11T08:57:33.551685+0000","last_undegraded":"2024-09-11T08:57:33.551685+0000","last_fullsized":"2024-09-11T08:57:33.551685+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:42.551816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:56:42.551816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:49.859574+0000","scrub_duration":0.0011279549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":380,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.550436+0000","last_change":"2024-09-11T08:56:51.873475+0000","last_active":"2024-09-11T08:57:33.550436+0000","last_peered":"2024-09-11T08:57:33.550436+0000","last_clean":"2024-09-11T08:57:33.550436+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T08:57:33.550436+0000","last_undegraded":"2024-09-11T08:57:33.550436+0000","last_fullsized":"2024-09-11T08:57:33.550436+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:51.873420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:56:51.873420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:17.933859+0000","scrub_duration":0.001764933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"374'1439","reported_seq":1455,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:36.901420+0000","last_change":"2024-09-11T08:57:14.474857+0000","last_active":"2024-09-11T08:57:36.901420+0000","last_peered":"2024-09-11T08:57:36.901420+0000","last_clean":"2024-09-11T08:57:36.901420+0000","last_became_active":"2024-09-11T08:57:14.474567+0000","last_became_peered":"2024-09-11T08:57:14.474567+0000","last_unstale":"2024-09-11T08:57:36.901420+0000","last_undegraded":"2024-09-11T08:57:36.901420+0000","last_fullsized":"2024-09-11T08:57:36.901420+0000","mapping_epoch":366,"log_start":"0'0","ondisk_log_start":"0'0","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":1439,"log_dups_size":0,"ondisk_log_size":1439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:41.871042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94306304,"num_objects":1439,"num_object_clones":0,"num_object_copies":4317,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1439,"num_write_kb":92096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"200'34","reported_seq":502,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.545925+0000","last_change":"2024-09-11T08:57:26.108938+0000","last_active":"2024-09-11T08:57:33.545925+0000","last_peered":"2024-09-11T08:57:33.545925+0000","last_clean":"2024-09-11T08:57:33.545925+0000","last_became_active":"2024-09-11T08:46:01.981928+0000","last_became_peered":"2024-09-11T08:46:01.981928+0000","last_unstale":"2024-09-11T08:57:33.545925+0000","last_undegraded":"2024-09-11T08:57:33.545925+0000","last_fullsized":"2024-09-11T08:57:33.545925+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:57:26.108866+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:57:26.108866+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:51.965531+0000","scrub_duration":0.0023139469999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":129,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548567+0000","last_change":"2024-09-11T08:56:32.686448+0000","last_active":"2024-09-11T08:57:33.548567+0000","last_peered":"2024-09-11T08:57:33.548567+0000","last_clean":"2024-09-11T08:57:33.548567+0000","last_became_active":"2024-09-11T08:52:38.484199+0000","last_became_peered":"2024-09-11T08:52:38.484199+0000","last_unstale":"2024-09-11T08:57:33.548567+0000","last_undegraded":"2024-09-11T08:57:33.548567+0000","last_fullsized":"2024-09-11T08:57:33.548567+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:56:32.686360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:56:32.686360+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:02.159812+0000","scrub_duration":0.0082837799999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"346'12","reported_seq":79,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546484+0000","last_change":"2024-09-11T08:56:08.480204+0000","last_active":"2024-09-11T08:57:33.546484+0000","last_peered":"2024-09-11T08:57:33.546484+0000","last_clean":"2024-09-11T08:57:33.546484+0000","last_became_active":"2024-09-11T08:56:08.479354+0000","last_became_peered":"2024-09-11T08:56:08.479354+0000","last_unstale":"2024-09-11T08:57:33.546484+0000","last_undegraded":"2024-09-11T08:57:33.546484+0000","last_fullsized":"2024-09-11T08:57:33.546484+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"287'11","last_scrub_stamp":"2024-09-11T08:54:13.613459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:13.613459+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":406,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546217+0000","last_change":"2024-09-11T08:54:56.341981+0000","last_active":"2024-09-11T08:57:33.546217+0000","last_peered":"2024-09-11T08:57:33.546217+0000","last_clean":"2024-09-11T08:57:33.546217+0000","last_became_active":"2024-09-11T08:52:18.573947+0000","last_became_peered":"2024-09-11T08:52:18.573947+0000","last_unstale":"2024-09-11T08:57:33.546217+0000","last_undegraded":"2024-09-11T08:57:33.546217+0000","last_fullsized":"2024-09-11T08:57:33.546217+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:56.341896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:54:56.341896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.13742462999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":371,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.553613+0000","last_change":"2024-09-11T08:56:28.649422+0000","last_active":"2024-09-11T08:57:33.553613+0000","last_peered":"2024-09-11T08:57:33.553613+0000","last_clean":"2024-09-11T08:57:33.553613+0000","last_became_active":"2024-09-11T08:56:08.480043+0000","last_became_peered":"2024-09-11T08:56:08.480043+0000","last_unstale":"2024-09-11T08:57:33.553613+0000","last_undegraded":"2024-09-11T08:57:33.553613+0000","last_fullsized":"2024-09-11T08:57:33.553613+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:28.649338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T08:56:28.649338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:48.748129+0000","scrub_duration":0.0036325789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"200'12","reported_seq":420,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.553669+0000","last_change":"2024-09-11T08:55:12.678723+0000","last_active":"2024-09-11T08:57:33.553669+0000","last_peered":"2024-09-11T08:57:33.553669+0000","last_clean":"2024-09-11T08:57:33.553669+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T08:57:33.553669+0000","last_undegraded":"2024-09-11T08:57:33.553669+0000","last_fullsized":"2024-09-11T08:57:33.553669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'12","last_scrub_stamp":"2024-09-11T08:55:12.678247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:12.678247+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020917618999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":392,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547482+0000","last_change":"2024-09-11T08:56:36.281964+0000","last_active":"2024-09-11T08:57:33.547482+0000","last_peered":"2024-09-11T08:57:33.547482+0000","last_clean":"2024-09-11T08:57:33.547482+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T08:57:33.547482+0000","last_undegraded":"2024-09-11T08:57:33.547482+0000","last_fullsized":"2024-09-11T08:57:33.547482+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:36.281886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:56:36.281886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:41.829744+0000","scrub_duration":0.0018509690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":429,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.555647+0000","last_change":"2024-09-11T08:57:21.766142+0000","last_active":"2024-09-11T08:57:33.555647+0000","last_peered":"2024-09-11T08:57:33.555647+0000","last_clean":"2024-09-11T08:57:33.555647+0000","last_became_active":"2024-09-11T08:46:01.979426+0000","last_became_peered":"2024-09-11T08:46:01.979426+0000","last_unstale":"2024-09-11T08:57:33.555647+0000","last_undegraded":"2024-09-11T08:57:33.555647+0000","last_fullsized":"2024-09-11T08:57:33.555647+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:21.766084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:57:21.766084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:25.894516+0000","scrub_duration":0.0031144699999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"200'15","reported_seq":426,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.553003+0000","last_change":"2024-09-11T08:57:27.752958+0000","last_active":"2024-09-11T08:57:33.553003+0000","last_peered":"2024-09-11T08:57:33.553003+0000","last_clean":"2024-09-11T08:57:33.553003+0000","last_became_active":"2024-09-11T08:52:49.569850+0000","last_became_peered":"2024-09-11T08:52:49.569850+0000","last_unstale":"2024-09-11T08:57:33.553003+0000","last_undegraded":"2024-09-11T08:57:33.553003+0000","last_fullsized":"2024-09-11T08:57:33.553003+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:57:27.752906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:57:27.752906+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:40.758576+0000","scrub_duration":0.0020717140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":110,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546692+0000","last_change":"2024-09-11T08:54:53.712104+0000","last_active":"2024-09-11T08:57:33.546692+0000","last_peered":"2024-09-11T08:57:33.546692+0000","last_clean":"2024-09-11T08:57:33.546692+0000","last_became_active":"2024-09-11T08:52:38.485708+0000","last_became_peered":"2024-09-11T08:52:38.485708+0000","last_unstale":"2024-09-11T08:57:33.546692+0000","last_undegraded":"2024-09-11T08:57:33.546692+0000","last_fullsized":"2024-09-11T08:57:33.546692+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:53.709804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:54:53.709804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042052879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":382,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:34.759441+0000","last_change":"2024-09-11T08:57:34.759414+0000","last_active":"2024-09-11T08:57:34.759441+0000","last_peered":"2024-09-11T08:57:34.759441+0000","last_clean":"2024-09-11T08:57:34.759441+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T08:57:34.759441+0000","last_undegraded":"2024-09-11T08:57:34.759441+0000","last_fullsized":"2024-09-11T08:57:34.759441+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:34.759355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:57:34.759355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:59.527293+0000","scrub_duration":0.0039078510000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":430,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.554358+0000","last_change":"2024-09-11T08:57:21.567429+0000","last_active":"2024-09-11T08:57:33.554358+0000","last_peered":"2024-09-11T08:57:33.554358+0000","last_clean":"2024-09-11T08:57:33.554358+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T08:57:33.554358+0000","last_undegraded":"2024-09-11T08:57:33.554358+0000","last_fullsized":"2024-09-11T08:57:33.554358+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:21.567295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:47:34.420932+0000","last_clean_scrub_stamp":"2024-09-11T08:57:21.567295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:51.197559+0000","scrub_duration":0.001589088,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"200'15","reported_seq":435,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547763+0000","last_change":"2024-09-11T08:54:08.406799+0000","last_active":"2024-09-11T08:57:33.547763+0000","last_peered":"2024-09-11T08:57:33.547763+0000","last_clean":"2024-09-11T08:57:33.547763+0000","last_became_active":"2024-09-11T08:53:58.555987+0000","last_became_peered":"2024-09-11T08:53:58.555987+0000","last_unstale":"2024-09-11T08:57:33.547763+0000","last_undegraded":"2024-09-11T08:57:33.547763+0000","last_fullsized":"2024-09-11T08:57:33.547763+0000","mapping_epoch":315,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:54:08.406695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:08.406695+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001945411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":110,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.558907+0000","last_change":"2024-09-11T08:55:03.641087+0000","last_active":"2024-09-11T08:57:33.558907+0000","last_peered":"2024-09-11T08:57:33.558907+0000","last_clean":"2024-09-11T08:57:33.558907+0000","last_became_active":"2024-09-11T08:52:49.566759+0000","last_became_peered":"2024-09-11T08:52:49.566759+0000","last_unstale":"2024-09-11T08:57:33.558907+0000","last_undegraded":"2024-09-11T08:57:33.558907+0000","last_fullsized":"2024-09-11T08:57:33.558907+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:03.641009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:03.641009+0000","last_clean_scrub_stamp":"2024-09-11T08:55:03.641009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057082440000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":617,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.554092+0000","last_change":"2024-09-11T08:57:28.788665+0000","last_active":"2024-09-11T08:57:33.554092+0000","last_peered":"2024-09-11T08:57:33.554092+0000","last_clean":"2024-09-11T08:57:33.554092+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T08:57:33.554092+0000","last_undegraded":"2024-09-11T08:57:33.554092+0000","last_fullsized":"2024-09-11T08:57:33.554092+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:57:28.788610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:57:28.788610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:52.269963+0000","scrub_duration":0.001594315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":374,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548290+0000","last_change":"2024-09-11T08:55:50.590487+0000","last_active":"2024-09-11T08:57:33.548290+0000","last_peered":"2024-09-11T08:57:33.548290+0000","last_clean":"2024-09-11T08:57:33.548290+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T08:57:33.548290+0000","last_undegraded":"2024-09-11T08:57:33.548290+0000","last_fullsized":"2024-09-11T08:57:33.548290+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:50.590410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:55:50.590410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044493969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"200'25","reported_seq":441,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.554343+0000","last_change":"2024-09-11T08:56:25.484599+0000","last_active":"2024-09-11T08:57:33.554343+0000","last_peered":"2024-09-11T08:57:33.554343+0000","last_clean":"2024-09-11T08:57:33.554343+0000","last_became_active":"2024-09-11T08:56:25.465124+0000","last_became_peered":"2024-09-11T08:56:25.465124+0000","last_unstale":"2024-09-11T08:57:33.554343+0000","last_undegraded":"2024-09-11T08:57:33.554343+0000","last_fullsized":"2024-09-11T08:57:33.554343+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'25","last_scrub_stamp":"2024-09-11T08:55:37.744946+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T08:55:37.744946+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064657020000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":425,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.555966+0000","last_change":"2024-09-11T08:57:20.773714+0000","last_active":"2024-09-11T08:57:33.555966+0000","last_peered":"2024-09-11T08:57:33.555966+0000","last_clean":"2024-09-11T08:57:33.555966+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T08:57:33.555966+0000","last_undegraded":"2024-09-11T08:57:33.555966+0000","last_fullsized":"2024-09-11T08:57:33.555966+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:20.773645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:15.235519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:20.773645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:43.082460+0000","scrub_duration":0.001477262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"277'10","reported_seq":655,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547683+0000","last_change":"2024-09-11T08:56:38.309684+0000","last_active":"2024-09-11T08:57:33.547683+0000","last_peered":"2024-09-11T08:57:33.547683+0000","last_clean":"2024-09-11T08:57:33.547683+0000","last_became_active":"2024-09-11T08:52:18.585914+0000","last_became_peered":"2024-09-11T08:52:18.585914+0000","last_unstale":"2024-09-11T08:57:33.547683+0000","last_undegraded":"2024-09-11T08:57:33.547683+0000","last_fullsized":"2024-09-11T08:57:33.547683+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'10","last_scrub_stamp":"2024-09-11T08:56:38.309615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:56:38.309615+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:07.275945+0000","scrub_duration":0.0019391720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":120,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.553075+0000","last_change":"2024-09-11T08:55:46.880868+0000","last_active":"2024-09-11T08:57:33.553075+0000","last_peered":"2024-09-11T08:57:33.553075+0000","last_clean":"2024-09-11T08:57:33.553075+0000","last_became_active":"2024-09-11T08:52:49.567693+0000","last_became_peered":"2024-09-11T08:52:49.567693+0000","last_unstale":"2024-09-11T08:57:33.553075+0000","last_undegraded":"2024-09-11T08:57:33.553075+0000","last_fullsized":"2024-09-11T08:57:33.553075+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:46.880773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:55:46.880773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0067893529999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"200'19","reported_seq":428,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546882+0000","last_change":"2024-09-11T08:54:15.800180+0000","last_active":"2024-09-11T08:57:33.546882+0000","last_peered":"2024-09-11T08:57:33.546882+0000","last_clean":"2024-09-11T08:57:33.546882+0000","last_became_active":"2024-09-11T08:49:16.937185+0000","last_became_peered":"2024-09-11T08:49:16.937185+0000","last_unstale":"2024-09-11T08:57:33.546882+0000","last_undegraded":"2024-09-11T08:57:33.546882+0000","last_fullsized":"2024-09-11T08:57:33.546882+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:54:15.800092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:15.800092+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014286116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":387,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548144+0000","last_change":"2024-09-11T08:57:22.682385+0000","last_active":"2024-09-11T08:57:33.548144+0000","last_peered":"2024-09-11T08:57:33.548144+0000","last_clean":"2024-09-11T08:57:33.548144+0000","last_became_active":"2024-09-11T08:38:38.952327+0000","last_became_peered":"2024-09-11T08:38:38.952327+0000","last_unstale":"2024-09-11T08:57:33.548144+0000","last_undegraded":"2024-09-11T08:57:33.548144+0000","last_fullsized":"2024-09-11T08:57:33.548144+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:22.682317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:57:22.682317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:32.056388+0000","scrub_duration":0.0014903729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":412,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.552089+0000","last_change":"2024-09-11T08:57:23.706522+0000","last_active":"2024-09-11T08:57:33.552089+0000","last_peered":"2024-09-11T08:57:33.552089+0000","last_clean":"2024-09-11T08:57:33.552089+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T08:57:33.552089+0000","last_undegraded":"2024-09-11T08:57:33.552089+0000","last_fullsized":"2024-09-11T08:57:33.552089+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:23.706463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:57:23.706463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:31.865709+0000","scrub_duration":0.0013292969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":400,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.558143+0000","last_change":"2024-09-11T08:56:47.833618+0000","last_active":"2024-09-11T08:57:33.558143+0000","last_peered":"2024-09-11T08:57:33.558143+0000","last_clean":"2024-09-11T08:57:33.558143+0000","last_became_active":"2024-09-11T08:52:18.585089+0000","last_became_peered":"2024-09-11T08:52:18.585089+0000","last_unstale":"2024-09-11T08:57:33.558143+0000","last_undegraded":"2024-09-11T08:57:33.558143+0000","last_fullsized":"2024-09-11T08:57:33.558143+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:47.833532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:53:21.391849+0000","last_clean_scrub_stamp":"2024-09-11T08:56:47.833532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:04.207503+0000","scrub_duration":0.001512191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"200'15","reported_seq":285,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.551202+0000","last_change":"2024-09-11T08:56:33.318042+0000","last_active":"2024-09-11T08:57:33.551202+0000","last_peered":"2024-09-11T08:57:33.551202+0000","last_clean":"2024-09-11T08:57:33.551202+0000","last_became_active":"2024-09-11T08:56:25.462496+0000","last_became_peered":"2024-09-11T08:56:25.462496+0000","last_unstale":"2024-09-11T08:57:33.551202+0000","last_undegraded":"2024-09-11T08:57:33.551202+0000","last_fullsized":"2024-09-11T08:57:33.551202+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:56:33.317985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:56:33.317985+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:54.108040+0000","scrub_duration":0.001689821,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":120,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.552518+0000","last_change":"2024-09-11T08:56:31.849778+0000","last_active":"2024-09-11T08:57:33.552518+0000","last_peered":"2024-09-11T08:57:33.552518+0000","last_clean":"2024-09-11T08:57:33.552518+0000","last_became_active":"2024-09-11T08:52:38.485748+0000","last_became_peered":"2024-09-11T08:52:38.485748+0000","last_unstale":"2024-09-11T08:57:33.552518+0000","last_undegraded":"2024-09-11T08:57:33.552518+0000","last_fullsized":"2024-09-11T08:57:33.552518+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:31.849658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:21.208684+0000","last_clean_scrub_stamp":"2024-09-11T08:56:31.849658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:00.515178+0000","scrub_duration":0.0079402989999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":402,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.550511+0000","last_change":"2024-09-11T08:55:42.795101+0000","last_active":"2024-09-11T08:57:33.550511+0000","last_peered":"2024-09-11T08:57:33.550511+0000","last_clean":"2024-09-11T08:57:33.550511+0000","last_became_active":"2024-09-11T08:52:18.585691+0000","last_became_peered":"2024-09-11T08:52:18.585691+0000","last_unstale":"2024-09-11T08:57:33.550511+0000","last_undegraded":"2024-09-11T08:57:33.550511+0000","last_fullsized":"2024-09-11T08:57:33.550511+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:42.795019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:55:42.795019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020422750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":202,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.989147+0000","last_change":"2024-09-11T08:57:33.989083+0000","last_active":"2024-09-11T08:57:33.989147+0000","last_peered":"2024-09-11T08:57:33.989147+0000","last_clean":"2024-09-11T08:57:33.989147+0000","last_became_active":"2024-09-11T08:52:49.566694+0000","last_became_peered":"2024-09-11T08:52:49.566694+0000","last_unstale":"2024-09-11T08:57:33.989147+0000","last_undegraded":"2024-09-11T08:57:33.989147+0000","last_fullsized":"2024-09-11T08:57:33.989147+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:33.988992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:24.401480+0000","last_clean_scrub_stamp":"2024-09-11T08:57:33.988992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:53.239255+0000","scrub_duration":0.0020405710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"200'19","reported_seq":405,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.557975+0000","last_change":"2024-09-11T08:54:53.565356+0000","last_active":"2024-09-11T08:57:33.557975+0000","last_peered":"2024-09-11T08:57:33.557975+0000","last_clean":"2024-09-11T08:57:33.557975+0000","last_became_active":"2024-09-11T08:38:41.039583+0000","last_became_peered":"2024-09-11T08:38:41.039583+0000","last_unstale":"2024-09-11T08:57:33.557975+0000","last_undegraded":"2024-09-11T08:57:33.557975+0000","last_fullsized":"2024-09-11T08:57:33.557975+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:54:53.565275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:54:53.565275+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061943880000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":406,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.547654+0000","last_change":"2024-09-11T08:56:25.460703+0000","last_active":"2024-09-11T08:57:33.547654+0000","last_peered":"2024-09-11T08:57:33.547654+0000","last_clean":"2024-09-11T08:57:33.547654+0000","last_became_active":"2024-09-11T08:56:25.460327+0000","last_became_peered":"2024-09-11T08:56:25.460327+0000","last_unstale":"2024-09-11T08:57:33.547654+0000","last_undegraded":"2024-09-11T08:57:33.547654+0000","last_fullsized":"2024-09-11T08:57:33.547654+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:55:40.622753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:55:40.622753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011839215,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":424,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.555145+0000","last_change":"2024-09-11T08:56:48.554012+0000","last_active":"2024-09-11T08:57:33.555145+0000","last_peered":"2024-09-11T08:57:33.555145+0000","last_clean":"2024-09-11T08:57:33.555145+0000","last_became_active":"2024-09-11T08:38:42.985380+0000","last_became_peered":"2024-09-11T08:38:42.985380+0000","last_unstale":"2024-09-11T08:57:33.555145+0000","last_undegraded":"2024-09-11T08:57:33.555145+0000","last_fullsized":"2024-09-11T08:57:33.555145+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:48.553940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:56:48.553940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:09.335240+0000","scrub_duration":0.0012299329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":420,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.552020+0000","last_change":"2024-09-11T08:57:32.723407+0000","last_active":"2024-09-11T08:57:33.552020+0000","last_peered":"2024-09-11T08:57:33.552020+0000","last_clean":"2024-09-11T08:57:33.552020+0000","last_became_active":"2024-09-11T08:52:18.585315+0000","last_became_peered":"2024-09-11T08:52:18.585315+0000","last_unstale":"2024-09-11T08:57:33.552020+0000","last_undegraded":"2024-09-11T08:57:33.552020+0000","last_fullsized":"2024-09-11T08:57:33.552020+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:32.723309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:25.255332+0000","last_clean_scrub_stamp":"2024-09-11T08:57:32.723309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:49.879256+0000","scrub_duration":0.0030645970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"200'10","reported_seq":433,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.550115+0000","last_change":"2024-09-11T08:55:00.665641+0000","last_active":"2024-09-11T08:57:33.550115+0000","last_peered":"2024-09-11T08:57:33.550115+0000","last_clean":"2024-09-11T08:57:33.550115+0000","last_became_active":"2024-09-11T08:46:01.982857+0000","last_became_peered":"2024-09-11T08:46:01.982857+0000","last_unstale":"2024-09-11T08:57:33.550115+0000","last_undegraded":"2024-09-11T08:57:33.550115+0000","last_fullsized":"2024-09-11T08:57:33.550115+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'10","last_scrub_stamp":"2024-09-11T08:55:00.665557+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:55:00.665557+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031074079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":214,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548801+0000","last_change":"2024-09-11T08:57:25.060216+0000","last_active":"2024-09-11T08:57:33.548801+0000","last_peered":"2024-09-11T08:57:33.548801+0000","last_clean":"2024-09-11T08:57:33.548801+0000","last_became_active":"2024-09-11T08:52:49.569954+0000","last_became_peered":"2024-09-11T08:52:49.569954+0000","last_unstale":"2024-09-11T08:57:33.548801+0000","last_undegraded":"2024-09-11T08:57:33.548801+0000","last_fullsized":"2024-09-11T08:57:33.548801+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:57:25.060145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:57:25.060145+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:39.440943+0000","scrub_duration":0.0017002149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":95,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546429+0000","last_change":"2024-09-11T08:56:08.486420+0000","last_active":"2024-09-11T08:57:33.546429+0000","last_peered":"2024-09-11T08:57:33.546429+0000","last_clean":"2024-09-11T08:57:33.546429+0000","last_became_active":"2024-09-11T08:56:08.483212+0000","last_became_peered":"2024-09-11T08:56:08.483212+0000","last_unstale":"2024-09-11T08:57:33.546429+0000","last_undegraded":"2024-09-11T08:57:33.546429+0000","last_fullsized":"2024-09-11T08:57:33.546429+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:55.481627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:54:55.481627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":71,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.546674+0000","last_change":"2024-09-11T08:54:17.155339+0000","last_active":"2024-09-11T08:57:33.546674+0000","last_peered":"2024-09-11T08:57:33.546674+0000","last_clean":"2024-09-11T08:57:33.546674+0000","last_became_active":"2024-09-11T08:54:17.154933+0000","last_became_peered":"2024-09-11T08:54:17.154933+0000","last_unstale":"2024-09-11T08:57:33.546674+0000","last_undegraded":"2024-09-11T08:57:33.546674+0000","last_fullsized":"2024-09-11T08:57:33.546674+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:54:11.694784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:30.788303+0000","last_clean_scrub_stamp":"2024-09-11T08:54:11.694784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":410,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.552493+0000","last_change":"2024-09-11T08:56:35.508830+0000","last_active":"2024-09-11T08:57:33.552493+0000","last_peered":"2024-09-11T08:57:33.552493+0000","last_clean":"2024-09-11T08:57:33.552493+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T08:57:33.552493+0000","last_undegraded":"2024-09-11T08:57:33.552493+0000","last_fullsized":"2024-09-11T08:57:33.552493+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:35.508746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:56:35.508746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:37.718672+0000","scrub_duration":0.0016595760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":426,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-11T08:57:35.763565+0000","last_change":"2024-09-11T08:57:35.763188+0000","last_active":"2024-09-11T08:57:35.763565+0000","last_peered":"2024-09-11T08:57:35.763565+0000","last_clean":"2024-09-11T08:57:35.763565+0000","last_became_active":"2024-09-11T08:56:25.458500+0000","last_became_peered":"2024-09-11T08:56:25.458500+0000","last_unstale":"2024-09-11T08:57:35.763565+0000","last_undegraded":"2024-09-11T08:57:35.763565+0000","last_fullsized":"2024-09-11T08:57:35.763565+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:35.763114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:32.393197+0000","last_clean_scrub_stamp":"2024-09-11T08:57:35.763114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:56.760811+0000","scrub_duration":0.0042514889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"200'19","reported_seq":115,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.548751+0000","last_change":"2024-09-11T08:56:08.482039+0000","last_active":"2024-09-11T08:57:33.548751+0000","last_peered":"2024-09-11T08:57:33.548751+0000","last_clean":"2024-09-11T08:57:33.548751+0000","last_became_active":"2024-09-11T08:56:08.481037+0000","last_became_peered":"2024-09-11T08:56:08.481037+0000","last_unstale":"2024-09-11T08:57:33.548751+0000","last_undegraded":"2024-09-11T08:57:33.548751+0000","last_fullsized":"2024-09-11T08:57:33.548751+0000","mapping_epoch":345,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":346,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:55:48.566907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:55:48.566907+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0096371630000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":211,"reported_epoch":373,"state":"active+clean","last_fresh":"2024-09-11T08:57:33.555819+0000","last_change":"2024-09-11T08:56:40.575923+0000","last_active":"2024-09-11T08:57:33.555819+0000","last_peered":"2024-09-11T08:57:33.555819+0000","last_clean":"2024-09-11T08:57:33.555819+0000","last_became_active":"2024-09-11T08:52:49.568537+0000","last_became_peered":"2024-09-11T08:52:49.568537+0000","last_unstale":"2024-09-11T08:57:33.555819+0000","last_undegraded":"2024-09-11T08:57:33.555819+0000","last_fullsized":"2024-09-11T08:57:33.555819+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:40.575825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:56:40.575825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:58:06.734704+0000","scrub_duration":0.00153857,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":1454505984,"num_objects":22194,"num_object_clones":0,"num_object_copies":66582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22194,"num_write_kb":1420416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4392484864,"data_stored":4392484864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22194,"ondisk_log_size":22194,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1112,"num_read_kb":869,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4582,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":779,"ondisk_log_size":779,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":166,"num_read_kb":161,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":697708,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":42,"num_write":20,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299625,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1372892,"kb_used_data":471508,"kb_used_omap":1,"kb_used_meta":901374,"kb_avail":92372260,"statfs":{"total":95995035648,"available":94589194240,"internally_reserved":0,"allocated":482824192,"data_stored":482453992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":923007320},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.92100000000000004,"15min":1.0449999999999999},"min":{"1min":0.379,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.8779999999999999,"5min":5.0449999999999999,"15min":32.171999999999997},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.82599999999999996,"15min":1.109},"min":{"1min":0.30399999999999999,"5min":0.23200000000000001,"15min":0.21199999999999999},"max":{"1min":3.4780000000000002,"5min":3.4780000000000002,"15min":32.149999999999999},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed Sep 11 08:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.85599999999999998,"15min":1.1879999999999999},"min":{"1min":0.38500000000000001,"5min":0.251,"15min":0.224},"max":{"1min":3.7719999999999998,"5min":4.3449999999999998,"15min":33.692999999999998},"last":2.363},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.67500000000000004,"15min":1.0509999999999999},"min":{"1min":0.26200000000000001,"5min":0.26000000000000001,"15min":0.23300000000000001},"max":{"1min":1.6359999999999999,"5min":2.9590000000000001,"15min":35.591000000000001},"last":0.53600000000000003}]},{"osd":2,"last update":"Wed Sep 11 08:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.97099999999999997,"15min":1.046},"min":{"1min":0.27200000000000002,"5min":0.27000000000000002,"15min":0.249},"max":{"1min":2.7530000000000001,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.92800000000000005,"15min":0.98799999999999999},"min":{"1min":0.42399999999999999,"5min":0.29199999999999998,"15min":0.25800000000000001},"max":{"1min":3.1989999999999998,"5min":4.282,"15min":7.5259999999999998},"last":0.86399999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:57:01 2024","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.2070000000000001,"15min":1.3759999999999999},"min":{"1min":0.30399999999999999,"5min":0.28899999999999998,"15min":0.25},"max":{"1min":12.452999999999999,"5min":12.452999999999999,"15min":56.265999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":1.498,"5min":1.1459999999999999,"15min":1.413},"min":{"1min":0.46700000000000003,"5min":0.30599999999999999,"15min":0.27400000000000002},"max":{"1min":12.481,"5min":12.481,"15min":57.207000000000001},"last":0.90800000000000003}]},{"osd":4,"last update":"Wed Sep 11 08:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":2.766,"15min":1.4019999999999999},"min":{"1min":0.27700000000000002,"5min":0.224,"15min":0.224},"max":{"1min":1.3280000000000001,"5min":163.60499999999999,"15min":163.60499999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":2.919,"15min":1.4990000000000001},"min":{"1min":0.307,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.919,"5min":163.88300000000001,"15min":163.88300000000001},"last":0.38800000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":1.4610000000000001,"15min":0.997},"min":{"1min":0.33800000000000002,"5min":0.28999999999999998,"15min":0.23699999999999999},"max":{"1min":1.2130000000000001,"5min":50.128,"15min":50.128},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":1.472,"15min":1.046},"min":{"1min":0.29499999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.224,"5min":50.241,"15min":50.241},"last":1.1699999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.95099999999999996,"15min":0.88500000000000001},"min":{"1min":0.36499999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.2849999999999999,"5min":3.4769999999999999,"15min":4.5449999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.92900000000000005,"15min":0.89700000000000002},"min":{"1min":0.309,"5min":0.25,"15min":0.25},"max":{"1min":2.9529999999999998,"5min":5.0620000000000003,"15min":5.0620000000000003},"last":0.73899999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528558,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1587936,"kb_used_data":690704,"kb_used_omap":1,"kb_used_meta":897214,"kb_avail":92157216,"statfs":{"total":95995035648,"available":94368989184,"internally_reserved":0,"allocated":707280896,"data_stored":706914766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":918747480},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:56:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":2.4790000000000001,"15min":1.7989999999999999},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":5.577,"5min":121.208,"15min":121.208},"last":0.28699999999999998},{"interface":"front","average":{"1min":1.1299999999999999,"5min":2.8330000000000002,"15min":1.9099999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.23300000000000001},"max":{"1min":3.4220000000000002,"5min":121.05200000000001,"15min":121.05200000000001},"last":1.155}]},{"osd":1,"last update":"Wed Sep 11 08:56:39 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1479999999999999,"15min":1.3360000000000001},"min":{"1min":0.36799999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":3.004,"5min":5.8689999999999998,"15min":27.984000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.044,"5min":0.96999999999999997,"15min":1.238},"min":{"1min":0.32600000000000001,"5min":0.254,"15min":0.253},"max":{"1min":2.8439999999999999,"5min":3.411,"15min":28.106000000000002},"last":0.52000000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:56:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.9850000000000001,"15min":1.2949999999999999},"min":{"1min":0.38,"5min":0.23999999999999999,"15min":0.23200000000000001},"max":{"1min":2.8839999999999999,"5min":67.486999999999995,"15min":67.486999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":1.0680000000000001,"5min":2.472,"15min":1.4910000000000001},"min":{"1min":0.28599999999999998,"5min":0.27400000000000002,"15min":0.26600000000000001},"max":{"1min":3.4550000000000001,"5min":67.272000000000006,"15min":67.272000000000006},"last":1.0209999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:56:39 2024","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":1.329,"15min":1.419},"min":{"1min":0.38200000000000001,"5min":0.27200000000000002,"15min":0.26700000000000002},"max":{"1min":5.8070000000000004,"5min":5.8470000000000004,"15min":27.795000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":1.726,"5min":1.135,"15min":1.3540000000000001},"min":{"1min":0.39100000000000001,"5min":0.28199999999999997,"15min":0.224},"max":{"1min":6.8879999999999999,"5min":6.8879999999999999,"15min":27.841999999999999},"last":0.628}]},{"osd":4,"last update":"Wed Sep 11 08:56:39 2024","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.8100000000000001,"15min":1.024},"min":{"1min":0.27600000000000002,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":5.4829999999999997,"5min":40.927999999999997,"15min":40.927999999999997},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.099,"5min":1.6719999999999999,"15min":0.96599999999999997},"min":{"1min":0.28599999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.8959999999999999,"5min":40.706000000000003,"15min":40.706000000000003},"last":0.57699999999999996}]},{"osd":5,"last update":"Wed Sep 11 08:56:39 2024","interfaces":[{"interface":"back","average":{"1min":1.823,"5min":1.9419999999999999,"15min":1.177},"min":{"1min":0.315,"5min":0.29899999999999999,"15min":0.23899999999999999},"max":{"1min":19.309999999999999,"5min":40.847000000000001,"15min":40.847000000000001},"last":1.0469999999999999},{"interface":"front","average":{"1min":2.0249999999999999,"5min":1.8049999999999999,"15min":1.075},"min":{"1min":0.34200000000000003,"5min":0.29899999999999999,"15min":0.27100000000000002},"max":{"1min":19.411000000000001,"5min":41.401000000000003,"15min":41.401000000000003},"last":1.202}]},{"osd":7,"last update":"Wed Sep 11 08:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":1.365,"15min":1.077},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.25900000000000001},"max":{"1min":3.133,"5min":40.426000000000002,"15min":40.426000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":1.3779999999999999,"15min":1.0409999999999999},"min":{"1min":0.32200000000000001,"5min":0.309,"15min":0.27700000000000002},"max":{"1min":2.0070000000000001,"5min":45.401000000000003,"15min":45.401000000000003},"last":0.496}]}]},{"osd":5,"up_from":38,"seq":163208757490,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1309336,"kb_used_data":415376,"kb_used_omap":1,"kb_used_meta":893950,"kb_avail":92435816,"statfs":{"total":95995035648,"available":94654275584,"internally_reserved":0,"allocated":425345024,"data_stored":424979877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":915405144},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:57:19 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":1.252,"15min":1.115},"min":{"1min":0.29199999999999998,"5min":0.221,"15min":0.20899999999999999},"max":{"1min":0.69699999999999995,"5min":10.372,"15min":10.372},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.89600000000000002,"15min":1.115},"min":{"1min":0.25700000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.67700000000000005,"5min":5.444,"15min":8.0410000000000004},"last":0.40000000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:57:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.95499999999999996,"15min":1.0920000000000001},"min":{"1min":0.35899999999999999,"5min":0.248,"15min":0.216},"max":{"1min":0.72699999999999998,"5min":5.3799999999999999,"15min":8},"last":2.605},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.88,"15min":1.087},"min":{"1min":0.32000000000000001,"5min":0.252,"15min":0.23400000000000001},"max":{"1min":0.60599999999999998,"5min":12.173,"15min":12.173},"last":0.30599999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:57:19 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":1.024,"15min":1.1160000000000001},"min":{"1min":0.34000000000000002,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.65000000000000002,"5min":8.7789999999999999,"15min":8.7789999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":1.0920000000000001,"15min":1.0960000000000001},"min":{"1min":0.41699999999999998,"5min":0.26500000000000001,"15min":0.25900000000000001},"max":{"1min":0.68200000000000005,"5min":8.0619999999999994,"15min":8.0640000000000001},"last":0.34000000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:57:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":1.9119999999999999,"15min":1.3899999999999999},"min":{"1min":0.40799999999999997,"5min":0.29499999999999998,"15min":0.26900000000000002},"max":{"1min":0.66100000000000003,"5min":85.299999999999997,"15min":85.299999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":1.6990000000000001,"15min":1.482},"min":{"1min":0.35799999999999998,"5min":0.29599999999999999,"15min":0.27400000000000002},"max":{"1min":0.747,"5min":85.941000000000003,"15min":85.941000000000003},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed Sep 11 08:57:19 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.94899999999999995,"15min":0.85499999999999998},"min":{"1min":0.31900000000000001,"5min":0.23300000000000001,"15min":0.20699999999999999},"max":{"1min":0.55300000000000005,"5min":16.021999999999998,"15min":16.021999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.93200000000000005,"15min":0.80700000000000005},"min":{"1min":0.308,"5min":0.28000000000000003,"15min":0.20499999999999999},"max":{"1min":0.63900000000000001,"5min":9.6140000000000008,"15min":9.6140000000000008},"last":0.48999999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:56:44 2024","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.179,"15min":0.90000000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.23599999999999999},"max":{"1min":6.8979999999999997,"5min":8.8059999999999992,"15min":8.8059999999999992},"last":0.51400000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.236,"15min":0.91100000000000003},"min":{"1min":0.32600000000000001,"5min":0.27400000000000002,"15min":0.251},"max":{"1min":6.5999999999999996,"5min":6.5999999999999996,"15min":8.1920000000000002},"last":0.52700000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":1.0589999999999999,"15min":1.196},"min":{"1min":0.32900000000000001,"5min":0.30199999999999999,"15min":0.27800000000000002},"max":{"1min":3.948,"5min":16.062999999999999,"15min":88.513000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.01,"15min":1.198},"min":{"1min":0.36699999999999999,"5min":0.315,"15min":0.29599999999999999},"max":{"1min":6.9340000000000002,"5min":8.8300000000000001,"15min":86.963999999999999},"last":0.55900000000000005}]}]},{"osd":4,"up_from":31,"seq":133143986422,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1292508,"kb_used_data":394956,"kb_used_omap":1,"kb_used_meta":897534,"kb_avail":92452644,"statfs":{"total":95995035648,"available":94671507456,"internally_reserved":0,"allocated":404434944,"data_stored":404072178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":919075160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:56:58 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.1579999999999999,"15min":1.8160000000000001},"min":{"1min":0.24299999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":3.581,"5min":5.2400000000000002,"15min":89.510000000000005},"last":0.30199999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.268,"15min":1.9139999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.22600000000000001},"max":{"1min":6.242,"5min":6.242,"15min":89.765000000000001},"last":1.3460000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:56:58 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.173,"15min":1.5800000000000001},"min":{"1min":0.28299999999999997,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":6.7279999999999998,"5min":18.065999999999999,"15min":48.976999999999997},"last":0.442},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2529999999999999,"15min":1.548},"min":{"1min":0.34499999999999997,"5min":0.28699999999999998,"15min":0.22800000000000001},"max":{"1min":6.2670000000000003,"5min":18.713999999999999,"15min":45.787999999999997},"last":1.7230000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:56:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":3.9849999999999999,"15min":2.0059999999999998},"min":{"1min":0.27300000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":3.351,"5min":255.88900000000001,"15min":255.88900000000001},"last":2.2000000000000002},{"interface":"front","average":{"1min":1.5800000000000001,"5min":4.2229999999999999,"15min":2.0649999999999999},"min":{"1min":0.28699999999999998,"5min":0.28299999999999997,"15min":0.25800000000000001},"max":{"1min":9.9870000000000001,"5min":255.41200000000001,"15min":255.41200000000001},"last":0.39500000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:56:58 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.4279999999999999,"15min":2.1699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.252},"max":{"1min":6.6699999999999999,"5min":16.663,"15min":89.881},"last":0.41999999999999998},{"interface":"front","average":{"1min":1.411,"5min":1.3,"15min":2.0790000000000002},"min":{"1min":0.33000000000000002,"5min":0.26600000000000001,"15min":0.25900000000000001},"max":{"1min":5.0389999999999997,"5min":16.795999999999999,"15min":89.715000000000003},"last":0.40500000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.875,"15min":0.83799999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.23999999999999999},"max":{"1min":4.9870000000000001,"5min":4.9870000000000001,"15min":11.045999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.86299999999999999,"15min":0.82199999999999995},"min":{"1min":0.28899999999999998,"5min":0.27000000000000002,"15min":0.254},"max":{"1min":3.2829999999999999,"5min":4.7869999999999999,"15min":10.526999999999999},"last":0.34100000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:56:47 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":2.1280000000000001,"15min":1.129},"min":{"1min":0.316,"5min":0.26800000000000002,"15min":0.22900000000000001},"max":{"1min":6.766,"5min":72.100999999999999,"15min":72.100999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":1.236,"5min":2.2650000000000001,"15min":1.1970000000000001},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.221},"max":{"1min":6.5,"5min":74.527000000000001,"15min":74.527000000000001},"last":0.372}]},{"osd":7,"last update":"Wed Sep 11 08:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.90000000000000002,"15min":0.76900000000000002},"min":{"1min":0.33300000000000002,"5min":0.253,"15min":0.252},"max":{"1min":6.907,"5min":6.907,"15min":6.907},"last":0.38300000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.1659999999999999,"15min":0.90500000000000003},"min":{"1min":0.315,"5min":0.29899999999999999,"15min":0.29599999999999999},"max":{"1min":6.8029999999999999,"5min":6.8029999999999999,"15min":6.8029999999999999},"last":0.35799999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149948,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1461432,"kb_used_data":566384,"kb_used_omap":1,"kb_used_meta":895038,"kb_avail":92283720,"statfs":{"total":95995035648,"available":94498529280,"internally_reserved":0,"allocated":579977216,"data_stored":579580210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":916519256},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:56:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1539999999999999,"15min":1.325},"min":{"1min":0.214,"5min":0.20899999999999999,"15min":0.16500000000000001},"max":{"1min":3.9399999999999999,"5min":10.393000000000001,"15min":24.280000000000001},"last":2.2160000000000002},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.1919999999999999,"15min":1.5449999999999999},"min":{"1min":0.219,"5min":0.19900000000000001,"15min":0.17000000000000001},"max":{"1min":5.2140000000000004,"5min":7.5599999999999996,"15min":22.225000000000001},"last":0.25}]},{"osd":1,"last update":"Wed Sep 11 08:56:37 2024","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.9530000000000001,"15min":2.1880000000000002},"min":{"1min":0.24399999999999999,"5min":0.20100000000000001,"15min":0.19800000000000001},"max":{"1min":4.8170000000000002,"5min":84.284999999999997,"15min":115.601},"last":3.8399999999999999},{"interface":"front","average":{"1min":1.6279999999999999,"5min":2.0390000000000001,"15min":2.0920000000000001},"min":{"1min":0.22600000000000001,"5min":0.20499999999999999,"15min":0.17999999999999999},"max":{"1min":6.7080000000000002,"5min":85.832999999999998,"15min":115.474},"last":1.2490000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:56:37 2024","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":2.2149999999999999,"15min":1.7110000000000001},"min":{"1min":0.30099999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":6.117,"5min":86.766000000000005,"15min":86.766000000000005},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.05,"5min":2.0960000000000001,"15min":1.6459999999999999},"min":{"1min":0.32200000000000001,"5min":0.23400000000000001,"15min":0.20799999999999999},"max":{"1min":3.8079999999999998,"5min":85.367999999999995,"15min":85.367999999999995},"last":1.71}]},{"osd":4,"last update":"Wed Sep 11 08:56:49 2024","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.407,"15min":1.405},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.27000000000000002},"max":{"1min":6.0890000000000004,"5min":6.0890000000000004,"15min":8.9969999999999999},"last":1.224},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.1479999999999999,"15min":1.1599999999999999},"min":{"1min":0.35099999999999998,"5min":0.33300000000000002,"15min":0.27700000000000002},"max":{"1min":3.726,"5min":5.617,"15min":7.234},"last":0.42099999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:57:08 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.333,"15min":1.3819999999999999},"min":{"1min":0.373,"5min":0.30599999999999999,"15min":0.28299999999999997},"max":{"1min":6.1440000000000001,"5min":6.1440000000000001,"15min":11.526999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.0920000000000001,"15min":1.208},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":3.758,"5min":5.5190000000000001,"15min":7.1689999999999996},"last":0.65700000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:56:37 2024","interfaces":[{"interface":"back","average":{"1min":1.917,"5min":1.6060000000000001,"15min":1.635},"min":{"1min":0.39900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":6.1959999999999997,"5min":11.007999999999999,"15min":33.106999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.6699999999999999,"5min":1.262,"15min":1.51},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":5.819,"5min":7.4809999999999999,"15min":33.076000000000001},"last":0.32500000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:56:58 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.3660000000000001,"15min":1.639},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":6.0410000000000004,"5min":6.0410000000000004,"15min":35.073},"last":1.296},{"interface":"front","average":{"1min":0.91900000000000004,"5min":1.1899999999999999,"15min":1.444},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.34699999999999998},"max":{"1min":3.7869999999999999,"5min":5.6529999999999996,"15min":34.844999999999999},"last":0.70199999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411584,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1368916,"kb_used_data":468740,"kb_used_omap":1,"kb_used_meta":900094,"kb_avail":92376236,"statfs":{"total":95995035648,"available":94593265664,"internally_reserved":0,"allocated":479989760,"data_stored":479635661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":921696600},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:57:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.94999999999999996,"15min":1.698},"min":{"1min":0.20300000000000001,"5min":0.189,"15min":0.159},"max":{"1min":2.6139999999999999,"5min":7.3380000000000001,"15min":34.713999999999999},"last":0.37},{"interface":"front","average":{"1min":0.45100000000000001,"5min":1.1339999999999999,"15min":1.8109999999999999},"min":{"1min":0.17499999999999999,"5min":0.17499999999999999,"15min":0.17399999999999999},"max":{"1min":2.379,"5min":9.0630000000000006,"15min":34.399000000000001},"last":0.44800000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:57:15 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.77900000000000003,"15min":1.7569999999999999},"min":{"1min":0.21099999999999999,"5min":0.189,"15min":0.189},"max":{"1min":1.262,"5min":5.9850000000000003,"15min":72.867000000000004},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.89800000000000002,"15min":1.8919999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.20399999999999999},"max":{"1min":4.0460000000000003,"5min":4.8570000000000002,"15min":67.799000000000007},"last":0.35099999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:57:23 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":1.599,"15min":1.851},"min":{"1min":0.248,"5min":0.186,"15min":0.186},"max":{"1min":0.91500000000000004,"5min":64.795000000000002,"15min":64.795000000000002},"last":3.5619999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":1.595,"15min":1.792},"min":{"1min":0.22700000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.53700000000000003,"5min":64.632999999999996,"15min":64.632999999999996},"last":1.827}]},{"osd":4,"last update":"Wed Sep 11 08:56:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.2150000000000001,"15min":1.3859999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.255},"max":{"1min":2.1989999999999998,"5min":6.0659999999999998,"15min":52.113999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":1.242,"5min":1.3100000000000001,"15min":1.363},"min":{"1min":0.31,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4420000000000002,"5min":5.5780000000000003,"15min":52.097999999999999},"last":0.57499999999999996}]},{"osd":5,"last update":"Wed Sep 11 08:57:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":1.1739999999999999,"15min":1.3500000000000001},"min":{"1min":0.34399999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67300000000000004,"5min":5.8860000000000001,"15min":52.145000000000003},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":1.01,"15min":1.171},"min":{"1min":0.34499999999999997,"5min":0.29799999999999999,"15min":0.28399999999999997},"max":{"1min":1.0329999999999999,"5min":4.8109999999999999,"15min":52.216000000000001},"last":0.94899999999999995}]},{"osd":6,"last update":"Wed Sep 11 08:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":1.254,"15min":1.415},"min":{"1min":0.434,"5min":0.34000000000000002,"15min":0.28699999999999998},"max":{"1min":1.647,"5min":7.298,"15min":52.235999999999997},"last":0.434},{"interface":"front","average":{"1min":0.65200000000000002,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.38100000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.458,"5min":5.6440000000000001,"15min":52.186},"last":0.60999999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:56:48 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.091,"15min":1.3300000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.30399999999999999},"max":{"1min":4.0220000000000002,"5min":4.8570000000000002,"15min":84.108999999999995},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":1.3069999999999999,"15min":1.589},"min":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.30299999999999999},"max":{"1min":2.2170000000000001,"5min":7.0720000000000001,"15min":85.938000000000002},"last":0.52600000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575109,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1446532,"kb_used_data":547004,"kb_used_omap":1,"kb_used_meta":899518,"kb_avail":92298620,"statfs":{"total":95995035648,"available":94513786880,"internally_reserved":0,"allocated":560132096,"data_stored":559785267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":921106776},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:56:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.9430000000000001,"15min":1.966},"min":{"1min":0.215,"5min":0.17499999999999999,"15min":0.16600000000000001},"max":{"1min":4.3819999999999997,"5min":56.613,"15min":117.01600000000001},"last":1.748},{"interface":"front","average":{"1min":1.0609999999999999,"5min":2.0609999999999999,"15min":2.2829999999999999},"min":{"1min":0.25700000000000001,"5min":0.187,"15min":0.17499999999999999},"max":{"1min":5.7599999999999998,"5min":55.533999999999999,"15min":117.62},"last":0.443}]},{"osd":2,"last update":"Wed Sep 11 08:57:15 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":3.1920000000000002,"15min":2.016},"min":{"1min":0.253,"5min":0.19,"15min":0.19},"max":{"1min":3.0049999999999999,"5min":169.607,"15min":169.607},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.504,"5min":2.9430000000000001,"15min":1.9570000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20000000000000001},"max":{"1min":2.0179999999999998,"5min":169.762,"15min":169.762},"last":0.23100000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:57:36 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.94699999999999995,"15min":1.323},"min":{"1min":0.249,"5min":0.224,"15min":0.191},"max":{"1min":1.7749999999999999,"5min":8.1270000000000007,"15min":8.3550000000000004},"last":1.7749999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.84699999999999998,"15min":1.27},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.22},"max":{"1min":1.462,"5min":5.9029999999999996,"15min":10.379},"last":1.462}]},{"osd":4,"last update":"Wed Sep 11 08:56:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.052,"15min":1.163},"min":{"1min":0.375,"5min":0.28699999999999998,"15min":0.26200000000000001},"max":{"1min":3.7269999999999999,"5min":7.9450000000000003,"15min":11.19},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":1.1479999999999999,"15min":1.113},"min":{"1min":0.312,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":3.9140000000000001,"5min":6.952,"15min":7.7169999999999996},"last":0.32900000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:57:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.2789999999999999,"15min":1.762},"min":{"1min":0.30599999999999999,"5min":0.30299999999999999,"15min":0.27600000000000002},"max":{"1min":1.881,"5min":38.098999999999997,"15min":103.337},"last":1.373},{"interface":"front","average":{"1min":0.56999999999999995,"5min":1.3069999999999999,"15min":1.7749999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.27400000000000002},"max":{"1min":1.3680000000000001,"5min":38.130000000000003,"15min":103.163},"last":0.39100000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:56:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.3640000000000001,"15min":1.2130000000000001},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.29399999999999998},"max":{"1min":5.2130000000000001,"5min":6.2249999999999996,"15min":8.9149999999999991},"last":0.37},{"interface":"front","average":{"1min":0.85299999999999998,"5min":1.0669999999999999,"15min":1.2689999999999999},"min":{"1min":0.36799999999999999,"5min":0.315,"15min":0.31},"max":{"1min":2.0249999999999999,"5min":6.2869999999999999,"15min":9.5199999999999996},"last":1.415}]},{"osd":7,"last update":"Wed Sep 11 08:56:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.89700000000000002,"15min":1.1599999999999999},"min":{"1min":0.34100000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":2.2309999999999999,"5min":8.3390000000000004,"15min":11.315},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.99399999999999999,"15min":1.151},"min":{"1min":0.34999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.9380000000000002,"5min":3.9380000000000002,"15min":10.266},"last":1.7989999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738633,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1365472,"kb_used_data":460372,"kb_used_omap":1,"kb_used_meta":905086,"kb_avail":92379680,"statfs":{"total":95995035648,"available":94596792320,"internally_reserved":0,"allocated":471420928,"data_stored":471050677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":926808408},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:56:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":1.0329999999999999,"15min":1.645},"min":{"1min":0.21299999999999999,"5min":0.17399999999999999,"15min":0.16300000000000001},"max":{"1min":2.25,"5min":8.5649999999999995,"15min":56.685000000000002},"last":4.194},{"interface":"front","average":{"1min":1.198,"5min":1.2050000000000001,"15min":1.611},"min":{"1min":0.19500000000000001,"5min":0.188,"15min":0.186},"max":{"1min":7.1109999999999998,"5min":7.694,"15min":61.012999999999998},"last":3.7000000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:57:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":1.194,"15min":1.2769999999999999},"min":{"1min":0.26500000000000001,"5min":0.222,"15min":0.21099999999999999},"max":{"1min":1.9470000000000001,"5min":8.1059999999999999,"15min":12.481},"last":0.255},{"interface":"front","average":{"1min":0.71699999999999997,"5min":1.1379999999999999,"15min":1.3320000000000001},"min":{"1min":0.27300000000000002,"5min":0.20899999999999999,"15min":0.20000000000000001},"max":{"1min":5.2300000000000004,"5min":5.4279999999999999,"15min":8.3049999999999997},"last":2.6709999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:56:36 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.25,"15min":1.7629999999999999},"min":{"1min":0.22900000000000001,"5min":0.218,"15min":0.19800000000000001},"max":{"1min":4.9059999999999997,"5min":7.7709999999999999,"15min":56.095999999999997},"last":0.37},{"interface":"front","average":{"1min":1.8109999999999999,"5min":1.264,"15min":1.7509999999999999},"min":{"1min":0.23999999999999999,"5min":0.21199999999999999,"15min":0.20899999999999999},"max":{"1min":8.0329999999999995,"5min":8.5419999999999998,"15min":59.651000000000003},"last":0.95299999999999996}]},{"osd":4,"last update":"Wed Sep 11 08:56:50 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":3.419,"15min":2.0049999999999999},"min":{"1min":0.42799999999999999,"5min":0.27200000000000002,"15min":0.25},"max":{"1min":4.702,"5min":168.72200000000001,"15min":168.72200000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":2.052,"5min":3.5910000000000002,"15min":2.0129999999999999},"min":{"1min":0.40999999999999998,"5min":0.34799999999999998,"15min":0.255},"max":{"1min":7.7039999999999997,"5min":168.88999999999999,"15min":168.88999999999999},"last":0.83999999999999997}]},{"osd":5,"last update":"Wed Sep 11 08:57:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":1.3520000000000001,"15min":1.3380000000000001},"min":{"1min":0.36799999999999999,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.95599999999999996,"5min":7.7830000000000004,"15min":7.7830000000000004},"last":0.315},{"interface":"front","average":{"1min":0.65100000000000002,"5min":1.3480000000000001,"15min":1.4039999999999999},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":1.214,"5min":9.5239999999999991,"15min":9.5239999999999991},"last":0.71299999999999997}]},{"osd":6,"last update":"Wed Sep 11 08:56:36 2024","interfaces":[{"interface":"back","average":{"1min":2.762,"5min":1.784,"15min":1.536},"min":{"1min":0.38900000000000001,"5min":0.33300000000000002,"15min":0.30099999999999999},"max":{"1min":25.027000000000001,"5min":25.027000000000001,"15min":25.027000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":3.383,"5min":1.893,"15min":1.6180000000000001},"min":{"1min":0.499,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":25.076000000000001,"5min":25.076000000000001,"15min":25.076000000000001},"last":0.80100000000000005}]},{"osd":7,"last update":"Wed Sep 11 08:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.3169999999999999,"15min":1.5960000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.313},"max":{"1min":3.802,"5min":5.0739999999999998,"15min":56.119},"last":0.871},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.595,"15min":1.7330000000000001},"min":{"1min":0.41999999999999998,"5min":0.35099999999999998,"15min":0.307},"max":{"1min":4.7039999999999997,"5min":9.9459999999999997,"15min":56.100000000000001},"last":0.51500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":495910912,"data_stored":495910912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":561709056,"data_stored":561709056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":522846208,"data_stored":522846208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":612106240,"data_stored":612106240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":451280896,"data_stored":451280896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":475201536,"data_stored":475201536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":728236032,"data_stored":728236032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":545193984,"data_stored":545193984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:57:48.939 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-11T08:57:48.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool set .mgr pg_num 21 2024-09-11T08:57:48.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:48.964 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:48.964 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:49.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:49.126 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:49.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:49.128 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:49.228 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:49.261 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:49.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:49.262 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:49.335 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:57:49.336871+0000 min lat: 0.0011062 max lat: 0.172662 avg lat: 0.00695127 2024-09-11T08:57:49.336 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:57:49.336 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 20 16 46028 46012 143.754 111.188 0.00436679 0.00695127 2024-09-11T08:57:49.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:49.403 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:49.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:49.404 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:49.504 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:49.540 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:49.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:49.542 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:49.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:49 smithi092 ceph-mon[29647]: 4.13 scrub starts 2024-09-11T08:57:49.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:49 smithi092 ceph-mon[29647]: 4.13 scrub ok 2024-09-11T08:57:49.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:49 smithi092 ceph-mon[29647]: 3.1d scrub starts 2024-09-11T08:57:49.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:49 smithi092 ceph-mon[29647]: 3.1d scrub ok 2024-09-11T08:57:49.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:49 smithi092 ceph-mon[29647]: osdmap e378: 8 total, 8 up, 8 in 2024-09-11T08:57:49.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:49 smithi092 ceph-mon[41114]: 4.13 scrub starts 2024-09-11T08:57:49.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:49 smithi092 ceph-mon[41114]: 4.13 scrub ok 2024-09-11T08:57:49.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:49 smithi092 ceph-mon[41114]: 3.1d scrub starts 2024-09-11T08:57:49.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:49 smithi092 ceph-mon[41114]: 3.1d scrub ok 2024-09-11T08:57:49.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:49 smithi092 ceph-mon[41114]: osdmap e378: 8 total, 8 up, 8 in 2024-09-11T08:57:49.642 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:49 smithi145 ceph-mon[33384]: 4.13 scrub starts 2024-09-11T08:57:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:49 smithi145 ceph-mon[33384]: 4.13 scrub ok 2024-09-11T08:57:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:49 smithi145 ceph-mon[33384]: 3.1d scrub starts 2024-09-11T08:57:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:49 smithi145 ceph-mon[33384]: 3.1d scrub ok 2024-09-11T08:57:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:49 smithi145 ceph-mon[33384]: osdmap e378: 8 total, 8 up, 8 in 2024-09-11T08:57:49.678 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:49.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:49.679 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:49.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-11T08:57:49.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:49.840 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:49.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:49.842 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:49.942 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:49.972 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:49.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:49.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:50.074 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:50.102 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:50.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:50.103 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:50.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:57:50.181 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:50.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:50.227 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:57:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:50.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:50.265 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:50.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:50 smithi092 ceph-mon[41114]: 5.18 scrub starts 2024-09-11T08:57:50.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:50 smithi092 ceph-mon[41114]: 5.18 scrub ok 2024-09-11T08:57:50.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:50 smithi092 ceph-mon[41114]: 3.1c scrub starts 2024-09-11T08:57:50.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:50 smithi092 ceph-mon[41114]: 3.1c scrub ok 2024-09-11T08:57:50.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:50 smithi092 ceph-mon[41114]: pgmap v869: 158 pgs: 158 active+clean; 2.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 109 MiB/s wr, 1.75k op/s 2024-09-11T08:57:50.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:50 smithi092 ceph-mon[29647]: 5.18 scrub starts 2024-09-11T08:57:50.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:50 smithi092 ceph-mon[29647]: 5.18 scrub ok 2024-09-11T08:57:50.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:50 smithi092 ceph-mon[29647]: 3.1c scrub starts 2024-09-11T08:57:50.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:50 smithi092 ceph-mon[29647]: 3.1c scrub ok 2024-09-11T08:57:50.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:50 smithi092 ceph-mon[29647]: pgmap v869: 158 pgs: 158 active+clean; 2.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 109 MiB/s wr, 1.75k op/s 2024-09-11T08:57:50.335 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 21 16 47664 47648 141.778 102.25 0.00664748 0.00704801 2024-09-11T08:57:50.367 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:50.393 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:50.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:50.395 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:50.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:50.543 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:50.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:50.544 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:50.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:50.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:50 smithi145 ceph-mon[33384]: 5.18 scrub starts 2024-09-11T08:57:50.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:50 smithi145 ceph-mon[33384]: 5.18 scrub ok 2024-09-11T08:57:50.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:50 smithi145 ceph-mon[33384]: 3.1c scrub starts 2024-09-11T08:57:50.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:50 smithi145 ceph-mon[33384]: 3.1c scrub ok 2024-09-11T08:57:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:50 smithi145 ceph-mon[33384]: pgmap v869: 158 pgs: 158 active+clean; 2.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 109 MiB/s wr, 1.75k op/s 2024-09-11T08:57:50.678 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:50.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:50.679 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:50.780 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:50.815 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:50.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:50.816 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:50.917 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:50.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:50.953 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:50.953 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:51.054 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:51.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:51.089 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:51.089 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:51.190 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:51.210 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:57:51.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:51.229 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:51.230 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:51.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:51.335 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 22 16 49255 49239 139.853 99.4375 0.0074934 0.00714509 2024-09-11T08:57:51.370 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:51.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:51.371 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:51.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[41114]: 1.2 deep-scrub starts 2024-09-11T08:57:51.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[41114]: 1.2 deep-scrub ok 2024-09-11T08:57:51.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[41114]: 5.7 scrub starts 2024-09-11T08:57:51.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[41114]: 5.7 scrub ok 2024-09-11T08:57:51.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/747803138' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:57:51.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:57:51.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[41114]: 3.0 scrub starts 2024-09-11T08:57:51.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[41114]: 3.0 scrub ok 2024-09-11T08:57:51.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[41114]: 1.9 scrub starts 2024-09-11T08:57:51.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[41114]: 1.9 scrub ok 2024-09-11T08:57:51.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[29647]: 1.2 deep-scrub starts 2024-09-11T08:57:51.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[29647]: 1.2 deep-scrub ok 2024-09-11T08:57:51.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[29647]: 5.7 scrub starts 2024-09-11T08:57:51.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[29647]: 5.7 scrub ok 2024-09-11T08:57:51.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/747803138' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:57:51.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:57:51.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[29647]: 3.0 scrub starts 2024-09-11T08:57:51.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[29647]: 3.0 scrub ok 2024-09-11T08:57:51.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[29647]: 1.9 scrub starts 2024-09-11T08:57:51.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-mon[29647]: 1.9 scrub ok 2024-09-11T08:57:51.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:51.504 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:51.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:51.505 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:51.606 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:51.637 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:51.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:51.638 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:51.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:51 smithi145 ceph-mon[33384]: 1.2 deep-scrub starts 2024-09-11T08:57:51.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:51 smithi145 ceph-mon[33384]: 1.2 deep-scrub ok 2024-09-11T08:57:51.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:51 smithi145 ceph-mon[33384]: 5.7 scrub starts 2024-09-11T08:57:51.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:51 smithi145 ceph-mon[33384]: 5.7 scrub ok 2024-09-11T08:57:51.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:51 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/747803138' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:57:51.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:51 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:57:51.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:51 smithi145 ceph-mon[33384]: 3.0 scrub starts 2024-09-11T08:57:51.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:51 smithi145 ceph-mon[33384]: 3.0 scrub ok 2024-09-11T08:57:51.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:51 smithi145 ceph-mon[33384]: 1.9 scrub starts 2024-09-11T08:57:51.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:51 smithi145 ceph-mon[33384]: 1.9 scrub ok 2024-09-11T08:57:51.739 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:51.760 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:51.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:51.761 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:51.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:57:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:57:51] "GET /metrics HTTP/1.1" 200 225062 "" "Prometheus/2.43.0" 2024-09-11T08:57:51.862 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:51.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:51.884 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:51.884 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:51.985 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:52.025 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:52.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:52.026 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:52.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:52.166 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:52.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:52.168 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:52.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:52.307 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:52.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:52.308 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:52.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:52 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:57:52.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:52 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:57:52.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:52 smithi092 ceph-mon[29647]: osdmap e379: 8 total, 8 up, 8 in 2024-09-11T08:57:52.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:52 smithi092 ceph-mon[29647]: pgmap v871: 158 pgs: 158 active+clean; 2.7 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 102 MiB/s wr, 1.63k op/s 2024-09-11T08:57:52.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:52 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:57:52.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:52 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:57:52.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:52 smithi092 ceph-mon[41114]: osdmap e379: 8 total, 8 up, 8 in 2024-09-11T08:57:52.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:52 smithi092 ceph-mon[41114]: pgmap v871: 158 pgs: 158 active+clean; 2.7 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 102 MiB/s wr, 1.63k op/s 2024-09-11T08:57:52.336 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 23 15 50483 50468 137.112 76.8125 0.00795999 0.00728593 2024-09-11T08:57:52.410 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:52.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:52.440 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:52.441 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:52.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:52.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:52.565 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:52.565 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:52 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:57:52.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:52 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:57:52.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:52 smithi145 ceph-mon[33384]: osdmap e379: 8 total, 8 up, 8 in 2024-09-11T08:57:52.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:52 smithi145 ceph-mon[33384]: pgmap v871: 158 pgs: 158 active+clean; 2.7 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 102 MiB/s wr, 1.63k op/s 2024-09-11T08:57:52.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:52.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:52.702 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:52.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:52.804 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:52.836 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:52.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:52.837 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:52.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:52.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:52.976 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:52.977 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:53.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:53.113 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:53.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:53.114 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:53.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:57:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:57:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:57:53.214 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:53.247 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:53.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:53.249 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:53.336 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 24 16 51995 51979 135.334 94.4375 0.0041275 0.00738457 2024-09-11T08:57:53.350 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:53.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:53.384 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:53.384 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:53.485 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:53.506 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:53.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:53.507 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:53.608 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:53.640 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:53.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:53.641 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:53.742 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:53.780 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:53.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:53.781 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:53.883 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:53.920 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:53.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:53.921 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:54.022 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:54.059 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:54.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:54.060 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:54.161 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:54.192 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:54.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:54.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:54.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:54.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:54.331 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:54.336 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 25 16 53541 53525 133.786 96.625 0.00479784 0.00746935 2024-09-11T08:57:54.432 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:54.468 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:54.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:54.469 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:54.570 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:54.574 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:54 smithi092 ceph-mon[29647]: pgmap v872: 158 pgs: 158 active+clean; 3.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 131 MiB/s wr, 2.09k op/s 2024-09-11T08:57:54.574 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:54 smithi092 ceph-mon[29647]: osdmap e380: 8 total, 8 up, 8 in 2024-09-11T08:57:54.574 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:54 smithi092 ceph-mon[41114]: pgmap v872: 158 pgs: 158 active+clean; 3.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 131 MiB/s wr, 2.09k op/s 2024-09-11T08:57:54.575 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:54 smithi092 ceph-mon[41114]: osdmap e380: 8 total, 8 up, 8 in 2024-09-11T08:57:54.600 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:54.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:54.602 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:54.702 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:54.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:54.744 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:54.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:54.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:54.883 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:54.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:54.884 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:54.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:54 smithi145 ceph-mon[33384]: pgmap v872: 158 pgs: 158 active+clean; 3.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 131 MiB/s wr, 2.09k op/s 2024-09-11T08:57:54.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:54 smithi145 ceph-mon[33384]: osdmap e380: 8 total, 8 up, 8 in 2024-09-11T08:57:54.985 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:55.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:55.022 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:55.022 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:55.123 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:55.157 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:55.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:55.158 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:55.259 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:55.289 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:55.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:55.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:55.336 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 26 16 55240 55224 132.724 106.188 0.014933 0.00752999 2024-09-11T08:57:55.391 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:55.422 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:55.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:55.423 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:55.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:55.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:55.579 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:55.579 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:55.680 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:55.717 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:55.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:55.718 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:55.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:55 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:57:55.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:55 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:57:55.768 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:55 smithi092 ceph-mon[29647]: osdmap e381: 8 total, 8 up, 8 in 2024-09-11T08:57:55.768 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:55 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:57:55.768 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:55 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:57:55.768 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:55 smithi092 ceph-mon[41114]: osdmap e381: 8 total, 8 up, 8 in 2024-09-11T08:57:55.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:55.861 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:55.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:55.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:55.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:55 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:57:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:55 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:57:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:55 smithi145 ceph-mon[33384]: osdmap e381: 8 total, 8 up, 8 in 2024-09-11T08:57:55.963 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:55.995 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:55.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:55.997 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:56.097 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:56.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:56.139 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:56.139 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:56.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:56.278 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:56.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:56.279 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:56.336 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 27 16 56899 56883 131.648 103.688 0.00369999 0.00759049 2024-09-11T08:57:56.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:57:56.380 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:57:56.412 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:56.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:56.413 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:57:56.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:56.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:56.552 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:56.552 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:56.653 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:56.682 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:56.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:56.684 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:57:56.784 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:56.807 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:56.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:56.808 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:56.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:56 smithi092 ceph-mon[29647]: pgmap v875: 158 pgs: 158 active+clean; 3.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-11T08:57:56.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:56 smithi092 ceph-mon[41114]: pgmap v875: 158 pgs: 158 active+clean; 3.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-11T08:57:56.909 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:56.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:56 smithi145 ceph-mon[33384]: pgmap v875: 158 pgs: 158 active+clean; 3.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-11T08:57:56.943 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:56.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:56.944 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:57.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:57.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:57.088 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:57.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:57.225 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:57.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:57.226 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:57.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:57.364 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 28 16 58278 58262 130.024 86.1875 0.00898368 0.007686 2024-09-11T08:57:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:57.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:57.395 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:57.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:57:57.530 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:57.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:57.531 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:57:57.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:57.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:57:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:57:57.670 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:57.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:57.671 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:57.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:57.808 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:57.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:57.809 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:57.910 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:57.942 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:57.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:57.943 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:58.044 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:58.073 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:58.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:58.074 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:58.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:58.211 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:58.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:58.212 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:58.313 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:58.336 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 29 16 59727 59711 128.663 90.5625 0.0116012 0.00776678 2024-09-11T08:57:58.347 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:58.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:58.349 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:58.450 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:58.481 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:58.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:58.482 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:58.583 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:58.612 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:58.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:58.614 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:58.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:57:58.762 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:58.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:58.763 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:57:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:58 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:58 smithi092 ceph-mon[29647]: pgmap v876: 158 pgs: 158 active+clean; 3.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 824 B/s rd, 126 MiB/s wr, 2.02k op/s 2024-09-11T08:57:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:58 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:58 smithi092 ceph-mon[41114]: pgmap v876: 158 pgs: 158 active+clean; 3.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 824 B/s rd, 126 MiB/s wr, 2.02k op/s 2024-09-11T08:57:58.864 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:58.897 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:58.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:58.898 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:58.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:58 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:57:58.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:58 smithi145 ceph-mon[33384]: pgmap v876: 158 pgs: 158 active+clean; 3.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 824 B/s rd, 126 MiB/s wr, 2.02k op/s 2024-09-11T08:57:58.999 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:59.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:59.037 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:59.037 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:59.101 INFO:teuthology.orchestra.run.smithi092.stderr:set pool 1 pg_num to 21 2024-09-11T08:57:59.137 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:59.168 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:59.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:59.169 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:59.270 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:57:59.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:59.301 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:59.301 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:57:59.337 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 30 16 61272 61256 127.593 96.5625 0.00685096 0.00783212 2024-09-11T08:57:59.402 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:57:59.433 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:59.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:59.434 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:57:59.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:57:59.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:59 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1180373413' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-11T08:57:59.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:59 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-11T08:57:59.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:59 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]': finished 2024-09-11T08:57:59.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:59 smithi092 ceph-mon[29647]: osdmap e382: 8 total, 8 up, 8 in 2024-09-11T08:57:59.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:59 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-09-11T08:57:59.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:57:59 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-09-11T08:57:59.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:59 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1180373413' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-11T08:57:59.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:59 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-11T08:57:59.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:59 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]': finished 2024-09-11T08:57:59.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:59 smithi092 ceph-mon[41114]: osdmap e382: 8 total, 8 up, 8 in 2024-09-11T08:57:59.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:59 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-09-11T08:57:59.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:57:59 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-09-11T08:57:59.577 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:59.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:59.578 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:57:59.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:59.714 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:59.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:59.715 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:59.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:57:59.856 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:57:59.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:59.857 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:57:59.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:59 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1180373413' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-11T08:57:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:59 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-11T08:57:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:59 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]': finished 2024-09-11T08:57:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:59 smithi145 ceph-mon[33384]: osdmap e382: 8 total, 8 up, 8 in 2024-09-11T08:57:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:59 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-09-11T08:57:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:57:59 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-09-11T08:57:59.957 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:57:59.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:57:59.993 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:57:59.993 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:00.094 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:00.101 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:58:00.169 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:00.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:00.170 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:00.271 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:00.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:00.306 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:00.307 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:00.337 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 31 16 62779 62763 126.513 94.1875 0.00656198 0.00789955 2024-09-11T08:58:00.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:00.410 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:58:00.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:00.469 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:00.469 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:00.516 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:00 smithi092 ceph-mon[29647]: pgmap v878: 158 pgs: 158 active+clean; 3.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 81 MiB/s wr, 1.29k op/s 2024-09-11T08:58:00.516 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:00 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-11T08:58:00.516 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:00 smithi092 ceph-mon[29647]: osdmap e383: 8 total, 8 up, 8 in 2024-09-11T08:58:00.516 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:00 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:58:00.517 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:58:00.517 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:00 smithi092 ceph-mon[41114]: pgmap v878: 158 pgs: 158 active+clean; 3.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 81 MiB/s wr, 1.29k op/s 2024-09-11T08:58:00.517 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:00 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-11T08:58:00.517 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:00 smithi092 ceph-mon[41114]: osdmap e383: 8 total, 8 up, 8 in 2024-09-11T08:58:00.517 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:00 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:58:00.517 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:58:00.571 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:00.606 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:00.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:00.607 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:00.708 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:00.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:00.750 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:00.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:00.894 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:00.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:00.895 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:00.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:00 smithi145 ceph-mon[33384]: pgmap v878: 158 pgs: 158 active+clean; 3.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 81 MiB/s wr, 1.29k op/s 2024-09-11T08:58:00.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:00 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-11T08:58:00.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:00 smithi145 ceph-mon[33384]: osdmap e383: 8 total, 8 up, 8 in 2024-09-11T08:58:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:00 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:58:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:58:00.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:01.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:01.039 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:01.039 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:01.115 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:58:01.115 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-11T08:58:01.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-11T08:58:01.115 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:58:01.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.115 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120753 9.1 9:80027676:::benchmark_data_smithi092_145083_object120752:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e234)", 2024-09-11T08:58:01.115 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.482645+0000", 2024-09-11T08:58:01.116 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 455.62644775400003, 2024-09-11T08:58:01.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.202329761, 2024-09-11T08:58:01.116 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.116 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:58:01.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:58:01.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:01.116 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120753 2024-09-11T08:58:01.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.116 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482645+0000", 2024-09-11T08:58:01.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482645+0000", 2024-09-11T08:58:01.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482644+0000", 2024-09-11T08:58:01.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:58:01.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482670+0000", 2024-09-11T08:58:01.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6781999999999999e-05 2024-09-11T08:58:01.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482682+0000", 2024-09-11T08:58:01.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2051e-05 2024-09-11T08:58:01.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.482688+0000", 2024-09-11T08:58:01.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0870000000000003e-06 2024-09-11T08:58:01.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.487390+0000", 2024-09-11T08:58:01.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047011450000000003 2024-09-11T08:58:01.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.487514+0000", 2024-09-11T08:58:01.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000124664 2024-09-11T08:58:01.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6,7", 2024-09-11T08:58:01.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.487660+0000", 2024-09-11T08:58:01.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014545500000000001 2024-09-11T08:58:01.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.496777+0000", 2024-09-11T08:58:01.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0091171150000000003 2024-09-11T08:58:01.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.497628+0000", 2024-09-11T08:58:01.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00085127100000000002 2024-09-11T08:58:01.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684968+0000", 2024-09-11T08:58:01.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.187339704 2024-09-11T08:58:01.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.684975+0000", 2024-09-11T08:58:01.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-11T08:58:01.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.123 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.124 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120753 9.1 9:80027676:::benchmark_data_smithi092_145083_object120752:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:58:01.124 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.504067+0000", 2024-09-11T08:58:01.124 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 455.60502558600001, 2024-09-11T08:58:01.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0247133180000001, 2024-09-11T08:58:01.124 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.125 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:58:01.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:58:01.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:01.125 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120753 2024-09-11T08:58:01.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.125 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504067+0000", 2024-09-11T08:58:01.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504067+0000", 2024-09-11T08:58:01.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504066+0000", 2024-09-11T08:58:01.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:01.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504100+0000", 2024-09-11T08:58:01.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4513e-05 2024-09-11T08:58:01.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504114+0000", 2024-09-11T08:58:01.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305999999999999e-05 2024-09-11T08:58:01.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504120+0000", 2024-09-11T08:58:01.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2450000000000003e-06 2024-09-11T08:58:01.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685178+0000", 2024-09-11T08:58:01.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.181057575 2024-09-11T08:58:01.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:58:01.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685184+0000", 2024-09-11T08:58:01.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9600000000000003e-06 2024-09-11T08:58:01.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.528703+0000", 2024-09-11T08:58:01.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84351812699999995 2024-09-11T08:58:01.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.528781+0000", 2024-09-11T08:58:01.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8183999999999995e-05 2024-09-11T08:58:01.130 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.131 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.131 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.131 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120754 9.1 9:8766ca32:::benchmark_data_smithi092_145083_object120753:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:58:01.131 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.504271+0000", 2024-09-11T08:58:01.131 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 455.604822001, 2024-09-11T08:58:01.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0247913390000001, 2024-09-11T08:58:01.131 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.131 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-11T08:58:01.131 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.131 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:58:01.131 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:01.132 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120754 2024-09-11T08:58:01.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.132 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504271+0000", 2024-09-11T08:58:01.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504271+0000", 2024-09-11T08:58:01.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504270+0000", 2024-09-11T08:58:01.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:01.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504292+0000", 2024-09-11T08:58:01.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1151000000000001e-05 2024-09-11T08:58:01.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504299+0000", 2024-09-11T08:58:01.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9079999999999995e-06 2024-09-11T08:58:01.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.504304+0000", 2024-09-11T08:58:01.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5109999999999997e-06 2024-09-11T08:58:01.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685200+0000", 2024-09-11T08:58:01.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18089587500000001 2024-09-11T08:58:01.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:58:01.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685205+0000", 2024-09-11T08:58:01.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9439999999999996e-06 2024-09-11T08:58:01.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529010+0000", 2024-09-11T08:58:01.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84380478599999997 2024-09-11T08:58:01.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529062+0000", 2024-09-11T08:58:01.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2797999999999998e-05 2024-09-11T08:58:01.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.138 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.139 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:120773 9.1 9:84313f40:::benchmark_data_smithi092_145083_object120772:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:58:01.139 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:25.524403+0000", 2024-09-11T08:58:01.139 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 455.58468998900003, 2024-09-11T08:58:01.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0196523550000001, 2024-09-11T08:58:01.139 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.139 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:58:01.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:01.139 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 120773 2024-09-11T08:58:01.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.140 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524403+0000", 2024-09-11T08:58:01.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524403+0000", 2024-09-11T08:58:01.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524402+0000", 2024-09-11T08:58:01.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:01.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524428+0000", 2024-09-11T08:58:01.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6698000000000001e-05 2024-09-11T08:58:01.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524442+0000", 2024-09-11T08:58:01.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3806e-05 2024-09-11T08:58:01.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.524448+0000", 2024-09-11T08:58:01.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3109999999999996e-06 2024-09-11T08:58:01.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685220+0000", 2024-09-11T08:58:01.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16077285399999999 2024-09-11T08:58:01.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:58:01.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:25.685225+0000", 2024-09-11T08:58:01.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8810000000000004e-06 2024-09-11T08:58:01.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529083+0000", 2024-09-11T08:58:01.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84385757900000002 2024-09-11T08:58:01.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529204+0000", 2024-09-11T08:58:01.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121352 2024-09-11T08:58:01.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6,7", 2024-09-11T08:58:01.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.529650+0000", 2024-09-11T08:58:01.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044596399999999997 2024-09-11T08:58:01.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.533815+0000", 2024-09-11T08:58:01.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041648370000000002 2024-09-11T08:58:01.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.536911+0000", 2024-09-11T08:58:01.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030961280000000001 2024-09-11T08:58:01.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.543805+0000", 2024-09-11T08:58:01.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006893939 2024-09-11T08:58:01.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.544015+0000", 2024-09-11T08:58:01.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020939999999999999 2024-09-11T08:58:01.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:26.544055+0000", 2024-09-11T08:58:01.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0908000000000002e-05 2024-09-11T08:58:01.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.147 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.148 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:122641 9.19 9:9f7f102d:::benchmark_data_smithi092_145083_object122640:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:58:01.148 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:27.538908+0000", 2024-09-11T08:58:01.148 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 453.57018459099999, 2024-09-11T08:58:01.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.439093481, 2024-09-11T08:58:01.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.148 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:58:01.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:01.148 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122641 2024-09-11T08:58:01.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.149 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538908+0000", 2024-09-11T08:58:01.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538908+0000", 2024-09-11T08:58:01.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538907+0000", 2024-09-11T08:58:01.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:01.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538933+0000", 2024-09-11T08:58:01.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5627e-05 2024-09-11T08:58:01.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538945+0000", 2024-09-11T08:58:01.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2721000000000001e-05 2024-09-11T08:58:01.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.538951+0000", 2024-09-11T08:58:01.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5849999999999999e-06 2024-09-11T08:58:01.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.539347+0000", 2024-09-11T08:58:01.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000395936 2024-09-11T08:58:01.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:58:01.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.539355+0000", 2024-09-11T08:58:01.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6629999999999999e-06 2024-09-11T08:58:01.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.534990+0000", 2024-09-11T08:58:01.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99563493400000003 2024-09-11T08:58:01.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:58:01.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535002+0000", 2024-09-11T08:58:01.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1333e-05 2024-09-11T08:58:01.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.963852+0000", 2024-09-11T08:58:01.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.42885004700000001 2024-09-11T08:58:01.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.964026+0000", 2024-09-11T08:58:01.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017450399999999999 2024-09-11T08:58:01.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6,7", 2024-09-11T08:58:01.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.964280+0000", 2024-09-11T08:58:01.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025399599999999999 2024-09-11T08:58:01.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.965296+0000", 2024-09-11T08:58:01.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010152290000000001 2024-09-11T08:58:01.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.969974+0000", 2024-09-11T08:58:01.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046781729999999999 2024-09-11T08:58:01.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.971213+0000", 2024-09-11T08:58:01.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001239431 2024-09-11T08:58:01.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.977816+0000", 2024-09-11T08:58:01.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066024580000000003 2024-09-11T08:58:01.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.977893+0000", 2024-09-11T08:58:01.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7600000000000002e-05 2024-09-11T08:58:01.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.978002+0000", 2024-09-11T08:58:01.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010877199999999999 2024-09-11T08:58:01.158 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.158 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.158 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.158 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:122656 9.19 9:9b532791:::benchmark_data_smithi092_145083_object122655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:58:01.158 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:50:27.549187+0000", 2024-09-11T08:58:01.159 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 453.55990580000002, 2024-09-11T08:58:01.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4294275089999999, 2024-09-11T08:58:01.159 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.159 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.159 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.159 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:58:01.159 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:01.159 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 122656 2024-09-11T08:58:01.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.159 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549187+0000", 2024-09-11T08:58:01.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549187+0000", 2024-09-11T08:58:01.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549186+0000", 2024-09-11T08:58:01.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:01.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549208+0000", 2024-09-11T08:58:01.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2378000000000002e-05 2024-09-11T08:58:01.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549220+0000", 2024-09-11T08:58:01.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1711e-05 2024-09-11T08:58:01.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549226+0000", 2024-09-11T08:58:01.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7810000000000002e-06 2024-09-11T08:58:01.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549320+0000", 2024-09-11T08:58:01.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3955000000000004e-05 2024-09-11T08:58:01.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:58:01.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:27.549329+0000", 2024-09-11T08:58:01.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.952e-06 2024-09-11T08:58:01.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535136+0000", 2024-09-11T08:58:01.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98580697900000003 2024-09-11T08:58:01.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:58:01.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.535143+0000", 2024-09-11T08:58:01.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4440000000000002e-06 2024-09-11T08:58:01.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.965346+0000", 2024-09-11T08:58:01.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43020250500000001 2024-09-11T08:58:01.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.965438+0000", 2024-09-11T08:58:01.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2334000000000001e-05 2024-09-11T08:58:01.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6,7", 2024-09-11T08:58:01.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.965597+0000", 2024-09-11T08:58:01.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015942300000000001 2024-09-11T08:58:01.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.966745+0000", 2024-09-11T08:58:01.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001147482 2024-09-11T08:58:01.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.971264+0000", 2024-09-11T08:58:01.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045190960000000002 2024-09-11T08:58:01.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.978072+0000", 2024-09-11T08:58:01.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068077540000000001 2024-09-11T08:58:01.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.978396+0000", 2024-09-11T08:58:01.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000324247 2024-09-11T08:58:01.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.978508+0000", 2024-09-11T08:58:01.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112379 2024-09-11T08:58:01.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:50:28.978615+0000", 2024-09-11T08:58:01.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010634800000000001 2024-09-11T08:58:01.167 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.167 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.167 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.168 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272348 9.8 9:15db9131:::benchmark_data_smithi092_145083_object128950:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:58:01.168 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.641892+0000", 2024-09-11T08:58:01.168 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 336.46720132899998, 2024-09-11T08:58:01.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98679250600000001, 2024-09-11T08:58:01.168 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.168 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.168 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.168 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:58:01.168 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:01.168 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272348 2024-09-11T08:58:01.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.169 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641892+0000", 2024-09-11T08:58:01.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641892+0000", 2024-09-11T08:58:01.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641890+0000", 2024-09-11T08:58:01.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:58:01.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641898+0000", 2024-09-11T08:58:01.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7849999999999992e-06 2024-09-11T08:58:01.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641915+0000", 2024-09-11T08:58:01.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7677e-05 2024-09-11T08:58:01.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641923+0000", 2024-09-11T08:58:01.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6199999999999999e-06 2024-09-11T08:58:01.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641981+0000", 2024-09-11T08:58:01.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7901e-05 2024-09-11T08:58:01.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:58:01.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641989+0000", 2024-09-11T08:58:01.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3040000000000005e-06 2024-09-11T08:58:01.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.624095+0000", 2024-09-11T08:58:01.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.982105699 2024-09-11T08:58:01.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.624185+0000", 2024-09-11T08:58:01.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0476999999999994e-05 2024-09-11T08:58:01.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:58:01.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.624299+0000", 2024-09-11T08:58:01.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113366 2024-09-11T08:58:01.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628589+0000", 2024-09-11T08:58:01.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004290622 2024-09-11T08:58:01.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628628+0000", 2024-09-11T08:58:01.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8549e-05 2024-09-11T08:58:01.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628633+0000", 2024-09-11T08:58:01.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1059999999999999e-06 2024-09-11T08:58:01.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628662+0000", 2024-09-11T08:58:01.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8758000000000001e-05 2024-09-11T08:58:01.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628684+0000", 2024-09-11T08:58:01.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2416999999999999e-05 2024-09-11T08:58:01.175 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.175 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.175 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.175 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272383 9.8 9:12173f2a:::benchmark_data_smithi092_145083_object128985:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:58:01.175 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.665664+0000", 2024-09-11T08:58:01.175 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 336.44342943700002, 2024-09-11T08:58:01.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.96324268999999996, 2024-09-11T08:58:01.175 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.175 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.175 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.175 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:58:01.176 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:01.176 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272383 2024-09-11T08:58:01.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.176 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665664+0000", 2024-09-11T08:58:01.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665664+0000", 2024-09-11T08:58:01.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665663+0000", 2024-09-11T08:58:01.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:58:01.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665668+0000", 2024-09-11T08:58:01.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9049999999999996e-06 2024-09-11T08:58:01.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665678+0000", 2024-09-11T08:58:01.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0057e-05 2024-09-11T08:58:01.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665683+0000", 2024-09-11T08:58:01.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2270000000000001e-06 2024-09-11T08:58:01.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665724+0000", 2024-09-11T08:58:01.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000003e-05 2024-09-11T08:58:01.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:58:01.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665731+0000", 2024-09-11T08:58:01.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6030000000000001e-06 2024-09-11T08:58:01.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.626022+0000", 2024-09-11T08:58:01.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.96029152699999998 2024-09-11T08:58:01.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.626136+0000", 2024-09-11T08:58:01.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114009 2024-09-11T08:58:01.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:58:01.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.626246+0000", 2024-09-11T08:58:01.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010950000000000001 2024-09-11T08:58:01.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628734+0000", 2024-09-11T08:58:01.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024882149999999998 2024-09-11T08:58:01.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628762+0000", 2024-09-11T08:58:01.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8354e-05 2024-09-11T08:58:01.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628852+0000", 2024-09-11T08:58:01.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9708000000000002e-05 2024-09-11T08:58:01.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628888+0000", 2024-09-11T08:58:01.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6128999999999999e-05 2024-09-11T08:58:01.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628906+0000", 2024-09-11T08:58:01.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8218000000000001e-05 2024-09-11T08:58:01.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.182 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.183 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.183 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272428 9.8 9:14391abd:::benchmark_data_smithi092_145083_object129030:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:58:01.183 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.737328+0000", 2024-09-11T08:58:01.183 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 336.37176490500002, 2024-09-11T08:58:01.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.89314801799999999, 2024-09-11T08:58:01.183 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.183 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.183 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.183 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:58:01.183 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:01.183 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272428 2024-09-11T08:58:01.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.184 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737328+0000", 2024-09-11T08:58:01.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737328+0000", 2024-09-11T08:58:01.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737327+0000", 2024-09-11T08:58:01.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:01.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:58:01.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737333+0000", 2024-09-11T08:58:01.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6720000000000001e-06 2024-09-11T08:58:01.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737345+0000", 2024-09-11T08:58:01.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2007999999999999e-05 2024-09-11T08:58:01.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737353+0000", 2024-09-11T08:58:01.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0930000000000003e-06 2024-09-11T08:58:01.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737385+0000", 2024-09-11T08:58:01.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2394999999999997e-05 2024-09-11T08:58:01.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:58:01.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737391+0000", 2024-09-11T08:58:01.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1800000000000004e-06 2024-09-11T08:58:01.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.627276+0000", 2024-09-11T08:58:01.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.88988573100000001 2024-09-11T08:58:01.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.627371+0000", 2024-09-11T08:58:01.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4772000000000007e-05 2024-09-11T08:58:01.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:58:01.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.627432+0000", 2024-09-11T08:58:01.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1106999999999996e-05 2024-09-11T08:58:01.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628813+0000", 2024-09-11T08:58:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001381195 2024-09-11T08:58:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628822+0000", 2024-09-11T08:58:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5169999999999999e-06 2024-09-11T08:58:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630397+0000", 2024-09-11T08:58:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001574717 2024-09-11T08:58:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630443+0000", 2024-09-11T08:58:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6513e-05 2024-09-11T08:58:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630476+0000", 2024-09-11T08:58:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3039e-05 2024-09-11T08:58:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.195 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272436 9.8 9:16049abc:::benchmark_data_smithi092_145083_object129038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:58:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.749067+0000", 2024-09-11T08:58:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 336.36002624100001, 2024-09-11T08:58:01.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.88488331399999998, 2024-09-11T08:58:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:58:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272436 2024-09-11T08:58:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.197 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749067+0000", 2024-09-11T08:58:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749067+0000", 2024-09-11T08:58:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749065+0000", 2024-09-11T08:58:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:58:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749073+0000", 2024-09-11T08:58:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9009999999999994e-06 2024-09-11T08:58:01.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749094+0000", 2024-09-11T08:58:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1129999999999999e-05 2024-09-11T08:58:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749101+0000", 2024-09-11T08:58:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3699999999999997e-06 2024-09-11T08:58:01.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749220+0000", 2024-09-11T08:58:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118566 2024-09-11T08:58:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:58:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749229+0000", 2024-09-11T08:58:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1370000000000008e-06 2024-09-11T08:58:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628938+0000", 2024-09-11T08:58:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.87970911100000004 2024-09-11T08:58:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.629008+0000", 2024-09-11T08:58:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9467000000000004e-05 2024-09-11T08:58:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:58:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.629069+0000", 2024-09-11T08:58:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1353999999999996e-05 2024-09-11T08:58:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633156+0000", 2024-09-11T08:58:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040865729999999996 2024-09-11T08:58:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633192+0000", 2024-09-11T08:58:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5939999999999998e-05 2024-09-11T08:58:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633837+0000", 2024-09-11T08:58:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00064550800000000004 2024-09-11T08:58:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633902+0000", 2024-09-11T08:58:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5283000000000005e-05 2024-09-11T08:58:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633950+0000", 2024-09-11T08:58:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7713999999999998e-05 2024-09-11T08:58:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.204 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272445 9.8 9:14cf6d17:::benchmark_data_smithi092_145083_object129047:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:58:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.755404+0000", 2024-09-11T08:58:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 336.35368955899997, 2024-09-11T08:58:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.87888003299999995, 2024-09-11T08:58:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:58:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:01.205 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272445 2024-09-11T08:58:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755404+0000", 2024-09-11T08:58:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755404+0000", 2024-09-11T08:58:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755402+0000", 2024-09-11T08:58:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755410+0000", 2024-09-11T08:58:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8029999999999997e-06 2024-09-11T08:58:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755424+0000", 2024-09-11T08:58:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3912e-05 2024-09-11T08:58:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755429+0000", 2024-09-11T08:58:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7339999999999996e-06 2024-09-11T08:58:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755475+0000", 2024-09-11T08:58:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5982999999999997e-05 2024-09-11T08:58:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:58:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755485+0000", 2024-09-11T08:58:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3170000000000008e-06 2024-09-11T08:58:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630509+0000", 2024-09-11T08:58:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.87502458000000005 2024-09-11T08:58:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630612+0000", 2024-09-11T08:58:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102422 2024-09-11T08:58:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T08:58:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630716+0000", 2024-09-11T08:58:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104014 2024-09-11T08:58:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633274+0000", 2024-09-11T08:58:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025580450000000001 2024-09-11T08:58:01.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.634132+0000", 2024-09-11T08:58:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00085821300000000003 2024-09-11T08:58:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.634183+0000", 2024-09-11T08:58:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1190000000000003e-05 2024-09-11T08:58:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.634236+0000", 2024-09-11T08:58:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2466000000000002e-05 2024-09-11T08:58:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.634284+0000", 2024-09-11T08:58:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7987e-05 2024-09-11T08:58:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:244905 10.7 10:e8d6c095:::benchmark_data_smithi092_173109_object107946:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T08:58:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:07.472236+0000", 2024-09-11T08:58:01.212 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 113.63685746, 2024-09-11T08:58:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0816275449999999, 2024-09-11T08:58:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:58:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:58:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 244905 2024-09-11T08:58:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.472236+0000", 2024-09-11T08:58:01.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.472236+0000", 2024-09-11T08:58:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.472234+0000", 2024-09-11T08:58:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.472240+0000", 2024-09-11T08:58:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8529999999999997e-06 2024-09-11T08:58:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.472248+0000", 2024-09-11T08:58:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999999e-06 2024-09-11T08:58:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.472254+0000", 2024-09-11T08:58:01.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4289999999999998e-06 2024-09-11T08:58:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.477108+0000", 2024-09-11T08:58:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048533680000000003 2024-09-11T08:58:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:58:01.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.477116+0000", 2024-09-11T08:58:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1489999999999994e-06 2024-09-11T08:58:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.523932+0000", 2024-09-11T08:58:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0468164870000001 2024-09-11T08:58:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.526492+0000", 2024-09-11T08:58:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025598610000000001 2024-09-11T08:58:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:58:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.526779+0000", 2024-09-11T08:58:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028662499999999999 2024-09-11T08:58:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.540305+0000", 2024-09-11T08:58:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013526404000000001 2024-09-11T08:58:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.540317+0000", 2024-09-11T08:58:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1440999999999999e-05 2024-09-11T08:58:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.553771+0000", 2024-09-11T08:58:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01345447 2024-09-11T08:58:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.553820+0000", 2024-09-11T08:58:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8513999999999997e-05 2024-09-11T08:58:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.553863+0000", 2024-09-11T08:58:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3473000000000002e-05 2024-09-11T08:58:01.219 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:244916 10.f 10:fbfc297d:::benchmark_data_smithi092_173109_object107957:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T08:58:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:07.479877+0000", 2024-09-11T08:58:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 113.629215891, 2024-09-11T08:58:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.074429866, 2024-09-11T08:58:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:58:01.220 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:58:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 244916 2024-09-11T08:58:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479877+0000", 2024-09-11T08:58:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479877+0000", 2024-09-11T08:58:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479876+0000", 2024-09-11T08:58:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479882+0000", 2024-09-11T08:58:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3230000000000003e-06 2024-09-11T08:58:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479893+0000", 2024-09-11T08:58:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1497e-05 2024-09-11T08:58:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479899+0000", 2024-09-11T08:58:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3789999999999997e-06 2024-09-11T08:58:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479983+0000", 2024-09-11T08:58:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3843000000000003e-05 2024-09-11T08:58:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:58:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479993+0000", 2024-09-11T08:58:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0549e-05 2024-09-11T08:58:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.521849+0000", 2024-09-11T08:58:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0418553269999999 2024-09-11T08:58:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.524337+0000", 2024-09-11T08:58:01.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024885490000000001 2024-09-11T08:58:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,6", 2024-09-11T08:58:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.524582+0000", 2024-09-11T08:58:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024493600000000001 2024-09-11T08:58:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.537423+0000", 2024-09-11T08:58:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012840991 2024-09-11T08:58:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.540312+0000", 2024-09-11T08:58:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028889229999999998 2024-09-11T08:58:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.554139+0000", 2024-09-11T08:58:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01382735 2024-09-11T08:58:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.554270+0000", 2024-09-11T08:58:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013089199999999999 2024-09-11T08:58:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.554307+0000", 2024-09-11T08:58:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6869000000000002e-05 2024-09-11T08:58:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:244927 10.7 10:e600bc70:::benchmark_data_smithi092_173109_object107968:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T08:58:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:07.498963+0000", 2024-09-11T08:58:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 113.610129642, 2024-09-11T08:58:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0561594940000001, 2024-09-11T08:58:01.228 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:58:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:58:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 244927 2024-09-11T08:58:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.498963+0000", 2024-09-11T08:58:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.498963+0000", 2024-09-11T08:58:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.498962+0000", 2024-09-11T08:58:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.498971+0000", 2024-09-11T08:58:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9940000000000006e-06 2024-09-11T08:58:01.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.498989+0000", 2024-09-11T08:58:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8427000000000002e-05 2024-09-11T08:58:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.498996+0000", 2024-09-11T08:58:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-11T08:58:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.499040+0000", 2024-09-11T08:58:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3906999999999998e-05 2024-09-11T08:58:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:58:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.499048+0000", 2024-09-11T08:58:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2849999999999995e-06 2024-09-11T08:58:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.537427+0000", 2024-09-11T08:58:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.038378536 2024-09-11T08:58:01.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.537552+0000", 2024-09-11T08:58:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000125418 2024-09-11T08:58:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:58:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.537677+0000", 2024-09-11T08:58:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012447900000000001 2024-09-11T08:58:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.540462+0000", 2024-09-11T08:58:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027854339999999998 2024-09-11T08:58:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.543405+0000", 2024-09-11T08:58:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029425369999999998 2024-09-11T08:58:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.553899+0000", 2024-09-11T08:58:01.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010494768999999999 2024-09-11T08:58:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.555084+0000", 2024-09-11T08:58:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00118486 2024-09-11T08:58:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.555123+0000", 2024-09-11T08:58:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8571000000000001e-05 2024-09-11T08:58:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.236 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:49863 11.2 11:47a5097e:::benchmark_data_smithi092_205252_object49862:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-11T08:58:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:51.719001+0000", 2024-09-11T08:58:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3900924860000003, 2024-09-11T08:58:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23083320700000001, 2024-09-11T08:58:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T08:58:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T08:58:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49863 2024-09-11T08:58:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719001+0000", 2024-09-11T08:58:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719001+0000", 2024-09-11T08:58:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.718999+0000", 2024-09-11T08:58:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719024+0000", 2024-09-11T08:58:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4508e-05 2024-09-11T08:58:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719037+0000", 2024-09-11T08:58:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3295000000000001e-05 2024-09-11T08:58:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719043+0000", 2024-09-11T08:58:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8200000000000002e-06 2024-09-11T08:58:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719079+0000", 2024-09-11T08:58:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6334999999999997e-05 2024-09-11T08:58:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719192+0000", 2024-09-11T08:58:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112426 2024-09-11T08:58:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:58:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719312+0000", 2024-09-11T08:58:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119926 2024-09-11T08:58:01.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.723980+0000", 2024-09-11T08:58:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004668681 2024-09-11T08:58:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.725139+0000", 2024-09-11T08:58:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011589650000000001 2024-09-11T08:58:01.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.949708+0000", 2024-09-11T08:58:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22456922900000001 2024-09-11T08:58:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.949787+0000", 2024-09-11T08:58:01.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8739000000000006e-05 2024-09-11T08:58:01.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.949834+0000", 2024-09-11T08:58:01.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6640000000000001e-05 2024-09-11T08:58:01.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.244 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.245 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:49867 11.2 11:4412ca95:::benchmark_data_smithi092_205252_object49866:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-11T08:58:01.245 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:51.721317+0000", 2024-09-11T08:58:01.245 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3877760099999996, 2024-09-11T08:58:01.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23993163000000001, 2024-09-11T08:58:01.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T08:58:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T08:58:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49867 2024-09-11T08:58:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721317+0000", 2024-09-11T08:58:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721317+0000", 2024-09-11T08:58:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721316+0000", 2024-09-11T08:58:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:58:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721341+0000", 2024-09-11T08:58:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.516e-05 2024-09-11T08:58:01.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721358+0000", 2024-09-11T08:58:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6738000000000001e-05 2024-09-11T08:58:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721365+0000", 2024-09-11T08:58:01.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.041e-06 2024-09-11T08:58:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721489+0000", 2024-09-11T08:58:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012353700000000001 2024-09-11T08:58:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721613+0000", 2024-09-11T08:58:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012459600000000001 2024-09-11T08:58:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:58:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721830+0000", 2024-09-11T08:58:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021665500000000001 2024-09-11T08:58:01.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.728332+0000", 2024-09-11T08:58:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065019960000000003 2024-09-11T08:58:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.728338+0000", 2024-09-11T08:58:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6280000000000002e-06 2024-09-11T08:58:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961116+0000", 2024-09-11T08:58:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23277804399999999 2024-09-11T08:58:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961209+0000", 2024-09-11T08:58:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3527000000000001e-05 2024-09-11T08:58:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961249+0000", 2024-09-11T08:58:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9564000000000003e-05 2024-09-11T08:58:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.254 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:49875 11.2 11:41e4a4f5:::benchmark_data_smithi092_205252_object49874:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-11T08:58:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:51.724742+0000", 2024-09-11T08:58:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3843513420000004, 2024-09-11T08:58:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23689321699999999, 2024-09-11T08:58:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T08:58:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T08:58:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49875 2024-09-11T08:58:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.724742+0000", 2024-09-11T08:58:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.724742+0000", 2024-09-11T08:58:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.724740+0000", 2024-09-11T08:58:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.724762+0000", 2024-09-11T08:58:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1639000000000001e-05 2024-09-11T08:58:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.724781+0000", 2024-09-11T08:58:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8861000000000001e-05 2024-09-11T08:58:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.724787+0000", 2024-09-11T08:58:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0789999999999997e-06 2024-09-11T08:58:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.725161+0000", 2024-09-11T08:58:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037398200000000003 2024-09-11T08:58:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.725426+0000", 2024-09-11T08:58:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026499100000000001 2024-09-11T08:58:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:58:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.725741+0000", 2024-09-11T08:58:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031497399999999998 2024-09-11T08:58:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.730610+0000", 2024-09-11T08:58:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048688070000000002 2024-09-11T08:58:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732667+0000", 2024-09-11T08:58:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002057101 2024-09-11T08:58:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961283+0000", 2024-09-11T08:58:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.228616554 2024-09-11T08:58:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961599+0000", 2024-09-11T08:58:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031523100000000002 2024-09-11T08:58:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961635+0000", 2024-09-11T08:58:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6337999999999997e-05 2024-09-11T08:58:01.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:49880 11.2 11:41f0ab8a:::benchmark_data_smithi092_205252_object49879:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-11T08:58:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:51.727728+0000", 2024-09-11T08:58:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3813651260000004, 2024-09-11T08:58:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.234053021, 2024-09-11T08:58:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T08:58:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T08:58:01.263 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49880 2024-09-11T08:58:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.727728+0000", 2024-09-11T08:58:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.727728+0000", 2024-09-11T08:58:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.727727+0000", 2024-09-11T08:58:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.727751+0000", 2024-09-11T08:58:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4093000000000001e-05 2024-09-11T08:58:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.727764+0000", 2024-09-11T08:58:01.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3315e-05 2024-09-11T08:58:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.727770+0000", 2024-09-11T08:58:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-06 2024-09-11T08:58:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.728358+0000", 2024-09-11T08:58:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00058712599999999999 2024-09-11T08:58:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.728472+0000", 2024-09-11T08:58:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114449 2024-09-11T08:58:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:58:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.728601+0000", 2024-09-11T08:58:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129238 2024-09-11T08:58:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732735+0000", 2024-09-11T08:58:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041335929999999996 2024-09-11T08:58:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.734415+0000", 2024-09-11T08:58:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016805209999999999 2024-09-11T08:58:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961694+0000", 2024-09-11T08:58:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.227278969 2024-09-11T08:58:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961745+0000", 2024-09-11T08:58:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0729999999999997e-05 2024-09-11T08:58:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961781+0000", 2024-09-11T08:58:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5941e-05 2024-09-11T08:58:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:49884 11.2 11:4a5a3be8:::benchmark_data_smithi092_205252_object49883:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-11T08:58:01.269 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:51.729597+0000", 2024-09-11T08:58:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3794956319999994, 2024-09-11T08:58:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23231478699999999, 2024-09-11T08:58:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T08:58:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T08:58:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49884 2024-09-11T08:58:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.729597+0000", 2024-09-11T08:58:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.729597+0000", 2024-09-11T08:58:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.729596+0000", 2024-09-11T08:58:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:58:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.729623+0000", 2024-09-11T08:58:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6237e-05 2024-09-11T08:58:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.729638+0000", 2024-09-11T08:58:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5699999999999999e-05 2024-09-11T08:58:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.729644+0000", 2024-09-11T08:58:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.896e-06 2024-09-11T08:58:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.730629+0000", 2024-09-11T08:58:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0009843740000000001 2024-09-11T08:58:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.730730+0000", 2024-09-11T08:58:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010111300000000001 2024-09-11T08:58:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:58:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.730860+0000", 2024-09-11T08:58:01.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000130257 2024-09-11T08:58:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.734454+0000", 2024-09-11T08:58:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035937040000000001 2024-09-11T08:58:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.734470+0000", 2024-09-11T08:58:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6396e-05 2024-09-11T08:58:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961822+0000", 2024-09-11T08:58:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22735159699999999 2024-09-11T08:58:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961880+0000", 2024-09-11T08:58:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8118000000000002e-05 2024-09-11T08:58:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961912+0000", 2024-09-11T08:58:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2489e-05 2024-09-11T08:58:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.275 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:49887 11.2 11:4fe36588:::benchmark_data_smithi092_205252_object49886:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-11T08:58:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:51.732433+0000", 2024-09-11T08:58:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3766604430000005, 2024-09-11T08:58:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22958763600000001, 2024-09-11T08:58:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:58:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:58:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T08:58:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T08:58:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49887 2024-09-11T08:58:01.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:58:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:58:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732433+0000", 2024-09-11T08:58:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:58:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732433+0000", 2024-09-11T08:58:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:58:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:58:01.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732431+0000", 2024-09-11T08:58:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:58:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:58:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732456+0000", 2024-09-11T08:58:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4791e-05 2024-09-11T08:58:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:58:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732466+0000", 2024-09-11T08:58:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7759999999999992e-06 2024-09-11T08:58:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:58:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732471+0000", 2024-09-11T08:58:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2689999999999999e-06 2024-09-11T08:58:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:58:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732846+0000", 2024-09-11T08:58:01.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037487000000000002 2024-09-11T08:58:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:58:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732970+0000", 2024-09-11T08:58:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012358700000000001 2024-09-11T08:58:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T08:58:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.733108+0000", 2024-09-11T08:58:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013874100000000001 2024-09-11T08:58:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:58:01.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.734544+0000", 2024-09-11T08:58:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014350509999999999 2024-09-11T08:58:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.734743+0000", 2024-09-11T08:58:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019981400000000001 2024-09-11T08:58:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961947+0000", 2024-09-11T08:58:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22720348600000001 2024-09-11T08:58:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:58:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961989+0000", 2024-09-11T08:58:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1922999999999998e-05 2024-09-11T08:58:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:58:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:58:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:58:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.962020+0000", 2024-09-11T08:58:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1520000000000003e-05 2024-09-11T08:58:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:58:01.283 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:58:01.283 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:58:01.290 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:01.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:01.291 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:01.337 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 32 15 64218 64203 125.372 90 0.0160101 0.0079711 2024-09-11T08:58:01.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:01.434 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:01.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:01.435 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:01.535 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:01.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:01.574 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:01.574 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:01.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:01.714 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:01.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:01.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:01.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:58:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:58:01] "GET /metrics HTTP/1.1" 200 225081 "" "Prometheus/2.43.0" 2024-09-11T08:58:01.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:01.855 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:01.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:01.856 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:01.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:01.998 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:01.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:01.999 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:02.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:02.146 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:02.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:02.147 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:02.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:02.281 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:02.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:02.284 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:02.337 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 33 15 65691 65676 124.363 92.0625 0.0158652 0.00803645 2024-09-11T08:58:02.385 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:02.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:02 smithi145 ceph-mon[33384]: osdmap e384: 8 total, 8 up, 8 in 2024-09-11T08:58:02.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:02 smithi145 ceph-mon[33384]: pgmap v881: 168 pgs: 6 unknown, 162 active+clean; 3.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 114 MiB/s wr, 1.82k op/s 2024-09-11T08:58:02.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:02 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]: dispatch 2024-09-11T08:58:02.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:02 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]: dispatch 2024-09-11T08:58:02.422 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:02.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:02.423 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:02.491 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:02 smithi092 ceph-mon[29647]: osdmap e384: 8 total, 8 up, 8 in 2024-09-11T08:58:02.491 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:02 smithi092 ceph-mon[29647]: pgmap v881: 168 pgs: 6 unknown, 162 active+clean; 3.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 114 MiB/s wr, 1.82k op/s 2024-09-11T08:58:02.491 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:02 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]: dispatch 2024-09-11T08:58:02.491 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:02 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]: dispatch 2024-09-11T08:58:02.491 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:02 smithi092 ceph-mon[41114]: osdmap e384: 8 total, 8 up, 8 in 2024-09-11T08:58:02.491 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:02 smithi092 ceph-mon[41114]: pgmap v881: 168 pgs: 6 unknown, 162 active+clean; 3.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 114 MiB/s wr, 1.82k op/s 2024-09-11T08:58:02.491 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:02 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]: dispatch 2024-09-11T08:58:02.491 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:02 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]: dispatch 2024-09-11T08:58:02.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:02.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:02.563 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:02.563 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:02.664 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:02.691 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:02.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:02.693 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:02.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:02.825 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:02.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:02.826 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:02.928 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:02.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:02.964 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:02.965 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:03.066 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:03.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:03.104 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:03.104 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:03.129 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:58:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:58:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:58:03.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:03.246 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:03.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:03.247 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:03.337 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 34 15 67144 67129 123.376 90.8125 0.011676 0.00810007 2024-09-11T08:58:03.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:03.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:03.384 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:03.384 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:03.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:03 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:58:03.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:03 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:58:03.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:03 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]': finished 2024-09-11T08:58:03.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:03 smithi145 ceph-mon[33384]: osdmap e385: 8 total, 8 up, 8 in 2024-09-11T08:58:03.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:03 smithi145 ceph-mon[33384]: osdmap e386: 8 total, 8 up, 8 in 2024-09-11T08:58:03.451 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:03 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:58:03.451 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:03 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:58:03.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:03 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]': finished 2024-09-11T08:58:03.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:03 smithi092 ceph-mon[29647]: osdmap e385: 8 total, 8 up, 8 in 2024-09-11T08:58:03.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:03 smithi092 ceph-mon[29647]: osdmap e386: 8 total, 8 up, 8 in 2024-09-11T08:58:03.452 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:03 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:58:03.452 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:03 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:58:03.452 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:03 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]': finished 2024-09-11T08:58:03.452 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:03 smithi092 ceph-mon[41114]: osdmap e385: 8 total, 8 up, 8 in 2024-09-11T08:58:03.452 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:03 smithi092 ceph-mon[41114]: osdmap e386: 8 total, 8 up, 8 in 2024-09-11T08:58:03.486 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:03.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:03.516 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:03.517 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:03.618 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:03.653 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:03.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:03.654 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:03.755 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:03.795 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:03.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:03.796 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:03.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:03.937 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-11T08:58:03.947 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:03.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:03.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:04.049 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:04.089 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:04.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:04.090 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:04.191 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:04.227 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:04.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:04.228 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:04.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:04.337 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 35 16 68583 68567 122.418 89.875 0.00894999 0.00816357 2024-09-11T08:58:04.343 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:58:04.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:04.368 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:04.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:04.403 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:04 smithi092 ceph-mon[29647]: pgmap v884: 168 pgs: 168 active+clean; 3.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-11T08:58:04.403 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:04 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:58:04.403 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:04 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:58:04.403 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:04 smithi092 ceph-mon[41114]: pgmap v884: 168 pgs: 168 active+clean; 3.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-11T08:58:04.403 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:04 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:58:04.403 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:04 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:58:04.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:04 smithi145 ceph-mon[33384]: pgmap v884: 168 pgs: 168 active+clean; 3.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-11T08:58:04.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:04 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:58:04.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:04 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:58:04.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:04.511 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:04.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:04.512 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:04.612 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:04.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:04.650 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:04.650 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:04.751 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:04.791 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:04.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:04.792 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:04.893 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:04.924 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:04.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:04.925 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:04.984 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:58:04.984 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:58:04.984 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-11T08:58:04.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd out 6 2024-09-11T08:58:05.018 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:58:05.018 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:58:05.019 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:58:05.019 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:58:05.019 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:58:05.019 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:58:05.019 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:58:05.019 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:58:05.019 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:58:05.019 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:58:05.019 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:58:05.019 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:58:05.020 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:58:05.020 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:58:05.020 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:58:05.020 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:58:05.026 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:05.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:05.067 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:05.067 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:05.168 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:05.211 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:05.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:05.212 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:05.313 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:05.337 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 36 16 69961 69945 121.409 86.125 0.0112135 0.00823091 2024-09-11T08:58:05.349 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:05.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:05.350 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:05.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:05 smithi145 ceph-mon[33384]: osdmap e387: 8 total, 8 up, 8 in 2024-09-11T08:58:05.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:05 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/407224939' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:58:05.451 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:05.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:05.484 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:05.484 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:05.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:05 smithi092 ceph-mon[29647]: osdmap e387: 8 total, 8 up, 8 in 2024-09-11T08:58:05.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:05 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/407224939' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:58:05.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:05 smithi092 ceph-mon[41114]: osdmap e387: 8 total, 8 up, 8 in 2024-09-11T08:58:05.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:05 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/407224939' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:58:05.585 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:05.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:05.619 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:05.619 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:05.720 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:05.754 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:05.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:05.755 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:05.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:05.891 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:05.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:05.892 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:05.993 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:06.033 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:06.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:06.034 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:06.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:06.177 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:06.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:06.178 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:06.279 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:06.310 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:06.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:06.311 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:06.339 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 37 16 71431 71415 120.606 91.875 0.0123985 0.00828672 2024-09-11T08:58:06.412 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:06.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:06.457 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:06.457 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:06.558 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:06.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:06 smithi092 ceph-mon[29647]: pgmap v886: 168 pgs: 168 active+clean; 3.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 107 MiB/s wr, 1.71k op/s 2024-09-11T08:58:06.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:06 smithi092 ceph-mon[41114]: pgmap v886: 168 pgs: 168 active+clean; 3.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 107 MiB/s wr, 1.71k op/s 2024-09-11T08:58:06.593 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:06.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:06.594 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:06.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:06 smithi145 ceph-mon[33384]: pgmap v886: 168 pgs: 168 active+clean; 3.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 107 MiB/s wr, 1.71k op/s 2024-09-11T08:58:06.695 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:06.730 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:06.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:06.731 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:06.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:06.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:06.873 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:06.873 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:06.938 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:58:06.975 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:07.006 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:07.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:07.007 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:07.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:07.148 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:07.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:07.149 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:07.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:07.295 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:58:07.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:07.298 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:07.339 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 38 16 72675 72659 119.478 77.75 0.0212608 0.00836492 2024-09-11T08:58:07.399 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:07.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:07.435 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:07.435 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:07.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:07 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2081323911' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:58:07.512 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:07 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2081323911' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:58:07.536 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:07.564 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:07.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:07.565 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:07.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:07 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2081323911' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:58:07.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:58:07 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:58:07.666 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:07.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:07.708 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:07.708 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:07.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:07.833 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:58:07.834 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:58:07.840 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:07.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:07.841 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:07.942 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:07.980 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:07.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:07.981 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:08.082 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:08.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:08.117 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:08.117 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:08.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:08.251 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:08.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:08.252 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:08.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:58:08.340 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 39 16 74096 74080 118.692 88.8125 0.0145385 0.00841967 2024-09-11T08:58:08.353 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:08.388 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:08.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:08.389 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:08.490 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:08.516 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:08 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:08.516 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:08 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:58:08.516 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:08 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2081323911' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:58:08.516 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:08 smithi145 ceph-mon[33384]: osdmap e388: 8 total, 8 up, 8 in 2024-09-11T08:58:08.516 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:08 smithi145 ceph-mon[33384]: pgmap v888: 168 pgs: 168 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 5.1 KiB/s rd, 84 MiB/s wr, 1.35k op/s; 80 KiB/s, 0 objects/s recovering 2024-09-11T08:58:08.524 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:08.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:08.525 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:08.553 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-11T08:58:08.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:08 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:08.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:08 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:58:08.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:08 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2081323911' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:58:08.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:08 smithi092 ceph-mon[29647]: osdmap e388: 8 total, 8 up, 8 in 2024-09-11T08:58:08.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:08 smithi092 ceph-mon[29647]: pgmap v888: 168 pgs: 168 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 5.1 KiB/s rd, 84 MiB/s wr, 1.35k op/s; 80 KiB/s, 0 objects/s recovering 2024-09-11T08:58:08.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:08 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:08.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:08 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:58:08.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:08 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2081323911' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:58:08.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:08 smithi092 ceph-mon[41114]: osdmap e388: 8 total, 8 up, 8 in 2024-09-11T08:58:08.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:08 smithi092 ceph-mon[41114]: pgmap v888: 168 pgs: 168 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 5.1 KiB/s rd, 84 MiB/s wr, 1.35k op/s; 80 KiB/s, 0 objects/s recovering 2024-09-11T08:58:08.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:08.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:08.660 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:08.761 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:08.798 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:08.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:08.799 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:08.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:08.935 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:08.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:08.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:08.992 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:58:09.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:09.075 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:09.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:09.076 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:09.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:09.214 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:09.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:09.215 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:09.316 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:09.340 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:58:09.341005+0000 min lat: 0.0011062 max lat: 0.241699 avg lat: 0.00848816 2024-09-11T08:58:09.340 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:58:09.340 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 40 16 75382 75366 117.734 80.375 0.0178168 0.00848816 2024-09-11T08:58:09.351 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:09.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:09.352 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:09.453 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:09.492 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:09.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:09.493 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:09.593 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:09.621 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:09.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:09.622 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:09.629 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:58:09.723 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:09.764 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:09.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:09.765 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:09.866 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:09.909 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:09.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:09.910 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:10.011 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:10.049 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:10.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:10.050 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:10.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:09 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a[29624]: 2024-09-11T08:58:09.898+0000 7f9ae2e34640 -1 mon.a@0(leader).osd e388 definitely_dead 0 2024-09-11T08:58:10.123 INFO:teuthology.orchestra.run.smithi092.stderr:marked out osd.6. 2024-09-11T08:58:10.151 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:10.178 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:10.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:10.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:10.280 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:10.302 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:10.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:10.303 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:10.340 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 41 16 76407 76391 116.425 64.0625 0.00319917 0.00854048 2024-09-11T08:58:10.358 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:10 smithi145 ceph-mon[33384]: pgmap v889: 168 pgs: 168 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 4.3 KiB/s rd, 71 MiB/s wr, 1.14k op/s; 67 KiB/s, 0 objects/s recovering 2024-09-11T08:58:10.359 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:10 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1735136654' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-09-11T08:58:10.359 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:10 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-09-11T08:58:10.359 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:10 smithi145 ceph-mon[33384]: Client client.admin marked osd.6 out, while it was still marked up 2024-09-11T08:58:10.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:10.426 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:10.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:10.427 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:10.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:10.549 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:10.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:10.550 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:10.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:10 smithi092 ceph-mon[29647]: pgmap v889: 168 pgs: 168 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 4.3 KiB/s rd, 71 MiB/s wr, 1.14k op/s; 67 KiB/s, 0 objects/s recovering 2024-09-11T08:58:10.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:10 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1735136654' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-09-11T08:58:10.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:10 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-09-11T08:58:10.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:10 smithi092 ceph-mon[29647]: Client client.admin marked osd.6 out, while it was still marked up 2024-09-11T08:58:10.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:10 smithi092 ceph-mon[41114]: pgmap v889: 168 pgs: 168 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 4.3 KiB/s rd, 71 MiB/s wr, 1.14k op/s; 67 KiB/s, 0 objects/s recovering 2024-09-11T08:58:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:10 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1735136654' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-09-11T08:58:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:10 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-09-11T08:58:10.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:10 smithi092 ceph-mon[41114]: Client client.admin marked osd.6 out, while it was still marked up 2024-09-11T08:58:10.651 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:10.672 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:10.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:10.673 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:10.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:10.796 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:10.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:10.797 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:10.897 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:10.904 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:58:10.904 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:58:10.920 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:10.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:10.920 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:11.021 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:11.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:58:11.042 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:11.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:11.043 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:11.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:11.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:11 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["6"]}]': finished 2024-09-11T08:58:11.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:11 smithi145 ceph-mon[33384]: osdmap e389: 8 total, 8 up, 7 in 2024-09-11T08:58:11.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:11 smithi145 ceph-mon[33384]: osdmap e390: 8 total, 8 up, 7 in 2024-09-11T08:58:11.167 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:11.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:11.168 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:11.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:11.289 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:11.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:11.290 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:11.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:11 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["6"]}]': finished 2024-09-11T08:58:11.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:11 smithi092 ceph-mon[29647]: osdmap e389: 8 total, 8 up, 7 in 2024-09-11T08:58:11.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:11 smithi092 ceph-mon[29647]: osdmap e390: 8 total, 8 up, 7 in 2024-09-11T08:58:11.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:11 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["6"]}]': finished 2024-09-11T08:58:11.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:11 smithi092 ceph-mon[41114]: osdmap e389: 8 total, 8 up, 7 in 2024-09-11T08:58:11.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:11 smithi092 ceph-mon[41114]: osdmap e390: 8 total, 8 up, 7 in 2024-09-11T08:58:11.340 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 42 16 76407 76391 113.653 0 - 0.00854048 2024-09-11T08:58:11.391 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:11.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:11.413 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:11.413 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:11.514 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:11.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:11.536 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:11.536 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:11.616 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-11T08:58:11.636 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:11.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:11.681 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:11.681 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:11.782 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:11.805 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:11.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:11.806 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:11.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:58:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:58:11] "GET /metrics HTTP/1.1" 200 225081 "" "Prometheus/2.43.0" 2024-09-11T08:58:11.862 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:58:11.907 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:11.928 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:11.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:11.929 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:12.030 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:12.053 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:12.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:12.054 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:12.155 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:12.180 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:12.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:12.181 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:12.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:12 smithi092 ceph-mon[29647]: pgmap v892: 168 pgs: 5 remapped+peering, 1 active+recovery_wait+degraded, 1 active+recovering, 161 active+clean; 4.5 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 5.3 KiB/s rd, 98 MiB/s wr, 1.57k op/s; 10/222555 objects degraded (0.004%); 1/222555 objects misplaced (0.000%); 124 KiB/s, 1 objects/s recovering 2024-09-11T08:58:12.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:12 smithi092 ceph-mon[29647]: osdmap e391: 8 total, 8 up, 7 in 2024-09-11T08:58:12.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:12 smithi092 ceph-mon[41114]: pgmap v892: 168 pgs: 5 remapped+peering, 1 active+recovery_wait+degraded, 1 active+recovering, 161 active+clean; 4.5 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 5.3 KiB/s rd, 98 MiB/s wr, 1.57k op/s; 10/222555 objects degraded (0.004%); 1/222555 objects misplaced (0.000%); 124 KiB/s, 1 objects/s recovering 2024-09-11T08:58:12.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:12 smithi092 ceph-mon[41114]: osdmap e391: 8 total, 8 up, 7 in 2024-09-11T08:58:12.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:12.295 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:12 smithi145 ceph-mon[33384]: pgmap v892: 168 pgs: 5 remapped+peering, 1 active+recovery_wait+degraded, 1 active+recovering, 161 active+clean; 4.5 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 5.3 KiB/s rd, 98 MiB/s wr, 1.57k op/s; 10/222555 objects degraded (0.004%); 1/222555 objects misplaced (0.000%); 124 KiB/s, 1 objects/s recovering 2024-09-11T08:58:12.295 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:12 smithi145 ceph-mon[33384]: osdmap e391: 8 total, 8 up, 7 in 2024-09-11T08:58:12.323 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:12.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:12.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:12.340 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 43 16 76442 76426 111.061 1.09375 0.00834174 0.00885052 2024-09-11T08:58:12.425 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:12.454 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:12.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:12.455 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:12.555 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:12.582 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:12.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:12.583 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:12.684 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:12.712 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:12.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:12.713 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:12.813 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:12.834 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:12.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:12.835 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:12.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:12.975 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:12.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:12.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:13.077 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:13.101 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:58:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:58:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:58:13.105 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:13.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:13.106 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:13.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:13.248 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:13.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:13.249 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:13.340 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 44 16 77162 77146 109.56 45 0.0154179 0.00912077 2024-09-11T08:58:13.349 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:13.377 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:13.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:13.378 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:13.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:13 smithi145 ceph-mon[33384]: 11.8 starting backfill to osd.0 from (0'0,0'0] MAX to 388'4796 2024-09-11T08:58:13.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:13 smithi145 ceph-mon[33384]: 11.1 starting backfill to osd.2 from (0'0,0'0] MAX to 388'4774 2024-09-11T08:58:13.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:13 smithi145 ceph-mon[33384]: 11.a starting backfill to osd.2 from (0'0,0'0] MAX to 388'4845 2024-09-11T08:58:13.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:13 smithi145 ceph-mon[33384]: 11.9 starting backfill to osd.1 from (0'0,0'0] MAX to 388'4683 2024-09-11T08:58:13.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:13 smithi145 ceph-mon[33384]: 11.3 starting backfill to osd.2 from (0'0,0'0] MAX to 388'4823 2024-09-11T08:58:13.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:13 smithi145 ceph-mon[33384]: 11.5 starting backfill to osd.3 from (0'0,0'0] MAX to 388'4806 2024-09-11T08:58:13.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:13 smithi145 ceph-mon[33384]: 11.4 starting backfill to osd.1 from (0'0,0'0] MAX to 388'4796 2024-09-11T08:58:13.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:13 smithi145 ceph-mon[33384]: Health check failed: Degraded data redundancy: 10/222555 objects degraded (0.004%), 1 pg degraded (PG_DEGRADED) 2024-09-11T08:58:13.479 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:13.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:13.507 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:13.507 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:13.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[29647]: 11.8 starting backfill to osd.0 from (0'0,0'0] MAX to 388'4796 2024-09-11T08:58:13.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[29647]: 11.1 starting backfill to osd.2 from (0'0,0'0] MAX to 388'4774 2024-09-11T08:58:13.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[29647]: 11.a starting backfill to osd.2 from (0'0,0'0] MAX to 388'4845 2024-09-11T08:58:13.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[29647]: 11.9 starting backfill to osd.1 from (0'0,0'0] MAX to 388'4683 2024-09-11T08:58:13.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[29647]: 11.3 starting backfill to osd.2 from (0'0,0'0] MAX to 388'4823 2024-09-11T08:58:13.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[29647]: 11.5 starting backfill to osd.3 from (0'0,0'0] MAX to 388'4806 2024-09-11T08:58:13.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[29647]: 11.4 starting backfill to osd.1 from (0'0,0'0] MAX to 388'4796 2024-09-11T08:58:13.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[29647]: Health check failed: Degraded data redundancy: 10/222555 objects degraded (0.004%), 1 pg degraded (PG_DEGRADED) 2024-09-11T08:58:13.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[41114]: 11.8 starting backfill to osd.0 from (0'0,0'0] MAX to 388'4796 2024-09-11T08:58:13.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[41114]: 11.1 starting backfill to osd.2 from (0'0,0'0] MAX to 388'4774 2024-09-11T08:58:13.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[41114]: 11.a starting backfill to osd.2 from (0'0,0'0] MAX to 388'4845 2024-09-11T08:58:13.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[41114]: 11.9 starting backfill to osd.1 from (0'0,0'0] MAX to 388'4683 2024-09-11T08:58:13.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[41114]: 11.3 starting backfill to osd.2 from (0'0,0'0] MAX to 388'4823 2024-09-11T08:58:13.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[41114]: 11.5 starting backfill to osd.3 from (0'0,0'0] MAX to 388'4806 2024-09-11T08:58:13.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[41114]: 11.4 starting backfill to osd.1 from (0'0,0'0] MAX to 388'4796 2024-09-11T08:58:13.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:13 smithi092 ceph-mon[41114]: Health check failed: Degraded data redundancy: 10/222555 objects degraded (0.004%), 1 pg degraded (PG_DEGRADED) 2024-09-11T08:58:13.608 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:13.639 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:13.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:13.640 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:13.733 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:58:13.733 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:58:13.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:13.766 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:13.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:13.768 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:13.868 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:13.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:13.911 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:13.911 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:14.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:14.061 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:14.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:14.062 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:14.163 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:14.190 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:14.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:14.191 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:14.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:14.335 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:14.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:14.336 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:14.340 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 45 16 77871 77855 108.11 44.3125 0.0113569 0.00924352 2024-09-11T08:58:14.370 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:14 smithi145 ceph-mon[33384]: pgmap v894: 168 pgs: 8 remapped+peering, 1 active+recovery_wait+degraded, 1 active+recovering, 158 active+clean; 4.7 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 2.5 KiB/s rd, 49 MiB/s wr, 779 op/s; 10/229794 objects degraded (0.004%); 1/229794 objects misplaced (0.000%); 62 KiB/s, 11 objects/s recovering 2024-09-11T08:58:14.370 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:14 smithi145 ceph-mon[33384]: osdmap e392: 8 total, 8 up, 7 in 2024-09-11T08:58:14.411 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-11T08:58:14.436 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:14.477 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:14.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:14.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:14.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:14 smithi092 ceph-mon[29647]: pgmap v894: 168 pgs: 8 remapped+peering, 1 active+recovery_wait+degraded, 1 active+recovering, 158 active+clean; 4.7 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 2.5 KiB/s rd, 49 MiB/s wr, 779 op/s; 10/229794 objects degraded (0.004%); 1/229794 objects misplaced (0.000%); 62 KiB/s, 11 objects/s recovering 2024-09-11T08:58:14.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:14 smithi092 ceph-mon[29647]: osdmap e392: 8 total, 8 up, 7 in 2024-09-11T08:58:14.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:14 smithi092 ceph-mon[41114]: pgmap v894: 168 pgs: 8 remapped+peering, 1 active+recovery_wait+degraded, 1 active+recovering, 158 active+clean; 4.7 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 2.5 KiB/s rd, 49 MiB/s wr, 779 op/s; 10/229794 objects degraded (0.004%); 1/229794 objects misplaced (0.000%); 62 KiB/s, 11 objects/s recovering 2024-09-11T08:58:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:14 smithi092 ceph-mon[41114]: osdmap e392: 8 total, 8 up, 7 in 2024-09-11T08:58:14.579 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:14.610 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:14.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:14.611 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:14.711 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:14.743 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:14.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:14.744 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:14.760 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:58:14.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:14.879 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:14.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:14.880 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:14.980 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:15.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:15.023 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:15.023 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:15.124 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:15.154 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:15.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:15.156 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:15.256 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:15.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:15.302 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:15.302 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:15.340 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 46 16 78699 78683 106.885 51.75 0.00439164 0.00934104 2024-09-11T08:58:15.404 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:15.432 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:15.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:15.434 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:15.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:15.585 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:15.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:15.586 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:15.687 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:15.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:15.710 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:15.711 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:15.812 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:15.842 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:15.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:15.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:15.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:15.980 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:15.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:15.981 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:16.082 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:16.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:16.123 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:16.123 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:16.224 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:16.254 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:16.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:16.255 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:16.343 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 47 16 79510 79494 105.682 50.6875 0.0153833 0.00945666 2024-09-11T08:58:16.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:16.359 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:16 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:58:16.359 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:16 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:58:16.359 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:16 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:58:16.359 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:16 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:58:16.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:16 smithi145 ceph-mon[33384]: osdmap e393: 8 total, 8 up, 7 in 2024-09-11T08:58:16.360 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:16 smithi145 ceph-mon[33384]: pgmap v897: 168 pgs: 8 remapped+peering, 1 active+recovery_wait+degraded, 1 active+recovering, 158 active+clean; 4.7 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 3.5 KiB/s rd, 68 MiB/s wr, 1.08k op/s; 10/229794 objects degraded (0.004%); 1/229794 objects misplaced (0.000%); 87 KiB/s, 15 objects/s recovering 2024-09-11T08:58:16.401 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:16.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:16.402 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:16.502 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:16.536 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:16.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:16.538 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:16.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:16 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:58:16.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:16 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:58:16.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:16 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:58:16.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:16 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:58:16.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:16 smithi092 ceph-mon[29647]: osdmap e393: 8 total, 8 up, 7 in 2024-09-11T08:58:16.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:16 smithi092 ceph-mon[29647]: pgmap v897: 168 pgs: 8 remapped+peering, 1 active+recovery_wait+degraded, 1 active+recovering, 158 active+clean; 4.7 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 3.5 KiB/s rd, 68 MiB/s wr, 1.08k op/s; 10/229794 objects degraded (0.004%); 1/229794 objects misplaced (0.000%); 87 KiB/s, 15 objects/s recovering 2024-09-11T08:58:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:16 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:58:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:16 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:58:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:16 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:58:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:16 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:58:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:16 smithi092 ceph-mon[41114]: osdmap e393: 8 total, 8 up, 7 in 2024-09-11T08:58:16.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:16 smithi092 ceph-mon[41114]: pgmap v897: 168 pgs: 8 remapped+peering, 1 active+recovery_wait+degraded, 1 active+recovering, 158 active+clean; 4.7 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 3.5 KiB/s rd, 68 MiB/s wr, 1.08k op/s; 10/229794 objects degraded (0.004%); 1/229794 objects misplaced (0.000%); 87 KiB/s, 15 objects/s recovering 2024-09-11T08:58:16.638 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:16.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:16.670 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:16.670 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:16.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:16.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:16.814 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:16.814 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:16.824 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:58:16.824 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:58:16.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:16.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:16.957 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:17.058 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:17.087 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:17.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:17.089 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:17.190 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:17.226 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:17.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:17.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:17.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:17.344 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 48 16 80409 80393 104.649 56.1875 0.0123243 0.00955062 2024-09-11T08:58:17.408 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:17.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:17.410 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:17.511 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:17.516 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:58:17 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:58:17.546 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:17.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:17.547 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:17.561 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-11T08:58:17.624 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:58:17.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:17.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:17.674 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:17.674 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:17.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:17.814 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:17 smithi092 ceph-mon[29647]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 10/229794 objects degraded (0.004%), 1 pg degraded) 2024-09-11T08:58:17.814 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:17 smithi092 ceph-mon[41114]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 10/229794 objects degraded (0.004%), 1 pg degraded) 2024-09-11T08:58:17.819 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:17.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:17.820 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:17.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:17 smithi145 ceph-mon[33384]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 10/229794 objects degraded (0.004%), 1 pg degraded) 2024-09-11T08:58:17.921 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:17.948 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:58:17.960 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:17.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:17.961 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:18.062 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:18.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:18.091 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:18.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:18.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:18.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:18.237 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:18.238 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:18.338 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:18.345 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 49 16 81252 81236 103.588 52.6875 0.0124261 0.00964799 2024-09-11T08:58:18.368 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:18.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:18.369 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:18.386 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:58:18.386 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:58:18.422 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:58:18.423 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:58:18.459 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:58:18.459 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:58:18.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:18.496 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:58:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:58:18.512 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:18.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:18.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:18.528 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:58:18.529 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:58:18.557 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:58:18.557 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:58:18.589 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:58:18.590 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:58:18.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:18.622 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:58:18.622 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:58:18.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:18.649 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:18.649 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:18.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:18 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:18.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:18 smithi092 ceph-mon[29647]: pgmap v898: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 4.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 2.5 KiB/s rd, 60 MiB/s wr, 955 op/s; 36444/239727 objects misplaced (15.202%); 34 MiB/s, 552 objects/s recovering 2024-09-11T08:58:18.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:18 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4209148313' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:58:18.745 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:18 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:18.746 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:18 smithi092 ceph-mon[41114]: pgmap v898: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 4.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 2.5 KiB/s rd, 60 MiB/s wr, 955 op/s; 36444/239727 objects misplaced (15.202%); 34 MiB/s, 552 objects/s recovering 2024-09-11T08:58:18.746 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:18 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4209148313' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:58:18.750 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:18.777 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:18.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:18.778 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:18.879 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:18.910 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:18 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:18.910 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:18 smithi145 ceph-mon[33384]: pgmap v898: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 4.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 2.5 KiB/s rd, 60 MiB/s wr, 955 op/s; 36444/239727 objects misplaced (15.202%); 34 MiB/s, 552 objects/s recovering 2024-09-11T08:58:18.910 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:18 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4209148313' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:58:18.918 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:18.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:18.920 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:19.020 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:19.046 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:19.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:19.047 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:19.148 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:19.187 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:19.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:19.188 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:19.289 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:19.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:19.327 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:19.328 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:19.345 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 50 16 82164 82148 102.657 57 0.0456159 0.00973583 2024-09-11T08:58:19.428 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:19.456 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:19.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:19.457 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:19.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:19.604 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:19.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:19.605 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:19.706 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:19.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:19.747 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:19.747 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:19.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:19.849 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:58:19.849 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T08:58:19.849 INFO:teuthology.orchestra.run.smithi145.stdout: "complaint_time": 30, 2024-09-11T08:58:19.849 INFO:teuthology.orchestra.run.smithi145.stdout: "num_blocked_ops": 0 2024-09-11T08:58:19.850 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:58:19.891 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:19.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:19.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:19.993 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:20.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:20.041 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:20.042 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:20.142 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:20.174 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:20.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:20.175 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:20.184 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:20 smithi092 ceph-mon[29647]: osdmap e394: 8 total, 8 up, 7 in 2024-09-11T08:58:20.184 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:20 smithi092 ceph-mon[29647]: pgmap v900: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 4.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 34 MiB/s wr, 552 op/s; 36444/239727 objects misplaced (15.202%); 34 MiB/s, 542 objects/s recovering 2024-09-11T08:58:20.184 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:20 smithi092 ceph-mon[41114]: osdmap e394: 8 total, 8 up, 7 in 2024-09-11T08:58:20.184 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:20 smithi092 ceph-mon[41114]: pgmap v900: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 4.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 34 MiB/s wr, 552 op/s; 36444/239727 objects misplaced (15.202%); 34 MiB/s, 542 objects/s recovering 2024-09-11T08:58:20.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:20.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:20.327 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:20.345 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 51 16 83093 83077 101.782 58.0625 0.015297 0.00981917 2024-09-11T08:58:20.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:20 smithi145 ceph-mon[33384]: osdmap e394: 8 total, 8 up, 7 in 2024-09-11T08:58:20.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:20 smithi145 ceph-mon[33384]: pgmap v900: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 4.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 34 MiB/s wr, 552 op/s; 36444/239727 objects misplaced (15.202%); 34 MiB/s, 542 objects/s recovering 2024-09-11T08:58:20.428 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:20.456 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:20.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:20.457 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:20.558 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:20.607 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-11T08:58:20.612 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:20.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:20.614 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:20.714 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:20.740 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:20.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:20.742 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:20.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:20.884 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:20.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:20.885 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:20.946 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:58:20.985 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:21.032 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:21.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:21.033 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:21.134 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:21.138 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:58:21.161 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:21.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:21.162 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:21.262 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:21.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:21.298 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:21.298 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:21.345 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 52 16 83912 83896 100.809 51.1875 0.0100735 0.00991408 2024-09-11T08:58:21.400 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:21.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:21 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2118660190' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:58:21.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:21 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:58:21.444 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:21.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:21.445 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:21.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:21 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2118660190' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:58:21.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:21 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:58:21.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:21 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2118660190' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:58:21.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:21 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:58:21.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:21.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:58:21.589 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:21.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:21.592 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:21.693 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:21.718 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:21.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:21.719 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:21.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:58:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:58:21] "GET /metrics HTTP/1.1" 200 225181 "" "Prometheus/2.43.0" 2024-09-11T08:58:21.820 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:21.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:21.862 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:21.963 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:21.992 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:21.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:21.993 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:22.094 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:22.146 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:22.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:22.148 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:22.249 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:22.278 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:22.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:22.280 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:22.317 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 7, 0] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:58:22.318 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:58:22.318 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-09-11T08:58:22.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd in 6 2024-09-11T08:58:22.345 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 53 16 84801 84785 99.9557 55.5625 0.0106392 0.00999957 2024-09-11T08:58:22.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:22.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:22 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:58:22.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:22 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:58:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:22 smithi145 ceph-mon[33384]: osdmap e395: 8 total, 8 up, 7 in 2024-09-11T08:58:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:22 smithi145 ceph-mon[33384]: pgmap v902: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.6 KiB/s rd, 52 MiB/s wr, 839 op/s; 36120/245553 objects misplaced (14.710%); 46 MiB/s, 735 objects/s recovering 2024-09-11T08:58:22.426 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:22.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:22.427 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:22.528 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:22.553 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:22.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:22.554 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:22 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:58:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:22 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:58:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:22 smithi092 ceph-mon[29647]: osdmap e395: 8 total, 8 up, 7 in 2024-09-11T08:58:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:22 smithi092 ceph-mon[29647]: pgmap v902: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.6 KiB/s rd, 52 MiB/s wr, 839 op/s; 36120/245553 objects misplaced (14.710%); 46 MiB/s, 735 objects/s recovering 2024-09-11T08:58:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:22 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:58:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:22 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:58:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:22 smithi092 ceph-mon[41114]: osdmap e395: 8 total, 8 up, 7 in 2024-09-11T08:58:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:22 smithi092 ceph-mon[41114]: pgmap v902: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.6 KiB/s rd, 52 MiB/s wr, 839 op/s; 36120/245553 objects misplaced (14.710%); 46 MiB/s, 735 objects/s recovering 2024-09-11T08:58:22.655 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:22.684 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:22.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:22.686 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:22.727 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:58:22.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:22.807 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:58:22.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:22.850 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:22.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:22.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:23.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:23.004 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:23.019 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:58:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:58:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:58:23.105 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:23.135 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:23.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:23.136 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:23.237 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:23.264 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:23.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:23.265 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:23.347 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 54 16 85625 85609 99.0546 51.5 0.0208604 0.0100897 2024-09-11T08:58:23.365 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:23.410 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:23.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:23.411 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:23.512 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:23.549 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:23.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:23.550 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:23.623 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:58:23.623 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T08:58:23.623 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T08:58:23.623 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:58:23.623 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.623 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120742 9.16 9:6eef19e6:::benchmark_data_smithi092_145083_object120741:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:58:23.624 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.505791+0000", 2024-09-11T08:58:23.624 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 478.11437574799999, 2024-09-11T08:58:23.624 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0374483940000001, 2024-09-11T08:58:23.624 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.624 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:58:23.624 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.624 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:23.624 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:23.624 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120742 2024-09-11T08:58:23.624 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.624 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.624 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.624 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.625 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505791+0000", 2024-09-11T08:58:23.625 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.625 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.625 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.625 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.625 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505791+0000", 2024-09-11T08:58:23.625 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.625 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.625 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.625 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.625 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505788+0000", 2024-09-11T08:58:23.625 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999976 2024-09-11T08:58:23.625 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.626 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.626 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.626 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505833+0000", 2024-09-11T08:58:23.626 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4573000000000002e-05 2024-09-11T08:58:23.626 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.626 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.626 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.626 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505846+0000", 2024-09-11T08:58:23.626 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3033e-05 2024-09-11T08:58:23.626 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.626 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.626 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.626 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505851+0000", 2024-09-11T08:58:23.627 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.2719999999999997e-06 2024-09-11T08:58:23.627 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.627 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.627 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.627 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642292+0000", 2024-09-11T08:58:23.627 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.136440541 2024-09-11T08:58:23.627 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.627 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.627 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.627 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642298+0000", 2024-09-11T08:58:23.627 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5309999999999998e-06 2024-09-11T08:58:23.627 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.627 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.627 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.627 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543169+0000", 2024-09-11T08:58:23.628 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90087110199999998 2024-09-11T08:58:23.628 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.628 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.628 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.628 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543239+0000", 2024-09-11T08:58:23.628 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9496e-05 2024-09-11T08:58:23.628 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.628 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.628 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.628 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.628 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.628 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120746 9.6 9:633191e9:::benchmark_data_smithi092_145083_object120745:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:58:23.628 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.505860+0000", 2024-09-11T08:58:23.629 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 478.11430610500003, 2024-09-11T08:58:23.629 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0273401719999999, 2024-09-11T08:58:23.629 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.629 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:58:23.629 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.629 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:23.629 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:23.629 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120746 2024-09-11T08:58:23.629 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.629 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.629 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.629 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.629 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505860+0000", 2024-09-11T08:58:23.630 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.630 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.630 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.630 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.630 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505860+0000", 2024-09-11T08:58:23.630 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.630 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.630 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.630 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.630 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505860+0000", 2024-09-11T08:58:23.630 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:23.630 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.630 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.630 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.631 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505904+0000", 2024-09-11T08:58:23.631 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4440999999999997e-05 2024-09-11T08:58:23.631 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.631 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.631 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.631 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505910+0000", 2024-09-11T08:58:23.631 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1469999999999998e-06 2024-09-11T08:58:23.631 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.631 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.631 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.631 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505914+0000", 2024-09-11T08:58:23.631 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4620000000000001e-06 2024-09-11T08:58:23.631 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.632 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.632 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.632 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.641584+0000", 2024-09-11T08:58:23.632 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.13567022500000001 2024-09-11T08:58:23.632 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.632 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.632 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.632 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.641609+0000", 2024-09-11T08:58:23.632 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4556999999999999e-05 2024-09-11T08:58:23.632 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.632 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.632 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.633 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.533129+0000", 2024-09-11T08:58:23.633 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89152070299999997 2024-09-11T08:58:23.633 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.633 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.633 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.633 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.533200+0000", 2024-09-11T08:58:23.633 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1083999999999999e-05 2024-09-11T08:58:23.633 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.633 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.633 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.633 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.633 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.633 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120752 9.16 9:6c917c0c:::benchmark_data_smithi092_145083_object120751:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:58:23.633 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.505918+0000", 2024-09-11T08:58:23.634 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 478.11424857600002, 2024-09-11T08:58:23.634 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.037401907, 2024-09-11T08:58:23.634 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.634 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:58:23.634 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.634 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:23.634 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:23.634 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120752 2024-09-11T08:58:23.634 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.634 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.634 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.634 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.635 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505918+0000", 2024-09-11T08:58:23.635 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.635 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.635 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.635 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.635 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505918+0000", 2024-09-11T08:58:23.635 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.635 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.635 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.635 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.635 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505917+0000", 2024-09-11T08:58:23.635 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:23.635 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.635 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.636 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.636 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505987+0000", 2024-09-11T08:58:23.636 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9468000000000006e-05 2024-09-11T08:58:23.636 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.636 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.636 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.636 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505994+0000", 2024-09-11T08:58:23.636 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2119999999999997e-06 2024-09-11T08:58:23.636 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.636 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.636 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.636 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.505998+0000", 2024-09-11T08:58:23.636 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6629999999999999e-06 2024-09-11T08:58:23.637 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.637 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.637 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.637 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642315+0000", 2024-09-11T08:58:23.637 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.136317302 2024-09-11T08:58:23.637 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.637 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.637 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.637 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642320+0000", 2024-09-11T08:58:23.637 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6290000000000001e-06 2024-09-11T08:58:23.637 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.637 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.637 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.638 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543263+0000", 2024-09-11T08:58:23.638 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90094333100000001 2024-09-11T08:58:23.638 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.638 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.638 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.638 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543320+0000", 2024-09-11T08:58:23.638 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6672999999999998e-05 2024-09-11T08:58:23.638 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.638 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.638 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.638 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.638 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.638 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120762 9.16 9:6d28e92c:::benchmark_data_smithi092_145083_object120761:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:58:23.639 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.506002+0000", 2024-09-11T08:58:23.639 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 478.11416437399998, 2024-09-11T08:58:23.639 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.037390528, 2024-09-11T08:58:23.639 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.639 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:58:23.639 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.639 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:23.639 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:23.639 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120762 2024-09-11T08:58:23.639 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.639 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.639 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.640 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.640 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506002+0000", 2024-09-11T08:58:23.640 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.640 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.640 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.640 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.640 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506002+0000", 2024-09-11T08:58:23.640 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.640 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.640 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.640 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.640 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506002+0000", 2024-09-11T08:58:23.640 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:23.640 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.641 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.641 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.641 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506056+0000", 2024-09-11T08:58:23.641 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4002999999999998e-05 2024-09-11T08:58:23.641 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.641 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.641 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.641 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506062+0000", 2024-09-11T08:58:23.641 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4069999999999998e-06 2024-09-11T08:58:23.641 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.641 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.641 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.641 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506065+0000", 2024-09-11T08:58:23.641 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4709999999999999e-06 2024-09-11T08:58:23.642 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.642 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.642 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.642 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642336+0000", 2024-09-11T08:58:23.642 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.136270262 2024-09-11T08:58:23.642 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.642 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.642 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.642 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642339+0000", 2024-09-11T08:58:23.642 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5209999999999998e-06 2024-09-11T08:58:23.642 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.642 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.642 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.643 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543343+0000", 2024-09-11T08:58:23.643 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90100398000000004 2024-09-11T08:58:23.643 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.643 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.643 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.643 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543392+0000", 2024-09-11T08:58:23.643 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9227999999999997e-05 2024-09-11T08:58:23.643 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.643 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.643 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.643 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.643 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.643 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120768 9.17 9:e9fb5cfc:::benchmark_data_smithi092_145083_object120767:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:58:23.644 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.506070+0000", 2024-09-11T08:58:23.644 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 478.11409671799998, 2024-09-11T08:58:23.644 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.039930231, 2024-09-11T08:58:23.644 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.644 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:23.644 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.644 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:23.644 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:23.644 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120768 2024-09-11T08:58:23.644 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.644 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.644 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.644 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.644 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506070+0000", 2024-09-11T08:58:23.645 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.645 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.645 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.645 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.645 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506070+0000", 2024-09-11T08:58:23.645 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.645 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.645 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.645 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.645 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506069+0000", 2024-09-11T08:58:23.645 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:23.645 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.645 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.645 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.646 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506110+0000", 2024-09-11T08:58:23.646 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0395999999999997e-05 2024-09-11T08:58:23.646 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.646 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.646 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.646 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506116+0000", 2024-09-11T08:58:23.646 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.19e-06 2024-09-11T08:58:23.646 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.646 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.646 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.646 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506120+0000", 2024-09-11T08:58:23.646 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9620000000000004e-06 2024-09-11T08:58:23.646 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.647 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.647 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.647 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.653509+0000", 2024-09-11T08:58:23.647 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.147388988 2024-09-11T08:58:23.647 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.647 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.647 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.647 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.653516+0000", 2024-09-11T08:58:23.647 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9909999999999996e-06 2024-09-11T08:58:23.647 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.647 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.647 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.647 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.541363+0000", 2024-09-11T08:58:23.647 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.88784689000000006 2024-09-11T08:58:23.647 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.648 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.648 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:23.648 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.541484+0000", 2024-09-11T08:58:23.648 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012157100000000001 2024-09-11T08:58:23.648 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.648 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.648 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,7", 2024-09-11T08:58:23.648 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.541711+0000", 2024-09-11T08:58:23.648 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000226314 2024-09-11T08:58:23.648 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.648 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.648 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:23.648 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543128+0000", 2024-09-11T08:58:23.648 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001417115 2024-09-11T08:58:23.649 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.649 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.649 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.649 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.545051+0000", 2024-09-11T08:58:23.649 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0019233410000000001 2024-09-11T08:58:23.649 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.649 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.649 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.649 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.545904+0000", 2024-09-11T08:58:23.649 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00085261600000000005 2024-09-11T08:58:23.649 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.649 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.649 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:23.649 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.545953+0000", 2024-09-11T08:58:23.650 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9363000000000002e-05 2024-09-11T08:58:23.650 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.650 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.650 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.650 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.546000+0000", 2024-09-11T08:58:23.650 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6848999999999998e-05 2024-09-11T08:58:23.650 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.650 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.650 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.650 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.650 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.650 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120769 9.16 9:6ce1939e:::benchmark_data_smithi092_145083_object120768:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:58:23.650 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.506125+0000", 2024-09-11T08:58:23.651 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 478.11404142399999, 2024-09-11T08:58:23.651 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0425137600000001, 2024-09-11T08:58:23.651 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.651 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:23.651 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.651 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:23.651 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:23.652 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:23.652 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120769 2024-09-11T08:58:23.653 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.653 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.653 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.653 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.653 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506125+0000", 2024-09-11T08:58:23.653 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.653 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.653 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.653 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.653 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506125+0000", 2024-09-11T08:58:23.653 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.653 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.653 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.654 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.654 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506124+0000", 2024-09-11T08:58:23.654 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:23.654 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.654 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.654 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.654 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506160+0000", 2024-09-11T08:58:23.654 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5710000000000002e-05 2024-09-11T08:58:23.654 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.654 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.654 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.654 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506167+0000", 2024-09-11T08:58:23.654 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3389999999999998e-06 2024-09-11T08:58:23.654 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.655 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.655 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.655 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506170+0000", 2024-09-11T08:58:23.655 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7139999999999999e-06 2024-09-11T08:58:23.655 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.655 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.655 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.655 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642352+0000", 2024-09-11T08:58:23.655 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.13618196299999999 2024-09-11T08:58:23.655 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.655 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.655 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.655 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.642356+0000", 2024-09-11T08:58:23.656 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4209999999999999e-06 2024-09-11T08:58:23.656 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.656 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.656 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.656 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543476+0000", 2024-09-11T08:58:23.656 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90112084699999995 2024-09-11T08:58:23.656 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.656 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.656 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:23.656 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543604+0000", 2024-09-11T08:58:23.656 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000127396 2024-09-11T08:58:23.656 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.656 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.657 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:58:23.657 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.543929+0000", 2024-09-11T08:58:23.657 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00032481300000000002 2024-09-11T08:58:23.657 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.657 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.657 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:23.657 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.546522+0000", 2024-09-11T08:58:23.657 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0025933559999999998 2024-09-11T08:58:23.657 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.657 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.657 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.657 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547191+0000", 2024-09-11T08:58:23.657 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00066896499999999999 2024-09-11T08:58:23.657 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.658 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.658 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.658 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.548558+0000", 2024-09-11T08:58:23.658 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001367385 2024-09-11T08:58:23.658 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.658 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.658 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:23.658 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.548605+0000", 2024-09-11T08:58:23.658 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6139e-05 2024-09-11T08:58:23.658 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.658 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.658 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.658 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.548639+0000", 2024-09-11T08:58:23.658 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4088999999999998e-05 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120771 9.5 9:a002ebf5:::benchmark_data_smithi092_145083_object120770:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.506174+0000", 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 478.11399209500001, 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0255135049999999, 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120771 2024-09-11T08:58:23.659 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.660 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.660 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.660 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.660 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506174+0000", 2024-09-11T08:58:23.660 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.660 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.660 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.660 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.660 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506174+0000", 2024-09-11T08:58:23.660 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.660 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.660 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.660 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.660 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506174+0000", 2024-09-11T08:58:23.660 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:23.661 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.661 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.661 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.661 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506204+0000", 2024-09-11T08:58:23.661 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0658999999999999e-05 2024-09-11T08:58:23.661 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.661 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.661 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.661 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506211+0000", 2024-09-11T08:58:23.661 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.127e-06 2024-09-11T08:58:23.661 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.661 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.661 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.662 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506214+0000", 2024-09-11T08:58:23.662 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6820000000000001e-06 2024-09-11T08:58:23.662 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.662 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.662 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.662 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.653029+0000", 2024-09-11T08:58:23.662 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.146814588 2024-09-11T08:58:23.662 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.662 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.662 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.662 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.653040+0000", 2024-09-11T08:58:23.662 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0954e-05 2024-09-11T08:58:23.662 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.662 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.663 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.663 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.513562+0000", 2024-09-11T08:58:23.663 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.86052253899999998 2024-09-11T08:58:23.663 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.663 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.663 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:23.663 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.513719+0000", 2024-09-11T08:58:23.663 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015641799999999999 2024-09-11T08:58:23.663 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.663 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.663 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,7", 2024-09-11T08:58:23.663 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.513988+0000", 2024-09-11T08:58:23.663 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00026880900000000001 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.517404+0000", 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003416681 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.521873+0000", 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0044682899999999998 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.531603+0000", 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0097300000000000008 2024-09-11T08:58:23.664 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.665 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.665 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:23.665 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.531650+0000", 2024-09-11T08:58:23.665 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7855999999999997e-05 2024-09-11T08:58:23.665 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.665 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.665 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.665 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.531688+0000", 2024-09-11T08:58:23.665 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7243999999999998e-05 2024-09-11T08:58:23.665 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.665 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.665 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.665 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.665 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.666 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120772 9.5 9:a5113f5f:::benchmark_data_smithi092_145083_object120771:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:58:23.666 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.506219+0000", 2024-09-11T08:58:23.666 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 478.11394694900002, 2024-09-11T08:58:23.666 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.032593828, 2024-09-11T08:58:23.666 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.666 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:23.666 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.666 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:23.666 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:23.666 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120772 2024-09-11T08:58:23.666 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.666 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.666 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.666 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.667 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506219+0000", 2024-09-11T08:58:23.667 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.667 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.667 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.667 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.667 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506219+0000", 2024-09-11T08:58:23.667 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.667 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.667 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.667 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.667 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506219+0000", 2024-09-11T08:58:23.667 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:23.667 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.667 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.667 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.668 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506254+0000", 2024-09-11T08:58:23.668 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4820000000000002e-05 2024-09-11T08:58:23.668 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.668 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.668 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.668 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506260+0000", 2024-09-11T08:58:23.668 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3090000000000001e-06 2024-09-11T08:58:23.668 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.668 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.668 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.668 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.506264+0000", 2024-09-11T08:58:23.668 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5319999999999998e-06 2024-09-11T08:58:23.668 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.668 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.669 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.669 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.657026+0000", 2024-09-11T08:58:23.669 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.15076241700000001 2024-09-11T08:58:23.669 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.669 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.669 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.669 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.657040+0000", 2024-09-11T08:58:23.669 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3667e-05 2024-09-11T08:58:23.669 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.669 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.669 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.669 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.515201+0000", 2024-09-11T08:58:23.669 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.85816157100000001 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.515331+0000", 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000130211 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,7", 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.515639+0000", 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00030726899999999999 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.517591+0000", 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001952147 2024-09-11T08:58:23.670 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.671 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.671 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.671 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.526102+0000", 2024-09-11T08:58:23.671 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0085114309999999999 2024-09-11T08:58:23.671 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.671 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.671 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.671 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538708+0000", 2024-09-11T08:58:23.671 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.01260591 2024-09-11T08:58:23.671 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.671 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.671 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:23.671 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538778+0000", 2024-09-11T08:58:23.671 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9960000000000001e-05 2024-09-11T08:58:23.671 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.672 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.672 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.672 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538813+0000", 2024-09-11T08:58:23.672 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4953000000000002e-05 2024-09-11T08:58:23.672 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.672 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.672 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.672 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.672 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.672 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272038 9.12 9:4c1395bf:::benchmark_data_smithi092_145083_object128640:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:58:23.672 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.599637+0000", 2024-09-11T08:58:23.672 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 360.02052907500001, 2024-09-11T08:58:23.672 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0332527600000001, 2024-09-11T08:58:23.672 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.673 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:23.673 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.673 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:23.673 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:23.673 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272038 2024-09-11T08:58:23.673 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.673 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.673 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.673 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.673 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599637+0000", 2024-09-11T08:58:23.673 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.673 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.673 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.673 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.674 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599637+0000", 2024-09-11T08:58:23.674 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.674 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.674 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.674 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.674 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599635+0000", 2024-09-11T08:58:23.674 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:58:23.674 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.674 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.674 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.674 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599643+0000", 2024-09-11T08:58:23.674 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.1759999999999993e-06 2024-09-11T08:58:23.674 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.674 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.674 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.675 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599660+0000", 2024-09-11T08:58:23.675 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7118000000000001e-05 2024-09-11T08:58:23.675 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.675 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.675 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.675 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599667+0000", 2024-09-11T08:58:23.675 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8260000000000004e-06 2024-09-11T08:58:23.675 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.675 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.675 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.675 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599717+0000", 2024-09-11T08:58:23.675 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0219000000000001e-05 2024-09-11T08:58:23.675 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.675 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.676 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op must wait for map", 2024-09-11T08:58:23.676 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599724+0000", 2024-09-11T08:58:23.676 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4130000000000003e-06 2024-09-11T08:58:23.676 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.676 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.676 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.676 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.611082+0000", 2024-09-11T08:58:23.676 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.01135824 2024-09-11T08:58:23.676 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.676 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.676 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.676 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.611093+0000", 2024-09-11T08:58:23.677 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0729e-05 2024-09-11T08:58:23.677 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.677 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.677 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.677 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.627111+0000", 2024-09-11T08:58:23.677 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0160185660000001 2024-09-11T08:58:23.677 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.677 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.677 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:23.677 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.627211+0000", 2024-09-11T08:58:23.677 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.9397999999999999e-05 2024-09-11T08:58:23.677 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.677 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.678 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:58:23.678 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.627282+0000", 2024-09-11T08:58:23.678 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1217000000000006e-05 2024-09-11T08:58:23.678 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.678 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.678 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:23.678 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.631283+0000", 2024-09-11T08:58:23.678 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0040011389999999999 2024-09-11T08:58:23.678 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.678 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.678 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.678 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.632714+0000", 2024-09-11T08:58:23.678 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0014306360000000001 2024-09-11T08:58:23.678 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.679 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.679 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.679 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.632844+0000", 2024-09-11T08:58:23.679 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013048000000000001 2024-09-11T08:58:23.679 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.679 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.679 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:23.679 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.632869+0000", 2024-09-11T08:58:23.679 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4278999999999999e-05 2024-09-11T08:58:23.679 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.679 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.679 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.679 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.632890+0000", 2024-09-11T08:58:23.680 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.1382e-05 2024-09-11T08:58:23.680 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.680 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.680 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.680 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.680 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.680 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272048 9.12 9:4dde8efd:::benchmark_data_smithi092_145083_object128650:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:58:23.680 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.614452+0000", 2024-09-11T08:58:23.680 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 360.00571403499998, 2024-09-11T08:58:23.680 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.031362157, 2024-09-11T08:58:23.680 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.680 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:23.681 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.681 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:23.681 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:23.681 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272048 2024-09-11T08:58:23.681 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.681 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.681 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.681 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.681 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614452+0000", 2024-09-11T08:58:23.681 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.681 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.681 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.681 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.682 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614452+0000", 2024-09-11T08:58:23.682 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.682 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.682 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.682 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.682 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614451+0000", 2024-09-11T08:58:23.682 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:23.682 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.682 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.682 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.682 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614458+0000", 2024-09-11T08:58:23.682 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4889999999999998e-06 2024-09-11T08:58:23.683 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.683 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.683 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.683 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614474+0000", 2024-09-11T08:58:23.683 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5764e-05 2024-09-11T08:58:23.683 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.683 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.683 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.683 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614481+0000", 2024-09-11T08:58:23.683 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.7859999999999999e-06 2024-09-11T08:58:23.683 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.683 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.683 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.684 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614573+0000", 2024-09-11T08:58:23.684 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.2504999999999996e-05 2024-09-11T08:58:23.684 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.684 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.684 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.684 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614581+0000", 2024-09-11T08:58:23.684 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7680000000000008e-06 2024-09-11T08:58:23.684 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.684 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.684 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.684 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.629158+0000", 2024-09-11T08:58:23.684 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.014577072 2024-09-11T08:58:23.684 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.685 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.685 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:23.685 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.629222+0000", 2024-09-11T08:58:23.685 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3751000000000007e-05 2024-09-11T08:58:23.685 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.685 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.685 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:58:23.685 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.629291+0000", 2024-09-11T08:58:23.685 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9078999999999994e-05 2024-09-11T08:58:23.685 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.685 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.685 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.686 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.634043+0000", 2024-09-11T08:58:23.686 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0047520009999999996 2024-09-11T08:58:23.686 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.686 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.686 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:23.686 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.634048+0000", 2024-09-11T08:58:23.686 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0679999999999996e-06 2024-09-11T08:58:23.686 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.686 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.686 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.686 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.645758+0000", 2024-09-11T08:58:23.686 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011710124000000001 2024-09-11T08:58:23.686 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.686 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.687 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:23.687 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.645789+0000", 2024-09-11T08:58:23.687 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1124000000000002e-05 2024-09-11T08:58:23.687 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.687 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.687 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.687 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.645814+0000", 2024-09-11T08:58:23.687 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5244000000000001e-05 2024-09-11T08:58:23.687 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.687 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.687 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.687 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.687 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.687 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272066 9.12 9:4968a467:::benchmark_data_smithi092_145083_object128668:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:58:23.688 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.629493+0000", 2024-09-11T08:58:23.688 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 359.99067317999999, 2024-09-11T08:58:23.688 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0189994769999999, 2024-09-11T08:58:23.688 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.688 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:23.688 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.688 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:23.688 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:23.688 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272066 2024-09-11T08:58:23.688 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.688 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.688 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.688 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.688 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629493+0000", 2024-09-11T08:58:23.689 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.689 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.689 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.689 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.689 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629493+0000", 2024-09-11T08:58:23.689 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.689 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.689 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.689 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.689 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629492+0000", 2024-09-11T08:58:23.689 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:23.689 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.689 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.689 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.690 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629499+0000", 2024-09-11T08:58:23.690 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4390000000000003e-06 2024-09-11T08:58:23.690 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.690 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.690 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.690 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629516+0000", 2024-09-11T08:58:23.690 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6617e-05 2024-09-11T08:58:23.690 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.690 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.690 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.690 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629522+0000", 2024-09-11T08:58:23.690 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2009999999999996e-06 2024-09-11T08:58:23.690 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.691 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.691 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.691 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629586+0000", 2024-09-11T08:58:23.691 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4300000000000004e-05 2024-09-11T08:58:23.691 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.691 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.691 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.691 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629594+0000", 2024-09-11T08:58:23.691 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.5959999999999998e-06 2024-09-11T08:58:23.691 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.691 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.691 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.691 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.630309+0000", 2024-09-11T08:58:23.691 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0007151240000001 2024-09-11T08:58:23.692 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.692 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.692 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:23.692 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.630368+0000", 2024-09-11T08:58:23.692 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9344e-05 2024-09-11T08:58:23.692 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.692 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.692 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:58:23.692 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.630435+0000", 2024-09-11T08:58:23.692 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6285999999999995e-05 2024-09-11T08:58:23.692 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.692 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.692 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:23.693 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.635154+0000", 2024-09-11T08:58:23.693 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0047192019999999996 2024-09-11T08:58:23.693 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.693 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.693 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.693 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.635174+0000", 2024-09-11T08:58:23.693 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0749000000000001e-05 2024-09-11T08:58:23.693 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.693 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.693 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.693 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648406+0000", 2024-09-11T08:58:23.693 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013231463000000001 2024-09-11T08:58:23.693 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.693 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.693 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:23.694 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648455+0000", 2024-09-11T08:58:23.695 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9174e-05 2024-09-11T08:58:23.695 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.695 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.695 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.695 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648493+0000", 2024-09-11T08:58:23.695 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7438999999999998e-05 2024-09-11T08:58:23.695 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.695 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.695 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.695 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.695 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.696 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272071 9.12 9:4c320382:::benchmark_data_smithi092_145083_object128673:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:58:23.696 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.632070+0000", 2024-09-11T08:58:23.696 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 359.98809590000002, 2024-09-11T08:58:23.696 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.016575234, 2024-09-11T08:58:23.696 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.696 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:23.696 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.696 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:23.696 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:23.696 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272071 2024-09-11T08:58:23.696 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.696 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.696 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.696 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.696 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632070+0000", 2024-09-11T08:58:23.697 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.697 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.697 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.697 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.697 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632070+0000", 2024-09-11T08:58:23.697 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.697 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.697 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.697 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.697 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632070+0000", 2024-09-11T08:58:23.697 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:58:23.697 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632074+0000", 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9110000000000001e-06 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632083+0000", 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.6370000000000005e-06 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632089+0000", 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6810000000000003e-06 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.698 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632285+0000", 2024-09-11T08:58:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00019620999999999999 2024-09-11T08:58:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632292+0000", 2024-09-11T08:58:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5289999999999997e-06 2024-09-11T08:58:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.631306+0000", 2024-09-11T08:58:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99901447799999998 2024-09-11T08:58:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.699 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.631400+0000", 2024-09-11T08:58:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.4510000000000001e-05 2024-09-11T08:58:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:58:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.631493+0000", 2024-09-11T08:58:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.2555999999999999e-05 2024-09-11T08:58:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.635158+0000", 2024-09-11T08:58:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0036650089999999999 2024-09-11T08:58:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.700 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.643385+0000", 2024-09-11T08:58:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0082264529999999999 2024-09-11T08:58:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648566+0000", 2024-09-11T08:58:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0051818469999999998 2024-09-11T08:58:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648610+0000", 2024-09-11T08:58:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3442999999999997e-05 2024-09-11T08:58:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.701 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648646+0000", 2024-09-11T08:58:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5800000000000003e-05 2024-09-11T08:58:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272083 9.12 9:4e00b3e7:::benchmark_data_smithi092_145083_object128685:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:58:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.640138+0000", 2024-09-11T08:58:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 359.98002782399999, 2024-09-11T08:58:23.702 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.008620208, 2024-09-11T08:58:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272083 2024-09-11T08:58:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.703 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640138+0000", 2024-09-11T08:58:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640138+0000", 2024-09-11T08:58:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640137+0000", 2024-09-11T08:58:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.704 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640144+0000", 2024-09-11T08:58:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4889999999999998e-06 2024-09-11T08:58:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640156+0000", 2024-09-11T08:58:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1718e-05 2024-09-11T08:58:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640162+0000", 2024-09-11T08:58:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8159999999999999e-06 2024-09-11T08:58:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.705 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640202+0000", 2024-09-11T08:58:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0370999999999997e-05 2024-09-11T08:58:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640210+0000", 2024-09-11T08:58:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4039999999999998e-06 2024-09-11T08:58:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.632939+0000", 2024-09-11T08:58:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99272926299999997 2024-09-11T08:58:23.706 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.633019+0000", 2024-09-11T08:58:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.9963000000000007e-05 2024-09-11T08:58:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T08:58:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.633114+0000", 2024-09-11T08:58:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.5613999999999996e-05 2024-09-11T08:58:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.636389+0000", 2024-09-11T08:58:23.707 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0032742420000000001 2024-09-11T08:58:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.643483+0000", 2024-09-11T08:58:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0070939260000000004 2024-09-11T08:58:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648683+0000", 2024-09-11T08:58:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0052004089999999996 2024-09-11T08:58:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648726+0000", 2024-09-11T08:58:23.708 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3457000000000001e-05 2024-09-11T08:58:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648759+0000", 2024-09-11T08:58:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2200999999999999e-05 2024-09-11T08:58:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:115609 10.3 10:c6226159:::benchmark_data_smithi092_173109_object115608:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T08:58:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:16.116557+0000", 2024-09-11T08:58:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 247.50360921000001, 2024-09-11T08:58:23.709 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0273496980000001, 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 115609 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116557+0000", 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.710 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116557+0000", 2024-09-11T08:58:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116557+0000", 2024-09-11T08:58:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116631+0000", 2024-09-11T08:58:23.711 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4128000000000006e-05 2024-09-11T08:58:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116640+0000", 2024-09-11T08:58:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.4280000000000002e-06 2024-09-11T08:58:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116644+0000", 2024-09-11T08:58:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3139999999999997e-06 2024-09-11T08:58:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116788+0000", 2024-09-11T08:58:23.712 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000143425 2024-09-11T08:58:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116794+0000", 2024-09-11T08:58:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.618e-06 2024-09-11T08:58:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.124868+0000", 2024-09-11T08:58:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0080731979999999 2024-09-11T08:58:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.713 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for missing object", 2024-09-11T08:58:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.124955+0000", 2024-09-11T08:58:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.7103999999999998e-05 2024-09-11T08:58:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.143857+0000", 2024-09-11T08:58:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.018902209999999999 2024-09-11T08:58:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.143907+0000", 2024-09-11T08:58:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9792e-05 2024-09-11T08:58:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.714 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:115621 10.3 10:ca57ac07:::benchmark_data_smithi092_173109_object115620:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T08:58:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:16.127920+0000", 2024-09-11T08:58:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 247.49224580699999, 2024-09-11T08:58:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.016159238, 2024-09-11T08:58:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:58:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:58:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 115621 2024-09-11T08:58:23.715 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.127920+0000", 2024-09-11T08:58:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.127920+0000", 2024-09-11T08:58:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.716 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.127919+0000", 2024-09-11T08:58:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.128018+0000", 2024-09-11T08:58:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.9053000000000002e-05 2024-09-11T08:58:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.128032+0000", 2024-09-11T08:58:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4107000000000001e-05 2024-09-11T08:58:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.717 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.128038+0000", 2024-09-11T08:58:23.718 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8030000000000002e-06 2024-09-11T08:58:23.718 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.128095+0000", 2024-09-11T08:58:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6344000000000001e-05 2024-09-11T08:58:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.720 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.128106+0000", 2024-09-11T08:58:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1985e-05 2024-09-11T08:58:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.124982+0000", 2024-09-11T08:58:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99687584600000001 2024-09-11T08:58:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.125098+0000", 2024-09-11T08:58:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000115944 2024-09-11T08:58:23.721 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,3", 2024-09-11T08:58:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.125523+0000", 2024-09-11T08:58:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00042452599999999999 2024-09-11T08:58:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.134514+0000", 2024-09-11T08:58:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0089912310000000006 2024-09-11T08:58:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.143952+0000", 2024-09-11T08:58:23.722 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0094376159999999994 2024-09-11T08:58:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.143992+0000", 2024-09-11T08:58:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9969000000000003e-05 2024-09-11T08:58:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.144020+0000", 2024-09-11T08:58:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.8540000000000001e-05 2024-09-11T08:58:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.723 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.144080+0000", 2024-09-11T08:58:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9534999999999998e-05 2024-09-11T08:58:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:115626 10.3 10:ceb6be78:::benchmark_data_smithi092_173109_object115625:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T08:58:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:16.133512+0000", 2024-09-11T08:58:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 247.48665436100001, 2024-09-11T08:58:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0108069260000001, 2024-09-11T08:58:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.724 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:58:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:58:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 115626 2024-09-11T08:58:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133512+0000", 2024-09-11T08:58:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.725 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133512+0000", 2024-09-11T08:58:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133510+0000", 2024-09-11T08:58:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133565+0000", 2024-09-11T08:58:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4809999999999999e-05 2024-09-11T08:58:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.726 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133582+0000", 2024-09-11T08:58:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.715e-05 2024-09-11T08:58:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133590+0000", 2024-09-11T08:58:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2679999999999997e-06 2024-09-11T08:58:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133653+0000", 2024-09-11T08:58:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3268999999999993e-05 2024-09-11T08:58:23.727 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133662+0000", 2024-09-11T08:58:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.9299999999999992e-06 2024-09-11T08:58:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.132033+0000", 2024-09-11T08:58:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99837082200000005 2024-09-11T08:58:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.728 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.132120+0000", 2024-09-11T08:58:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.7138999999999994e-05 2024-09-11T08:58:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,3", 2024-09-11T08:58:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.132222+0000", 2024-09-11T08:58:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010197800000000001 2024-09-11T08:58:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.137321+0000", 2024-09-11T08:58:23.729 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0050989479999999998 2024-09-11T08:58:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.144111+0000", 2024-09-11T08:58:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0067903490000000002 2024-09-11T08:58:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.144194+0000", 2024-09-11T08:58:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2897000000000003e-05 2024-09-11T08:58:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.730 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.144226+0000", 2024-09-11T08:58:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2466999999999999e-05 2024-09-11T08:58:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.144319+0000", 2024-09-11T08:58:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.2509999999999993e-05 2024-09-11T08:58:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:115640 10.3 10:c94ff23e:::benchmark_data_smithi092_173109_object115639:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T08:58:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:16.141138+0000", 2024-09-11T08:58:23.731 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 247.47902791800001, 2024-09-11T08:58:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0107861440000001, 2024-09-11T08:58:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:58:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:58:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 115640 2024-09-11T08:58:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.141138+0000", 2024-09-11T08:58:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.732 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.141138+0000", 2024-09-11T08:58:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.141137+0000", 2024-09-11T08:58:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.141259+0000", 2024-09-11T08:58:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000121923 2024-09-11T08:58:23.733 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.141267+0000", 2024-09-11T08:58:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.5939999999999994e-06 2024-09-11T08:58:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.141272+0000", 2024-09-11T08:58:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7770000000000002e-06 2024-09-11T08:58:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.152765+0000", 2024-09-11T08:58:23.734 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011493412 2024-09-11T08:58:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.152775+0000", 2024-09-11T08:58:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.7699999999999996e-06 2024-09-11T08:58:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.133348+0000", 2024-09-11T08:58:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.98057306799999999 2024-09-11T08:58:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.735 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.133429+0000", 2024-09-11T08:58:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.0222000000000005e-05 2024-09-11T08:58:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,3", 2024-09-11T08:58:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.133520+0000", 2024-09-11T08:58:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.1508999999999994e-05 2024-09-11T08:58:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.139818+0000", 2024-09-11T08:58:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0062979840000000004 2024-09-11T08:58:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.736 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.150267+0000", 2024-09-11T08:58:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.010448737 2024-09-11T08:58:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.151828+0000", 2024-09-11T08:58:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0015615220000000001 2024-09-11T08:58:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.151863+0000", 2024-09-11T08:58:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4477999999999998e-05 2024-09-11T08:58:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.737 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.151924+0000", 2024-09-11T08:58:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1764e-05 2024-09-11T08:58:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14448.0:2416 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 94136053445360 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T08:58:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.450967+0000", 2024-09-11T08:58:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 136.16919896100001, 2024-09-11T08:58:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0358244990000001, 2024-09-11T08:58:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.738 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T08:58:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14448", 2024-09-11T08:58:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3990203193", 2024-09-11T08:58:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 2416 2024-09-11T08:58:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450967+0000", 2024-09-11T08:58:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.739 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450967+0000", 2024-09-11T08:58:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450965+0000", 2024-09-11T08:58:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:58:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450973+0000", 2024-09-11T08:58:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.3540000000000002e-06 2024-09-11T08:58:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.740 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450985+0000", 2024-09-11T08:58:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2714999999999999e-05 2024-09-11T08:58:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450991+0000", 2024-09-11T08:58:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8429999999999998e-06 2024-09-11T08:58:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.451040+0000", 2024-09-11T08:58:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9215000000000002e-05 2024-09-11T08:58:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op must wait for map", 2024-09-11T08:58:23.741 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.451047+0000", 2024-09-11T08:58:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4139999999999999e-06 2024-09-11T08:58:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.462274+0000", 2024-09-11T08:58:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011227098 2024-09-11T08:58:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.462284+0000", 2024-09-11T08:58:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.9499999999999996e-06 2024-09-11T08:58:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.742 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.483120+0000", 2024-09-11T08:58:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0208363410000001 2024-09-11T08:58:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.483249+0000", 2024-09-11T08:58:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000128658 2024-09-11T08:58:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.486792+0000", 2024-09-11T08:58:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0035430489999999999 2024-09-11T08:58:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.743 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244899 10.a 10:55016aaa:::benchmark_data_smithi092_173109_object107940:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T08:58:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.461819+0000", 2024-09-11T08:58:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 136.158347628, 2024-09-11T08:58:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0650427090000001, 2024-09-11T08:58:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:58:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:58:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:58:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244899 2024-09-11T08:58:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.744 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461819+0000", 2024-09-11T08:58:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461819+0000", 2024-09-11T08:58:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461817+0000", 2024-09-11T08:58:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:58:23.745 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461824+0000", 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6669999999999999e-06 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461834+0000", 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.029e-05 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461840+0000", 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7830000000000004e-06 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.746 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.466710+0000", 2024-09-11T08:58:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.004870592 2024-09-11T08:58:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.466719+0000", 2024-09-11T08:58:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8759999999999994e-06 2024-09-11T08:58:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.524699+0000", 2024-09-11T08:58:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0579804180000001 2024-09-11T08:58:23.747 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.526861+0000", 2024-09-11T08:58:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0021618840000000002 2024-09-11T08:58:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244920 10.a 10:501159a3:::benchmark_data_smithi092_173109_object107961:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T08:58:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.484501+0000", 2024-09-11T08:58:23.748 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 136.135665622, 2024-09-11T08:58:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0561753199999999, 2024-09-11T08:58:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T08:58:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:58:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244920 2024-09-11T08:58:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484501+0000", 2024-09-11T08:58:23.749 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484501+0000", 2024-09-11T08:58:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484500+0000", 2024-09-11T08:58:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:58:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.750 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484505+0000", 2024-09-11T08:58:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8470000000000003e-06 2024-09-11T08:58:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484515+0000", 2024-09-11T08:58:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0786e-05 2024-09-11T08:58:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:23.751 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484521+0000", 2024-09-11T08:58:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3070000000000002e-06 2024-09-11T08:58:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484568+0000", 2024-09-11T08:58:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6848999999999998e-05 2024-09-11T08:58:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484576+0000", 2024-09-11T08:58:23.753 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2349999999999999e-06 2024-09-11T08:58:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.527274+0000", 2024-09-11T08:58:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0426980720000001 2024-09-11T08:58:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.527843+0000", 2024-09-11T08:58:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00056964299999999999 2024-09-11T08:58:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.754 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,6", 2024-09-11T08:58:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.528148+0000", 2024-09-11T08:58:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00030435400000000001 2024-09-11T08:58:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.537204+0000", 2024-09-11T08:58:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0090564110000000003 2024-09-11T08:58:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.540357+0000", 2024-09-11T08:58:23.755 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0031531599999999999 2024-09-11T08:58:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.540384+0000", 2024-09-11T08:58:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.6625e-05 2024-09-11T08:58:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.540636+0000", 2024-09-11T08:58:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00025166300000000001 2024-09-11T08:58:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:23.756 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.540676+0000", 2024-09-11T08:58:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0258e-05 2024-09-11T08:58:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:23.757 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:23.757 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:58:23.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:23.758 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:23.758 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:23.859 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:23.886 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:23.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:23.887 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:23.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:24.025 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:24.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:24.026 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:24.127 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:24.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:24.153 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:24.154 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:24.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:24.298 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:24.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:24.299 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:24.347 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 55 16 86541 86525 98.2946 57.25 0.0137918 0.0101685 2024-09-11T08:58:24.400 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:24.430 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:24.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:24.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:24.532 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:24.579 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:24.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:24.581 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:24.630 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:24 smithi145 ceph-mon[33384]: pgmap v903: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.1 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 60 MiB/s wr, 965 op/s; 34434/252954 objects misplaced (13.613%); 54 MiB/s, 864 objects/s recovering 2024-09-11T08:58:24.630 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:24 smithi145 ceph-mon[33384]: osdmap e396: 8 total, 8 up, 7 in 2024-09-11T08:58:24.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:58:24.666 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-11T08:58:24.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:24.701 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:24 smithi092 ceph-mon[29647]: pgmap v903: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.1 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 60 MiB/s wr, 965 op/s; 34434/252954 objects misplaced (13.613%); 54 MiB/s, 864 objects/s recovering 2024-09-11T08:58:24.701 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:24 smithi092 ceph-mon[29647]: osdmap e396: 8 total, 8 up, 7 in 2024-09-11T08:58:24.702 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:24 smithi092 ceph-mon[41114]: pgmap v903: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.1 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 60 MiB/s wr, 965 op/s; 34434/252954 objects misplaced (13.613%); 54 MiB/s, 864 objects/s recovering 2024-09-11T08:58:24.702 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:24 smithi092 ceph-mon[41114]: osdmap e396: 8 total, 8 up, 7 in 2024-09-11T08:58:24.768 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:24.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:24.769 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:24.870 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:24.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:24.901 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:24.901 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:25.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:25.007 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:58:25.047 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:25.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:25.049 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:25.149 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:25.178 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:25.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:25.179 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:25.280 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:25.310 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:25.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:25.311 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:25.347 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 56 16 87429 87413 97.5305 55.5 0.00756979 0.0102471 2024-09-11T08:58:25.412 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:25.439 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:25.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:25.440 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:25.541 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:25.570 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:25.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:25.571 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:25.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:25 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:58:25.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:25 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:58:25.672 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:25 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:58:25.672 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:25 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:58:25.672 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:25.697 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:25.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:25.698 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:25.799 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:25.824 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:25.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:25.825 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:25.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:25 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:58:25.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:25 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:58:25.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:25.992 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:25.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:25.994 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:26.094 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:26.138 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:26.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:26.139 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:26.239 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:26.287 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:26.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:26.289 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:26.347 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 57 16 88268 88252 96.7394 52.4375 0.0122346 0.0103319 2024-09-11T08:58:26.390 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:26.413 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:26.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:26.414 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:26.515 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:26.540 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:26.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:26.541 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:26.642 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:26.670 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:26.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:26.671 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:26.772 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:26.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:26.802 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:26.803 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:26.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:26 smithi092 ceph-mon[29647]: pgmap v905: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.1 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 809 B/s rd, 44 MiB/s wr, 697 op/s; 34434/252954 objects misplaced (13.613%); 36 MiB/s, 579 objects/s recovering 2024-09-11T08:58:26.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:26 smithi092 ceph-mon[29647]: osdmap e397: 8 total, 8 up, 7 in 2024-09-11T08:58:26.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:26 smithi092 ceph-mon[41114]: pgmap v905: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.1 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 809 B/s rd, 44 MiB/s wr, 697 op/s; 34434/252954 objects misplaced (13.613%); 36 MiB/s, 579 objects/s recovering 2024-09-11T08:58:26.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:26 smithi092 ceph-mon[41114]: osdmap e397: 8 total, 8 up, 7 in 2024-09-11T08:58:26.903 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:26.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:26 smithi145 ceph-mon[33384]: pgmap v905: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.1 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 809 B/s rd, 44 MiB/s wr, 697 op/s; 34434/252954 objects misplaced (13.613%); 36 MiB/s, 579 objects/s recovering 2024-09-11T08:58:26.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:26 smithi145 ceph-mon[33384]: osdmap e397: 8 total, 8 up, 7 in 2024-09-11T08:58:26.942 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:26.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:26.943 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:27.044 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:27.073 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:27.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:27.074 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:27.174 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:27.205 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:27.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:27.206 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:27.307 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:27.346 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:27.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:27.347 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:27.347 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 58 16 89077 89061 95.9434 50.5625 0.0114611 0.0104176 2024-09-11T08:58:27.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:27.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:27.472 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:27.506 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:58:27 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:58:27.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:27.613 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:27 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/214067967' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:58:27.613 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:27 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/214067967' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:58:27.620 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:27.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:27.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:27.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:27.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:27.752 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:27.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:27.875 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:27.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:27.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:27.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:27 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/214067967' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:58:27.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:28.021 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:28.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:28.022 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:28.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:28.170 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:28.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:28.171 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:28.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:28.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:28.293 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:28.293 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:28.347 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 59 16 89618 89602 94.8904 33.8125 0.0891348 0.0105283 2024-09-11T08:58:28.394 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:28.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:28.442 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:28.442 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:28.543 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:28.565 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:28.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:28.566 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:28.667 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:28.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:28.708 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:28.808 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:28.811 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:28 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:28.812 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:28 smithi145 ceph-mon[33384]: pgmap v907: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.4 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.6 KiB/s rd, 86 MiB/s wr, 1.38k op/s; 32211/265863 objects misplaced (12.116%); 72 MiB/s, 1.16k objects/s recovering 2024-09-11T08:58:28.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:28 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:28.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:28 smithi092 ceph-mon[29647]: pgmap v907: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.4 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.6 KiB/s rd, 86 MiB/s wr, 1.38k op/s; 32211/265863 objects misplaced (12.116%); 72 MiB/s, 1.16k objects/s recovering 2024-09-11T08:58:28.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:28 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:28.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:28 smithi092 ceph-mon[41114]: pgmap v907: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.4 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.6 KiB/s rd, 86 MiB/s wr, 1.38k op/s; 32211/265863 objects misplaced (12.116%); 72 MiB/s, 1.16k objects/s recovering 2024-09-11T08:58:28.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:28.832 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:28.832 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:28.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:28.968 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:28.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:28.969 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:28.981 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:58:28.982 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:58:28.982 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:58:28.982 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:58:28.982 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:58:28.982 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:58:28.982 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:58:28.982 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:58:28.982 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:58:28.982 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:58:28.983 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:58:28.983 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:58:28.983 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:58:28.983 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:58:28.983 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:58:28.983 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:58:29.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:29.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:29.095 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:29.095 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:29.196 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:29.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:29.227 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:29.227 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:29.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:29.348 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:58:29.349123+0000 min lat: 0.0011062 max lat: 2.29632 avg lat: 0.0106141 2024-09-11T08:58:29.348 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:58:29.348 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 60 16 90442 90426 94.1673 51.5 0.0119379 0.0106141 2024-09-11T08:58:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:29.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:29.370 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:29.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:29.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:29.502 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:29.502 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:29.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:29.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:29.641 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:29.641 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:29.742 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:29.764 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:29.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:29.765 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:29.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:29 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:58:29.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:29 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:58:29.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:29 smithi092 ceph-mon[29647]: osdmap e398: 8 total, 8 up, 7 in 2024-09-11T08:58:29.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:29 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:58:29.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:29 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:58:29.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:29 smithi092 ceph-mon[41114]: osdmap e398: 8 total, 8 up, 7 in 2024-09-11T08:58:29.866 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:29.895 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:29.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:29.896 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:29.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:29 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:58:29.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:29 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:58:29.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:29 smithi145 ceph-mon[33384]: osdmap e398: 8 total, 8 up, 7 in 2024-09-11T08:58:29.997 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:30.019 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:30.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:30.020 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:30.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:30.167 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:30.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:30.168 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:30.269 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:30.290 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:30.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:30.291 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:30.348 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 61 16 91193 91177 93.3931 46.9375 0.00704415 0.0107016 2024-09-11T08:58:30.391 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:30.419 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:30.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:30.420 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:30.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:30.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:30.548 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:30.549 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:30.634 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:58:30.634 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:58:30.649 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:30.678 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:30.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:30.679 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:30.780 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:30.808 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:30.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:30.809 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:30 smithi092 ceph-mon[29647]: pgmap v909: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.4 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 45 MiB/s wr, 717 op/s; 32211/265863 objects misplaced (12.116%); 38 MiB/s, 603 objects/s recovering 2024-09-11T08:58:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:30 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:58:30.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:58:30.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:30 smithi092 ceph-mon[41114]: pgmap v909: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.4 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 45 MiB/s wr, 717 op/s; 32211/265863 objects misplaced (12.116%); 38 MiB/s, 603 objects/s recovering 2024-09-11T08:58:30.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:30 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:58:30.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:58:30.909 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:30.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:30 smithi145 ceph-mon[33384]: pgmap v909: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.4 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 45 MiB/s wr, 717 op/s; 32211/265863 objects misplaced (12.116%); 38 MiB/s, 603 objects/s recovering 2024-09-11T08:58:30.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:30 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:58:30.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:58:30.953 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:30.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:30.954 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:31.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:31.095 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:31.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:31.096 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:31.197 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:31.223 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:31.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:31.224 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:31.325 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:31.348 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 62 16 92052 92036 92.7527 53.6875 0.0124314 0.0107758 2024-09-11T08:58:31.354 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:31.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:31.355 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:31.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:31.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:31.494 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:31.494 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:31.595 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:31.639 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:31.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:31.641 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:31.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:31.781 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:31.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:31.782 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:31.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:58:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:58:31] "GET /metrics HTTP/1.1" 200 225239 "" "Prometheus/2.43.0" 2024-09-11T08:58:31.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:31.927 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:31.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:31.928 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:32.029 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:32.053 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:32.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:32.054 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:32.155 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:32.188 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:32.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:32.190 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:32.291 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:32.311 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:32.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:32.312 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:32.348 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 63 16 92723 92707 91.9462 41.9375 0.0622071 0.01087 2024-09-11T08:58:32.413 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:32.450 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:32.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:32.451 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:32.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:32.585 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:32.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:32.586 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:32.688 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:32.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:32.733 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:32.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:32.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:32 smithi092 ceph-mon[29647]: pgmap v910: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.5 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.4 KiB/s rd, 51 MiB/s wr, 813 op/s; 31584/270816 objects misplaced (11.663%); 43 MiB/s, 684 objects/s recovering 2024-09-11T08:58:32.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:32 smithi092 ceph-mon[41114]: pgmap v910: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.5 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.4 KiB/s rd, 51 MiB/s wr, 813 op/s; 31584/270816 objects misplaced (11.663%); 43 MiB/s, 684 objects/s recovering 2024-09-11T08:58:32.834 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:32.865 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:32.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:32.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:32.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:32 smithi145 ceph-mon[33384]: pgmap v910: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.5 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.4 KiB/s rd, 51 MiB/s wr, 813 op/s; 31584/270816 objects misplaced (11.663%); 43 MiB/s, 684 objects/s recovering 2024-09-11T08:58:32.912 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:58:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:58:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:58:32.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:33.009 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:33.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:33.011 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:33.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:33.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:33.134 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:33.134 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:33.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:33.274 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:33.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:33.275 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:33.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:32 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a[29624]: 2024-09-11T08:58:32.972+0000 7f9ae2e34640 -1 mon.a@0(leader).osd e398 definitely_dead 0 2024-09-11T08:58:33.348 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 64 16 93459 93443 91.2284 46 0.00414915 0.0109521 2024-09-11T08:58:33.375 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:33.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:33.402 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:33.402 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:33.503 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:33.541 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:33.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:33.542 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:marked in osd.6. 2024-09-11T08:58:33.643 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:33.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:33.665 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:33.665 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:33.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:33.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:33.787 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:33.787 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:33.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:33 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4236534613' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["6"]}]: dispatch 2024-09-11T08:58:33.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:33 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4236534613' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["6"]}]: dispatch 2024-09-11T08:58:33.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:33.910 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:33.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:33.911 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:33.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:33 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4236534613' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["6"]}]: dispatch 2024-09-11T08:58:34.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:34.034 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:34.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:34.035 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:34.136 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:34.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:34.169 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:34.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:34.269 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:34.290 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:34.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:34.291 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:34.348 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 65 16 93611 93595 89.9712 9.5 0.172085 0.0110074 2024-09-11T08:58:34.392 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:34.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:34.415 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:34.415 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:34.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:34.537 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:34.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:34.539 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:34.640 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:34.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:34.663 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:34.764 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:34.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:34.786 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:34.787 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:34.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:34 smithi092 ceph-mon[29647]: pgmap v911: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.6 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 63 MiB/s wr, 1.01k op/s; 29823/277209 objects misplaced (10.758%); 56 MiB/s, 889 objects/s recovering 2024-09-11T08:58:34.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:34 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4236534613' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["6"]}]': finished 2024-09-11T08:58:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:34 smithi092 ceph-mon[29647]: osdmap e399: 8 total, 8 up, 8 in 2024-09-11T08:58:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:34 smithi092 ceph-mon[29647]: 3.4 scrub starts 2024-09-11T08:58:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:34 smithi092 ceph-mon[29647]: 3.4 scrub ok 2024-09-11T08:58:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:34 smithi092 ceph-mon[29647]: osdmap e400: 8 total, 8 up, 8 in 2024-09-11T08:58:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:34 smithi092 ceph-mon[41114]: pgmap v911: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.6 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 63 MiB/s wr, 1.01k op/s; 29823/277209 objects misplaced (10.758%); 56 MiB/s, 889 objects/s recovering 2024-09-11T08:58:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:34 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4236534613' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["6"]}]': finished 2024-09-11T08:58:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:34 smithi092 ceph-mon[41114]: osdmap e399: 8 total, 8 up, 8 in 2024-09-11T08:58:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:34 smithi092 ceph-mon[41114]: 3.4 scrub starts 2024-09-11T08:58:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:34 smithi092 ceph-mon[41114]: 3.4 scrub ok 2024-09-11T08:58:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:34 smithi092 ceph-mon[41114]: osdmap e400: 8 total, 8 up, 8 in 2024-09-11T08:58:34.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:34.908 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:34.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:34.909 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:34.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:34 smithi145 ceph-mon[33384]: pgmap v911: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.6 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 63 MiB/s wr, 1.01k op/s; 29823/277209 objects misplaced (10.758%); 56 MiB/s, 889 objects/s recovering 2024-09-11T08:58:34.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:34 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4236534613' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["6"]}]': finished 2024-09-11T08:58:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:34 smithi145 ceph-mon[33384]: osdmap e399: 8 total, 8 up, 8 in 2024-09-11T08:58:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:34 smithi145 ceph-mon[33384]: 3.4 scrub starts 2024-09-11T08:58:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:34 smithi145 ceph-mon[33384]: 3.4 scrub ok 2024-09-11T08:58:34.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:34 smithi145 ceph-mon[33384]: osdmap e400: 8 total, 8 up, 8 in 2024-09-11T08:58:35.010 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:35.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:35.034 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:35.034 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:35.135 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:35.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:35.170 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:35.170 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:35.271 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:35.317 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:35.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:35.318 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:35.348 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 66 16 93881 93865 88.8639 16.875 0.00299429 0.0112444 2024-09-11T08:58:35.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:35.440 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:35.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:35.441 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:35.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:35.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:35.579 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:35.579 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:35.680 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:35.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:35.713 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:35.713 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:35.814 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:35.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:35 smithi092 ceph-mon[29647]: 5.2 scrub starts 2024-09-11T08:58:35.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:35 smithi092 ceph-mon[29647]: osdmap e401: 8 total, 8 up, 8 in 2024-09-11T08:58:35.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:35 smithi092 ceph-mon[41114]: 5.2 scrub starts 2024-09-11T08:58:35.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:35 smithi092 ceph-mon[41114]: osdmap e401: 8 total, 8 up, 8 in 2024-09-11T08:58:35.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:35.851 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:35.851 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:35.874 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:35 smithi145 ceph-mon[33384]: 5.2 scrub starts 2024-09-11T08:58:35.874 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:35 smithi145 ceph-mon[33384]: osdmap e401: 8 total, 8 up, 8 in 2024-09-11T08:58:35.915 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-11T08:58:35.951 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:36.005 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:36.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:36.007 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:36.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:36.142 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:36.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:36.143 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:36.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:36.282 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:36.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:36.284 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:36.288 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-09-11T08:58:36.333 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:58:36.348 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 67 16 95041 95025 88.6196 72.5 0.0110175 0.0112794 2024-09-11T08:58:36.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:36.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:36.421 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:36.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:36.562 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:36.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:36.564 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:36.664 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:36.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:36.697 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:36.697 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:36.759 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:58:36.792 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:36 smithi145 ceph-mon[33384]: pgmap v915: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.6 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 852 B/s rd, 39 MiB/s wr, 630 op/s; 29823/277209 objects misplaced (10.758%); 36 MiB/s, 582 objects/s recovering 2024-09-11T08:58:36.792 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:36 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:58:36.798 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:36 smithi092 ceph-mon[29647]: pgmap v915: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.6 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 852 B/s rd, 39 MiB/s wr, 630 op/s; 29823/277209 objects misplaced (10.758%); 36 MiB/s, 582 objects/s recovering 2024-09-11T08:58:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:36 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:58:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:36 smithi092 ceph-mon[41114]: pgmap v915: 168 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 160 active+clean; 5.6 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 852 B/s rd, 39 MiB/s wr, 630 op/s; 29823/277209 objects misplaced (10.758%); 36 MiB/s, 582 objects/s recovering 2024-09-11T08:58:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:36 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:58:36.835 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:36.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:36.836 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:36.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:36.973 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:36.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:36.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:37.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:37.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:37.109 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:37.110 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:37.210 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:37.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:37.251 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:37.251 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:37.348 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 68 16 96371 96355 88.5387 83.125 0.0140635 0.0112901 2024-09-11T08:58:37.352 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:37.385 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:37.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:37.386 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:37.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:58:37 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:58:37.487 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:37.521 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:37.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:37.522 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:37.623 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:37.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:37.660 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:37.661 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:37.762 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:37.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:37.795 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:37.795 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:37.896 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:37.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:37 smithi145 ceph-mon[33384]: 6.a scrub starts 2024-09-11T08:58:37.931 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:37.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:37.932 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:38.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:38.063 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:38.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:38.064 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:38.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:37 smithi092 ceph-mon[29647]: 6.a scrub starts 2024-09-11T08:58:38.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:37 smithi092 ceph-mon[41114]: 6.a scrub starts 2024-09-11T08:58:38.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:38.213 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:38.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:38.215 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:38.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:38.348 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 69 16 97682 97666 88.443 81.9375 0.0136504 0.011302 2024-09-11T08:58:38.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:38.354 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:38.354 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:38.456 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:38.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:38.499 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:38.499 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:38.600 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:38.642 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:38.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:38.643 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:38.666 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:58:38.666 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:58:38.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:38 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:38.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:38 smithi092 ceph-mon[29647]: pgmap v916: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 5.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 55 MiB/s wr, 880 op/s; 845/286680 objects misplaced (0.295%); 38 MiB/s, 612 objects/s recovering 2024-09-11T08:58:38.723 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:38 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:38.723 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:38 smithi092 ceph-mon[41114]: pgmap v916: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 5.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 55 MiB/s wr, 880 op/s; 845/286680 objects misplaced (0.295%); 38 MiB/s, 612 objects/s recovering 2024-09-11T08:58:38.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:38.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:38.787 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:38.787 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:38.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:58:38.888 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:38.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:38 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:38.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:38 smithi145 ceph-mon[33384]: pgmap v916: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 5.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 55 MiB/s wr, 880 op/s; 845/286680 objects misplaced (0.295%); 38 MiB/s, 612 objects/s recovering 2024-09-11T08:58:38.924 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:38.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:38.925 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:39.025 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:39.053 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:39.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:39.054 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:58:39.155 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:39.186 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:39.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:39.187 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:39.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:39.328 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:39.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:39.329 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:39.349 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 70 16 98833 98817 88.2071 71.9375 0.00569484 0.0113322 2024-09-11T08:58:39.398 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-11T08:58:39.431 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:39.490 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:39.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:39.491 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:39.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:39.621 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:39.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:39.622 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:39.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:39 smithi145 ceph-mon[33384]: 3.10 scrub starts 2024-09-11T08:58:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:39 smithi145 ceph-mon[33384]: 5.c scrub starts 2024-09-11T08:58:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:39 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2992129212' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:58:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:39 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:58:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:39 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:58:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:39 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:58:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:39 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:58:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:39 smithi145 ceph-mon[33384]: osdmap e402: 8 total, 8 up, 8 in 2024-09-11T08:58:39.723 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:39.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:39.772 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:39.772 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:39.850 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:58:39.873 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:39.928 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:39.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:39.929 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:40.030 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:40.064 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:40.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:40.066 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:40.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[29647]: 3.10 scrub starts 2024-09-11T08:58:40.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[29647]: 5.c scrub starts 2024-09-11T08:58:40.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2992129212' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:58:40.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:58:40.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:58:40.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:58:40.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:58:40.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[29647]: osdmap e402: 8 total, 8 up, 8 in 2024-09-11T08:58:40.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[41114]: 3.10 scrub starts 2024-09-11T08:58:40.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[41114]: 5.c scrub starts 2024-09-11T08:58:40.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2992129212' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:58:40.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:58:40.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:58:40.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:58:40.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:58:40.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:39 smithi092 ceph-mon[41114]: osdmap e402: 8 total, 8 up, 8 in 2024-09-11T08:58:40.167 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:40.201 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:40.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:40.203 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:40.303 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:40.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:40.387 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:40.387 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:40.388 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 71 15 100049 100034 88.036 76.0625 0.0217088 0.0113543 2024-09-11T08:58:40.489 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:40.522 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:40.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:40.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:40.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:40.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:40 smithi145 ceph-mon[33384]: 5.1a scrub starts 2024-09-11T08:58:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:40 smithi145 ceph-mon[33384]: pgmap v918: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 5.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 34 MiB/s wr, 535 op/s; 845/286680 objects misplaced (0.295%); 16 MiB/s, 265 objects/s recovering 2024-09-11T08:58:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:40 smithi145 ceph-mon[33384]: 6.1 deep-scrub starts 2024-09-11T08:58:40.664 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:40.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:40.665 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:40.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:40.803 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:40.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:40.804 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:40.905 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:40.936 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:40.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:40.940 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:41.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:41.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:40 smithi092 ceph-mon[29647]: 5.1a scrub starts 2024-09-11T08:58:41.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:40 smithi092 ceph-mon[29647]: pgmap v918: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 5.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 34 MiB/s wr, 535 op/s; 845/286680 objects misplaced (0.295%); 16 MiB/s, 265 objects/s recovering 2024-09-11T08:58:41.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:40 smithi092 ceph-mon[29647]: 6.1 deep-scrub starts 2024-09-11T08:58:41.061 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:40 smithi092 ceph-mon[41114]: 5.1a scrub starts 2024-09-11T08:58:41.061 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:40 smithi092 ceph-mon[41114]: pgmap v918: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 5.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 34 MiB/s wr, 535 op/s; 845/286680 objects misplaced (0.295%); 16 MiB/s, 265 objects/s recovering 2024-09-11T08:58:41.061 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:40 smithi092 ceph-mon[41114]: 6.1 deep-scrub starts 2024-09-11T08:58:41.097 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:41.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:41.098 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:41.199 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:41.234 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:41.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:41.236 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:41.289 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 7, 0, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T08:58:41.289 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T08:58:41.290 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-11T08:58:41.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.1 reset_purged_snaps_last 2024-09-11T08:58:41.331 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:58:41.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:41.353 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 72 16 101291 101275 87.8894 77.5625 0.0244976 0.0113721 2024-09-11T08:58:41.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:41.400 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:41.401 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:41.501 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:41.534 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:41.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:41.535 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:41.636 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:41.660 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:58:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:58:41] "GET /metrics HTTP/1.1" 200 225239 "" "Prometheus/2.43.0" 2024-09-11T08:58:41.662 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:41 smithi092 ceph-mon[29647]: 5.10 scrub starts 2024-09-11T08:58:41.663 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:41 smithi092 ceph-mon[29647]: 6.4 scrub starts 2024-09-11T08:58:41.663 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:41 smithi092 ceph-mon[41114]: 5.10 scrub starts 2024-09-11T08:58:41.663 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:41 smithi092 ceph-mon[41114]: 6.4 scrub starts 2024-09-11T08:58:41.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:41.694 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:41.694 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:41.795 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:41.840 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:41.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:41.841 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:41.853 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:41 smithi145 ceph-mon[33384]: 5.10 scrub starts 2024-09-11T08:58:41.853 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:41 smithi145 ceph-mon[33384]: 6.4 scrub starts 2024-09-11T08:58:41.902 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:58:41.902 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:58:41.942 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:41.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:41.970 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:41.970 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:42.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:42.113 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:42.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:42.114 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:42.215 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:42.252 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:42.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:42.253 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:42.350 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 73 16 102370 102354 87.6092 67.4375 0.00913001 0.0114081 2024-09-11T08:58:42.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:42.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:42.378 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:42.379 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:42.480 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:42.513 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:42.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:42.515 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:42.613 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T08:58:42.615 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:42.637 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:42.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:42.638 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:42.739 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:42.769 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:42.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:42.770 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:42.871 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:42.903 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:42.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:42.904 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:42.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:58:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:58:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:58:42.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:42 smithi145 ceph-mon[33384]: 5.3 scrub starts 2024-09-11T08:58:42.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:42 smithi145 ceph-mon[33384]: 3.1e scrub starts 2024-09-11T08:58:42.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:42 smithi145 ceph-mon[33384]: pgmap v919: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 6.0 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 0 B/s rd, 50 MiB/s wr, 802 op/s; 546/294867 objects misplaced (0.185%); 15 MiB/s, 254 objects/s recovering 2024-09-11T08:58:42.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:42 smithi145 ceph-mon[33384]: 1.b scrub starts 2024-09-11T08:58:42.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:42 smithi145 ceph-mon[33384]: 1.b scrub ok 2024-09-11T08:58:42.951 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:58:43.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:43.045 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:43.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:43.046 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:43.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:42 smithi092 ceph-mon[29647]: 5.3 scrub starts 2024-09-11T08:58:43.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:42 smithi092 ceph-mon[29647]: 3.1e scrub starts 2024-09-11T08:58:43.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:42 smithi092 ceph-mon[29647]: pgmap v919: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 6.0 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 0 B/s rd, 50 MiB/s wr, 802 op/s; 546/294867 objects misplaced (0.185%); 15 MiB/s, 254 objects/s recovering 2024-09-11T08:58:43.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:42 smithi092 ceph-mon[29647]: 1.b scrub starts 2024-09-11T08:58:43.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:42 smithi092 ceph-mon[29647]: 1.b scrub ok 2024-09-11T08:58:43.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:42 smithi092 ceph-mon[41114]: 5.3 scrub starts 2024-09-11T08:58:43.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:42 smithi092 ceph-mon[41114]: 3.1e scrub starts 2024-09-11T08:58:43.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:42 smithi092 ceph-mon[41114]: pgmap v919: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 6.0 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 0 B/s rd, 50 MiB/s wr, 802 op/s; 546/294867 objects misplaced (0.185%); 15 MiB/s, 254 objects/s recovering 2024-09-11T08:58:43.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:42 smithi092 ceph-mon[41114]: 1.b scrub starts 2024-09-11T08:58:43.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:42 smithi092 ceph-mon[41114]: 1.b scrub ok 2024-09-11T08:58:43.147 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:43.175 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:43.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:43.176 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:43.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:58:43.276 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:43.316 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:43.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:43.317 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:43.350 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 74 16 103189 103173 87.1171 51.1875 0.00771331 0.0114731 2024-09-11T08:58:43.418 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:43.452 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:43.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:43.453 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:43.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:43.596 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:43.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:43.600 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:43.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:43.736 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:43.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:43.737 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:43.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:43 smithi092 ceph-mon[29647]: 4.10 scrub starts 2024-09-11T08:58:43.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:43 smithi092 ceph-mon[29647]: 5.1e scrub starts 2024-09-11T08:58:43.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:43 smithi092 ceph-mon[29647]: 4.b deep-scrub starts 2024-09-11T08:58:43.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:43 smithi092 ceph-mon[29647]: 1.13 scrub starts 2024-09-11T08:58:43.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:43 smithi092 ceph-mon[41114]: 4.10 scrub starts 2024-09-11T08:58:43.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:43 smithi092 ceph-mon[41114]: 5.1e scrub starts 2024-09-11T08:58:43.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:43 smithi092 ceph-mon[41114]: 4.b deep-scrub starts 2024-09-11T08:58:43.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:43 smithi092 ceph-mon[41114]: 1.13 scrub starts 2024-09-11T08:58:43.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:43.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:43.873 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:43.873 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:43.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:43 smithi145 ceph-mon[33384]: 4.10 scrub starts 2024-09-11T08:58:43.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:43 smithi145 ceph-mon[33384]: 5.1e scrub starts 2024-09-11T08:58:43.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:43 smithi145 ceph-mon[33384]: 4.b deep-scrub starts 2024-09-11T08:58:43.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:43 smithi145 ceph-mon[33384]: 1.13 scrub starts 2024-09-11T08:58:43.974 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:44.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:44.010 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:44.011 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:44.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:44.133 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:44.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:44.135 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:44.235 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:44.267 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:44.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:44.268 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:44.350 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 75 16 104339 104323 86.9138 71.875 0.0102409 0.0115012 2024-09-11T08:58:44.369 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:44.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:44.405 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:44.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:44.506 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:44.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:44.535 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:44.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:44.636 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:44.671 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:44.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:44.672 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:44.773 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:44.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:44.795 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:44.795 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:44.852 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:44 smithi145 ceph-mon[33384]: pgmap v920: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 6.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 0 B/s rd, 70 MiB/s wr, 1.12k op/s; 546/305337 objects misplaced (0.179%); 13 MiB/s, 223 objects/s recovering 2024-09-11T08:58:44.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:44.935 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:44.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:44.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:44.994 INFO:teuthology.orchestra.run.smithi145.stdout:ERROR: (22) Invalid argument 2024-09-11T08:58:44.994 INFO:teuthology.orchestra.run.smithi145.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:58:45.037 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:45.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:44 smithi092 ceph-mon[29647]: pgmap v920: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 6.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 0 B/s rd, 70 MiB/s wr, 1.12k op/s; 546/305337 objects misplaced (0.179%); 13 MiB/s, 223 objects/s recovering 2024-09-11T08:58:45.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:44 smithi092 ceph-mon[41114]: pgmap v920: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 6.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 0 B/s rd, 70 MiB/s wr, 1.12k op/s; 546/305337 objects misplaced (0.179%); 13 MiB/s, 223 objects/s recovering 2024-09-11T08:58:45.071 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:45.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:45.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:45.173 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:45.217 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:45.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:45.218 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:45.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:45.350 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 76 16 105517 105501 86.7389 73.625 0.00855311 0.0115244 2024-09-11T08:58:45.355 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:45.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:45.356 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:45.457 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:45.493 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:45.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:45.494 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:45.594 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:45.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:45.629 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:45.630 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:45.668 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-11T08:58:45.699 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:58:45.700 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:58:45.731 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:45.794 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:45.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:45.795 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:45.812 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:45 smithi092 ceph-mon[29647]: 6.d deep-scrub starts 2024-09-11T08:58:45.812 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:45 smithi092 ceph-mon[29647]: 3.6 scrub starts 2024-09-11T08:58:45.812 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:45 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:58:45.812 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:58:45.813 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:45 smithi092 ceph-mon[41114]: 6.d deep-scrub starts 2024-09-11T08:58:45.813 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:45 smithi092 ceph-mon[41114]: 3.6 scrub starts 2024-09-11T08:58:45.813 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:45 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:58:45.813 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:58:45.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:45.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:45 smithi145 ceph-mon[33384]: 6.d deep-scrub starts 2024-09-11T08:58:45.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:45 smithi145 ceph-mon[33384]: 3.6 scrub starts 2024-09-11T08:58:45.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:45 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:58:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:58:45.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:45.935 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:45.936 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:46.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:46.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:46.072 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:46.103 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:58:46.173 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:46.205 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:46.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:46.206 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:46.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:46.340 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:46.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:46.341 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:46.350 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 77 16 106739 106723 86.6042 76.375 0.0111239 0.0115419 2024-09-11T08:58:46.369 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:58:46.369 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:58:46.407 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:58:46.408 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:58:46.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:46.448 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:58:46.449 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:58:46.492 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:46.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:46.493 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:46.506 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:58:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:58:46.541 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:58:46.541 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:58:46.570 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:58:46.570 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:58:46.594 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:46.600 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:58:46.601 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:58:46.628 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:46.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:46.629 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:46.633 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:58:46.633 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:58:46.634 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:46 smithi092 ceph-mon[29647]: 6.19 scrub starts 2024-09-11T08:58:46.634 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:46 smithi092 ceph-mon[29647]: 6.19 scrub ok 2024-09-11T08:58:46.634 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:46 smithi092 ceph-mon[29647]: pgmap v921: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 6.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 0 B/s rd, 59 MiB/s wr, 937 op/s; 546/305337 objects misplaced (0.179%); 11 MiB/s, 186 objects/s recovering 2024-09-11T08:58:46.634 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:46 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2943944152' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:58:46.634 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:46 smithi092 ceph-mon[41114]: 6.19 scrub starts 2024-09-11T08:58:46.634 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:46 smithi092 ceph-mon[41114]: 6.19 scrub ok 2024-09-11T08:58:46.634 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:46 smithi092 ceph-mon[41114]: pgmap v921: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 6.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 0 B/s rd, 59 MiB/s wr, 937 op/s; 546/305337 objects misplaced (0.179%); 11 MiB/s, 186 objects/s recovering 2024-09-11T08:58:46.634 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:46 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2943944152' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:58:46.730 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:46.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:46.770 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:46.770 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:46.871 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:46.907 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:46.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:46.909 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:46.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:46 smithi145 ceph-mon[33384]: 6.19 scrub starts 2024-09-11T08:58:46.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:46 smithi145 ceph-mon[33384]: 6.19 scrub ok 2024-09-11T08:58:46.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:46 smithi145 ceph-mon[33384]: pgmap v921: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 6.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 0 B/s rd, 59 MiB/s wr, 937 op/s; 546/305337 objects misplaced (0.179%); 11 MiB/s, 186 objects/s recovering 2024-09-11T08:58:46.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:46 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2943944152' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:58:47.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:47.050 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:47.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:47.051 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:47.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:47.195 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:47.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:47.196 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:47.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:47.334 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:47.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:47.335 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:47.350 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 78 16 107948 107932 86.4626 75.5625 0.00719561 0.0115609 2024-09-11T08:58:47.360 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:58:47 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:58:47.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:47.473 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:47.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:47.474 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:47.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:47.613 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:47.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:47.614 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:47.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:47.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:47.748 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:47.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:47.884 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:47.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:47.885 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:47.907 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:47 smithi145 ceph-mon[33384]: 5.f scrub starts 2024-09-11T08:58:47.907 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:47 smithi145 ceph-mon[33384]: 5.5 scrub starts 2024-09-11T08:58:47.989 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:48.023 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:48.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:48.024 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:48.061 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:58:48.061 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T08:58:48.061 INFO:teuthology.orchestra.run.smithi145.stdout: "complaint_time": 30, 2024-09-11T08:58:48.061 INFO:teuthology.orchestra.run.smithi145.stdout: "num_blocked_ops": 0 2024-09-11T08:58:48.061 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:58:48.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:47 smithi092 ceph-mon[29647]: 5.f scrub starts 2024-09-11T08:58:48.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:47 smithi092 ceph-mon[29647]: 5.5 scrub starts 2024-09-11T08:58:48.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:47 smithi092 ceph-mon[41114]: 5.f scrub starts 2024-09-11T08:58:48.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:47 smithi092 ceph-mon[41114]: 5.5 scrub starts 2024-09-11T08:58:48.126 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:48.155 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:48.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:48.156 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:48.256 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:48.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:48.302 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:48.303 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:48.350 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 79 16 109174 109158 86.338 76.625 0.0121496 0.0115775 2024-09-11T08:58:48.404 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:48.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:48.446 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:48.446 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:48.547 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:48.575 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:48.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:48.576 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:48.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:48.718 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:48.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:48.719 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:48.763 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:48 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:48.763 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:48 smithi145 ceph-mon[33384]: pgmap v922: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 6.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 9.7 KiB/s rd, 73 MiB/s wr, 1.18k op/s; 162/321684 objects misplaced (0.050%); 4.3 MiB/s, 68 objects/s recovering 2024-09-11T08:58:48.763 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:48 smithi145 ceph-mon[33384]: 6.15 deep-scrub starts 2024-09-11T08:58:48.763 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:48 smithi145 ceph-mon[33384]: 4.f scrub starts 2024-09-11T08:58:48.763 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:48 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:58:48.789 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-11T08:58:48.820 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:48.855 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:48.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:48.856 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:48.916 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:48 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:48.916 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:48 smithi092 ceph-mon[41114]: pgmap v922: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 6.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 9.7 KiB/s rd, 73 MiB/s wr, 1.18k op/s; 162/321684 objects misplaced (0.050%); 4.3 MiB/s, 68 objects/s recovering 2024-09-11T08:58:48.916 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:48 smithi092 ceph-mon[41114]: 6.15 deep-scrub starts 2024-09-11T08:58:48.916 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:48 smithi092 ceph-mon[41114]: 4.f scrub starts 2024-09-11T08:58:48.916 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:48 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:58:48.917 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:48 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:48.917 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:48 smithi092 ceph-mon[29647]: pgmap v922: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 6.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 9.7 KiB/s rd, 73 MiB/s wr, 1.18k op/s; 162/321684 objects misplaced (0.050%); 4.3 MiB/s, 68 objects/s recovering 2024-09-11T08:58:48.917 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:48 smithi092 ceph-mon[29647]: 6.15 deep-scrub starts 2024-09-11T08:58:48.917 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:48 smithi092 ceph-mon[29647]: 4.f scrub starts 2024-09-11T08:58:48.917 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:48 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:58:48.957 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:48.978 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:48.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:48.979 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:49.057 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:58:49.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:49.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:49.103 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:49.104 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:49.204 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:49.227 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:49.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:49.228 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:49.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:49.350 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:2024-09-11T08:58:49.351733+0000 min lat: 0.0011062 max lat: 2.29632 avg lat: 0.0115754 2024-09-11T08:58:49.350 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T08:58:49.350 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 80 16 109700 109684 85.6698 32.875 0.00299814 0.0115754 2024-09-11T08:58:49.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:49.351 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:49.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:49.452 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:49.476 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:49.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:49.477 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:49.577 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:49.600 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:49.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:49.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:49.701 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:49.722 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:49.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:49.723 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:49.803 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:49 smithi145 ceph-mon[33384]: 4.0 scrub starts 2024-09-11T08:58:49.803 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:49 smithi145 ceph-mon[33384]: 4.0 scrub ok 2024-09-11T08:58:49.803 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:49 smithi145 ceph-mon[33384]: osdmap e403: 8 total, 8 up, 8 in 2024-09-11T08:58:49.804 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:49 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:58:49.804 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:49 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:58:49.804 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:49 smithi145 ceph-mon[33384]: 1.8 scrub starts 2024-09-11T08:58:49.804 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:49 smithi145 ceph-mon[33384]: 1.8 scrub ok 2024-09-11T08:58:49.804 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:49 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:58:49.804 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:49 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:58:49.804 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:49 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:58:49.804 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:49 smithi145 ceph-mon[33384]: osdmap e404: 8 total, 8 up, 8 in 2024-09-11T08:58:49.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[29647]: 4.0 scrub starts 2024-09-11T08:58:49.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[29647]: 4.0 scrub ok 2024-09-11T08:58:49.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[29647]: osdmap e403: 8 total, 8 up, 8 in 2024-09-11T08:58:49.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:58:49.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:58:49.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[29647]: 1.8 scrub starts 2024-09-11T08:58:49.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[29647]: 1.8 scrub ok 2024-09-11T08:58:49.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:58:49.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:58:49.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:58:49.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[29647]: osdmap e404: 8 total, 8 up, 8 in 2024-09-11T08:58:49.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[41114]: 4.0 scrub starts 2024-09-11T08:58:49.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[41114]: 4.0 scrub ok 2024-09-11T08:58:49.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[41114]: osdmap e403: 8 total, 8 up, 8 in 2024-09-11T08:58:49.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:58:49.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:58:49.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[41114]: 1.8 scrub starts 2024-09-11T08:58:49.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[41114]: 1.8 scrub ok 2024-09-11T08:58:49.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:58:49.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:58:49.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:58:49.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:49 smithi092 ceph-mon[41114]: osdmap e404: 8 total, 8 up, 8 in 2024-09-11T08:58:49.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:49.845 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:49.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:49.846 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:49.947 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:49.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:49.970 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:50.071 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:50.098 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:50.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:50.099 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:50.199 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:50.237 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:50.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:50.238 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:50.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:58:50.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:50.350 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 81 16 110144 110128 84.9548 27.75 0.0139329 0.0117657 2024-09-11T08:58:50.378 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:50.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:50.379 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:50.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:50.527 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:50.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:50.528 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:50.629 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:50.672 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:50.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:50.673 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:50.774 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:50.815 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:50.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:50.816 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:50.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:50 smithi145 ceph-mon[33384]: pgmap v925: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 6.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 11 KiB/s rd, 70 MiB/s wr, 1.14k op/s; 162/321684 objects misplaced (0.050%); 3.0 MiB/s, 47 objects/s recovering 2024-09-11T08:58:50.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:50 smithi145 ceph-mon[33384]: 4.6 scrub starts 2024-09-11T08:58:50.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:50 smithi145 ceph-mon[33384]: 4.6 scrub ok 2024-09-11T08:58:50.917 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:50.949 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:50.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:50.950 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:50.959 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T08:58:50.959 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T08:58:50.959 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T08:58:50.959 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T08:58:50.959 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.959 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120775 9.19 9:9f3327b4:::benchmark_data_smithi092_145083_object120774:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:58:50.959 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.527015+0000", 2024-09-11T08:58:50.959 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 505.42968980000001, 2024-09-11T08:58:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0204559310000001, 2024-09-11T08:58:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120775 2024-09-11T08:58:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527015+0000", 2024-09-11T08:58:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527015+0000", 2024-09-11T08:58:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527008+0000", 2024-09-11T08:58:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999933 2024-09-11T08:58:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527071+0000", 2024-09-11T08:58:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2667000000000002e-05 2024-09-11T08:58:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527089+0000", 2024-09-11T08:58:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.8498000000000001e-05 2024-09-11T08:58:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.527099+0000", 2024-09-11T08:58:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0016e-05 2024-09-11T08:58:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.676418+0000", 2024-09-11T08:58:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.14931815200000001 2024-09-11T08:58:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.676427+0000", 2024-09-11T08:58:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.8320000000000001e-06 2024-09-11T08:58:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538222+0000", 2024-09-11T08:58:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.86179418299999999 2024-09-11T08:58:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538346+0000", 2024-09-11T08:58:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012482300000000001 2024-09-11T08:58:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,5", 2024-09-11T08:58:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538574+0000", 2024-09-11T08:58:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00022776599999999999 2024-09-11T08:58:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.539918+0000", 2024-09-11T08:58:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001344065 2024-09-11T08:58:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.539927+0000", 2024-09-11T08:58:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.4710000000000007e-06 2024-09-11T08:58:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547401+0000", 2024-09-11T08:58:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.007474217 2024-09-11T08:58:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547440+0000", 2024-09-11T08:58:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9490999999999998e-05 2024-09-11T08:58:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547471+0000", 2024-09-11T08:58:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0651000000000002e-05 2024-09-11T08:58:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120776 9.13 9:cecf0514:::benchmark_data_smithi092_145083_object120775:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:58:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.655311+0000", 2024-09-11T08:58:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 505.30139345800001, 2024-09-11T08:58:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89210484800000001, 2024-09-11T08:58:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120776 2024-09-11T08:58:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655311+0000", 2024-09-11T08:58:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655311+0000", 2024-09-11T08:58:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655309+0000", 2024-09-11T08:58:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T08:58:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655369+0000", 2024-09-11T08:58:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0052e-05 2024-09-11T08:58:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655395+0000", 2024-09-11T08:58:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5205999999999999e-05 2024-09-11T08:58:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.655405+0000", 2024-09-11T08:58:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0399e-05 2024-09-11T08:58:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.659087+0000", 2024-09-11T08:58:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0036818240000000002 2024-09-11T08:58:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.659098+0000", 2024-09-11T08:58:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1600000000000001e-05 2024-09-11T08:58:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.537736+0000", 2024-09-11T08:58:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.87863754699999996 2024-09-11T08:58:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.537834+0000", 2024-09-11T08:58:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.7986999999999995e-05 2024-09-11T08:58:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:58:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538018+0000", 2024-09-11T08:58:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00018384599999999999 2024-09-11T08:58:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538962+0000", 2024-09-11T08:58:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00094433400000000002 2024-09-11T08:58:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.540658+0000", 2024-09-11T08:58:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0016956969999999999 2024-09-11T08:58:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.973 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:50.973 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547326+0000", 2024-09-11T08:58:50.973 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0066684819999999999 2024-09-11T08:58:50.973 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.973 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.973 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:50.973 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547381+0000", 2024-09-11T08:58:50.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:50 smithi092 ceph-mon[29647]: pgmap v925: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 6.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 11 KiB/s rd, 70 MiB/s wr, 1.14k op/s; 162/321684 objects misplaced (0.050%); 3.0 MiB/s, 47 objects/s recovering 2024-09-11T08:58:50.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:50 smithi092 ceph-mon[29647]: 4.6 scrub starts 2024-09-11T08:58:50.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:50 smithi092 ceph-mon[29647]: 4.6 scrub ok 2024-09-11T08:58:50.974 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:50 smithi092 ceph-mon[41114]: pgmap v925: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 6.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 11 KiB/s rd, 70 MiB/s wr, 1.14k op/s; 162/321684 objects misplaced (0.050%); 3.0 MiB/s, 47 objects/s recovering 2024-09-11T08:58:50.975 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:50 smithi092 ceph-mon[41114]: 4.6 scrub starts 2024-09-11T08:58:50.975 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:50 smithi092 ceph-mon[41114]: 4.6 scrub ok 2024-09-11T08:58:50.975 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4537999999999999e-05 2024-09-11T08:58:50.975 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.975 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.975 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:50.975 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.547416+0000", 2024-09-11T08:58:50.975 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5389000000000003e-05 2024-09-11T08:58:50.975 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:50.975 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:50.975 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:120777 9.3 9:c63642ab:::benchmark_data_smithi092_145083_object120776:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-11T08:58:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:25.681266+0000", 2024-09-11T08:58:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 505.27543905200002, 2024-09-11T08:58:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.85751633400000005, 2024-09-11T08:58:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 120777 2024-09-11T08:58:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681266+0000", 2024-09-11T08:58:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681266+0000", 2024-09-11T08:58:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681264+0000", 2024-09-11T08:58:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681297+0000", 2024-09-11T08:58:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2461e-05 2024-09-11T08:58:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681309+0000", 2024-09-11T08:58:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2519e-05 2024-09-11T08:58:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681314+0000", 2024-09-11T08:58:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1279999999999999e-06 2024-09-11T08:58:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681360+0000", 2024-09-11T08:58:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5438000000000001e-05 2024-09-11T08:58:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:25.681368+0000", 2024-09-11T08:58:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2949999999999994e-06 2024-09-11T08:58:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.525926+0000", 2024-09-11T08:58:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.84455778599999998 2024-09-11T08:58:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.526080+0000", 2024-09-11T08:58:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015359799999999999 2024-09-11T08:58:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:58:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.526368+0000", 2024-09-11T08:58:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000288027 2024-09-11T08:58:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.527948+0000", 2024-09-11T08:58:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0015808269999999999 2024-09-11T08:58:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.535215+0000", 2024-09-11T08:58:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0072669259999999999 2024-09-11T08:58:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538703+0000", 2024-09-11T08:58:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0034881640000000002 2024-09-11T08:58:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538748+0000", 2024-09-11T08:58:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4304999999999997e-05 2024-09-11T08:58:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:26.538782+0000", 2024-09-11T08:58:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4400000000000003e-05 2024-09-11T08:58:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122628 9.13 9:cbe7bc43:::benchmark_data_smithi092_145083_object122627:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236)", 2024-09-11T08:58:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.520279+0000", 2024-09-11T08:58:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 503.43642578800001, 2024-09-11T08:58:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.435847541, 2024-09-11T08:58:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:58:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122628 2024-09-11T08:58:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520279+0000", 2024-09-11T08:58:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520279+0000", 2024-09-11T08:58:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520278+0000", 2024-09-11T08:58:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520339+0000", 2024-09-11T08:58:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1049000000000005e-05 2024-09-11T08:58:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520349+0000", 2024-09-11T08:58:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.6190000000000006e-06 2024-09-11T08:58:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520353+0000", 2024-09-11T08:58:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3220000000000003e-06 2024-09-11T08:58:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520397+0000", 2024-09-11T08:58:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3739000000000003e-05 2024-09-11T08:58:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520511+0000", 2024-09-11T08:58:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011360499999999999 2024-09-11T08:58:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:58:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.520760+0000", 2024-09-11T08:58:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00024918800000000001 2024-09-11T08:58:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.522187+0000", 2024-09-11T08:58:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0014270960000000001 2024-09-11T08:58:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.527785+0000", 2024-09-11T08:58:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0055977650000000002 2024-09-11T08:58:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.536750+0000", 2024-09-11T08:58:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0089646959999999994 2024-09-11T08:58:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.536758+0000", 2024-09-11T08:58:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.1410000000000005e-06 2024-09-11T08:58:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537393+0000", 2024-09-11T08:58:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.000635798 2024-09-11T08:58:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537404+0000", 2024-09-11T08:58:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0471e-05 2024-09-11T08:58:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.956039+0000", 2024-09-11T08:58:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.41863534800000002 2024-09-11T08:58:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.956127+0000", 2024-09-11T08:58:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.7373999999999994e-05 2024-09-11T08:58:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122632 9.13 9:cb01d006:::benchmark_data_smithi092_145083_object122631:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236)", 2024-09-11T08:58:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.523542+0000", 2024-09-11T08:58:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 503.43316264200001, 2024-09-11T08:58:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.43267765, 2024-09-11T08:58:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T08:58:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122632 2024-09-11T08:58:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523542+0000", 2024-09-11T08:58:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523542+0000", 2024-09-11T08:58:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523541+0000", 2024-09-11T08:58:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:58:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523573+0000", 2024-09-11T08:58:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2162000000000001e-05 2024-09-11T08:58:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523585+0000", 2024-09-11T08:58:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1269e-05 2024-09-11T08:58:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523590+0000", 2024-09-11T08:58:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5720000000000002e-06 2024-09-11T08:58:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523636+0000", 2024-09-11T08:58:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6233999999999998e-05 2024-09-11T08:58:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.523944+0000", 2024-09-11T08:58:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00030767799999999998 2024-09-11T08:58:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:58:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.524202+0000", 2024-09-11T08:58:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00025762300000000001 2024-09-11T08:58:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.528196+0000", 2024-09-11T08:58:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0039945980000000002 2024-09-11T08:58:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.530011+0000", 2024-09-11T08:58:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001814525 2024-09-11T08:58:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.537490+0000", 2024-09-11T08:58:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0074790059999999998 2024-09-11T08:58:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.537523+0000", 2024-09-11T08:58:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3389000000000002e-05 2024-09-11T08:58:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537576+0000", 2024-09-11T08:58:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0000530000000001 2024-09-11T08:58:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537583+0000", 2024-09-11T08:58:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9650000000000002e-06 2024-09-11T08:58:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.956160+0000", 2024-09-11T08:58:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.41857655500000002 2024-09-11T08:58:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.956220+0000", 2024-09-11T08:58:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0102000000000002e-05 2024-09-11T08:58:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122660 9.17 9:e8b491cd:::benchmark_data_smithi092_145083_object122659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:58:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.551000+0000", 2024-09-11T08:58:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 503.40570505400001, 2024-09-11T08:58:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4263746720000001, 2024-09-11T08:58:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122660 2024-09-11T08:58:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551000+0000", 2024-09-11T08:58:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551000+0000", 2024-09-11T08:58:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.550999+0000", 2024-09-11T08:58:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:58:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551060+0000", 2024-09-11T08:58:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1168999999999997e-05 2024-09-11T08:58:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551072+0000", 2024-09-11T08:58:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2099000000000001e-05 2024-09-11T08:58:51.001 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.001 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.001 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:51.001 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551078+0000", 2024-09-11T08:58:51.001 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5629999999999999e-06 2024-09-11T08:58:51.001 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.001 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.001 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.001 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551141+0000", 2024-09-11T08:58:51.001 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3082000000000004e-05 2024-09-11T08:58:51.002 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.002 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.002 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.002 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.551150+0000", 2024-09-11T08:58:51.002 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8459999999999997e-06 2024-09-11T08:58:51.002 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.002 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.002 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.002 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537577+0000", 2024-09-11T08:58:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.98642732899999996 2024-09-11T08:58:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537585+0000", 2024-09-11T08:58:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2230000000000007e-06 2024-09-11T08:58:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.961506+0000", 2024-09-11T08:58:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.42392071100000001 2024-09-11T08:58:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.961706+0000", 2024-09-11T08:58:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00020044799999999999 2024-09-11T08:58:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2,3,5", 2024-09-11T08:58:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.962037+0000", 2024-09-11T08:58:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00033083199999999998 2024-09-11T08:58:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.964043+0000", 2024-09-11T08:58:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0020053179999999999 2024-09-11T08:58:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.965851+0000", 2024-09-11T08:58:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0018087610000000001 2024-09-11T08:58:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.967059+0000", 2024-09-11T08:58:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0012074080000000001 2024-09-11T08:58:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.967101+0000", 2024-09-11T08:58:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2601000000000001e-05 2024-09-11T08:58:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.977211+0000", 2024-09-11T08:58:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.010109315000000001 2024-09-11T08:58:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.977264+0000", 2024-09-11T08:58:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3288999999999998e-05 2024-09-11T08:58:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.977375+0000", 2024-09-11T08:58:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000110554 2024-09-11T08:58:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:122671 9.15 9:ab433584:::benchmark_data_smithi092_145083_object122670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237)", 2024-09-11T08:58:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:27.558491+0000", 2024-09-11T08:58:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 503.398214236, 2024-09-11T08:58:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3936105059999999, 2024-09-11T08:58:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 122671 2024-09-11T08:58:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558491+0000", 2024-09-11T08:58:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558491+0000", 2024-09-11T08:58:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558490+0000", 2024-09-11T08:58:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:58:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558704+0000", 2024-09-11T08:58:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00021395600000000001 2024-09-11T08:58:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558716+0000", 2024-09-11T08:58:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.278e-05 2024-09-11T08:58:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558722+0000", 2024-09-11T08:58:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4999999999999999e-06 2024-09-11T08:58:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558765+0000", 2024-09-11T08:58:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2815999999999998e-05 2024-09-11T08:58:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:27.558770+0000", 2024-09-11T08:58:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1660000000000002e-06 2024-09-11T08:58:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537512+0000", 2024-09-11T08:58:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97874157399999995 2024-09-11T08:58:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.537520+0000", 2024-09-11T08:58:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.5860000000000005e-06 2024-09-11T08:58:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.932889+0000", 2024-09-11T08:58:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.39536916900000002 2024-09-11T08:58:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.933050+0000", 2024-09-11T08:58:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00016028799999999999 2024-09-11T08:58:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3,5", 2024-09-11T08:58:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.933393+0000", 2024-09-11T08:58:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00034307499999999998 2024-09-11T08:58:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.937335+0000", 2024-09-11T08:58:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0039422570000000002 2024-09-11T08:58:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.939190+0000", 2024-09-11T08:58:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0018552729999999999 2024-09-11T08:58:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.942079+0000", 2024-09-11T08:58:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.002888767 2024-09-11T08:58:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.951900+0000", 2024-09-11T08:58:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0098207899999999994 2024-09-11T08:58:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.951972+0000", 2024-09-11T08:58:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2611e-05 2024-09-11T08:58:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:28.952101+0000", 2024-09-11T08:58:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012884499999999999 2024-09-11T08:58:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:161920 9.15 9:afdf2969:::benchmark_data_smithi092_145083_object18522:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:58:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:58.943979+0000", 2024-09-11T08:58:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 472.01272624299997, 2024-09-11T08:58:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.001253038, 2024-09-11T08:58:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 161920 2024-09-11T08:58:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.943979+0000", 2024-09-11T08:58:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.017 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:51.017 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.943979+0000", 2024-09-11T08:58:51.017 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.017 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.017 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.017 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:51.017 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.943978+0000", 2024-09-11T08:58:51.017 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:58:51.017 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.017 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.018 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:51.018 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.943982+0000", 2024-09-11T08:58:51.018 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5090000000000004e-06 2024-09-11T08:58:51.018 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.018 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.018 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:51.018 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.944019+0000", 2024-09-11T08:58:51.018 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6127999999999997e-05 2024-09-11T08:58:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.944024+0000", 2024-09-11T08:58:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9320000000000001e-06 2024-09-11T08:58:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.945966+0000", 2024-09-11T08:58:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001941047 2024-09-11T08:58:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.945973+0000", 2024-09-11T08:58:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.267e-06 2024-09-11T08:58:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.924511+0000", 2024-09-11T08:58:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97853800700000004 2024-09-11T08:58:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.924629+0000", 2024-09-11T08:58:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000117945 2024-09-11T08:58:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:58:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.924992+0000", 2024-09-11T08:58:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00036293600000000001 2024-09-11T08:58:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.931573+0000", 2024-09-11T08:58:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0065809639999999999 2024-09-11T08:58:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945131+0000", 2024-09-11T08:58:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013558374999999999 2024-09-11T08:58:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945175+0000", 2024-09-11T08:58:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3695000000000001e-05 2024-09-11T08:58:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945208+0000", 2024-09-11T08:58:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3596000000000002e-05 2024-09-11T08:58:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945232+0000", 2024-09-11T08:58:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3382000000000001e-05 2024-09-11T08:58:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:161947 9.15 9:a9036c16:::benchmark_data_smithi092_145083_object18549:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:58:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:58.962388+0000", 2024-09-11T08:58:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 471.99431705900002, 2024-09-11T08:58:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.98294829399999994, 2024-09-11T08:58:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 161947 2024-09-11T08:58:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962388+0000", 2024-09-11T08:58:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962388+0000", 2024-09-11T08:58:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962387+0000", 2024-09-11T08:58:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962392+0000", 2024-09-11T08:58:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8999999999999997e-06 2024-09-11T08:58:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962400+0000", 2024-09-11T08:58:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6109999999999996e-06 2024-09-11T08:58:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962404+0000", 2024-09-11T08:58:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5549999999999996e-06 2024-09-11T08:58:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962462+0000", 2024-09-11T08:58:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7262999999999997e-05 2024-09-11T08:58:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.962468+0000", 2024-09-11T08:58:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3049999999999998e-06 2024-09-11T08:58:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.926681+0000", 2024-09-11T08:58:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.96421365999999997 2024-09-11T08:58:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.926819+0000", 2024-09-11T08:58:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013747 2024-09-11T08:58:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:58:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.927081+0000", 2024-09-11T08:58:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00026203400000000002 2024-09-11T08:58:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.931577+0000", 2024-09-11T08:58:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.004495644 2024-09-11T08:58:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945265+0000", 2024-09-11T08:58:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013688709 2024-09-11T08:58:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945295+0000", 2024-09-11T08:58:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9221e-05 2024-09-11T08:58:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945318+0000", 2024-09-11T08:58:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3699000000000001e-05 2024-09-11T08:58:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945336+0000", 2024-09-11T08:58:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7876999999999998e-05 2024-09-11T08:58:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:161955 9.15 9:aef2dade:::benchmark_data_smithi092_145083_object18557:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:58:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:58.968477+0000", 2024-09-11T08:58:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 471.98822783999998, 2024-09-11T08:58:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.97706887499999995, 2024-09-11T08:58:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 161955 2024-09-11T08:58:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968477+0000", 2024-09-11T08:58:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968477+0000", 2024-09-11T08:58:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968477+0000", 2024-09-11T08:58:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968479+0000", 2024-09-11T08:58:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5830000000000001e-06 2024-09-11T08:58:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968485+0000", 2024-09-11T08:58:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0410000000000003e-06 2024-09-11T08:58:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968489+0000", 2024-09-11T08:58:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4520000000000002e-06 2024-09-11T08:58:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968632+0000", 2024-09-11T08:58:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014362199999999999 2024-09-11T08:58:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:58.968639+0000", 2024-09-11T08:58:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8290000000000002e-06 2024-09-11T08:58:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.928106+0000", 2024-09-11T08:58:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.95946673199999999 2024-09-11T08:58:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.928211+0000", 2024-09-11T08:58:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000104817 2024-09-11T08:58:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:58:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.928417+0000", 2024-09-11T08:58:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00020603400000000001 2024-09-11T08:58:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.933865+0000", 2024-09-11T08:58:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0054485180000000003 2024-09-11T08:58:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945414+0000", 2024-09-11T08:58:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011548388999999999 2024-09-11T08:58:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945462+0000", 2024-09-11T08:58:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8201000000000001e-05 2024-09-11T08:58:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945511+0000", 2024-09-11T08:58:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9518999999999998e-05 2024-09-11T08:58:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945546+0000", 2024-09-11T08:58:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4551999999999997e-05 2024-09-11T08:58:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:161996 9.15 9:a871d7df:::benchmark_data_smithi092_145083_object18598:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:58:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:59.038626+0000", 2024-09-11T08:58:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 471.918078718, 2024-09-11T08:58:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90711806399999995, 2024-09-11T08:58:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 161996 2024-09-11T08:58:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038626+0000", 2024-09-11T08:58:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038626+0000", 2024-09-11T08:58:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038625+0000", 2024-09-11T08:58:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:58:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038633+0000", 2024-09-11T08:58:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.9629999999999998e-06 2024-09-11T08:58:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038645+0000", 2024-09-11T08:58:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2421e-05 2024-09-11T08:58:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038651+0000", 2024-09-11T08:58:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3399999999999997e-06 2024-09-11T08:58:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038936+0000", 2024-09-11T08:58:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00028560199999999998 2024-09-11T08:58:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.038943+0000", 2024-09-11T08:58:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.7309999999999996e-06 2024-09-11T08:58:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.929410+0000", 2024-09-11T08:58:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89046684799999998 2024-09-11T08:58:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.929473+0000", 2024-09-11T08:58:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3482e-05 2024-09-11T08:58:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:58:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.929657+0000", 2024-09-11T08:58:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00018351000000000001 2024-09-11T08:58:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.933869+0000", 2024-09-11T08:58:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0042121370000000003 2024-09-11T08:58:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945597+0000", 2024-09-11T08:58:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011727609999999999 2024-09-11T08:58:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945665+0000", 2024-09-11T08:58:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8650000000000002e-05 2024-09-11T08:58:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945708+0000", 2024-09-11T08:58:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2342000000000002e-05 2024-09-11T08:58:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945744+0000", 2024-09-11T08:58:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6569999999999997e-05 2024-09-11T08:58:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:161999 9.15 9:acecc340:::benchmark_data_smithi092_145083_object18601:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:58:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:59.040350+0000", 2024-09-11T08:58:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 471.91635483300001, 2024-09-11T08:58:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90555341600000006, 2024-09-11T08:58:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 161999 2024-09-11T08:58:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040350+0000", 2024-09-11T08:58:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040350+0000", 2024-09-11T08:58:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040349+0000", 2024-09-11T08:58:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:58:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040354+0000", 2024-09-11T08:58:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8829999999999997e-06 2024-09-11T08:58:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040366+0000", 2024-09-11T08:58:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.238e-05 2024-09-11T08:58:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040372+0000", 2024-09-11T08:58:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8139999999999997e-06 2024-09-11T08:58:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040516+0000", 2024-09-11T08:58:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000143313 2024-09-11T08:58:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.040520+0000", 2024-09-11T08:58:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4089999999999996e-06 2024-09-11T08:58:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.930467+0000", 2024-09-11T08:58:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.88994658900000001 2024-09-11T08:58:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.930588+0000", 2024-09-11T08:58:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012184 2024-09-11T08:58:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:58:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.930801+0000", 2024-09-11T08:58:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000212124 2024-09-11T08:58:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.937385+0000", 2024-09-11T08:58:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0065848369999999996 2024-09-11T08:58:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945800+0000", 2024-09-11T08:58:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0084145890000000001 2024-09-11T08:58:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945832+0000", 2024-09-11T08:58:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2202000000000001e-05 2024-09-11T08:58:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945872+0000", 2024-09-11T08:58:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0256000000000002e-05 2024-09-11T08:58:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945903+0000", 2024-09-11T08:58:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1078000000000003e-05 2024-09-11T08:58:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162018 9.15 9:ac3e5f86:::benchmark_data_smithi092_145083_object18620:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:58:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:59.052029+0000", 2024-09-11T08:58:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 471.90467610000002, 2024-09-11T08:58:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89400514099999995, 2024-09-11T08:58:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162018 2024-09-11T08:58:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052029+0000", 2024-09-11T08:58:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052029+0000", 2024-09-11T08:58:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052027+0000", 2024-09-11T08:58:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052034+0000", 2024-09-11T08:58:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3389999999999998e-06 2024-09-11T08:58:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052046+0000", 2024-09-11T08:58:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2517999999999999e-05 2024-09-11T08:58:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052056+0000", 2024-09-11T08:58:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.023e-05 2024-09-11T08:58:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052119+0000", 2024-09-11T08:58:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2984999999999994e-05 2024-09-11T08:58:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.052127+0000", 2024-09-11T08:58:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.0330000000000008e-06 2024-09-11T08:58:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.931615+0000", 2024-09-11T08:58:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.87948807200000001 2024-09-11T08:58:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.931701+0000", 2024-09-11T08:58:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.5131999999999997e-05 2024-09-11T08:58:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:58:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.931887+0000", 2024-09-11T08:58:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00018613399999999999 2024-09-11T08:58:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.937393+0000", 2024-09-11T08:58:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0055058060000000002 2024-09-11T08:58:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945937+0000", 2024-09-11T08:58:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0085445950000000003 2024-09-11T08:58:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.945969+0000", 2024-09-11T08:58:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2131000000000001e-05 2024-09-11T08:58:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946003+0000", 2024-09-11T08:58:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4224000000000003e-05 2024-09-11T08:58:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946034+0000", 2024-09-11T08:58:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0470000000000001e-05 2024-09-11T08:58:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162035 9.15 9:ac737bbe:::benchmark_data_smithi092_145083_object18637:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:58:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:59.065230+0000", 2024-09-11T08:58:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 471.89147510800001, 2024-09-11T08:58:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.88095609799999997, 2024-09-11T08:58:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162035 2024-09-11T08:58:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065230+0000", 2024-09-11T08:58:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:51.060 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:51.061 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065230+0000", 2024-09-11T08:58:51.061 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.061 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.061 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.061 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:51.061 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065229+0000", 2024-09-11T08:58:51.061 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065232+0000", 2024-09-11T08:58:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9340000000000002e-06 2024-09-11T08:58:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065239+0000", 2024-09-11T08:58:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.708e-06 2024-09-11T08:58:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065243+0000", 2024-09-11T08:58:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6430000000000001e-06 2024-09-11T08:58:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065366+0000", 2024-09-11T08:58:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000123511 2024-09-11T08:58:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.065372+0000", 2024-09-11T08:58:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1909999999999997e-06 2024-09-11T08:58:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.932683+0000", 2024-09-11T08:58:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.86731089299999997 2024-09-11T08:58:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.932780+0000", 2024-09-11T08:58:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.6373999999999996e-05 2024-09-11T08:58:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:58:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.933044+0000", 2024-09-11T08:58:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00026401899999999999 2024-09-11T08:58:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.939749+0000", 2024-09-11T08:58:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.006705529 2024-09-11T08:58:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946068+0000", 2024-09-11T08:58:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0063190540000000002 2024-09-11T08:58:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946106+0000", 2024-09-11T08:58:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7839000000000001e-05 2024-09-11T08:58:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946144+0000", 2024-09-11T08:58:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8343000000000002e-05 2024-09-11T08:58:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946186+0000", 2024-09-11T08:58:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1564999999999999e-05 2024-09-11T08:58:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162098 9.15 9:ab1b879d:::benchmark_data_smithi092_145083_object18700:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-11T08:58:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:50:59.112673+0000", 2024-09-11T08:58:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 471.84403209800001, 2024-09-11T08:58:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.83383003600000005, 2024-09-11T08:58:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162098 2024-09-11T08:58:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112673+0000", 2024-09-11T08:58:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112673+0000", 2024-09-11T08:58:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112672+0000", 2024-09-11T08:58:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:58:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112676+0000", 2024-09-11T08:58:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4020000000000004e-06 2024-09-11T08:58:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112685+0000", 2024-09-11T08:58:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.9390000000000003e-06 2024-09-11T08:58:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112689+0000", 2024-09-11T08:58:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5210000000000004e-06 2024-09-11T08:58:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112725+0000", 2024-09-11T08:58:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5176000000000003e-05 2024-09-11T08:58:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.112731+0000", 2024-09-11T08:58:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.088e-06 2024-09-11T08:58:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.933898+0000", 2024-09-11T08:58:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.82116687399999999 2024-09-11T08:58:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.933969+0000", 2024-09-11T08:58:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1109e-05 2024-09-11T08:58:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,3", 2024-09-11T08:58:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.934117+0000", 2024-09-11T08:58:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000148527 2024-09-11T08:58:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.939757+0000", 2024-09-11T08:58:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0056393290000000002 2024-09-11T08:58:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946218+0000", 2024-09-11T08:58:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0064610709999999997 2024-09-11T08:58:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946415+0000", 2024-09-11T08:58:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00019775899999999999 2024-09-11T08:58:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946453+0000", 2024-09-11T08:58:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8075999999999999e-05 2024-09-11T08:58:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:50:59.946503+0000", 2024-09-11T08:58:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9345999999999998e-05 2024-09-11T08:58:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162378 9.17 9:eb84acd7:::benchmark_data_smithi092_145083_object18980:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-11T08:58:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:51:00.001027+0000", 2024-09-11T08:58:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 470.955677756, 2024-09-11T08:58:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.95130415599999996, 2024-09-11T08:58:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162378 2024-09-11T08:58:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001027+0000", 2024-09-11T08:58:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001027+0000", 2024-09-11T08:58:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001027+0000", 2024-09-11T08:58:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001029+0000", 2024-09-11T08:58:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0640000000000001e-06 2024-09-11T08:58:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001035+0000", 2024-09-11T08:58:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1e-06 2024-09-11T08:58:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001039+0000", 2024-09-11T08:58:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7749999999999999e-06 2024-09-11T08:58:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001115+0000", 2024-09-11T08:58:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6666000000000001e-05 2024-09-11T08:58:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.001124+0000", 2024-09-11T08:58:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8990000000000007e-06 2024-09-11T08:58:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.933510+0000", 2024-09-11T08:58:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.93238578100000002 2024-09-11T08:58:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.933633+0000", 2024-09-11T08:58:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012321599999999999 2024-09-11T08:58:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:58:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.933908+0000", 2024-09-11T08:58:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00027423600000000002 2024-09-11T08:58:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.939240+0000", 2024-09-11T08:58:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0053320060000000002 2024-09-11T08:58:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952168+0000", 2024-09-11T08:58:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.012928439 2024-09-11T08:58:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952206+0000", 2024-09-11T08:58:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8479999999999997e-05 2024-09-11T08:58:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952254+0000", 2024-09-11T08:58:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7126999999999998e-05 2024-09-11T08:58:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952331+0000", 2024-09-11T08:58:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7713999999999995e-05 2024-09-11T08:58:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162412 9.17 9:ee197516:::benchmark_data_smithi092_145083_object19014:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-11T08:58:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:51:00.024372+0000", 2024-09-11T08:58:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 470.93233312900003, 2024-09-11T08:58:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.92838030800000004, 2024-09-11T08:58:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162412 2024-09-11T08:58:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024372+0000", 2024-09-11T08:58:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024372+0000", 2024-09-11T08:58:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024370+0000", 2024-09-11T08:58:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T08:58:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024377+0000", 2024-09-11T08:58:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1179999999999997e-06 2024-09-11T08:58:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024388+0000", 2024-09-11T08:58:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1260999999999999e-05 2024-09-11T08:58:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024393+0000", 2024-09-11T08:58:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5600000000000001e-06 2024-09-11T08:58:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024653+0000", 2024-09-11T08:58:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00025985299999999999 2024-09-11T08:58:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.024661+0000", 2024-09-11T08:58:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2760000000000003e-06 2024-09-11T08:58:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.935055+0000", 2024-09-11T08:58:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.91039429400000005 2024-09-11T08:58:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.935124+0000", 2024-09-11T08:58:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8872000000000001e-05 2024-09-11T08:58:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:58:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.935310+0000", 2024-09-11T08:58:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000185823 2024-09-11T08:58:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.939246+0000", 2024-09-11T08:58:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0039369030000000003 2024-09-11T08:58:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952414+0000", 2024-09-11T08:58:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013167935 2024-09-11T08:58:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952690+0000", 2024-09-11T08:58:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00027535 2024-09-11T08:58:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952724+0000", 2024-09-11T08:58:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4767000000000001e-05 2024-09-11T08:58:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952752+0000", 2024-09-11T08:58:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7591999999999999e-05 2024-09-11T08:58:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162425 9.17 9:ee96b337:::benchmark_data_smithi092_145083_object19027:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-11T08:58:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:51:00.030921+0000", 2024-09-11T08:58:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 470.92578411199997, 2024-09-11T08:58:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.921978466, 2024-09-11T08:58:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162425 2024-09-11T08:58:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030921+0000", 2024-09-11T08:58:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030921+0000", 2024-09-11T08:58:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030920+0000", 2024-09-11T08:58:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030923+0000", 2024-09-11T08:58:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.362e-06 2024-09-11T08:58:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030929+0000", 2024-09-11T08:58:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8769999999999998e-06 2024-09-11T08:58:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030932+0000", 2024-09-11T08:58:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.32e-06 2024-09-11T08:58:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030982+0000", 2024-09-11T08:58:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9997999999999998e-05 2024-09-11T08:58:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.030989+0000", 2024-09-11T08:58:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8190000000000003e-06 2024-09-11T08:58:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.936138+0000", 2024-09-11T08:58:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90514870999999997 2024-09-11T08:58:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.936278+0000", 2024-09-11T08:58:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014009 2024-09-11T08:58:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:58:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.936509+0000", 2024-09-11T08:58:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00023174599999999999 2024-09-11T08:58:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.941634+0000", 2024-09-11T08:58:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0051242279999999998 2024-09-11T08:58:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952645+0000", 2024-09-11T08:58:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011011099 2024-09-11T08:58:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952820+0000", 2024-09-11T08:58:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00017567400000000001 2024-09-11T08:58:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952863+0000", 2024-09-11T08:58:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3123e-05 2024-09-11T08:58:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952899+0000", 2024-09-11T08:58:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.578e-05 2024-09-11T08:58:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162450 9.17 9:e99b70b0:::benchmark_data_smithi092_145083_object19052:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-11T08:58:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:51:00.045692+0000", 2024-09-11T08:58:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 470.91101250000003, 2024-09-11T08:58:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.90749497800000001, 2024-09-11T08:58:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162450 2024-09-11T08:58:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045692+0000", 2024-09-11T08:58:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045692+0000", 2024-09-11T08:58:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045692+0000", 2024-09-11T08:58:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T08:58:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045696+0000", 2024-09-11T08:58:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0729999999999998e-06 2024-09-11T08:58:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045706+0000", 2024-09-11T08:58:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.9299999999999998e-06 2024-09-11T08:58:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045712+0000", 2024-09-11T08:58:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6829999999999996e-06 2024-09-11T08:58:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045759+0000", 2024-09-11T08:58:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7027999999999998e-05 2024-09-11T08:58:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.045766+0000", 2024-09-11T08:58:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9979999999999997e-06 2024-09-11T08:58:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.937223+0000", 2024-09-11T08:58:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89145709200000001 2024-09-11T08:58:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.937283+0000", 2024-09-11T08:58:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0676e-05 2024-09-11T08:58:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:58:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.937427+0000", 2024-09-11T08:58:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000143642 2024-09-11T08:58:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.941640+0000", 2024-09-11T08:58:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0042131299999999998 2024-09-11T08:58:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952942+0000", 2024-09-11T08:58:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011301774000000001 2024-09-11T08:58:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953099+0000", 2024-09-11T08:58:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015739500000000001 2024-09-11T08:58:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953141+0000", 2024-09-11T08:58:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1414e-05 2024-09-11T08:58:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953187+0000", 2024-09-11T08:58:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6687999999999998e-05 2024-09-11T08:58:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:162513 9.17 9:e993ae52:::benchmark_data_smithi092_145083_object19115:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-11T08:58:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:51:00.087447+0000", 2024-09-11T08:58:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 470.86925797499998, 2024-09-11T08:58:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.86587709199999996, 2024-09-11T08:58:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T08:58:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:58:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T08:58:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T08:58:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:58:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 162513 2024-09-11T08:58:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T08:58:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T08:58:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087447+0000", 2024-09-11T08:58:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T08:58:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087447+0000", 2024-09-11T08:58:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T08:58:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T08:58:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087446+0000", 2024-09-11T08:58:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T08:58:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T08:58:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087452+0000", 2024-09-11T08:58:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6439999999999997e-06 2024-09-11T08:58:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T08:58:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087466+0000", 2024-09-11T08:58:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3966e-05 2024-09-11T08:58:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T08:58:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087471+0000", 2024-09-11T08:58:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6849999999999998e-06 2024-09-11T08:58:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087515+0000", 2024-09-11T08:58:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3890000000000002e-05 2024-09-11T08:58:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T08:58:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.087522+0000", 2024-09-11T08:58:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5899999999999996e-06 2024-09-11T08:58:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T08:58:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.938122+0000", 2024-09-11T08:58:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.85060005800000005 2024-09-11T08:58:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T08:58:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.938185+0000", 2024-09-11T08:58:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2726999999999998e-05 2024-09-11T08:58:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,2", 2024-09-11T08:58:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.938349+0000", 2024-09-11T08:58:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000164535 2024-09-11T08:58:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T08:58:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.944372+0000", 2024-09-11T08:58:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0060229369999999999 2024-09-11T08:58:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.952983+0000", 2024-09-11T08:58:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0086107600000000003 2024-09-11T08:58:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:58:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953253+0000", 2024-09-11T08:58:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00027028600000000001 2024-09-11T08:58:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T08:58:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953290+0000", 2024-09-11T08:58:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6860000000000003e-05 2024-09-11T08:58:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T08:58:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T08:58:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T08:58:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:51:00.953324+0000", 2024-09-11T08:58:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4007999999999997e-05 2024-09-11T08:58:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T08:58:51.109 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T08:58:51.109 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T08:58:51.138 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:51.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:51.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:51.240 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:51.261 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:51.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:51.262 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:51.350 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 82 16 111261 111245 84.7701 69.8125 0.00847942 0.0117917 2024-09-11T08:58:51.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:51.393 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:51.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:51.394 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:51.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:51.531 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:51.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:51.533 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:51.633 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:51.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:51.670 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:51.670 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:51.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:51.806 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:51.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:51.807 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:51.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[29647]: 4.12 deep-scrub starts 2024-09-11T08:58:51.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[29647]: 4.12 deep-scrub ok 2024-09-11T08:58:51.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:58:51.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:58:51.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[29647]: 4.11 scrub starts 2024-09-11T08:58:51.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[29647]: 4.11 scrub ok 2024-09-11T08:58:51.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[29647]: 3.16 scrub starts 2024-09-11T08:58:51.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[29647]: 3.16 scrub ok 2024-09-11T08:58:51.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[29647]: 3.10 scrub starts 2024-09-11T08:58:51.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[29647]: 3.10 scrub ok 2024-09-11T08:58:51.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[29647]: 5.6 deep-scrub starts 2024-09-11T08:58:51.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[29647]: 5.6 deep-scrub ok 2024-09-11T08:58:51.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[29647]: osdmap e405: 8 total, 8 up, 8 in 2024-09-11T08:58:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[41114]: 4.12 deep-scrub starts 2024-09-11T08:58:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[41114]: 4.12 deep-scrub ok 2024-09-11T08:58:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:58:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:58:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[41114]: 4.11 scrub starts 2024-09-11T08:58:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[41114]: 4.11 scrub ok 2024-09-11T08:58:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[41114]: 3.16 scrub starts 2024-09-11T08:58:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[41114]: 3.16 scrub ok 2024-09-11T08:58:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[41114]: 3.10 scrub starts 2024-09-11T08:58:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[41114]: 3.10 scrub ok 2024-09-11T08:58:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[41114]: 5.6 deep-scrub starts 2024-09-11T08:58:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[41114]: 5.6 deep-scrub ok 2024-09-11T08:58:51.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-mon[41114]: osdmap e405: 8 total, 8 up, 8 in 2024-09-11T08:58:51.818 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:58:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:58:51] "GET /metrics HTTP/1.1" 200 225207 "" "Prometheus/2.43.0" 2024-09-11T08:58:51.909 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:51 smithi145 ceph-mon[33384]: 4.12 deep-scrub starts 2024-09-11T08:58:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:51 smithi145 ceph-mon[33384]: 4.12 deep-scrub ok 2024-09-11T08:58:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:51 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:58:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:51 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:58:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:51 smithi145 ceph-mon[33384]: 4.11 scrub starts 2024-09-11T08:58:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:51 smithi145 ceph-mon[33384]: 4.11 scrub ok 2024-09-11T08:58:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:51 smithi145 ceph-mon[33384]: 3.16 scrub starts 2024-09-11T08:58:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:51 smithi145 ceph-mon[33384]: 3.16 scrub ok 2024-09-11T08:58:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:51 smithi145 ceph-mon[33384]: 3.10 scrub starts 2024-09-11T08:58:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:51 smithi145 ceph-mon[33384]: 3.10 scrub ok 2024-09-11T08:58:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:51 smithi145 ceph-mon[33384]: 5.6 deep-scrub starts 2024-09-11T08:58:51.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:51 smithi145 ceph-mon[33384]: 5.6 deep-scrub ok 2024-09-11T08:58:51.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:51 smithi145 ceph-mon[33384]: osdmap e405: 8 total, 8 up, 8 in 2024-09-11T08:58:51.938 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:51.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:51.939 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:52.040 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:52.074 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:52.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:52.075 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:52.176 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:52.208 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:52.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:52.209 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:52.310 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:52.345 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:52.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:52.346 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:52.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-11T08:58:52.350 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 83 16 112502 112486 84.6832 77.5625 0.00755745 0.0118043 2024-09-11T08:58:52.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.2 reset_purged_snaps_last 2024-09-11T08:58:52.447 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:52.479 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:52.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:52.480 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:52.581 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:52.610 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:52.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:52.611 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:52.642 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[41114]: pgmap v927: 168 pgs: 168 active+clean; 6.7 GiB data, 28 GiB used, 688 GiB / 715 GiB avail; 54 KiB/s rd, 80 MiB/s wr, 1.37k op/s; 5.7 MiB/s, 91 objects/s recovering 2024-09-11T08:58:52.642 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[41114]: 3.12 scrub starts 2024-09-11T08:58:52.642 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[41114]: 3.12 scrub ok 2024-09-11T08:58:52.642 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[41114]: 5.11 deep-scrub starts 2024-09-11T08:58:52.642 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[41114]: 5.11 deep-scrub ok 2024-09-11T08:58:52.642 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[41114]: 4.7 scrub starts 2024-09-11T08:58:52.643 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[41114]: 4.7 scrub ok 2024-09-11T08:58:52.643 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T08:58:52.643 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[41114]: 5.d scrub ok 2024-09-11T08:58:52.643 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[41114]: 11.f scrub starts 2024-09-11T08:58:52.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:52.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:52.787 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:52.888 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:52.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[29647]: pgmap v927: 168 pgs: 168 active+clean; 6.7 GiB data, 28 GiB used, 688 GiB / 715 GiB avail; 54 KiB/s rd, 80 MiB/s wr, 1.37k op/s; 5.7 MiB/s, 91 objects/s recovering 2024-09-11T08:58:52.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[29647]: 3.12 scrub starts 2024-09-11T08:58:52.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[29647]: 3.12 scrub ok 2024-09-11T08:58:52.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[29647]: 5.11 deep-scrub starts 2024-09-11T08:58:52.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[29647]: 5.11 deep-scrub ok 2024-09-11T08:58:52.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[29647]: 4.7 scrub starts 2024-09-11T08:58:52.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[29647]: 4.7 scrub ok 2024-09-11T08:58:52.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T08:58:52.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[29647]: 5.d scrub ok 2024-09-11T08:58:52.896 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:52 smithi092 ceph-mon[29647]: 11.f scrub starts 2024-09-11T08:58:52.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:58:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:58:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:58:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:52 smithi145 ceph-mon[33384]: pgmap v927: 168 pgs: 168 active+clean; 6.7 GiB data, 28 GiB used, 688 GiB / 715 GiB avail; 54 KiB/s rd, 80 MiB/s wr, 1.37k op/s; 5.7 MiB/s, 91 objects/s recovering 2024-09-11T08:58:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:52 smithi145 ceph-mon[33384]: 3.12 scrub starts 2024-09-11T08:58:52.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:52 smithi145 ceph-mon[33384]: 3.12 scrub ok 2024-09-11T08:58:52.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:52 smithi145 ceph-mon[33384]: 5.11 deep-scrub starts 2024-09-11T08:58:52.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:52 smithi145 ceph-mon[33384]: 5.11 deep-scrub ok 2024-09-11T08:58:52.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:52 smithi145 ceph-mon[33384]: 4.7 scrub starts 2024-09-11T08:58:52.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:52 smithi145 ceph-mon[33384]: 4.7 scrub ok 2024-09-11T08:58:52.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:52 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T08:58:52.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:52 smithi145 ceph-mon[33384]: 5.d scrub ok 2024-09-11T08:58:52.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:52 smithi145 ceph-mon[33384]: 11.f scrub starts 2024-09-11T08:58:52.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:52.923 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:52.923 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:53.024 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:53.050 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:53.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:53.051 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:53.063 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:58:53.111 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:58:53.143 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:58:53.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:53.207 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:53.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:53.208 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:53.309 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:53.353 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 84 16 113513 113497 84.4252 63.1875 0.0133257 0.0118398 2024-09-11T08:58:53.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:53.360 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:53.361 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:53.461 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:53.501 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:53.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:53.502 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:53.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:53.638 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:53.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:53.639 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:53.663 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T08:58:53.740 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:53.775 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:53.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:53.776 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:53.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[29647]: 3.7 deep-scrub starts 2024-09-11T08:58:53.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[29647]: 3.7 deep-scrub ok 2024-09-11T08:58:53.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[29647]: 3.11 scrub starts 2024-09-11T08:58:53.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[29647]: 3.11 scrub ok 2024-09-11T08:58:53.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[29647]: 4.1f scrub starts 2024-09-11T08:58:53.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[29647]: 4.1f scrub ok 2024-09-11T08:58:53.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[29647]: 5.4 scrub starts 2024-09-11T08:58:53.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[29647]: 5.4 scrub ok 2024-09-11T08:58:53.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1878715647' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:58:53.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:58:53.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[41114]: 3.7 deep-scrub starts 2024-09-11T08:58:53.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[41114]: 3.7 deep-scrub ok 2024-09-11T08:58:53.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[41114]: 3.11 scrub starts 2024-09-11T08:58:53.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[41114]: 3.11 scrub ok 2024-09-11T08:58:53.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[41114]: 4.1f scrub starts 2024-09-11T08:58:53.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[41114]: 4.1f scrub ok 2024-09-11T08:58:53.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[41114]: 5.4 scrub starts 2024-09-11T08:58:53.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[41114]: 5.4 scrub ok 2024-09-11T08:58:53.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1878715647' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:58:53.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:53 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:58:53.877 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:53.907 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:53.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:53.909 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:53.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:53 smithi145 ceph-mon[33384]: 3.7 deep-scrub starts 2024-09-11T08:58:53.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:53 smithi145 ceph-mon[33384]: 3.7 deep-scrub ok 2024-09-11T08:58:53.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:53 smithi145 ceph-mon[33384]: 3.11 scrub starts 2024-09-11T08:58:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:53 smithi145 ceph-mon[33384]: 3.11 scrub ok 2024-09-11T08:58:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:53 smithi145 ceph-mon[33384]: 4.1f scrub starts 2024-09-11T08:58:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:53 smithi145 ceph-mon[33384]: 4.1f scrub ok 2024-09-11T08:58:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:53 smithi145 ceph-mon[33384]: 5.4 scrub starts 2024-09-11T08:58:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:53 smithi145 ceph-mon[33384]: 5.4 scrub ok 2024-09-11T08:58:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:53 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1878715647' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:58:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:53 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T08:58:54.010 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:54.042 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:54.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:54.043 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:54.143 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:54.183 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:54.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:54.185 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:54.285 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:54.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:54.320 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:54.320 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:54.353 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 85 16 114818 114802 84.3915 81.5625 0.0133381 0.0118447 2024-09-11T08:58:54.421 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:54.451 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:54.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:54.452 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:54.553 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:54.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:54.585 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:54.586 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:54.686 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:54.720 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:54.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:54.721 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:54.822 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:54.860 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:54.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:54.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:54.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:54 smithi145 ceph-mon[33384]: pgmap v928: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.8 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 58 KiB/s rd, 47 MiB/s wr, 854 op/s; 1.7 MiB/s, 27 objects/s recovering 2024-09-11T08:58:54.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:54 smithi145 ceph-mon[33384]: 1.0 scrub starts 2024-09-11T08:58:54.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:54 smithi145 ceph-mon[33384]: 1.0 scrub ok 2024-09-11T08:58:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:54 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:58:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:54 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:58:54.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:54 smithi145 ceph-mon[33384]: osdmap e406: 8 total, 8 up, 8 in 2024-09-11T08:58:54.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:55.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:54 smithi092 ceph-mon[41114]: pgmap v928: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.8 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 58 KiB/s rd, 47 MiB/s wr, 854 op/s; 1.7 MiB/s, 27 objects/s recovering 2024-09-11T08:58:55.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:54 smithi092 ceph-mon[41114]: 1.0 scrub starts 2024-09-11T08:58:55.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:54 smithi092 ceph-mon[41114]: 1.0 scrub ok 2024-09-11T08:58:55.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:54 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:58:55.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:54 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:58:55.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:54 smithi092 ceph-mon[41114]: osdmap e406: 8 total, 8 up, 8 in 2024-09-11T08:58:55.007 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:54 smithi092 ceph-mon[29647]: pgmap v928: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.8 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 58 KiB/s rd, 47 MiB/s wr, 854 op/s; 1.7 MiB/s, 27 objects/s recovering 2024-09-11T08:58:55.007 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:54 smithi092 ceph-mon[29647]: 1.0 scrub starts 2024-09-11T08:58:55.007 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:54 smithi092 ceph-mon[29647]: 1.0 scrub ok 2024-09-11T08:58:55.007 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:54 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:58:55.008 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:54 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T08:58:55.008 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:54 smithi092 ceph-mon[29647]: osdmap e406: 8 total, 8 up, 8 in 2024-09-11T08:58:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:55.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:55.014 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:55.115 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:55.145 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:55.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:55.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:55.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:55.284 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:55.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:55.285 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:55.353 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 86 16 116150 116134 84.3781 83.25 0.0109663 0.0118463 2024-09-11T08:58:55.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:55.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:55.428 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:55.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:55.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:55.573 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:55.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:55.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:55.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:55.814 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:55.849 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:55.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:55.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:55.951 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:55.976 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:55.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:55.978 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:56.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:56.123 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:56.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:56.125 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:56.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:56.267 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:56.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:56.268 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:56.353 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 87 16 117456 117440 84.3464 81.625 0.00846581 0.0118513 2024-09-11T08:58:56.369 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:56.408 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:56.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:56.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:56.510 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:56.543 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:56.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:56.544 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:56.644 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:56.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:56.681 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:56.681 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:56.782 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:56.810 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:56.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:56.811 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:56 smithi145 ceph-mon[33384]: pgmap v930: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.8 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 55 KiB/s rd, 45 MiB/s wr, 809 op/s; 1.6 MiB/s, 25 objects/s recovering 2024-09-11T08:58:56.912 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:56.946 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:56.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:56.948 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:57.028 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:56 smithi092 ceph-mon[29647]: pgmap v930: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.8 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 55 KiB/s rd, 45 MiB/s wr, 809 op/s; 1.6 MiB/s, 25 objects/s recovering 2024-09-11T08:58:57.028 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:56 smithi092 ceph-mon[41114]: pgmap v930: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.8 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 55 KiB/s rd, 45 MiB/s wr, 809 op/s; 1.6 MiB/s, 25 objects/s recovering 2024-09-11T08:58:57.049 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:57.086 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:57.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:57.087 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:57.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:57.227 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:57.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:57.228 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:57.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:57.353 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 88 16 118835 118819 84.3671 86.1875 0.0125196 0.011848 2024-09-11T08:58:57.364 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:57.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:57.365 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:57.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:57.504 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:57.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:57.507 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:57.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:57.644 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:57.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:57.645 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:57.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:58:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:58:57.746 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:57.768 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T08:58:57.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:57.781 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:57.781 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:57.882 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:57.907 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:57.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:57.908 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:58.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:58.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:58.058 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:58.058 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:58.159 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:58:58.196 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:58.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:58.198 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:58:58.298 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:58.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:58.334 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:58.335 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:58.353 INFO:tasks.radosbench.radosbench.0.smithi092.stdout: 89 16 120068 120052 84.285 77.0625 0.0131347 0.0118603 2024-09-11T08:58:58.436 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:58:58.468 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:58.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:58.469 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:58:58.570 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:58.610 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:58.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:58.612 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:58.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:58.754 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:58.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:58.755 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:58.856 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:58.893 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:58.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:58.894 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:58.945 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:58 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:58.946 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:58 smithi092 ceph-mon[29647]: pgmap v931: 168 pgs: 168 active+clean; 7.2 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 44 KiB/s rd, 84 MiB/s wr, 1.42k op/s; 1.3 MiB/s, 20 objects/s recovering 2024-09-11T08:58:58.946 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:58 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:58.946 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:58 smithi092 ceph-mon[41114]: pgmap v931: 168 pgs: 168 active+clean; 7.2 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 44 KiB/s rd, 84 MiB/s wr, 1.42k op/s; 1.3 MiB/s, 20 objects/s recovering 2024-09-11T08:58:58.995 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:58:59.020 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:58:59.020 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:58:59.027 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:59.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:59.028 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:58:59.063 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:58:59.063 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:58:59.096 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:58:59.096 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:58:59.129 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:58:59.129 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:58:59.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:58:59.160 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:58:59.160 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:58:59.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:58 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:58:59.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:58 smithi145 ceph-mon[33384]: pgmap v931: 168 pgs: 168 active+clean; 7.2 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 44 KiB/s rd, 84 MiB/s wr, 1.42k op/s; 1.3 MiB/s, 20 objects/s recovering 2024-09-11T08:58:59.165 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:59.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:59.166 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:58:59.195 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:58:59.195 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:58:59.233 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:58:59.233 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:58:59.267 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:58:59.272 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:58:59.273 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:58:59.299 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:58:59.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:59.300 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:58:59.353 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Total time run: 90.0042 2024-09-11T08:58:59.354 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Total writes made: 121318 2024-09-11T08:58:59.354 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Write size: 65536 2024-09-11T08:58:59.354 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Object size: 65536 2024-09-11T08:58:59.354 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Bandwidth (MB/sec): 84.2447 2024-09-11T08:58:59.354 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Stddev Bandwidth: 43.2781 2024-09-11T08:58:59.354 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Max bandwidth (MB/sec): 228.5 2024-09-11T08:58:59.354 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Min bandwidth (MB/sec): 0 2024-09-11T08:58:59.354 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Average IOPS: 1347 2024-09-11T08:58:59.354 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Stddev IOPS: 692.46 2024-09-11T08:58:59.354 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Max IOPS: 3656 2024-09-11T08:58:59.354 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Min IOPS: 0 2024-09-11T08:58:59.354 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Average Latency(s): 0.0118663 2024-09-11T08:58:59.354 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Stddev Latency(s): 0.0349238 2024-09-11T08:58:59.354 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Max latency(s): 2.29632 2024-09-11T08:58:59.355 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Min latency(s): 0.0011062 2024-09-11T08:58:59.358 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Cleaning up (deleting benchmark objects) 2024-09-11T08:58:59.401 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:59.436 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:59.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:59.437 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:59.538 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:59.573 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:59.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:59.574 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:58:59.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:58:59.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:59.718 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:58:59.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:58:59 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2423227271' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:58:59.746 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:58:59 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2423227271' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:58:59.792 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:58:59.792 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:58:59.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:58:59.857 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:59.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:59.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:58:59.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:58:59.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:58:59.996 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:58:59.996 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:00.057 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:58:59 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2423227271' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:59:00.097 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:00.131 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:00.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:00.133 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:00.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:00.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:00.269 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:00.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:00.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:00.411 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:00.411 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:00.512 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:00.546 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:00.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:00.547 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:00.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:00.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:00.684 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:00.684 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:00.742 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-11T08:59:00.785 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:00.822 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:00.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:00.823 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:00 smithi145 ceph-mon[33384]: pgmap v932: 168 pgs: 168 active+clean; 7.2 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 42 KiB/s rd, 81 MiB/s wr, 1.37k op/s; 1.2 MiB/s, 19 objects/s recovering 2024-09-11T08:59:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:00 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:59:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:59:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:00 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:59:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:00 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:59:00.924 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:00.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:00.959 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:00.959 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:01.060 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:01.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:00 smithi092 ceph-mon[29647]: pgmap v932: 168 pgs: 168 active+clean; 7.2 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 42 KiB/s rd, 81 MiB/s wr, 1.37k op/s; 1.2 MiB/s, 19 objects/s recovering 2024-09-11T08:59:01.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:00 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:59:01.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:59:01.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:00 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:59:01.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:00 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:59:01.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:00 smithi092 ceph-mon[41114]: pgmap v932: 168 pgs: 168 active+clean; 7.2 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 42 KiB/s rd, 81 MiB/s wr, 1.37k op/s; 1.2 MiB/s, 19 objects/s recovering 2024-09-11T08:59:01.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:00 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:59:01.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:59:01.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:00 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:59:01.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:00 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:59:01.093 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:01.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:01.094 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:01.195 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:01.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:01.226 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:01.226 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:01.327 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:01.368 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:01.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:01.369 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:01.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:01.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:01.505 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:01.505 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:01.606 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:01.639 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:01.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:01.640 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:01.741 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:01.770 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:01.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:01.772 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:01.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:59:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:59:01] "GET /metrics HTTP/1.1" 200 225187 "" "Prometheus/2.43.0" 2024-09-11T08:59:01.873 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:01.904 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:01.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:01.905 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:02.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:02.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.3 reset_purged_snaps_last 2024-09-11T08:59:02.044 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:02.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:02.045 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:02.137 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:02.146 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:02.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:02.174 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:02.174 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:02.218 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:02.275 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:02.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:02 smithi092 ceph-mon[29647]: osdmap e407: 8 total, 8 up, 8 in 2024-09-11T08:59:02.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:02 smithi092 ceph-mon[29647]: pgmap v934: 168 pgs: 168 active+clean; 7.2 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.4 KiB/s rd, 62 MiB/s wr, 1.13k op/s 2024-09-11T08:59:02.291 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:02 smithi092 ceph-mon[41114]: osdmap e407: 8 total, 8 up, 8 in 2024-09-11T08:59:02.291 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:02 smithi092 ceph-mon[41114]: pgmap v934: 168 pgs: 168 active+clean; 7.2 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.4 KiB/s rd, 62 MiB/s wr, 1.13k op/s 2024-09-11T08:59:02.301 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:02.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:02.302 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:02.403 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:02.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:02 smithi145 ceph-mon[33384]: osdmap e407: 8 total, 8 up, 8 in 2024-09-11T08:59:02.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:02 smithi145 ceph-mon[33384]: pgmap v934: 168 pgs: 168 active+clean; 7.2 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.4 KiB/s rd, 62 MiB/s wr, 1.13k op/s 2024-09-11T08:59:02.433 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:02.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:02.434 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:02.535 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:02.568 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:02.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:02.569 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:02.670 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:02.702 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:02.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:02.704 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:02.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:02.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:02.845 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:02.946 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:02.980 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:02.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:02.981 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:03.082 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:03.121 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:03.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:03.122 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:03.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:59:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:59:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:59:03.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:59:03.223 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:03.253 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:03.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:03.254 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:03.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:03.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:03.403 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:03.504 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:03.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:03.536 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:03.536 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:03.637 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:03.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:03.672 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:03.672 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:03.773 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:03.810 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:03.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:03.811 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:03.912 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:03.947 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:03.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:03.949 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:04.050 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:04.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:04.083 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:04.084 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:04.184 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:04.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:04.222 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:04.223 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:04.323 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:04.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:04.360 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:04.360 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:04.461 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:04.489 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:04.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:04.490 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:04.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:04.620 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:04.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:04.621 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:04.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:04.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:04.756 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:04.756 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:04.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:04 smithi092 ceph-mon[29647]: pgmap v935: 168 pgs: 168 active+clean; 7.2 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.1 KiB/s rd, 62 MiB/s wr, 1.30k op/s 2024-09-11T08:59:04.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:04 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:59:04.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:04 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:59:04.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:04 smithi092 ceph-mon[29647]: osdmap e408: 8 total, 8 up, 8 in 2024-09-11T08:59:04.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:04 smithi092 ceph-mon[41114]: pgmap v935: 168 pgs: 168 active+clean; 7.2 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.1 KiB/s rd, 62 MiB/s wr, 1.30k op/s 2024-09-11T08:59:04.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:04 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:59:04.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:04 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:59:04.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:04 smithi092 ceph-mon[41114]: osdmap e408: 8 total, 8 up, 8 in 2024-09-11T08:59:04.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:04.892 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:04.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:04.893 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:04 smithi145 ceph-mon[33384]: pgmap v935: 168 pgs: 168 active+clean; 7.2 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.1 KiB/s rd, 62 MiB/s wr, 1.30k op/s 2024-09-11T08:59:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:04 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:59:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:04 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:59:04.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:04 smithi145 ceph-mon[33384]: osdmap e408: 8 total, 8 up, 8 in 2024-09-11T08:59:04.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:05.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:05.027 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:05.128 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:05.162 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:05.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:05.163 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:05.264 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:05.298 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:05.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:05.299 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:05.400 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:05.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:05.435 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:05.435 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:05.536 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:05.576 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:05.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:05.577 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:05.678 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:05.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:05.710 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:05.711 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:05.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:05.843 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:05.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:05.845 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:05.945 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:06.005 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:06.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:06.006 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:06.108 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:06.144 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:06.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:06.146 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:06.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:06.285 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:06.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:06.287 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:06.387 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:06.417 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:06.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:06.418 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:06.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:06.540 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:06.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:06.541 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:06.642 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:06.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:06.667 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:06.667 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:06.767 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:06.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:06.801 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:06.802 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:06.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:06 smithi092 ceph-mon[29647]: pgmap v937: 168 pgs: 168 active+clean; 7.2 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 767 B/s rd, 27 MiB/s wr, 810 op/s 2024-09-11T08:59:06.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:06 smithi092 ceph-mon[41114]: pgmap v937: 168 pgs: 168 active+clean; 7.2 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 767 B/s rd, 27 MiB/s wr, 810 op/s 2024-09-11T08:59:06.903 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:06.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:06 smithi145 ceph-mon[33384]: pgmap v937: 168 pgs: 168 active+clean; 7.2 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 767 B/s rd, 27 MiB/s wr, 810 op/s 2024-09-11T08:59:06.937 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:06.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:06.938 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:07.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:07.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:07.063 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:07.164 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:07.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:07.196 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:07.196 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:07.297 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:07.330 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:07.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:07.331 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:07.431 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:07.463 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:07.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:07.465 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:07.565 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:07.586 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:07.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:07.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:07.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:59:07 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:59:07.688 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:07.724 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:07.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:07.725 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:07.826 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:07.847 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:07.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:07.849 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:07.949 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:07.981 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:07.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:07.982 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:08.083 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:08.110 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:08.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:08.111 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:08.212 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:08.241 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:08.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:08.242 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:08.343 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:08.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:08.380 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:08.380 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:08.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:08.522 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:08.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:08.523 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:08.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:08.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:08.676 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:08.676 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:08.777 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:08.803 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:08.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:08.804 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:08.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:08 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:08.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:08 smithi092 ceph-mon[29647]: pgmap v938: 168 pgs: 168 active+clean; 6.8 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.4 KiB/s rd, 27 MiB/s wr, 1.61k op/s 2024-09-11T08:59:08.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:08 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:59:08.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:08 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:59:08.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:08 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:08.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:08 smithi092 ceph-mon[41114]: pgmap v938: 168 pgs: 168 active+clean; 6.8 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.4 KiB/s rd, 27 MiB/s wr, 1.61k op/s 2024-09-11T08:59:08.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:08 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:59:08.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:08 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:59:08.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:08 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:08 smithi145 ceph-mon[33384]: pgmap v938: 168 pgs: 168 active+clean; 6.8 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.4 KiB/s rd, 27 MiB/s wr, 1.61k op/s 2024-09-11T08:59:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:08 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:59:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:08 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:59:08.948 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:08.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:08.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:09.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:09.054 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:09.139 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:09.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:09.140 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:09.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:09.281 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:09.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:09.283 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:09.383 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:09.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:09.419 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:09.419 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:09.520 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:09.554 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:09.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:09.558 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:09.625 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:09 smithi092 ceph-mon[29647]: osdmap e409: 8 total, 8 up, 8 in 2024-09-11T08:59:09.625 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:09 smithi092 ceph-mon[29647]: osdmap e410: 8 total, 8 up, 8 in 2024-09-11T08:59:09.625 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:09 smithi092 ceph-mon[41114]: osdmap e409: 8 total, 8 up, 8 in 2024-09-11T08:59:09.626 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:09 smithi092 ceph-mon[41114]: osdmap e410: 8 total, 8 up, 8 in 2024-09-11T08:59:09.659 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:09.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:09.687 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:09.688 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:09.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:09.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:09.822 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:09.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:09 smithi145 ceph-mon[33384]: osdmap e409: 8 total, 8 up, 8 in 2024-09-11T08:59:09.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:09 smithi145 ceph-mon[33384]: osdmap e410: 8 total, 8 up, 8 in 2024-09-11T08:59:09.923 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:09.959 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:09.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:09.961 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:10.061 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:10.092 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:10.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:10.093 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:10.193 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:10.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:10.227 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:10.228 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:10.329 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:10.359 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:10.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:10.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:10.461 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:10.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:10.498 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:10.498 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:10.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:10.636 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:10.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:10.638 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:10.739 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:10.771 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:10.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:10.772 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:10 smithi092 ceph-mon[29647]: pgmap v941: 168 pgs: 168 active+clean; 6.8 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T08:59:10.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:10 smithi092 ceph-mon[41114]: pgmap v941: 168 pgs: 168 active+clean; 6.8 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T08:59:10.873 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:10.899 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:10.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:10.900 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:10.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:10 smithi145 ceph-mon[33384]: pgmap v941: 168 pgs: 168 active+clean; 6.8 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T08:59:11.002 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:11.033 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:11.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:11.034 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:11.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:11.170 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:11.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:11.171 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:11.272 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:11.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:11.295 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:11.295 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:11.396 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:11.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:11.418 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:11.418 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:11.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:11.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:11.559 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:11.560 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:11.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:11.681 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:11.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:11.682 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:11.780 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:59:11.781 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:59:11.781 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:59:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:59:11] "GET /metrics HTTP/1.1" 200 225187 "" "Prometheus/2.43.0" 2024-09-11T08:59:11.783 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:11.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:11.821 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:11.921 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:11.955 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:11.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:11.956 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:12.057 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:12.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:12.089 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:12.089 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:12.190 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:12.222 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:12.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:12.223 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:12.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:12.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:12.361 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:12.361 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:12.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:12.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:12.503 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:12.503 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:12.604 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:12.635 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:12.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:12.636 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:12.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:12 smithi092 ceph-mon[29647]: pgmap v942: 168 pgs: 168 active+clean; 6.7 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T08:59:12.684 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:12 smithi092 ceph-mon[41114]: pgmap v942: 168 pgs: 168 active+clean; 6.7 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T08:59:12.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:12.772 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:12.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:12.773 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:12.874 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:12.898 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:12.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:12.899 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:12.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:59:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:59:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:59:12.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:12 smithi145 ceph-mon[33384]: pgmap v942: 168 pgs: 168 active+clean; 6.7 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T08:59:13.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:13.037 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:13.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:13.038 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:13.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:13.174 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:13.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:13.175 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:13.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-11T08:59:13.276 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:13.303 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:13.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:13.304 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:13.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:13.448 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:13.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:13.449 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:13.550 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:13.574 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:13.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:13.575 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:13.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:13.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:13.720 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:13.720 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:13.821 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:13.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:13.848 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:13.848 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:13.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:13.981 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:13.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:13.982 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:14.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:14.121 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:14.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:14.122 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:14.223 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:14.263 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:14.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:14.264 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:14.365 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:14.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:14.395 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:14.395 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:14.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:14.531 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:14.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:14.533 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:14.549 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T08:59:14.633 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:14.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:14.662 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:14.662 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:14.763 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:14.795 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:14.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:14.796 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:14.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:14 smithi092 ceph-mon[29647]: pgmap v943: 168 pgs: 168 active+clean; 6.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-11T08:59:14.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:14 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4198622625' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:59:14.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:14 smithi092 ceph-mon[41114]: pgmap v943: 168 pgs: 168 active+clean; 6.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-11T08:59:14.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:14 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4198622625' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:59:14.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:14.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:14 smithi145 ceph-mon[33384]: pgmap v943: 168 pgs: 168 active+clean; 6.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-11T08:59:14.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:14 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4198622625' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:59:14.932 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:14.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:14.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:15.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:15.066 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:15.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:15.067 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:15.168 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:15.206 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:15.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:15.207 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:15.308 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:15.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:15.337 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:15.338 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:15.438 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:15.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:15.466 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:15.467 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:15.568 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:15.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:15.598 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:15.599 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:15.626 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:15 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:59:15.626 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:15 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4198622625' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:59:15.627 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:15 smithi092 ceph-mon[29647]: osdmap e411: 8 total, 8 up, 8 in 2024-09-11T08:59:15.627 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:59:15.627 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:15 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:59:15.627 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:15 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4198622625' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:59:15.627 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:15 smithi092 ceph-mon[41114]: osdmap e411: 8 total, 8 up, 8 in 2024-09-11T08:59:15.627 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:59:15.672 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:15.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:15.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:15.744 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:15.745 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:15.845 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:15.871 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:15.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:15.872 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:15.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:15 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:59:15.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:15 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4198622625' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T08:59:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:15 smithi145 ceph-mon[33384]: osdmap e411: 8 total, 8 up, 8 in 2024-09-11T08:59:15.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:59:15.973 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:16.003 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:16.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:16.004 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:16.105 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:16.145 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:16.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:16.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:16.247 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:16.275 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:16.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:16.276 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:16.377 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:16.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:16.412 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:16.412 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:16.414 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:16.513 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:16.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:16.546 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:16.546 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:16.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:16.679 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:16.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:16.680 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:16.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:16.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:16 smithi092 ceph-mon[29647]: pgmap v945: 168 pgs: 168 active+clean; 6.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 738 B/s rd, 0 B/s wr, 912 op/s 2024-09-11T08:59:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:16 smithi092 ceph-mon[41114]: pgmap v945: 168 pgs: 168 active+clean; 6.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 738 B/s rd, 0 B/s wr, 912 op/s 2024-09-11T08:59:16.818 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:16.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:16.819 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:16.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:16 smithi145 ceph-mon[33384]: pgmap v945: 168 pgs: 168 active+clean; 6.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 738 B/s rd, 0 B/s wr, 912 op/s 2024-09-11T08:59:16.920 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:16.954 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:16.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:16.955 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:17.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.4 reset_purged_snaps_last 2024-09-11T08:59:17.056 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:17.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:17.088 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:17.089 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:17.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:17.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:17.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:17.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:17.381 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:17.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:17.383 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:17.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T08:59:17.484 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:17.530 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:17.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:17.531 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:17.579 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:59:17 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:59:17.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:17.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:17.672 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:17.747 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:17 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [3, 5]}]: dispatch 2024-09-11T08:59:17.748 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:17 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.18", "id": [1, 2]}]: dispatch 2024-09-11T08:59:17.748 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:17 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [3, 5]}]: dispatch 2024-09-11T08:59:17.748 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:17 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.18", "id": [1, 2]}]: dispatch 2024-09-11T08:59:17.749 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:17 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [3, 5]}]: dispatch 2024-09-11T08:59:17.749 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:17 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.18", "id": [1, 2]}]: dispatch 2024-09-11T08:59:17.749 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:17 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [3, 5]}]: dispatch 2024-09-11T08:59:17.749 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:17 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.18", "id": [1, 2]}]: dispatch 2024-09-11T08:59:17.773 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:17.808 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:17.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:17.809 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:17.910 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:17.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:17 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [3, 5]}]: dispatch 2024-09-11T08:59:17.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:17 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.18", "id": [1, 2]}]: dispatch 2024-09-11T08:59:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:17 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [3, 5]}]: dispatch 2024-09-11T08:59:17.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:17 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.18", "id": [1, 2]}]: dispatch 2024-09-11T08:59:17.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:17.943 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:17.944 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:18.045 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:18.073 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:18.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:18.074 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:18.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:18.177 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:18.235 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:18.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:18.236 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:18.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:18.387 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:18.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:18.388 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:18.489 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:18.524 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:18.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:18.525 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:18.619 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:18 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:18.619 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:18 smithi092 ceph-mon[29647]: pgmap v946: 168 pgs: 168 active+clean; 6.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.54k op/s 2024-09-11T08:59:18.620 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:18 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [3, 5]}]': finished 2024-09-11T08:59:18.620 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:18 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.18", "id": [1, 2]}]': finished 2024-09-11T08:59:18.620 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:18 smithi092 ceph-mon[29647]: osdmap e412: 8 total, 8 up, 8 in 2024-09-11T08:59:18.620 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:18 smithi092 ceph-mon[29647]: osdmap e413: 8 total, 8 up, 8 in 2024-09-11T08:59:18.620 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:18 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:18.620 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:18 smithi092 ceph-mon[41114]: pgmap v946: 168 pgs: 168 active+clean; 6.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.54k op/s 2024-09-11T08:59:18.620 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:18 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [3, 5]}]': finished 2024-09-11T08:59:18.620 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:18 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.18", "id": [1, 2]}]': finished 2024-09-11T08:59:18.621 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:18 smithi092 ceph-mon[41114]: osdmap e412: 8 total, 8 up, 8 in 2024-09-11T08:59:18.621 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:18 smithi092 ceph-mon[41114]: osdmap e413: 8 total, 8 up, 8 in 2024-09-11T08:59:18.626 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:18.651 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:18.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:18.652 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:18.753 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:18.795 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:18.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:18.796 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:18.897 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:18.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:18 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:18 smithi145 ceph-mon[33384]: pgmap v946: 168 pgs: 168 active+clean; 6.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.54k op/s 2024-09-11T08:59:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:18 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [3, 5]}]': finished 2024-09-11T08:59:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:18 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.18", "id": [1, 2]}]': finished 2024-09-11T08:59:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:18 smithi145 ceph-mon[33384]: osdmap e412: 8 total, 8 up, 8 in 2024-09-11T08:59:18.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:18 smithi145 ceph-mon[33384]: osdmap e413: 8 total, 8 up, 8 in 2024-09-11T08:59:18.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:18.933 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:18.934 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:19.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:19.077 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:19.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:19.078 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:19.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:19.212 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:19.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:19.213 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:19.314 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:19.351 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:19.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:19.352 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:19.452 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:19.481 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:19.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:19.482 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:19.502 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:59:19.502 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:59:19.544 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:59:19.544 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:59:19.574 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:59:19.574 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:59:19.583 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:19.604 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:59:19.604 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:59:19.619 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:19.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:19.621 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:19.633 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:59:19.633 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:59:19.662 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:59:19.663 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:59:19.693 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:59:19.693 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:59:19.721 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:19.726 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:59:19.726 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:59:19.750 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:19.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:19.751 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:19.852 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:19.877 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:19.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:19.878 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:19.979 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:20.010 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:20.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:20.011 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:20.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:20.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:20.152 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:20.253 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:20.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:20.281 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:20.281 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:20.382 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:20.406 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:20.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:20.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:20.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:20 smithi145 ceph-mon[33384]: pgmap v949: 168 pgs: 168 active+clean; 6.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T08:59:20.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:20 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1310178662' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:59:20.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:20.545 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:20.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:20.546 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:20.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:20 smithi092 ceph-mon[29647]: pgmap v949: 168 pgs: 168 active+clean; 6.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T08:59:20.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:20 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1310178662' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:59:20.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:20 smithi092 ceph-mon[41114]: pgmap v949: 168 pgs: 168 active+clean; 6.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T08:59:20.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:20 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1310178662' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:59:20.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:20.682 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:20.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:20.683 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:20.784 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:20.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:20.814 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:20.815 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:20.915 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:20.940 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:20.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:20.941 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:21.042 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:21.075 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:21.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:21.076 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:21.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:21.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:21.213 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:21.213 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:21.314 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:21.342 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:21.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:21.343 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:21.444 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:21.475 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:21.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:21.476 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:21.577 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:21.614 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:21.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:21.615 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:21.716 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:21.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:21.748 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:21.748 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:21.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:59:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:59:21] "GET /metrics HTTP/1.1" 200 225157 "" "Prometheus/2.43.0" 2024-09-11T08:59:21.850 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:21.888 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:21.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:21.889 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:21.990 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:22.023 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:22.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:22.024 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:22.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:22.162 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:22.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:22.163 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:22.263 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:22.290 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:22.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:22.291 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:22.391 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:22.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:22 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:59:22.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:22 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:59:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:22 smithi145 ceph-mon[33384]: osdmap e414: 8 total, 8 up, 8 in 2024-09-11T08:59:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:22 smithi145 ceph-mon[33384]: pgmap v951: 168 pgs: 168 active+clean; 5.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.57k op/s 2024-09-11T08:59:22.424 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:22.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:22.425 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:22.525 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:22.559 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:22.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:22.560 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:22 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:59:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:22 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:59:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:22 smithi092 ceph-mon[29647]: osdmap e414: 8 total, 8 up, 8 in 2024-09-11T08:59:22.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:22 smithi092 ceph-mon[29647]: pgmap v951: 168 pgs: 168 active+clean; 5.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.57k op/s 2024-09-11T08:59:22.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:22 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:59:22.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:22 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:59:22.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:22 smithi092 ceph-mon[41114]: osdmap e414: 8 total, 8 up, 8 in 2024-09-11T08:59:22.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:22 smithi092 ceph-mon[41114]: pgmap v951: 168 pgs: 168 active+clean; 5.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.57k op/s 2024-09-11T08:59:22.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:22.692 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:22.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:22.693 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:22.793 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:22.821 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:22.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:22.823 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:22.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:22.959 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:22.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:22.960 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:23.061 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:23.089 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:23.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:23.090 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:23.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:59:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:59:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:59:23.192 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:23.221 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:23.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:23.222 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:23.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:23.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:23.359 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:23.359 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:23.460 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:23.490 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:23.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:23.491 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:23.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:23.625 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:23.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:23.626 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:23.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:23.768 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:23.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:23.769 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:23.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:23.906 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:23.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:23.907 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:24.008 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:24.038 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:24.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:24.039 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:24.039 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:24.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:24.188 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:24.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:24.189 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:24.290 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:24.328 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:24.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:24.329 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:24.430 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:24.462 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:24.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:24.463 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:24.564 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:24.601 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:24.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:24.602 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:24.703 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:24.737 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:24 smithi092 ceph-mon[29647]: pgmap v952: 168 pgs: 168 active+clean; 5.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 1.08k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:59:24.737 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:24 smithi092 ceph-mon[41114]: pgmap v952: 168 pgs: 168 active+clean; 5.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 1.08k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:59:24.743 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:24.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:24.744 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:24.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:24.885 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:24.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:24.886 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:24.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:24 smithi145 ceph-mon[33384]: pgmap v952: 168 pgs: 168 active+clean; 5.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 1.08k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:59:24.987 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:25.017 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:25.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:25.018 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:25.119 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:25.151 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:25.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:25.152 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:25.253 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:25.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:25.285 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:25.286 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:25.387 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:25.424 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:25.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:25.425 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:25.526 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:25.552 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:25.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:25.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:25.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:25.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:25.696 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:25.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:25.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:25.833 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:25.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:25.970 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:25.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:25.971 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:26.072 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:26.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:59:26.101 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:26.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:26.102 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:26.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:26.235 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:26.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:26.236 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:26.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:26.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:26.378 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:26.379 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:26.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:26.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:26 smithi092 ceph-mon[29647]: pgmap v953: 168 pgs: 168 active+clean; 5.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 519 B/s rd, 0 B/s wr, 822 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:59:26.529 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:26.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:26.530 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:26.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:26.665 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:26.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:26.666 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:26.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.5 reset_purged_snaps_last 2024-09-11T08:59:26.766 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:26.794 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:26.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:26.795 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:26.797 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:26 smithi092 ceph-mon[41114]: pgmap v953: 168 pgs: 168 active+clean; 5.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 519 B/s rd, 0 B/s wr, 822 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:59:26.895 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:26.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:26 smithi145 ceph-mon[33384]: pgmap v953: 168 pgs: 168 active+clean; 5.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 519 B/s rd, 0 B/s wr, 822 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:59:26.928 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:26.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:26.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:27.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:27.082 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:27.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:27.084 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:27.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:27.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:27.228 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:27.228 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:27.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:27.361 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:27.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:27.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:27.462 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:27.489 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:27.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:27.491 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:27.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:27.627 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:27.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:27.629 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:27.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:59:27 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:59:27.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:27.762 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:27.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:27.763 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:27.864 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:27.899 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:27.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:27.901 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:28.001 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:28.024 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:28.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:28.025 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:28.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:28.169 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:28.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:28.170 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:28.255 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:59:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-11T08:59:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-11T08:59:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T08:59:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272053 9.5 9:a00bb8b6:::benchmark_data_smithi092_145083_object128655:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T08:59:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:23.616160+0000", 2024-09-11T08:59:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 424.63710296099998, 2024-09-11T08:59:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0208401620000001, 2024-09-11T08:59:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:59:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:59:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:59:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272053 2024-09-11T08:59:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616160+0000", 2024-09-11T08:59:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616160+0000", 2024-09-11T08:59:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616158+0000", 2024-09-11T08:59:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:59:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616165+0000", 2024-09-11T08:59:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7240000000000003e-06 2024-09-11T08:59:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616179+0000", 2024-09-11T08:59:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3740999999999999e-05 2024-09-11T08:59:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616185+0000", 2024-09-11T08:59:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9529999999999996e-06 2024-09-11T08:59:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616245+0000", 2024-09-11T08:59:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0341999999999999e-05 2024-09-11T08:59:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:59:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:23.616257+0000", 2024-09-11T08:59:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1643e-05 2024-09-11T08:59:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.630442+0000", 2024-09-11T08:59:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.014185804 2024-09-11T08:59:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.630590+0000", 2024-09-11T08:59:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014730400000000001 2024-09-11T08:59:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,7", 2024-09-11T08:59:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.630711+0000", 2024-09-11T08:59:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012151599999999999 2024-09-11T08:59:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.636837+0000", 2024-09-11T08:59:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061256540000000003 2024-09-11T08:59:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.636891+0000", 2024-09-11T08:59:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3708999999999997e-05 2024-09-11T08:59:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:59:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.636903+0000", 2024-09-11T08:59:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2651e-05 2024-09-11T08:59:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.636959+0000", 2024-09-11T08:59:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6181000000000003e-05 2024-09-11T08:59:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.637000+0000", 2024-09-11T08:59:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0586000000000001e-05 2024-09-11T08:59:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272343 9.b 9:deb81e83:::benchmark_data_smithi092_145083_object128945:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:59:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.633667+0000", 2024-09-11T08:59:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 423.619596022, 2024-09-11T08:59:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0859924860000001, 2024-09-11T08:59:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:59:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:59:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:59:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272343 2024-09-11T08:59:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633667+0000", 2024-09-11T08:59:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633667+0000", 2024-09-11T08:59:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633665+0000", 2024-09-11T08:59:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:59:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633716+0000", 2024-09-11T08:59:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0563999999999999e-05 2024-09-11T08:59:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633726+0000", 2024-09-11T08:59:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7629999999999995e-06 2024-09-11T08:59:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633730+0000", 2024-09-11T08:59:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4610000000000001e-06 2024-09-11T08:59:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633789+0000", 2024-09-11T08:59:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9329000000000001e-05 2024-09-11T08:59:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:59:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.633797+0000", 2024-09-11T08:59:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7049999999999999e-06 2024-09-11T08:59:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.707365+0000", 2024-09-11T08:59:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.073567543 2024-09-11T08:59:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.707459+0000", 2024-09-11T08:59:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4926999999999994e-05 2024-09-11T08:59:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:59:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.707557+0000", 2024-09-11T08:59:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7717e-05 2024-09-11T08:59:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:59:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.714212+0000", 2024-09-11T08:59:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066545939999999998 2024-09-11T08:59:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719510+0000", 2024-09-11T08:59:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005298395 2024-09-11T08:59:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719541+0000", 2024-09-11T08:59:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0528999999999999e-05 2024-09-11T08:59:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719606+0000", 2024-09-11T08:59:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5350000000000003e-05 2024-09-11T08:59:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719659+0000", 2024-09-11T08:59:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3238000000000001e-05 2024-09-11T08:59:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272372 9.b 9:dabfe76a:::benchmark_data_smithi092_145083_object128974:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:59:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.660754+0000", 2024-09-11T08:59:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 423.59250904700002, 2024-09-11T08:59:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.059110105, 2024-09-11T08:59:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:59:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:59:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:59:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272372 2024-09-11T08:59:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660754+0000", 2024-09-11T08:59:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660754+0000", 2024-09-11T08:59:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660752+0000", 2024-09-11T08:59:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:59:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660759+0000", 2024-09-11T08:59:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9929999999999998e-06 2024-09-11T08:59:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660778+0000", 2024-09-11T08:59:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.855e-05 2024-09-11T08:59:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660785+0000", 2024-09-11T08:59:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0960000000000003e-06 2024-09-11T08:59:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660850+0000", 2024-09-11T08:59:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5032999999999999e-05 2024-09-11T08:59:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:59:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.660857+0000", 2024-09-11T08:59:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7219999999999999e-06 2024-09-11T08:59:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.709617+0000", 2024-09-11T08:59:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.048759955 2024-09-11T08:59:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.709729+0000", 2024-09-11T08:59:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111554 2024-09-11T08:59:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:59:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.709875+0000", 2024-09-11T08:59:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000145597 2024-09-11T08:59:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:59:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.716482+0000", 2024-09-11T08:59:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066076379999999999 2024-09-11T08:59:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719695+0000", 2024-09-11T08:59:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032126479999999998 2024-09-11T08:59:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719780+0000", 2024-09-11T08:59:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4653000000000004e-05 2024-09-11T08:59:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719820+0000", 2024-09-11T08:59:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0395999999999997e-05 2024-09-11T08:59:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719864+0000", 2024-09-11T08:59:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3959999999999999e-05 2024-09-11T08:59:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272378 9.b 9:da18d24a:::benchmark_data_smithi092_145083_object128980:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:59:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.664541+0000", 2024-09-11T08:59:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 423.58872198699999, 2024-09-11T08:59:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0630863310000001, 2024-09-11T08:59:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:59:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:59:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:59:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272378 2024-09-11T08:59:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664541+0000", 2024-09-11T08:59:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664541+0000", 2024-09-11T08:59:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664539+0000", 2024-09-11T08:59:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:59:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664547+0000", 2024-09-11T08:59:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1960000000000008e-06 2024-09-11T08:59:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664568+0000", 2024-09-11T08:59:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0577999999999998e-05 2024-09-11T08:59:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664574+0000", 2024-09-11T08:59:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7619999999999998e-06 2024-09-11T08:59:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664646+0000", 2024-09-11T08:59:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1848999999999996e-05 2024-09-11T08:59:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:59:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.664656+0000", 2024-09-11T08:59:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6889999999999999e-06 2024-09-11T08:59:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.711155+0000", 2024-09-11T08:59:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0464992740000001 2024-09-11T08:59:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.711226+0000", 2024-09-11T08:59:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1174999999999995e-05 2024-09-11T08:59:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:59:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.711339+0000", 2024-09-11T08:59:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011281399999999999 2024-09-11T08:59:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:59:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.716490+0000", 2024-09-11T08:59:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051504020000000001 2024-09-11T08:59:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719723+0000", 2024-09-11T08:59:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032333890000000001 2024-09-11T08:59:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727554+0000", 2024-09-11T08:59:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0078313399999999991 2024-09-11T08:59:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727595+0000", 2024-09-11T08:59:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0383000000000003e-05 2024-09-11T08:59:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727627+0000", 2024-09-11T08:59:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2435999999999999e-05 2024-09-11T08:59:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272400 9.b 9:d843832b:::benchmark_data_smithi092_145083_object129002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:59:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.678382+0000", 2024-09-11T08:59:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 423.57488110899999, 2024-09-11T08:59:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.049380773, 2024-09-11T08:59:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:59:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:59:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:59:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272400 2024-09-11T08:59:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678382+0000", 2024-09-11T08:59:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678382+0000", 2024-09-11T08:59:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678380+0000", 2024-09-11T08:59:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:59:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678386+0000", 2024-09-11T08:59:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5840000000000002e-06 2024-09-11T08:59:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678401+0000", 2024-09-11T08:59:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4601e-05 2024-09-11T08:59:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678407+0000", 2024-09-11T08:59:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3409999999999999e-06 2024-09-11T08:59:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678457+0000", 2024-09-11T08:59:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9512000000000003e-05 2024-09-11T08:59:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:59:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.678465+0000", 2024-09-11T08:59:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2870000000000004e-06 2024-09-11T08:59:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.712993+0000", 2024-09-11T08:59:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0345277070000001 2024-09-11T08:59:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.713060+0000", 2024-09-11T08:59:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7471999999999994e-05 2024-09-11T08:59:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:59:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.713135+0000", 2024-09-11T08:59:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5100999999999998e-05 2024-09-11T08:59:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:59:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719168+0000", 2024-09-11T08:59:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060324829999999999 2024-09-11T08:59:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727669+0000", 2024-09-11T08:59:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085014749999999997 2024-09-11T08:59:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727695+0000", 2024-09-11T08:59:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6137000000000001e-05 2024-09-11T08:59:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727732+0000", 2024-09-11T08:59:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6380000000000001e-05 2024-09-11T08:59:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727762+0000", 2024-09-11T08:59:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0917000000000002e-05 2024-09-11T08:59:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272419 9.b 9:dbacabf2:::benchmark_data_smithi092_145083_object129021:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T08:59:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.733096+0000", 2024-09-11T08:59:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 423.520167141, 2024-09-11T08:59:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99494694900000002, 2024-09-11T08:59:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:59:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T08:59:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T08:59:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272419 2024-09-11T08:59:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733096+0000", 2024-09-11T08:59:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733096+0000", 2024-09-11T08:59:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733094+0000", 2024-09-11T08:59:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:59:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733101+0000", 2024-09-11T08:59:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4499999999999998e-06 2024-09-11T08:59:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733119+0000", 2024-09-11T08:59:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7252999999999999e-05 2024-09-11T08:59:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733125+0000", 2024-09-11T08:59:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5629999999999997e-06 2024-09-11T08:59:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733204+0000", 2024-09-11T08:59:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9022000000000003e-05 2024-09-11T08:59:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:59:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.733286+0000", 2024-09-11T08:59:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1546999999999997e-05 2024-09-11T08:59:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.714254+0000", 2024-09-11T08:59:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98096810099999998 2024-09-11T08:59:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.714361+0000", 2024-09-11T08:59:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010746100000000001 2024-09-11T08:59:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T08:59:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.714441+0000", 2024-09-11T08:59:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9348e-05 2024-09-11T08:59:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:59:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.719175+0000", 2024-09-11T08:59:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047343079999999996 2024-09-11T08:59:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727872+0000", 2024-09-11T08:59:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0086973250000000005 2024-09-11T08:59:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.727961+0000", 2024-09-11T08:59:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8913999999999996e-05 2024-09-11T08:59:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.298 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:28.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.728006+0000", 2024-09-11T08:59:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4579999999999997e-05 2024-09-11T08:59:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.728043+0000", 2024-09-11T08:59:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.676e-05 2024-09-11T08:59:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:115615 10.5 10:a6dfdb06:::benchmark_data_smithi092_173109_object115614:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T08:59:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:54:16.126276+0000", 2024-09-11T08:59:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 312.126986993, 2024-09-11T08:59:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0208769630000001, 2024-09-11T08:59:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:59:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:59:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:59:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 115615 2024-09-11T08:59:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.126276+0000", 2024-09-11T08:59:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.126276+0000", 2024-09-11T08:59:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.126274+0000", 2024-09-11T08:59:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:59:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.126299+0000", 2024-09-11T08:59:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4856000000000001e-05 2024-09-11T08:59:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.126316+0000", 2024-09-11T08:59:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7295999999999998e-05 2024-09-11T08:59:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.126322+0000", 2024-09-11T08:59:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2530000000000001e-06 2024-09-11T08:59:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.126899+0000", 2024-09-11T08:59:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00057623500000000003 2024-09-11T08:59:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:59:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.126909+0000", 2024-09-11T08:59:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0162e-05 2024-09-11T08:59:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.126905+0000", 2024-09-11T08:59:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99999626399999997 2024-09-11T08:59:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.128959+0000", 2024-09-11T08:59:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020535079999999999 2024-09-11T08:59:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,3", 2024-09-11T08:59:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.129247+0000", 2024-09-11T08:59:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028803500000000002 2024-09-11T08:59:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.146906+0000", 2024-09-11T08:59:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017658831999999999 2024-09-11T08:59:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:59:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.146913+0000", 2024-09-11T08:59:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1110000000000002e-06 2024-09-11T08:59:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.147058+0000", 2024-09-11T08:59:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014540000000000001 2024-09-11T08:59:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.147113+0000", 2024-09-11T08:59:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5442999999999998e-05 2024-09-11T08:59:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.147153+0000", 2024-09-11T08:59:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9337e-05 2024-09-11T08:59:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:115624 10.5 10:a8e1fd8e:::benchmark_data_smithi092_173109_object115623:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T08:59:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:54:16.128087+0000", 2024-09-11T08:59:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 312.125176001, 2024-09-11T08:59:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.019222251, 2024-09-11T08:59:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:59:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:59:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:59:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 115624 2024-09-11T08:59:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.128087+0000", 2024-09-11T08:59:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.128087+0000", 2024-09-11T08:59:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.128085+0000", 2024-09-11T08:59:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:59:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.128108+0000", 2024-09-11T08:59:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2795000000000001e-05 2024-09-11T08:59:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.128125+0000", 2024-09-11T08:59:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6843000000000001e-05 2024-09-11T08:59:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.128130+0000", 2024-09-11T08:59:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6890000000000001e-06 2024-09-11T08:59:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.128666+0000", 2024-09-11T08:59:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00053514800000000005 2024-09-11T08:59:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:59:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.128674+0000", 2024-09-11T08:59:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6079999999999996e-06 2024-09-11T08:59:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.137500+0000", 2024-09-11T08:59:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0088259429999999 2024-09-11T08:59:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.137625+0000", 2024-09-11T08:59:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000124828 2024-09-11T08:59:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,3", 2024-09-11T08:59:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.137764+0000", 2024-09-11T08:59:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013914099999999999 2024-09-11T08:59:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:59:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.146948+0000", 2024-09-11T08:59:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0091838549999999994 2024-09-11T08:59:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.147027+0000", 2024-09-11T08:59:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9042000000000007e-05 2024-09-11T08:59:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.147219+0000", 2024-09-11T08:59:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019186 2024-09-11T08:59:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.147280+0000", 2024-09-11T08:59:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0738e-05 2024-09-11T08:59:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.147309+0000", 2024-09-11T08:59:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9516e-05 2024-09-11T08:59:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:115635 10.5 10:a494612f:::benchmark_data_smithi092_173109_object115634:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T08:59:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:54:16.140012+0000", 2024-09-11T08:59:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 312.11325123500001, 2024-09-11T08:59:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0076486010000001, 2024-09-11T08:59:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:59:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:59:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:59:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 115635 2024-09-11T08:59:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.140012+0000", 2024-09-11T08:59:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.140012+0000", 2024-09-11T08:59:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.140010+0000", 2024-09-11T08:59:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:59:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.140034+0000", 2024-09-11T08:59:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3977000000000001e-05 2024-09-11T08:59:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.140050+0000", 2024-09-11T08:59:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6662000000000001e-05 2024-09-11T08:59:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.140058+0000", 2024-09-11T08:59:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0430000000000002e-06 2024-09-11T08:59:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.140114+0000", 2024-09-11T08:59:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6292999999999998e-05 2024-09-11T08:59:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T08:59:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:16.140135+0000", 2024-09-11T08:59:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0876000000000001e-05 2024-09-11T08:59:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.140679+0000", 2024-09-11T08:59:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0005445749999999 2024-09-11T08:59:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.140803+0000", 2024-09-11T08:59:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012416799999999999 2024-09-11T08:59:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,3", 2024-09-11T08:59:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.140957+0000", 2024-09-11T08:59:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015373200000000001 2024-09-11T08:59:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:59:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.146994+0000", 2024-09-11T08:59:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060364160000000002 2024-09-11T08:59:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.147189+0000", 2024-09-11T08:59:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019536999999999999 2024-09-11T08:59:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.147581+0000", 2024-09-11T08:59:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00039193000000000002 2024-09-11T08:59:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.147626+0000", 2024-09-11T08:59:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5346999999999998e-05 2024-09-11T08:59:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:54:17.147660+0000", 2024-09-11T08:59:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3967000000000002e-05 2024-09-11T08:59:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:180206 10.5 10:a20b1f22:::benchmark_data_smithi092_173109_object43247:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-11T08:59:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:55:10.331214+0000", 2024-09-11T08:59:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 257.92204921400003, 2024-09-11T08:59:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.25704467399999997, 2024-09-11T08:59:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:59:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:59:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:59:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 180206 2024-09-11T08:59:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.331214+0000", 2024-09-11T08:59:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.331214+0000", 2024-09-11T08:59:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.331212+0000", 2024-09-11T08:59:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T08:59:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.331219+0000", 2024-09-11T08:59:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0990000000000001e-06 2024-09-11T08:59:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.331235+0000", 2024-09-11T08:59:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5616000000000001e-05 2024-09-11T08:59:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.331240+0000", 2024-09-11T08:59:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.575e-06 2024-09-11T08:59:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.331485+0000", 2024-09-11T08:59:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024503600000000002 2024-09-11T08:59:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.331766+0000", 2024-09-11T08:59:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028036400000000001 2024-09-11T08:59:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,3", 2024-09-11T08:59:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.332114+0000", 2024-09-11T08:59:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034843800000000001 2024-09-11T08:59:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:59:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.336311+0000", 2024-09-11T08:59:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041972989999999998 2024-09-11T08:59:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.337542+0000", 2024-09-11T08:59:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001230428 2024-09-11T08:59:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.588156+0000", 2024-09-11T08:59:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.25061372799999998 2024-09-11T08:59:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.588213+0000", 2024-09-11T08:59:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7095000000000001e-05 2024-09-11T08:59:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:55:10.588258+0000", 2024-09-11T08:59:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5671999999999999e-05 2024-09-11T08:59:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:245437 10.5 10:abaaab65:::benchmark_data_smithi092_173109_object108478:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e346)", 2024-09-11T08:59:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:08.882078+0000", 2024-09-11T08:59:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 199.371184535, 2024-09-11T08:59:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12820742600000001, 2024-09-11T08:59:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:59:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:59:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:59:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 245437 2024-09-11T08:59:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.882078+0000", 2024-09-11T08:59:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.882078+0000", 2024-09-11T08:59:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.882077+0000", 2024-09-11T08:59:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:59:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.882082+0000", 2024-09-11T08:59:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9309999999999999e-06 2024-09-11T08:59:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.882091+0000", 2024-09-11T08:59:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2609999999999999e-06 2024-09-11T08:59:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.882096+0000", 2024-09-11T08:59:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2490000000000003e-06 2024-09-11T08:59:28.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.995880+0000", 2024-09-11T08:59:28.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.113784596 2024-09-11T08:59:28.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.995981+0000", 2024-09-11T08:59:28.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100546 2024-09-11T08:59:28.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,3", 2024-09-11T08:59:28.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.996084+0000", 2024-09-11T08:59:28.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103491 2024-09-11T08:59:28.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.999264+0000", 2024-09-11T08:59:28.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031795249999999999 2024-09-11T08:59:28.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:59:28.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:09.004234+0000", 2024-09-11T08:59:28.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004970169 2024-09-11T08:59:28.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:09.010181+0000", 2024-09-11T08:59:28.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059472070000000004 2024-09-11T08:59:28.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:09.010244+0000", 2024-09-11T08:59:28.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3059999999999996e-05 2024-09-11T08:59:28.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:09.010286+0000", 2024-09-11T08:59:28.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1619000000000003e-05 2024-09-11T08:59:28.334 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.334 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.334 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.334 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:245438 10.5 10:a87cc065:::benchmark_data_smithi092_173109_object108479:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e346)", 2024-09-11T08:59:28.334 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:08.884503+0000", 2024-09-11T08:59:28.334 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 199.36876004000001, 2024-09-11T08:59:28.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12589113199999999, 2024-09-11T08:59:28.334 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.334 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.334 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T08:59:28.334 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T08:59:28.335 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T08:59:28.335 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 245438 2024-09-11T08:59:28.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.335 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.884503+0000", 2024-09-11T08:59:28.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.884503+0000", 2024-09-11T08:59:28.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.884502+0000", 2024-09-11T08:59:28.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:59:28.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.884508+0000", 2024-09-11T08:59:28.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5209999999999999e-06 2024-09-11T08:59:28.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.993454+0000", 2024-09-11T08:59:28.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.108945526 2024-09-11T08:59:28.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.993462+0000", 2024-09-11T08:59:28.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5909999999999996e-06 2024-09-11T08:59:28.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.999587+0000", 2024-09-11T08:59:28.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061248170000000003 2024-09-11T08:59:28.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.999714+0000", 2024-09-11T08:59:28.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012717300000000001 2024-09-11T08:59:28.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,3", 2024-09-11T08:59:28.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.999846+0000", 2024-09-11T08:59:28.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000131977 2024-09-11T08:59:28.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:09.003203+0000", 2024-09-11T08:59:28.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003356969 2024-09-11T08:59:28.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T08:59:28.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:09.007475+0000", 2024-09-11T08:59:28.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042719899999999998 2024-09-11T08:59:28.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T08:59:28.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:09.010322+0000", 2024-09-11T08:59:28.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028471529999999998 2024-09-11T08:59:28.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:09.010360+0000", 2024-09-11T08:59:28.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8164999999999998e-05 2024-09-11T08:59:28.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:09.010394+0000", 2024-09-11T08:59:28.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3497000000000001e-05 2024-09-11T08:59:28.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.340 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.340 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15918.0:41007 11.1 e377/366 11:8214b659:::benchmark_data_smithi092_205252_object41006:head v 377'2607, mlcod=377'2605)", 2024-09-11T08:59:28.340 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:46.469587+0000", 2024-09-11T08:59:28.340 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 101.783676083, 2024-09-11T08:59:28.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.168520636, 2024-09-11T08:59:28.340 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.341 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.342 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.469587+0000", 2024-09-11T08:59:28.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.469587+0000", 2024-09-11T08:59:28.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.469586+0000", 2024-09-11T08:59:28.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:59:28.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.469700+0000", 2024-09-11T08:59:28.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011459900000000001 2024-09-11T08:59:28.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.469712+0000", 2024-09-11T08:59:28.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1482e-05 2024-09-11T08:59:28.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.469718+0000", 2024-09-11T08:59:28.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1569999999999997e-06 2024-09-11T08:59:28.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.469794+0000", 2024-09-11T08:59:28.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5492000000000002e-05 2024-09-11T08:59:28.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.469818+0000", 2024-09-11T08:59:28.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4564999999999999e-05 2024-09-11T08:59:28.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.471270+0000", 2024-09-11T08:59:28.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014521720000000001 2024-09-11T08:59:28.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.638107+0000", 2024-09-11T08:59:28.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.166837071 2024-09-11T08:59:28.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.345 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.346 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.346 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15918.0:41012 11.b e377/366 11:d3df7b3b:::benchmark_data_smithi092_205252_object41011:head v 377'2572, mlcod=377'2570)", 2024-09-11T08:59:28.346 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:46.471104+0000", 2024-09-11T08:59:28.346 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 101.78215919, 2024-09-11T08:59:28.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.171140771, 2024-09-11T08:59:28.346 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.346 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.346 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.471104+0000", 2024-09-11T08:59:28.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.471104+0000", 2024-09-11T08:59:28.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.471103+0000", 2024-09-11T08:59:28.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T08:59:28.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.471122+0000", 2024-09-11T08:59:28.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9176000000000001e-05 2024-09-11T08:59:28.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.471131+0000", 2024-09-11T08:59:28.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3520000000000007e-06 2024-09-11T08:59:28.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.471135+0000", 2024-09-11T08:59:28.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4059999999999998e-06 2024-09-11T08:59:28.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.638192+0000", 2024-09-11T08:59:28.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.167057434 2024-09-11T08:59:28.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.638216+0000", 2024-09-11T08:59:28.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4117999999999998e-05 2024-09-11T08:59:28.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.642192+0000", 2024-09-11T08:59:28.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039754689999999997 2024-09-11T08:59:28.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.642244+0000", 2024-09-11T08:59:28.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2457000000000003e-05 2024-09-11T08:59:28.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.350 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.350 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15918.0:41019 11.b e377/366 11:df106f02:::benchmark_data_smithi092_205252_object41018:head v 377'2573, mlcod=377'2570)", 2024-09-11T08:59:28.350 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:46.476161+0000", 2024-09-11T08:59:28.350 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 101.77710132999999, 2024-09-11T08:59:28.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16731412500000001, 2024-09-11T08:59:28.351 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.351 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.351 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.476161+0000", 2024-09-11T08:59:28.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.476161+0000", 2024-09-11T08:59:28.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.476160+0000", 2024-09-11T08:59:28.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:59:28.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.476252+0000", 2024-09-11T08:59:28.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1681999999999994e-05 2024-09-11T08:59:28.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.638128+0000", 2024-09-11T08:59:28.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16187626899999999 2024-09-11T08:59:28.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.638136+0000", 2024-09-11T08:59:28.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.481e-06 2024-09-11T08:59:28.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.639845+0000", 2024-09-11T08:59:28.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017089850000000001 2024-09-11T08:59:28.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.639862+0000", 2024-09-11T08:59:28.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7164e-05 2024-09-11T08:59:28.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.643444+0000", 2024-09-11T08:59:28.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035826090000000001 2024-09-11T08:59:28.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.643476+0000", 2024-09-11T08:59:28.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1265e-05 2024-09-11T08:59:28.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.355 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.355 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15918.0:41022 11.6 e377/366 11:63196349:::benchmark_data_smithi092_205252_object41021:head v 377'2530, mlcod=377'2528)", 2024-09-11T08:59:28.355 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:46.477582+0000", 2024-09-11T08:59:28.355 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 101.775681185, 2024-09-11T08:59:28.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16330604400000001, 2024-09-11T08:59:28.355 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.355 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.355 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.477582+0000", 2024-09-11T08:59:28.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.477582+0000", 2024-09-11T08:59:28.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.477580+0000", 2024-09-11T08:59:28.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:59:28.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.477634+0000", 2024-09-11T08:59:28.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3436999999999997e-05 2024-09-11T08:59:28.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.638134+0000", 2024-09-11T08:59:28.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16049980899999999 2024-09-11T08:59:28.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.638140+0000", 2024-09-11T08:59:28.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5459999999999997e-06 2024-09-11T08:59:28.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.638394+0000", 2024-09-11T08:59:28.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025410100000000001 2024-09-11T08:59:28.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.638415+0000", 2024-09-11T08:59:28.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0414000000000001e-05 2024-09-11T08:59:28.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.640836+0000", 2024-09-11T08:59:28.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024214649999999998 2024-09-11T08:59:28.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:46.640888+0000", 2024-09-11T08:59:28.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1479e-05 2024-09-11T08:59:28.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.359 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.359 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15918.0:148252 11.b e413/366 11:d18531c4:::benchmark_data_smithi092_205252_object26931:head v 413'9150, mlcod=413'9148)", 2024-09-11T08:59:28.359 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:59:20.248814+0000", 2024-09-11T08:59:28.359 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.0044487170000007, 2024-09-11T08:59:28.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.14128896299999999, 2024-09-11T08:59:28.360 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.360 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.360 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.248814+0000", 2024-09-11T08:59:28.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.248814+0000", 2024-09-11T08:59:28.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.248813+0000", 2024-09-11T08:59:28.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T08:59:28.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.248820+0000", 2024-09-11T08:59:28.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8450000000000003e-06 2024-09-11T08:59:28.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.248839+0000", 2024-09-11T08:59:28.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8244999999999999e-05 2024-09-11T08:59:28.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.248845+0000", 2024-09-11T08:59:28.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0569999999999998e-06 2024-09-11T08:59:28.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.248901+0000", 2024-09-11T08:59:28.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6419000000000003e-05 2024-09-11T08:59:28.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.248920+0000", 2024-09-11T08:59:28.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8839e-05 2024-09-11T08:59:28.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.252715+0000", 2024-09-11T08:59:28.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037950979999999998 2024-09-11T08:59:28.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.390103+0000", 2024-09-11T08:59:28.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.13738801 2024-09-11T08:59:28.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.363 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.364 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15918.0:148256 11.2 e413/366 11:41c50033:::benchmark_data_smithi092_205252_object26935:head v 413'9316, mlcod=413'9314)", 2024-09-11T08:59:28.364 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:59:20.251039+0000", 2024-09-11T08:59:28.364 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.0022235009999996, 2024-09-11T08:59:28.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.146583343, 2024-09-11T08:59:28.364 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.364 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.364 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.251039+0000", 2024-09-11T08:59:28.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.251039+0000", 2024-09-11T08:59:28.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.251039+0000", 2024-09-11T08:59:28.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:59:28.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.251046+0000", 2024-09-11T08:59:28.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6899999999999992e-06 2024-09-11T08:59:28.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.251056+0000", 2024-09-11T08:59:28.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9539999999999999e-06 2024-09-11T08:59:28.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.251062+0000", 2024-09-11T08:59:28.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5929999999999996e-06 2024-09-11T08:59:28.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.251100+0000", 2024-09-11T08:59:28.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8602000000000001e-05 2024-09-11T08:59:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.251120+0000", 2024-09-11T08:59:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9939e-05 2024-09-11T08:59:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.397567+0000", 2024-09-11T08:59:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.146447155 2024-09-11T08:59:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.397623+0000", 2024-09-11T08:59:28.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5208999999999999e-05 2024-09-11T08:59:28.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.368 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.368 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15918.0:148258 11.1 e413/400 11:86a3724d:::benchmark_data_smithi092_205252_object26937:head v 413'9292, mlcod=413'9290)", 2024-09-11T08:59:28.368 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:59:20.253783+0000", 2024-09-11T08:59:28.368 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.9994795090000004, 2024-09-11T08:59:28.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.14219041599999999, 2024-09-11T08:59:28.368 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.368 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.368 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.253783+0000", 2024-09-11T08:59:28.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.253783+0000", 2024-09-11T08:59:28.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.253782+0000", 2024-09-11T08:59:28.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:59:28.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.253790+0000", 2024-09-11T08:59:28.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4019999999999996e-06 2024-09-11T08:59:28.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.390075+0000", 2024-09-11T08:59:28.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.13628553400000001 2024-09-11T08:59:28.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.390086+0000", 2024-09-11T08:59:28.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0562e-05 2024-09-11T08:59:28.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.390131+0000", 2024-09-11T08:59:28.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5512000000000001e-05 2024-09-11T08:59:28.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.390153+0000", 2024-09-11T08:59:28.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1818e-05 2024-09-11T08:59:28.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.395942+0000", 2024-09-11T08:59:28.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005789153 2024-09-11T08:59:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.395974+0000", 2024-09-11T08:59:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1449999999999999e-05 2024-09-11T08:59:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15918.0:148282 11.6 e413/366 11:6dfe47d2:::benchmark_data_smithi092_205252_object26961:head v 413'9282, mlcod=413'9280)", 2024-09-11T08:59:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:59:20.271194+0000", 2024-09-11T08:59:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.9820691579999998, 2024-09-11T08:59:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.125017929, 2024-09-11T08:59:28.373 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T08:59:28.373 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T08:59:28.373 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T08:59:28.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T08:59:28.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.271194+0000", 2024-09-11T08:59:28.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T08:59:28.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.271194+0000", 2024-09-11T08:59:28.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T08:59:28.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T08:59:28.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.271193+0000", 2024-09-11T08:59:28.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T08:59:28.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T08:59:28.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.271202+0000", 2024-09-11T08:59:28.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9450000000000005e-06 2024-09-11T08:59:28.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T08:59:28.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.390076+0000", 2024-09-11T08:59:28.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11887397800000001 2024-09-11T08:59:28.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T08:59:28.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.390086+0000", 2024-09-11T08:59:28.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.842e-06 2024-09-11T08:59:28.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T08:59:28.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.390132+0000", 2024-09-11T08:59:28.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5401000000000001e-05 2024-09-11T08:59:28.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T08:59:28.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.390154+0000", 2024-09-11T08:59:28.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2524999999999999e-05 2024-09-11T08:59:28.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T08:59:28.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.396126+0000", 2024-09-11T08:59:28.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059720950000000002 2024-09-11T08:59:28.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T08:59:28.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T08:59:28.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T08:59:28.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:59:20.396212+0000", 2024-09-11T08:59:28.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5295000000000002e-05 2024-09-11T08:59:28.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.376 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T08:59:28.377 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T08:59:28.377 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:59:28.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:28.378 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:28.378 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:28.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:28.522 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:28.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:28.523 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:28.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:28.660 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:28.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:28.661 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:28.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:28.804 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:28.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:28.805 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:28.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:28 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:28.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:28 smithi092 ceph-mon[29647]: pgmap v954: 168 pgs: 168 active+clean; 5.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 922 B/s rd, 0 B/s wr, 1.50k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:59:28.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:28 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:28.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:28 smithi092 ceph-mon[41114]: pgmap v954: 168 pgs: 168 active+clean; 5.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 922 B/s rd, 0 B/s wr, 1.50k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:59:28.905 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:28.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:28 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:28.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:28 smithi145 ceph-mon[33384]: pgmap v954: 168 pgs: 168 active+clean; 5.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 922 B/s rd, 0 B/s wr, 1.50k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:59:28.937 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:28.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:28.938 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:29.039 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:29.075 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:29.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:29.076 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:29.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:29.220 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:29.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:29.221 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:29.322 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:29.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:29.350 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:29.351 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:29.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:29.486 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:29.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:29.487 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:29.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:29.627 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:29.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:29.628 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:29.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:29.768 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-11T08:59:29.773 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:29.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:29.776 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:29.877 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:29.903 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:29.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:29.904 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:30.001 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:30.005 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:30.031 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:30.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:30.032 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:30.034 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:30.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:30.182 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:30.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:30.183 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:30.284 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:30.310 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:30.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:30.311 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:30.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:30 smithi145 ceph-mon[33384]: pgmap v955: 168 pgs: 168 active+clean; 5.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 818 B/s rd, 0 B/s wr, 1.33k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:59:30.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:59:30.412 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:30.432 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:30 smithi092 ceph-mon[29647]: pgmap v955: 168 pgs: 168 active+clean; 5.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 818 B/s rd, 0 B/s wr, 1.33k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:59:30.433 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:59:30.433 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:30 smithi092 ceph-mon[41114]: pgmap v955: 168 pgs: 168 active+clean; 5.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 818 B/s rd, 0 B/s wr, 1.33k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:59:30.433 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:59:30.448 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:30.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:30.449 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:30.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:30.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:30.589 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:30.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:30.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:30.733 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:30.733 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:30.837 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:30.865 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:30.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:30.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:30.967 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:31.003 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:31.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:31.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:31.105 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:31.140 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:31.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:31.141 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:31.176 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T08:59:31.242 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:31.274 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:31.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:31.275 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:31.376 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:31.408 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:31.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:31.409 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:31.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:31 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/675215909' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:59:31.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:31 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/675215909' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:59:31.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:31 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/675215909' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T08:59:31.510 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:31.535 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:31.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:31.536 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:31.637 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:31.671 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:31.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:31.672 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:31.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:31.795 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:59:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:59:31] "GET /metrics HTTP/1.1" 200 225133 "" "Prometheus/2.43.0" 2024-09-11T08:59:31.813 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:31.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:31.814 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:31.915 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:31.944 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:31.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:31.945 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:32.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:32.079 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:32.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:32.080 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:32.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:32.221 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:32.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:32.222 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:32.323 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:32.350 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:32.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:32.351 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:32.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:32 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:59:32.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:32 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/675215909' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:59:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:32 smithi145 ceph-mon[33384]: osdmap e415: 8 total, 8 up, 8 in 2024-09-11T08:59:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:32 smithi145 ceph-mon[33384]: pgmap v957: 168 pgs: 168 active+clean; 5.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 818 B/s rd, 0 B/s wr, 1.34k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:59:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:32 smithi145 ceph-mon[33384]: 5.1a scrub starts 2024-09-11T08:59:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:32 smithi145 ceph-mon[33384]: 4.b scrub starts 2024-09-11T08:59:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:32 smithi145 ceph-mon[33384]: 4.b scrub ok 2024-09-11T08:59:32.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:32.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:32.488 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:32.489 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:32 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:59:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:32 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/675215909' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:59:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:32 smithi092 ceph-mon[29647]: osdmap e415: 8 total, 8 up, 8 in 2024-09-11T08:59:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:32 smithi092 ceph-mon[29647]: pgmap v957: 168 pgs: 168 active+clean; 5.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 818 B/s rd, 0 B/s wr, 1.34k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:59:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:32 smithi092 ceph-mon[29647]: 5.1a scrub starts 2024-09-11T08:59:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:32 smithi092 ceph-mon[29647]: 4.b scrub starts 2024-09-11T08:59:32.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:32 smithi092 ceph-mon[29647]: 4.b scrub ok 2024-09-11T08:59:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:32 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T08:59:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:32 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/675215909' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T08:59:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:32 smithi092 ceph-mon[41114]: osdmap e415: 8 total, 8 up, 8 in 2024-09-11T08:59:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:32 smithi092 ceph-mon[41114]: pgmap v957: 168 pgs: 168 active+clean; 5.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 818 B/s rd, 0 B/s wr, 1.34k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T08:59:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:32 smithi092 ceph-mon[41114]: 5.1a scrub starts 2024-09-11T08:59:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:32 smithi092 ceph-mon[41114]: 4.b scrub starts 2024-09-11T08:59:32.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:32 smithi092 ceph-mon[41114]: 4.b scrub ok 2024-09-11T08:59:32.590 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:32.625 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:32.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:32.626 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:32.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:32.772 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:32.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:32.774 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:32.874 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:32.902 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:32.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:32.903 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:33.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:33.041 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:33.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:33.042 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:33.143 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:33.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:59:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:59:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:59:33.172 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:33.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:33.174 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:33.250 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:33 smithi092 ceph-mon[29647]: 5.1a scrub ok 2024-09-11T08:59:33.250 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:33 smithi092 ceph-mon[29647]: 3.1e scrub starts 2024-09-11T08:59:33.250 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:33 smithi092 ceph-mon[29647]: 3.1e scrub ok 2024-09-11T08:59:33.250 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:33 smithi092 ceph-mon[29647]: 1.14 scrub starts 2024-09-11T08:59:33.250 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:33 smithi092 ceph-mon[29647]: 1.14 scrub ok 2024-09-11T08:59:33.251 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:33 smithi092 ceph-mon[41114]: 5.1a scrub ok 2024-09-11T08:59:33.251 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:33 smithi092 ceph-mon[41114]: 3.1e scrub starts 2024-09-11T08:59:33.251 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:33 smithi092 ceph-mon[41114]: 3.1e scrub ok 2024-09-11T08:59:33.251 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:33 smithi092 ceph-mon[41114]: 1.14 scrub starts 2024-09-11T08:59:33.251 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:33 smithi092 ceph-mon[41114]: 1.14 scrub ok 2024-09-11T08:59:33.275 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:33.302 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:33.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:33.303 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:33.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:33.436 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:33.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:33.438 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:33.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:33.569 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:33.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:33.570 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:33.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:33 smithi145 ceph-mon[33384]: 5.1a scrub ok 2024-09-11T08:59:33.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:33 smithi145 ceph-mon[33384]: 3.1e scrub starts 2024-09-11T08:59:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:33 smithi145 ceph-mon[33384]: 3.1e scrub ok 2024-09-11T08:59:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:33 smithi145 ceph-mon[33384]: 1.14 scrub starts 2024-09-11T08:59:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:33 smithi145 ceph-mon[33384]: 1.14 scrub ok 2024-09-11T08:59:33.671 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:33.704 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:33.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:33.705 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:33.806 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:33.838 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:33.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:33.839 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:33.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:33.962 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:34.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:34.005 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:34.005 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:34.012 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:59:34.012 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:59:34.048 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:59:34.049 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:59:34.078 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:59:34.079 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:59:34.105 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:34.106 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:59:34.106 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:59:34.131 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:34.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:34.132 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:34.141 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:59:34.142 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:59:34.175 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:59:34.175 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:59:34.204 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:59:34.205 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:59:34.205 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:34 smithi092 ceph-mon[29647]: 4.e scrub starts 2024-09-11T08:59:34.205 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:34 smithi092 ceph-mon[29647]: 4.e scrub ok 2024-09-11T08:59:34.205 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:34 smithi092 ceph-mon[29647]: pgmap v958: 168 pgs: 168 active+clean; 4.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-11T08:59:34.205 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:34 smithi092 ceph-mon[29647]: 3.6 scrub starts 2024-09-11T08:59:34.206 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:34 smithi092 ceph-mon[29647]: 3.6 scrub ok 2024-09-11T08:59:34.206 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:34 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2564857557' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:59:34.206 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:34 smithi092 ceph-mon[41114]: 4.e scrub starts 2024-09-11T08:59:34.206 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:34 smithi092 ceph-mon[41114]: 4.e scrub ok 2024-09-11T08:59:34.206 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:34 smithi092 ceph-mon[41114]: pgmap v958: 168 pgs: 168 active+clean; 4.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-11T08:59:34.206 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:34 smithi092 ceph-mon[41114]: 3.6 scrub starts 2024-09-11T08:59:34.206 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:34 smithi092 ceph-mon[41114]: 3.6 scrub ok 2024-09-11T08:59:34.206 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:34 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2564857557' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:59:34.232 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:59:34.232 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:59:34.232 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:34.279 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:34.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:34.280 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:34.382 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:34.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:34.417 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:34.418 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:34.518 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:34.557 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:34.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:34.558 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:34.659 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:34.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:34 smithi145 ceph-mon[33384]: 4.e scrub starts 2024-09-11T08:59:34.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:34 smithi145 ceph-mon[33384]: 4.e scrub ok 2024-09-11T08:59:34.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:34 smithi145 ceph-mon[33384]: pgmap v958: 168 pgs: 168 active+clean; 4.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-11T08:59:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:34 smithi145 ceph-mon[33384]: 3.6 scrub starts 2024-09-11T08:59:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:34 smithi145 ceph-mon[33384]: 3.6 scrub ok 2024-09-11T08:59:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:34 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2564857557' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:59:34.696 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:34.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:34.698 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:34.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:34.832 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:34.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:34.834 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:34.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:34.977 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:34.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:34.978 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:35.079 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:35.107 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:35.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:35.108 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:35.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:35.253 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:35.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:35.254 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:35.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:35 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T08:59:35.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:35 smithi092 ceph-mon[29647]: 5.8 scrub ok 2024-09-11T08:59:35.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:35 smithi092 ceph-mon[29647]: 4.10 scrub starts 2024-09-11T08:59:35.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:35 smithi092 ceph-mon[29647]: 4.10 scrub ok 2024-09-11T08:59:35.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:35 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T08:59:35.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:35 smithi092 ceph-mon[41114]: 5.8 scrub ok 2024-09-11T08:59:35.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:35 smithi092 ceph-mon[41114]: 4.10 scrub starts 2024-09-11T08:59:35.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:35 smithi092 ceph-mon[41114]: 4.10 scrub ok 2024-09-11T08:59:35.355 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:35.380 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:35.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:35.381 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:35.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:35.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:35.524 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:35.524 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:35.624 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:35.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:35.653 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:35.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:35.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:35 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T08:59:35.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:35 smithi145 ceph-mon[33384]: 5.8 scrub ok 2024-09-11T08:59:35.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:35 smithi145 ceph-mon[33384]: 4.10 scrub starts 2024-09-11T08:59:35.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:35 smithi145 ceph-mon[33384]: 4.10 scrub ok 2024-09-11T08:59:35.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T08:59:35.754 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:35.792 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:35.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:35.793 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:35.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:35.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:35.935 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:35.935 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:36.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:36.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:36.088 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:36.088 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:36.189 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:36.217 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:36.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:36.218 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:36.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:36.356 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:36.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:36.357 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:36.458 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:36.489 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:36.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:36.490 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:36.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:36 smithi092 ceph-mon[29647]: 5.c scrub starts 2024-09-11T08:59:36.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:36 smithi092 ceph-mon[29647]: 5.c scrub ok 2024-09-11T08:59:36.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:36 smithi092 ceph-mon[29647]: 3.e scrub starts 2024-09-11T08:59:36.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:36 smithi092 ceph-mon[29647]: 3.e scrub ok 2024-09-11T08:59:36.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:36 smithi092 ceph-mon[29647]: pgmap v959: 168 pgs: 168 active+clean; 4.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-11T08:59:36.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:36 smithi092 ceph-mon[29647]: 5.2 scrub starts 2024-09-11T08:59:36.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:36 smithi092 ceph-mon[29647]: 5.2 scrub ok 2024-09-11T08:59:36.561 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:36 smithi092 ceph-mon[41114]: 5.c scrub starts 2024-09-11T08:59:36.561 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:36 smithi092 ceph-mon[41114]: 5.c scrub ok 2024-09-11T08:59:36.561 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:36 smithi092 ceph-mon[41114]: 3.e scrub starts 2024-09-11T08:59:36.561 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:36 smithi092 ceph-mon[41114]: 3.e scrub ok 2024-09-11T08:59:36.561 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:36 smithi092 ceph-mon[41114]: pgmap v959: 168 pgs: 168 active+clean; 4.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-11T08:59:36.561 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:36 smithi092 ceph-mon[41114]: 5.2 scrub starts 2024-09-11T08:59:36.562 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:36 smithi092 ceph-mon[41114]: 5.2 scrub ok 2024-09-11T08:59:36.591 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:36.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.7 reset_purged_snaps_last 2024-09-11T08:59:36.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:36.624 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:36.624 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:36.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:36 smithi145 ceph-mon[33384]: 5.c scrub starts 2024-09-11T08:59:36.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:36 smithi145 ceph-mon[33384]: 5.c scrub ok 2024-09-11T08:59:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:36 smithi145 ceph-mon[33384]: 3.e scrub starts 2024-09-11T08:59:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:36 smithi145 ceph-mon[33384]: 3.e scrub ok 2024-09-11T08:59:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:36 smithi145 ceph-mon[33384]: pgmap v959: 168 pgs: 168 active+clean; 4.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-11T08:59:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:36 smithi145 ceph-mon[33384]: 5.2 scrub starts 2024-09-11T08:59:36.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:36 smithi145 ceph-mon[33384]: 5.2 scrub ok 2024-09-11T08:59:36.725 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:36.751 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:36.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:36.753 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:36.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:36.894 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:36.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:36.895 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:36.996 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:37.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:37.031 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:37.032 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:37.085 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T08:59:37.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:37.181 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:37.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:37.183 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:37.284 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:37.312 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:37.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:37.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:37.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[29647]: 6.1 scrub starts 2024-09-11T08:59:37.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[29647]: 6.1 scrub ok 2024-09-11T08:59:37.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[29647]: 5.b scrub starts 2024-09-11T08:59:37.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[29647]: 5.b scrub ok 2024-09-11T08:59:37.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[29647]: 6.a scrub starts 2024-09-11T08:59:37.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[29647]: 6.a scrub ok 2024-09-11T08:59:37.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[29647]: 4.1e scrub starts 2024-09-11T08:59:37.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[29647]: 4.1e scrub ok 2024-09-11T08:59:37.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:59:37.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:59:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[41114]: 6.1 scrub starts 2024-09-11T08:59:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[41114]: 6.1 scrub ok 2024-09-11T08:59:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[41114]: 5.b scrub starts 2024-09-11T08:59:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[41114]: 5.b scrub ok 2024-09-11T08:59:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[41114]: 6.a scrub starts 2024-09-11T08:59:37.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[41114]: 6.a scrub ok 2024-09-11T08:59:37.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[41114]: 4.1e scrub starts 2024-09-11T08:59:37.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[41114]: 4.1e scrub ok 2024-09-11T08:59:37.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:59:37.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:37 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:59:37.413 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:37.442 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:37.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:37.443 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:37.544 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:37.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:37.587 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:37.633 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:37.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:59:37 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:59:37.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:37 smithi145 ceph-mon[33384]: 6.1 scrub starts 2024-09-11T08:59:37.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:37 smithi145 ceph-mon[33384]: 6.1 scrub ok 2024-09-11T08:59:37.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:37 smithi145 ceph-mon[33384]: 5.b scrub starts 2024-09-11T08:59:37.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:37 smithi145 ceph-mon[33384]: 5.b scrub ok 2024-09-11T08:59:37.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:37 smithi145 ceph-mon[33384]: 6.a scrub starts 2024-09-11T08:59:37.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:37 smithi145 ceph-mon[33384]: 6.a scrub ok 2024-09-11T08:59:37.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:37 smithi145 ceph-mon[33384]: 4.1e scrub starts 2024-09-11T08:59:37.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:37 smithi145 ceph-mon[33384]: 4.1e scrub ok 2024-09-11T08:59:37.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:37 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:59:37.663 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:37 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:59:37.688 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:37.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:37.739 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:37.739 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:37.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:37.883 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:37.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:37.884 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:37.985 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:38.018 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:38.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:38.019 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:38.120 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:38.155 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:38.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:38.156 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:38.256 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:38.289 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:38.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:38.290 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:38.390 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:38.432 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:38.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:38.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:38.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[29647]: 1.13 scrub starts 2024-09-11T08:59:38.511 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[29647]: 1.13 scrub ok 2024-09-11T08:59:38.512 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[29647]: 4.4 scrub starts 2024-09-11T08:59:38.512 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[29647]: 4.4 scrub ok 2024-09-11T08:59:38.512 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[29647]: osdmap e416: 8 total, 8 up, 8 in 2024-09-11T08:59:38.512 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:38.512 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[29647]: pgmap v961: 168 pgs: 168 active+clean; 4.4 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.70k op/s 2024-09-11T08:59:38.512 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[29647]: 6.4 scrub starts 2024-09-11T08:59:38.512 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[29647]: 6.4 scrub ok 2024-09-11T08:59:38.512 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[29647]: 5.10 scrub starts 2024-09-11T08:59:38.512 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[29647]: 5.10 scrub ok 2024-09-11T08:59:38.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[41114]: 1.13 scrub starts 2024-09-11T08:59:38.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[41114]: 1.13 scrub ok 2024-09-11T08:59:38.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[41114]: 4.4 scrub starts 2024-09-11T08:59:38.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[41114]: 4.4 scrub ok 2024-09-11T08:59:38.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[41114]: osdmap e416: 8 total, 8 up, 8 in 2024-09-11T08:59:38.514 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:38.514 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[41114]: pgmap v961: 168 pgs: 168 active+clean; 4.4 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.70k op/s 2024-09-11T08:59:38.514 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[41114]: 6.4 scrub starts 2024-09-11T08:59:38.514 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[41114]: 6.4 scrub ok 2024-09-11T08:59:38.514 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[41114]: 5.10 scrub starts 2024-09-11T08:59:38.514 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:38 smithi092 ceph-mon[41114]: 5.10 scrub ok 2024-09-11T08:59:38.534 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:38.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:38.571 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:38.572 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:38.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:38 smithi145 ceph-mon[33384]: 1.13 scrub starts 2024-09-11T08:59:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:38 smithi145 ceph-mon[33384]: 1.13 scrub ok 2024-09-11T08:59:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:38 smithi145 ceph-mon[33384]: 4.4 scrub starts 2024-09-11T08:59:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:38 smithi145 ceph-mon[33384]: 4.4 scrub ok 2024-09-11T08:59:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:38 smithi145 ceph-mon[33384]: osdmap e416: 8 total, 8 up, 8 in 2024-09-11T08:59:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:38 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:38 smithi145 ceph-mon[33384]: pgmap v961: 168 pgs: 168 active+clean; 4.4 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.70k op/s 2024-09-11T08:59:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:38 smithi145 ceph-mon[33384]: 6.4 scrub starts 2024-09-11T08:59:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:38 smithi145 ceph-mon[33384]: 6.4 scrub ok 2024-09-11T08:59:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:38 smithi145 ceph-mon[33384]: 5.10 scrub starts 2024-09-11T08:59:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:38 smithi145 ceph-mon[33384]: 5.10 scrub ok 2024-09-11T08:59:38.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:38.713 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:38.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:38.714 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:38.815 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:38.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:38.844 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:38.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:38.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:38.982 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:38.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:38.983 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:39.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:39.120 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:39.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:39.122 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:39.223 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:39.229 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[29647]: 5.5 scrub starts 2024-09-11T08:59:39.229 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[29647]: 5.5 scrub ok 2024-09-11T08:59:39.229 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[29647]: 3.3 scrub starts 2024-09-11T08:59:39.229 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[29647]: 3.3 scrub ok 2024-09-11T08:59:39.229 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[29647]: 5.1e scrub starts 2024-09-11T08:59:39.229 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[29647]: 5.1e scrub ok 2024-09-11T08:59:39.229 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[29647]: 4.17 scrub starts 2024-09-11T08:59:39.229 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[29647]: 4.17 scrub ok 2024-09-11T08:59:39.230 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[41114]: 5.5 scrub starts 2024-09-11T08:59:39.230 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[41114]: 5.5 scrub ok 2024-09-11T08:59:39.230 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[41114]: 3.3 scrub starts 2024-09-11T08:59:39.230 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[41114]: 3.3 scrub ok 2024-09-11T08:59:39.230 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[41114]: 5.1e scrub starts 2024-09-11T08:59:39.230 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[41114]: 5.1e scrub ok 2024-09-11T08:59:39.230 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[41114]: 4.17 scrub starts 2024-09-11T08:59:39.230 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:39 smithi092 ceph-mon[41114]: 4.17 scrub ok 2024-09-11T08:59:39.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:39.270 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:39.271 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:39.371 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:39.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:39.404 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:39.405 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:39.498 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T08:59:39.499 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T08:59:39.505 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:39.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:39.546 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:39.547 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:39.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:39.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:39 smithi145 ceph-mon[33384]: 5.5 scrub starts 2024-09-11T08:59:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:39 smithi145 ceph-mon[33384]: 5.5 scrub ok 2024-09-11T08:59:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:39 smithi145 ceph-mon[33384]: 3.3 scrub starts 2024-09-11T08:59:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:39 smithi145 ceph-mon[33384]: 3.3 scrub ok 2024-09-11T08:59:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:39 smithi145 ceph-mon[33384]: 5.1e scrub starts 2024-09-11T08:59:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:39 smithi145 ceph-mon[33384]: 5.1e scrub ok 2024-09-11T08:59:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:39 smithi145 ceph-mon[33384]: 4.17 scrub starts 2024-09-11T08:59:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:39 smithi145 ceph-mon[33384]: 4.17 scrub ok 2024-09-11T08:59:39.683 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:39.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:39.684 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:39.784 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:39.816 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:39.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:39.817 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:39.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:39.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:39.965 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:39.965 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:40.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:40.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:40.104 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:40.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:40.245 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:40.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:40.246 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:40.347 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:40.378 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:40.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:40.380 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:40.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:40.488 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:40 smithi092 ceph-mon[29647]: 4.f scrub starts 2024-09-11T08:59:40.489 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:40 smithi092 ceph-mon[29647]: 4.f scrub ok 2024-09-11T08:59:40.489 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:40 smithi092 ceph-mon[29647]: pgmap v962: 168 pgs: 168 active+clean; 4.4 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-11T08:59:40.489 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:40 smithi092 ceph-mon[29647]: 2.0 scrub starts 2024-09-11T08:59:40.489 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:40 smithi092 ceph-mon[29647]: 2.0 scrub ok 2024-09-11T08:59:40.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:40 smithi092 ceph-mon[41114]: 4.f scrub starts 2024-09-11T08:59:40.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:40 smithi092 ceph-mon[41114]: 4.f scrub ok 2024-09-11T08:59:40.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:40 smithi092 ceph-mon[41114]: pgmap v962: 168 pgs: 168 active+clean; 4.4 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-11T08:59:40.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:40 smithi092 ceph-mon[41114]: 2.0 scrub starts 2024-09-11T08:59:40.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:40 smithi092 ceph-mon[41114]: 2.0 scrub ok 2024-09-11T08:59:40.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:40.533 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:40.533 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:40.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-11T08:59:40.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:40.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:40 smithi145 ceph-mon[33384]: 4.f scrub starts 2024-09-11T08:59:40.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:40 smithi145 ceph-mon[33384]: 4.f scrub ok 2024-09-11T08:59:40.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:40 smithi145 ceph-mon[33384]: pgmap v962: 168 pgs: 168 active+clean; 4.4 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-11T08:59:40.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:40 smithi145 ceph-mon[33384]: 2.0 scrub starts 2024-09-11T08:59:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:40 smithi145 ceph-mon[33384]: 2.0 scrub ok 2024-09-11T08:59:40.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:40.692 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:40.692 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:40.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:40.843 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:40.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:40.845 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:40.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:40.986 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:40.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:40.987 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:41.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:41.130 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:41.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:41.131 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:41.232 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:41.261 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:41.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:41.262 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:41.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:41.402 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:41.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:41.403 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:41.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:41 smithi092 ceph-mon[29647]: 11.6 scrub starts 2024-09-11T08:59:41.451 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:41 smithi092 ceph-mon[29647]: 3.a scrub starts 2024-09-11T08:59:41.451 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:41 smithi092 ceph-mon[29647]: 3.a scrub ok 2024-09-11T08:59:41.451 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:41 smithi092 ceph-mon[29647]: 5.3 scrub starts 2024-09-11T08:59:41.451 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:41 smithi092 ceph-mon[29647]: 5.3 scrub ok 2024-09-11T08:59:41.451 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:41 smithi092 ceph-mon[41114]: 11.6 scrub starts 2024-09-11T08:59:41.451 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:41 smithi092 ceph-mon[41114]: 3.a scrub starts 2024-09-11T08:59:41.451 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:41 smithi092 ceph-mon[41114]: 3.a scrub ok 2024-09-11T08:59:41.451 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:41 smithi092 ceph-mon[41114]: 5.3 scrub starts 2024-09-11T08:59:41.451 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:41 smithi092 ceph-mon[41114]: 5.3 scrub ok 2024-09-11T08:59:41.504 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:41.532 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:41.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:41.559 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:41.559 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:41.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:41 smithi145 ceph-mon[33384]: 11.6 scrub starts 2024-09-11T08:59:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:41 smithi145 ceph-mon[33384]: 3.a scrub starts 2024-09-11T08:59:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:41 smithi145 ceph-mon[33384]: 3.a scrub ok 2024-09-11T08:59:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:41 smithi145 ceph-mon[33384]: 5.3 scrub starts 2024-09-11T08:59:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:41 smithi145 ceph-mon[33384]: 5.3 scrub ok 2024-09-11T08:59:41.706 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:41.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:41.707 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:41.807 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:41.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:59:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:59:41] "GET /metrics HTTP/1.1" 200 225133 "" "Prometheus/2.43.0" 2024-09-11T08:59:41.834 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:41.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:41.835 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:41.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:41.978 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:41.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:41.979 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:42.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:42.113 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:42.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:42.114 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:42.215 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:42.244 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:42.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:42.245 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:42.346 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:42.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:42.379 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:42.379 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:42.480 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:42.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:42.507 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:42.508 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:42.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:42 smithi092 ceph-mon[29647]: 11.f scrub starts 2024-09-11T08:59:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:42 smithi092 ceph-mon[29647]: 11.d scrub starts 2024-09-11T08:59:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:42 smithi092 ceph-mon[29647]: 6.d scrub starts 2024-09-11T08:59:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:42 smithi092 ceph-mon[29647]: pgmap v963: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 4.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-11T08:59:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:42 smithi092 ceph-mon[41114]: 11.f scrub starts 2024-09-11T08:59:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:42 smithi092 ceph-mon[41114]: 11.d scrub starts 2024-09-11T08:59:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:42 smithi092 ceph-mon[41114]: 6.d scrub starts 2024-09-11T08:59:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:42 smithi092 ceph-mon[41114]: pgmap v963: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 4.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-11T08:59:42.609 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:42.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:42.640 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:42.641 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:42 smithi145 ceph-mon[33384]: 11.f scrub starts 2024-09-11T08:59:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:42 smithi145 ceph-mon[33384]: 11.d scrub starts 2024-09-11T08:59:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:42 smithi145 ceph-mon[33384]: 6.d scrub starts 2024-09-11T08:59:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:42 smithi145 ceph-mon[33384]: pgmap v963: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 4.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-11T08:59:42.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:42.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:42.768 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:42.869 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:42.901 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:42.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:42.903 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:43.003 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:43.030 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:43.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:43.032 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:43.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:43.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:59:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:59:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:59:43.168 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:43.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:43.169 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:43.270 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:43.291 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:43.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:43.292 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:43.393 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:43.418 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:43.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:43.419 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:43.520 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:43.549 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:43.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:43.550 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:43.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:43 smithi092 ceph-mon[29647]: 11.c scrub starts 2024-09-11T08:59:43.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:43 smithi092 ceph-mon[29647]: 11.6 scrub ok 2024-09-11T08:59:43.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:43 smithi092 ceph-mon[29647]: 6.15 scrub starts 2024-09-11T08:59:43.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:43 smithi092 ceph-mon[29647]: 6.15 scrub ok 2024-09-11T08:59:43.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:43 smithi092 ceph-mon[41114]: 11.c scrub starts 2024-09-11T08:59:43.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:43 smithi092 ceph-mon[41114]: 11.6 scrub ok 2024-09-11T08:59:43.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:43 smithi092 ceph-mon[41114]: 6.15 scrub starts 2024-09-11T08:59:43.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:43 smithi092 ceph-mon[41114]: 6.15 scrub ok 2024-09-11T08:59:43.651 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:43.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:43 smithi145 ceph-mon[33384]: 11.c scrub starts 2024-09-11T08:59:43.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:43 smithi145 ceph-mon[33384]: 11.6 scrub ok 2024-09-11T08:59:43.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:43 smithi145 ceph-mon[33384]: 6.15 scrub starts 2024-09-11T08:59:43.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:43 smithi145 ceph-mon[33384]: 6.15 scrub ok 2024-09-11T08:59:43.682 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:43.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:43.683 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:43.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:43.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:43.810 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:43.810 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:43.911 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:43.947 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:43.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:43.948 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:44.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:44.086 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:44.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:44.087 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:44.188 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:44.225 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:44.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:44.226 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:44.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:44 smithi092 ceph-mon[29647]: 11.d scrub ok 2024-09-11T08:59:44.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:44 smithi092 ceph-mon[29647]: 2.1 scrub starts 2024-09-11T08:59:44.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:44 smithi092 ceph-mon[29647]: 2.1 scrub ok 2024-09-11T08:59:44.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:44 smithi092 ceph-mon[29647]: 11.f scrub ok 2024-09-11T08:59:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:44 smithi092 ceph-mon[29647]: 11.4 scrub starts 2024-09-11T08:59:44.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:44 smithi092 ceph-mon[29647]: pgmap v964: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 4.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T08:59:44.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:44 smithi092 ceph-mon[41114]: 11.d scrub ok 2024-09-11T08:59:44.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:44 smithi092 ceph-mon[41114]: 2.1 scrub starts 2024-09-11T08:59:44.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:44 smithi092 ceph-mon[41114]: 2.1 scrub ok 2024-09-11T08:59:44.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:44 smithi092 ceph-mon[41114]: 11.f scrub ok 2024-09-11T08:59:44.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:44 smithi092 ceph-mon[41114]: 11.4 scrub starts 2024-09-11T08:59:44.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:44 smithi092 ceph-mon[41114]: pgmap v964: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 4.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T08:59:44.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:44.358 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:44.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:44.359 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:44.460 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:44.486 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:44.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:44.487 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:44.589 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:44.620 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:44.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:44.621 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:44.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:44 smithi145 ceph-mon[33384]: 11.d scrub ok 2024-09-11T08:59:44.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:44 smithi145 ceph-mon[33384]: 2.1 scrub starts 2024-09-11T08:59:44.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:44 smithi145 ceph-mon[33384]: 2.1 scrub ok 2024-09-11T08:59:44.664 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:44 smithi145 ceph-mon[33384]: 11.f scrub ok 2024-09-11T08:59:44.665 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:44 smithi145 ceph-mon[33384]: 11.4 scrub starts 2024-09-11T08:59:44.665 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:44 smithi145 ceph-mon[33384]: pgmap v964: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 4.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T08:59:44.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:44.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:44.768 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:44.768 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:44.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:44.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:44.891 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:44.891 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:44.992 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:45.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:45.014 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:45.014 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:45.115 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:45.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:45.157 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:45.157 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:45.258 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:45.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:45.290 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:45.290 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:45.391 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:45.428 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:45.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:45.429 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:45.530 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:45.551 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:45.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:45.552 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:45.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:45 smithi092 ceph-mon[41114]: 3.9 scrub starts 2024-09-11T08:59:45.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:45 smithi092 ceph-mon[41114]: 3.9 scrub ok 2024-09-11T08:59:45.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:45 smithi092 ceph-mon[41114]: 5.f scrub starts 2024-09-11T08:59:45.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:45 smithi092 ceph-mon[41114]: 5.f scrub ok 2024-09-11T08:59:45.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:59:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:45 smithi092 ceph-mon[29647]: 3.9 scrub starts 2024-09-11T08:59:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:45 smithi092 ceph-mon[29647]: 3.9 scrub ok 2024-09-11T08:59:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:45 smithi092 ceph-mon[29647]: 5.f scrub starts 2024-09-11T08:59:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:45 smithi092 ceph-mon[29647]: 5.f scrub ok 2024-09-11T08:59:45.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:59:45.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:45 smithi145 ceph-mon[33384]: 3.9 scrub starts 2024-09-11T08:59:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:45 smithi145 ceph-mon[33384]: 3.9 scrub ok 2024-09-11T08:59:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:45 smithi145 ceph-mon[33384]: 5.f scrub starts 2024-09-11T08:59:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:45 smithi145 ceph-mon[33384]: 5.f scrub ok 2024-09-11T08:59:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:59:45.689 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:45.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:45.690 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:45.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:45.826 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:45.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:45.828 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:45.928 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:45.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:45.960 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:45.960 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:46.061 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:46.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:46.097 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:46.098 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:46.198 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:46.231 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:46.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:46.232 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:46.332 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:46.363 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:46.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:46.364 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:46.465 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:46.495 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:46.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:46.496 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[29647]: 5.1f scrub starts 2024-09-11T08:59:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[29647]: 5.1f scrub ok 2024-09-11T08:59:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[29647]: 11.4 scrub ok 2024-09-11T08:59:46.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[29647]: 3.13 scrub starts 2024-09-11T08:59:46.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[29647]: 3.13 scrub ok 2024-09-11T08:59:46.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[29647]: pgmap v965: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 4.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T08:59:46.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1932753570' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:59:46.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:59:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[41114]: 5.1f scrub starts 2024-09-11T08:59:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[41114]: 5.1f scrub ok 2024-09-11T08:59:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[41114]: 11.4 scrub ok 2024-09-11T08:59:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[41114]: 3.13 scrub starts 2024-09-11T08:59:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[41114]: 3.13 scrub ok 2024-09-11T08:59:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[41114]: pgmap v965: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 4.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T08:59:46.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1932753570' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:59:46.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:46 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:59:46.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:46.634 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:46.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:46.635 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:46.644 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T08:59:46.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:46 smithi145 ceph-mon[33384]: 5.1f scrub starts 2024-09-11T08:59:46.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:46 smithi145 ceph-mon[33384]: 5.1f scrub ok 2024-09-11T08:59:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:46 smithi145 ceph-mon[33384]: 11.4 scrub ok 2024-09-11T08:59:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:46 smithi145 ceph-mon[33384]: 3.13 scrub starts 2024-09-11T08:59:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:46 smithi145 ceph-mon[33384]: 3.13 scrub ok 2024-09-11T08:59:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:46 smithi145 ceph-mon[33384]: pgmap v965: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 4.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T08:59:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:46 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1932753570' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:59:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:46 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T08:59:46.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:46.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:46.775 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:46.775 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:46.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:46.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:46.912 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:46.913 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:47.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:47.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:47.050 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:47.050 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:47.151 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:47.183 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:47.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:47.184 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:47.285 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:47.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:47.318 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:47.318 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:47.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:47.455 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:47.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:47.457 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:47.557 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:47.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:47 smithi092 ceph-mon[29647]: 4.1b scrub starts 2024-09-11T08:59:47.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:47 smithi092 ceph-mon[29647]: 4.1b scrub ok 2024-09-11T08:59:47.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:47 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:59:47.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:47 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T08:59:47.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:47 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:59:47.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:47 smithi092 ceph-mon[29647]: osdmap e417: 8 total, 8 up, 8 in 2024-09-11T08:59:47.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:47 smithi092 ceph-mon[41114]: 4.1b scrub starts 2024-09-11T08:59:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:47 smithi092 ceph-mon[41114]: 4.1b scrub ok 2024-09-11T08:59:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:47 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:59:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:47 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T08:59:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:47 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:59:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:47 smithi092 ceph-mon[41114]: osdmap e417: 8 total, 8 up, 8 in 2024-09-11T08:59:47.591 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:47.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:47.592 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:47.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:59:47 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:59:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:47 smithi145 ceph-mon[33384]: 4.1b scrub starts 2024-09-11T08:59:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:47 smithi145 ceph-mon[33384]: 4.1b scrub ok 2024-09-11T08:59:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:47 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T08:59:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:47 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T08:59:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:47 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T08:59:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:47 smithi145 ceph-mon[33384]: osdmap e417: 8 total, 8 up, 8 in 2024-09-11T08:59:47.693 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:47.731 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:47.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:47.732 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:47.833 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:47.866 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:47.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:47.867 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:47.967 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:47.991 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:47.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:47.992 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:48.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:48.127 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:48.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:48.128 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:48.229 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:48.265 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:48.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:48.266 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:48.367 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:48.399 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:48.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:48.400 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:48.500 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:48.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:48.525 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:48.525 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:48 smithi092 ceph-mon[29647]: 4.1 scrub starts 2024-09-11T08:59:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:48 smithi092 ceph-mon[29647]: 4.1 scrub ok 2024-09-11T08:59:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:48 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:48.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:48 smithi092 ceph-mon[29647]: pgmap v967: 168 pgs: 168 active+clean; 3.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-11T08:59:48.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:48 smithi092 ceph-mon[41114]: 4.1 scrub starts 2024-09-11T08:59:48.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:48 smithi092 ceph-mon[41114]: 4.1 scrub ok 2024-09-11T08:59:48.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:48 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:48.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:48 smithi092 ceph-mon[41114]: pgmap v967: 168 pgs: 168 active+clean; 3.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-11T08:59:48.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:48.660 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:48.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:48.661 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:48 smithi145 ceph-mon[33384]: 4.1 scrub starts 2024-09-11T08:59:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:48 smithi145 ceph-mon[33384]: 4.1 scrub ok 2024-09-11T08:59:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:48 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:48 smithi145 ceph-mon[33384]: pgmap v967: 168 pgs: 168 active+clean; 3.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-11T08:59:48.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:48.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:48.802 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:48.903 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:48.936 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:48.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:48.938 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:49.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:49.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:49.072 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:49.072 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:49.173 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:49.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:49.221 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:49.222 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:49.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:49.361 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:49.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:49.366 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:49.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:49.508 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:49.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:49.509 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:49.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:49 smithi092 ceph-mon[29647]: 4.18 scrub starts 2024-09-11T08:59:49.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:49 smithi092 ceph-mon[29647]: 4.18 scrub ok 2024-09-11T08:59:49.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:49 smithi092 ceph-mon[29647]: 6.3 scrub starts 2024-09-11T08:59:49.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:49 smithi092 ceph-mon[29647]: 6.3 scrub ok 2024-09-11T08:59:49.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:49 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:59:49.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:49 smithi092 ceph-mon[29647]: 11.0 scrub starts 2024-09-11T08:59:49.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:49 smithi092 ceph-mon[41114]: 4.18 scrub starts 2024-09-11T08:59:49.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:49 smithi092 ceph-mon[41114]: 4.18 scrub ok 2024-09-11T08:59:49.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:49 smithi092 ceph-mon[41114]: 6.3 scrub starts 2024-09-11T08:59:49.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:49 smithi092 ceph-mon[41114]: 6.3 scrub ok 2024-09-11T08:59:49.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:49 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:59:49.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:49 smithi092 ceph-mon[41114]: 11.0 scrub starts 2024-09-11T08:59:49.611 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:49.651 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:49.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:49.652 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:49 smithi145 ceph-mon[33384]: 4.18 scrub starts 2024-09-11T08:59:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:49 smithi145 ceph-mon[33384]: 4.18 scrub ok 2024-09-11T08:59:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:49 smithi145 ceph-mon[33384]: 6.3 scrub starts 2024-09-11T08:59:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:49 smithi145 ceph-mon[33384]: 6.3 scrub ok 2024-09-11T08:59:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:49 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:59:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:49 smithi145 ceph-mon[33384]: 11.0 scrub starts 2024-09-11T08:59:49.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:49.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:49.792 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:49.792 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:49.893 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:49.922 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:49.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:49.923 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:50.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:50.061 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:50.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:50.062 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:50.162 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:50.193 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:50.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:50.194 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:50.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:50.332 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:50.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:50.333 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:50.434 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:50.467 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:50.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:50.468 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:50.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[29647]: 4.c scrub starts 2024-09-11T08:59:50.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[29647]: 4.c scrub ok 2024-09-11T08:59:50.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:59:50.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:59:50.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[29647]: osdmap e418: 8 total, 8 up, 8 in 2024-09-11T08:59:50.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[29647]: pgmap v969: 168 pgs: 168 active+clean; 3.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T08:59:50.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[29647]: 3.8 scrub starts 2024-09-11T08:59:50.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[29647]: 3.8 scrub ok 2024-09-11T08:59:50.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[29647]: 5.12 scrub starts 2024-09-11T08:59:50.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[29647]: 5.12 scrub ok 2024-09-11T08:59:50.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[41114]: 4.c scrub starts 2024-09-11T08:59:50.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[41114]: 4.c scrub ok 2024-09-11T08:59:50.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:59:50.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:59:50.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[41114]: osdmap e418: 8 total, 8 up, 8 in 2024-09-11T08:59:50.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[41114]: pgmap v969: 168 pgs: 168 active+clean; 3.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T08:59:50.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[41114]: 3.8 scrub starts 2024-09-11T08:59:50.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[41114]: 3.8 scrub ok 2024-09-11T08:59:50.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[41114]: 5.12 scrub starts 2024-09-11T08:59:50.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:50 smithi092 ceph-mon[41114]: 5.12 scrub ok 2024-09-11T08:59:50.568 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:50.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:50.608 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:50.608 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:50.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:50 smithi145 ceph-mon[33384]: 4.c scrub starts 2024-09-11T08:59:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:50 smithi145 ceph-mon[33384]: 4.c scrub ok 2024-09-11T08:59:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:50 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:59:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:50 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:59:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:50 smithi145 ceph-mon[33384]: osdmap e418: 8 total, 8 up, 8 in 2024-09-11T08:59:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:50 smithi145 ceph-mon[33384]: pgmap v969: 168 pgs: 168 active+clean; 3.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T08:59:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:50 smithi145 ceph-mon[33384]: 3.8 scrub starts 2024-09-11T08:59:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:50 smithi145 ceph-mon[33384]: 3.8 scrub ok 2024-09-11T08:59:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:50 smithi145 ceph-mon[33384]: 5.12 scrub starts 2024-09-11T08:59:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:50 smithi145 ceph-mon[33384]: 5.12 scrub ok 2024-09-11T08:59:50.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:50.746 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:50.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:50.748 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:50.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:50.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:50.879 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:50.879 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:50.980 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:51.001 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:51.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:51.002 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:51.103 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:51.131 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:51.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:51.132 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:51.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:51.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:51.272 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:51.273 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:51.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:51.407 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:51.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:51.408 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:51.509 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:51.541 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:51.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:51.542 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:51.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:51 smithi092 ceph-mon[29647]: 3.5 scrub starts 2024-09-11T08:59:51.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:51 smithi092 ceph-mon[29647]: 3.5 scrub ok 2024-09-11T08:59:51.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 08:59:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:59:51] "GET /metrics HTTP/1.1" 200 225146 "" "Prometheus/2.43.0" 2024-09-11T08:59:51.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:51 smithi092 ceph-mon[41114]: 3.5 scrub starts 2024-09-11T08:59:51.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:51 smithi092 ceph-mon[41114]: 3.5 scrub ok 2024-09-11T08:59:51.643 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:51.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:51 smithi145 ceph-mon[33384]: 3.5 scrub starts 2024-09-11T08:59:51.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:51 smithi145 ceph-mon[33384]: 3.5 scrub ok 2024-09-11T08:59:51.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:51.677 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:51.677 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:51.778 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:51.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:51.809 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:51.809 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:51.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:51.945 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:51.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:51.946 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:52.047 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:52.082 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:52.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:52.083 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:52.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:52.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:52.230 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:52.230 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:52.331 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:52.360 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:52.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:52.361 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:52.462 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:52.492 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:52.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:52.493 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:52.593 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:52.629 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:52.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:52.631 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:52 smithi145 ceph-mon[33384]: 4.5 scrub starts 2024-09-11T08:59:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:52 smithi145 ceph-mon[33384]: 4.5 scrub ok 2024-09-11T08:59:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:52 smithi145 ceph-mon[33384]: 11.0 scrub ok 2024-09-11T08:59:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:52 smithi145 ceph-mon[33384]: pgmap v970: 168 pgs: 168 active+clean; 3.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T08:59:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:52 smithi145 ceph-mon[33384]: 3.b scrub starts 2024-09-11T08:59:52.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:52 smithi145 ceph-mon[33384]: 3.b scrub ok 2024-09-11T08:59:52.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:52 smithi092 ceph-mon[29647]: 4.5 scrub starts 2024-09-11T08:59:52.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:52 smithi092 ceph-mon[29647]: 4.5 scrub ok 2024-09-11T08:59:52.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:52 smithi092 ceph-mon[29647]: 11.0 scrub ok 2024-09-11T08:59:52.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:52 smithi092 ceph-mon[29647]: pgmap v970: 168 pgs: 168 active+clean; 3.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T08:59:52.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:52 smithi092 ceph-mon[29647]: 3.b scrub starts 2024-09-11T08:59:52.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:52 smithi092 ceph-mon[29647]: 3.b scrub ok 2024-09-11T08:59:52.681 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:52 smithi092 ceph-mon[41114]: 4.5 scrub starts 2024-09-11T08:59:52.681 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:52 smithi092 ceph-mon[41114]: 4.5 scrub ok 2024-09-11T08:59:52.681 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:52 smithi092 ceph-mon[41114]: 11.0 scrub ok 2024-09-11T08:59:52.682 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:52 smithi092 ceph-mon[41114]: pgmap v970: 168 pgs: 168 active+clean; 3.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T08:59:52.682 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:52 smithi092 ceph-mon[41114]: 3.b scrub starts 2024-09-11T08:59:52.682 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:52 smithi092 ceph-mon[41114]: 3.b scrub ok 2024-09-11T08:59:52.731 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:52.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:52.761 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:52.761 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:52.862 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:52.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:52.895 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:52.895 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:52.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:53.039 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:53.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:53.040 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:53.140 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:53.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 08:59:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:08:59:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:59:53.174 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:53.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:53.175 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:53.225 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T08:59:53.226 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T08:59:53.226 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T08:59:53.226 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T08:59:53.226 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T08:59:53.226 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T08:59:53.226 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T08:59:53.226 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T08:59:53.226 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T08:59:53.226 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T08:59:53.226 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T08:59:53.227 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T08:59:53.227 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T08:59:53.227 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T08:59:53.227 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T08:59:53.227 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T08:59:53.276 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:53.312 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:53.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:53.313 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:53.413 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:53.446 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:53.453 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:53.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:53.454 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:53.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:53.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[29647]: 6.14 deep-scrub starts 2024-09-11T08:59:53.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[29647]: 6.14 deep-scrub ok 2024-09-11T08:59:53.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:59:53.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:59:53.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:59:53.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:59:53.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[29647]: osdmap e419: 8 total, 8 up, 8 in 2024-09-11T08:59:53.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:59:53.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[29647]: 5.16 scrub starts 2024-09-11T08:59:53.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[29647]: 5.16 scrub ok 2024-09-11T08:59:53.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[29647]: 6.1f deep-scrub starts 2024-09-11T08:59:53.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[29647]: 6.1f deep-scrub ok 2024-09-11T08:59:53.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1281054245' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:59:53.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[41114]: 6.14 deep-scrub starts 2024-09-11T08:59:53.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[41114]: 6.14 deep-scrub ok 2024-09-11T08:59:53.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:59:53.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:59:53.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:59:53.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:59:53.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[41114]: osdmap e419: 8 total, 8 up, 8 in 2024-09-11T08:59:53.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:59:53.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[41114]: 5.16 scrub starts 2024-09-11T08:59:53.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[41114]: 5.16 scrub ok 2024-09-11T08:59:53.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[41114]: 6.1f deep-scrub starts 2024-09-11T08:59:53.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[41114]: 6.1f deep-scrub ok 2024-09-11T08:59:53.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:53 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1281054245' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:59:53.591 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:53.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:53.592 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:53.665 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:53 smithi145 ceph-mon[33384]: 6.14 deep-scrub starts 2024-09-11T08:59:53.665 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:53 smithi145 ceph-mon[33384]: 6.14 deep-scrub ok 2024-09-11T08:59:53.665 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:53 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:59:53.665 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:53 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:59:53.665 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:53 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:59:53.665 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:53 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:59:53.665 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:53 smithi145 ceph-mon[33384]: osdmap e419: 8 total, 8 up, 8 in 2024-09-11T08:59:53.666 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:53 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T08:59:53.666 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:53 smithi145 ceph-mon[33384]: 5.16 scrub starts 2024-09-11T08:59:53.666 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:53 smithi145 ceph-mon[33384]: 5.16 scrub ok 2024-09-11T08:59:53.666 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:53 smithi145 ceph-mon[33384]: 6.1f deep-scrub starts 2024-09-11T08:59:53.666 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:53 smithi145 ceph-mon[33384]: 6.1f deep-scrub ok 2024-09-11T08:59:53.666 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:53 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1281054245' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T08:59:53.693 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:53.727 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:53.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:53.728 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:53.829 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:53.866 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:53.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:53.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:53.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:54.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:54.009 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:54.009 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:54.110 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:54.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:54.145 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:54.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:54.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:54.286 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:54.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:54.287 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:54.388 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:54.420 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:54.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:54.421 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:54.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:54.556 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T08:59:54.563 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:54.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:54.564 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:54.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:54 smithi092 ceph-mon[29647]: pgmap v972: 168 pgs: 168 active+clean; 3.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.57k op/s 2024-09-11T08:59:54.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:54 smithi092 ceph-mon[41114]: pgmap v972: 168 pgs: 168 active+clean; 3.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.57k op/s 2024-09-11T08:59:54.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:54 smithi145 ceph-mon[33384]: pgmap v972: 168 pgs: 168 active+clean; 3.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.57k op/s 2024-09-11T08:59:54.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:54.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:54.705 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:54.705 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:54.806 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:54.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:54.842 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:54.842 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:54.943 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:54.978 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:54.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:54.979 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:55.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:55.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:55.117 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:55.118 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:55.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:55.259 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:55.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:55.260 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:55.361 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:55.399 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:55.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:55.400 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:55.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:55.534 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:55.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:55.535 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:55.636 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:55.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:55 smithi145 ceph-mon[33384]: 6.17 scrub starts 2024-09-11T08:59:55.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:55 smithi145 ceph-mon[33384]: 6.17 scrub ok 2024-09-11T08:59:55.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:55 smithi145 ceph-mon[33384]: 6.9 scrub starts 2024-09-11T08:59:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:55 smithi145 ceph-mon[33384]: 6.9 scrub ok 2024-09-11T08:59:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:55 smithi145 ceph-mon[33384]: 1.0 scrub starts 2024-09-11T08:59:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:55 smithi145 ceph-mon[33384]: 1.0 scrub ok 2024-09-11T08:59:55.664 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:55.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:55.665 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:55.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:55 smithi092 ceph-mon[29647]: 6.17 scrub starts 2024-09-11T08:59:55.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:55 smithi092 ceph-mon[29647]: 6.17 scrub ok 2024-09-11T08:59:55.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:55 smithi092 ceph-mon[29647]: 6.9 scrub starts 2024-09-11T08:59:55.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:55 smithi092 ceph-mon[29647]: 6.9 scrub ok 2024-09-11T08:59:55.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:55 smithi092 ceph-mon[29647]: 1.0 scrub starts 2024-09-11T08:59:55.758 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:55 smithi092 ceph-mon[29647]: 1.0 scrub ok 2024-09-11T08:59:55.759 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:55 smithi092 ceph-mon[41114]: 6.17 scrub starts 2024-09-11T08:59:55.759 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:55 smithi092 ceph-mon[41114]: 6.17 scrub ok 2024-09-11T08:59:55.759 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:55 smithi092 ceph-mon[41114]: 6.9 scrub starts 2024-09-11T08:59:55.759 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:55 smithi092 ceph-mon[41114]: 6.9 scrub ok 2024-09-11T08:59:55.759 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:55 smithi092 ceph-mon[41114]: 1.0 scrub starts 2024-09-11T08:59:55.759 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:55 smithi092 ceph-mon[41114]: 1.0 scrub ok 2024-09-11T08:59:55.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:55.803 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.0 reset_purged_snaps_last 2024-09-11T08:59:55.808 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:55.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:55.809 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:55.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:55.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:55.958 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:56.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:56.099 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:56.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:56.100 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:56.201 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:56.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:56.240 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:56.240 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:56.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:56.369 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:56.401 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:56.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:56.402 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:56.503 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:56.545 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:56.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:56.546 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:56.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:56.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:56 smithi145 ceph-mon[33384]: pgmap v973: 168 pgs: 168 active+clean; 3.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 762 op/s 2024-09-11T08:59:56.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:56 smithi145 ceph-mon[33384]: 1.3 scrub starts 2024-09-11T08:59:56.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:56 smithi145 ceph-mon[33384]: 1.3 scrub ok 2024-09-11T08:59:56.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T08:59:56.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:56.688 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:56.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:56 smithi092 ceph-mon[29647]: pgmap v973: 168 pgs: 168 active+clean; 3.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 762 op/s 2024-09-11T08:59:56.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:56 smithi092 ceph-mon[29647]: 1.3 scrub starts 2024-09-11T08:59:56.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:56 smithi092 ceph-mon[29647]: 1.3 scrub ok 2024-09-11T08:59:56.723 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:56 smithi092 ceph-mon[41114]: pgmap v973: 168 pgs: 168 active+clean; 3.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 762 op/s 2024-09-11T08:59:56.723 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:56 smithi092 ceph-mon[41114]: 1.3 scrub starts 2024-09-11T08:59:56.723 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:56 smithi092 ceph-mon[41114]: 1.3 scrub ok 2024-09-11T08:59:56.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:56.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:56.831 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:56.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:56.973 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:56.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:56.976 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:57.076 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:57.108 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:57.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:57.109 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:57.210 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:57.246 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:57.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:57.247 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:57.348 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:57.381 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:57.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:57.382 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:57.483 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:57.512 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:57.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:57.513 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:57.531 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:57 smithi092 ceph-mon[29647]: 3.1 scrub starts 2024-09-11T08:59:57.531 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:57 smithi092 ceph-mon[29647]: 3.1 scrub ok 2024-09-11T08:59:57.531 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:57 smithi092 ceph-mon[41114]: 3.1 scrub starts 2024-09-11T08:59:57.531 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:57 smithi092 ceph-mon[41114]: 3.1 scrub ok 2024-09-11T08:59:57.615 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T08:59:57.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:57.646 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:57.646 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T08:59:57.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:57 smithi145 ceph-mon[33384]: 3.1 scrub starts 2024-09-11T08:59:57.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:57 smithi145 ceph-mon[33384]: 3.1 scrub ok 2024-09-11T08:59:57.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 08:59:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T08:59:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:59:57.747 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:57.791 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:57.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:57.792 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:57.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:57.937 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:57.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:57.938 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:58.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:58.075 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:58.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:58.076 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:58.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:58.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:58.209 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:58.209 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:58.310 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:58.348 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:58.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:58.349 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:58.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:58.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:58.487 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:58.487 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:58.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:58.630 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:58.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:58.631 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:58.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:58 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:58.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:58 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T08:59:58.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:58 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T08:59:58.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:58 smithi145 ceph-mon[33384]: pgmap v974: 168 pgs: 168 active+clean; 2.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.55k op/s 2024-09-11T08:59:58.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:58 smithi145 ceph-mon[33384]: 5.19 scrub starts 2024-09-11T08:59:58.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:58 smithi145 ceph-mon[33384]: 5.19 scrub ok 2024-09-11T08:59:58.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T08:59:58.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:58.774 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:58.774 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T08:59:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:58 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:58 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T08:59:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:58 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T08:59:58.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:58 smithi092 ceph-mon[29647]: pgmap v974: 168 pgs: 168 active+clean; 2.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.55k op/s 2024-09-11T08:59:58.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:58 smithi092 ceph-mon[29647]: 5.19 scrub starts 2024-09-11T08:59:58.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:58 smithi092 ceph-mon[29647]: 5.19 scrub ok 2024-09-11T08:59:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:58 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:59:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:58 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T08:59:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:58 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T08:59:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:58 smithi092 ceph-mon[41114]: pgmap v974: 168 pgs: 168 active+clean; 2.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.55k op/s 2024-09-11T08:59:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:58 smithi092 ceph-mon[41114]: 5.19 scrub starts 2024-09-11T08:59:58.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:58 smithi092 ceph-mon[41114]: 5.19 scrub ok 2024-09-11T08:59:58.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T08:59:58.907 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:58.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:58.908 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T08:59:59.009 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:59.038 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:59.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:59.039 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:59.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:59.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:59.178 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T08:59:59.279 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T08:59:59.311 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:59.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:59.312 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T08:59:59.413 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:59.424 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T08:59:59.424 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T08:59:59.424 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T08:59:59.424 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T08:59:59.424 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T08:59:59.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:59.442 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:59.442 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:59.527 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:59 smithi092 ceph-mon[29647]: osdmap e420: 8 total, 8 up, 8 in 2024-09-11T08:59:59.527 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:59 smithi092 ceph-mon[29647]: 5.4 scrub starts 2024-09-11T08:59:59.527 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 08:59:59 smithi092 ceph-mon[29647]: 5.4 scrub ok 2024-09-11T08:59:59.527 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:59 smithi092 ceph-mon[41114]: osdmap e420: 8 total, 8 up, 8 in 2024-09-11T08:59:59.527 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:59 smithi092 ceph-mon[41114]: 5.4 scrub starts 2024-09-11T08:59:59.528 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 08:59:59 smithi092 ceph-mon[41114]: 5.4 scrub ok 2024-09-11T08:59:59.543 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T08:59:59.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:59.576 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:59.576 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T08:59:59.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:59 smithi145 ceph-mon[33384]: osdmap e420: 8 total, 8 up, 8 in 2024-09-11T08:59:59.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:59 smithi145 ceph-mon[33384]: 5.4 scrub starts 2024-09-11T08:59:59.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 08:59:59 smithi145 ceph-mon[33384]: 5.4 scrub ok 2024-09-11T08:59:59.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T08:59:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:59.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:59.718 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T08:59:59.819 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T08:59:59.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:59.850 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T08:59:59.851 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T08:59:59.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T08:59:59.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:59:59.998 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T08:59:59.998 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:00.100 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:00.128 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:00.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:00.129 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:00.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:00.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:00.273 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:00.374 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:00.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:00.418 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:00.418 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:00.520 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:00.550 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:00.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:00.551 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:00.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-11T09:00:00.652 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:00.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:00 smithi145 ceph-mon[33384]: 3.c scrub starts 2024-09-11T09:00:00.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:00 smithi145 ceph-mon[33384]: 3.c scrub ok 2024-09-11T09:00:00.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:00 smithi145 ceph-mon[33384]: pgmap v976: 168 pgs: 168 active+clean; 2.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T09:00:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:00 smithi145 ceph-mon[33384]: overall HEALTH_OK 2024-09-11T09:00:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:00 smithi145 ceph-mon[33384]: 5.1d scrub starts 2024-09-11T09:00:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:00 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:00:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:00 smithi145 ceph-mon[33384]: 5.1d scrub ok 2024-09-11T09:00:00.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:00:00.683 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:00.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:00.684 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:00.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[29647]: 3.c scrub starts 2024-09-11T09:00:00.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[29647]: 3.c scrub ok 2024-09-11T09:00:00.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[29647]: pgmap v976: 168 pgs: 168 active+clean; 2.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T09:00:00.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[29647]: overall HEALTH_OK 2024-09-11T09:00:00.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[29647]: 5.1d scrub starts 2024-09-11T09:00:00.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:00:00.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[29647]: 5.1d scrub ok 2024-09-11T09:00:00.746 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:00:00.746 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[41114]: 3.c scrub starts 2024-09-11T09:00:00.746 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[41114]: 3.c scrub ok 2024-09-11T09:00:00.746 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[41114]: pgmap v976: 168 pgs: 168 active+clean; 2.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T09:00:00.746 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[41114]: overall HEALTH_OK 2024-09-11T09:00:00.746 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[41114]: 5.1d scrub starts 2024-09-11T09:00:00.746 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:00:00.747 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[41114]: 5.1d scrub ok 2024-09-11T09:00:00.747 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:00:00.785 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:00.810 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:00.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:00.811 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:00.912 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:00.940 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:00.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:00.941 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:01.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:01.077 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:01.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:01.078 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:01.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:01.214 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:01.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:01.215 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:01.316 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:01.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:01.346 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:01.347 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:01.448 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:01.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:01 smithi092 ceph-mon[29647]: 4.14 scrub starts 2024-09-11T09:00:01.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:01 smithi092 ceph-mon[29647]: 4.14 scrub ok 2024-09-11T09:00:01.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:01 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T09:00:01.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:01 smithi092 ceph-mon[29647]: 4.1c scrub ok 2024-09-11T09:00:01.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:01 smithi092 ceph-mon[41114]: 4.14 scrub starts 2024-09-11T09:00:01.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:01 smithi092 ceph-mon[41114]: 4.14 scrub ok 2024-09-11T09:00:01.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:01 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T09:00:01.450 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:01 smithi092 ceph-mon[41114]: 4.1c scrub ok 2024-09-11T09:00:01.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:01.479 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:01.479 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:01.580 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:01.605 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:01.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:01.606 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:01.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:01 smithi145 ceph-mon[33384]: 4.14 scrub starts 2024-09-11T09:00:01.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:01 smithi145 ceph-mon[33384]: 4.14 scrub ok 2024-09-11T09:00:01.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:01 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T09:00:01.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:01 smithi145 ceph-mon[33384]: 4.1c scrub ok 2024-09-11T09:00:01.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:01.745 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:01.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:01.746 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:01.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:00:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:00:01] "GET /metrics HTTP/1.1" 200 225160 "" "Prometheus/2.43.0" 2024-09-11T09:00:01.847 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:01.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:01.879 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:01.879 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:01.980 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:02.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:02.011 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:02.011 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:02.112 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:02.144 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:02.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:02.145 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:02.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:02.282 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:02.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:02.283 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:02.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:02.406 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:02.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:02.407 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:02.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:02.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:02.543 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:02.543 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:02.644 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:02 smithi145 ceph-mon[33384]: 5.e scrub starts 2024-09-11T09:00:02.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:02 smithi145 ceph-mon[33384]: 5.e scrub ok 2024-09-11T09:00:02.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:02 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4102545675' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T09:00:02.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:02 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T09:00:02.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:02 smithi145 ceph-mon[33384]: 5.1b scrub starts 2024-09-11T09:00:02.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:02 smithi145 ceph-mon[33384]: 5.1b scrub ok 2024-09-11T09:00:02.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:02 smithi145 ceph-mon[33384]: pgmap v977: 168 pgs: 168 active+clean; 2.7 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T09:00:02.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:02 smithi145 ceph-mon[33384]: 3.10 scrub starts 2024-09-11T09:00:02.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:02 smithi145 ceph-mon[33384]: 3.10 scrub ok 2024-09-11T09:00:02.671 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:02.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:02.673 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:02.773 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:02.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:02.805 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:02.805 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:02.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[29647]: 5.e scrub starts 2024-09-11T09:00:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[29647]: 5.e scrub ok 2024-09-11T09:00:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4102545675' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T09:00:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T09:00:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[29647]: 5.1b scrub starts 2024-09-11T09:00:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[29647]: 5.1b scrub ok 2024-09-11T09:00:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[29647]: pgmap v977: 168 pgs: 168 active+clean; 2.7 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T09:00:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[29647]: 3.10 scrub starts 2024-09-11T09:00:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[29647]: 3.10 scrub ok 2024-09-11T09:00:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[41114]: 5.e scrub starts 2024-09-11T09:00:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[41114]: 5.e scrub ok 2024-09-11T09:00:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4102545675' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T09:00:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T09:00:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[41114]: 5.1b scrub starts 2024-09-11T09:00:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[41114]: 5.1b scrub ok 2024-09-11T09:00:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[41114]: pgmap v977: 168 pgs: 168 active+clean; 2.7 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T09:00:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[41114]: 3.10 scrub starts 2024-09-11T09:00:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:02 smithi092 ceph-mon[41114]: 3.10 scrub ok 2024-09-11T09:00:02.906 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:02.940 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:02.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:02.941 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:03.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:03.079 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:03.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:03.080 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:03.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:00:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:00:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:00:03.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:03.205 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:03.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:03.206 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:03.307 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:03.340 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:03.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:03.343 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:03.444 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:03.481 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:03.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:03.482 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:03.583 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:03.624 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:03.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:03.625 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:03.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:03 smithi145 ceph-mon[33384]: 6.e scrub starts 2024-09-11T09:00:03.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:03 smithi145 ceph-mon[33384]: 6.e scrub ok 2024-09-11T09:00:03.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:03 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T09:00:03.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:03 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T09:00:03.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:03 smithi145 ceph-mon[33384]: osdmap e421: 8 total, 8 up, 8 in 2024-09-11T09:00:03.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:03.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:03.764 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:03.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:03.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:03 smithi092 ceph-mon[29647]: 6.e scrub starts 2024-09-11T09:00:03.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:03 smithi092 ceph-mon[29647]: 6.e scrub ok 2024-09-11T09:00:03.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:03 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T09:00:03.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:03 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T09:00:03.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:03 smithi092 ceph-mon[29647]: osdmap e421: 8 total, 8 up, 8 in 2024-09-11T09:00:03.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:03 smithi092 ceph-mon[41114]: 6.e scrub starts 2024-09-11T09:00:03.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:03 smithi092 ceph-mon[41114]: 6.e scrub ok 2024-09-11T09:00:03.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:03 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T09:00:03.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:03 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T09:00:03.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:03 smithi092 ceph-mon[41114]: osdmap e421: 8 total, 8 up, 8 in 2024-09-11T09:00:03.853 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:03.865 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:03.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:03.897 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:03.897 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:03.979 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T09:00:03.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:04.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:04.036 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:04.036 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:04.137 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:04.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:04.171 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:04.172 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:04.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:04.306 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:04.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:04.307 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:04.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:04.444 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:04.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:04.445 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:04.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:04.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:04 smithi092 ceph-mon[29647]: pgmap v979: 168 pgs: 168 active+clean; 2.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-11T09:00:04.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:04 smithi092 ceph-mon[41114]: pgmap v979: 168 pgs: 168 active+clean; 2.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-11T09:00:04.578 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:04.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:04.579 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:04.657 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:04 smithi145 ceph-mon[33384]: pgmap v979: 168 pgs: 168 active+clean; 2.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-11T09:00:04.680 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:04.701 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:04.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:04.702 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:04.803 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:04.826 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:04.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:04.827 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:04.912 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 09:00:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a[66445]: logger=cleanup t=2024-09-11T09:00:04.657995375Z level=info msg="Completed cleanup jobs" duration=11.86166ms 2024-09-11T09:00:04.928 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:04.962 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:04.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:04.963 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:05.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:05.099 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:05.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:05.100 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:05.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:05.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:05.237 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:05.237 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:05.339 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:05.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:05.379 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:05.380 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:05.480 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:05.511 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:05.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:05.512 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:05.613 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:05.644 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:05.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:05.645 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:05.748 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:05.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:05.786 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:05.786 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:05.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:05.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:05.909 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:05.909 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:06.010 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:06.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:06.047 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:06.048 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:06.149 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:06.174 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:06.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:06.176 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:06.276 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:06.304 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:06.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:06.305 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:06.406 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:06.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:06.444 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:06.444 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:06.545 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:06.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:06 smithi092 ceph-mon[29647]: pgmap v980: 168 pgs: 168 active+clean; 2.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 817 op/s 2024-09-11T09:00:06.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:06 smithi092 ceph-mon[41114]: pgmap v980: 168 pgs: 168 active+clean; 2.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 817 op/s 2024-09-11T09:00:06.578 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:06.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:06.580 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:06.680 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:06.709 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:06.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:06.710 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:06.811 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:06.840 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:06.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:06.841 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:06.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:06 smithi145 ceph-mon[33384]: pgmap v980: 168 pgs: 168 active+clean; 2.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 817 op/s 2024-09-11T09:00:06.942 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:06.972 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:06.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:06.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:07.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:07.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:07.112 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:07.112 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:07.213 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:07.247 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:07.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:07.249 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:07.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:07.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:07.385 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:07.385 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:07.486 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:07.487 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T09:00:07.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:07.544 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:07.544 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:07.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:07.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:07 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:00:07.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:07 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:00:07.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:00:07 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:00:07.690 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:07.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:07.691 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:07.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:07 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:00:07.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:07 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:00:07.760 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:07 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:00:07.760 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:07 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:00:07.791 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:07.818 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:07.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:07.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:07.920 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:07.950 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:07.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:07.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:07.971 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:08.052 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:08.078 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:08.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:08.079 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:08.180 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:08.213 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:08.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:08.214 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:08.315 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:08.341 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:08.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:08.342 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:08.443 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:08.475 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:08.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:08.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:08.577 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:08.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:08.601 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:08.602 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:08.702 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:08.733 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:08.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:08.734 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:08.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:08 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:08.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:08 smithi092 ceph-mon[29647]: pgmap v981: 168 pgs: 168 active+clean; 2.1 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-11T09:00:08.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:08 smithi092 ceph-mon[29647]: osdmap e422: 8 total, 8 up, 8 in 2024-09-11T09:00:08.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:08 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:08.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:08 smithi092 ceph-mon[41114]: pgmap v981: 168 pgs: 168 active+clean; 2.1 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-11T09:00:08.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:08 smithi092 ceph-mon[41114]: osdmap e422: 8 total, 8 up, 8 in 2024-09-11T09:00:08.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:08.873 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:08.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:08.874 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:08 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:08 smithi145 ceph-mon[33384]: pgmap v981: 168 pgs: 168 active+clean; 2.1 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-11T09:00:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:08 smithi145 ceph-mon[33384]: osdmap e422: 8 total, 8 up, 8 in 2024-09-11T09:00:08.975 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:09.007 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:09.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:09.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:09.109 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:09.137 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:09.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:09.139 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:09.239 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:09.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:09.279 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:09.279 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:09.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:09.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:09.418 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:09.418 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:09.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:09.552 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:09.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:09.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:09.654 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:09.680 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:09.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:09.680 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:09.781 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:09.809 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:09.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:09.810 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:09.911 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:09.941 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:09.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:09.942 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:10.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:10.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:10.087 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:10.087 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:10.188 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:10.221 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:10.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:10.222 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:10.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:10.360 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:10.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:10.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:10.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:10 smithi145 ceph-mon[33384]: pgmap v983: 168 pgs: 168 active+clean; 2.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T09:00:10.463 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:10.502 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:10.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:10.503 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:10.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:10 smithi092 ceph-mon[29647]: pgmap v983: 168 pgs: 168 active+clean; 2.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T09:00:10.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:10 smithi092 ceph-mon[41114]: pgmap v983: 168 pgs: 168 active+clean; 2.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T09:00:10.603 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:10.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:10.637 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:10.638 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:10.738 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:10.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:10.773 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:10.773 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:10.831 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:10.874 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:10.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:10.916 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:10.916 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:11.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:11.049 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:11.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:11.050 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:11.152 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:11.187 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:11.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:11.188 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:11.289 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:11.320 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:11.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:11.321 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:11.422 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:11.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.6 reset_purged_snaps_last 2024-09-11T09:00:11.455 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:11.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:11.456 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:11.556 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:11.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:11.591 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:11.591 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:11.637 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:00:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:00:11] "GET /metrics HTTP/1.1" 200 225160 "" "Prometheus/2.43.0" 2024-09-11T09:00:11.691 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:11.735 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:11.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:11.737 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:11.837 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:11.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:11.871 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:11.871 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:11.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:12.013 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:12.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:12.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:12.115 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:12.148 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:12.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:12.149 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:12.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:12.289 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:12.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:12.290 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:12.390 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:12.430 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:12.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:12.431 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:12.532 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:12.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:12.572 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:12.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:12.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:12.713 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:12.713 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:12.780 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:12 smithi092 ceph-mon[29647]: pgmap v984: 168 pgs: 168 active+clean; 2.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 982 op/s 2024-09-11T09:00:12.780 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:12 smithi092 ceph-mon[41114]: pgmap v984: 168 pgs: 168 active+clean; 2.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 982 op/s 2024-09-11T09:00:12.815 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:12.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:12.838 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:12.838 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:12.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:12 smithi145 ceph-mon[33384]: pgmap v984: 168 pgs: 168 active+clean; 2.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 982 op/s 2024-09-11T09:00:12.912 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:00:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:00:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:00:12.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:12.980 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:12.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:12.982 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:13.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:13.085 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T09:00:13.085 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T09:00:13.085 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T09:00:13.085 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T09:00:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:00:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:00:13.085 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:00:13.086 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:00:13.086 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T09:00:13.086 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T09:00:13.086 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T09:00:13.086 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:00:13.086 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:00:13.086 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T09:00:13.118 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T09:00:13.119 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T09:00:13.121 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:13.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:13.122 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:13.223 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:13.251 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:13.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:13.252 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:13.353 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:13.379 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:13.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:13.380 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:13.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:13.515 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:13.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:13.516 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:13.617 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:13.652 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:13.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:13.653 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:13.754 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:13.785 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:13.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:13.786 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:13.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:13 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3262585344' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T09:00:13.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:13 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3262585344' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T09:00:13.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:13.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:13 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3262585344' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T09:00:13.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:13.921 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:13.921 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:14.022 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:14.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:14.061 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:14.162 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:14.199 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:14.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:14.200 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:14.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:14.335 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:14.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:14.336 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:14.437 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:14.466 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:14.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:14.467 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:14.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:14.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:14.607 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:14.607 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:14.608 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:14 smithi092 ceph-mon[29647]: pgmap v985: 168 pgs: 168 active+clean; 1.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T09:00:14.608 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:14 smithi092 ceph-mon[41114]: pgmap v985: 168 pgs: 168 active+clean; 1.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T09:00:14.708 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:14.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:14.744 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:14.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:14.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:14.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:14.886 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:14.886 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:14.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:14 smithi145 ceph-mon[33384]: pgmap v985: 168 pgs: 168 active+clean; 1.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T09:00:14.988 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:15.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:15.021 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:15.021 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:15.122 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:15.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:15.150 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:15.150 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:15.251 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:15.278 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:15.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:15.279 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:15.380 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:15.408 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:15.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:15.409 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:15.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:15.545 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:15.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:15.546 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:15.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:15.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:15.684 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:15.684 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:15.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:15.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:15.845 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:15.845 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:15.876 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:15.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:15.985 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:15.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:15.987 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:16.088 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:16.118 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:16.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:16.119 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:16.220 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:16.257 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:16.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:16.258 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:16.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:16.382 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:16.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:16.383 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:16.386 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:16 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:00:16.386 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:16 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:00:16.386 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:16 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:00:16.386 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:16 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:00:16.386 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:16 smithi092 ceph-mon[29647]: pgmap v986: 168 pgs: 168 active+clean; 1.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T09:00:16.387 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:16 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:00:16.387 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:16 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:00:16.387 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:16 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:00:16.387 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:16 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:00:16.387 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:16 smithi092 ceph-mon[41114]: pgmap v986: 168 pgs: 168 active+clean; 1.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T09:00:16.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:16 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:00:16.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:16 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:00:16.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:16 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:00:16.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:16 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:00:16.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:16 smithi145 ceph-mon[33384]: pgmap v986: 168 pgs: 168 active+clean; 1.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-11T09:00:16.484 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:16.507 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T09:00:16.507 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:00:16.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:16.510 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:16.510 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:16.611 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:16.632 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:16.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:16.633 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:16.734 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:16.756 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:16.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:16.757 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:16.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:16.879 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:16.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:16.880 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:16.980 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:17.005 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:17.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:17.006 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:17.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:17.139 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:17.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:17.141 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:17.242 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:17.264 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:17.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:17.265 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:17.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:17 smithi092 ceph-mon[29647]: osdmap e423: 8 total, 8 up, 8 in 2024-09-11T09:00:17.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:17 smithi092 ceph-mon[41114]: osdmap e423: 8 total, 8 up, 8 in 2024-09-11T09:00:17.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:00:17.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:17.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:17 smithi145 ceph-mon[33384]: osdmap e423: 8 total, 8 up, 8 in 2024-09-11T09:00:17.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:00:17 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:00:17.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:17.426 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:17.426 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:17.527 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:17.556 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:17.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:17.557 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:17.658 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:17.682 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:17.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:17.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:17.783 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:17.818 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:17.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:17.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:17.920 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:17.954 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:17.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:17.955 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:18.056 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:18.090 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:18.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:18.091 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:18.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:18.223 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:18.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:18.224 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:18.325 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:18.358 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:18.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:18.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:18.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:18 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:18.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:18 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/146022256' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T09:00:18.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:18 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T09:00:18.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:18 smithi145 ceph-mon[33384]: pgmap v988: 168 pgs: 168 active+clean; 1.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T09:00:18.461 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:18.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:18.499 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:18.546 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:18 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:18.546 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:18 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/146022256' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T09:00:18.547 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:18 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T09:00:18.547 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:18 smithi092 ceph-mon[41114]: pgmap v988: 168 pgs: 168 active+clean; 1.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T09:00:18.547 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:18 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:18.547 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:18 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/146022256' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T09:00:18.547 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:18 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T09:00:18.547 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:18 smithi092 ceph-mon[29647]: pgmap v988: 168 pgs: 168 active+clean; 1.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T09:00:18.601 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:18.636 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:18.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:18.637 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:18.738 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:18.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:18.773 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:18.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:18.874 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:18.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:18.909 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:18.909 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:19.010 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:19.044 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:19.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:19.045 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:19.145 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:19.184 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:19.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:19.185 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:19.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:19.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:19.323 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:19.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:19 smithi145 ceph-mon[33384]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T09:00:19.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:19 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T09:00:19.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:19 smithi145 ceph-mon[33384]: osdmap e424: 8 total, 8 up, 8 in 2024-09-11T09:00:19.424 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:19.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:19.453 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:19.454 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:19.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:19.564 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-11T09:00:19.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:19 smithi092 ceph-mon[29647]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T09:00:19.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:19 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T09:00:19.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:19 smithi092 ceph-mon[29647]: osdmap e424: 8 total, 8 up, 8 in 2024-09-11T09:00:19.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:19 smithi092 ceph-mon[41114]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T09:00:19.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:19 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T09:00:19.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:19 smithi092 ceph-mon[41114]: osdmap e424: 8 total, 8 up, 8 in 2024-09-11T09:00:19.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:19.586 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:19.586 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:19.687 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:19.718 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:19.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:19.719 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:19.820 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:19.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:19.858 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:19.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:19.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:19.994 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:19.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:19.995 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:20.096 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:20.129 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:20.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:20.130 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:20.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:20.267 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:20.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:20.268 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:20.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:20 smithi092 ceph-mon[29647]: pgmap v990: 168 pgs: 168 active+clean; 1.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-11T09:00:20.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:20 smithi092 ceph-mon[41114]: pgmap v990: 168 pgs: 168 active+clean; 1.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-11T09:00:20.369 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:20.398 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:20.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:20.399 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:20.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:20 smithi145 ceph-mon[33384]: pgmap v990: 168 pgs: 168 active+clean; 1.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-11T09:00:20.501 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:20.531 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:20.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:20.532 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:20.633 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:20.668 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:20.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:20.670 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:20.770 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:20.802 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:20.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:20.803 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:20.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:20.946 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:20.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:20.947 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:20.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T09:00:21.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:21.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:21.093 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:21.094 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:21.194 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:21.229 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:21.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:21.230 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:21.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:21.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:21.372 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:21.472 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:21.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:21.499 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:21.500 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:21.601 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:21.633 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:00:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:00:21] "GET /metrics HTTP/1.1" 200 225113 "" "Prometheus/2.43.0" 2024-09-11T09:00:21.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:21.650 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:21.650 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:21.751 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:21.772 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:21.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:21.796 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:21.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:21.938 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:21.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:21.939 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:22.040 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:22.072 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:22.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:22.073 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:22.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:22.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:22.214 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:22.315 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:22.347 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:22.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:22.348 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:22.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T09:00:22.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:22.491 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:22.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:22.493 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:22.593 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:22.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:22.625 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:22.625 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:22.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:22.763 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:22.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:22.764 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:22.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:22 smithi092 ceph-mon[29647]: pgmap v991: 168 pgs: 168 active+clean; 1.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T09:00:22.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:22 smithi092 ceph-mon[41114]: pgmap v991: 168 pgs: 168 active+clean; 1.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T09:00:22.829 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:22.865 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:22.891 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:22.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:22.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:22.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:22 smithi145 ceph-mon[33384]: pgmap v991: 168 pgs: 168 active+clean; 1.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T09:00:22.912 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:00:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:00:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:00:22.994 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:23.024 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:23.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:23.026 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:23.126 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:23.155 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:23.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:23.156 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:23.257 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:23.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:23.292 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:23.292 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:23.393 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:23.430 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:23.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:23.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:23.532 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:23.559 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:23.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:23.561 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:23.661 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:23.690 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:23.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:23.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:23.792 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:23.826 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:23.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:23.827 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:23.928 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:23.967 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:23.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:23.968 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:24.069 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:24.099 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:24.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:24.100 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:24.201 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:24.228 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:24.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:24.229 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:24.330 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:24.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:24.368 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:24.368 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:24.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:24.510 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:24.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:24.511 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:24.612 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:24.642 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:24.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:24.643 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:24.720 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:24 smithi092 ceph-mon[29647]: pgmap v992: 168 pgs: 168 active+clean; 1.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.48k op/s 2024-09-11T09:00:24.720 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:24 smithi092 ceph-mon[41114]: pgmap v992: 168 pgs: 168 active+clean; 1.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.48k op/s 2024-09-11T09:00:24.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:24.782 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:24.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:24.783 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:24.884 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:24.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:24 smithi145 ceph-mon[33384]: pgmap v992: 168 pgs: 168 active+clean; 1.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.48k op/s 2024-09-11T09:00:24.920 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:24.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:24.921 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:25.021 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:25.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:25.056 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:25.056 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:25.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:25.191 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:25.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:25.193 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:25.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:25.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:25.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:25.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:25.465 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:25.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:25.466 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:25.567 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:25.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:25.601 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:25.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:25.627 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:25.703 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:25.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:25.741 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:25.742 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:25.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:25.882 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:25.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:25.883 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:25.984 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:26.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:26.021 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:26.021 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:26.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:26.152 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:26.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:26.153 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:26.254 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:26.290 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:26.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:26.291 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:26.392 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:26.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:26.428 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:26.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:26.529 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:26.562 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:26.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:26.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:26.663 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:26.707 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:26.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:26.708 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:26.780 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:26 smithi092 ceph-mon[29647]: pgmap v993: 168 pgs: 168 active+clean; 1.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T09:00:26.780 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:26 smithi092 ceph-mon[41114]: pgmap v993: 168 pgs: 168 active+clean; 1.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T09:00:26.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:26.851 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:26.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:26.852 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:26.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:26 smithi145 ceph-mon[33384]: pgmap v993: 168 pgs: 168 active+clean; 1.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T09:00:26.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:26.984 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:26.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:26.985 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:27.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:27.125 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:27.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:27.126 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:27.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:27.263 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:27.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:27.264 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:27.365 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:27.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:27.399 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:27.400 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:27.500 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:27.543 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:27.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:27.544 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:27.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:27.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:00:27 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:00:27.689 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:27.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:27.690 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:27.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:27.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:27.834 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:27.834 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:27.935 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:27.965 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:27.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:27.966 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:27.995 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T09:00:27.996 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:00:28.037 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T09:00:28.037 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:00:28.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:28.070 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T09:00:28.071 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:00:28.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:28.105 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:28.106 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:28.107 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T09:00:28.108 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:00:28.139 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T09:00:28.140 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:00:28.169 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T09:00:28.169 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:00:28.205 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T09:00:28.205 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T09:00:28.207 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:28.234 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:28.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:28.235 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:28.241 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T09:00:28.242 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T09:00:28.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:28.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:28.372 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:28.372 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:28.473 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T09:00:28.473 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T09:00:28.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.473 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15918.0:234924 11.c e424/366 11:372b8099:::benchmark_data_smithi092_205252_object113603:head v 424'14483, mlcod=424'14480)", 2024-09-11T09:00:28.474 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T09:00:28.448427+0000", 2024-09-11T09:00:28.474 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.023365230000000001, 2024-09-11T09:00:28.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023392241000000001, 2024-09-11T09:00:28.474 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:00:28.474 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T09:00:28.475 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:00:28.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:00:28.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.448427+0000", 2024-09-11T09:00:28.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:00:28.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:00:28.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.448427+0000", 2024-09-11T09:00:28.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:00:28.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:00:28.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.448426+0000", 2024-09-11T09:00:28.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T09:00:28.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:00:28.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.448433+0000", 2024-09-11T09:00:28.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7830000000000001e-06 2024-09-11T09:00:28.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:00:28.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.448449+0000", 2024-09-11T09:00:28.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5582000000000001e-05 2024-09-11T09:00:28.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:00:28.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.448454+0000", 2024-09-11T09:00:28.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5969999999999999e-06 2024-09-11T09:00:28.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:00:28.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.449174+0000", 2024-09-11T09:00:28.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071924499999999998 2024-09-11T09:00:28.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:00:28.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.449204+0000", 2024-09-11T09:00:28.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0694000000000001e-05 2024-09-11T09:00:28.480 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:28.480 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:28.480 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:28.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.481 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15918.0:234928 11.c e424/366 11:374e97d5:::benchmark_data_smithi092_205252_object113607:head v 424'14484, mlcod=424'14480)", 2024-09-11T09:00:28.481 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T09:00:28.453565+0000", 2024-09-11T09:00:28.481 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.018227766999999999, 2024-09-11T09:00:28.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018335840999999999, 2024-09-11T09:00:28.481 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:00:28.481 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T09:00:28.481 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:00:28.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:00:28.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.453565+0000", 2024-09-11T09:00:28.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:00:28.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:00:28.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.453565+0000", 2024-09-11T09:00:28.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:00:28.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:00:28.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.453564+0000", 2024-09-11T09:00:28.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T09:00:28.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:00:28.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.453572+0000", 2024-09-11T09:00:28.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3529999999999995e-06 2024-09-11T09:00:28.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:00:28.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.453588+0000", 2024-09-11T09:00:28.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5781999999999999e-05 2024-09-11T09:00:28.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:00:28.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.453594+0000", 2024-09-11T09:00:28.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4690000000000002e-06 2024-09-11T09:00:28.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:00:28.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.460963+0000", 2024-09-11T09:00:28.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073683419999999999 2024-09-11T09:00:28.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:00:28.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.460986+0000", 2024-09-11T09:00:28.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3428999999999999e-05 2024-09-11T09:00:28.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:28.485 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:28.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:28.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.485 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15918.0:234938 11.4 e424/400 11:29eac844:::benchmark_data_smithi092_205252_object113617:head v 424'14791, mlcod=424'14789)", 2024-09-11T09:00:28.485 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T09:00:28.457401+0000", 2024-09-11T09:00:28.485 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.014391796, 2024-09-11T09:00:28.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014559299, 2024-09-11T09:00:28.485 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:00:28.486 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T09:00:28.486 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:00:28.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:00:28.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.457401+0000", 2024-09-11T09:00:28.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:00:28.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:00:28.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.457401+0000", 2024-09-11T09:00:28.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:00:28.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:00:28.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.457400+0000", 2024-09-11T09:00:28.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T09:00:28.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:00:28.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.457407+0000", 2024-09-11T09:00:28.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:00:28.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:00:28.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.457423+0000", 2024-09-11T09:00:28.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5732000000000001e-05 2024-09-11T09:00:28.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:00:28.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.457428+0000", 2024-09-11T09:00:28.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4949999999999999e-06 2024-09-11T09:00:28.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:00:28.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.464291+0000", 2024-09-11T09:00:28.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068630169999999999 2024-09-11T09:00:28.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:00:28.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.464338+0000", 2024-09-11T09:00:28.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6711000000000001e-05 2024-09-11T09:00:28.489 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:28.490 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:28.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:28.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.490 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15918.0:234929 11.c e424/366 11:30818a3c:::benchmark_data_smithi092_205252_object113608:head v 424'14485, mlcod=424'14480)", 2024-09-11T09:00:28.490 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T09:00:28.458322+0000", 2024-09-11T09:00:28.490 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.013470413000000001, 2024-09-11T09:00:28.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013698297999999999, 2024-09-11T09:00:28.490 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:00:28.490 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T09:00:28.491 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:00:28.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:00:28.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.458322+0000", 2024-09-11T09:00:28.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:00:28.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:00:28.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.458322+0000", 2024-09-11T09:00:28.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:00:28.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:00:28.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.458321+0000", 2024-09-11T09:00:28.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T09:00:28.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:00:28.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.458329+0000", 2024-09-11T09:00:28.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6879999999999999e-06 2024-09-11T09:00:28.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:00:28.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.458346+0000", 2024-09-11T09:00:28.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7397999999999998e-05 2024-09-11T09:00:28.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:00:28.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.458354+0000", 2024-09-11T09:00:28.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-11T09:00:28.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:00:28.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.467051+0000", 2024-09-11T09:00:28.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0086972709999999995 2024-09-11T09:00:28.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:00:28.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.467068+0000", 2024-09-11T09:00:28.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6614999999999999e-05 2024-09-11T09:00:28.495 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:28.495 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:28.495 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:28.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.496 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15918.0:234931 11.c e424/366 11:3bca756c:::benchmark_data_smithi092_205252_object113610:head v 424'14486, mlcod=424'14480)", 2024-09-11T09:00:28.496 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T09:00:28.462170+0000", 2024-09-11T09:00:28.496 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0096224259999999999, 2024-09-11T09:00:28.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099115200000000001, 2024-09-11T09:00:28.496 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:00:28.496 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T09:00:28.496 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:00:28.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:00:28.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.462170+0000", 2024-09-11T09:00:28.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:00:28.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:00:28.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.462170+0000", 2024-09-11T09:00:28.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:00:28.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:00:28.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.462169+0000", 2024-09-11T09:00:28.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T09:00:28.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:00:28.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.462177+0000", 2024-09-11T09:00:28.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0120000000000005e-06 2024-09-11T09:00:28.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:00:28.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.462192+0000", 2024-09-11T09:00:28.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5211000000000001e-05 2024-09-11T09:00:28.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:00:28.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.462198+0000", 2024-09-11T09:00:28.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8969999999999996e-06 2024-09-11T09:00:28.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:00:28.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.470129+0000", 2024-09-11T09:00:28.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079312889999999994 2024-09-11T09:00:28.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:00:28.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.470148+0000", 2024-09-11T09:00:28.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8329e-05 2024-09-11T09:00:28.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:28.500 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:28.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:28.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.500 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15918.0:234946 11.c e424/366)", 2024-09-11T09:00:28.500 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T09:00:28.465488+0000", 2024-09-11T09:00:28.500 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0063041190000000004, 2024-09-11T09:00:28.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066493000000000003, 2024-09-11T09:00:28.501 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:00:28.501 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-11T09:00:28.501 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:00:28.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:00:28.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.465488+0000", 2024-09-11T09:00:28.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:00:28.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:00:28.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.465488+0000", 2024-09-11T09:00:28.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:00:28.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:00:28.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.465488+0000", 2024-09-11T09:00:28.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T09:00:28.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:00:28.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.465492+0000", 2024-09-11T09:00:28.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7060000000000002e-06 2024-09-11T09:00:28.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:00:28.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.465497+0000", 2024-09-11T09:00:28.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-06 2024-09-11T09:00:28.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:00:28.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.465501+0000", 2024-09-11T09:00:28.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.27e-06 2024-09-11T09:00:28.504 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:28.504 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:28.504 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:28.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.504 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.15918.0:234944 11.2 e424/366 11:4dbdf327:::benchmark_data_smithi092_205252_object113623:head v 424'14632, mlcod=424'14629)", 2024-09-11T09:00:28.504 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T09:00:28.470928+0000", 2024-09-11T09:00:28.504 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.000864195, 2024-09-11T09:00:28.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012578960000000001, 2024-09-11T09:00:28.504 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:00:28.504 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-11T09:00:28.504 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:00:28.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:00:28.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.470928+0000", 2024-09-11T09:00:28.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:00:28.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:00:28.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.470928+0000", 2024-09-11T09:00:28.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:00:28.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:00:28.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.470927+0000", 2024-09-11T09:00:28.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T09:00:28.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:00:28.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.470934+0000", 2024-09-11T09:00:28.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9319999999999999e-06 2024-09-11T09:00:28.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:00:28.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.470948+0000", 2024-09-11T09:00:28.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3579e-05 2024-09-11T09:00:28.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:00:28.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.470953+0000", 2024-09-11T09:00:28.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1710000000000001e-06 2024-09-11T09:00:28.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:00:28.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.471010+0000", 2024-09-11T09:00:28.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7043999999999998e-05 2024-09-11T09:00:28.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:28.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:28.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:00:28.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:28.471026+0000", 2024-09-11T09:00:28.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6322000000000001e-05 2024-09-11T09:00:28.507 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:28.507 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:28.507 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:28.507 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:28.508 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T09:00:28.508 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 7 2024-09-11T09:00:28.508 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T09:00:28.508 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:28.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:28.541 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:28.541 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:28.642 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:28.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:28.676 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:28.677 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:28.777 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:28.811 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:28.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:28.812 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:28.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:28 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:28.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:28 smithi092 ceph-mon[29647]: pgmap v994: 168 pgs: 168 active+clean; 625 MiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-11T09:00:28.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:28 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4069210513' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T09:00:28.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:28 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:28.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:28 smithi092 ceph-mon[41114]: pgmap v994: 168 pgs: 168 active+clean; 625 MiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-11T09:00:28.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:28 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4069210513' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T09:00:28.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:28 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:28.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:28 smithi145 ceph-mon[33384]: pgmap v994: 168 pgs: 168 active+clean; 625 MiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-11T09:00:28.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:28 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4069210513' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T09:00:28.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:28.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:28.957 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:28.957 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:29.058 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:29.092 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:29.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:29.094 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:29.194 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:29.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:29.228 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:29.228 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:29.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:29.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:29.373 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:29.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:29.491 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 7, 0, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T09:00:29.491 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:00:29.492 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-11T09:00:29.492 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-11T09:00:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:29.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:29.511 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:29.611 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:29.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:29.659 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:29.659 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:29.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:29.802 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:29.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:29.803 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:29.904 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:29.927 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:29.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:29.928 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:30.029 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:30.053 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:30.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:30.055 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:30.155 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:30.198 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T09:00:30.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:30.200 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:30.200 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:30.301 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:30.338 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:30.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:30.339 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:30.435 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:30 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4126670916' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T09:00:30.435 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:30 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T09:00:30.435 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:30 smithi092 ceph-mon[29647]: pgmap v995: 168 pgs: 168 active+clean; 625 MiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 900 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-11T09:00:30.435 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:00:30.435 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:30 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4126670916' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T09:00:30.436 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:30 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T09:00:30.436 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:30 smithi092 ceph-mon[41114]: pgmap v995: 168 pgs: 168 active+clean; 625 MiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 900 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-11T09:00:30.436 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:00:30.440 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:30.473 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:30.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-11T09:00:30.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:30.475 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:30.576 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:30.600 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:30.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:30.601 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:30.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:30 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4126670916' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T09:00:30.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:30 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T09:00:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:30 smithi145 ceph-mon[33384]: pgmap v995: 168 pgs: 168 active+clean; 625 MiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 900 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-11T09:00:30.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:00:30.702 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:30.763 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:30.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:30.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:30.865 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:30.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:30.900 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:30.900 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:31.001 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:31.036 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:31.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:31.037 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:31.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:31.206 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:31.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:31.207 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:31.216 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:31.308 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:31.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:31.349 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:31.401 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:31 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:00:31.401 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:31 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:00:31.401 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:31 smithi092 ceph-mon[29647]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T09:00:31.402 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:31 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T09:00:31.402 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:31 smithi092 ceph-mon[29647]: osdmap e425: 8 total, 8 up, 8 in 2024-09-11T09:00:31.402 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:31 smithi092 ceph-mon[29647]: 6.d scrub starts 2024-09-11T09:00:31.402 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:31 smithi092 ceph-mon[29647]: 6.d scrub ok 2024-09-11T09:00:31.402 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:31 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:00:31.402 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:31 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:00:31.402 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:31 smithi092 ceph-mon[41114]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T09:00:31.402 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:31 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T09:00:31.402 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:31 smithi092 ceph-mon[41114]: osdmap e425: 8 total, 8 up, 8 in 2024-09-11T09:00:31.402 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:31 smithi092 ceph-mon[41114]: 6.d scrub starts 2024-09-11T09:00:31.403 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:31 smithi092 ceph-mon[41114]: 6.d scrub ok 2024-09-11T09:00:31.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:31.485 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:31.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:31.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:31.533 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:31 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:00:31.533 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:31 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:00:31.533 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:31 smithi145 ceph-mon[33384]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T09:00:31.533 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:31 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T09:00:31.533 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:31 smithi145 ceph-mon[33384]: osdmap e425: 8 total, 8 up, 8 in 2024-09-11T09:00:31.533 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:31 smithi145 ceph-mon[33384]: 6.d scrub starts 2024-09-11T09:00:31.533 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:31 smithi145 ceph-mon[33384]: 6.d scrub ok 2024-09-11T09:00:31.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:31.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:31.628 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:31.729 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:00:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:00:31] "GET /metrics HTTP/1.1" 200 225076 "" "Prometheus/2.43.0" 2024-09-11T09:00:31.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:31.770 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:31.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:31.771 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:31.872 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:31.908 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:31.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:31.909 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:31.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:00:32.011 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:32.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:32.048 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:32.048 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:32.082 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:32.149 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:32.196 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:32.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:32.197 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:32.243 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:32 smithi092 ceph-mon[29647]: 11.c scrub starts 2024-09-11T09:00:32.244 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:32 smithi092 ceph-mon[29647]: 11.c scrub ok 2024-09-11T09:00:32.244 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:32 smithi092 ceph-mon[29647]: 1.a scrub starts 2024-09-11T09:00:32.244 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:32 smithi092 ceph-mon[29647]: 1.a scrub ok 2024-09-11T09:00:32.244 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:32 smithi092 ceph-mon[29647]: pgmap v997: 168 pgs: 168 active+clean; 417 MiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-11T09:00:32.244 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:32 smithi092 ceph-mon[29647]: 4.6 scrub starts 2024-09-11T09:00:32.244 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:32 smithi092 ceph-mon[29647]: 4.6 scrub ok 2024-09-11T09:00:32.244 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:32 smithi092 ceph-mon[41114]: 11.c scrub starts 2024-09-11T09:00:32.244 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:32 smithi092 ceph-mon[41114]: 11.c scrub ok 2024-09-11T09:00:32.244 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:32 smithi092 ceph-mon[41114]: 1.a scrub starts 2024-09-11T09:00:32.244 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:32 smithi092 ceph-mon[41114]: 1.a scrub ok 2024-09-11T09:00:32.245 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:32 smithi092 ceph-mon[41114]: pgmap v997: 168 pgs: 168 active+clean; 417 MiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-11T09:00:32.245 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:32 smithi092 ceph-mon[41114]: 4.6 scrub starts 2024-09-11T09:00:32.245 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:32 smithi092 ceph-mon[41114]: 4.6 scrub ok 2024-09-11T09:00:32.298 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:32.335 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:32.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:32.336 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:32.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:32 smithi145 ceph-mon[33384]: 11.c scrub starts 2024-09-11T09:00:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:32 smithi145 ceph-mon[33384]: 11.c scrub ok 2024-09-11T09:00:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:32 smithi145 ceph-mon[33384]: 1.a scrub starts 2024-09-11T09:00:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:32 smithi145 ceph-mon[33384]: 1.a scrub ok 2024-09-11T09:00:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:32 smithi145 ceph-mon[33384]: pgmap v997: 168 pgs: 168 active+clean; 417 MiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-11T09:00:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:32 smithi145 ceph-mon[33384]: 4.6 scrub starts 2024-09-11T09:00:32.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:32 smithi145 ceph-mon[33384]: 4.6 scrub ok 2024-09-11T09:00:32.437 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:32.471 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:32.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:32.472 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:32.573 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:32.609 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:32.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:32.610 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:32.710 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:32.741 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:32.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:32.742 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:32.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:32.885 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:32.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:32.887 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:32.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:33.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:33.032 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:33.033 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:33.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:33.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:00:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:00:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:00:33.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T09:00:33.197 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:33.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:33.198 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:33.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:33.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:33.346 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:33.347 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:33.399 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:33 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T09:00:33.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:33 smithi092 ceph-mon[29647]: 6.f scrub ok 2024-09-11T09:00:33.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:33 smithi092 ceph-mon[29647]: 1.1 scrub starts 2024-09-11T09:00:33.400 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:33 smithi092 ceph-mon[29647]: 1.1 scrub ok 2024-09-11T09:00:33.400 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:33 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T09:00:33.400 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:33 smithi092 ceph-mon[41114]: 6.f scrub ok 2024-09-11T09:00:33.400 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:33 smithi092 ceph-mon[41114]: 1.1 scrub starts 2024-09-11T09:00:33.400 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:33 smithi092 ceph-mon[41114]: 1.1 scrub ok 2024-09-11T09:00:33.448 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:33.475 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:33.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:33.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:33.577 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:33.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:33.615 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:33.615 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:33.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:33 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T09:00:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:33 smithi145 ceph-mon[33384]: 6.f scrub ok 2024-09-11T09:00:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:33 smithi145 ceph-mon[33384]: 1.1 scrub starts 2024-09-11T09:00:33.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:33 smithi145 ceph-mon[33384]: 1.1 scrub ok 2024-09-11T09:00:33.715 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:33.745 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:33.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:33.746 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:33.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:33.885 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:33.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:33.886 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:33.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:34.021 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:34.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:34.022 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:34.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:34.154 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:34.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:34.155 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:34.256 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:34.285 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:34.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:34.286 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:34.387 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:34.419 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:34.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:34.420 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:34.448 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Removed 121318 objects 2024-09-11T09:00:34.448 INFO:tasks.radosbench.radosbench.0.smithi092.stdout:Clean up completed and total clean up time :95.0901 2024-09-11T09:00:34.519 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:34 smithi092 ceph-mon[41114]: 1.2 scrub starts 2024-09-11T09:00:34.519 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:34 smithi092 ceph-mon[41114]: 1.2 scrub ok 2024-09-11T09:00:34.519 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:34 smithi092 ceph-mon[41114]: 1.11 scrub starts 2024-09-11T09:00:34.519 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:34 smithi092 ceph-mon[41114]: 1.11 scrub ok 2024-09-11T09:00:34.519 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:34 smithi092 ceph-mon[41114]: pgmap v998: 168 pgs: 168 active+clean; 220 MiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T09:00:34.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:34 smithi092 ceph-mon[29647]: 1.2 scrub starts 2024-09-11T09:00:34.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:34 smithi092 ceph-mon[29647]: 1.2 scrub ok 2024-09-11T09:00:34.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:34 smithi092 ceph-mon[29647]: 1.11 scrub starts 2024-09-11T09:00:34.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:34 smithi092 ceph-mon[29647]: 1.11 scrub ok 2024-09-11T09:00:34.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:34 smithi092 ceph-mon[29647]: pgmap v998: 168 pgs: 168 active+clean; 220 MiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T09:00:34.521 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:34.542 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:34.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:34.543 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:34.644 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:34.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:34 smithi145 ceph-mon[33384]: 1.2 scrub starts 2024-09-11T09:00:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:34 smithi145 ceph-mon[33384]: 1.2 scrub ok 2024-09-11T09:00:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:34 smithi145 ceph-mon[33384]: 1.11 scrub starts 2024-09-11T09:00:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:34 smithi145 ceph-mon[33384]: 1.11 scrub ok 2024-09-11T09:00:34.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:34 smithi145 ceph-mon[33384]: pgmap v998: 168 pgs: 168 active+clean; 220 MiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T09:00:34.664 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:34.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:34.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:34.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:34.788 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:34.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:34.789 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:34.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:34.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:34.913 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:34.913 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:35.014 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:35.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:35.037 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:35.037 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:35.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:35.163 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:35.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:35.164 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:35.265 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:35.290 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:35.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:35.291 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:35.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:35.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:35.417 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:35.418 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:35.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:35.543 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:35.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:35.545 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:35.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:35 smithi092 ceph-mon[29647]: 4.a scrub starts 2024-09-11T09:00:35.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:35 smithi092 ceph-mon[29647]: 4.a scrub ok 2024-09-11T09:00:35.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:35 smithi092 ceph-mon[29647]: 4.12 scrub starts 2024-09-11T09:00:35.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:35 smithi092 ceph-mon[29647]: 4.12 scrub ok 2024-09-11T09:00:35.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:35 smithi092 ceph-mon[41114]: 4.a scrub starts 2024-09-11T09:00:35.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:35 smithi092 ceph-mon[41114]: 4.a scrub ok 2024-09-11T09:00:35.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:35 smithi092 ceph-mon[41114]: 4.12 scrub starts 2024-09-11T09:00:35.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:35 smithi092 ceph-mon[41114]: 4.12 scrub ok 2024-09-11T09:00:35.646 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:35.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:35 smithi145 ceph-mon[33384]: 4.a scrub starts 2024-09-11T09:00:35.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:35 smithi145 ceph-mon[33384]: 4.a scrub ok 2024-09-11T09:00:35.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:35 smithi145 ceph-mon[33384]: 4.12 scrub starts 2024-09-11T09:00:35.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:35 smithi145 ceph-mon[33384]: 4.12 scrub ok 2024-09-11T09:00:35.668 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:35.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:35.669 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:35.770 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:35.796 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:35.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:35.797 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:35.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:35.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:35.924 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:35.925 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:36.026 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:36.049 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:36.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:36.050 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:36.151 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:36.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:36.176 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:36.176 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:36.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:36 smithi092 ceph-mon[29647]: 5.15 scrub starts 2024-09-11T09:00:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:36 smithi092 ceph-mon[29647]: 5.15 scrub ok 2024-09-11T09:00:36.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:36 smithi092 ceph-mon[29647]: pgmap v999: 168 pgs: 168 active+clean; 220 MiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T09:00:36.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:36 smithi092 ceph-mon[41114]: 5.15 scrub starts 2024-09-11T09:00:36.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:36 smithi092 ceph-mon[41114]: 5.15 scrub ok 2024-09-11T09:00:36.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:36 smithi092 ceph-mon[41114]: pgmap v999: 168 pgs: 168 active+clean; 220 MiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T09:00:36.299 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:36.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:36.300 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:36.401 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:36.405 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T09:00:36.405 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T09:00:36.405 INFO:teuthology.orchestra.run.smithi092.stdout: "epoch": 425, 2024-09-11T09:00:36.405 INFO:teuthology.orchestra.run.smithi092.stdout: "fsid": "211625d4-7018-11ef-bcea-c7b262605968", 2024-09-11T09:00:36.405 INFO:teuthology.orchestra.run.smithi092.stdout: "created": "2024-09-11T08:33:32.174444+0000", 2024-09-11T09:00:36.405 INFO:teuthology.orchestra.run.smithi092.stdout: "modified": "2024-09-11T09:00:30.165203+0000", 2024-09-11T09:00:36.405 INFO:teuthology.orchestra.run.smithi092.stdout: "last_up_change": "2024-09-11T08:38:23.097480+0000", 2024-09-11T09:00:36.405 INFO:teuthology.orchestra.run.smithi092.stdout: "last_in_change": "2024-09-11T08:58:33.533202+0000", 2024-09-11T09:00:36.406 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T09:00:36.406 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_num": 5804032, 2024-09-11T09:00:36.406 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_set": [ 2024-09-11T09:00:36.406 INFO:teuthology.orchestra.run.smithi092.stdout: "nodeep-scrub", 2024-09-11T09:00:36.406 INFO:teuthology.orchestra.run.smithi092.stdout: "pglog_hardlimit", 2024-09-11T09:00:36.406 INFO:teuthology.orchestra.run.smithi092.stdout: "purged_snapdirs", 2024-09-11T09:00:36.406 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_deletes", 2024-09-11T09:00:36.406 INFO:teuthology.orchestra.run.smithi092.stdout: "sortbitwise" 2024-09-11T09:00:36.406 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T09:00:36.406 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_version": 18, 2024-09-11T09:00:36.406 INFO:teuthology.orchestra.run.smithi092.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T09:00:36.406 INFO:teuthology.orchestra.run.smithi092.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T09:00:36.406 INFO:teuthology.orchestra.run.smithi092.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T09:00:36.407 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_snapshot": "", 2024-09-11T09:00:36.407 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_max": 11, 2024-09-11T09:00:36.407 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd": 8, 2024-09-11T09:00:36.407 INFO:teuthology.orchestra.run.smithi092.stdout: "require_min_compat_client": "luminous", 2024-09-11T09:00:36.407 INFO:teuthology.orchestra.run.smithi092.stdout: "min_compat_client": "luminous", 2024-09-11T09:00:36.407 INFO:teuthology.orchestra.run.smithi092.stdout: "require_osd_release": "quincy", 2024-09-11T09:00:36.407 INFO:teuthology.orchestra.run.smithi092.stdout: "pools": [ 2024-09-11T09:00:36.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.407 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 1, 2024-09-11T09:00:36.407 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": ".mgr", 2024-09-11T09:00:36.408 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:36:29.606460+0000", 2024-09-11T09:00:36.408 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T09:00:36.408 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T09:00:36.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T09:00:36.408 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T09:00:36.408 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T09:00:36.408 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T09:00:36.408 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:00:36.408 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:00:36.408 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:00:36.408 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:00:36.408 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T09:00:36.408 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:00:36.409 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 21, 2024-09-11T09:00:36.409 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 21, 2024-09-11T09:00:36.409 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 21, 2024-09-11T09:00:36.409 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 21, 2024-09-11T09:00:36.409 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 21, 2024-09-11T09:00:36.409 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T09:00:36.409 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T09:00:36.409 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T09:00:36.409 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T09:00:36.409 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T09:00:36.409 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T09:00:36.409 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T09:00:36.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.410 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "385", 2024-09-11T09:00:36.410 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T09:00:36.410 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:00:36.410 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "383", 2024-09-11T09:00:36.410 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T09:00:36.410 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:00:36.410 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T09:00:36.410 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T09:00:36.410 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T09:00:36.410 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T09:00:36.410 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T09:00:36.410 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T09:00:36.410 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T09:00:36.410 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T09:00:36.411 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T09:00:36.411 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T09:00:36.411 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T09:00:36.411 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T09:00:36.411 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T09:00:36.411 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:00:36.411 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:00:36.411 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:00:36.411 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T09:00:36.411 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T09:00:36.411 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T09:00:36.411 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T09:00:36.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T09:00:36.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.412 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T09:00:36.412 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T09:00:36.412 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T09:00:36.412 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:00:36.412 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:00:36.412 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:00:36.412 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:00:36.412 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T09:00:36.412 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T09:00:36.412 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T09:00:36.412 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T09:00:36.412 INFO:teuthology.orchestra.run.smithi092.stdout: "options": { 2024-09-11T09:00:36.413 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_max": 32, 2024-09-11T09:00:36.413 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_min": 1 2024-09-11T09:00:36.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.413 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T09:00:36.413 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": {} 2024-09-11T09:00:36.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.413 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 2, 2024-09-11T09:00:36.413 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "datapool", 2024-09-11T09:00:36.413 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:37.674272+0000", 2024-09-11T09:00:36.413 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 8193, 2024-09-11T09:00:36.413 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T09:00:36.414 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T09:00:36.414 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T09:00:36.414 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T09:00:36.414 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T09:00:36.414 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:00:36.414 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:00:36.414 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:00:36.414 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:00:36.414 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T09:00:36.414 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:00:36.414 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 3, 2024-09-11T09:00:36.414 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 3, 2024-09-11T09:00:36.414 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 3, 2024-09-11T09:00:36.414 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 3, 2024-09-11T09:00:36.415 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 3, 2024-09-11T09:00:36.415 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T09:00:36.415 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T09:00:36.415 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T09:00:36.415 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T09:00:36.415 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T09:00:36.415 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T09:00:36.415 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T09:00:36.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.415 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "60", 2024-09-11T09:00:36.415 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T09:00:36.415 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:00:36.415 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:00:36.415 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T09:00:36.416 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:00:36.416 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 3, 2024-09-11T09:00:36.416 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 60, 2024-09-11T09:00:36.416 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T09:00:36.416 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T09:00:36.416 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T09:00:36.416 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T09:00:36.416 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T09:00:36.416 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T09:00:36.416 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T09:00:36.416 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T09:00:36.416 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T09:00:36.416 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T09:00:36.416 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T09:00:36.417 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:00:36.417 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:00:36.417 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:00:36.417 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T09:00:36.417 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T09:00:36.417 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T09:00:36.417 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T09:00:36.417 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T09:00:36.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.417 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T09:00:36.417 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T09:00:36.417 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T09:00:36.417 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:00:36.417 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:00:36.418 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:00:36.418 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:00:36.418 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T09:00:36.418 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T09:00:36.418 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T09:00:36.418 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T09:00:36.418 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T09:00:36.418 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T09:00:36.418 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd": {} 2024-09-11T09:00:36.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.418 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 3, 2024-09-11T09:00:36.419 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": ".rgw.root", 2024-09-11T09:00:36.419 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:37.912906+0000", 2024-09-11T09:00:36.419 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T09:00:36.419 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T09:00:36.419 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T09:00:36.419 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T09:00:36.419 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T09:00:36.419 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T09:00:36.419 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:00:36.419 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:00:36.419 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:00:36.419 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:00:36.419 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T09:00:36.419 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:00:36.420 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T09:00:36.420 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T09:00:36.420 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T09:00:36.420 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T09:00:36.420 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T09:00:36.420 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T09:00:36.420 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T09:00:36.420 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T09:00:36.420 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T09:00:36.420 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T09:00:36.420 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T09:00:36.420 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T09:00:36.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.420 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "57", 2024-09-11T09:00:36.421 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T09:00:36.421 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:00:36.421 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:00:36.421 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T09:00:36.421 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:00:36.421 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T09:00:36.421 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T09:00:36.421 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T09:00:36.421 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T09:00:36.421 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T09:00:36.421 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T09:00:36.421 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T09:00:36.421 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T09:00:36.422 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T09:00:36.422 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T09:00:36.422 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T09:00:36.422 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T09:00:36.422 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T09:00:36.422 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:00:36.422 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:00:36.422 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:00:36.422 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T09:00:36.422 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T09:00:36.422 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T09:00:36.422 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T09:00:36.422 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T09:00:36.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.423 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T09:00:36.423 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T09:00:36.423 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T09:00:36.423 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:00:36.423 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:00:36.423 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:00:36.423 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:00:36.423 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T09:00:36.423 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T09:00:36.423 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T09:00:36.423 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T09:00:36.423 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T09:00:36.423 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T09:00:36.424 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T09:00:36.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.424 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 4, 2024-09-11T09:00:36.424 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "default.rgw.log", 2024-09-11T09:00:36.424 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:39.234173+0000", 2024-09-11T09:00:36.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T09:00:36.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T09:00:36.424 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T09:00:36.424 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T09:00:36.424 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T09:00:36.424 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T09:00:36.425 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:00:36.425 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:00:36.425 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:00:36.425 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:00:36.425 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T09:00:36.425 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:00:36.425 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T09:00:36.425 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T09:00:36.425 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T09:00:36.425 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T09:00:36.425 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T09:00:36.425 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T09:00:36.425 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T09:00:36.425 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T09:00:36.426 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T09:00:36.426 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T09:00:36.426 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T09:00:36.426 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T09:00:36.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.426 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "59", 2024-09-11T09:00:36.426 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T09:00:36.426 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:00:36.426 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:00:36.426 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T09:00:36.426 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:00:36.426 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T09:00:36.426 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T09:00:36.426 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T09:00:36.427 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T09:00:36.427 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T09:00:36.427 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T09:00:36.427 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T09:00:36.427 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T09:00:36.427 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T09:00:36.427 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T09:00:36.427 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T09:00:36.427 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T09:00:36.427 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T09:00:36.427 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:00:36.427 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:00:36.427 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:00:36.428 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T09:00:36.428 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T09:00:36.428 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T09:00:36.428 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T09:00:36.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T09:00:36.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.428 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T09:00:36.428 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T09:00:36.428 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T09:00:36.428 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:00:36.428 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:00:36.428 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:00:36.428 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:00:36.429 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T09:00:36.429 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T09:00:36.429 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T09:00:36.429 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T09:00:36.429 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T09:00:36.429 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T09:00:36.429 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T09:00:36.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.429 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 5, 2024-09-11T09:00:36.429 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "default.rgw.control", 2024-09-11T09:00:36.429 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:41.216386+0000", 2024-09-11T09:00:36.429 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T09:00:36.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T09:00:36.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T09:00:36.430 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T09:00:36.430 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T09:00:36.430 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T09:00:36.430 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:00:36.430 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:00:36.430 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:00:36.430 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:00:36.430 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T09:00:36.430 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:00:36.430 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T09:00:36.430 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T09:00:36.431 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T09:00:36.431 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T09:00:36.431 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T09:00:36.431 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T09:00:36.431 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T09:00:36.431 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T09:00:36.431 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T09:00:36.431 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T09:00:36.431 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T09:00:36.431 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T09:00:36.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.431 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "61", 2024-09-11T09:00:36.431 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T09:00:36.432 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:00:36.432 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:00:36.432 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T09:00:36.432 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:00:36.432 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T09:00:36.432 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T09:00:36.432 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T09:00:36.432 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T09:00:36.432 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T09:00:36.432 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T09:00:36.432 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T09:00:36.432 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T09:00:36.432 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T09:00:36.432 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T09:00:36.433 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T09:00:36.433 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T09:00:36.433 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T09:00:36.433 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:00:36.433 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:00:36.433 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:00:36.433 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T09:00:36.433 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T09:00:36.433 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T09:00:36.433 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T09:00:36.433 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T09:00:36.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.433 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T09:00:36.433 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T09:00:36.434 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T09:00:36.434 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:00:36.434 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:00:36.434 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:00:36.434 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:00:36.434 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T09:00:36.434 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T09:00:36.434 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T09:00:36.434 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T09:00:36.434 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T09:00:36.434 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T09:00:36.434 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T09:00:36.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.435 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 6, 2024-09-11T09:00:36.435 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "default.rgw.meta", 2024-09-11T09:00:36.435 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:38:43.285497+0000", 2024-09-11T09:00:36.435 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T09:00:36.435 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T09:00:36.435 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T09:00:36.435 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T09:00:36.435 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T09:00:36.435 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T09:00:36.435 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:00:36.435 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:00:36.435 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:00:36.436 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:00:36.436 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T09:00:36.436 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:00:36.436 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-11T09:00:36.436 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-11T09:00:36.436 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-11T09:00:36.436 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-11T09:00:36.436 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-11T09:00:36.436 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T09:00:36.436 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T09:00:36.436 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T09:00:36.436 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T09:00:36.436 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T09:00:36.436 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T09:00:36.437 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T09:00:36.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.437 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "63", 2024-09-11T09:00:36.437 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T09:00:36.437 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:00:36.437 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:00:36.437 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T09:00:36.437 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:00:36.437 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T09:00:36.437 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T09:00:36.437 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T09:00:36.437 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T09:00:36.437 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T09:00:36.437 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T09:00:36.438 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T09:00:36.438 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T09:00:36.438 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T09:00:36.438 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T09:00:36.438 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T09:00:36.438 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T09:00:36.438 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T09:00:36.438 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:00:36.438 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:00:36.438 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:00:36.438 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T09:00:36.438 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T09:00:36.438 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T09:00:36.439 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T09:00:36.439 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T09:00:36.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.439 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T09:00:36.439 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T09:00:36.439 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T09:00:36.439 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:00:36.439 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:00:36.439 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:00:36.439 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:00:36.439 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T09:00:36.439 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T09:00:36.439 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T09:00:36.439 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T09:00:36.440 INFO:teuthology.orchestra.run.smithi092.stdout: "options": { 2024-09-11T09:00:36.440 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_bias": 4 2024-09-11T09:00:36.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.440 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T09:00:36.440 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw": {} 2024-09-11T09:00:36.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.440 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 11, 2024-09-11T09:00:36.440 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "unique_pool_4", 2024-09-11T09:00:36.440 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-11T08:57:13.063288+0000", 2024-09-11T09:00:36.440 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-11T09:00:36.440 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-11T09:00:36.441 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-11T09:00:36.441 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 3, 2024-09-11T09:00:36.441 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 2, 2024-09-11T09:00:36.441 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-11T09:00:36.441 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:00:36.441 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:00:36.441 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:00:36.441 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:00:36.441 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-11T09:00:36.441 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:00:36.441 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 16, 2024-09-11T09:00:36.441 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 16, 2024-09-11T09:00:36.441 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 16, 2024-09-11T09:00:36.441 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 16, 2024-09-11T09:00:36.442 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 16, 2024-09-11T09:00:36.442 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-11T09:00:36.442 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-11T09:00:36.442 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-11T09:00:36.442 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-11T09:00:36.442 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-11T09:00:36.442 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-11T09:00:36.442 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-11T09:00:36.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.442 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "371", 2024-09-11T09:00:36.442 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-11T09:00:36.442 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:00:36.442 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:00:36.443 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-11T09:00:36.443 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:00:36.443 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-11T09:00:36.443 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-11T09:00:36.443 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-11T09:00:36.443 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-11T09:00:36.443 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-11T09:00:36.443 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-11T09:00:36.443 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-11T09:00:36.443 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-11T09:00:36.443 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-11T09:00:36.443 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-11T09:00:36.443 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-11T09:00:36.443 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-11T09:00:36.444 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-11T09:00:36.444 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:00:36.444 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:00:36.444 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:00:36.444 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-11T09:00:36.444 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-11T09:00:36.444 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-11T09:00:36.444 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-11T09:00:36.444 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-11T09:00:36.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.444 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-11T09:00:36.444 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-11T09:00:36.444 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-11T09:00:36.444 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:00:36.445 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:00:36.445 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:00:36.445 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:00:36.445 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-11T09:00:36.445 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-11T09:00:36.445 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-11T09:00:36.445 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-11T09:00:36.445 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-11T09:00:36.445 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-11T09:00:36.445 INFO:teuthology.orchestra.run.smithi092.stdout: "rados": {} 2024-09-11T09:00:36.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.445 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T09:00:36.446 INFO:teuthology.orchestra.run.smithi092.stdout: "osds": [ 2024-09-11T09:00:36.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.446 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-09-11T09:00:36.446 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "9d8a3811-e801-4b6a-9a63-83936283cb7d", 2024-09-11T09:00:36.446 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T09:00:36.446 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T09:00:36.446 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T09:00:36.446 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 0.1711883544921875, 2024-09-11T09:00:36.446 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T09:00:36.446 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T09:00:36.446 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 8, 2024-09-11T09:00:36.446 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 399, 2024-09-11T09:00:36.446 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T09:00:36.446 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T09:00:36.447 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T09:00:36.447 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.447 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.447 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6802", 2024-09-11T09:00:36.447 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T09:00:36.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.447 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.447 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6803", 2024-09-11T09:00:36.447 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T09:00:36.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.447 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.448 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T09:00:36.448 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.448 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.448 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6804", 2024-09-11T09:00:36.448 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T09:00:36.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.448 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.448 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6805", 2024-09-11T09:00:36.448 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T09:00:36.448 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.448 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.449 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T09:00:36.449 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.449 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.449 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6808", 2024-09-11T09:00:36.449 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T09:00:36.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.449 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.449 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6809", 2024-09-11T09:00:36.449 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T09:00:36.449 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.450 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.450 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T09:00:36.450 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.450 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.450 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6806", 2024-09-11T09:00:36.450 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T09:00:36.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.450 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.450 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6807", 2024-09-11T09:00:36.451 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2040613580 2024-09-11T09:00:36.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.451 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.451 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6803/2040613580", 2024-09-11T09:00:36.451 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6805/2040613580", 2024-09-11T09:00:36.451 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6809/2040613580", 2024-09-11T09:00:36.451 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6807/2040613580", 2024-09-11T09:00:36.451 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T09:00:36.451 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T09:00:36.451 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T09:00:36.451 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.451 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-09-11T09:00:36.452 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "e7a18e45-0137-45ef-8b85-e458cf3737ac", 2024-09-11T09:00:36.452 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T09:00:36.452 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T09:00:36.452 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 0.7004241943359375, 2024-09-11T09:00:36.452 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T09:00:36.452 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T09:00:36.452 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T09:00:36.452 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 13, 2024-09-11T09:00:36.452 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 399, 2024-09-11T09:00:36.452 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T09:00:36.452 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T09:00:36.452 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T09:00:36.452 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.453 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.453 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6810", 2024-09-11T09:00:36.453 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T09:00:36.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.453 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.453 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6811", 2024-09-11T09:00:36.453 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T09:00:36.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.453 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.453 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T09:00:36.453 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.454 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.454 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6812", 2024-09-11T09:00:36.454 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T09:00:36.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.454 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.454 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6813", 2024-09-11T09:00:36.454 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T09:00:36.454 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.454 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.454 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T09:00:36.455 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.455 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.455 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6816", 2024-09-11T09:00:36.455 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T09:00:36.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.455 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.455 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6817", 2024-09-11T09:00:36.455 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T09:00:36.455 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.455 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.455 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T09:00:36.456 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.456 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.456 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6814", 2024-09-11T09:00:36.456 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T09:00:36.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.456 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.456 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6815", 2024-09-11T09:00:36.456 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 544779328 2024-09-11T09:00:36.456 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.456 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.456 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6811/544779328", 2024-09-11T09:00:36.457 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6813/544779328", 2024-09-11T09:00:36.457 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6817/544779328", 2024-09-11T09:00:36.457 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6815/544779328", 2024-09-11T09:00:36.457 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T09:00:36.457 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T09:00:36.457 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T09:00:36.457 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.457 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-09-11T09:00:36.457 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "017282e2-8053-4fa7-affc-a05408fa481e", 2024-09-11T09:00:36.457 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T09:00:36.457 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T09:00:36.458 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T09:00:36.458 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 0.5283050537109375, 2024-09-11T09:00:36.458 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T09:00:36.458 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T09:00:36.458 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 18, 2024-09-11T09:00:36.458 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 403, 2024-09-11T09:00:36.458 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T09:00:36.458 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T09:00:36.458 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T09:00:36.458 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.458 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6818", 2024-09-11T09:00:36.459 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T09:00:36.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.459 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.459 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6819", 2024-09-11T09:00:36.459 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T09:00:36.459 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.459 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.459 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T09:00:36.459 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.459 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.460 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6820", 2024-09-11T09:00:36.460 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T09:00:36.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.460 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.460 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6821", 2024-09-11T09:00:36.460 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T09:00:36.460 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.460 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.460 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T09:00:36.460 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.461 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.461 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6824", 2024-09-11T09:00:36.461 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T09:00:36.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.461 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.461 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6825", 2024-09-11T09:00:36.461 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T09:00:36.461 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.461 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.461 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T09:00:36.461 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.462 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.462 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6822", 2024-09-11T09:00:36.462 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T09:00:36.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.462 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.462 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6823", 2024-09-11T09:00:36.462 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 933736777 2024-09-11T09:00:36.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.462 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.462 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6819/933736777", 2024-09-11T09:00:36.462 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6821/933736777", 2024-09-11T09:00:36.463 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6825/933736777", 2024-09-11T09:00:36.463 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6823/933736777", 2024-09-11T09:00:36.463 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T09:00:36.463 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T09:00:36.463 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T09:00:36.463 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.463 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-09-11T09:00:36.463 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "58e5a8a4-9e58-40ad-9b40-90cb5ff5414b", 2024-09-11T09:00:36.463 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T09:00:36.463 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T09:00:36.463 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T09:00:36.463 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T09:00:36.464 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T09:00:36.464 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T09:00:36.464 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 26, 2024-09-11T09:00:36.464 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 400, 2024-09-11T09:00:36.464 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T09:00:36.464 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T09:00:36.464 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T09:00:36.464 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.464 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.464 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6826", 2024-09-11T09:00:36.464 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T09:00:36.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.465 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.465 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6827", 2024-09-11T09:00:36.465 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T09:00:36.465 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.465 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.465 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T09:00:36.465 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.465 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.465 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6828", 2024-09-11T09:00:36.465 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T09:00:36.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.466 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.466 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6829", 2024-09-11T09:00:36.466 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T09:00:36.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.466 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.466 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T09:00:36.466 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.466 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.466 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6832", 2024-09-11T09:00:36.466 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T09:00:36.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.467 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.467 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6833", 2024-09-11T09:00:36.467 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T09:00:36.467 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.467 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.467 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T09:00:36.467 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.467 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.467 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6830", 2024-09-11T09:00:36.467 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T09:00:36.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.468 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.468 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6831", 2024-09-11T09:00:36.468 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 878892459 2024-09-11T09:00:36.468 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.468 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.468 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6827/878892459", 2024-09-11T09:00:36.468 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6829/878892459", 2024-09-11T09:00:36.468 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6833/878892459", 2024-09-11T09:00:36.468 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6831/878892459", 2024-09-11T09:00:36.468 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T09:00:36.468 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T09:00:36.468 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T09:00:36.469 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.469 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-09-11T09:00:36.469 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "e144e102-963b-48b7-92a9-9391706164bf", 2024-09-11T09:00:36.469 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T09:00:36.469 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T09:00:36.469 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T09:00:36.469 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T09:00:36.469 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T09:00:36.469 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T09:00:36.469 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 31, 2024-09-11T09:00:36.469 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 412, 2024-09-11T09:00:36.469 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T09:00:36.470 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T09:00:36.470 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T09:00:36.470 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.470 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.470 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6800", 2024-09-11T09:00:36.470 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T09:00:36.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.470 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.470 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6801", 2024-09-11T09:00:36.470 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T09:00:36.470 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.470 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.471 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T09:00:36.471 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.471 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.471 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6802", 2024-09-11T09:00:36.471 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T09:00:36.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.471 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.471 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6803", 2024-09-11T09:00:36.471 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T09:00:36.471 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T09:00:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6806", 2024-09-11T09:00:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T09:00:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6807", 2024-09-11T09:00:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T09:00:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T09:00:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6804", 2024-09-11T09:00:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T09:00:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6805", 2024-09-11T09:00:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2792751669 2024-09-11T09:00:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.474 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.474 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6801/2792751669", 2024-09-11T09:00:36.474 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6803/2792751669", 2024-09-11T09:00:36.474 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6807/2792751669", 2024-09-11T09:00:36.474 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6805/2792751669", 2024-09-11T09:00:36.474 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T09:00:36.474 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T09:00:36.474 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T09:00:36.474 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.474 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-09-11T09:00:36.474 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "811b0dc9-b7a7-439a-86c1-5dc9d0d54e16", 2024-09-11T09:00:36.475 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T09:00:36.475 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T09:00:36.475 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T09:00:36.475 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-11T09:00:36.475 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T09:00:36.475 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T09:00:36.475 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 38, 2024-09-11T09:00:36.475 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 412, 2024-09-11T09:00:36.475 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T09:00:36.475 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T09:00:36.475 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T09:00:36.475 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.475 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.476 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6808", 2024-09-11T09:00:36.476 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T09:00:36.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.476 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.476 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6809", 2024-09-11T09:00:36.476 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T09:00:36.476 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.476 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.476 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T09:00:36.476 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.477 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.477 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6810", 2024-09-11T09:00:36.477 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T09:00:36.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.477 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.477 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6811", 2024-09-11T09:00:36.477 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T09:00:36.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.477 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.477 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T09:00:36.478 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.478 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.478 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6814", 2024-09-11T09:00:36.478 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T09:00:36.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.478 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.478 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6815", 2024-09-11T09:00:36.478 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T09:00:36.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.478 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.478 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T09:00:36.479 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.479 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.479 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6812", 2024-09-11T09:00:36.479 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T09:00:36.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.479 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.479 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6813", 2024-09-11T09:00:36.479 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3569637188 2024-09-11T09:00:36.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.479 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.479 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6809/3569637188", 2024-09-11T09:00:36.480 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6811/3569637188", 2024-09-11T09:00:36.480 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6815/3569637188", 2024-09-11T09:00:36.480 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6813/3569637188", 2024-09-11T09:00:36.480 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T09:00:36.480 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T09:00:36.480 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T09:00:36.480 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.480 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-09-11T09:00:36.480 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "bd162f52-097c-4c5d-9d37-1289779a196b", 2024-09-11T09:00:36.480 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T09:00:36.480 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T09:00:36.481 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T09:00:36.481 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 0, 2024-09-11T09:00:36.481 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T09:00:36.481 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T09:00:36.481 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 45, 2024-09-11T09:00:36.481 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 399, 2024-09-11T09:00:36.481 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T09:00:36.481 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T09:00:36.481 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T09:00:36.481 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.481 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.481 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6816", 2024-09-11T09:00:36.481 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T09:00:36.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.482 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.482 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6817", 2024-09-11T09:00:36.482 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T09:00:36.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.482 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.482 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T09:00:36.482 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.482 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.482 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6818", 2024-09-11T09:00:36.483 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T09:00:36.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.483 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.483 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6819", 2024-09-11T09:00:36.483 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T09:00:36.483 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.483 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.483 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T09:00:36.483 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.483 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.483 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6822", 2024-09-11T09:00:36.484 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T09:00:36.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.484 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.484 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6823", 2024-09-11T09:00:36.484 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T09:00:36.484 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.484 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.484 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T09:00:36.484 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.485 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.485 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6820", 2024-09-11T09:00:36.485 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T09:00:36.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.485 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.485 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6821", 2024-09-11T09:00:36.485 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2615047110 2024-09-11T09:00:36.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.486 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.486 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6817/2615047110", 2024-09-11T09:00:36.486 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6819/2615047110", 2024-09-11T09:00:36.486 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6823/2615047110", 2024-09-11T09:00:36.486 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6821/2615047110", 2024-09-11T09:00:36.486 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T09:00:36.486 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T09:00:36.486 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T09:00:36.486 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.486 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-09-11T09:00:36.487 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "4e4021bf-78c7-4bc6-b222-3494f724ddec", 2024-09-11T09:00:36.487 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-11T09:00:36.487 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-11T09:00:36.487 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-11T09:00:36.487 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 0.2356109619140625, 2024-09-11T09:00:36.487 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-11T09:00:36.487 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-11T09:00:36.487 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 52, 2024-09-11T09:00:36.487 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 400, 2024-09-11T09:00:36.487 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-11T09:00:36.487 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-11T09:00:36.487 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-11T09:00:36.487 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.488 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.488 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6824", 2024-09-11T09:00:36.488 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T09:00:36.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.488 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.488 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6825", 2024-09-11T09:00:36.488 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T09:00:36.488 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.488 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.488 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-11T09:00:36.488 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.489 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.489 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6826", 2024-09-11T09:00:36.489 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T09:00:36.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.489 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.489 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6827", 2024-09-11T09:00:36.489 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T09:00:36.489 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.489 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.489 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-11T09:00:36.489 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.490 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.490 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6830", 2024-09-11T09:00:36.490 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T09:00:36.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.490 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.490 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6831", 2024-09-11T09:00:36.490 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T09:00:36.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.490 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.490 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-11T09:00:36.491 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-11T09:00:36.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.491 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-11T09:00:36.491 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6828", 2024-09-11T09:00:36.491 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T09:00:36.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.491 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-11T09:00:36.491 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.145:6829", 2024-09-11T09:00:36.491 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3195898530 2024-09-11T09:00:36.491 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.491 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.491 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.145:6825/3195898530", 2024-09-11T09:00:36.492 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.145:6827/3195898530", 2024-09-11T09:00:36.492 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.145:6831/3195898530", 2024-09-11T09:00:36.492 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.145:6829/3195898530", 2024-09-11T09:00:36.492 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-11T09:00:36.492 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-11T09:00:36.492 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-11T09:00:36.492 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.492 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.492 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T09:00:36.492 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_xinfo": [ 2024-09-11T09:00:36.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.492 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-09-11T09:00:36.492 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T09:00:36.492 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T09:00:36.493 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T09:00:36.493 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T09:00:36.493 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T09:00:36.493 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T09:00:15.178481+0000", 2024-09-11T09:00:36.493 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T09:00:36.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.493 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-09-11T09:00:36.493 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T09:00:36.493 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T09:00:36.493 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T09:00:36.493 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T09:00:36.493 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T09:00:36.494 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:59:57.379530+0000", 2024-09-11T09:00:36.494 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T09:00:36.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.494 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-09-11T09:00:36.494 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T09:00:36.494 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T09:00:36.494 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T09:00:36.494 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T09:00:36.494 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T09:00:36.494 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T09:00:07.248516+0000", 2024-09-11T09:00:36.494 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T09:00:36.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.494 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-09-11T09:00:36.495 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T09:00:36.495 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T09:00:36.495 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T09:00:36.495 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T09:00:36.495 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T09:00:36.495 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:59:36.796306+0000", 2024-09-11T09:00:36.495 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T09:00:36.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.495 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-09-11T09:00:36.495 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T09:00:36.495 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T09:00:36.495 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T09:00:36.496 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T09:00:36.496 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T09:00:36.496 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T09:00:30.140729+0000", 2024-09-11T09:00:36.496 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T09:00:36.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.496 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-09-11T09:00:36.496 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T09:00:36.496 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T09:00:36.496 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T09:00:36.496 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T09:00:36.496 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T09:00:36.496 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:59:52.095440+0000", 2024-09-11T09:00:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T09:00:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-09-11T09:00:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T09:00:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T09:00:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T09:00:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T09:00:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T09:00:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:59:49.035523+0000", 2024-09-11T09:00:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T09:00:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.497 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-09-11T09:00:36.498 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-11T09:00:36.498 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-11T09:00:36.498 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-11T09:00:36.498 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540138320759226367, 2024-09-11T09:00:36.498 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-11T09:00:36.498 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-11T08:59:20.774663+0000", 2024-09-11T09:00:36.498 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-11T09:00:36.498 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.498 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T09:00:36.498 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap": [ 2024-09-11T09:00:36.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.498 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "3.1", 2024-09-11T09:00:36.498 INFO:teuthology.orchestra.run.smithi092.stdout: "osds": [ 2024-09-11T09:00:36.498 INFO:teuthology.orchestra.run.smithi092.stdout: 5, 2024-09-11T09:00:36.498 INFO:teuthology.orchestra.run.smithi092.stdout: 4, 2024-09-11T09:00:36.499 INFO:teuthology.orchestra.run.smithi092.stdout: 2 2024-09-11T09:00:36.499 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.499 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.499 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T09:00:36.499 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap_items": [ 2024-09-11T09:00:36.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.499 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "3.5", 2024-09-11T09:00:36.499 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T09:00:36.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.499 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 5, 2024-09-11T09:00:36.499 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T09:00:36.499 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.499 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.500 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.10", 2024-09-11T09:00:36.500 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T09:00:36.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.500 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 3, 2024-09-11T09:00:36.500 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 5 2024-09-11T09:00:36.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.500 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.500 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.13", 2024-09-11T09:00:36.500 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T09:00:36.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.501 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T09:00:36.501 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 5 2024-09-11T09:00:36.501 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.501 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.501 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.15", 2024-09-11T09:00:36.501 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T09:00:36.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.501 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 7, 2024-09-11T09:00:36.501 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 1 2024-09-11T09:00:36.501 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.501 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.502 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.17", 2024-09-11T09:00:36.502 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T09:00:36.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.502 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T09:00:36.502 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T09:00:36.502 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.502 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.502 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "4.1d", 2024-09-11T09:00:36.502 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T09:00:36.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.502 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T09:00:36.502 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T09:00:36.503 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.503 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.503 INFO:teuthology.orchestra.run.smithi092.stdout: "pgid": "5.18", 2024-09-11T09:00:36.503 INFO:teuthology.orchestra.run.smithi092.stdout: "mappings": [ 2024-09-11T09:00:36.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:00:36.503 INFO:teuthology.orchestra.run.smithi092.stdout: "from": 1, 2024-09-11T09:00:36.503 INFO:teuthology.orchestra.run.smithi092.stdout: "to": 2 2024-09-11T09:00:36.503 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.503 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:00:36.503 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.503 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-11T09:00:36.504 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_temp": [], 2024-09-11T09:00:36.504 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_temp": [], 2024-09-11T09:00:36.504 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklist": { 2024-09-11T09:00:36.504 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6800/1861854063": "2024-09-12T08:39:14.984442+0000", 2024-09-11T09:00:36.504 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/4113910808": "2024-09-12T08:33:54.195816+0000", 2024-09-11T09:00:36.504 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/3501902574": "2024-09-12T08:39:14.984442+0000", 2024-09-11T09:00:36.504 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/1583695466": "2024-09-12T08:33:54.195816+0000", 2024-09-11T09:00:36.504 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/736380628": "2024-09-12T08:33:54.195816+0000", 2024-09-11T09:00:36.504 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6800/4060453592": "2024-09-12T08:33:54.195816+0000", 2024-09-11T09:00:36.504 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/2234574262": "2024-09-12T08:39:14.984442+0000", 2024-09-11T09:00:36.504 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6800/2480574848": "2024-09-12T08:34:17.306389+0000", 2024-09-11T09:00:36.504 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/1540079149": "2024-09-12T08:39:14.984442+0000", 2024-09-11T09:00:36.504 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/2680744738": "2024-09-12T08:34:17.306389+0000", 2024-09-11T09:00:36.504 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6801/4060453592": "2024-09-12T08:33:54.195816+0000", 2024-09-11T09:00:36.504 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6801/1861854063": "2024-09-12T08:39:14.984442+0000", 2024-09-11T09:00:36.505 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6801/2480574848": "2024-09-12T08:34:17.306389+0000", 2024-09-11T09:00:36.505 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/12697418": "2024-09-12T08:34:17.306389+0000", 2024-09-11T09:00:36.505 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/4280958069": "2024-09-12T08:34:17.306389+0000", 2024-09-11T09:00:36.505 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/2686720854": "2024-09-12T08:39:14.984442+0000", 2024-09-11T09:00:36.505 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:0/969221731": "2024-09-12T08:39:14.984442+0000" 2024-09-11T09:00:36.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.505 INFO:teuthology.orchestra.run.smithi092.stdout: "range_blocklist": {}, 2024-09-11T09:00:36.505 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profiles": { 2024-09-11T09:00:36.505 INFO:teuthology.orchestra.run.smithi092.stdout: "default": { 2024-09-11T09:00:36.505 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-failure-domain": "osd", 2024-09-11T09:00:36.505 INFO:teuthology.orchestra.run.smithi092.stdout: "k": "2", 2024-09-11T09:00:36.505 INFO:teuthology.orchestra.run.smithi092.stdout: "m": "1", 2024-09-11T09:00:36.505 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin": "jerasure", 2024-09-11T09:00:36.506 INFO:teuthology.orchestra.run.smithi092.stdout: "technique": "reed_sol_van" 2024-09-11T09:00:36.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:00:36.506 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps_queue": [], 2024-09-11T09:00:36.506 INFO:teuthology.orchestra.run.smithi092.stdout: "new_removed_snaps": [], 2024-09-11T09:00:36.506 INFO:teuthology.orchestra.run.smithi092.stdout: "new_purged_snaps": [], 2024-09-11T09:00:36.506 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_node_flags": {}, 2024-09-11T09:00:36.506 INFO:teuthology.orchestra.run.smithi092.stdout: "device_class_flags": {}, 2024-09-11T09:00:36.506 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode": { 2024-09-11T09:00:36.506 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_enabled": false, 2024-09-11T09:00:36.506 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_bucket_count": 0, 2024-09-11T09:00:36.506 INFO:teuthology.orchestra.run.smithi092.stdout: "degraded_stretch_mode": 0, 2024-09-11T09:00:36.506 INFO:teuthology.orchestra.run.smithi092.stdout: "recovering_stretch_mode": 0, 2024-09-11T09:00:36.506 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_bucket": 0 2024-09-11T09:00:36.507 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:00:36.507 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T09:00:36.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:36.508 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:36.508 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:36.609 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:36.634 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:36.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:36.635 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:36.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:36 smithi145 ceph-mon[33384]: 5.15 scrub starts 2024-09-11T09:00:36.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:36 smithi145 ceph-mon[33384]: 5.15 scrub ok 2024-09-11T09:00:36.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:36 smithi145 ceph-mon[33384]: pgmap v999: 168 pgs: 168 active+clean; 220 MiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T09:00:36.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:36.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:36.759 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:36.759 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:36.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:36.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:36.882 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:36.882 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:36.983 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:37.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:37.010 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:37.010 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:37.111 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:37.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:37.141 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:37.141 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:37.242 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:37.265 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:37.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:37.267 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:37.367 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:37.393 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:37.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:37.394 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:37.427 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:37 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2292481115' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T09:00:37.427 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:37 smithi092 ceph-mon[29647]: 3.6 scrub starts 2024-09-11T09:00:37.427 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:37 smithi092 ceph-mon[29647]: 3.6 scrub ok 2024-09-11T09:00:37.428 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:37 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2292481115' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T09:00:37.428 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:37 smithi092 ceph-mon[41114]: 3.6 scrub starts 2024-09-11T09:00:37.428 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:37 smithi092 ceph-mon[41114]: 3.6 scrub ok 2024-09-11T09:00:37.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:37.519 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:37.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:37.520 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:37.621 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:37.647 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:37.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:37.648 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:37 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2292481115' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T09:00:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:37 smithi145 ceph-mon[33384]: 3.6 scrub starts 2024-09-11T09:00:37.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:37 smithi145 ceph-mon[33384]: 3.6 scrub ok 2024-09-11T09:00:37.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:00:37 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:00:37.748 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:37.776 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:37.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:37.777 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:37.878 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:37.902 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:37.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:37.903 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:38.004 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:38.028 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:38.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:38.029 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:38.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:38.153 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:38.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:38.154 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:38.254 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:38.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:38.281 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:38.281 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:38.382 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:38.408 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:38.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:38.409 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:38.428 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_4 2024-09-11T09:00:38.428 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool rm unique_pool_4 unique_pool_4 --yes-i-really-really-mean-it 2024-09-11T09:00:38.451 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[29647]: 1.d scrub starts 2024-09-11T09:00:38.451 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[29647]: 1.d scrub ok 2024-09-11T09:00:38.451 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[29647]: 5.6 scrub starts 2024-09-11T09:00:38.451 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[29647]: 5.6 scrub ok 2024-09-11T09:00:38.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[29647]: 4.2 scrub starts 2024-09-11T09:00:38.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[29647]: 4.2 scrub ok 2024-09-11T09:00:38.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:38.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[29647]: pgmap v1000: 168 pgs: 168 active+clean; 528 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1k op/s 2024-09-11T09:00:38.452 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[41114]: 1.d scrub starts 2024-09-11T09:00:38.452 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[41114]: 1.d scrub ok 2024-09-11T09:00:38.452 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[41114]: 5.6 scrub starts 2024-09-11T09:00:38.452 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[41114]: 5.6 scrub ok 2024-09-11T09:00:38.452 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[41114]: 4.2 scrub starts 2024-09-11T09:00:38.453 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[41114]: 4.2 scrub ok 2024-09-11T09:00:38.453 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:38.453 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:38 smithi092 ceph-mon[41114]: pgmap v1000: 168 pgs: 168 active+clean; 528 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1k op/s 2024-09-11T09:00:38.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:38.532 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:38.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:38.533 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:38.621 INFO:tasks.thrashosds.thrasher:j is {'epoch': 425, 'fsid': '211625d4-7018-11ef-bcea-c7b262605968', 'created': '2024-09-11T08:33:32.174444+0000', 'modified': '2024-09-11T09:00:30.165203+0000', 'last_up_change': '2024-09-11T08:38:23.097480+0000', 'last_in_change': '2024-09-11T08:58:33.533202+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 11, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T08:36:29.606460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '385', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '383', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-11T08:38:37.674272+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-11T08:38:37.912906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-11T08:38:39.234173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-11T08:38:41.216386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-11T08:38:43.285497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 11, 'pool_name': 'unique_pool_4', 'create_time': '2024-09-11T08:57:13.063288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '371', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '9d8a3811-e801-4b6a-9a63-83936283cb7d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1711883544921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 399, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 2040613580}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 2040613580}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 2040613580}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 2040613580}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 2040613580}]}, 'public_addr': '172.21.15.92:6803/2040613580', 'cluster_addr': '172.21.15.92:6805/2040613580', 'heartbeat_back_addr': '172.21.15.92:6809/2040613580', 'heartbeat_front_addr': '172.21.15.92:6807/2040613580', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e7a18e45-0137-45ef-8b85-e458cf3737ac', 'up': 1, 'in': 1, 'weight': 0.7004241943359375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 399, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 544779328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 544779328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 544779328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 544779328}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 544779328}]}, 'public_addr': '172.21.15.92:6811/544779328', 'cluster_addr': '172.21.15.92:6813/544779328', 'heartbeat_back_addr': '172.21.15.92:6817/544779328', 'heartbeat_front_addr': '172.21.15.92:6815/544779328', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '017282e2-8053-4fa7-affc-a05408fa481e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5283050537109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 403, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 933736777}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 933736777}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 933736777}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 933736777}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 933736777}]}, 'public_addr': '172.21.15.92:6819/933736777', 'cluster_addr': '172.21.15.92:6821/933736777', 'heartbeat_back_addr': '172.21.15.92:6825/933736777', 'heartbeat_front_addr': '172.21.15.92:6823/933736777', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '58e5a8a4-9e58-40ad-9b40-90cb5ff5414b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 400, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 878892459}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 878892459}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 878892459}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 878892459}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 878892459}]}, 'public_addr': '172.21.15.92:6827/878892459', 'cluster_addr': '172.21.15.92:6829/878892459', 'heartbeat_back_addr': '172.21.15.92:6833/878892459', 'heartbeat_front_addr': '172.21.15.92:6831/878892459', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e144e102-963b-48b7-92a9-9391706164bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 412, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6800', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6801', 'nonce': 2792751669}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6802', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6803', 'nonce': 2792751669}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6806', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6807', 'nonce': 2792751669}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6804', 'nonce': 2792751669}, {'type': 'v1', 'addr': '172.21.15.145:6805', 'nonce': 2792751669}]}, 'public_addr': '172.21.15.145:6801/2792751669', 'cluster_addr': '172.21.15.145:6803/2792751669', 'heartbeat_back_addr': '172.21.15.145:6807/2792751669', 'heartbeat_front_addr': '172.21.15.145:6805/2792751669', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '811b0dc9-b7a7-439a-86c1-5dc9d0d54e16', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 38, 'up_thru': 412, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6808', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6809', 'nonce': 3569637188}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6810', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6811', 'nonce': 3569637188}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6814', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6815', 'nonce': 3569637188}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6812', 'nonce': 3569637188}, {'type': 'v1', 'addr': '172.21.15.145:6813', 'nonce': 3569637188}]}, 'public_addr': '172.21.15.145:6809/3569637188', 'cluster_addr': '172.21.15.145:6811/3569637188', 'heartbeat_back_addr': '172.21.15.145:6815/3569637188', 'heartbeat_front_addr': '172.21.15.145:6813/3569637188', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'bd162f52-097c-4c5d-9d37-1289779a196b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 399, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6816', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6817', 'nonce': 2615047110}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6818', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6819', 'nonce': 2615047110}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6822', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6823', 'nonce': 2615047110}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6820', 'nonce': 2615047110}, {'type': 'v1', 'addr': '172.21.15.145:6821', 'nonce': 2615047110}]}, 'public_addr': '172.21.15.145:6817/2615047110', 'cluster_addr': '172.21.15.145:6819/2615047110', 'heartbeat_back_addr': '172.21.15.145:6823/2615047110', 'heartbeat_front_addr': '172.21.15.145:6821/2615047110', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '4e4021bf-78c7-4bc6-b222-3494f724ddec', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.2356109619140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 52, 'up_thru': 400, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6824', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6825', 'nonce': 3195898530}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6826', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6827', 'nonce': 3195898530}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6830', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6831', 'nonce': 3195898530}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6828', 'nonce': 3195898530}, {'type': 'v1', 'addr': '172.21.15.145:6829', 'nonce': 3195898530}]}, 'public_addr': '172.21.15.145:6825/3195898530', 'cluster_addr': '172.21.15.145:6827/3195898530', 'heartbeat_back_addr': '172.21.15.145:6831/3195898530', 'heartbeat_front_addr': '172.21.15.145:6829/3195898530', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:00:15.178481+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:59:57.379530+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:00:07.248516+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:59:36.796306+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:00:30.140729+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:59:52.095440+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:59:49.035523+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T08:59:20.774663+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1', 'osds': [5, 4, 2]}], 'pg_upmap_items': [{'pgid': '3.5', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '4.10', 'mappings': [{'from': 3, 'to': 5}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.15', 'mappings': [{'from': 7, 'to': 1}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.1d', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '5.18', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.92:6800/1861854063': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/4113910808': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:0/3501902574': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/1583695466': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:0/736380628': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:6800/4060453592': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:0/2234574262': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:6800/2480574848': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/1540079149': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/2680744738': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:6801/4060453592': '2024-09-12T08:33:54.195816+0000', '172.21.15.92:6801/1861854063': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:6801/2480574848': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/12697418': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/4280958069': '2024-09-12T08:34:17.306389+0000', '172.21.15.92:0/2686720854': '2024-09-12T08:39:14.984442+0000', '172.21.15.92:0/969221731': '2024-09-12T08:39:14.984442+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T09:00:38.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T09:00:38.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:38.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:38 smithi145 ceph-mon[33384]: 1.d scrub starts 2024-09-11T09:00:38.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:38 smithi145 ceph-mon[33384]: 1.d scrub ok 2024-09-11T09:00:38.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:38 smithi145 ceph-mon[33384]: 5.6 scrub starts 2024-09-11T09:00:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:38 smithi145 ceph-mon[33384]: 5.6 scrub ok 2024-09-11T09:00:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:38 smithi145 ceph-mon[33384]: 4.2 scrub starts 2024-09-11T09:00:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:38 smithi145 ceph-mon[33384]: 4.2 scrub ok 2024-09-11T09:00:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:38 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:38.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:38 smithi145 ceph-mon[33384]: pgmap v1000: 168 pgs: 168 active+clean; 528 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1k op/s 2024-09-11T09:00:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:38.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:38.688 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:38.790 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:38.815 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:38.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:38.817 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:38.918 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:38.943 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:38.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:38.944 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:39.044 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:39.066 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:39.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:39.068 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:39.169 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:39.193 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:39.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:39.195 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:39.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:39.316 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:39.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:39.317 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:39.418 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:39.440 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:39.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:39.441 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:39.541 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:39 smithi092 ceph-mon[29647]: 6.18 scrub starts 2024-09-11T09:00:39.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:39 smithi092 ceph-mon[29647]: 6.18 scrub ok 2024-09-11T09:00:39.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:39 smithi092 ceph-mon[41114]: 6.18 scrub starts 2024-09-11T09:00:39.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:39 smithi092 ceph-mon[41114]: 6.18 scrub ok 2024-09-11T09:00:39.568 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:39.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:39.569 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:39.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:39 smithi145 ceph-mon[33384]: 6.18 scrub starts 2024-09-11T09:00:39.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:39 smithi145 ceph-mon[33384]: 6.18 scrub ok 2024-09-11T09:00:39.669 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:39.695 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:39.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:39.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:39.797 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:39.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:39.824 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:39.824 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:39.925 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:39.950 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:39.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:39.951 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:39.986 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T09:00:39.986 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T09:00:39.986 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T09:00:39.986 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T09:00:39.986 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T09:00:40.052 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:40.073 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:40.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:40.074 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:40.175 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:40.201 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:40.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:40.202 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:40.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:40.324 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:40.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:40.325 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:40.426 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:40.452 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:40.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:40.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:40.554 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:40.579 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:40.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:40.580 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:40.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:40 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T09:00:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:40 smithi145 ceph-mon[33384]: 5.d scrub ok 2024-09-11T09:00:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:40 smithi145 ceph-mon[33384]: pgmap v1001: 168 pgs: 168 active+clean; 528 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1k op/s 2024-09-11T09:00:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:40 smithi145 ceph-mon[33384]: 3.0 scrub starts 2024-09-11T09:00:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:40 smithi145 ceph-mon[33384]: 3.0 scrub ok 2024-09-11T09:00:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:40 smithi145 ceph-mon[33384]: 6.16 scrub starts 2024-09-11T09:00:40.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:40 smithi145 ceph-mon[33384]: 6.16 scrub ok 2024-09-11T09:00:40.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:40.706 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:40.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:40.707 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:40.808 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:40.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:40 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T09:00:40.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:40 smithi092 ceph-mon[29647]: 5.d scrub ok 2024-09-11T09:00:40.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:40 smithi092 ceph-mon[29647]: pgmap v1001: 168 pgs: 168 active+clean; 528 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1k op/s 2024-09-11T09:00:40.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:40 smithi092 ceph-mon[29647]: 3.0 scrub starts 2024-09-11T09:00:40.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:40 smithi092 ceph-mon[29647]: 3.0 scrub ok 2024-09-11T09:00:40.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:40 smithi092 ceph-mon[29647]: 6.16 scrub starts 2024-09-11T09:00:40.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:40 smithi092 ceph-mon[29647]: 6.16 scrub ok 2024-09-11T09:00:40.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:40 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T09:00:40.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:40 smithi092 ceph-mon[41114]: 5.d scrub ok 2024-09-11T09:00:40.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:40 smithi092 ceph-mon[41114]: pgmap v1001: 168 pgs: 168 active+clean; 528 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1k op/s 2024-09-11T09:00:40.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:40 smithi092 ceph-mon[41114]: 3.0 scrub starts 2024-09-11T09:00:40.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:40 smithi092 ceph-mon[41114]: 3.0 scrub ok 2024-09-11T09:00:40.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:40 smithi092 ceph-mon[41114]: 6.16 scrub starts 2024-09-11T09:00:40.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:40 smithi092 ceph-mon[41114]: 6.16 scrub ok 2024-09-11T09:00:40.833 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:40.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:40.835 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:40.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:40.962 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:40.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:40.963 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:41.064 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:41.090 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:41.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:41.091 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:41.192 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:41.217 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:41.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:41.218 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:41.319 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:41.342 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:41.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:41.343 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:41.444 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:41.468 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:41.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:41.469 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:41.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:00:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:00:41] "GET /metrics HTTP/1.1" 200 225076 "" "Prometheus/2.43.0" 2024-09-11T09:00:41.570 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:41.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:41.593 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:41.594 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:41.697 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:41.723 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:41.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:41.724 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:41.825 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:41.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:41.851 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:41.851 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:41.952 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:41.976 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:41.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:41.977 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:42.078 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:42.101 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:42.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:42.102 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:42.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:42.231 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:42.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:42.232 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:42.333 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:42.349 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:42.366 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:42.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:42.367 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:42.381 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-11T09:00:42.468 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:42.490 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:42.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:42.491 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:42.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:42.619 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:42.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:42.620 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:42 smithi145 ceph-mon[33384]: pgmap v1002: 168 pgs: 168 active+clean; 528 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 885 op/s 2024-09-11T09:00:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:42 smithi145 ceph-mon[33384]: 5.13 scrub starts 2024-09-11T09:00:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:42 smithi145 ceph-mon[33384]: 5.13 scrub ok 2024-09-11T09:00:42.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:42.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:42.743 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:42.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:42 smithi092 ceph-mon[29647]: pgmap v1002: 168 pgs: 168 active+clean; 528 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 885 op/s 2024-09-11T09:00:42.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:42 smithi092 ceph-mon[29647]: 5.13 scrub starts 2024-09-11T09:00:42.795 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:42 smithi092 ceph-mon[29647]: 5.13 scrub ok 2024-09-11T09:00:42.796 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:42 smithi092 ceph-mon[41114]: pgmap v1002: 168 pgs: 168 active+clean; 528 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 885 op/s 2024-09-11T09:00:42.796 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:42 smithi092 ceph-mon[41114]: 5.13 scrub starts 2024-09-11T09:00:42.796 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:42 smithi092 ceph-mon[41114]: 5.13 scrub ok 2024-09-11T09:00:42.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:42.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:42.866 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:42.867 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:42.919 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T09:00:42.919 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T09:00:42.919 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:00:42.920 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:00:42.927 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:42.928 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:42.928 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T09:00:42.929 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:00:42.945 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T09:00:42.945 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:00:42.964 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T09:00:42.964 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:00:42.967 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:42.980 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T09:00:42.980 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:00:42.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:42.992 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:42.993 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:42.995 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T09:00:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T09:00:43.011 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T09:00:43.011 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T09:00:43.093 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:43.116 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:43.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:43.117 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:43.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:00:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:00:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:00:43.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:43.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:43.241 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:43.241 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:43.342 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:43.366 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:43.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:43.367 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:43.468 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:43.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:43.491 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:43.491 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:43.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:43.608 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:43 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1556952385' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T09:00:43.608 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:43 smithi092 ceph-mon[29647]: 6.5 scrub starts 2024-09-11T09:00:43.609 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:43 smithi092 ceph-mon[29647]: 6.5 scrub ok 2024-09-11T09:00:43.609 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:43 smithi092 ceph-mon[29647]: 3.19 scrub starts 2024-09-11T09:00:43.609 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:43 smithi092 ceph-mon[29647]: 3.19 scrub ok 2024-09-11T09:00:43.609 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:43 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1556952385' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T09:00:43.609 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:43 smithi092 ceph-mon[41114]: 6.5 scrub starts 2024-09-11T09:00:43.609 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:43 smithi092 ceph-mon[41114]: 6.5 scrub ok 2024-09-11T09:00:43.609 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:43 smithi092 ceph-mon[41114]: 3.19 scrub starts 2024-09-11T09:00:43.609 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:43 smithi092 ceph-mon[41114]: 3.19 scrub ok 2024-09-11T09:00:43.613 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:43.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:43.614 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:43.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:43 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1556952385' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T09:00:43.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:43 smithi145 ceph-mon[33384]: 6.5 scrub starts 2024-09-11T09:00:43.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:43 smithi145 ceph-mon[33384]: 6.5 scrub ok 2024-09-11T09:00:43.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:43 smithi145 ceph-mon[33384]: 3.19 scrub starts 2024-09-11T09:00:43.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:43 smithi145 ceph-mon[33384]: 3.19 scrub ok 2024-09-11T09:00:43.715 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:43.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:43.739 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:43.739 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:43.840 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:43.862 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:43.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:43.863 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:43.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:43.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:43.988 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:44.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:44.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:44.115 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:44.116 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:44.217 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:44.239 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:44.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:44.240 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:44.341 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:44.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:44.365 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:44.365 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:44.466 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:44.488 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:44.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:44.489 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:44.590 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:44.612 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:44.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:44.614 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:44.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:44.736 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:44.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:44.737 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:44.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:44 smithi092 ceph-mon[29647]: 3.d scrub starts 2024-09-11T09:00:44.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:44 smithi092 ceph-mon[29647]: 3.d scrub ok 2024-09-11T09:00:44.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:44 smithi092 ceph-mon[29647]: pgmap v1003: 168 pgs: 168 active+clean; 528 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 555 op/s 2024-09-11T09:00:44.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:44 smithi092 ceph-mon[29647]: 4.1d scrub starts 2024-09-11T09:00:44.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:44 smithi092 ceph-mon[29647]: 4.1d scrub ok 2024-09-11T09:00:44.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:44 smithi092 ceph-mon[29647]: 6.1d scrub starts 2024-09-11T09:00:44.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:44 smithi092 ceph-mon[29647]: 6.1d scrub ok 2024-09-11T09:00:44.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:44 smithi092 ceph-mon[41114]: 3.d scrub starts 2024-09-11T09:00:44.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:44 smithi092 ceph-mon[41114]: 3.d scrub ok 2024-09-11T09:00:44.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:44 smithi092 ceph-mon[41114]: pgmap v1003: 168 pgs: 168 active+clean; 528 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 555 op/s 2024-09-11T09:00:44.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:44 smithi092 ceph-mon[41114]: 4.1d scrub starts 2024-09-11T09:00:44.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:44 smithi092 ceph-mon[41114]: 4.1d scrub ok 2024-09-11T09:00:44.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:44 smithi092 ceph-mon[41114]: 6.1d scrub starts 2024-09-11T09:00:44.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:44 smithi092 ceph-mon[41114]: 6.1d scrub ok 2024-09-11T09:00:44.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:44.859 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:44.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:44.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:44 smithi145 ceph-mon[33384]: 3.d scrub starts 2024-09-11T09:00:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:44 smithi145 ceph-mon[33384]: 3.d scrub ok 2024-09-11T09:00:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:44 smithi145 ceph-mon[33384]: pgmap v1003: 168 pgs: 168 active+clean; 528 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 555 op/s 2024-09-11T09:00:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:44 smithi145 ceph-mon[33384]: 4.1d scrub starts 2024-09-11T09:00:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:44 smithi145 ceph-mon[33384]: 4.1d scrub ok 2024-09-11T09:00:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:44 smithi145 ceph-mon[33384]: 6.1d scrub starts 2024-09-11T09:00:44.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:44 smithi145 ceph-mon[33384]: 6.1d scrub ok 2024-09-11T09:00:44.913 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:44.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:44.983 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:44.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:44.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:45.085 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:45.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:45.110 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:45.110 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:45.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:45.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:45.233 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:45.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:45.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:45.357 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:45.357 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:45.458 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:45.484 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:45.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:45.485 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:45.586 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:45.610 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:45.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:45.611 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:45.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:00:45.706 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:45 smithi092 ceph-mon[29647]: 11.3 scrub starts 2024-09-11T09:00:45.706 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:45 smithi092 ceph-mon[29647]: 11.3 scrub ok 2024-09-11T09:00:45.706 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:45 smithi092 ceph-mon[29647]: 6.c scrub starts 2024-09-11T09:00:45.706 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:45 smithi092 ceph-mon[29647]: 6.c scrub ok 2024-09-11T09:00:45.706 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:00:45.707 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:45 smithi092 ceph-mon[41114]: 11.3 scrub starts 2024-09-11T09:00:45.707 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:45 smithi092 ceph-mon[41114]: 11.3 scrub ok 2024-09-11T09:00:45.707 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:45 smithi092 ceph-mon[41114]: 6.c scrub starts 2024-09-11T09:00:45.707 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:45 smithi092 ceph-mon[41114]: 6.c scrub ok 2024-09-11T09:00:45.707 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:00:45.711 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:45.735 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:45.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:45.736 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:45.837 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:45.862 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:45.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:45.863 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:45.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:45 smithi145 ceph-mon[33384]: 11.3 scrub starts 2024-09-11T09:00:45.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:45 smithi145 ceph-mon[33384]: 11.3 scrub ok 2024-09-11T09:00:45.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:45 smithi145 ceph-mon[33384]: 6.c scrub starts 2024-09-11T09:00:45.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:45 smithi145 ceph-mon[33384]: 6.c scrub ok 2024-09-11T09:00:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:00:45.964 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:45.990 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:45.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:45.991 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:46.092 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:46.117 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:46.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:46.118 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:46.219 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:46.244 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:46.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:46.245 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:46.346 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:46.370 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:46.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:46.371 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:46.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:46.496 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:46.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:46.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:46.600 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:46.625 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:46.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:46.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:46.727 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:46.751 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:46.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:46.752 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:46.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:00:46.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:00:46.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[29647]: pgmap v1004: 168 pgs: 168 active+clean; 528 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 293 op/s 2024-09-11T09:00:46.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[29647]: 11.5 scrub starts 2024-09-11T09:00:46.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[29647]: 11.5 scrub ok 2024-09-11T09:00:46.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[29647]: osdmap e426: 8 total, 8 up, 8 in 2024-09-11T09:00:46.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[29647]: 6.6 scrub starts 2024-09-11T09:00:46.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[29647]: 6.6 scrub ok 2024-09-11T09:00:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:00:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:00:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[41114]: pgmap v1004: 168 pgs: 168 active+clean; 528 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 293 op/s 2024-09-11T09:00:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[41114]: 11.5 scrub starts 2024-09-11T09:00:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[41114]: 11.5 scrub ok 2024-09-11T09:00:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[41114]: osdmap e426: 8 total, 8 up, 8 in 2024-09-11T09:00:46.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[41114]: 6.6 scrub starts 2024-09-11T09:00:46.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:46 smithi092 ceph-mon[41114]: 6.6 scrub ok 2024-09-11T09:00:46.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:46.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:46.875 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:46.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:46.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:46 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:00:46.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:46 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:00:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:46 smithi145 ceph-mon[33384]: pgmap v1004: 168 pgs: 168 active+clean; 528 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 293 op/s 2024-09-11T09:00:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:46 smithi145 ceph-mon[33384]: 11.5 scrub starts 2024-09-11T09:00:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:46 smithi145 ceph-mon[33384]: 11.5 scrub ok 2024-09-11T09:00:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:46 smithi145 ceph-mon[33384]: osdmap e426: 8 total, 8 up, 8 in 2024-09-11T09:00:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:46 smithi145 ceph-mon[33384]: 6.6 scrub starts 2024-09-11T09:00:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:46 smithi145 ceph-mon[33384]: 6.6 scrub ok 2024-09-11T09:00:46.976 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:47.003 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:47.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:47.004 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:47.105 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:47.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:47.131 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:47.131 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:47.232 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:47.254 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:47.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:47.255 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:47.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:47.377 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:47.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:47.378 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:47.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:47.500 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:47.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:47.501 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:47.570 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T09:00:47.602 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:47.628 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:47.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:47.629 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:47.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:47 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4255081656' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T09:00:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:47 smithi145 ceph-mon[33384]: 1.e scrub starts 2024-09-11T09:00:47.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:47 smithi145 ceph-mon[33384]: 1.e scrub ok 2024-09-11T09:00:47.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:00:47 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:00:47.730 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:47.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:47.757 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:47.757 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:47.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:47 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4255081656' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T09:00:47.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:47 smithi092 ceph-mon[29647]: 1.e scrub starts 2024-09-11T09:00:47.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:47 smithi092 ceph-mon[29647]: 1.e scrub ok 2024-09-11T09:00:47.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:47 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4255081656' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T09:00:47.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:47 smithi092 ceph-mon[41114]: 1.e scrub starts 2024-09-11T09:00:47.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:47 smithi092 ceph-mon[41114]: 1.e scrub ok 2024-09-11T09:00:47.858 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:47.884 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:47.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:47.886 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:47.986 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:48.012 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:48.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:48.013 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:48.114 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:48.139 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:48.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:48.140 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:48.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:48.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:48.265 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:48.365 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:48.390 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:48.390 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:48.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:48.391 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:48.492 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:48.514 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:48.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:48.515 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:48.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:48.637 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:48.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:48.638 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:48.694 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:48.694 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[29647]: pgmap v1006: 168 pgs: 168 active+clean; 528 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:00:48.694 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T09:00:48.694 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T09:00:48.694 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4255081656' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T09:00:48.694 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[29647]: osdmap e427: 8 total, 8 up, 8 in 2024-09-11T09:00:48.694 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[29647]: 1.4 scrub starts 2024-09-11T09:00:48.694 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[29647]: 1.4 scrub ok 2024-09-11T09:00:48.695 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:48.695 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[41114]: pgmap v1006: 168 pgs: 168 active+clean; 528 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:00:48.695 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T09:00:48.695 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T09:00:48.695 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4255081656' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T09:00:48.695 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[41114]: osdmap e427: 8 total, 8 up, 8 in 2024-09-11T09:00:48.695 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[41114]: 1.4 scrub starts 2024-09-11T09:00:48.695 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:48 smithi092 ceph-mon[41114]: 1.4 scrub ok 2024-09-11T09:00:48.739 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:48.765 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:48.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:48.766 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:48.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:48.889 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:48.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:48.890 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:48.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:48 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:48.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:48 smithi145 ceph-mon[33384]: pgmap v1006: 168 pgs: 168 active+clean; 528 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:00:48.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:48 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T09:00:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:48 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T09:00:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:48 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4255081656' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T09:00:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:48 smithi145 ceph-mon[33384]: osdmap e427: 8 total, 8 up, 8 in 2024-09-11T09:00:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:48 smithi145 ceph-mon[33384]: 1.4 scrub starts 2024-09-11T09:00:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:48 smithi145 ceph-mon[33384]: 1.4 scrub ok 2024-09-11T09:00:48.990 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:49.017 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:49.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:49.017 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:49.118 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:49.143 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:49.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:49.144 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:49.245 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:49.266 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:49.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:49.267 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:49.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:49.390 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:49.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:49.391 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:49.492 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:49.518 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:49.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:49.519 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:49.620 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:49.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:49.643 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:49.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:49.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:49.766 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:49.766 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:49.867 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:49.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:49.893 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:49.894 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:49.900 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[29647]: 4.3 scrub starts 2024-09-11T09:00:49.900 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[29647]: 4.3 scrub ok 2024-09-11T09:00:49.900 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[29647]: 5.10 scrub starts 2024-09-11T09:00:49.900 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[29647]: 5.10 scrub ok 2024-09-11T09:00:49.900 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[29647]: 3.f scrub starts 2024-09-11T09:00:49.900 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[29647]: 3.f scrub ok 2024-09-11T09:00:49.900 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[29647]: 3.15 scrub starts 2024-09-11T09:00:49.900 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[29647]: 3.15 scrub ok 2024-09-11T09:00:49.900 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[41114]: 4.3 scrub starts 2024-09-11T09:00:49.900 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[41114]: 4.3 scrub ok 2024-09-11T09:00:49.901 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[41114]: 5.10 scrub starts 2024-09-11T09:00:49.901 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[41114]: 5.10 scrub ok 2024-09-11T09:00:49.901 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[41114]: 3.f scrub starts 2024-09-11T09:00:49.901 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[41114]: 3.f scrub ok 2024-09-11T09:00:49.901 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[41114]: 3.15 scrub starts 2024-09-11T09:00:49.901 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:49 smithi092 ceph-mon[41114]: 3.15 scrub ok 2024-09-11T09:00:49.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:49 smithi145 ceph-mon[33384]: 4.3 scrub starts 2024-09-11T09:00:49.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:49 smithi145 ceph-mon[33384]: 4.3 scrub ok 2024-09-11T09:00:49.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:49 smithi145 ceph-mon[33384]: 5.10 scrub starts 2024-09-11T09:00:49.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:49 smithi145 ceph-mon[33384]: 5.10 scrub ok 2024-09-11T09:00:49.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:49 smithi145 ceph-mon[33384]: 3.f scrub starts 2024-09-11T09:00:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:49 smithi145 ceph-mon[33384]: 3.f scrub ok 2024-09-11T09:00:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:49 smithi145 ceph-mon[33384]: 3.15 scrub starts 2024-09-11T09:00:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:49 smithi145 ceph-mon[33384]: 3.15 scrub ok 2024-09-11T09:00:49.994 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:50.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:50.021 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:50.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:50.049 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T09:00:50.052 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T09:00:50.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:50.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:50.144 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:50.144 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:50.245 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:50.270 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:50.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:50.271 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:50.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T09:00:50.372 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:50.397 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:50.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:50.398 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:50.499 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:50.520 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:50.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:50.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:50.622 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:50.648 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:50.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:50.649 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:50.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:50.772 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:50.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:50.773 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:50.825 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:50 smithi092 ceph-mon[29647]: pgmap v1008: 168 pgs: 168 active+clean; 528 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T09:00:50.825 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:50 smithi092 ceph-mon[29647]: 5.1 deep-scrub starts 2024-09-11T09:00:50.825 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:50 smithi092 ceph-mon[29647]: 5.1 deep-scrub ok 2024-09-11T09:00:50.825 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:50 smithi092 ceph-mon[29647]: 3.4 scrub starts 2024-09-11T09:00:50.826 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:50 smithi092 ceph-mon[29647]: 3.4 scrub ok 2024-09-11T09:00:50.826 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:50 smithi092 ceph-mon[41114]: pgmap v1008: 168 pgs: 168 active+clean; 528 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T09:00:50.826 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:50 smithi092 ceph-mon[41114]: 5.1 deep-scrub starts 2024-09-11T09:00:50.826 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:50 smithi092 ceph-mon[41114]: 5.1 deep-scrub ok 2024-09-11T09:00:50.826 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:50 smithi092 ceph-mon[41114]: 3.4 scrub starts 2024-09-11T09:00:50.826 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:50 smithi092 ceph-mon[41114]: 3.4 scrub ok 2024-09-11T09:00:50.874 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:50.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:50.896 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:50.896 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:50.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:50 smithi145 ceph-mon[33384]: pgmap v1008: 168 pgs: 168 active+clean; 528 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T09:00:50.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:50 smithi145 ceph-mon[33384]: 5.1 deep-scrub starts 2024-09-11T09:00:50.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:50 smithi145 ceph-mon[33384]: 5.1 deep-scrub ok 2024-09-11T09:00:50.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:50 smithi145 ceph-mon[33384]: 3.4 scrub starts 2024-09-11T09:00:50.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:50 smithi145 ceph-mon[33384]: 3.4 scrub ok 2024-09-11T09:00:50.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:51.018 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:51.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:51.019 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:51.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:51.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:51.143 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:51.152 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":1008,"stamp":"2024-09-11T09:00:49.496985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":540631,"num_objects":197,"num_object_clones":0,"num_object_copies":591,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":197,"num_whiteouts":0,"num_read":1916,"num_read_kb":1678,"num_write":243651,"num_write_kb":7764544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16763,"num_bytes_recovered":1073924473,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79799,"ondisk_log_size":79799,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":504,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":7181764,"kb_used_data":8484,"kb_used_omap":13,"kb_used_meta":7173170,"kb_avail":742779452,"statfs":{"total":767960285184,"available":760606158848,"internally_reserved":0,"allocated":8687616,"data_stored":5812844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":7345326784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002175"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":254,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.187133+0000","last_change":"2024-09-11T08:57:41.078011+0000","last_active":"2024-09-11T09:00:30.187133+0000","last_peered":"2024-09-11T09:00:30.187133+0000","last_clean":"2024-09-11T09:00:30.187133+0000","last_became_active":"2024-09-11T08:52:49.568416+0000","last_became_peered":"2024-09-11T08:52:49.568416+0000","last_unstale":"2024-09-11T09:00:30.187133+0000","last_undegraded":"2024-09-11T09:00:30.187133+0000","last_fullsized":"2024-09-11T09:00:30.187133+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:41.077950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:41.077950+0000","last_clean_scrub_stamp":"2024-09-11T08:57:41.077950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037516300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"402'25","reported_seq":565,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.188781+0000","last_change":"2024-09-11T08:59:47.853825+0000","last_active":"2024-09-11T09:00:30.188781+0000","last_peered":"2024-09-11T09:00:30.188781+0000","last_clean":"2024-09-11T09:00:30.188781+0000","last_became_active":"2024-09-11T08:58:34.124732+0000","last_became_peered":"2024-09-11T08:58:34.124732+0000","last_unstale":"2024-09-11T09:00:30.188781+0000","last_undegraded":"2024-09-11T09:00:30.188781+0000","last_fullsized":"2024-09-11T09:00:30.188781+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'25","last_scrub_stamp":"2024-09-11T08:59:47.853740+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T08:59:47.853740+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:08.785490+0000","scrub_duration":0.0029801060000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":474,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.206413+0000","last_change":"2024-09-11T08:59:58.149931+0000","last_active":"2024-09-11T09:00:30.206413+0000","last_peered":"2024-09-11T09:00:30.206413+0000","last_clean":"2024-09-11T09:00:30.206413+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T09:00:30.206413+0000","last_undegraded":"2024-09-11T09:00:30.206413+0000","last_fullsized":"2024-09-11T09:00:30.206413+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:58.149880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:36.907966+0000","last_clean_scrub_stamp":"2024-09-11T08:59:58.149880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:25.942941+0000","scrub_duration":0.017946175000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":443,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189335+0000","last_change":"2024-09-11T08:57:44.612014+0000","last_active":"2024-09-11T09:00:30.189335+0000","last_peered":"2024-09-11T09:00:30.189335+0000","last_clean":"2024-09-11T09:00:30.189335+0000","last_became_active":"2024-09-11T08:56:25.459294+0000","last_became_peered":"2024-09-11T08:56:25.459294+0000","last_unstale":"2024-09-11T09:00:30.189335+0000","last_undegraded":"2024-09-11T09:00:30.189335+0000","last_fullsized":"2024-09-11T09:00:30.189335+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:44.611899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:57:44.611899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034089490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":482,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.196205+0000","last_change":"2024-09-11T08:58:34.136920+0000","last_active":"2024-09-11T09:00:30.196205+0000","last_peered":"2024-09-11T09:00:30.196205+0000","last_clean":"2024-09-11T09:00:30.196205+0000","last_became_active":"2024-09-11T08:58:34.135162+0000","last_became_peered":"2024-09-11T08:58:34.135162+0000","last_unstale":"2024-09-11T09:00:30.196205+0000","last_undegraded":"2024-09-11T09:00:30.196205+0000","last_fullsized":"2024-09-11T09:00:30.196205+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:46.987503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:57:46.987503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054907039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":491,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.196077+0000","last_change":"2024-09-11T08:59:31.639818+0000","last_active":"2024-09-11T09:00:30.196077+0000","last_peered":"2024-09-11T09:00:30.196077+0000","last_clean":"2024-09-11T09:00:30.196077+0000","last_became_active":"2024-09-11T08:58:34.316048+0000","last_became_peered":"2024-09-11T08:58:34.316048+0000","last_unstale":"2024-09-11T09:00:30.196077+0000","last_undegraded":"2024-09-11T09:00:30.196077+0000","last_fullsized":"2024-09-11T09:00:30.196077+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:31.639729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:59:31.639729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:49.255282+0000","scrub_duration":0.0063344730000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"402'46","reported_seq":566,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.194437+0000","last_change":"2024-09-11T08:57:20.749529+0000","last_active":"2024-09-11T09:00:30.194437+0000","last_peered":"2024-09-11T09:00:30.194437+0000","last_clean":"2024-09-11T09:00:30.194437+0000","last_became_active":"2024-09-11T08:52:18.585600+0000","last_became_peered":"2024-09-11T08:52:18.585600+0000","last_unstale":"2024-09-11T09:00:30.194437+0000","last_undegraded":"2024-09-11T09:00:30.194437+0000","last_fullsized":"2024-09-11T09:00:30.194437+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:57:20.749472+0000","last_deep_scrub":"200'34","last_deep_scrub_stamp":"2024-09-11T08:51:31.358660+0000","last_clean_scrub_stamp":"2024-09-11T08:57:20.749472+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017730809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":465,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.188709+0000","last_change":"2024-09-11T08:59:18.590510+0000","last_active":"2024-09-11T09:00:30.188709+0000","last_peered":"2024-09-11T09:00:30.188709+0000","last_clean":"2024-09-11T09:00:30.188709+0000","last_became_active":"2024-09-11T08:59:18.590212+0000","last_became_peered":"2024-09-11T08:59:18.590212+0000","last_unstale":"2024-09-11T09:00:30.188709+0000","last_undegraded":"2024-09-11T09:00:30.188709+0000","last_fullsized":"2024-09-11T09:00:30.188709+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:48.634491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T08:57:48.634491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051186900000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":494,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.191409+0000","last_change":"2024-09-11T08:58:34.154108+0000","last_active":"2024-09-11T09:00:30.191409+0000","last_peered":"2024-09-11T09:00:30.191409+0000","last_clean":"2024-09-11T09:00:30.191409+0000","last_became_active":"2024-09-11T08:58:34.153391+0000","last_became_peered":"2024-09-11T08:58:34.153391+0000","last_unstale":"2024-09-11T09:00:30.191409+0000","last_undegraded":"2024-09-11T09:00:30.191409+0000","last_fullsized":"2024-09-11T09:00:30.191409+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:48.014030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T08:57:48.014030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060467710000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"402'25","reported_seq":525,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189483+0000","last_change":"2024-09-11T08:57:43.648861+0000","last_active":"2024-09-11T09:00:30.189483+0000","last_peered":"2024-09-11T09:00:30.189483+0000","last_clean":"2024-09-11T09:00:30.189483+0000","last_became_active":"2024-09-11T08:52:18.577411+0000","last_became_peered":"2024-09-11T08:52:18.577411+0000","last_unstale":"2024-09-11T09:00:30.189483+0000","last_undegraded":"2024-09-11T09:00:30.189483+0000","last_fullsized":"2024-09-11T09:00:30.189483+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:57:43.648772+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T08:57:43.648772+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.038205647000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":480,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.186750+0000","last_change":"2024-09-11T09:00:01.402698+0000","last_active":"2024-09-11T09:00:30.186750+0000","last_peered":"2024-09-11T09:00:30.186750+0000","last_clean":"2024-09-11T09:00:30.186750+0000","last_became_active":"2024-09-11T08:52:18.573461+0000","last_became_peered":"2024-09-11T08:52:18.573461+0000","last_unstale":"2024-09-11T09:00:30.186750+0000","last_undegraded":"2024-09-11T09:00:30.186750+0000","last_fullsized":"2024-09-11T09:00:30.186750+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:01.402615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:00:01.402615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:18.007058+0000","scrub_duration":0.0026449749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":274,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:39.010896+0000","last_change":"2024-09-11T09:00:39.010833+0000","last_active":"2024-09-11T09:00:39.010896+0000","last_peered":"2024-09-11T09:00:39.010896+0000","last_clean":"2024-09-11T09:00:39.010896+0000","last_became_active":"2024-09-11T08:52:49.568147+0000","last_became_peered":"2024-09-11T08:52:49.568147+0000","last_unstale":"2024-09-11T09:00:39.010896+0000","last_undegraded":"2024-09-11T09:00:39.010896+0000","last_fullsized":"2024-09-11T09:00:39.010896+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:39.010738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:33.520462+0000","last_clean_scrub_stamp":"2024-09-11T09:00:39.010738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:43.746978+0000","scrub_duration":0.0010920210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":454,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.186040+0000","last_change":"2024-09-11T08:57:48.987018+0000","last_active":"2024-09-11T09:00:30.186040+0000","last_peered":"2024-09-11T09:00:30.186040+0000","last_clean":"2024-09-11T09:00:30.186040+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T09:00:30.186040+0000","last_undegraded":"2024-09-11T09:00:30.186040+0000","last_fullsized":"2024-09-11T09:00:30.186040+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:48.986930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:30.465003+0000","last_clean_scrub_stamp":"2024-09-11T08:57:48.986930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031690020000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"402'9","reported_seq":472,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189676+0000","last_change":"2024-09-11T08:59:45.817071+0000","last_active":"2024-09-11T09:00:30.189676+0000","last_peered":"2024-09-11T09:00:30.189676+0000","last_clean":"2024-09-11T09:00:30.189676+0000","last_became_active":"2024-09-11T08:56:25.465945+0000","last_became_peered":"2024-09-11T08:56:25.465945+0000","last_unstale":"2024-09-11T09:00:30.189676+0000","last_undegraded":"2024-09-11T09:00:30.189676+0000","last_fullsized":"2024-09-11T09:00:30.189676+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'9","last_scrub_stamp":"2024-09-11T08:59:45.816978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:59:45.816978+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:11.478081+0000","scrub_duration":0.0099802310000000009,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":476,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.191727+0000","last_change":"2024-09-11T08:59:31.591175+0000","last_active":"2024-09-11T09:00:30.191727+0000","last_peered":"2024-09-11T09:00:30.191727+0000","last_clean":"2024-09-11T09:00:30.191727+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T09:00:30.191727+0000","last_undegraded":"2024-09-11T09:00:30.191727+0000","last_fullsized":"2024-09-11T09:00:30.191727+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:31.591088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:18.199651+0000","last_clean_scrub_stamp":"2024-09-11T08:59:31.591088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:53.925123+0000","scrub_duration":0.0059973370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":448,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.185772+0000","last_change":"2024-09-11T08:58:44.937454+0000","last_active":"2024-09-11T09:00:30.185772+0000","last_peered":"2024-09-11T09:00:30.185772+0000","last_clean":"2024-09-11T09:00:30.185772+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T09:00:30.185772+0000","last_undegraded":"2024-09-11T09:00:30.185772+0000","last_fullsized":"2024-09-11T09:00:30.185772+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:58:44.937350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T08:58:44.937350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040866110000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":473,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189796+0000","last_change":"2024-09-11T08:58:34.126432+0000","last_active":"2024-09-11T09:00:30.189796+0000","last_peered":"2024-09-11T09:00:30.189796+0000","last_clean":"2024-09-11T09:00:30.189796+0000","last_became_active":"2024-09-11T08:58:34.126085+0000","last_became_peered":"2024-09-11T08:58:34.126085+0000","last_unstale":"2024-09-11T09:00:30.189796+0000","last_undegraded":"2024-09-11T09:00:30.189796+0000","last_fullsized":"2024-09-11T09:00:30.189796+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:28.557623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:57:28.557623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001331053,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":254,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189849+0000","last_change":"2024-09-11T08:57:32.534621+0000","last_active":"2024-09-11T09:00:30.189849+0000","last_peered":"2024-09-11T09:00:30.189849+0000","last_clean":"2024-09-11T09:00:30.189849+0000","last_became_active":"2024-09-11T08:52:49.566575+0000","last_became_peered":"2024-09-11T08:52:49.566575+0000","last_unstale":"2024-09-11T09:00:30.189849+0000","last_undegraded":"2024-09-11T09:00:30.189849+0000","last_fullsized":"2024-09-11T09:00:30.189849+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:32.534529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:57:32.534529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066860009999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"402'46","reported_seq":431,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.207368+0000","last_change":"2024-09-11T09:00:01.063220+0000","last_active":"2024-09-11T09:00:30.207368+0000","last_peered":"2024-09-11T09:00:30.207368+0000","last_clean":"2024-09-11T09:00:30.207368+0000","last_became_active":"2024-09-11T08:54:17.121459+0000","last_became_peered":"2024-09-11T08:54:17.121459+0000","last_unstale":"2024-09-11T09:00:30.207368+0000","last_undegraded":"2024-09-11T09:00:30.207368+0000","last_fullsized":"2024-09-11T09:00:30.207368+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'46","last_scrub_stamp":"2024-09-11T09:00:01.063132+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T09:00:01.063132+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:21.388713+0000","scrub_duration":0.0081816179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":494,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.208282+0000","last_change":"2024-09-11T09:00:00.108429+0000","last_active":"2024-09-11T09:00:30.208282+0000","last_peered":"2024-09-11T09:00:30.208282+0000","last_clean":"2024-09-11T09:00:30.208282+0000","last_became_active":"2024-09-11T08:52:18.579026+0000","last_became_peered":"2024-09-11T08:52:18.579026+0000","last_unstale":"2024-09-11T09:00:30.208282+0000","last_undegraded":"2024-09-11T09:00:30.208282+0000","last_fullsized":"2024-09-11T09:00:30.208282+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:00.108347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:31.860874+0000","last_clean_scrub_stamp":"2024-09-11T09:00:00.108347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:16.257463+0000","scrub_duration":0.0068836330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"419'2","reported_seq":500,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.196293+0000","last_change":"2024-09-11T08:59:52.794454+0000","last_active":"2024-09-11T09:00:30.196293+0000","last_peered":"2024-09-11T09:00:30.196293+0000","last_clean":"2024-09-11T09:00:30.196293+0000","last_became_active":"2024-09-11T08:58:34.135094+0000","last_became_peered":"2024-09-11T08:58:34.135094+0000","last_unstale":"2024-09-11T09:00:30.196293+0000","last_undegraded":"2024-09-11T09:00:30.196293+0000","last_fullsized":"2024-09-11T09:00:30.196293+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"419'2","last_scrub_stamp":"2024-09-11T08:59:52.794363+0000","last_deep_scrub":"419'2","last_deep_scrub_stamp":"2024-09-11T08:59:52.794363+0000","last_clean_scrub_stamp":"2024-09-11T08:59:52.794363+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:56.504438+0000","scrub_duration":0.01289274,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":439,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189629+0000","last_change":"2024-09-11T08:57:34.558305+0000","last_active":"2024-09-11T09:00:30.189629+0000","last_peered":"2024-09-11T09:00:30.189629+0000","last_clean":"2024-09-11T09:00:30.189629+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T09:00:30.189629+0000","last_undegraded":"2024-09-11T09:00:30.189629+0000","last_fullsized":"2024-09-11T09:00:30.189629+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:34.558209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:57:34.558209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040844999999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"402'24","reported_seq":509,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:43.950719+0000","last_change":"2024-09-11T09:00:43.950649+0000","last_active":"2024-09-11T09:00:43.950719+0000","last_peered":"2024-09-11T09:00:43.950719+0000","last_clean":"2024-09-11T09:00:43.950719+0000","last_became_active":"2024-09-11T08:49:16.943886+0000","last_became_peered":"2024-09-11T08:49:16.943886+0000","last_unstale":"2024-09-11T09:00:43.950719+0000","last_undegraded":"2024-09-11T09:00:43.950719+0000","last_fullsized":"2024-09-11T09:00:43.950719+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'24","last_scrub_stamp":"2024-09-11T09:00:43.950557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:00:43.950557+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:46.103145+0000","scrub_duration":0.002215692,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":435,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.190148+0000","last_change":"2024-09-11T08:57:17.636602+0000","last_active":"2024-09-11T09:00:30.190148+0000","last_peered":"2024-09-11T09:00:30.190148+0000","last_clean":"2024-09-11T09:00:30.190148+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T09:00:30.190148+0000","last_undegraded":"2024-09-11T09:00:30.190148+0000","last_fullsized":"2024-09-11T09:00:30.190148+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:17.636528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:57:17.636528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039989719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":181,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.185468+0000","last_change":"2024-09-11T08:57:30.044915+0000","last_active":"2024-09-11T09:00:30.185468+0000","last_peered":"2024-09-11T09:00:30.185468+0000","last_clean":"2024-09-11T09:00:30.185468+0000","last_became_active":"2024-09-11T08:52:38.483603+0000","last_became_peered":"2024-09-11T08:52:38.483603+0000","last_unstale":"2024-09-11T09:00:30.185468+0000","last_undegraded":"2024-09-11T09:00:30.185468+0000","last_fullsized":"2024-09-11T09:00:30.185468+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T08:57:30.044827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:57:30.044827+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031105490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":478,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:43.016688+0000","last_change":"2024-09-11T09:00:43.016635+0000","last_active":"2024-09-11T09:00:43.016688+0000","last_peered":"2024-09-11T09:00:43.016688+0000","last_clean":"2024-09-11T09:00:43.016688+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T09:00:43.016688+0000","last_undegraded":"2024-09-11T09:00:43.016688+0000","last_fullsized":"2024-09-11T09:00:43.016688+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:43.016578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T09:00:43.016578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:51.822449+0000","scrub_duration":0.001396362,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"402'35","reported_seq":212,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.195845+0000","last_change":"2024-09-11T08:59:36.784471+0000","last_active":"2024-09-11T09:00:30.195845+0000","last_peered":"2024-09-11T09:00:30.195845+0000","last_clean":"2024-09-11T09:00:30.195845+0000","last_became_active":"2024-09-11T08:52:49.568074+0000","last_became_peered":"2024-09-11T08:52:49.568074+0000","last_unstale":"2024-09-11T09:00:30.195845+0000","last_undegraded":"2024-09-11T09:00:30.195845+0000","last_fullsized":"2024-09-11T09:00:30.195845+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'35","last_scrub_stamp":"2024-09-11T08:59:36.784384+0000","last_deep_scrub":"280'25","last_deep_scrub_stamp":"2024-09-11T08:52:24.584533+0000","last_clean_scrub_stamp":"2024-09-11T08:59:36.784384+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:59.827339+0000","scrub_duration":0.0089222299999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"401'11","reported_seq":210,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:42.923455+0000","last_change":"2024-09-11T08:59:44.834984+0000","last_active":"2024-09-11T09:00:42.923455+0000","last_peered":"2024-09-11T09:00:42.923455+0000","last_clean":"2024-09-11T09:00:42.923455+0000","last_became_active":"2024-09-11T08:58:34.136152+0000","last_became_peered":"2024-09-11T08:58:34.136152+0000","last_unstale":"2024-09-11T09:00:42.923455+0000","last_undegraded":"2024-09-11T09:00:42.923455+0000","last_fullsized":"2024-09-11T09:00:42.923455+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"401'11","last_scrub_stamp":"2024-09-11T08:59:44.834892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:59:44.834892+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:04.284189+0000","scrub_duration":0.001910653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.0","version":"425'15052","reported_seq":15266,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.445581+0000","last_change":"2024-09-11T08:59:51.421747+0000","last_active":"2024-09-11T09:00:34.445581+0000","last_peered":"2024-09-11T09:00:34.445581+0000","last_clean":"2024-09-11T09:00:34.445581+0000","last_became_active":"2024-09-11T08:57:14.476693+0000","last_became_peered":"2024-09-11T08:57:14.476693+0000","last_unstale":"2024-09-11T09:00:34.445581+0000","last_undegraded":"2024-09-11T09:00:34.445581+0000","last_fullsized":"2024-09-11T09:00:34.445581+0000","mapping_epoch":366,"log_start":"411'8900","ondisk_log_start":"411'8900","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"418'11739","last_scrub_stamp":"2024-09-11T08:59:51.421659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:59:51.421659+0000","objects_scrubbed":3396,"log_size":6152,"log_dups_size":0,"ondisk_log_size":6152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:04.590096+0000","scrub_duration":2.2078821350000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15052,"num_write_kb":481664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"199'81","reported_seq":235,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:31.340329+0000","last_change":"2024-09-11T09:00:31.340141+0000","last_active":"2024-09-11T09:00:31.340329+0000","last_peered":"2024-09-11T09:00:31.340329+0000","last_clean":"2024-09-11T09:00:31.340329+0000","last_became_active":"2024-09-11T08:58:34.313408+0000","last_became_peered":"2024-09-11T08:58:34.313408+0000","last_unstale":"2024-09-11T09:00:31.340329+0000","last_undegraded":"2024-09-11T09:00:31.340329+0000","last_fullsized":"2024-09-11T09:00:31.340329+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:00:31.339980+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:00:31.339980+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:39.117639+0000","scrub_duration":0.0030218139999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"402'46","reported_seq":537,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.208634+0000","last_change":"2024-09-11T08:59:38.423750+0000","last_active":"2024-09-11T09:00:30.208634+0000","last_peered":"2024-09-11T09:00:30.208634+0000","last_clean":"2024-09-11T09:00:30.208634+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T09:00:30.208634+0000","last_undegraded":"2024-09-11T09:00:30.208634+0000","last_fullsized":"2024-09-11T09:00:30.208634+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'46","last_scrub_stamp":"2024-09-11T08:59:38.423658+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T08:59:38.423658+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:47.092676+0000","scrub_duration":0.014428547999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":448,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.197815+0000","last_change":"2024-09-11T08:59:49.780722+0000","last_active":"2024-09-11T09:00:30.197815+0000","last_peered":"2024-09-11T09:00:30.197815+0000","last_clean":"2024-09-11T09:00:30.197815+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T09:00:30.197815+0000","last_undegraded":"2024-09-11T09:00:30.197815+0000","last_fullsized":"2024-09-11T09:00:30.197815+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:49.780653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:59:49.780653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:59.843753+0000","scrub_duration":0.023162815999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"277'11","reported_seq":747,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:42.923486+0000","last_change":"2024-09-11T09:00:01.040962+0000","last_active":"2024-09-11T09:00:42.923486+0000","last_peered":"2024-09-11T09:00:42.923486+0000","last_clean":"2024-09-11T09:00:42.923486+0000","last_became_active":"2024-09-11T08:52:18.578328+0000","last_became_peered":"2024-09-11T08:52:18.578328+0000","last_unstale":"2024-09-11T09:00:42.923486+0000","last_undegraded":"2024-09-11T09:00:42.923486+0000","last_fullsized":"2024-09-11T09:00:42.923486+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'11","last_scrub_stamp":"2024-09-11T09:00:01.040867+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T09:00:01.040867+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:05.908694+0000","scrub_duration":0.0029381519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":481,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.321604+0000","last_change":"2024-09-11T09:00:30.321377+0000","last_active":"2024-09-11T09:00:30.321604+0000","last_peered":"2024-09-11T09:00:30.321604+0000","last_clean":"2024-09-11T09:00:30.321604+0000","last_became_active":"2024-09-11T08:56:25.462088+0000","last_became_peered":"2024-09-11T08:56:25.462088+0000","last_unstale":"2024-09-11T09:00:30.321604+0000","last_undegraded":"2024-09-11T09:00:30.321604+0000","last_fullsized":"2024-09-11T09:00:30.321604+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:30.321250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:00:30.321250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:41.284205+0000","scrub_duration":0.0034720300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"425'15092","reported_seq":15466,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.445687+0000","last_change":"2024-09-11T08:59:43.036436+0000","last_active":"2024-09-11T09:00:34.445687+0000","last_peered":"2024-09-11T09:00:34.445687+0000","last_clean":"2024-09-11T09:00:34.445687+0000","last_became_active":"2024-09-11T08:57:14.473631+0000","last_became_peered":"2024-09-11T08:57:14.473631+0000","last_unstale":"2024-09-11T09:00:34.445687+0000","last_undegraded":"2024-09-11T09:00:34.445687+0000","last_fullsized":"2024-09-11T09:00:34.445687+0000","mapping_epoch":366,"log_start":"411'8900","ondisk_log_start":"411'8900","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"416'11053","last_scrub_stamp":"2024-09-11T08:59:43.036344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:59:43.036344+0000","objects_scrubbed":4104,"log_size":6192,"log_dups_size":0,"ondisk_log_size":6192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:53.371347+0000","scrub_duration":2.6013681530000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15092,"num_write_kb":482944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"199'81","reported_seq":235,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.190432+0000","last_change":"2024-09-11T08:58:34.324876+0000","last_active":"2024-09-11T09:00:30.190432+0000","last_peered":"2024-09-11T09:00:30.190432+0000","last_clean":"2024-09-11T09:00:30.190432+0000","last_became_active":"2024-09-11T08:58:34.324527+0000","last_became_peered":"2024-09-11T08:58:34.324527+0000","last_unstale":"2024-09-11T09:00:30.190432+0000","last_undegraded":"2024-09-11T09:00:30.190432+0000","last_fullsized":"2024-09-11T09:00:30.190432+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:56:33.671306+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:56:33.671306+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011424930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"402'50","reported_seq":578,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.197532+0000","last_change":"2024-09-11T08:58:48.517831+0000","last_active":"2024-09-11T09:00:30.197532+0000","last_peered":"2024-09-11T09:00:30.197532+0000","last_clean":"2024-09-11T09:00:30.197532+0000","last_became_active":"2024-09-11T08:52:18.595013+0000","last_became_peered":"2024-09-11T08:52:18.595013+0000","last_unstale":"2024-09-11T09:00:30.197532+0000","last_undegraded":"2024-09-11T09:00:30.197532+0000","last_fullsized":"2024-09-11T09:00:30.197532+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'50","last_scrub_stamp":"2024-09-11T08:58:48.517737+0000","last_deep_scrub":"199'26","last_deep_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_clean_scrub_stamp":"2024-09-11T08:58:48.517737+0000","objects_scrubbed":8,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046192949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":471,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.196677+0000","last_change":"2024-09-11T08:58:52.524400+0000","last_active":"2024-09-11T09:00:30.196677+0000","last_peered":"2024-09-11T09:00:30.196677+0000","last_clean":"2024-09-11T09:00:30.196677+0000","last_became_active":"2024-09-11T08:52:18.586084+0000","last_became_peered":"2024-09-11T08:52:18.586084+0000","last_unstale":"2024-09-11T09:00:30.196677+0000","last_undegraded":"2024-09-11T09:00:30.196677+0000","last_fullsized":"2024-09-11T09:00:30.196677+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:58:52.524310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:58:52.524310+0000","last_clean_scrub_stamp":"2024-09-11T08:58:52.524310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027447449999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":435,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.188998+0000","last_change":"2024-09-11T08:57:18.592168+0000","last_active":"2024-09-11T09:00:30.188998+0000","last_peered":"2024-09-11T09:00:30.188998+0000","last_clean":"2024-09-11T09:00:30.188998+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T09:00:30.188998+0000","last_undegraded":"2024-09-11T09:00:30.188998+0000","last_fullsized":"2024-09-11T09:00:30.188998+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:18.592099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:29.687577+0000","last_clean_scrub_stamp":"2024-09-11T08:57:18.592099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001725077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":433,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189693+0000","last_change":"2024-09-11T08:57:27.597138+0000","last_active":"2024-09-11T09:00:30.189693+0000","last_peered":"2024-09-11T09:00:30.189693+0000","last_clean":"2024-09-11T09:00:30.189693+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T09:00:30.189693+0000","last_undegraded":"2024-09-11T09:00:30.189693+0000","last_fullsized":"2024-09-11T09:00:30.189693+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:27.597069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:57:27.597069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001643186,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"425'15206","reported_seq":15274,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.441053+0000","last_change":"2024-09-11T08:57:14.474398+0000","last_active":"2024-09-11T09:00:34.441053+0000","last_peered":"2024-09-11T09:00:34.441053+0000","last_clean":"2024-09-11T09:00:34.441053+0000","last_became_active":"2024-09-11T08:57:14.474062+0000","last_became_peered":"2024-09-11T08:57:14.474062+0000","last_unstale":"2024-09-11T09:00:34.441053+0000","last_undegraded":"2024-09-11T09:00:34.441053+0000","last_fullsized":"2024-09-11T09:00:34.441053+0000","mapping_epoch":366,"log_start":"416'10964","ondisk_log_start":"416'10964","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":4242,"log_dups_size":0,"ondisk_log_size":4242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15206,"num_write_kb":486592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"199'81","reported_seq":220,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.194315+0000","last_change":"2024-09-11T08:58:01.125510+0000","last_active":"2024-09-11T09:00:30.194315+0000","last_peered":"2024-09-11T09:00:30.194315+0000","last_clean":"2024-09-11T09:00:30.194315+0000","last_became_active":"2024-09-11T08:58:01.125235+0000","last_became_peered":"2024-09-11T08:58:01.125235+0000","last_unstale":"2024-09-11T09:00:30.194315+0000","last_undegraded":"2024-09-11T09:00:30.194315+0000","last_fullsized":"2024-09-11T09:00:30.194315+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:55:53.823609+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:55:53.823609+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052441400000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109226,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"402'49","reported_seq":562,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189133+0000","last_change":"2024-09-11T08:59:46.855924+0000","last_active":"2024-09-11T09:00:30.189133+0000","last_peered":"2024-09-11T09:00:30.189133+0000","last_clean":"2024-09-11T09:00:30.189133+0000","last_became_active":"2024-09-11T08:58:34.127547+0000","last_became_peered":"2024-09-11T08:58:34.127547+0000","last_unstale":"2024-09-11T09:00:30.189133+0000","last_undegraded":"2024-09-11T09:00:30.189133+0000","last_fullsized":"2024-09-11T09:00:30.189133+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'49","last_scrub_stamp":"2024-09-11T08:59:46.855861+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T08:59:46.855861+0000","objects_scrubbed":7,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:53.917065+0000","scrub_duration":0.0018591650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":63,"num_read_kb":56,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":221,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:36.602411+0000","last_change":"2024-09-11T09:00:36.602365+0000","last_active":"2024-09-11T09:00:36.602411+0000","last_peered":"2024-09-11T09:00:36.602411+0000","last_clean":"2024-09-11T09:00:36.602411+0000","last_became_active":"2024-09-11T08:52:18.586549+0000","last_became_peered":"2024-09-11T08:52:18.586549+0000","last_unstale":"2024-09-11T09:00:36.602411+0000","last_undegraded":"2024-09-11T09:00:36.602411+0000","last_fullsized":"2024-09-11T09:00:36.602411+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:36.602311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:28.487573+0000","last_clean_scrub_stamp":"2024-09-11T09:00:36.602311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:46.380660+0000","scrub_duration":0.0014473520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":710,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:42.923355+0000","last_change":"2024-09-11T08:57:22.782387+0000","last_active":"2024-09-11T09:00:42.923355+0000","last_peered":"2024-09-11T09:00:42.923355+0000","last_clean":"2024-09-11T09:00:42.923355+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T09:00:42.923355+0000","last_undegraded":"2024-09-11T09:00:42.923355+0000","last_fullsized":"2024-09-11T09:00:42.923355+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:57:22.782332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:57:22.782332+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001789934,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":302,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.203188+0000","last_change":"2024-09-11T08:59:48.072943+0000","last_active":"2024-09-11T09:00:30.203188+0000","last_peered":"2024-09-11T09:00:30.203188+0000","last_clean":"2024-09-11T09:00:30.203188+0000","last_became_active":"2024-09-11T08:58:34.316988+0000","last_became_peered":"2024-09-11T08:58:34.316988+0000","last_unstale":"2024-09-11T09:00:30.203188+0000","last_undegraded":"2024-09-11T09:00:30.203188+0000","last_fullsized":"2024-09-11T09:00:30.203188+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:48.072866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:59:48.072866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:02.483768+0000","scrub_duration":0.0019439800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"425'15222","reported_seq":18331,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.451718+0000","last_change":"2024-09-11T08:59:42.633506+0000","last_active":"2024-09-11T09:00:34.451718+0000","last_peered":"2024-09-11T09:00:34.451718+0000","last_clean":"2024-09-11T09:00:34.451718+0000","last_became_active":"2024-09-11T08:58:49.953605+0000","last_became_peered":"2024-09-11T08:58:49.953605+0000","last_unstale":"2024-09-11T09:00:34.451718+0000","last_undegraded":"2024-09-11T09:00:34.451718+0000","last_fullsized":"2024-09-11T09:00:34.451718+0000","mapping_epoch":403,"log_start":"414'9932","ondisk_log_start":"414'9932","created":366,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"416'11204","last_scrub_stamp":"2024-09-11T08:59:42.633425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:59:42.633425+0000","objects_scrubbed":4075,"log_size":5290,"log_dups_size":0,"ondisk_log_size":5290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:57.016549+0000","scrub_duration":1.640879924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15222,"num_write_kb":487041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1418,"num_bytes_recovered":92930048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"199'81","reported_seq":376,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.196485+0000","last_change":"2024-09-11T08:58:34.135605+0000","last_active":"2024-09-11T09:00:30.196485+0000","last_peered":"2024-09-11T09:00:30.196485+0000","last_clean":"2024-09-11T09:00:30.196485+0000","last_became_active":"2024-09-11T08:58:34.135067+0000","last_became_peered":"2024-09-11T08:58:34.135067+0000","last_unstale":"2024-09-11T09:00:30.196485+0000","last_undegraded":"2024-09-11T09:00:30.196485+0000","last_fullsized":"2024-09-11T09:00:30.196485+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:57:41.844437+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:57:41.844437+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0081335720000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"402'35","reported_seq":371,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:37.349662+0000","last_change":"2024-09-11T09:00:37.349588+0000","last_active":"2024-09-11T09:00:37.349662+0000","last_peered":"2024-09-11T09:00:37.349662+0000","last_clean":"2024-09-11T09:00:37.349662+0000","last_became_active":"2024-09-11T08:56:25.462200+0000","last_became_peered":"2024-09-11T08:56:25.462200+0000","last_unstale":"2024-09-11T09:00:37.349662+0000","last_undegraded":"2024-09-11T09:00:37.349662+0000","last_fullsized":"2024-09-11T09:00:37.349662+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'35","last_scrub_stamp":"2024-09-11T09:00:37.349520+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_clean_scrub_stamp":"2024-09-11T09:00:37.349520+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:01.441169+0000","scrub_duration":0.0022442370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":454,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.191468+0000","last_change":"2024-09-11T08:59:50.520317+0000","last_active":"2024-09-11T09:00:30.191468+0000","last_peered":"2024-09-11T09:00:30.191468+0000","last_clean":"2024-09-11T09:00:30.191468+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T09:00:30.191468+0000","last_undegraded":"2024-09-11T09:00:30.191468+0000","last_fullsized":"2024-09-11T09:00:30.191468+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:50.520207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:59:50.520207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:04.993928+0000","scrub_duration":0.0066407949999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"401'14","reported_seq":210,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:42.923488+0000","last_change":"2024-09-11T08:59:40.420154+0000","last_active":"2024-09-11T09:00:42.923488+0000","last_peered":"2024-09-11T09:00:42.923488+0000","last_clean":"2024-09-11T09:00:42.923488+0000","last_became_active":"2024-09-11T08:58:34.140748+0000","last_became_peered":"2024-09-11T08:58:34.140748+0000","last_unstale":"2024-09-11T09:00:42.923488+0000","last_undegraded":"2024-09-11T09:00:42.923488+0000","last_fullsized":"2024-09-11T09:00:42.923488+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"401'14","last_scrub_stamp":"2024-09-11T08:59:40.420037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:59:40.420037+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:51.882993+0000","scrub_duration":0.0053309489999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":254,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.197361+0000","last_change":"2024-09-11T08:57:18.779086+0000","last_active":"2024-09-11T09:00:30.197361+0000","last_peered":"2024-09-11T09:00:30.197361+0000","last_clean":"2024-09-11T09:00:30.197361+0000","last_became_active":"2024-09-11T08:52:49.571683+0000","last_became_peered":"2024-09-11T08:52:49.571683+0000","last_unstale":"2024-09-11T09:00:30.197361+0000","last_undegraded":"2024-09-11T09:00:30.197361+0000","last_fullsized":"2024-09-11T09:00:30.197361+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:18.779033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:51.896002+0000","last_clean_scrub_stamp":"2024-09-11T08:57:18.779033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001147438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"425'14946","reported_seq":15038,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.447277+0000","last_change":"2024-09-11T09:00:31.250917+0000","last_active":"2024-09-11T09:00:34.447277+0000","last_peered":"2024-09-11T09:00:34.447277+0000","last_clean":"2024-09-11T09:00:34.447277+0000","last_became_active":"2024-09-11T08:57:14.473651+0000","last_became_peered":"2024-09-11T08:57:14.473651+0000","last_unstale":"2024-09-11T09:00:34.447277+0000","last_undegraded":"2024-09-11T09:00:34.447277+0000","last_fullsized":"2024-09-11T09:00:34.447277+0000","mapping_epoch":366,"log_start":"411'8800","ondisk_log_start":"411'8800","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"425'14685","last_scrub_stamp":"2024-09-11T09:00:31.250726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T09:00:31.250726+0000","objects_scrubbed":264,"log_size":6146,"log_dups_size":0,"ondisk_log_size":6146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:48.963280+0000","scrub_duration":0.16207967100000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14946,"num_write_kb":478272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"199'81","reported_seq":376,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.195646+0000","last_change":"2024-09-11T08:58:34.136787+0000","last_active":"2024-09-11T09:00:30.195646+0000","last_peered":"2024-09-11T09:00:30.195646+0000","last_clean":"2024-09-11T09:00:30.195646+0000","last_became_active":"2024-09-11T08:58:34.134963+0000","last_became_peered":"2024-09-11T08:58:34.134963+0000","last_unstale":"2024-09-11T09:00:30.195646+0000","last_undegraded":"2024-09-11T09:00:30.195646+0000","last_fullsized":"2024-09-11T09:00:30.195646+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:57:45.946559+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:57:45.946559+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044386419999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"402'39","reported_seq":517,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.187202+0000","last_change":"2024-09-11T08:57:19.131847+0000","last_active":"2024-09-11T09:00:30.187202+0000","last_peered":"2024-09-11T09:00:30.187202+0000","last_clean":"2024-09-11T09:00:30.187202+0000","last_became_active":"2024-09-11T08:52:49.563364+0000","last_became_peered":"2024-09-11T08:52:49.563364+0000","last_unstale":"2024-09-11T09:00:30.187202+0000","last_undegraded":"2024-09-11T09:00:30.187202+0000","last_fullsized":"2024-09-11T09:00:30.187202+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'29","last_scrub_stamp":"2024-09-11T08:57:19.131782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:57:19.131782+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021894190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":68,"num_read_kb":54,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":478,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.207533+0000","last_change":"2024-09-11T08:58:33.985777+0000","last_active":"2024-09-11T09:00:30.207533+0000","last_peered":"2024-09-11T09:00:30.207533+0000","last_clean":"2024-09-11T09:00:30.207533+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T09:00:30.207533+0000","last_undegraded":"2024-09-11T09:00:30.207533+0000","last_fullsized":"2024-09-11T09:00:30.207533+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:58:33.985723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:58:33.985723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013827310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":432,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.186050+0000","last_change":"2024-09-11T08:59:35.556255+0000","last_active":"2024-09-11T09:00:30.186050+0000","last_peered":"2024-09-11T09:00:30.186050+0000","last_clean":"2024-09-11T09:00:30.186050+0000","last_became_active":"2024-09-11T08:58:34.130160+0000","last_became_peered":"2024-09-11T08:58:34.130160+0000","last_unstale":"2024-09-11T09:00:30.186050+0000","last_undegraded":"2024-09-11T09:00:30.186050+0000","last_fullsized":"2024-09-11T09:00:30.186050+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:35.556051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_clean_scrub_stamp":"2024-09-11T08:59:35.556051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.008463623,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":515,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.206344+0000","last_change":"2024-09-11T08:59:35.424456+0000","last_active":"2024-09-11T09:00:30.206344+0000","last_peered":"2024-09-11T09:00:30.206344+0000","last_clean":"2024-09-11T09:00:30.206344+0000","last_became_active":"2024-09-11T08:58:34.315290+0000","last_became_peered":"2024-09-11T08:58:34.315290+0000","last_unstale":"2024-09-11T09:00:30.206344+0000","last_undegraded":"2024-09-11T09:00:30.206344+0000","last_fullsized":"2024-09-11T09:00:30.206344+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:35.424382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:31.631810+0000","last_clean_scrub_stamp":"2024-09-11T08:59:35.424382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:01.798707+0000","scrub_duration":0.0076198460000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"425'14966","reported_seq":15034,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.432876+0000","last_change":"2024-09-11T08:57:14.485776+0000","last_active":"2024-09-11T09:00:34.432876+0000","last_peered":"2024-09-11T09:00:34.432876+0000","last_clean":"2024-09-11T09:00:34.432876+0000","last_became_active":"2024-09-11T08:57:14.485494+0000","last_became_peered":"2024-09-11T08:57:14.485494+0000","last_unstale":"2024-09-11T09:00:34.432876+0000","last_undegraded":"2024-09-11T09:00:34.432876+0000","last_fullsized":"2024-09-11T09:00:34.432876+0000","mapping_epoch":366,"log_start":"416'10892","ondisk_log_start":"416'10892","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":4074,"log_dups_size":0,"ondisk_log_size":4074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14966,"num_write_kb":478912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"199'81","reported_seq":235,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:32.381052+0000","last_change":"2024-09-11T09:00:32.380997+0000","last_active":"2024-09-11T09:00:32.381052+0000","last_peered":"2024-09-11T09:00:32.381052+0000","last_clean":"2024-09-11T09:00:32.381052+0000","last_became_active":"2024-09-11T08:58:01.120225+0000","last_became_peered":"2024-09-11T08:58:01.120225+0000","last_unstale":"2024-09-11T09:00:32.381052+0000","last_undegraded":"2024-09-11T09:00:32.381052+0000","last_fullsized":"2024-09-11T09:00:32.381052+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:00:32.380911+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:00:32.380911+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:57.002279+0000","scrub_duration":0.0032049819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"402'65","reported_seq":666,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:42.923297+0000","last_change":"2024-09-11T08:59:37.010134+0000","last_active":"2024-09-11T09:00:42.923297+0000","last_peered":"2024-09-11T09:00:42.923297+0000","last_clean":"2024-09-11T09:00:42.923297+0000","last_became_active":"2024-09-11T08:58:34.317818+0000","last_became_peered":"2024-09-11T08:58:34.317818+0000","last_unstale":"2024-09-11T09:00:42.923297+0000","last_undegraded":"2024-09-11T09:00:42.923297+0000","last_fullsized":"2024-09-11T09:00:42.923297+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'65","last_scrub_stamp":"2024-09-11T08:59:37.010051+0000","last_deep_scrub":"200'50","last_deep_scrub_stamp":"2024-09-11T08:53:09.745322+0000","last_clean_scrub_stamp":"2024-09-11T08:59:37.010051+0000","objects_scrubbed":10,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:46.224332+0000","scrub_duration":0.0076550569999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":489,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.190433+0000","last_change":"2024-09-11T08:59:37.883646+0000","last_active":"2024-09-11T09:00:30.190433+0000","last_peered":"2024-09-11T09:00:30.190433+0000","last_clean":"2024-09-11T09:00:30.190433+0000","last_became_active":"2024-09-11T08:58:34.136762+0000","last_became_peered":"2024-09-11T08:58:34.136762+0000","last_unstale":"2024-09-11T09:00:30.190433+0000","last_undegraded":"2024-09-11T09:00:30.190433+0000","last_fullsized":"2024-09-11T09:00:30.190433+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:37.883566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_clean_scrub_stamp":"2024-09-11T08:59:37.883566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:46.932540+0000","scrub_duration":0.0038426889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":517,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.186357+0000","last_change":"2024-09-11T08:58:34.160917+0000","last_active":"2024-09-11T09:00:30.186357+0000","last_peered":"2024-09-11T09:00:30.186357+0000","last_clean":"2024-09-11T09:00:30.186357+0000","last_became_active":"2024-09-11T08:58:34.138191+0000","last_became_peered":"2024-09-11T08:58:34.138191+0000","last_unstale":"2024-09-11T09:00:30.186357+0000","last_undegraded":"2024-09-11T09:00:30.186357+0000","last_fullsized":"2024-09-11T09:00:30.186357+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T08:57:40.077657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:57:40.077657+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.039451288000000001,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":277,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.208215+0000","last_change":"2024-09-11T08:59:37.431255+0000","last_active":"2024-09-11T09:00:30.208215+0000","last_peered":"2024-09-11T09:00:30.208215+0000","last_clean":"2024-09-11T09:00:30.208215+0000","last_became_active":"2024-09-11T08:52:49.569137+0000","last_became_peered":"2024-09-11T08:52:49.569137+0000","last_unstale":"2024-09-11T09:00:30.208215+0000","last_undegraded":"2024-09-11T09:00:30.208215+0000","last_fullsized":"2024-09-11T09:00:30.208215+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:37.431175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:59:37.431175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:49.002882+0000","scrub_duration":0.0057539649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":442,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T09:00:45.936286+0000","last_change":"2024-09-11T09:00:45.936261+0000","last_active":"2024-09-11T09:00:45.936286+0000","last_peered":"2024-09-11T09:00:45.936286+0000","last_clean":"2024-09-11T09:00:45.936286+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T09:00:45.936286+0000","last_undegraded":"2024-09-11T09:00:45.936286+0000","last_fullsized":"2024-09-11T09:00:45.936286+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:45.936206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:00:45.936206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:48.400735+0000","scrub_duration":0.0043292230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"425'15338","reported_seq":19694,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.448382+0000","last_change":"2024-09-11T08:58:35.132707+0000","last_active":"2024-09-11T09:00:34.448382+0000","last_peered":"2024-09-11T09:00:34.448382+0000","last_clean":"2024-09-11T09:00:34.448382+0000","last_became_active":"2024-09-11T08:58:35.129015+0000","last_became_peered":"2024-09-11T08:58:35.129015+0000","last_unstale":"2024-09-11T09:00:34.448382+0000","last_undegraded":"2024-09-11T09:00:34.448382+0000","last_fullsized":"2024-09-11T09:00:34.448382+0000","mapping_epoch":400,"log_start":"416'11282","ondisk_log_start":"416'11282","created":366,"last_epoch_clean":401,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15338,"num_write_kb":490816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2132,"num_bytes_recovered":139722752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"199'81","reported_seq":514,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:32.945666+0000","last_change":"2024-09-11T09:00:32.945604+0000","last_active":"2024-09-11T09:00:32.945666+0000","last_peered":"2024-09-11T09:00:32.945666+0000","last_clean":"2024-09-11T09:00:32.945666+0000","last_became_active":"2024-09-11T08:58:01.121000+0000","last_became_peered":"2024-09-11T08:58:01.121000+0000","last_unstale":"2024-09-11T09:00:32.945666+0000","last_undegraded":"2024-09-11T09:00:32.945666+0000","last_fullsized":"2024-09-11T09:00:32.945666+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:00:32.945519+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:57:49.622788+0000","last_clean_scrub_stamp":"2024-09-11T09:00:32.945519+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:58.536705+0000","scrub_duration":0.005091496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"402'21","reported_seq":350,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.186573+0000","last_change":"2024-09-11T08:58:51.834835+0000","last_active":"2024-09-11T09:00:30.186573+0000","last_peered":"2024-09-11T09:00:30.186573+0000","last_clean":"2024-09-11T09:00:30.186573+0000","last_became_active":"2024-09-11T08:56:25.459902+0000","last_became_peered":"2024-09-11T08:56:25.459902+0000","last_unstale":"2024-09-11T09:00:30.186573+0000","last_undegraded":"2024-09-11T09:00:30.186573+0000","last_fullsized":"2024-09-11T09:00:30.186573+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'21","last_scrub_stamp":"2024-09-11T08:58:51.834743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:58:51.834743+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026214160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":30,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":524,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:40.033512+0000","last_change":"2024-09-11T09:00:40.033466+0000","last_active":"2024-09-11T09:00:40.033512+0000","last_peered":"2024-09-11T09:00:40.033512+0000","last_clean":"2024-09-11T09:00:40.033512+0000","last_became_active":"2024-09-11T08:58:34.311119+0000","last_became_peered":"2024-09-11T08:58:34.311119+0000","last_unstale":"2024-09-11T09:00:40.033512+0000","last_undegraded":"2024-09-11T09:00:40.033512+0000","last_fullsized":"2024-09-11T09:00:40.033512+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:40.033409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:00:40.033409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:53.374284+0000","scrub_duration":0.0021261729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"263'2","reported_seq":549,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.203125+0000","last_change":"2024-09-11T08:59:43.021664+0000","last_active":"2024-09-11T09:00:30.203125+0000","last_peered":"2024-09-11T09:00:30.203125+0000","last_clean":"2024-09-11T09:00:30.203125+0000","last_became_active":"2024-09-11T08:52:18.586545+0000","last_became_peered":"2024-09-11T08:52:18.586545+0000","last_unstale":"2024-09-11T09:00:30.203125+0000","last_undegraded":"2024-09-11T09:00:30.203125+0000","last_fullsized":"2024-09-11T09:00:30.203125+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'2","last_scrub_stamp":"2024-09-11T08:59:43.021582+0000","last_deep_scrub":"263'2","last_deep_scrub_stamp":"2024-09-11T08:53:08.772979+0000","last_clean_scrub_stamp":"2024-09-11T08:59:43.021582+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:59.796734+0000","scrub_duration":0.024279913,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":509,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:37.024806+0000","last_change":"2024-09-11T09:00:37.024774+0000","last_active":"2024-09-11T09:00:37.024806+0000","last_peered":"2024-09-11T09:00:37.024806+0000","last_clean":"2024-09-11T09:00:37.024806+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T09:00:37.024806+0000","last_undegraded":"2024-09-11T09:00:37.024806+0000","last_fullsized":"2024-09-11T09:00:37.024806+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:37.024723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:58:50.901121+0000","last_clean_scrub_stamp":"2024-09-11T09:00:37.024723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:02.079734+0000","scrub_duration":0.0016047870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":292,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:42.990938+0000","last_change":"2024-09-11T09:00:42.990904+0000","last_active":"2024-09-11T09:00:42.990938+0000","last_peered":"2024-09-11T09:00:42.990938+0000","last_clean":"2024-09-11T09:00:42.990938+0000","last_became_active":"2024-09-11T08:58:34.135340+0000","last_became_peered":"2024-09-11T08:58:34.135340+0000","last_unstale":"2024-09-11T09:00:42.990938+0000","last_undegraded":"2024-09-11T09:00:42.990938+0000","last_fullsized":"2024-09-11T09:00:42.990938+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:42.990841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:00:42.990841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:04.365607+0000","scrub_duration":0.001380722,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"425'14894","reported_seq":14985,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.447560+0000","last_change":"2024-09-11T08:58:35.141925+0000","last_active":"2024-09-11T09:00:34.447560+0000","last_peered":"2024-09-11T09:00:34.447560+0000","last_clean":"2024-09-11T09:00:34.447560+0000","last_became_active":"2024-09-11T08:58:35.135801+0000","last_became_peered":"2024-09-11T08:58:35.135801+0000","last_unstale":"2024-09-11T09:00:34.447560+0000","last_undegraded":"2024-09-11T09:00:34.447560+0000","last_fullsized":"2024-09-11T09:00:34.447560+0000","mapping_epoch":400,"log_start":"416'10869","ondisk_log_start":"416'10869","created":366,"last_epoch_clean":401,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":4025,"log_dups_size":0,"ondisk_log_size":4025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14894,"num_write_kb":476608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"199'81","reported_seq":518,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.207061+0000","last_change":"2024-09-11T08:59:56.129414+0000","last_active":"2024-09-11T09:00:30.207061+0000","last_peered":"2024-09-11T09:00:30.207061+0000","last_clean":"2024-09-11T09:00:30.207061+0000","last_became_active":"2024-09-11T08:58:34.130607+0000","last_became_peered":"2024-09-11T08:58:34.130607+0000","last_unstale":"2024-09-11T09:00:30.207061+0000","last_undegraded":"2024-09-11T09:00:30.207061+0000","last_fullsized":"2024-09-11T09:00:30.207061+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:59:56.129324+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:59:56.129324+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:00.343202+0000","scrub_duration":0.0022328249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"402'36","reported_seq":250,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:31.599359+0000","last_change":"2024-09-11T09:00:31.598948+0000","last_active":"2024-09-11T09:00:31.599359+0000","last_peered":"2024-09-11T09:00:31.599359+0000","last_clean":"2024-09-11T09:00:31.599359+0000","last_became_active":"2024-09-11T08:52:18.580950+0000","last_became_peered":"2024-09-11T08:52:18.580950+0000","last_unstale":"2024-09-11T09:00:31.599359+0000","last_undegraded":"2024-09-11T09:00:31.599359+0000","last_fullsized":"2024-09-11T09:00:31.599359+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'36","last_scrub_stamp":"2024-09-11T09:00:31.598656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:00:31.598656+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:55.347477+0000","scrub_duration":0.0070966370000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":47,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":120,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.185651+0000","last_change":"2024-09-11T08:59:56.451380+0000","last_active":"2024-09-11T09:00:30.185651+0000","last_peered":"2024-09-11T09:00:30.185651+0000","last_clean":"2024-09-11T09:00:30.185651+0000","last_became_active":"2024-09-11T08:55:31.080178+0000","last_became_peered":"2024-09-11T08:55:31.080178+0000","last_unstale":"2024-09-11T09:00:30.185651+0000","last_undegraded":"2024-09-11T09:00:30.185651+0000","last_fullsized":"2024-09-11T09:00:30.185651+0000","mapping_epoch":335,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":336,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:56.451306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:59:56.451306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:12.388075+0000","scrub_duration":0.0026035519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"318'6","reported_seq":361,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T09:00:45.564258+0000","last_change":"2024-09-11T08:59:39.758650+0000","last_active":"2024-09-11T09:00:45.564258+0000","last_peered":"2024-09-11T09:00:45.564258+0000","last_clean":"2024-09-11T09:00:45.564258+0000","last_became_active":"2024-09-11T08:56:25.462503+0000","last_became_peered":"2024-09-11T08:56:25.462503+0000","last_unstale":"2024-09-11T09:00:45.564258+0000","last_undegraded":"2024-09-11T09:00:45.564258+0000","last_fullsized":"2024-09-11T09:00:45.564258+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'6","last_scrub_stamp":"2024-09-11T08:59:39.758574+0000","last_deep_scrub":"318'6","last_deep_scrub_stamp":"2024-09-11T08:54:12.873653+0000","last_clean_scrub_stamp":"2024-09-11T08:59:39.758574+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:02.780178+0000","scrub_duration":0.026258813999999998,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":355,"num_read_kb":350,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":470,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.187127+0000","last_change":"2024-09-11T08:57:49.964787+0000","last_active":"2024-09-11T09:00:30.187127+0000","last_peered":"2024-09-11T09:00:30.187127+0000","last_clean":"2024-09-11T09:00:30.187127+0000","last_became_active":"2024-09-11T08:52:18.578530+0000","last_became_peered":"2024-09-11T08:52:18.578530+0000","last_unstale":"2024-09-11T09:00:30.187127+0000","last_undegraded":"2024-09-11T09:00:30.187127+0000","last_fullsized":"2024-09-11T09:00:30.187127+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:49.964601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:16.367873+0000","last_clean_scrub_stamp":"2024-09-11T08:57:49.964601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028402520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":123,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.186123+0000","last_change":"2024-09-11T08:59:37.478313+0000","last_active":"2024-09-11T09:00:30.186123+0000","last_peered":"2024-09-11T09:00:30.186123+0000","last_clean":"2024-09-11T09:00:30.186123+0000","last_became_active":"2024-09-11T08:58:34.139181+0000","last_became_peered":"2024-09-11T08:58:34.139181+0000","last_unstale":"2024-09-11T09:00:30.186123+0000","last_undegraded":"2024-09-11T09:00:30.186123+0000","last_fullsized":"2024-09-11T09:00:30.186123+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:37.478243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T08:59:37.478243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:06.027555+0000","scrub_duration":0.0048852050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"425'15344","reported_seq":18680,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.418384+0000","last_change":"2024-09-11T08:58:35.134759+0000","last_active":"2024-09-11T09:00:34.418384+0000","last_peered":"2024-09-11T09:00:34.418384+0000","last_clean":"2024-09-11T09:00:34.418384+0000","last_became_active":"2024-09-11T08:58:35.129003+0000","last_became_peered":"2024-09-11T08:58:35.129003+0000","last_unstale":"2024-09-11T09:00:34.418384+0000","last_undegraded":"2024-09-11T09:00:34.418384+0000","last_fullsized":"2024-09-11T09:00:34.418384+0000","mapping_epoch":400,"log_start":"416'11331","ondisk_log_start":"416'11331","created":366,"last_epoch_clean":401,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":4013,"log_dups_size":0,"ondisk_log_size":4013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15344,"num_write_kb":491008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1621,"num_bytes_recovered":106233856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"402'39","reported_seq":264,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189976+0000","last_change":"2024-09-11T08:59:50.940860+0000","last_active":"2024-09-11T09:00:30.189976+0000","last_peered":"2024-09-11T09:00:30.189976+0000","last_clean":"2024-09-11T09:00:30.189976+0000","last_became_active":"2024-09-11T08:58:34.136011+0000","last_became_peered":"2024-09-11T08:58:34.136011+0000","last_unstale":"2024-09-11T09:00:30.189976+0000","last_undegraded":"2024-09-11T09:00:30.189976+0000","last_fullsized":"2024-09-11T09:00:30.189976+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'39","last_scrub_stamp":"2024-09-11T08:59:50.940760+0000","last_deep_scrub":"201'27","last_deep_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_clean_scrub_stamp":"2024-09-11T08:59:50.940760+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:02.926447+0000","scrub_duration":0.0086383280000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":50,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":475,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.195820+0000","last_change":"2024-09-11T08:58:34.158996+0000","last_active":"2024-09-11T09:00:30.195820+0000","last_peered":"2024-09-11T09:00:30.195820+0000","last_clean":"2024-09-11T09:00:30.195820+0000","last_became_active":"2024-09-11T08:58:34.135492+0000","last_became_peered":"2024-09-11T08:58:34.135492+0000","last_unstale":"2024-09-11T09:00:30.195820+0000","last_undegraded":"2024-09-11T09:00:30.195820+0000","last_fullsized":"2024-09-11T09:00:30.195820+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:56:50.853747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:56:50.853747+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018093129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1748,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"199'81","reported_seq":512,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.191469+0000","last_change":"2024-09-11T08:59:54.346142+0000","last_active":"2024-09-11T09:00:30.191469+0000","last_peered":"2024-09-11T09:00:30.191469+0000","last_clean":"2024-09-11T09:00:30.191469+0000","last_became_active":"2024-09-11T08:58:34.135132+0000","last_became_peered":"2024-09-11T08:58:34.135132+0000","last_unstale":"2024-09-11T09:00:30.191469+0000","last_undegraded":"2024-09-11T09:00:30.191469+0000","last_fullsized":"2024-09-11T09:00:30.191469+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:59:54.346071+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:55:08.518651+0000","last_clean_scrub_stamp":"2024-09-11T08:59:54.346071+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:13.850687+0000","scrub_duration":0.0063267250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":201,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.203843+0000","last_change":"2024-09-11T08:59:58.895303+0000","last_active":"2024-09-11T09:00:30.203843+0000","last_peered":"2024-09-11T09:00:30.203843+0000","last_clean":"2024-09-11T09:00:30.203843+0000","last_became_active":"2024-09-11T08:52:38.485446+0000","last_became_peered":"2024-09-11T08:52:38.485446+0000","last_unstale":"2024-09-11T09:00:30.203843+0000","last_undegraded":"2024-09-11T09:00:30.203843+0000","last_fullsized":"2024-09-11T09:00:30.203843+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:58.895153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:59:58.895153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:10.425611+0000","scrub_duration":0.005864256,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":448,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.186735+0000","last_change":"2024-09-11T08:57:27.101488+0000","last_active":"2024-09-11T09:00:30.186735+0000","last_peered":"2024-09-11T09:00:30.186735+0000","last_clean":"2024-09-11T09:00:30.186735+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T09:00:30.186735+0000","last_undegraded":"2024-09-11T09:00:30.186735+0000","last_fullsized":"2024-09-11T09:00:30.186735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:27.101421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:57:27.101421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014808950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"199'81","reported_seq":507,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.206374+0000","last_change":"2024-09-11T08:58:41.961086+0000","last_active":"2024-09-11T09:00:30.206374+0000","last_peered":"2024-09-11T09:00:30.206374+0000","last_clean":"2024-09-11T09:00:30.206374+0000","last_became_active":"2024-09-11T08:58:03.122171+0000","last_became_peered":"2024-09-11T08:58:03.122171+0000","last_unstale":"2024-09-11T09:00:30.206374+0000","last_undegraded":"2024-09-11T09:00:30.206374+0000","last_fullsized":"2024-09-11T09:00:30.206374+0000","mapping_epoch":385,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":386,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:58:41.960976+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:58:41.960976+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052535719999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"425'15158","reported_seq":18818,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.443752+0000","last_change":"2024-09-11T08:58:35.136329+0000","last_active":"2024-09-11T09:00:34.443752+0000","last_peered":"2024-09-11T09:00:34.443752+0000","last_clean":"2024-09-11T09:00:34.443752+0000","last_became_active":"2024-09-11T08:58:35.129642+0000","last_became_peered":"2024-09-11T08:58:35.129642+0000","last_unstale":"2024-09-11T09:00:34.443752+0000","last_undegraded":"2024-09-11T09:00:34.443752+0000","last_fullsized":"2024-09-11T09:00:34.443752+0000","mapping_epoch":400,"log_start":"416'11060","ondisk_log_start":"416'11060","created":366,"last_epoch_clean":401,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":4098,"log_dups_size":0,"ondisk_log_size":4098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15158,"num_write_kb":485056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1782,"num_bytes_recovered":116785152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"402'27","reported_seq":615,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189969+0000","last_change":"2024-09-11T08:59:32.783481+0000","last_active":"2024-09-11T09:00:30.189969+0000","last_peered":"2024-09-11T09:00:30.189969+0000","last_clean":"2024-09-11T09:00:30.189969+0000","last_became_active":"2024-09-11T08:58:34.127518+0000","last_became_peered":"2024-09-11T08:58:34.127518+0000","last_unstale":"2024-09-11T09:00:30.189969+0000","last_undegraded":"2024-09-11T09:00:30.189969+0000","last_fullsized":"2024-09-11T09:00:30.189969+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'27","last_scrub_stamp":"2024-09-11T08:59:32.783420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:59:32.783420+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029123970000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":439,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189209+0000","last_change":"2024-09-11T08:59:43.797565+0000","last_active":"2024-09-11T09:00:30.189209+0000","last_peered":"2024-09-11T09:00:30.189209+0000","last_clean":"2024-09-11T09:00:30.189209+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T09:00:30.189209+0000","last_undegraded":"2024-09-11T09:00:30.189209+0000","last_fullsized":"2024-09-11T09:00:30.189209+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:43.797507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:54.671832+0000","last_clean_scrub_stamp":"2024-09-11T08:59:43.797507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:00.898356+0000","scrub_duration":0.0053197610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":480,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.186497+0000","last_change":"2024-09-11T08:59:44.379807+0000","last_active":"2024-09-11T09:00:30.186497+0000","last_peered":"2024-09-11T09:00:30.186497+0000","last_clean":"2024-09-11T09:00:30.186497+0000","last_became_active":"2024-09-11T08:58:34.135221+0000","last_became_peered":"2024-09-11T08:58:34.135221+0000","last_unstale":"2024-09-11T09:00:30.186497+0000","last_undegraded":"2024-09-11T09:00:30.186497+0000","last_fullsized":"2024-09-11T09:00:30.186497+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:44.379743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T08:59:44.379743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:54.178632+0000","scrub_duration":0.0034415499999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":482,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:44.918108+0000","last_change":"2024-09-11T09:00:44.918043+0000","last_active":"2024-09-11T09:00:44.918108+0000","last_peered":"2024-09-11T09:00:44.918108+0000","last_clean":"2024-09-11T09:00:44.918108+0000","last_became_active":"2024-09-11T08:58:34.135447+0000","last_became_peered":"2024-09-11T08:58:34.135447+0000","last_unstale":"2024-09-11T09:00:44.918108+0000","last_undegraded":"2024-09-11T09:00:44.918108+0000","last_fullsized":"2024-09-11T09:00:44.918108+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:44.917980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:00:44.917980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:07.792196+0000","scrub_duration":0.001425852,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"425'15136","reported_seq":15204,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.448174+0000","last_change":"2024-09-11T08:57:14.487415+0000","last_active":"2024-09-11T09:00:34.448174+0000","last_peered":"2024-09-11T09:00:34.448174+0000","last_clean":"2024-09-11T09:00:34.448174+0000","last_became_active":"2024-09-11T08:57:14.487157+0000","last_became_peered":"2024-09-11T08:57:14.487157+0000","last_unstale":"2024-09-11T09:00:34.448174+0000","last_undegraded":"2024-09-11T09:00:34.448174+0000","last_fullsized":"2024-09-11T09:00:34.448174+0000","mapping_epoch":366,"log_start":"416'11049","ondisk_log_start":"416'11049","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":4087,"log_dups_size":0,"ondisk_log_size":4087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15136,"num_write_kb":484352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"199'81","reported_seq":222,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.185467+0000","last_change":"2024-09-11T08:58:48.849757+0000","last_active":"2024-09-11T09:00:30.185467+0000","last_peered":"2024-09-11T09:00:30.185467+0000","last_clean":"2024-09-11T09:00:30.185467+0000","last_became_active":"2024-09-11T08:51:03.020941+0000","last_became_peered":"2024-09-11T08:51:03.020941+0000","last_unstale":"2024-09-11T09:00:30.185467+0000","last_undegraded":"2024-09-11T09:00:30.185467+0000","last_fullsized":"2024-09-11T09:00:30.185467+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:58:48.849665+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:53:14.809894+0000","last_clean_scrub_stamp":"2024-09-11T08:58:48.849665+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001934304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"402'53","reported_seq":525,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.190310+0000","last_change":"2024-09-11T08:57:33.510922+0000","last_active":"2024-09-11T09:00:30.190310+0000","last_peered":"2024-09-11T09:00:30.190310+0000","last_clean":"2024-09-11T09:00:30.190310+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T09:00:30.190310+0000","last_undegraded":"2024-09-11T09:00:30.190310+0000","last_fullsized":"2024-09-11T09:00:30.190310+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"317'39","last_scrub_stamp":"2024-09-11T08:57:33.510860+0000","last_deep_scrub":"317'39","last_deep_scrub_stamp":"2024-09-11T08:54:06.928266+0000","last_clean_scrub_stamp":"2024-09-11T08:57:33.510860+0000","objects_scrubbed":8,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036910879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":71,"num_read_kb":61,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":133,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189551+0000","last_change":"2024-09-11T08:59:39.837079+0000","last_active":"2024-09-11T09:00:30.189551+0000","last_peered":"2024-09-11T09:00:30.189551+0000","last_clean":"2024-09-11T09:00:30.189551+0000","last_became_active":"2024-09-11T08:58:34.125891+0000","last_became_peered":"2024-09-11T08:58:34.125891+0000","last_unstale":"2024-09-11T09:00:30.189551+0000","last_undegraded":"2024-09-11T09:00:30.189551+0000","last_fullsized":"2024-09-11T09:00:30.189551+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:39.836999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:07.504031+0000","last_clean_scrub_stamp":"2024-09-11T08:59:39.836999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:59.127435+0000","scrub_duration":0.0078184029999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":487,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.207065+0000","last_change":"2024-09-11T08:59:34.400338+0000","last_active":"2024-09-11T09:00:30.207065+0000","last_peered":"2024-09-11T09:00:30.207065+0000","last_clean":"2024-09-11T09:00:30.207065+0000","last_became_active":"2024-09-11T08:52:18.577201+0000","last_became_peered":"2024-09-11T08:52:18.577201+0000","last_unstale":"2024-09-11T09:00:30.207065+0000","last_undegraded":"2024-09-11T09:00:30.207065+0000","last_fullsized":"2024-09-11T09:00:30.207065+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:34.400154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:59:34.400154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033943699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":139,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:32.055790+0000","last_change":"2024-09-11T09:00:32.055720+0000","last_active":"2024-09-11T09:00:32.055790+0000","last_peered":"2024-09-11T09:00:32.055790+0000","last_clean":"2024-09-11T09:00:32.055790+0000","last_became_active":"2024-09-11T08:54:17.121372+0000","last_became_peered":"2024-09-11T08:54:17.121372+0000","last_unstale":"2024-09-11T09:00:32.055790+0000","last_undegraded":"2024-09-11T09:00:32.055790+0000","last_fullsized":"2024-09-11T09:00:32.055790+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:32.055635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:00:32.055635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:47.408419+0000","scrub_duration":0.0039920800000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3","version":"425'15174","reported_seq":22202,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:44.499394+0000","last_change":"2024-09-11T09:00:44.499329+0000","last_active":"2024-09-11T09:00:44.499394+0000","last_peered":"2024-09-11T09:00:44.499394+0000","last_clean":"2024-09-11T09:00:44.499394+0000","last_became_active":"2024-09-11T08:58:35.115965+0000","last_became_peered":"2024-09-11T08:58:35.115965+0000","last_unstale":"2024-09-11T09:00:44.499394+0000","last_undegraded":"2024-09-11T09:00:44.499394+0000","last_fullsized":"2024-09-11T09:00:44.499394+0000","mapping_epoch":400,"log_start":"415'10514","ondisk_log_start":"415'10514","created":366,"last_epoch_clean":401,"parent":"0.0","parent_split_bits":0,"last_scrub":"425'15174","last_scrub_stamp":"2024-09-11T09:00:44.499262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T09:00:44.499262+0000","objects_scrubbed":0,"log_size":4660,"log_dups_size":0,"ondisk_log_size":4660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:13.872329+0000","scrub_duration":0.0086068329999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15174,"num_write_kb":485568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3461,"num_bytes_recovered":226820096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"199'81","reported_seq":498,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.194585+0000","last_change":"2024-09-11T08:57:50.853394+0000","last_active":"2024-09-11T09:00:30.194585+0000","last_peered":"2024-09-11T09:00:30.194585+0000","last_clean":"2024-09-11T09:00:30.194585+0000","last_became_active":"2024-09-11T08:52:38.481475+0000","last_became_peered":"2024-09-11T08:52:38.481475+0000","last_unstale":"2024-09-11T09:00:30.194585+0000","last_undegraded":"2024-09-11T09:00:30.194585+0000","last_fullsized":"2024-09-11T09:00:30.194585+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:57:50.853055+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:57:50.853055+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0098307800000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"402'30","reported_seq":576,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.190540+0000","last_change":"2024-09-11T08:59:48.900913+0000","last_active":"2024-09-11T09:00:30.190540+0000","last_peered":"2024-09-11T09:00:30.190540+0000","last_clean":"2024-09-11T09:00:30.190540+0000","last_became_active":"2024-09-11T08:58:34.136623+0000","last_became_peered":"2024-09-11T08:58:34.136623+0000","last_unstale":"2024-09-11T09:00:30.190540+0000","last_undegraded":"2024-09-11T09:00:30.190540+0000","last_fullsized":"2024-09-11T09:00:30.190540+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'30","last_scrub_stamp":"2024-09-11T08:59:48.900760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:59:48.900760+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:02.180475+0000","scrub_duration":0.0062324750000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":468,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.196664+0000","last_change":"2024-09-11T08:59:51.772473+0000","last_active":"2024-09-11T09:00:30.196664+0000","last_peered":"2024-09-11T09:00:30.196664+0000","last_clean":"2024-09-11T09:00:30.196664+0000","last_became_active":"2024-09-11T08:52:18.586343+0000","last_became_peered":"2024-09-11T08:52:18.586343+0000","last_unstale":"2024-09-11T09:00:30.196664+0000","last_undegraded":"2024-09-11T09:00:30.196664+0000","last_fullsized":"2024-09-11T09:00:30.196664+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:51.772391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:59:51.772391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:08.733038+0000","scrub_duration":0.012525537,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"299'9","reported_seq":780,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:42.923274+0000","last_change":"2024-09-11T09:00:39.007959+0000","last_active":"2024-09-11T09:00:42.923274+0000","last_peered":"2024-09-11T09:00:42.923274+0000","last_clean":"2024-09-11T09:00:42.923274+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T09:00:42.923274+0000","last_undegraded":"2024-09-11T09:00:42.923274+0000","last_fullsized":"2024-09-11T09:00:42.923274+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'9","last_scrub_stamp":"2024-09-11T09:00:39.007901+0000","last_deep_scrub":"299'9","last_deep_scrub_stamp":"2024-09-11T08:53:05.835177+0000","last_clean_scrub_stamp":"2024-09-11T09:00:39.007901+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:08.824010+0000","scrub_duration":0.0022187919999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":453,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189725+0000","last_change":"2024-09-11T09:00:02.102856+0000","last_active":"2024-09-11T09:00:30.189725+0000","last_peered":"2024-09-11T09:00:30.189725+0000","last_clean":"2024-09-11T09:00:30.189725+0000","last_became_active":"2024-09-11T08:56:25.457566+0000","last_became_peered":"2024-09-11T08:56:25.457566+0000","last_unstale":"2024-09-11T09:00:30.189725+0000","last_undegraded":"2024-09-11T09:00:30.189725+0000","last_fullsized":"2024-09-11T09:00:30.189725+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:02.102619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T09:00:02.102619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:13.645575+0000","scrub_duration":0.014453644999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"199'81","reported_seq":50,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189638+0000","last_change":"2024-09-11T08:58:03.137874+0000","last_active":"2024-09-11T09:00:30.189638+0000","last_peered":"2024-09-11T09:00:30.189638+0000","last_clean":"2024-09-11T09:00:30.189638+0000","last_became_active":"2024-09-11T08:58:03.137484+0000","last_became_peered":"2024-09-11T08:58:03.137484+0000","last_unstale":"2024-09-11T09:00:30.189638+0000","last_undegraded":"2024-09-11T09:00:30.189638+0000","last_fullsized":"2024-09-11T09:00:30.189638+0000","mapping_epoch":385,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":386,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:57:45.946559+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:57:45.946559+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025551829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"425'15302","reported_seq":20847,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.424066+0000","last_change":"2024-09-11T08:59:45.210539+0000","last_active":"2024-09-11T09:00:34.424066+0000","last_peered":"2024-09-11T09:00:34.424066+0000","last_clean":"2024-09-11T09:00:34.424066+0000","last_became_active":"2024-09-11T08:58:35.117318+0000","last_became_peered":"2024-09-11T08:58:35.117318+0000","last_unstale":"2024-09-11T09:00:34.424066+0000","last_undegraded":"2024-09-11T09:00:34.424066+0000","last_fullsized":"2024-09-11T09:00:34.424066+0000","mapping_epoch":400,"log_start":"416'10930","ondisk_log_start":"416'10930","created":366,"last_epoch_clean":401,"parent":"0.0","parent_split_bits":0,"last_scrub":"416'11386","last_scrub_stamp":"2024-09-11T08:59:45.210447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:59:45.210447+0000","objects_scrubbed":3973,"log_size":4372,"log_dups_size":0,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:53.218463+0000","scrub_duration":1.74350603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15302,"num_write_kb":489664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2642,"num_bytes_recovered":173146112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"402'21","reported_seq":230,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.190005+0000","last_change":"2024-09-11T08:59:31.811393+0000","last_active":"2024-09-11T09:00:30.190005+0000","last_peered":"2024-09-11T09:00:30.190005+0000","last_clean":"2024-09-11T09:00:30.190005+0000","last_became_active":"2024-09-11T08:52:18.588329+0000","last_became_peered":"2024-09-11T08:52:18.588329+0000","last_unstale":"2024-09-11T09:00:30.190005+0000","last_undegraded":"2024-09-11T09:00:30.190005+0000","last_fullsized":"2024-09-11T09:00:30.190005+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'21","last_scrub_stamp":"2024-09-11T08:59:31.811231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:59:31.811231+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:49.260388+0000","scrub_duration":0.0050544370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":481,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.186656+0000","last_change":"2024-09-11T08:59:59.434653+0000","last_active":"2024-09-11T09:00:30.186656+0000","last_peered":"2024-09-11T09:00:30.186656+0000","last_clean":"2024-09-11T09:00:30.186656+0000","last_became_active":"2024-09-11T08:58:34.139430+0000","last_became_peered":"2024-09-11T08:58:34.139430+0000","last_unstale":"2024-09-11T09:00:30.186656+0000","last_undegraded":"2024-09-11T09:00:30.186656+0000","last_fullsized":"2024-09-11T09:00:30.186656+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:59.434593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T08:59:59.434593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:26.917005+0000","scrub_duration":0.0094122749999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":129,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189345+0000","last_change":"2024-09-11T08:57:40.577143+0000","last_active":"2024-09-11T09:00:30.189345+0000","last_peered":"2024-09-11T09:00:30.189345+0000","last_clean":"2024-09-11T09:00:30.189345+0000","last_became_active":"2024-09-11T08:54:17.119864+0000","last_became_peered":"2024-09-11T08:54:17.119864+0000","last_unstale":"2024-09-11T09:00:30.189345+0000","last_undegraded":"2024-09-11T09:00:30.189345+0000","last_fullsized":"2024-09-11T09:00:30.189345+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:40.577018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:57:40.577018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066650809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":201,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.202975+0000","last_change":"2024-09-11T08:59:54.048266+0000","last_active":"2024-09-11T09:00:30.202975+0000","last_peered":"2024-09-11T09:00:30.202975+0000","last_clean":"2024-09-11T09:00:30.202975+0000","last_became_active":"2024-09-11T08:52:49.569749+0000","last_became_peered":"2024-09-11T08:52:49.569749+0000","last_unstale":"2024-09-11T09:00:30.202975+0000","last_undegraded":"2024-09-11T09:00:30.202975+0000","last_fullsized":"2024-09-11T09:00:30.202975+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:54.048191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:13.770061+0000","last_clean_scrub_stamp":"2024-09-11T08:59:54.048191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:56.829396+0000","scrub_duration":0.0072192979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.f","version":"199'81","reported_seq":70,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.186649+0000","last_change":"2024-09-11T08:58:34.140231+0000","last_active":"2024-09-11T09:00:30.186649+0000","last_peered":"2024-09-11T09:00:30.186649+0000","last_clean":"2024-09-11T09:00:30.186649+0000","last_became_active":"2024-09-11T08:58:34.139067+0000","last_became_peered":"2024-09-11T08:58:34.139067+0000","last_unstale":"2024-09-11T09:00:30.186649+0000","last_undegraded":"2024-09-11T09:00:30.186649+0000","last_fullsized":"2024-09-11T09:00:30.186649+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:57:41.844437+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:57:41.844437+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0081335720000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.5","version":"425'15244","reported_seq":21974,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:45.526030+0000","last_change":"2024-09-11T09:00:45.525978+0000","last_active":"2024-09-11T09:00:45.526030+0000","last_peered":"2024-09-11T09:00:45.526030+0000","last_clean":"2024-09-11T09:00:45.526030+0000","last_became_active":"2024-09-11T08:58:34.141573+0000","last_became_peered":"2024-09-11T08:58:34.141573+0000","last_unstale":"2024-09-11T09:00:45.526030+0000","last_undegraded":"2024-09-11T09:00:45.526030+0000","last_fullsized":"2024-09-11T09:00:45.526030+0000","mapping_epoch":399,"log_start":"415'10597","ondisk_log_start":"415'10597","created":366,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"425'15244","last_scrub_stamp":"2024-09-11T09:00:45.525888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T09:00:45.525888+0000","objects_scrubbed":0,"log_size":4647,"log_dups_size":0,"ondisk_log_size":4647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:46.965764+0000","scrub_duration":0.0084139550000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15244,"num_write_kb":487808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3314,"num_bytes_recovered":217186304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"402'60","reported_seq":300,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.079048+0000","last_change":"2024-09-11T09:00:34.078961+0000","last_active":"2024-09-11T09:00:34.079048+0000","last_peered":"2024-09-11T09:00:34.079048+0000","last_clean":"2024-09-11T09:00:34.079048+0000","last_became_active":"2024-09-11T08:58:34.126635+0000","last_became_peered":"2024-09-11T08:58:34.126635+0000","last_unstale":"2024-09-11T09:00:34.079048+0000","last_undegraded":"2024-09-11T09:00:34.079048+0000","last_fullsized":"2024-09-11T09:00:34.079048+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'60","last_scrub_stamp":"2024-09-11T09:00:34.078883+0000","last_deep_scrub":"374'44","last_deep_scrub_stamp":"2024-09-11T08:57:34.896912+0000","last_clean_scrub_stamp":"2024-09-11T09:00:34.078883+0000","objects_scrubbed":9,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:51.410010+0000","scrub_duration":0.0057466679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":78,"num_read_kb":67,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":202,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:43.442249+0000","last_change":"2024-09-11T09:00:43.442213+0000","last_active":"2024-09-11T09:00:43.442249+0000","last_peered":"2024-09-11T09:00:43.442249+0000","last_clean":"2024-09-11T09:00:43.442249+0000","last_became_active":"2024-09-11T08:58:34.152326+0000","last_became_peered":"2024-09-11T08:58:34.152326+0000","last_unstale":"2024-09-11T09:00:43.442249+0000","last_undegraded":"2024-09-11T09:00:43.442249+0000","last_fullsized":"2024-09-11T09:00:43.442249+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:43.442157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:00:43.442157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:58.967046+0000","scrub_duration":0.0012200539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":516,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.203259+0000","last_change":"2024-09-11T08:59:36.028846+0000","last_active":"2024-09-11T09:00:30.203259+0000","last_peered":"2024-09-11T09:00:30.203259+0000","last_clean":"2024-09-11T09:00:30.203259+0000","last_became_active":"2024-09-11T08:52:18.584748+0000","last_became_peered":"2024-09-11T08:52:18.584748+0000","last_unstale":"2024-09-11T09:00:30.203259+0000","last_undegraded":"2024-09-11T09:00:30.203259+0000","last_fullsized":"2024-09-11T09:00:30.203259+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:36.028751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:56:46.744542+0000","last_clean_scrub_stamp":"2024-09-11T08:59:36.028751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:50.601973+0000","scrub_duration":0.0039260609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":133,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.197616+0000","last_change":"2024-09-11T08:57:40.804069+0000","last_active":"2024-09-11T09:00:30.197616+0000","last_peered":"2024-09-11T09:00:30.197616+0000","last_clean":"2024-09-11T09:00:30.197616+0000","last_became_active":"2024-09-11T08:54:17.155625+0000","last_became_peered":"2024-09-11T08:54:17.155625+0000","last_unstale":"2024-09-11T09:00:30.197616+0000","last_undegraded":"2024-09-11T09:00:30.197616+0000","last_fullsized":"2024-09-11T09:00:30.197616+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:40.803996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:57:40.803996+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073961749999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.c","version":"199'81","reported_seq":263,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.194648+0000","last_change":"2024-09-11T08:58:34.158711+0000","last_active":"2024-09-11T09:00:30.194648+0000","last_peered":"2024-09-11T09:00:30.194648+0000","last_clean":"2024-09-11T09:00:30.194648+0000","last_became_active":"2024-09-11T08:58:34.135770+0000","last_became_peered":"2024-09-11T08:58:34.135770+0000","last_unstale":"2024-09-11T09:00:30.194648+0000","last_undegraded":"2024-09-11T09:00:30.194648+0000","last_fullsized":"2024-09-11T09:00:30.194648+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:55:53.823609+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:55:53.823609+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052441400000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109227,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"425'15266","reported_seq":15514,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.444970+0000","last_change":"2024-09-11T08:59:41.953820+0000","last_active":"2024-09-11T09:00:34.444970+0000","last_peered":"2024-09-11T09:00:34.444970+0000","last_clean":"2024-09-11T09:00:34.444970+0000","last_became_active":"2024-09-11T08:57:14.485531+0000","last_became_peered":"2024-09-11T08:57:14.485531+0000","last_unstale":"2024-09-11T09:00:34.444970+0000","last_undegraded":"2024-09-11T09:00:34.444970+0000","last_fullsized":"2024-09-11T09:00:34.444970+0000","mapping_epoch":366,"log_start":"412'9100","ondisk_log_start":"412'9100","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"416'11138","last_scrub_stamp":"2024-09-11T08:59:41.953669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:59:41.953669+0000","objects_scrubbed":4224,"log_size":6166,"log_dups_size":0,"ondisk_log_size":6166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:57.476989+0000","scrub_duration":2.552648569,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15266,"num_write_kb":488512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"402'29","reported_seq":483,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189065+0000","last_change":"2024-09-11T08:57:19.603379+0000","last_active":"2024-09-11T09:00:30.189065+0000","last_peered":"2024-09-11T09:00:30.189065+0000","last_clean":"2024-09-11T09:00:30.189065+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T09:00:30.189065+0000","last_undegraded":"2024-09-11T09:00:30.189065+0000","last_fullsized":"2024-09-11T09:00:30.189065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"302'23","last_scrub_stamp":"2024-09-11T08:57:19.603310+0000","last_deep_scrub":"302'23","last_deep_scrub_stamp":"2024-09-11T08:53:16.896643+0000","last_clean_scrub_stamp":"2024-09-11T08:57:19.603310+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023914409999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":172,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.190221+0000","last_change":"2024-09-11T08:59:34.863455+0000","last_active":"2024-09-11T09:00:30.190221+0000","last_peered":"2024-09-11T09:00:30.190221+0000","last_clean":"2024-09-11T09:00:30.190221+0000","last_became_active":"2024-09-11T08:52:38.487881+0000","last_became_peered":"2024-09-11T08:52:38.487881+0000","last_unstale":"2024-09-11T09:00:30.190221+0000","last_undegraded":"2024-09-11T09:00:30.190221+0000","last_fullsized":"2024-09-11T09:00:30.190221+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:34.863401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:24.331966+0000","last_clean_scrub_stamp":"2024-09-11T08:59:34.863401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:59.865565+0000","scrub_duration":0.0091876590000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":461,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.206995+0000","last_change":"2024-09-11T08:59:33.370613+0000","last_active":"2024-09-11T09:00:30.206995+0000","last_peered":"2024-09-11T09:00:30.206995+0000","last_clean":"2024-09-11T09:00:30.206995+0000","last_became_active":"2024-09-11T08:54:17.122020+0000","last_became_peered":"2024-09-11T08:54:17.122020+0000","last_unstale":"2024-09-11T09:00:30.206995+0000","last_undegraded":"2024-09-11T09:00:30.206995+0000","last_fullsized":"2024-09-11T09:00:30.206995+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:33.370427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:59:33.370427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:58.470953+0000","scrub_duration":0.010864649000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":432,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.196352+0000","last_change":"2024-09-11T08:56:51.873475+0000","last_active":"2024-09-11T09:00:30.196352+0000","last_peered":"2024-09-11T09:00:30.196352+0000","last_clean":"2024-09-11T09:00:30.196352+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T09:00:30.196352+0000","last_undegraded":"2024-09-11T09:00:30.196352+0000","last_fullsized":"2024-09-11T09:00:30.196352+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:51.873420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T08:56:51.873420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001764933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"199'81","reported_seq":234,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:36.856755+0000","last_change":"2024-09-11T09:00:36.856680+0000","last_active":"2024-09-11T09:00:36.856755+0000","last_peered":"2024-09-11T09:00:36.856755+0000","last_clean":"2024-09-11T09:00:36.856755+0000","last_became_active":"2024-09-11T08:58:03.124503+0000","last_became_peered":"2024-09-11T08:58:03.124503+0000","last_unstale":"2024-09-11T09:00:36.856755+0000","last_undegraded":"2024-09-11T09:00:36.856755+0000","last_fullsized":"2024-09-11T09:00:36.856755+0000","mapping_epoch":385,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":386,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:00:36.856576+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:00:36.856576+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:58.969813+0000","scrub_duration":0.001547697,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"425'15298","reported_seq":15366,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.446727+0000","last_change":"2024-09-11T08:57:14.474857+0000","last_active":"2024-09-11T09:00:34.446727+0000","last_peered":"2024-09-11T09:00:34.446727+0000","last_clean":"2024-09-11T09:00:34.446727+0000","last_became_active":"2024-09-11T08:57:14.474567+0000","last_became_peered":"2024-09-11T08:57:14.474567+0000","last_unstale":"2024-09-11T09:00:34.446727+0000","last_undegraded":"2024-09-11T09:00:34.446727+0000","last_fullsized":"2024-09-11T09:00:34.446727+0000","mapping_epoch":366,"log_start":"415'10590","ondisk_log_start":"415'10590","created":366,"last_epoch_clean":367,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:13.451519+0000","last_clean_scrub_stamp":"2024-09-11T08:57:13.451519+0000","objects_scrubbed":0,"log_size":4708,"log_dups_size":0,"ondisk_log_size":4708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15298,"num_write_kb":489536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"402'46","reported_seq":624,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.185614+0000","last_change":"2024-09-11T08:58:34.160655+0000","last_active":"2024-09-11T09:00:30.185614+0000","last_peered":"2024-09-11T09:00:30.185614+0000","last_clean":"2024-09-11T09:00:30.185614+0000","last_became_active":"2024-09-11T08:58:34.130937+0000","last_became_peered":"2024-09-11T08:58:34.130937+0000","last_unstale":"2024-09-11T09:00:30.185614+0000","last_undegraded":"2024-09-11T09:00:30.185614+0000","last_fullsized":"2024-09-11T09:00:30.185614+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'34","last_scrub_stamp":"2024-09-11T08:57:26.108866+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T08:57:26.108866+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023139469999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":181,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.190438+0000","last_change":"2024-09-11T08:56:32.686448+0000","last_active":"2024-09-11T09:00:30.190438+0000","last_peered":"2024-09-11T09:00:30.190438+0000","last_clean":"2024-09-11T09:00:30.190438+0000","last_became_active":"2024-09-11T08:52:38.484199+0000","last_became_peered":"2024-09-11T08:52:38.484199+0000","last_unstale":"2024-09-11T09:00:30.190438+0000","last_undegraded":"2024-09-11T09:00:30.190438+0000","last_fullsized":"2024-09-11T09:00:30.190438+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T08:56:32.686360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:56:32.686360+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082837799999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"401'14","reported_seq":207,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:42.923481+0000","last_change":"2024-09-11T08:58:34.163453+0000","last_active":"2024-09-11T09:00:42.923481+0000","last_peered":"2024-09-11T09:00:42.923481+0000","last_clean":"2024-09-11T09:00:42.923481+0000","last_became_active":"2024-09-11T08:58:34.135340+0000","last_became_peered":"2024-09-11T08:58:34.135340+0000","last_unstale":"2024-09-11T09:00:42.923481+0000","last_undegraded":"2024-09-11T09:00:42.923481+0000","last_fullsized":"2024-09-11T09:00:42.923481+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"346'12","last_scrub_stamp":"2024-09-11T08:57:42.673365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:57:42.673365+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.035195890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":491,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.185982+0000","last_change":"2024-09-11T08:59:36.505032+0000","last_active":"2024-09-11T09:00:30.185982+0000","last_peered":"2024-09-11T09:00:30.185982+0000","last_clean":"2024-09-11T09:00:30.185982+0000","last_became_active":"2024-09-11T08:58:34.128990+0000","last_became_peered":"2024-09-11T08:58:34.128990+0000","last_unstale":"2024-09-11T09:00:30.185982+0000","last_undegraded":"2024-09-11T09:00:30.185982+0000","last_fullsized":"2024-09-11T09:00:30.185982+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:36.504891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:59:36.504891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:57.766274+0000","scrub_duration":0.0058145430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"199'81","reported_seq":70,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.186167+0000","last_change":"2024-09-11T08:58:34.131260+0000","last_active":"2024-09-11T09:00:30.186167+0000","last_peered":"2024-09-11T09:00:30.186167+0000","last_clean":"2024-09-11T09:00:30.186167+0000","last_became_active":"2024-09-11T08:58:34.130930+0000","last_became_peered":"2024-09-11T08:58:34.130930+0000","last_unstale":"2024-09-11T09:00:30.186167+0000","last_undegraded":"2024-09-11T09:00:30.186167+0000","last_fullsized":"2024-09-11T09:00:30.186167+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":5,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:57:49.622788+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:57:49.622788+0000","last_clean_scrub_stamp":"2024-09-11T08:57:49.622788+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":11,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":466,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189184+0000","last_change":"2024-09-11T09:00:01.636347+0000","last_active":"2024-09-11T09:00:30.189184+0000","last_peered":"2024-09-11T09:00:30.189184+0000","last_clean":"2024-09-11T09:00:30.189184+0000","last_became_active":"2024-09-11T08:58:34.134966+0000","last_became_peered":"2024-09-11T08:58:34.134966+0000","last_unstale":"2024-09-11T09:00:30.189184+0000","last_undegraded":"2024-09-11T09:00:30.189184+0000","last_fullsized":"2024-09-11T09:00:30.189184+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:01.636269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T09:00:01.636269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:22.434253+0000","scrub_duration":0.0048176729999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"402'18","reported_seq":491,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.196559+0000","last_change":"2024-09-11T08:59:38.763105+0000","last_active":"2024-09-11T09:00:30.196559+0000","last_peered":"2024-09-11T09:00:30.196559+0000","last_clean":"2024-09-11T09:00:30.196559+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T09:00:30.196559+0000","last_undegraded":"2024-09-11T09:00:30.196559+0000","last_fullsized":"2024-09-11T09:00:30.196559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'18","last_scrub_stamp":"2024-09-11T08:59:38.763022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:59:38.763022+0000","objects_scrubbed":3,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:57.606101+0000","scrub_duration":0.0050463779999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":454,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.186462+0000","last_change":"2024-09-11T08:59:52.511520+0000","last_active":"2024-09-11T09:00:30.186462+0000","last_peered":"2024-09-11T09:00:30.186462+0000","last_clean":"2024-09-11T09:00:30.186462+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T09:00:30.186462+0000","last_undegraded":"2024-09-11T09:00:30.186462+0000","last_fullsized":"2024-09-11T09:00:30.186462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:52.511447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:59:52.511447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:04.810836+0000","scrub_duration":0.0045834229999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":514,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.191633+0000","last_change":"2024-09-11T08:59:42.519978+0000","last_active":"2024-09-11T09:00:30.191633+0000","last_peered":"2024-09-11T09:00:30.191633+0000","last_clean":"2024-09-11T09:00:30.191633+0000","last_became_active":"2024-09-11T08:58:34.135112+0000","last_became_peered":"2024-09-11T08:58:34.135112+0000","last_unstale":"2024-09-11T09:00:30.191633+0000","last_undegraded":"2024-09-11T09:00:30.191633+0000","last_fullsized":"2024-09-11T09:00:30.191633+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:42.519821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:59:42.519821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:50.077400+0000","scrub_duration":0.0034672069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"199'81","reported_seq":510,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.206291+0000","last_change":"2024-09-11T08:59:36.427908+0000","last_active":"2024-09-11T09:00:30.206291+0000","last_peered":"2024-09-11T09:00:30.206291+0000","last_clean":"2024-09-11T09:00:30.206291+0000","last_became_active":"2024-09-11T08:58:03.129022+0000","last_became_peered":"2024-09-11T08:58:03.129022+0000","last_unstale":"2024-09-11T09:00:30.206291+0000","last_undegraded":"2024-09-11T09:00:30.206291+0000","last_fullsized":"2024-09-11T09:00:30.206291+0000","mapping_epoch":385,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":386,"parent":"0.0","parent_split_bits":5,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:59:36.427363+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:59:36.427363+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:55.635190+0000","scrub_duration":0.0036244279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"402'21","reported_seq":487,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.195922+0000","last_change":"2024-09-11T08:57:27.752958+0000","last_active":"2024-09-11T09:00:30.195922+0000","last_peered":"2024-09-11T09:00:30.195922+0000","last_clean":"2024-09-11T09:00:30.195922+0000","last_became_active":"2024-09-11T08:52:49.569850+0000","last_became_peered":"2024-09-11T08:52:49.569850+0000","last_unstale":"2024-09-11T09:00:30.195922+0000","last_undegraded":"2024-09-11T09:00:30.195922+0000","last_fullsized":"2024-09-11T09:00:30.195922+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:57:27.752906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:57:27.752906+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020717140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":192,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189141+0000","last_change":"2024-09-11T08:58:52.798593+0000","last_active":"2024-09-11T09:00:30.189141+0000","last_peered":"2024-09-11T09:00:30.189141+0000","last_clean":"2024-09-11T09:00:30.189141+0000","last_became_active":"2024-09-11T08:58:34.124303+0000","last_became_peered":"2024-09-11T08:58:34.124303+0000","last_unstale":"2024-09-11T09:00:30.189141+0000","last_undegraded":"2024-09-11T09:00:30.189141+0000","last_fullsized":"2024-09-11T09:00:30.189141+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:58:52.798520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T08:58:52.798520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025654160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":434,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.196357+0000","last_change":"2024-09-11T08:57:34.759414+0000","last_active":"2024-09-11T09:00:30.196357+0000","last_peered":"2024-09-11T09:00:30.196357+0000","last_clean":"2024-09-11T09:00:30.196357+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T09:00:30.196357+0000","last_undegraded":"2024-09-11T09:00:30.196357+0000","last_fullsized":"2024-09-11T09:00:30.196357+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:34.759355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:57:34.759355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039078510000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":492,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.203774+0000","last_change":"2024-09-11T08:59:52.054494+0000","last_active":"2024-09-11T09:00:30.203774+0000","last_peered":"2024-09-11T09:00:30.203774+0000","last_clean":"2024-09-11T09:00:30.203774+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T09:00:30.203774+0000","last_undegraded":"2024-09-11T09:00:30.203774+0000","last_fullsized":"2024-09-11T09:00:30.203774+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:52.054422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:59:52.054422+0000","last_clean_scrub_stamp":"2024-09-11T08:59:52.054422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:10.272603+0000","scrub_duration":0.023663457999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"199'81","reported_seq":70,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.185336+0000","last_change":"2024-09-11T08:58:34.131215+0000","last_active":"2024-09-11T09:00:30.185336+0000","last_peered":"2024-09-11T09:00:30.185336+0000","last_clean":"2024-09-11T09:00:30.185336+0000","last_became_active":"2024-09-11T08:58:34.130875+0000","last_became_peered":"2024-09-11T08:58:34.130875+0000","last_unstale":"2024-09-11T09:00:30.185336+0000","last_undegraded":"2024-09-11T09:00:30.185336+0000","last_fullsized":"2024-09-11T09:00:30.185336+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":5,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:57:46.823248+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:55:08.518651+0000","last_clean_scrub_stamp":"2024-09-11T08:57:46.823248+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":11,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"402'21","reported_seq":506,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.191346+0000","last_change":"2024-09-11T08:57:42.104866+0000","last_active":"2024-09-11T09:00:30.191346+0000","last_peered":"2024-09-11T09:00:30.191346+0000","last_clean":"2024-09-11T09:00:30.191346+0000","last_became_active":"2024-09-11T08:53:58.555987+0000","last_became_peered":"2024-09-11T08:53:58.555987+0000","last_unstale":"2024-09-11T09:00:30.191346+0000","last_undegraded":"2024-09-11T09:00:30.191346+0000","last_fullsized":"2024-09-11T09:00:30.191346+0000","mapping_epoch":315,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'15","last_scrub_stamp":"2024-09-11T08:57:42.104756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:57:42.104756+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.062049017999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":172,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.195673+0000","last_change":"2024-09-11T08:58:51.491740+0000","last_active":"2024-09-11T09:00:30.195673+0000","last_peered":"2024-09-11T09:00:30.195673+0000","last_clean":"2024-09-11T09:00:30.195673+0000","last_became_active":"2024-09-11T08:52:49.566759+0000","last_became_peered":"2024-09-11T08:52:49.566759+0000","last_unstale":"2024-09-11T09:00:30.195673+0000","last_undegraded":"2024-09-11T09:00:30.195673+0000","last_fullsized":"2024-09-11T09:00:30.195673+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:58:51.491665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:03.641009+0000","last_clean_scrub_stamp":"2024-09-11T08:58:51.491665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005754065,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":707,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:42.923278+0000","last_change":"2024-09-11T08:57:28.788665+0000","last_active":"2024-09-11T09:00:42.923278+0000","last_peered":"2024-09-11T09:00:42.923278+0000","last_clean":"2024-09-11T09:00:42.923278+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T09:00:42.923278+0000","last_undegraded":"2024-09-11T09:00:42.923278+0000","last_fullsized":"2024-09-11T09:00:42.923278+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T08:57:28.788610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:57:28.788610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001594315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":436,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.190034+0000","last_change":"2024-09-11T08:59:53.976457+0000","last_active":"2024-09-11T09:00:30.190034+0000","last_peered":"2024-09-11T09:00:30.190034+0000","last_clean":"2024-09-11T09:00:30.190034+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T09:00:30.190034+0000","last_undegraded":"2024-09-11T09:00:30.190034+0000","last_fullsized":"2024-09-11T09:00:30.190034+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:53.976399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T08:59:53.976399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:05.705092+0000","scrub_duration":0.0018491969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"199'81","reported_seq":244,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:33.377129+0000","last_change":"2024-09-11T09:00:33.377066+0000","last_active":"2024-09-11T09:00:33.377129+0000","last_peered":"2024-09-11T09:00:33.377129+0000","last_clean":"2024-09-11T09:00:33.377129+0000","last_became_active":"2024-09-11T08:58:03.126550+0000","last_became_peered":"2024-09-11T08:58:03.126550+0000","last_unstale":"2024-09-11T09:00:33.377129+0000","last_undegraded":"2024-09-11T09:00:33.377129+0000","last_fullsized":"2024-09-11T09:00:33.377129+0000","mapping_epoch":385,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":386,"parent":"0.0","parent_split_bits":5,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:00:33.376978+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:00:33.376978+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:43.719565+0000","scrub_duration":0.0050545779999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"402'35","reported_seq":518,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.194893+0000","last_change":"2024-09-11T09:00:00.778073+0000","last_active":"2024-09-11T09:00:30.194893+0000","last_peered":"2024-09-11T09:00:30.194893+0000","last_clean":"2024-09-11T09:00:30.194893+0000","last_became_active":"2024-09-11T08:56:25.465124+0000","last_became_peered":"2024-09-11T08:56:25.465124+0000","last_unstale":"2024-09-11T09:00:30.194893+0000","last_undegraded":"2024-09-11T09:00:30.194893+0000","last_fullsized":"2024-09-11T09:00:30.194893+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'35","last_scrub_stamp":"2024-09-11T09:00:00.777986+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T09:00:00.777986+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:14.708768+0000","scrub_duration":0.013426545999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":487,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.192196+0000","last_change":"2024-09-11T08:59:45.377712+0000","last_active":"2024-09-11T09:00:30.192196+0000","last_peered":"2024-09-11T09:00:30.192196+0000","last_clean":"2024-09-11T09:00:30.192196+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T09:00:30.192196+0000","last_undegraded":"2024-09-11T09:00:30.192196+0000","last_fullsized":"2024-09-11T09:00:30.192196+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:45.377656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:15.235519+0000","last_clean_scrub_stamp":"2024-09-11T08:59:45.377656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:53.408511+0000","scrub_duration":0.00160103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"277'10","reported_seq":755,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:42.923474+0000","last_change":"2024-09-11T09:00:35.376518+0000","last_active":"2024-09-11T09:00:42.923474+0000","last_peered":"2024-09-11T09:00:42.923474+0000","last_clean":"2024-09-11T09:00:42.923474+0000","last_became_active":"2024-09-11T08:52:18.585914+0000","last_became_peered":"2024-09-11T08:52:18.585914+0000","last_unstale":"2024-09-11T09:00:42.923474+0000","last_undegraded":"2024-09-11T09:00:42.923474+0000","last_fullsized":"2024-09-11T09:00:42.923474+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'10","last_scrub_stamp":"2024-09-11T09:00:35.376455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:00:35.376455+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:38.345184+0000","scrub_duration":0.0016380909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":182,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:40.067570+0000","last_change":"2024-09-11T09:00:40.067532+0000","last_active":"2024-09-11T09:00:40.067570+0000","last_peered":"2024-09-11T09:00:40.067570+0000","last_clean":"2024-09-11T09:00:40.067570+0000","last_became_active":"2024-09-11T08:52:49.567693+0000","last_became_peered":"2024-09-11T08:52:49.567693+0000","last_unstale":"2024-09-11T09:00:40.067570+0000","last_undegraded":"2024-09-11T09:00:40.067570+0000","last_fullsized":"2024-09-11T09:00:40.067570+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:40.067474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:00:40.067474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:53.098022+0000","scrub_duration":0.001476527,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"402'27","reported_seq":546,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.189775+0000","last_change":"2024-09-11T08:58:34.168360+0000","last_active":"2024-09-11T09:00:30.189775+0000","last_peered":"2024-09-11T09:00:30.189775+0000","last_clean":"2024-09-11T09:00:30.189775+0000","last_became_active":"2024-09-11T08:58:34.135591+0000","last_became_peered":"2024-09-11T08:58:34.135591+0000","last_unstale":"2024-09-11T09:00:30.189775+0000","last_undegraded":"2024-09-11T09:00:30.189775+0000","last_fullsized":"2024-09-11T09:00:30.189775+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'19","last_scrub_stamp":"2024-09-11T08:57:47.679894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:57:47.679894+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026090039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":459,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.190610+0000","last_change":"2024-09-11T08:58:34.139289+0000","last_active":"2024-09-11T09:00:30.190610+0000","last_peered":"2024-09-11T09:00:30.190610+0000","last_clean":"2024-09-11T09:00:30.190610+0000","last_became_active":"2024-09-11T08:58:34.138928+0000","last_became_peered":"2024-09-11T08:58:34.138928+0000","last_unstale":"2024-09-11T09:00:30.190610+0000","last_undegraded":"2024-09-11T09:00:30.190610+0000","last_fullsized":"2024-09-11T09:00:30.190610+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:22.682317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:57:22.682317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014903729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":474,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.206952+0000","last_change":"2024-09-11T08:59:50.187401+0000","last_active":"2024-09-11T09:00:30.206952+0000","last_peered":"2024-09-11T09:00:30.206952+0000","last_clean":"2024-09-11T09:00:30.206952+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T09:00:30.206952+0000","last_undegraded":"2024-09-11T09:00:30.206952+0000","last_fullsized":"2024-09-11T09:00:30.206952+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:50.187325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:59:50.187325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:11.527817+0000","scrub_duration":0.0082628449999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":452,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.194696+0000","last_change":"2024-09-11T08:56:47.833618+0000","last_active":"2024-09-11T09:00:30.194696+0000","last_peered":"2024-09-11T09:00:30.194696+0000","last_clean":"2024-09-11T09:00:30.194696+0000","last_became_active":"2024-09-11T08:52:18.585089+0000","last_became_peered":"2024-09-11T08:52:18.585089+0000","last_unstale":"2024-09-11T09:00:30.194696+0000","last_undegraded":"2024-09-11T09:00:30.194696+0000","last_fullsized":"2024-09-11T09:00:30.194696+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:47.833532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:53:21.391849+0000","last_clean_scrub_stamp":"2024-09-11T08:56:47.833532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001512191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"404'22","reported_seq":76,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:34.192072+0000","last_change":"2024-09-11T09:00:34.192001+0000","last_active":"2024-09-11T09:00:34.192072+0000","last_peered":"2024-09-11T09:00:34.192072+0000","last_clean":"2024-09-11T09:00:34.192072+0000","last_became_active":"2024-09-11T08:58:34.315719+0000","last_became_peered":"2024-09-11T08:58:34.315719+0000","last_unstale":"2024-09-11T09:00:34.192072+0000","last_undegraded":"2024-09-11T09:00:34.192072+0000","last_fullsized":"2024-09-11T09:00:34.192072+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"404'22","last_scrub_stamp":"2024-09-11T09:00:34.191902+0000","last_deep_scrub":"404'22","last_deep_scrub_stamp":"2024-09-11T08:58:50.028197+0000","last_clean_scrub_stamp":"2024-09-11T09:00:34.191902+0000","objects_scrubbed":4,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:44.813454+0000","scrub_duration":0.0030888600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":172,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.195358+0000","last_change":"2024-09-11T08:56:31.849778+0000","last_active":"2024-09-11T09:00:30.195358+0000","last_peered":"2024-09-11T09:00:30.195358+0000","last_clean":"2024-09-11T09:00:30.195358+0000","last_became_active":"2024-09-11T08:52:38.485748+0000","last_became_peered":"2024-09-11T08:52:38.485748+0000","last_unstale":"2024-09-11T09:00:30.195358+0000","last_undegraded":"2024-09-11T09:00:30.195358+0000","last_fullsized":"2024-09-11T09:00:30.195358+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:56:31.849658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:21.208684+0000","last_clean_scrub_stamp":"2024-09-11T08:56:31.849658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079402989999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":464,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:42.027497+0000","last_change":"2024-09-11T09:00:42.027449+0000","last_active":"2024-09-11T09:00:42.027497+0000","last_peered":"2024-09-11T09:00:42.027497+0000","last_clean":"2024-09-11T09:00:42.027497+0000","last_became_active":"2024-09-11T08:52:18.585691+0000","last_became_peered":"2024-09-11T08:52:18.585691+0000","last_unstale":"2024-09-11T09:00:42.027497+0000","last_undegraded":"2024-09-11T09:00:42.027497+0000","last_fullsized":"2024-09-11T09:00:42.027497+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:42.027377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:00:42.027377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:49.375067+0000","scrub_duration":0.001578217,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":254,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.185538+0000","last_change":"2024-09-11T08:57:33.989083+0000","last_active":"2024-09-11T09:00:30.185538+0000","last_peered":"2024-09-11T09:00:30.185538+0000","last_clean":"2024-09-11T09:00:30.185538+0000","last_became_active":"2024-09-11T08:52:49.566694+0000","last_became_peered":"2024-09-11T08:52:49.566694+0000","last_unstale":"2024-09-11T09:00:30.185538+0000","last_undegraded":"2024-09-11T09:00:30.185538+0000","last_fullsized":"2024-09-11T09:00:30.185538+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:33.988992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:24.401480+0000","last_clean_scrub_stamp":"2024-09-11T08:57:33.988992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020405710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.14","version":"199'81","reported_seq":60,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.203589+0000","last_change":"2024-09-11T08:59:32.035683+0000","last_active":"2024-09-11T09:00:30.203589+0000","last_peered":"2024-09-11T09:00:30.203589+0000","last_clean":"2024-09-11T09:00:30.203589+0000","last_became_active":"2024-09-11T08:58:03.128935+0000","last_became_peered":"2024-09-11T08:58:03.128935+0000","last_unstale":"2024-09-11T09:00:30.203589+0000","last_undegraded":"2024-09-11T09:00:30.203589+0000","last_fullsized":"2024-09-11T09:00:30.203589+0000","mapping_epoch":385,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":386,"parent":"0.0","parent_split_bits":5,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T08:59:32.035548+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T08:59:32.035548+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:01:00.050546+0000","scrub_duration":0.0089104370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"402'27","reported_seq":523,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.194510+0000","last_change":"2024-09-11T08:58:50.511223+0000","last_active":"2024-09-11T09:00:30.194510+0000","last_peered":"2024-09-11T09:00:30.194510+0000","last_clean":"2024-09-11T09:00:30.194510+0000","last_became_active":"2024-09-11T08:58:34.136417+0000","last_became_peered":"2024-09-11T08:58:34.136417+0000","last_unstale":"2024-09-11T09:00:30.194510+0000","last_undegraded":"2024-09-11T09:00:30.194510+0000","last_fullsized":"2024-09-11T09:00:30.194510+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'27","last_scrub_stamp":"2024-09-11T08:58:50.511078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:58:50.511078+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021957132000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":468,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.186585+0000","last_change":"2024-09-11T08:58:50.816299+0000","last_active":"2024-09-11T09:00:30.186585+0000","last_peered":"2024-09-11T09:00:30.186585+0000","last_clean":"2024-09-11T09:00:30.186585+0000","last_became_active":"2024-09-11T08:56:25.460327+0000","last_became_peered":"2024-09-11T08:56:25.460327+0000","last_unstale":"2024-09-11T09:00:30.186585+0000","last_undegraded":"2024-09-11T09:00:30.186585+0000","last_fullsized":"2024-09-11T09:00:30.186585+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:58:50.816221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:58:50.816221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090614000000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":509,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.190940+0000","last_change":"2024-09-11T08:59:37.509820+0000","last_active":"2024-09-11T09:00:30.190940+0000","last_peered":"2024-09-11T09:00:30.190940+0000","last_clean":"2024-09-11T09:00:30.190940+0000","last_became_active":"2024-09-11T08:58:34.135692+0000","last_became_peered":"2024-09-11T08:58:34.135692+0000","last_unstale":"2024-09-11T09:00:30.190940+0000","last_undegraded":"2024-09-11T09:00:30.190940+0000","last_fullsized":"2024-09-11T09:00:30.190940+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:37.509735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:59:37.509735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038713910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":492,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.196598+0000","last_change":"2024-09-11T08:58:34.136856+0000","last_active":"2024-09-11T09:00:30.196598+0000","last_peered":"2024-09-11T09:00:30.196598+0000","last_clean":"2024-09-11T09:00:30.196598+0000","last_became_active":"2024-09-11T08:58:34.135699+0000","last_became_peered":"2024-09-11T08:58:34.135699+0000","last_unstale":"2024-09-11T09:00:30.196598+0000","last_undegraded":"2024-09-11T09:00:30.196598+0000","last_fullsized":"2024-09-11T09:00:30.196598+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:32.723309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:25.255332+0000","last_clean_scrub_stamp":"2024-09-11T08:57:32.723309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030645970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"402'14","reported_seq":39,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.185245+0000","last_change":"2024-09-11T08:59:34.566349+0000","last_active":"2024-09-11T09:00:30.185245+0000","last_peered":"2024-09-11T09:00:30.185245+0000","last_clean":"2024-09-11T09:00:30.185245+0000","last_became_active":"2024-09-11T08:59:18.588313+0000","last_became_peered":"2024-09-11T08:59:18.588313+0000","last_unstale":"2024-09-11T09:00:30.185245+0000","last_undegraded":"2024-09-11T09:00:30.185245+0000","last_fullsized":"2024-09-11T09:00:30.185245+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'14","last_scrub_stamp":"2024-09-11T08:59:34.566295+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T08:59:34.566295+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:50.203626+0000","scrub_duration":0.0052631730000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":266,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.187203+0000","last_change":"2024-09-11T08:57:25.060216+0000","last_active":"2024-09-11T09:00:30.187203+0000","last_peered":"2024-09-11T09:00:30.187203+0000","last_clean":"2024-09-11T09:00:30.187203+0000","last_became_active":"2024-09-11T08:52:49.569954+0000","last_became_peered":"2024-09-11T08:52:49.569954+0000","last_unstale":"2024-09-11T09:00:30.187203+0000","last_undegraded":"2024-09-11T09:00:30.187203+0000","last_fullsized":"2024-09-11T09:00:30.187203+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T08:57:25.060145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T08:57:25.060145+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017002149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":177,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.188861+0000","last_change":"2024-09-11T08:58:51.777702+0000","last_active":"2024-09-11T09:00:30.188861+0000","last_peered":"2024-09-11T09:00:30.188861+0000","last_clean":"2024-09-11T09:00:30.188861+0000","last_became_active":"2024-09-11T08:58:34.123709+0000","last_became_peered":"2024-09-11T08:58:34.123709+0000","last_unstale":"2024-09-11T09:00:30.188861+0000","last_undegraded":"2024-09-11T09:00:30.188861+0000","last_fullsized":"2024-09-11T09:00:30.188861+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:58:51.777616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:58:51.777616+0000","last_clean_scrub_stamp":"2024-09-11T08:58:51.777616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019640949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":133,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.188736+0000","last_change":"2024-09-11T08:57:39.555701+0000","last_active":"2024-09-11T09:00:30.188736+0000","last_peered":"2024-09-11T09:00:30.188736+0000","last_clean":"2024-09-11T09:00:30.188736+0000","last_became_active":"2024-09-11T08:54:17.154933+0000","last_became_peered":"2024-09-11T08:54:17.154933+0000","last_unstale":"2024-09-11T09:00:30.188736+0000","last_undegraded":"2024-09-11T09:00:30.188736+0000","last_fullsized":"2024-09-11T09:00:30.188736+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:39.555623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:30.788303+0000","last_clean_scrub_stamp":"2024-09-11T08:57:39.555623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019664629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":482,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:44.038628+0000","last_change":"2024-09-11T09:00:44.038582+0000","last_active":"2024-09-11T09:00:44.038628+0000","last_peered":"2024-09-11T09:00:44.038628+0000","last_clean":"2024-09-11T09:00:44.038628+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T09:00:44.038628+0000","last_undegraded":"2024-09-11T09:00:44.038628+0000","last_fullsized":"2024-09-11T09:00:44.038628+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:00:44.038527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:00:44.038527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:12.698123+0000","scrub_duration":0.001517352,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":478,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.197690+0000","last_change":"2024-09-11T08:57:35.763188+0000","last_active":"2024-09-11T09:00:30.197690+0000","last_peered":"2024-09-11T09:00:30.197690+0000","last_clean":"2024-09-11T09:00:30.197690+0000","last_became_active":"2024-09-11T08:56:25.458500+0000","last_became_peered":"2024-09-11T08:56:25.458500+0000","last_unstale":"2024-09-11T09:00:30.197690+0000","last_undegraded":"2024-09-11T09:00:30.197690+0000","last_fullsized":"2024-09-11T09:00:30.197690+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:57:35.763114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:32.393197+0000","last_clean_scrub_stamp":"2024-09-11T08:57:35.763114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042514889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"402'27","reported_seq":233,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.187054+0000","last_change":"2024-09-11T08:58:52.873049+0000","last_active":"2024-09-11T09:00:30.187054+0000","last_peered":"2024-09-11T09:00:30.187054+0000","last_clean":"2024-09-11T09:00:30.187054+0000","last_became_active":"2024-09-11T08:58:34.136082+0000","last_became_peered":"2024-09-11T08:58:34.136082+0000","last_unstale":"2024-09-11T09:00:30.187054+0000","last_undegraded":"2024-09-11T09:00:30.187054+0000","last_fullsized":"2024-09-11T09:00:30.187054+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'27","last_scrub_stamp":"2024-09-11T08:58:52.872949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T08:58:52.872949+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095344599999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":276,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T09:00:30.191974+0000","last_change":"2024-09-11T08:59:38.525039+0000","last_active":"2024-09-11T09:00:30.191974+0000","last_peered":"2024-09-11T09:00:30.191974+0000","last_clean":"2024-09-11T09:00:30.191974+0000","last_became_active":"2024-09-11T08:52:49.568537+0000","last_became_peered":"2024-09-11T08:52:49.568537+0000","last_unstale":"2024-09-11T09:00:30.191974+0000","last_undegraded":"2024-09-11T09:00:30.191974+0000","last_fullsized":"2024-09-11T09:00:30.191974+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:59:38.524935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T08:59:38.524935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061219220000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242638,"num_write_kb":7764353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16370,"num_bytes_recovered":1072824320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76928,"ondisk_log_size":76928,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1540,"num_read_kb":1297,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":7146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1068,"ondisk_log_size":1068,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":355,"num_read_kb":350,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":465,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":534811,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":22,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1090380,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299663,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":889756,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":888894,"kb_avail":92855396,"statfs":{"total":95995035648,"available":95083925504,"internally_reserved":0,"allocated":868352,"data_stored":505359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":910227800},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:00:08 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.86199999999999999,"15min":0.97699999999999998},"min":{"1min":0.26500000000000001,"5min":0.22500000000000001,"15min":0.20599999999999999},"max":{"1min":4.133,"5min":4.133,"15min":32.171999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":1.046,"5min":0.88300000000000001,"15min":1.0249999999999999},"min":{"1min":0.23999999999999999,"5min":0.23200000000000001,"15min":0.221},"max":{"1min":4.5289999999999999,"5min":4.5289999999999999,"15min":32.149999999999999},"last":0.44400000000000001}]},{"osd":1,"last update":"Wed Sep 11 09:00:08 2024","interfaces":[{"interface":"back","average":{"1min":2.1299999999999999,"5min":1.1279999999999999,"15min":1.1639999999999999},"min":{"1min":0.29399999999999998,"5min":0.249,"15min":0.224},"max":{"1min":26.193999999999999,"5min":26.193999999999999,"15min":33.692999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":2.4369999999999998,"5min":1.042,"15min":1.1000000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.23799999999999999},"max":{"1min":27.058,"5min":27.058,"15min":35.591000000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Wed Sep 11 09:00:08 2024","interfaces":[{"interface":"back","average":{"1min":6.7800000000000002,"5min":1.9830000000000001,"15min":1.351},"min":{"1min":0.35199999999999998,"5min":0.27000000000000002,"15min":0.249},"max":{"1min":116.639,"5min":116.639,"15min":116.639},"last":0.36599999999999999},{"interface":"front","average":{"1min":6.484,"5min":2.008,"15min":1.304},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.25800000000000001},"max":{"1min":116.68899999999999,"5min":116.68899999999999,"15min":116.68899999999999},"last":0.47999999999999998}]},{"osd":3,"last update":"Wed Sep 11 09:00:08 2024","interfaces":[{"interface":"back","average":{"1min":2.5259999999999998,"5min":1.5660000000000001,"15min":1.431},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":26.225000000000001,"5min":26.225000000000001,"15min":56.265999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":2.8490000000000002,"5min":1.7629999999999999,"15min":1.5009999999999999},"min":{"1min":0.317,"5min":0.29599999999999999,"15min":0.27700000000000002},"max":{"1min":26.280999999999999,"5min":26.280999999999999,"15min":57.207000000000001},"last":0.42899999999999999}]},{"osd":4,"last update":"Wed Sep 11 09:00:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":0.874,"15min":1.486},"min":{"1min":0.24199999999999999,"5min":0.224,"15min":0.224},"max":{"1min":9.0879999999999992,"5min":9.0879999999999992,"15min":163.60499999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.74099999999999999,"15min":1.534},"min":{"1min":0.25,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":3.3620000000000001,"5min":5.1529999999999996,"15min":163.88300000000001},"last":0.33100000000000002}]},{"osd":5,"last update":"Wed Sep 11 09:00:08 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":1.2509999999999999,"15min":1.0489999999999999},"min":{"1min":0.28899999999999998,"5min":0.28799999999999998,"15min":0.23699999999999999},"max":{"1min":4.7000000000000002,"5min":50.128,"15min":50.128},"last":0.39600000000000002},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.294,"15min":1.1040000000000001},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.26400000000000001},"max":{"1min":4.992,"5min":50.241,"15min":50.241},"last":0.374}]},{"osd":6,"last update":"Wed Sep 11 09:00:08 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.046,"15min":0.98699999999999999},"min":{"1min":0.317,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.4359999999999999,"5min":8.4359999999999999,"15min":8.4359999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.71499999999999997,"15min":0.90700000000000003},"min":{"1min":0.32600000000000001,"5min":0.25,"15min":0.25},"max":{"1min":1.8680000000000001,"5min":4.1180000000000003,"15min":5.0620000000000003},"last":0.40899999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528596,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":898392,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":897214,"kb_avail":92846760,"statfs":{"total":95995035648,"available":95075082240,"internally_reserved":0,"allocated":1191936,"data_stored":833013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":918747480},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.91900000000000004,"15min":1.5800000000000001},"min":{"1min":0.25700000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":4.4290000000000003,"5min":5.577,"15min":121.208},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":1.1080000000000001,"15min":1.7030000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":2.024,"5min":5.0090000000000003,"15min":121.05200000000001},"last":0.55400000000000005}]},{"osd":1,"last update":"Wed Sep 11 08:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.258,"15min":1.387},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.23000000000000001},"max":{"1min":3.3959999999999999,"5min":7.4180000000000001,"15min":27.984000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.038,"15min":1.2629999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.254},"max":{"1min":6.8659999999999997,"5min":7.3049999999999997,"15min":28.106000000000002},"last":0.63300000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":2.0209999999999999,"15min":1.3120000000000001},"min":{"1min":0.376,"5min":0.23999999999999999,"15min":0.23200000000000001},"max":{"1min":3.1659999999999999,"5min":67.486999999999995,"15min":67.486999999999995},"last":0.504},{"interface":"front","average":{"1min":1.2330000000000001,"5min":2.1219999999999999,"15min":1.496},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.26600000000000001},"max":{"1min":4.3150000000000004,"5min":67.272000000000006,"15min":67.272000000000006},"last":0.39700000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.49,"15min":1.4319999999999999},"min":{"1min":0.36299999999999999,"5min":0.27200000000000002,"15min":0.26700000000000002},"max":{"1min":3.3279999999999998,"5min":13.871,"15min":27.795000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":1.627,"5min":1.369,"15min":1.369},"min":{"1min":0.48799999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":5.5949999999999998,"5min":12.318,"15min":27.841999999999999},"last":0.59599999999999997}]},{"osd":4,"last update":"Wed Sep 11 08:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.1739999999999999,"15min":1.101},"min":{"1min":0.26700000000000002,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":5.2640000000000002,"5min":5.4829999999999997,"15min":40.927999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.96899999999999997,"15min":1.0289999999999999},"min":{"1min":0.30299999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.2509999999999999,"5min":5.891,"15min":40.706000000000003},"last":0.52900000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.298,"15min":1.254},"min":{"1min":0.311,"5min":0.29499999999999998,"15min":0.23899999999999999},"max":{"1min":5.3040000000000003,"5min":19.309999999999999,"15min":40.847000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.272,"15min":1.167},"min":{"1min":0.318,"5min":0.28199999999999997,"15min":0.27100000000000002},"max":{"1min":3.3690000000000002,"5min":19.411000000000001,"15min":41.401000000000003},"last":0.57699999999999996}]},{"osd":7,"last update":"Wed Sep 11 09:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.84899999999999998,"15min":1.131},"min":{"1min":0.34799999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.0419999999999998,"5min":3.133,"15min":40.426000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.79100000000000004,"15min":1.0840000000000001},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.309},"max":{"1min":3.0099999999999998,"5min":3.0099999999999998,"15min":45.401000000000003},"last":0.68300000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757528,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":895064,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":893950,"kb_avail":92850088,"statfs":{"total":95995035648,"available":95078490112,"internally_reserved":0,"allocated":1126400,"data_stored":768460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":915405144},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:00:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":1.431,"15min":1.1830000000000001},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.21099999999999999},"max":{"1min":2.6509999999999998,"5min":25.09,"15min":25.09},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.212,"5min":1.2450000000000001,"15min":1.1579999999999999},"min":{"1min":0.26000000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":4.8760000000000003,"5min":23.628,"15min":23.628},"last":0.73299999999999998}]},{"osd":1,"last update":"Wed Sep 11 09:00:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":1.734,"15min":1.3300000000000001},"min":{"1min":0.29799999999999999,"5min":0.23999999999999999,"15min":0.216},"max":{"1min":3.032,"5min":49.965000000000003,"15min":49.965000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.7270000000000001,"15min":1.3169999999999999},"min":{"1min":0.26600000000000001,"5min":0.24399999999999999,"15min":0.23400000000000001},"max":{"1min":4.4320000000000004,"5min":48.692,"15min":48.692},"last":0.54100000000000004}]},{"osd":2,"last update":"Wed Sep 11 09:00:24 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.8149999999999999,"15min":1.365},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.253},"max":{"1min":7.5019999999999998,"5min":48.331000000000003,"15min":48.331000000000003},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.625,"5min":1.7070000000000001,"15min":1.306},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.6699999999999999,"5min":50.441000000000003,"15min":50.441000000000003},"last":0.79700000000000004}]},{"osd":3,"last update":"Wed Sep 11 09:00:24 2024","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":2.0990000000000002,"15min":1.7330000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":7.7169999999999996,"5min":51.340000000000003,"15min":85.299999999999997},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.0169999999999999,"5min":2.117,"15min":1.788},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29099999999999998},"max":{"1min":7.9420000000000002,"5min":51.889000000000003,"15min":85.941000000000003},"last":0.66300000000000003}]},{"osd":4,"last update":"Wed Sep 11 09:00:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.69699999999999995,"15min":0.88300000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.6950000000000001,"5min":3.8570000000000002,"15min":16.021999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.81899999999999995,"15min":0.85699999999999998},"min":{"1min":0.31,"5min":0.27800000000000002,"15min":0.25800000000000001},"max":{"1min":3.0369999999999999,"5min":6.9909999999999997,"15min":9.6140000000000008},"last":0.34599999999999997}]},{"osd":6,"last update":"Wed Sep 11 08:59:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.85399999999999998,"15min":0.93400000000000005},"min":{"1min":0.316,"5min":0.253,"15min":0.253},"max":{"1min":1.3169999999999999,"5min":6.8979999999999997,"15min":8.8059999999999992},"last":0.31},{"interface":"front","average":{"1min":0.622,"5min":1.0509999999999999,"15min":0.94599999999999995},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.27200000000000002},"max":{"1min":1.6579999999999999,"5min":6.5999999999999996,"15min":8.1920000000000002},"last":0.29799999999999999}]},{"osd":7,"last update":"Wed Sep 11 09:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.86299999999999999,"15min":1.2490000000000001},"min":{"1min":0.28799999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.7829999999999999,"5min":6.2969999999999997,"15min":88.513000000000005},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.874,"15min":1.246},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.29599999999999999},"max":{"1min":1.8260000000000001,"5min":6.9340000000000002,"15min":86.963999999999999},"last":0.36499999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986460,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":890068,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":889214,"kb_avail":92855084,"statfs":{"total":95995035648,"available":95083606016,"internally_reserved":0,"allocated":860160,"data_stored":504601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":910555480},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:00:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":1.1399999999999999,"15min":1.1910000000000001},"min":{"1min":0.246,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.8860000000000001,"5min":5.069,"15min":44.140999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0249999999999999,"15min":1.2869999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.22600000000000001},"max":{"1min":4.1580000000000004,"5min":6.242,"15min":44.450000000000003},"last":0.51200000000000001}]},{"osd":1,"last update":"Wed Sep 11 09:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.302,"15min":1.5820000000000001},"min":{"1min":0.32300000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":2.5680000000000001,"5min":24.946000000000002,"15min":48.976999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.468,"15min":1.613},"min":{"1min":0.28000000000000003,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.9079999999999999,"5min":25.035,"15min":45.787999999999997},"last":0.56999999999999995}]},{"osd":2,"last update":"Wed Sep 11 09:00:02 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":1.548,"15min":2.1709999999999998},"min":{"1min":0.33800000000000002,"5min":0.248,"15min":0.21099999999999999},"max":{"1min":2.9279999999999999,"5min":44.311,"15min":255.88900000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.6180000000000001,"15min":2.1960000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.399,"5min":40.161999999999999,"15min":255.41200000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Wed Sep 11 09:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.498,"15min":1.605},"min":{"1min":0.26900000000000002,"5min":0.26500000000000001,"15min":0.252},"max":{"1min":3.0489999999999999,"5min":21.652999999999999,"15min":45.268000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.627,"5min":1.4390000000000001,"15min":1.5489999999999999},"min":{"1min":0.312,"5min":0.26600000000000001,"15min":0.25900000000000001},"max":{"1min":5.3090000000000002,"5min":21.556000000000001,"15min":44.234000000000002},"last":0.53000000000000003}]},{"osd":5,"last update":"Wed Sep 11 09:00:24 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.80900000000000005,"15min":0.89900000000000002},"min":{"1min":0.26200000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.524,"5min":4.9870000000000001,"15min":11.045999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.81699999999999995,"15min":0.89800000000000002},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.254},"max":{"1min":2.8279999999999998,"5min":4.7119999999999997,"15min":10.526999999999999},"last":0.40100000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:59:53 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.998,"15min":1.21},"min":{"1min":0.311,"5min":0.27000000000000002,"15min":0.22900000000000001},"max":{"1min":3.23,"5min":72.100999999999999,"15min":72.100999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":2.141,"15min":1.2569999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.221},"max":{"1min":2.383,"5min":74.527000000000001,"15min":74.527000000000001},"last":0.42399999999999999}]},{"osd":7,"last update":"Wed Sep 11 09:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.95099999999999996,"15min":0.85599999999999998},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.252},"max":{"1min":2.8620000000000001,"5min":6.907,"15min":6.907},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":2.0209999999999999,"5min":6.8029999999999999,"15min":6.8029999999999999},"last":0.41299999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149986,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":904824,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":903358,"kb_avail":92840328,"statfs":{"total":95995035648,"available":95068495872,"internally_reserved":0,"allocated":1486848,"data_stored":1097049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":925038936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:00:43 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.238,"15min":1.1559999999999999},"min":{"1min":0.18099999999999999,"5min":0.17499999999999999,"15min":0.16500000000000001},"max":{"1min":4.4290000000000003,"5min":15.601000000000001,"15min":24.280000000000001},"last":0.27400000000000002},{"interface":"front","average":{"1min":1.9910000000000001,"5min":1.5940000000000001,"15min":1.3360000000000001},"min":{"1min":0.219,"5min":0.20000000000000001,"15min":0.17000000000000001},"max":{"1min":7.9279999999999999,"5min":16.492999999999999,"15min":22.225000000000001},"last":0.28899999999999998}]},{"osd":1,"last update":"Wed Sep 11 09:00:43 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.595,"15min":2.0449999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.19800000000000001},"max":{"1min":5.8959999999999999,"5min":16.559000000000001,"15min":115.601},"last":0.33100000000000002},{"interface":"front","average":{"1min":1.367,"5min":1.7370000000000001,"15min":2.0259999999999998},"min":{"1min":0.23499999999999999,"5min":0.22600000000000001,"15min":0.20499999999999999},"max":{"1min":5.3899999999999997,"5min":13.686,"15min":115.474},"last":0.308}]},{"osd":2,"last update":"Wed Sep 11 09:00:43 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.5609999999999999,"15min":1.583},"min":{"1min":0.28699999999999998,"5min":0.23100000000000001,"15min":0.22500000000000001},"max":{"1min":4.7569999999999997,"5min":13.627000000000001,"15min":86.766000000000005},"last":0.25600000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.5620000000000001,"15min":1.55},"min":{"1min":0.28299999999999997,"5min":0.23200000000000001,"15min":0.20799999999999999},"max":{"1min":3.1339999999999999,"5min":15.621,"15min":85.367999999999995},"last":0.318}]},{"osd":4,"last update":"Wed Sep 11 08:59:50 2024","interfaces":[{"interface":"back","average":{"1min":2.3570000000000002,"5min":1.6439999999999999,"15min":1.4419999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":9.7550000000000008,"5min":13.663,"15min":13.663},"last":0.45400000000000001},{"interface":"front","average":{"1min":2.028,"5min":1.403,"15min":1.1950000000000001},"min":{"1min":0.38,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":17.079999999999998,"5min":17.079999999999998,"15min":17.079999999999998},"last":0.39000000000000001}]},{"osd":5,"last update":"Wed Sep 11 09:00:15 2024","interfaces":[{"interface":"back","average":{"1min":2.2909999999999999,"5min":1.617,"15min":1.375},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.29799999999999999},"max":{"1min":9.7859999999999996,"5min":13.736000000000001,"15min":13.736000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":1.9159999999999999,"5min":1.373,"15min":1.248},"min":{"1min":0.36699999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":17.048999999999999,"5min":17.048999999999999,"15min":17.048999999999999},"last":0.41799999999999998}]},{"osd":6,"last update":"Wed Sep 11 09:00:43 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.5660000000000001,"15min":1.5760000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.29999999999999999},"max":{"1min":3.5310000000000001,"5min":13.755000000000001,"15min":33.106999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":1.393,"5min":1.6040000000000001,"15min":1.4770000000000001},"min":{"1min":0.33200000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.3620000000000001,"5min":16.527000000000001,"15min":33.076000000000001},"last":0.49399999999999999}]},{"osd":7,"last update":"Wed Sep 11 09:00:06 2024","interfaces":[{"interface":"back","average":{"1min":2.2370000000000001,"5min":1.5609999999999999,"15min":1.607},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":9.8520000000000003,"5min":13.712999999999999,"15min":35.073},"last":0.40999999999999998},{"interface":"front","average":{"1min":2.2429999999999999,"5min":1.4810000000000001,"15min":1.4890000000000001},"min":{"1min":0.40699999999999997,"5min":0.39600000000000002,"15min":0.372},"max":{"1min":17,"5min":17,"15min":34.844999999999999},"last":0.48499999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411622,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":892876,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":891774,"kb_avail":92852276,"statfs":{"total":95995035648,"available":95080730624,"internally_reserved":0,"allocated":1114112,"data_stored":767220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":913176920},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:00:19 2024","interfaces":[{"interface":"back","average":{"1min":8.8200000000000003,"5min":2.5600000000000001,"15min":1.901},"min":{"1min":0.24399999999999999,"5min":0.192,"15min":0.16500000000000001},"max":{"1min":130.703,"5min":130.703,"15min":130.703},"last":0.45200000000000001},{"interface":"front","average":{"1min":8.3330000000000002,"5min":2.5550000000000002,"15min":1.9099999999999999},"min":{"1min":0.21199999999999999,"5min":0.17499999999999999,"15min":0.17399999999999999},"max":{"1min":130.64699999999999,"5min":130.64699999999999,"15min":130.64699999999999},"last":0.39900000000000002}]},{"osd":1,"last update":"Wed Sep 11 09:00:19 2024","interfaces":[{"interface":"back","average":{"1min":4.0179999999999998,"5min":1.6559999999999999,"15min":1.875},"min":{"1min":0.25600000000000001,"5min":0.182,"15min":0.182},"max":{"1min":51.231999999999999,"5min":51.231999999999999,"15min":72.867000000000004},"last":0.33800000000000002},{"interface":"front","average":{"1min":4.5099999999999998,"5min":1.9399999999999999,"15min":2.0059999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20399999999999999},"max":{"1min":47.674999999999997,"5min":47.674999999999997,"15min":67.799000000000007},"last":0.35899999999999999}]},{"osd":3,"last update":"Wed Sep 11 09:00:30 2024","interfaces":[{"interface":"back","average":{"1min":6.5250000000000004,"5min":2.4239999999999999,"15min":2.1440000000000001},"min":{"1min":0.25800000000000001,"5min":0.248,"15min":0.186},"max":{"1min":50.173000000000002,"5min":50.173000000000002,"15min":64.795000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":6.3390000000000004,"5min":2.3050000000000002,"15min":2.056},"min":{"1min":0.23300000000000001,"5min":0.22700000000000001,"15min":0.216},"max":{"1min":49.935000000000002,"5min":49.935000000000002,"15min":64.632999999999996},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:59:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6120000000000001,"5min":1.258,"15min":1.3380000000000001},"min":{"1min":0.32900000000000001,"5min":0.255,"15min":0.255},"max":{"1min":6.298,"5min":7.3650000000000002,"15min":52.113999999999997},"last":0.63},{"interface":"front","average":{"1min":1.097,"5min":1.194,"15min":1.3160000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.27300000000000002},"max":{"1min":3.117,"5min":7.0810000000000004,"15min":52.097999999999999},"last":0.77600000000000002}]},{"osd":5,"last update":"Wed Sep 11 09:00:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.181,"15min":1.331},"min":{"1min":0.311,"5min":0.311,"15min":0.29799999999999999},"max":{"1min":3.5419999999999998,"5min":7.1219999999999999,"15min":52.145000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":1.1259999999999999,"15min":1.1879999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.28399999999999997},"max":{"1min":3.3799999999999999,"5min":6.5880000000000001,"15min":52.216000000000001},"last":0.68200000000000005}]},{"osd":6,"last update":"Wed Sep 11 09:00:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":3.6480000000000001,"15min":2.2450000000000001},"min":{"1min":0.45400000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.2729999999999997,"5min":246.482,"15min":246.482},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.7569999999999999,"5min":3.6739999999999999,"15min":2.1949999999999998},"min":{"1min":0.38200000000000001,"5min":0.34899999999999998,"15min":0.33500000000000002},"max":{"1min":11.725,"5min":246.184,"15min":246.184},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed Sep 11 08:59:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.137,"15min":1.3300000000000001},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.32200000000000001},"max":{"1min":3.5720000000000001,"5min":6.6289999999999996,"15min":84.108999999999995},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.24,"15min":1.532},"min":{"1min":0.37,"5min":0.30599999999999999,"15min":0.30299999999999999},"max":{"1min":6.2699999999999996,"5min":7.415,"15min":85.938000000000002},"last":0.58299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575147,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":904516,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":903678,"kb_avail":92840636,"statfs":{"total":95995035648,"available":95068811264,"internally_reserved":0,"allocated":843776,"data_stored":504154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":925366616},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:59:56 2024","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.4299999999999999,"15min":1.377},"min":{"1min":0.26000000000000001,"5min":0.186,"15min":0.16600000000000001},"max":{"1min":6.1950000000000003,"5min":18.27,"15min":56.613},"last":0.24399999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.456,"15min":1.502},"min":{"1min":0.19,"5min":0.187,"15min":0.17499999999999999},"max":{"1min":6.0289999999999999,"5min":18.210999999999999,"15min":55.533999999999999},"last":0.26300000000000001}]},{"osd":2,"last update":"Wed Sep 11 09:00:26 2024","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.47,"15min":1.9510000000000001},"min":{"1min":0.34999999999999998,"5min":0.20100000000000001,"15min":0.19},"max":{"1min":9.0530000000000008,"5min":12.596,"15min":169.607},"last":0.34300000000000003},{"interface":"front","average":{"1min":1.8029999999999999,"5min":1.29,"15min":1.881},"min":{"1min":0.27700000000000002,"5min":0.20300000000000001,"15min":0.20000000000000001},"max":{"1min":7.7839999999999998,"5min":10.243,"15min":169.762},"last":0.29699999999999999}]},{"osd":3,"last update":"Wed Sep 11 09:00:43 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":2.7610000000000001,"15min":1.742},"min":{"1min":0.23200000000000001,"5min":0.19600000000000001,"15min":0.191},"max":{"1min":5.9989999999999997,"5min":70.478999999999999,"15min":70.478999999999999},"last":0.318},{"interface":"front","average":{"1min":1.4690000000000001,"5min":3.1179999999999999,"15min":1.774},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":3.8849999999999998,"5min":70.337999999999994,"15min":70.337999999999994},"last":0.307}]},{"osd":4,"last update":"Wed Sep 11 08:59:56 2024","interfaces":[{"interface":"back","average":{"1min":1.7230000000000001,"5min":1.0489999999999999,"15min":1.119},"min":{"1min":0.30399999999999999,"5min":0.27900000000000003,"15min":0.27500000000000002},"max":{"1min":7.6680000000000001,"5min":7.6680000000000001,"15min":7.9450000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.234,"15min":1.133},"min":{"1min":0.35499999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.609,"5min":7.8559999999999999,"15min":7.8559999999999999},"last":0.66100000000000003}]},{"osd":5,"last update":"Wed Sep 11 09:00:26 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.5880000000000001,"15min":1.3500000000000001},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.27600000000000002},"max":{"1min":3.77,"5min":38.098999999999997,"15min":38.098999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.694,"5min":1.5569999999999999,"15min":1.3839999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":7.7140000000000004,"5min":38.130000000000003,"15min":38.130000000000003},"last":0.60699999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.343,"15min":1.2330000000000001},"min":{"1min":0.40699999999999997,"5min":0.33800000000000002,"15min":0.30499999999999999},"max":{"1min":4.5549999999999997,"5min":7.8840000000000003,"15min":7.8840000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.208,"15min":1.2290000000000001},"min":{"1min":0.34200000000000003,"5min":0.315,"15min":0.315},"max":{"1min":6.1740000000000004,"5min":6.1740000000000004,"15min":6.2869999999999999},"last":0.72499999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:59:56 2024","interfaces":[{"interface":"back","average":{"1min":1.786,"5min":1.2769999999999999,"15min":1.2},"min":{"1min":0.45000000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":7.7549999999999999,"5min":14.994,"15min":14.994},"last":0.56100000000000005},{"interface":"front","average":{"1min":1.472,"5min":1.3380000000000001,"15min":1.2430000000000001},"min":{"1min":0.35999999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":4.71,"5min":15.721,"15min":15.721},"last":0.71799999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738671,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":906268,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":905086,"kb_avail":92838884,"statfs":{"total":95995035648,"available":95067017216,"internally_reserved":0,"allocated":1196032,"data_stored":832988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":926808408},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:59:54 2024","interfaces":[{"interface":"back","average":{"1min":2.5670000000000002,"5min":1.706,"15min":1.615},"min":{"1min":0.19900000000000001,"5min":0.17399999999999999,"15min":0.16300000000000001},"max":{"1min":30.419,"5min":42.197000000000003,"15min":56.685000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":2.581,"5min":1.7629999999999999,"15min":1.7030000000000001},"min":{"1min":0.20399999999999999,"5min":0.19,"15min":0.188},"max":{"1min":30.388000000000002,"5min":41.822000000000003,"15min":61.012999999999998},"last":0.34499999999999997}]},{"osd":2,"last update":"Wed Sep 11 09:00:21 2024","interfaces":[{"interface":"back","average":{"1min":2.008,"5min":2.5710000000000002,"15min":1.611},"min":{"1min":0.23699999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":22.585000000000001,"5min":111.41200000000001,"15min":111.41200000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":2.1579999999999999,"5min":2.7850000000000001,"15min":1.671},"min":{"1min":0.23799999999999999,"5min":0.20100000000000001,"15min":0.20000000000000001},"max":{"1min":22.611999999999998,"5min":111.637,"15min":111.637},"last":0.30199999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:59:47 2024","interfaces":[{"interface":"back","average":{"1min":2.9740000000000002,"5min":2.1389999999999998,"15min":1.7549999999999999},"min":{"1min":0.28000000000000003,"5min":0.22900000000000001,"15min":0.19800000000000001},"max":{"1min":22.859999999999999,"5min":41.131999999999998,"15min":56.095999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":2.3559999999999999,"5min":1.9330000000000001,"15min":1.667},"min":{"1min":0.24199999999999999,"5min":0.23999999999999999,"15min":0.20899999999999999},"max":{"1min":22.782,"5min":41.173999999999999,"15min":59.651000000000003},"last":0.438}]},{"osd":4,"last update":"Wed Sep 11 08:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.522,"15min":1.804},"min":{"1min":0.30499999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":5.2119999999999997,"5min":35.066000000000003,"15min":168.72200000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":1.6679999999999999,"15min":1.8500000000000001},"min":{"1min":0.38500000000000001,"5min":0.30099999999999999,"15min":0.27600000000000002},"max":{"1min":1.8,"5min":34.652000000000001,"15min":168.88999999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Wed Sep 11 09:00:21 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.1970000000000001,"15min":1.2330000000000001},"min":{"1min":0.317,"5min":0.307,"15min":0.30599999999999999},"max":{"1min":3.3679999999999999,"5min":7.2649999999999997,"15min":7.7830000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":1.032,"15min":1.232},"min":{"1min":0.40600000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.8939999999999999,"5min":5.2359999999999998,"15min":9.5239999999999991},"last":0.65900000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.714,"5min":1.6000000000000001,"15min":1.397},"min":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.30099999999999999},"max":{"1min":7.2050000000000001,"5min":25.027000000000001,"15min":25.027000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":1.034,"5min":1.6200000000000001,"15min":1.3939999999999999},"min":{"1min":0.40400000000000003,"5min":0.311,"15min":0.29399999999999998},"max":{"1min":5.2569999999999997,"5min":25.076000000000001,"15min":25.076000000000001},"last":0.61899999999999999}]},{"osd":7,"last update":"Wed Sep 11 09:00:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":1.29,"15min":1.494},"min":{"1min":0.45000000000000001,"5min":0.33300000000000002,"15min":0.32700000000000001},"max":{"1min":2.008,"5min":20.399000000000001,"15min":56.119},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.482,"5min":1.5,"15min":1.677},"min":{"1min":0.46800000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":4.9429999999999996,"5min":20.286999999999999,"15min":56.100000000000001},"last":0.53000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:00:51.161 INFO:tasks.thrashosds.thrasher:Setting 3.1 to [6, 4, 0] 2024-09-11T09:00:51.161 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1', '6', '4', '0'] 2024-09-11T09:00:51.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pg-upmap 3.1 6 4 0 2024-09-11T09:00:51.245 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:51.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:51.271 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:51.271 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:51.372 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:51.396 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:51.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:51.397 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:51.498 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:51.522 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:51.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:51.523 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:51.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:51.646 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:51.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:51.647 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:51.704 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:00:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:00:51] "GET /metrics HTTP/1.1" 200 224955 "" "Prometheus/2.43.0" 2024-09-11T09:00:51.704 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:51 smithi092 ceph-mon[29647]: from='client.25910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T09:00:51.704 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:51 smithi092 ceph-mon[29647]: 6.7 scrub starts 2024-09-11T09:00:51.705 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:51 smithi092 ceph-mon[29647]: 6.7 scrub ok 2024-09-11T09:00:51.705 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:51 smithi092 ceph-mon[29647]: 5.1c scrub starts 2024-09-11T09:00:51.705 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:51 smithi092 ceph-mon[29647]: 5.1c scrub ok 2024-09-11T09:00:51.705 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:51 smithi092 ceph-mon[41114]: from='client.25910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T09:00:51.705 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:51 smithi092 ceph-mon[41114]: 6.7 scrub starts 2024-09-11T09:00:51.705 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:51 smithi092 ceph-mon[41114]: 6.7 scrub ok 2024-09-11T09:00:51.705 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:51 smithi092 ceph-mon[41114]: 5.1c scrub starts 2024-09-11T09:00:51.705 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:51 smithi092 ceph-mon[41114]: 5.1c scrub ok 2024-09-11T09:00:51.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:51.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:51.770 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:51.770 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:51.871 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:51.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:51.894 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:51.894 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:51.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:51 smithi145 ceph-mon[33384]: from='client.25910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T09:00:51.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:51 smithi145 ceph-mon[33384]: 6.7 scrub starts 2024-09-11T09:00:51.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:51 smithi145 ceph-mon[33384]: 6.7 scrub ok 2024-09-11T09:00:51.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:51 smithi145 ceph-mon[33384]: 5.1c scrub starts 2024-09-11T09:00:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:51 smithi145 ceph-mon[33384]: 5.1c scrub ok 2024-09-11T09:00:51.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:52.016 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:52.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:52.017 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:52.118 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:52.144 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:52.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:52.145 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:52.246 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:52.271 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:52.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:52.273 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:52.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:52.396 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:52.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:52.397 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:52.497 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:52.518 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:52.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:52.519 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:52.579 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'unique_pool_4' removed 2024-09-11T09:00:52.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:52.658 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:52.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:52.659 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:52.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1 2024-09-11T09:00:52.782 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:52.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:52.782 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.1' 2024-09-11T09:00:52.880 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[29647]: 1.10 scrub starts 2024-09-11T09:00:52.880 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[29647]: 1.10 scrub ok 2024-09-11T09:00:52.880 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[29647]: pgmap v1009: 168 pgs: 168 active+clean; 421 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:00:52.880 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[29647]: 5.1e scrub starts 2024-09-11T09:00:52.880 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[29647]: 5.1e scrub ok 2024-09-11T09:00:52.880 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[29647]: 4.b scrub starts 2024-09-11T09:00:52.880 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[29647]: 4.b scrub ok 2024-09-11T09:00:52.880 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[29647]: 6.11 scrub starts 2024-09-11T09:00:52.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[29647]: 6.11 scrub ok 2024-09-11T09:00:52.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[29647]: 1.b scrub starts 2024-09-11T09:00:52.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[29647]: 1.b scrub ok 2024-09-11T09:00:52.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/885257334' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T09:00:52.881 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T09:00:52.881 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[41114]: 1.10 scrub starts 2024-09-11T09:00:52.881 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[41114]: 1.10 scrub ok 2024-09-11T09:00:52.881 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[41114]: pgmap v1009: 168 pgs: 168 active+clean; 421 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:00:52.881 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[41114]: 5.1e scrub starts 2024-09-11T09:00:52.881 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[41114]: 5.1e scrub ok 2024-09-11T09:00:52.881 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[41114]: 4.b scrub starts 2024-09-11T09:00:52.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[41114]: 4.b scrub ok 2024-09-11T09:00:52.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[41114]: 6.11 scrub starts 2024-09-11T09:00:52.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[41114]: 6.11 scrub ok 2024-09-11T09:00:52.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[41114]: 1.b scrub starts 2024-09-11T09:00:52.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[41114]: 1.b scrub ok 2024-09-11T09:00:52.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/885257334' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T09:00:52.882 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:52 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T09:00:52.883 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:52.905 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:52.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:52.906 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:52.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:00:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:00:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:00:52.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:52 smithi145 ceph-mon[33384]: 1.10 scrub starts 2024-09-11T09:00:52.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:52 smithi145 ceph-mon[33384]: 1.10 scrub ok 2024-09-11T09:00:52.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:52 smithi145 ceph-mon[33384]: pgmap v1009: 168 pgs: 168 active+clean; 421 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:00:52.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:52 smithi145 ceph-mon[33384]: 5.1e scrub starts 2024-09-11T09:00:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:52 smithi145 ceph-mon[33384]: 5.1e scrub ok 2024-09-11T09:00:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:52 smithi145 ceph-mon[33384]: 4.b scrub starts 2024-09-11T09:00:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:52 smithi145 ceph-mon[33384]: 4.b scrub ok 2024-09-11T09:00:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:52 smithi145 ceph-mon[33384]: 6.11 scrub starts 2024-09-11T09:00:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:52 smithi145 ceph-mon[33384]: 6.11 scrub ok 2024-09-11T09:00:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:52 smithi145 ceph-mon[33384]: 1.b scrub starts 2024-09-11T09:00:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:52 smithi145 ceph-mon[33384]: 1.b scrub ok 2024-09-11T09:00:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:52 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/885257334' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T09:00:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:52 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T09:00:53.007 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:53.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:53.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:53.105 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-11T09:00:53.106 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:00:53.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:53.160 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:53.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:53.161 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:53.262 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:53.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:53.284 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:53.285 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:53.385 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4 2024-09-11T09:00:53.411 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:53.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:53.412 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.4' 2024-09-11T09:00:53.513 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:53.535 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:53.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:53.536 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:53.636 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:53.659 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:53.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:53.660 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:53.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:53.783 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:53.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:53.784 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:53.885 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:53.908 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:53.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:53.910 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:53.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:53 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/885257334' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T09:00:53.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:53 smithi145 ceph-mon[33384]: osdmap e428: 8 total, 8 up, 8 in 2024-09-11T09:00:53.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:53 smithi145 ceph-mon[33384]: 1.5 deep-scrub starts 2024-09-11T09:00:53.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:53 smithi145 ceph-mon[33384]: 1.5 deep-scrub ok 2024-09-11T09:00:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:53 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T09:00:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:53 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T09:00:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:53 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:00:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:53 smithi145 ceph-mon[33384]: 5.c scrub starts 2024-09-11T09:00:53.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:53 smithi145 ceph-mon[33384]: 5.c scrub ok 2024-09-11T09:00:54.011 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7 2024-09-11T09:00:54.034 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:54.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:54.035 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.7' 2024-09-11T09:00:54.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/885257334' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T09:00:54.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[41114]: osdmap e428: 8 total, 8 up, 8 in 2024-09-11T09:00:54.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[41114]: 1.5 deep-scrub starts 2024-09-11T09:00:54.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[41114]: 1.5 deep-scrub ok 2024-09-11T09:00:54.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T09:00:54.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T09:00:54.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:00:54.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[41114]: 5.c scrub starts 2024-09-11T09:00:54.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[41114]: 5.c scrub ok 2024-09-11T09:00:54.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/885257334' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T09:00:54.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[29647]: osdmap e428: 8 total, 8 up, 8 in 2024-09-11T09:00:54.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[29647]: 1.5 deep-scrub starts 2024-09-11T09:00:54.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[29647]: 1.5 deep-scrub ok 2024-09-11T09:00:54.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T09:00:54.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T09:00:54.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:00:54.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[29647]: 5.c scrub starts 2024-09-11T09:00:54.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:53 smithi092 ceph-mon[29647]: 5.c scrub ok 2024-09-11T09:00:54.136 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5 2024-09-11T09:00:54.159 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:54.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:54.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.5' 2024-09-11T09:00:54.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-11T09:00:54.217 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:54.260 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:54.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:54.284 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:54.285 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:54.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:54.407 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:54.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:54.408 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:54.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:54.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:54.532 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:54.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0 2024-09-11T09:00:54.654 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:54.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:54.655 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.0' 2024-09-11T09:00:54.733 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:54 smithi092 ceph-mon[41114]: 1.12 scrub starts 2024-09-11T09:00:54.733 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:54 smithi092 ceph-mon[41114]: 1.12 scrub ok 2024-09-11T09:00:54.733 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:54 smithi092 ceph-mon[41114]: pgmap v1011: 152 pgs: 152 active+clean; 395 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-11T09:00:54.733 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:54 smithi092 ceph-mon[41114]: 3.2 scrub starts 2024-09-11T09:00:54.733 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:54 smithi092 ceph-mon[41114]: 3.2 scrub ok 2024-09-11T09:00:54.733 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:54 smithi092 ceph-mon[29647]: 1.12 scrub starts 2024-09-11T09:00:54.733 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:54 smithi092 ceph-mon[29647]: 1.12 scrub ok 2024-09-11T09:00:54.733 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:54 smithi092 ceph-mon[29647]: pgmap v1011: 152 pgs: 152 active+clean; 395 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-11T09:00:54.733 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:54 smithi092 ceph-mon[29647]: 3.2 scrub starts 2024-09-11T09:00:54.733 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:54 smithi092 ceph-mon[29647]: 3.2 scrub ok 2024-09-11T09:00:54.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:54.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:54.778 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:54.879 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2 2024-09-11T09:00:54.900 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:54.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:54.901 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.2' 2024-09-11T09:00:54.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:54 smithi145 ceph-mon[33384]: 1.12 scrub starts 2024-09-11T09:00:54.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:54 smithi145 ceph-mon[33384]: 1.12 scrub ok 2024-09-11T09:00:54.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:54 smithi145 ceph-mon[33384]: pgmap v1011: 152 pgs: 152 active+clean; 395 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-11T09:00:54.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:54 smithi145 ceph-mon[33384]: 3.2 scrub starts 2024-09-11T09:00:54.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:54 smithi145 ceph-mon[33384]: 3.2 scrub ok 2024-09-11T09:00:55.002 DEBUG:teuthology.orchestra.run.smithi145:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6 2024-09-11T09:00:55.028 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: docker: command not found 2024-09-11T09:00:55.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:55.029 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi145 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.6' 2024-09-11T09:00:55.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:55.152 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:55.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:55.153 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:55.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3 2024-09-11T09:00:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: docker: command not found 2024-09-11T09:00:55.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:00:55.276 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi092 with status 1: 'sudo docker kill -s 1 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd.3' 2024-09-11T09:00:55.286 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-11T09:00:55.297 INFO:tasks.thrashosds:joining thrashosds 2024-09-11T09:00:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:55 smithi145 ceph-mon[33384]: 6.15 deep-scrub starts 2024-09-11T09:00:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:55 smithi145 ceph-mon[33384]: 6.15 deep-scrub ok 2024-09-11T09:00:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:55 smithi145 ceph-mon[33384]: 6.b scrub starts 2024-09-11T09:00:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:55 smithi145 ceph-mon[33384]: 6.b scrub ok 2024-09-11T09:00:55.945 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:55 smithi092 ceph-mon[29647]: 6.15 deep-scrub starts 2024-09-11T09:00:55.945 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:55 smithi092 ceph-mon[29647]: 6.15 deep-scrub ok 2024-09-11T09:00:55.945 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:55 smithi092 ceph-mon[29647]: 6.b scrub starts 2024-09-11T09:00:55.945 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:55 smithi092 ceph-mon[29647]: 6.b scrub ok 2024-09-11T09:00:55.945 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:55 smithi092 ceph-mon[41114]: 6.15 deep-scrub starts 2024-09-11T09:00:55.945 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:55 smithi092 ceph-mon[41114]: 6.15 deep-scrub ok 2024-09-11T09:00:55.945 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:55 smithi092 ceph-mon[41114]: 6.b scrub starts 2024-09-11T09:00:55.946 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:55 smithi092 ceph-mon[41114]: 6.b scrub ok 2024-09-11T09:00:56.008 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-11T09:00:56.008 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-11T09:00:56.008 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-11T09:00:56.008 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-11T09:00:56.008 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-11T09:00:56.008 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:00:56.008 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:00:56.009 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:00:56.009 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:00:56.009 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:00:56.021 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-11T09:00:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:00:56.037 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-11T09:00:56.037 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T09:00:56.053 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-11T09:00:56.053 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T09:00:56.871 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:56 smithi092 ceph-mon[29647]: pgmap v1012: 152 pgs: 152 active+clean; 395 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:00:56.871 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:56 smithi092 ceph-mon[29647]: 3.13 scrub starts 2024-09-11T09:00:56.872 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:56 smithi092 ceph-mon[29647]: 3.13 scrub ok 2024-09-11T09:00:56.872 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:56 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2829502184' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T09:00:56.872 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:56 smithi092 ceph-mon[29647]: 4.f scrub starts 2024-09-11T09:00:56.872 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:56 smithi092 ceph-mon[29647]: 4.f scrub ok 2024-09-11T09:00:56.872 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:56 smithi092 ceph-mon[41114]: pgmap v1012: 152 pgs: 152 active+clean; 395 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:00:56.872 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:56 smithi092 ceph-mon[41114]: 3.13 scrub starts 2024-09-11T09:00:56.872 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:56 smithi092 ceph-mon[41114]: 3.13 scrub ok 2024-09-11T09:00:56.872 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:56 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2829502184' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T09:00:56.872 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:56 smithi092 ceph-mon[41114]: 4.f scrub starts 2024-09-11T09:00:56.872 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:56 smithi092 ceph-mon[41114]: 4.f scrub ok 2024-09-11T09:00:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:56 smithi145 ceph-mon[33384]: pgmap v1012: 152 pgs: 152 active+clean; 395 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:00:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:56 smithi145 ceph-mon[33384]: 3.13 scrub starts 2024-09-11T09:00:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:56 smithi145 ceph-mon[33384]: 3.13 scrub ok 2024-09-11T09:00:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:56 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2829502184' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T09:00:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:56 smithi145 ceph-mon[33384]: 4.f scrub starts 2024-09-11T09:00:56.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:56 smithi145 ceph-mon[33384]: 4.f scrub ok 2024-09-11T09:00:57.001 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:57.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:00:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:00:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:57 smithi145 ceph-mon[33384]: 1.f scrub starts 2024-09-11T09:00:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:57 smithi145 ceph-mon[33384]: 1.f scrub ok 2024-09-11T09:00:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:57 smithi145 ceph-mon[33384]: 5.1a scrub starts 2024-09-11T09:00:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:57 smithi145 ceph-mon[33384]: 5.1a scrub ok 2024-09-11T09:00:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:57 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:00:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:57 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:00:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:57 smithi145 ceph-mon[33384]: 1.6 deep-scrub starts 2024-09-11T09:00:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:57 smithi145 ceph-mon[33384]: 1.6 deep-scrub ok 2024-09-11T09:00:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:57 smithi145 ceph-mon[33384]: 5.5 scrub starts 2024-09-11T09:00:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:57 smithi145 ceph-mon[33384]: 5.5 scrub ok 2024-09-11T09:00:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:57 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:00:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:57 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:00:57.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[29647]: 1.f scrub starts 2024-09-11T09:00:57.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[29647]: 1.f scrub ok 2024-09-11T09:00:57.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[29647]: 5.1a scrub starts 2024-09-11T09:00:57.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[29647]: 5.1a scrub ok 2024-09-11T09:00:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:00:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:00:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[29647]: 1.6 deep-scrub starts 2024-09-11T09:00:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[29647]: 1.6 deep-scrub ok 2024-09-11T09:00:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[29647]: 5.5 scrub starts 2024-09-11T09:00:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[29647]: 5.5 scrub ok 2024-09-11T09:00:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:00:57.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:00:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[41114]: 1.f scrub starts 2024-09-11T09:00:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[41114]: 1.f scrub ok 2024-09-11T09:00:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[41114]: 5.1a scrub starts 2024-09-11T09:00:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[41114]: 5.1a scrub ok 2024-09-11T09:00:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:00:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:00:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[41114]: 1.6 deep-scrub starts 2024-09-11T09:00:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[41114]: 1.6 deep-scrub ok 2024-09-11T09:00:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[41114]: 5.5 scrub starts 2024-09-11T09:00:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[41114]: 5.5 scrub ok 2024-09-11T09:00:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:00:57.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:57 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:00:58.520 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:00:58.626 INFO:teuthology.orchestra.run.smithi092.stderr:set 3.1 pg_upmap mapping to [6,4,0] 2024-09-11T09:00:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:58 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:58 smithi145 ceph-mon[33384]: pgmap v1013: 152 pgs: 152 active+clean; 369 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:00:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:58 smithi145 ceph-mon[33384]: osdmap e429: 8 total, 8 up, 8 in 2024-09-11T09:00:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:58 smithi145 ceph-mon[33384]: 3.14 scrub starts 2024-09-11T09:00:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:58 smithi145 ceph-mon[33384]: 3.14 scrub ok 2024-09-11T09:00:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:58 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2560415133' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.1", "id": [6, 4, 0]}]: dispatch 2024-09-11T09:00:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:58 smithi145 ceph-mon[33384]: 1.7 scrub starts 2024-09-11T09:00:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:58 smithi145 ceph-mon[33384]: 1.7 scrub ok 2024-09-11T09:00:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:58 smithi145 ceph-mon[33384]: 1.13 scrub starts 2024-09-11T09:00:58.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:58 smithi145 ceph-mon[33384]: 1.13 scrub ok 2024-09-11T09:00:58.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:58 smithi145 ceph-mon[33384]: 4.4 scrub starts 2024-09-11T09:00:58.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:58 smithi145 ceph-mon[33384]: 4.4 scrub ok 2024-09-11T09:00:59.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:59.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[29647]: pgmap v1013: 152 pgs: 152 active+clean; 369 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:00:59.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[29647]: osdmap e429: 8 total, 8 up, 8 in 2024-09-11T09:00:59.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[29647]: 3.14 scrub starts 2024-09-11T09:00:59.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[29647]: 3.14 scrub ok 2024-09-11T09:00:59.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2560415133' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.1", "id": [6, 4, 0]}]: dispatch 2024-09-11T09:00:59.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[29647]: 1.7 scrub starts 2024-09-11T09:00:59.063 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[29647]: 1.7 scrub ok 2024-09-11T09:00:59.063 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[29647]: 1.13 scrub starts 2024-09-11T09:00:59.063 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[29647]: 1.13 scrub ok 2024-09-11T09:00:59.063 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[29647]: 4.4 scrub starts 2024-09-11T09:00:59.063 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[29647]: 4.4 scrub ok 2024-09-11T09:00:59.063 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:00:59.063 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[41114]: pgmap v1013: 152 pgs: 152 active+clean; 369 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:00:59.063 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[41114]: osdmap e429: 8 total, 8 up, 8 in 2024-09-11T09:00:59.063 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[41114]: 3.14 scrub starts 2024-09-11T09:00:59.063 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[41114]: 3.14 scrub ok 2024-09-11T09:00:59.064 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2560415133' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.1", "id": [6, 4, 0]}]: dispatch 2024-09-11T09:00:59.064 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[41114]: 1.7 scrub starts 2024-09-11T09:00:59.064 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[41114]: 1.7 scrub ok 2024-09-11T09:00:59.064 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[41114]: 1.13 scrub starts 2024-09-11T09:00:59.064 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[41114]: 1.13 scrub ok 2024-09-11T09:00:59.064 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[41114]: 4.4 scrub starts 2024-09-11T09:00:59.064 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:58 smithi092 ceph-mon[41114]: 4.4 scrub ok 2024-09-11T09:00:59.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:59 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2560415133' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.1", "id": [6, 4, 0]}]': finished 2024-09-11T09:00:59.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:59 smithi092 ceph-mon[29647]: osdmap e430: 8 total, 8 up, 8 in 2024-09-11T09:00:59.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:59 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T09:00:59.765 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:00:59 smithi092 ceph-mon[29647]: 5.8 scrub ok 2024-09-11T09:00:59.765 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:59 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2560415133' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.1", "id": [6, 4, 0]}]': finished 2024-09-11T09:00:59.765 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:59 smithi092 ceph-mon[41114]: osdmap e430: 8 total, 8 up, 8 in 2024-09-11T09:00:59.765 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:59 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T09:00:59.765 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:00:59 smithi092 ceph-mon[41114]: 5.8 scrub ok 2024-09-11T09:00:59.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:59 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2560415133' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.1", "id": [6, 4, 0]}]': finished 2024-09-11T09:00:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:59 smithi145 ceph-mon[33384]: osdmap e430: 8 total, 8 up, 8 in 2024-09-11T09:00:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:59 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T09:00:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:00:59 smithi145 ceph-mon[33384]: 5.8 scrub ok 2024-09-11T09:01:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:00 smithi145 ceph-mon[33384]: pgmap v1016: 152 pgs: 152 active+clean; 369 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:01:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:00 smithi145 ceph-mon[33384]: osdmap e431: 8 total, 8 up, 8 in 2024-09-11T09:01:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:00 smithi145 ceph-mon[33384]: 1.c scrub starts 2024-09-11T09:01:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:00 smithi145 ceph-mon[33384]: 1.c scrub ok 2024-09-11T09:01:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:00 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:01:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:01:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:00 smithi145 ceph-mon[33384]: 5.b scrub starts 2024-09-11T09:01:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:00 smithi145 ceph-mon[33384]: 5.b scrub ok 2024-09-11T09:01:00.958 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[29647]: pgmap v1016: 152 pgs: 152 active+clean; 369 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:01:00.958 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[29647]: osdmap e431: 8 total, 8 up, 8 in 2024-09-11T09:01:00.958 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[29647]: 1.c scrub starts 2024-09-11T09:01:00.959 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[29647]: 1.c scrub ok 2024-09-11T09:01:00.959 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:01:00.959 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:01:00.959 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[29647]: 5.b scrub starts 2024-09-11T09:01:00.959 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[29647]: 5.b scrub ok 2024-09-11T09:01:00.959 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[41114]: pgmap v1016: 152 pgs: 152 active+clean; 369 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:01:00.959 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[41114]: osdmap e431: 8 total, 8 up, 8 in 2024-09-11T09:01:00.959 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[41114]: 1.c scrub starts 2024-09-11T09:01:00.959 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[41114]: 1.c scrub ok 2024-09-11T09:01:00.959 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:01:00.959 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:01:00.960 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[41114]: 5.b scrub starts 2024-09-11T09:01:00.960 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:00 smithi092 ceph-mon[41114]: 5.b scrub ok 2024-09-11T09:01:01.555 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:01:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:01:01] "GET /metrics HTTP/1.1" 200 222528 "" "Prometheus/2.43.0" 2024-09-11T09:01:01.638 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-11T09:01:01.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:01 smithi092 ceph-mon[29647]: 3.17 scrub starts 2024-09-11T09:01:01.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:01 smithi092 ceph-mon[29647]: 3.17 scrub ok 2024-09-11T09:01:01.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:01 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/901404712' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T09:01:01.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:01 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T09:01:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:01 smithi092 ceph-mon[41114]: 3.17 scrub starts 2024-09-11T09:01:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:01 smithi092 ceph-mon[41114]: 3.17 scrub ok 2024-09-11T09:01:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:01 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/901404712' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T09:01:01.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:01 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T09:01:01.824 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T09:01:01.824 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T09:01:01.825 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-11T09:01:01.825 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T09:01:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:01 smithi145 ceph-mon[33384]: 3.17 scrub starts 2024-09-11T09:01:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:01 smithi145 ceph-mon[33384]: 3.17 scrub ok 2024-09-11T09:01:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:01 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/901404712' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T09:01:01.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:01 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T09:01:02.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:01:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:01:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[29647]: 4.8 scrub starts 2024-09-11T09:01:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[29647]: 4.8 scrub ok 2024-09-11T09:01:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[29647]: pgmap v1018: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-11T09:01:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[29647]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T09:01:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T09:01:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[29647]: osdmap e432: 8 total, 8 up, 8 in 2024-09-11T09:01:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:01:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:01:02.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[29647]: 6.9 scrub starts 2024-09-11T09:01:02.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:01:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:01:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[41114]: 4.8 scrub starts 2024-09-11T09:01:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[41114]: 4.8 scrub ok 2024-09-11T09:01:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[41114]: pgmap v1018: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-11T09:01:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[41114]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T09:01:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T09:01:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[41114]: osdmap e432: 8 total, 8 up, 8 in 2024-09-11T09:01:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:01:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:01:02.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:02 smithi092 ceph-mon[41114]: 6.9 scrub starts 2024-09-11T09:01:02.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-11T09:01:02.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:01:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:01:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:01:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:02 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:01:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:02 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:01:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:02 smithi145 ceph-mon[33384]: 4.8 scrub starts 2024-09-11T09:01:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:02 smithi145 ceph-mon[33384]: 4.8 scrub ok 2024-09-11T09:01:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:02 smithi145 ceph-mon[33384]: pgmap v1018: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-11T09:01:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:02 smithi145 ceph-mon[33384]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T09:01:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:02 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T09:01:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:02 smithi145 ceph-mon[33384]: osdmap e432: 8 total, 8 up, 8 in 2024-09-11T09:01:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:02 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:01:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:02 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:01:02.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:02 smithi145 ceph-mon[33384]: 6.9 scrub starts 2024-09-11T09:01:03.560 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:03.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:03 smithi092 ceph-mon[29647]: osdmap e433: 8 total, 8 up, 8 in 2024-09-11T09:01:03.681 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:03 smithi092 ceph-mon[41114]: osdmap e433: 8 total, 8 up, 8 in 2024-09-11T09:01:03.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:03 smithi145 ceph-mon[33384]: osdmap e433: 8 total, 8 up, 8 in 2024-09-11T09:01:04.529 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-11T09:01:04.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd pool set .mgr pgp_num 21 2024-09-11T09:01:04.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:04 smithi145 ceph-mon[33384]: pgmap v1021: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T09:01:04.921 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:04 smithi092 ceph-mon[29647]: pgmap v1021: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T09:01:04.922 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:04 smithi092 ceph-mon[41114]: pgmap v1021: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T09:01:05.067 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:05.374 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T09:01:05.374 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-11T09:01:05.374 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-11T09:01:05.374 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-11T09:01:05.374 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T09:01:05.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T09:01:05.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-11T09:01:06.796 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:06.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:06 smithi092 ceph-mon[29647]: pgmap v1022: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T09:01:06.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:06 smithi092 ceph-mon[41114]: pgmap v1022: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T09:01:06.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:06 smithi145 ceph-mon[33384]: pgmap v1022: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T09:01:07.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:01:07 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:01:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:set pool 1 pgp_num to 21 2024-09-11T09:01:08.682 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:08.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:08 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:08 smithi145 ceph-mon[33384]: pgmap v1023: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T09:01:08.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:08 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2969350626' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"}]: dispatch 2024-09-11T09:01:08.972 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:08 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:08.972 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:08 smithi092 ceph-mon[29647]: pgmap v1023: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T09:01:08.972 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:08 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2969350626' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"}]: dispatch 2024-09-11T09:01:08.973 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:08 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:08.973 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:08 smithi092 ceph-mon[41114]: pgmap v1023: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T09:01:08.973 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:08 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2969350626' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"}]: dispatch 2024-09-11T09:01:09.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-11T09:01:09.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:09 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2969350626' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"}]': finished 2024-09-11T09:01:09.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:09 smithi092 ceph-mon[29647]: osdmap e434: 8 total, 8 up, 8 in 2024-09-11T09:01:09.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:09 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2969350626' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"}]': finished 2024-09-11T09:01:09.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:09 smithi092 ceph-mon[41114]: osdmap e434: 8 total, 8 up, 8 in 2024-09-11T09:01:09.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:09 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2969350626' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"}]': finished 2024-09-11T09:01:09.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:09 smithi145 ceph-mon[33384]: osdmap e434: 8 total, 8 up, 8 in 2024-09-11T09:01:10.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:10 smithi145 ceph-mon[33384]: pgmap v1025: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 650 B/s rd, 0 op/s 2024-09-11T09:01:11.049 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:10 smithi092 ceph-mon[41114]: pgmap v1025: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 650 B/s rd, 0 op/s 2024-09-11T09:01:11.050 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:10 smithi092 ceph-mon[29647]: pgmap v1025: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 650 B/s rd, 0 op/s 2024-09-11T09:01:11.101 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:11.633 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:01:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:01:11] "GET /metrics HTTP/1.1" 200 222528 "" "Prometheus/2.43.0" 2024-09-11T09:01:12.142 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-11T09:01:12.142 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-11T09:01:12.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-11T09:01:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-11T09:01:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272348 9.8 9:15db9131:::benchmark_data_smithi092_145083_object128950:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T09:01:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.641892+0000", 2024-09-11T09:01:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 527.29444370900001, 2024-09-11T09:01:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98679250600000001, 2024-09-11T09:01:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T09:01:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T09:01:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272348 2024-09-11T09:01:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641892+0000", 2024-09-11T09:01:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641892+0000", 2024-09-11T09:01:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641890+0000", 2024-09-11T09:01:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T09:01:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641898+0000", 2024-09-11T09:01:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7849999999999992e-06 2024-09-11T09:01:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641915+0000", 2024-09-11T09:01:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7677e-05 2024-09-11T09:01:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641923+0000", 2024-09-11T09:01:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6199999999999999e-06 2024-09-11T09:01:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641981+0000", 2024-09-11T09:01:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7901e-05 2024-09-11T09:01:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T09:01:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.641989+0000", 2024-09-11T09:01:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3040000000000005e-06 2024-09-11T09:01:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.624095+0000", 2024-09-11T09:01:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.982105699 2024-09-11T09:01:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.624185+0000", 2024-09-11T09:01:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0476999999999994e-05 2024-09-11T09:01:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T09:01:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.624299+0000", 2024-09-11T09:01:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113366 2024-09-11T09:01:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628589+0000", 2024-09-11T09:01:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004290622 2024-09-11T09:01:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628628+0000", 2024-09-11T09:01:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8549e-05 2024-09-11T09:01:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628633+0000", 2024-09-11T09:01:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1059999999999999e-06 2024-09-11T09:01:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628662+0000", 2024-09-11T09:01:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8758000000000001e-05 2024-09-11T09:01:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628684+0000", 2024-09-11T09:01:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2416999999999999e-05 2024-09-11T09:01:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272383 9.8 9:12173f2a:::benchmark_data_smithi092_145083_object128985:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T09:01:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.665664+0000", 2024-09-11T09:01:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 527.27067181699999, 2024-09-11T09:01:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.96324268999999996, 2024-09-11T09:01:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T09:01:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T09:01:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272383 2024-09-11T09:01:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665664+0000", 2024-09-11T09:01:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665664+0000", 2024-09-11T09:01:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665663+0000", 2024-09-11T09:01:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T09:01:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665668+0000", 2024-09-11T09:01:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9049999999999996e-06 2024-09-11T09:01:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665678+0000", 2024-09-11T09:01:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0057e-05 2024-09-11T09:01:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665683+0000", 2024-09-11T09:01:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:01:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665724+0000", 2024-09-11T09:01:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000003e-05 2024-09-11T09:01:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T09:01:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.665731+0000", 2024-09-11T09:01:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6030000000000001e-06 2024-09-11T09:01:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.626022+0000", 2024-09-11T09:01:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.96029152699999998 2024-09-11T09:01:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.626136+0000", 2024-09-11T09:01:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114009 2024-09-11T09:01:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T09:01:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.626246+0000", 2024-09-11T09:01:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010950000000000001 2024-09-11T09:01:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628734+0000", 2024-09-11T09:01:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024882149999999998 2024-09-11T09:01:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628762+0000", 2024-09-11T09:01:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8354e-05 2024-09-11T09:01:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628852+0000", 2024-09-11T09:01:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9708000000000002e-05 2024-09-11T09:01:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628888+0000", 2024-09-11T09:01:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6128999999999999e-05 2024-09-11T09:01:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628906+0000", 2024-09-11T09:01:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8218000000000001e-05 2024-09-11T09:01:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272428 9.8 9:14391abd:::benchmark_data_smithi092_145083_object129030:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T09:01:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.737328+0000", 2024-09-11T09:01:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 527.19900728499999, 2024-09-11T09:01:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.89314801799999999, 2024-09-11T09:01:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T09:01:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T09:01:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272428 2024-09-11T09:01:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737328+0000", 2024-09-11T09:01:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737328+0000", 2024-09-11T09:01:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737327+0000", 2024-09-11T09:01:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T09:01:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737333+0000", 2024-09-11T09:01:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6720000000000001e-06 2024-09-11T09:01:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737345+0000", 2024-09-11T09:01:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2007999999999999e-05 2024-09-11T09:01:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737353+0000", 2024-09-11T09:01:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0930000000000003e-06 2024-09-11T09:01:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737385+0000", 2024-09-11T09:01:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2394999999999997e-05 2024-09-11T09:01:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T09:01:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.737391+0000", 2024-09-11T09:01:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1800000000000004e-06 2024-09-11T09:01:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.627276+0000", 2024-09-11T09:01:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.88988573100000001 2024-09-11T09:01:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.627371+0000", 2024-09-11T09:01:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4772000000000007e-05 2024-09-11T09:01:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T09:01:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.627432+0000", 2024-09-11T09:01:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1106999999999996e-05 2024-09-11T09:01:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628813+0000", 2024-09-11T09:01:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001381195 2024-09-11T09:01:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628822+0000", 2024-09-11T09:01:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5169999999999999e-06 2024-09-11T09:01:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630397+0000", 2024-09-11T09:01:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001574717 2024-09-11T09:01:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630443+0000", 2024-09-11T09:01:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6513e-05 2024-09-11T09:01:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630476+0000", 2024-09-11T09:01:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3039e-05 2024-09-11T09:01:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272436 9.8 9:16049abc:::benchmark_data_smithi092_145083_object129038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T09:01:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.749067+0000", 2024-09-11T09:01:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 527.18726862100004, 2024-09-11T09:01:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.88488331399999998, 2024-09-11T09:01:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T09:01:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T09:01:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272436 2024-09-11T09:01:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749067+0000", 2024-09-11T09:01:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749067+0000", 2024-09-11T09:01:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749065+0000", 2024-09-11T09:01:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999981 2024-09-11T09:01:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749073+0000", 2024-09-11T09:01:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9009999999999994e-06 2024-09-11T09:01:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749094+0000", 2024-09-11T09:01:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1129999999999999e-05 2024-09-11T09:01:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749101+0000", 2024-09-11T09:01:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3699999999999997e-06 2024-09-11T09:01:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749220+0000", 2024-09-11T09:01:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118566 2024-09-11T09:01:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T09:01:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.749229+0000", 2024-09-11T09:01:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1370000000000008e-06 2024-09-11T09:01:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.628938+0000", 2024-09-11T09:01:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.87970911100000004 2024-09-11T09:01:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.629008+0000", 2024-09-11T09:01:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9467000000000004e-05 2024-09-11T09:01:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T09:01:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.629069+0000", 2024-09-11T09:01:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1353999999999996e-05 2024-09-11T09:01:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633156+0000", 2024-09-11T09:01:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040865729999999996 2024-09-11T09:01:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633192+0000", 2024-09-11T09:01:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5939999999999998e-05 2024-09-11T09:01:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633837+0000", 2024-09-11T09:01:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00064550800000000004 2024-09-11T09:01:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633902+0000", 2024-09-11T09:01:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5283000000000005e-05 2024-09-11T09:01:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633950+0000", 2024-09-11T09:01:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7713999999999998e-05 2024-09-11T09:01:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15366.0:272445 9.8 9:14cf6d17:::benchmark_data_smithi092_145083_object129047:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T09:01:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:52:24.755404+0000", 2024-09-11T09:01:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 527.18093193899995, 2024-09-11T09:01:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.87888003299999995, 2024-09-11T09:01:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15366", 2024-09-11T09:01:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T09:01:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 272445 2024-09-11T09:01:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755404+0000", 2024-09-11T09:01:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755404+0000", 2024-09-11T09:01:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755402+0000", 2024-09-11T09:01:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755410+0000", 2024-09-11T09:01:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8029999999999997e-06 2024-09-11T09:01:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755424+0000", 2024-09-11T09:01:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3912e-05 2024-09-11T09:01:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755429+0000", 2024-09-11T09:01:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7339999999999996e-06 2024-09-11T09:01:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755475+0000", 2024-09-11T09:01:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5982999999999997e-05 2024-09-11T09:01:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T09:01:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:24.755485+0000", 2024-09-11T09:01:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3170000000000008e-06 2024-09-11T09:01:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630509+0000", 2024-09-11T09:01:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.87502458000000005 2024-09-11T09:01:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630612+0000", 2024-09-11T09:01:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102422 2024-09-11T09:01:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,5", 2024-09-11T09:01:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.630716+0000", 2024-09-11T09:01:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104014 2024-09-11T09:01:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.633274+0000", 2024-09-11T09:01:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025580450000000001 2024-09-11T09:01:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.634132+0000", 2024-09-11T09:01:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00085821300000000003 2024-09-11T09:01:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.634183+0000", 2024-09-11T09:01:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1190000000000003e-05 2024-09-11T09:01:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.634236+0000", 2024-09-11T09:01:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2466000000000002e-05 2024-09-11T09:01:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:52:25.634284+0000", 2024-09-11T09:01:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7987e-05 2024-09-11T09:01:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:244905 10.7 10:e8d6c095:::benchmark_data_smithi092_173109_object107946:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T09:01:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:07.472236+0000", 2024-09-11T09:01:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 304.46409984000002, 2024-09-11T09:01:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0816275449999999, 2024-09-11T09:01:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T09:01:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 244905 2024-09-11T09:01:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.472236+0000", 2024-09-11T09:01:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.472236+0000", 2024-09-11T09:01:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.472234+0000", 2024-09-11T09:01:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.472240+0000", 2024-09-11T09:01:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8529999999999997e-06 2024-09-11T09:01:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.472248+0000", 2024-09-11T09:01:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999999e-06 2024-09-11T09:01:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.472254+0000", 2024-09-11T09:01:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4289999999999998e-06 2024-09-11T09:01:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.477108+0000", 2024-09-11T09:01:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048533680000000003 2024-09-11T09:01:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T09:01:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.477116+0000", 2024-09-11T09:01:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1489999999999994e-06 2024-09-11T09:01:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.523932+0000", 2024-09-11T09:01:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0468164870000001 2024-09-11T09:01:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.526492+0000", 2024-09-11T09:01:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025598610000000001 2024-09-11T09:01:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T09:01:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.526779+0000", 2024-09-11T09:01:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028662499999999999 2024-09-11T09:01:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.540305+0000", 2024-09-11T09:01:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013526404000000001 2024-09-11T09:01:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.540317+0000", 2024-09-11T09:01:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1440999999999999e-05 2024-09-11T09:01:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.553771+0000", 2024-09-11T09:01:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01345447 2024-09-11T09:01:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.553820+0000", 2024-09-11T09:01:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8513999999999997e-05 2024-09-11T09:01:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.553863+0000", 2024-09-11T09:01:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3473000000000002e-05 2024-09-11T09:01:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:244916 10.f 10:fbfc297d:::benchmark_data_smithi092_173109_object107957:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T09:01:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:07.479877+0000", 2024-09-11T09:01:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 304.45645827099997, 2024-09-11T09:01:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.074429866, 2024-09-11T09:01:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T09:01:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 244916 2024-09-11T09:01:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479877+0000", 2024-09-11T09:01:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479877+0000", 2024-09-11T09:01:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479876+0000", 2024-09-11T09:01:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479882+0000", 2024-09-11T09:01:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3230000000000003e-06 2024-09-11T09:01:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479893+0000", 2024-09-11T09:01:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1497e-05 2024-09-11T09:01:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479899+0000", 2024-09-11T09:01:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3789999999999997e-06 2024-09-11T09:01:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479983+0000", 2024-09-11T09:01:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3843000000000003e-05 2024-09-11T09:01:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T09:01:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.479993+0000", 2024-09-11T09:01:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0549e-05 2024-09-11T09:01:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.521849+0000", 2024-09-11T09:01:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0418553269999999 2024-09-11T09:01:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.524337+0000", 2024-09-11T09:01:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024885490000000001 2024-09-11T09:01:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,6", 2024-09-11T09:01:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.524582+0000", 2024-09-11T09:01:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024493600000000001 2024-09-11T09:01:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.537423+0000", 2024-09-11T09:01:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012840991 2024-09-11T09:01:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.540312+0000", 2024-09-11T09:01:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028889229999999998 2024-09-11T09:01:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.554139+0000", 2024-09-11T09:01:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01382735 2024-09-11T09:01:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.554270+0000", 2024-09-11T09:01:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013089199999999999 2024-09-11T09:01:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.554307+0000", 2024-09-11T09:01:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6869000000000002e-05 2024-09-11T09:01:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15621.0:244927 10.7 10:e600bc70:::benchmark_data_smithi092_173109_object107968:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T09:01:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:56:07.498963+0000", 2024-09-11T09:01:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 304.43737202199998, 2024-09-11T09:01:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0561594940000001, 2024-09-11T09:01:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15621", 2024-09-11T09:01:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 244927 2024-09-11T09:01:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.498963+0000", 2024-09-11T09:01:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.498963+0000", 2024-09-11T09:01:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.498962+0000", 2024-09-11T09:01:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.498971+0000", 2024-09-11T09:01:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9940000000000006e-06 2024-09-11T09:01:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.498989+0000", 2024-09-11T09:01:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8427000000000002e-05 2024-09-11T09:01:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.498996+0000", 2024-09-11T09:01:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-11T09:01:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.499040+0000", 2024-09-11T09:01:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3906999999999998e-05 2024-09-11T09:01:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-11T09:01:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:07.499048+0000", 2024-09-11T09:01:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2849999999999995e-06 2024-09-11T09:01:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.537427+0000", 2024-09-11T09:01:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.038378536 2024-09-11T09:01:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.537552+0000", 2024-09-11T09:01:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000125418 2024-09-11T09:01:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T09:01:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.537677+0000", 2024-09-11T09:01:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012447900000000001 2024-09-11T09:01:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.540462+0000", 2024-09-11T09:01:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027854339999999998 2024-09-11T09:01:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.543405+0000", 2024-09-11T09:01:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029425369999999998 2024-09-11T09:01:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.553899+0000", 2024-09-11T09:01:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010494768999999999 2024-09-11T09:01:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.555084+0000", 2024-09-11T09:01:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00118486 2024-09-11T09:01:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:56:08.555123+0000", 2024-09-11T09:01:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8571000000000001e-05 2024-09-11T09:01:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:49863 11.2 11:47a5097e:::benchmark_data_smithi092_205252_object49862:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-11T09:01:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:51.719001+0000", 2024-09-11T09:01:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 200.21733486599999, 2024-09-11T09:01:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23083320700000001, 2024-09-11T09:01:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T09:01:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49863 2024-09-11T09:01:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719001+0000", 2024-09-11T09:01:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719001+0000", 2024-09-11T09:01:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.718999+0000", 2024-09-11T09:01:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719024+0000", 2024-09-11T09:01:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4508e-05 2024-09-11T09:01:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719037+0000", 2024-09-11T09:01:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3295000000000001e-05 2024-09-11T09:01:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719043+0000", 2024-09-11T09:01:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8200000000000002e-06 2024-09-11T09:01:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719079+0000", 2024-09-11T09:01:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6334999999999997e-05 2024-09-11T09:01:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719192+0000", 2024-09-11T09:01:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112426 2024-09-11T09:01:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T09:01:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.719312+0000", 2024-09-11T09:01:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119926 2024-09-11T09:01:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.723980+0000", 2024-09-11T09:01:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004668681 2024-09-11T09:01:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.725139+0000", 2024-09-11T09:01:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011589650000000001 2024-09-11T09:01:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.949708+0000", 2024-09-11T09:01:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22456922900000001 2024-09-11T09:01:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.949787+0000", 2024-09-11T09:01:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8739000000000006e-05 2024-09-11T09:01:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.949834+0000", 2024-09-11T09:01:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6640000000000001e-05 2024-09-11T09:01:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:49867 11.2 11:4412ca95:::benchmark_data_smithi092_205252_object49866:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-11T09:01:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:51.721317+0000", 2024-09-11T09:01:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 200.21501839000001, 2024-09-11T09:01:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23993163000000001, 2024-09-11T09:01:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T09:01:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49867 2024-09-11T09:01:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721317+0000", 2024-09-11T09:01:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721317+0000", 2024-09-11T09:01:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721316+0000", 2024-09-11T09:01:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T09:01:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721341+0000", 2024-09-11T09:01:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.516e-05 2024-09-11T09:01:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721358+0000", 2024-09-11T09:01:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6738000000000001e-05 2024-09-11T09:01:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721365+0000", 2024-09-11T09:01:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.041e-06 2024-09-11T09:01:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721489+0000", 2024-09-11T09:01:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012353700000000001 2024-09-11T09:01:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721613+0000", 2024-09-11T09:01:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012459600000000001 2024-09-11T09:01:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T09:01:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.721830+0000", 2024-09-11T09:01:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021665500000000001 2024-09-11T09:01:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.728332+0000", 2024-09-11T09:01:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065019960000000003 2024-09-11T09:01:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.728338+0000", 2024-09-11T09:01:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6280000000000002e-06 2024-09-11T09:01:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961116+0000", 2024-09-11T09:01:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23277804399999999 2024-09-11T09:01:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961209+0000", 2024-09-11T09:01:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3527000000000001e-05 2024-09-11T09:01:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961249+0000", 2024-09-11T09:01:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9564000000000003e-05 2024-09-11T09:01:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:49875 11.2 11:41e4a4f5:::benchmark_data_smithi092_205252_object49874:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-11T09:01:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:51.724742+0000", 2024-09-11T09:01:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 200.211593722, 2024-09-11T09:01:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23689321699999999, 2024-09-11T09:01:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T09:01:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49875 2024-09-11T09:01:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.724742+0000", 2024-09-11T09:01:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.724742+0000", 2024-09-11T09:01:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.724740+0000", 2024-09-11T09:01:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.724762+0000", 2024-09-11T09:01:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1639000000000001e-05 2024-09-11T09:01:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.724781+0000", 2024-09-11T09:01:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8861000000000001e-05 2024-09-11T09:01:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.724787+0000", 2024-09-11T09:01:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0789999999999997e-06 2024-09-11T09:01:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.725161+0000", 2024-09-11T09:01:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037398200000000003 2024-09-11T09:01:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.725426+0000", 2024-09-11T09:01:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026499100000000001 2024-09-11T09:01:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T09:01:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.725741+0000", 2024-09-11T09:01:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031497399999999998 2024-09-11T09:01:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.730610+0000", 2024-09-11T09:01:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048688070000000002 2024-09-11T09:01:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732667+0000", 2024-09-11T09:01:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002057101 2024-09-11T09:01:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961283+0000", 2024-09-11T09:01:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.228616554 2024-09-11T09:01:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961599+0000", 2024-09-11T09:01:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031523100000000002 2024-09-11T09:01:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961635+0000", 2024-09-11T09:01:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6337999999999997e-05 2024-09-11T09:01:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:49880 11.2 11:41f0ab8a:::benchmark_data_smithi092_205252_object49879:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-11T09:01:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:51.727728+0000", 2024-09-11T09:01:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 200.20860750599999, 2024-09-11T09:01:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.234053021, 2024-09-11T09:01:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T09:01:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49880 2024-09-11T09:01:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.727728+0000", 2024-09-11T09:01:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.727728+0000", 2024-09-11T09:01:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.727727+0000", 2024-09-11T09:01:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.727751+0000", 2024-09-11T09:01:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4093000000000001e-05 2024-09-11T09:01:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.727764+0000", 2024-09-11T09:01:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3315e-05 2024-09-11T09:01:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.727770+0000", 2024-09-11T09:01:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-06 2024-09-11T09:01:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.728358+0000", 2024-09-11T09:01:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00058712599999999999 2024-09-11T09:01:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.728472+0000", 2024-09-11T09:01:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114449 2024-09-11T09:01:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T09:01:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.728601+0000", 2024-09-11T09:01:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129238 2024-09-11T09:01:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732735+0000", 2024-09-11T09:01:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041335929999999996 2024-09-11T09:01:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.734415+0000", 2024-09-11T09:01:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016805209999999999 2024-09-11T09:01:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961694+0000", 2024-09-11T09:01:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.227278969 2024-09-11T09:01:12.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961745+0000", 2024-09-11T09:01:12.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0729999999999997e-05 2024-09-11T09:01:12.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961781+0000", 2024-09-11T09:01:12.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5941e-05 2024-09-11T09:01:12.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.230 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.231 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:49884 11.2 11:4a5a3be8:::benchmark_data_smithi092_205252_object49883:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-11T09:01:12.231 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:51.729597+0000", 2024-09-11T09:01:12.231 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 200.20673801199999, 2024-09-11T09:01:12.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23231478699999999, 2024-09-11T09:01:12.231 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.231 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T09:01:12.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:12.231 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49884 2024-09-11T09:01:12.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.231 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.729597+0000", 2024-09-11T09:01:12.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.729597+0000", 2024-09-11T09:01:12.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.729596+0000", 2024-09-11T09:01:12.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T09:01:12.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.729623+0000", 2024-09-11T09:01:12.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6237e-05 2024-09-11T09:01:12.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.729638+0000", 2024-09-11T09:01:12.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5699999999999999e-05 2024-09-11T09:01:12.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.729644+0000", 2024-09-11T09:01:12.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.896e-06 2024-09-11T09:01:12.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.730629+0000", 2024-09-11T09:01:12.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0009843740000000001 2024-09-11T09:01:12.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.730730+0000", 2024-09-11T09:01:12.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010111300000000001 2024-09-11T09:01:12.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T09:01:12.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.730860+0000", 2024-09-11T09:01:12.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000130257 2024-09-11T09:01:12.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.734454+0000", 2024-09-11T09:01:12.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035937040000000001 2024-09-11T09:01:12.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.734470+0000", 2024-09-11T09:01:12.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6396e-05 2024-09-11T09:01:12.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961822+0000", 2024-09-11T09:01:12.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22735159699999999 2024-09-11T09:01:12.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961880+0000", 2024-09-11T09:01:12.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8118000000000002e-05 2024-09-11T09:01:12.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961912+0000", 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2489e-05 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:49887 11.2 11:4fe36588:::benchmark_data_smithi092_205252_object49886:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T08:57:51.732433+0000", 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 200.20390282299999, 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22958763600000001, 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.237 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T09:01:12.238 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:12.238 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49887 2024-09-11T09:01:12.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.238 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732433+0000", 2024-09-11T09:01:12.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732433+0000", 2024-09-11T09:01:12.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732431+0000", 2024-09-11T09:01:12.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.999999 2024-09-11T09:01:12.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732456+0000", 2024-09-11T09:01:12.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4791e-05 2024-09-11T09:01:12.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732466+0000", 2024-09-11T09:01:12.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7759999999999992e-06 2024-09-11T09:01:12.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732471+0000", 2024-09-11T09:01:12.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2689999999999999e-06 2024-09-11T09:01:12.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732846+0000", 2024-09-11T09:01:12.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037487000000000002 2024-09-11T09:01:12.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.732970+0000", 2024-09-11T09:01:12.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012358700000000001 2024-09-11T09:01:12.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T09:01:12.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.733108+0000", 2024-09-11T09:01:12.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013874100000000001 2024-09-11T09:01:12.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.734544+0000", 2024-09-11T09:01:12.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014350509999999999 2024-09-11T09:01:12.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.734743+0000", 2024-09-11T09:01:12.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019981400000000001 2024-09-11T09:01:12.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961947+0000", 2024-09-11T09:01:12.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22720348600000001 2024-09-11T09:01:12.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.961989+0000", 2024-09-11T09:01:12.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1922999999999998e-05 2024-09-11T09:01:12.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T08:57:51.962020+0000", 2024-09-11T09:01:12.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1520000000000003e-05 2024-09-11T09:01:12.243 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.243 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.243 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.243 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:209914 11.9 11:9da878fb:::benchmark_data_smithi092_205252_object88593:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e422)", 2024-09-11T09:01:12.243 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T09:00:08.466955+0000", 2024-09-11T09:01:12.244 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.469380354999998, 2024-09-11T09:01:12.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.24026382900000001, 2024-09-11T09:01:12.244 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.244 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.244 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.244 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T09:01:12.244 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:12.244 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 209914 2024-09-11T09:01:12.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.244 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:08.466955+0000", 2024-09-11T09:01:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:08.466955+0000", 2024-09-11T09:01:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:08.466954+0000", 2024-09-11T09:01:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:08.466960+0000", 2024-09-11T09:01:12.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.993e-06 2024-09-11T09:01:12.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:08.466974+0000", 2024-09-11T09:01:12.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3910000000000001e-05 2024-09-11T09:01:12.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:08.466979+0000", 2024-09-11T09:01:12.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2830000000000001e-06 2024-09-11T09:01:12.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:08.467017+0000", 2024-09-11T09:01:12.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7803999999999999e-05 2024-09-11T09:01:12.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:08.467124+0000", 2024-09-11T09:01:12.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106928 2024-09-11T09:01:12.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,6", 2024-09-11T09:01:12.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:08.467230+0000", 2024-09-11T09:01:12.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010592800000000001 2024-09-11T09:01:12.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:08.469848+0000", 2024-09-11T09:01:12.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026187390000000001 2024-09-11T09:01:12.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:08.476083+0000", 2024-09-11T09:01:12.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062345450000000002 2024-09-11T09:01:12.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:08.707080+0000", 2024-09-11T09:01:12.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.230997067 2024-09-11T09:01:12.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:08.707115+0000", 2024-09-11T09:01:12.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5299000000000002e-05 2024-09-11T09:01:12.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:08.707219+0000", 2024-09-11T09:01:12.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103594 2024-09-11T09:01:12.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.249 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:238428 11.a 11:551d838c:::benchmark_data_smithi092_205252_object117107:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T09:00:31.166222+0000", 2024-09-11T09:01:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.770113641999998, 2024-09-11T09:01:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038815277000000002, 2024-09-11T09:01:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T09:01:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 238428 2024-09-11T09:01:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.166222+0000", 2024-09-11T09:01:12.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.166222+0000", 2024-09-11T09:01:12.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.166221+0000", 2024-09-11T09:01:12.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:12.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.166224+0000", 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8590000000000001e-06 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.166230+0000", 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.93e-06 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.166234+0000", 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3220000000000001e-06 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.169879+0000", 2024-09-11T09:01:12.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036456470000000001 2024-09-11T09:01:12.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.170002+0000", 2024-09-11T09:01:12.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012317599999999999 2024-09-11T09:01:12.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5,6", 2024-09-11T09:01:12.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.170125+0000", 2024-09-11T09:01:12.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000123127 2024-09-11T09:01:12.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.173243+0000", 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003117276 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.173433+0000", 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000189872 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.204668+0000", 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031235881 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.204989+0000", 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032076700000000001 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.205037+0000", 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7833000000000001e-05 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:238432 11.2 11:41bc98d2:::benchmark_data_smithi092_205252_object117111:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T09:00:31.169702+0000", 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.766633261999999, 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035421324999999997, 2024-09-11T09:01:12.255 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.256 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T09:01:12.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:12.256 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 238432 2024-09-11T09:01:12.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.256 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.169702+0000", 2024-09-11T09:01:12.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.169702+0000", 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.169701+0000", 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.169708+0000", 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2459999999999997e-06 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.169720+0000", 2024-09-11T09:01:12.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1926999999999999e-05 2024-09-11T09:01:12.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.169725+0000", 2024-09-11T09:01:12.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5570000000000003e-06 2024-09-11T09:01:12.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.173312+0000", 2024-09-11T09:01:12.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035862889999999999 2024-09-11T09:01:12.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.173425+0000", 2024-09-11T09:01:12.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112872 2024-09-11T09:01:12.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T09:01:12.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.173543+0000", 2024-09-11T09:01:12.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117992 2024-09-11T09:01:12.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.194003+0000", 2024-09-11T09:01:12.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020460234000000001 2024-09-11T09:01:12.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.194164+0000", 2024-09-11T09:01:12.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000161044 2024-09-11T09:01:12.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.205049+0000", 2024-09-11T09:01:12.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010884868000000001 2024-09-11T09:01:12.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.205092+0000", 2024-09-11T09:01:12.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3016000000000003e-05 2024-09-11T09:01:12.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.205124+0000", 2024-09-11T09:01:12.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1776000000000002e-05 2024-09-11T09:01:12.261 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.261 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.261 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.261 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:238436 11.2 11:4ea71df3:::benchmark_data_smithi092_205252_object117115:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:12.261 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T09:00:31.172916+0000", 2024-09-11T09:01:12.261 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.763419644999999, 2024-09-11T09:01:12.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035505198000000002, 2024-09-11T09:01:12.261 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.261 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.261 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.261 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T09:01:12.261 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:12.262 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 238436 2024-09-11T09:01:12.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.262 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.172916+0000", 2024-09-11T09:01:12.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.172916+0000", 2024-09-11T09:01:12.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.172915+0000", 2024-09-11T09:01:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.172918+0000", 2024-09-11T09:01:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2369999999999999e-06 2024-09-11T09:01:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.172923+0000", 2024-09-11T09:01:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0660000000000003e-06 2024-09-11T09:01:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.172926+0000", 2024-09-11T09:01:12.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3349999999999998e-06 2024-09-11T09:01:12.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.190151+0000", 2024-09-11T09:01:12.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017224962 2024-09-11T09:01:12.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.191358+0000", 2024-09-11T09:01:12.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012072579999999999 2024-09-11T09:01:12.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,2", 2024-09-11T09:01:12.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.191466+0000", 2024-09-11T09:01:12.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107376 2024-09-11T09:01:12.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.204727+0000", 2024-09-11T09:01:12.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013261304 2024-09-11T09:01:12.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.205323+0000", 2024-09-11T09:01:12.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000596154 2024-09-11T09:01:12.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.208227+0000", 2024-09-11T09:01:12.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029037070000000002 2024-09-11T09:01:12.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.208386+0000", 2024-09-11T09:01:12.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000159006 2024-09-11T09:01:12.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:31.208421+0000", 2024-09-11T09:01:12.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5138999999999997e-05 2024-09-11T09:01:12.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.267 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.267 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:240940 11.9 11:96cce66f:::benchmark_data_smithi092_205252_object119619:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:12.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T09:00:33.162834+0000", 2024-09-11T09:01:12.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.773501852999999, 2024-09-11T09:01:12.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034897986999999998, 2024-09-11T09:01:12.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T09:01:12.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:12.267 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 240940 2024-09-11T09:01:12.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.268 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.162834+0000", 2024-09-11T09:01:12.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.162834+0000", 2024-09-11T09:01:12.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.162832+0000", 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.162838+0000", 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0730000000000001e-06 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.162852+0000", 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3687e-05 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.162859+0000", 2024-09-11T09:01:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8750000000000002e-06 2024-09-11T09:01:12.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.174165+0000", 2024-09-11T09:01:12.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011306249000000001 2024-09-11T09:01:12.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.174269+0000", 2024-09-11T09:01:12.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010406 2024-09-11T09:01:12.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,6", 2024-09-11T09:01:12.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.174378+0000", 2024-09-11T09:01:12.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010874099999999999 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.182189+0000", 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0078114689999999997 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.182989+0000", 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00079917699999999998 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.197536+0000", 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014547183999999999 2024-09-11T09:01:12.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.197689+0000", 2024-09-11T09:01:12.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015356300000000001 2024-09-11T09:01:12.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.197732+0000", 2024-09-11T09:01:12.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2200000000000003e-05 2024-09-11T09:01:12.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.272 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.273 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.15918.0:240942 11.1 11:8ffb295b:::benchmark_data_smithi092_205252_object119621:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:12.273 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-11T09:00:33.162866+0000", 2024-09-11T09:01:12.273 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.773469804999998, 2024-09-11T09:01:12.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038771462, 2024-09-11T09:01:12.273 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-11T09:01:12.273 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:12.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-11T09:01:12.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.15918", 2024-09-11T09:01:12.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:12.273 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 240942 2024-09-11T09:01:12.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.273 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-11T09:01:12.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-11T09:01:12.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.162866+0000", 2024-09-11T09:01:12.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-11T09:01:12.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.162866+0000", 2024-09-11T09:01:12.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-11T09:01:12.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-11T09:01:12.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.162865+0000", 2024-09-11T09:01:12.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:12.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-11T09:01:12.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.162868+0000", 2024-09-11T09:01:12.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7690000000000001e-06 2024-09-11T09:01:12.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-11T09:01:12.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.162875+0000", 2024-09-11T09:01:12.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1779999999999997e-06 2024-09-11T09:01:12.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-11T09:01:12.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.162880+0000", 2024-09-11T09:01:12.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9599999999999999e-06 2024-09-11T09:01:12.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-11T09:01:12.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.182294+0000", 2024-09-11T09:01:12.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019414092000000001 2024-09-11T09:01:12.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-11T09:01:12.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.182397+0000", 2024-09-11T09:01:12.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103222 2024-09-11T09:01:12.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0,6", 2024-09-11T09:01:12.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.182513+0000", 2024-09-11T09:01:12.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011589700000000001 2024-09-11T09:01:12.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-11T09:01:12.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.197741+0000", 2024-09-11T09:01:12.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015227350000000001 2024-09-11T09:01:12.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.201502+0000", 2024-09-11T09:01:12.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037613 2024-09-11T09:01:12.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:12.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.201542+0000", 2024-09-11T09:01:12.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0132999999999997e-05 2024-09-11T09:01:12.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-11T09:01:12.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.201598+0000", 2024-09-11T09:01:12.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5787e-05 2024-09-11T09:01:12.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-11T09:01:12.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-11T09:01:12.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-11T09:01:12.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-11T09:00:33.201637+0000", 2024-09-11T09:01:12.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9304000000000002e-05 2024-09-11T09:01:12.278 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.279 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-11T09:01:12.279 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-11T09:01:12.279 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-11T09:01:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-11T09:01:12.280 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:12 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:01:12.280 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:12 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:01:12.280 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:12 smithi092 ceph-mon[29647]: osdmap e435: 8 total, 8 up, 8 in 2024-09-11T09:01:12.280 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:12 smithi092 ceph-mon[29647]: pgmap v1027: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:12.280 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:12 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/458485950' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T09:01:12.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:12 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:01:12.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:12 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:01:12.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:12 smithi092 ceph-mon[41114]: osdmap e435: 8 total, 8 up, 8 in 2024-09-11T09:01:12.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:12 smithi092 ceph-mon[41114]: pgmap v1027: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:12.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:12 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/458485950' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T09:01:12.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:12 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:01:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:12 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:01:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:12 smithi145 ceph-mon[33384]: osdmap e435: 8 total, 8 up, 8 in 2024-09-11T09:01:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:12 smithi145 ceph-mon[33384]: pgmap v1027: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:12.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:12 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/458485950' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T09:01:13.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:01:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:01:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:01:13.410 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:13 smithi092 ceph-mon[41114]: Health check cleared: OSDMAP_FLAGS (was: noscrub flag(s) set) 2024-09-11T09:01:13.410 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:13 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T09:01:13.411 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:13 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/458485950' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T09:01:13.411 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:13 smithi092 ceph-mon[41114]: osdmap e436: 8 total, 8 up, 8 in 2024-09-11T09:01:13.411 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:13 smithi092 ceph-mon[29647]: Health check cleared: OSDMAP_FLAGS (was: noscrub flag(s) set) 2024-09-11T09:01:13.411 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:13 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T09:01:13.411 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:13 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/458485950' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T09:01:13.411 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:13 smithi092 ceph-mon[29647]: osdmap e436: 8 total, 8 up, 8 in 2024-09-11T09:01:13.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:13 smithi145 ceph-mon[33384]: Health check cleared: OSDMAP_FLAGS (was: noscrub flag(s) set) 2024-09-11T09:01:13.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:13 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T09:01:13.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:13 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/458485950' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T09:01:13.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:13 smithi145 ceph-mon[33384]: osdmap e436: 8 total, 8 up, 8 in 2024-09-11T09:01:13.835 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-11T09:01:14.094 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T09:01:14.548 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:14 smithi145 ceph-mon[33384]: 1.3 scrub starts 2024-09-11T09:01:14.549 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:14 smithi145 ceph-mon[33384]: 1.3 scrub ok 2024-09-11T09:01:14.549 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:14 smithi145 ceph-mon[33384]: 2.1 scrub starts 2024-09-11T09:01:14.549 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:14 smithi145 ceph-mon[33384]: 2.1 scrub ok 2024-09-11T09:01:14.549 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:14 smithi145 ceph-mon[33384]: pgmap v1029: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T09:01:14.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:14 smithi092 ceph-mon[29647]: 1.3 scrub starts 2024-09-11T09:01:14.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:14 smithi092 ceph-mon[29647]: 1.3 scrub ok 2024-09-11T09:01:14.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:14 smithi092 ceph-mon[29647]: 2.1 scrub starts 2024-09-11T09:01:14.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:14 smithi092 ceph-mon[29647]: 2.1 scrub ok 2024-09-11T09:01:14.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:14 smithi092 ceph-mon[29647]: pgmap v1029: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T09:01:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:14 smithi092 ceph-mon[41114]: 1.3 scrub starts 2024-09-11T09:01:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:14 smithi092 ceph-mon[41114]: 1.3 scrub ok 2024-09-11T09:01:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:14 smithi092 ceph-mon[41114]: 2.1 scrub starts 2024-09-11T09:01:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:14 smithi092 ceph-mon[41114]: 2.1 scrub ok 2024-09-11T09:01:14.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:14 smithi092 ceph-mon[41114]: pgmap v1029: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T09:01:15.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T09:01:15.386 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:15 smithi145 ceph-mon[33384]: 4.16 scrub starts 2024-09-11T09:01:15.386 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:15 smithi145 ceph-mon[33384]: 4.16 scrub ok 2024-09-11T09:01:15.386 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:15 smithi145 ceph-mon[33384]: 4.9 deep-scrub starts 2024-09-11T09:01:15.386 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:15 smithi145 ceph-mon[33384]: 4.9 deep-scrub ok 2024-09-11T09:01:15.386 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:01:15.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:15 smithi092 ceph-mon[29647]: 4.16 scrub starts 2024-09-11T09:01:15.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:15 smithi092 ceph-mon[29647]: 4.16 scrub ok 2024-09-11T09:01:15.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:15 smithi092 ceph-mon[29647]: 4.9 deep-scrub starts 2024-09-11T09:01:15.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:15 smithi092 ceph-mon[29647]: 4.9 deep-scrub ok 2024-09-11T09:01:15.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:01:15.458 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:15 smithi092 ceph-mon[41114]: 4.16 scrub starts 2024-09-11T09:01:15.458 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:15 smithi092 ceph-mon[41114]: 4.16 scrub ok 2024-09-11T09:01:15.458 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:15 smithi092 ceph-mon[41114]: 4.9 deep-scrub starts 2024-09-11T09:01:15.458 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:15 smithi092 ceph-mon[41114]: 4.9 deep-scrub ok 2024-09-11T09:01:15.458 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:01:15.714 INFO:teuthology.orchestra.run.smithi092.stdout:mon.a: {} 2024-09-11T09:01:15.714 INFO:teuthology.orchestra.run.smithi092.stdout:mon.b: {} 2024-09-11T09:01:15.714 INFO:teuthology.orchestra.run.smithi092.stdout:mon.c: {} 2024-09-11T09:01:15.714 INFO:teuthology.orchestra.run.smithi092.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T09:01:15.714 INFO:teuthology.orchestra.run.smithi092.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T09:01:15.714 INFO:teuthology.orchestra.run.smithi092.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T09:01:15.918 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T09:01:15.918 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T09:01:15.918 INFO:teuthology.orchestra.run.smithi145.stdout: "num_ops": 0 2024-09-11T09:01:15.918 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T09:01:15.945 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:16.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:16 smithi092 ceph-mon[29647]: 1.14 scrub starts 2024-09-11T09:01:16.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:16 smithi092 ceph-mon[29647]: 1.14 scrub ok 2024-09-11T09:01:16.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:16 smithi092 ceph-mon[29647]: 1.0 scrub starts 2024-09-11T09:01:16.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:16 smithi092 ceph-mon[29647]: 1.0 scrub ok 2024-09-11T09:01:16.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:16 smithi092 ceph-mon[29647]: pgmap v1030: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 747 B/s rd, 0 op/s 2024-09-11T09:01:16.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:16 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/858067125' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T09:01:16.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:16 smithi092 ceph-mon[41114]: 1.14 scrub starts 2024-09-11T09:01:16.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:16 smithi092 ceph-mon[41114]: 1.14 scrub ok 2024-09-11T09:01:16.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:16 smithi092 ceph-mon[41114]: 1.0 scrub starts 2024-09-11T09:01:16.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:16 smithi092 ceph-mon[41114]: 1.0 scrub ok 2024-09-11T09:01:16.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:16 smithi092 ceph-mon[41114]: pgmap v1030: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 747 B/s rd, 0 op/s 2024-09-11T09:01:16.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:16 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/858067125' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T09:01:16.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 0 1 2024-09-11T09:01:16.560 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:16 smithi145 ceph-mon[33384]: 1.14 scrub starts 2024-09-11T09:01:16.560 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:16 smithi145 ceph-mon[33384]: 1.14 scrub ok 2024-09-11T09:01:16.560 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:16 smithi145 ceph-mon[33384]: 1.0 scrub starts 2024-09-11T09:01:16.560 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:16 smithi145 ceph-mon[33384]: 1.0 scrub ok 2024-09-11T09:01:16.561 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:16 smithi145 ceph-mon[33384]: pgmap v1030: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 747 B/s rd, 0 op/s 2024-09-11T09:01:16.561 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:16 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/858067125' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T09:01:16.586 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-11T09:01:16.830 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T09:01:17.255 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:17 smithi145 ceph-mon[33384]: 6.1 deep-scrub starts 2024-09-11T09:01:17.255 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:17 smithi145 ceph-mon[33384]: 6.1 deep-scrub ok 2024-09-11T09:01:17.255 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:17 smithi145 ceph-mon[33384]: 6.3 scrub starts 2024-09-11T09:01:17.255 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:17 smithi145 ceph-mon[33384]: 6.3 scrub ok 2024-09-11T09:01:17.342 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:17.468 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:17 smithi092 ceph-mon[29647]: 6.1 deep-scrub starts 2024-09-11T09:01:17.469 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:17 smithi092 ceph-mon[29647]: 6.1 deep-scrub ok 2024-09-11T09:01:17.469 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:17 smithi092 ceph-mon[29647]: 6.3 scrub starts 2024-09-11T09:01:17.469 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:17 smithi092 ceph-mon[29647]: 6.3 scrub ok 2024-09-11T09:01:17.469 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:17 smithi092 ceph-mon[41114]: 6.1 deep-scrub starts 2024-09-11T09:01:17.469 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:17 smithi092 ceph-mon[41114]: 6.1 deep-scrub ok 2024-09-11T09:01:17.469 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:17 smithi092 ceph-mon[41114]: 6.3 scrub starts 2024-09-11T09:01:17.469 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:17 smithi092 ceph-mon[41114]: 6.3 scrub ok 2024-09-11T09:01:17.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:01:17 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:01:18.257 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:18 smithi145 ceph-mon[33384]: 6.14 scrub starts 2024-09-11T09:01:18.257 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:18 smithi145 ceph-mon[33384]: 6.14 scrub ok 2024-09-11T09:01:18.257 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:18 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:18.257 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:18 smithi145 ceph-mon[33384]: pgmap v1031: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:18.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:18 smithi092 ceph-mon[29647]: 6.14 scrub starts 2024-09-11T09:01:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:18 smithi092 ceph-mon[29647]: 6.14 scrub ok 2024-09-11T09:01:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:18 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:18.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:18 smithi092 ceph-mon[29647]: pgmap v1031: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:18.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:18 smithi092 ceph-mon[41114]: 6.14 scrub starts 2024-09-11T09:01:18.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:18 smithi092 ceph-mon[41114]: 6.14 scrub ok 2024-09-11T09:01:18.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:18 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:18.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:18 smithi092 ceph-mon[41114]: pgmap v1031: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:18.489 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T09:01:18.489 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T09:01:18.489 INFO:teuthology.orchestra.run.smithi145.stdout: "complaint_time": 30, 2024-09-11T09:01:18.489 INFO:teuthology.orchestra.run.smithi145.stdout: "num_blocked_ops": 0 2024-09-11T09:01:18.489 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T09:01:19.186 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-11T09:01:19.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:19 smithi092 ceph-mon[29647]: 5.4 scrub starts 2024-09-11T09:01:19.374 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:19 smithi092 ceph-mon[29647]: 5.4 scrub ok 2024-09-11T09:01:19.374 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:19 smithi092 ceph-mon[41114]: 5.4 scrub starts 2024-09-11T09:01:19.375 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:19 smithi092 ceph-mon[41114]: 5.4 scrub ok 2024-09-11T09:01:19.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:19 smithi145 ceph-mon[33384]: 5.4 scrub starts 2024-09-11T09:01:19.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:19 smithi145 ceph-mon[33384]: 5.4 scrub ok 2024-09-11T09:01:19.430 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T09:01:20.200 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-11T09:01:20.502 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[29647]: 6.0 scrub starts 2024-09-11T09:01:20.502 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[29647]: 6.0 scrub ok 2024-09-11T09:01:20.502 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[29647]: 3.1a scrub starts 2024-09-11T09:01:20.502 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[29647]: 3.1a scrub ok 2024-09-11T09:01:20.503 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3341808279' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T09:01:20.503 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T09:01:20.503 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[29647]: 6.1c scrub starts 2024-09-11T09:01:20.503 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[29647]: 6.1c scrub ok 2024-09-11T09:01:20.503 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[29647]: pgmap v1032: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:20.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[41114]: 6.0 scrub starts 2024-09-11T09:01:20.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[41114]: 6.0 scrub ok 2024-09-11T09:01:20.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[41114]: 3.1a scrub starts 2024-09-11T09:01:20.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[41114]: 3.1a scrub ok 2024-09-11T09:01:20.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3341808279' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T09:01:20.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T09:01:20.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[41114]: 6.1c scrub starts 2024-09-11T09:01:20.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[41114]: 6.1c scrub ok 2024-09-11T09:01:20.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:20 smithi092 ceph-mon[41114]: pgmap v1032: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:20.624 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:20 smithi145 ceph-mon[33384]: 6.0 scrub starts 2024-09-11T09:01:20.624 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:20 smithi145 ceph-mon[33384]: 6.0 scrub ok 2024-09-11T09:01:20.624 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:20 smithi145 ceph-mon[33384]: 3.1a scrub starts 2024-09-11T09:01:20.624 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:20 smithi145 ceph-mon[33384]: 3.1a scrub ok 2024-09-11T09:01:20.624 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:20 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3341808279' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T09:01:20.625 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:20 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T09:01:20.625 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:20 smithi145 ceph-mon[33384]: 6.1c scrub starts 2024-09-11T09:01:20.625 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:20 smithi145 ceph-mon[33384]: 6.1c scrub ok 2024-09-11T09:01:20.625 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:20 smithi145 ceph-mon[33384]: pgmap v1032: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:21.169 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T09:01:21.169 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T09:01:21.170 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T09:01:21.170 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T09:01:21.170 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.170 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272398 9.10 9:0afe1f3f:::benchmark_data_smithi092_145083_object129000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T09:01:21.170 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:24.678144+0000", 2024-09-11T09:01:21.170 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 536.47978531599995, 2024-09-11T09:01:21.170 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.95568126099999995, 2024-09-11T09:01:21.170 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.170 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:21.170 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.171 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T09:01:21.171 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T09:01:21.171 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272398 2024-09-11T09:01:21.171 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.171 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.171 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.171 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.171 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678144+0000", 2024-09-11T09:01:21.171 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.171 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.171 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.172 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.172 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678144+0000", 2024-09-11T09:01:21.172 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.172 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.172 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.172 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.172 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678143+0000", 2024-09-11T09:01:21.172 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:21.172 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.172 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.172 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.173 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678149+0000", 2024-09-11T09:01:21.173 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2489999999999998e-06 2024-09-11T09:01:21.173 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.173 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.173 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.173 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678161+0000", 2024-09-11T09:01:21.173 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1782e-05 2024-09-11T09:01:21.173 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.173 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.173 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.173 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678167+0000", 2024-09-11T09:01:21.173 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8449999999999999e-06 2024-09-11T09:01:21.173 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.173 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.174 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.174 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678218+0000", 2024-09-11T09:01:21.174 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1109000000000001e-05 2024-09-11T09:01:21.174 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.174 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.174 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:21.174 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.678225+0000", 2024-09-11T09:01:21.174 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6890000000000004e-06 2024-09-11T09:01:21.174 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.174 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.174 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.174 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.627423+0000", 2024-09-11T09:01:21.174 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.94919719899999999 2024-09-11T09:01:21.175 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.175 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.175 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.175 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.627524+0000", 2024-09-11T09:01:21.175 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000101258 2024-09-11T09:01:21.175 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.175 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.175 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,7", 2024-09-11T09:01:21.175 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.627606+0000", 2024-09-11T09:01:21.175 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2095000000000006e-05 2024-09-11T09:01:21.175 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.175 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.176 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.176 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.633662+0000", 2024-09-11T09:01:21.176 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0060555619999999996 2024-09-11T09:01:21.176 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.176 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.176 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.176 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.633731+0000", 2024-09-11T09:01:21.176 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9865999999999998e-05 2024-09-11T09:01:21.176 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.177 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.177 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:21.177 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.633739+0000", 2024-09-11T09:01:21.177 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.379e-06 2024-09-11T09:01:21.177 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.177 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.177 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:21.177 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.633783+0000", 2024-09-11T09:01:21.177 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3732000000000001e-05 2024-09-11T09:01:21.177 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.177 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.177 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.177 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.633826+0000", 2024-09-11T09:01:21.177 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3099999999999997e-05 2024-09-11T09:01:21.178 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.178 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.178 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.178 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.178 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.178 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272433 9.10 9:0a114e2d:::benchmark_data_smithi092_145083_object129035:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e281)", 2024-09-11T09:01:21.179 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:24.742605+0000", 2024-09-11T09:01:21.179 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 536.41532483000003, 2024-09-11T09:01:21.179 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.89161613399999995, 2024-09-11T09:01:21.179 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.179 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:21.179 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.179 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T09:01:21.179 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T09:01:21.179 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272433 2024-09-11T09:01:21.179 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.179 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.179 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.179 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.180 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742605+0000", 2024-09-11T09:01:21.180 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.180 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.180 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.180 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.180 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742605+0000", 2024-09-11T09:01:21.180 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.180 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.180 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.180 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.180 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742604+0000", 2024-09-11T09:01:21.180 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:21.180 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.181 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.181 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.181 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742610+0000", 2024-09-11T09:01:21.181 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2670000000000003e-06 2024-09-11T09:01:21.181 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.181 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.181 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.181 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742622+0000", 2024-09-11T09:01:21.181 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2029e-05 2024-09-11T09:01:21.181 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.181 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.181 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.182 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742627+0000", 2024-09-11T09:01:21.182 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8099999999999997e-06 2024-09-11T09:01:21.182 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.182 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.182 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.182 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742665+0000", 2024-09-11T09:01:21.182 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8421999999999999e-05 2024-09-11T09:01:21.182 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.182 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.182 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:21.182 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.742671+0000", 2024-09-11T09:01:21.182 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4890000000000003e-06 2024-09-11T09:01:21.183 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.183 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.183 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.183 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.629333+0000", 2024-09-11T09:01:21.183 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.88666230800000001 2024-09-11T09:01:21.183 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.183 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.183 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.183 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.629439+0000", 2024-09-11T09:01:21.183 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000106183 2024-09-11T09:01:21.183 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.183 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.183 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,7", 2024-09-11T09:01:21.183 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.629559+0000", 2024-09-11T09:01:21.184 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000120099 2024-09-11T09:01:21.184 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.184 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.184 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:21.184 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.633833+0000", 2024-09-11T09:01:21.184 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0042741630000000001 2024-09-11T09:01:21.184 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.184 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.184 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.184 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.634007+0000", 2024-09-11T09:01:21.184 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00017368400000000001 2024-09-11T09:01:21.184 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.185 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.185 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.185 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.634126+0000", 2024-09-11T09:01:21.185 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000118921 2024-09-11T09:01:21.185 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.185 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.185 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:21.186 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.634185+0000", 2024-09-11T09:01:21.186 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8594999999999997e-05 2024-09-11T09:01:21.186 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.186 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.186 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.186 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:25.634221+0000", 2024-09-11T09:01:21.186 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6502999999999999e-05 2024-09-11T09:01:21.186 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.186 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.187 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.187 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.187 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.187 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:180218 10.e 10:730e92fe:::benchmark_data_smithi092_173109_object43259:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-11T09:01:21.187 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:55:10.343242+0000", 2024-09-11T09:01:21.187 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 370.81468764800002, 2024-09-11T09:01:21.187 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.232542421, 2024-09-11T09:01:21.187 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.187 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:21.187 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.188 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:21.188 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:21.188 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 180218 2024-09-11T09:01:21.188 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.188 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.188 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.188 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.188 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343242+0000", 2024-09-11T09:01:21.188 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.189 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.189 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.189 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.189 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343242+0000", 2024-09-11T09:01:21.189 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.189 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.189 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.189 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.190 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343241+0000", 2024-09-11T09:01:21.190 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:21.190 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.190 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.190 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.190 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343245+0000", 2024-09-11T09:01:21.190 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6430000000000001e-06 2024-09-11T09:01:21.190 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.190 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.190 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.190 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343255+0000", 2024-09-11T09:01:21.190 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.8700000000000004e-06 2024-09-11T09:01:21.191 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.191 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.191 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.191 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343260+0000", 2024-09-11T09:01:21.191 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4909999999999996e-06 2024-09-11T09:01:21.191 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.191 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.191 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.191 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343301+0000", 2024-09-11T09:01:21.191 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0436999999999999e-05 2024-09-11T09:01:21.191 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.192 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.192 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.192 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343411+0000", 2024-09-11T09:01:21.192 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011047799999999999 2024-09-11T09:01:21.192 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.192 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.192 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,6", 2024-09-11T09:01:21.192 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.343523+0000", 2024-09-11T09:01:21.192 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000111548 2024-09-11T09:01:21.192 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.192 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.193 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.193 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.350779+0000", 2024-09-11T09:01:21.193 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0072566460000000003 2024-09-11T09:01:21.193 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.193 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.193 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:21.193 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.350932+0000", 2024-09-11T09:01:21.193 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000152578 2024-09-11T09:01:21.193 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.194 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.194 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.194 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.575671+0000", 2024-09-11T09:01:21.194 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.22473905299999999 2024-09-11T09:01:21.194 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.194 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.194 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:21.194 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.575737+0000", 2024-09-11T09:01:21.194 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6239000000000001e-05 2024-09-11T09:01:21.194 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.195 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.195 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.195 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:55:10.575784+0000", 2024-09-11T09:01:21.195 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7122e-05 2024-09-11T09:01:21.195 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.195 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.195 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.195 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.195 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.196 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14448.0:2415 5.1f 5:f95f44c2:::notify.0:head [watch reconnect cookie 94136053430192 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T09:01:21.196 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.450928+0000", 2024-09-11T09:01:21.196 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 313.70700154899998, 2024-09-11T09:01:21.196 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.034864671, 2024-09-11T09:01:21.196 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.196 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:21.197 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.197 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14448", 2024-09-11T09:01:21.197 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3990203193", 2024-09-11T09:01:21.197 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 2415 2024-09-11T09:01:21.197 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.197 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.197 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.197 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.197 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450928+0000", 2024-09-11T09:01:21.197 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.197 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.197 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.197 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.197 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450928+0000", 2024-09-11T09:01:21.198 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.198 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.198 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.198 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.198 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450926+0000", 2024-09-11T09:01:21.198 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T09:01:21.198 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.198 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.198 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.199 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450932+0000", 2024-09-11T09:01:21.199 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2349999999999996e-06 2024-09-11T09:01:21.199 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.199 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.199 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.199 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450944+0000", 2024-09-11T09:01:21.199 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.136e-05 2024-09-11T09:01:21.199 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.199 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.199 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.200 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450949+0000", 2024-09-11T09:01:21.200 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3489999999999999e-06 2024-09-11T09:01:21.200 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.200 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.200 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.200 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.451189+0000", 2024-09-11T09:01:21.200 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000239423 2024-09-11T09:01:21.200 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.200 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.200 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op must wait for map", 2024-09-11T09:01:21.200 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.451195+0000", 2024-09-11T09:01:21.200 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4189999999999999e-06 2024-09-11T09:01:21.200 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.201 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.201 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.201 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.457535+0000", 2024-09-11T09:01:21.201 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0063400519999999997 2024-09-11T09:01:21.201 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.201 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.201 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:21.201 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.457543+0000", 2024-09-11T09:01:21.201 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.9300000000000003e-06 2024-09-11T09:01:21.201 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.201 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.201 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.201 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.480385+0000", 2024-09-11T09:01:21.201 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0228418239999999 2024-09-11T09:01:21.202 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.202 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.202 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.202 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.480643+0000", 2024-09-11T09:01:21.202 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000258624 2024-09-11T09:01:21.202 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.202 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.202 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.202 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.485793+0000", 2024-09-11T09:01:21.202 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0051494269999999998 2024-09-11T09:01:21.202 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.203 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.203 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.203 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.203 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.203 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14448.0:2417 5.9 5:93e5b521:::notify.7:head [watch reconnect cookie 94136053465648 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T09:01:21.203 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.450972+0000", 2024-09-11T09:01:21.203 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 313.70695775000002, 2024-09-11T09:01:21.203 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0359644020000001, 2024-09-11T09:01:21.203 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.203 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:21.203 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.203 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14448", 2024-09-11T09:01:21.203 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3990203193", 2024-09-11T09:01:21.203 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 2417 2024-09-11T09:01:21.204 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.204 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.204 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.204 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.204 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450972+0000", 2024-09-11T09:01:21.204 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.204 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.204 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.204 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.204 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450972+0000", 2024-09-11T09:01:21.204 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.205 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.205 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.205 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.205 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450971+0000", 2024-09-11T09:01:21.205 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:21.205 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.205 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.205 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.205 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450975+0000", 2024-09-11T09:01:21.205 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.117e-06 2024-09-11T09:01:21.205 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.205 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.205 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.206 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450981+0000", 2024-09-11T09:01:21.206 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9090000000000003e-06 2024-09-11T09:01:21.206 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.206 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.206 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.206 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450986+0000", 2024-09-11T09:01:21.206 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2270000000000004e-06 2024-09-11T09:01:21.206 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.206 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.206 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.206 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.451059+0000", 2024-09-11T09:01:21.206 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2923999999999995e-05 2024-09-11T09:01:21.206 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.207 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.207 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op must wait for map", 2024-09-11T09:01:21.207 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.451070+0000", 2024-09-11T09:01:21.207 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1797000000000001e-05 2024-09-11T09:01:21.207 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.207 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.207 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.207 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.464512+0000", 2024-09-11T09:01:21.207 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013441761999999999 2024-09-11T09:01:21.207 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.208 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.208 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:21.208 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.464521+0000", 2024-09-11T09:01:21.208 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.85e-06 2024-09-11T09:01:21.208 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.208 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.208 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.208 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.481720+0000", 2024-09-11T09:01:21.208 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0171985960000001 2024-09-11T09:01:21.208 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.208 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.208 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.208 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.481871+0000", 2024-09-11T09:01:21.208 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015174700000000001 2024-09-11T09:01:21.209 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.209 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.209 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.209 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.486936+0000", 2024-09-11T09:01:21.209 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0050649980000000002 2024-09-11T09:01:21.209 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.209 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.209 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.209 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.209 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.209 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244894 10.d 10:b5f3791e:::benchmark_data_smithi092_173109_object107935:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T09:01:21.209 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.461820+0000", 2024-09-11T09:01:21.209 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 313.69610996300003, 2024-09-11T09:01:21.210 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0661809230000001, 2024-09-11T09:01:21.210 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.210 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T09:01:21.210 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.210 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:21.210 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:21.210 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244894 2024-09-11T09:01:21.210 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.210 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.210 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.210 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.210 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461820+0000", 2024-09-11T09:01:21.211 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.211 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.211 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.211 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.211 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461820+0000", 2024-09-11T09:01:21.211 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.211 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.211 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.211 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.211 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461819+0000", 2024-09-11T09:01:21.211 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:21.211 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.212 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.212 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.212 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461823+0000", 2024-09-11T09:01:21.212 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7960000000000003e-06 2024-09-11T09:01:21.212 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.212 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.212 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.212 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461834+0000", 2024-09-11T09:01:21.212 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0325e-05 2024-09-11T09:01:21.212 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.212 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.212 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.212 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461839+0000", 2024-09-11T09:01:21.213 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7529999999999998e-06 2024-09-11T09:01:21.213 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.213 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.213 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.213 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.462395+0000", 2024-09-11T09:01:21.213 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000555238 2024-09-11T09:01:21.213 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.213 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.214 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:21.214 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.462403+0000", 2024-09-11T09:01:21.214 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.1300000000000001e-06 2024-09-11T09:01:21.214 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.214 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.214 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.214 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.525952+0000", 2024-09-11T09:01:21.214 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.063549627 2024-09-11T09:01:21.214 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.214 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.214 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.214 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.528001+0000", 2024-09-11T09:01:21.214 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0020481840000000002 2024-09-11T09:01:21.214 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.215 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.215 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.215 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.215 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.215 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244917 10.d 10:b66065da:::benchmark_data_smithi092_173109_object107958:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T09:01:21.215 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.479915+0000", 2024-09-11T09:01:21.215 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 313.67801472799999, 2024-09-11T09:01:21.215 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.068724894, 2024-09-11T09:01:21.215 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.215 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:21.215 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.215 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:21.215 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:21.216 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244917 2024-09-11T09:01:21.216 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.216 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.216 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.216 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.216 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.479915+0000", 2024-09-11T09:01:21.216 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.216 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.216 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.216 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.216 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.479915+0000", 2024-09-11T09:01:21.216 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.216 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.216 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.217 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.217 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.479914+0000", 2024-09-11T09:01:21.217 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:21.217 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.217 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.217 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.217 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.479919+0000", 2024-09-11T09:01:21.217 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9609999999999996e-06 2024-09-11T09:01:21.217 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.217 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.217 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.217 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.479929+0000", 2024-09-11T09:01:21.217 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.7489999999999993e-06 2024-09-11T09:01:21.218 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.218 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.218 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.218 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.479933+0000", 2024-09-11T09:01:21.218 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2339999999999996e-06 2024-09-11T09:01:21.218 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.218 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.218 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.218 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.479988+0000", 2024-09-11T09:01:21.218 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5248999999999999e-05 2024-09-11T09:01:21.218 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.218 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.218 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:21.219 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.479998+0000", 2024-09-11T09:01:21.219 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.008e-05 2024-09-11T09:01:21.219 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.219 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.219 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.219 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.528028+0000", 2024-09-11T09:01:21.219 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0480296259999999 2024-09-11T09:01:21.219 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.219 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.219 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.219 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.528627+0000", 2024-09-11T09:01:21.219 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00059896799999999996 2024-09-11T09:01:21.219 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.219 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.220 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T09:01:21.221 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,6", 2024-09-11T09:01:21.221 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.528954+0000", 2024-09-11T09:01:21.221 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00032738299999999999 2024-09-11T09:01:21.221 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.222 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.222 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:21.222 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.545019+0000", 2024-09-11T09:01:21.222 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.016064750999999999 2024-09-11T09:01:21.222 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.222 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.222 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.222 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.545889+0000", 2024-09-11T09:01:21.222 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00087030300000000001 2024-09-11T09:01:21.222 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.222 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.222 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.222 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.548498+0000", 2024-09-11T09:01:21.223 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0026087530000000001 2024-09-11T09:01:21.223 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.223 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.223 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:21.223 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.548602+0000", 2024-09-11T09:01:21.223 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000103711 2024-09-11T09:01:21.223 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.223 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.223 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.223 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.548640+0000", 2024-09-11T09:01:21.223 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8034999999999998e-05 2024-09-11T09:01:21.223 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.223 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.223 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.224 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.224 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.224 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244918 10.d 10:bd6b17d1:::benchmark_data_smithi092_173109_object107959:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T09:01:21.224 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.483339+0000", 2024-09-11T09:01:21.224 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 313.67459053900001, 2024-09-11T09:01:21.224 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.065437599, 2024-09-11T09:01:21.224 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.224 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:21.224 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.224 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:21.224 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:21.224 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244918 2024-09-11T09:01:21.224 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.224 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.225 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.225 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.225 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.483339+0000", 2024-09-11T09:01:21.225 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.225 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.225 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.225 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.225 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.483339+0000", 2024-09-11T09:01:21.225 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.225 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.225 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.225 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.225 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.483339+0000", 2024-09-11T09:01:21.226 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:21.226 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.226 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.226 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.226 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.483343+0000", 2024-09-11T09:01:21.226 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3390000000000003e-06 2024-09-11T09:01:21.226 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.226 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.226 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.226 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.483350+0000", 2024-09-11T09:01:21.226 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.3490000000000003e-06 2024-09-11T09:01:21.226 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.226 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.227 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.227 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.483354+0000", 2024-09-11T09:01:21.227 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3919999999999999e-06 2024-09-11T09:01:21.227 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.227 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.227 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.227 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.483390+0000", 2024-09-11T09:01:21.227 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6739999999999997e-05 2024-09-11T09:01:21.227 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.227 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.227 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:21.227 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.483396+0000", 2024-09-11T09:01:21.227 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1200000000000001e-06 2024-09-11T09:01:21.228 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.228 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.228 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.228 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.538486+0000", 2024-09-11T09:01:21.228 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0550904480000001 2024-09-11T09:01:21.228 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.228 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.228 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.228 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.538602+0000", 2024-09-11T09:01:21.228 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011561300000000001 2024-09-11T09:01:21.228 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.228 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.228 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,6", 2024-09-11T09:01:21.228 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.538727+0000", 2024-09-11T09:01:21.229 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012533600000000001 2024-09-11T09:01:21.229 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.229 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.229 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:21.229 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.545040+0000", 2024-09-11T09:01:21.229 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0063130729999999998 2024-09-11T09:01:21.229 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.229 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.229 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.229 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.548680+0000", 2024-09-11T09:01:21.229 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0036400780000000002 2024-09-11T09:01:21.229 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.229 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.229 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.230 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.548709+0000", 2024-09-11T09:01:21.230 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9264999999999998e-05 2024-09-11T09:01:21.230 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.230 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.230 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:21.230 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.548745+0000", 2024-09-11T09:01:21.230 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5927999999999999e-05 2024-09-11T09:01:21.230 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.230 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.230 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.230 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.548777+0000", 2024-09-11T09:01:21.230 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1486000000000003e-05 2024-09-11T09:01:21.230 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.231 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.231 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.231 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.231 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.231 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244925 10.d 10:b9d19df4:::benchmark_data_smithi092_173109_object107966:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T09:01:21.231 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.496138+0000", 2024-09-11T09:01:21.231 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 313.66179135700003, 2024-09-11T09:01:21.231 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0539990480000001, 2024-09-11T09:01:21.231 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.231 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:21.231 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.231 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:21.231 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:21.231 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244925 2024-09-11T09:01:21.232 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.232 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.232 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.232 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.232 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.496138+0000", 2024-09-11T09:01:21.232 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.232 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.232 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.232 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.232 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.496138+0000", 2024-09-11T09:01:21.232 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.232 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.232 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.233 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.233 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.496137+0000", 2024-09-11T09:01:21.233 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:21.233 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.233 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.233 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.233 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.496143+0000", 2024-09-11T09:01:21.233 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5040000000000002e-06 2024-09-11T09:01:21.233 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.233 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.233 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.233 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.496154+0000", 2024-09-11T09:01:21.233 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1333e-05 2024-09-11T09:01:21.234 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.234 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.234 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.234 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.496160+0000", 2024-09-11T09:01:21.234 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0950000000000001e-06 2024-09-11T09:01:21.234 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.234 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.234 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.234 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.496210+0000", 2024-09-11T09:01:21.234 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9610000000000001e-05 2024-09-11T09:01:21.234 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.234 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.234 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:21.234 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.496228+0000", 2024-09-11T09:01:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.8068999999999999e-05 2024-09-11T09:01:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.541561+0000", 2024-09-11T09:01:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0453332360000001 2024-09-11T09:01:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.541674+0000", 2024-09-11T09:01:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011276 2024-09-11T09:01:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,6", 2024-09-11T09:01:21.236 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.541812+0000", 2024-09-11T09:01:21.236 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000137897 2024-09-11T09:01:21.236 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.236 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.236 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:21.236 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.545045+0000", 2024-09-11T09:01:21.236 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003233568 2024-09-11T09:01:21.236 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.236 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.236 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.236 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.548806+0000", 2024-09-11T09:01:21.236 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003760289 2024-09-11T09:01:21.236 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.550060+0000", 2024-09-11T09:01:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0012548749999999999 2024-09-11T09:01:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.550102+0000", 2024-09-11T09:01:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1971000000000002e-05 2024-09-11T09:01:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.550137+0000", 2024-09-11T09:01:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5030000000000002e-05 2024-09-11T09:01:21.238 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.238 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.238 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.238 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.238 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.238 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14448.0:2422 5.1f 5:f95f44c2:::notify.0:head [watch ping cookie 94136053430192 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T09:01:21.238 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.870280+0000", 2024-09-11T09:01:21.238 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 313.28765004299999, 2024-09-11T09:01:21.239 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.61251766299999999, 2024-09-11T09:01:21.239 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.239 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:21.239 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.239 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14448", 2024-09-11T09:01:21.239 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3990203193", 2024-09-11T09:01:21.239 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 2422 2024-09-11T09:01:21.239 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.240 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.240 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.240 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.240 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870280+0000", 2024-09-11T09:01:21.240 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.240 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.240 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.240 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.240 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870280+0000", 2024-09-11T09:01:21.240 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.240 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.240 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.240 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.241 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870279+0000", 2024-09-11T09:01:21.241 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:21.241 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.241 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.241 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.241 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870283+0000", 2024-09-11T09:01:21.241 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2459999999999997e-06 2024-09-11T09:01:21.241 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.241 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.241 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.241 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870289+0000", 2024-09-11T09:01:21.241 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.302e-06 2024-09-11T09:01:21.242 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.242 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.242 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.242 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870296+0000", 2024-09-11T09:01:21.242 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8469999999999998e-06 2024-09-11T09:01:21.242 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.242 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.242 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.242 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870369+0000", 2024-09-11T09:01:21.242 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2342999999999995e-05 2024-09-11T09:01:21.242 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.243 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.243 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:21.243 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870379+0000", 2024-09-11T09:01:21.243 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0169999999999999e-05 2024-09-11T09:01:21.243 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.243 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.243 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.244 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.482569+0000", 2024-09-11T09:01:21.244 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.61218998700000005 2024-09-11T09:01:21.244 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.244 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.244 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.245 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.482644+0000", 2024-09-11T09:01:21.245 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4948000000000001e-05 2024-09-11T09:01:21.245 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.245 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.245 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.246 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.482797+0000", 2024-09-11T09:01:21.246 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015360999999999999 2024-09-11T09:01:21.246 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.246 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.246 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.246 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.247 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.247 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14448.0:2424 5.9 5:93e5b521:::notify.7:head [watch ping cookie 94136053465648 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T09:01:21.247 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.870347+0000", 2024-09-11T09:01:21.247 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 313.28758286300001, 2024-09-11T09:01:21.247 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.61277905799999999, 2024-09-11T09:01:21.247 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.248 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:21.248 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.248 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14448", 2024-09-11T09:01:21.248 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3990203193", 2024-09-11T09:01:21.248 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 2424 2024-09-11T09:01:21.248 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.249 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.249 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.249 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.249 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870347+0000", 2024-09-11T09:01:21.249 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.250 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.250 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.250 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.250 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870347+0000", 2024-09-11T09:01:21.250 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.250 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.251 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.251 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.251 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870346+0000", 2024-09-11T09:01:21.251 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:21.251 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.251 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.252 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.252 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870349+0000", 2024-09-11T09:01:21.252 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7700000000000002e-06 2024-09-11T09:01:21.252 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.252 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.252 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.252 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870355+0000", 2024-09-11T09:01:21.253 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8819999999999998e-06 2024-09-11T09:01:21.253 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.253 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.253 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.253 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870359+0000", 2024-09-11T09:01:21.253 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0929999999999996e-06 2024-09-11T09:01:21.253 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.254 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.254 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.254 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870411+0000", 2024-09-11T09:01:21.254 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1829000000000001e-05 2024-09-11T09:01:21.254 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.254 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.254 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:21.255 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.870419+0000", 2024-09-11T09:01:21.255 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6930000000000007e-06 2024-09-11T09:01:21.255 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.255 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.255 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.256 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.482932+0000", 2024-09-11T09:01:21.256 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.61251346399999995 2024-09-11T09:01:21.256 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.256 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.256 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.256 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.483032+0000", 2024-09-11T09:01:21.257 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000100299 2024-09-11T09:01:21.257 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.257 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.257 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.257 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.483126+0000", 2024-09-11T09:01:21.257 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.3410000000000002e-05 2024-09-11T09:01:21.258 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.258 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.258 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.258 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.258 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.258 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:236053 11.e 11:7b1b4c74:::benchmark_data_smithi092_205252_object114732:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T09:01:21.259 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:29.310490+0000", 2024-09-11T09:01:21.259 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 51.847439538000003, 2024-09-11T09:01:21.259 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.033290107999999999, 2024-09-11T09:01:21.259 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.259 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:21.259 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.260 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:21.260 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:21.260 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 236053 2024-09-11T09:01:21.260 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.260 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.260 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.261 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.261 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.310490+0000", 2024-09-11T09:01:21.261 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.261 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.261 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.261 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.261 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.310490+0000", 2024-09-11T09:01:21.262 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.262 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.262 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.262 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.262 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.310488+0000", 2024-09-11T09:01:21.262 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T09:01:21.262 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.262 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.263 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.263 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.310496+0000", 2024-09-11T09:01:21.263 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1350000000000003e-06 2024-09-11T09:01:21.263 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.263 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.263 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.263 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.310510+0000", 2024-09-11T09:01:21.264 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.398e-05 2024-09-11T09:01:21.264 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.264 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.264 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.264 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.310515+0000", 2024-09-11T09:01:21.264 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6659999999999996e-06 2024-09-11T09:01:21.264 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.265 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.265 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.265 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.310559+0000", 2024-09-11T09:01:21.265 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4063000000000001e-05 2024-09-11T09:01:21.265 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.266 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.266 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.266 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.310687+0000", 2024-09-11T09:01:21.266 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012820999999999999 2024-09-11T09:01:21.266 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.267 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.267 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,7", 2024-09-11T09:01:21.267 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.311104+0000", 2024-09-11T09:01:21.267 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00041638999999999999 2024-09-11T09:01:21.267 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.267 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.267 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:21.267 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.314028+0000", 2024-09-11T09:01:21.267 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0029245790000000001 2024-09-11T09:01:21.267 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.267 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.267 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.267 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.314334+0000", 2024-09-11T09:01:21.268 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00030540299999999999 2024-09-11T09:01:21.268 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.268 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.268 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.268 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.343693+0000", 2024-09-11T09:01:21.268 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.029358945000000001 2024-09-11T09:01:21.268 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.268 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.268 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:21.268 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.343743+0000", 2024-09-11T09:01:21.268 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9824000000000003e-05 2024-09-11T09:01:21.268 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.268 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.269 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.269 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.343780+0000", 2024-09-11T09:01:21.269 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.769e-05 2024-09-11T09:01:21.269 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.269 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.269 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.269 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.269 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.269 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:236393 11.e 11:77585a52:::benchmark_data_smithi092_205252_object115072:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T09:01:21.269 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:29.584567+0000", 2024-09-11T09:01:21.269 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 51.573363071999999, 2024-09-11T09:01:21.269 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.026905074000000001, 2024-09-11T09:01:21.269 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.269 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:21.270 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.270 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:21.270 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:21.270 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 236393 2024-09-11T09:01:21.270 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.270 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.270 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.270 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.270 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.584567+0000", 2024-09-11T09:01:21.270 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.270 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.270 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.270 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.270 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.584567+0000", 2024-09-11T09:01:21.271 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.271 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.271 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.271 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.271 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.584565+0000", 2024-09-11T09:01:21.271 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T09:01:21.271 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.271 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.271 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.271 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.584573+0000", 2024-09-11T09:01:21.271 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.1839999999999999e-06 2024-09-11T09:01:21.271 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.271 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.271 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.272 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.584587+0000", 2024-09-11T09:01:21.272 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3675999999999999e-05 2024-09-11T09:01:21.272 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.272 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.272 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.272 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.584592+0000", 2024-09-11T09:01:21.272 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3870000000000003e-06 2024-09-11T09:01:21.272 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.272 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.272 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.272 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.584649+0000", 2024-09-11T09:01:21.272 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6962000000000001e-05 2024-09-11T09:01:21.272 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.273 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.273 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.273 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.584771+0000", 2024-09-11T09:01:21.273 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000122342 2024-09-11T09:01:21.273 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.273 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.273 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,7", 2024-09-11T09:01:21.273 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.584883+0000", 2024-09-11T09:01:21.273 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011166399999999999 2024-09-11T09:01:21.273 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.273 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.273 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:21.273 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.587215+0000", 2024-09-11T09:01:21.273 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0023316000000000001 2024-09-11T09:01:21.274 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.274 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.274 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.274 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.587895+0000", 2024-09-11T09:01:21.274 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00068063300000000004 2024-09-11T09:01:21.274 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.274 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.274 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.274 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.611395+0000", 2024-09-11T09:01:21.274 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.023499296999999999 2024-09-11T09:01:21.274 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.274 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.274 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:21.274 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.611433+0000", 2024-09-11T09:01:21.275 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8046000000000001e-05 2024-09-11T09:01:21.275 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.275 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.275 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.275 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.611472+0000", 2024-09-11T09:01:21.275 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9088000000000003e-05 2024-09-11T09:01:21.275 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.275 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.275 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.275 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.275 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.275 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:237763 11.e 11:798054cc:::benchmark_data_smithi092_205252_object116442:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:21.275 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:30.657971+0000", 2024-09-11T09:01:21.275 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 50.499958581999998, 2024-09-11T09:01:21.275 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.031069701000000002, 2024-09-11T09:01:21.276 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.276 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:21.276 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.276 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:21.276 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:21.276 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 237763 2024-09-11T09:01:21.276 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.276 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.276 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.276 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.276 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.657971+0000", 2024-09-11T09:01:21.276 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.276 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.276 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.277 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.277 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.657971+0000", 2024-09-11T09:01:21.277 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.277 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.277 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.277 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.277 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.657971+0000", 2024-09-11T09:01:21.277 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:21.277 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.277 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.277 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.277 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.657974+0000", 2024-09-11T09:01:21.277 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:01:21.278 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.278 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.278 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.278 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.657981+0000", 2024-09-11T09:01:21.278 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4059999999999999e-06 2024-09-11T09:01:21.278 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.278 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.278 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.278 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.657986+0000", 2024-09-11T09:01:21.278 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6129999999999997e-06 2024-09-11T09:01:21.278 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.278 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.278 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.279 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.660796+0000", 2024-09-11T09:01:21.279 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0028103849999999999 2024-09-11T09:01:21.279 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.279 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.279 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.279 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.660902+0000", 2024-09-11T09:01:21.279 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000105198 2024-09-11T09:01:21.279 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.279 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.279 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,7", 2024-09-11T09:01:21.279 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.661014+0000", 2024-09-11T09:01:21.279 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011254799999999999 2024-09-11T09:01:21.279 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.279 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.280 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:21.280 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.663629+0000", 2024-09-11T09:01:21.280 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.002615081 2024-09-11T09:01:21.280 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.280 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.280 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.280 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.665498+0000", 2024-09-11T09:01:21.280 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001868544 2024-09-11T09:01:21.280 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.280 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.280 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.280 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.688973+0000", 2024-09-11T09:01:21.280 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.023474918000000001 2024-09-11T09:01:21.281 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.281 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.281 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:21.281 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.689005+0000", 2024-09-11T09:01:21.281 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2375000000000001e-05 2024-09-11T09:01:21.281 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.281 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.281 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.281 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.689041+0000", 2024-09-11T09:01:21.281 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5858000000000001e-05 2024-09-11T09:01:21.281 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.281 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.281 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.281 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.282 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.282 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:239509 11.e 11:74dc1856:::benchmark_data_smithi092_205252_object118188:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:21.282 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:32.029396+0000", 2024-09-11T09:01:21.282 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 49.128533625999999, 2024-09-11T09:01:21.282 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.035319151999999999, 2024-09-11T09:01:21.282 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.282 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:21.282 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.282 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:21.282 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:21.282 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 239509 2024-09-11T09:01:21.282 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.282 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.283 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.283 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.283 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.029396+0000", 2024-09-11T09:01:21.283 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.283 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.283 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.283 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.283 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.029396+0000", 2024-09-11T09:01:21.283 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.283 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.283 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.283 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.283 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.029394+0000", 2024-09-11T09:01:21.283 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T09:01:21.284 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.284 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.284 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.284 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.029400+0000", 2024-09-11T09:01:21.284 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7679999999999997e-06 2024-09-11T09:01:21.284 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.284 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.284 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.284 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.029415+0000", 2024-09-11T09:01:21.284 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5315000000000002e-05 2024-09-11T09:01:21.284 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.284 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.284 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.285 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.029421+0000", 2024-09-11T09:01:21.285 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1449999999999996e-06 2024-09-11T09:01:21.285 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.285 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.285 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.285 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.035174+0000", 2024-09-11T09:01:21.285 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0057523280000000001 2024-09-11T09:01:21.285 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.285 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.285 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.285 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.035283+0000", 2024-09-11T09:01:21.285 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000109664 2024-09-11T09:01:21.285 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.285 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.286 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,7", 2024-09-11T09:01:21.286 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.035394+0000", 2024-09-11T09:01:21.286 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000110627 2024-09-11T09:01:21.286 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.286 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.286 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.286 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.057346+0000", 2024-09-11T09:01:21.286 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.021952372000000001 2024-09-11T09:01:21.286 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.286 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.286 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.286 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.057822+0000", 2024-09-11T09:01:21.286 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00047576999999999998 2024-09-11T09:01:21.286 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.287 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.287 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:21.287 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.064463+0000", 2024-09-11T09:01:21.287 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0066408919999999998 2024-09-11T09:01:21.287 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.287 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.287 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:21.287 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.064597+0000", 2024-09-11T09:01:21.287 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013359299999999999 2024-09-11T09:01:21.287 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.287 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.287 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.287 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.064715+0000", 2024-09-11T09:01:21.287 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000118707 2024-09-11T09:01:21.288 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.289 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.289 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.289 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.289 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.289 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:239517 11.e 11:7cedb95e:::benchmark_data_smithi092_205252_object118196:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:21.289 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:32.034825+0000", 2024-09-11T09:01:21.289 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 49.123104521999998, 2024-09-11T09:01:21.289 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.030124472999999999, 2024-09-11T09:01:21.289 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.289 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:21.289 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.289 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:21.290 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:21.290 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 239517 2024-09-11T09:01:21.290 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.290 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.290 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.290 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.290 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.034825+0000", 2024-09-11T09:01:21.290 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.290 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.290 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.290 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.290 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.034825+0000", 2024-09-11T09:01:21.290 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.290 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.291 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.291 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.291 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.034823+0000", 2024-09-11T09:01:21.291 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T09:01:21.291 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.291 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.291 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.291 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.034831+0000", 2024-09-11T09:01:21.291 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6299999999999998e-06 2024-09-11T09:01:21.291 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.291 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.291 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.291 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.034845+0000", 2024-09-11T09:01:21.291 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3474e-05 2024-09-11T09:01:21.292 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.292 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.292 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.292 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.034851+0000", 2024-09-11T09:01:21.292 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4320000000000004e-06 2024-09-11T09:01:21.292 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.292 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.292 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.292 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.037920+0000", 2024-09-11T09:01:21.292 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0030695369999999998 2024-09-11T09:01:21.292 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.292 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.292 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.292 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.038207+0000", 2024-09-11T09:01:21.293 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00028669299999999997 2024-09-11T09:01:21.293 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.293 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.293 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,7", 2024-09-11T09:01:21.293 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.038548+0000", 2024-09-11T09:01:21.293 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00034038900000000002 2024-09-11T09:01:21.293 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.293 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.293 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.293 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.057670+0000", 2024-09-11T09:01:21.293 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.019122516999999999 2024-09-11T09:01:21.293 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.293 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.294 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.294 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.057954+0000", 2024-09-11T09:01:21.294 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00028372500000000003 2024-09-11T09:01:21.294 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.294 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.294 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:21.294 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.064738+0000", 2024-09-11T09:01:21.294 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0067842830000000003 2024-09-11T09:01:21.294 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.294 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.294 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:21.294 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.064843+0000", 2024-09-11T09:01:21.294 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010444499999999999 2024-09-11T09:01:21.294 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.295 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.295 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.295 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.064950+0000", 2024-09-11T09:01:21.295 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000107142 2024-09-11T09:01:21.295 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.295 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.295 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.295 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.295 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.295 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:239520 11.e 11:72e31567:::benchmark_data_smithi092_205252_object118199:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:21.295 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:32.035827+0000", 2024-09-11T09:01:21.295 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 49.122103189999997, 2024-09-11T09:01:21.295 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.029668168000000002, 2024-09-11T09:01:21.295 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.296 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:21.296 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.296 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:21.296 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:21.296 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 239520 2024-09-11T09:01:21.296 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.296 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.296 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.296 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.296 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.035827+0000", 2024-09-11T09:01:21.296 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.296 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.296 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.297 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.297 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.035827+0000", 2024-09-11T09:01:21.297 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.297 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.297 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.297 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.297 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.035826+0000", 2024-09-11T09:01:21.297 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:21.297 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.297 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.297 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.297 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.035832+0000", 2024-09-11T09:01:21.297 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0809999999999999e-06 2024-09-11T09:01:21.297 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.298 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.298 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.298 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.035845+0000", 2024-09-11T09:01:21.298 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3740999999999999e-05 2024-09-11T09:01:21.298 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.298 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.298 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.298 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.035851+0000", 2024-09-11T09:01:21.298 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6300000000000003e-06 2024-09-11T09:01:21.298 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.298 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.298 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.298 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.041733+0000", 2024-09-11T09:01:21.298 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0058816650000000003 2024-09-11T09:01:21.299 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.299 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.299 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.299 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.042011+0000", 2024-09-11T09:01:21.299 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00027795000000000001 2024-09-11T09:01:21.299 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.299 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.299 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,7", 2024-09-11T09:01:21.299 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.042336+0000", 2024-09-11T09:01:21.299 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00032580499999999997 2024-09-11T09:01:21.299 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.299 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.299 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.299 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.058109+0000", 2024-09-11T09:01:21.300 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.015772234 2024-09-11T09:01:21.300 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.300 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.300 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.300 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.058263+0000", 2024-09-11T09:01:21.300 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015418300000000001 2024-09-11T09:01:21.300 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.300 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.300 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:21.300 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.064976+0000", 2024-09-11T09:01:21.300 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0067130469999999998 2024-09-11T09:01:21.300 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.300 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.300 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:21.301 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.065302+0000", 2024-09-11T09:01:21.301 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00032614699999999999 2024-09-11T09:01:21.301 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.301 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.301 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.301 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.065495+0000", 2024-09-11T09:01:21.301 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000192683 2024-09-11T09:01:21.301 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.301 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.301 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.301 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.301 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.301 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:239521 11.e 11:7f6cf0bf:::benchmark_data_smithi092_205252_object118200:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:21.302 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:32.038857+0000", 2024-09-11T09:01:21.302 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 49.119072367000001, 2024-09-11T09:01:21.302 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.026825025999999998, 2024-09-11T09:01:21.302 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.302 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:21.302 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.302 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:21.302 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:21.302 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 239521 2024-09-11T09:01:21.302 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.302 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.302 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.302 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.302 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.038857+0000", 2024-09-11T09:01:21.303 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.303 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.303 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.303 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.303 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.038857+0000", 2024-09-11T09:01:21.303 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.303 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.303 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.303 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.303 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.038856+0000", 2024-09-11T09:01:21.303 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:21.303 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.303 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.303 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.304 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.038863+0000", 2024-09-11T09:01:21.304 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1930000000000004e-06 2024-09-11T09:01:21.304 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.304 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.304 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.304 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.038891+0000", 2024-09-11T09:01:21.304 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.8320999999999999e-05 2024-09-11T09:01:21.304 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.304 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.304 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.304 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.038898+0000", 2024-09-11T09:01:21.304 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6250000000000001e-06 2024-09-11T09:01:21.304 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.304 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.305 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.305 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.052053+0000", 2024-09-11T09:01:21.305 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013155205999999999 2024-09-11T09:01:21.305 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.305 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.305 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.305 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.052276+0000", 2024-09-11T09:01:21.305 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00022228699999999999 2024-09-11T09:01:21.305 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.305 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.305 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,7", 2024-09-11T09:01:21.305 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.052522+0000", 2024-09-11T09:01:21.305 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00024643400000000002 2024-09-11T09:01:21.306 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.306 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.306 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.306 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.058438+0000", 2024-09-11T09:01:21.306 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0059165210000000001 2024-09-11T09:01:21.306 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.306 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.306 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:21.306 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.065520+0000", 2024-09-11T09:01:21.306 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.007081899 2024-09-11T09:01:21.306 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.306 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.306 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.306 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.065603+0000", 2024-09-11T09:01:21.307 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2546000000000006e-05 2024-09-11T09:01:21.307 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.307 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.307 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:21.307 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.065648+0000", 2024-09-11T09:01:21.307 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.4728000000000003e-05 2024-09-11T09:01:21.307 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.307 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.307 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.307 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.065682+0000", 2024-09-11T09:01:21.307 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4724999999999997e-05 2024-09-11T09:01:21.307 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.307 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.308 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.308 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.308 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.308 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:239528 11.e 11:7c68ab9e:::benchmark_data_smithi092_205252_object118207:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:21.308 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:32.043657+0000", 2024-09-11T09:01:21.308 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 49.114272634000002, 2024-09-11T09:01:21.308 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.029881673000000001, 2024-09-11T09:01:21.308 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.308 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:21.308 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.308 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:21.308 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:21.308 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 239528 2024-09-11T09:01:21.308 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.309 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.309 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.309 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.309 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.043657+0000", 2024-09-11T09:01:21.309 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.309 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.309 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.309 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.309 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.043657+0000", 2024-09-11T09:01:21.309 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.309 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.309 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.309 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.310 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.043655+0000", 2024-09-11T09:01:21.310 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:21.310 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.310 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.310 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.310 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.043666+0000", 2024-09-11T09:01:21.310 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0827e-05 2024-09-11T09:01:21.310 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.310 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.310 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.310 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.043684+0000", 2024-09-11T09:01:21.310 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7856e-05 2024-09-11T09:01:21.310 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.310 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.311 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.311 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.043691+0000", 2024-09-11T09:01:21.311 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9600000000000003e-06 2024-09-11T09:01:21.311 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.311 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.311 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.311 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.058561+0000", 2024-09-11T09:01:21.311 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.014869795999999999 2024-09-11T09:01:21.311 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.311 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.311 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.311 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.058837+0000", 2024-09-11T09:01:21.311 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00027618400000000001 2024-09-11T09:01:21.311 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.312 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.312 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,7", 2024-09-11T09:01:21.312 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.059187+0000", 2024-09-11T09:01:21.312 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00035020599999999999 2024-09-11T09:01:21.312 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.312 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.312 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:21.312 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.065540+0000", 2024-09-11T09:01:21.312 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0063529889999999999 2024-09-11T09:01:21.312 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.312 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.312 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.312 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.065722+0000", 2024-09-11T09:01:21.312 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00018183699999999999 2024-09-11T09:01:21.313 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.313 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.313 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.313 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.073167+0000", 2024-09-11T09:01:21.313 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0074449030000000001 2024-09-11T09:01:21.313 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.313 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.313 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:21.313 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.073374+0000", 2024-09-11T09:01:21.313 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000207414 2024-09-11T09:01:21.313 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.313 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.313 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.313 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.073539+0000", 2024-09-11T09:01:21.314 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00016436700000000001 2024-09-11T09:01:21.314 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.314 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.314 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.314 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.314 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.314 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:241004 11.e 11:7134175e:::benchmark_data_smithi092_205252_object119683:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:21.314 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:33.222337+0000", 2024-09-11T09:01:21.314 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 47.935592622999998, 2024-09-11T09:01:21.314 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.028851396000000001, 2024-09-11T09:01:21.314 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:21.314 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:21.314 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:21.315 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:21.315 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:21.315 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 241004 2024-09-11T09:01:21.315 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.315 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:21.315 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.315 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:21.315 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.222337+0000", 2024-09-11T09:01:21.315 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.315 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.315 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.315 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:21.315 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.222337+0000", 2024-09-11T09:01:21.315 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:21.316 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.316 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.316 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:21.316 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.222337+0000", 2024-09-11T09:01:21.316 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:21.316 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.316 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.317 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:21.317 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.222340+0000", 2024-09-11T09:01:21.317 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0960000000000001e-06 2024-09-11T09:01:21.317 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.317 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.317 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:21.318 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.222351+0000", 2024-09-11T09:01:21.318 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.095e-05 2024-09-11T09:01:21.318 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.318 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.318 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:21.318 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.222355+0000", 2024-09-11T09:01:21.319 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9029999999999997e-06 2024-09-11T09:01:21.319 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.319 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.319 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:21.319 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.225085+0000", 2024-09-11T09:01:21.319 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0027303570000000001 2024-09-11T09:01:21.320 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.320 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.320 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:21.320 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.226346+0000", 2024-09-11T09:01:21.320 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0012612280000000001 2024-09-11T09:01:21.320 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.320 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.320 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,7", 2024-09-11T09:01:21.320 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.226453+0000", 2024-09-11T09:01:21.320 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010678799999999999 2024-09-11T09:01:21.321 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.321 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.321 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:21.321 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.229321+0000", 2024-09-11T09:01:21.321 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0028684779999999998 2024-09-11T09:01:21.321 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.321 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.321 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.321 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.230460+0000", 2024-09-11T09:01:21.321 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0011387280000000001 2024-09-11T09:01:21.321 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.321 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.321 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:21.321 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.251103+0000", 2024-09-11T09:01:21.322 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.020643172000000001 2024-09-11T09:01:21.322 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.322 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.322 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:21.322 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.251152+0000", 2024-09-11T09:01:21.322 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9005999999999998e-05 2024-09-11T09:01:21.322 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:21.322 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:21.322 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:21.322 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.251188+0000", 2024-09-11T09:01:21.322 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6189999999999997e-05 2024-09-11T09:01:21.322 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.322 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.323 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.323 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:21.323 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:21.323 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T09:01:21.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:01:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:01:21] "GET /metrics HTTP/1.1" 200 222491 "" "Prometheus/2.43.0" 2024-09-11T09:01:21.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:21 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T09:01:21.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:21 smithi092 ceph-mon[41114]: osdmap e437: 8 total, 8 up, 8 in 2024-09-11T09:01:21.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:21 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/4018219405' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T09:01:21.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:21 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T09:01:21.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:21 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T09:01:21.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:21 smithi092 ceph-mon[29647]: osdmap e437: 8 total, 8 up, 8 in 2024-09-11T09:01:21.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:21 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/4018219405' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T09:01:21.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:21 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T09:01:21.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:21 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T09:01:21.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:21 smithi145 ceph-mon[33384]: osdmap e437: 8 total, 8 up, 8 in 2024-09-11T09:01:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:21 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/4018219405' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T09:01:21.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:21 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T09:01:21.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 0 1 2024-09-11T09:01:22.078 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-11T09:01:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:22.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[29647]: 6.8 scrub starts 2024-09-11T09:01:22.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[29647]: 6.8 scrub ok 2024-09-11T09:01:22.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[29647]: 5.12 scrub starts 2024-09-11T09:01:22.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[29647]: 5.12 scrub ok 2024-09-11T09:01:22.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T09:01:22.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[29647]: osdmap e438: 8 total, 8 up, 8 in 2024-09-11T09:01:22.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[29647]: 2.2 scrub starts 2024-09-11T09:01:22.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[29647]: pgmap v1035: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:22.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[29647]: 2.2 scrub ok 2024-09-11T09:01:22.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[41114]: 6.8 scrub starts 2024-09-11T09:01:22.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[41114]: 6.8 scrub ok 2024-09-11T09:01:22.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[41114]: 5.12 scrub starts 2024-09-11T09:01:22.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[41114]: 5.12 scrub ok 2024-09-11T09:01:22.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T09:01:22.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[41114]: osdmap e438: 8 total, 8 up, 8 in 2024-09-11T09:01:22.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[41114]: 2.2 scrub starts 2024-09-11T09:01:22.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[41114]: pgmap v1035: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:22.317 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:22 smithi092 ceph-mon[41114]: 2.2 scrub ok 2024-09-11T09:01:22.331 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T09:01:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:22 smithi145 ceph-mon[33384]: 6.8 scrub starts 2024-09-11T09:01:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:22 smithi145 ceph-mon[33384]: 6.8 scrub ok 2024-09-11T09:01:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:22 smithi145 ceph-mon[33384]: 5.12 scrub starts 2024-09-11T09:01:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:22 smithi145 ceph-mon[33384]: 5.12 scrub ok 2024-09-11T09:01:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:22 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T09:01:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:22 smithi145 ceph-mon[33384]: osdmap e438: 8 total, 8 up, 8 in 2024-09-11T09:01:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:22 smithi145 ceph-mon[33384]: 2.2 scrub starts 2024-09-11T09:01:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:22 smithi145 ceph-mon[33384]: pgmap v1035: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:22.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:22 smithi145 ceph-mon[33384]: 2.2 scrub ok 2024-09-11T09:01:23.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:01:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:01:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:01:23.525 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:23 smithi145 ceph-mon[33384]: 4.19 scrub starts 2024-09-11T09:01:23.525 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:23 smithi145 ceph-mon[33384]: 4.19 scrub ok 2024-09-11T09:01:23.736 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:23 smithi092 ceph-mon[29647]: 4.19 scrub starts 2024-09-11T09:01:23.736 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:23 smithi092 ceph-mon[29647]: 4.19 scrub ok 2024-09-11T09:01:23.736 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:23 smithi092 ceph-mon[41114]: 4.19 scrub starts 2024-09-11T09:01:23.736 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:23 smithi092 ceph-mon[41114]: 4.19 scrub ok 2024-09-11T09:01:24.020 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T09:01:24.020 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T09:01:24.020 INFO:teuthology.orchestra.run.smithi145.stdout: "num_ops": 0 2024-09-11T09:01:24.020 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T09:01:24.381 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-11T09:01:24.633 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:24 smithi145 ceph-mon[33384]: 6.1a scrub starts 2024-09-11T09:01:24.633 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:24 smithi145 ceph-mon[33384]: 6.1a scrub ok 2024-09-11T09:01:24.633 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:24 smithi145 ceph-mon[33384]: 6.9 scrub starts 2024-09-11T09:01:24.633 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:24 smithi145 ceph-mon[33384]: 6.9 scrub ok 2024-09-11T09:01:24.633 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:24 smithi145 ceph-mon[33384]: pgmap v1036: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:24.633 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:24 smithi145 ceph-mon[33384]: 3.10 scrub starts 2024-09-11T09:01:24.634 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:24 smithi145 ceph-mon[33384]: 3.10 scrub ok 2024-09-11T09:01:24.634 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:24 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/373945263' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T09:01:24.658 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-11T09:01:24.731 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[29647]: 6.1a scrub starts 2024-09-11T09:01:24.732 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[29647]: 6.1a scrub ok 2024-09-11T09:01:24.732 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[29647]: 6.9 scrub starts 2024-09-11T09:01:24.732 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[29647]: 6.9 scrub ok 2024-09-11T09:01:24.732 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[29647]: pgmap v1036: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:24.732 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[29647]: 3.10 scrub starts 2024-09-11T09:01:24.732 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[29647]: 3.10 scrub ok 2024-09-11T09:01:24.732 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/373945263' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T09:01:24.732 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[41114]: 6.1a scrub starts 2024-09-11T09:01:24.733 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[41114]: 6.1a scrub ok 2024-09-11T09:01:24.733 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[41114]: 6.9 scrub starts 2024-09-11T09:01:24.733 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[41114]: 6.9 scrub ok 2024-09-11T09:01:24.733 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[41114]: pgmap v1036: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:24.733 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[41114]: 3.10 scrub starts 2024-09-11T09:01:24.733 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[41114]: 3.10 scrub ok 2024-09-11T09:01:24.733 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:24 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/373945263' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T09:01:24.901 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T09:01:25.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 1 1 2024-09-11T09:01:25.425 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:25.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:25 smithi145 ceph-mon[33384]: 4.1a scrub starts 2024-09-11T09:01:25.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:25 smithi145 ceph-mon[33384]: 4.1a scrub ok 2024-09-11T09:01:25.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:25 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/373945263' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-11T09:01:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:25 smithi145 ceph-mon[33384]: osdmap e439: 8 total, 8 up, 8 in 2024-09-11T09:01:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:25 smithi145 ceph-mon[33384]: 3.1c scrub starts 2024-09-11T09:01:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:25 smithi145 ceph-mon[33384]: 3.1c scrub ok 2024-09-11T09:01:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:25 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:01:25.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:25 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:01:25.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[29647]: 4.1a scrub starts 2024-09-11T09:01:25.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[29647]: 4.1a scrub ok 2024-09-11T09:01:25.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/373945263' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-11T09:01:25.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[29647]: osdmap e439: 8 total, 8 up, 8 in 2024-09-11T09:01:25.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[29647]: 3.1c scrub starts 2024-09-11T09:01:25.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[29647]: 3.1c scrub ok 2024-09-11T09:01:25.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:01:25.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:01:25.667 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[41114]: 4.1a scrub starts 2024-09-11T09:01:25.667 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[41114]: 4.1a scrub ok 2024-09-11T09:01:25.667 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/373945263' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-11T09:01:25.667 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[41114]: osdmap e439: 8 total, 8 up, 8 in 2024-09-11T09:01:25.667 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[41114]: 3.1c scrub starts 2024-09-11T09:01:25.667 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[41114]: 3.1c scrub ok 2024-09-11T09:01:25.667 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:01:25.667 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:25 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:01:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[29647]: 4.13 scrub starts 2024-09-11T09:01:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[29647]: 4.13 scrub ok 2024-09-11T09:01:26.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[29647]: osdmap e440: 8 total, 8 up, 8 in 2024-09-11T09:01:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[29647]: pgmap v1039: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 964 B/s rd, 0 op/s 2024-09-11T09:01:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[29647]: 5.0 deep-scrub starts 2024-09-11T09:01:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[29647]: 5.0 deep-scrub ok 2024-09-11T09:01:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[29647]: 5.1d scrub starts 2024-09-11T09:01:26.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[29647]: 5.1d scrub ok 2024-09-11T09:01:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[41114]: 4.13 scrub starts 2024-09-11T09:01:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[41114]: 4.13 scrub ok 2024-09-11T09:01:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[41114]: osdmap e440: 8 total, 8 up, 8 in 2024-09-11T09:01:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[41114]: pgmap v1039: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 964 B/s rd, 0 op/s 2024-09-11T09:01:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[41114]: 5.0 deep-scrub starts 2024-09-11T09:01:26.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[41114]: 5.0 deep-scrub ok 2024-09-11T09:01:26.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[41114]: 5.1d scrub starts 2024-09-11T09:01:26.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:26 smithi092 ceph-mon[41114]: 5.1d scrub ok 2024-09-11T09:01:26.628 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T09:01:26.628 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T09:01:26.628 INFO:teuthology.orchestra.run.smithi145.stdout: "complaint_time": 30, 2024-09-11T09:01:26.628 INFO:teuthology.orchestra.run.smithi145.stdout: "num_blocked_ops": 0 2024-09-11T09:01:26.629 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T09:01:26.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:26 smithi145 ceph-mon[33384]: 4.13 scrub starts 2024-09-11T09:01:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:26 smithi145 ceph-mon[33384]: 4.13 scrub ok 2024-09-11T09:01:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:26 smithi145 ceph-mon[33384]: osdmap e440: 8 total, 8 up, 8 in 2024-09-11T09:01:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:26 smithi145 ceph-mon[33384]: pgmap v1039: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 964 B/s rd, 0 op/s 2024-09-11T09:01:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:26 smithi145 ceph-mon[33384]: 5.0 deep-scrub starts 2024-09-11T09:01:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:26 smithi145 ceph-mon[33384]: 5.0 deep-scrub ok 2024-09-11T09:01:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:26 smithi145 ceph-mon[33384]: 5.1d scrub starts 2024-09-11T09:01:26.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:26 smithi145 ceph-mon[33384]: 5.1d scrub ok 2024-09-11T09:01:27.299 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-11T09:01:27.487 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.1 to 1 (10000) 2024-09-11T09:01:27.548 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T09:01:27.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:27 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/332974502' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T09:01:27.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:27 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T09:01:27.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:27 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/332974502' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T09:01:27.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:27 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T09:01:27.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:01:27 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:01:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:27 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/332974502' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T09:01:27.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:27 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T09:01:28.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 1 1 2024-09-11T09:01:28.500 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:28.500 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T09:01:28.500 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[41114]: osdmap e441: 8 total, 8 up, 8 in 2024-09-11T09:01:28.500 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[41114]: pgmap v1041: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:01:28.500 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[41114]: 3.1f scrub starts 2024-09-11T09:01:28.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[41114]: 3.1f scrub ok 2024-09-11T09:01:28.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[41114]: 4.1c scrub starts 2024-09-11T09:01:28.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[41114]: 4.1c scrub ok 2024-09-11T09:01:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:28.698 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:28 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:28.698 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:28 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T09:01:28.698 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:28 smithi145 ceph-mon[33384]: osdmap e441: 8 total, 8 up, 8 in 2024-09-11T09:01:28.698 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:28 smithi145 ceph-mon[33384]: pgmap v1041: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:01:28.698 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:28 smithi145 ceph-mon[33384]: 3.1f scrub starts 2024-09-11T09:01:28.698 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:28 smithi145 ceph-mon[33384]: 3.1f scrub ok 2024-09-11T09:01:28.698 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:28 smithi145 ceph-mon[33384]: 4.1c scrub starts 2024-09-11T09:01:28.699 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:28 smithi145 ceph-mon[33384]: 4.1c scrub ok 2024-09-11T09:01:28.751 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:28.751 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T09:01:28.751 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[29647]: osdmap e441: 8 total, 8 up, 8 in 2024-09-11T09:01:28.751 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[29647]: pgmap v1041: 152 pgs: 152 active+clean; 582 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:01:28.751 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[29647]: 3.1f scrub starts 2024-09-11T09:01:28.751 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[29647]: 3.1f scrub ok 2024-09-11T09:01:28.751 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[29647]: 4.1c scrub starts 2024-09-11T09:01:28.752 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:28 smithi092 ceph-mon[29647]: 4.1c scrub ok 2024-09-11T09:01:29.251 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T09:01:29.251 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T09:01:29.251 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T09:01:29.252 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T09:01:29.252 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.252 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272038 9.12 9:4c1395bf:::benchmark_data_smithi092_145083_object128640:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T09:01:29.252 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.599637+0000", 2024-09-11T09:01:29.252 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 545.63489110199998, 2024-09-11T09:01:29.252 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0332527600000001, 2024-09-11T09:01:29.252 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.252 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.252 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.252 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T09:01:29.252 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T09:01:29.252 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272038 2024-09-11T09:01:29.253 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.253 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.253 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.253 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.253 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599637+0000", 2024-09-11T09:01:29.253 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.253 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.253 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.253 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.253 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599637+0000", 2024-09-11T09:01:29.253 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.253 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.254 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.254 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.254 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599635+0000", 2024-09-11T09:01:29.254 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T09:01:29.254 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.254 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.254 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.254 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599643+0000", 2024-09-11T09:01:29.254 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.1759999999999993e-06 2024-09-11T09:01:29.254 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.254 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.254 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.254 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599660+0000", 2024-09-11T09:01:29.255 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7118000000000001e-05 2024-09-11T09:01:29.255 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.255 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.255 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.255 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599667+0000", 2024-09-11T09:01:29.255 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8260000000000004e-06 2024-09-11T09:01:29.255 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.255 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.255 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.255 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599717+0000", 2024-09-11T09:01:29.255 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0219000000000001e-05 2024-09-11T09:01:29.255 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.256 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.256 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op must wait for map", 2024-09-11T09:01:29.256 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.599724+0000", 2024-09-11T09:01:29.256 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4130000000000003e-06 2024-09-11T09:01:29.256 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.256 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.256 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.256 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.611082+0000", 2024-09-11T09:01:29.257 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.01135824 2024-09-11T09:01:29.257 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.257 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.257 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:29.257 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.611093+0000", 2024-09-11T09:01:29.257 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0729e-05 2024-09-11T09:01:29.257 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.257 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.257 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.257 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.627111+0000", 2024-09-11T09:01:29.257 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0160185660000001 2024-09-11T09:01:29.257 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.258 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.258 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.258 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.627211+0000", 2024-09-11T09:01:29.258 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.9397999999999999e-05 2024-09-11T09:01:29.258 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.258 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.258 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T09:01:29.259 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.627282+0000", 2024-09-11T09:01:29.259 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1217000000000006e-05 2024-09-11T09:01:29.259 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.259 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.259 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.259 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.631283+0000", 2024-09-11T09:01:29.259 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0040011389999999999 2024-09-11T09:01:29.259 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.259 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.259 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.259 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.632714+0000", 2024-09-11T09:01:29.260 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0014306360000000001 2024-09-11T09:01:29.260 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.260 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.260 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.260 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.632844+0000", 2024-09-11T09:01:29.260 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013048000000000001 2024-09-11T09:01:29.260 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.260 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.260 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.260 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.632869+0000", 2024-09-11T09:01:29.260 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4278999999999999e-05 2024-09-11T09:01:29.260 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.261 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.261 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.261 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.632890+0000", 2024-09-11T09:01:29.261 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.1382e-05 2024-09-11T09:01:29.261 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.261 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.261 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.261 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.261 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.261 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272048 9.12 9:4dde8efd:::benchmark_data_smithi092_145083_object128650:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T09:01:29.262 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.614452+0000", 2024-09-11T09:01:29.262 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 545.62007606199995, 2024-09-11T09:01:29.262 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.031362157, 2024-09-11T09:01:29.262 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.262 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.262 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.262 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T09:01:29.262 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T09:01:29.262 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272048 2024-09-11T09:01:29.263 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.263 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.263 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.263 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.263 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614452+0000", 2024-09-11T09:01:29.263 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.263 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.263 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.263 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.263 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614452+0000", 2024-09-11T09:01:29.263 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.263 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.264 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.264 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.264 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614451+0000", 2024-09-11T09:01:29.264 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:29.264 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.264 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.264 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.264 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614458+0000", 2024-09-11T09:01:29.264 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4889999999999998e-06 2024-09-11T09:01:29.264 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.264 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.265 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.265 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614474+0000", 2024-09-11T09:01:29.265 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5764e-05 2024-09-11T09:01:29.265 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.265 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.265 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.265 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614481+0000", 2024-09-11T09:01:29.265 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.7859999999999999e-06 2024-09-11T09:01:29.265 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.265 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.265 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.265 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614573+0000", 2024-09-11T09:01:29.265 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.2504999999999996e-05 2024-09-11T09:01:29.266 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.266 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.266 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:29.266 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.614581+0000", 2024-09-11T09:01:29.266 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.7680000000000008e-06 2024-09-11T09:01:29.266 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.266 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.266 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.266 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.629158+0000", 2024-09-11T09:01:29.266 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.014577072 2024-09-11T09:01:29.267 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.267 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.267 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.267 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.629222+0000", 2024-09-11T09:01:29.267 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3751000000000007e-05 2024-09-11T09:01:29.267 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.267 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.268 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T09:01:29.268 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.629291+0000", 2024-09-11T09:01:29.268 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9078999999999994e-05 2024-09-11T09:01:29.268 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.268 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.268 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.268 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.634043+0000", 2024-09-11T09:01:29.268 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0047520009999999996 2024-09-11T09:01:29.268 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.268 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.268 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.269 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.634048+0000", 2024-09-11T09:01:29.269 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.0679999999999996e-06 2024-09-11T09:01:29.269 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.269 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.269 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.269 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.645758+0000", 2024-09-11T09:01:29.269 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011710124000000001 2024-09-11T09:01:29.269 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.269 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.269 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.270 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.645789+0000", 2024-09-11T09:01:29.270 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1124000000000002e-05 2024-09-11T09:01:29.270 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.270 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.270 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.270 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.645814+0000", 2024-09-11T09:01:29.270 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5244000000000001e-05 2024-09-11T09:01:29.270 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.270 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.270 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.270 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.270 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.270 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272066 9.12 9:4968a467:::benchmark_data_smithi092_145083_object128668:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T09:01:29.270 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.629493+0000", 2024-09-11T09:01:29.271 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 545.60503520700001, 2024-09-11T09:01:29.271 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0189994769999999, 2024-09-11T09:01:29.271 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.271 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.271 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.271 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T09:01:29.271 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T09:01:29.271 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272066 2024-09-11T09:01:29.271 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.271 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.271 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.272 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.272 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629493+0000", 2024-09-11T09:01:29.272 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.272 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.272 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.272 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.272 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629493+0000", 2024-09-11T09:01:29.272 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.272 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.272 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.272 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.272 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629492+0000", 2024-09-11T09:01:29.272 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:29.273 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.273 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.273 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.273 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629499+0000", 2024-09-11T09:01:29.273 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4390000000000003e-06 2024-09-11T09:01:29.273 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.273 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.273 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.273 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629516+0000", 2024-09-11T09:01:29.273 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6617e-05 2024-09-11T09:01:29.273 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.273 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.273 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.274 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629522+0000", 2024-09-11T09:01:29.274 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2009999999999996e-06 2024-09-11T09:01:29.274 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.274 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.274 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.274 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629586+0000", 2024-09-11T09:01:29.274 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4300000000000004e-05 2024-09-11T09:01:29.274 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.274 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.274 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:29.274 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.629594+0000", 2024-09-11T09:01:29.275 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.5959999999999998e-06 2024-09-11T09:01:29.275 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.275 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.275 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.275 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.630309+0000", 2024-09-11T09:01:29.276 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0007151240000001 2024-09-11T09:01:29.276 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.276 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.276 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.276 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.630368+0000", 2024-09-11T09:01:29.276 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9344e-05 2024-09-11T09:01:29.277 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.277 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.277 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T09:01:29.277 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.630435+0000", 2024-09-11T09:01:29.277 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6285999999999995e-05 2024-09-11T09:01:29.277 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.278 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.278 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.278 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.635154+0000", 2024-09-11T09:01:29.278 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0047192019999999996 2024-09-11T09:01:29.278 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.278 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.279 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.279 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.635174+0000", 2024-09-11T09:01:29.279 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0749000000000001e-05 2024-09-11T09:01:29.279 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.279 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.279 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.280 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648406+0000", 2024-09-11T09:01:29.280 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013231463000000001 2024-09-11T09:01:29.280 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.280 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.280 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.280 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648455+0000", 2024-09-11T09:01:29.281 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9174e-05 2024-09-11T09:01:29.281 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.281 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.281 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.281 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648493+0000", 2024-09-11T09:01:29.282 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7438999999999998e-05 2024-09-11T09:01:29.282 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.282 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.282 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.282 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.282 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.283 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272071 9.12 9:4c320382:::benchmark_data_smithi092_145083_object128673:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T09:01:29.283 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.632070+0000", 2024-09-11T09:01:29.283 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 545.60245792700005, 2024-09-11T09:01:29.283 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.016575234, 2024-09-11T09:01:29.283 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.283 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.283 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.284 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T09:01:29.284 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T09:01:29.284 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272071 2024-09-11T09:01:29.284 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.284 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.284 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.284 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.284 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632070+0000", 2024-09-11T09:01:29.284 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.284 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.284 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.284 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.284 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632070+0000", 2024-09-11T09:01:29.285 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.285 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.285 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.285 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.285 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632070+0000", 2024-09-11T09:01:29.285 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:29.285 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.285 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.285 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.285 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632074+0000", 2024-09-11T09:01:29.285 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9110000000000001e-06 2024-09-11T09:01:29.285 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.285 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.286 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.286 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632083+0000", 2024-09-11T09:01:29.286 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.6370000000000005e-06 2024-09-11T09:01:29.286 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.286 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.286 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.286 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632089+0000", 2024-09-11T09:01:29.286 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6810000000000003e-06 2024-09-11T09:01:29.286 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.286 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.286 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.286 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632285+0000", 2024-09-11T09:01:29.286 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00019620999999999999 2024-09-11T09:01:29.286 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.287 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.287 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:29.287 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.632292+0000", 2024-09-11T09:01:29.287 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5289999999999997e-06 2024-09-11T09:01:29.287 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.287 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.287 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.287 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.631306+0000", 2024-09-11T09:01:29.287 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99901447799999998 2024-09-11T09:01:29.287 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.287 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.287 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.287 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.631400+0000", 2024-09-11T09:01:29.287 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.4510000000000001e-05 2024-09-11T09:01:29.288 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.288 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.288 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T09:01:29.288 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.631493+0000", 2024-09-11T09:01:29.288 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.2555999999999999e-05 2024-09-11T09:01:29.288 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.288 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.288 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.288 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.635158+0000", 2024-09-11T09:01:29.288 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0036650089999999999 2024-09-11T09:01:29.288 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.288 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.288 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.289 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.643385+0000", 2024-09-11T09:01:29.289 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0082264529999999999 2024-09-11T09:01:29.289 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.289 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.289 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.289 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648566+0000", 2024-09-11T09:01:29.289 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0051818469999999998 2024-09-11T09:01:29.289 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.289 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.289 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.289 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648610+0000", 2024-09-11T09:01:29.289 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3442999999999997e-05 2024-09-11T09:01:29.290 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.290 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.290 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.290 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648646+0000", 2024-09-11T09:01:29.290 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5800000000000003e-05 2024-09-11T09:01:29.290 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.290 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.290 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.290 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.290 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.290 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15366.0:272083 9.12 9:4e00b3e7:::benchmark_data_smithi092_145083_object128685:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e280)", 2024-09-11T09:01:29.290 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:52:23.640138+0000", 2024-09-11T09:01:29.290 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 545.59438985099996, 2024-09-11T09:01:29.291 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.008620208, 2024-09-11T09:01:29.291 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.291 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.291 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.291 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15366", 2024-09-11T09:01:29.291 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/1639244674", 2024-09-11T09:01:29.291 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 272083 2024-09-11T09:01:29.291 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.291 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.291 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.291 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.291 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640138+0000", 2024-09-11T09:01:29.291 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.292 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.292 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.292 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.292 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640138+0000", 2024-09-11T09:01:29.292 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.292 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.292 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.292 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.292 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640137+0000", 2024-09-11T09:01:29.292 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:29.292 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.293 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.293 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.293 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640144+0000", 2024-09-11T09:01:29.293 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4889999999999998e-06 2024-09-11T09:01:29.293 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.293 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.293 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.293 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640156+0000", 2024-09-11T09:01:29.293 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1718e-05 2024-09-11T09:01:29.293 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.293 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.293 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.293 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640162+0000", 2024-09-11T09:01:29.294 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8159999999999999e-06 2024-09-11T09:01:29.294 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.294 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.294 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.294 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640202+0000", 2024-09-11T09:01:29.294 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0370999999999997e-05 2024-09-11T09:01:29.294 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.294 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.294 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:29.294 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:23.640210+0000", 2024-09-11T09:01:29.295 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4039999999999998e-06 2024-09-11T09:01:29.295 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.295 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.295 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.295 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.632939+0000", 2024-09-11T09:01:29.295 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99272926299999997 2024-09-11T09:01:29.295 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.295 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.295 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.295 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.633019+0000", 2024-09-11T09:01:29.295 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.9963000000000007e-05 2024-09-11T09:01:29.295 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.295 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.295 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T09:01:29.296 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.633114+0000", 2024-09-11T09:01:29.296 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.5613999999999996e-05 2024-09-11T09:01:29.296 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.296 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.296 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.296 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.636389+0000", 2024-09-11T09:01:29.296 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0032742420000000001 2024-09-11T09:01:29.296 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.296 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.297 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.297 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.643483+0000", 2024-09-11T09:01:29.297 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0070939260000000004 2024-09-11T09:01:29.297 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.297 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.297 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.297 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648683+0000", 2024-09-11T09:01:29.297 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0052004089999999996 2024-09-11T09:01:29.297 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.297 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.297 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.297 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648726+0000", 2024-09-11T09:01:29.297 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3457000000000001e-05 2024-09-11T09:01:29.298 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.298 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.298 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.298 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:52:24.648759+0000", 2024-09-11T09:01:29.298 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2200999999999999e-05 2024-09-11T09:01:29.298 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.298 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.298 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.298 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.298 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.298 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:115609 10.3 10:c6226159:::benchmark_data_smithi092_173109_object115608:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T09:01:29.299 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:16.116557+0000", 2024-09-11T09:01:29.299 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 433.11797123700001, 2024-09-11T09:01:29.299 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0273496980000001, 2024-09-11T09:01:29.299 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.299 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T09:01:29.299 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.299 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:29.299 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:29.299 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 115609 2024-09-11T09:01:29.299 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.299 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.299 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.299 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.299 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116557+0000", 2024-09-11T09:01:29.300 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.300 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.300 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.300 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.300 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116557+0000", 2024-09-11T09:01:29.300 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.300 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.300 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.300 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.300 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116557+0000", 2024-09-11T09:01:29.300 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:29.300 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.300 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.301 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.301 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116631+0000", 2024-09-11T09:01:29.301 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4128000000000006e-05 2024-09-11T09:01:29.301 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.301 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.301 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.301 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116640+0000", 2024-09-11T09:01:29.301 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.4280000000000002e-06 2024-09-11T09:01:29.301 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.301 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.301 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.301 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116644+0000", 2024-09-11T09:01:29.301 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3139999999999997e-06 2024-09-11T09:01:29.301 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.302 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.302 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.302 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116788+0000", 2024-09-11T09:01:29.302 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000143425 2024-09-11T09:01:29.302 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.302 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.302 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:29.302 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.116794+0000", 2024-09-11T09:01:29.302 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.618e-06 2024-09-11T09:01:29.303 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.303 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.303 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.303 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.124868+0000", 2024-09-11T09:01:29.303 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0080731979999999 2024-09-11T09:01:29.303 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.303 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.303 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for missing object", 2024-09-11T09:01:29.303 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.124955+0000", 2024-09-11T09:01:29.303 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.7103999999999998e-05 2024-09-11T09:01:29.303 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.303 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.304 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.304 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.143857+0000", 2024-09-11T09:01:29.304 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.018902209999999999 2024-09-11T09:01:29.304 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.304 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.304 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.304 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.143907+0000", 2024-09-11T09:01:29.304 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9792e-05 2024-09-11T09:01:29.304 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.304 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.304 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.304 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.304 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.304 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:115621 10.3 10:ca57ac07:::benchmark_data_smithi092_173109_object115620:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T09:01:29.305 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:16.127920+0000", 2024-09-11T09:01:29.305 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 433.10660783399999, 2024-09-11T09:01:29.305 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.016159238, 2024-09-11T09:01:29.305 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.305 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.305 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.305 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:29.305 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:29.305 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 115621 2024-09-11T09:01:29.305 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.305 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.305 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.306 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.306 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.127920+0000", 2024-09-11T09:01:29.306 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.306 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.306 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.306 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.306 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.127920+0000", 2024-09-11T09:01:29.306 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.307 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.307 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.307 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.307 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.127919+0000", 2024-09-11T09:01:29.307 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:29.307 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.307 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.307 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.307 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.128018+0000", 2024-09-11T09:01:29.308 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.9053000000000002e-05 2024-09-11T09:01:29.308 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.308 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.308 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.308 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.128032+0000", 2024-09-11T09:01:29.308 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.4107000000000001e-05 2024-09-11T09:01:29.308 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.309 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.309 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.309 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.128038+0000", 2024-09-11T09:01:29.309 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8030000000000002e-06 2024-09-11T09:01:29.309 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.309 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.309 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.309 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.128095+0000", 2024-09-11T09:01:29.310 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6344000000000001e-05 2024-09-11T09:01:29.310 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.310 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.310 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:29.310 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.128106+0000", 2024-09-11T09:01:29.310 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1985e-05 2024-09-11T09:01:29.310 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.310 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.310 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.311 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.124982+0000", 2024-09-11T09:01:29.311 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99687584600000001 2024-09-11T09:01:29.311 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.311 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.311 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.311 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.125098+0000", 2024-09-11T09:01:29.311 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000115944 2024-09-11T09:01:29.311 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.312 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.312 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,3", 2024-09-11T09:01:29.312 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.125523+0000", 2024-09-11T09:01:29.312 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00042452599999999999 2024-09-11T09:01:29.312 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.312 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.312 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.312 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.134514+0000", 2024-09-11T09:01:29.313 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0089912310000000006 2024-09-11T09:01:29.313 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.313 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.313 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.313 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.143952+0000", 2024-09-11T09:01:29.313 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0094376159999999994 2024-09-11T09:01:29.313 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.314 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.314 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.314 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.143992+0000", 2024-09-11T09:01:29.314 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9969000000000003e-05 2024-09-11T09:01:29.314 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.314 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.314 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.314 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.144020+0000", 2024-09-11T09:01:29.315 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.8540000000000001e-05 2024-09-11T09:01:29.315 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.315 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.315 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.315 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.144080+0000", 2024-09-11T09:01:29.315 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9534999999999998e-05 2024-09-11T09:01:29.315 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.315 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.315 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.316 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.316 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.316 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:115626 10.3 10:ceb6be78:::benchmark_data_smithi092_173109_object115625:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T09:01:29.316 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:16.133512+0000", 2024-09-11T09:01:29.316 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 433.10101638800001, 2024-09-11T09:01:29.316 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0108069260000001, 2024-09-11T09:01:29.316 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.316 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.317 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.317 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:29.317 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:29.317 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 115626 2024-09-11T09:01:29.317 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.317 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.317 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.317 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.318 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133512+0000", 2024-09-11T09:01:29.318 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.318 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.318 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.318 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.318 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133512+0000", 2024-09-11T09:01:29.318 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.318 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.318 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.319 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.319 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133510+0000", 2024-09-11T09:01:29.319 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:29.319 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.319 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.319 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.319 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133565+0000", 2024-09-11T09:01:29.319 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4809999999999999e-05 2024-09-11T09:01:29.320 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.320 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.320 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.320 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133582+0000", 2024-09-11T09:01:29.320 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.715e-05 2024-09-11T09:01:29.320 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.320 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.320 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.320 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133590+0000", 2024-09-11T09:01:29.321 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2679999999999997e-06 2024-09-11T09:01:29.321 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.321 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.321 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.321 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133653+0000", 2024-09-11T09:01:29.321 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3268999999999993e-05 2024-09-11T09:01:29.321 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.321 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.322 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:29.322 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.133662+0000", 2024-09-11T09:01:29.322 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.9299999999999992e-06 2024-09-11T09:01:29.322 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.322 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.322 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.322 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.132033+0000", 2024-09-11T09:01:29.322 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.99837082200000005 2024-09-11T09:01:29.322 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.322 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.322 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.322 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.132120+0000", 2024-09-11T09:01:29.322 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.7138999999999994e-05 2024-09-11T09:01:29.323 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.323 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.323 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,3", 2024-09-11T09:01:29.323 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.132222+0000", 2024-09-11T09:01:29.323 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010197800000000001 2024-09-11T09:01:29.323 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.323 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.323 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.323 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.137321+0000", 2024-09-11T09:01:29.323 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0050989479999999998 2024-09-11T09:01:29.323 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.323 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.323 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.323 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.144111+0000", 2024-09-11T09:01:29.324 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0067903490000000002 2024-09-11T09:01:29.324 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.324 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.324 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.324 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.144194+0000", 2024-09-11T09:01:29.324 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2897000000000003e-05 2024-09-11T09:01:29.324 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.324 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.324 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.324 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.144226+0000", 2024-09-11T09:01:29.324 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2466999999999999e-05 2024-09-11T09:01:29.326 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.326 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.326 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.326 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.144319+0000", 2024-09-11T09:01:29.326 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.2509999999999993e-05 2024-09-11T09:01:29.326 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.326 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.326 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.326 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.327 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.327 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:115640 10.3 10:c94ff23e:::benchmark_data_smithi092_173109_object115639:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T09:01:29.327 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:54:16.141138+0000", 2024-09-11T09:01:29.327 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 433.09338994500001, 2024-09-11T09:01:29.327 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0107861440000001, 2024-09-11T09:01:29.327 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.327 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.327 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.327 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:29.327 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:29.327 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 115640 2024-09-11T09:01:29.328 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.328 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.328 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.328 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.328 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.141138+0000", 2024-09-11T09:01:29.328 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.328 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.328 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.328 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.328 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.141138+0000", 2024-09-11T09:01:29.328 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.328 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.328 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.328 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.329 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.141137+0000", 2024-09-11T09:01:29.329 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:29.329 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.329 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.329 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.329 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.141259+0000", 2024-09-11T09:01:29.329 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000121923 2024-09-11T09:01:29.329 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.329 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.329 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.329 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.141267+0000", 2024-09-11T09:01:29.329 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.5939999999999994e-06 2024-09-11T09:01:29.329 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.330 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.330 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.330 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.141272+0000", 2024-09-11T09:01:29.330 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7770000000000002e-06 2024-09-11T09:01:29.330 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.330 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.330 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.330 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.152765+0000", 2024-09-11T09:01:29.330 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011493412 2024-09-11T09:01:29.330 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.330 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.330 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:29.330 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:16.152775+0000", 2024-09-11T09:01:29.330 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.7699999999999996e-06 2024-09-11T09:01:29.331 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.331 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.331 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.331 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.133348+0000", 2024-09-11T09:01:29.331 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.98057306799999999 2024-09-11T09:01:29.331 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.331 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.331 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.331 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.133429+0000", 2024-09-11T09:01:29.331 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.0222000000000005e-05 2024-09-11T09:01:29.331 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.331 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.331 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,3", 2024-09-11T09:01:29.332 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.133520+0000", 2024-09-11T09:01:29.332 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.1508999999999994e-05 2024-09-11T09:01:29.332 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.332 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.332 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.332 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.139818+0000", 2024-09-11T09:01:29.332 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0062979840000000004 2024-09-11T09:01:29.332 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.332 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.332 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.332 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.150267+0000", 2024-09-11T09:01:29.332 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.010448737 2024-09-11T09:01:29.333 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.333 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.333 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.333 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.151828+0000", 2024-09-11T09:01:29.333 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0015615220000000001 2024-09-11T09:01:29.333 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.333 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.333 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.333 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.151863+0000", 2024-09-11T09:01:29.333 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4477999999999998e-05 2024-09-11T09:01:29.333 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.333 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.334 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.334 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:54:17.151924+0000", 2024-09-11T09:01:29.334 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1764e-05 2024-09-11T09:01:29.334 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.334 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.334 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.334 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.334 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.334 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.14448.0:2416 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 94136053445360 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T09:01:29.334 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.450967+0000", 2024-09-11T09:01:29.334 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 321.78356098799998, 2024-09-11T09:01:29.334 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0358244990000001, 2024-09-11T09:01:29.334 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.335 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:29.335 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.335 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.14448", 2024-09-11T09:01:29.335 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/3990203193", 2024-09-11T09:01:29.335 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 2416 2024-09-11T09:01:29.335 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.335 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.335 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.335 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.335 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450967+0000", 2024-09-11T09:01:29.335 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.335 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.335 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.336 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.336 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450967+0000", 2024-09-11T09:01:29.336 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.336 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.336 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.336 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.336 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450965+0000", 2024-09-11T09:01:29.336 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T09:01:29.336 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.336 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.336 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.336 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450973+0000", 2024-09-11T09:01:29.336 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.3540000000000002e-06 2024-09-11T09:01:29.337 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.337 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.337 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.337 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450985+0000", 2024-09-11T09:01:29.337 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2714999999999999e-05 2024-09-11T09:01:29.337 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.337 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.337 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.337 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.450991+0000", 2024-09-11T09:01:29.337 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8429999999999998e-06 2024-09-11T09:01:29.337 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.337 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.337 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.338 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.451040+0000", 2024-09-11T09:01:29.338 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9215000000000002e-05 2024-09-11T09:01:29.338 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.338 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.338 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op must wait for map", 2024-09-11T09:01:29.338 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.451047+0000", 2024-09-11T09:01:29.338 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:01:29.338 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.338 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.338 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.338 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.462274+0000", 2024-09-11T09:01:29.338 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.011227098 2024-09-11T09:01:29.338 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.338 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.339 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:29.339 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.462284+0000", 2024-09-11T09:01:29.339 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.9499999999999996e-06 2024-09-11T09:01:29.339 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.339 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.339 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.339 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.483120+0000", 2024-09-11T09:01:29.339 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0208363410000001 2024-09-11T09:01:29.339 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.339 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.339 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.339 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.483249+0000", 2024-09-11T09:01:29.339 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000128658 2024-09-11T09:01:29.339 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.340 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.340 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.340 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.486792+0000", 2024-09-11T09:01:29.340 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0035430489999999999 2024-09-11T09:01:29.340 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.340 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.340 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.340 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.340 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.340 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244899 10.a 10:55016aaa:::benchmark_data_smithi092_173109_object107940:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T09:01:29.340 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.461819+0000", 2024-09-11T09:01:29.340 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 321.77270965500003, 2024-09-11T09:01:29.340 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0650427090000001, 2024-09-11T09:01:29.341 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.341 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T09:01:29.341 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.341 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:29.341 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:29.341 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244899 2024-09-11T09:01:29.341 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.341 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.341 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.341 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.341 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461819+0000", 2024-09-11T09:01:29.341 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.341 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.342 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.342 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.342 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461819+0000", 2024-09-11T09:01:29.342 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.342 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.342 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.342 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.342 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461817+0000", 2024-09-11T09:01:29.342 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T09:01:29.342 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.342 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.343 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.343 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461824+0000", 2024-09-11T09:01:29.343 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6669999999999999e-06 2024-09-11T09:01:29.343 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.343 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.343 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.343 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461834+0000", 2024-09-11T09:01:29.343 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.029e-05 2024-09-11T09:01:29.343 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.343 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.343 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.343 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.461840+0000", 2024-09-11T09:01:29.343 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7830000000000004e-06 2024-09-11T09:01:29.344 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.344 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.344 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.344 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.466710+0000", 2024-09-11T09:01:29.344 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.004870592 2024-09-11T09:01:29.344 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.344 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.344 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:29.344 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.466719+0000", 2024-09-11T09:01:29.344 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8759999999999994e-06 2024-09-11T09:01:29.345 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.345 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.345 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.345 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.524699+0000", 2024-09-11T09:01:29.345 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0579804180000001 2024-09-11T09:01:29.345 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.345 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.345 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.345 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.526861+0000", 2024-09-11T09:01:29.345 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0021618840000000002 2024-09-11T09:01:29.345 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.345 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.345 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.346 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.346 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.346 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244920 10.a 10:501159a3:::benchmark_data_smithi092_173109_object107961:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e345)", 2024-09-11T09:01:29.346 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.484501+0000", 2024-09-11T09:01:29.346 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 321.750027649, 2024-09-11T09:01:29.346 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0561753199999999, 2024-09-11T09:01:29.346 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.346 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.346 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.346 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:29.346 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:29.346 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244920 2024-09-11T09:01:29.346 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.346 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.347 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.347 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.347 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484501+0000", 2024-09-11T09:01:29.347 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.347 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.347 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.347 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.347 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484501+0000", 2024-09-11T09:01:29.347 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.347 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.347 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.347 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.347 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484500+0000", 2024-09-11T09:01:29.347 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:29.348 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.348 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.348 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.348 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484505+0000", 2024-09-11T09:01:29.348 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8470000000000003e-06 2024-09-11T09:01:29.348 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.348 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.348 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.348 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484515+0000", 2024-09-11T09:01:29.348 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0786e-05 2024-09-11T09:01:29.348 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.348 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.348 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.349 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484521+0000", 2024-09-11T09:01:29.349 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3070000000000002e-06 2024-09-11T09:01:29.349 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.349 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.349 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.349 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484568+0000", 2024-09-11T09:01:29.349 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6848999999999998e-05 2024-09-11T09:01:29.349 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.349 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.349 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for peered", 2024-09-11T09:01:29.349 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.484576+0000", 2024-09-11T09:01:29.349 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.2349999999999999e-06 2024-09-11T09:01:29.349 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.349 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.350 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.350 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.527274+0000", 2024-09-11T09:01:29.350 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0426980720000001 2024-09-11T09:01:29.350 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.350 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.350 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.350 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.527843+0000", 2024-09-11T09:01:29.350 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00056964299999999999 2024-09-11T09:01:29.350 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.350 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.350 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,6", 2024-09-11T09:01:29.350 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.528148+0000", 2024-09-11T09:01:29.350 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00030435400000000001 2024-09-11T09:01:29.350 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.351 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.351 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.351 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.537204+0000", 2024-09-11T09:01:29.351 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0090564110000000003 2024-09-11T09:01:29.351 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.351 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.351 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.351 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.540357+0000", 2024-09-11T09:01:29.351 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0031531599999999999 2024-09-11T09:01:29.351 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.351 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.351 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.351 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.540384+0000", 2024-09-11T09:01:29.351 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.6625e-05 2024-09-11T09:01:29.352 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.352 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.352 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.352 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.540636+0000", 2024-09-11T09:01:29.352 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00025166300000000001 2024-09-11T09:01:29.352 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.352 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.352 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.352 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:08.540676+0000", 2024-09-11T09:01:29.352 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0258e-05 2024-09-11T09:01:29.352 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.352 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.352 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.353 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.353 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.353 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:234624 11.7 11:e1bb99d3:::benchmark_data_smithi092_205252_object113303:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T09:01:29.353 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:28.195181+0000", 2024-09-11T09:01:29.353 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 61.039347435000003, 2024-09-11T09:01:29.353 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.030873919999999999, 2024-09-11T09:01:29.353 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.353 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.353 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.353 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:29.353 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:29.353 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 234624 2024-09-11T09:01:29.353 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.353 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.354 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.354 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.354 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.195181+0000", 2024-09-11T09:01:29.354 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.354 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.354 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.354 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.354 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.195181+0000", 2024-09-11T09:01:29.354 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.354 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.354 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.354 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.354 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.195179+0000", 2024-09-11T09:01:29.354 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:29.355 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.355 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.355 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.355 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.195186+0000", 2024-09-11T09:01:29.355 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8889999999999996e-06 2024-09-11T09:01:29.355 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.355 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.355 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.355 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.195198+0000", 2024-09-11T09:01:29.355 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1810999999999999e-05 2024-09-11T09:01:29.355 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.355 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.355 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.356 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.195203+0000", 2024-09-11T09:01:29.356 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1089999999999997e-06 2024-09-11T09:01:29.356 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.356 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.356 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.356 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.195239+0000", 2024-09-11T09:01:29.356 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6040000000000001e-05 2024-09-11T09:01:29.356 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.356 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.356 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.356 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.195361+0000", 2024-09-11T09:01:29.356 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000121619 2024-09-11T09:01:29.356 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.357 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.357 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,4", 2024-09-11T09:01:29.357 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.195568+0000", 2024-09-11T09:01:29.357 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00020729599999999999 2024-09-11T09:01:29.357 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.357 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.357 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.357 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.198426+0000", 2024-09-11T09:01:29.357 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0028582500000000001 2024-09-11T09:01:29.357 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.357 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.357 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.357 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.199195+0000", 2024-09-11T09:01:29.358 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00076847999999999999 2024-09-11T09:01:29.358 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.358 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.358 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.358 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.226002+0000", 2024-09-11T09:01:29.358 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.026807220999999999 2024-09-11T09:01:29.358 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.358 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.358 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.358 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.226030+0000", 2024-09-11T09:01:29.358 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.8277e-05 2024-09-11T09:01:29.359 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.359 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.359 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.359 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.226055+0000", 2024-09-11T09:01:29.359 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4513999999999999e-05 2024-09-11T09:01:29.359 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.359 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.359 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.359 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.359 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.359 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:234831 11.7 11:e8e9b125:::benchmark_data_smithi092_205252_object113510:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T09:01:29.359 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:28.364696+0000", 2024-09-11T09:01:29.360 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 60.869832465999998, 2024-09-11T09:01:29.360 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.028405709000000001, 2024-09-11T09:01:29.360 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.360 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.360 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.360 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:29.360 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:29.360 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 234831 2024-09-11T09:01:29.360 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.361 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.361 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.361 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.361 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.364696+0000", 2024-09-11T09:01:29.361 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.361 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.361 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.361 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.361 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.364696+0000", 2024-09-11T09:01:29.361 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.361 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.361 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.362 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.362 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.364695+0000", 2024-09-11T09:01:29.362 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:29.362 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.362 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.362 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.362 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.364698+0000", 2024-09-11T09:01:29.362 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.2649999999999999e-06 2024-09-11T09:01:29.362 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.362 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.362 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.362 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.364704+0000", 2024-09-11T09:01:29.362 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4189999999999999e-06 2024-09-11T09:01:29.363 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.363 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.363 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.363 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.364708+0000", 2024-09-11T09:01:29.363 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4850000000000001e-06 2024-09-11T09:01:29.363 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.363 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.363 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.363 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.369690+0000", 2024-09-11T09:01:29.363 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0049819390000000003 2024-09-11T09:01:29.363 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.363 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.364 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.364 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.369802+0000", 2024-09-11T09:01:29.364 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0001125 2024-09-11T09:01:29.364 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.364 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.364 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,4", 2024-09-11T09:01:29.364 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.369918+0000", 2024-09-11T09:01:29.364 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000116004 2024-09-11T09:01:29.364 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.364 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.364 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.365 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.372241+0000", 2024-09-11T09:01:29.365 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.002323341 2024-09-11T09:01:29.365 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.365 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.365 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.365 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.372260+0000", 2024-09-11T09:01:29.365 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.8201000000000001e-05 2024-09-11T09:01:29.365 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.366 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.366 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.366 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.393016+0000", 2024-09-11T09:01:29.366 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.020756204 2024-09-11T09:01:29.366 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.366 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.366 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.366 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.393062+0000", 2024-09-11T09:01:29.366 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6063000000000003e-05 2024-09-11T09:01:29.366 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.366 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.366 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.366 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.393102+0000", 2024-09-11T09:01:29.367 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9632000000000003e-05 2024-09-11T09:01:29.367 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.367 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.367 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.367 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.367 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.367 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:234847 11.7 11:e4cd34a8:::benchmark_data_smithi092_205252_object113526:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T09:01:29.367 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:28.380117+0000", 2024-09-11T09:01:29.368 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 60.854411425000002, 2024-09-11T09:01:29.368 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.032912891, 2024-09-11T09:01:29.368 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.368 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.368 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.368 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:29.368 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:29.368 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 234847 2024-09-11T09:01:29.368 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.368 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.369 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.369 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.369 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.380117+0000", 2024-09-11T09:01:29.369 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.369 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.369 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.369 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.369 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.380117+0000", 2024-09-11T09:01:29.369 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.369 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.370 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.370 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.370 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.380116+0000", 2024-09-11T09:01:29.370 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:29.370 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.370 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.370 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.370 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.380124+0000", 2024-09-11T09:01:29.370 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.6600000000000001e-06 2024-09-11T09:01:29.371 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.371 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.371 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.371 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.380136+0000", 2024-09-11T09:01:29.371 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1347e-05 2024-09-11T09:01:29.371 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.371 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.371 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.371 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.380141+0000", 2024-09-11T09:01:29.371 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:01:29.371 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.371 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.371 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.372 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.380184+0000", 2024-09-11T09:01:29.372 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3170999999999997e-05 2024-09-11T09:01:29.372 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.372 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.372 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.372 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.380262+0000", 2024-09-11T09:01:29.372 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.8328e-05 2024-09-11T09:01:29.372 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.372 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.372 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,4", 2024-09-11T09:01:29.372 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.380353+0000", 2024-09-11T09:01:29.373 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.0340000000000006e-05 2024-09-11T09:01:29.373 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.373 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.373 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.373 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.382844+0000", 2024-09-11T09:01:29.373 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0024911209999999998 2024-09-11T09:01:29.373 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.373 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.373 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.373 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.382861+0000", 2024-09-11T09:01:29.373 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7581e-05 2024-09-11T09:01:29.373 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.374 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.374 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.374 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.412933+0000", 2024-09-11T09:01:29.374 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.030071482 2024-09-11T09:01:29.374 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.374 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.374 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.374 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.412989+0000", 2024-09-11T09:01:29.374 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6681000000000002e-05 2024-09-11T09:01:29.374 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.374 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.374 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.374 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.413030+0000", 2024-09-11T09:01:29.375 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0391e-05 2024-09-11T09:01:29.375 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.375 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.375 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.375 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.375 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.375 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:235934 11.7 11:e9f9b730:::benchmark_data_smithi092_205252_object114613:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T09:01:29.375 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:29.201510+0000", 2024-09-11T09:01:29.375 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 60.033018456999997, 2024-09-11T09:01:29.375 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.034009686999999997, 2024-09-11T09:01:29.375 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.375 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.376 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.376 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:29.376 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:29.376 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 235934 2024-09-11T09:01:29.376 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.376 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.376 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.376 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.376 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.201510+0000", 2024-09-11T09:01:29.376 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.376 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.376 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.377 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.377 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.201510+0000", 2024-09-11T09:01:29.377 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.377 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.377 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.377 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.377 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.201509+0000", 2024-09-11T09:01:29.377 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:29.377 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.377 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.377 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.377 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.201515+0000", 2024-09-11T09:01:29.377 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.6619999999999999e-06 2024-09-11T09:01:29.377 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.378 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.378 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.378 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.201531+0000", 2024-09-11T09:01:29.378 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5526e-05 2024-09-11T09:01:29.378 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.378 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.378 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.378 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.201537+0000", 2024-09-11T09:01:29.378 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7450000000000001e-06 2024-09-11T09:01:29.378 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.378 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.378 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.379 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.202332+0000", 2024-09-11T09:01:29.379 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00079535499999999998 2024-09-11T09:01:29.379 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.379 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.379 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.379 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.202438+0000", 2024-09-11T09:01:29.379 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00010652799999999999 2024-09-11T09:01:29.379 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.379 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.379 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,4", 2024-09-11T09:01:29.379 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.202566+0000", 2024-09-11T09:01:29.380 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012761900000000001 2024-09-11T09:01:29.380 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.380 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.380 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.380 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.205030+0000", 2024-09-11T09:01:29.380 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0024639810000000001 2024-09-11T09:01:29.380 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.380 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.380 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.380 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.205895+0000", 2024-09-11T09:01:29.380 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00086550199999999998 2024-09-11T09:01:29.380 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.380 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.380 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.381 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.235448+0000", 2024-09-11T09:01:29.381 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.029552163999999999 2024-09-11T09:01:29.381 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.381 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.381 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.381 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.235496+0000", 2024-09-11T09:01:29.381 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8223999999999998e-05 2024-09-11T09:01:29.381 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.381 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.381 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.381 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.235520+0000", 2024-09-11T09:01:29.381 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3649999999999999e-05 2024-09-11T09:01:29.381 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.382 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.382 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.382 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.382 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.382 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:236390 11.7 11:ea13cb95:::benchmark_data_smithi092_205252_object115069:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T09:01:29.382 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:29.583557+0000", 2024-09-11T09:01:29.382 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 59.650971579, 2024-09-11T09:01:29.382 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.032548303000000001, 2024-09-11T09:01:29.382 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.382 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.382 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.382 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:29.382 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:29.382 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 236390 2024-09-11T09:01:29.383 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.383 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.383 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.383 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.383 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.583557+0000", 2024-09-11T09:01:29.383 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.383 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.383 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.383 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.383 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.583557+0000", 2024-09-11T09:01:29.383 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.383 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.383 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.383 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.384 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.583556+0000", 2024-09-11T09:01:29.384 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:29.384 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.384 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.384 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.384 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.583561+0000", 2024-09-11T09:01:29.384 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.916e-06 2024-09-11T09:01:29.384 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.384 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.384 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.384 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.583571+0000", 2024-09-11T09:01:29.384 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0162e-05 2024-09-11T09:01:29.384 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.385 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.385 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.385 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.583577+0000", 2024-09-11T09:01:29.385 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7509999999999997e-06 2024-09-11T09:01:29.385 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.385 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.385 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.385 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.583614+0000", 2024-09-11T09:01:29.385 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7271000000000003e-05 2024-09-11T09:01:29.385 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.385 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.386 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.386 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.583691+0000", 2024-09-11T09:01:29.386 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.6846000000000003e-05 2024-09-11T09:01:29.386 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.386 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.386 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,4", 2024-09-11T09:01:29.386 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.583760+0000", 2024-09-11T09:01:29.386 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9189000000000005e-05 2024-09-11T09:01:29.386 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.386 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.386 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.386 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.586377+0000", 2024-09-11T09:01:29.386 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0026175880000000001 2024-09-11T09:01:29.387 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.387 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.387 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.387 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.586727+0000", 2024-09-11T09:01:29.387 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000349577 2024-09-11T09:01:29.387 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.387 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.387 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.387 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.616008+0000", 2024-09-11T09:01:29.387 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.029280595 2024-09-11T09:01:29.387 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.387 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.388 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.388 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.616068+0000", 2024-09-11T09:01:29.388 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0729000000000001e-05 2024-09-11T09:01:29.388 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.388 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.388 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.388 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:29.616105+0000", 2024-09-11T09:01:29.388 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6643999999999997e-05 2024-09-11T09:01:29.388 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.388 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.388 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.388 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.388 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.388 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:239566 11.7 11:eb0025bc:::benchmark_data_smithi092_205252_object118245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:29.389 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:32.075547+0000", 2024-09-11T09:01:29.389 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 57.158981775999997, 2024-09-11T09:01:29.389 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.033902733999999997, 2024-09-11T09:01:29.389 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.389 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.389 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.389 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:29.389 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:29.389 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 239566 2024-09-11T09:01:29.389 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.390 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.390 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.390 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.390 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.075547+0000", 2024-09-11T09:01:29.390 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.390 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.390 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.390 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.390 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.075547+0000", 2024-09-11T09:01:29.390 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.390 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.390 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.390 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.390 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.075545+0000", 2024-09-11T09:01:29.391 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:29.391 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.391 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.391 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.391 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.075552+0000", 2024-09-11T09:01:29.391 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1509999999999998e-06 2024-09-11T09:01:29.391 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.391 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.391 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.391 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.075564+0000", 2024-09-11T09:01:29.391 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1917e-05 2024-09-11T09:01:29.391 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.391 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.391 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.392 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.075570+0000", 2024-09-11T09:01:29.392 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2580000000000001e-06 2024-09-11T09:01:29.392 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.392 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.392 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.392 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.075618+0000", 2024-09-11T09:01:29.392 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7663000000000001e-05 2024-09-11T09:01:29.392 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.392 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.392 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.392 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.075746+0000", 2024-09-11T09:01:29.392 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000127537 2024-09-11T09:01:29.393 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.393 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.393 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 1,4", 2024-09-11T09:01:29.393 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.075875+0000", 2024-09-11T09:01:29.393 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012956900000000001 2024-09-11T09:01:29.393 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.393 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.393 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.393 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.079132+0000", 2024-09-11T09:01:29.393 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003256714 2024-09-11T09:01:29.394 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.394 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.394 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.394 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.079190+0000", 2024-09-11T09:01:29.394 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8004999999999998e-05 2024-09-11T09:01:29.394 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.394 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.394 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.394 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.109374+0000", 2024-09-11T09:01:29.395 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.030184632999999999 2024-09-11T09:01:29.395 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.395 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.395 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.395 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.109409+0000", 2024-09-11T09:01:29.395 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4402e-05 2024-09-11T09:01:29.395 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.395 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.395 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.395 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.109449+0000", 2024-09-11T09:01:29.395 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0410000000000001e-05 2024-09-11T09:01:29.395 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.395 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.396 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.396 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.396 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.396 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:239889 11.5 11:acf52f3d:::benchmark_data_smithi092_205252_object118568:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:29.396 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:32.351417+0000", 2024-09-11T09:01:29.396 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 56.883111407999998, 2024-09-11T09:01:29.396 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.029672714999999999, 2024-09-11T09:01:29.396 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.396 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.396 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.396 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:29.396 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:29.396 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 239889 2024-09-11T09:01:29.397 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.397 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.397 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.397 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.397 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.351417+0000", 2024-09-11T09:01:29.397 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.397 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.397 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.397 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.398 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.351417+0000", 2024-09-11T09:01:29.398 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.398 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.398 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.398 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.398 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.351417+0000", 2024-09-11T09:01:29.398 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:29.398 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.398 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.398 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.398 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.351419+0000", 2024-09-11T09:01:29.399 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3819999999999998e-06 2024-09-11T09:01:29.399 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.399 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.399 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.399 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.351424+0000", 2024-09-11T09:01:29.399 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:01:29.399 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.399 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.399 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.399 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.351427+0000", 2024-09-11T09:01:29.399 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1499999999999999e-06 2024-09-11T09:01:29.400 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.400 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.400 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.400 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.357619+0000", 2024-09-11T09:01:29.400 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0061913899999999997 2024-09-11T09:01:29.400 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.400 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.400 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.400 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.357738+0000", 2024-09-11T09:01:29.400 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000119555 2024-09-11T09:01:29.400 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.400 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.400 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 6,7", 2024-09-11T09:01:29.400 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.357840+0000", 2024-09-11T09:01:29.401 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000102062 2024-09-11T09:01:29.401 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.401 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.401 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.401 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.363464+0000", 2024-09-11T09:01:29.401 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0056231349999999996 2024-09-11T09:01:29.401 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.401 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.401 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.401 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.377051+0000", 2024-09-11T09:01:29.401 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013587309000000001 2024-09-11T09:01:29.401 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.401 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.402 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.402 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.380945+0000", 2024-09-11T09:01:29.402 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003893798 2024-09-11T09:01:29.402 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.402 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.402 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.402 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.381049+0000", 2024-09-11T09:01:29.402 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000104416 2024-09-11T09:01:29.402 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.402 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.402 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.402 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.381090+0000", 2024-09-11T09:01:29.403 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0499e-05 2024-09-11T09:01:29.403 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.403 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.403 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.403 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.403 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.403 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:239891 11.5 11:a1abaf5f:::benchmark_data_smithi092_205252_object118570:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:29.403 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:32.352930+0000", 2024-09-11T09:01:29.403 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 56.881598216, 2024-09-11T09:01:29.403 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.028353753999999998, 2024-09-11T09:01:29.404 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:29.404 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:29.404 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:29.404 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:29.404 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:29.404 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 239891 2024-09-11T09:01:29.404 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.404 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:29.404 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.404 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:29.404 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.352930+0000", 2024-09-11T09:01:29.404 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.404 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.404 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.405 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:29.405 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.352930+0000", 2024-09-11T09:01:29.405 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:29.405 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.405 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.405 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:29.405 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.352929+0000", 2024-09-11T09:01:29.406 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:29.406 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.406 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.406 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:29.406 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.352934+0000", 2024-09-11T09:01:29.406 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:01:29.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.407 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.407 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:29.407 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.352949+0000", 2024-09-11T09:01:29.407 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5585000000000001e-05 2024-09-11T09:01:29.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.407 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.407 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:29.407 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.352955+0000", 2024-09-11T09:01:29.407 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7950000000000004e-06 2024-09-11T09:01:29.407 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:29.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.363506+0000", 2024-09-11T09:01:29.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.010551404 2024-09-11T09:01:29.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:29.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.363620+0000", 2024-09-11T09:01:29.408 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000113397 2024-09-11T09:01:29.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.408 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.408 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 6,7", 2024-09-11T09:01:29.408 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.363736+0000", 2024-09-11T09:01:29.409 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011644199999999999 2024-09-11T09:01:29.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.409 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.377017+0000", 2024-09-11T09:01:29.409 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.013281139000000001 2024-09-11T09:01:29.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.409 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.409 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:29.409 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.377076+0000", 2024-09-11T09:01:29.409 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8319999999999997e-05 2024-09-11T09:01:29.410 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.410 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.410 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:29.410 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.381160+0000", 2024-09-11T09:01:29.410 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0040842689999999997 2024-09-11T09:01:29.410 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.410 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.410 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:29.410 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.381228+0000", 2024-09-11T09:01:29.410 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.8330999999999993e-05 2024-09-11T09:01:29.410 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:29.410 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:29.410 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:29.410 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.381284+0000", 2024-09-11T09:01:29.411 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5475e-05 2024-09-11T09:01:29.411 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.411 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.411 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.411 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:29.411 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:29.411 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T09:01:29.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:29 smithi145 ceph-mon[33384]: osdmap e442: 8 total, 8 up, 8 in 2024-09-11T09:01:29.713 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:29 smithi092 ceph-mon[29647]: osdmap e442: 8 total, 8 up, 8 in 2024-09-11T09:01:29.714 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:29 smithi092 ceph-mon[41114]: osdmap e442: 8 total, 8 up, 8 in 2024-09-11T09:01:30.161 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-11T09:01:30.409 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T09:01:30.508 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-11T09:01:30.519 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:30 smithi092 ceph-mon[29647]: 5.7 deep-scrub starts 2024-09-11T09:01:30.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:30 smithi092 ceph-mon[29647]: 5.7 deep-scrub ok 2024-09-11T09:01:30.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:30 smithi092 ceph-mon[29647]: pgmap v1043: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 5.5 KiB/s rd, 43 KiB/s wr, 5 op/s; 26 KiB/s, 0 objects/s recovering 2024-09-11T09:01:30.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:01:30.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:30 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/209219669' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T09:01:30.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:30 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T09:01:30.520 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:30 smithi092 ceph-mon[41114]: 5.7 deep-scrub starts 2024-09-11T09:01:30.520 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:30 smithi092 ceph-mon[41114]: 5.7 deep-scrub ok 2024-09-11T09:01:30.520 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:30 smithi092 ceph-mon[41114]: pgmap v1043: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 5.5 KiB/s rd, 43 KiB/s wr, 5 op/s; 26 KiB/s, 0 objects/s recovering 2024-09-11T09:01:30.520 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:01:30.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:30 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/209219669' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T09:01:30.521 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:30 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T09:01:30.853 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:30 smithi145 ceph-mon[33384]: 5.7 deep-scrub starts 2024-09-11T09:01:30.853 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:30 smithi145 ceph-mon[33384]: 5.7 deep-scrub ok 2024-09-11T09:01:30.853 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:30 smithi145 ceph-mon[33384]: pgmap v1043: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 5.5 KiB/s rd, 43 KiB/s wr, 5 op/s; 26 KiB/s, 0 objects/s recovering 2024-09-11T09:01:30.853 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:01:30.853 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:30 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/209219669' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T09:01:30.854 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:30 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T09:01:31.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 2 1 2024-09-11T09:01:31.591 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:31.599 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:31 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-11T09:01:31.600 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:31 smithi145 ceph-mon[33384]: osdmap e443: 8 total, 8 up, 8 in 2024-09-11T09:01:31.600 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:31 smithi145 ceph-mon[33384]: 4.7 scrub starts 2024-09-11T09:01:31.600 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:31 smithi145 ceph-mon[33384]: 4.7 scrub ok 2024-09-11T09:01:31.712 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:31 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-11T09:01:31.712 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:31 smithi092 ceph-mon[29647]: osdmap e443: 8 total, 8 up, 8 in 2024-09-11T09:01:31.712 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:31 smithi092 ceph-mon[29647]: 4.7 scrub starts 2024-09-11T09:01:31.712 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:31 smithi092 ceph-mon[29647]: 4.7 scrub ok 2024-09-11T09:01:31.712 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:01:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:01:31] "GET /metrics HTTP/1.1" 200 222520 "" "Prometheus/2.43.0" 2024-09-11T09:01:31.713 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:31 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-11T09:01:31.713 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:31 smithi092 ceph-mon[41114]: osdmap e443: 8 total, 8 up, 8 in 2024-09-11T09:01:31.713 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:31 smithi092 ceph-mon[41114]: 4.7 scrub starts 2024-09-11T09:01:31.713 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:31 smithi092 ceph-mon[41114]: 4.7 scrub ok 2024-09-11T09:01:32.178 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T09:01:32.178 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T09:01:32.178 INFO:teuthology.orchestra.run.smithi145.stdout: "num_ops": 0 2024-09-11T09:01:32.178 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T09:01:32.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:32 smithi092 ceph-mon[29647]: 4.15 deep-scrub starts 2024-09-11T09:01:32.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:32 smithi092 ceph-mon[29647]: 4.15 deep-scrub ok 2024-09-11T09:01:32.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:32 smithi092 ceph-mon[29647]: pgmap v1045: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 4.7 KiB/s rd, 37 KiB/s wr, 4 op/s; 22 KiB/s, 1 objects/s recovering 2024-09-11T09:01:32.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:32 smithi092 ceph-mon[41114]: 4.15 deep-scrub starts 2024-09-11T09:01:32.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:32 smithi092 ceph-mon[41114]: 4.15 deep-scrub ok 2024-09-11T09:01:32.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:32 smithi092 ceph-mon[41114]: pgmap v1045: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 4.7 KiB/s rd, 37 KiB/s wr, 4 op/s; 22 KiB/s, 1 objects/s recovering 2024-09-11T09:01:32.823 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:32 smithi145 ceph-mon[33384]: 4.15 deep-scrub starts 2024-09-11T09:01:32.823 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:32 smithi145 ceph-mon[33384]: 4.15 deep-scrub ok 2024-09-11T09:01:32.823 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:32 smithi145 ceph-mon[33384]: pgmap v1045: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 4.7 KiB/s rd, 37 KiB/s wr, 4 op/s; 22 KiB/s, 1 objects/s recovering 2024-09-11T09:01:32.823 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:01:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:01:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:01:32.849 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-11T09:01:33.108 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T09:01:34.523 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.2 to 1 (10000) 2024-09-11T09:01:34.538 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:34 smithi145 ceph-mon[33384]: 3.1d scrub starts 2024-09-11T09:01:34.538 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:34 smithi145 ceph-mon[33384]: 3.1d scrub ok 2024-09-11T09:01:34.538 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:34 smithi145 ceph-mon[33384]: pgmap v1046: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 37 KiB/s wr, 2 op/s; 22 KiB/s, 5 objects/s recovering 2024-09-11T09:01:34.538 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:34 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/971955760' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T09:01:34.538 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:34 smithi145 ceph-mon[33384]: 6.10 scrub starts 2024-09-11T09:01:34.538 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:34 smithi145 ceph-mon[33384]: 6.10 scrub ok 2024-09-11T09:01:34.539 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:34 smithi145 ceph-mon[33384]: 2.0 scrub starts 2024-09-11T09:01:34.539 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:34 smithi145 ceph-mon[33384]: 2.0 scrub ok 2024-09-11T09:01:34.776 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T09:01:34.776 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T09:01:34.776 INFO:teuthology.orchestra.run.smithi145.stdout: "complaint_time": 30, 2024-09-11T09:01:34.776 INFO:teuthology.orchestra.run.smithi145.stdout: "num_blocked_ops": 0 2024-09-11T09:01:34.776 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T09:01:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[29647]: 3.1d scrub starts 2024-09-11T09:01:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[29647]: 3.1d scrub ok 2024-09-11T09:01:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[29647]: pgmap v1046: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 37 KiB/s wr, 2 op/s; 22 KiB/s, 5 objects/s recovering 2024-09-11T09:01:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/971955760' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T09:01:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[29647]: 6.10 scrub starts 2024-09-11T09:01:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[29647]: 6.10 scrub ok 2024-09-11T09:01:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[29647]: 2.0 scrub starts 2024-09-11T09:01:34.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[29647]: 2.0 scrub ok 2024-09-11T09:01:34.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[41114]: 3.1d scrub starts 2024-09-11T09:01:34.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[41114]: 3.1d scrub ok 2024-09-11T09:01:34.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[41114]: pgmap v1046: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 37 KiB/s wr, 2 op/s; 22 KiB/s, 5 objects/s recovering 2024-09-11T09:01:34.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/971955760' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T09:01:34.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[41114]: 6.10 scrub starts 2024-09-11T09:01:34.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[41114]: 6.10 scrub ok 2024-09-11T09:01:34.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[41114]: 2.0 scrub starts 2024-09-11T09:01:34.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:34 smithi092 ceph-mon[41114]: 2.0 scrub ok 2024-09-11T09:01:35.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 2 1 2024-09-11T09:01:35.439 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-11T09:01:35.520 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:35.688 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T09:01:35.780 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:35 smithi092 ceph-mon[29647]: 1.8 deep-scrub starts 2024-09-11T09:01:35.780 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:35 smithi092 ceph-mon[29647]: 1.8 deep-scrub ok 2024-09-11T09:01:35.780 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:35 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/971955760' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T09:01:35.781 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:35 smithi092 ceph-mon[29647]: osdmap e444: 8 total, 8 up, 8 in 2024-09-11T09:01:35.781 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:35 smithi092 ceph-mon[41114]: 1.8 deep-scrub starts 2024-09-11T09:01:35.781 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:35 smithi092 ceph-mon[41114]: 1.8 deep-scrub ok 2024-09-11T09:01:35.781 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:35 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/971955760' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T09:01:35.781 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:35 smithi092 ceph-mon[41114]: osdmap e444: 8 total, 8 up, 8 in 2024-09-11T09:01:35.809 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:35 smithi145 ceph-mon[33384]: 1.8 deep-scrub starts 2024-09-11T09:01:35.810 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:35 smithi145 ceph-mon[33384]: 1.8 deep-scrub ok 2024-09-11T09:01:35.810 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:35 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/971955760' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T09:01:35.810 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:35 smithi145 ceph-mon[33384]: osdmap e444: 8 total, 8 up, 8 in 2024-09-11T09:01:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:36 smithi092 ceph-mon[29647]: 3.16 scrub starts 2024-09-11T09:01:36.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:36 smithi092 ceph-mon[29647]: 3.16 scrub ok 2024-09-11T09:01:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:36 smithi092 ceph-mon[29647]: pgmap v1048: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 144 B/s, 5 objects/s recovering 2024-09-11T09:01:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:36 smithi092 ceph-mon[29647]: 3.1b scrub starts 2024-09-11T09:01:36.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:36 smithi092 ceph-mon[29647]: 3.1b scrub ok 2024-09-11T09:01:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:36 smithi092 ceph-mon[41114]: 3.16 scrub starts 2024-09-11T09:01:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:36 smithi092 ceph-mon[41114]: 3.16 scrub ok 2024-09-11T09:01:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:36 smithi092 ceph-mon[41114]: pgmap v1048: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 144 B/s, 5 objects/s recovering 2024-09-11T09:01:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:36 smithi092 ceph-mon[41114]: 3.1b scrub starts 2024-09-11T09:01:36.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:36 smithi092 ceph-mon[41114]: 3.1b scrub ok 2024-09-11T09:01:36.896 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:36 smithi145 ceph-mon[33384]: 3.16 scrub starts 2024-09-11T09:01:36.896 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:36 smithi145 ceph-mon[33384]: 3.16 scrub ok 2024-09-11T09:01:36.896 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:36 smithi145 ceph-mon[33384]: pgmap v1048: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 144 B/s, 5 objects/s recovering 2024-09-11T09:01:36.896 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:36 smithi145 ceph-mon[33384]: 3.1b scrub starts 2024-09-11T09:01:36.896 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:36 smithi145 ceph-mon[33384]: 3.1b scrub ok 2024-09-11T09:01:37.442 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T09:01:37.442 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T09:01:37.442 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T09:01:37.442 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T09:01:37.442 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.442 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:243930 10.d 10:b3a5164c:::benchmark_data_smithi092_173109_object106971:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e344)", 2024-09-11T09:01:37.442 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:06.588586+0000", 2024-09-11T09:01:37.443 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 330.84003970100002, 2024-09-11T09:01:37.443 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.039286862999999998, 2024-09-11T09:01:37.443 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.443 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.443 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:37.443 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:37.443 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:37.443 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 243930 2024-09-11T09:01:37.443 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.443 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.443 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.443 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.443 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.588586+0000", 2024-09-11T09:01:37.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.444 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.588586+0000", 2024-09-11T09:01:37.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.444 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.444 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.588584+0000", 2024-09-11T09:01:37.444 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:37.444 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.444 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.445 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.588591+0000", 2024-09-11T09:01:37.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2539999999999998e-06 2024-09-11T09:01:37.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.445 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.445 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.588602+0000", 2024-09-11T09:01:37.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1532000000000001e-05 2024-09-11T09:01:37.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.445 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.445 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.445 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.588608+0000", 2024-09-11T09:01:37.445 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6690000000000003e-06 2024-09-11T09:01:37.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.589852+0000", 2024-09-11T09:01:37.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001243819 2024-09-11T09:01:37.446 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.589949+0000", 2024-09-11T09:01:37.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.6848999999999994e-05 2024-09-11T09:01:37.446 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.446 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,4", 2024-09-11T09:01:37.446 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.590053+0000", 2024-09-11T09:01:37.446 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000104777 2024-09-11T09:01:37.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:37.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.599481+0000", 2024-09-11T09:01:37.447 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0094272350000000008 2024-09-11T09:01:37.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.447 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.603292+0000", 2024-09-11T09:01:37.447 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0038110010000000001 2024-09-11T09:01:37.447 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.447 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.447 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.448 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.627773+0000", 2024-09-11T09:01:37.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.024481105999999999 2024-09-11T09:01:37.448 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.448 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.448 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.448 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.627820+0000", 2024-09-11T09:01:37.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7855999999999997e-05 2024-09-11T09:01:37.448 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.448 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.448 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.448 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.627873+0000", 2024-09-11T09:01:37.448 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.2012000000000003e-05 2024-09-11T09:01:37.448 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.449 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.449 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.449 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.449 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.449 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244355 10.b 10:dbeeb021:::benchmark_data_smithi092_173109_object107396:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e344)", 2024-09-11T09:01:37.449 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:06.912463+0000", 2024-09-11T09:01:37.449 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 330.51616254599998, 2024-09-11T09:01:37.449 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.038872979000000002, 2024-09-11T09:01:37.449 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.449 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.449 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:37.449 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:37.449 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:37.450 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244355 2024-09-11T09:01:37.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.450 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.912463+0000", 2024-09-11T09:01:37.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.450 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.912463+0000", 2024-09-11T09:01:37.450 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.450 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.450 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.912461+0000", 2024-09-11T09:01:37.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:37.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.912469+0000", 2024-09-11T09:01:37.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4610000000000002e-06 2024-09-11T09:01:37.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.451 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.451 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.912478+0000", 2024-09-11T09:01:37.451 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.2150000000000007e-06 2024-09-11T09:01:37.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.912483+0000", 2024-09-11T09:01:37.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.2460000000000003e-06 2024-09-11T09:01:37.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.912569+0000", 2024-09-11T09:01:37.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.5896999999999994e-05 2024-09-11T09:01:37.452 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.452 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.452 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.452 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.912683+0000", 2024-09-11T09:01:37.452 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000113681 2024-09-11T09:01:37.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,1", 2024-09-11T09:01:37.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.912825+0000", 2024-09-11T09:01:37.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00014156600000000001 2024-09-11T09:01:37.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.453 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.916016+0000", 2024-09-11T09:01:37.453 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0031914420000000001 2024-09-11T09:01:37.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.453 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.453 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:37.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.916398+0000", 2024-09-11T09:01:37.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000381624 2024-09-11T09:01:37.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.951223+0000", 2024-09-11T09:01:37.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.034825229999999999 2024-09-11T09:01:37.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.454 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.454 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.951297+0000", 2024-09-11T09:01:37.454 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.3702999999999996e-05 2024-09-11T09:01:37.454 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.454 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.455 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.455 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:06.951336+0000", 2024-09-11T09:01:37.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9251000000000001e-05 2024-09-11T09:01:37.455 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.455 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.455 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.455 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.455 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.455 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244556 10.7 10:e73822a7:::benchmark_data_smithi092_173109_object107597:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e344)", 2024-09-11T09:01:37.455 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.077393+0000", 2024-09-11T09:01:37.455 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 330.35123245800003, 2024-09-11T09:01:37.455 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.149305357, 2024-09-11T09:01:37.455 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.455 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.455 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:37.456 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:37.456 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:37.456 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244556 2024-09-11T09:01:37.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.456 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.456 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.456 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.456 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.077393+0000", 2024-09-11T09:01:37.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.456 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.456 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.456 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.077393+0000", 2024-09-11T09:01:37.456 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.077392+0000", 2024-09-11T09:01:37.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:37.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.077402+0000", 2024-09-11T09:01:37.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.025e-05 2024-09-11T09:01:37.457 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.457 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.457 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.457 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.077411+0000", 2024-09-11T09:01:37.457 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.7229999999999993e-06 2024-09-11T09:01:37.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.077416+0000", 2024-09-11T09:01:37.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5800000000000002e-06 2024-09-11T09:01:37.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.077449+0000", 2024-09-11T09:01:37.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3574000000000001e-05 2024-09-11T09:01:37.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.458 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.458 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.458 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.077614+0000", 2024-09-11T09:01:37.458 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000164239 2024-09-11T09:01:37.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,5", 2024-09-11T09:01:37.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.077768+0000", 2024-09-11T09:01:37.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00015399899999999999 2024-09-11T09:01:37.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:37.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.081331+0000", 2024-09-11T09:01:37.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00356376 2024-09-11T09:01:37.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.459 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.459 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.459 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.082126+0000", 2024-09-11T09:01:37.459 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00079426200000000001 2024-09-11T09:01:37.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.226614+0000", 2024-09-11T09:01:37.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.14448802999999999 2024-09-11T09:01:37.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.226660+0000", 2024-09-11T09:01:37.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6767000000000001e-05 2024-09-11T09:01:37.460 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.460 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.460 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.460 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.226698+0000", 2024-09-11T09:01:37.460 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7870000000000002e-05 2024-09-11T09:01:37.461 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.461 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.461 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.461 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.461 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.461 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244566 10.7 10:e5876987:::benchmark_data_smithi092_173109_object107607:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e344)", 2024-09-11T09:01:37.461 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.085451+0000", 2024-09-11T09:01:37.461 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 330.34317448500002, 2024-09-11T09:01:37.461 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.14435182699999999, 2024-09-11T09:01:37.461 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.461 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.461 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:37.461 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:37.461 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:37.462 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244566 2024-09-11T09:01:37.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.462 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085451+0000", 2024-09-11T09:01:37.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.462 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.462 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085451+0000", 2024-09-11T09:01:37.462 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085450+0000", 2024-09-11T09:01:37.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:37.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085456+0000", 2024-09-11T09:01:37.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8950000000000003e-06 2024-09-11T09:01:37.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.463 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.463 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085467+0000", 2024-09-11T09:01:37.463 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1094000000000001e-05 2024-09-11T09:01:37.463 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.463 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085472+0000", 2024-09-11T09:01:37.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3689999999999998e-06 2024-09-11T09:01:37.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085543+0000", 2024-09-11T09:01:37.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.1200999999999998e-05 2024-09-11T09:01:37.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.464 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.464 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085679+0000", 2024-09-11T09:01:37.464 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013611 2024-09-11T09:01:37.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.464 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,5", 2024-09-11T09:01:37.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085791+0000", 2024-09-11T09:01:37.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011146600000000001 2024-09-11T09:01:37.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.089488+0000", 2024-09-11T09:01:37.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0036973589999999999 2024-09-11T09:01:37.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.465 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:37.465 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.089492+0000", 2024-09-11T09:01:37.465 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5939999999999998e-06 2024-09-11T09:01:37.465 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.465 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.466 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.229691+0000", 2024-09-11T09:01:37.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.14019895199999999 2024-09-11T09:01:37.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.466 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.229764+0000", 2024-09-11T09:01:37.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.3631e-05 2024-09-11T09:01:37.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.466 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.466 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.229803+0000", 2024-09-11T09:01:37.466 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8376e-05 2024-09-11T09:01:37.466 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.467 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.467 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.467 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.467 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244567 10.a 10:59740626:::benchmark_data_smithi092_173109_object107608:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e344)", 2024-09-11T09:01:37.467 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.085683+0000", 2024-09-11T09:01:37.467 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 330.342941956, 2024-09-11T09:01:37.467 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.16020936, 2024-09-11T09:01:37.467 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.467 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.467 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:37.467 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:37.467 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:37.467 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244567 2024-09-11T09:01:37.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.468 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085683+0000", 2024-09-11T09:01:37.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.468 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085683+0000", 2024-09-11T09:01:37.468 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.468 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.468 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.468 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.469 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085683+0000", 2024-09-11T09:01:37.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:37.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.469 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.469 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085687+0000", 2024-09-11T09:01:37.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2869999999999998e-06 2024-09-11T09:01:37.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.469 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.469 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.469 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085695+0000", 2024-09-11T09:01:37.469 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.028e-06 2024-09-11T09:01:37.469 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085700+0000", 2024-09-11T09:01:37.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6330000000000004e-06 2024-09-11T09:01:37.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085737+0000", 2024-09-11T09:01:37.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7098999999999998e-05 2024-09-11T09:01:37.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.470 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.470 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.470 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.085832+0000", 2024-09-11T09:01:37.470 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.5262999999999999e-05 2024-09-11T09:01:37.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,5", 2024-09-11T09:01:37.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.086285+0000", 2024-09-11T09:01:37.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00045268400000000002 2024-09-11T09:01:37.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:37.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.091194+0000", 2024-09-11T09:01:37.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0049094719999999998 2024-09-11T09:01:37.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.471 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.471 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.471 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.096638+0000", 2024-09-11T09:01:37.471 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0054436969999999999 2024-09-11T09:01:37.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.245819+0000", 2024-09-11T09:01:37.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.14918115900000001 2024-09-11T09:01:37.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.245860+0000", 2024-09-11T09:01:37.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.1248999999999998e-05 2024-09-11T09:01:37.472 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.472 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.472 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.472 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.245893+0000", 2024-09-11T09:01:37.472 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2456999999999998e-05 2024-09-11T09:01:37.473 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.473 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.473 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.473 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.473 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244570 10.a 10:512ba5fb:::benchmark_data_smithi092_173109_object107611:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e344)", 2024-09-11T09:01:37.473 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.086648+0000", 2024-09-11T09:01:37.473 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 330.34197685200002, 2024-09-11T09:01:37.473 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.16322981, 2024-09-11T09:01:37.473 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.473 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.473 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:37.473 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:37.473 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:37.474 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244570 2024-09-11T09:01:37.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.474 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.086648+0000", 2024-09-11T09:01:37.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.474 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.086648+0000", 2024-09-11T09:01:37.474 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.474 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.474 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.086647+0000", 2024-09-11T09:01:37.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:37.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.086652+0000", 2024-09-11T09:01:37.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3989999999999997e-06 2024-09-11T09:01:37.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.475 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.086661+0000", 2024-09-11T09:01:37.475 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.2890000000000003e-06 2024-09-11T09:01:37.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.475 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.475 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.086666+0000", 2024-09-11T09:01:37.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7600000000000002e-06 2024-09-11T09:01:37.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.089158+0000", 2024-09-11T09:01:37.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0024924610000000001 2024-09-11T09:01:37.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.476 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.089271+0000", 2024-09-11T09:01:37.476 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000112483 2024-09-11T09:01:37.476 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.476 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.476 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,5", 2024-09-11T09:01:37.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.089360+0000", 2024-09-11T09:01:37.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8875999999999994e-05 2024-09-11T09:01:37.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:37.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.093155+0000", 2024-09-11T09:01:37.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0037951080000000002 2024-09-11T09:01:37.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.477 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.096672+0000", 2024-09-11T09:01:37.477 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0035176080000000002 2024-09-11T09:01:37.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.477 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.249807+0000", 2024-09-11T09:01:37.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.15313496600000001 2024-09-11T09:01:37.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.249847+0000", 2024-09-11T09:01:37.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9774999999999998e-05 2024-09-11T09:01:37.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.478 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.478 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.478 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.249878+0000", 2024-09-11T09:01:37.478 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1288000000000003e-05 2024-09-11T09:01:37.478 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.478 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.478 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.479 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.479 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.479 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244575 10.a 10:56142644:::benchmark_data_smithi092_173109_object107616:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e344)", 2024-09-11T09:01:37.479 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.089320+0000", 2024-09-11T09:01:37.479 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 330.33930512199998, 2024-09-11T09:01:37.479 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.160681989, 2024-09-11T09:01:37.479 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.479 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.479 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:37.479 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:37.479 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:37.479 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244575 2024-09-11T09:01:37.479 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.479 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.480 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.480 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.480 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.089320+0000", 2024-09-11T09:01:37.480 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.480 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.480 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.480 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.089320+0000", 2024-09-11T09:01:37.480 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.480 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.480 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.480 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.089319+0000", 2024-09-11T09:01:37.480 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:37.480 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.481 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.089324+0000", 2024-09-11T09:01:37.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.3080000000000001e-06 2024-09-11T09:01:37.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.481 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.089332+0000", 2024-09-11T09:01:37.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.7409999999999998e-06 2024-09-11T09:01:37.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.481 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.481 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.089337+0000", 2024-09-11T09:01:37.481 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6750000000000001e-06 2024-09-11T09:01:37.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.091216+0000", 2024-09-11T09:01:37.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001878783 2024-09-11T09:01:37.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.091305+0000", 2024-09-11T09:01:37.482 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8709999999999996e-05 2024-09-11T09:01:37.482 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.482 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.482 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,5", 2024-09-11T09:01:37.482 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.091375+0000", 2024-09-11T09:01:37.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.0963000000000005e-05 2024-09-11T09:01:37.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:37.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.093353+0000", 2024-09-11T09:01:37.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0019772349999999999 2024-09-11T09:01:37.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.483 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.096705+0000", 2024-09-11T09:01:37.483 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0033524819999999999 2024-09-11T09:01:37.483 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.483 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.483 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.249910+0000", 2024-09-11T09:01:37.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.15320519599999999 2024-09-11T09:01:37.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.484 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.484 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.249970+0000", 2024-09-11T09:01:37.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.9435000000000003e-05 2024-09-11T09:01:37.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.484 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.484 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.484 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.250002+0000", 2024-09-11T09:01:37.484 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2364999999999999e-05 2024-09-11T09:01:37.484 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.485 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.485 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.485 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.485 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.485 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244578 10.7 10:e29ba64b:::benchmark_data_smithi092_173109_object107619:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e344)", 2024-09-11T09:01:37.485 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.093300+0000", 2024-09-11T09:01:37.485 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 330.33532519800002, 2024-09-11T09:01:37.485 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.13824492899999999, 2024-09-11T09:01:37.485 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.485 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.485 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:37.485 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:37.485 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:37.485 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244578 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.093300+0000", 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.093300+0000", 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.093300+0000", 2024-09-11T09:01:37.486 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:37.487 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.487 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.487 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.487 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.093302+0000", 2024-09-11T09:01:37.487 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3939999999999999e-06 2024-09-11T09:01:37.487 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.487 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.487 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.487 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.093307+0000", 2024-09-11T09:01:37.487 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.78e-06 2024-09-11T09:01:37.487 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.487 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.487 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.093310+0000", 2024-09-11T09:01:37.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.6089999999999999e-06 2024-09-11T09:01:37.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.093329+0000", 2024-09-11T09:01:37.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.9204000000000001e-05 2024-09-11T09:01:37.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.488 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.488 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.093417+0000", 2024-09-11T09:01:37.488 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.8540000000000003e-05 2024-09-11T09:01:37.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.488 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 3,5", 2024-09-11T09:01:37.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.093529+0000", 2024-09-11T09:01:37.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011202800000000001 2024-09-11T09:01:37.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:37.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.096367+0000", 2024-09-11T09:01:37.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0028377889999999998 2024-09-11T09:01:37.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.489 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.489 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.489 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.097251+0000", 2024-09-11T09:01:37.489 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00088340999999999995 2024-09-11T09:01:37.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.231434+0000", 2024-09-11T09:01:37.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.13418333099999999 2024-09-11T09:01:37.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.231508+0000", 2024-09-11T09:01:37.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.3675000000000002e-05 2024-09-11T09:01:37.490 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.490 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.490 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.490 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.231545+0000", 2024-09-11T09:01:37.490 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7537000000000003e-05 2024-09-11T09:01:37.490 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.491 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.491 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.491 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.491 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244587 10.a 10:584e3abb:::benchmark_data_smithi092_173109_object107628:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e344)", 2024-09-11T09:01:37.491 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.100120+0000", 2024-09-11T09:01:37.491 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 330.32850564699999, 2024-09-11T09:01:37.491 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.15000371300000001, 2024-09-11T09:01:37.491 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.491 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.491 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:37.491 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:37.491 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:37.491 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244587 2024-09-11T09:01:37.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.492 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.492 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.492 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.100120+0000", 2024-09-11T09:01:37.492 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.492 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.492 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.492 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.100120+0000", 2024-09-11T09:01:37.492 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.492 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.492 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.492 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.492 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.100119+0000", 2024-09-11T09:01:37.492 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:37.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.493 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.493 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.493 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.100123+0000", 2024-09-11T09:01:37.493 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.771e-06 2024-09-11T09:01:37.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.493 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.493 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.493 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.100133+0000", 2024-09-11T09:01:37.493 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0540000000000001e-05 2024-09-11T09:01:37.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.493 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.493 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.100137+0000", 2024-09-11T09:01:37.494 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8909999999999997e-06 2024-09-11T09:01:37.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.494 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.494 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.100165+0000", 2024-09-11T09:01:37.494 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7790999999999999e-05 2024-09-11T09:01:37.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.494 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.494 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.494 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.100237+0000", 2024-09-11T09:01:37.494 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2026999999999993e-05 2024-09-11T09:01:37.494 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.494 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,5", 2024-09-11T09:01:37.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.100300+0000", 2024-09-11T09:01:37.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2776999999999999e-05 2024-09-11T09:01:37.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:37.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.102899+0000", 2024-09-11T09:01:37.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.002598632 2024-09-11T09:01:37.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.495 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.495 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.495 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.103192+0000", 2024-09-11T09:01:37.495 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00029305200000000003 2024-09-11T09:01:37.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.250060+0000", 2024-09-11T09:01:37.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.14686840500000001 2024-09-11T09:01:37.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.496 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.250094+0000", 2024-09-11T09:01:37.496 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4190000000000003e-05 2024-09-11T09:01:37.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.496 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.497 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.250123+0000", 2024-09-11T09:01:37.497 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9255999999999999e-05 2024-09-11T09:01:37.497 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.497 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.497 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.497 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.497 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15621.0:244597 10.a 10:57ae75f6:::benchmark_data_smithi092_173109_object107638:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e344)", 2024-09-11T09:01:37.497 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:56:07.115243+0000", 2024-09-11T09:01:37.497 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 330.31338258400001, 2024-09-11T09:01:37.497 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.134966536, 2024-09-11T09:01:37.497 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.497 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.498 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:37.498 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15621", 2024-09-11T09:01:37.498 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/4033048146", 2024-09-11T09:01:37.498 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 244597 2024-09-11T09:01:37.498 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.498 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.498 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.498 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.498 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.115243+0000", 2024-09-11T09:01:37.498 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.498 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.498 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.498 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.498 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.115243+0000", 2024-09-11T09:01:37.499 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.499 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.499 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.499 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.499 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.115242+0000", 2024-09-11T09:01:37.499 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:37.499 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.499 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.499 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.499 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.115246+0000", 2024-09-11T09:01:37.499 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.777e-06 2024-09-11T09:01:37.499 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.499 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.499 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.500 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.115253+0000", 2024-09-11T09:01:37.500 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2060000000000001e-06 2024-09-11T09:01:37.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.500 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.500 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.500 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.115256+0000", 2024-09-11T09:01:37.500 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.287e-06 2024-09-11T09:01:37.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.500 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.500 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.500 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.115296+0000", 2024-09-11T09:01:37.500 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9894000000000001e-05 2024-09-11T09:01:37.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.501 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.501 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.501 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.115412+0000", 2024-09-11T09:01:37.501 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000116397 2024-09-11T09:01:37.501 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.501 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.501 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 0,5", 2024-09-11T09:01:37.501 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.115532+0000", 2024-09-11T09:01:37.501 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000119837 2024-09-11T09:01:37.501 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.501 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.501 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:37.501 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.117739+0000", 2024-09-11T09:01:37.501 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0022066220000000001 2024-09-11T09:01:37.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.502 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.502 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.502 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.117905+0000", 2024-09-11T09:01:37.502 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000165718 2024-09-11T09:01:37.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.502 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.502 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:37.502 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.250151+0000", 2024-09-11T09:01:37.502 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.132246421 2024-09-11T09:01:37.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.502 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.502 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.502 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.250183+0000", 2024-09-11T09:01:37.503 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1569999999999998e-05 2024-09-11T09:01:37.503 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.503 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.503 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.503 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:56:07.250209+0000", 2024-09-11T09:01:37.503 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.6684999999999999e-05 2024-09-11T09:01:37.503 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.503 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.503 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.503 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.503 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.503 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15918.0:66340 11.d e385/366 11:b9ea60ee:::benchmark_data_smithi092_205252_object66339:head v 385'4204, mlcod=385'4202)", 2024-09-11T09:01:37.503 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:58:02.784207+0000", 2024-09-11T09:01:37.503 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 214.64441784300001, 2024-09-11T09:01:37.504 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.027075189999999999, 2024-09-11T09:01:37.504 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.504 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.504 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.504 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.504 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.504 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:02.784207+0000", 2024-09-11T09:01:37.504 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.504 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.504 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.504 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.504 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:02.784207+0000", 2024-09-11T09:01:37.504 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.504 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.505 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.505 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.505 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:02.784207+0000", 2024-09-11T09:01:37.505 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:37.505 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.505 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.505 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.505 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:02.784302+0000", 2024-09-11T09:01:37.505 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.4734999999999993e-05 2024-09-11T09:01:37.505 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.505 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.505 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.505 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:02.784311+0000", 2024-09-11T09:01:37.506 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.8989999999999996e-06 2024-09-11T09:01:37.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.506 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.506 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.506 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:02.784317+0000", 2024-09-11T09:01:37.506 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3600000000000004e-06 2024-09-11T09:01:37.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.506 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.506 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.506 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:02.786235+0000", 2024-09-11T09:01:37.506 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0019184849999999999 2024-09-11T09:01:37.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.506 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.506 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.507 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:02.786251+0000", 2024-09-11T09:01:37.507 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6178999999999999e-05 2024-09-11T09:01:37.507 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.507 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.507 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.507 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:02.811196+0000", 2024-09-11T09:01:37.507 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.024944397 2024-09-11T09:01:37.507 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.507 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.507 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.507 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:02.811283+0000", 2024-09-11T09:01:37.507 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.6793999999999996e-05 2024-09-11T09:01:37.507 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.507 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.507 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.508 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.508 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.508 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15918.0:89251 11.4 e397/390 11:27b27c35:::benchmark_data_smithi092_205252_object89250:head v 397'5600, mlcod=397'5598)", 2024-09-11T09:01:37.508 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:58:27.632091+0000", 2024-09-11T09:01:37.508 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 189.79653465300001, 2024-09-11T09:01:37.508 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.25449839200000002, 2024-09-11T09:01:37.508 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.508 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.508 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.508 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.508 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.508 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.632091+0000", 2024-09-11T09:01:37.508 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.508 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.509 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.509 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.509 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.632091+0000", 2024-09-11T09:01:37.509 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.509 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.509 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.509 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.509 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.632090+0000", 2024-09-11T09:01:37.509 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:37.509 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.509 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.509 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.509 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.632113+0000", 2024-09-11T09:01:37.510 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3387000000000002e-05 2024-09-11T09:01:37.510 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.510 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.510 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.510 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.632126+0000", 2024-09-11T09:01:37.511 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2741999999999999e-05 2024-09-11T09:01:37.511 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.511 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.511 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.511 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.632132+0000", 2024-09-11T09:01:37.511 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8730000000000004e-06 2024-09-11T09:01:37.511 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.511 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.511 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.511 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.632179+0000", 2024-09-11T09:01:37.511 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7355000000000003e-05 2024-09-11T09:01:37.511 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.511 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.511 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.512 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.632200+0000", 2024-09-11T09:01:37.512 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.141e-05 2024-09-11T09:01:37.512 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.512 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.512 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.512 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.886559+0000", 2024-09-11T09:01:37.512 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.254358362 2024-09-11T09:01:37.512 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.512 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.512 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.512 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.886589+0000", 2024-09-11T09:01:37.512 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0250999999999999e-05 2024-09-11T09:01:37.512 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.512 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.513 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.513 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.513 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15918.0:89255 11.9 e397/390 11:9af78481:::benchmark_data_smithi092_205252_object89254:head v 397'5489, mlcod=397'5487)", 2024-09-11T09:01:37.513 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:58:27.636280+0000", 2024-09-11T09:01:37.513 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 189.79234583600001, 2024-09-11T09:01:37.513 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.255467944, 2024-09-11T09:01:37.513 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.513 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.513 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.513 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.513 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.513 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.636280+0000", 2024-09-11T09:01:37.513 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.514 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.514 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.514 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.636280+0000", 2024-09-11T09:01:37.514 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.514 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.514 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.514 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.514 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.636278+0000", 2024-09-11T09:01:37.514 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:37.514 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.514 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.514 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.514 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.636349+0000", 2024-09-11T09:01:37.514 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.0937000000000002e-05 2024-09-11T09:01:37.515 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.515 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.515 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.515 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.882462+0000", 2024-09-11T09:01:37.515 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.246113097 2024-09-11T09:01:37.515 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.515 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.515 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.515 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.882473+0000", 2024-09-11T09:01:37.515 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1270999999999999e-05 2024-09-11T09:01:37.515 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.515 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.515 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.515 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.882546+0000", 2024-09-11T09:01:37.515 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.2951e-05 2024-09-11T09:01:37.516 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.516 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.516 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.516 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.882575+0000", 2024-09-11T09:01:37.516 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.8328999999999999e-05 2024-09-11T09:01:37.516 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.516 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.516 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.516 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.891716+0000", 2024-09-11T09:01:37.516 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0091417119999999998 2024-09-11T09:01:37.516 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.516 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.516 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.516 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:27.891747+0000", 2024-09-11T09:01:37.517 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1022000000000002e-05 2024-09-11T09:01:37.517 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.517 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.517 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.517 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.517 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.517 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15918.0:107971 11.5 e402/399 11:a77ebd7a:::benchmark_data_smithi092_205252_object107970:head v 402'6727, mlcod=402'6725)", 2024-09-11T09:01:37.517 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:58:47.371342+0000", 2024-09-11T09:01:37.517 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 170.05728376600001, 2024-09-11T09:01:37.517 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.028400983000000001, 2024-09-11T09:01:37.517 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.517 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.517 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.517 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.518 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.518 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.371342+0000", 2024-09-11T09:01:37.518 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.518 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.518 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.518 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.518 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.371342+0000", 2024-09-11T09:01:37.518 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.518 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.518 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.518 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.518 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.371341+0000", 2024-09-11T09:01:37.518 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:37.518 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.518 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.519 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.519 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.371361+0000", 2024-09-11T09:01:37.519 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.9468000000000001e-05 2024-09-11T09:01:37.519 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.519 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.519 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.519 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.371367+0000", 2024-09-11T09:01:37.519 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.0410000000000003e-06 2024-09-11T09:01:37.519 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.519 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.519 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.519 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.371370+0000", 2024-09-11T09:01:37.519 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.5860000000000001e-06 2024-09-11T09:01:37.519 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.520 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.520 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.520 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.374386+0000", 2024-09-11T09:01:37.520 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0030151510000000002 2024-09-11T09:01:37.520 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.520 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.520 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.520 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.374406+0000", 2024-09-11T09:01:37.520 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0058e-05 2024-09-11T09:01:37.520 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.520 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.520 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.520 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.399711+0000", 2024-09-11T09:01:37.520 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.025305195999999999 2024-09-11T09:01:37.521 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.521 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.521 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.521 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.399743+0000", 2024-09-11T09:01:37.521 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.18e-05 2024-09-11T09:01:37.521 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.521 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.521 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.521 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.521 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.521 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15918.0:107966 11.d e402/399 11:b4b843cd:::benchmark_data_smithi092_205252_object107965:head v 402'6807, mlcod=398'5835)", 2024-09-11T09:01:37.521 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:58:47.374153+0000", 2024-09-11T09:01:37.521 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 170.054472777, 2024-09-11T09:01:37.522 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.025614872, 2024-09-11T09:01:37.522 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.522 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.522 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.522 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.522 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.522 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.374153+0000", 2024-09-11T09:01:37.522 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.522 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.522 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.522 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.374153+0000", 2024-09-11T09:01:37.522 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.523 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.523 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.523 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.374152+0000", 2024-09-11T09:01:37.523 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:37.523 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.523 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.523 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.523 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.374189+0000", 2024-09-11T09:01:37.523 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7307e-05 2024-09-11T09:01:37.523 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.523 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.523 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.523 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.374201+0000", 2024-09-11T09:01:37.523 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1999e-05 2024-09-11T09:01:37.523 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.524 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.524 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.524 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.374206+0000", 2024-09-11T09:01:37.524 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4949999999999999e-06 2024-09-11T09:01:37.524 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.524 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.524 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.524 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.375547+0000", 2024-09-11T09:01:37.524 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001340993 2024-09-11T09:01:37.524 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.524 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.524 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.524 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.375570+0000", 2024-09-11T09:01:37.524 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.2768999999999998e-05 2024-09-11T09:01:37.525 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.525 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.525 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.525 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.399745+0000", 2024-09-11T09:01:37.525 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.024175248999999999 2024-09-11T09:01:37.525 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.525 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.525 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.525 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:47.399767+0000", 2024-09-11T09:01:37.525 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.2087999999999999e-05 2024-09-11T09:01:37.525 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.525 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.525 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.525 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.525 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.526 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15918.0:108822 11.5 e402/399 11:ada63594:::benchmark_data_smithi092_205252_object108821:head v 402'6767, mlcod=402'6765)", 2024-09-11T09:01:37.526 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:58:48.058817+0000", 2024-09-11T09:01:37.526 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 169.369808355, 2024-09-11T09:01:37.526 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.025756101999999999, 2024-09-11T09:01:37.526 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.526 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.526 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.526 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.526 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.526 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:48.058817+0000", 2024-09-11T09:01:37.526 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.526 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.526 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.527 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.527 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:48.058817+0000", 2024-09-11T09:01:37.527 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.527 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.527 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.527 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.527 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:48.058816+0000", 2024-09-11T09:01:37.527 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:37.527 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.527 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.527 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.527 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:48.058874+0000", 2024-09-11T09:01:37.527 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7821000000000001e-05 2024-09-11T09:01:37.527 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.528 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.528 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.528 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:48.058884+0000", 2024-09-11T09:01:37.528 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0244999999999999e-05 2024-09-11T09:01:37.528 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.528 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.528 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.528 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:48.058889+0000", 2024-09-11T09:01:37.528 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.853e-06 2024-09-11T09:01:37.528 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.528 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.528 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.528 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:48.061308+0000", 2024-09-11T09:01:37.528 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0024183249999999998 2024-09-11T09:01:37.529 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.529 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.529 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.529 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:48.061324+0000", 2024-09-11T09:01:37.529 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6458999999999999e-05 2024-09-11T09:01:37.529 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.529 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.529 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.529 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:48.084541+0000", 2024-09-11T09:01:37.529 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.023216391999999999 2024-09-11T09:01:37.529 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.529 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.529 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.529 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:48.084573+0000", 2024-09-11T09:01:37.530 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2580000000000003e-05 2024-09-11T09:01:37.530 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.530 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.530 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.530 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.530 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.530 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15918.0:110461 11.5 e404/399 11:a4c52f28:::benchmark_data_smithi092_205252_object110460:head v 404'6881, mlcod=404'6878)", 2024-09-11T09:01:37.530 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:58:50.656123+0000", 2024-09-11T09:01:37.530 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 166.77250186800001, 2024-09-11T09:01:37.530 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.023713788999999999, 2024-09-11T09:01:37.530 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.530 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.530 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.530 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.531 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.531 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:50.656123+0000", 2024-09-11T09:01:37.531 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.531 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.531 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.531 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.531 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:50.656123+0000", 2024-09-11T09:01:37.531 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.531 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.531 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.531 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.531 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:50.656123+0000", 2024-09-11T09:01:37.531 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:37.531 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.532 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.532 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.532 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:50.656147+0000", 2024-09-11T09:01:37.532 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.3875000000000001e-05 2024-09-11T09:01:37.532 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.532 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.532 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.532 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:50.656154+0000", 2024-09-11T09:01:37.532 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.0690000000000004e-06 2024-09-11T09:01:37.532 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.532 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.532 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.532 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:50.656158+0000", 2024-09-11T09:01:37.532 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.8089999999999999e-06 2024-09-11T09:01:37.533 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.533 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.533 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.533 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:50.656572+0000", 2024-09-11T09:01:37.533 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000413998 2024-09-11T09:01:37.533 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.533 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.533 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.533 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:50.656588+0000", 2024-09-11T09:01:37.533 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6392000000000002e-05 2024-09-11T09:01:37.533 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.533 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.533 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:50.679625+0000", 2024-09-11T09:01:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.023036976000000001 2024-09-11T09:01:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:50.679837+0000", 2024-09-11T09:01:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00021217299999999999 2024-09-11T09:01:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15918.0:112440 11.5 e405/399 11:a95e779a:::benchmark_data_smithi092_205252_object112439:head v 405'7012, mlcod=405'7010)", 2024-09-11T09:01:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:58:52.302003+0000", 2024-09-11T09:01:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 165.12662205800001, 2024-09-11T09:01:37.535 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.026746037, 2024-09-11T09:01:37.535 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.535 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.535 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.535 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.535 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.535 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:52.302003+0000", 2024-09-11T09:01:37.535 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.535 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.535 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.535 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.535 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:52.302003+0000", 2024-09-11T09:01:37.535 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.536 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.536 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.536 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.536 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:52.302002+0000", 2024-09-11T09:01:37.536 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:37.536 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.536 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.536 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.536 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:52.302026+0000", 2024-09-11T09:01:37.536 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4057999999999999e-05 2024-09-11T09:01:37.536 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.536 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.536 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.536 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:52.302038+0000", 2024-09-11T09:01:37.537 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1467999999999999e-05 2024-09-11T09:01:37.537 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.537 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.537 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.537 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:52.302043+0000", 2024-09-11T09:01:37.537 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.5430000000000001e-06 2024-09-11T09:01:37.537 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.537 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.537 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.537 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:52.302101+0000", 2024-09-11T09:01:37.537 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.7590999999999998e-05 2024-09-11T09:01:37.537 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.537 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.538 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.538 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:52.302120+0000", 2024-09-11T09:01:37.538 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.9233999999999999e-05 2024-09-11T09:01:37.538 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.538 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.538 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.538 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:52.328658+0000", 2024-09-11T09:01:37.538 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.026537551 2024-09-11T09:01:37.538 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.538 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.538 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.538 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:58:52.328749+0000", 2024-09-11T09:01:37.538 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.1463000000000002e-05 2024-09-11T09:01:37.538 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.539 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.539 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.539 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.539 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.539 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15918.0:154172 11.d e414/403 11:bb20286c:::benchmark_data_smithi092_205252_object32851:head v 414'9702, mlcod=414'9699)", 2024-09-11T09:01:37.539 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T08:59:24.722397+0000", 2024-09-11T09:01:37.539 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 132.706228189, 2024-09-11T09:01:37.539 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.025181335999999999, 2024-09-11T09:01:37.539 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.539 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.539 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.539 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.539 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.540 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:59:24.722397+0000", 2024-09-11T09:01:37.540 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.540 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.540 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.540 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.540 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:59:24.722397+0000", 2024-09-11T09:01:37.540 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.540 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.540 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.540 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.540 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:59:24.722396+0000", 2024-09-11T09:01:37.540 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:37.540 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.540 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.541 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.541 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:59:24.722403+0000", 2024-09-11T09:01:37.541 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.4389999999999997e-06 2024-09-11T09:01:37.541 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.541 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.541 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.541 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:59:24.722414+0000", 2024-09-11T09:01:37.541 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0745e-05 2024-09-11T09:01:37.541 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.541 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.541 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.541 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:59:24.722419+0000", 2024-09-11T09:01:37.541 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3129999999999998e-06 2024-09-11T09:01:37.542 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.542 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.542 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.542 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:59:24.726294+0000", 2024-09-11T09:01:37.542 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0038751720000000001 2024-09-11T09:01:37.542 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.542 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.542 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.542 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:59:24.726315+0000", 2024-09-11T09:01:37.542 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0534e-05 2024-09-11T09:01:37.542 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.542 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.543 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.543 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:59:24.747562+0000", 2024-09-11T09:01:37.543 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.021247818000000002 2024-09-11T09:01:37.543 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.543 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.543 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.543 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T08:59:24.747578+0000", 2024-09-11T09:01:37.543 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6007000000000001e-05 2024-09-11T09:01:37.543 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.543 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.543 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.543 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.543 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.543 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15918.0:241096 11.5 e425/399 11:ad8aa82a:::benchmark_data_smithi092_205252_object119775:head v 425'15118, mlcod=425'15115)", 2024-09-11T09:01:37.544 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:33.290399+0000", 2024-09-11T09:01:37.544 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 64.138226669999995, 2024-09-11T09:01:37.544 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.024243614, 2024-09-11T09:01:37.544 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:37.544 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:37.544 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:37.544 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.544 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:37.544 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.290399+0000", 2024-09-11T09:01:37.544 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.544 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.544 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.544 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:37.544 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.290399+0000", 2024-09-11T09:01:37.545 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:37.545 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.545 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.545 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:37.545 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.290398+0000", 2024-09-11T09:01:37.545 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:37.545 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.545 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.545 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:37.545 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.290404+0000", 2024-09-11T09:01:37.545 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.609e-06 2024-09-11T09:01:37.545 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.545 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.545 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:37.546 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.290413+0000", 2024-09-11T09:01:37.546 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.6239999999999997e-06 2024-09-11T09:01:37.546 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.546 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.546 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:37.546 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.290418+0000", 2024-09-11T09:01:37.546 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:01:37.546 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.546 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.546 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:37.546 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.293846+0000", 2024-09-11T09:01:37.546 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0034277069999999999 2024-09-11T09:01:37.546 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.546 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.547 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:37.547 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.293868+0000", 2024-09-11T09:01:37.547 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.1917e-05 2024-09-11T09:01:37.547 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.547 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.547 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:37.547 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.314604+0000", 2024-09-11T09:01:37.547 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.020736570999999999 2024-09-11T09:01:37.547 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:37.547 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:37.547 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:37.547 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.314642+0000", 2024-09-11T09:01:37.547 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7982999999999999e-05 2024-09-11T09:01:37.547 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.548 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.548 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.548 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:37.548 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:37.548 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T09:01:37.548 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:01:37 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:01:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:37 smithi092 ceph-mon[29647]: 6.1e deep-scrub starts 2024-09-11T09:01:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:37 smithi092 ceph-mon[29647]: 6.1e deep-scrub ok 2024-09-11T09:01:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:37 smithi092 ceph-mon[29647]: 3.18 scrub starts 2024-09-11T09:01:37.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:37 smithi092 ceph-mon[29647]: 3.18 scrub ok 2024-09-11T09:01:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:37 smithi092 ceph-mon[41114]: 6.1e deep-scrub starts 2024-09-11T09:01:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:37 smithi092 ceph-mon[41114]: 6.1e deep-scrub ok 2024-09-11T09:01:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:37 smithi092 ceph-mon[41114]: 3.18 scrub starts 2024-09-11T09:01:37.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:37 smithi092 ceph-mon[41114]: 3.18 scrub ok 2024-09-11T09:01:37.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:37 smithi145 ceph-mon[33384]: 6.1e deep-scrub starts 2024-09-11T09:01:37.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:37 smithi145 ceph-mon[33384]: 6.1e deep-scrub ok 2024-09-11T09:01:37.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:37 smithi145 ceph-mon[33384]: 3.18 scrub starts 2024-09-11T09:01:37.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:37 smithi145 ceph-mon[33384]: 3.18 scrub ok 2024-09-11T09:01:38.539 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-11T09:01:38.774 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T09:01:38.796 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:38 smithi145 ceph-mon[33384]: 6.12 deep-scrub starts 2024-09-11T09:01:38.797 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:38 smithi145 ceph-mon[33384]: 6.12 deep-scrub ok 2024-09-11T09:01:38.797 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:38 smithi145 ceph-mon[33384]: 6.19 scrub starts 2024-09-11T09:01:38.797 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:38 smithi145 ceph-mon[33384]: 6.19 scrub ok 2024-09-11T09:01:38.797 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:38 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:38.797 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:38 smithi145 ceph-mon[33384]: pgmap v1049: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 127 B/s, 4 objects/s recovering 2024-09-11T09:01:38.797 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:38 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/926015161' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T09:01:38.797 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:38 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T09:01:38.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[29647]: 6.12 deep-scrub starts 2024-09-11T09:01:38.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[29647]: 6.12 deep-scrub ok 2024-09-11T09:01:38.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[29647]: 6.19 scrub starts 2024-09-11T09:01:38.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[29647]: 6.19 scrub ok 2024-09-11T09:01:38.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:38.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[29647]: pgmap v1049: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 127 B/s, 4 objects/s recovering 2024-09-11T09:01:38.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/926015161' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T09:01:38.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T09:01:38.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[41114]: 6.12 deep-scrub starts 2024-09-11T09:01:38.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[41114]: 6.12 deep-scrub ok 2024-09-11T09:01:38.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[41114]: 6.19 scrub starts 2024-09-11T09:01:38.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[41114]: 6.19 scrub ok 2024-09-11T09:01:38.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:38.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[41114]: pgmap v1049: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 127 B/s, 4 objects/s recovering 2024-09-11T09:01:38.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/926015161' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T09:01:38.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:38 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T09:01:39.017 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T09:01:39.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 3 1 2024-09-11T09:01:39.605 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:39.726 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:39 smithi092 ceph-mon[29647]: 6.2 deep-scrub starts 2024-09-11T09:01:39.726 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:39 smithi092 ceph-mon[29647]: 6.2 deep-scrub ok 2024-09-11T09:01:39.726 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:39 smithi092 ceph-mon[29647]: 4.1f scrub starts 2024-09-11T09:01:39.726 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:39 smithi092 ceph-mon[29647]: 4.1f scrub ok 2024-09-11T09:01:39.726 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:39 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-11T09:01:39.726 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:39 smithi092 ceph-mon[29647]: osdmap e445: 8 total, 8 up, 8 in 2024-09-11T09:01:39.727 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:39 smithi092 ceph-mon[41114]: 6.2 deep-scrub starts 2024-09-11T09:01:39.727 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:39 smithi092 ceph-mon[41114]: 6.2 deep-scrub ok 2024-09-11T09:01:39.727 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:39 smithi092 ceph-mon[41114]: 4.1f scrub starts 2024-09-11T09:01:39.727 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:39 smithi092 ceph-mon[41114]: 4.1f scrub ok 2024-09-11T09:01:39.727 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:39 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-11T09:01:39.727 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:39 smithi092 ceph-mon[41114]: osdmap e445: 8 total, 8 up, 8 in 2024-09-11T09:01:39.907 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:39 smithi145 ceph-mon[33384]: 6.2 deep-scrub starts 2024-09-11T09:01:39.908 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:39 smithi145 ceph-mon[33384]: 6.2 deep-scrub ok 2024-09-11T09:01:39.908 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:39 smithi145 ceph-mon[33384]: 4.1f scrub starts 2024-09-11T09:01:39.908 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:39 smithi145 ceph-mon[33384]: 4.1f scrub ok 2024-09-11T09:01:39.908 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:39 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-11T09:01:39.908 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:39 smithi145 ceph-mon[33384]: osdmap e445: 8 total, 8 up, 8 in 2024-09-11T09:01:40.730 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T09:01:40.730 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T09:01:40.731 INFO:teuthology.orchestra.run.smithi145.stdout: "num_ops": 0 2024-09-11T09:01:40.731 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T09:01:40.809 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[29647]: 4.d scrub starts 2024-09-11T09:01:40.809 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[29647]: 4.d scrub ok 2024-09-11T09:01:40.809 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[29647]: 4.10 scrub starts 2024-09-11T09:01:40.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[29647]: 4.10 scrub ok 2024-09-11T09:01:40.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:01:40.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:01:40.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[29647]: pgmap v1051: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 99 B/s, 2 objects/s recovering 2024-09-11T09:01:40.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[29647]: osdmap e446: 8 total, 8 up, 8 in 2024-09-11T09:01:40.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[29647]: 6.4 scrub starts 2024-09-11T09:01:40.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[29647]: 6.4 scrub ok 2024-09-11T09:01:40.810 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[41114]: 4.d scrub starts 2024-09-11T09:01:40.810 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[41114]: 4.d scrub ok 2024-09-11T09:01:40.810 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[41114]: 4.10 scrub starts 2024-09-11T09:01:40.810 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[41114]: 4.10 scrub ok 2024-09-11T09:01:40.811 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:01:40.811 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:01:40.811 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[41114]: pgmap v1051: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 99 B/s, 2 objects/s recovering 2024-09-11T09:01:40.811 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[41114]: osdmap e446: 8 total, 8 up, 8 in 2024-09-11T09:01:40.811 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[41114]: 6.4 scrub starts 2024-09-11T09:01:40.811 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:40 smithi092 ceph-mon[41114]: 6.4 scrub ok 2024-09-11T09:01:40.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:40 smithi145 ceph-mon[33384]: 4.d scrub starts 2024-09-11T09:01:40.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:40 smithi145 ceph-mon[33384]: 4.d scrub ok 2024-09-11T09:01:40.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:40 smithi145 ceph-mon[33384]: 4.10 scrub starts 2024-09-11T09:01:40.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:40 smithi145 ceph-mon[33384]: 4.10 scrub ok 2024-09-11T09:01:40.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:40 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:01:40.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:40 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:01:40.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:40 smithi145 ceph-mon[33384]: pgmap v1051: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 99 B/s, 2 objects/s recovering 2024-09-11T09:01:40.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:40 smithi145 ceph-mon[33384]: osdmap e446: 8 total, 8 up, 8 in 2024-09-11T09:01:40.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:40 smithi145 ceph-mon[33384]: 6.4 scrub starts 2024-09-11T09:01:40.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:40 smithi145 ceph-mon[33384]: 6.4 scrub ok 2024-09-11T09:01:41.374 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-11T09:01:41.548 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T09:01:41.659 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T09:01:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:41 smithi145 ceph-mon[33384]: 5.a scrub starts 2024-09-11T09:01:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:41 smithi145 ceph-mon[33384]: 5.a scrub ok 2024-09-11T09:01:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:41 smithi145 ceph-mon[33384]: 5.9 scrub starts 2024-09-11T09:01:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:41 smithi145 ceph-mon[33384]: 5.9 scrub ok 2024-09-11T09:01:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:41 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2574624958' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T09:01:41.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:01:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:01:41] "GET /metrics HTTP/1.1" 200 222520 "" "Prometheus/2.43.0" 2024-09-11T09:01:41.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:41 smithi092 ceph-mon[29647]: 5.a scrub starts 2024-09-11T09:01:41.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:41 smithi092 ceph-mon[29647]: 5.a scrub ok 2024-09-11T09:01:41.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:41 smithi092 ceph-mon[29647]: 5.9 scrub starts 2024-09-11T09:01:41.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:41 smithi092 ceph-mon[29647]: 5.9 scrub ok 2024-09-11T09:01:41.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:41 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2574624958' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T09:01:41.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:41 smithi092 ceph-mon[41114]: 5.a scrub starts 2024-09-11T09:01:41.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:41 smithi092 ceph-mon[41114]: 5.a scrub ok 2024-09-11T09:01:41.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:41 smithi092 ceph-mon[41114]: 5.9 scrub starts 2024-09-11T09:01:41.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:41 smithi092 ceph-mon[41114]: 5.9 scrub ok 2024-09-11T09:01:41.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:41 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2574624958' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T09:01:42.333 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 3 1 2024-09-11T09:01:42.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:42 smithi092 ceph-mon[41114]: pgmap v1053: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 586 B/s rd, 0 op/s 2024-09-11T09:01:42.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:42 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2574624958' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T09:01:42.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:42 smithi092 ceph-mon[41114]: osdmap e447: 8 total, 8 up, 8 in 2024-09-11T09:01:42.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:42 smithi092 ceph-mon[41114]: 3.12 scrub starts 2024-09-11T09:01:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:42 smithi092 ceph-mon[41114]: 3.12 scrub ok 2024-09-11T09:01:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:42 smithi092 ceph-mon[29647]: pgmap v1053: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 586 B/s rd, 0 op/s 2024-09-11T09:01:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:42 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2574624958' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T09:01:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:42 smithi092 ceph-mon[29647]: osdmap e447: 8 total, 8 up, 8 in 2024-09-11T09:01:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:42 smithi092 ceph-mon[29647]: 3.12 scrub starts 2024-09-11T09:01:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:42 smithi092 ceph-mon[29647]: 3.12 scrub ok 2024-09-11T09:01:42.576 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:42.822 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:01:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:01:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:01:42.823 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:42 smithi145 ceph-mon[33384]: pgmap v1053: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 586 B/s rd, 0 op/s 2024-09-11T09:01:42.823 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:42 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2574624958' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T09:01:42.823 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:42 smithi145 ceph-mon[33384]: osdmap e447: 8 total, 8 up, 8 in 2024-09-11T09:01:42.823 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:42 smithi145 ceph-mon[33384]: 3.12 scrub starts 2024-09-11T09:01:42.823 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:42 smithi145 ceph-mon[33384]: 3.12 scrub ok 2024-09-11T09:01:43.393 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T09:01:43.393 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [], 2024-09-11T09:01:43.393 INFO:teuthology.orchestra.run.smithi145.stdout: "complaint_time": 30, 2024-09-11T09:01:43.393 INFO:teuthology.orchestra.run.smithi145.stdout: "num_blocked_ops": 0 2024-09-11T09:01:43.393 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T09:01:43.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:43 smithi092 ceph-mon[41114]: 6.1b scrub starts 2024-09-11T09:01:43.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:43 smithi092 ceph-mon[41114]: 6.1b scrub ok 2024-09-11T09:01:43.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:43 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T09:01:43.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:43 smithi092 ceph-mon[41114]: 6.f scrub ok 2024-09-11T09:01:43.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:43 smithi092 ceph-mon[29647]: 6.1b scrub starts 2024-09-11T09:01:43.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:43 smithi092 ceph-mon[29647]: 6.1b scrub ok 2024-09-11T09:01:43.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:43 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T09:01:43.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:43 smithi092 ceph-mon[29647]: 6.f scrub ok 2024-09-11T09:01:43.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:43 smithi145 ceph-mon[33384]: 6.1b scrub starts 2024-09-11T09:01:43.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:43 smithi145 ceph-mon[33384]: 6.1b scrub ok 2024-09-11T09:01:43.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:43 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T09:01:43.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:43 smithi145 ceph-mon[33384]: 6.f scrub ok 2024-09-11T09:01:44.031 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-11T09:01:44.277 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T09:01:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:44 smithi145 ceph-mon[33384]: 5.f scrub starts 2024-09-11T09:01:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:44 smithi145 ceph-mon[33384]: 5.f scrub ok 2024-09-11T09:01:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:44 smithi145 ceph-mon[33384]: pgmap v1055: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T09:01:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:44 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3884911096' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T09:01:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:44 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T09:01:44.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:44 smithi092 ceph-mon[29647]: 5.f scrub starts 2024-09-11T09:01:44.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:44 smithi092 ceph-mon[29647]: 5.f scrub ok 2024-09-11T09:01:44.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:44 smithi092 ceph-mon[29647]: pgmap v1055: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T09:01:44.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:44 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3884911096' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T09:01:44.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:44 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T09:01:44.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:44 smithi092 ceph-mon[41114]: 5.f scrub starts 2024-09-11T09:01:44.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:44 smithi092 ceph-mon[41114]: 5.f scrub ok 2024-09-11T09:01:44.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:44 smithi092 ceph-mon[41114]: pgmap v1055: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T09:01:44.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:44 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3884911096' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T09:01:44.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:44 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T09:01:45.172 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-11T09:01:45.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:45 smithi092 ceph-mon[29647]: 6.a scrub starts 2024-09-11T09:01:45.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:45 smithi092 ceph-mon[29647]: 6.a scrub ok 2024-09-11T09:01:45.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:01:45.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:45 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-11T09:01:45.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:45 smithi092 ceph-mon[29647]: osdmap e448: 8 total, 8 up, 8 in 2024-09-11T09:01:45.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:45 smithi092 ceph-mon[29647]: 5.19 scrub starts 2024-09-11T09:01:45.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:45 smithi092 ceph-mon[29647]: 5.19 scrub ok 2024-09-11T09:01:45.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:45 smithi092 ceph-mon[41114]: 6.a scrub starts 2024-09-11T09:01:45.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:45 smithi092 ceph-mon[41114]: 6.a scrub ok 2024-09-11T09:01:45.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:01:45.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:45 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-11T09:01:45.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:45 smithi092 ceph-mon[41114]: osdmap e448: 8 total, 8 up, 8 in 2024-09-11T09:01:45.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:45 smithi092 ceph-mon[41114]: 5.19 scrub starts 2024-09-11T09:01:45.818 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:45 smithi092 ceph-mon[41114]: 5.19 scrub ok 2024-09-11T09:01:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:45 smithi145 ceph-mon[33384]: 6.a scrub starts 2024-09-11T09:01:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:45 smithi145 ceph-mon[33384]: 6.a scrub ok 2024-09-11T09:01:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:01:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:45 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-11T09:01:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:45 smithi145 ceph-mon[33384]: osdmap e448: 8 total, 8 up, 8 in 2024-09-11T09:01:45.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:45 smithi145 ceph-mon[33384]: 5.19 scrub starts 2024-09-11T09:01:45.913 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:45 smithi145 ceph-mon[33384]: 5.19 scrub ok 2024-09-11T09:01:46.011 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 4 1 2024-09-11T09:01:46.149 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-09-11T09:01:46.149 INFO:teuthology.orchestra.run.smithi145.stdout: "size": 20, 2024-09-11T09:01:46.149 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 600, 2024-09-11T09:01:46.149 INFO:teuthology.orchestra.run.smithi145.stdout: "ops": [ 2024-09-11T09:01:46.149 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.149 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:235478 11.4 11:2655db11:::benchmark_data_smithi092_205252_object114157:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T09:01:46.149 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:28.845726+0000", 2024-09-11T09:01:46.149 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 77.302383227000007, 2024-09-11T09:01:46.150 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.025246459999999998, 2024-09-11T09:01:46.150 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.150 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:46.150 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:46.150 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:46.150 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:46.150 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 235478 2024-09-11T09:01:46.150 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.150 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.150 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.150 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.150 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.845726+0000", 2024-09-11T09:01:46.150 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.151 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.151 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.151 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.151 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.845726+0000", 2024-09-11T09:01:46.151 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.151 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.151 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.151 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.151 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.845725+0000", 2024-09-11T09:01:46.151 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:46.151 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.151 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.151 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.151 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.845730+0000", 2024-09-11T09:01:46.151 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8790000000000002e-06 2024-09-11T09:01:46.152 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.152 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.152 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.152 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.845740+0000", 2024-09-11T09:01:46.152 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.8989999999999996e-06 2024-09-11T09:01:46.152 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.152 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.152 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.152 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.845745+0000", 2024-09-11T09:01:46.152 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.3190000000000002e-06 2024-09-11T09:01:46.152 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.152 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.152 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.152 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.850827+0000", 2024-09-11T09:01:46.153 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0050820350000000004 2024-09-11T09:01:46.153 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.153 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.153 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.153 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.850944+0000", 2024-09-11T09:01:46.153 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000117556 2024-09-11T09:01:46.153 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.153 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.153 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,6", 2024-09-11T09:01:46.153 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.851055+0000", 2024-09-11T09:01:46.153 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011014999999999999 2024-09-11T09:01:46.153 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.153 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.153 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:46.154 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.853142+0000", 2024-09-11T09:01:46.154 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0020872400000000002 2024-09-11T09:01:46.154 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.154 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.154 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:46.154 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.854985+0000", 2024-09-11T09:01:46.154 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001843194 2024-09-11T09:01:46.154 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.154 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.154 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:46.154 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.870877+0000", 2024-09-11T09:01:46.154 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.015891760000000001 2024-09-11T09:01:46.154 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.154 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.154 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:46.155 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.870926+0000", 2024-09-11T09:01:46.155 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.9063000000000001e-05 2024-09-11T09:01:46.155 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.155 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.155 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.155 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.870972+0000", 2024-09-11T09:01:46.155 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6181999999999999e-05 2024-09-11T09:01:46.155 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.155 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.155 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.155 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.155 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.155 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:235575 11.4 11:2eb0e8e9:::benchmark_data_smithi092_205252_object114254:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T09:01:46.156 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:28.913463+0000", 2024-09-11T09:01:46.156 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 77.234645404999995, 2024-09-11T09:01:46.156 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.025607568000000001, 2024-09-11T09:01:46.156 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.156 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:46.156 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:46.156 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:46.156 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:46.156 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 235575 2024-09-11T09:01:46.156 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.156 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.156 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.156 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.156 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.913463+0000", 2024-09-11T09:01:46.156 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.157 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.157 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.157 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.157 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.913463+0000", 2024-09-11T09:01:46.157 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.157 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.157 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.157 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.157 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.913462+0000", 2024-09-11T09:01:46.157 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:46.157 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.157 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.157 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.157 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.913468+0000", 2024-09-11T09:01:46.158 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.2950000000000001e-06 2024-09-11T09:01:46.158 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.158 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.158 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.158 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.913483+0000", 2024-09-11T09:01:46.158 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.5068999999999999e-05 2024-09-11T09:01:46.158 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.158 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.158 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.158 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.913489+0000", 2024-09-11T09:01:46.158 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.8050000000000003e-06 2024-09-11T09:01:46.158 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.159 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.159 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.159 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.915456+0000", 2024-09-11T09:01:46.159 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0019673289999999999 2024-09-11T09:01:46.159 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.159 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.159 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.159 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.915568+0000", 2024-09-11T09:01:46.159 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000112447 2024-09-11T09:01:46.159 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.159 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.159 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,6", 2024-09-11T09:01:46.159 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.915690+0000", 2024-09-11T09:01:46.159 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012129999999999999 2024-09-11T09:01:46.160 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.160 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.160 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:46.160 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.919080+0000", 2024-09-11T09:01:46.160 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.003389834 2024-09-11T09:01:46.160 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.160 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.160 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:46.160 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.923294+0000", 2024-09-11T09:01:46.160 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0042148799999999998 2024-09-11T09:01:46.160 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.160 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.160 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:46.160 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.939021+0000", 2024-09-11T09:01:46.161 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.015726698000000001 2024-09-11T09:01:46.161 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.161 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.161 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:46.161 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.939049+0000", 2024-09-11T09:01:46.161 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.8186000000000001e-05 2024-09-11T09:01:46.161 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.161 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.161 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.161 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:28.939071+0000", 2024-09-11T09:01:46.161 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.1750999999999999e-05 2024-09-11T09:01:46.161 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.161 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.162 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.162 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.162 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.162 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:237862 11.4 11:29cb27b3:::benchmark_data_smithi092_205252_object116541:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:46.162 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:30.739551+0000", 2024-09-11T09:01:46.162 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 75.408558127000006, 2024-09-11T09:01:46.162 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.026735302999999998, 2024-09-11T09:01:46.162 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.162 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:46.162 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:46.162 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:46.162 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:46.162 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 237862 2024-09-11T09:01:46.163 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.163 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.163 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.163 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.164 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.739551+0000", 2024-09-11T09:01:46.164 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.164 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.164 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.164 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.164 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.739551+0000", 2024-09-11T09:01:46.164 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.164 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.164 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.164 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.164 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.739550+0000", 2024-09-11T09:01:46.164 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:46.165 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.165 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.165 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.165 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.739556+0000", 2024-09-11T09:01:46.165 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2729999999999999e-06 2024-09-11T09:01:46.165 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.165 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.165 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.165 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.739567+0000", 2024-09-11T09:01:46.165 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1246e-05 2024-09-11T09:01:46.165 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.165 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.165 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.165 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.739573+0000", 2024-09-11T09:01:46.166 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.1530000000000002e-06 2024-09-11T09:01:46.166 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.166 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.166 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.166 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.741977+0000", 2024-09-11T09:01:46.166 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0024033460000000002 2024-09-11T09:01:46.166 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.166 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.166 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.166 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.742088+0000", 2024-09-11T09:01:46.166 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011162699999999999 2024-09-11T09:01:46.166 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.166 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.166 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,6", 2024-09-11T09:01:46.166 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.742230+0000", 2024-09-11T09:01:46.167 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000141724 2024-09-11T09:01:46.167 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.167 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.167 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:46.167 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.746836+0000", 2024-09-11T09:01:46.167 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0046062259999999997 2024-09-11T09:01:46.167 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.167 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.167 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:46.167 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.746845+0000", 2024-09-11T09:01:46.167 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.1829999999999994e-06 2024-09-11T09:01:46.167 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.167 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.167 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:46.167 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.766229+0000", 2024-09-11T09:01:46.168 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.019383944 2024-09-11T09:01:46.168 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.168 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.168 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:46.168 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.766260+0000", 2024-09-11T09:01:46.168 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1498999999999998e-05 2024-09-11T09:01:46.168 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.168 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.168 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.168 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.766286+0000", 2024-09-11T09:01:46.168 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5984999999999999e-05 2024-09-11T09:01:46.168 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.168 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.168 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.168 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.169 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.169 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:237866 11.4 11:296f6833:::benchmark_data_smithi092_205252_object116545:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:46.169 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:30.740117+0000", 2024-09-11T09:01:46.169 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 75.407992297999996, 2024-09-11T09:01:46.169 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.029071577000000001, 2024-09-11T09:01:46.169 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.169 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:46.169 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:46.169 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:46.169 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:46.169 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 237866 2024-09-11T09:01:46.169 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.169 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.169 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.170 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.170 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.740117+0000", 2024-09-11T09:01:46.170 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.170 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.170 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.170 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.170 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.740117+0000", 2024-09-11T09:01:46.170 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.170 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.170 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.170 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.170 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.740116+0000", 2024-09-11T09:01:46.170 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:46.170 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.171 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.171 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.171 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.740120+0000", 2024-09-11T09:01:46.171 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9999999999999998e-06 2024-09-11T09:01:46.171 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.171 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.171 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.171 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.740128+0000", 2024-09-11T09:01:46.171 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 8.5709999999999998e-06 2024-09-11T09:01:46.171 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.171 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.171 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.171 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.740133+0000", 2024-09-11T09:01:46.171 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.87e-06 2024-09-11T09:01:46.172 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.172 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.172 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.172 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.744668+0000", 2024-09-11T09:01:46.172 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0045347360000000001 2024-09-11T09:01:46.172 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.172 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.172 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.172 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.744778+0000", 2024-09-11T09:01:46.172 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000110392 2024-09-11T09:01:46.172 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.172 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.173 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,6", 2024-09-11T09:01:46.173 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.744891+0000", 2024-09-11T09:01:46.173 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000112254 2024-09-11T09:01:46.173 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.173 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.173 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:46.173 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.746873+0000", 2024-09-11T09:01:46.173 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0019825419999999999 2024-09-11T09:01:46.174 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.174 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.174 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:46.174 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.746880+0000", 2024-09-11T09:01:46.174 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.199e-06 2024-09-11T09:01:46.174 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.174 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.174 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:46.174 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.769138+0000", 2024-09-11T09:01:46.174 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.022257097 2024-09-11T09:01:46.174 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.174 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.175 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:46.175 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.769163+0000", 2024-09-11T09:01:46.175 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5891999999999999e-05 2024-09-11T09:01:46.175 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.175 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.175 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.175 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:30.769188+0000", 2024-09-11T09:01:46.175 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4719000000000002e-05 2024-09-11T09:01:46.175 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.175 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.175 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.175 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.175 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.175 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:238392 11.4 11:2ccc0f2e:::benchmark_data_smithi092_205252_object117071:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:46.175 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:31.136824+0000", 2024-09-11T09:01:46.176 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 75.011284970999995, 2024-09-11T09:01:46.176 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.028687354000000002, 2024-09-11T09:01:46.176 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.176 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:46.176 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:46.176 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:46.176 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:46.176 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 238392 2024-09-11T09:01:46.176 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.176 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.176 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.176 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.176 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:31.136824+0000", 2024-09-11T09:01:46.176 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.177 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.177 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.177 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.177 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:31.136824+0000", 2024-09-11T09:01:46.177 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.177 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.177 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.177 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.177 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:31.136823+0000", 2024-09-11T09:01:46.177 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:46.177 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.177 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.177 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.177 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:31.136829+0000", 2024-09-11T09:01:46.177 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.3790000000000003e-06 2024-09-11T09:01:46.178 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.178 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.178 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.178 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:31.136841+0000", 2024-09-11T09:01:46.178 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1919999999999999e-05 2024-09-11T09:01:46.178 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.178 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.178 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.178 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:31.136847+0000", 2024-09-11T09:01:46.178 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.6810000000000003e-06 2024-09-11T09:01:46.178 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.178 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.178 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.178 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:31.138121+0000", 2024-09-11T09:01:46.178 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0012738249999999999 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:31.138231+0000", 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011027999999999999 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,6", 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:31.138335+0000", 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000104492 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:31.141296+0000", 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0029603920000000001 2024-09-11T09:01:46.179 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.180 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.180 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:46.180 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:31.142349+0000", 2024-09-11T09:01:46.180 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.001052854 2024-09-11T09:01:46.180 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.180 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.180 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:46.180 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:31.165418+0000", 2024-09-11T09:01:46.180 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.023069075000000001 2024-09-11T09:01:46.180 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.180 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.180 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:46.180 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:31.165466+0000", 2024-09-11T09:01:46.180 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.7920999999999997e-05 2024-09-11T09:01:46.181 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.181 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.181 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.181 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:31.165511+0000", 2024-09-11T09:01:46.181 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5516999999999998e-05 2024-09-11T09:01:46.181 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.181 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.181 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.181 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.181 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.181 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_repop(client.15918.0:240168 11.0 e425/366 11:0a55e90b:::benchmark_data_smithi092_205252_object118847:head v 425'14889, mlcod=425'14886)", 2024-09-11T09:01:46.181 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:32.559152+0000", 2024-09-11T09:01:46.181 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 73.588956620000005, 2024-09-11T09:01:46.181 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.027145870999999999, 2024-09-11T09:01:46.182 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.182 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:46.182 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.182 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.182 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.182 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.559152+0000", 2024-09-11T09:01:46.182 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.182 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.182 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.182 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.182 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.559152+0000", 2024-09-11T09:01:46.182 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.182 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.182 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.183 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.183 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.559151+0000", 2024-09-11T09:01:46.183 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.999999 2024-09-11T09:01:46.183 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.183 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.183 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.183 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.559162+0000", 2024-09-11T09:01:46.183 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0674e-05 2024-09-11T09:01:46.183 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.183 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.183 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.183 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.559180+0000", 2024-09-11T09:01:46.183 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7498000000000001e-05 2024-09-11T09:01:46.184 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.184 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.184 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.184 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.559186+0000", 2024-09-11T09:01:46.184 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.2419999999999997e-06 2024-09-11T09:01:46.184 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.184 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.184 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.184 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.564979+0000", 2024-09-11T09:01:46.184 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0057929670000000004 2024-09-11T09:01:46.184 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.184 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.184 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.185 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.565000+0000", 2024-09-11T09:01:46.185 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.0950000000000001e-05 2024-09-11T09:01:46.185 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.185 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.185 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:46.185 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.586258+0000", 2024-09-11T09:01:46.185 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.021258549000000002 2024-09-11T09:01:46.185 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.185 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.185 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.185 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:32.586298+0000", 2024-09-11T09:01:46.185 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.9823999999999997e-05 2024-09-11T09:01:46.185 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.186 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.186 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.186 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.186 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.186 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "osd_op(client.15918.0:241941 11.4 11:291e243d:::benchmark_data_smithi092_205252_object120620:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e425)", 2024-09-11T09:01:46.186 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:00:33.929722+0000", 2024-09-11T09:01:46.186 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 72.218386811000002, 2024-09-11T09:01:46.186 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.027207011999999999, 2024-09-11T09:01:46.186 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.186 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:01:46.186 INFO:teuthology.orchestra.run.smithi145.stdout: "client_info": { 2024-09-11T09:01:46.186 INFO:teuthology.orchestra.run.smithi145.stdout: "client": "client.15918", 2024-09-11T09:01:46.186 INFO:teuthology.orchestra.run.smithi145.stdout: "client_addr": "172.21.15.92:0/2039650841", 2024-09-11T09:01:46.186 INFO:teuthology.orchestra.run.smithi145.stdout: "tid": 241941 2024-09-11T09:01:46.186 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.187 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.187 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.187 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.187 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.929722+0000", 2024-09-11T09:01:46.187 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.187 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.187 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.187 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.187 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.929722+0000", 2024-09-11T09:01:46.187 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.187 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.187 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.187 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.187 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.929721+0000", 2024-09-11T09:01:46.188 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:46.188 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.188 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.188 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.188 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.929728+0000", 2024-09-11T09:01:46.188 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.5350000000000001e-06 2024-09-11T09:01:46.188 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.188 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.188 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.188 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.929741+0000", 2024-09-11T09:01:46.188 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2924e-05 2024-09-11T09:01:46.188 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.188 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.188 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.189 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.929746+0000", 2024-09-11T09:01:46.189 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1120000000000004e-06 2024-09-11T09:01:46.189 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.189 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.189 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.189 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.932072+0000", 2024-09-11T09:01:46.189 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0023258620000000002 2024-09-11T09:01:46.189 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.189 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.189 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.189 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.932185+0000", 2024-09-11T09:01:46.189 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00011288700000000001 2024-09-11T09:01:46.189 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.190 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.190 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "waiting for subops from 2,6", 2024-09-11T09:01:46.190 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.932298+0000", 2024-09-11T09:01:46.190 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000113201 2024-09-11T09:01:46.190 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.190 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.190 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op_commit", 2024-09-11T09:01:46.190 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.935709+0000", 2024-09-11T09:01:46.190 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0034107249999999999 2024-09-11T09:01:46.190 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.190 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.190 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:46.190 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.936339+0000", 2024-09-11T09:01:46.190 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00063052099999999999 2024-09-11T09:01:46.191 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.191 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.191 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:01:46.191 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.956817+0000", 2024-09-11T09:01:46.191 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.020477787000000001 2024-09-11T09:01:46.191 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.191 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.191 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "commit_sent", 2024-09-11T09:01:46.191 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.956892+0000", 2024-09-11T09:01:46.191 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.4789999999999994e-05 2024-09-11T09:01:46.191 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.191 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.191 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.191 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:00:33.956929+0000", 2024-09-11T09:01:46.192 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.7378e-05 2024-09-11T09:01:46.192 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.192 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.192 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.192 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.192 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.192 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "MOSDScrubReserve(6.9 REQUEST e431)", 2024-09-11T09:01:46.192 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:01:02.197029+0000", 2024-09-11T09:01:46.192 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 43.951079624999998, 2024-09-11T09:01:46.192 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00026104999999999998, 2024-09-11T09:01:46.192 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.192 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:46.192 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.193 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.193 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.193 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.197029+0000", 2024-09-11T09:01:46.193 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.193 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.193 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.193 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.193 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.197029+0000", 2024-09-11T09:01:46.193 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.193 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.193 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.193 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.193 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.197027+0000", 2024-09-11T09:01:46.193 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T09:01:46.194 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.194 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.194 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.194 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.197041+0000", 2024-09-11T09:01:46.194 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3862999999999999e-05 2024-09-11T09:01:46.194 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.194 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.194 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.194 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.197071+0000", 2024-09-11T09:01:46.194 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9810000000000001e-05 2024-09-11T09:01:46.194 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.194 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.194 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.194 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.197084+0000", 2024-09-11T09:01:46.195 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2839e-05 2024-09-11T09:01:46.195 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.195 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.195 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.195 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.197255+0000", 2024-09-11T09:01:46.195 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000171278 2024-09-11T09:01:46.195 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.195 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.195 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.195 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.197272+0000", 2024-09-11T09:01:46.195 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7306999999999999e-05 2024-09-11T09:01:46.195 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.195 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.195 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.195 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.197290+0000", 2024-09-11T09:01:46.196 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.8070000000000001e-05 2024-09-11T09:01:46.196 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.196 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.196 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.196 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.196 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.196 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "replica_scrub(pg: 6.9,from:0'0,to:0'0,epoch:431/290,start:6:90000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-11T09:01:46.196 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:01:02.198055+0000", 2024-09-11T09:01:46.196 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 43.950053574999998, 2024-09-11T09:01:46.196 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00026342200000000003, 2024-09-11T09:01:46.196 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.196 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:46.196 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.196 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.197 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.197 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.198055+0000", 2024-09-11T09:01:46.197 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.197 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.197 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.197 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.197 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.198055+0000", 2024-09-11T09:01:46.197 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.197 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.197 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.197 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.197 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.198054+0000", 2024-09-11T09:01:46.197 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:46.197 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.197 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.198 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.198 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.198067+0000", 2024-09-11T09:01:46.198 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2916e-05 2024-09-11T09:01:46.198 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.198 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.198 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.198 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.198098+0000", 2024-09-11T09:01:46.198 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.1319999999999998e-05 2024-09-11T09:01:46.198 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.198 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.198 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.198 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.198110+0000", 2024-09-11T09:01:46.198 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2452999999999999e-05 2024-09-11T09:01:46.199 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.199 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.199 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.199 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.198250+0000", 2024-09-11T09:01:46.199 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00013908099999999999 2024-09-11T09:01:46.199 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.199 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.199 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.199 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.198271+0000", 2024-09-11T09:01:46.199 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.1112e-05 2024-09-11T09:01:46.199 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.199 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.199 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.199 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:02.198319+0000", 2024-09-11T09:01:46.200 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8025999999999997e-05 2024-09-11T09:01:46.200 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.200 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.200 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.200 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.200 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.200 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "MOSDScrubReserve(1.14 REQUEST e436)", 2024-09-11T09:01:46.200 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:01:15.237111+0000", 2024-09-11T09:01:46.200 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 30.910997698999999, 2024-09-11T09:01:46.200 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000221506, 2024-09-11T09:01:46.200 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.201 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:46.201 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.201 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.201 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.201 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.237111+0000", 2024-09-11T09:01:46.201 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.201 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.201 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.201 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.201 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.237111+0000", 2024-09-11T09:01:46.201 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.201 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.201 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.202 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.202 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.237110+0000", 2024-09-11T09:01:46.202 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:46.202 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.202 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.202 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.202 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.237120+0000", 2024-09-11T09:01:46.202 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.012e-05 2024-09-11T09:01:46.202 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.202 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.202 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.202 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.237146+0000", 2024-09-11T09:01:46.202 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.6723000000000001e-05 2024-09-11T09:01:46.202 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.203 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.203 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.203 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.237163+0000", 2024-09-11T09:01:46.203 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6305999999999999e-05 2024-09-11T09:01:46.203 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.203 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.203 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.203 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.237254+0000", 2024-09-11T09:01:46.203 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.0744999999999999e-05 2024-09-11T09:01:46.203 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.203 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.203 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.204 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.237278+0000", 2024-09-11T09:01:46.204 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.4263000000000001e-05 2024-09-11T09:01:46.204 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.204 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.204 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.204 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.237333+0000", 2024-09-11T09:01:46.204 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.4892000000000002e-05 2024-09-11T09:01:46.204 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.204 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.204 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.204 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.204 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.204 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "replica_scrub(pg: 1.14,from:0'0,to:0'0,epoch:436/385,start:1:28000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-11T09:01:46.205 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:01:15.237941+0000", 2024-09-11T09:01:46.205 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 30.910167814000001, 2024-09-11T09:01:46.205 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00025297200000000002, 2024-09-11T09:01:46.205 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.205 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:46.205 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.205 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.205 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.205 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.237941+0000", 2024-09-11T09:01:46.205 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.205 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.205 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.205 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.206 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.237941+0000", 2024-09-11T09:01:46.206 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.206 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.206 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.206 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.206 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.237940+0000", 2024-09-11T09:01:46.206 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:46.207 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.207 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.207 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.207 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.237952+0000", 2024-09-11T09:01:46.207 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2072999999999999e-05 2024-09-11T09:01:46.207 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.207 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.207 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.207 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.238000+0000", 2024-09-11T09:01:46.208 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8600000000000002e-05 2024-09-11T09:01:46.208 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.208 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.208 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.208 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.238013+0000", 2024-09-11T09:01:46.208 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2225999999999999e-05 2024-09-11T09:01:46.208 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.208 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.208 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.208 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.238134+0000", 2024-09-11T09:01:46.208 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000121141 2024-09-11T09:01:46.208 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.209 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.209 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.209 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.238151+0000", 2024-09-11T09:01:46.209 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7580000000000001e-05 2024-09-11T09:01:46.209 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.209 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.209 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.209 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:15.238194+0000", 2024-09-11T09:01:46.209 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2762000000000002e-05 2024-09-11T09:01:46.209 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.209 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.209 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.210 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.210 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.210 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "MOSDScrubReserve(6.3 REQUEST e436)", 2024-09-11T09:01:46.210 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:01:16.258536+0000", 2024-09-11T09:01:46.210 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 29.889572645000001, 2024-09-11T09:01:46.210 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00029372799999999999, 2024-09-11T09:01:46.210 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.210 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:46.210 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.210 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.210 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.210 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:16.258536+0000", 2024-09-11T09:01:46.210 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.211 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.211 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.211 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.211 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:16.258536+0000", 2024-09-11T09:01:46.211 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.211 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.211 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.211 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.211 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:16.258534+0000", 2024-09-11T09:01:46.211 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T09:01:46.211 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.211 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.211 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.211 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:16.258548+0000", 2024-09-11T09:01:46.212 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3923e-05 2024-09-11T09:01:46.212 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.212 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.212 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.212 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:16.258576+0000", 2024-09-11T09:01:46.212 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7396e-05 2024-09-11T09:01:46.212 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.212 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.212 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.212 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:16.258588+0000", 2024-09-11T09:01:46.212 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2782999999999999e-05 2024-09-11T09:01:46.212 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.212 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.213 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.213 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:16.258711+0000", 2024-09-11T09:01:46.213 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00012283199999999999 2024-09-11T09:01:46.213 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.213 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.213 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.213 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:16.258752+0000", 2024-09-11T09:01:46.213 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0312000000000003e-05 2024-09-11T09:01:46.213 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.213 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.213 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.213 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:16.258830+0000", 2024-09-11T09:01:46.213 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.8409000000000001e-05 2024-09-11T09:01:46.213 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.214 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.214 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.214 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.214 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.214 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "MOSDScrubReserve(5.4 REQUEST e436)", 2024-09-11T09:01:46.214 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:01:18.310832+0000", 2024-09-11T09:01:46.214 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 27.837276981999999, 2024-09-11T09:01:46.214 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00042560599999999998, 2024-09-11T09:01:46.214 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.214 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:46.214 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.214 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.214 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.215 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:18.310832+0000", 2024-09-11T09:01:46.215 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.215 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.215 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.215 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.215 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:18.310832+0000", 2024-09-11T09:01:46.215 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.215 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.215 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.215 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.215 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:18.310830+0000", 2024-09-11T09:01:46.215 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T09:01:46.215 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.216 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.216 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.216 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:18.310843+0000", 2024-09-11T09:01:46.216 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2677000000000001e-05 2024-09-11T09:01:46.216 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.216 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.216 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.216 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:18.310873+0000", 2024-09-11T09:01:46.216 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9946000000000001e-05 2024-09-11T09:01:46.216 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.216 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.216 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.216 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:18.310888+0000", 2024-09-11T09:01:46.216 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.508e-05 2024-09-11T09:01:46.216 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.217 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.217 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.217 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:18.311015+0000", 2024-09-11T09:01:46.217 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000127494 2024-09-11T09:01:46.217 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.217 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.217 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.217 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:18.311058+0000", 2024-09-11T09:01:46.217 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.2533000000000001e-05 2024-09-11T09:01:46.217 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.217 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.217 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.217 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:18.311257+0000", 2024-09-11T09:01:46.217 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00019969000000000001 2024-09-11T09:01:46.218 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.218 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.218 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.218 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.218 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.218 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "MOSDScrubReserve(6.8 REQUEST e437)", 2024-09-11T09:01:46.218 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:01:20.957365+0000", 2024-09-11T09:01:46.218 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 25.190744173999999, 2024-09-11T09:01:46.218 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00023923299999999999, 2024-09-11T09:01:46.218 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.218 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:46.218 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.218 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.218 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.218 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:20.957365+0000", 2024-09-11T09:01:46.219 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.219 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.219 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.219 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.219 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:20.957365+0000", 2024-09-11T09:01:46.219 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.219 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.219 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.219 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.219 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:20.957362+0000", 2024-09-11T09:01:46.219 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999976 2024-09-11T09:01:46.219 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.219 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.219 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.219 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:20.957379+0000", 2024-09-11T09:01:46.220 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.6844e-05 2024-09-11T09:01:46.220 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.220 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.220 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.220 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:20.957414+0000", 2024-09-11T09:01:46.220 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.4484999999999999e-05 2024-09-11T09:01:46.220 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.220 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.220 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.220 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:20.957427+0000", 2024-09-11T09:01:46.220 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3044000000000001e-05 2024-09-11T09:01:46.220 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.220 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.220 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.221 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:20.957522+0000", 2024-09-11T09:01:46.221 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.5611000000000003e-05 2024-09-11T09:01:46.221 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.221 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.221 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.221 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:20.957555+0000", 2024-09-11T09:01:46.221 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.2802999999999997e-05 2024-09-11T09:01:46.221 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.221 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.221 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.221 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:20.957604+0000", 2024-09-11T09:01:46.221 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.8823000000000003e-05 2024-09-11T09:01:46.221 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.221 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.221 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.222 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.222 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.222 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "replica_scrub(pg: 6.9,from:0'0,to:0'0,epoch:438/290,start:6:90000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-11T09:01:46.222 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:01:23.331284+0000", 2024-09-11T09:01:46.222 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 22.816824572000002, 2024-09-11T09:01:46.222 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0002076, 2024-09-11T09:01:46.222 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.222 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:46.222 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.222 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.222 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.222 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:23.331284+0000", 2024-09-11T09:01:46.222 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.223 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.223 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.223 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.223 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:23.331284+0000", 2024-09-11T09:01:46.223 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.223 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.223 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.223 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.223 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:23.331283+0000", 2024-09-11T09:01:46.223 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:46.224 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.224 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.224 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.224 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:23.331295+0000", 2024-09-11T09:01:46.224 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1712999999999999e-05 2024-09-11T09:01:46.224 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.224 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.224 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.224 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:23.331324+0000", 2024-09-11T09:01:46.224 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.9125999999999999e-05 2024-09-11T09:01:46.224 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.224 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.224 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.225 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:23.331341+0000", 2024-09-11T09:01:46.225 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.7583000000000001e-05 2024-09-11T09:01:46.225 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.225 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.225 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.225 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:23.331461+0000", 2024-09-11T09:01:46.225 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000119926 2024-09-11T09:01:46.225 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.225 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.225 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.225 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:23.331469+0000", 2024-09-11T09:01:46.225 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 7.8019999999999992e-06 2024-09-11T09:01:46.225 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.225 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.226 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.226 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:23.331492+0000", 2024-09-11T09:01:46.226 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.2840000000000002e-05 2024-09-11T09:01:46.226 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.226 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.226 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.226 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.226 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.226 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "MOSDPGPull(4.7 e442/441 cost 1049576)", 2024-09-11T09:01:46.226 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:01:28.507184+0000", 2024-09-11T09:01:46.226 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 17.64092441, 2024-09-11T09:01:46.226 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00026655299999999999, 2024-09-11T09:01:46.227 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.227 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "reached pg", 2024-09-11T09:01:46.227 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.227 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.227 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.227 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:28.507184+0000", 2024-09-11T09:01:46.227 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.227 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.227 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.227 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.227 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:28.507184+0000", 2024-09-11T09:01:46.227 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.228 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.228 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.228 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.228 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:28.507184+0000", 2024-09-11T09:01:46.228 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:46.228 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.228 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.228 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.228 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:28.507193+0000", 2024-09-11T09:01:46.228 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.0299999999999999e-06 2024-09-11T09:01:46.228 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.228 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.229 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.229 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:28.507200+0000", 2024-09-11T09:01:46.229 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.742e-06 2024-09-11T09:01:46.229 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.229 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.229 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.229 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:28.507205+0000", 2024-09-11T09:01:46.229 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.2449999999999998e-06 2024-09-11T09:01:46.229 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.229 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.229 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.229 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:28.507246+0000", 2024-09-11T09:01:46.229 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.0599000000000002e-05 2024-09-11T09:01:46.229 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.230 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.230 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.230 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:28.507451+0000", 2024-09-11T09:01:46.230 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000205286 2024-09-11T09:01:46.230 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.230 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.230 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.230 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.230 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.230 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "MOSDScrubReserve(2.0 REQUEST e443)", 2024-09-11T09:01:46.230 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:01:34.143596+0000", 2024-09-11T09:01:46.230 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 12.004512870999999, 2024-09-11T09:01:46.230 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000214966, 2024-09-11T09:01:46.231 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.231 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:46.231 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.231 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.231 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.231 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.143596+0000", 2024-09-11T09:01:46.231 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.231 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.231 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.231 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.231 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.143596+0000", 2024-09-11T09:01:46.231 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.231 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.232 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.232 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.232 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.143594+0000", 2024-09-11T09:01:46.232 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999981 2024-09-11T09:01:46.232 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.232 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.232 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.232 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.143605+0000", 2024-09-11T09:01:46.232 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.0509999999999999e-05 2024-09-11T09:01:46.232 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.232 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.232 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.232 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.143631+0000", 2024-09-11T09:01:46.233 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.5993e-05 2024-09-11T09:01:46.233 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.233 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.233 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.233 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.143642+0000", 2024-09-11T09:01:46.233 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1066e-05 2024-09-11T09:01:46.233 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.233 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.233 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.233 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.143732+0000", 2024-09-11T09:01:46.233 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 9.0066000000000001e-05 2024-09-11T09:01:46.233 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.233 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.234 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.234 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.143759+0000", 2024-09-11T09:01:46.234 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7039e-05 2024-09-11T09:01:46.234 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.234 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.234 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.234 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.143811+0000", 2024-09-11T09:01:46.234 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1992e-05 2024-09-11T09:01:46.234 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.234 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.234 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.234 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.234 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.234 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "replica_scrub(pg: 2.0,from:0'0,to:318'6,epoch:443/441,start:2:00000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-11T09:01:46.235 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:01:34.144410+0000", 2024-09-11T09:01:46.235 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 12.003698922, 2024-09-11T09:01:46.235 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000231048, 2024-09-11T09:01:46.235 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.235 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:46.235 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.235 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.235 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.235 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.144410+0000", 2024-09-11T09:01:46.235 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.235 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.235 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.235 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.236 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.144410+0000", 2024-09-11T09:01:46.236 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.236 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.236 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.236 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.236 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.144409+0000", 2024-09-11T09:01:46.236 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999986 2024-09-11T09:01:46.236 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.236 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.236 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.236 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.144421+0000", 2024-09-11T09:01:46.236 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2e-05 2024-09-11T09:01:46.236 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.237 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.237 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.237 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.144451+0000", 2024-09-11T09:01:46.237 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.0624999999999999e-05 2024-09-11T09:01:46.237 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.237 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.237 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.237 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.144463+0000", 2024-09-11T09:01:46.237 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1834000000000001e-05 2024-09-11T09:01:46.237 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.237 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.237 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.238 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.144577+0000", 2024-09-11T09:01:46.238 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.000114318 2024-09-11T09:01:46.238 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.238 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.238 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.238 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.144596+0000", 2024-09-11T09:01:46.238 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.8363e-05 2024-09-11T09:01:46.238 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.238 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.238 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.238 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:34.144641+0000", 2024-09-11T09:01:46.238 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.5327999999999997e-05 2024-09-11T09:01:46.239 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.239 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.239 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.239 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.239 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.239 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "MOSDScrubReserve(3.16 REQUEST e444)", 2024-09-11T09:01:46.239 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:01:35.303671+0000", 2024-09-11T09:01:46.239 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 10.844437629, 2024-09-11T09:01:46.239 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0017005259999999999, 2024-09-11T09:01:46.239 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.239 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:46.239 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.239 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.240 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.240 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:35.303671+0000", 2024-09-11T09:01:46.240 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.240 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.240 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.240 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.240 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:35.303671+0000", 2024-09-11T09:01:46.240 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.240 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.240 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.240 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.240 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:35.303671+0000", 2024-09-11T09:01:46.241 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967296 2024-09-11T09:01:46.241 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.241 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.241 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.241 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:35.303674+0000", 2024-09-11T09:01:46.241 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7259999999999998e-06 2024-09-11T09:01:46.241 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.241 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.241 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.241 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:35.303687+0000", 2024-09-11T09:01:46.241 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.3013999999999999e-05 2024-09-11T09:01:46.241 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.241 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.242 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.242 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:35.303698+0000", 2024-09-11T09:01:46.242 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1092e-05 2024-09-11T09:01:46.242 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.242 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.242 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.242 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:35.303744+0000", 2024-09-11T09:01:46.242 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.6587e-05 2024-09-11T09:01:46.242 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.242 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.242 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op must wait for map", 2024-09-11T09:01:46.242 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:35.303751+0000", 2024-09-11T09:01:46.242 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 6.9410000000000001e-06 2024-09-11T09:01:46.243 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.243 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.243 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.243 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:35.305332+0000", 2024-09-11T09:01:46.243 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.0015806069999999999 2024-09-11T09:01:46.243 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.243 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.243 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.243 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:35.305345+0000", 2024-09-11T09:01:46.243 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.2634e-05 2024-09-11T09:01:46.243 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.244 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.244 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.244 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:35.305372+0000", 2024-09-11T09:01:46.244 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.7095000000000001e-05 2024-09-11T09:01:46.244 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.244 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.244 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.244 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.244 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.244 INFO:teuthology.orchestra.run.smithi145.stdout: "description": "MOSDScrubReserve(5.19 RELEASE e448)", 2024-09-11T09:01:46.244 INFO:teuthology.orchestra.run.smithi145.stdout: "initiated_at": "2024-09-11T09:01:45.250453+0000", 2024-09-11T09:01:46.244 INFO:teuthology.orchestra.run.smithi145.stdout: "age": 0.89765590299999998, 2024-09-11T09:01:46.245 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00070524599999999998, 2024-09-11T09:01:46.245 INFO:teuthology.orchestra.run.smithi145.stdout: "type_data": { 2024-09-11T09:01:46.245 INFO:teuthology.orchestra.run.smithi145.stdout: "flag_point": "started", 2024-09-11T09:01:46.245 INFO:teuthology.orchestra.run.smithi145.stdout: "events": [ 2024-09-11T09:01:46.245 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.245 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "initiated", 2024-09-11T09:01:46.245 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:45.250453+0000", 2024-09-11T09:01:46.245 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.245 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.245 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.245 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "throttled", 2024-09-11T09:01:46.245 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:45.250453+0000", 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "header_read", 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:45.250452+0000", 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4294967295.9999995 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "all_read", 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:45.250456+0000", 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.6770000000000001e-06 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "dispatched", 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:45.250468+0000", 2024-09-11T09:01:46.246 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.1999e-05 2024-09-11T09:01:46.247 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.247 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.247 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "queued_for_pg", 2024-09-11T09:01:46.247 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:45.250473+0000", 2024-09-11T09:01:46.247 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 5.1370000000000001e-06 2024-09-11T09:01:46.247 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.247 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.247 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.247 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:45.250507+0000", 2024-09-11T09:01:46.247 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 3.3719e-05 2024-09-11T09:01:46.247 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.247 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.247 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "op must wait for map", 2024-09-11T09:01:46.247 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:45.250512+0000", 2024-09-11T09:01:46.247 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 4.904e-06 2024-09-11T09:01:46.248 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.248 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.248 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "reached_pg", 2024-09-11T09:01:46.248 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:45.251124+0000", 2024-09-11T09:01:46.248 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 0.00061246900000000003 2024-09-11T09:01:46.248 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.248 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.248 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "started", 2024-09-11T09:01:46.248 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:45.251136+0000", 2024-09-11T09:01:46.248 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 1.189e-05 2024-09-11T09:01:46.248 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-09-11T09:01:46.248 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-09-11T09:01:46.248 INFO:teuthology.orchestra.run.smithi145.stdout: "event": "done", 2024-09-11T09:01:46.248 INFO:teuthology.orchestra.run.smithi145.stdout: "time": "2024-09-11T09:01:45.251158+0000", 2024-09-11T09:01:46.248 INFO:teuthology.orchestra.run.smithi145.stdout: "duration": 2.2011e-05 2024-09-11T09:01:46.249 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.249 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.249 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.249 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-09-11T09:01:46.249 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-09-11T09:01:46.249 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-09-11T09:01:46.262 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:46.719 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:46 smithi092 ceph-mon[29647]: pgmap v1057: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T09:01:46.719 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:46 smithi092 ceph-mon[29647]: 5.3 deep-scrub starts 2024-09-11T09:01:46.719 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:46 smithi092 ceph-mon[29647]: 5.3 deep-scrub ok 2024-09-11T09:01:46.719 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:46 smithi092 ceph-mon[29647]: 4.12 deep-scrub starts 2024-09-11T09:01:46.719 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:46 smithi092 ceph-mon[29647]: 4.12 deep-scrub ok 2024-09-11T09:01:46.720 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:46 smithi092 ceph-mon[41114]: pgmap v1057: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T09:01:46.720 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:46 smithi092 ceph-mon[41114]: 5.3 deep-scrub starts 2024-09-11T09:01:46.720 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:46 smithi092 ceph-mon[41114]: 5.3 deep-scrub ok 2024-09-11T09:01:46.720 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:46 smithi092 ceph-mon[41114]: 4.12 deep-scrub starts 2024-09-11T09:01:46.720 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:46 smithi092 ceph-mon[41114]: 4.12 deep-scrub ok 2024-09-11T09:01:46.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:46 smithi145 ceph-mon[33384]: pgmap v1057: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T09:01:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:46 smithi145 ceph-mon[33384]: 5.3 deep-scrub starts 2024-09-11T09:01:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:46 smithi145 ceph-mon[33384]: 5.3 deep-scrub ok 2024-09-11T09:01:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:46 smithi145 ceph-mon[33384]: 4.12 deep-scrub starts 2024-09-11T09:01:46.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:46 smithi145 ceph-mon[33384]: 4.12 deep-scrub ok 2024-09-11T09:01:47.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:47 smithi092 ceph-mon[29647]: 5.16 scrub starts 2024-09-11T09:01:47.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:47 smithi092 ceph-mon[29647]: 5.16 scrub ok 2024-09-11T09:01:47.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:47 smithi092 ceph-mon[29647]: 5.18 scrub starts 2024-09-11T09:01:47.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:47 smithi092 ceph-mon[29647]: 5.18 scrub ok 2024-09-11T09:01:47.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:47 smithi092 ceph-mon[41114]: 5.16 scrub starts 2024-09-11T09:01:47.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:47 smithi092 ceph-mon[41114]: 5.16 scrub ok 2024-09-11T09:01:47.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:47 smithi092 ceph-mon[41114]: 5.18 scrub starts 2024-09-11T09:01:47.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:47 smithi092 ceph-mon[41114]: 5.18 scrub ok 2024-09-11T09:01:47.911 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:01:47 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:01:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:47 smithi145 ceph-mon[33384]: 5.16 scrub starts 2024-09-11T09:01:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:47 smithi145 ceph-mon[33384]: 5.16 scrub ok 2024-09-11T09:01:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:47 smithi145 ceph-mon[33384]: 5.18 scrub starts 2024-09-11T09:01:47.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:47 smithi145 ceph-mon[33384]: 5.18 scrub ok 2024-09-11T09:01:48.569 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T09:01:48.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:48 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:48.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:48 smithi092 ceph-mon[29647]: pgmap v1058: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T09:01:48.898 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:48 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/263793784' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T09:01:48.898 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:48 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:48.898 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:48 smithi092 ceph-mon[41114]: pgmap v1058: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T09:01:48.898 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:48 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/263793784' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T09:01:48.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:48 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:48 smithi145 ceph-mon[33384]: pgmap v1058: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T09:01:48.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:48 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/263793784' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T09:01:49.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 4 1 2024-09-11T09:01:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:49.821 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:49 smithi092 ceph-mon[29647]: 3.5 scrub starts 2024-09-11T09:01:49.822 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:49 smithi092 ceph-mon[29647]: 3.5 scrub ok 2024-09-11T09:01:49.822 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:49 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/263793784' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T09:01:49.822 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:49 smithi092 ceph-mon[29647]: osdmap e449: 8 total, 8 up, 8 in 2024-09-11T09:01:49.822 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:49 smithi092 ceph-mon[41114]: 3.5 scrub starts 2024-09-11T09:01:49.822 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:49 smithi092 ceph-mon[41114]: 3.5 scrub ok 2024-09-11T09:01:49.822 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:49 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/263793784' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T09:01:49.822 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:49 smithi092 ceph-mon[41114]: osdmap e449: 8 total, 8 up, 8 in 2024-09-11T09:01:49.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:49 smithi145 ceph-mon[33384]: 3.5 scrub starts 2024-09-11T09:01:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:49 smithi145 ceph-mon[33384]: 3.5 scrub ok 2024-09-11T09:01:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:49 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/263793784' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T09:01:49.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:49 smithi145 ceph-mon[33384]: osdmap e449: 8 total, 8 up, 8 in 2024-09-11T09:01:50.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:50 smithi092 ceph-mon[41114]: pgmap v1060: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T09:01:50.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:50 smithi092 ceph-mon[41114]: 5.14 scrub starts 2024-09-11T09:01:50.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:50 smithi092 ceph-mon[41114]: 5.14 scrub ok 2024-09-11T09:01:50.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:50 smithi092 ceph-mon[29647]: pgmap v1060: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T09:01:50.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:50 smithi092 ceph-mon[29647]: 5.14 scrub starts 2024-09-11T09:01:50.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:50 smithi092 ceph-mon[29647]: 5.14 scrub ok 2024-09-11T09:01:50.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:50 smithi145 ceph-mon[33384]: pgmap v1060: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T09:01:50.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:50 smithi145 ceph-mon[33384]: 5.14 scrub starts 2024-09-11T09:01:50.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:50 smithi145 ceph-mon[33384]: 5.14 scrub ok 2024-09-11T09:01:51.581 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-11T09:01:51.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:51 smithi092 ceph-mon[29647]: 6.13 scrub starts 2024-09-11T09:01:51.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:51 smithi092 ceph-mon[29647]: 6.13 scrub ok 2024-09-11T09:01:51.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:51 smithi092 ceph-mon[29647]: 4.2 scrub starts 2024-09-11T09:01:51.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:51 smithi092 ceph-mon[29647]: 4.2 scrub ok 2024-09-11T09:01:51.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:51 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1631126596' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T09:01:51.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:51 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T09:01:51.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:51 smithi092 ceph-mon[41114]: 6.13 scrub starts 2024-09-11T09:01:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:51 smithi092 ceph-mon[41114]: 6.13 scrub ok 2024-09-11T09:01:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:51 smithi092 ceph-mon[41114]: 4.2 scrub starts 2024-09-11T09:01:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:51 smithi092 ceph-mon[41114]: 4.2 scrub ok 2024-09-11T09:01:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:51 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1631126596' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T09:01:51.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:51 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T09:01:51.817 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:01:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:01:51] "GET /metrics HTTP/1.1" 200 222514 "" "Prometheus/2.43.0" 2024-09-11T09:01:51.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:51 smithi145 ceph-mon[33384]: 6.13 scrub starts 2024-09-11T09:01:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:51 smithi145 ceph-mon[33384]: 6.13 scrub ok 2024-09-11T09:01:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:51 smithi145 ceph-mon[33384]: 4.2 scrub starts 2024-09-11T09:01:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:51 smithi145 ceph-mon[33384]: 4.2 scrub ok 2024-09-11T09:01:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:51 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1631126596' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T09:01:51.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:51 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T09:01:52.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 5 1 2024-09-11T09:01:52.613 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:52.733 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:52 smithi092 ceph-mon[29647]: pgmap v1061: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T09:01:52.733 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:52 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-11T09:01:52.734 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:52 smithi092 ceph-mon[29647]: osdmap e450: 8 total, 8 up, 8 in 2024-09-11T09:01:52.734 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:52 smithi092 ceph-mon[29647]: 5.17 scrub starts 2024-09-11T09:01:52.734 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:52 smithi092 ceph-mon[29647]: 5.17 scrub ok 2024-09-11T09:01:52.734 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:52 smithi092 ceph-mon[41114]: pgmap v1061: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T09:01:52.734 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:52 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-11T09:01:52.734 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:52 smithi092 ceph-mon[41114]: osdmap e450: 8 total, 8 up, 8 in 2024-09-11T09:01:52.734 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:52 smithi092 ceph-mon[41114]: 5.17 scrub starts 2024-09-11T09:01:52.734 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:52 smithi092 ceph-mon[41114]: 5.17 scrub ok 2024-09-11T09:01:52.911 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:01:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:01:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:01:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:52 smithi145 ceph-mon[33384]: pgmap v1061: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T09:01:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:52 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-11T09:01:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:52 smithi145 ceph-mon[33384]: osdmap e450: 8 total, 8 up, 8 in 2024-09-11T09:01:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:52 smithi145 ceph-mon[33384]: 5.17 scrub starts 2024-09-11T09:01:52.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:52 smithi145 ceph-mon[33384]: 5.17 scrub ok 2024-09-11T09:01:53.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:53 smithi092 ceph-mon[29647]: 1.9 scrub starts 2024-09-11T09:01:53.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:53 smithi092 ceph-mon[29647]: 1.9 scrub ok 2024-09-11T09:01:53.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:53 smithi092 ceph-mon[29647]: 5.2 scrub starts 2024-09-11T09:01:53.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:53 smithi092 ceph-mon[29647]: 5.2 scrub ok 2024-09-11T09:01:53.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:53 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T09:01:53.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:53 smithi092 ceph-mon[29647]: 4.a scrub starts 2024-09-11T09:01:53.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:53 smithi092 ceph-mon[29647]: 4.a scrub ok 2024-09-11T09:01:53.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:53 smithi092 ceph-mon[41114]: 1.9 scrub starts 2024-09-11T09:01:53.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:53 smithi092 ceph-mon[41114]: 1.9 scrub ok 2024-09-11T09:01:53.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:53 smithi092 ceph-mon[41114]: 5.2 scrub starts 2024-09-11T09:01:53.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:53 smithi092 ceph-mon[41114]: 5.2 scrub ok 2024-09-11T09:01:53.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:53 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T09:01:53.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:53 smithi092 ceph-mon[41114]: 4.a scrub starts 2024-09-11T09:01:53.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:53 smithi092 ceph-mon[41114]: 4.a scrub ok 2024-09-11T09:01:53.864 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:53 smithi145 ceph-mon[33384]: 1.9 scrub starts 2024-09-11T09:01:53.864 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:53 smithi145 ceph-mon[33384]: 1.9 scrub ok 2024-09-11T09:01:53.864 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:53 smithi145 ceph-mon[33384]: 5.2 scrub starts 2024-09-11T09:01:53.864 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:53 smithi145 ceph-mon[33384]: 5.2 scrub ok 2024-09-11T09:01:53.864 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:53 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T09:01:53.864 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:53 smithi145 ceph-mon[33384]: 4.a scrub starts 2024-09-11T09:01:53.865 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:53 smithi145 ceph-mon[33384]: 4.a scrub ok 2024-09-11T09:01:54.843 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:54 smithi092 ceph-mon[29647]: pgmap v1063: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:54.843 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:54 smithi092 ceph-mon[41114]: pgmap v1063: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:54.891 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:54 smithi145 ceph-mon[33384]: pgmap v1063: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:55.591 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.5 to 1 (10000) 2024-09-11T09:01:55.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:55 smithi092 ceph-mon[29647]: 3.19 scrub starts 2024-09-11T09:01:55.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:55 smithi092 ceph-mon[29647]: 3.19 scrub ok 2024-09-11T09:01:55.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:55 smithi092 ceph-mon[29647]: 5.1b scrub starts 2024-09-11T09:01:55.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:55 smithi092 ceph-mon[29647]: 5.1b scrub ok 2024-09-11T09:01:55.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:55 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3832723312' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T09:01:55.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:55 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T09:01:55.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:55 smithi092 ceph-mon[41114]: 3.19 scrub starts 2024-09-11T09:01:55.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:55 smithi092 ceph-mon[41114]: 3.19 scrub ok 2024-09-11T09:01:55.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:55 smithi092 ceph-mon[41114]: 5.1b scrub starts 2024-09-11T09:01:55.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:55 smithi092 ceph-mon[41114]: 5.1b scrub ok 2024-09-11T09:01:55.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:55 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3832723312' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T09:01:55.907 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:55 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T09:01:55.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:55 smithi145 ceph-mon[33384]: 3.19 scrub starts 2024-09-11T09:01:55.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:55 smithi145 ceph-mon[33384]: 3.19 scrub ok 2024-09-11T09:01:55.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:55 smithi145 ceph-mon[33384]: 5.1b scrub starts 2024-09-11T09:01:55.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:55 smithi145 ceph-mon[33384]: 5.1b scrub ok 2024-09-11T09:01:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:55 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3832723312' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T09:01:55.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:55 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T09:01:56.350 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 5 1 2024-09-11T09:01:56.894 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:56 smithi092 ceph-mon[29647]: 3.0 scrub starts 2024-09-11T09:01:56.894 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:56 smithi092 ceph-mon[29647]: 3.0 scrub ok 2024-09-11T09:01:56.895 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:56 smithi092 ceph-mon[29647]: pgmap v1064: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T09:01:56.895 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:56 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T09:01:56.895 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:56 smithi092 ceph-mon[29647]: osdmap e451: 8 total, 8 up, 8 in 2024-09-11T09:01:56.895 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:56 smithi092 ceph-mon[29647]: 4.0 scrub starts 2024-09-11T09:01:56.895 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:56 smithi092 ceph-mon[29647]: 4.0 scrub ok 2024-09-11T09:01:56.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:56 smithi092 ceph-mon[41114]: 3.0 scrub starts 2024-09-11T09:01:56.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:56 smithi092 ceph-mon[41114]: 3.0 scrub ok 2024-09-11T09:01:56.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:56 smithi092 ceph-mon[41114]: pgmap v1064: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T09:01:56.895 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:56 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T09:01:56.896 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:56 smithi092 ceph-mon[41114]: osdmap e451: 8 total, 8 up, 8 in 2024-09-11T09:01:56.896 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:56 smithi092 ceph-mon[41114]: 4.0 scrub starts 2024-09-11T09:01:56.896 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:56 smithi092 ceph-mon[41114]: 4.0 scrub ok 2024-09-11T09:01:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:56 smithi145 ceph-mon[33384]: 3.0 scrub starts 2024-09-11T09:01:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:56 smithi145 ceph-mon[33384]: 3.0 scrub ok 2024-09-11T09:01:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:56 smithi145 ceph-mon[33384]: pgmap v1064: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T09:01:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:56 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T09:01:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:56 smithi145 ceph-mon[33384]: osdmap e451: 8 total, 8 up, 8 in 2024-09-11T09:01:56.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:56 smithi145 ceph-mon[33384]: 4.0 scrub starts 2024-09-11T09:01:56.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:56 smithi145 ceph-mon[33384]: 4.0 scrub ok 2024-09-11T09:01:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:01:57.533 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:01:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:01:57.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:57 smithi092 ceph-mon[29647]: 3.c scrub starts 2024-09-11T09:01:57.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:57 smithi092 ceph-mon[29647]: 3.c scrub ok 2024-09-11T09:01:57.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:57 smithi092 ceph-mon[41114]: 3.c scrub starts 2024-09-11T09:01:57.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:57 smithi092 ceph-mon[41114]: 3.c scrub ok 2024-09-11T09:01:57.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:57 smithi145 ceph-mon[33384]: 3.c scrub starts 2024-09-11T09:01:57.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:57 smithi145 ceph-mon[33384]: 3.c scrub ok 2024-09-11T09:01:58.863 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:58.864 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[29647]: pgmap v1066: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:58.864 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[29647]: 5.10 scrub starts 2024-09-11T09:01:58.864 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[29647]: 5.10 scrub ok 2024-09-11T09:01:58.864 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:01:58.864 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:01:58.864 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[29647]: 5.11 deep-scrub starts 2024-09-11T09:01:58.864 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[29647]: 5.11 deep-scrub ok 2024-09-11T09:01:58.864 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[29647]: 4.1e scrub starts 2024-09-11T09:01:58.864 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[29647]: 4.1e scrub ok 2024-09-11T09:01:58.864 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:58.865 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[41114]: pgmap v1066: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:58.865 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[41114]: 5.10 scrub starts 2024-09-11T09:01:58.865 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[41114]: 5.10 scrub ok 2024-09-11T09:01:58.865 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:01:58.865 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:01:58.865 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[41114]: 5.11 deep-scrub starts 2024-09-11T09:01:58.865 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[41114]: 5.11 deep-scrub ok 2024-09-11T09:01:58.865 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[41114]: 4.1e scrub starts 2024-09-11T09:01:58.865 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:58 smithi092 ceph-mon[41114]: 4.1e scrub ok 2024-09-11T09:01:58.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:58 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:01:58.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:58 smithi145 ceph-mon[33384]: pgmap v1066: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:01:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:58 smithi145 ceph-mon[33384]: 5.10 scrub starts 2024-09-11T09:01:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:58 smithi145 ceph-mon[33384]: 5.10 scrub ok 2024-09-11T09:01:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:58 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:01:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:58 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:01:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:58 smithi145 ceph-mon[33384]: 5.11 deep-scrub starts 2024-09-11T09:01:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:58 smithi145 ceph-mon[33384]: 5.11 deep-scrub ok 2024-09-11T09:01:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:58 smithi145 ceph-mon[33384]: 4.1e scrub starts 2024-09-11T09:01:58.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:58 smithi145 ceph-mon[33384]: 4.1e scrub ok 2024-09-11T09:01:59.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:59 smithi092 ceph-mon[29647]: 5.15 scrub starts 2024-09-11T09:01:59.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:59 smithi092 ceph-mon[29647]: 5.15 scrub ok 2024-09-11T09:01:59.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:59 smithi092 ceph-mon[29647]: 3.11 scrub starts 2024-09-11T09:01:59.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:59 smithi092 ceph-mon[29647]: 3.11 scrub ok 2024-09-11T09:01:59.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:59 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:01:59.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:01:59 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:01:59.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:59 smithi092 ceph-mon[41114]: 5.15 scrub starts 2024-09-11T09:01:59.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:59 smithi092 ceph-mon[41114]: 5.15 scrub ok 2024-09-11T09:01:59.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:59 smithi092 ceph-mon[41114]: 3.11 scrub starts 2024-09-11T09:01:59.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:59 smithi092 ceph-mon[41114]: 3.11 scrub ok 2024-09-11T09:01:59.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:59 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:01:59.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:01:59 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:01:59.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:59 smithi145 ceph-mon[33384]: 5.15 scrub starts 2024-09-11T09:01:59.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:59 smithi145 ceph-mon[33384]: 5.15 scrub ok 2024-09-11T09:01:59.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:59 smithi145 ceph-mon[33384]: 3.11 scrub starts 2024-09-11T09:01:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:59 smithi145 ceph-mon[33384]: 3.11 scrub ok 2024-09-11T09:01:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:59 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:01:59.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:01:59 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:02:00.714 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-11T09:02:00.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:00 smithi145 ceph-mon[33384]: 1.a scrub starts 2024-09-11T09:02:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:00 smithi145 ceph-mon[33384]: 1.a scrub ok 2024-09-11T09:02:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:00 smithi145 ceph-mon[33384]: pgmap v1067: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:02:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T09:02:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T09:02:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:00 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:02:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:00 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/364142424' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T09:02:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:00 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T09:02:00.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:02:01.023 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[29647]: 1.a scrub starts 2024-09-11T09:02:01.023 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[29647]: 1.a scrub ok 2024-09-11T09:02:01.023 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[29647]: pgmap v1067: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:02:01.023 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T09:02:01.023 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T09:02:01.023 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:02:01.023 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/364142424' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T09:02:01.024 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T09:02:01.024 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:02:01.024 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[41114]: 1.a scrub starts 2024-09-11T09:02:01.024 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[41114]: 1.a scrub ok 2024-09-11T09:02:01.024 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[41114]: pgmap v1067: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:02:01.024 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T09:02:01.024 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T09:02:01.024 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:02:01.024 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/364142424' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T09:02:01.025 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T09:02:01.025 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:02:01.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 6 1 2024-09-11T09:02:01.713 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:02:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:02:01] "GET /metrics HTTP/1.1" 200 222520 "" "Prometheus/2.43.0" 2024-09-11T09:02:01.736 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:02.009 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:01 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-11T09:02:02.009 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:01 smithi092 ceph-mon[41114]: osdmap e452: 8 total, 8 up, 8 in 2024-09-11T09:02:02.009 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:01 smithi092 ceph-mon[41114]: 4.e scrub starts 2024-09-11T09:02:02.009 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:01 smithi092 ceph-mon[41114]: 4.e scrub ok 2024-09-11T09:02:02.009 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:01 smithi092 ceph-mon[41114]: 5.1a scrub starts 2024-09-11T09:02:02.009 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:01 smithi092 ceph-mon[41114]: 5.1a scrub ok 2024-09-11T09:02:02.010 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:01 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-11T09:02:02.010 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:01 smithi092 ceph-mon[29647]: osdmap e452: 8 total, 8 up, 8 in 2024-09-11T09:02:02.010 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:01 smithi092 ceph-mon[29647]: 4.e scrub starts 2024-09-11T09:02:02.010 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:01 smithi092 ceph-mon[29647]: 4.e scrub ok 2024-09-11T09:02:02.010 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:01 smithi092 ceph-mon[29647]: 5.1a scrub starts 2024-09-11T09:02:02.010 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:01 smithi092 ceph-mon[29647]: 5.1a scrub ok 2024-09-11T09:02:02.052 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:01 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-11T09:02:02.052 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:01 smithi145 ceph-mon[33384]: osdmap e452: 8 total, 8 up, 8 in 2024-09-11T09:02:02.052 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:01 smithi145 ceph-mon[33384]: 4.e scrub starts 2024-09-11T09:02:02.052 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:01 smithi145 ceph-mon[33384]: 4.e scrub ok 2024-09-11T09:02:02.053 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:01 smithi145 ceph-mon[33384]: 5.1a scrub starts 2024-09-11T09:02:02.053 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:01 smithi145 ceph-mon[33384]: 5.1a scrub ok 2024-09-11T09:02:03.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:02 smithi092 ceph-mon[29647]: pgmap v1069: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T09:02:03.056 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:02 smithi092 ceph-mon[41114]: pgmap v1069: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T09:02:03.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:02 smithi145 ceph-mon[33384]: pgmap v1069: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T09:02:03.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:02:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:02:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:02:04.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[29647]: 6.d scrub starts 2024-09-11T09:02:04.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[29647]: 6.d scrub ok 2024-09-11T09:02:04.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[29647]: 1.b scrub starts 2024-09-11T09:02:04.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[29647]: 1.b scrub ok 2024-09-11T09:02:04.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[29647]: 5.6 scrub starts 2024-09-11T09:02:04.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[29647]: 5.6 scrub ok 2024-09-11T09:02:04.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[29647]: 3.7 scrub starts 2024-09-11T09:02:04.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[29647]: 3.7 scrub ok 2024-09-11T09:02:04.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[41114]: 6.d scrub starts 2024-09-11T09:02:04.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[41114]: 6.d scrub ok 2024-09-11T09:02:04.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[41114]: 1.b scrub starts 2024-09-11T09:02:04.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[41114]: 1.b scrub ok 2024-09-11T09:02:04.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[41114]: 5.6 scrub starts 2024-09-11T09:02:04.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[41114]: 5.6 scrub ok 2024-09-11T09:02:04.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[41114]: 3.7 scrub starts 2024-09-11T09:02:04.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:03 smithi092 ceph-mon[41114]: 3.7 scrub ok 2024-09-11T09:02:04.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:03 smithi145 ceph-mon[33384]: 6.d scrub starts 2024-09-11T09:02:04.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:03 smithi145 ceph-mon[33384]: 6.d scrub ok 2024-09-11T09:02:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:03 smithi145 ceph-mon[33384]: 1.b scrub starts 2024-09-11T09:02:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:03 smithi145 ceph-mon[33384]: 1.b scrub ok 2024-09-11T09:02:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:03 smithi145 ceph-mon[33384]: 5.6 scrub starts 2024-09-11T09:02:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:03 smithi145 ceph-mon[33384]: 5.6 scrub ok 2024-09-11T09:02:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:03 smithi145 ceph-mon[33384]: 3.7 scrub starts 2024-09-11T09:02:04.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:03 smithi145 ceph-mon[33384]: 3.7 scrub ok 2024-09-11T09:02:04.729 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.6 to 1 (10000) 2024-09-11T09:02:05.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:04 smithi092 ceph-mon[29647]: pgmap v1070: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:02:05.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:04 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1260770306' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T09:02:05.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:04 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T09:02:05.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:04 smithi092 ceph-mon[41114]: pgmap v1070: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:02:05.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:04 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1260770306' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T09:02:05.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:04 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T09:02:05.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:04 smithi145 ceph-mon[33384]: pgmap v1070: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:02:05.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:04 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1260770306' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T09:02:05.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:04 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T09:02:05.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 6 1 2024-09-11T09:02:05.800 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:05.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[41114]: 1.12 scrub starts 2024-09-11T09:02:05.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[41114]: 1.12 scrub ok 2024-09-11T09:02:05.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[41114]: 3.4 scrub starts 2024-09-11T09:02:05.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[41114]: 3.4 scrub ok 2024-09-11T09:02:05.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:02:05.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:02:05.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T09:02:05.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[41114]: osdmap e453: 8 total, 8 up, 8 in 2024-09-11T09:02:05.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[41114]: 1.6 scrub starts 2024-09-11T09:02:05.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[41114]: 1.6 scrub ok 2024-09-11T09:02:05.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[29647]: 1.12 scrub starts 2024-09-11T09:02:05.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[29647]: 1.12 scrub ok 2024-09-11T09:02:05.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[29647]: 3.4 scrub starts 2024-09-11T09:02:05.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[29647]: 3.4 scrub ok 2024-09-11T09:02:05.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:02:05.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:02:05.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T09:02:05.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[29647]: osdmap e453: 8 total, 8 up, 8 in 2024-09-11T09:02:05.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[29647]: 1.6 scrub starts 2024-09-11T09:02:05.817 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:05 smithi092 ceph-mon[29647]: 1.6 scrub ok 2024-09-11T09:02:06.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:05 smithi145 ceph-mon[33384]: 1.12 scrub starts 2024-09-11T09:02:06.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:05 smithi145 ceph-mon[33384]: 1.12 scrub ok 2024-09-11T09:02:06.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:05 smithi145 ceph-mon[33384]: 3.4 scrub starts 2024-09-11T09:02:06.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:05 smithi145 ceph-mon[33384]: 3.4 scrub ok 2024-09-11T09:02:06.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:05 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:02:06.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:05 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:02:06.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:05 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T09:02:06.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:05 smithi145 ceph-mon[33384]: osdmap e453: 8 total, 8 up, 8 in 2024-09-11T09:02:06.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:05 smithi145 ceph-mon[33384]: 1.6 scrub starts 2024-09-11T09:02:06.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:05 smithi145 ceph-mon[33384]: 1.6 scrub ok 2024-09-11T09:02:07.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:06 smithi092 ceph-mon[29647]: pgmap v1072: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T09:02:07.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:06 smithi092 ceph-mon[29647]: 1.11 scrub starts 2024-09-11T09:02:07.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:06 smithi092 ceph-mon[29647]: 1.11 scrub ok 2024-09-11T09:02:07.055 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:06 smithi092 ceph-mon[41114]: pgmap v1072: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T09:02:07.055 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:06 smithi092 ceph-mon[41114]: 1.11 scrub starts 2024-09-11T09:02:07.055 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:06 smithi092 ceph-mon[41114]: 1.11 scrub ok 2024-09-11T09:02:07.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:06 smithi145 ceph-mon[33384]: pgmap v1072: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T09:02:07.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:06 smithi145 ceph-mon[33384]: 1.11 scrub starts 2024-09-11T09:02:07.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:06 smithi145 ceph-mon[33384]: 1.11 scrub ok 2024-09-11T09:02:07.742 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:02:07 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:02:08.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:07 smithi092 ceph-mon[29647]: 4.11 scrub starts 2024-09-11T09:02:08.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:07 smithi092 ceph-mon[29647]: 4.11 scrub ok 2024-09-11T09:02:08.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:07 smithi092 ceph-mon[29647]: 3.3 scrub starts 2024-09-11T09:02:08.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:07 smithi092 ceph-mon[29647]: 3.3 scrub ok 2024-09-11T09:02:08.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:07 smithi092 ceph-mon[41114]: 4.11 scrub starts 2024-09-11T09:02:08.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:07 smithi092 ceph-mon[41114]: 4.11 scrub ok 2024-09-11T09:02:08.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:07 smithi092 ceph-mon[41114]: 3.3 scrub starts 2024-09-11T09:02:08.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:07 smithi092 ceph-mon[41114]: 3.3 scrub ok 2024-09-11T09:02:08.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:07 smithi145 ceph-mon[33384]: 4.11 scrub starts 2024-09-11T09:02:08.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:07 smithi145 ceph-mon[33384]: 4.11 scrub ok 2024-09-11T09:02:08.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:07 smithi145 ceph-mon[33384]: 3.3 scrub starts 2024-09-11T09:02:08.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:07 smithi145 ceph-mon[33384]: 3.3 scrub ok 2024-09-11T09:02:08.747 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-11T09:02:09.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[29647]: 6.7 deep-scrub starts 2024-09-11T09:02:09.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[29647]: 6.7 deep-scrub ok 2024-09-11T09:02:09.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:09.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:02:09.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:02:09.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[29647]: pgmap v1073: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:02:09.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[29647]: osdmap e454: 8 total, 8 up, 8 in 2024-09-11T09:02:09.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[29647]: 4.1b scrub starts 2024-09-11T09:02:09.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3421843387' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T09:02:09.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[29647]: 4.1b scrub ok 2024-09-11T09:02:09.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[41114]: 6.7 deep-scrub starts 2024-09-11T09:02:09.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[41114]: 6.7 deep-scrub ok 2024-09-11T09:02:09.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:09.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:02:09.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:02:09.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[41114]: pgmap v1073: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:02:09.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[41114]: osdmap e454: 8 total, 8 up, 8 in 2024-09-11T09:02:09.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[41114]: 4.1b scrub starts 2024-09-11T09:02:09.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3421843387' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T09:02:09.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:08 smithi092 ceph-mon[41114]: 4.1b scrub ok 2024-09-11T09:02:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:08 smithi145 ceph-mon[33384]: 6.7 deep-scrub starts 2024-09-11T09:02:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:08 smithi145 ceph-mon[33384]: 6.7 deep-scrub ok 2024-09-11T09:02:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:08 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:08 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:02:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:08 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:02:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:08 smithi145 ceph-mon[33384]: pgmap v1073: 152 pgs: 152 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T09:02:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:08 smithi145 ceph-mon[33384]: osdmap e454: 8 total, 8 up, 8 in 2024-09-11T09:02:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:08 smithi145 ceph-mon[33384]: 4.1b scrub starts 2024-09-11T09:02:09.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:08 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3421843387' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T09:02:09.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:08 smithi145 ceph-mon[33384]: 4.1b scrub ok 2024-09-11T09:02:09.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd reweight 7 1 2024-09-11T09:02:09.801 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:09.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:09 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3421843387' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-11T09:02:09.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:09 smithi092 ceph-mon[29647]: osdmap e455: 8 total, 8 up, 8 in 2024-09-11T09:02:09.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:09 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3421843387' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-11T09:02:09.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:09 smithi092 ceph-mon[41114]: osdmap e455: 8 total, 8 up, 8 in 2024-09-11T09:02:10.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:09 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3421843387' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-11T09:02:10.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:09 smithi145 ceph-mon[33384]: osdmap e455: 8 total, 8 up, 8 in 2024-09-11T09:02:10.922 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:10 smithi092 ceph-mon[29647]: 4.4 scrub starts 2024-09-11T09:02:10.922 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:10 smithi092 ceph-mon[29647]: 4.4 scrub ok 2024-09-11T09:02:10.922 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:10 smithi092 ceph-mon[29647]: pgmap v1076: 152 pgs: 152 active+clean; 635 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:10.922 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:10 smithi092 ceph-mon[29647]: 5.d scrub starts 2024-09-11T09:02:10.922 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:10 smithi092 ceph-mon[29647]: 5.d scrub ok 2024-09-11T09:02:10.922 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:10 smithi092 ceph-mon[29647]: osdmap e456: 8 total, 8 up, 8 in 2024-09-11T09:02:10.923 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:10 smithi092 ceph-mon[41114]: 4.4 scrub starts 2024-09-11T09:02:10.923 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:10 smithi092 ceph-mon[41114]: 4.4 scrub ok 2024-09-11T09:02:10.923 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:10 smithi092 ceph-mon[41114]: pgmap v1076: 152 pgs: 152 active+clean; 635 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:10.923 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:10 smithi092 ceph-mon[41114]: 5.d scrub starts 2024-09-11T09:02:10.923 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:10 smithi092 ceph-mon[41114]: 5.d scrub ok 2024-09-11T09:02:10.923 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:10 smithi092 ceph-mon[41114]: osdmap e456: 8 total, 8 up, 8 in 2024-09-11T09:02:11.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:10 smithi145 ceph-mon[33384]: 4.4 scrub starts 2024-09-11T09:02:11.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:10 smithi145 ceph-mon[33384]: 4.4 scrub ok 2024-09-11T09:02:11.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:10 smithi145 ceph-mon[33384]: pgmap v1076: 152 pgs: 152 active+clean; 635 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:11.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:10 smithi145 ceph-mon[33384]: 5.d scrub starts 2024-09-11T09:02:11.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:10 smithi145 ceph-mon[33384]: 5.d scrub ok 2024-09-11T09:02:11.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:10 smithi145 ceph-mon[33384]: osdmap e456: 8 total, 8 up, 8 in 2024-09-11T09:02:11.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:02:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:02:11] "GET /metrics HTTP/1.1" 200 222520 "" "Prometheus/2.43.0" 2024-09-11T09:02:11.760 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.7 to 1 (10000) 2024-09-11T09:02:11.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:11 smithi145 ceph-mon[33384]: 3.1e deep-scrub starts 2024-09-11T09:02:11.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:11 smithi145 ceph-mon[33384]: 3.1e deep-scrub ok 2024-09-11T09:02:11.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:11 smithi145 ceph-mon[33384]: 3.a scrub starts 2024-09-11T09:02:11.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:11 smithi145 ceph-mon[33384]: 3.a scrub ok 2024-09-11T09:02:11.912 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:11 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/710562971' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T09:02:12.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:11 smithi092 ceph-mon[29647]: 3.1e deep-scrub starts 2024-09-11T09:02:12.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:11 smithi092 ceph-mon[29647]: 3.1e deep-scrub ok 2024-09-11T09:02:12.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:11 smithi092 ceph-mon[29647]: 3.a scrub starts 2024-09-11T09:02:12.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:11 smithi092 ceph-mon[29647]: 3.a scrub ok 2024-09-11T09:02:12.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:11 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/710562971' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T09:02:12.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:11 smithi092 ceph-mon[41114]: 3.1e deep-scrub starts 2024-09-11T09:02:12.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:11 smithi092 ceph-mon[41114]: 3.1e deep-scrub ok 2024-09-11T09:02:12.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:11 smithi092 ceph-mon[41114]: 3.a scrub starts 2024-09-11T09:02:12.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:11 smithi092 ceph-mon[41114]: 3.a scrub ok 2024-09-11T09:02:12.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:11 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/710562971' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T09:02:12.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 7 1 2024-09-11T09:02:12.810 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[29647]: 1.1 scrub starts 2024-09-11T09:02:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[29647]: 1.1 scrub ok 2024-09-11T09:02:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[29647]: pgmap v1078: 152 pgs: 152 active+clean; 635 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/710562971' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T09:02:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[29647]: osdmap e457: 8 total, 8 up, 8 in 2024-09-11T09:02:12.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[29647]: 6.1f scrub starts 2024-09-11T09:02:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[29647]: 6.1f scrub ok 2024-09-11T09:02:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[29647]: 3.13 scrub starts 2024-09-11T09:02:12.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[29647]: 3.13 scrub ok 2024-09-11T09:02:12.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[41114]: 1.1 scrub starts 2024-09-11T09:02:12.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[41114]: 1.1 scrub ok 2024-09-11T09:02:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[41114]: pgmap v1078: 152 pgs: 152 active+clean; 635 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/710562971' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T09:02:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[41114]: osdmap e457: 8 total, 8 up, 8 in 2024-09-11T09:02:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[41114]: 6.1f scrub starts 2024-09-11T09:02:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[41114]: 6.1f scrub ok 2024-09-11T09:02:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[41114]: 3.13 scrub starts 2024-09-11T09:02:12.817 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:12 smithi092 ceph-mon[41114]: 3.13 scrub ok 2024-09-11T09:02:13.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:02:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:02:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:02:13.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:12 smithi145 ceph-mon[33384]: 1.1 scrub starts 2024-09-11T09:02:13.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:12 smithi145 ceph-mon[33384]: 1.1 scrub ok 2024-09-11T09:02:13.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:12 smithi145 ceph-mon[33384]: pgmap v1078: 152 pgs: 152 active+clean; 635 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:13.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:12 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/710562971' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T09:02:13.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:12 smithi145 ceph-mon[33384]: osdmap e457: 8 total, 8 up, 8 in 2024-09-11T09:02:13.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:12 smithi145 ceph-mon[33384]: 6.1f scrub starts 2024-09-11T09:02:13.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:12 smithi145 ceph-mon[33384]: 6.1f scrub ok 2024-09-11T09:02:13.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:12 smithi145 ceph-mon[33384]: 3.13 scrub starts 2024-09-11T09:02:13.163 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:12 smithi145 ceph-mon[33384]: 3.13 scrub ok 2024-09-11T09:02:15.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[29647]: 3.d scrub starts 2024-09-11T09:02:15.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[29647]: 3.d scrub ok 2024-09-11T09:02:15.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[29647]: pgmap v1080: 152 pgs: 152 active+clean; 635 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 354 B/s rd, 0 op/s 2024-09-11T09:02:15.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[29647]: 4.1 scrub starts 2024-09-11T09:02:15.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[29647]: 4.1 scrub ok 2024-09-11T09:02:15.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[29647]: 4.5 scrub starts 2024-09-11T09:02:15.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[29647]: 4.5 scrub ok 2024-09-11T09:02:15.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[29647]: 4.17 scrub starts 2024-09-11T09:02:15.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[29647]: 4.17 scrub ok 2024-09-11T09:02:15.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[41114]: 3.d scrub starts 2024-09-11T09:02:15.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[41114]: 3.d scrub ok 2024-09-11T09:02:15.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[41114]: pgmap v1080: 152 pgs: 152 active+clean; 635 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 354 B/s rd, 0 op/s 2024-09-11T09:02:15.068 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[41114]: 4.1 scrub starts 2024-09-11T09:02:15.068 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[41114]: 4.1 scrub ok 2024-09-11T09:02:15.068 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[41114]: 4.5 scrub starts 2024-09-11T09:02:15.068 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[41114]: 4.5 scrub ok 2024-09-11T09:02:15.068 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[41114]: 4.17 scrub starts 2024-09-11T09:02:15.068 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:14 smithi092 ceph-mon[41114]: 4.17 scrub ok 2024-09-11T09:02:15.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:14 smithi145 ceph-mon[33384]: 3.d scrub starts 2024-09-11T09:02:15.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:14 smithi145 ceph-mon[33384]: 3.d scrub ok 2024-09-11T09:02:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:14 smithi145 ceph-mon[33384]: pgmap v1080: 152 pgs: 152 active+clean; 635 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 354 B/s rd, 0 op/s 2024-09-11T09:02:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:14 smithi145 ceph-mon[33384]: 4.1 scrub starts 2024-09-11T09:02:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:14 smithi145 ceph-mon[33384]: 4.1 scrub ok 2024-09-11T09:02:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:14 smithi145 ceph-mon[33384]: 4.5 scrub starts 2024-09-11T09:02:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:14 smithi145 ceph-mon[33384]: 4.5 scrub ok 2024-09-11T09:02:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:14 smithi145 ceph-mon[33384]: 4.17 scrub starts 2024-09-11T09:02:15.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:14 smithi145 ceph-mon[33384]: 4.17 scrub ok 2024-09-11T09:02:15.780 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-11T09:02:16.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:02:16.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:02:16.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[29647]: 3.e deep-scrub starts 2024-09-11T09:02:16.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[29647]: 3.e deep-scrub ok 2024-09-11T09:02:16.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[29647]: osdmap e458: 8 total, 8 up, 8 in 2024-09-11T09:02:16.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2684724233' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T09:02:16.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T09:02:16.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[29647]: 1.2 scrub starts 2024-09-11T09:02:16.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[29647]: 1.2 scrub ok 2024-09-11T09:02:16.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:02:16.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:02:16.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:02:16.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[41114]: 3.e deep-scrub starts 2024-09-11T09:02:16.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[41114]: 3.e deep-scrub ok 2024-09-11T09:02:16.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[41114]: osdmap e458: 8 total, 8 up, 8 in 2024-09-11T09:02:16.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2684724233' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T09:02:16.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T09:02:16.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[41114]: 1.2 scrub starts 2024-09-11T09:02:16.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[41114]: 1.2 scrub ok 2024-09-11T09:02:16.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:15 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:02:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:15 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:02:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:15 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:02:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:15 smithi145 ceph-mon[33384]: 3.e deep-scrub starts 2024-09-11T09:02:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:15 smithi145 ceph-mon[33384]: 3.e deep-scrub ok 2024-09-11T09:02:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:15 smithi145 ceph-mon[33384]: osdmap e458: 8 total, 8 up, 8 in 2024-09-11T09:02:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:15 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2684724233' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T09:02:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:15 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T09:02:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:15 smithi145 ceph-mon[33384]: 1.2 scrub starts 2024-09-11T09:02:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:15 smithi145 ceph-mon[33384]: 1.2 scrub ok 2024-09-11T09:02:16.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:15 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:02:16.584 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-11T09:02:16.585 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-11T09:02:16.585 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-11T09:02:16.585 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-11T09:02:16.585 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T09:02:16.585 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T09:02:16.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:16 smithi092 ceph-mon[29647]: pgmap v1082: 152 pgs: 152 active+clean; 635 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 341 B/s rd, 0 op/s 2024-09-11T09:02:16.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:16 smithi092 ceph-mon[29647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-11T09:02:16.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:16 smithi092 ceph-mon[29647]: osdmap e459: 8 total, 8 up, 8 in 2024-09-11T09:02:16.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:16 smithi092 ceph-mon[29647]: 6.16 scrub starts 2024-09-11T09:02:16.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:16 smithi092 ceph-mon[29647]: 6.16 scrub ok 2024-09-11T09:02:16.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:16 smithi092 ceph-mon[29647]: 1.13 scrub starts 2024-09-11T09:02:16.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:16 smithi092 ceph-mon[29647]: 1.13 scrub ok 2024-09-11T09:02:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:16 smithi092 ceph-mon[41114]: pgmap v1082: 152 pgs: 152 active+clean; 635 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 341 B/s rd, 0 op/s 2024-09-11T09:02:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:16 smithi092 ceph-mon[41114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-11T09:02:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:16 smithi092 ceph-mon[41114]: osdmap e459: 8 total, 8 up, 8 in 2024-09-11T09:02:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:16 smithi092 ceph-mon[41114]: 6.16 scrub starts 2024-09-11T09:02:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:16 smithi092 ceph-mon[41114]: 6.16 scrub ok 2024-09-11T09:02:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:16 smithi092 ceph-mon[41114]: 1.13 scrub starts 2024-09-11T09:02:16.816 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:16 smithi092 ceph-mon[41114]: 1.13 scrub ok 2024-09-11T09:02:16.839 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:17.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:16 smithi145 ceph-mon[33384]: pgmap v1082: 152 pgs: 152 active+clean; 635 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 341 B/s rd, 0 op/s 2024-09-11T09:02:17.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:16 smithi145 ceph-mon[33384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-11T09:02:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:16 smithi145 ceph-mon[33384]: osdmap e459: 8 total, 8 up, 8 in 2024-09-11T09:02:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:16 smithi145 ceph-mon[33384]: 6.16 scrub starts 2024-09-11T09:02:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:16 smithi145 ceph-mon[33384]: 6.16 scrub ok 2024-09-11T09:02:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:16 smithi145 ceph-mon[33384]: 1.13 scrub starts 2024-09-11T09:02:17.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:16 smithi145 ceph-mon[33384]: 1.13 scrub ok 2024-09-11T09:02:17.792 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:02:17 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:02:18.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:17 smithi092 ceph-mon[29647]: 5.b scrub starts 2024-09-11T09:02:18.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:17 smithi092 ceph-mon[29647]: 5.b scrub ok 2024-09-11T09:02:18.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:17 smithi092 ceph-mon[29647]: 3.9 deep-scrub starts 2024-09-11T09:02:18.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:17 smithi092 ceph-mon[29647]: 3.9 deep-scrub ok 2024-09-11T09:02:18.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:17 smithi092 ceph-mon[29647]: osdmap e460: 8 total, 8 up, 8 in 2024-09-11T09:02:18.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:17 smithi092 ceph-mon[41114]: 5.b scrub starts 2024-09-11T09:02:18.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:17 smithi092 ceph-mon[41114]: 5.b scrub ok 2024-09-11T09:02:18.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:17 smithi092 ceph-mon[41114]: 3.9 deep-scrub starts 2024-09-11T09:02:18.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:17 smithi092 ceph-mon[41114]: 3.9 deep-scrub ok 2024-09-11T09:02:18.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:17 smithi092 ceph-mon[41114]: osdmap e460: 8 total, 8 up, 8 in 2024-09-11T09:02:18.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:17 smithi145 ceph-mon[33384]: 5.b scrub starts 2024-09-11T09:02:18.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:17 smithi145 ceph-mon[33384]: 5.b scrub ok 2024-09-11T09:02:18.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:17 smithi145 ceph-mon[33384]: 3.9 deep-scrub starts 2024-09-11T09:02:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:17 smithi145 ceph-mon[33384]: 3.9 deep-scrub ok 2024-09-11T09:02:18.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:17 smithi145 ceph-mon[33384]: osdmap e460: 8 total, 8 up, 8 in 2024-09-11T09:02:18.849 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T09:02:18.849 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":460,"fsid":"211625d4-7018-11ef-bcea-c7b262605968","created":"2024-09-11T08:33:32.174444+0000","modified":"2024-09-11T09:02:16.783249+0000","last_up_change":"2024-09-11T08:38:23.097480+0000","last_in_change":"2024-09-11T08:58:33.533202+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:36:29.606460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":21,"pg_placement_num":21,"pg_placement_num_target":21,"pg_num_target":21,"pg_num_pending":21,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"434","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"383","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-11T08:38:37.674272+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-11T08:38:37.912906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T08:38:39.234173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T08:38:41.216386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T08:38:43.285497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"9d8a3811-e801-4b6a-9a63-83936283cb7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":439,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6803","nonce":2040613580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6805","nonce":2040613580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6809","nonce":2040613580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6807","nonce":2040613580}]},"public_addr":"172.21.15.92:6803/2040613580","cluster_addr":"172.21.15.92:6805/2040613580","heartbeat_back_addr":"172.21.15.92:6809/2040613580","heartbeat_front_addr":"172.21.15.92:6807/2040613580","state":["exists","up"]},{"osd":1,"uuid":"e7a18e45-0137-45ef-8b85-e458cf3737ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":441,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6811","nonce":544779328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6813","nonce":544779328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6817","nonce":544779328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6815","nonce":544779328}]},"public_addr":"172.21.15.92:6811/544779328","cluster_addr":"172.21.15.92:6813/544779328","heartbeat_back_addr":"172.21.15.92:6817/544779328","heartbeat_front_addr":"172.21.15.92:6815/544779328","state":["exists","up"]},{"osd":2,"uuid":"017282e2-8053-4fa7-affc-a05408fa481e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":445,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6819","nonce":933736777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6821","nonce":933736777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6825","nonce":933736777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6823","nonce":933736777}]},"public_addr":"172.21.15.92:6819/933736777","cluster_addr":"172.21.15.92:6821/933736777","heartbeat_back_addr":"172.21.15.92:6825/933736777","heartbeat_front_addr":"172.21.15.92:6823/933736777","state":["exists","up"]},{"osd":3,"uuid":"58e5a8a4-9e58-40ad-9b40-90cb5ff5414b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":441,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6827","nonce":878892459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6829","nonce":878892459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6833","nonce":878892459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6831","nonce":878892459}]},"public_addr":"172.21.15.92:6827/878892459","cluster_addr":"172.21.15.92:6829/878892459","heartbeat_back_addr":"172.21.15.92:6833/878892459","heartbeat_front_addr":"172.21.15.92:6831/878892459","state":["exists","up"]},{"osd":4,"uuid":"e144e102-963b-48b7-92a9-9391706164bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":441,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6801","nonce":2792751669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6803","nonce":2792751669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6807","nonce":2792751669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6805","nonce":2792751669}]},"public_addr":"172.21.15.145:6801/2792751669","cluster_addr":"172.21.15.145:6803/2792751669","heartbeat_back_addr":"172.21.15.145:6807/2792751669","heartbeat_front_addr":"172.21.15.145:6805/2792751669","state":["exists","up"]},{"osd":5,"uuid":"811b0dc9-b7a7-439a-86c1-5dc9d0d54e16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":441,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6809","nonce":3569637188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6811","nonce":3569637188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6815","nonce":3569637188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6813","nonce":3569637188}]},"public_addr":"172.21.15.145:6809/3569637188","cluster_addr":"172.21.15.145:6811/3569637188","heartbeat_back_addr":"172.21.15.145:6815/3569637188","heartbeat_front_addr":"172.21.15.145:6813/3569637188","state":["exists","up"]},{"osd":6,"uuid":"bd162f52-097c-4c5d-9d37-1289779a196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":455,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6817","nonce":2615047110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6819","nonce":2615047110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6823","nonce":2615047110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6821","nonce":2615047110}]},"public_addr":"172.21.15.145:6817/2615047110","cluster_addr":"172.21.15.145:6819/2615047110","heartbeat_back_addr":"172.21.15.145:6823/2615047110","heartbeat_front_addr":"172.21.15.145:6821/2615047110","state":["exists","up"]},{"osd":7,"uuid":"4e4021bf-78c7-4bc6-b222-3494f724ddec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":459,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6825","nonce":3195898530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6827","nonce":3195898530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6831","nonce":3195898530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6829","nonce":3195898530}]},"public_addr":"172.21.15.145:6825/3195898530","cluster_addr":"172.21.15.145:6827/3195898530","heartbeat_back_addr":"172.21.15.145:6831/3195898530","heartbeat_front_addr":"172.21.15.145:6829/3195898530","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:01:24.924394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:02:07.469142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:01:10.463612+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:02:04.351403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:01:39.385549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:00:56.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:01:01.770235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:02:14.659361+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1","osds":[6,4,0]}],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":3,"to":5}]},{"pgid":"4.13","mappings":[{"from":1,"to":5}]},{"pgid":"4.15","mappings":[{"from":7,"to":1}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":2}]},{"pgid":"5.18","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6800/1861854063":"2024-09-12T08:39:14.984442+0000","172.21.15.92:0/4113910808":"2024-09-12T08:33:54.195816+0000","172.21.15.92:0/3501902574":"2024-09-12T08:39:14.984442+0000","172.21.15.92:0/1583695466":"2024-09-12T08:33:54.195816+0000","172.21.15.92:0/736380628":"2024-09-12T08:33:54.195816+0000","172.21.15.92:6800/4060453592":"2024-09-12T08:33:54.195816+0000","172.21.15.92:0/2234574262":"2024-09-12T08:39:14.984442+0000","172.21.15.92:6800/2480574848":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/1540079149":"2024-09-12T08:39:14.984442+0000","172.21.15.92:0/2680744738":"2024-09-12T08:34:17.306389+0000","172.21.15.92:6801/4060453592":"2024-09-12T08:33:54.195816+0000","172.21.15.92:6801/1861854063":"2024-09-12T08:39:14.984442+0000","172.21.15.92:6801/2480574848":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/12697418":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/4280958069":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/2686720854":"2024-09-12T08:39:14.984442+0000","172.21.15.92:0/969221731":"2024-09-12T08:39:14.984442+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T09:02:19.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[29647]: 1.10 deep-scrub starts 2024-09-11T09:02:19.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[29647]: 1.10 deep-scrub ok 2024-09-11T09:02:19.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:19.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[29647]: pgmap v1085: 152 pgs: 17 peering, 135 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 13 KiB/s wr, 0 op/s; 9.1 KiB/s, 0 objects/s recovering 2024-09-11T09:02:19.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[29647]: 4.c scrub starts 2024-09-11T09:02:19.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[29647]: Health check failed: Reduced data availability: 10 pgs inactive, 16 pgs peering (PG_AVAILABILITY) 2024-09-11T09:02:19.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[29647]: 4.c scrub ok 2024-09-11T09:02:19.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[29647]: 3.15 scrub starts 2024-09-11T09:02:19.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[29647]: 3.15 scrub ok 2024-09-11T09:02:19.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[41114]: 1.10 deep-scrub starts 2024-09-11T09:02:19.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[41114]: 1.10 deep-scrub ok 2024-09-11T09:02:19.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:19.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[41114]: pgmap v1085: 152 pgs: 17 peering, 135 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 13 KiB/s wr, 0 op/s; 9.1 KiB/s, 0 objects/s recovering 2024-09-11T09:02:19.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[41114]: 4.c scrub starts 2024-09-11T09:02:19.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[41114]: Health check failed: Reduced data availability: 10 pgs inactive, 16 pgs peering (PG_AVAILABILITY) 2024-09-11T09:02:19.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[41114]: 4.c scrub ok 2024-09-11T09:02:19.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[41114]: 3.15 scrub starts 2024-09-11T09:02:19.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:18 smithi092 ceph-mon[41114]: 3.15 scrub ok 2024-09-11T09:02:19.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:18 smithi145 ceph-mon[33384]: 1.10 deep-scrub starts 2024-09-11T09:02:19.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:18 smithi145 ceph-mon[33384]: 1.10 deep-scrub ok 2024-09-11T09:02:19.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:18 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:19.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:18 smithi145 ceph-mon[33384]: pgmap v1085: 152 pgs: 17 peering, 135 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 13 KiB/s wr, 0 op/s; 9.1 KiB/s, 0 objects/s recovering 2024-09-11T09:02:19.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:18 smithi145 ceph-mon[33384]: 4.c scrub starts 2024-09-11T09:02:19.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:18 smithi145 ceph-mon[33384]: Health check failed: Reduced data availability: 10 pgs inactive, 16 pgs peering (PG_AVAILABILITY) 2024-09-11T09:02:19.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:18 smithi145 ceph-mon[33384]: 4.c scrub ok 2024-09-11T09:02:19.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:18 smithi145 ceph-mon[33384]: 3.15 scrub starts 2024-09-11T09:02:19.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:18 smithi145 ceph-mon[33384]: 3.15 scrub ok 2024-09-11T09:02:19.665 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T09:02:19.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T09:02:19.909 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:20.024 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:19 smithi092 ceph-mon[29647]: 5.1f scrub starts 2024-09-11T09:02:20.024 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:19 smithi092 ceph-mon[29647]: 5.1f scrub ok 2024-09-11T09:02:20.024 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:19 smithi092 ceph-mon[29647]: 6.17 scrub starts 2024-09-11T09:02:20.024 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:19 smithi092 ceph-mon[29647]: 6.17 scrub ok 2024-09-11T09:02:20.024 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:19 smithi092 ceph-mon[29647]: 3.6 scrub starts 2024-09-11T09:02:20.024 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:19 smithi092 ceph-mon[29647]: 3.6 scrub ok 2024-09-11T09:02:20.024 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:19 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/2085412230' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T09:02:20.025 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:19 smithi092 ceph-mon[41114]: 5.1f scrub starts 2024-09-11T09:02:20.025 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:19 smithi092 ceph-mon[41114]: 5.1f scrub ok 2024-09-11T09:02:20.025 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:19 smithi092 ceph-mon[41114]: 6.17 scrub starts 2024-09-11T09:02:20.025 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:19 smithi092 ceph-mon[41114]: 6.17 scrub ok 2024-09-11T09:02:20.025 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:19 smithi092 ceph-mon[41114]: 3.6 scrub starts 2024-09-11T09:02:20.025 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:19 smithi092 ceph-mon[41114]: 3.6 scrub ok 2024-09-11T09:02:20.025 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:19 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/2085412230' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T09:02:20.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:19 smithi145 ceph-mon[33384]: 5.1f scrub starts 2024-09-11T09:02:20.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:19 smithi145 ceph-mon[33384]: 5.1f scrub ok 2024-09-11T09:02:20.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:19 smithi145 ceph-mon[33384]: 6.17 scrub starts 2024-09-11T09:02:20.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:19 smithi145 ceph-mon[33384]: 6.17 scrub ok 2024-09-11T09:02:20.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:19 smithi145 ceph-mon[33384]: 3.6 scrub starts 2024-09-11T09:02:20.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:19 smithi145 ceph-mon[33384]: 3.6 scrub ok 2024-09-11T09:02:20.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:19 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/2085412230' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T09:02:21.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[29647]: 6.1d scrub starts 2024-09-11T09:02:21.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[29647]: 6.1d scrub ok 2024-09-11T09:02:21.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[29647]: pgmap v1086: 152 pgs: 17 peering, 135 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 12 KiB/s wr, 0 op/s; 8.7 KiB/s, 0 objects/s recovering 2024-09-11T09:02:21.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[29647]: 6.18 deep-scrub starts 2024-09-11T09:02:21.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[29647]: 6.18 deep-scrub ok 2024-09-11T09:02:21.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[29647]: 3.8 scrub starts 2024-09-11T09:02:21.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[29647]: 3.8 scrub ok 2024-09-11T09:02:21.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[29647]: 6.5 scrub starts 2024-09-11T09:02:21.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[29647]: 6.5 scrub ok 2024-09-11T09:02:21.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[41114]: 6.1d scrub starts 2024-09-11T09:02:21.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[41114]: 6.1d scrub ok 2024-09-11T09:02:21.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[41114]: pgmap v1086: 152 pgs: 17 peering, 135 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 12 KiB/s wr, 0 op/s; 8.7 KiB/s, 0 objects/s recovering 2024-09-11T09:02:21.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[41114]: 6.18 deep-scrub starts 2024-09-11T09:02:21.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[41114]: 6.18 deep-scrub ok 2024-09-11T09:02:21.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[41114]: 3.8 scrub starts 2024-09-11T09:02:21.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[41114]: 3.8 scrub ok 2024-09-11T09:02:21.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[41114]: 6.5 scrub starts 2024-09-11T09:02:21.067 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:20 smithi092 ceph-mon[41114]: 6.5 scrub ok 2024-09-11T09:02:21.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:20 smithi145 ceph-mon[33384]: 6.1d scrub starts 2024-09-11T09:02:21.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:20 smithi145 ceph-mon[33384]: 6.1d scrub ok 2024-09-11T09:02:21.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:20 smithi145 ceph-mon[33384]: pgmap v1086: 152 pgs: 17 peering, 135 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 12 KiB/s wr, 0 op/s; 8.7 KiB/s, 0 objects/s recovering 2024-09-11T09:02:21.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:20 smithi145 ceph-mon[33384]: 6.18 deep-scrub starts 2024-09-11T09:02:21.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:20 smithi145 ceph-mon[33384]: 6.18 deep-scrub ok 2024-09-11T09:02:21.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:20 smithi145 ceph-mon[33384]: 3.8 scrub starts 2024-09-11T09:02:21.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:20 smithi145 ceph-mon[33384]: 3.8 scrub ok 2024-09-11T09:02:21.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:20 smithi145 ceph-mon[33384]: 6.5 scrub starts 2024-09-11T09:02:21.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:20 smithi145 ceph-mon[33384]: 6.5 scrub ok 2024-09-11T09:02:21.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:02:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:02:21] "GET /metrics HTTP/1.1" 200 222524 "" "Prometheus/2.43.0" 2024-09-11T09:02:21.876 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:21 smithi092 ceph-mon[29647]: 5.8 scrub starts 2024-09-11T09:02:21.876 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:21 smithi092 ceph-mon[29647]: 5.8 scrub ok 2024-09-11T09:02:21.876 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:21 smithi092 ceph-mon[41114]: 5.8 scrub starts 2024-09-11T09:02:21.876 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:21 smithi092 ceph-mon[41114]: 5.8 scrub ok 2024-09-11T09:02:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T09:02:22.011 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":460,"fsid":"211625d4-7018-11ef-bcea-c7b262605968","created":"2024-09-11T08:33:32.174444+0000","modified":"2024-09-11T09:02:16.783249+0000","last_up_change":"2024-09-11T08:38:23.097480+0000","last_in_change":"2024-09-11T08:58:33.533202+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:36:29.606460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":21,"pg_placement_num":21,"pg_placement_num_target":21,"pg_num_target":21,"pg_num_pending":21,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"434","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"383","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-11T08:38:37.674272+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-11T08:38:37.912906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T08:38:39.234173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T08:38:41.216386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T08:38:43.285497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"9d8a3811-e801-4b6a-9a63-83936283cb7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":439,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6803","nonce":2040613580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6805","nonce":2040613580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6809","nonce":2040613580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2040613580},{"type":"v1","addr":"172.21.15.92:6807","nonce":2040613580}]},"public_addr":"172.21.15.92:6803/2040613580","cluster_addr":"172.21.15.92:6805/2040613580","heartbeat_back_addr":"172.21.15.92:6809/2040613580","heartbeat_front_addr":"172.21.15.92:6807/2040613580","state":["exists","up"]},{"osd":1,"uuid":"e7a18e45-0137-45ef-8b85-e458cf3737ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":441,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6811","nonce":544779328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6813","nonce":544779328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6817","nonce":544779328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":544779328},{"type":"v1","addr":"172.21.15.92:6815","nonce":544779328}]},"public_addr":"172.21.15.92:6811/544779328","cluster_addr":"172.21.15.92:6813/544779328","heartbeat_back_addr":"172.21.15.92:6817/544779328","heartbeat_front_addr":"172.21.15.92:6815/544779328","state":["exists","up"]},{"osd":2,"uuid":"017282e2-8053-4fa7-affc-a05408fa481e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":445,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6819","nonce":933736777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6821","nonce":933736777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6825","nonce":933736777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":933736777},{"type":"v1","addr":"172.21.15.92:6823","nonce":933736777}]},"public_addr":"172.21.15.92:6819/933736777","cluster_addr":"172.21.15.92:6821/933736777","heartbeat_back_addr":"172.21.15.92:6825/933736777","heartbeat_front_addr":"172.21.15.92:6823/933736777","state":["exists","up"]},{"osd":3,"uuid":"58e5a8a4-9e58-40ad-9b40-90cb5ff5414b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":441,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6827","nonce":878892459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6829","nonce":878892459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6833","nonce":878892459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":878892459},{"type":"v1","addr":"172.21.15.92:6831","nonce":878892459}]},"public_addr":"172.21.15.92:6827/878892459","cluster_addr":"172.21.15.92:6829/878892459","heartbeat_back_addr":"172.21.15.92:6833/878892459","heartbeat_front_addr":"172.21.15.92:6831/878892459","state":["exists","up"]},{"osd":4,"uuid":"e144e102-963b-48b7-92a9-9391706164bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":441,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6801","nonce":2792751669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6803","nonce":2792751669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6807","nonce":2792751669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":2792751669},{"type":"v1","addr":"172.21.15.145:6805","nonce":2792751669}]},"public_addr":"172.21.15.145:6801/2792751669","cluster_addr":"172.21.15.145:6803/2792751669","heartbeat_back_addr":"172.21.15.145:6807/2792751669","heartbeat_front_addr":"172.21.15.145:6805/2792751669","state":["exists","up"]},{"osd":5,"uuid":"811b0dc9-b7a7-439a-86c1-5dc9d0d54e16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":441,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6809","nonce":3569637188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6811","nonce":3569637188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6815","nonce":3569637188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3569637188},{"type":"v1","addr":"172.21.15.145:6813","nonce":3569637188}]},"public_addr":"172.21.15.145:6809/3569637188","cluster_addr":"172.21.15.145:6811/3569637188","heartbeat_back_addr":"172.21.15.145:6815/3569637188","heartbeat_front_addr":"172.21.15.145:6813/3569637188","state":["exists","up"]},{"osd":6,"uuid":"bd162f52-097c-4c5d-9d37-1289779a196b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":455,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6817","nonce":2615047110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6819","nonce":2615047110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6823","nonce":2615047110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2615047110},{"type":"v1","addr":"172.21.15.145:6821","nonce":2615047110}]},"public_addr":"172.21.15.145:6817/2615047110","cluster_addr":"172.21.15.145:6819/2615047110","heartbeat_back_addr":"172.21.15.145:6823/2615047110","heartbeat_front_addr":"172.21.15.145:6821/2615047110","state":["exists","up"]},{"osd":7,"uuid":"4e4021bf-78c7-4bc6-b222-3494f724ddec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":459,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6825","nonce":3195898530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6827","nonce":3195898530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6831","nonce":3195898530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3195898530},{"type":"v1","addr":"172.21.15.145:6829","nonce":3195898530}]},"public_addr":"172.21.15.145:6825/3195898530","cluster_addr":"172.21.15.145:6827/3195898530","heartbeat_back_addr":"172.21.15.145:6831/3195898530","heartbeat_front_addr":"172.21.15.145:6829/3195898530","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:01:24.924394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:02:07.469142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:01:10.463612+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:02:04.351403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:01:39.385549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:00:56.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:01:01.770235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:02:14.659361+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1","osds":[6,4,0]}],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"4.10","mappings":[{"from":3,"to":5}]},{"pgid":"4.13","mappings":[{"from":1,"to":5}]},{"pgid":"4.15","mappings":[{"from":7,"to":1}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":2}]},{"pgid":"5.18","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6800/1861854063":"2024-09-12T08:39:14.984442+0000","172.21.15.92:0/4113910808":"2024-09-12T08:33:54.195816+0000","172.21.15.92:0/3501902574":"2024-09-12T08:39:14.984442+0000","172.21.15.92:0/1583695466":"2024-09-12T08:33:54.195816+0000","172.21.15.92:0/736380628":"2024-09-12T08:33:54.195816+0000","172.21.15.92:6800/4060453592":"2024-09-12T08:33:54.195816+0000","172.21.15.92:0/2234574262":"2024-09-12T08:39:14.984442+0000","172.21.15.92:6800/2480574848":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/1540079149":"2024-09-12T08:39:14.984442+0000","172.21.15.92:0/2680744738":"2024-09-12T08:34:17.306389+0000","172.21.15.92:6801/4060453592":"2024-09-12T08:33:54.195816+0000","172.21.15.92:6801/1861854063":"2024-09-12T08:39:14.984442+0000","172.21.15.92:6801/2480574848":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/12697418":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/4280958069":"2024-09-12T08:34:17.306389+0000","172.21.15.92:0/2686720854":"2024-09-12T08:39:14.984442+0000","172.21.15.92:0/969221731":"2024-09-12T08:39:14.984442+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T09:02:22.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:21 smithi145 ceph-mon[33384]: 5.8 scrub starts 2024-09-11T09:02:22.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:21 smithi145 ceph-mon[33384]: 5.8 scrub ok 2024-09-11T09:02:22.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T09:02:22.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T09:02:22.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T09:02:22.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T09:02:22.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T09:02:22.686 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T09:02:22.686 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T09:02:22.687 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T09:02:23.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[29647]: pgmap v1087: 152 pgs: 17 peering, 135 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 11 KiB/s wr, 0 op/s; 7.7 KiB/s, 0 objects/s recovering 2024-09-11T09:02:23.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[29647]: 5.e scrub starts 2024-09-11T09:02:23.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[29647]: 5.e scrub ok 2024-09-11T09:02:23.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[29647]: 4.b scrub starts 2024-09-11T09:02:23.067 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[29647]: 4.b scrub ok 2024-09-11T09:02:23.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[29647]: 3.b deep-scrub starts 2024-09-11T09:02:23.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[29647]: 3.b deep-scrub ok 2024-09-11T09:02:23.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1137262255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T09:02:23.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[29647]: 3.f scrub starts 2024-09-11T09:02:23.068 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[29647]: 3.f scrub ok 2024-09-11T09:02:23.068 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[41114]: pgmap v1087: 152 pgs: 17 peering, 135 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 11 KiB/s wr, 0 op/s; 7.7 KiB/s, 0 objects/s recovering 2024-09-11T09:02:23.068 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[41114]: 5.e scrub starts 2024-09-11T09:02:23.068 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[41114]: 5.e scrub ok 2024-09-11T09:02:23.068 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[41114]: 4.b scrub starts 2024-09-11T09:02:23.068 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[41114]: 4.b scrub ok 2024-09-11T09:02:23.068 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[41114]: 3.b deep-scrub starts 2024-09-11T09:02:23.069 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[41114]: 3.b deep-scrub ok 2024-09-11T09:02:23.069 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1137262255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T09:02:23.069 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[41114]: 3.f scrub starts 2024-09-11T09:02:23.069 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:22 smithi092 ceph-mon[41114]: 3.f scrub ok 2024-09-11T09:02:23.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:02:22 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:02:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:02:23.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:22 smithi145 ceph-mon[33384]: pgmap v1087: 152 pgs: 17 peering, 135 active+clean; 661 KiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 11 KiB/s wr, 0 op/s; 7.7 KiB/s, 0 objects/s recovering 2024-09-11T09:02:23.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:22 smithi145 ceph-mon[33384]: 5.e scrub starts 2024-09-11T09:02:23.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:22 smithi145 ceph-mon[33384]: 5.e scrub ok 2024-09-11T09:02:23.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:22 smithi145 ceph-mon[33384]: 4.b scrub starts 2024-09-11T09:02:23.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:22 smithi145 ceph-mon[33384]: 4.b scrub ok 2024-09-11T09:02:23.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:22 smithi145 ceph-mon[33384]: 3.b deep-scrub starts 2024-09-11T09:02:23.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:22 smithi145 ceph-mon[33384]: 3.b deep-scrub ok 2024-09-11T09:02:23.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:22 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1137262255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T09:02:23.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:22 smithi145 ceph-mon[33384]: 3.f scrub starts 2024-09-11T09:02:23.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:22 smithi145 ceph-mon[33384]: 3.f scrub ok 2024-09-11T09:02:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:23.208 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:23.965 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:02:23.965 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:02:23.965 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:02:23.965 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:02:23.965 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[29647]: osdmap e461: 8 total, 8 up, 8 in 2024-09-11T09:02:23.965 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[29647]: 4.14 scrub starts 2024-09-11T09:02:23.965 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[29647]: 4.3 scrub starts 2024-09-11T09:02:23.965 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[29647]: 4.14 scrub ok 2024-09-11T09:02:23.966 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[29647]: 4.3 scrub ok 2024-09-11T09:02:23.966 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[29647]: 3.1 scrub starts 2024-09-11T09:02:23.966 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[29647]: 3.1 scrub ok 2024-09-11T09:02:23.966 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:02:23.966 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:02:23.966 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:02:23.966 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:02:23.966 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[41114]: osdmap e461: 8 total, 8 up, 8 in 2024-09-11T09:02:23.966 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[41114]: 4.14 scrub starts 2024-09-11T09:02:23.966 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[41114]: 4.3 scrub starts 2024-09-11T09:02:23.967 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[41114]: 4.14 scrub ok 2024-09-11T09:02:23.967 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[41114]: 4.3 scrub ok 2024-09-11T09:02:23.967 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[41114]: 3.1 scrub starts 2024-09-11T09:02:23.967 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:23 smithi092 ceph-mon[41114]: 3.1 scrub ok 2024-09-11T09:02:24.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:23 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:02:24.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:23 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:02:24.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:23 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:02:24.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:23 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:02:24.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:23 smithi145 ceph-mon[33384]: osdmap e461: 8 total, 8 up, 8 in 2024-09-11T09:02:24.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:23 smithi145 ceph-mon[33384]: 4.14 scrub starts 2024-09-11T09:02:24.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:23 smithi145 ceph-mon[33384]: 4.3 scrub starts 2024-09-11T09:02:24.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:23 smithi145 ceph-mon[33384]: 4.14 scrub ok 2024-09-11T09:02:24.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:23 smithi145 ceph-mon[33384]: 4.3 scrub ok 2024-09-11T09:02:24.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:23 smithi145 ceph-mon[33384]: 3.1 scrub starts 2024-09-11T09:02:24.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:23 smithi145 ceph-mon[33384]: 3.1 scrub ok 2024-09-11T09:02:24.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[29647]: pgmap v1089: 152 pgs: 152 active+clean; 661 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 9.6 KiB/s wr, 0 op/s; 6.7 KiB/s, 0 objects/s recovering 2024-09-11T09:02:24.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[29647]: 2.1 scrub starts 2024-09-11T09:02:24.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[29647]: 2.1 scrub ok 2024-09-11T09:02:24.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[29647]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs inactive, 17 pgs peering) 2024-09-11T09:02:24.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[29647]: Cluster is now healthy 2024-09-11T09:02:24.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[29647]: 4.18 scrub starts 2024-09-11T09:02:24.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[29647]: 4.18 scrub ok 2024-09-11T09:02:24.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[29647]: 1.9 scrub starts 2024-09-11T09:02:24.816 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[29647]: 1.9 scrub ok 2024-09-11T09:02:25.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[41114]: pgmap v1089: 152 pgs: 152 active+clean; 661 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 9.6 KiB/s wr, 0 op/s; 6.7 KiB/s, 0 objects/s recovering 2024-09-11T09:02:25.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[41114]: 2.1 scrub starts 2024-09-11T09:02:25.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[41114]: 2.1 scrub ok 2024-09-11T09:02:25.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[41114]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs inactive, 17 pgs peering) 2024-09-11T09:02:25.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[41114]: Cluster is now healthy 2024-09-11T09:02:25.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[41114]: 4.18 scrub starts 2024-09-11T09:02:25.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[41114]: 4.18 scrub ok 2024-09-11T09:02:25.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[41114]: 1.9 scrub starts 2024-09-11T09:02:25.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:24 smithi092 ceph-mon[41114]: 1.9 scrub ok 2024-09-11T09:02:25.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:24 smithi145 ceph-mon[33384]: pgmap v1089: 152 pgs: 152 active+clean; 661 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 9.6 KiB/s wr, 0 op/s; 6.7 KiB/s, 0 objects/s recovering 2024-09-11T09:02:25.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:24 smithi145 ceph-mon[33384]: 2.1 scrub starts 2024-09-11T09:02:25.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:24 smithi145 ceph-mon[33384]: 2.1 scrub ok 2024-09-11T09:02:25.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:24 smithi145 ceph-mon[33384]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs inactive, 17 pgs peering) 2024-09-11T09:02:25.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:24 smithi145 ceph-mon[33384]: Cluster is now healthy 2024-09-11T09:02:25.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:24 smithi145 ceph-mon[33384]: 4.18 scrub starts 2024-09-11T09:02:25.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:24 smithi145 ceph-mon[33384]: 4.18 scrub ok 2024-09-11T09:02:25.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:24 smithi145 ceph-mon[33384]: 1.9 scrub starts 2024-09-11T09:02:25.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:24 smithi145 ceph-mon[33384]: 1.9 scrub ok 2024-09-11T09:02:25.831 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[29647]: 4.8 scrub starts 2024-09-11T09:02:25.831 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[29647]: 4.8 scrub ok 2024-09-11T09:02:25.831 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[29647]: 6.e scrub starts 2024-09-11T09:02:25.831 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[29647]: 6.e scrub ok 2024-09-11T09:02:25.831 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[29647]: 4.1d scrub starts 2024-09-11T09:02:25.832 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[29647]: 4.1d scrub ok 2024-09-11T09:02:25.832 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[29647]: 1.d scrub starts 2024-09-11T09:02:25.832 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[29647]: 1.d scrub ok 2024-09-11T09:02:25.832 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[41114]: 4.8 scrub starts 2024-09-11T09:02:25.832 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[41114]: 4.8 scrub ok 2024-09-11T09:02:25.832 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[41114]: 6.e scrub starts 2024-09-11T09:02:25.832 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[41114]: 6.e scrub ok 2024-09-11T09:02:25.832 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[41114]: 4.1d scrub starts 2024-09-11T09:02:25.832 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[41114]: 4.1d scrub ok 2024-09-11T09:02:25.832 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[41114]: 1.d scrub starts 2024-09-11T09:02:25.832 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:25 smithi092 ceph-mon[41114]: 1.d scrub ok 2024-09-11T09:02:26.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:25 smithi145 ceph-mon[33384]: 4.8 scrub starts 2024-09-11T09:02:26.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:25 smithi145 ceph-mon[33384]: 4.8 scrub ok 2024-09-11T09:02:26.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:25 smithi145 ceph-mon[33384]: 6.e scrub starts 2024-09-11T09:02:26.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:25 smithi145 ceph-mon[33384]: 6.e scrub ok 2024-09-11T09:02:26.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:25 smithi145 ceph-mon[33384]: 4.1d scrub starts 2024-09-11T09:02:26.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:25 smithi145 ceph-mon[33384]: 4.1d scrub ok 2024-09-11T09:02:26.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:25 smithi145 ceph-mon[33384]: 1.d scrub starts 2024-09-11T09:02:26.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:25 smithi145 ceph-mon[33384]: 1.d scrub ok 2024-09-11T09:02:27.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:26 smithi092 ceph-mon[29647]: 4.f scrub starts 2024-09-11T09:02:27.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:26 smithi092 ceph-mon[29647]: 4.f scrub ok 2024-09-11T09:02:27.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:26 smithi092 ceph-mon[29647]: pgmap v1090: 152 pgs: 152 active+clean; 661 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 586 B/s rd, 0 op/s 2024-09-11T09:02:27.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:26 smithi092 ceph-mon[29647]: 4.6 scrub starts 2024-09-11T09:02:27.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:26 smithi092 ceph-mon[29647]: 4.6 scrub ok 2024-09-11T09:02:27.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:26 smithi092 ceph-mon[41114]: 4.f scrub starts 2024-09-11T09:02:27.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:26 smithi092 ceph-mon[41114]: 4.f scrub ok 2024-09-11T09:02:27.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:26 smithi092 ceph-mon[41114]: pgmap v1090: 152 pgs: 152 active+clean; 661 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 586 B/s rd, 0 op/s 2024-09-11T09:02:27.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:26 smithi092 ceph-mon[41114]: 4.6 scrub starts 2024-09-11T09:02:27.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:26 smithi092 ceph-mon[41114]: 4.6 scrub ok 2024-09-11T09:02:27.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:26 smithi145 ceph-mon[33384]: 4.f scrub starts 2024-09-11T09:02:27.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:26 smithi145 ceph-mon[33384]: 4.f scrub ok 2024-09-11T09:02:27.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:26 smithi145 ceph-mon[33384]: pgmap v1090: 152 pgs: 152 active+clean; 661 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 586 B/s rd, 0 op/s 2024-09-11T09:02:27.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:26 smithi145 ceph-mon[33384]: 4.6 scrub starts 2024-09-11T09:02:27.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:26 smithi145 ceph-mon[33384]: 4.6 scrub ok 2024-09-11T09:02:27.825 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:02:27 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:02:28.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:27 smithi145 ceph-mon[33384]: 1.14 deep-scrub starts 2024-09-11T09:02:28.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:27 smithi145 ceph-mon[33384]: 1.14 deep-scrub ok 2024-09-11T09:02:28.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:27 smithi145 ceph-mon[33384]: 6.6 scrub starts 2024-09-11T09:02:28.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:27 smithi145 ceph-mon[33384]: 6.6 scrub ok 2024-09-11T09:02:28.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:27 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:28.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:27 smithi145 ceph-mon[33384]: pgmap v1091: 152 pgs: 152 active+clean; 608 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:28.218 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:27 smithi092 ceph-mon[29647]: 1.14 deep-scrub starts 2024-09-11T09:02:28.218 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:27 smithi092 ceph-mon[29647]: 1.14 deep-scrub ok 2024-09-11T09:02:28.218 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:27 smithi092 ceph-mon[29647]: 6.6 scrub starts 2024-09-11T09:02:28.218 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:27 smithi092 ceph-mon[29647]: 6.6 scrub ok 2024-09-11T09:02:28.218 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:27 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:28.218 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:27 smithi092 ceph-mon[29647]: pgmap v1091: 152 pgs: 152 active+clean; 608 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:28.219 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:27 smithi092 ceph-mon[41114]: 1.14 deep-scrub starts 2024-09-11T09:02:28.219 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:27 smithi092 ceph-mon[41114]: 1.14 deep-scrub ok 2024-09-11T09:02:28.219 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:27 smithi092 ceph-mon[41114]: 6.6 scrub starts 2024-09-11T09:02:28.219 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:27 smithi092 ceph-mon[41114]: 6.6 scrub ok 2024-09-11T09:02:28.219 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:27 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:28.219 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:27 smithi092 ceph-mon[41114]: pgmap v1091: 152 pgs: 152 active+clean; 608 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:29.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:28 smithi092 ceph-mon[29647]: 1.6 scrub starts 2024-09-11T09:02:29.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:28 smithi092 ceph-mon[29647]: 5.1c scrub starts 2024-09-11T09:02:29.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:28 smithi092 ceph-mon[29647]: 1.6 scrub ok 2024-09-11T09:02:29.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:28 smithi092 ceph-mon[29647]: 5.1c scrub ok 2024-09-11T09:02:29.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:28 smithi092 ceph-mon[29647]: 1.2 deep-scrub starts 2024-09-11T09:02:29.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:28 smithi092 ceph-mon[29647]: 1.2 deep-scrub ok 2024-09-11T09:02:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:28 smithi092 ceph-mon[41114]: 1.6 scrub starts 2024-09-11T09:02:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:28 smithi092 ceph-mon[41114]: 5.1c scrub starts 2024-09-11T09:02:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:28 smithi092 ceph-mon[41114]: 1.6 scrub ok 2024-09-11T09:02:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:28 smithi092 ceph-mon[41114]: 5.1c scrub ok 2024-09-11T09:02:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:28 smithi092 ceph-mon[41114]: 1.2 deep-scrub starts 2024-09-11T09:02:29.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:28 smithi092 ceph-mon[41114]: 1.2 deep-scrub ok 2024-09-11T09:02:29.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:28 smithi145 ceph-mon[33384]: 1.6 scrub starts 2024-09-11T09:02:29.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:28 smithi145 ceph-mon[33384]: 5.1c scrub starts 2024-09-11T09:02:29.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:28 smithi145 ceph-mon[33384]: 1.6 scrub ok 2024-09-11T09:02:29.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:28 smithi145 ceph-mon[33384]: 5.1c scrub ok 2024-09-11T09:02:29.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:28 smithi145 ceph-mon[33384]: 1.2 deep-scrub starts 2024-09-11T09:02:29.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:28 smithi145 ceph-mon[33384]: 1.2 deep-scrub ok 2024-09-11T09:02:30.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:29 smithi145 ceph-mon[33384]: 2.2 scrub starts 2024-09-11T09:02:30.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:29 smithi145 ceph-mon[33384]: 2.2 scrub ok 2024-09-11T09:02:30.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:29 smithi145 ceph-mon[33384]: 5.c deep-scrub starts 2024-09-11T09:02:30.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:29 smithi145 ceph-mon[33384]: 5.c deep-scrub ok 2024-09-11T09:02:30.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:29 smithi145 ceph-mon[33384]: 5.13 deep-scrub starts 2024-09-11T09:02:30.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:29 smithi145 ceph-mon[33384]: 5.13 deep-scrub ok 2024-09-11T09:02:30.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:29 smithi145 ceph-mon[33384]: 6.15 deep-scrub starts 2024-09-11T09:02:30.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:29 smithi145 ceph-mon[33384]: 6.15 deep-scrub ok 2024-09-11T09:02:30.082 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:29 smithi145 ceph-mon[33384]: pgmap v1092: 152 pgs: 152 active+clean; 608 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:30.123 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[29647]: 2.2 scrub starts 2024-09-11T09:02:30.123 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[29647]: 2.2 scrub ok 2024-09-11T09:02:30.123 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[29647]: 5.c deep-scrub starts 2024-09-11T09:02:30.123 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[29647]: 5.c deep-scrub ok 2024-09-11T09:02:30.123 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[29647]: 5.13 deep-scrub starts 2024-09-11T09:02:30.123 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[29647]: 5.13 deep-scrub ok 2024-09-11T09:02:30.123 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[29647]: 6.15 deep-scrub starts 2024-09-11T09:02:30.124 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[29647]: 6.15 deep-scrub ok 2024-09-11T09:02:30.124 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[29647]: pgmap v1092: 152 pgs: 152 active+clean; 608 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:30.124 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[41114]: 2.2 scrub starts 2024-09-11T09:02:30.124 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[41114]: 2.2 scrub ok 2024-09-11T09:02:30.124 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[41114]: 5.c deep-scrub starts 2024-09-11T09:02:30.124 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[41114]: 5.c deep-scrub ok 2024-09-11T09:02:30.124 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[41114]: 5.13 deep-scrub starts 2024-09-11T09:02:30.124 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[41114]: 5.13 deep-scrub ok 2024-09-11T09:02:30.124 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[41114]: 6.15 deep-scrub starts 2024-09-11T09:02:30.124 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[41114]: 6.15 deep-scrub ok 2024-09-11T09:02:30.125 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:29 smithi092 ceph-mon[41114]: pgmap v1092: 152 pgs: 152 active+clean; 608 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:31.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:30 smithi092 ceph-mon[29647]: 1.f scrub starts 2024-09-11T09:02:31.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:30 smithi092 ceph-mon[29647]: 1.f scrub ok 2024-09-11T09:02:31.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:30 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:02:31.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:30 smithi092 ceph-mon[41114]: 1.f scrub starts 2024-09-11T09:02:31.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:30 smithi092 ceph-mon[41114]: 1.f scrub ok 2024-09-11T09:02:31.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:30 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:02:31.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:30 smithi145 ceph-mon[33384]: 1.f scrub starts 2024-09-11T09:02:31.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:30 smithi145 ceph-mon[33384]: 1.f scrub ok 2024-09-11T09:02:31.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:30 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:02:31.225 INFO:teuthology.orchestra.run.smithi092.stdout:193273528617 2024-09-11T09:02:31.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T09:02:31.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:02:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:02:31] "GET /metrics HTTP/1.1" 200 222527 "" "Prometheus/2.43.0" 2024-09-11T09:02:32.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:31 smithi092 ceph-mon[41114]: 5.1 scrub starts 2024-09-11T09:02:32.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:31 smithi092 ceph-mon[41114]: 5.1 scrub ok 2024-09-11T09:02:32.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:31 smithi092 ceph-mon[41114]: 1.7 scrub starts 2024-09-11T09:02:32.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:31 smithi092 ceph-mon[41114]: 1.7 scrub ok 2024-09-11T09:02:32.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:31 smithi092 ceph-mon[41114]: pgmap v1093: 152 pgs: 152 active+clean; 608 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:32.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:31 smithi092 ceph-mon[29647]: 5.1 scrub starts 2024-09-11T09:02:32.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:31 smithi092 ceph-mon[29647]: 5.1 scrub ok 2024-09-11T09:02:32.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:31 smithi092 ceph-mon[29647]: 1.7 scrub starts 2024-09-11T09:02:32.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:31 smithi092 ceph-mon[29647]: 1.7 scrub ok 2024-09-11T09:02:32.066 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:31 smithi092 ceph-mon[29647]: pgmap v1093: 152 pgs: 152 active+clean; 608 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:32.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:31 smithi145 ceph-mon[33384]: 5.1 scrub starts 2024-09-11T09:02:32.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:31 smithi145 ceph-mon[33384]: 5.1 scrub ok 2024-09-11T09:02:32.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:31 smithi145 ceph-mon[33384]: 1.7 scrub starts 2024-09-11T09:02:32.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:31 smithi145 ceph-mon[33384]: 1.7 scrub ok 2024-09-11T09:02:32.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:31 smithi145 ceph-mon[33384]: pgmap v1093: 152 pgs: 152 active+clean; 608 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:33.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:32 smithi092 ceph-mon[29647]: 5.1e scrub starts 2024-09-11T09:02:33.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:32 smithi092 ceph-mon[29647]: 5.1e scrub ok 2024-09-11T09:02:33.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:32 smithi092 ceph-mon[41114]: 5.1e scrub starts 2024-09-11T09:02:33.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:32 smithi092 ceph-mon[41114]: 5.1e scrub ok 2024-09-11T09:02:33.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:32 smithi145 ceph-mon[33384]: 5.1e scrub starts 2024-09-11T09:02:33.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:32 smithi145 ceph-mon[33384]: 5.1e scrub ok 2024-09-11T09:02:33.162 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:02:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:02:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:02:33.547 INFO:teuthology.orchestra.run.smithi092.stdout:55834575169 2024-09-11T09:02:33.548 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T09:02:34.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:33 smithi145 ceph-mon[33384]: 1.3 scrub starts 2024-09-11T09:02:34.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:33 smithi145 ceph-mon[33384]: 1.3 scrub ok 2024-09-11T09:02:34.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:33 smithi145 ceph-mon[33384]: 6.b scrub starts 2024-09-11T09:02:34.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:33 smithi145 ceph-mon[33384]: 6.b scrub ok 2024-09-11T09:02:34.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:33 smithi145 ceph-mon[33384]: 1.0 scrub starts 2024-09-11T09:02:34.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:33 smithi145 ceph-mon[33384]: 1.0 scrub ok 2024-09-11T09:02:34.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:33 smithi145 ceph-mon[33384]: pgmap v1094: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:34.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:33 smithi092 ceph-mon[41114]: 1.3 scrub starts 2024-09-11T09:02:34.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:33 smithi092 ceph-mon[41114]: 1.3 scrub ok 2024-09-11T09:02:34.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:33 smithi092 ceph-mon[41114]: 6.b scrub starts 2024-09-11T09:02:34.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:33 smithi092 ceph-mon[41114]: 6.b scrub ok 2024-09-11T09:02:34.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:33 smithi092 ceph-mon[41114]: 1.0 scrub starts 2024-09-11T09:02:34.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:33 smithi092 ceph-mon[41114]: 1.0 scrub ok 2024-09-11T09:02:34.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:33 smithi092 ceph-mon[41114]: pgmap v1094: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:33 smithi092 ceph-mon[29647]: 1.3 scrub starts 2024-09-11T09:02:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:33 smithi092 ceph-mon[29647]: 1.3 scrub ok 2024-09-11T09:02:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:33 smithi092 ceph-mon[29647]: 6.b scrub starts 2024-09-11T09:02:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:33 smithi092 ceph-mon[29647]: 6.b scrub ok 2024-09-11T09:02:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:33 smithi092 ceph-mon[29647]: 1.0 scrub starts 2024-09-11T09:02:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:33 smithi092 ceph-mon[29647]: 1.0 scrub ok 2024-09-11T09:02:34.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:33 smithi092 ceph-mon[29647]: pgmap v1094: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:34.482 INFO:teuthology.orchestra.run.smithi092.stdout:223338299685 2024-09-11T09:02:34.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T09:02:35.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:34 smithi145 ceph-mon[33384]: 6.1c scrub starts 2024-09-11T09:02:35.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:34 smithi145 ceph-mon[33384]: 6.1c scrub ok 2024-09-11T09:02:35.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:34 smithi092 ceph-mon[29647]: 6.1c scrub starts 2024-09-11T09:02:35.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:34 smithi092 ceph-mon[29647]: 6.1c scrub ok 2024-09-11T09:02:35.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:34 smithi092 ceph-mon[41114]: 6.1c scrub starts 2024-09-11T09:02:35.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:34 smithi092 ceph-mon[41114]: 6.1c scrub ok 2024-09-11T09:02:36.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:35 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:02:36.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:35 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:02:36.065 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:35 smithi092 ceph-mon[29647]: pgmap v1095: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:36.065 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:35 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:02:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:35 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:02:36.066 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:35 smithi092 ceph-mon[41114]: pgmap v1095: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:36.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:35 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:02:36.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:35 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:02:36.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:35 smithi145 ceph-mon[33384]: pgmap v1095: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail 2024-09-11T09:02:36.993 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:36 smithi092 ceph-mon[41114]: osdmap e462: 8 total, 8 up, 8 in 2024-09-11T09:02:36.993 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:36 smithi092 ceph-mon[41114]: 6.c deep-scrub starts 2024-09-11T09:02:36.993 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:36 smithi092 ceph-mon[41114]: 6.c deep-scrub ok 2024-09-11T09:02:36.993 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:36 smithi092 ceph-mon[29647]: osdmap e462: 8 total, 8 up, 8 in 2024-09-11T09:02:36.993 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:36 smithi092 ceph-mon[29647]: 6.c deep-scrub starts 2024-09-11T09:02:36.993 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:36 smithi092 ceph-mon[29647]: 6.c deep-scrub ok 2024-09-11T09:02:37.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:36 smithi145 ceph-mon[33384]: osdmap e462: 8 total, 8 up, 8 in 2024-09-11T09:02:37.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:36 smithi145 ceph-mon[33384]: 6.c deep-scrub starts 2024-09-11T09:02:37.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:36 smithi145 ceph-mon[33384]: 6.c deep-scrub ok 2024-09-11T09:02:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:37.851 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:02:37 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:02:38.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:37 smithi145 ceph-mon[33384]: 4.16 deep-scrub starts 2024-09-11T09:02:38.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:37 smithi145 ceph-mon[33384]: 4.16 deep-scrub ok 2024-09-11T09:02:38.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:37 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:38.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:37 smithi145 ceph-mon[33384]: pgmap v1097: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-11T09:02:38.197 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:37 smithi092 ceph-mon[29647]: 4.16 deep-scrub starts 2024-09-11T09:02:38.197 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:37 smithi092 ceph-mon[29647]: 4.16 deep-scrub ok 2024-09-11T09:02:38.197 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:37 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:38.197 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:37 smithi092 ceph-mon[29647]: pgmap v1097: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-11T09:02:38.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:37 smithi092 ceph-mon[41114]: 4.16 deep-scrub starts 2024-09-11T09:02:38.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:37 smithi092 ceph-mon[41114]: 4.16 deep-scrub ok 2024-09-11T09:02:38.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:37 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:38.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:37 smithi092 ceph-mon[41114]: pgmap v1097: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-11T09:02:38.221 INFO:teuthology.orchestra.run.smithi092.stdout:163208757550 2024-09-11T09:02:38.222 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T09:02:39.111 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:38 smithi092 ceph-mon[41114]: 5.7 scrub starts 2024-09-11T09:02:39.112 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:38 smithi092 ceph-mon[41114]: 5.7 scrub ok 2024-09-11T09:02:39.112 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:38 smithi092 ceph-mon[41114]: 3.2 scrub starts 2024-09-11T09:02:39.112 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:38 smithi092 ceph-mon[41114]: 3.2 scrub ok 2024-09-11T09:02:39.112 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:38 smithi092 ceph-mon[29647]: 5.7 scrub starts 2024-09-11T09:02:39.112 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:38 smithi092 ceph-mon[29647]: 5.7 scrub ok 2024-09-11T09:02:39.112 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:38 smithi092 ceph-mon[29647]: 3.2 scrub starts 2024-09-11T09:02:39.112 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:38 smithi092 ceph-mon[29647]: 3.2 scrub ok 2024-09-11T09:02:39.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:38 smithi145 ceph-mon[33384]: 5.7 scrub starts 2024-09-11T09:02:39.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:38 smithi145 ceph-mon[33384]: 5.7 scrub ok 2024-09-11T09:02:39.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:38 smithi145 ceph-mon[33384]: 3.2 scrub starts 2024-09-11T09:02:39.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:38 smithi145 ceph-mon[33384]: 3.2 scrub ok 2024-09-11T09:02:39.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:39 smithi092 ceph-mon[29647]: 1.e scrub starts 2024-09-11T09:02:39.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:39 smithi092 ceph-mon[29647]: 1.e scrub ok 2024-09-11T09:02:39.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:39 smithi092 ceph-mon[29647]: 6.11 deep-scrub starts 2024-09-11T09:02:39.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:39 smithi092 ceph-mon[29647]: 6.11 deep-scrub ok 2024-09-11T09:02:39.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:39 smithi092 ceph-mon[29647]: 6.0 deep-scrub starts 2024-09-11T09:02:39.974 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:39 smithi092 ceph-mon[29647]: 6.0 deep-scrub ok 2024-09-11T09:02:39.975 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:39 smithi092 ceph-mon[29647]: pgmap v1098: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-11T09:02:39.975 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:39 smithi092 ceph-mon[41114]: 1.e scrub starts 2024-09-11T09:02:39.975 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:39 smithi092 ceph-mon[41114]: 1.e scrub ok 2024-09-11T09:02:39.975 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:39 smithi092 ceph-mon[41114]: 6.11 deep-scrub starts 2024-09-11T09:02:39.975 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:39 smithi092 ceph-mon[41114]: 6.11 deep-scrub ok 2024-09-11T09:02:39.975 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:39 smithi092 ceph-mon[41114]: 6.0 deep-scrub starts 2024-09-11T09:02:39.975 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:39 smithi092 ceph-mon[41114]: 6.0 deep-scrub ok 2024-09-11T09:02:39.975 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:39 smithi092 ceph-mon[41114]: pgmap v1098: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-11T09:02:40.161 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:39 smithi145 ceph-mon[33384]: 1.e scrub starts 2024-09-11T09:02:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:39 smithi145 ceph-mon[33384]: 1.e scrub ok 2024-09-11T09:02:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:39 smithi145 ceph-mon[33384]: 6.11 deep-scrub starts 2024-09-11T09:02:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:39 smithi145 ceph-mon[33384]: 6.11 deep-scrub ok 2024-09-11T09:02:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:39 smithi145 ceph-mon[33384]: 6.0 deep-scrub starts 2024-09-11T09:02:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:39 smithi145 ceph-mon[33384]: 6.0 deep-scrub ok 2024-09-11T09:02:40.162 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:39 smithi145 ceph-mon[33384]: pgmap v1098: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-11T09:02:40.415 INFO:teuthology.orchestra.run.smithi092.stdout:111669150009 2024-09-11T09:02:40.416 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T09:02:41.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:41 smithi092 ceph-mon[29647]: 5.5 scrub starts 2024-09-11T09:02:41.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:41 smithi092 ceph-mon[29647]: 5.5 scrub ok 2024-09-11T09:02:41.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:41 smithi092 ceph-mon[41114]: 5.5 scrub starts 2024-09-11T09:02:41.447 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:41 smithi092 ceph-mon[41114]: 5.5 scrub ok 2024-09-11T09:02:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:41 smithi145 ceph-mon[33384]: 5.5 scrub starts 2024-09-11T09:02:41.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:41 smithi145 ceph-mon[33384]: 5.5 scrub ok 2024-09-11T09:02:41.791 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:02:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:02:41] "GET /metrics HTTP/1.1" 200 222527 "" "Prometheus/2.43.0" 2024-09-11T09:02:41.816 INFO:teuthology.orchestra.run.smithi092.stdout:133143986483 2024-09-11T09:02:41.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T09:02:42.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:42 smithi092 ceph-mon[29647]: 5.4 deep-scrub starts 2024-09-11T09:02:42.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:42 smithi092 ceph-mon[29647]: 5.4 deep-scrub ok 2024-09-11T09:02:42.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:42 smithi092 ceph-mon[29647]: pgmap v1099: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-11T09:02:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:42 smithi092 ceph-mon[29647]: 6.9 scrub starts 2024-09-11T09:02:42.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:42 smithi092 ceph-mon[29647]: 6.9 scrub ok 2024-09-11T09:02:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:42 smithi092 ceph-mon[41114]: 5.4 deep-scrub starts 2024-09-11T09:02:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:42 smithi092 ceph-mon[41114]: 5.4 deep-scrub ok 2024-09-11T09:02:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:42 smithi092 ceph-mon[41114]: pgmap v1099: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-11T09:02:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:42 smithi092 ceph-mon[41114]: 6.9 scrub starts 2024-09-11T09:02:42.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:42 smithi092 ceph-mon[41114]: 6.9 scrub ok 2024-09-11T09:02:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:42 smithi145 ceph-mon[33384]: 5.4 deep-scrub starts 2024-09-11T09:02:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:42 smithi145 ceph-mon[33384]: 5.4 deep-scrub ok 2024-09-11T09:02:42.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:42 smithi145 ceph-mon[33384]: pgmap v1099: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-11T09:02:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:42 smithi145 ceph-mon[33384]: 6.9 scrub starts 2024-09-11T09:02:42.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:42 smithi145 ceph-mon[33384]: 6.9 scrub ok 2024-09-11T09:02:43.000 INFO:teuthology.orchestra.run.smithi092.stdout:34359738694 2024-09-11T09:02:43.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T09:02:43.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:02:42 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:02:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:02:43.239 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:43 smithi092 ceph-mon[41114]: 1.4 scrub starts 2024-09-11T09:02:43.239 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:43 smithi092 ceph-mon[41114]: 1.4 scrub ok 2024-09-11T09:02:43.239 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:43 smithi092 ceph-mon[29647]: 1.4 scrub starts 2024-09-11T09:02:43.239 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:43 smithi092 ceph-mon[29647]: 1.4 scrub ok 2024-09-11T09:02:43.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:43 smithi145 ceph-mon[33384]: 1.4 scrub starts 2024-09-11T09:02:43.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:43 smithi145 ceph-mon[33384]: 1.4 scrub ok 2024-09-11T09:02:44.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:44 smithi092 ceph-mon[29647]: 6.1 scrub starts 2024-09-11T09:02:44.367 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:44 smithi092 ceph-mon[29647]: 6.1 scrub ok 2024-09-11T09:02:44.368 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:44 smithi092 ceph-mon[29647]: 1.c scrub starts 2024-09-11T09:02:44.368 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:44 smithi092 ceph-mon[29647]: 1.c scrub ok 2024-09-11T09:02:44.368 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:44 smithi092 ceph-mon[29647]: pgmap v1100: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-11T09:02:44.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:44 smithi092 ceph-mon[41114]: 6.1 scrub starts 2024-09-11T09:02:44.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:44 smithi092 ceph-mon[41114]: 6.1 scrub ok 2024-09-11T09:02:44.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:44 smithi092 ceph-mon[41114]: 1.c scrub starts 2024-09-11T09:02:44.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:44 smithi092 ceph-mon[41114]: 1.c scrub ok 2024-09-11T09:02:44.368 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:44 smithi092 ceph-mon[41114]: pgmap v1100: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-11T09:02:44.393 INFO:teuthology.orchestra.run.smithi092.stdout:77309411646 2024-09-11T09:02:44.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T09:02:44.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:44 smithi145 ceph-mon[33384]: 6.1 scrub starts 2024-09-11T09:02:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:44 smithi145 ceph-mon[33384]: 6.1 scrub ok 2024-09-11T09:02:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:44 smithi145 ceph-mon[33384]: 1.c scrub starts 2024-09-11T09:02:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:44 smithi145 ceph-mon[33384]: 1.c scrub ok 2024-09-11T09:02:44.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:44 smithi145 ceph-mon[33384]: pgmap v1100: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-11T09:02:44.781 INFO:teuthology.orchestra.run.smithi092.stdout:223338299687 2024-09-11T09:02:45.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:45 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/1752670695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T09:02:45.451 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:45 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:02:45.451 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:45 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/1752670695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T09:02:45.451 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:45 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:02:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:45 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/1752670695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T09:02:45.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:45 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:02:46.005 INFO:teuthology.orchestra.run.smithi092.stdout:55834575171 2024-09-11T09:02:46.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:46 smithi092 ceph-mon[29647]: 4.19 scrub starts 2024-09-11T09:02:46.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:46 smithi092 ceph-mon[29647]: 4.19 scrub ok 2024-09-11T09:02:46.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:46 smithi092 ceph-mon[29647]: pgmap v1101: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-11T09:02:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:46 smithi092 ceph-mon[29647]: 3.17 scrub starts 2024-09-11T09:02:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:46 smithi092 ceph-mon[29647]: 3.17 scrub ok 2024-09-11T09:02:46.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:46 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3484225372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T09:02:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:46 smithi092 ceph-mon[41114]: 4.19 scrub starts 2024-09-11T09:02:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:46 smithi092 ceph-mon[41114]: 4.19 scrub ok 2024-09-11T09:02:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:46 smithi092 ceph-mon[41114]: pgmap v1101: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-11T09:02:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:46 smithi092 ceph-mon[41114]: 3.17 scrub starts 2024-09-11T09:02:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:46 smithi092 ceph-mon[41114]: 3.17 scrub ok 2024-09-11T09:02:46.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:46 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3484225372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T09:02:46.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:46 smithi145 ceph-mon[33384]: 4.19 scrub starts 2024-09-11T09:02:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:46 smithi145 ceph-mon[33384]: 4.19 scrub ok 2024-09-11T09:02:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:46 smithi145 ceph-mon[33384]: pgmap v1101: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-11T09:02:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:46 smithi145 ceph-mon[33384]: 3.17 scrub starts 2024-09-11T09:02:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:46 smithi145 ceph-mon[33384]: 3.17 scrub ok 2024-09-11T09:02:46.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:46 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3484225372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T09:02:46.834 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299685 got 223338299687 for osd.7 2024-09-11T09:02:46.834 DEBUG:teuthology.parallel:result is None 2024-09-11T09:02:47.048 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:47.050 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:47.050 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:47.484 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:47 smithi145 ceph-mon[33384]: 4.10 scrub starts 2024-09-11T09:02:47.484 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:47 smithi145 ceph-mon[33384]: 4.10 scrub ok 2024-09-11T09:02:47.484 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:47 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:02:47.484 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:47 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:02:47.484 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:47 smithi145 ceph-mon[33384]: 3.1f scrub starts 2024-09-11T09:02:47.484 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:47 smithi145 ceph-mon[33384]: 3.1f scrub ok 2024-09-11T09:02:47.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:47 smithi092 ceph-mon[29647]: 4.10 scrub starts 2024-09-11T09:02:47.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:47 smithi092 ceph-mon[29647]: 4.10 scrub ok 2024-09-11T09:02:47.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:47 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:02:47.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:47 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:02:47.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:47 smithi092 ceph-mon[29647]: 3.1f scrub starts 2024-09-11T09:02:47.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:47 smithi092 ceph-mon[29647]: 3.1f scrub ok 2024-09-11T09:02:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:47 smithi092 ceph-mon[41114]: 4.10 scrub starts 2024-09-11T09:02:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:47 smithi092 ceph-mon[41114]: 4.10 scrub ok 2024-09-11T09:02:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:47 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:02:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:47 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:02:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:47 smithi092 ceph-mon[41114]: 3.1f scrub starts 2024-09-11T09:02:47.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:47 smithi092 ceph-mon[41114]: 3.1f scrub ok 2024-09-11T09:02:47.801 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834575169 got 55834575171 for osd.1 2024-09-11T09:02:47.802 DEBUG:teuthology.parallel:result is None 2024-09-11T09:02:47.911 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:02:47 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:02:48.328 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:48 smithi092 ceph-mon[41114]: 5.a deep-scrub starts 2024-09-11T09:02:48.328 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:48 smithi092 ceph-mon[41114]: 5.a deep-scrub ok 2024-09-11T09:02:48.328 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:48 smithi092 ceph-mon[41114]: osdmap e463: 8 total, 8 up, 8 in 2024-09-11T09:02:48.329 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:48 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:48.329 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:48 smithi092 ceph-mon[41114]: pgmap v1103: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:02:48.329 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:48 smithi092 ceph-mon[29647]: 5.a deep-scrub starts 2024-09-11T09:02:48.329 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:48 smithi092 ceph-mon[29647]: 5.a deep-scrub ok 2024-09-11T09:02:48.329 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:48 smithi092 ceph-mon[29647]: osdmap e463: 8 total, 8 up, 8 in 2024-09-11T09:02:48.329 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:48 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:48.329 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:48 smithi092 ceph-mon[29647]: pgmap v1103: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:02:48.469 INFO:teuthology.orchestra.run.smithi092.stdout:193273528621 2024-09-11T09:02:48.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:48 smithi145 ceph-mon[33384]: 5.a deep-scrub starts 2024-09-11T09:02:48.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:48 smithi145 ceph-mon[33384]: 5.a deep-scrub ok 2024-09-11T09:02:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:48 smithi145 ceph-mon[33384]: osdmap e463: 8 total, 8 up, 8 in 2024-09-11T09:02:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:48 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:48.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:48 smithi145 ceph-mon[33384]: pgmap v1103: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:02:49.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:49 smithi092 ceph-mon[29647]: 3.1d scrub starts 2024-09-11T09:02:49.315 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:49 smithi092 ceph-mon[29647]: 3.1d scrub ok 2024-09-11T09:02:49.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:49 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/840658033' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T09:02:49.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:49 smithi092 ceph-mon[41114]: 3.1d scrub starts 2024-09-11T09:02:49.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:49 smithi092 ceph-mon[41114]: 3.1d scrub ok 2024-09-11T09:02:49.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:49 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/840658033' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T09:02:49.348 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:49 smithi145 ceph-mon[33384]: 3.1d scrub starts 2024-09-11T09:02:49.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:49 smithi145 ceph-mon[33384]: 3.1d scrub ok 2024-09-11T09:02:49.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:49 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/840658033' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T09:02:50.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:50 smithi092 ceph-mon[29647]: 5.1d scrub starts 2024-09-11T09:02:50.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:50 smithi092 ceph-mon[29647]: 5.1d scrub ok 2024-09-11T09:02:50.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:50 smithi092 ceph-mon[29647]: 6.3 scrub starts 2024-09-11T09:02:50.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:50 smithi092 ceph-mon[29647]: 6.3 scrub ok 2024-09-11T09:02:50.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:50 smithi092 ceph-mon[29647]: pgmap v1104: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:02:50.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:50 smithi092 ceph-mon[41114]: 5.1d scrub starts 2024-09-11T09:02:50.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:50 smithi092 ceph-mon[41114]: 5.1d scrub ok 2024-09-11T09:02:50.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:50 smithi092 ceph-mon[41114]: 6.3 scrub starts 2024-09-11T09:02:50.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:50 smithi092 ceph-mon[41114]: 6.3 scrub ok 2024-09-11T09:02:50.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:50 smithi092 ceph-mon[41114]: pgmap v1104: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:02:50.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:50 smithi145 ceph-mon[33384]: 5.1d scrub starts 2024-09-11T09:02:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:50 smithi145 ceph-mon[33384]: 5.1d scrub ok 2024-09-11T09:02:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:50 smithi145 ceph-mon[33384]: 6.3 scrub starts 2024-09-11T09:02:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:50 smithi145 ceph-mon[33384]: 6.3 scrub ok 2024-09-11T09:02:50.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:50 smithi145 ceph-mon[33384]: pgmap v1104: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:02:50.973 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:02:51.003 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528617 got 193273528621 for osd.6 2024-09-11T09:02:51.004 DEBUG:teuthology.parallel:result is None 2024-09-11T09:02:51.491 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:51 smithi092 ceph-mon[29647]: 3.14 scrub starts 2024-09-11T09:02:51.491 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:51 smithi092 ceph-mon[29647]: 3.14 scrub ok 2024-09-11T09:02:51.492 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:02:51 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:02:51] "GET /metrics HTTP/1.1" 200 222484 "" "Prometheus/2.43.0" 2024-09-11T09:02:51.492 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:51 smithi092 ceph-mon[41114]: 3.14 scrub starts 2024-09-11T09:02:51.492 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:51 smithi092 ceph-mon[41114]: 3.14 scrub ok 2024-09-11T09:02:51.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:51 smithi145 ceph-mon[33384]: 3.14 scrub starts 2024-09-11T09:02:51.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:51 smithi145 ceph-mon[33384]: 3.14 scrub ok 2024-09-11T09:02:52.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:52 smithi092 ceph-mon[41114]: 4.7 scrub starts 2024-09-11T09:02:52.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:52 smithi092 ceph-mon[41114]: 4.7 scrub ok 2024-09-11T09:02:52.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:52 smithi092 ceph-mon[41114]: 6.8 scrub starts 2024-09-11T09:02:52.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:52 smithi092 ceph-mon[41114]: 6.8 scrub ok 2024-09-11T09:02:52.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:52 smithi092 ceph-mon[41114]: pgmap v1105: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:02:52.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:52 smithi092 ceph-mon[29647]: 4.7 scrub starts 2024-09-11T09:02:52.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:52 smithi092 ceph-mon[29647]: 4.7 scrub ok 2024-09-11T09:02:52.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:52 smithi092 ceph-mon[29647]: 6.8 scrub starts 2024-09-11T09:02:52.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:52 smithi092 ceph-mon[29647]: 6.8 scrub ok 2024-09-11T09:02:52.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:52 smithi092 ceph-mon[29647]: pgmap v1105: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:02:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:52 smithi145 ceph-mon[33384]: 4.7 scrub starts 2024-09-11T09:02:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:52 smithi145 ceph-mon[33384]: 4.7 scrub ok 2024-09-11T09:02:52.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:52 smithi145 ceph-mon[33384]: 6.8 scrub starts 2024-09-11T09:02:52.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:52 smithi145 ceph-mon[33384]: 6.8 scrub ok 2024-09-11T09:02:52.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:52 smithi145 ceph-mon[33384]: pgmap v1105: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:02:53.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:02:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:02:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:02:53.344 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:53 smithi092 ceph-mon[41114]: 3.1c scrub starts 2024-09-11T09:02:53.344 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:53 smithi092 ceph-mon[41114]: 3.1c scrub ok 2024-09-11T09:02:53.344 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:53 smithi092 ceph-mon[29647]: 3.1c scrub starts 2024-09-11T09:02:53.344 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:53 smithi092 ceph-mon[29647]: 3.1c scrub ok 2024-09-11T09:02:53.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:53 smithi145 ceph-mon[33384]: 3.1c scrub starts 2024-09-11T09:02:53.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:53 smithi145 ceph-mon[33384]: 3.1c scrub ok 2024-09-11T09:02:54.345 INFO:teuthology.orchestra.run.smithi092.stdout:77309411648 2024-09-11T09:02:54.485 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:54 smithi092 ceph-mon[29647]: 6.14 scrub starts 2024-09-11T09:02:54.486 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:54 smithi092 ceph-mon[29647]: 6.14 scrub ok 2024-09-11T09:02:54.486 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:54 smithi092 ceph-mon[29647]: 6.1b scrub starts 2024-09-11T09:02:54.486 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:54 smithi092 ceph-mon[29647]: 6.1b scrub ok 2024-09-11T09:02:54.486 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:54 smithi092 ceph-mon[29647]: 3.16 scrub starts 2024-09-11T09:02:54.486 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:54 smithi092 ceph-mon[29647]: 3.16 scrub ok 2024-09-11T09:02:54.486 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:54 smithi092 ceph-mon[29647]: pgmap v1106: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:02:54.486 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:54 smithi092 ceph-mon[41114]: 6.14 scrub starts 2024-09-11T09:02:54.486 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:54 smithi092 ceph-mon[41114]: 6.14 scrub ok 2024-09-11T09:02:54.486 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:54 smithi092 ceph-mon[41114]: 6.1b scrub starts 2024-09-11T09:02:54.486 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:54 smithi092 ceph-mon[41114]: 6.1b scrub ok 2024-09-11T09:02:54.487 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:54 smithi092 ceph-mon[41114]: 3.16 scrub starts 2024-09-11T09:02:54.487 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:54 smithi092 ceph-mon[41114]: 3.16 scrub ok 2024-09-11T09:02:54.487 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:54 smithi092 ceph-mon[41114]: pgmap v1106: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:02:54.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:54 smithi145 ceph-mon[33384]: 6.14 scrub starts 2024-09-11T09:02:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:54 smithi145 ceph-mon[33384]: 6.14 scrub ok 2024-09-11T09:02:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:54 smithi145 ceph-mon[33384]: 6.1b scrub starts 2024-09-11T09:02:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:54 smithi145 ceph-mon[33384]: 6.1b scrub ok 2024-09-11T09:02:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:54 smithi145 ceph-mon[33384]: 3.16 scrub starts 2024-09-11T09:02:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:54 smithi145 ceph-mon[33384]: 3.16 scrub ok 2024-09-11T09:02:54.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:54 smithi145 ceph-mon[33384]: pgmap v1106: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:02:55.514 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:55 smithi092 ceph-mon[29647]: 1.5 scrub starts 2024-09-11T09:02:55.514 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:55 smithi092 ceph-mon[29647]: 1.5 scrub ok 2024-09-11T09:02:55.514 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:55 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:02:55.514 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:55 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:02:55.514 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:55 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/390198492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T09:02:55.514 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:55 smithi092 ceph-mon[29647]: 3.10 scrub starts 2024-09-11T09:02:55.514 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:55 smithi092 ceph-mon[29647]: 3.10 scrub ok 2024-09-11T09:02:55.514 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:55 smithi092 ceph-mon[41114]: 1.5 scrub starts 2024-09-11T09:02:55.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:55 smithi092 ceph-mon[41114]: 1.5 scrub ok 2024-09-11T09:02:55.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:55 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:02:55.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:55 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:02:55.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:55 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/390198492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T09:02:55.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:55 smithi092 ceph-mon[41114]: 3.10 scrub starts 2024-09-11T09:02:55.515 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:55 smithi092 ceph-mon[41114]: 3.10 scrub ok 2024-09-11T09:02:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:55 smithi145 ceph-mon[33384]: 1.5 scrub starts 2024-09-11T09:02:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:55 smithi145 ceph-mon[33384]: 1.5 scrub ok 2024-09-11T09:02:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:55 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:02:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:55 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:02:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:55 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/390198492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T09:02:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:55 smithi145 ceph-mon[33384]: 3.10 scrub starts 2024-09-11T09:02:55.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:55 smithi145 ceph-mon[33384]: 3.10 scrub ok 2024-09-11T09:02:55.760 INFO:teuthology.orchestra.run.smithi092.stdout:163208757554 2024-09-11T09:02:56.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[29647]: 5.19 deep-scrub starts 2024-09-11T09:02:56.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[29647]: 5.19 deep-scrub ok 2024-09-11T09:02:56.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[29647]: 6.1a scrub starts 2024-09-11T09:02:56.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[29647]: 6.1a scrub ok 2024-09-11T09:02:56.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[29647]: 4.15 scrub starts 2024-09-11T09:02:56.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[29647]: 4.15 scrub ok 2024-09-11T09:02:56.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[29647]: osdmap e464: 8 total, 8 up, 8 in 2024-09-11T09:02:56.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[29647]: pgmap v1108: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 615 B/s rd, 0 op/s 2024-09-11T09:02:56.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3632728974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T09:02:56.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[41114]: 5.19 deep-scrub starts 2024-09-11T09:02:56.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[41114]: 5.19 deep-scrub ok 2024-09-11T09:02:56.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[41114]: 6.1a scrub starts 2024-09-11T09:02:56.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[41114]: 6.1a scrub ok 2024-09-11T09:02:56.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[41114]: 4.15 scrub starts 2024-09-11T09:02:56.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[41114]: 4.15 scrub ok 2024-09-11T09:02:56.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[41114]: osdmap e464: 8 total, 8 up, 8 in 2024-09-11T09:02:56.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[41114]: pgmap v1108: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 615 B/s rd, 0 op/s 2024-09-11T09:02:56.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:56 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3632728974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T09:02:56.642 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757550 got 163208757554 for osd.5 2024-09-11T09:02:56.642 DEBUG:teuthology.parallel:result is None 2024-09-11T09:02:56.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:56 smithi145 ceph-mon[33384]: 5.19 deep-scrub starts 2024-09-11T09:02:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:56 smithi145 ceph-mon[33384]: 5.19 deep-scrub ok 2024-09-11T09:02:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:56 smithi145 ceph-mon[33384]: 6.1a scrub starts 2024-09-11T09:02:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:56 smithi145 ceph-mon[33384]: 6.1a scrub ok 2024-09-11T09:02:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:56 smithi145 ceph-mon[33384]: 4.15 scrub starts 2024-09-11T09:02:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:56 smithi145 ceph-mon[33384]: 4.15 scrub ok 2024-09-11T09:02:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:56 smithi145 ceph-mon[33384]: osdmap e464: 8 total, 8 up, 8 in 2024-09-11T09:02:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:56 smithi145 ceph-mon[33384]: pgmap v1108: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 615 B/s rd, 0 op/s 2024-09-11T09:02:56.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:56 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3632728974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T09:02:57.022 INFO:teuthology.orchestra.run.smithi092.stdout:133143986486 2024-09-11T09:02:57.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:57 smithi092 ceph-mon[29647]: 5.0 scrub starts 2024-09-11T09:02:57.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:57 smithi092 ceph-mon[29647]: 5.0 scrub ok 2024-09-11T09:02:57.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:57 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3803976038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T09:02:57.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:57 smithi092 ceph-mon[41114]: 5.0 scrub starts 2024-09-11T09:02:57.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:57 smithi092 ceph-mon[41114]: 5.0 scrub ok 2024-09-11T09:02:57.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:57 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3803976038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T09:02:57.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:57 smithi145 ceph-mon[33384]: 5.0 scrub starts 2024-09-11T09:02:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:57 smithi145 ceph-mon[33384]: 5.0 scrub ok 2024-09-11T09:02:57.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:57 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3803976038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T09:02:57.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:02:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:02:57.980 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411646 got 77309411648 for osd.2 2024-09-11T09:02:57.980 DEBUG:teuthology.parallel:result is None 2024-09-11T09:02:58.312 INFO:teuthology.orchestra.run.smithi092.stdout:34359738698 2024-09-11T09:02:58.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:58 smithi092 ceph-mon[29647]: 3.5 deep-scrub starts 2024-09-11T09:02:58.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:58 smithi092 ceph-mon[29647]: 3.5 deep-scrub ok 2024-09-11T09:02:58.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:58 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:58.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:58 smithi092 ceph-mon[29647]: pgmap v1109: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:02:58.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:58 smithi092 ceph-mon[29647]: 5.9 scrub starts 2024-09-11T09:02:58.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:58 smithi092 ceph-mon[29647]: 5.9 scrub ok 2024-09-11T09:02:58.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:58 smithi092 ceph-mon[41114]: 3.5 deep-scrub starts 2024-09-11T09:02:58.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:58 smithi092 ceph-mon[41114]: 3.5 deep-scrub ok 2024-09-11T09:02:58.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:58 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:58.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:58 smithi092 ceph-mon[41114]: pgmap v1109: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:02:58.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:58 smithi092 ceph-mon[41114]: 5.9 scrub starts 2024-09-11T09:02:58.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:58 smithi092 ceph-mon[41114]: 5.9 scrub ok 2024-09-11T09:02:58.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:58 smithi145 ceph-mon[33384]: 3.5 deep-scrub starts 2024-09-11T09:02:58.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:58 smithi145 ceph-mon[33384]: 3.5 deep-scrub ok 2024-09-11T09:02:58.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:58 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:02:58.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:58 smithi145 ceph-mon[33384]: pgmap v1109: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:02:58.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:58 smithi145 ceph-mon[33384]: 5.9 scrub starts 2024-09-11T09:02:58.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:58 smithi145 ceph-mon[33384]: 5.9 scrub ok 2024-09-11T09:02:58.877 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986483 got 133143986486 for osd.4 2024-09-11T09:02:58.877 DEBUG:teuthology.parallel:result is None 2024-09-11T09:02:59.493 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[29647]: 5.12 scrub starts 2024-09-11T09:02:59.493 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[29647]: 5.12 scrub ok 2024-09-11T09:02:59.494 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[29647]: 4.13 scrub starts 2024-09-11T09:02:59.494 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[29647]: 4.13 scrub ok 2024-09-11T09:02:59.494 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[29647]: 3.18 scrub starts 2024-09-11T09:02:59.494 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[29647]: 3.18 scrub ok 2024-09-11T09:02:59.494 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[29647]: 6.19 scrub starts 2024-09-11T09:02:59.494 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[29647]: 6.19 scrub ok 2024-09-11T09:02:59.494 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/3898540448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T09:02:59.494 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[41114]: 5.12 scrub starts 2024-09-11T09:02:59.494 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[41114]: 5.12 scrub ok 2024-09-11T09:02:59.494 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[41114]: 4.13 scrub starts 2024-09-11T09:02:59.494 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[41114]: 4.13 scrub ok 2024-09-11T09:02:59.494 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[41114]: 3.18 scrub starts 2024-09-11T09:02:59.495 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[41114]: 3.18 scrub ok 2024-09-11T09:02:59.495 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[41114]: 6.19 scrub starts 2024-09-11T09:02:59.495 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[41114]: 6.19 scrub ok 2024-09-11T09:02:59.495 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:02:59 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/3898540448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T09:02:59.583 INFO:teuthology.orchestra.run.smithi092.stdout:111669150013 2024-09-11T09:02:59.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:59 smithi145 ceph-mon[33384]: 5.12 scrub starts 2024-09-11T09:02:59.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:59 smithi145 ceph-mon[33384]: 5.12 scrub ok 2024-09-11T09:02:59.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:59 smithi145 ceph-mon[33384]: 4.13 scrub starts 2024-09-11T09:02:59.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:59 smithi145 ceph-mon[33384]: 4.13 scrub ok 2024-09-11T09:02:59.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:59 smithi145 ceph-mon[33384]: 3.18 scrub starts 2024-09-11T09:02:59.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:59 smithi145 ceph-mon[33384]: 3.18 scrub ok 2024-09-11T09:02:59.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:59 smithi145 ceph-mon[33384]: 6.19 scrub starts 2024-09-11T09:02:59.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:59 smithi145 ceph-mon[33384]: 6.19 scrub ok 2024-09-11T09:02:59.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:02:59 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/3898540448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T09:03:00.378 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738694 got 34359738698 for osd.0 2024-09-11T09:03:00.378 DEBUG:teuthology.parallel:result is None 2024-09-11T09:03:00.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:00 smithi145 ceph-mon[33384]: 6.f scrub starts 2024-09-11T09:03:00.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:00 smithi145 ceph-mon[33384]: 6.f scrub ok 2024-09-11T09:03:00.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:00 smithi145 ceph-mon[33384]: 6.4 scrub starts 2024-09-11T09:03:00.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:00 smithi145 ceph-mon[33384]: 6.4 scrub ok 2024-09-11T09:03:00.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:00 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/711169356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T09:03:00.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:00 smithi145 ceph-mon[33384]: pgmap v1110: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:03:00.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T09:03:00.563 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:00 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:03:00.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[29647]: 6.f scrub starts 2024-09-11T09:03:00.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[29647]: 6.f scrub ok 2024-09-11T09:03:00.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[29647]: 6.4 scrub starts 2024-09-11T09:03:00.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[29647]: 6.4 scrub ok 2024-09-11T09:03:00.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/711169356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T09:03:00.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[29647]: pgmap v1110: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:03:00.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T09:03:00.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:03:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[41114]: 6.f scrub starts 2024-09-11T09:03:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[41114]: 6.f scrub ok 2024-09-11T09:03:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[41114]: 6.4 scrub starts 2024-09-11T09:03:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[41114]: 6.4 scrub ok 2024-09-11T09:03:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/711169356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T09:03:00.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[41114]: pgmap v1110: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:03:00.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T09:03:00.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:00 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T09:03:00.940 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669150009 got 111669150013 for osd.3 2024-09-11T09:03:00.940 DEBUG:teuthology.parallel:result is None 2024-09-11T09:03:00.941 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T09:03:00.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T09:03:01.184 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:03:01.565 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:01 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:03:01] "GET /metrics HTTP/1.1" 200 222501 "" "Prometheus/2.43.0" 2024-09-11T09:03:01.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:01 smithi092 ceph-mon[29647]: 5.17 deep-scrub starts 2024-09-11T09:03:01.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:01 smithi092 ceph-mon[29647]: 5.17 deep-scrub ok 2024-09-11T09:03:01.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:01 smithi092 ceph-mon[29647]: 1.8 scrub starts 2024-09-11T09:03:01.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:01 smithi092 ceph-mon[29647]: 1.8 scrub ok 2024-09-11T09:03:01.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:01 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T09:03:01.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:01 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T09:03:01.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:01 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:03:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:01 smithi092 ceph-mon[41114]: 5.17 deep-scrub starts 2024-09-11T09:03:01.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:01 smithi092 ceph-mon[41114]: 5.17 deep-scrub ok 2024-09-11T09:03:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:01 smithi092 ceph-mon[41114]: 1.8 scrub starts 2024-09-11T09:03:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:01 smithi092 ceph-mon[41114]: 1.8 scrub ok 2024-09-11T09:03:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:01 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T09:03:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:01 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T09:03:01.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:01 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:03:01.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:01 smithi145 ceph-mon[33384]: 5.17 deep-scrub starts 2024-09-11T09:03:01.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:01 smithi145 ceph-mon[33384]: 5.17 deep-scrub ok 2024-09-11T09:03:01.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:01 smithi145 ceph-mon[33384]: 1.8 scrub starts 2024-09-11T09:03:01.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:01 smithi145 ceph-mon[33384]: 1.8 scrub ok 2024-09-11T09:03:01.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:01 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T09:03:01.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:01 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T09:03:01.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:01 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:03:02.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:02 smithi092 ceph-mon[29647]: 4.2 scrub starts 2024-09-11T09:03:02.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:02 smithi092 ceph-mon[29647]: 4.2 scrub ok 2024-09-11T09:03:02.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:02 smithi092 ceph-mon[29647]: 6.2 scrub starts 2024-09-11T09:03:02.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:02 smithi092 ceph-mon[29647]: 6.2 scrub ok 2024-09-11T09:03:02.515 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:02 smithi092 ceph-mon[29647]: pgmap v1111: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:03:02.516 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:02 smithi092 ceph-mon[41114]: 4.2 scrub starts 2024-09-11T09:03:02.516 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:02 smithi092 ceph-mon[41114]: 4.2 scrub ok 2024-09-11T09:03:02.516 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:02 smithi092 ceph-mon[41114]: 6.2 scrub starts 2024-09-11T09:03:02.516 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:02 smithi092 ceph-mon[41114]: 6.2 scrub ok 2024-09-11T09:03:02.516 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:02 smithi092 ceph-mon[41114]: pgmap v1111: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:03:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:02 smithi145 ceph-mon[33384]: 4.2 scrub starts 2024-09-11T09:03:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:02 smithi145 ceph-mon[33384]: 4.2 scrub ok 2024-09-11T09:03:02.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:02 smithi145 ceph-mon[33384]: 6.2 scrub starts 2024-09-11T09:03:02.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:02 smithi145 ceph-mon[33384]: 6.2 scrub ok 2024-09-11T09:03:02.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:02 smithi145 ceph-mon[33384]: pgmap v1111: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:03:03.161 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:03:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:03:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:03:03.240 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T09:03:03.243 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T09:03:03.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:03 smithi092 ceph-mon[29647]: 2.0 scrub starts 2024-09-11T09:03:03.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:03 smithi092 ceph-mon[29647]: 2.0 scrub ok 2024-09-11T09:03:03.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:03 smithi092 ceph-mon[29647]: 5.2 scrub starts 2024-09-11T09:03:03.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:03 smithi092 ceph-mon[29647]: 5.2 scrub ok 2024-09-11T09:03:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:03 smithi092 ceph-mon[41114]: 2.0 scrub starts 2024-09-11T09:03:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:03 smithi092 ceph-mon[41114]: 2.0 scrub ok 2024-09-11T09:03:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:03 smithi092 ceph-mon[41114]: 5.2 scrub starts 2024-09-11T09:03:03.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:03 smithi092 ceph-mon[41114]: 5.2 scrub ok 2024-09-11T09:03:03.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:03 smithi145 ceph-mon[33384]: 2.0 scrub starts 2024-09-11T09:03:03.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:03 smithi145 ceph-mon[33384]: 2.0 scrub ok 2024-09-11T09:03:03.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:03 smithi145 ceph-mon[33384]: 5.2 scrub starts 2024-09-11T09:03:03.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:03 smithi145 ceph-mon[33384]: 5.2 scrub ok 2024-09-11T09:03:03.993 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":1111,"stamp":"2024-09-11T09:03:01.532388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1689,"num_read_kb":1441,"num_write":1002,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":426,"num_bytes_recovered":10586,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2889,"ondisk_log_size":2889,"up":456,"acting":456,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":456,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":7185176,"kb_used_data":7748,"kb_used_omap":13,"kb_used_meta":7177330,"kb_avail":742776040,"statfs":{"total":767960285184,"available":760602664960,"internally_reserved":0,"allocated":7933952,"data_stored":5344076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":7349586624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002839"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":147,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932752+0000","last_change":"2024-09-11T09:02:46.954818+0000","last_active":"2024-09-11T09:02:47.932752+0000","last_peered":"2024-09-11T09:02:47.932752+0000","last_clean":"2024-09-11T09:02:47.932752+0000","last_became_active":"2024-09-11T09:01:25.400173+0000","last_became_peered":"2024-09-11T09:01:25.400173+0000","last_unstale":"2024-09-11T09:02:47.932752+0000","last_undegraded":"2024-09-11T09:02:47.932752+0000","last_fullsized":"2024-09-11T09:02:47.932752+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:46.954763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:41.077950+0000","last_clean_scrub_stamp":"2024-09-11T09:02:46.954763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:00.566135+0000","scrub_duration":0.001632618,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"402'25","reported_seq":613,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212043+0000","last_change":"2024-09-11T09:02:23.846135+0000","last_active":"2024-09-11T09:02:47.212043+0000","last_peered":"2024-09-11T09:02:47.212043+0000","last_clean":"2024-09-11T09:02:47.212043+0000","last_became_active":"2024-09-11T08:58:34.124732+0000","last_became_peered":"2024-09-11T08:58:34.124732+0000","last_unstale":"2024-09-11T09:02:47.212043+0000","last_undegraded":"2024-09-11T09:02:47.212043+0000","last_fullsized":"2024-09-11T09:02:47.212043+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'25","last_scrub_stamp":"2024-09-11T09:02:23.846064+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T09:02:23.846064+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:52.598290+0000","scrub_duration":0.0021792809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":532,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:54.777107+0000","last_change":"2024-09-11T09:02:54.777070+0000","last_active":"2024-09-11T09:02:54.777107+0000","last_peered":"2024-09-11T09:02:54.777107+0000","last_clean":"2024-09-11T09:02:54.777107+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T09:02:54.777107+0000","last_undegraded":"2024-09-11T09:02:54.777107+0000","last_fullsized":"2024-09-11T09:02:54.777107+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:54.777016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:54.777016+0000","last_clean_scrub_stamp":"2024-09-11T09:02:54.777016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:01.347708+0000","scrub_duration":0.0014251509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":511,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:54.836251+0000","last_change":"2024-09-11T09:02:54.836201+0000","last_active":"2024-09-11T09:02:54.836251+0000","last_peered":"2024-09-11T09:02:54.836251+0000","last_clean":"2024-09-11T09:02:54.836251+0000","last_became_active":"2024-09-11T09:01:28.501615+0000","last_became_peered":"2024-09-11T09:01:28.501615+0000","last_unstale":"2024-09-11T09:02:54.836251+0000","last_undegraded":"2024-09-11T09:02:54.836251+0000","last_fullsized":"2024-09-11T09:02:54.836251+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:54.836130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:54.836130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:56.488493+0000","scrub_duration":0.00153531,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":540,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:52.867364+0000","last_change":"2024-09-11T09:02:52.867322+0000","last_active":"2024-09-11T09:02:52.867364+0000","last_peered":"2024-09-11T09:02:52.867364+0000","last_clean":"2024-09-11T09:02:52.867364+0000","last_became_active":"2024-09-11T08:58:34.135162+0000","last_became_peered":"2024-09-11T08:58:34.135162+0000","last_unstale":"2024-09-11T09:02:52.867364+0000","last_undegraded":"2024-09-11T09:02:52.867364+0000","last_fullsized":"2024-09-11T09:02:52.867364+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:52.867256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:52.867256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:57.786100+0000","scrub_duration":0.0019533269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":539,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214940+0000","last_change":"2024-09-11T09:02:10.795660+0000","last_active":"2024-09-11T09:02:47.214940+0000","last_peered":"2024-09-11T09:02:47.214940+0000","last_clean":"2024-09-11T09:02:47.214940+0000","last_became_active":"2024-09-11T08:58:34.316048+0000","last_became_peered":"2024-09-11T08:58:34.316048+0000","last_unstale":"2024-09-11T09:02:47.214940+0000","last_undegraded":"2024-09-11T09:02:47.214940+0000","last_fullsized":"2024-09-11T09:02:47.214940+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:10.795606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:10.795606+0000","last_clean_scrub_stamp":"2024-09-11T09:02:10.795606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:18.892727+0000","scrub_duration":0.0017057629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"402'46","reported_seq":624,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214252+0000","last_change":"2024-09-11T09:02:45.005804+0000","last_active":"2024-09-11T09:02:47.214252+0000","last_peered":"2024-09-11T09:02:47.214252+0000","last_clean":"2024-09-11T09:02:47.214252+0000","last_became_active":"2024-09-11T08:52:18.585600+0000","last_became_peered":"2024-09-11T08:52:18.585600+0000","last_unstale":"2024-09-11T09:02:47.214252+0000","last_undegraded":"2024-09-11T09:02:47.214252+0000","last_fullsized":"2024-09-11T09:02:47.214252+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'46","last_scrub_stamp":"2024-09-11T09:02:45.005742+0000","last_deep_scrub":"200'34","last_deep_scrub_stamp":"2024-09-11T08:51:31.358660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:45.005742+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:10.137781+0000","scrub_duration":0.0017452469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":513,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212030+0000","last_change":"2024-09-11T09:01:46.683759+0000","last_active":"2024-09-11T09:02:47.212030+0000","last_peered":"2024-09-11T09:02:47.212030+0000","last_clean":"2024-09-11T09:02:47.212030+0000","last_became_active":"2024-09-11T08:59:18.590212+0000","last_became_peered":"2024-09-11T08:59:18.590212+0000","last_unstale":"2024-09-11T09:02:47.212030+0000","last_undegraded":"2024-09-11T09:02:47.212030+0000","last_fullsized":"2024-09-11T09:02:47.212030+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:46.683708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T09:01:46.683708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001532423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":552,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:48.073383+0000","last_change":"2024-09-11T09:02:48.073333+0000","last_active":"2024-09-11T09:02:48.073383+0000","last_peered":"2024-09-11T09:02:48.073383+0000","last_clean":"2024-09-11T09:02:48.073383+0000","last_became_active":"2024-09-11T08:58:34.153391+0000","last_became_peered":"2024-09-11T08:58:34.153391+0000","last_unstale":"2024-09-11T09:02:48.073383+0000","last_undegraded":"2024-09-11T09:02:48.073383+0000","last_fullsized":"2024-09-11T09:02:48.073383+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:48.073266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T09:02:48.073266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:02.982251+0000","scrub_duration":0.001572642,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"402'25","reported_seq":573,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212521+0000","last_change":"2024-09-11T09:01:23.873666+0000","last_active":"2024-09-11T09:02:47.212521+0000","last_peered":"2024-09-11T09:02:47.212521+0000","last_clean":"2024-09-11T09:02:47.212521+0000","last_became_active":"2024-09-11T08:52:18.577411+0000","last_became_peered":"2024-09-11T08:52:18.577411+0000","last_unstale":"2024-09-11T09:02:47.212521+0000","last_undegraded":"2024-09-11T09:02:47.212521+0000","last_fullsized":"2024-09-11T09:02:47.212521+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'25","last_scrub_stamp":"2024-09-11T09:01:23.873601+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T09:01:23.873601+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001935911,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":528,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210444+0000","last_change":"2024-09-11T09:01:54.331887+0000","last_active":"2024-09-11T09:02:47.210444+0000","last_peered":"2024-09-11T09:02:47.210444+0000","last_clean":"2024-09-11T09:02:47.210444+0000","last_became_active":"2024-09-11T08:52:18.573461+0000","last_became_peered":"2024-09-11T08:52:18.573461+0000","last_unstale":"2024-09-11T09:02:47.210444+0000","last_undegraded":"2024-09-11T09:02:47.210444+0000","last_fullsized":"2024-09-11T09:02:47.210444+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:54.331833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:01:54.331833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:22.433718+0000","scrub_duration":0.0012485619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":137,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932274+0000","last_change":"2024-09-11T09:02:19.871321+0000","last_active":"2024-09-11T09:02:47.932274+0000","last_peered":"2024-09-11T09:02:47.932274+0000","last_clean":"2024-09-11T09:02:47.932274+0000","last_became_active":"2024-09-11T09:01:25.419172+0000","last_became_peered":"2024-09-11T09:01:25.419172+0000","last_unstale":"2024-09-11T09:02:47.932274+0000","last_undegraded":"2024-09-11T09:02:47.932274+0000","last_fullsized":"2024-09-11T09:02:47.932274+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:19.871233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:19.871233+0000","last_clean_scrub_stamp":"2024-09-11T09:02:19.871233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:31.660758+0000","scrub_duration":0.001544405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":512,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:52.037618+0000","last_change":"2024-09-11T09:02:52.037560+0000","last_active":"2024-09-11T09:02:52.037618+0000","last_peered":"2024-09-11T09:02:52.037618+0000","last_clean":"2024-09-11T09:02:52.037618+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T09:02:52.037618+0000","last_undegraded":"2024-09-11T09:02:52.037618+0000","last_fullsized":"2024-09-11T09:02:52.037618+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:52.037494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:30.465003+0000","last_clean_scrub_stamp":"2024-09-11T09:02:52.037494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:10.885814+0000","scrub_duration":0.0018698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"402'9","reported_seq":536,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212501+0000","last_change":"2024-09-11T09:02:07.768204+0000","last_active":"2024-09-11T09:02:47.212501+0000","last_peered":"2024-09-11T09:02:47.212501+0000","last_clean":"2024-09-11T09:02:47.212501+0000","last_became_active":"2024-09-11T09:01:28.501477+0000","last_became_peered":"2024-09-11T09:01:28.501477+0000","last_unstale":"2024-09-11T09:02:47.212501+0000","last_undegraded":"2024-09-11T09:02:47.212501+0000","last_fullsized":"2024-09-11T09:02:47.212501+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'9","last_scrub_stamp":"2024-09-11T09:02:07.768083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:07.768083+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:23.707991+0000","scrub_duration":0.0024195089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":534,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931672+0000","last_change":"2024-09-11T09:02:01.515128+0000","last_active":"2024-09-11T09:02:47.931672+0000","last_peered":"2024-09-11T09:02:47.931672+0000","last_clean":"2024-09-11T09:02:47.931672+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T09:02:47.931672+0000","last_undegraded":"2024-09-11T09:02:47.931672+0000","last_fullsized":"2024-09-11T09:02:47.931672+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:01.515072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:18.199651+0000","last_clean_scrub_stamp":"2024-09-11T09:02:01.515072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:25.439344+0000","scrub_duration":0.001295526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":496,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210282+0000","last_change":"2024-09-11T09:01:37.318284+0000","last_active":"2024-09-11T09:02:47.210282+0000","last_peered":"2024-09-11T09:02:47.210282+0000","last_clean":"2024-09-11T09:02:47.210282+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T09:02:47.210282+0000","last_undegraded":"2024-09-11T09:02:47.210282+0000","last_fullsized":"2024-09-11T09:02:47.210282+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:37.318233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T09:01:37.318233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:55.327410+0000","scrub_duration":0.0012707490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":521,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212884+0000","last_change":"2024-09-11T09:01:35.704717+0000","last_active":"2024-09-11T09:02:47.212884+0000","last_peered":"2024-09-11T09:02:47.212884+0000","last_clean":"2024-09-11T09:02:47.212884+0000","last_became_active":"2024-09-11T08:58:34.126085+0000","last_became_peered":"2024-09-11T08:58:34.126085+0000","last_unstale":"2024-09-11T09:02:47.212884+0000","last_undegraded":"2024-09-11T09:02:47.212884+0000","last_fullsized":"2024-09-11T09:02:47.212884+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:35.704629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:01:35.704629+0000","last_clean_scrub_stamp":"2024-09-11T09:01:35.704629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015126740000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":137,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931969+0000","last_change":"2024-09-11T09:01:35.972995+0000","last_active":"2024-09-11T09:02:47.931969+0000","last_peered":"2024-09-11T09:02:47.931969+0000","last_clean":"2024-09-11T09:02:47.931969+0000","last_became_active":"2024-09-11T09:01:25.404021+0000","last_became_peered":"2024-09-11T09:01:25.404021+0000","last_unstale":"2024-09-11T09:02:47.931969+0000","last_undegraded":"2024-09-11T09:02:47.931969+0000","last_fullsized":"2024-09-11T09:02:47.931969+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:35.972939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:01:35.972939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:03.356777+0000","scrub_duration":0.0014110139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"402'46","reported_seq":426,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931149+0000","last_change":"2024-09-11T09:01:39.553753+0000","last_active":"2024-09-11T09:02:47.931149+0000","last_peered":"2024-09-11T09:02:47.931149+0000","last_clean":"2024-09-11T09:02:47.931149+0000","last_became_active":"2024-09-11T09:01:39.553406+0000","last_became_peered":"2024-09-11T09:01:39.553406+0000","last_unstale":"2024-09-11T09:02:47.931149+0000","last_undegraded":"2024-09-11T09:02:47.931149+0000","last_fullsized":"2024-09-11T09:02:47.931149+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'46","last_scrub_stamp":"2024-09-11T09:01:28.169023+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T09:01:28.169023+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:56.682751+0000","scrub_duration":0.006483768,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":51,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":552,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:48.710058+0000","last_change":"2024-09-11T09:02:48.710014+0000","last_active":"2024-09-11T09:02:48.710058+0000","last_peered":"2024-09-11T09:02:48.710058+0000","last_clean":"2024-09-11T09:02:48.710058+0000","last_became_active":"2024-09-11T08:52:18.579026+0000","last_became_peered":"2024-09-11T08:52:18.579026+0000","last_unstale":"2024-09-11T09:02:48.710058+0000","last_undegraded":"2024-09-11T09:02:48.710058+0000","last_fullsized":"2024-09-11T09:02:48.710058+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:48.709961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:31.860874+0000","last_clean_scrub_stamp":"2024-09-11T09:02:48.709961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:56.818710+0000","scrub_duration":0.0015477080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"419'2","reported_seq":548,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214727+0000","last_change":"2024-09-11T09:02:11.792219+0000","last_active":"2024-09-11T09:02:47.214727+0000","last_peered":"2024-09-11T09:02:47.214727+0000","last_clean":"2024-09-11T09:02:47.214727+0000","last_became_active":"2024-09-11T08:58:34.135094+0000","last_became_peered":"2024-09-11T08:58:34.135094+0000","last_unstale":"2024-09-11T09:02:47.214727+0000","last_undegraded":"2024-09-11T09:02:47.214727+0000","last_fullsized":"2024-09-11T09:02:47.214727+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"419'2","last_scrub_stamp":"2024-09-11T09:02:11.792137+0000","last_deep_scrub":"419'2","last_deep_scrub_stamp":"2024-09-11T08:59:52.794363+0000","last_clean_scrub_stamp":"2024-09-11T09:02:11.792137+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:27.055519+0000","scrub_duration":0.0032077220000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":487,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212576+0000","last_change":"2024-09-11T09:01:18.943321+0000","last_active":"2024-09-11T09:02:47.212576+0000","last_peered":"2024-09-11T09:02:47.212576+0000","last_clean":"2024-09-11T09:02:47.212576+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T09:02:47.212576+0000","last_undegraded":"2024-09-11T09:02:47.212576+0000","last_fullsized":"2024-09-11T09:02:47.212576+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:18.943265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:01:18.943265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001474944,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"402'24","reported_seq":557,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214755+0000","last_change":"2024-09-11T09:02:24.851545+0000","last_active":"2024-09-11T09:02:47.214755+0000","last_peered":"2024-09-11T09:02:47.214755+0000","last_clean":"2024-09-11T09:02:47.214755+0000","last_became_active":"2024-09-11T08:49:16.943886+0000","last_became_peered":"2024-09-11T08:49:16.943886+0000","last_unstale":"2024-09-11T09:02:47.214755+0000","last_undegraded":"2024-09-11T09:02:47.214755+0000","last_fullsized":"2024-09-11T09:02:47.214755+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'24","last_scrub_stamp":"2024-09-11T09:02:24.851487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:24.851487+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:31.266405+0000","scrub_duration":0.002063998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":493,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212478+0000","last_change":"2024-09-11T09:02:27.820405+0000","last_active":"2024-09-11T09:02:47.212478+0000","last_peered":"2024-09-11T09:02:47.212478+0000","last_clean":"2024-09-11T09:02:47.212478+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T09:02:47.212478+0000","last_undegraded":"2024-09-11T09:02:47.212478+0000","last_fullsized":"2024-09-11T09:02:47.212478+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:27.820336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:02:27.820336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:56.964184+0000","scrub_duration":0.001624096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":308,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932155+0000","last_change":"2024-09-11T09:02:34.325436+0000","last_active":"2024-09-11T09:02:47.932155+0000","last_peered":"2024-09-11T09:02:47.932155+0000","last_clean":"2024-09-11T09:02:47.932155+0000","last_became_active":"2024-09-11T09:02:16.803219+0000","last_became_peered":"2024-09-11T09:02:16.803219+0000","last_unstale":"2024-09-11T09:02:47.932155+0000","last_undegraded":"2024-09-11T09:02:47.932155+0000","last_fullsized":"2024-09-11T09:02:47.932155+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T09:02:34.325356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:34.325356+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:38.320294+0000","scrub_duration":0.0018443520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":526,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930756+0000","last_change":"2024-09-11T09:01:54.321984+0000","last_active":"2024-09-11T09:02:47.930756+0000","last_peered":"2024-09-11T09:02:47.930756+0000","last_clean":"2024-09-11T09:02:47.930756+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T09:02:47.930756+0000","last_undegraded":"2024-09-11T09:02:47.930756+0000","last_fullsized":"2024-09-11T09:02:47.930756+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:54.321929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T09:01:54.321929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:19.000474+0000","scrub_duration":0.0014839110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"402'35","reported_seq":223,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932658+0000","last_change":"2024-09-11T09:01:57.879021+0000","last_active":"2024-09-11T09:02:47.932658+0000","last_peered":"2024-09-11T09:02:47.932658+0000","last_clean":"2024-09-11T09:02:47.932658+0000","last_became_active":"2024-09-11T09:01:25.406496+0000","last_became_peered":"2024-09-11T09:01:25.406496+0000","last_unstale":"2024-09-11T09:02:47.932658+0000","last_undegraded":"2024-09-11T09:02:47.932658+0000","last_fullsized":"2024-09-11T09:02:47.932658+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'35","last_scrub_stamp":"2024-09-11T09:01:57.878967+0000","last_deep_scrub":"280'25","last_deep_scrub_stamp":"2024-09-11T08:52:24.584533+0000","last_clean_scrub_stamp":"2024-09-11T09:01:57.878967+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:13.854861+0000","scrub_duration":0.001700167,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"457'12","reported_seq":175,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:52.946526+0000","last_change":"2024-09-11T09:02:18.745062+0000","last_active":"2024-09-11T09:02:52.946526+0000","last_peered":"2024-09-11T09:02:52.946526+0000","last_clean":"2024-09-11T09:02:52.946526+0000","last_became_active":"2024-09-11T09:02:09.769212+0000","last_became_peered":"2024-09-11T09:02:09.769212+0000","last_unstale":"2024-09-11T09:02:52.946526+0000","last_undegraded":"2024-09-11T09:02:52.946526+0000","last_fullsized":"2024-09-11T09:02:52.946526+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'12","last_scrub_stamp":"2024-09-11T09:02:18.744984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:02:18.744984+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:22.092997+0000","scrub_duration":0.001728668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"199'81","reported_seq":283,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210110+0000","last_change":"2024-09-11T09:01:59.346242+0000","last_active":"2024-09-11T09:02:47.210110+0000","last_peered":"2024-09-11T09:02:47.210110+0000","last_clean":"2024-09-11T09:02:47.210110+0000","last_became_active":"2024-09-11T08:58:34.313408+0000","last_became_peered":"2024-09-11T08:58:34.313408+0000","last_unstale":"2024-09-11T09:02:47.210110+0000","last_undegraded":"2024-09-11T09:02:47.210110+0000","last_fullsized":"2024-09-11T09:02:47.210110+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:01:59.346179+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:01:59.346179+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:24.232572+0000","scrub_duration":0.00144864,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"402'46","reported_seq":595,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932229+0000","last_change":"2024-09-11T09:02:25.476477+0000","last_active":"2024-09-11T09:02:47.932229+0000","last_peered":"2024-09-11T09:02:47.932229+0000","last_clean":"2024-09-11T09:02:47.932229+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T09:02:47.932229+0000","last_undegraded":"2024-09-11T09:02:47.932229+0000","last_fullsized":"2024-09-11T09:02:47.932229+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'46","last_scrub_stamp":"2024-09-11T09:02:25.476423+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T09:02:25.476423+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:47.010191+0000","scrub_duration":0.0019200389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":496,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214276+0000","last_change":"2024-09-11T09:02:19.879820+0000","last_active":"2024-09-11T09:02:47.214276+0000","last_peered":"2024-09-11T09:02:47.214276+0000","last_clean":"2024-09-11T09:02:47.214276+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T09:02:47.214276+0000","last_undegraded":"2024-09-11T09:02:47.214276+0000","last_fullsized":"2024-09-11T09:02:47.214276+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:19.879733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:19.879733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:32.453233+0000","scrub_duration":0.0013971019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"277'11","reported_seq":822,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:55.294225+0000","last_change":"2024-09-11T09:02:21.792473+0000","last_active":"2024-09-11T09:02:55.294225+0000","last_peered":"2024-09-11T09:02:55.294225+0000","last_clean":"2024-09-11T09:02:55.294225+0000","last_became_active":"2024-09-11T08:52:18.578328+0000","last_became_peered":"2024-09-11T08:52:18.578328+0000","last_unstale":"2024-09-11T09:02:55.294225+0000","last_undegraded":"2024-09-11T09:02:55.294225+0000","last_fullsized":"2024-09-11T09:02:55.294225+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'11","last_scrub_stamp":"2024-09-11T09:02:21.792402+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T09:02:21.792402+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:50.492725+0000","scrub_duration":0.0017782640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":539,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210503+0000","last_change":"2024-09-11T09:02:02.279129+0000","last_active":"2024-09-11T09:02:47.210503+0000","last_peered":"2024-09-11T09:02:47.210503+0000","last_clean":"2024-09-11T09:02:47.210503+0000","last_became_active":"2024-09-11T09:01:28.499311+0000","last_became_peered":"2024-09-11T09:01:28.499311+0000","last_unstale":"2024-09-11T09:02:47.210503+0000","last_undegraded":"2024-09-11T09:02:47.210503+0000","last_fullsized":"2024-09-11T09:02:47.210503+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:02.279076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:02.279076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:24.854919+0000","scrub_duration":0.0015422739999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"199'81","reported_seq":293,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:53.847396+0000","last_change":"2024-09-11T09:02:53.847350+0000","last_active":"2024-09-11T09:02:53.847396+0000","last_peered":"2024-09-11T09:02:53.847396+0000","last_clean":"2024-09-11T09:02:53.847396+0000","last_became_active":"2024-09-11T08:58:34.324527+0000","last_became_peered":"2024-09-11T08:58:34.324527+0000","last_unstale":"2024-09-11T09:02:53.847396+0000","last_undegraded":"2024-09-11T09:02:53.847396+0000","last_fullsized":"2024-09-11T09:02:53.847396+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:53.847283+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T09:00:52.859779+0000","last_clean_scrub_stamp":"2024-09-11T09:02:53.847283+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:03.846789+0000","scrub_duration":0.001609826,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"402'50","reported_seq":626,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214121+0000","last_change":"2024-09-11T09:01:55.795454+0000","last_active":"2024-09-11T09:02:47.214121+0000","last_peered":"2024-09-11T09:02:47.214121+0000","last_clean":"2024-09-11T09:02:47.214121+0000","last_became_active":"2024-09-11T08:52:18.595013+0000","last_became_peered":"2024-09-11T08:52:18.595013+0000","last_unstale":"2024-09-11T09:02:47.214121+0000","last_undegraded":"2024-09-11T09:02:47.214121+0000","last_fullsized":"2024-09-11T09:02:47.214121+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'50","last_scrub_stamp":"2024-09-11T09:01:55.795401+0000","last_deep_scrub":"199'26","last_deep_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_clean_scrub_stamp":"2024-09-11T09:01:55.795401+0000","objects_scrubbed":8,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:14.858041+0000","scrub_duration":0.0024891940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":519,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214970+0000","last_change":"2024-09-11T09:02:02.807138+0000","last_active":"2024-09-11T09:02:47.214970+0000","last_peered":"2024-09-11T09:02:47.214970+0000","last_clean":"2024-09-11T09:02:47.214970+0000","last_became_active":"2024-09-11T08:52:18.586084+0000","last_became_peered":"2024-09-11T08:52:18.586084+0000","last_unstale":"2024-09-11T09:02:47.214970+0000","last_undegraded":"2024-09-11T09:02:47.214970+0000","last_fullsized":"2024-09-11T09:02:47.214970+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:02.807084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:58:52.524310+0000","last_clean_scrub_stamp":"2024-09-11T09:02:02.807084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:13.559910+0000","scrub_duration":0.0014273700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":493,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212095+0000","last_change":"2024-09-11T09:02:30.830289+0000","last_active":"2024-09-11T09:02:47.212095+0000","last_peered":"2024-09-11T09:02:47.212095+0000","last_clean":"2024-09-11T09:02:47.212095+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T09:02:47.212095+0000","last_undegraded":"2024-09-11T09:02:47.212095+0000","last_fullsized":"2024-09-11T09:02:47.212095+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:30.830227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:00:49.851783+0000","last_clean_scrub_stamp":"2024-09-11T09:02:30.830227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:41.684266+0000","scrub_duration":0.0014479510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":491,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:03:00.815289+0000","last_change":"2024-09-11T09:03:00.815258+0000","last_active":"2024-09-11T09:03:00.815289+0000","last_peered":"2024-09-11T09:03:00.815289+0000","last_clean":"2024-09-11T09:03:00.815289+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T09:03:00.815289+0000","last_undegraded":"2024-09-11T09:03:00.815289+0000","last_fullsized":"2024-09-11T09:03:00.815289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:03:00.815205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:01:37.682866+0000","last_clean_scrub_stamp":"2024-09-11T09:03:00.815205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:02.910968+0000","scrub_duration":0.0014020619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"199'81","reported_seq":278,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214494+0000","last_change":"2024-09-11T09:02:42.011760+0000","last_active":"2024-09-11T09:02:47.214494+0000","last_peered":"2024-09-11T09:02:47.214494+0000","last_clean":"2024-09-11T09:02:47.214494+0000","last_became_active":"2024-09-11T08:58:01.125235+0000","last_became_peered":"2024-09-11T08:58:01.125235+0000","last_unstale":"2024-09-11T09:02:47.214494+0000","last_undegraded":"2024-09-11T09:02:47.214494+0000","last_fullsized":"2024-09-11T09:02:47.214494+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:42.011689+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:42.011689+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:48.723451+0000","scrub_duration":0.0013268659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"402'49","reported_seq":610,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212265+0000","last_change":"2024-09-11T09:02:13.712585+0000","last_active":"2024-09-11T09:02:47.212265+0000","last_peered":"2024-09-11T09:02:47.212265+0000","last_clean":"2024-09-11T09:02:47.212265+0000","last_became_active":"2024-09-11T08:58:34.127547+0000","last_became_peered":"2024-09-11T08:58:34.127547+0000","last_unstale":"2024-09-11T09:02:47.212265+0000","last_undegraded":"2024-09-11T09:02:47.212265+0000","last_fullsized":"2024-09-11T09:02:47.212265+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'49","last_scrub_stamp":"2024-09-11T09:02:13.712514+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T09:02:13.712514+0000","objects_scrubbed":7,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:22.194203+0000","scrub_duration":0.0021497080000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":63,"num_read_kb":56,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":269,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932721+0000","last_change":"2024-09-11T09:02:18.832369+0000","last_active":"2024-09-11T09:02:47.932721+0000","last_peered":"2024-09-11T09:02:47.932721+0000","last_clean":"2024-09-11T09:02:47.932721+0000","last_became_active":"2024-09-11T08:52:18.586549+0000","last_became_peered":"2024-09-11T08:52:18.586549+0000","last_unstale":"2024-09-11T09:02:47.932721+0000","last_undegraded":"2024-09-11T09:02:47.932721+0000","last_fullsized":"2024-09-11T09:02:47.932721+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:18.832318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:28.487573+0000","last_clean_scrub_stamp":"2024-09-11T09:02:18.832318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:45.872311+0000","scrub_duration":0.001382568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"440'9","reported_seq":796,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:55.900130+0000","last_change":"2024-09-11T09:02:55.900092+0000","last_active":"2024-09-11T09:02:55.900130+0000","last_peered":"2024-09-11T09:02:55.900130+0000","last_clean":"2024-09-11T09:02:55.900130+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T09:02:55.900130+0000","last_undegraded":"2024-09-11T09:02:55.900130+0000","last_fullsized":"2024-09-11T09:02:55.900130+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"440'9","last_scrub_stamp":"2024-09-11T09:02:55.900035+0000","last_deep_scrub":"440'9","last_deep_scrub_stamp":"2024-09-11T09:01:25.808118+0000","last_clean_scrub_stamp":"2024-09-11T09:02:55.900035+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:02.572075+0000","scrub_duration":0.0018779599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":327,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:49.467388+0000","last_change":"2024-09-11T09:02:49.467337+0000","last_active":"2024-09-11T09:02:49.467388+0000","last_peered":"2024-09-11T09:02:49.467388+0000","last_clean":"2024-09-11T09:02:49.467388+0000","last_became_active":"2024-09-11T09:02:16.798218+0000","last_became_peered":"2024-09-11T09:02:16.798218+0000","last_unstale":"2024-09-11T09:02:49.467388+0000","last_undegraded":"2024-09-11T09:02:49.467388+0000","last_fullsized":"2024-09-11T09:02:49.467388+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:49.467264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:49.467264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:06.612572+0000","scrub_duration":0.001586795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"199'81","reported_seq":372,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932066+0000","last_change":"2024-09-11T09:02:30.900172+0000","last_active":"2024-09-11T09:02:47.932066+0000","last_peered":"2024-09-11T09:02:47.932066+0000","last_clean":"2024-09-11T09:02:47.932066+0000","last_became_active":"2024-09-11T09:02:09.772929+0000","last_became_peered":"2024-09-11T09:02:09.772929+0000","last_unstale":"2024-09-11T09:02:47.932066+0000","last_undegraded":"2024-09-11T09:02:47.932066+0000","last_fullsized":"2024-09-11T09:02:47.932066+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:30.900115+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:30.900115+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:00.807339+0000","scrub_duration":0.0012564060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"402'35","reported_seq":54,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930784+0000","last_change":"2024-09-11T09:01:51.297495+0000","last_active":"2024-09-11T09:02:47.930784+0000","last_peered":"2024-09-11T09:02:47.930784+0000","last_clean":"2024-09-11T09:02:47.930784+0000","last_became_active":"2024-09-11T09:01:28.502156+0000","last_became_peered":"2024-09-11T09:01:28.502156+0000","last_unstale":"2024-09-11T09:02:47.930784+0000","last_undegraded":"2024-09-11T09:02:47.930784+0000","last_fullsized":"2024-09-11T09:02:47.930784+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'35","last_scrub_stamp":"2024-09-11T09:01:51.297442+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_clean_scrub_stamp":"2024-09-11T09:01:51.297442+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:55.420882+0000","scrub_duration":0.0022843949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":502,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210914+0000","last_change":"2024-09-11T09:01:48.343306+0000","last_active":"2024-09-11T09:02:47.210914+0000","last_peered":"2024-09-11T09:02:47.210914+0000","last_clean":"2024-09-11T09:02:47.210914+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T09:02:47.210914+0000","last_undegraded":"2024-09-11T09:02:47.210914+0000","last_fullsized":"2024-09-11T09:02:47.210914+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:48.343238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:01:48.343238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001431047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"448'16","reported_seq":98,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:52.946368+0000","last_change":"2024-09-11T09:01:45.930676+0000","last_active":"2024-09-11T09:02:52.946368+0000","last_peered":"2024-09-11T09:02:52.946368+0000","last_clean":"2024-09-11T09:02:52.946368+0000","last_became_active":"2024-09-11T09:01:25.399797+0000","last_became_peered":"2024-09-11T09:01:25.399797+0000","last_unstale":"2024-09-11T09:02:52.946368+0000","last_undegraded":"2024-09-11T09:02:52.946368+0000","last_fullsized":"2024-09-11T09:02:52.946368+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"448'16","last_scrub_stamp":"2024-09-11T09:01:45.930603+0000","last_deep_scrub":"448'16","last_deep_scrub_stamp":"2024-09-11T09:01:45.930603+0000","last_clean_scrub_stamp":"2024-09-11T09:01:45.930603+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:02.286957+0000","scrub_duration":0.002995836,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":137,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932351+0000","last_change":"2024-09-11T09:02:38.967684+0000","last_active":"2024-09-11T09:02:47.932351+0000","last_peered":"2024-09-11T09:02:47.932351+0000","last_clean":"2024-09-11T09:02:47.932351+0000","last_became_active":"2024-09-11T09:01:25.400182+0000","last_became_peered":"2024-09-11T09:01:25.400182+0000","last_unstale":"2024-09-11T09:02:47.932351+0000","last_undegraded":"2024-09-11T09:02:47.932351+0000","last_fullsized":"2024-09-11T09:02:47.932351+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:38.967626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:38.967626+0000","last_clean_scrub_stamp":"2024-09-11T09:02:38.967626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:54.059817+0000","scrub_duration":0.0010909590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"199'81","reported_seq":372,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932125+0000","last_change":"2024-09-11T09:02:27.819558+0000","last_active":"2024-09-11T09:02:47.932125+0000","last_peered":"2024-09-11T09:02:47.932125+0000","last_clean":"2024-09-11T09:02:47.932125+0000","last_became_active":"2024-09-11T09:02:09.767659+0000","last_became_peered":"2024-09-11T09:02:09.767659+0000","last_unstale":"2024-09-11T09:02:47.932125+0000","last_undegraded":"2024-09-11T09:02:47.932125+0000","last_fullsized":"2024-09-11T09:02:47.932125+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:27.819487+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T09:00:57.089686+0000","last_clean_scrub_stamp":"2024-09-11T09:02:27.819487+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:34.788158+0000","scrub_duration":0.0019620150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"402'39","reported_seq":538,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931766+0000","last_change":"2024-09-11T09:02:22.824681+0000","last_active":"2024-09-11T09:02:47.931766+0000","last_peered":"2024-09-11T09:02:47.931766+0000","last_clean":"2024-09-11T09:02:47.931766+0000","last_became_active":"2024-09-11T09:01:25.404892+0000","last_became_peered":"2024-09-11T09:01:25.404892+0000","last_unstale":"2024-09-11T09:02:47.931766+0000","last_undegraded":"2024-09-11T09:02:47.931766+0000","last_fullsized":"2024-09-11T09:02:47.931766+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'39","last_scrub_stamp":"2024-09-11T09:02:22.824392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:22.824392+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:40.236394+0000","scrub_duration":0.0081303699999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":67,"num_read_kb":53,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":536,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931478+0000","last_change":"2024-09-11T09:02:04.348546+0000","last_active":"2024-09-11T09:02:47.931478+0000","last_peered":"2024-09-11T09:02:47.931478+0000","last_clean":"2024-09-11T09:02:47.931478+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T09:02:47.931478+0000","last_undegraded":"2024-09-11T09:02:47.931478+0000","last_fullsized":"2024-09-11T09:02:47.931478+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:04.348493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:04.348493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:10.942122+0000","scrub_duration":0.001483713,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":409,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930999+0000","last_change":"2024-09-11T09:02:09.768372+0000","last_active":"2024-09-11T09:02:47.930999+0000","last_peered":"2024-09-11T09:02:47.930999+0000","last_clean":"2024-09-11T09:02:47.930999+0000","last_became_active":"2024-09-11T09:02:09.768092+0000","last_became_peered":"2024-09-11T09:02:09.768092+0000","last_unstale":"2024-09-11T09:02:47.930999+0000","last_undegraded":"2024-09-11T09:02:47.930999+0000","last_fullsized":"2024-09-11T09:02:47.930999+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:52.871083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_clean_scrub_stamp":"2024-09-11T09:01:52.871083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:00.827308+0000","scrub_duration":0.011823264,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":573,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930344+0000","last_change":"2024-09-11T09:02:42.691486+0000","last_active":"2024-09-11T09:02:47.930344+0000","last_peered":"2024-09-11T09:02:47.930344+0000","last_clean":"2024-09-11T09:02:47.930344+0000","last_became_active":"2024-09-11T08:58:34.315290+0000","last_became_peered":"2024-09-11T08:58:34.315290+0000","last_unstale":"2024-09-11T09:02:47.930344+0000","last_undegraded":"2024-09-11T09:02:47.930344+0000","last_fullsized":"2024-09-11T09:02:47.930344+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:42.691434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:01:16.084845+0000","last_clean_scrub_stamp":"2024-09-11T09:02:42.691434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:11.902949+0000","scrub_duration":0.0014238320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"199'81","reported_seq":283,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210047+0000","last_change":"2024-09-11T09:02:11.272886+0000","last_active":"2024-09-11T09:02:47.210047+0000","last_peered":"2024-09-11T09:02:47.210047+0000","last_clean":"2024-09-11T09:02:47.210047+0000","last_became_active":"2024-09-11T08:58:01.120225+0000","last_became_peered":"2024-09-11T08:58:01.120225+0000","last_unstale":"2024-09-11T09:02:47.210047+0000","last_undegraded":"2024-09-11T09:02:47.210047+0000","last_fullsized":"2024-09-11T09:02:47.210047+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:11.272812+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:11.272812+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:31.608940+0000","scrub_duration":0.0016332790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"442'66","reported_seq":82,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:52.946396+0000","last_change":"2024-09-11T09:02:09.314628+0000","last_active":"2024-09-11T09:02:52.946396+0000","last_peered":"2024-09-11T09:02:52.946396+0000","last_clean":"2024-09-11T09:02:52.946396+0000","last_became_active":"2024-09-11T09:01:28.505013+0000","last_became_peered":"2024-09-11T09:01:28.505013+0000","last_unstale":"2024-09-11T09:02:52.946396+0000","last_undegraded":"2024-09-11T09:02:52.946396+0000","last_fullsized":"2024-09-11T09:02:52.946396+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"442'66","last_scrub_stamp":"2024-09-11T09:02:09.314564+0000","last_deep_scrub":"200'50","last_deep_scrub_stamp":"2024-09-11T08:53:09.745322+0000","last_clean_scrub_stamp":"2024-09-11T09:02:09.314564+0000","objects_scrubbed":10,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:32.955018+0000","scrub_duration":0.0022008150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":537,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212983+0000","last_change":"2024-09-11T09:02:06.785768+0000","last_active":"2024-09-11T09:02:47.212983+0000","last_peered":"2024-09-11T09:02:47.212983+0000","last_clean":"2024-09-11T09:02:47.212983+0000","last_became_active":"2024-09-11T08:58:34.136762+0000","last_became_peered":"2024-09-11T08:58:34.136762+0000","last_unstale":"2024-09-11T09:02:47.212983+0000","last_undegraded":"2024-09-11T09:02:47.212983+0000","last_fullsized":"2024-09-11T09:02:47.212983+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:06.785697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_clean_scrub_stamp":"2024-09-11T09:02:06.785697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:20.724085+0000","scrub_duration":0.0017665090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":575,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210509+0000","last_change":"2024-09-11T09:02:28.296073+0000","last_active":"2024-09-11T09:02:47.210509+0000","last_peered":"2024-09-11T09:02:47.210509+0000","last_clean":"2024-09-11T09:02:47.210509+0000","last_became_active":"2024-09-11T08:58:34.138191+0000","last_became_peered":"2024-09-11T08:58:34.138191+0000","last_unstale":"2024-09-11T09:02:47.210509+0000","last_undegraded":"2024-09-11T09:02:47.210509+0000","last_fullsized":"2024-09-11T09:02:47.210509+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T09:02:28.295996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:28.295996+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:53.153378+0000","scrub_duration":0.0021093420000000002,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":137,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932439+0000","last_change":"2024-09-11T09:02:40.942123+0000","last_active":"2024-09-11T09:02:47.932439+0000","last_peered":"2024-09-11T09:02:47.932439+0000","last_clean":"2024-09-11T09:02:47.932439+0000","last_became_active":"2024-09-11T09:01:25.420792+0000","last_became_peered":"2024-09-11T09:01:25.420792+0000","last_unstale":"2024-09-11T09:02:47.932439+0000","last_undegraded":"2024-09-11T09:02:47.932439+0000","last_fullsized":"2024-09-11T09:02:47.932439+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:40.942069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:02:40.942069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:05.731958+0000","scrub_duration":0.001662312,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":490,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214615+0000","last_change":"2024-09-11T09:02:26.773611+0000","last_active":"2024-09-11T09:02:47.214615+0000","last_peered":"2024-09-11T09:02:47.214615+0000","last_clean":"2024-09-11T09:02:47.214615+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T09:02:47.214615+0000","last_undegraded":"2024-09-11T09:02:47.214615+0000","last_fullsized":"2024-09-11T09:02:47.214615+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:26.773556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:26.773556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:49.354353+0000","scrub_duration":0.0016365150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"199'81","reported_seq":405,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931495+0000","last_change":"2024-09-11T09:02:28.310057+0000","last_active":"2024-09-11T09:02:47.931495+0000","last_peered":"2024-09-11T09:02:47.931495+0000","last_clean":"2024-09-11T09:02:47.931495+0000","last_became_active":"2024-09-11T09:02:16.797286+0000","last_became_peered":"2024-09-11T09:02:16.797286+0000","last_unstale":"2024-09-11T09:02:47.931495+0000","last_undegraded":"2024-09-11T09:02:47.931495+0000","last_fullsized":"2024-09-11T09:02:47.931495+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:28.309987+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T09:02:28.309987+0000","last_clean_scrub_stamp":"2024-09-11T09:02:28.309987+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:52.189981+0000","scrub_duration":0.001601211,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"402'21","reported_seq":64,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:50.753409+0000","last_change":"2024-09-11T09:02:50.753355+0000","last_active":"2024-09-11T09:02:50.753409+0000","last_peered":"2024-09-11T09:02:50.753409+0000","last_clean":"2024-09-11T09:02:50.753409+0000","last_became_active":"2024-09-11T09:01:28.499434+0000","last_became_peered":"2024-09-11T09:01:28.499434+0000","last_unstale":"2024-09-11T09:02:50.753409+0000","last_undegraded":"2024-09-11T09:02:50.753409+0000","last_fullsized":"2024-09-11T09:02:50.753409+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'21","last_scrub_stamp":"2024-09-11T09:02:50.753291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:50.753291+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:12.239614+0000","scrub_duration":0.0022493639999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":29,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":572,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930342+0000","last_change":"2024-09-11T09:01:55.352768+0000","last_active":"2024-09-11T09:02:47.930342+0000","last_peered":"2024-09-11T09:02:47.930342+0000","last_clean":"2024-09-11T09:02:47.930342+0000","last_became_active":"2024-09-11T08:58:34.311119+0000","last_became_peered":"2024-09-11T08:58:34.311119+0000","last_unstale":"2024-09-11T09:02:47.930342+0000","last_undegraded":"2024-09-11T09:02:47.930342+0000","last_fullsized":"2024-09-11T09:02:47.930342+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:55.352709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:01:55.352709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:04.203148+0000","scrub_duration":0.001469171,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"263'2","reported_seq":607,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931105+0000","last_change":"2024-09-11T09:02:23.661666+0000","last_active":"2024-09-11T09:02:47.931105+0000","last_peered":"2024-09-11T09:02:47.931105+0000","last_clean":"2024-09-11T09:02:47.931105+0000","last_became_active":"2024-09-11T08:52:18.586545+0000","last_became_peered":"2024-09-11T08:52:18.586545+0000","last_unstale":"2024-09-11T09:02:47.931105+0000","last_undegraded":"2024-09-11T09:02:47.931105+0000","last_fullsized":"2024-09-11T09:02:47.931105+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'2","last_scrub_stamp":"2024-09-11T09:02:23.661568+0000","last_deep_scrub":"263'2","last_deep_scrub_stamp":"2024-09-11T08:53:08.772979+0000","last_clean_scrub_stamp":"2024-09-11T09:02:23.661568+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:53.535710+0000","scrub_duration":0.0016331290000000001,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":557,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931470+0000","last_change":"2024-09-11T09:02:02.558793+0000","last_active":"2024-09-11T09:02:47.931470+0000","last_peered":"2024-09-11T09:02:47.931470+0000","last_clean":"2024-09-11T09:02:47.931470+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T09:02:47.931470+0000","last_undegraded":"2024-09-11T09:02:47.931470+0000","last_fullsized":"2024-09-11T09:02:47.931470+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:02.558740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:58:50.901121+0000","last_clean_scrub_stamp":"2024-09-11T09:02:02.558740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:05.166107+0000","scrub_duration":0.0014589900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":337,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932111+0000","last_change":"2024-09-11T09:02:20.438199+0000","last_active":"2024-09-11T09:02:47.932111+0000","last_peered":"2024-09-11T09:02:47.932111+0000","last_clean":"2024-09-11T09:02:47.932111+0000","last_became_active":"2024-09-11T09:02:16.796628+0000","last_became_peered":"2024-09-11T09:02:16.796628+0000","last_unstale":"2024-09-11T09:02:47.932111+0000","last_undegraded":"2024-09-11T09:02:47.932111+0000","last_fullsized":"2024-09-11T09:02:47.932111+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:20.438132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:20.438132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:45.057300+0000","scrub_duration":0.001693908,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"199'81","reported_seq":576,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930990+0000","last_change":"2024-09-11T09:02:32.595769+0000","last_active":"2024-09-11T09:02:47.930990+0000","last_peered":"2024-09-11T09:02:47.930990+0000","last_clean":"2024-09-11T09:02:47.930990+0000","last_became_active":"2024-09-11T08:58:34.130607+0000","last_became_peered":"2024-09-11T08:58:34.130607+0000","last_unstale":"2024-09-11T09:02:47.930990+0000","last_undegraded":"2024-09-11T09:02:47.930990+0000","last_fullsized":"2024-09-11T09:02:47.930990+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:32.595714+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:32.595714+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:36.254682+0000","scrub_duration":0.001862371,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"402'36","reported_seq":298,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932597+0000","last_change":"2024-09-11T09:02:25.858393+0000","last_active":"2024-09-11T09:02:47.932597+0000","last_peered":"2024-09-11T09:02:47.932597+0000","last_clean":"2024-09-11T09:02:47.932597+0000","last_became_active":"2024-09-11T08:52:18.580950+0000","last_became_peered":"2024-09-11T08:52:18.580950+0000","last_unstale":"2024-09-11T09:02:47.932597+0000","last_undegraded":"2024-09-11T09:02:47.932597+0000","last_fullsized":"2024-09-11T09:02:47.932597+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'36","last_scrub_stamp":"2024-09-11T09:02:25.858325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:25.858325+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:29.622507+0000","scrub_duration":0.0020603610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":47,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":186,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930758+0000","last_change":"2024-09-11T09:02:22.827670+0000","last_active":"2024-09-11T09:02:47.930758+0000","last_peered":"2024-09-11T09:02:47.930758+0000","last_clean":"2024-09-11T09:02:47.930758+0000","last_became_active":"2024-09-11T09:02:09.765000+0000","last_became_peered":"2024-09-11T09:02:09.765000+0000","last_unstale":"2024-09-11T09:02:47.930758+0000","last_undegraded":"2024-09-11T09:02:47.930758+0000","last_fullsized":"2024-09-11T09:02:47.930758+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:22.827603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:22.827603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:40.390028+0000","scrub_duration":0.002742179,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"318'6","reported_seq":237,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:55.293121+0000","last_change":"2024-09-11T09:02:16.801653+0000","last_active":"2024-09-11T09:02:55.293121+0000","last_peered":"2024-09-11T09:02:55.293121+0000","last_clean":"2024-09-11T09:02:55.293121+0000","last_became_active":"2024-09-11T09:02:16.801234+0000","last_became_peered":"2024-09-11T09:02:16.801234+0000","last_unstale":"2024-09-11T09:02:55.293121+0000","last_undegraded":"2024-09-11T09:02:55.293121+0000","last_fullsized":"2024-09-11T09:02:55.293121+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'6","last_scrub_stamp":"2024-09-11T09:01:34.145178+0000","last_deep_scrub":"318'6","last_deep_scrub_stamp":"2024-09-11T08:54:12.873653+0000","last_clean_scrub_stamp":"2024-09-11T09:01:34.145178+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:57.940712+0000","scrub_duration":0.0017088629999999999,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":135,"num_read_kb":130,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":528,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.211287+0000","last_change":"2024-09-11T09:02:37.230413+0000","last_active":"2024-09-11T09:02:47.211287+0000","last_peered":"2024-09-11T09:02:47.211287+0000","last_clean":"2024-09-11T09:02:47.211287+0000","last_became_active":"2024-09-11T08:52:18.578530+0000","last_became_peered":"2024-09-11T08:52:18.578530+0000","last_unstale":"2024-09-11T09:02:47.211287+0000","last_undegraded":"2024-09-11T09:02:47.211287+0000","last_fullsized":"2024-09-11T09:02:47.211287+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:37.230344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:01:29.426515+0000","last_clean_scrub_stamp":"2024-09-11T09:02:37.230344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:57.375943+0000","scrub_duration":0.001363782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":42,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931286+0000","last_change":"2024-09-11T09:01:40.070553+0000","last_active":"2024-09-11T09:02:47.931286+0000","last_peered":"2024-09-11T09:02:47.931286+0000","last_clean":"2024-09-11T09:02:47.931286+0000","last_became_active":"2024-09-11T09:01:28.502749+0000","last_became_peered":"2024-09-11T09:01:28.502749+0000","last_unstale":"2024-09-11T09:02:47.931286+0000","last_undegraded":"2024-09-11T09:02:47.931286+0000","last_fullsized":"2024-09-11T09:02:47.931286+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:40.070501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T09:01:40.070501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014735569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"402'39","reported_seq":259,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932049+0000","last_change":"2024-09-11T09:02:13.800886+0000","last_active":"2024-09-11T09:02:47.932049+0000","last_peered":"2024-09-11T09:02:47.932049+0000","last_clean":"2024-09-11T09:02:47.932049+0000","last_became_active":"2024-09-11T09:02:09.772033+0000","last_became_peered":"2024-09-11T09:02:09.772033+0000","last_unstale":"2024-09-11T09:02:47.932049+0000","last_undegraded":"2024-09-11T09:02:47.932049+0000","last_fullsized":"2024-09-11T09:02:47.932049+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'39","last_scrub_stamp":"2024-09-11T09:02:13.800816+0000","last_deep_scrub":"201'27","last_deep_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_clean_scrub_stamp":"2024-09-11T09:02:13.800816+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:23.711614+0000","scrub_duration":0.0021351090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":59,"num_read_kb":49,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":533,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214370+0000","last_change":"2024-09-11T09:02:37.948332+0000","last_active":"2024-09-11T09:02:47.214370+0000","last_peered":"2024-09-11T09:02:47.214370+0000","last_clean":"2024-09-11T09:02:47.214370+0000","last_became_active":"2024-09-11T08:58:34.135492+0000","last_became_peered":"2024-09-11T08:58:34.135492+0000","last_unstale":"2024-09-11T09:02:47.214370+0000","last_undegraded":"2024-09-11T09:02:47.214370+0000","last_fullsized":"2024-09-11T09:02:47.214370+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T09:02:37.948275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:37.948275+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:43.681244+0000","scrub_duration":0.0018152579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1748,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"199'81","reported_seq":560,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931375+0000","last_change":"2024-09-11T09:02:33.363939+0000","last_active":"2024-09-11T09:02:47.931375+0000","last_peered":"2024-09-11T09:02:47.931375+0000","last_clean":"2024-09-11T09:02:47.931375+0000","last_became_active":"2024-09-11T09:02:16.797435+0000","last_became_peered":"2024-09-11T09:02:16.797435+0000","last_unstale":"2024-09-11T09:02:47.931375+0000","last_undegraded":"2024-09-11T09:02:47.931375+0000","last_fullsized":"2024-09-11T09:02:47.931375+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:33.363867+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:55:08.518651+0000","last_clean_scrub_stamp":"2024-09-11T09:02:33.363867+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:33.671552+0000","scrub_duration":0.0015999829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":319,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932205+0000","last_change":"2024-09-11T09:02:41.300284+0000","last_active":"2024-09-11T09:02:47.932205+0000","last_peered":"2024-09-11T09:02:47.932205+0000","last_clean":"2024-09-11T09:02:47.932205+0000","last_became_active":"2024-09-11T09:02:16.802882+0000","last_became_peered":"2024-09-11T09:02:16.802882+0000","last_unstale":"2024-09-11T09:02:47.932205+0000","last_undegraded":"2024-09-11T09:02:47.932205+0000","last_fullsized":"2024-09-11T09:02:47.932205+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:41.300221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:41.300221+0000","last_clean_scrub_stamp":"2024-09-11T09:02:41.300221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:06.538379+0000","scrub_duration":0.0012995229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":506,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.211217+0000","last_change":"2024-09-11T09:02:07.276096+0000","last_active":"2024-09-11T09:02:47.211217+0000","last_peered":"2024-09-11T09:02:47.211217+0000","last_clean":"2024-09-11T09:02:47.211217+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T09:02:47.211217+0000","last_undegraded":"2024-09-11T09:02:47.211217+0000","last_fullsized":"2024-09-11T09:02:47.211217+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:07.276030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:07.276030+0000","last_clean_scrub_stamp":"2024-09-11T09:02:07.276030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:32.126766+0000","scrub_duration":0.001431938,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"199'81","reported_seq":565,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930784+0000","last_change":"2024-09-11T09:02:02.394136+0000","last_active":"2024-09-11T09:02:47.930784+0000","last_peered":"2024-09-11T09:02:47.930784+0000","last_clean":"2024-09-11T09:02:47.930784+0000","last_became_active":"2024-09-11T08:58:03.122171+0000","last_became_peered":"2024-09-11T08:58:03.122171+0000","last_unstale":"2024-09-11T09:02:47.930784+0000","last_undegraded":"2024-09-11T09:02:47.930784+0000","last_fullsized":"2024-09-11T09:02:47.930784+0000","mapping_epoch":385,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":386,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:02.394080+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:02.394080+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:28.463436+0000","scrub_duration":0.0012231379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"402'27","reported_seq":685,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.213051+0000","last_change":"2024-09-11T09:02:00.763868+0000","last_active":"2024-09-11T09:02:47.213051+0000","last_peered":"2024-09-11T09:02:47.213051+0000","last_clean":"2024-09-11T09:02:47.213051+0000","last_became_active":"2024-09-11T09:01:28.505591+0000","last_became_peered":"2024-09-11T09:01:28.505591+0000","last_unstale":"2024-09-11T09:02:47.213051+0000","last_undegraded":"2024-09-11T09:02:47.213051+0000","last_fullsized":"2024-09-11T09:02:47.213051+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'27","last_scrub_stamp":"2024-09-11T09:02:00.763791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:00.763791+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:27.138185+0000","scrub_duration":0.0026385029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":770,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":487,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212348+0000","last_change":"2024-09-11T09:02:16.741968+0000","last_active":"2024-09-11T09:02:47.212348+0000","last_peered":"2024-09-11T09:02:47.212348+0000","last_clean":"2024-09-11T09:02:47.212348+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T09:02:47.212348+0000","last_undegraded":"2024-09-11T09:02:47.212348+0000","last_fullsized":"2024-09-11T09:02:47.212348+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:16.741894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:16.741894+0000","last_clean_scrub_stamp":"2024-09-11T09:02:16.741894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:17.247396+0000","scrub_duration":0.001389007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":528,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.211141+0000","last_change":"2024-09-11T09:01:43.311203+0000","last_active":"2024-09-11T09:02:47.211141+0000","last_peered":"2024-09-11T09:02:47.211141+0000","last_clean":"2024-09-11T09:02:47.211141+0000","last_became_active":"2024-09-11T08:58:34.135221+0000","last_became_peered":"2024-09-11T08:58:34.135221+0000","last_unstale":"2024-09-11T09:02:47.211141+0000","last_undegraded":"2024-09-11T09:02:47.211141+0000","last_fullsized":"2024-09-11T09:02:47.211141+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:43.311148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T09:01:43.311148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:12.616485+0000","scrub_duration":0.0012877260000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":530,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214555+0000","last_change":"2024-09-11T09:02:35.905921+0000","last_active":"2024-09-11T09:02:47.214555+0000","last_peered":"2024-09-11T09:02:47.214555+0000","last_clean":"2024-09-11T09:02:47.214555+0000","last_became_active":"2024-09-11T08:58:34.135447+0000","last_became_peered":"2024-09-11T08:58:34.135447+0000","last_unstale":"2024-09-11T09:02:47.214555+0000","last_undegraded":"2024-09-11T09:02:47.214555+0000","last_fullsized":"2024-09-11T09:02:47.214555+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:35.905869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:35.905869+0000","last_clean_scrub_stamp":"2024-09-11T09:02:35.905869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:01.580035+0000","scrub_duration":0.0032310490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"199'81","reported_seq":270,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210153+0000","last_change":"2024-09-11T09:01:34.302855+0000","last_active":"2024-09-11T09:02:47.210153+0000","last_peered":"2024-09-11T09:02:47.210153+0000","last_clean":"2024-09-11T09:02:47.210153+0000","last_became_active":"2024-09-11T08:51:03.020941+0000","last_became_peered":"2024-09-11T08:51:03.020941+0000","last_unstale":"2024-09-11T09:02:47.210153+0000","last_undegraded":"2024-09-11T09:02:47.210153+0000","last_fullsized":"2024-09-11T09:02:47.210153+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:01:34.302799+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T09:01:34.302799+0000","last_clean_scrub_stamp":"2024-09-11T09:01:34.302799+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:56.836810+0000","scrub_duration":0.0024238559999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"402'53","reported_seq":573,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212880+0000","last_change":"2024-09-11T09:01:38.676516+0000","last_active":"2024-09-11T09:02:47.212880+0000","last_peered":"2024-09-11T09:02:47.212880+0000","last_clean":"2024-09-11T09:02:47.212880+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T09:02:47.212880+0000","last_undegraded":"2024-09-11T09:02:47.212880+0000","last_fullsized":"2024-09-11T09:02:47.212880+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'53","last_scrub_stamp":"2024-09-11T09:01:38.676458+0000","last_deep_scrub":"317'39","last_deep_scrub_stamp":"2024-09-11T08:54:06.928266+0000","last_clean_scrub_stamp":"2024-09-11T09:01:38.676458+0000","objects_scrubbed":8,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:00.311673+0000","scrub_duration":0.0021813280000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":71,"num_read_kb":61,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":181,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931184+0000","last_change":"2024-09-11T09:02:10.871339+0000","last_active":"2024-09-11T09:02:47.931184+0000","last_peered":"2024-09-11T09:02:47.931184+0000","last_clean":"2024-09-11T09:02:47.931184+0000","last_became_active":"2024-09-11T09:02:09.765158+0000","last_became_peered":"2024-09-11T09:02:09.765158+0000","last_unstale":"2024-09-11T09:02:47.931184+0000","last_undegraded":"2024-09-11T09:02:47.931184+0000","last_fullsized":"2024-09-11T09:02:47.931184+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:10.871277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:07.504031+0000","last_clean_scrub_stamp":"2024-09-11T09:02:10.871277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:16.543811+0000","scrub_duration":0.001535876,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":545,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931355+0000","last_change":"2024-09-11T09:02:28.577701+0000","last_active":"2024-09-11T09:02:47.931355+0000","last_peered":"2024-09-11T09:02:47.931355+0000","last_clean":"2024-09-11T09:02:47.931355+0000","last_became_active":"2024-09-11T08:52:18.577201+0000","last_became_peered":"2024-09-11T08:52:18.577201+0000","last_unstale":"2024-09-11T09:02:47.931355+0000","last_undegraded":"2024-09-11T09:02:47.931355+0000","last_fullsized":"2024-09-11T09:02:47.931355+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:28.577617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:28.577617+0000","last_clean_scrub_stamp":"2024-09-11T09:02:28.577617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:36.603773+0000","scrub_duration":0.0017497859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":412,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:58.491117+0000","last_change":"2024-09-11T09:02:58.491092+0000","last_active":"2024-09-11T09:02:58.491117+0000","last_peered":"2024-09-11T09:02:58.491117+0000","last_clean":"2024-09-11T09:02:58.491117+0000","last_became_active":"2024-09-11T09:01:39.644254+0000","last_became_peered":"2024-09-11T09:01:39.644254+0000","last_unstale":"2024-09-11T09:02:58.491117+0000","last_undegraded":"2024-09-11T09:02:58.491117+0000","last_fullsized":"2024-09-11T09:02:58.491117+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:58.491037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:58.491037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:05.537774+0000","scrub_duration":0.0016928869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"199'81","reported_seq":395,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931537+0000","last_change":"2024-09-11T09:02:24.396150+0000","last_active":"2024-09-11T09:02:47.931537+0000","last_peered":"2024-09-11T09:02:47.931537+0000","last_clean":"2024-09-11T09:02:47.931537+0000","last_became_active":"2024-09-11T09:02:16.798411+0000","last_became_peered":"2024-09-11T09:02:16.798411+0000","last_unstale":"2024-09-11T09:02:47.931537+0000","last_undegraded":"2024-09-11T09:02:47.931537+0000","last_fullsized":"2024-09-11T09:02:47.931537+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:24.396086+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:24.396086+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:26.784550+0000","scrub_duration":0.001620535,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"402'30","reported_seq":624,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212633+0000","last_change":"2024-09-11T09:02:17.789888+0000","last_active":"2024-09-11T09:02:47.212633+0000","last_peered":"2024-09-11T09:02:47.212633+0000","last_clean":"2024-09-11T09:02:47.212633+0000","last_became_active":"2024-09-11T08:58:34.136623+0000","last_became_peered":"2024-09-11T08:58:34.136623+0000","last_unstale":"2024-09-11T09:02:47.212633+0000","last_undegraded":"2024-09-11T09:02:47.212633+0000","last_fullsized":"2024-09-11T09:02:47.212633+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'30","last_scrub_stamp":"2024-09-11T09:02:17.789818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:17.789818+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:34.662655+0000","scrub_duration":0.0020786559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":516,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214640+0000","last_change":"2024-09-11T09:02:21.859173+0000","last_active":"2024-09-11T09:02:47.214640+0000","last_peered":"2024-09-11T09:02:47.214640+0000","last_clean":"2024-09-11T09:02:47.214640+0000","last_became_active":"2024-09-11T08:52:18.586343+0000","last_became_peered":"2024-09-11T08:52:18.586343+0000","last_unstale":"2024-09-11T09:02:47.214640+0000","last_undegraded":"2024-09-11T09:02:47.214640+0000","last_fullsized":"2024-09-11T09:02:47.214640+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:21.859117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:21.859117+0000","last_clean_scrub_stamp":"2024-09-11T09:02:21.859117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:32.572516+0000","scrub_duration":0.001611924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"299'9","reported_seq":855,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:55.242453+0000","last_change":"2024-09-11T09:02:09.612482+0000","last_active":"2024-09-11T09:02:55.242453+0000","last_peered":"2024-09-11T09:02:55.242453+0000","last_clean":"2024-09-11T09:02:55.242453+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T09:02:55.242453+0000","last_undegraded":"2024-09-11T09:02:55.242453+0000","last_fullsized":"2024-09-11T09:02:55.242453+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'9","last_scrub_stamp":"2024-09-11T09:02:09.612396+0000","last_deep_scrub":"299'9","last_deep_scrub_stamp":"2024-09-11T08:53:05.835177+0000","last_clean_scrub_stamp":"2024-09-11T09:02:09.612396+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:30.537560+0000","scrub_duration":0.0023718680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":511,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212723+0000","last_change":"2024-09-11T09:02:24.847255+0000","last_active":"2024-09-11T09:02:47.212723+0000","last_peered":"2024-09-11T09:02:47.212723+0000","last_clean":"2024-09-11T09:02:47.212723+0000","last_became_active":"2024-09-11T09:01:28.501529+0000","last_became_peered":"2024-09-11T09:01:28.501529+0000","last_unstale":"2024-09-11T09:02:47.212723+0000","last_undegraded":"2024-09-11T09:02:47.212723+0000","last_fullsized":"2024-09-11T09:02:47.212723+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:24.847190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T09:02:24.847190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:27.957728+0000","scrub_duration":0.001626099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"199'81","reported_seq":108,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212831+0000","last_change":"2024-09-11T09:02:38.883940+0000","last_active":"2024-09-11T09:02:47.212831+0000","last_peered":"2024-09-11T09:02:47.212831+0000","last_clean":"2024-09-11T09:02:47.212831+0000","last_became_active":"2024-09-11T08:58:03.137484+0000","last_became_peered":"2024-09-11T08:58:03.137484+0000","last_unstale":"2024-09-11T09:02:47.212831+0000","last_undegraded":"2024-09-11T09:02:47.212831+0000","last_fullsized":"2024-09-11T09:02:47.212831+0000","mapping_epoch":385,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":386,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:38.883866+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:38.883866+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:40.765064+0000","scrub_duration":0.0014340259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"402'21","reported_seq":288,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932047+0000","last_change":"2024-09-11T09:02:21.818101+0000","last_active":"2024-09-11T09:02:47.932047+0000","last_peered":"2024-09-11T09:02:47.932047+0000","last_clean":"2024-09-11T09:02:47.932047+0000","last_became_active":"2024-09-11T08:52:18.588329+0000","last_became_peered":"2024-09-11T08:52:18.588329+0000","last_unstale":"2024-09-11T09:02:47.932047+0000","last_undegraded":"2024-09-11T09:02:47.932047+0000","last_fullsized":"2024-09-11T09:02:47.932047+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'21","last_scrub_stamp":"2024-09-11T09:02:21.818018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:21.818018+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:49.056254+0000","scrub_duration":0.0018483709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":529,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210774+0000","last_change":"2024-09-11T09:01:56.336631+0000","last_active":"2024-09-11T09:02:47.210774+0000","last_peered":"2024-09-11T09:02:47.210774+0000","last_clean":"2024-09-11T09:02:47.210774+0000","last_became_active":"2024-09-11T08:58:34.139430+0000","last_became_peered":"2024-09-11T08:58:34.139430+0000","last_unstale":"2024-09-11T09:02:47.210774+0000","last_undegraded":"2024-09-11T09:02:47.210774+0000","last_fullsized":"2024-09-11T09:02:47.210774+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:56.336547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T09:01:56.336547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:25.568418+0000","scrub_duration":0.0013432450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":409,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931260+0000","last_change":"2024-09-11T09:02:46.500211+0000","last_active":"2024-09-11T09:02:47.931260+0000","last_peered":"2024-09-11T09:02:47.931260+0000","last_clean":"2024-09-11T09:02:47.931260+0000","last_became_active":"2024-09-11T09:01:39.644407+0000","last_became_peered":"2024-09-11T09:01:39.644407+0000","last_unstale":"2024-09-11T09:02:47.931260+0000","last_undegraded":"2024-09-11T09:02:47.931260+0000","last_fullsized":"2024-09-11T09:02:47.931260+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:46.500156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:46.500156+0000","last_clean_scrub_stamp":"2024-09-11T09:02:46.500156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:12.245878+0000","scrub_duration":0.0013442370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":68,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931085+0000","last_change":"2024-09-11T09:02:41.921567+0000","last_active":"2024-09-11T09:02:47.931085+0000","last_peered":"2024-09-11T09:02:47.931085+0000","last_clean":"2024-09-11T09:02:47.931085+0000","last_became_active":"2024-09-11T09:01:25.402638+0000","last_became_peered":"2024-09-11T09:01:25.402638+0000","last_unstale":"2024-09-11T09:02:47.931085+0000","last_undegraded":"2024-09-11T09:02:47.931085+0000","last_fullsized":"2024-09-11T09:02:47.931085+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:41.921508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:13.770061+0000","last_clean_scrub_stamp":"2024-09-11T09:02:41.921508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:55.879252+0000","scrub_duration":0.00148383,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"199'81","reported_seq":66,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931616+0000","last_change":"2024-09-11T09:02:29.865227+0000","last_active":"2024-09-11T09:02:47.931616+0000","last_peered":"2024-09-11T09:02:47.931616+0000","last_clean":"2024-09-11T09:02:47.931616+0000","last_became_active":"2024-09-11T09:02:09.770746+0000","last_became_peered":"2024-09-11T09:02:09.770746+0000","last_unstale":"2024-09-11T09:02:47.931616+0000","last_undegraded":"2024-09-11T09:02:47.931616+0000","last_fullsized":"2024-09-11T09:02:47.931616+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:29.865153+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:29.865153+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:57.496797+0000","scrub_duration":0.00152848,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"402'60","reported_seq":285,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931293+0000","last_change":"2024-09-11T09:02:09.767797+0000","last_active":"2024-09-11T09:02:47.931293+0000","last_peered":"2024-09-11T09:02:47.931293+0000","last_clean":"2024-09-11T09:02:47.931293+0000","last_became_active":"2024-09-11T09:02:09.767382+0000","last_became_peered":"2024-09-11T09:02:09.767382+0000","last_unstale":"2024-09-11T09:02:47.931293+0000","last_undegraded":"2024-09-11T09:02:47.931293+0000","last_fullsized":"2024-09-11T09:02:47.931293+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'60","last_scrub_stamp":"2024-09-11T09:01:53.308889+0000","last_deep_scrub":"374'44","last_deep_scrub_stamp":"2024-09-11T08:57:34.896912+0000","last_clean_scrub_stamp":"2024-09-11T09:01:53.308889+0000","objects_scrubbed":9,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:22.530451+0000","scrub_duration":0.011841266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":77,"num_read_kb":66,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":313,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932543+0000","last_change":"2024-09-11T09:02:16.804258+0000","last_active":"2024-09-11T09:02:47.932543+0000","last_peered":"2024-09-11T09:02:47.932543+0000","last_clean":"2024-09-11T09:02:47.932543+0000","last_became_active":"2024-09-11T09:02:16.803424+0000","last_became_peered":"2024-09-11T09:02:16.803424+0000","last_unstale":"2024-09-11T09:02:47.932543+0000","last_undegraded":"2024-09-11T09:02:47.932543+0000","last_fullsized":"2024-09-11T09:02:47.932543+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:13.239031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:13.239031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:20.173694+0000","scrub_duration":0.001520041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":574,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930592+0000","last_change":"2024-09-11T09:02:16.655092+0000","last_active":"2024-09-11T09:02:47.930592+0000","last_peered":"2024-09-11T09:02:47.930592+0000","last_clean":"2024-09-11T09:02:47.930592+0000","last_became_active":"2024-09-11T08:52:18.584748+0000","last_became_peered":"2024-09-11T08:52:18.584748+0000","last_unstale":"2024-09-11T09:02:47.930592+0000","last_undegraded":"2024-09-11T09:02:47.930592+0000","last_fullsized":"2024-09-11T09:02:47.930592+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:16.655040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:56:46.744542+0000","last_clean_scrub_stamp":"2024-09-11T09:02:16.655040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:19.302489+0000","scrub_duration":0.001669696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":310,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:51.491632+0000","last_change":"2024-09-11T09:02:51.491550+0000","last_active":"2024-09-11T09:02:51.491632+0000","last_peered":"2024-09-11T09:02:51.491632+0000","last_clean":"2024-09-11T09:02:51.491632+0000","last_became_active":"2024-09-11T09:02:16.797777+0000","last_became_peered":"2024-09-11T09:02:16.797777+0000","last_unstale":"2024-09-11T09:02:51.491632+0000","last_undegraded":"2024-09-11T09:02:51.491632+0000","last_fullsized":"2024-09-11T09:02:51.491632+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:51.491474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:51.491474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:58.716857+0000","scrub_duration":0.0014864139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"199'81","reported_seq":321,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214831+0000","last_change":"2024-09-11T09:02:42.998461+0000","last_active":"2024-09-11T09:02:47.214831+0000","last_peered":"2024-09-11T09:02:47.214831+0000","last_clean":"2024-09-11T09:02:47.214831+0000","last_became_active":"2024-09-11T08:58:34.135770+0000","last_became_peered":"2024-09-11T08:58:34.135770+0000","last_unstale":"2024-09-11T09:02:47.214831+0000","last_undegraded":"2024-09-11T09:02:47.214831+0000","last_fullsized":"2024-09-11T09:02:47.214831+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:42.998406+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:42.998406+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:02.837909+0000","scrub_duration":0.001872744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"402'29","reported_seq":531,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212162+0000","last_change":"2024-09-11T09:01:13.988154+0000","last_active":"2024-09-11T09:02:47.212162+0000","last_peered":"2024-09-11T09:02:47.212162+0000","last_clean":"2024-09-11T09:02:47.212162+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T09:02:47.212162+0000","last_undegraded":"2024-09-11T09:02:47.212162+0000","last_fullsized":"2024-09-11T09:02:47.212162+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'29","last_scrub_stamp":"2024-09-11T09:01:13.988098+0000","last_deep_scrub":"402'29","last_deep_scrub_stamp":"2024-09-11T09:01:13.988098+0000","last_clean_scrub_stamp":"2024-09-11T09:01:13.988098+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002328933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":313,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932781+0000","last_change":"2024-09-11T09:02:16.801599+0000","last_active":"2024-09-11T09:02:47.932781+0000","last_peered":"2024-09-11T09:02:47.932781+0000","last_clean":"2024-09-11T09:02:47.932781+0000","last_became_active":"2024-09-11T09:02:16.801373+0000","last_became_peered":"2024-09-11T09:02:16.801373+0000","last_unstale":"2024-09-11T09:02:47.932781+0000","last_undegraded":"2024-09-11T09:02:47.932781+0000","last_fullsized":"2024-09-11T09:02:47.932781+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:14.700205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:14.700205+0000","last_clean_scrub_stamp":"2024-09-11T09:02:14.700205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:44.080217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":418,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930647+0000","last_change":"2024-09-11T09:02:20.652457+0000","last_active":"2024-09-11T09:02:47.930647+0000","last_peered":"2024-09-11T09:02:47.930647+0000","last_clean":"2024-09-11T09:02:47.930647+0000","last_became_active":"2024-09-11T09:01:39.869277+0000","last_became_peered":"2024-09-11T09:01:39.869277+0000","last_unstale":"2024-09-11T09:02:47.930647+0000","last_undegraded":"2024-09-11T09:02:47.930647+0000","last_fullsized":"2024-09-11T09:02:47.930647+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:20.652405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:02:20.652405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:31.207876+0000","scrub_duration":0.0011182480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":490,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214424+0000","last_change":"2024-09-11T09:02:32.868637+0000","last_active":"2024-09-11T09:02:47.214424+0000","last_peered":"2024-09-11T09:02:47.214424+0000","last_clean":"2024-09-11T09:02:47.214424+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T09:02:47.214424+0000","last_undegraded":"2024-09-11T09:02:47.214424+0000","last_fullsized":"2024-09-11T09:02:47.214424+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:32.868587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T09:02:32.868587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:49.468361+0000","scrub_duration":0.001569478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"199'81","reported_seq":202,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932084+0000","last_change":"2024-09-11T09:02:25.397821+0000","last_active":"2024-09-11T09:02:47.932084+0000","last_peered":"2024-09-11T09:02:47.932084+0000","last_clean":"2024-09-11T09:02:47.932084+0000","last_became_active":"2024-09-11T09:02:16.803579+0000","last_became_peered":"2024-09-11T09:02:16.803579+0000","last_unstale":"2024-09-11T09:02:47.932084+0000","last_undegraded":"2024-09-11T09:02:47.932084+0000","last_fullsized":"2024-09-11T09:02:47.932084+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:25.397751+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:25.397751+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:32.968716+0000","scrub_duration":0.001575592,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"402'46","reported_seq":682,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210212+0000","last_change":"2024-09-11T09:02:24.263032+0000","last_active":"2024-09-11T09:02:47.210212+0000","last_peered":"2024-09-11T09:02:47.210212+0000","last_clean":"2024-09-11T09:02:47.210212+0000","last_became_active":"2024-09-11T08:58:34.130937+0000","last_became_peered":"2024-09-11T08:58:34.130937+0000","last_unstale":"2024-09-11T09:02:47.210212+0000","last_undegraded":"2024-09-11T09:02:47.210212+0000","last_fullsized":"2024-09-11T09:02:47.210212+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'46","last_scrub_stamp":"2024-09-11T09:02:24.262947+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T09:02:24.262947+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:49.238019+0000","scrub_duration":0.0021749249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":362,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932746+0000","last_change":"2024-09-11T09:02:22.420296+0000","last_active":"2024-09-11T09:02:47.932746+0000","last_peered":"2024-09-11T09:02:47.932746+0000","last_clean":"2024-09-11T09:02:47.932746+0000","last_became_active":"2024-09-11T09:02:16.797349+0000","last_became_peered":"2024-09-11T09:02:16.797349+0000","last_unstale":"2024-09-11T09:02:47.932746+0000","last_undegraded":"2024-09-11T09:02:47.932746+0000","last_fullsized":"2024-09-11T09:02:47.932746+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T09:02:22.420155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:22.420155+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:46.820846+0000","scrub_duration":0.0020171049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"460'16","reported_seq":520,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:52.946565+0000","last_change":"2024-09-11T09:02:16.796915+0000","last_active":"2024-09-11T09:02:52.946565+0000","last_peered":"2024-09-11T09:02:52.946565+0000","last_clean":"2024-09-11T09:02:52.946565+0000","last_became_active":"2024-09-11T09:02:16.796406+0000","last_became_peered":"2024-09-11T09:02:16.796406+0000","last_unstale":"2024-09-11T09:02:52.946565+0000","last_undegraded":"2024-09-11T09:02:52.946565+0000","last_fullsized":"2024-09-11T09:02:52.946565+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"401'14","last_scrub_stamp":"2024-09-11T09:01:40.596701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:01:40.596701+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071142979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":539,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210232+0000","last_change":"2024-09-11T09:01:44.323080+0000","last_active":"2024-09-11T09:02:47.210232+0000","last_peered":"2024-09-11T09:02:47.210232+0000","last_clean":"2024-09-11T09:02:47.210232+0000","last_became_active":"2024-09-11T08:58:34.128990+0000","last_became_peered":"2024-09-11T08:58:34.128990+0000","last_unstale":"2024-09-11T09:02:47.210232+0000","last_undegraded":"2024-09-11T09:02:47.210232+0000","last_fullsized":"2024-09-11T09:02:47.210232+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:44.323026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:01:44.323026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:04.370889+0000","scrub_duration":0.001236406,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"199'81","reported_seq":138,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210384+0000","last_change":"2024-09-11T09:02:04.278619+0000","last_active":"2024-09-11T09:02:47.210384+0000","last_peered":"2024-09-11T09:02:47.210384+0000","last_clean":"2024-09-11T09:02:47.210384+0000","last_became_active":"2024-09-11T09:01:28.496162+0000","last_became_peered":"2024-09-11T09:01:28.496162+0000","last_unstale":"2024-09-11T09:02:47.210384+0000","last_undegraded":"2024-09-11T09:02:47.210384+0000","last_fullsized":"2024-09-11T09:02:47.210384+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":5,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:04.278532+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:57:49.622788+0000","last_clean_scrub_stamp":"2024-09-11T09:02:04.278532+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:09.874878+0000","scrub_duration":0.0018337760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":439,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:54.808356+0000","last_change":"2024-09-11T09:02:54.808307+0000","last_active":"2024-09-11T09:02:54.808356+0000","last_peered":"2024-09-11T09:02:54.808356+0000","last_clean":"2024-09-11T09:02:54.808356+0000","last_became_active":"2024-09-11T09:02:09.770004+0000","last_became_peered":"2024-09-11T09:02:09.770004+0000","last_unstale":"2024-09-11T09:02:54.808356+0000","last_undegraded":"2024-09-11T09:02:54.808356+0000","last_fullsized":"2024-09-11T09:02:54.808356+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:54.808243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T09:02:54.808243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:57.208244+0000","scrub_duration":0.0014588870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"402'18","reported_seq":539,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214884+0000","last_change":"2024-09-11T09:02:13.832716+0000","last_active":"2024-09-11T09:02:47.214884+0000","last_peered":"2024-09-11T09:02:47.214884+0000","last_clean":"2024-09-11T09:02:47.214884+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T09:02:47.214884+0000","last_undegraded":"2024-09-11T09:02:47.214884+0000","last_fullsized":"2024-09-11T09:02:47.214884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'18","last_scrub_stamp":"2024-09-11T09:02:13.832656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:13.832656+0000","objects_scrubbed":3,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:33.245733+0000","scrub_duration":0.001811434,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":502,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.211016+0000","last_change":"2024-09-11T09:01:46.346364+0000","last_active":"2024-09-11T09:02:47.211016+0000","last_peered":"2024-09-11T09:02:47.211016+0000","last_clean":"2024-09-11T09:02:47.211016+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T09:02:47.211016+0000","last_undegraded":"2024-09-11T09:02:47.211016+0000","last_fullsized":"2024-09-11T09:02:47.211016+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:46.346312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:01:46.346312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:14.064974+0000","scrub_duration":0.0015947590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":572,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932016+0000","last_change":"2024-09-11T09:02:29.344893+0000","last_active":"2024-09-11T09:02:47.932016+0000","last_peered":"2024-09-11T09:02:47.932016+0000","last_clean":"2024-09-11T09:02:47.932016+0000","last_became_active":"2024-09-11T08:58:34.135112+0000","last_became_peered":"2024-09-11T08:58:34.135112+0000","last_unstale":"2024-09-11T09:02:47.932016+0000","last_undegraded":"2024-09-11T09:02:47.932016+0000","last_fullsized":"2024-09-11T09:02:47.932016+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:29.344825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:29.344825+0000","last_clean_scrub_stamp":"2024-09-11T09:02:29.344825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:30.040040+0000","scrub_duration":0.001588768,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"199'81","reported_seq":568,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930693+0000","last_change":"2024-09-11T09:02:16.221457+0000","last_active":"2024-09-11T09:02:47.930693+0000","last_peered":"2024-09-11T09:02:47.930693+0000","last_clean":"2024-09-11T09:02:47.930693+0000","last_became_active":"2024-09-11T08:58:03.129022+0000","last_became_peered":"2024-09-11T08:58:03.129022+0000","last_unstale":"2024-09-11T09:02:47.930693+0000","last_undegraded":"2024-09-11T09:02:47.930693+0000","last_fullsized":"2024-09-11T09:02:47.930693+0000","mapping_epoch":385,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":386,"parent":"0.0","parent_split_bits":5,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:16.221383+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:16.221383+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:21.455145+0000","scrub_duration":0.0019473030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"462'25","reported_seq":512,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932678+0000","last_change":"2024-09-11T09:02:36.944835+0000","last_active":"2024-09-11T09:02:47.932678+0000","last_peered":"2024-09-11T09:02:47.932678+0000","last_clean":"2024-09-11T09:02:47.932678+0000","last_became_active":"2024-09-11T09:01:25.405993+0000","last_became_peered":"2024-09-11T09:01:25.405993+0000","last_unstale":"2024-09-11T09:02:47.932678+0000","last_undegraded":"2024-09-11T09:02:47.932678+0000","last_fullsized":"2024-09-11T09:02:47.932678+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"462'25","last_scrub_stamp":"2024-09-11T09:02:36.944772+0000","last_deep_scrub":"462'25","last_deep_scrub_stamp":"2024-09-11T09:02:36.944772+0000","last_clean_scrub_stamp":"2024-09-11T09:02:36.944772+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:45.238295+0000","scrub_duration":0.0036256919999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":316,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931328+0000","last_change":"2024-09-11T09:02:16.801305+0000","last_active":"2024-09-11T09:02:47.931328+0000","last_peered":"2024-09-11T09:02:47.931328+0000","last_clean":"2024-09-11T09:02:47.931328+0000","last_became_active":"2024-09-11T09:02:16.800720+0000","last_became_peered":"2024-09-11T09:02:16.800720+0000","last_unstale":"2024-09-11T09:02:47.931328+0000","last_undegraded":"2024-09-11T09:02:47.931328+0000","last_fullsized":"2024-09-11T09:02:47.931328+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:58.761306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T09:01:58.761306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:27.423944+0000","scrub_duration":0.0042052879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":492,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:59.890496+0000","last_change":"2024-09-11T09:02:59.890451+0000","last_active":"2024-09-11T09:02:59.890496+0000","last_peered":"2024-09-11T09:02:59.890496+0000","last_clean":"2024-09-11T09:02:59.890496+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T09:02:59.890496+0000","last_undegraded":"2024-09-11T09:02:59.890496+0000","last_fullsized":"2024-09-11T09:02:59.890496+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:59.890391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:59.890391+0000","last_clean_scrub_stamp":"2024-09-11T09:02:59.890391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:28.635200+0000","scrub_duration":0.001261755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":550,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:52.464364+0000","last_change":"2024-09-11T09:02:52.464309+0000","last_active":"2024-09-11T09:02:52.464364+0000","last_peered":"2024-09-11T09:02:52.464364+0000","last_clean":"2024-09-11T09:02:52.464364+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T09:02:52.464364+0000","last_undegraded":"2024-09-11T09:02:52.464364+0000","last_fullsized":"2024-09-11T09:02:52.464364+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:52.464244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:59:52.054422+0000","last_clean_scrub_stamp":"2024-09-11T09:02:52.464244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:05.769952+0000","scrub_duration":0.00165238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"199'81","reported_seq":128,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.209939+0000","last_change":"2024-09-11T09:02:17.278096+0000","last_active":"2024-09-11T09:02:47.209939+0000","last_peered":"2024-09-11T09:02:47.209939+0000","last_clean":"2024-09-11T09:02:47.209939+0000","last_became_active":"2024-09-11T08:58:34.130875+0000","last_became_peered":"2024-09-11T08:58:34.130875+0000","last_unstale":"2024-09-11T09:02:47.209939+0000","last_undegraded":"2024-09-11T09:02:47.209939+0000","last_fullsized":"2024-09-11T09:02:47.209939+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":5,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:17.278031+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T09:02:17.278031+0000","last_clean_scrub_stamp":"2024-09-11T09:02:17.278031+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:29.771668+0000","scrub_duration":0.0015397919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"443'25","reported_seq":568,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:55.045616+0000","last_change":"2024-09-11T09:02:55.045533+0000","last_active":"2024-09-11T09:02:55.045616+0000","last_peered":"2024-09-11T09:02:55.045616+0000","last_clean":"2024-09-11T09:02:55.045616+0000","last_became_active":"2024-09-11T08:53:58.555987+0000","last_became_peered":"2024-09-11T08:53:58.555987+0000","last_unstale":"2024-09-11T09:02:55.045616+0000","last_undegraded":"2024-09-11T09:02:55.045616+0000","last_fullsized":"2024-09-11T09:02:55.045616+0000","mapping_epoch":315,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'25","last_scrub_stamp":"2024-09-11T09:02:55.045428+0000","last_deep_scrub":"443'25","last_deep_scrub_stamp":"2024-09-11T09:01:31.380448+0000","last_clean_scrub_stamp":"2024-09-11T09:02:55.045428+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:04.876290+0000","scrub_duration":0.0021129880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":68,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931378+0000","last_change":"2024-09-11T09:01:41.930052+0000","last_active":"2024-09-11T09:02:47.931378+0000","last_peered":"2024-09-11T09:02:47.931378+0000","last_clean":"2024-09-11T09:02:47.931378+0000","last_became_active":"2024-09-11T09:01:25.402878+0000","last_became_peered":"2024-09-11T09:01:25.402878+0000","last_unstale":"2024-09-11T09:02:47.931378+0000","last_undegraded":"2024-09-11T09:02:47.931378+0000","last_fullsized":"2024-09-11T09:02:47.931378+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:41.929995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:03.641009+0000","last_clean_scrub_stamp":"2024-09-11T09:01:41.929995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:10.513168+0000","scrub_duration":0.0014327949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":782,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:55.241856+0000","last_change":"2024-09-11T09:01:49.788850+0000","last_active":"2024-09-11T09:02:55.241856+0000","last_peered":"2024-09-11T09:02:55.241856+0000","last_clean":"2024-09-11T09:02:55.241856+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T09:02:55.241856+0000","last_undegraded":"2024-09-11T09:02:55.241856+0000","last_fullsized":"2024-09-11T09:02:55.241856+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T09:01:49.788792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:01:49.788792+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:15.463889+0000","scrub_duration":0.0017970480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":484,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212994+0000","last_change":"2024-09-11T09:02:18.805812+0000","last_active":"2024-09-11T09:02:47.212994+0000","last_peered":"2024-09-11T09:02:47.212994+0000","last_clean":"2024-09-11T09:02:47.212994+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T09:02:47.212994+0000","last_undegraded":"2024-09-11T09:02:47.212994+0000","last_fullsized":"2024-09-11T09:02:47.212994+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:18.805748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:18.805748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:22.278065+0000","scrub_duration":0.0014510829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"199'81","reported_seq":84,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930784+0000","last_change":"2024-09-11T09:02:05.924533+0000","last_active":"2024-09-11T09:02:47.930784+0000","last_peered":"2024-09-11T09:02:47.930784+0000","last_clean":"2024-09-11T09:02:47.930784+0000","last_became_active":"2024-09-11T09:01:25.398637+0000","last_became_peered":"2024-09-11T09:01:25.398637+0000","last_unstale":"2024-09-11T09:02:47.930784+0000","last_undegraded":"2024-09-11T09:02:47.930784+0000","last_fullsized":"2024-09-11T09:02:47.930784+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":5,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:05.924475+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:05.924475+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:15.208417+0000","scrub_duration":0.0015511259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"402'35","reported_seq":588,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214908+0000","last_change":"2024-09-11T09:02:22.820209+0000","last_active":"2024-09-11T09:02:47.214908+0000","last_peered":"2024-09-11T09:02:47.214908+0000","last_clean":"2024-09-11T09:02:47.214908+0000","last_became_active":"2024-09-11T09:01:28.504757+0000","last_became_peered":"2024-09-11T09:01:28.504757+0000","last_unstale":"2024-09-11T09:02:47.214908+0000","last_undegraded":"2024-09-11T09:02:47.214908+0000","last_fullsized":"2024-09-11T09:02:47.214908+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'35","last_scrub_stamp":"2024-09-11T09:02:22.820112+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T09:02:22.820112+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:38.227044+0000","scrub_duration":0.0061296340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":545,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932022+0000","last_change":"2024-09-11T09:02:12.493237+0000","last_active":"2024-09-11T09:02:47.932022+0000","last_peered":"2024-09-11T09:02:47.932022+0000","last_clean":"2024-09-11T09:02:47.932022+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T09:02:47.932022+0000","last_undegraded":"2024-09-11T09:02:47.932022+0000","last_fullsized":"2024-09-11T09:02:47.932022+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:12.493172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:15.235519+0000","last_clean_scrub_stamp":"2024-09-11T09:02:12.493172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:22.197202+0000","scrub_duration":0.0025578910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"277'10","reported_seq":829,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:52.946562+0000","last_change":"2024-09-11T09:01:58.315885+0000","last_active":"2024-09-11T09:02:52.946562+0000","last_peered":"2024-09-11T09:02:52.946562+0000","last_clean":"2024-09-11T09:02:52.946562+0000","last_became_active":"2024-09-11T08:52:18.585914+0000","last_became_peered":"2024-09-11T08:52:18.585914+0000","last_unstale":"2024-09-11T09:02:52.946562+0000","last_undegraded":"2024-09-11T09:02:52.946562+0000","last_fullsized":"2024-09-11T09:02:52.946562+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'10","last_scrub_stamp":"2024-09-11T09:01:58.315822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:01:58.315822+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:00.336225+0000","scrub_duration":0.0013541219999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":68,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932758+0000","last_change":"2024-09-11T09:02:15.875014+0000","last_active":"2024-09-11T09:02:47.932758+0000","last_peered":"2024-09-11T09:02:47.932758+0000","last_clean":"2024-09-11T09:02:47.932758+0000","last_became_active":"2024-09-11T09:01:25.405725+0000","last_became_peered":"2024-09-11T09:01:25.405725+0000","last_unstale":"2024-09-11T09:02:47.932758+0000","last_undegraded":"2024-09-11T09:02:47.932758+0000","last_fullsized":"2024-09-11T09:02:47.932758+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:15.874955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:15.874955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:41.092556+0000","scrub_duration":0.0035339189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"402'27","reported_seq":604,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:57.851270+0000","last_change":"2024-09-11T09:02:57.851233+0000","last_active":"2024-09-11T09:02:57.851270+0000","last_peered":"2024-09-11T09:02:57.851270+0000","last_clean":"2024-09-11T09:02:57.851270+0000","last_became_active":"2024-09-11T08:58:34.135591+0000","last_became_peered":"2024-09-11T08:58:34.135591+0000","last_unstale":"2024-09-11T09:02:57.851270+0000","last_undegraded":"2024-09-11T09:02:57.851270+0000","last_fullsized":"2024-09-11T09:02:57.851270+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'27","last_scrub_stamp":"2024-09-11T09:02:57.851163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:57.851163+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:26.173111+0000","scrub_duration":0.002053095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":517,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:49.885810+0000","last_change":"2024-09-11T09:02:49.885750+0000","last_active":"2024-09-11T09:02:49.885810+0000","last_peered":"2024-09-11T09:02:49.885810+0000","last_clean":"2024-09-11T09:02:49.885810+0000","last_became_active":"2024-09-11T08:58:34.138928+0000","last_became_peered":"2024-09-11T08:58:34.138928+0000","last_unstale":"2024-09-11T09:02:49.885810+0000","last_undegraded":"2024-09-11T09:02:49.885810+0000","last_fullsized":"2024-09-11T09:02:49.885810+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:49.885681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:49.885681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:15.037792+0000","scrub_duration":0.001492761,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":522,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931033+0000","last_change":"2024-09-11T09:01:21.189150+0000","last_active":"2024-09-11T09:02:47.931033+0000","last_peered":"2024-09-11T09:02:47.931033+0000","last_clean":"2024-09-11T09:02:47.931033+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T09:02:47.931033+0000","last_undegraded":"2024-09-11T09:02:47.931033+0000","last_fullsized":"2024-09-11T09:02:47.931033+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:21.189096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:01:21.189096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013140070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":510,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214429+0000","last_change":"2024-09-11T09:02:38.917179+0000","last_active":"2024-09-11T09:02:47.214429+0000","last_peered":"2024-09-11T09:02:47.214429+0000","last_clean":"2024-09-11T09:02:47.214429+0000","last_became_active":"2024-09-11T08:52:18.585089+0000","last_became_peered":"2024-09-11T08:52:18.585089+0000","last_unstale":"2024-09-11T09:02:47.214429+0000","last_undegraded":"2024-09-11T09:02:47.214429+0000","last_fullsized":"2024-09-11T09:02:47.214429+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:38.917120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:38.917120+0000","last_clean_scrub_stamp":"2024-09-11T09:02:38.917120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:08.060616+0000","scrub_duration":0.001747572,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"448'25","reported_seq":55,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930707+0000","last_change":"2024-09-11T09:01:46.274566+0000","last_active":"2024-09-11T09:02:47.930707+0000","last_peered":"2024-09-11T09:02:47.930707+0000","last_clean":"2024-09-11T09:02:47.930707+0000","last_became_active":"2024-09-11T09:01:28.498888+0000","last_became_peered":"2024-09-11T09:01:28.498888+0000","last_unstale":"2024-09-11T09:02:47.930707+0000","last_undegraded":"2024-09-11T09:02:47.930707+0000","last_fullsized":"2024-09-11T09:02:47.930707+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"448'25","last_scrub_stamp":"2024-09-11T09:01:46.274515+0000","last_deep_scrub":"448'25","last_deep_scrub_stamp":"2024-09-11T09:01:46.274515+0000","last_clean_scrub_stamp":"2024-09-11T09:01:46.274515+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:06.032306+0000","scrub_duration":0.0032311929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":323,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932440+0000","last_change":"2024-09-11T09:02:18.456738+0000","last_active":"2024-09-11T09:02:47.932440+0000","last_peered":"2024-09-11T09:02:47.932440+0000","last_clean":"2024-09-11T09:02:47.932440+0000","last_became_active":"2024-09-11T09:02:16.797837+0000","last_became_peered":"2024-09-11T09:02:16.797837+0000","last_unstale":"2024-09-11T09:02:47.932440+0000","last_undegraded":"2024-09-11T09:02:47.932440+0000","last_fullsized":"2024-09-11T09:02:47.932440+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:18.456672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:21.208684+0000","last_clean_scrub_stamp":"2024-09-11T09:02:18.456672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:18.517939+0000","scrub_duration":0.001458336,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":512,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214462+0000","last_change":"2024-09-11T09:02:28.806185+0000","last_active":"2024-09-11T09:02:47.214462+0000","last_peered":"2024-09-11T09:02:47.214462+0000","last_clean":"2024-09-11T09:02:47.214462+0000","last_became_active":"2024-09-11T08:52:18.585691+0000","last_became_peered":"2024-09-11T08:52:18.585691+0000","last_unstale":"2024-09-11T09:02:47.214462+0000","last_undegraded":"2024-09-11T09:02:47.214462+0000","last_fullsized":"2024-09-11T09:02:47.214462+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:28.806128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:28.806128+0000","last_clean_scrub_stamp":"2024-09-11T09:02:28.806128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:52.254759+0000","scrub_duration":0.0012738910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":137,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932383+0000","last_change":"2024-09-11T09:01:33.974905+0000","last_active":"2024-09-11T09:02:47.932383+0000","last_peered":"2024-09-11T09:02:47.932383+0000","last_clean":"2024-09-11T09:02:47.932383+0000","last_became_active":"2024-09-11T09:01:25.419653+0000","last_became_peered":"2024-09-11T09:01:25.419653+0000","last_unstale":"2024-09-11T09:02:47.932383+0000","last_undegraded":"2024-09-11T09:02:47.932383+0000","last_fullsized":"2024-09-11T09:02:47.932383+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:33.974842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:24.401480+0000","last_clean_scrub_stamp":"2024-09-11T09:01:33.974842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:00.737680+0000","scrub_duration":0.0018484700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"199'81","reported_seq":118,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930736+0000","last_change":"2024-09-11T09:02:26.705339+0000","last_active":"2024-09-11T09:02:47.930736+0000","last_peered":"2024-09-11T09:02:47.930736+0000","last_clean":"2024-09-11T09:02:47.930736+0000","last_became_active":"2024-09-11T08:58:03.128935+0000","last_became_peered":"2024-09-11T08:58:03.128935+0000","last_unstale":"2024-09-11T09:02:47.930736+0000","last_undegraded":"2024-09-11T09:02:47.930736+0000","last_fullsized":"2024-09-11T09:02:47.930736+0000","mapping_epoch":385,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":386,"parent":"0.0","parent_split_bits":5,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:26.705254+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T09:02:26.705254+0000","last_clean_scrub_stamp":"2024-09-11T09:02:26.705254+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:41.908234+0000","scrub_duration":0.0019461490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"402'27","reported_seq":571,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214274+0000","last_change":"2024-09-11T09:02:06.720016+0000","last_active":"2024-09-11T09:02:47.214274+0000","last_peered":"2024-09-11T09:02:47.214274+0000","last_clean":"2024-09-11T09:02:47.214274+0000","last_became_active":"2024-09-11T08:58:34.136417+0000","last_became_peered":"2024-09-11T08:58:34.136417+0000","last_unstale":"2024-09-11T09:02:47.214274+0000","last_undegraded":"2024-09-11T09:02:47.214274+0000","last_fullsized":"2024-09-11T09:02:47.214274+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'27","last_scrub_stamp":"2024-09-11T09:02:06.719953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:06.719953+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:10.448704+0000","scrub_duration":0.0021473439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":536,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:53.069232+0000","last_change":"2024-09-11T09:02:53.069185+0000","last_active":"2024-09-11T09:02:53.069232+0000","last_peered":"2024-09-11T09:02:53.069232+0000","last_clean":"2024-09-11T09:02:53.069232+0000","last_became_active":"2024-09-11T09:01:28.499223+0000","last_became_peered":"2024-09-11T09:01:28.499223+0000","last_unstale":"2024-09-11T09:02:53.069232+0000","last_undegraded":"2024-09-11T09:02:53.069232+0000","last_fullsized":"2024-09-11T09:02:53.069232+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:53.069122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:53.069122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:13.418207+0000","scrub_duration":0.001505752,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":567,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930864+0000","last_change":"2024-09-11T09:01:57.539656+0000","last_active":"2024-09-11T09:02:47.930864+0000","last_peered":"2024-09-11T09:02:47.930864+0000","last_clean":"2024-09-11T09:02:47.930864+0000","last_became_active":"2024-09-11T08:58:34.135692+0000","last_became_peered":"2024-09-11T08:58:34.135692+0000","last_unstale":"2024-09-11T09:02:47.930864+0000","last_undegraded":"2024-09-11T09:02:47.930864+0000","last_fullsized":"2024-09-11T09:02:47.930864+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:57.539603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:01:57.539603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:18.127353+0000","scrub_duration":0.0011331589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":540,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214587+0000","last_change":"2024-09-11T09:01:50.781018+0000","last_active":"2024-09-11T09:02:47.214587+0000","last_peered":"2024-09-11T09:02:47.214587+0000","last_clean":"2024-09-11T09:02:47.214587+0000","last_became_active":"2024-09-11T08:58:34.135699+0000","last_became_peered":"2024-09-11T08:58:34.135699+0000","last_unstale":"2024-09-11T09:02:47.214587+0000","last_undegraded":"2024-09-11T09:02:47.214587+0000","last_fullsized":"2024-09-11T09:02:47.214587+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:50.780955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:25.255332+0000","last_clean_scrub_stamp":"2024-09-11T09:01:50.780955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:03.552990+0000","scrub_duration":0.001653409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"402'14","reported_seq":97,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.209925+0000","last_change":"2024-09-11T09:02:46.160216+0000","last_active":"2024-09-11T09:02:47.209925+0000","last_peered":"2024-09-11T09:02:47.209925+0000","last_clean":"2024-09-11T09:02:47.209925+0000","last_became_active":"2024-09-11T08:59:18.588313+0000","last_became_peered":"2024-09-11T08:59:18.588313+0000","last_unstale":"2024-09-11T09:02:47.209925+0000","last_undegraded":"2024-09-11T09:02:47.209925+0000","last_fullsized":"2024-09-11T09:02:47.209925+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'14","last_scrub_stamp":"2024-09-11T09:02:46.160141+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T09:02:46.160141+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:55.146328+0000","scrub_duration":0.0018706040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":72,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932731+0000","last_change":"2024-09-11T09:02:45.922265+0000","last_active":"2024-09-11T09:02:47.932731+0000","last_peered":"2024-09-11T09:02:47.932731+0000","last_clean":"2024-09-11T09:02:47.932731+0000","last_became_active":"2024-09-11T09:01:25.402119+0000","last_became_peered":"2024-09-11T09:01:25.402119+0000","last_unstale":"2024-09-11T09:02:47.932731+0000","last_undegraded":"2024-09-11T09:02:47.932731+0000","last_fullsized":"2024-09-11T09:02:47.932731+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T09:02:45.922211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:45.922211+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:12.899971+0000","scrub_duration":0.001718097,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":153,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931066+0000","last_change":"2024-09-11T09:02:09.772508+0000","last_active":"2024-09-11T09:02:47.931066+0000","last_peered":"2024-09-11T09:02:47.931066+0000","last_clean":"2024-09-11T09:02:47.931066+0000","last_became_active":"2024-09-11T09:02:09.772205+0000","last_became_peered":"2024-09-11T09:02:09.772205+0000","last_unstale":"2024-09-11T09:02:47.931066+0000","last_undegraded":"2024-09-11T09:02:47.931066+0000","last_fullsized":"2024-09-11T09:02:47.931066+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:57.769178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:01:57.769178+0000","last_clean_scrub_stamp":"2024-09-11T09:01:57.769178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:09.765725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":307,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931589+0000","last_change":"2024-09-11T09:02:16.799283+0000","last_active":"2024-09-11T09:02:47.931589+0000","last_peered":"2024-09-11T09:02:47.931589+0000","last_clean":"2024-09-11T09:02:47.931589+0000","last_became_active":"2024-09-11T09:02:16.798333+0000","last_became_peered":"2024-09-11T09:02:16.798333+0000","last_unstale":"2024-09-11T09:02:47.931589+0000","last_undegraded":"2024-09-11T09:02:47.931589+0000","last_fullsized":"2024-09-11T09:02:47.931589+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:36.694030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:01:36.694030+0000","last_clean_scrub_stamp":"2024-09-11T09:01:36.694030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:03.137902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":530,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932476+0000","last_change":"2024-09-11T09:02:19.321545+0000","last_active":"2024-09-11T09:02:47.932476+0000","last_peered":"2024-09-11T09:02:47.932476+0000","last_clean":"2024-09-11T09:02:47.932476+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T09:02:47.932476+0000","last_undegraded":"2024-09-11T09:02:47.932476+0000","last_fullsized":"2024-09-11T09:02:47.932476+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:19.321484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:19.321484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:29.810544+0000","scrub_duration":0.0018493750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":546,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:57.874790+0000","last_change":"2024-09-11T09:02:57.874764+0000","last_active":"2024-09-11T09:02:57.874790+0000","last_peered":"2024-09-11T09:02:57.874790+0000","last_clean":"2024-09-11T09:02:57.874790+0000","last_became_active":"2024-09-11T09:01:28.499661+0000","last_became_peered":"2024-09-11T09:01:28.499661+0000","last_unstale":"2024-09-11T09:02:57.874790+0000","last_undegraded":"2024-09-11T09:02:57.874790+0000","last_fullsized":"2024-09-11T09:02:57.874790+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:57.874711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:32.393197+0000","last_clean_scrub_stamp":"2024-09-11T09:02:57.874711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:20.411716+0000","scrub_duration":0.001187486,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"402'27","reported_seq":218,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931621+0000","last_change":"2024-09-11T09:02:09.769791+0000","last_active":"2024-09-11T09:02:47.931621+0000","last_peered":"2024-09-11T09:02:47.931621+0000","last_clean":"2024-09-11T09:02:47.931621+0000","last_became_active":"2024-09-11T09:02:09.769396+0000","last_became_peered":"2024-09-11T09:02:09.769396+0000","last_unstale":"2024-09-11T09:02:47.931621+0000","last_undegraded":"2024-09-11T09:02:47.931621+0000","last_fullsized":"2024-09-11T09:02:47.931621+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'27","last_scrub_stamp":"2024-09-11T09:01:38.350893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:01:38.350893+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:04.648781+0000","scrub_duration":0.0096371630000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":137,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932577+0000","last_change":"2024-09-11T09:02:31.845724+0000","last_active":"2024-09-11T09:02:47.932577+0000","last_peered":"2024-09-11T09:02:47.932577+0000","last_clean":"2024-09-11T09:02:47.932577+0000","last_became_active":"2024-09-11T09:01:25.405212+0000","last_became_peered":"2024-09-11T09:01:25.405212+0000","last_unstale":"2024-09-11T09:02:47.932577+0000","last_undegraded":"2024-09-11T09:02:47.932577+0000","last_fullsized":"2024-09-11T09:02:47.932577+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:31.845670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:02:31.845670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:43.176324+0000","scrub_duration":0.001355689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1531,"num_read_kb":1288,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":8164,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1080,"ondisk_log_size":1080,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":2294,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":135,"num_read_kb":130,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299690,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":893812,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":893054,"kb_avail":92851340,"statfs":{"total":95995035648,"available":95079772160,"internally_reserved":0,"allocated":761856,"data_stored":446433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":914487640},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.67500000000000004,"15min":0.82899999999999996},"min":{"1min":0.34200000000000003,"5min":0.22500000000000001,"15min":0.20599999999999999},"max":{"1min":0.70499999999999996,"5min":4.133,"15min":5.0449999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.73799999999999999,"15min":0.88},"min":{"1min":0.37,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":0.73299999999999998,"5min":4.5289999999999999,"15min":4.5289999999999999},"last":0.32700000000000001}]},{"osd":1,"last update":"Wed Sep 11 09:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.98199999999999998,"15min":1.01},"min":{"1min":0.379,"5min":0.249,"15min":0.224},"max":{"1min":0.73299999999999998,"5min":26.193999999999999,"15min":26.193999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":1.05,"15min":0.95399999999999996},"min":{"1min":0.31900000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.72099999999999997,"5min":27.058,"15min":27.058},"last":0.37}]},{"osd":2,"last update":"Wed Sep 11 09:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":1.925,"15min":1.306},"min":{"1min":0.35999999999999999,"5min":0.27600000000000002,"15min":0.27000000000000002},"max":{"1min":0.61199999999999999,"5min":116.639,"15min":116.639},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":1.8919999999999999,"15min":1.264},"min":{"1min":0.38700000000000001,"5min":0.28799999999999998,"15min":0.25800000000000001},"max":{"1min":1.145,"5min":116.68899999999999,"15min":116.68899999999999},"last":0.48799999999999999}]},{"osd":3,"last update":"Wed Sep 11 09:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":1.2629999999999999,"15min":1.403},"min":{"1min":0.40600000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.155,"5min":26.225000000000001,"15min":56.265999999999998},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":1.4570000000000001,"15min":1.4730000000000001},"min":{"1min":0.39200000000000002,"5min":0.29599999999999999,"15min":0.27700000000000002},"max":{"1min":0.70899999999999996,"5min":26.280999999999999,"15min":57.207000000000001},"last":0.38200000000000001}]},{"osd":4,"last update":"Wed Sep 11 09:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.73999999999999999,"15min":1.482},"min":{"1min":0.30299999999999999,"5min":0.24199999999999999,"15min":0.224},"max":{"1min":0.81399999999999995,"5min":9.0879999999999992,"15min":163.60499999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.59499999999999997,"15min":1.5349999999999999},"min":{"1min":0.29499999999999998,"5min":0.25,"15min":0.20799999999999999},"max":{"1min":0.79700000000000004,"5min":5.1529999999999996,"15min":163.88300000000001},"last":0.30499999999999999}]},{"osd":5,"last update":"Wed Sep 11 09:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.64600000000000002,"15min":1.04},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.23699999999999999},"max":{"1min":0.69999999999999996,"5min":4.7000000000000002,"15min":50.128},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.74099999999999999,"15min":1.085},"min":{"1min":0.29799999999999999,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.77900000000000003,"5min":7.1820000000000004,"15min":50.241},"last":0.46800000000000003}]},{"osd":6,"last update":"Wed Sep 11 09:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":2.8900000000000001,"15min":1.6339999999999999},"min":{"1min":0.316,"5min":0.30599999999999999,"15min":0.29399999999999998},"max":{"1min":0.85799999999999998,"5min":254.94999999999999,"15min":254.94999999999999},"last":0.501},{"interface":"front","average":{"1min":0.46999999999999997,"5min":2.581,"15min":1.554},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.25},"max":{"1min":0.71199999999999997,"5min":254.88,"15min":254.88},"last":0.443}]}]},{"osd":6,"up_from":45,"seq":193273528623,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":898284,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":897214,"kb_avail":92846868,"statfs":{"total":95995035648,"available":95075192832,"internally_reserved":0,"allocated":1081344,"data_stored":774059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":918747480},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.82299999999999995,"15min":1.5289999999999999},"min":{"1min":0.39700000000000002,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":1.5089999999999999,"5min":7.5270000000000001,"15min":121.208},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.90300000000000002,"15min":1.6930000000000001},"min":{"1min":0.35099999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":4.6619999999999999,"5min":5.2670000000000003,"15min":121.05200000000001},"last":0.63900000000000001}]},{"osd":1,"last update":"Wed Sep 11 09:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":1.0489999999999999,"15min":1.1879999999999999},"min":{"1min":0.39200000000000002,"5min":0.216,"15min":0.216},"max":{"1min":1.0580000000000001,"5min":7.4180000000000001,"15min":27.984000000000002},"last":0.503},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.996,"15min":1.1499999999999999},"min":{"1min":0.39600000000000002,"5min":0.26200000000000001,"15min":0.254},"max":{"1min":0.71399999999999997,"5min":7.3049999999999997,"15min":28.106000000000002},"last":0.61699999999999999}]},{"osd":2,"last update":"Wed Sep 11 09:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":1.0660000000000001,"15min":1.3140000000000001},"min":{"1min":0.45000000000000001,"5min":0.28899999999999998,"15min":0.23999999999999999},"max":{"1min":1.4159999999999999,"5min":5.7690000000000001,"15min":67.486999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.0349999999999999,"15min":1.464},"min":{"1min":0.33400000000000002,"5min":0.27700000000000002,"15min":0.27400000000000002},"max":{"1min":4.7789999999999999,"5min":5.306,"15min":67.272000000000006},"last":0.72199999999999998}]},{"osd":3,"last update":"Wed Sep 11 09:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":1.1499999999999999,"15min":1.302},"min":{"1min":0.36099999999999999,"5min":0.28199999999999997,"15min":0.26700000000000002},"max":{"1min":1.8069999999999999,"5min":13.871,"15min":27.795000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":1.161,"15min":1.3440000000000001},"min":{"1min":0.46600000000000003,"5min":0.29099999999999998,"15min":0.28199999999999997},"max":{"1min":1.4570000000000001,"5min":12.318,"15min":27.841999999999999},"last":0.67100000000000004}]},{"osd":4,"last update":"Wed Sep 11 09:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.77600000000000002,"15min":1.1200000000000001},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.23300000000000001},"max":{"1min":0.83499999999999996,"5min":5.5940000000000003,"15min":40.927999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.66800000000000004,"15min":1.0329999999999999},"min":{"1min":0.30499999999999999,"5min":0.26200000000000001,"15min":0.25700000000000001},"max":{"1min":0.80900000000000005,"5min":2.3860000000000001,"15min":40.706000000000003},"last":0.70399999999999996}]},{"osd":5,"last update":"Wed Sep 11 09:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.86399999999999999,"15min":1.274},"min":{"1min":0.34599999999999997,"5min":0.29599999999999999,"15min":0.23899999999999999},"max":{"1min":0.91400000000000003,"5min":5.75,"15min":40.847000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.83899999999999997,"15min":1.163},"min":{"1min":0.316,"5min":0.27500000000000002,"15min":0.27100000000000002},"max":{"1min":0.88100000000000001,"5min":3.746,"15min":41.401000000000003},"last":0.65700000000000003}]},{"osd":7,"last update":"Wed Sep 11 09:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.66400000000000003,"15min":1.093},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.74299999999999999,"5min":3.0419999999999998,"15min":40.426000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.68000000000000005,"15min":1.091},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.83499999999999996,"5min":3.0099999999999998,"15min":45.401000000000003},"last":0.76300000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757555,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":894972,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":893950,"kb_avail":92850180,"statfs":{"total":95995035648,"available":95078584320,"internally_reserved":0,"allocated":1032192,"data_stored":709864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":915405144},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":1.0449999999999999,"15min":1.113},"min":{"1min":0.34000000000000002,"5min":0.222,"15min":0.21099999999999999},"max":{"1min":0.97399999999999998,"5min":25.09,"15min":25.09},"last":0.628},{"interface":"front","average":{"1min":0.54400000000000004,"5min":1.1379999999999999,"15min":1.0860000000000001},"min":{"1min":0.309,"5min":0.23899999999999999,"15min":0.20999999999999999},"max":{"1min":1.0409999999999999,"5min":23.628,"15min":23.628},"last":0.51800000000000002}]},{"osd":1,"last update":"Wed Sep 11 09:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":1.601,"15min":1.274},"min":{"1min":0.41699999999999998,"5min":0.23999999999999999,"15min":0.216},"max":{"1min":0.98499999999999999,"5min":49.965000000000003,"15min":49.965000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":1.601,"15min":1.2589999999999999},"min":{"1min":0.32100000000000001,"5min":0.24399999999999999,"15min":0.23400000000000001},"max":{"1min":0.94399999999999995,"5min":48.692,"15min":48.692},"last":0.50800000000000001}]},{"osd":2,"last update":"Wed Sep 11 09:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":1.7410000000000001,"15min":1.296},"min":{"1min":0.34999999999999998,"5min":0.26100000000000001,"15min":0.253},"max":{"1min":0.93000000000000005,"5min":48.331000000000003,"15min":48.331000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":1.52,"15min":1.2749999999999999},"min":{"1min":0.33300000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.92700000000000005,"5min":50.441000000000003,"15min":50.441000000000003},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Sep 11 09:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":1.9350000000000001,"15min":1.6870000000000001},"min":{"1min":0.41399999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.94699999999999995,"5min":51.340000000000003,"15min":85.299999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":1.988,"15min":1.7150000000000001},"min":{"1min":0.374,"5min":0.29399999999999998,"15min":0.29099999999999998},"max":{"1min":0.999,"5min":51.889000000000003,"15min":85.941000000000003},"last":0.58299999999999996}]},{"osd":4,"last update":"Wed Sep 11 09:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.51100000000000001,"15min":0.84199999999999997},"min":{"1min":0.27300000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.94499999999999995,"5min":2.5009999999999999,"15min":16.021999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.61599999999999999,"15min":0.83799999999999997},"min":{"1min":0.28599999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.96299999999999997,"5min":3.0369999999999999,"15min":9.6140000000000008},"last":0.35699999999999998}]},{"osd":6,"last update":"Wed Sep 11 09:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.64500000000000002,"15min":0.93999999999999995},"min":{"1min":0.29799999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.92000000000000004,"5min":2.4489999999999998,"15min":8.8059999999999992},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.67600000000000005,"15min":0.96299999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27300000000000002},"max":{"1min":0.88700000000000001,"5min":3.9470000000000001,"15min":8.1920000000000002},"last":0.45100000000000001}]},{"osd":7,"last update":"Wed Sep 11 09:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.68000000000000005,"15min":1.25},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.98099999999999998,"5min":3.448,"15min":88.513000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.72799999999999998,"15min":1.256},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.29599999999999999},"max":{"1min":1.0609999999999999,"5min":3.306,"15min":86.963999999999999},"last":0.54500000000000004}]}]},{"osd":4,"up_from":31,"seq":133143986488,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":889972,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":889214,"kb_avail":92855180,"statfs":{"total":95995035648,"available":95083704320,"internally_reserved":0,"allocated":765952,"data_stored":446005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":910555480},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":2.919,"15min":1.8979999999999999},"min":{"1min":0.42699999999999999,"5min":0.20999999999999999,"15min":0.20499999999999999},"max":{"1min":0.78400000000000003,"5min":206.922,"15min":206.922},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":2.9409999999999998,"15min":1.9850000000000001},"min":{"1min":0.35299999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.79200000000000004,"5min":206.636,"15min":206.636},"last":0.51000000000000001}]},{"osd":1,"last update":"Wed Sep 11 09:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":1.119,"15min":1.544},"min":{"1min":0.34200000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.65500000000000003,"5min":24.946000000000002,"15min":48.976999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":1.2729999999999999,"15min":1.5700000000000001},"min":{"1min":0.36699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.80200000000000005,"5min":25.035,"15min":45.787999999999997},"last":0.49199999999999999}]},{"osd":2,"last update":"Wed Sep 11 09:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":1.3169999999999999,"15min":2.1360000000000001},"min":{"1min":0.38600000000000001,"5min":0.248,"15min":0.21099999999999999},"max":{"1min":0.77800000000000002,"5min":44.311,"15min":255.88900000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":1.3640000000000001,"15min":2.214},"min":{"1min":0.40100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.85299999999999998,"5min":40.161999999999999,"15min":255.41200000000001},"last":0.60299999999999998}]},{"osd":3,"last update":"Wed Sep 11 09:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":1.3520000000000001,"15min":1.601},"min":{"1min":0.40200000000000002,"5min":0.26900000000000002,"15min":0.252},"max":{"1min":0.77000000000000002,"5min":21.652999999999999,"15min":45.268000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":1.181,"15min":1.504},"min":{"1min":0.35499999999999998,"5min":0.28599999999999998,"15min":0.25900000000000001},"max":{"1min":0.81499999999999995,"5min":21.556000000000001,"15min":44.234000000000002},"last":0.47499999999999998}]},{"osd":5,"last update":"Wed Sep 11 09:02:27 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.63900000000000001,"15min":0.873},"min":{"1min":0.316,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.73599999999999999,"5min":3.524,"15min":11.045999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.68200000000000005,"15min":0.86699999999999999},"min":{"1min":0.28999999999999998,"5min":0.26500000000000001,"15min":0.254},"max":{"1min":0.747,"5min":4.7119999999999997,"15min":10.526999999999999},"last":0.63300000000000001}]},{"osd":6,"last update":"Wed Sep 11 09:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.73499999999999999,"15min":1.2110000000000001},"min":{"1min":0.29999999999999999,"5min":0.27000000000000002,"15min":0.26800000000000002},"max":{"1min":0.61499999999999999,"5min":3.3519999999999999,"15min":72.100999999999999},"last":0.443},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.79800000000000004,"15min":1.272},"min":{"1min":0.33200000000000002,"5min":0.28000000000000003,"15min":0.22800000000000001},"max":{"1min":0.622,"5min":6.5359999999999996,"15min":74.527000000000001},"last":0.42699999999999999}]},{"osd":7,"last update":"Wed Sep 11 09:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.80700000000000005,"15min":0.87},"min":{"1min":0.31900000000000001,"5min":0.31,"15min":0.253},"max":{"1min":0.69899999999999995,"5min":5.2560000000000002,"15min":6.907},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.78600000000000003,"15min":0.98499999999999999},"min":{"1min":0.35499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73099999999999998,"5min":3.6560000000000001,"15min":6.8029999999999999},"last":0.55300000000000005}]}]},{"osd":3,"up_from":26,"seq":111669150014,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":904720,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":903358,"kb_avail":92840432,"statfs":{"total":95995035648,"available":95068602368,"internally_reserved":0,"allocated":1384448,"data_stored":1037954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":925038936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":1.038,"15min":1.022},"min":{"1min":0.20399999999999999,"5min":0.17499999999999999,"15min":0.16500000000000001},"max":{"1min":0.54200000000000004,"5min":15.601000000000001,"15min":24.280000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.34799999999999998,"5min":1.3100000000000001,"15min":1.1599999999999999},"min":{"1min":0.192,"5min":0.192,"15min":0.184},"max":{"1min":0.53000000000000003,"5min":16.492999999999999,"15min":22.225000000000001},"last":0.30199999999999999}]},{"osd":1,"last update":"Wed Sep 11 09:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":1.321,"15min":1.4630000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.20100000000000001},"max":{"1min":0.501,"5min":16.559000000000001,"15min":84.284999999999997},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":1.4239999999999999,"15min":1.4790000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20499999999999999},"max":{"1min":0.60199999999999998,"5min":13.686,"15min":85.832999999999998},"last":0.26200000000000001}]},{"osd":2,"last update":"Wed Sep 11 09:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":1.3120000000000001,"15min":1.45},"min":{"1min":0.23699999999999999,"5min":0.23100000000000001,"15min":0.22500000000000001},"max":{"1min":0.57199999999999995,"5min":13.627000000000001,"15min":86.766000000000005},"last":0.317},{"interface":"front","average":{"1min":0.36299999999999999,"5min":1.3939999999999999,"15min":1.45},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.22700000000000001},"max":{"1min":0.58499999999999996,"5min":15.621,"15min":85.367999999999995},"last":0.35599999999999998}]},{"osd":4,"last update":"Wed Sep 11 09:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":1.393,"15min":1.393},"min":{"1min":0.44,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.84899999999999998,"5min":13.663,"15min":13.663},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":1.2030000000000001,"15min":1.1459999999999999},"min":{"1min":0.36599999999999999,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":0.875,"5min":17.079999999999998,"15min":17.079999999999998},"last":0.57199999999999995}]},{"osd":5,"last update":"Wed Sep 11 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":1.321,"15min":1.288},"min":{"1min":0.41799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.74299999999999999,"5min":13.736000000000001,"15min":13.736000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":1.274,"15min":1.2010000000000001},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.28899999999999998},"max":{"1min":0.77200000000000002,"5min":17.048999999999999,"15min":17.048999999999999},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed Sep 11 09:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":1.266,"15min":1.446},"min":{"1min":0.42199999999999999,"5min":0.34599999999999997,"15min":0.29999999999999999},"max":{"1min":0.86799999999999999,"5min":13.755000000000001,"15min":33.106999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":1.371,"15min":1.365},"min":{"1min":0.41399999999999998,"5min":0.33200000000000002,"15min":0.29299999999999998},"max":{"1min":0.79400000000000004,"5min":16.527000000000001,"15min":33.076000000000001},"last":0.67000000000000004}]},{"osd":7,"last update":"Wed Sep 11 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":1.3580000000000001,"15min":1.5589999999999999},"min":{"1min":0.41499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94199999999999995,"5min":13.712999999999999,"15min":35.073},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":1.327,"15min":1.4379999999999999},"min":{"1min":0.41399999999999998,"5min":0.39300000000000002,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":17,"15min":34.844999999999999},"last":0.69399999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411650,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":892776,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":891774,"kb_avail":92852376,"statfs":{"total":95995035648,"available":95080833024,"internally_reserved":0,"allocated":1015808,"data_stored":708514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":913176920},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":2.3759999999999999,"15min":1.5589999999999999},"min":{"1min":0.25,"5min":0.192,"15min":0.189},"max":{"1min":0.47599999999999998,"5min":130.703,"15min":130.703},"last":0.439},{"interface":"front","average":{"1min":0.377,"5min":2.3730000000000002,"15min":1.6240000000000001},"min":{"1min":0.249,"5min":0.186,"15min":0.17399999999999999},"max":{"1min":0.52600000000000002,"5min":130.64699999999999,"15min":130.64699999999999},"last":0.34399999999999997}]},{"osd":1,"last update":"Wed Sep 11 09:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":1.6000000000000001,"15min":1.571},"min":{"1min":0.307,"5min":0.182,"15min":0.182},"max":{"1min":0.58399999999999996,"5min":51.231999999999999,"15min":72.867000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":1.8400000000000001,"15min":1.75},"min":{"1min":0.28899999999999998,"5min":0.20499999999999999,"15min":0.20399999999999999},"max":{"1min":0.53600000000000003,"5min":47.674999999999997,"15min":67.799000000000007},"last":0.40100000000000002}]},{"osd":3,"last update":"Wed Sep 11 09:02:39 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":2.3500000000000001,"15min":1.841},"min":{"1min":0.25900000000000001,"5min":0.219,"15min":0.186},"max":{"1min":0.54900000000000004,"5min":50.173000000000002,"15min":64.795000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":2.1890000000000001,"15min":1.7470000000000001},"min":{"1min":0.27900000000000003,"5min":0.23000000000000001,"15min":0.216},"max":{"1min":0.54900000000000004,"5min":49.935000000000002,"15min":64.632999999999996},"last":0.42799999999999999}]},{"osd":4,"last update":"Wed Sep 11 09:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":1.133,"15min":1.2549999999999999},"min":{"1min":0.41299999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.94299999999999995,"5min":7.3650000000000002,"15min":52.113999999999997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.98099999999999998,"15min":1.2150000000000001},"min":{"1min":0.40899999999999997,"5min":0.28899999999999998,"15min":0.27300000000000002},"max":{"1min":0.82099999999999995,"5min":7.0810000000000004,"15min":52.097999999999999},"last":0.49399999999999999}]},{"osd":5,"last update":"Wed Sep 11 09:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":1.0740000000000001,"15min":1.276},"min":{"1min":0.46600000000000003,"5min":0.311,"15min":0.29799999999999999},"max":{"1min":0.88700000000000001,"5min":7.1219999999999999,"15min":52.145000000000003},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.62,"5min":1.034,"15min":1.1519999999999999},"min":{"1min":0.41399999999999998,"5min":0.29399999999999998,"15min":0.28999999999999998},"max":{"1min":0.77500000000000002,"5min":6.5880000000000001,"15min":52.216000000000001},"last":0.44500000000000001}]},{"osd":6,"last update":"Wed Sep 11 09:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":3.5470000000000002,"15min":2.1070000000000002},"min":{"1min":0.45900000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.97299999999999998,"5min":246.482,"15min":246.482},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.625,"5min":3.5249999999999999,"15min":2.1299999999999999},"min":{"1min":0.48699999999999999,"5min":0.34899999999999998,"15min":0.33500000000000002},"max":{"1min":0.79000000000000004,"5min":246.184,"15min":246.184},"last":0.55900000000000005}]},{"osd":7,"last update":"Wed Sep 11 09:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":1.0369999999999999,"15min":1.264},"min":{"1min":0.40300000000000002,"5min":0.33800000000000002,"15min":0.32200000000000001},"max":{"1min":0.78000000000000003,"5min":6.6289999999999996,"15min":84.108999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.626,"5min":1.135,"15min":1.4199999999999999},"min":{"1min":0.47599999999999998,"5min":0.30599999999999999,"15min":0.30299999999999999},"max":{"1min":0.91100000000000003,"5min":7.415,"15min":85.938000000000002},"last":0.55100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575174,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":904468,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":903678,"kb_avail":92840684,"statfs":{"total":95995035648,"available":95068860416,"internally_reserved":0,"allocated":794624,"data_stored":446965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":925366616},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.94499999999999995,"15min":1.3160000000000001},"min":{"1min":0.249,"5min":0.186,"15min":0.16600000000000001},"max":{"1min":0.48899999999999999,"5min":7.8220000000000001,"15min":56.613},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.37,"5min":1.024,"15min":1.417},"min":{"1min":0.248,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.622,"5min":7.0439999999999996,"15min":55.533999999999999},"last":0.32000000000000001}]},{"osd":2,"last update":"Wed Sep 11 09:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":1.3029999999999999,"15min":1.885},"min":{"1min":0.219,"5min":0.219,"15min":0.19},"max":{"1min":0.64600000000000002,"5min":12.596,"15min":169.607},"last":0.436},{"interface":"front","average":{"1min":0.34799999999999998,"5min":1.1659999999999999,"15min":1.7969999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20000000000000001},"max":{"1min":0.47499999999999998,"5min":10.243,"15min":169.762},"last":0.378}]},{"osd":3,"last update":"Wed Sep 11 09:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":2.5150000000000001,"15min":1.577},"min":{"1min":0.29499999999999998,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.48199999999999998,"5min":70.478999999999999,"15min":70.478999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":2.9460000000000002,"15min":1.679},"min":{"1min":0.26300000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.76400000000000001,"5min":70.337999999999994,"15min":70.337999999999994},"last":0.49399999999999999}]},{"osd":4,"last update":"Wed Sep 11 09:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":1.1739999999999999,"15min":1.157},"min":{"1min":0.30099999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76200000000000001,"5min":21.786000000000001,"15min":21.786000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":1.206,"15min":1.1519999999999999},"min":{"1min":0.36199999999999999,"5min":0.32300000000000001,"15min":0.23599999999999999},"max":{"1min":0.86799999999999999,"5min":21.815999999999999,"15min":21.815999999999999},"last":0.42099999999999999}]},{"osd":5,"last update":"Wed Sep 11 09:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":1.1910000000000001,"15min":1.2849999999999999},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.27600000000000002},"max":{"1min":1.1739999999999999,"5min":15.664999999999999,"15min":38.098999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":1.1779999999999999,"15min":1.335},"min":{"1min":0.38900000000000001,"5min":0.307,"15min":0.307},"max":{"1min":1.2270000000000001,"5min":15.199,"15min":38.130000000000003},"last":0.64500000000000002}]},{"osd":6,"last update":"Wed Sep 11 09:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":1.0509999999999999,"15min":1.1559999999999999},"min":{"1min":0.376,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.1279999999999999,"5min":7.8840000000000003,"15min":7.8840000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.622,"5min":1.202,"15min":1.1479999999999999},"min":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.315},"max":{"1min":0.82999999999999996,"5min":6.1740000000000004,"15min":6.2869999999999999},"last":0.63}]},{"osd":7,"last update":"Wed Sep 11 09:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":1.323,"15min":1.1859999999999999},"min":{"1min":0.41899999999999998,"5min":0.34799999999999998,"15min":0.31900000000000001},"max":{"1min":0.82999999999999996,"5min":14.994,"15min":14.994},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":1.444,"15min":1.2649999999999999},"min":{"1min":0.46000000000000002,"5min":0.35999999999999999,"15min":0.30199999999999999},"max":{"1min":0.83799999999999997,"5min":22.215,"15min":22.215},"last":0.53600000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738698,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":906172,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":905086,"kb_avail":92838980,"statfs":{"total":95995035648,"available":95067115520,"internally_reserved":0,"allocated":1097728,"data_stored":774282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":926808408},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 09:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":1.6120000000000001,"15min":1.528},"min":{"1min":0.17100000000000001,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.65000000000000002,"5min":42.197000000000003,"15min":56.685000000000002},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":1.6319999999999999,"15min":1.5840000000000001},"min":{"1min":0.22600000000000001,"5min":0.19,"15min":0.188},"max":{"1min":5.3399999999999999,"5min":41.822000000000003,"15min":61.012999999999998},"last":0.27900000000000003}]},{"osd":2,"last update":"Wed Sep 11 09:02:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":2.4100000000000001,"15min":1.466},"min":{"1min":0.25800000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":5.4020000000000001,"5min":111.41200000000001,"15min":111.41200000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":2.6120000000000001,"15min":1.556},"min":{"1min":0.247,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":4.79,"5min":111.637,"15min":111.637},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed Sep 11 09:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":1.823,"15min":1.5640000000000001},"min":{"1min":0.27200000000000002,"5min":0.20000000000000001,"15min":0.19800000000000001},"max":{"1min":0.58099999999999996,"5min":41.131999999999998,"15min":56.095999999999997},"last":0.375},{"interface":"front","average":{"1min":0.36399999999999999,"5min":1.4770000000000001,"15min":1.502},"min":{"1min":0.26700000000000002,"5min":0.24199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52100000000000002,"5min":41.173999999999999,"15min":59.651000000000003},"last":0.34100000000000003}]},{"osd":4,"last update":"Wed Sep 11 09:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.68600000000000005,"15min":1.7430000000000001},"min":{"1min":0.378,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.81299999999999994,"5min":5.2119999999999997,"15min":168.72200000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.83399999999999996,"15min":1.833},"min":{"1min":0.36599999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":5.8579999999999997,"5min":5.8579999999999997,"15min":168.88999999999999},"last":0.54600000000000004}]},{"osd":5,"last update":"Wed Sep 11 09:02:26 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":1.0129999999999999,"15min":1.163},"min":{"1min":0.38600000000000001,"5min":0.311,"15min":0.30599999999999999},"max":{"1min":4.6639999999999997,"5min":7.2649999999999997,"15min":7.7830000000000004},"last":0.496},{"interface":"front","average":{"1min":0.621,"5min":0.79900000000000004,"15min":1.145},"min":{"1min":0.36099999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.85899999999999999,"5min":5.2359999999999998,"15min":9.5239999999999991},"last":0.42399999999999999}]},{"osd":6,"last update":"Wed Sep 11 09:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":1.1899999999999999,"15min":1.304},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.30099999999999999},"max":{"1min":0.96499999999999997,"5min":16.196000000000002,"15min":25.027000000000001},"last":0.435},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.90300000000000002,"15min":1.2789999999999999},"min":{"1min":0.47699999999999998,"5min":0.32200000000000001,"15min":0.29399999999999998},"max":{"1min":0.84499999999999997,"5min":16.585999999999999,"15min":25.076000000000001},"last":0.504}]},{"osd":7,"last update":"Wed Sep 11 09:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":1.264,"15min":1.484},"min":{"1min":0.40100000000000002,"5min":0.33300000000000002,"15min":0.32800000000000001},"max":{"1min":5.8929999999999998,"5min":20.399000000000001,"15min":56.119},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":1.468,"15min":1.6830000000000001},"min":{"1min":0.443,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":4.7610000000000001,"5min":20.286999999999999,"15min":56.100000000000001},"last":0.45200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:03:04.001 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T09:03:04.249 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:03:04.517 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:04 smithi092 ceph-mon[29647]: from='client.16389 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T09:03:04.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:04 smithi092 ceph-mon[29647]: 4.1c deep-scrub starts 2024-09-11T09:03:04.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:04 smithi092 ceph-mon[29647]: 4.1c deep-scrub ok 2024-09-11T09:03:04.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:04 smithi092 ceph-mon[29647]: pgmap v1112: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:03:04.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:04 smithi092 ceph-mon[29647]: 6.10 scrub starts 2024-09-11T09:03:04.518 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:04 smithi092 ceph-mon[29647]: 6.10 scrub ok 2024-09-11T09:03:04.518 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:04 smithi092 ceph-mon[41114]: from='client.16389 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T09:03:04.518 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:04 smithi092 ceph-mon[41114]: 4.1c deep-scrub starts 2024-09-11T09:03:04.518 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:04 smithi092 ceph-mon[41114]: 4.1c deep-scrub ok 2024-09-11T09:03:04.518 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:04 smithi092 ceph-mon[41114]: pgmap v1112: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:03:04.518 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:04 smithi092 ceph-mon[41114]: 6.10 scrub starts 2024-09-11T09:03:04.519 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:04 smithi092 ceph-mon[41114]: 6.10 scrub ok 2024-09-11T09:03:04.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:04 smithi145 ceph-mon[33384]: from='client.16389 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T09:03:04.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:04 smithi145 ceph-mon[33384]: 4.1c deep-scrub starts 2024-09-11T09:03:04.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:04 smithi145 ceph-mon[33384]: 4.1c deep-scrub ok 2024-09-11T09:03:04.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:04 smithi145 ceph-mon[33384]: pgmap v1112: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:03:04.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:04 smithi145 ceph-mon[33384]: 6.10 scrub starts 2024-09-11T09:03:04.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:04 smithi145 ceph-mon[33384]: 6.10 scrub ok 2024-09-11T09:03:05.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:05 smithi092 ceph-mon[41114]: 5.3 scrub starts 2024-09-11T09:03:05.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:05 smithi092 ceph-mon[41114]: 5.3 scrub ok 2024-09-11T09:03:05.489 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:05 smithi092 ceph-mon[29647]: 5.3 scrub starts 2024-09-11T09:03:05.489 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:05 smithi092 ceph-mon[29647]: 5.3 scrub ok 2024-09-11T09:03:05.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:05 smithi145 ceph-mon[33384]: 5.3 scrub starts 2024-09-11T09:03:05.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:05 smithi145 ceph-mon[33384]: 5.3 scrub ok 2024-09-11T09:03:06.190 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-11T09:03:06.192 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-11T09:03:06.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:06 smithi092 ceph-mon[29647]: 5.6 deep-scrub starts 2024-09-11T09:03:06.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:06 smithi092 ceph-mon[29647]: 5.6 deep-scrub ok 2024-09-11T09:03:06.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:06 smithi092 ceph-mon[29647]: pgmap v1113: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 994 B/s rd, 0 op/s 2024-09-11T09:03:06.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:06 smithi092 ceph-mon[29647]: 4.1f scrub starts 2024-09-11T09:03:06.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:06 smithi092 ceph-mon[29647]: 4.1f scrub ok 2024-09-11T09:03:06.564 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:06 smithi092 ceph-mon[41114]: 5.6 deep-scrub starts 2024-09-11T09:03:06.564 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:06 smithi092 ceph-mon[41114]: 5.6 deep-scrub ok 2024-09-11T09:03:06.564 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:06 smithi092 ceph-mon[41114]: pgmap v1113: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 994 B/s rd, 0 op/s 2024-09-11T09:03:06.564 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:06 smithi092 ceph-mon[41114]: 4.1f scrub starts 2024-09-11T09:03:06.564 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:06 smithi092 ceph-mon[41114]: 4.1f scrub ok 2024-09-11T09:03:06.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:06 smithi145 ceph-mon[33384]: 5.6 deep-scrub starts 2024-09-11T09:03:06.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:06 smithi145 ceph-mon[33384]: 5.6 deep-scrub ok 2024-09-11T09:03:06.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:06 smithi145 ceph-mon[33384]: pgmap v1113: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 994 B/s rd, 0 op/s 2024-09-11T09:03:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:06 smithi145 ceph-mon[33384]: 4.1f scrub starts 2024-09-11T09:03:06.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:06 smithi145 ceph-mon[33384]: 4.1f scrub ok 2024-09-11T09:03:06.903 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":1113,"stamp":"2024-09-11T09:03:05.533606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1694,"num_read_kb":1446,"num_write":1002,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":426,"num_bytes_recovered":10586,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2889,"ondisk_log_size":2889,"up":456,"acting":456,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":456,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":7185160,"kb_used_data":7748,"kb_used_omap":13,"kb_used_meta":7177330,"kb_avail":742776056,"statfs":{"total":767960285184,"available":760602681344,"internally_reserved":0,"allocated":7933952,"data_stored":5344076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":7349586624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.293796"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":147,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932752+0000","last_change":"2024-09-11T09:02:46.954818+0000","last_active":"2024-09-11T09:02:47.932752+0000","last_peered":"2024-09-11T09:02:47.932752+0000","last_clean":"2024-09-11T09:02:47.932752+0000","last_became_active":"2024-09-11T09:01:25.400173+0000","last_became_peered":"2024-09-11T09:01:25.400173+0000","last_unstale":"2024-09-11T09:02:47.932752+0000","last_undegraded":"2024-09-11T09:02:47.932752+0000","last_fullsized":"2024-09-11T09:02:47.932752+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:46.954763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:41.077950+0000","last_clean_scrub_stamp":"2024-09-11T09:02:46.954763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:00.566135+0000","scrub_duration":0.001632618,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"402'25","reported_seq":613,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212043+0000","last_change":"2024-09-11T09:02:23.846135+0000","last_active":"2024-09-11T09:02:47.212043+0000","last_peered":"2024-09-11T09:02:47.212043+0000","last_clean":"2024-09-11T09:02:47.212043+0000","last_became_active":"2024-09-11T08:58:34.124732+0000","last_became_peered":"2024-09-11T08:58:34.124732+0000","last_unstale":"2024-09-11T09:02:47.212043+0000","last_undegraded":"2024-09-11T09:02:47.212043+0000","last_fullsized":"2024-09-11T09:02:47.212043+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'25","last_scrub_stamp":"2024-09-11T09:02:23.846064+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:56.666703+0000","last_clean_scrub_stamp":"2024-09-11T09:02:23.846064+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:52.598290+0000","scrub_duration":0.0021792809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":532,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:54.777107+0000","last_change":"2024-09-11T09:02:54.777070+0000","last_active":"2024-09-11T09:02:54.777107+0000","last_peered":"2024-09-11T09:02:54.777107+0000","last_clean":"2024-09-11T09:02:54.777107+0000","last_became_active":"2024-09-11T08:38:42.980247+0000","last_became_peered":"2024-09-11T08:38:42.980247+0000","last_unstale":"2024-09-11T09:02:54.777107+0000","last_undegraded":"2024-09-11T09:02:54.777107+0000","last_fullsized":"2024-09-11T09:02:54.777107+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:54.777016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:54.777016+0000","last_clean_scrub_stamp":"2024-09-11T09:02:54.777016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:01.347708+0000","scrub_duration":0.0014251509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":511,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:54.836251+0000","last_change":"2024-09-11T09:02:54.836201+0000","last_active":"2024-09-11T09:02:54.836251+0000","last_peered":"2024-09-11T09:02:54.836251+0000","last_clean":"2024-09-11T09:02:54.836251+0000","last_became_active":"2024-09-11T09:01:28.501615+0000","last_became_peered":"2024-09-11T09:01:28.501615+0000","last_unstale":"2024-09-11T09:02:54.836251+0000","last_undegraded":"2024-09-11T09:02:54.836251+0000","last_fullsized":"2024-09-11T09:02:54.836251+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:54.836130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:54.836130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:56.488493+0000","scrub_duration":0.00153531,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":540,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:52.867364+0000","last_change":"2024-09-11T09:02:52.867322+0000","last_active":"2024-09-11T09:02:52.867364+0000","last_peered":"2024-09-11T09:02:52.867364+0000","last_clean":"2024-09-11T09:02:52.867364+0000","last_became_active":"2024-09-11T08:58:34.135162+0000","last_became_peered":"2024-09-11T08:58:34.135162+0000","last_unstale":"2024-09-11T09:02:52.867364+0000","last_undegraded":"2024-09-11T09:02:52.867364+0000","last_fullsized":"2024-09-11T09:02:52.867364+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:52.867256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:52.867256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:57.786100+0000","scrub_duration":0.0019533269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":539,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214940+0000","last_change":"2024-09-11T09:02:10.795660+0000","last_active":"2024-09-11T09:02:47.214940+0000","last_peered":"2024-09-11T09:02:47.214940+0000","last_clean":"2024-09-11T09:02:47.214940+0000","last_became_active":"2024-09-11T08:58:34.316048+0000","last_became_peered":"2024-09-11T08:58:34.316048+0000","last_unstale":"2024-09-11T09:02:47.214940+0000","last_undegraded":"2024-09-11T09:02:47.214940+0000","last_fullsized":"2024-09-11T09:02:47.214940+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:10.795606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:10.795606+0000","last_clean_scrub_stamp":"2024-09-11T09:02:10.795606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:18.892727+0000","scrub_duration":0.0017057629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"402'46","reported_seq":624,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214252+0000","last_change":"2024-09-11T09:02:45.005804+0000","last_active":"2024-09-11T09:02:47.214252+0000","last_peered":"2024-09-11T09:02:47.214252+0000","last_clean":"2024-09-11T09:02:47.214252+0000","last_became_active":"2024-09-11T08:52:18.585600+0000","last_became_peered":"2024-09-11T08:52:18.585600+0000","last_unstale":"2024-09-11T09:02:47.214252+0000","last_undegraded":"2024-09-11T09:02:47.214252+0000","last_fullsized":"2024-09-11T09:02:47.214252+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'46","last_scrub_stamp":"2024-09-11T09:02:45.005742+0000","last_deep_scrub":"200'34","last_deep_scrub_stamp":"2024-09-11T08:51:31.358660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:45.005742+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:10.137781+0000","scrub_duration":0.0017452469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":513,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212030+0000","last_change":"2024-09-11T09:01:46.683759+0000","last_active":"2024-09-11T09:02:47.212030+0000","last_peered":"2024-09-11T09:02:47.212030+0000","last_clean":"2024-09-11T09:02:47.212030+0000","last_became_active":"2024-09-11T08:59:18.590212+0000","last_became_peered":"2024-09-11T08:59:18.590212+0000","last_unstale":"2024-09-11T09:02:47.212030+0000","last_undegraded":"2024-09-11T09:02:47.212030+0000","last_fullsized":"2024-09-11T09:02:47.212030+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:46.683708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:43:28.988697+0000","last_clean_scrub_stamp":"2024-09-11T09:01:46.683708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001532423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":552,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:48.073383+0000","last_change":"2024-09-11T09:02:48.073333+0000","last_active":"2024-09-11T09:02:48.073383+0000","last_peered":"2024-09-11T09:02:48.073383+0000","last_clean":"2024-09-11T09:02:48.073383+0000","last_became_active":"2024-09-11T08:58:34.153391+0000","last_became_peered":"2024-09-11T08:58:34.153391+0000","last_unstale":"2024-09-11T09:02:48.073383+0000","last_undegraded":"2024-09-11T09:02:48.073383+0000","last_fullsized":"2024-09-11T09:02:48.073383+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:48.073266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:57.348997+0000","last_clean_scrub_stamp":"2024-09-11T09:02:48.073266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:02.982251+0000","scrub_duration":0.001572642,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"402'25","reported_seq":573,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212521+0000","last_change":"2024-09-11T09:01:23.873666+0000","last_active":"2024-09-11T09:02:47.212521+0000","last_peered":"2024-09-11T09:02:47.212521+0000","last_clean":"2024-09-11T09:02:47.212521+0000","last_became_active":"2024-09-11T08:52:18.577411+0000","last_became_peered":"2024-09-11T08:52:18.577411+0000","last_unstale":"2024-09-11T09:02:47.212521+0000","last_undegraded":"2024-09-11T09:02:47.212521+0000","last_fullsized":"2024-09-11T09:02:47.212521+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'25","last_scrub_stamp":"2024-09-11T09:01:23.873601+0000","last_deep_scrub":"78'13","last_deep_scrub_stamp":"2024-09-11T08:39:46.620624+0000","last_clean_scrub_stamp":"2024-09-11T09:01:23.873601+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001935911,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":528,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210444+0000","last_change":"2024-09-11T09:01:54.331887+0000","last_active":"2024-09-11T09:02:47.210444+0000","last_peered":"2024-09-11T09:02:47.210444+0000","last_clean":"2024-09-11T09:02:47.210444+0000","last_became_active":"2024-09-11T08:52:18.573461+0000","last_became_peered":"2024-09-11T08:52:18.573461+0000","last_unstale":"2024-09-11T09:02:47.210444+0000","last_undegraded":"2024-09-11T09:02:47.210444+0000","last_fullsized":"2024-09-11T09:02:47.210444+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:54.331833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:01:54.331833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:22.433718+0000","scrub_duration":0.0012485619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":137,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932274+0000","last_change":"2024-09-11T09:02:19.871321+0000","last_active":"2024-09-11T09:02:47.932274+0000","last_peered":"2024-09-11T09:02:47.932274+0000","last_clean":"2024-09-11T09:02:47.932274+0000","last_became_active":"2024-09-11T09:01:25.419172+0000","last_became_peered":"2024-09-11T09:01:25.419172+0000","last_unstale":"2024-09-11T09:02:47.932274+0000","last_undegraded":"2024-09-11T09:02:47.932274+0000","last_fullsized":"2024-09-11T09:02:47.932274+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:19.871233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:19.871233+0000","last_clean_scrub_stamp":"2024-09-11T09:02:19.871233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:31.660758+0000","scrub_duration":0.001544405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":512,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:52.037618+0000","last_change":"2024-09-11T09:02:52.037560+0000","last_active":"2024-09-11T09:02:52.037618+0000","last_peered":"2024-09-11T09:02:52.037618+0000","last_clean":"2024-09-11T09:02:52.037618+0000","last_became_active":"2024-09-11T08:38:38.950253+0000","last_became_peered":"2024-09-11T08:38:38.950253+0000","last_unstale":"2024-09-11T09:02:52.037618+0000","last_undegraded":"2024-09-11T09:02:52.037618+0000","last_fullsized":"2024-09-11T09:02:52.037618+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:52.037494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:30.465003+0000","last_clean_scrub_stamp":"2024-09-11T09:02:52.037494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:10.885814+0000","scrub_duration":0.0018698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"402'9","reported_seq":536,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212501+0000","last_change":"2024-09-11T09:02:07.768204+0000","last_active":"2024-09-11T09:02:47.212501+0000","last_peered":"2024-09-11T09:02:47.212501+0000","last_clean":"2024-09-11T09:02:47.212501+0000","last_became_active":"2024-09-11T09:01:28.501477+0000","last_became_peered":"2024-09-11T09:01:28.501477+0000","last_unstale":"2024-09-11T09:02:47.212501+0000","last_undegraded":"2024-09-11T09:02:47.212501+0000","last_fullsized":"2024-09-11T09:02:47.212501+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'9","last_scrub_stamp":"2024-09-11T09:02:07.768083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:07.768083+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:23.707991+0000","scrub_duration":0.0024195089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":534,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931672+0000","last_change":"2024-09-11T09:02:01.515128+0000","last_active":"2024-09-11T09:02:47.931672+0000","last_peered":"2024-09-11T09:02:47.931672+0000","last_clean":"2024-09-11T09:02:47.931672+0000","last_became_active":"2024-09-11T08:38:42.984579+0000","last_became_peered":"2024-09-11T08:38:42.984579+0000","last_unstale":"2024-09-11T09:02:47.931672+0000","last_undegraded":"2024-09-11T09:02:47.931672+0000","last_fullsized":"2024-09-11T09:02:47.931672+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:01.515072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:18.199651+0000","last_clean_scrub_stamp":"2024-09-11T09:02:01.515072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:25.439344+0000","scrub_duration":0.001295526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":506,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:58.064660+0000","last_change":"2024-09-11T09:02:58.064627+0000","last_active":"2024-09-11T09:02:58.064660+0000","last_peered":"2024-09-11T09:02:58.064660+0000","last_clean":"2024-09-11T09:02:58.064660+0000","last_became_active":"2024-09-11T08:38:44.993982+0000","last_became_peered":"2024-09-11T08:38:44.993982+0000","last_unstale":"2024-09-11T09:02:58.064660+0000","last_undegraded":"2024-09-11T09:02:58.064660+0000","last_fullsized":"2024-09-11T09:02:58.064660+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:58.064565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:42:40.554717+0000","last_clean_scrub_stamp":"2024-09-11T09:02:58.064565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:06.910515+0000","scrub_duration":0.0015018880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":521,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212884+0000","last_change":"2024-09-11T09:01:35.704717+0000","last_active":"2024-09-11T09:02:47.212884+0000","last_peered":"2024-09-11T09:02:47.212884+0000","last_clean":"2024-09-11T09:02:47.212884+0000","last_became_active":"2024-09-11T08:58:34.126085+0000","last_became_peered":"2024-09-11T08:58:34.126085+0000","last_unstale":"2024-09-11T09:02:47.212884+0000","last_undegraded":"2024-09-11T09:02:47.212884+0000","last_fullsized":"2024-09-11T09:02:47.212884+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:35.704629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:01:35.704629+0000","last_clean_scrub_stamp":"2024-09-11T09:01:35.704629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015126740000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":137,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931969+0000","last_change":"2024-09-11T09:01:35.972995+0000","last_active":"2024-09-11T09:02:47.931969+0000","last_peered":"2024-09-11T09:02:47.931969+0000","last_clean":"2024-09-11T09:02:47.931969+0000","last_became_active":"2024-09-11T09:01:25.404021+0000","last_became_peered":"2024-09-11T09:01:25.404021+0000","last_unstale":"2024-09-11T09:02:47.931969+0000","last_undegraded":"2024-09-11T09:02:47.931969+0000","last_fullsized":"2024-09-11T09:02:47.931969+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:35.972939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:01:35.972939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:03.356777+0000","scrub_duration":0.0014110139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"402'46","reported_seq":426,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931149+0000","last_change":"2024-09-11T09:01:39.553753+0000","last_active":"2024-09-11T09:02:47.931149+0000","last_peered":"2024-09-11T09:02:47.931149+0000","last_clean":"2024-09-11T09:02:47.931149+0000","last_became_active":"2024-09-11T09:01:39.553406+0000","last_became_peered":"2024-09-11T09:01:39.553406+0000","last_unstale":"2024-09-11T09:02:47.931149+0000","last_undegraded":"2024-09-11T09:02:47.931149+0000","last_fullsized":"2024-09-11T09:02:47.931149+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'46","last_scrub_stamp":"2024-09-11T09:01:28.169023+0000","last_deep_scrub":"78'22","last_deep_scrub_stamp":"2024-09-11T08:39:50.472368+0000","last_clean_scrub_stamp":"2024-09-11T09:01:28.169023+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:02:56.682751+0000","scrub_duration":0.006483768,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":51,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":552,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:48.710058+0000","last_change":"2024-09-11T09:02:48.710014+0000","last_active":"2024-09-11T09:02:48.710058+0000","last_peered":"2024-09-11T09:02:48.710058+0000","last_clean":"2024-09-11T09:02:48.710058+0000","last_became_active":"2024-09-11T08:52:18.579026+0000","last_became_peered":"2024-09-11T08:52:18.579026+0000","last_unstale":"2024-09-11T09:02:48.710058+0000","last_undegraded":"2024-09-11T09:02:48.710058+0000","last_fullsized":"2024-09-11T09:02:48.710058+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:48.709961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:57:31.860874+0000","last_clean_scrub_stamp":"2024-09-11T09:02:48.709961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:56.818710+0000","scrub_duration":0.0015477080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"419'2","reported_seq":548,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214727+0000","last_change":"2024-09-11T09:02:11.792219+0000","last_active":"2024-09-11T09:02:47.214727+0000","last_peered":"2024-09-11T09:02:47.214727+0000","last_clean":"2024-09-11T09:02:47.214727+0000","last_became_active":"2024-09-11T08:58:34.135094+0000","last_became_peered":"2024-09-11T08:58:34.135094+0000","last_unstale":"2024-09-11T09:02:47.214727+0000","last_undegraded":"2024-09-11T09:02:47.214727+0000","last_fullsized":"2024-09-11T09:02:47.214727+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"419'2","last_scrub_stamp":"2024-09-11T09:02:11.792137+0000","last_deep_scrub":"419'2","last_deep_scrub_stamp":"2024-09-11T08:59:52.794363+0000","last_clean_scrub_stamp":"2024-09-11T09:02:11.792137+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:27.055519+0000","scrub_duration":0.0032077220000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":487,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212576+0000","last_change":"2024-09-11T09:01:18.943321+0000","last_active":"2024-09-11T09:02:47.212576+0000","last_peered":"2024-09-11T09:02:47.212576+0000","last_clean":"2024-09-11T09:02:47.212576+0000","last_became_active":"2024-09-11T08:38:38.949401+0000","last_became_peered":"2024-09-11T08:38:38.949401+0000","last_unstale":"2024-09-11T09:02:47.212576+0000","last_undegraded":"2024-09-11T09:02:47.212576+0000","last_fullsized":"2024-09-11T09:02:47.212576+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:18.943265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:01:18.943265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001474944,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"402'24","reported_seq":557,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214755+0000","last_change":"2024-09-11T09:02:24.851545+0000","last_active":"2024-09-11T09:02:47.214755+0000","last_peered":"2024-09-11T09:02:47.214755+0000","last_clean":"2024-09-11T09:02:47.214755+0000","last_became_active":"2024-09-11T08:49:16.943886+0000","last_became_peered":"2024-09-11T08:49:16.943886+0000","last_unstale":"2024-09-11T09:02:47.214755+0000","last_undegraded":"2024-09-11T09:02:47.214755+0000","last_fullsized":"2024-09-11T09:02:47.214755+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'24","last_scrub_stamp":"2024-09-11T09:02:24.851487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:24.851487+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:31.266405+0000","scrub_duration":0.002063998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":493,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212478+0000","last_change":"2024-09-11T09:02:27.820405+0000","last_active":"2024-09-11T09:02:47.212478+0000","last_peered":"2024-09-11T09:02:47.212478+0000","last_clean":"2024-09-11T09:02:47.212478+0000","last_became_active":"2024-09-11T08:38:42.984211+0000","last_became_peered":"2024-09-11T08:38:42.984211+0000","last_unstale":"2024-09-11T09:02:47.212478+0000","last_undegraded":"2024-09-11T09:02:47.212478+0000","last_fullsized":"2024-09-11T09:02:47.212478+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:27.820336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:02:27.820336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:56.964184+0000","scrub_duration":0.001624096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"63'1","reported_seq":308,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932155+0000","last_change":"2024-09-11T09:02:34.325436+0000","last_active":"2024-09-11T09:02:47.932155+0000","last_peered":"2024-09-11T09:02:47.932155+0000","last_clean":"2024-09-11T09:02:47.932155+0000","last_became_active":"2024-09-11T09:02:16.803219+0000","last_became_peered":"2024-09-11T09:02:16.803219+0000","last_unstale":"2024-09-11T09:02:47.932155+0000","last_undegraded":"2024-09-11T09:02:47.932155+0000","last_fullsized":"2024-09-11T09:02:47.932155+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-11T09:02:34.325356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:34.325356+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:38.320294+0000","scrub_duration":0.0018443520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":526,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930756+0000","last_change":"2024-09-11T09:01:54.321984+0000","last_active":"2024-09-11T09:02:47.930756+0000","last_peered":"2024-09-11T09:02:47.930756+0000","last_clean":"2024-09-11T09:02:47.930756+0000","last_became_active":"2024-09-11T08:38:38.953819+0000","last_became_peered":"2024-09-11T08:38:38.953819+0000","last_unstale":"2024-09-11T09:02:47.930756+0000","last_undegraded":"2024-09-11T09:02:47.930756+0000","last_fullsized":"2024-09-11T09:02:47.930756+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:54.321929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:33.758018+0000","last_clean_scrub_stamp":"2024-09-11T09:01:54.321929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:19.000474+0000","scrub_duration":0.0014839110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"402'35","reported_seq":223,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932658+0000","last_change":"2024-09-11T09:01:57.879021+0000","last_active":"2024-09-11T09:02:47.932658+0000","last_peered":"2024-09-11T09:02:47.932658+0000","last_clean":"2024-09-11T09:02:47.932658+0000","last_became_active":"2024-09-11T09:01:25.406496+0000","last_became_peered":"2024-09-11T09:01:25.406496+0000","last_unstale":"2024-09-11T09:02:47.932658+0000","last_undegraded":"2024-09-11T09:02:47.932658+0000","last_fullsized":"2024-09-11T09:02:47.932658+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'35","last_scrub_stamp":"2024-09-11T09:01:57.878967+0000","last_deep_scrub":"280'25","last_deep_scrub_stamp":"2024-09-11T08:52:24.584533+0000","last_clean_scrub_stamp":"2024-09-11T09:01:57.878967+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:13.854861+0000","scrub_duration":0.001700167,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"457'12","reported_seq":176,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:55.294511+0000","last_change":"2024-09-11T09:02:18.745062+0000","last_active":"2024-09-11T09:02:55.294511+0000","last_peered":"2024-09-11T09:02:55.294511+0000","last_clean":"2024-09-11T09:02:55.294511+0000","last_became_active":"2024-09-11T09:02:09.769212+0000","last_became_peered":"2024-09-11T09:02:09.769212+0000","last_unstale":"2024-09-11T09:02:55.294511+0000","last_undegraded":"2024-09-11T09:02:55.294511+0000","last_fullsized":"2024-09-11T09:02:55.294511+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'12","last_scrub_stamp":"2024-09-11T09:02:18.744984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:02:18.744984+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:22.092997+0000","scrub_duration":0.001728668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"199'81","reported_seq":283,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210110+0000","last_change":"2024-09-11T09:01:59.346242+0000","last_active":"2024-09-11T09:02:47.210110+0000","last_peered":"2024-09-11T09:02:47.210110+0000","last_clean":"2024-09-11T09:02:47.210110+0000","last_became_active":"2024-09-11T08:58:34.313408+0000","last_became_peered":"2024-09-11T08:58:34.313408+0000","last_unstale":"2024-09-11T09:02:47.210110+0000","last_undegraded":"2024-09-11T09:02:47.210110+0000","last_fullsized":"2024-09-11T09:02:47.210110+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:01:59.346179+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:01:59.346179+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:24.232572+0000","scrub_duration":0.00144864,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"402'46","reported_seq":595,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932229+0000","last_change":"2024-09-11T09:02:25.476477+0000","last_active":"2024-09-11T09:02:47.932229+0000","last_peered":"2024-09-11T09:02:47.932229+0000","last_clean":"2024-09-11T09:02:47.932229+0000","last_became_active":"2024-09-11T08:38:40.978434+0000","last_became_peered":"2024-09-11T08:38:40.978434+0000","last_unstale":"2024-09-11T09:02:47.932229+0000","last_undegraded":"2024-09-11T09:02:47.932229+0000","last_fullsized":"2024-09-11T09:02:47.932229+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'46","last_scrub_stamp":"2024-09-11T09:02:25.476423+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:33.630901+0000","last_clean_scrub_stamp":"2024-09-11T09:02:25.476423+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:47.010191+0000","scrub_duration":0.0019200389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":496,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214276+0000","last_change":"2024-09-11T09:02:19.879820+0000","last_active":"2024-09-11T09:02:47.214276+0000","last_peered":"2024-09-11T09:02:47.214276+0000","last_clean":"2024-09-11T09:02:47.214276+0000","last_became_active":"2024-09-11T08:38:38.950608+0000","last_became_peered":"2024-09-11T08:38:38.950608+0000","last_unstale":"2024-09-11T09:02:47.214276+0000","last_undegraded":"2024-09-11T09:02:47.214276+0000","last_fullsized":"2024-09-11T09:02:47.214276+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:19.879733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:19.879733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:32.453233+0000","scrub_duration":0.0013971019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"277'11","reported_seq":822,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:55.294225+0000","last_change":"2024-09-11T09:02:21.792473+0000","last_active":"2024-09-11T09:02:55.294225+0000","last_peered":"2024-09-11T09:02:55.294225+0000","last_clean":"2024-09-11T09:02:55.294225+0000","last_became_active":"2024-09-11T08:52:18.578328+0000","last_became_peered":"2024-09-11T08:52:18.578328+0000","last_unstale":"2024-09-11T09:02:55.294225+0000","last_undegraded":"2024-09-11T09:02:55.294225+0000","last_fullsized":"2024-09-11T09:02:55.294225+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'11","last_scrub_stamp":"2024-09-11T09:02:21.792402+0000","last_deep_scrub":"154'9","last_deep_scrub_stamp":"2024-09-11T08:45:26.208818+0000","last_clean_scrub_stamp":"2024-09-11T09:02:21.792402+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:50.492725+0000","scrub_duration":0.0017782640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":539,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210503+0000","last_change":"2024-09-11T09:02:02.279129+0000","last_active":"2024-09-11T09:02:47.210503+0000","last_peered":"2024-09-11T09:02:47.210503+0000","last_clean":"2024-09-11T09:02:47.210503+0000","last_became_active":"2024-09-11T09:01:28.499311+0000","last_became_peered":"2024-09-11T09:01:28.499311+0000","last_unstale":"2024-09-11T09:02:47.210503+0000","last_undegraded":"2024-09-11T09:02:47.210503+0000","last_fullsized":"2024-09-11T09:02:47.210503+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:02.279076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:02.279076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:24.854919+0000","scrub_duration":0.0015422739999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"199'81","reported_seq":293,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:53.847396+0000","last_change":"2024-09-11T09:02:53.847350+0000","last_active":"2024-09-11T09:02:53.847396+0000","last_peered":"2024-09-11T09:02:53.847396+0000","last_clean":"2024-09-11T09:02:53.847396+0000","last_became_active":"2024-09-11T08:58:34.324527+0000","last_became_peered":"2024-09-11T08:58:34.324527+0000","last_unstale":"2024-09-11T09:02:53.847396+0000","last_undegraded":"2024-09-11T09:02:53.847396+0000","last_fullsized":"2024-09-11T09:02:53.847396+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:53.847283+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T09:00:52.859779+0000","last_clean_scrub_stamp":"2024-09-11T09:02:53.847283+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:03.846789+0000","scrub_duration":0.001609826,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"402'50","reported_seq":626,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214121+0000","last_change":"2024-09-11T09:01:55.795454+0000","last_active":"2024-09-11T09:02:47.214121+0000","last_peered":"2024-09-11T09:02:47.214121+0000","last_clean":"2024-09-11T09:02:47.214121+0000","last_became_active":"2024-09-11T08:52:18.595013+0000","last_became_peered":"2024-09-11T08:52:18.595013+0000","last_unstale":"2024-09-11T09:02:47.214121+0000","last_undegraded":"2024-09-11T09:02:47.214121+0000","last_fullsized":"2024-09-11T09:02:47.214121+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'50","last_scrub_stamp":"2024-09-11T09:01:55.795401+0000","last_deep_scrub":"199'26","last_deep_scrub_stamp":"2024-09-11T08:48:44.093994+0000","last_clean_scrub_stamp":"2024-09-11T09:01:55.795401+0000","objects_scrubbed":8,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:14.858041+0000","scrub_duration":0.0024891940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":519,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214970+0000","last_change":"2024-09-11T09:02:02.807138+0000","last_active":"2024-09-11T09:02:47.214970+0000","last_peered":"2024-09-11T09:02:47.214970+0000","last_clean":"2024-09-11T09:02:47.214970+0000","last_became_active":"2024-09-11T08:52:18.586084+0000","last_became_peered":"2024-09-11T08:52:18.586084+0000","last_unstale":"2024-09-11T09:02:47.214970+0000","last_undegraded":"2024-09-11T09:02:47.214970+0000","last_fullsized":"2024-09-11T09:02:47.214970+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:02.807084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:58:52.524310+0000","last_clean_scrub_stamp":"2024-09-11T09:02:02.807084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:13.559910+0000","scrub_duration":0.0014273700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":493,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212095+0000","last_change":"2024-09-11T09:02:30.830289+0000","last_active":"2024-09-11T09:02:47.212095+0000","last_peered":"2024-09-11T09:02:47.212095+0000","last_clean":"2024-09-11T09:02:47.212095+0000","last_became_active":"2024-09-11T08:38:42.983435+0000","last_became_peered":"2024-09-11T08:38:42.983435+0000","last_unstale":"2024-09-11T09:02:47.212095+0000","last_undegraded":"2024-09-11T09:02:47.212095+0000","last_fullsized":"2024-09-11T09:02:47.212095+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:30.830227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:00:49.851783+0000","last_clean_scrub_stamp":"2024-09-11T09:02:30.830227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:41.684266+0000","scrub_duration":0.0014479510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":491,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:03:00.815289+0000","last_change":"2024-09-11T09:03:00.815258+0000","last_active":"2024-09-11T09:03:00.815289+0000","last_peered":"2024-09-11T09:03:00.815289+0000","last_clean":"2024-09-11T09:03:00.815289+0000","last_became_active":"2024-09-11T08:38:44.995371+0000","last_became_peered":"2024-09-11T08:38:44.995371+0000","last_unstale":"2024-09-11T09:03:00.815289+0000","last_undegraded":"2024-09-11T09:03:00.815289+0000","last_fullsized":"2024-09-11T09:03:00.815289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:03:00.815205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:01:37.682866+0000","last_clean_scrub_stamp":"2024-09-11T09:03:00.815205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:02.910968+0000","scrub_duration":0.0014020619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"199'81","reported_seq":278,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214494+0000","last_change":"2024-09-11T09:02:42.011760+0000","last_active":"2024-09-11T09:02:47.214494+0000","last_peered":"2024-09-11T09:02:47.214494+0000","last_clean":"2024-09-11T09:02:47.214494+0000","last_became_active":"2024-09-11T08:58:01.125235+0000","last_became_peered":"2024-09-11T08:58:01.125235+0000","last_unstale":"2024-09-11T09:02:47.214494+0000","last_undegraded":"2024-09-11T09:02:47.214494+0000","last_fullsized":"2024-09-11T09:02:47.214494+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:42.011689+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:42.011689+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:48.723451+0000","scrub_duration":0.0013268659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"402'49","reported_seq":610,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212265+0000","last_change":"2024-09-11T09:02:13.712585+0000","last_active":"2024-09-11T09:02:47.212265+0000","last_peered":"2024-09-11T09:02:47.212265+0000","last_clean":"2024-09-11T09:02:47.212265+0000","last_became_active":"2024-09-11T08:58:34.127547+0000","last_became_peered":"2024-09-11T08:58:34.127547+0000","last_unstale":"2024-09-11T09:02:47.212265+0000","last_undegraded":"2024-09-11T09:02:47.212265+0000","last_fullsized":"2024-09-11T09:02:47.212265+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'49","last_scrub_stamp":"2024-09-11T09:02:13.712514+0000","last_deep_scrub":"86'21","last_deep_scrub_stamp":"2024-09-11T08:43:30.997710+0000","last_clean_scrub_stamp":"2024-09-11T09:02:13.712514+0000","objects_scrubbed":7,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:22.194203+0000","scrub_duration":0.0021497080000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":63,"num_read_kb":56,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":269,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932721+0000","last_change":"2024-09-11T09:02:18.832369+0000","last_active":"2024-09-11T09:02:47.932721+0000","last_peered":"2024-09-11T09:02:47.932721+0000","last_clean":"2024-09-11T09:02:47.932721+0000","last_became_active":"2024-09-11T08:52:18.586549+0000","last_became_peered":"2024-09-11T08:52:18.586549+0000","last_unstale":"2024-09-11T09:02:47.932721+0000","last_undegraded":"2024-09-11T09:02:47.932721+0000","last_fullsized":"2024-09-11T09:02:47.932721+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:18.832318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:52:28.487573+0000","last_clean_scrub_stamp":"2024-09-11T09:02:18.832318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:45.872311+0000","scrub_duration":0.001382568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"440'9","reported_seq":796,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:55.900130+0000","last_change":"2024-09-11T09:02:55.900092+0000","last_active":"2024-09-11T09:02:55.900130+0000","last_peered":"2024-09-11T09:02:55.900130+0000","last_clean":"2024-09-11T09:02:55.900130+0000","last_became_active":"2024-09-11T08:38:42.982540+0000","last_became_peered":"2024-09-11T08:38:42.982540+0000","last_unstale":"2024-09-11T09:02:55.900130+0000","last_undegraded":"2024-09-11T09:02:55.900130+0000","last_fullsized":"2024-09-11T09:02:55.900130+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"440'9","last_scrub_stamp":"2024-09-11T09:02:55.900035+0000","last_deep_scrub":"440'9","last_deep_scrub_stamp":"2024-09-11T09:01:25.808118+0000","last_clean_scrub_stamp":"2024-09-11T09:02:55.900035+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:02.572075+0000","scrub_duration":0.0018779599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":327,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:49.467388+0000","last_change":"2024-09-11T09:02:49.467337+0000","last_active":"2024-09-11T09:02:49.467388+0000","last_peered":"2024-09-11T09:02:49.467388+0000","last_clean":"2024-09-11T09:02:49.467388+0000","last_became_active":"2024-09-11T09:02:16.798218+0000","last_became_peered":"2024-09-11T09:02:16.798218+0000","last_unstale":"2024-09-11T09:02:49.467388+0000","last_undegraded":"2024-09-11T09:02:49.467388+0000","last_fullsized":"2024-09-11T09:02:49.467388+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:49.467264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:49.467264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:06.612572+0000","scrub_duration":0.001586795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"199'81","reported_seq":372,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932066+0000","last_change":"2024-09-11T09:02:30.900172+0000","last_active":"2024-09-11T09:02:47.932066+0000","last_peered":"2024-09-11T09:02:47.932066+0000","last_clean":"2024-09-11T09:02:47.932066+0000","last_became_active":"2024-09-11T09:02:09.772929+0000","last_became_peered":"2024-09-11T09:02:09.772929+0000","last_unstale":"2024-09-11T09:02:47.932066+0000","last_undegraded":"2024-09-11T09:02:47.932066+0000","last_fullsized":"2024-09-11T09:02:47.932066+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:30.900115+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:30.900115+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:00.807339+0000","scrub_duration":0.0012564060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"402'35","reported_seq":64,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:03:00.763995+0000","last_change":"2024-09-11T09:03:00.763967+0000","last_active":"2024-09-11T09:03:00.763995+0000","last_peered":"2024-09-11T09:03:00.763995+0000","last_clean":"2024-09-11T09:03:00.763995+0000","last_became_active":"2024-09-11T09:01:28.502156+0000","last_became_peered":"2024-09-11T09:01:28.502156+0000","last_unstale":"2024-09-11T09:03:00.763995+0000","last_undegraded":"2024-09-11T09:03:00.763995+0000","last_fullsized":"2024-09-11T09:03:00.763995+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'35","last_scrub_stamp":"2024-09-11T09:03:00.763914+0000","last_deep_scrub":"100'15","last_deep_scrub_stamp":"2024-09-11T08:48:38.032039+0000","last_clean_scrub_stamp":"2024-09-11T09:03:00.763914+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:21.925759+0000","scrub_duration":0.001924378,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":512,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:57.080603+0000","last_change":"2024-09-11T09:02:57.080553+0000","last_active":"2024-09-11T09:02:57.080603+0000","last_peered":"2024-09-11T09:02:57.080603+0000","last_clean":"2024-09-11T09:02:57.080603+0000","last_became_active":"2024-09-11T08:38:38.950904+0000","last_became_peered":"2024-09-11T08:38:38.950904+0000","last_unstale":"2024-09-11T09:02:57.080603+0000","last_undegraded":"2024-09-11T09:02:57.080603+0000","last_fullsized":"2024-09-11T09:02:57.080603+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:57.080485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:57.080485+0000","last_clean_scrub_stamp":"2024-09-11T09:02:57.080485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:10.990065+0000","scrub_duration":0.0019162490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"448'16","reported_seq":99,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:55.293413+0000","last_change":"2024-09-11T09:01:45.930676+0000","last_active":"2024-09-11T09:02:55.293413+0000","last_peered":"2024-09-11T09:02:55.293413+0000","last_clean":"2024-09-11T09:02:55.293413+0000","last_became_active":"2024-09-11T09:01:25.399797+0000","last_became_peered":"2024-09-11T09:01:25.399797+0000","last_unstale":"2024-09-11T09:02:55.293413+0000","last_undegraded":"2024-09-11T09:02:55.293413+0000","last_fullsized":"2024-09-11T09:02:55.293413+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"448'16","last_scrub_stamp":"2024-09-11T09:01:45.930603+0000","last_deep_scrub":"448'16","last_deep_scrub_stamp":"2024-09-11T09:01:45.930603+0000","last_clean_scrub_stamp":"2024-09-11T09:01:45.930603+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:02.286957+0000","scrub_duration":0.002995836,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":137,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932351+0000","last_change":"2024-09-11T09:02:38.967684+0000","last_active":"2024-09-11T09:02:47.932351+0000","last_peered":"2024-09-11T09:02:47.932351+0000","last_clean":"2024-09-11T09:02:47.932351+0000","last_became_active":"2024-09-11T09:01:25.400182+0000","last_became_peered":"2024-09-11T09:01:25.400182+0000","last_unstale":"2024-09-11T09:02:47.932351+0000","last_undegraded":"2024-09-11T09:02:47.932351+0000","last_fullsized":"2024-09-11T09:02:47.932351+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:38.967626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:38.967626+0000","last_clean_scrub_stamp":"2024-09-11T09:02:38.967626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:54.059817+0000","scrub_duration":0.0010909590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"199'81","reported_seq":372,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932125+0000","last_change":"2024-09-11T09:02:27.819558+0000","last_active":"2024-09-11T09:02:47.932125+0000","last_peered":"2024-09-11T09:02:47.932125+0000","last_clean":"2024-09-11T09:02:47.932125+0000","last_became_active":"2024-09-11T09:02:09.767659+0000","last_became_peered":"2024-09-11T09:02:09.767659+0000","last_unstale":"2024-09-11T09:02:47.932125+0000","last_undegraded":"2024-09-11T09:02:47.932125+0000","last_fullsized":"2024-09-11T09:02:47.932125+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:27.819487+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T09:00:57.089686+0000","last_clean_scrub_stamp":"2024-09-11T09:02:27.819487+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:34.788158+0000","scrub_duration":0.0019620150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"402'39","reported_seq":538,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931766+0000","last_change":"2024-09-11T09:02:22.824681+0000","last_active":"2024-09-11T09:02:47.931766+0000","last_peered":"2024-09-11T09:02:47.931766+0000","last_clean":"2024-09-11T09:02:47.931766+0000","last_became_active":"2024-09-11T09:01:25.404892+0000","last_became_peered":"2024-09-11T09:01:25.404892+0000","last_unstale":"2024-09-11T09:02:47.931766+0000","last_undegraded":"2024-09-11T09:02:47.931766+0000","last_fullsized":"2024-09-11T09:02:47.931766+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'39","last_scrub_stamp":"2024-09-11T09:02:22.824392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:22.824392+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:40.236394+0000","scrub_duration":0.0081303699999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":67,"num_read_kb":53,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":536,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931478+0000","last_change":"2024-09-11T09:02:04.348546+0000","last_active":"2024-09-11T09:02:47.931478+0000","last_peered":"2024-09-11T09:02:47.931478+0000","last_clean":"2024-09-11T09:02:47.931478+0000","last_became_active":"2024-09-11T08:38:38.952755+0000","last_became_peered":"2024-09-11T08:38:38.952755+0000","last_unstale":"2024-09-11T09:02:47.931478+0000","last_undegraded":"2024-09-11T09:02:47.931478+0000","last_fullsized":"2024-09-11T09:02:47.931478+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:04.348493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:04.348493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:10.942122+0000","scrub_duration":0.001483713,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":409,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930999+0000","last_change":"2024-09-11T09:02:09.768372+0000","last_active":"2024-09-11T09:02:47.930999+0000","last_peered":"2024-09-11T09:02:47.930999+0000","last_clean":"2024-09-11T09:02:47.930999+0000","last_became_active":"2024-09-11T09:02:09.768092+0000","last_became_peered":"2024-09-11T09:02:09.768092+0000","last_unstale":"2024-09-11T09:02:47.930999+0000","last_undegraded":"2024-09-11T09:02:47.930999+0000","last_fullsized":"2024-09-11T09:02:47.930999+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:52.871083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:28.069461+0000","last_clean_scrub_stamp":"2024-09-11T09:01:52.871083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:00.827308+0000","scrub_duration":0.011823264,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":573,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930344+0000","last_change":"2024-09-11T09:02:42.691486+0000","last_active":"2024-09-11T09:02:47.930344+0000","last_peered":"2024-09-11T09:02:47.930344+0000","last_clean":"2024-09-11T09:02:47.930344+0000","last_became_active":"2024-09-11T08:58:34.315290+0000","last_became_peered":"2024-09-11T08:58:34.315290+0000","last_unstale":"2024-09-11T09:02:47.930344+0000","last_undegraded":"2024-09-11T09:02:47.930344+0000","last_fullsized":"2024-09-11T09:02:47.930344+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:42.691434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:01:16.084845+0000","last_clean_scrub_stamp":"2024-09-11T09:02:42.691434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:11.902949+0000","scrub_duration":0.0014238320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"199'81","reported_seq":283,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210047+0000","last_change":"2024-09-11T09:02:11.272886+0000","last_active":"2024-09-11T09:02:47.210047+0000","last_peered":"2024-09-11T09:02:47.210047+0000","last_clean":"2024-09-11T09:02:47.210047+0000","last_became_active":"2024-09-11T08:58:01.120225+0000","last_became_peered":"2024-09-11T08:58:01.120225+0000","last_unstale":"2024-09-11T09:02:47.210047+0000","last_undegraded":"2024-09-11T09:02:47.210047+0000","last_fullsized":"2024-09-11T09:02:47.210047+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:11.272812+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:11.272812+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:31.608940+0000","scrub_duration":0.0016332790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"442'66","reported_seq":83,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:55.292964+0000","last_change":"2024-09-11T09:02:09.314628+0000","last_active":"2024-09-11T09:02:55.292964+0000","last_peered":"2024-09-11T09:02:55.292964+0000","last_clean":"2024-09-11T09:02:55.292964+0000","last_became_active":"2024-09-11T09:01:28.505013+0000","last_became_peered":"2024-09-11T09:01:28.505013+0000","last_unstale":"2024-09-11T09:02:55.292964+0000","last_undegraded":"2024-09-11T09:02:55.292964+0000","last_fullsized":"2024-09-11T09:02:55.292964+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"442'66","last_scrub_stamp":"2024-09-11T09:02:09.314564+0000","last_deep_scrub":"200'50","last_deep_scrub_stamp":"2024-09-11T08:53:09.745322+0000","last_clean_scrub_stamp":"2024-09-11T09:02:09.314564+0000","objects_scrubbed":10,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:32.955018+0000","scrub_duration":0.0022008150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":537,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212983+0000","last_change":"2024-09-11T09:02:06.785768+0000","last_active":"2024-09-11T09:02:47.212983+0000","last_peered":"2024-09-11T09:02:47.212983+0000","last_clean":"2024-09-11T09:02:47.212983+0000","last_became_active":"2024-09-11T08:58:34.136762+0000","last_became_peered":"2024-09-11T08:58:34.136762+0000","last_unstale":"2024-09-11T09:02:47.212983+0000","last_undegraded":"2024-09-11T09:02:47.212983+0000","last_fullsized":"2024-09-11T09:02:47.212983+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:06.785697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:49:25.626031+0000","last_clean_scrub_stamp":"2024-09-11T09:02:06.785697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:20.724085+0000","scrub_duration":0.0017665090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'2","reported_seq":575,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210509+0000","last_change":"2024-09-11T09:02:28.296073+0000","last_active":"2024-09-11T09:02:47.210509+0000","last_peered":"2024-09-11T09:02:47.210509+0000","last_clean":"2024-09-11T09:02:47.210509+0000","last_became_active":"2024-09-11T08:58:34.138191+0000","last_became_peered":"2024-09-11T08:58:34.138191+0000","last_unstale":"2024-09-11T09:02:47.210509+0000","last_undegraded":"2024-09-11T09:02:47.210509+0000","last_fullsized":"2024-09-11T09:02:47.210509+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-11T09:02:28.295996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:28.295996+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:53.153378+0000","scrub_duration":0.0021093420000000002,"objects_trimmed":0,"snaptrim_duration":0.000352214,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":137,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932439+0000","last_change":"2024-09-11T09:02:40.942123+0000","last_active":"2024-09-11T09:02:47.932439+0000","last_peered":"2024-09-11T09:02:47.932439+0000","last_clean":"2024-09-11T09:02:47.932439+0000","last_became_active":"2024-09-11T09:01:25.420792+0000","last_became_peered":"2024-09-11T09:01:25.420792+0000","last_unstale":"2024-09-11T09:02:47.932439+0000","last_undegraded":"2024-09-11T09:02:47.932439+0000","last_fullsized":"2024-09-11T09:02:47.932439+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:40.942069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:02:40.942069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:05.731958+0000","scrub_duration":0.001662312,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":490,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214615+0000","last_change":"2024-09-11T09:02:26.773611+0000","last_active":"2024-09-11T09:02:47.214615+0000","last_peered":"2024-09-11T09:02:47.214615+0000","last_clean":"2024-09-11T09:02:47.214615+0000","last_became_active":"2024-09-11T08:38:44.996842+0000","last_became_peered":"2024-09-11T08:38:44.996842+0000","last_unstale":"2024-09-11T09:02:47.214615+0000","last_undegraded":"2024-09-11T09:02:47.214615+0000","last_fullsized":"2024-09-11T09:02:47.214615+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:26.773556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:26.773556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:49.354353+0000","scrub_duration":0.0016365150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"199'81","reported_seq":405,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931495+0000","last_change":"2024-09-11T09:02:28.310057+0000","last_active":"2024-09-11T09:02:47.931495+0000","last_peered":"2024-09-11T09:02:47.931495+0000","last_clean":"2024-09-11T09:02:47.931495+0000","last_became_active":"2024-09-11T09:02:16.797286+0000","last_became_peered":"2024-09-11T09:02:16.797286+0000","last_unstale":"2024-09-11T09:02:47.931495+0000","last_undegraded":"2024-09-11T09:02:47.931495+0000","last_fullsized":"2024-09-11T09:02:47.931495+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:28.309987+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T09:02:28.309987+0000","last_clean_scrub_stamp":"2024-09-11T09:02:28.309987+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:52.189981+0000","scrub_duration":0.001601211,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"402'21","reported_seq":64,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:50.753409+0000","last_change":"2024-09-11T09:02:50.753355+0000","last_active":"2024-09-11T09:02:50.753409+0000","last_peered":"2024-09-11T09:02:50.753409+0000","last_clean":"2024-09-11T09:02:50.753409+0000","last_became_active":"2024-09-11T09:01:28.499434+0000","last_became_peered":"2024-09-11T09:01:28.499434+0000","last_unstale":"2024-09-11T09:02:50.753409+0000","last_undegraded":"2024-09-11T09:02:50.753409+0000","last_fullsized":"2024-09-11T09:02:50.753409+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'21","last_scrub_stamp":"2024-09-11T09:02:50.753291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:50.753291+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:12.239614+0000","scrub_duration":0.0022493639999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":29,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":572,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930342+0000","last_change":"2024-09-11T09:01:55.352768+0000","last_active":"2024-09-11T09:02:47.930342+0000","last_peered":"2024-09-11T09:02:47.930342+0000","last_clean":"2024-09-11T09:02:47.930342+0000","last_became_active":"2024-09-11T08:58:34.311119+0000","last_became_peered":"2024-09-11T08:58:34.311119+0000","last_unstale":"2024-09-11T09:02:47.930342+0000","last_undegraded":"2024-09-11T09:02:47.930342+0000","last_fullsized":"2024-09-11T09:02:47.930342+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:55.352709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:01:55.352709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:04.203148+0000","scrub_duration":0.001469171,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"263'2","reported_seq":607,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931105+0000","last_change":"2024-09-11T09:02:23.661666+0000","last_active":"2024-09-11T09:02:47.931105+0000","last_peered":"2024-09-11T09:02:47.931105+0000","last_clean":"2024-09-11T09:02:47.931105+0000","last_became_active":"2024-09-11T08:52:18.586545+0000","last_became_peered":"2024-09-11T08:52:18.586545+0000","last_unstale":"2024-09-11T09:02:47.931105+0000","last_undegraded":"2024-09-11T09:02:47.931105+0000","last_fullsized":"2024-09-11T09:02:47.931105+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"263'2","last_scrub_stamp":"2024-09-11T09:02:23.661568+0000","last_deep_scrub":"263'2","last_deep_scrub_stamp":"2024-09-11T08:53:08.772979+0000","last_clean_scrub_stamp":"2024-09-11T09:02:23.661568+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:53.535710+0000","scrub_duration":0.0016331290000000001,"objects_trimmed":0,"snaptrim_duration":0.001158826,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":557,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931470+0000","last_change":"2024-09-11T09:02:02.558793+0000","last_active":"2024-09-11T09:02:47.931470+0000","last_peered":"2024-09-11T09:02:47.931470+0000","last_clean":"2024-09-11T09:02:47.931470+0000","last_became_active":"2024-09-11T08:38:42.980537+0000","last_became_peered":"2024-09-11T08:38:42.980537+0000","last_unstale":"2024-09-11T09:02:47.931470+0000","last_undegraded":"2024-09-11T09:02:47.931470+0000","last_fullsized":"2024-09-11T09:02:47.931470+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:02.558740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:58:50.901121+0000","last_clean_scrub_stamp":"2024-09-11T09:02:02.558740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:05.166107+0000","scrub_duration":0.0014589900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":337,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932111+0000","last_change":"2024-09-11T09:02:20.438199+0000","last_active":"2024-09-11T09:02:47.932111+0000","last_peered":"2024-09-11T09:02:47.932111+0000","last_clean":"2024-09-11T09:02:47.932111+0000","last_became_active":"2024-09-11T09:02:16.796628+0000","last_became_peered":"2024-09-11T09:02:16.796628+0000","last_unstale":"2024-09-11T09:02:47.932111+0000","last_undegraded":"2024-09-11T09:02:47.932111+0000","last_fullsized":"2024-09-11T09:02:47.932111+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:20.438132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:20.438132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:45.057300+0000","scrub_duration":0.001693908,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"199'81","reported_seq":576,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930990+0000","last_change":"2024-09-11T09:02:32.595769+0000","last_active":"2024-09-11T09:02:47.930990+0000","last_peered":"2024-09-11T09:02:47.930990+0000","last_clean":"2024-09-11T09:02:47.930990+0000","last_became_active":"2024-09-11T08:58:34.130607+0000","last_became_peered":"2024-09-11T08:58:34.130607+0000","last_unstale":"2024-09-11T09:02:47.930990+0000","last_undegraded":"2024-09-11T09:02:47.930990+0000","last_fullsized":"2024-09-11T09:02:47.930990+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":3,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:32.595714+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:32.595714+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:36.254682+0000","scrub_duration":0.001862371,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"402'36","reported_seq":298,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932597+0000","last_change":"2024-09-11T09:02:25.858393+0000","last_active":"2024-09-11T09:02:47.932597+0000","last_peered":"2024-09-11T09:02:47.932597+0000","last_clean":"2024-09-11T09:02:47.932597+0000","last_became_active":"2024-09-11T08:52:18.580950+0000","last_became_peered":"2024-09-11T08:52:18.580950+0000","last_unstale":"2024-09-11T09:02:47.932597+0000","last_undegraded":"2024-09-11T09:02:47.932597+0000","last_fullsized":"2024-09-11T09:02:47.932597+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'36","last_scrub_stamp":"2024-09-11T09:02:25.858325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:25.858325+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:29.622507+0000","scrub_duration":0.0020603610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":47,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":186,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930758+0000","last_change":"2024-09-11T09:02:22.827670+0000","last_active":"2024-09-11T09:02:47.930758+0000","last_peered":"2024-09-11T09:02:47.930758+0000","last_clean":"2024-09-11T09:02:47.930758+0000","last_became_active":"2024-09-11T09:02:09.765000+0000","last_became_peered":"2024-09-11T09:02:09.765000+0000","last_unstale":"2024-09-11T09:02:47.930758+0000","last_undegraded":"2024-09-11T09:02:47.930758+0000","last_fullsized":"2024-09-11T09:02:47.930758+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:22.827603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:22.827603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:40.390028+0000","scrub_duration":0.002742179,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"318'6","reported_seq":242,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:59.870376+0000","last_change":"2024-09-11T09:02:16.801653+0000","last_active":"2024-09-11T09:02:59.870376+0000","last_peered":"2024-09-11T09:02:59.870376+0000","last_clean":"2024-09-11T09:02:59.870376+0000","last_became_active":"2024-09-11T09:02:16.801234+0000","last_became_peered":"2024-09-11T09:02:16.801234+0000","last_unstale":"2024-09-11T09:02:59.870376+0000","last_undegraded":"2024-09-11T09:02:59.870376+0000","last_fullsized":"2024-09-11T09:02:59.870376+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"318'6","last_scrub_stamp":"2024-09-11T09:01:34.145178+0000","last_deep_scrub":"318'6","last_deep_scrub_stamp":"2024-09-11T08:54:12.873653+0000","last_clean_scrub_stamp":"2024-09-11T09:01:34.145178+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017088629999999999,"objects_trimmed":0,"snaptrim_duration":0.00034408300000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":140,"num_read_kb":135,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":528,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.211287+0000","last_change":"2024-09-11T09:02:37.230413+0000","last_active":"2024-09-11T09:02:47.211287+0000","last_peered":"2024-09-11T09:02:47.211287+0000","last_clean":"2024-09-11T09:02:47.211287+0000","last_became_active":"2024-09-11T08:52:18.578530+0000","last_became_peered":"2024-09-11T08:52:18.578530+0000","last_unstale":"2024-09-11T09:02:47.211287+0000","last_undegraded":"2024-09-11T09:02:47.211287+0000","last_fullsized":"2024-09-11T09:02:47.211287+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:37.230344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:01:29.426515+0000","last_clean_scrub_stamp":"2024-09-11T09:02:37.230344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:57.375943+0000","scrub_duration":0.001363782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":52,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:58.804442+0000","last_change":"2024-09-11T09:02:58.804417+0000","last_active":"2024-09-11T09:02:58.804442+0000","last_peered":"2024-09-11T09:02:58.804442+0000","last_clean":"2024-09-11T09:02:58.804442+0000","last_became_active":"2024-09-11T09:01:28.502749+0000","last_became_peered":"2024-09-11T09:01:28.502749+0000","last_unstale":"2024-09-11T09:02:58.804442+0000","last_undegraded":"2024-09-11T09:02:58.804442+0000","last_fullsized":"2024-09-11T09:02:58.804442+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:58.804364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:41:53.259438+0000","last_clean_scrub_stamp":"2024-09-11T09:02:58.804364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:10.865232+0000","scrub_duration":0.001804344,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"402'39","reported_seq":259,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932049+0000","last_change":"2024-09-11T09:02:13.800886+0000","last_active":"2024-09-11T09:02:47.932049+0000","last_peered":"2024-09-11T09:02:47.932049+0000","last_clean":"2024-09-11T09:02:47.932049+0000","last_became_active":"2024-09-11T09:02:09.772033+0000","last_became_peered":"2024-09-11T09:02:09.772033+0000","last_unstale":"2024-09-11T09:02:47.932049+0000","last_undegraded":"2024-09-11T09:02:47.932049+0000","last_fullsized":"2024-09-11T09:02:47.932049+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'39","last_scrub_stamp":"2024-09-11T09:02:13.800816+0000","last_deep_scrub":"201'27","last_deep_scrub_stamp":"2024-09-11T08:48:47.827134+0000","last_clean_scrub_stamp":"2024-09-11T09:02:13.800816+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:23.711614+0000","scrub_duration":0.0021351090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":59,"num_read_kb":49,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":533,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214370+0000","last_change":"2024-09-11T09:02:37.948332+0000","last_active":"2024-09-11T09:02:47.214370+0000","last_peered":"2024-09-11T09:02:47.214370+0000","last_clean":"2024-09-11T09:02:47.214370+0000","last_became_active":"2024-09-11T08:58:34.135492+0000","last_became_peered":"2024-09-11T08:58:34.135492+0000","last_unstale":"2024-09-11T09:02:47.214370+0000","last_undegraded":"2024-09-11T09:02:47.214370+0000","last_fullsized":"2024-09-11T09:02:47.214370+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T09:02:37.948275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:37.948275+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:43.681244+0000","scrub_duration":0.0018152579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1748,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"199'81","reported_seq":560,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931375+0000","last_change":"2024-09-11T09:02:33.363939+0000","last_active":"2024-09-11T09:02:47.931375+0000","last_peered":"2024-09-11T09:02:47.931375+0000","last_clean":"2024-09-11T09:02:47.931375+0000","last_became_active":"2024-09-11T09:02:16.797435+0000","last_became_peered":"2024-09-11T09:02:16.797435+0000","last_unstale":"2024-09-11T09:02:47.931375+0000","last_undegraded":"2024-09-11T09:02:47.931375+0000","last_fullsized":"2024-09-11T09:02:47.931375+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:33.363867+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:55:08.518651+0000","last_clean_scrub_stamp":"2024-09-11T09:02:33.363867+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:33.671552+0000","scrub_duration":0.0015999829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":319,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932205+0000","last_change":"2024-09-11T09:02:41.300284+0000","last_active":"2024-09-11T09:02:47.932205+0000","last_peered":"2024-09-11T09:02:47.932205+0000","last_clean":"2024-09-11T09:02:47.932205+0000","last_became_active":"2024-09-11T09:02:16.802882+0000","last_became_peered":"2024-09-11T09:02:16.802882+0000","last_unstale":"2024-09-11T09:02:47.932205+0000","last_undegraded":"2024-09-11T09:02:47.932205+0000","last_fullsized":"2024-09-11T09:02:47.932205+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:41.300221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:41.300221+0000","last_clean_scrub_stamp":"2024-09-11T09:02:41.300221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:06.538379+0000","scrub_duration":0.0012995229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":506,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.211217+0000","last_change":"2024-09-11T09:02:07.276096+0000","last_active":"2024-09-11T09:02:47.211217+0000","last_peered":"2024-09-11T09:02:47.211217+0000","last_clean":"2024-09-11T09:02:47.211217+0000","last_became_active":"2024-09-11T08:38:44.994003+0000","last_became_peered":"2024-09-11T08:38:44.994003+0000","last_unstale":"2024-09-11T09:02:47.211217+0000","last_undegraded":"2024-09-11T09:02:47.211217+0000","last_fullsized":"2024-09-11T09:02:47.211217+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:07.276030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:07.276030+0000","last_clean_scrub_stamp":"2024-09-11T09:02:07.276030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:32.126766+0000","scrub_duration":0.001431938,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"199'81","reported_seq":565,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930784+0000","last_change":"2024-09-11T09:02:02.394136+0000","last_active":"2024-09-11T09:02:47.930784+0000","last_peered":"2024-09-11T09:02:47.930784+0000","last_clean":"2024-09-11T09:02:47.930784+0000","last_became_active":"2024-09-11T08:58:03.122171+0000","last_became_peered":"2024-09-11T08:58:03.122171+0000","last_unstale":"2024-09-11T09:02:47.930784+0000","last_undegraded":"2024-09-11T09:02:47.930784+0000","last_fullsized":"2024-09-11T09:02:47.930784+0000","mapping_epoch":385,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":386,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:02.394080+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:02.394080+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:28.463436+0000","scrub_duration":0.0012231379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"402'27","reported_seq":685,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.213051+0000","last_change":"2024-09-11T09:02:00.763868+0000","last_active":"2024-09-11T09:02:47.213051+0000","last_peered":"2024-09-11T09:02:47.213051+0000","last_clean":"2024-09-11T09:02:47.213051+0000","last_became_active":"2024-09-11T09:01:28.505591+0000","last_became_peered":"2024-09-11T09:01:28.505591+0000","last_unstale":"2024-09-11T09:02:47.213051+0000","last_undegraded":"2024-09-11T09:02:47.213051+0000","last_fullsized":"2024-09-11T09:02:47.213051+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'27","last_scrub_stamp":"2024-09-11T09:02:00.763791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:00.763791+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:27.138185+0000","scrub_duration":0.0026385029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":770,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":487,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212348+0000","last_change":"2024-09-11T09:02:16.741968+0000","last_active":"2024-09-11T09:02:47.212348+0000","last_peered":"2024-09-11T09:02:47.212348+0000","last_clean":"2024-09-11T09:02:47.212348+0000","last_became_active":"2024-09-11T08:38:38.947071+0000","last_became_peered":"2024-09-11T08:38:38.947071+0000","last_unstale":"2024-09-11T09:02:47.212348+0000","last_undegraded":"2024-09-11T09:02:47.212348+0000","last_fullsized":"2024-09-11T09:02:47.212348+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:16.741894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:16.741894+0000","last_clean_scrub_stamp":"2024-09-11T09:02:16.741894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:17.247396+0000","scrub_duration":0.001389007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":528,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.211141+0000","last_change":"2024-09-11T09:01:43.311203+0000","last_active":"2024-09-11T09:02:47.211141+0000","last_peered":"2024-09-11T09:02:47.211141+0000","last_clean":"2024-09-11T09:02:47.211141+0000","last_became_active":"2024-09-11T08:58:34.135221+0000","last_became_peered":"2024-09-11T08:58:34.135221+0000","last_unstale":"2024-09-11T09:02:47.211141+0000","last_undegraded":"2024-09-11T09:02:47.211141+0000","last_fullsized":"2024-09-11T09:02:47.211141+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:43.311148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:26.726573+0000","last_clean_scrub_stamp":"2024-09-11T09:01:43.311148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:12.616485+0000","scrub_duration":0.0012877260000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":530,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214555+0000","last_change":"2024-09-11T09:02:35.905921+0000","last_active":"2024-09-11T09:02:47.214555+0000","last_peered":"2024-09-11T09:02:47.214555+0000","last_clean":"2024-09-11T09:02:47.214555+0000","last_became_active":"2024-09-11T08:58:34.135447+0000","last_became_peered":"2024-09-11T08:58:34.135447+0000","last_unstale":"2024-09-11T09:02:47.214555+0000","last_undegraded":"2024-09-11T09:02:47.214555+0000","last_fullsized":"2024-09-11T09:02:47.214555+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:35.905869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:35.905869+0000","last_clean_scrub_stamp":"2024-09-11T09:02:35.905869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:01.580035+0000","scrub_duration":0.0032310490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"199'81","reported_seq":280,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:03:00.059545+0000","last_change":"2024-09-11T09:03:00.059495+0000","last_active":"2024-09-11T09:03:00.059545+0000","last_peered":"2024-09-11T09:03:00.059545+0000","last_clean":"2024-09-11T09:03:00.059545+0000","last_became_active":"2024-09-11T08:51:03.020941+0000","last_became_peered":"2024-09-11T08:51:03.020941+0000","last_unstale":"2024-09-11T09:03:00.059545+0000","last_undegraded":"2024-09-11T09:03:00.059545+0000","last_fullsized":"2024-09-11T09:03:00.059545+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:03:00.059397+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T09:01:34.302799+0000","last_clean_scrub_stamp":"2024-09-11T09:03:00.059397+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:21.230538+0000","scrub_duration":0.0015514789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"402'53","reported_seq":573,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212880+0000","last_change":"2024-09-11T09:01:38.676516+0000","last_active":"2024-09-11T09:02:47.212880+0000","last_peered":"2024-09-11T09:02:47.212880+0000","last_clean":"2024-09-11T09:02:47.212880+0000","last_became_active":"2024-09-11T08:38:40.973909+0000","last_became_peered":"2024-09-11T08:38:40.973909+0000","last_unstale":"2024-09-11T09:02:47.212880+0000","last_undegraded":"2024-09-11T09:02:47.212880+0000","last_fullsized":"2024-09-11T09:02:47.212880+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'53","last_scrub_stamp":"2024-09-11T09:01:38.676458+0000","last_deep_scrub":"317'39","last_deep_scrub_stamp":"2024-09-11T08:54:06.928266+0000","last_clean_scrub_stamp":"2024-09-11T09:01:38.676458+0000","objects_scrubbed":8,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:00.311673+0000","scrub_duration":0.0021813280000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":71,"num_read_kb":61,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":181,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931184+0000","last_change":"2024-09-11T09:02:10.871339+0000","last_active":"2024-09-11T09:02:47.931184+0000","last_peered":"2024-09-11T09:02:47.931184+0000","last_clean":"2024-09-11T09:02:47.931184+0000","last_became_active":"2024-09-11T09:02:09.765158+0000","last_became_peered":"2024-09-11T09:02:09.765158+0000","last_unstale":"2024-09-11T09:02:47.931184+0000","last_undegraded":"2024-09-11T09:02:47.931184+0000","last_fullsized":"2024-09-11T09:02:47.931184+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:10.871277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:07.504031+0000","last_clean_scrub_stamp":"2024-09-11T09:02:10.871277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:16.543811+0000","scrub_duration":0.001535876,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":545,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931355+0000","last_change":"2024-09-11T09:02:28.577701+0000","last_active":"2024-09-11T09:02:47.931355+0000","last_peered":"2024-09-11T09:02:47.931355+0000","last_clean":"2024-09-11T09:02:47.931355+0000","last_became_active":"2024-09-11T08:52:18.577201+0000","last_became_peered":"2024-09-11T08:52:18.577201+0000","last_unstale":"2024-09-11T09:02:47.931355+0000","last_undegraded":"2024-09-11T09:02:47.931355+0000","last_fullsized":"2024-09-11T09:02:47.931355+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:28.577617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:28.577617+0000","last_clean_scrub_stamp":"2024-09-11T09:02:28.577617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:36.603773+0000","scrub_duration":0.0017497859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":412,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:58.491117+0000","last_change":"2024-09-11T09:02:58.491092+0000","last_active":"2024-09-11T09:02:58.491117+0000","last_peered":"2024-09-11T09:02:58.491117+0000","last_clean":"2024-09-11T09:02:58.491117+0000","last_became_active":"2024-09-11T09:01:39.644254+0000","last_became_peered":"2024-09-11T09:01:39.644254+0000","last_unstale":"2024-09-11T09:02:58.491117+0000","last_undegraded":"2024-09-11T09:02:58.491117+0000","last_fullsized":"2024-09-11T09:02:58.491117+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:58.491037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:58.491037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:05.537774+0000","scrub_duration":0.0016928869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"199'81","reported_seq":395,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931537+0000","last_change":"2024-09-11T09:02:24.396150+0000","last_active":"2024-09-11T09:02:47.931537+0000","last_peered":"2024-09-11T09:02:47.931537+0000","last_clean":"2024-09-11T09:02:47.931537+0000","last_became_active":"2024-09-11T09:02:16.798411+0000","last_became_peered":"2024-09-11T09:02:16.798411+0000","last_unstale":"2024-09-11T09:02:47.931537+0000","last_undegraded":"2024-09-11T09:02:47.931537+0000","last_fullsized":"2024-09-11T09:02:47.931537+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:24.396086+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:24.396086+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:26.784550+0000","scrub_duration":0.001620535,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"402'30","reported_seq":624,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212633+0000","last_change":"2024-09-11T09:02:17.789888+0000","last_active":"2024-09-11T09:02:47.212633+0000","last_peered":"2024-09-11T09:02:47.212633+0000","last_clean":"2024-09-11T09:02:47.212633+0000","last_became_active":"2024-09-11T08:58:34.136623+0000","last_became_peered":"2024-09-11T08:58:34.136623+0000","last_unstale":"2024-09-11T09:02:47.212633+0000","last_undegraded":"2024-09-11T09:02:47.212633+0000","last_fullsized":"2024-09-11T09:02:47.212633+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'30","last_scrub_stamp":"2024-09-11T09:02:17.789818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:17.789818+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:34.662655+0000","scrub_duration":0.0020786559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":516,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214640+0000","last_change":"2024-09-11T09:02:21.859173+0000","last_active":"2024-09-11T09:02:47.214640+0000","last_peered":"2024-09-11T09:02:47.214640+0000","last_clean":"2024-09-11T09:02:47.214640+0000","last_became_active":"2024-09-11T08:52:18.586343+0000","last_became_peered":"2024-09-11T08:52:18.586343+0000","last_unstale":"2024-09-11T09:02:47.214640+0000","last_undegraded":"2024-09-11T09:02:47.214640+0000","last_fullsized":"2024-09-11T09:02:47.214640+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:21.859117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:21.859117+0000","last_clean_scrub_stamp":"2024-09-11T09:02:21.859117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:32.572516+0000","scrub_duration":0.001611924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"299'9","reported_seq":855,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:55.242453+0000","last_change":"2024-09-11T09:02:09.612482+0000","last_active":"2024-09-11T09:02:55.242453+0000","last_peered":"2024-09-11T09:02:55.242453+0000","last_clean":"2024-09-11T09:02:55.242453+0000","last_became_active":"2024-09-11T08:38:42.979695+0000","last_became_peered":"2024-09-11T08:38:42.979695+0000","last_unstale":"2024-09-11T09:02:55.242453+0000","last_undegraded":"2024-09-11T09:02:55.242453+0000","last_fullsized":"2024-09-11T09:02:55.242453+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'9","last_scrub_stamp":"2024-09-11T09:02:09.612396+0000","last_deep_scrub":"299'9","last_deep_scrub_stamp":"2024-09-11T08:53:05.835177+0000","last_clean_scrub_stamp":"2024-09-11T09:02:09.612396+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:30.537560+0000","scrub_duration":0.0023718680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":511,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212723+0000","last_change":"2024-09-11T09:02:24.847255+0000","last_active":"2024-09-11T09:02:47.212723+0000","last_peered":"2024-09-11T09:02:47.212723+0000","last_clean":"2024-09-11T09:02:47.212723+0000","last_became_active":"2024-09-11T09:01:28.501529+0000","last_became_peered":"2024-09-11T09:01:28.501529+0000","last_unstale":"2024-09-11T09:02:47.212723+0000","last_undegraded":"2024-09-11T09:02:47.212723+0000","last_fullsized":"2024-09-11T09:02:47.212723+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:24.847190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:40:38.663952+0000","last_clean_scrub_stamp":"2024-09-11T09:02:24.847190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:27.957728+0000","scrub_duration":0.001626099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"199'81","reported_seq":108,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212831+0000","last_change":"2024-09-11T09:02:38.883940+0000","last_active":"2024-09-11T09:02:47.212831+0000","last_peered":"2024-09-11T09:02:47.212831+0000","last_clean":"2024-09-11T09:02:47.212831+0000","last_became_active":"2024-09-11T08:58:03.137484+0000","last_became_peered":"2024-09-11T08:58:03.137484+0000","last_unstale":"2024-09-11T09:02:47.212831+0000","last_undegraded":"2024-09-11T09:02:47.212831+0000","last_fullsized":"2024-09-11T09:02:47.212831+0000","mapping_epoch":385,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":386,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:38.883866+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:38.883866+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:40.765064+0000","scrub_duration":0.0014340259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"402'21","reported_seq":288,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932047+0000","last_change":"2024-09-11T09:02:21.818101+0000","last_active":"2024-09-11T09:02:47.932047+0000","last_peered":"2024-09-11T09:02:47.932047+0000","last_clean":"2024-09-11T09:02:47.932047+0000","last_became_active":"2024-09-11T08:52:18.588329+0000","last_became_peered":"2024-09-11T08:52:18.588329+0000","last_unstale":"2024-09-11T09:02:47.932047+0000","last_undegraded":"2024-09-11T09:02:47.932047+0000","last_fullsized":"2024-09-11T09:02:47.932047+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'21","last_scrub_stamp":"2024-09-11T09:02:21.818018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:21.818018+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:49.056254+0000","scrub_duration":0.0018483709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":529,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210774+0000","last_change":"2024-09-11T09:01:56.336631+0000","last_active":"2024-09-11T09:02:47.210774+0000","last_peered":"2024-09-11T09:02:47.210774+0000","last_clean":"2024-09-11T09:02:47.210774+0000","last_became_active":"2024-09-11T08:58:34.139430+0000","last_became_peered":"2024-09-11T08:58:34.139430+0000","last_unstale":"2024-09-11T09:02:47.210774+0000","last_undegraded":"2024-09-11T09:02:47.210774+0000","last_fullsized":"2024-09-11T09:02:47.210774+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:56.336547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:39:50.452494+0000","last_clean_scrub_stamp":"2024-09-11T09:01:56.336547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:25.568418+0000","scrub_duration":0.0013432450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":409,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931260+0000","last_change":"2024-09-11T09:02:46.500211+0000","last_active":"2024-09-11T09:02:47.931260+0000","last_peered":"2024-09-11T09:02:47.931260+0000","last_clean":"2024-09-11T09:02:47.931260+0000","last_became_active":"2024-09-11T09:01:39.644407+0000","last_became_peered":"2024-09-11T09:01:39.644407+0000","last_unstale":"2024-09-11T09:02:47.931260+0000","last_undegraded":"2024-09-11T09:02:47.931260+0000","last_fullsized":"2024-09-11T09:02:47.931260+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:46.500156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:46.500156+0000","last_clean_scrub_stamp":"2024-09-11T09:02:46.500156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:12.245878+0000","scrub_duration":0.0013442370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":68,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931085+0000","last_change":"2024-09-11T09:02:41.921567+0000","last_active":"2024-09-11T09:02:47.931085+0000","last_peered":"2024-09-11T09:02:47.931085+0000","last_clean":"2024-09-11T09:02:47.931085+0000","last_became_active":"2024-09-11T09:01:25.402638+0000","last_became_peered":"2024-09-11T09:01:25.402638+0000","last_unstale":"2024-09-11T09:02:47.931085+0000","last_undegraded":"2024-09-11T09:02:47.931085+0000","last_fullsized":"2024-09-11T09:02:47.931085+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:41.921508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:13.770061+0000","last_clean_scrub_stamp":"2024-09-11T09:02:41.921508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:55.879252+0000","scrub_duration":0.00148383,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"199'81","reported_seq":66,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931616+0000","last_change":"2024-09-11T09:02:29.865227+0000","last_active":"2024-09-11T09:02:47.931616+0000","last_peered":"2024-09-11T09:02:47.931616+0000","last_clean":"2024-09-11T09:02:47.931616+0000","last_became_active":"2024-09-11T09:02:09.770746+0000","last_became_peered":"2024-09-11T09:02:09.770746+0000","last_unstale":"2024-09-11T09:02:47.931616+0000","last_undegraded":"2024-09-11T09:02:47.931616+0000","last_fullsized":"2024-09-11T09:02:47.931616+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:29.865153+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:29.865153+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:57.496797+0000","scrub_duration":0.00152848,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"402'60","reported_seq":285,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931293+0000","last_change":"2024-09-11T09:02:09.767797+0000","last_active":"2024-09-11T09:02:47.931293+0000","last_peered":"2024-09-11T09:02:47.931293+0000","last_clean":"2024-09-11T09:02:47.931293+0000","last_became_active":"2024-09-11T09:02:09.767382+0000","last_became_peered":"2024-09-11T09:02:09.767382+0000","last_unstale":"2024-09-11T09:02:47.931293+0000","last_undegraded":"2024-09-11T09:02:47.931293+0000","last_fullsized":"2024-09-11T09:02:47.931293+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'60","last_scrub_stamp":"2024-09-11T09:01:53.308889+0000","last_deep_scrub":"374'44","last_deep_scrub_stamp":"2024-09-11T08:57:34.896912+0000","last_clean_scrub_stamp":"2024-09-11T09:01:53.308889+0000","objects_scrubbed":9,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:22.530451+0000","scrub_duration":0.011841266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":77,"num_read_kb":66,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":313,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932543+0000","last_change":"2024-09-11T09:02:16.804258+0000","last_active":"2024-09-11T09:02:47.932543+0000","last_peered":"2024-09-11T09:02:47.932543+0000","last_clean":"2024-09-11T09:02:47.932543+0000","last_became_active":"2024-09-11T09:02:16.803424+0000","last_became_peered":"2024-09-11T09:02:16.803424+0000","last_unstale":"2024-09-11T09:02:47.932543+0000","last_undegraded":"2024-09-11T09:02:47.932543+0000","last_fullsized":"2024-09-11T09:02:47.932543+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:13.239031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:13.239031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:20.173694+0000","scrub_duration":0.001520041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":574,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930592+0000","last_change":"2024-09-11T09:02:16.655092+0000","last_active":"2024-09-11T09:02:47.930592+0000","last_peered":"2024-09-11T09:02:47.930592+0000","last_clean":"2024-09-11T09:02:47.930592+0000","last_became_active":"2024-09-11T08:52:18.584748+0000","last_became_peered":"2024-09-11T08:52:18.584748+0000","last_unstale":"2024-09-11T09:02:47.930592+0000","last_undegraded":"2024-09-11T09:02:47.930592+0000","last_fullsized":"2024-09-11T09:02:47.930592+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:16.655040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:56:46.744542+0000","last_clean_scrub_stamp":"2024-09-11T09:02:16.655040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:19.302489+0000","scrub_duration":0.001669696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":310,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:51.491632+0000","last_change":"2024-09-11T09:02:51.491550+0000","last_active":"2024-09-11T09:02:51.491632+0000","last_peered":"2024-09-11T09:02:51.491632+0000","last_clean":"2024-09-11T09:02:51.491632+0000","last_became_active":"2024-09-11T09:02:16.797777+0000","last_became_peered":"2024-09-11T09:02:16.797777+0000","last_unstale":"2024-09-11T09:02:51.491632+0000","last_undegraded":"2024-09-11T09:02:51.491632+0000","last_fullsized":"2024-09-11T09:02:51.491632+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:51.491474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:51.491474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:58.716857+0000","scrub_duration":0.0014864139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"199'81","reported_seq":321,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214831+0000","last_change":"2024-09-11T09:02:42.998461+0000","last_active":"2024-09-11T09:02:47.214831+0000","last_peered":"2024-09-11T09:02:47.214831+0000","last_clean":"2024-09-11T09:02:47.214831+0000","last_became_active":"2024-09-11T08:58:34.135770+0000","last_became_peered":"2024-09-11T08:58:34.135770+0000","last_unstale":"2024-09-11T09:02:47.214831+0000","last_undegraded":"2024-09-11T09:02:47.214831+0000","last_fullsized":"2024-09-11T09:02:47.214831+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:42.998406+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:42.998406+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:02.837909+0000","scrub_duration":0.001872744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"402'29","reported_seq":531,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212162+0000","last_change":"2024-09-11T09:01:13.988154+0000","last_active":"2024-09-11T09:02:47.212162+0000","last_peered":"2024-09-11T09:02:47.212162+0000","last_clean":"2024-09-11T09:02:47.212162+0000","last_became_active":"2024-09-11T08:38:40.973192+0000","last_became_peered":"2024-09-11T08:38:40.973192+0000","last_unstale":"2024-09-11T09:02:47.212162+0000","last_undegraded":"2024-09-11T09:02:47.212162+0000","last_fullsized":"2024-09-11T09:02:47.212162+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'29","last_scrub_stamp":"2024-09-11T09:01:13.988098+0000","last_deep_scrub":"402'29","last_deep_scrub_stamp":"2024-09-11T09:01:13.988098+0000","last_clean_scrub_stamp":"2024-09-11T09:01:13.988098+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002328933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":313,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932781+0000","last_change":"2024-09-11T09:02:16.801599+0000","last_active":"2024-09-11T09:02:47.932781+0000","last_peered":"2024-09-11T09:02:47.932781+0000","last_clean":"2024-09-11T09:02:47.932781+0000","last_became_active":"2024-09-11T09:02:16.801373+0000","last_became_peered":"2024-09-11T09:02:16.801373+0000","last_unstale":"2024-09-11T09:02:47.932781+0000","last_undegraded":"2024-09-11T09:02:47.932781+0000","last_fullsized":"2024-09-11T09:02:47.932781+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:14.700205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:14.700205+0000","last_clean_scrub_stamp":"2024-09-11T09:02:14.700205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:44.080217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":418,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930647+0000","last_change":"2024-09-11T09:02:20.652457+0000","last_active":"2024-09-11T09:02:47.930647+0000","last_peered":"2024-09-11T09:02:47.930647+0000","last_clean":"2024-09-11T09:02:47.930647+0000","last_became_active":"2024-09-11T09:01:39.869277+0000","last_became_peered":"2024-09-11T09:01:39.869277+0000","last_unstale":"2024-09-11T09:02:47.930647+0000","last_undegraded":"2024-09-11T09:02:47.930647+0000","last_fullsized":"2024-09-11T09:02:47.930647+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:20.652405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:02:20.652405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:31.207876+0000","scrub_duration":0.0011182480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":490,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214424+0000","last_change":"2024-09-11T09:02:32.868637+0000","last_active":"2024-09-11T09:02:47.214424+0000","last_peered":"2024-09-11T09:02:47.214424+0000","last_clean":"2024-09-11T09:02:47.214424+0000","last_became_active":"2024-09-11T08:38:44.998074+0000","last_became_peered":"2024-09-11T08:38:44.998074+0000","last_unstale":"2024-09-11T09:02:47.214424+0000","last_undegraded":"2024-09-11T09:02:47.214424+0000","last_fullsized":"2024-09-11T09:02:47.214424+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:32.868587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:33.228082+0000","last_clean_scrub_stamp":"2024-09-11T09:02:32.868587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:49.468361+0000","scrub_duration":0.001569478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"199'81","reported_seq":202,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932084+0000","last_change":"2024-09-11T09:02:25.397821+0000","last_active":"2024-09-11T09:02:47.932084+0000","last_peered":"2024-09-11T09:02:47.932084+0000","last_clean":"2024-09-11T09:02:47.932084+0000","last_became_active":"2024-09-11T09:02:16.803579+0000","last_became_peered":"2024-09-11T09:02:16.803579+0000","last_unstale":"2024-09-11T09:02:47.932084+0000","last_undegraded":"2024-09-11T09:02:47.932084+0000","last_fullsized":"2024-09-11T09:02:47.932084+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":4,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:25.397751+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:25.397751+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:32.968716+0000","scrub_duration":0.001575592,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"402'46","reported_seq":682,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210212+0000","last_change":"2024-09-11T09:02:24.263032+0000","last_active":"2024-09-11T09:02:47.210212+0000","last_peered":"2024-09-11T09:02:47.210212+0000","last_clean":"2024-09-11T09:02:47.210212+0000","last_became_active":"2024-09-11T08:58:34.130937+0000","last_became_peered":"2024-09-11T08:58:34.130937+0000","last_unstale":"2024-09-11T09:02:47.210212+0000","last_undegraded":"2024-09-11T09:02:47.210212+0000","last_fullsized":"2024-09-11T09:02:47.210212+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'46","last_scrub_stamp":"2024-09-11T09:02:24.262947+0000","last_deep_scrub":"124'22","last_deep_scrub_stamp":"2024-09-11T08:43:32.625376+0000","last_clean_scrub_stamp":"2024-09-11T09:02:24.262947+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:49.238019+0000","scrub_duration":0.0021749249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"56'2","reported_seq":362,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932746+0000","last_change":"2024-09-11T09:02:22.420296+0000","last_active":"2024-09-11T09:02:47.932746+0000","last_peered":"2024-09-11T09:02:47.932746+0000","last_clean":"2024-09-11T09:02:47.932746+0000","last_became_active":"2024-09-11T09:02:16.797349+0000","last_became_peered":"2024-09-11T09:02:16.797349+0000","last_unstale":"2024-09-11T09:02:47.932746+0000","last_undegraded":"2024-09-11T09:02:47.932746+0000","last_fullsized":"2024-09-11T09:02:47.932746+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-11T09:02:22.420155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:22.420155+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:46.820846+0000","scrub_duration":0.0020171049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"460'16","reported_seq":531,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:57.544147+0000","last_change":"2024-09-11T09:02:57.544088+0000","last_active":"2024-09-11T09:02:57.544147+0000","last_peered":"2024-09-11T09:02:57.544147+0000","last_clean":"2024-09-11T09:02:57.544147+0000","last_became_active":"2024-09-11T09:02:16.796406+0000","last_became_peered":"2024-09-11T09:02:16.796406+0000","last_unstale":"2024-09-11T09:02:57.544147+0000","last_undegraded":"2024-09-11T09:02:57.544147+0000","last_fullsized":"2024-09-11T09:02:57.544147+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"460'16","last_scrub_stamp":"2024-09-11T09:02:57.544016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:02:57.544016+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:19.084020+0000","scrub_duration":0.001677879,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":539,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210232+0000","last_change":"2024-09-11T09:01:44.323080+0000","last_active":"2024-09-11T09:02:47.210232+0000","last_peered":"2024-09-11T09:02:47.210232+0000","last_clean":"2024-09-11T09:02:47.210232+0000","last_became_active":"2024-09-11T08:58:34.128990+0000","last_became_peered":"2024-09-11T08:58:34.128990+0000","last_unstale":"2024-09-11T09:02:47.210232+0000","last_undegraded":"2024-09-11T09:02:47.210232+0000","last_fullsized":"2024-09-11T09:02:47.210232+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:44.323026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:01:44.323026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:04.370889+0000","scrub_duration":0.001236406,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"199'81","reported_seq":138,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.210384+0000","last_change":"2024-09-11T09:02:04.278619+0000","last_active":"2024-09-11T09:02:47.210384+0000","last_peered":"2024-09-11T09:02:47.210384+0000","last_clean":"2024-09-11T09:02:47.210384+0000","last_became_active":"2024-09-11T09:01:28.496162+0000","last_became_peered":"2024-09-11T09:01:28.496162+0000","last_unstale":"2024-09-11T09:02:47.210384+0000","last_undegraded":"2024-09-11T09:02:47.210384+0000","last_fullsized":"2024-09-11T09:02:47.210384+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":5,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:04.278532+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:57:49.622788+0000","last_clean_scrub_stamp":"2024-09-11T09:02:04.278532+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:09.874878+0000","scrub_duration":0.0018337760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":439,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:54.808356+0000","last_change":"2024-09-11T09:02:54.808307+0000","last_active":"2024-09-11T09:02:54.808356+0000","last_peered":"2024-09-11T09:02:54.808356+0000","last_clean":"2024-09-11T09:02:54.808356+0000","last_became_active":"2024-09-11T09:02:09.770004+0000","last_became_peered":"2024-09-11T09:02:09.770004+0000","last_unstale":"2024-09-11T09:02:54.808356+0000","last_undegraded":"2024-09-11T09:02:54.808356+0000","last_fullsized":"2024-09-11T09:02:54.808356+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:54.808243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:45:28.247771+0000","last_clean_scrub_stamp":"2024-09-11T09:02:54.808243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:57.208244+0000","scrub_duration":0.0014588870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"402'18","reported_seq":539,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214884+0000","last_change":"2024-09-11T09:02:13.832716+0000","last_active":"2024-09-11T09:02:47.214884+0000","last_peered":"2024-09-11T09:02:47.214884+0000","last_clean":"2024-09-11T09:02:47.214884+0000","last_became_active":"2024-09-11T08:40:16.184217+0000","last_became_peered":"2024-09-11T08:40:16.184217+0000","last_unstale":"2024-09-11T09:02:47.214884+0000","last_undegraded":"2024-09-11T09:02:47.214884+0000","last_fullsized":"2024-09-11T09:02:47.214884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'18","last_scrub_stamp":"2024-09-11T09:02:13.832656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:13.832656+0000","objects_scrubbed":3,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:33.245733+0000","scrub_duration":0.001811434,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":502,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.211016+0000","last_change":"2024-09-11T09:01:46.346364+0000","last_active":"2024-09-11T09:02:47.211016+0000","last_peered":"2024-09-11T09:02:47.211016+0000","last_clean":"2024-09-11T09:02:47.211016+0000","last_became_active":"2024-09-11T08:38:42.975621+0000","last_became_peered":"2024-09-11T08:38:42.975621+0000","last_unstale":"2024-09-11T09:02:47.211016+0000","last_undegraded":"2024-09-11T09:02:47.211016+0000","last_fullsized":"2024-09-11T09:02:47.211016+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:46.346312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:01:46.346312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:14.064974+0000","scrub_duration":0.0015947590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":572,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932016+0000","last_change":"2024-09-11T09:02:29.344893+0000","last_active":"2024-09-11T09:02:47.932016+0000","last_peered":"2024-09-11T09:02:47.932016+0000","last_clean":"2024-09-11T09:02:47.932016+0000","last_became_active":"2024-09-11T08:58:34.135112+0000","last_became_peered":"2024-09-11T08:58:34.135112+0000","last_unstale":"2024-09-11T09:02:47.932016+0000","last_undegraded":"2024-09-11T09:02:47.932016+0000","last_fullsized":"2024-09-11T09:02:47.932016+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:29.344825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:29.344825+0000","last_clean_scrub_stamp":"2024-09-11T09:02:29.344825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:30.040040+0000","scrub_duration":0.001588768,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"199'81","reported_seq":568,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930693+0000","last_change":"2024-09-11T09:02:16.221457+0000","last_active":"2024-09-11T09:02:47.930693+0000","last_peered":"2024-09-11T09:02:47.930693+0000","last_clean":"2024-09-11T09:02:47.930693+0000","last_became_active":"2024-09-11T08:58:03.129022+0000","last_became_peered":"2024-09-11T08:58:03.129022+0000","last_unstale":"2024-09-11T09:02:47.930693+0000","last_undegraded":"2024-09-11T09:02:47.930693+0000","last_fullsized":"2024-09-11T09:02:47.930693+0000","mapping_epoch":385,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":386,"parent":"0.0","parent_split_bits":5,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:16.221383+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:16.221383+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:21.455145+0000","scrub_duration":0.0019473030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"462'25","reported_seq":512,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932678+0000","last_change":"2024-09-11T09:02:36.944835+0000","last_active":"2024-09-11T09:02:47.932678+0000","last_peered":"2024-09-11T09:02:47.932678+0000","last_clean":"2024-09-11T09:02:47.932678+0000","last_became_active":"2024-09-11T09:01:25.405993+0000","last_became_peered":"2024-09-11T09:01:25.405993+0000","last_unstale":"2024-09-11T09:02:47.932678+0000","last_undegraded":"2024-09-11T09:02:47.932678+0000","last_fullsized":"2024-09-11T09:02:47.932678+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"462'25","last_scrub_stamp":"2024-09-11T09:02:36.944772+0000","last_deep_scrub":"462'25","last_deep_scrub_stamp":"2024-09-11T09:02:36.944772+0000","last_clean_scrub_stamp":"2024-09-11T09:02:36.944772+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:45.238295+0000","scrub_duration":0.0036256919999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":316,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931328+0000","last_change":"2024-09-11T09:02:16.801305+0000","last_active":"2024-09-11T09:02:47.931328+0000","last_peered":"2024-09-11T09:02:47.931328+0000","last_clean":"2024-09-11T09:02:47.931328+0000","last_became_active":"2024-09-11T09:02:16.800720+0000","last_became_peered":"2024-09-11T09:02:16.800720+0000","last_unstale":"2024-09-11T09:02:47.931328+0000","last_undegraded":"2024-09-11T09:02:47.931328+0000","last_fullsized":"2024-09-11T09:02:47.931328+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:58.761306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:44:32.483439+0000","last_clean_scrub_stamp":"2024-09-11T09:01:58.761306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:27.423944+0000","scrub_duration":0.0042052879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":492,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:59.890496+0000","last_change":"2024-09-11T09:02:59.890451+0000","last_active":"2024-09-11T09:02:59.890496+0000","last_peered":"2024-09-11T09:02:59.890496+0000","last_clean":"2024-09-11T09:02:59.890496+0000","last_became_active":"2024-09-11T08:38:42.981822+0000","last_became_peered":"2024-09-11T08:38:42.981822+0000","last_unstale":"2024-09-11T09:02:59.890496+0000","last_undegraded":"2024-09-11T09:02:59.890496+0000","last_fullsized":"2024-09-11T09:02:59.890496+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:59.890391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:59.890391+0000","last_clean_scrub_stamp":"2024-09-11T09:02:59.890391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:28.635200+0000","scrub_duration":0.001261755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":550,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:52.464364+0000","last_change":"2024-09-11T09:02:52.464309+0000","last_active":"2024-09-11T09:02:52.464364+0000","last_peered":"2024-09-11T09:02:52.464364+0000","last_clean":"2024-09-11T09:02:52.464364+0000","last_became_active":"2024-09-11T08:38:44.991493+0000","last_became_peered":"2024-09-11T08:38:44.991493+0000","last_unstale":"2024-09-11T09:02:52.464364+0000","last_undegraded":"2024-09-11T09:02:52.464364+0000","last_fullsized":"2024-09-11T09:02:52.464364+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:52.464244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:59:52.054422+0000","last_clean_scrub_stamp":"2024-09-11T09:02:52.464244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:05.769952+0000","scrub_duration":0.00165238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"199'81","reported_seq":128,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.209939+0000","last_change":"2024-09-11T09:02:17.278096+0000","last_active":"2024-09-11T09:02:47.209939+0000","last_peered":"2024-09-11T09:02:47.209939+0000","last_clean":"2024-09-11T09:02:47.209939+0000","last_became_active":"2024-09-11T08:58:34.130875+0000","last_became_peered":"2024-09-11T08:58:34.130875+0000","last_unstale":"2024-09-11T09:02:47.209939+0000","last_undegraded":"2024-09-11T09:02:47.209939+0000","last_fullsized":"2024-09-11T09:02:47.209939+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":5,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:17.278031+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T09:02:17.278031+0000","last_clean_scrub_stamp":"2024-09-11T09:02:17.278031+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:29.771668+0000","scrub_duration":0.0015397919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"443'25","reported_seq":568,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:55.045616+0000","last_change":"2024-09-11T09:02:55.045533+0000","last_active":"2024-09-11T09:02:55.045616+0000","last_peered":"2024-09-11T09:02:55.045616+0000","last_clean":"2024-09-11T09:02:55.045616+0000","last_became_active":"2024-09-11T08:53:58.555987+0000","last_became_peered":"2024-09-11T08:53:58.555987+0000","last_unstale":"2024-09-11T09:02:55.045616+0000","last_undegraded":"2024-09-11T09:02:55.045616+0000","last_fullsized":"2024-09-11T09:02:55.045616+0000","mapping_epoch":315,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'25","last_scrub_stamp":"2024-09-11T09:02:55.045428+0000","last_deep_scrub":"443'25","last_deep_scrub_stamp":"2024-09-11T09:01:31.380448+0000","last_clean_scrub_stamp":"2024-09-11T09:02:55.045428+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:04.876290+0000","scrub_duration":0.0021129880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":68,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931378+0000","last_change":"2024-09-11T09:01:41.930052+0000","last_active":"2024-09-11T09:02:47.931378+0000","last_peered":"2024-09-11T09:02:47.931378+0000","last_clean":"2024-09-11T09:02:47.931378+0000","last_became_active":"2024-09-11T09:01:25.402878+0000","last_became_peered":"2024-09-11T09:01:25.402878+0000","last_unstale":"2024-09-11T09:02:47.931378+0000","last_undegraded":"2024-09-11T09:02:47.931378+0000","last_fullsized":"2024-09-11T09:02:47.931378+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:41.929995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:55:03.641009+0000","last_clean_scrub_stamp":"2024-09-11T09:01:41.929995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:10.513168+0000","scrub_duration":0.0014327949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":782,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:55.241856+0000","last_change":"2024-09-11T09:01:49.788850+0000","last_active":"2024-09-11T09:02:55.241856+0000","last_peered":"2024-09-11T09:02:55.241856+0000","last_clean":"2024-09-11T09:02:55.241856+0000","last_became_active":"2024-09-11T08:38:42.981647+0000","last_became_peered":"2024-09-11T08:38:42.981647+0000","last_unstale":"2024-09-11T09:02:55.241856+0000","last_undegraded":"2024-09-11T09:02:55.241856+0000","last_fullsized":"2024-09-11T09:02:55.241856+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-11T09:01:49.788792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:01:49.788792+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:15.463889+0000","scrub_duration":0.0017970480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":484,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.212994+0000","last_change":"2024-09-11T09:02:18.805812+0000","last_active":"2024-09-11T09:02:47.212994+0000","last_peered":"2024-09-11T09:02:47.212994+0000","last_clean":"2024-09-11T09:02:47.212994+0000","last_became_active":"2024-09-11T08:38:44.993690+0000","last_became_peered":"2024-09-11T08:38:44.993690+0000","last_unstale":"2024-09-11T09:02:47.212994+0000","last_undegraded":"2024-09-11T09:02:47.212994+0000","last_fullsized":"2024-09-11T09:02:47.212994+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:18.805748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:18.805748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:22.278065+0000","scrub_duration":0.0014510829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"199'81","reported_seq":84,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930784+0000","last_change":"2024-09-11T09:02:05.924533+0000","last_active":"2024-09-11T09:02:47.930784+0000","last_peered":"2024-09-11T09:02:47.930784+0000","last_clean":"2024-09-11T09:02:47.930784+0000","last_became_active":"2024-09-11T09:01:25.398637+0000","last_became_peered":"2024-09-11T09:01:25.398637+0000","last_unstale":"2024-09-11T09:02:47.930784+0000","last_undegraded":"2024-09-11T09:02:47.930784+0000","last_fullsized":"2024-09-11T09:02:47.930784+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":5,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:05.924475+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T08:48:46.028660+0000","last_clean_scrub_stamp":"2024-09-11T09:02:05.924475+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:15.208417+0000","scrub_duration":0.0015511259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"402'35","reported_seq":588,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214908+0000","last_change":"2024-09-11T09:02:22.820209+0000","last_active":"2024-09-11T09:02:47.214908+0000","last_peered":"2024-09-11T09:02:47.214908+0000","last_clean":"2024-09-11T09:02:47.214908+0000","last_became_active":"2024-09-11T09:01:28.504757+0000","last_became_peered":"2024-09-11T09:01:28.504757+0000","last_unstale":"2024-09-11T09:02:47.214908+0000","last_undegraded":"2024-09-11T09:02:47.214908+0000","last_fullsized":"2024-09-11T09:02:47.214908+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'35","last_scrub_stamp":"2024-09-11T09:02:22.820112+0000","last_deep_scrub":"152'15","last_deep_scrub_stamp":"2024-09-11T08:45:21.132083+0000","last_clean_scrub_stamp":"2024-09-11T09:02:22.820112+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:38.227044+0000","scrub_duration":0.0061296340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":545,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932022+0000","last_change":"2024-09-11T09:02:12.493237+0000","last_active":"2024-09-11T09:02:47.932022+0000","last_peered":"2024-09-11T09:02:47.932022+0000","last_clean":"2024-09-11T09:02:47.932022+0000","last_became_active":"2024-09-11T08:38:38.949606+0000","last_became_peered":"2024-09-11T08:38:38.949606+0000","last_unstale":"2024-09-11T09:02:47.932022+0000","last_undegraded":"2024-09-11T09:02:47.932022+0000","last_fullsized":"2024-09-11T09:02:47.932022+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:12.493172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:54:15.235519+0000","last_clean_scrub_stamp":"2024-09-11T09:02:12.493172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:22.197202+0000","scrub_duration":0.0025578910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"277'10","reported_seq":830,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:55.241567+0000","last_change":"2024-09-11T09:01:58.315885+0000","last_active":"2024-09-11T09:02:55.241567+0000","last_peered":"2024-09-11T09:02:55.241567+0000","last_clean":"2024-09-11T09:02:55.241567+0000","last_became_active":"2024-09-11T08:52:18.585914+0000","last_became_peered":"2024-09-11T08:52:18.585914+0000","last_unstale":"2024-09-11T09:02:55.241567+0000","last_undegraded":"2024-09-11T09:02:55.241567+0000","last_fullsized":"2024-09-11T09:02:55.241567+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'10","last_scrub_stamp":"2024-09-11T09:01:58.315822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:01:58.315822+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:00.336225+0000","scrub_duration":0.0013541219999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":68,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932758+0000","last_change":"2024-09-11T09:02:15.875014+0000","last_active":"2024-09-11T09:02:47.932758+0000","last_peered":"2024-09-11T09:02:47.932758+0000","last_clean":"2024-09-11T09:02:47.932758+0000","last_became_active":"2024-09-11T09:01:25.405725+0000","last_became_peered":"2024-09-11T09:01:25.405725+0000","last_unstale":"2024-09-11T09:02:47.932758+0000","last_undegraded":"2024-09-11T09:02:47.932758+0000","last_fullsized":"2024-09-11T09:02:47.932758+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:15.874955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:15.874955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:41.092556+0000","scrub_duration":0.0035339189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"402'27","reported_seq":604,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:57.851270+0000","last_change":"2024-09-11T09:02:57.851233+0000","last_active":"2024-09-11T09:02:57.851270+0000","last_peered":"2024-09-11T09:02:57.851270+0000","last_clean":"2024-09-11T09:02:57.851270+0000","last_became_active":"2024-09-11T08:58:34.135591+0000","last_became_peered":"2024-09-11T08:58:34.135591+0000","last_unstale":"2024-09-11T09:02:57.851270+0000","last_undegraded":"2024-09-11T09:02:57.851270+0000","last_fullsized":"2024-09-11T09:02:57.851270+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'27","last_scrub_stamp":"2024-09-11T09:02:57.851163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:57.851163+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:26.173111+0000","scrub_duration":0.002053095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":517,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:49.885810+0000","last_change":"2024-09-11T09:02:49.885750+0000","last_active":"2024-09-11T09:02:49.885810+0000","last_peered":"2024-09-11T09:02:49.885810+0000","last_clean":"2024-09-11T09:02:49.885810+0000","last_became_active":"2024-09-11T08:58:34.138928+0000","last_became_peered":"2024-09-11T08:58:34.138928+0000","last_unstale":"2024-09-11T09:02:49.885810+0000","last_undegraded":"2024-09-11T09:02:49.885810+0000","last_fullsized":"2024-09-11T09:02:49.885810+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:49.885681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:49.885681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:15.037792+0000","scrub_duration":0.001492761,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":532,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:57.783055+0000","last_change":"2024-09-11T09:02:57.783029+0000","last_active":"2024-09-11T09:02:57.783055+0000","last_peered":"2024-09-11T09:02:57.783055+0000","last_clean":"2024-09-11T09:02:57.783055+0000","last_became_active":"2024-09-11T08:38:42.978872+0000","last_became_peered":"2024-09-11T08:38:42.978872+0000","last_unstale":"2024-09-11T09:02:57.783055+0000","last_undegraded":"2024-09-11T09:02:57.783055+0000","last_fullsized":"2024-09-11T09:02:57.783055+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:57.782972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:02:57.782972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:17.726614+0000","scrub_duration":0.001383057,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":510,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214429+0000","last_change":"2024-09-11T09:02:38.917179+0000","last_active":"2024-09-11T09:02:47.214429+0000","last_peered":"2024-09-11T09:02:47.214429+0000","last_clean":"2024-09-11T09:02:47.214429+0000","last_became_active":"2024-09-11T08:52:18.585089+0000","last_became_peered":"2024-09-11T08:52:18.585089+0000","last_unstale":"2024-09-11T09:02:47.214429+0000","last_undegraded":"2024-09-11T09:02:47.214429+0000","last_fullsized":"2024-09-11T09:02:47.214429+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:38.917120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:38.917120+0000","last_clean_scrub_stamp":"2024-09-11T09:02:38.917120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:08.060616+0000","scrub_duration":0.001747572,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"448'25","reported_seq":55,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930707+0000","last_change":"2024-09-11T09:01:46.274566+0000","last_active":"2024-09-11T09:02:47.930707+0000","last_peered":"2024-09-11T09:02:47.930707+0000","last_clean":"2024-09-11T09:02:47.930707+0000","last_became_active":"2024-09-11T09:01:28.498888+0000","last_became_peered":"2024-09-11T09:01:28.498888+0000","last_unstale":"2024-09-11T09:02:47.930707+0000","last_undegraded":"2024-09-11T09:02:47.930707+0000","last_fullsized":"2024-09-11T09:02:47.930707+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"448'25","last_scrub_stamp":"2024-09-11T09:01:46.274515+0000","last_deep_scrub":"448'25","last_deep_scrub_stamp":"2024-09-11T09:01:46.274515+0000","last_clean_scrub_stamp":"2024-09-11T09:01:46.274515+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:06.032306+0000","scrub_duration":0.0032311929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":323,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932440+0000","last_change":"2024-09-11T09:02:18.456738+0000","last_active":"2024-09-11T09:02:47.932440+0000","last_peered":"2024-09-11T09:02:47.932440+0000","last_clean":"2024-09-11T09:02:47.932440+0000","last_became_active":"2024-09-11T09:02:16.797837+0000","last_became_peered":"2024-09-11T09:02:16.797837+0000","last_unstale":"2024-09-11T09:02:47.932440+0000","last_undegraded":"2024-09-11T09:02:47.932440+0000","last_fullsized":"2024-09-11T09:02:47.932440+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:18.456672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:21.208684+0000","last_clean_scrub_stamp":"2024-09-11T09:02:18.456672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:18.517939+0000","scrub_duration":0.001458336,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":512,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214462+0000","last_change":"2024-09-11T09:02:28.806185+0000","last_active":"2024-09-11T09:02:47.214462+0000","last_peered":"2024-09-11T09:02:47.214462+0000","last_clean":"2024-09-11T09:02:47.214462+0000","last_became_active":"2024-09-11T08:52:18.585691+0000","last_became_peered":"2024-09-11T08:52:18.585691+0000","last_unstale":"2024-09-11T09:02:47.214462+0000","last_undegraded":"2024-09-11T09:02:47.214462+0000","last_fullsized":"2024-09-11T09:02:47.214462+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:28.806128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:02:28.806128+0000","last_clean_scrub_stamp":"2024-09-11T09:02:28.806128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:52.254759+0000","scrub_duration":0.0012738910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":137,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932383+0000","last_change":"2024-09-11T09:01:33.974905+0000","last_active":"2024-09-11T09:02:47.932383+0000","last_peered":"2024-09-11T09:02:47.932383+0000","last_clean":"2024-09-11T09:02:47.932383+0000","last_became_active":"2024-09-11T09:01:25.419653+0000","last_became_peered":"2024-09-11T09:01:25.419653+0000","last_unstale":"2024-09-11T09:02:47.932383+0000","last_undegraded":"2024-09-11T09:02:47.932383+0000","last_fullsized":"2024-09-11T09:02:47.932383+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:33.974842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:24.401480+0000","last_clean_scrub_stamp":"2024-09-11T09:01:33.974842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:00.737680+0000","scrub_duration":0.0018484700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"199'81","reported_seq":118,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930736+0000","last_change":"2024-09-11T09:02:26.705339+0000","last_active":"2024-09-11T09:02:47.930736+0000","last_peered":"2024-09-11T09:02:47.930736+0000","last_clean":"2024-09-11T09:02:47.930736+0000","last_became_active":"2024-09-11T08:58:03.128935+0000","last_became_peered":"2024-09-11T08:58:03.128935+0000","last_unstale":"2024-09-11T09:02:47.930736+0000","last_undegraded":"2024-09-11T09:02:47.930736+0000","last_fullsized":"2024-09-11T09:02:47.930736+0000","mapping_epoch":385,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":386,"parent":"0.0","parent_split_bits":5,"last_scrub":"199'81","last_scrub_stamp":"2024-09-11T09:02:26.705254+0000","last_deep_scrub":"199'81","last_deep_scrub_stamp":"2024-09-11T09:02:26.705254+0000","last_clean_scrub_stamp":"2024-09-11T09:02:26.705254+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:41.908234+0000","scrub_duration":0.0019461490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"402'27","reported_seq":571,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214274+0000","last_change":"2024-09-11T09:02:06.720016+0000","last_active":"2024-09-11T09:02:47.214274+0000","last_peered":"2024-09-11T09:02:47.214274+0000","last_clean":"2024-09-11T09:02:47.214274+0000","last_became_active":"2024-09-11T08:58:34.136417+0000","last_became_peered":"2024-09-11T08:58:34.136417+0000","last_unstale":"2024-09-11T09:02:47.214274+0000","last_undegraded":"2024-09-11T09:02:47.214274+0000","last_fullsized":"2024-09-11T09:02:47.214274+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'27","last_scrub_stamp":"2024-09-11T09:02:06.719953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:02:06.719953+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:10.448704+0000","scrub_duration":0.0021473439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":536,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:53.069232+0000","last_change":"2024-09-11T09:02:53.069185+0000","last_active":"2024-09-11T09:02:53.069232+0000","last_peered":"2024-09-11T09:02:53.069232+0000","last_clean":"2024-09-11T09:02:53.069232+0000","last_became_active":"2024-09-11T09:01:28.499223+0000","last_became_peered":"2024-09-11T09:01:28.499223+0000","last_unstale":"2024-09-11T09:02:53.069232+0000","last_undegraded":"2024-09-11T09:02:53.069232+0000","last_fullsized":"2024-09-11T09:02:53.069232+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:53.069122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:53.069122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:13.418207+0000","scrub_duration":0.001505752,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":567,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.930864+0000","last_change":"2024-09-11T09:01:57.539656+0000","last_active":"2024-09-11T09:02:47.930864+0000","last_peered":"2024-09-11T09:02:47.930864+0000","last_clean":"2024-09-11T09:02:47.930864+0000","last_became_active":"2024-09-11T08:58:34.135692+0000","last_became_peered":"2024-09-11T08:58:34.135692+0000","last_unstale":"2024-09-11T09:02:47.930864+0000","last_undegraded":"2024-09-11T09:02:47.930864+0000","last_fullsized":"2024-09-11T09:02:47.930864+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:57.539603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:01:57.539603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:18.127353+0000","scrub_duration":0.0011331589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":540,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.214587+0000","last_change":"2024-09-11T09:01:50.781018+0000","last_active":"2024-09-11T09:02:47.214587+0000","last_peered":"2024-09-11T09:02:47.214587+0000","last_clean":"2024-09-11T09:02:47.214587+0000","last_became_active":"2024-09-11T08:58:34.135699+0000","last_became_peered":"2024-09-11T08:58:34.135699+0000","last_unstale":"2024-09-11T09:02:47.214587+0000","last_undegraded":"2024-09-11T09:02:47.214587+0000","last_fullsized":"2024-09-11T09:02:47.214587+0000","mapping_epoch":399,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":400,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:50.780955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:50:25.255332+0000","last_clean_scrub_stamp":"2024-09-11T09:01:50.780955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:03.552990+0000","scrub_duration":0.001653409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"402'14","reported_seq":97,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.209925+0000","last_change":"2024-09-11T09:02:46.160216+0000","last_active":"2024-09-11T09:02:47.209925+0000","last_peered":"2024-09-11T09:02:47.209925+0000","last_clean":"2024-09-11T09:02:47.209925+0000","last_became_active":"2024-09-11T08:59:18.588313+0000","last_became_peered":"2024-09-11T08:59:18.588313+0000","last_unstale":"2024-09-11T09:02:47.209925+0000","last_undegraded":"2024-09-11T09:02:47.209925+0000","last_fullsized":"2024-09-11T09:02:47.209925+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'14","last_scrub_stamp":"2024-09-11T09:02:46.160141+0000","last_deep_scrub":"136'6","last_deep_scrub_stamp":"2024-09-11T08:44:21.302870+0000","last_clean_scrub_stamp":"2024-09-11T09:02:46.160141+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:55.146328+0000","scrub_duration":0.0018706040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"56'1","reported_seq":72,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932731+0000","last_change":"2024-09-11T09:02:45.922265+0000","last_active":"2024-09-11T09:02:47.932731+0000","last_peered":"2024-09-11T09:02:47.932731+0000","last_clean":"2024-09-11T09:02:47.932731+0000","last_became_active":"2024-09-11T09:01:25.402119+0000","last_became_peered":"2024-09-11T09:01:25.402119+0000","last_unstale":"2024-09-11T09:02:47.932731+0000","last_undegraded":"2024-09-11T09:02:47.932731+0000","last_fullsized":"2024-09-11T09:02:47.932731+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-11T09:02:45.922211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:37.932007+0000","last_clean_scrub_stamp":"2024-09-11T09:02:45.922211+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:12.899971+0000","scrub_duration":0.001718097,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":153,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931066+0000","last_change":"2024-09-11T09:02:09.772508+0000","last_active":"2024-09-11T09:02:47.931066+0000","last_peered":"2024-09-11T09:02:47.931066+0000","last_clean":"2024-09-11T09:02:47.931066+0000","last_became_active":"2024-09-11T09:02:09.772205+0000","last_became_peered":"2024-09-11T09:02:09.772205+0000","last_unstale":"2024-09-11T09:02:47.931066+0000","last_undegraded":"2024-09-11T09:02:47.931066+0000","last_fullsized":"2024-09-11T09:02:47.931066+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:57.769178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:01:57.769178+0000","last_clean_scrub_stamp":"2024-09-11T09:01:57.769178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:09.765725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":307,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931589+0000","last_change":"2024-09-11T09:02:16.799283+0000","last_active":"2024-09-11T09:02:47.931589+0000","last_peered":"2024-09-11T09:02:47.931589+0000","last_clean":"2024-09-11T09:02:47.931589+0000","last_became_active":"2024-09-11T09:02:16.798333+0000","last_became_peered":"2024-09-11T09:02:16.798333+0000","last_unstale":"2024-09-11T09:02:47.931589+0000","last_undegraded":"2024-09-11T09:02:47.931589+0000","last_fullsized":"2024-09-11T09:02:47.931589+0000","mapping_epoch":459,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:01:36.694030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:01:36.694030+0000","last_clean_scrub_stamp":"2024-09-11T09:01:36.694030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:03.137902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":530,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932476+0000","last_change":"2024-09-11T09:02:19.321545+0000","last_active":"2024-09-11T09:02:47.932476+0000","last_peered":"2024-09-11T09:02:47.932476+0000","last_clean":"2024-09-11T09:02:47.932476+0000","last_became_active":"2024-09-11T08:38:44.988630+0000","last_became_peered":"2024-09-11T08:38:44.988630+0000","last_unstale":"2024-09-11T09:02:47.932476+0000","last_undegraded":"2024-09-11T09:02:47.932476+0000","last_fullsized":"2024-09-11T09:02:47.932476+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:19.321484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:43.973425+0000","last_clean_scrub_stamp":"2024-09-11T09:02:19.321484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:29.810544+0000","scrub_duration":0.0018493750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":546,"reported_epoch":464,"state":"active+clean","last_fresh":"2024-09-11T09:02:57.874790+0000","last_change":"2024-09-11T09:02:57.874764+0000","last_active":"2024-09-11T09:02:57.874790+0000","last_peered":"2024-09-11T09:02:57.874790+0000","last_clean":"2024-09-11T09:02:57.874790+0000","last_became_active":"2024-09-11T09:01:28.499661+0000","last_became_peered":"2024-09-11T09:01:28.499661+0000","last_unstale":"2024-09-11T09:02:57.874790+0000","last_undegraded":"2024-09-11T09:02:57.874790+0000","last_fullsized":"2024-09-11T09:02:57.874790+0000","mapping_epoch":441,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:57.874711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:51:32.393197+0000","last_clean_scrub_stamp":"2024-09-11T09:02:57.874711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:20.411716+0000","scrub_duration":0.001187486,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"402'27","reported_seq":218,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.931621+0000","last_change":"2024-09-11T09:02:09.769791+0000","last_active":"2024-09-11T09:02:47.931621+0000","last_peered":"2024-09-11T09:02:47.931621+0000","last_clean":"2024-09-11T09:02:47.931621+0000","last_became_active":"2024-09-11T09:02:09.769396+0000","last_became_peered":"2024-09-11T09:02:09.769396+0000","last_unstale":"2024-09-11T09:02:47.931621+0000","last_undegraded":"2024-09-11T09:02:47.931621+0000","last_fullsized":"2024-09-11T09:02:47.931621+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"402'27","last_scrub_stamp":"2024-09-11T09:01:38.350893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:39.947194+0000","last_clean_scrub_stamp":"2024-09-11T09:01:38.350893+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:04.648781+0000","scrub_duration":0.0096371630000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":137,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-11T09:02:47.932577+0000","last_change":"2024-09-11T09:02:31.845724+0000","last_active":"2024-09-11T09:02:47.932577+0000","last_peered":"2024-09-11T09:02:47.932577+0000","last_clean":"2024-09-11T09:02:47.932577+0000","last_became_active":"2024-09-11T09:01:25.405212+0000","last_became_peered":"2024-09-11T09:01:25.405212+0000","last_unstale":"2024-09-11T09:02:47.932577+0000","last_undegraded":"2024-09-11T09:02:47.932577+0000","last_fullsized":"2024-09-11T09:02:47.932577+0000","mapping_epoch":439,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:02:31.845670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:38:41.959472+0000","last_clean_scrub_stamp":"2024-09-11T09:02:31.845670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:43.176324+0000","scrub_duration":0.001355689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1531,"num_read_kb":1288,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":8164,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1080,"ondisk_log_size":1080,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":2294,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":135,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299691,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":893808,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":893054,"kb_avail":92851344,"statfs":{"total":95995035648,"available":95079776256,"internally_reserved":0,"allocated":761856,"data_stored":446433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":914487640},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.67500000000000004,"15min":0.82899999999999996},"min":{"1min":0.34200000000000003,"5min":0.22500000000000001,"15min":0.20599999999999999},"max":{"1min":0.70499999999999996,"5min":4.133,"15min":5.0449999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.73799999999999999,"15min":0.88},"min":{"1min":0.37,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":0.73299999999999998,"5min":4.5289999999999999,"15min":4.5289999999999999},"last":0.42399999999999999}]},{"osd":1,"last update":"Wed Sep 11 09:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.98199999999999998,"15min":1.01},"min":{"1min":0.379,"5min":0.249,"15min":0.224},"max":{"1min":0.73299999999999998,"5min":26.193999999999999,"15min":26.193999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":1.05,"15min":0.95399999999999996},"min":{"1min":0.31900000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.72099999999999997,"5min":27.058,"15min":27.058},"last":0.46999999999999997}]},{"osd":2,"last update":"Wed Sep 11 09:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":1.925,"15min":1.306},"min":{"1min":0.35999999999999999,"5min":0.27600000000000002,"15min":0.27000000000000002},"max":{"1min":0.61199999999999999,"5min":116.639,"15min":116.639},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":1.8919999999999999,"15min":1.264},"min":{"1min":0.38700000000000001,"5min":0.28799999999999998,"15min":0.25800000000000001},"max":{"1min":1.145,"5min":116.68899999999999,"15min":116.68899999999999},"last":0.48399999999999999}]},{"osd":3,"last update":"Wed Sep 11 09:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":1.2629999999999999,"15min":1.403},"min":{"1min":0.40600000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.155,"5min":26.225000000000001,"15min":56.265999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":1.4570000000000001,"15min":1.4730000000000001},"min":{"1min":0.39200000000000002,"5min":0.29599999999999999,"15min":0.27700000000000002},"max":{"1min":0.70899999999999996,"5min":26.280999999999999,"15min":57.207000000000001},"last":0.51400000000000001}]},{"osd":4,"last update":"Wed Sep 11 09:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.73999999999999999,"15min":1.482},"min":{"1min":0.30299999999999999,"5min":0.24199999999999999,"15min":0.224},"max":{"1min":0.81399999999999995,"5min":9.0879999999999992,"15min":163.60499999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.59499999999999997,"15min":1.5349999999999999},"min":{"1min":0.29499999999999998,"5min":0.25,"15min":0.20799999999999999},"max":{"1min":0.79700000000000004,"5min":5.1529999999999996,"15min":163.88300000000001},"last":0.46000000000000002}]},{"osd":5,"last update":"Wed Sep 11 09:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.64600000000000002,"15min":1.04},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.23699999999999999},"max":{"1min":0.69999999999999996,"5min":4.7000000000000002,"15min":50.128},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.74099999999999999,"15min":1.085},"min":{"1min":0.29799999999999999,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.77900000000000003,"5min":7.1820000000000004,"15min":50.241},"last":0.56699999999999995}]},{"osd":6,"last update":"Wed Sep 11 09:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":2.8900000000000001,"15min":1.6339999999999999},"min":{"1min":0.316,"5min":0.30599999999999999,"15min":0.29399999999999998},"max":{"1min":0.85799999999999998,"5min":254.94999999999999,"15min":254.94999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":2.581,"15min":1.554},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.25},"max":{"1min":0.71199999999999997,"5min":254.88,"15min":254.88},"last":0.61699999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528624,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":898280,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":897214,"kb_avail":92846872,"statfs":{"total":95995035648,"available":95075196928,"internally_reserved":0,"allocated":1081344,"data_stored":774059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":918747480},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.82299999999999995,"15min":1.5289999999999999},"min":{"1min":0.39700000000000002,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":1.5089999999999999,"5min":7.5270000000000001,"15min":121.208},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.90300000000000002,"15min":1.6930000000000001},"min":{"1min":0.35099999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":4.6619999999999999,"5min":5.2670000000000003,"15min":121.05200000000001},"last":0.44}]},{"osd":1,"last update":"Wed Sep 11 09:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":1.0489999999999999,"15min":1.1879999999999999},"min":{"1min":0.39200000000000002,"5min":0.216,"15min":0.216},"max":{"1min":1.0580000000000001,"5min":7.4180000000000001,"15min":27.984000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.996,"15min":1.1499999999999999},"min":{"1min":0.39600000000000002,"5min":0.26200000000000001,"15min":0.254},"max":{"1min":0.71399999999999997,"5min":7.3049999999999997,"15min":28.106000000000002},"last":0.54000000000000004}]},{"osd":2,"last update":"Wed Sep 11 09:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":1.0660000000000001,"15min":1.3140000000000001},"min":{"1min":0.45000000000000001,"5min":0.28899999999999998,"15min":0.23999999999999999},"max":{"1min":1.4159999999999999,"5min":5.7690000000000001,"15min":67.486999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.0349999999999999,"15min":1.464},"min":{"1min":0.33400000000000002,"5min":0.27700000000000002,"15min":0.27400000000000002},"max":{"1min":4.7789999999999999,"5min":5.306,"15min":67.272000000000006},"last":0.49199999999999999}]},{"osd":3,"last update":"Wed Sep 11 09:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":1.1499999999999999,"15min":1.302},"min":{"1min":0.36099999999999999,"5min":0.28199999999999997,"15min":0.26700000000000002},"max":{"1min":1.8069999999999999,"5min":13.871,"15min":27.795000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":1.161,"15min":1.3440000000000001},"min":{"1min":0.46600000000000003,"5min":0.29099999999999998,"15min":0.28199999999999997},"max":{"1min":1.4570000000000001,"5min":12.318,"15min":27.841999999999999},"last":0.55500000000000005}]},{"osd":4,"last update":"Wed Sep 11 09:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.77600000000000002,"15min":1.1200000000000001},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.23300000000000001},"max":{"1min":0.83499999999999996,"5min":5.5940000000000003,"15min":40.927999999999997},"last":0.375},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.66800000000000004,"15min":1.0329999999999999},"min":{"1min":0.30499999999999999,"5min":0.26200000000000001,"15min":0.25700000000000001},"max":{"1min":0.80900000000000005,"5min":2.3860000000000001,"15min":40.706000000000003},"last":0.40699999999999997}]},{"osd":5,"last update":"Wed Sep 11 09:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.86399999999999999,"15min":1.274},"min":{"1min":0.34599999999999997,"5min":0.29599999999999999,"15min":0.23899999999999999},"max":{"1min":0.91400000000000003,"5min":5.75,"15min":40.847000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.83899999999999997,"15min":1.163},"min":{"1min":0.316,"5min":0.27500000000000002,"15min":0.27100000000000002},"max":{"1min":0.88100000000000001,"5min":3.746,"15min":41.401000000000003},"last":0.42999999999999999}]},{"osd":7,"last update":"Wed Sep 11 09:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.66400000000000003,"15min":1.093},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.74299999999999999,"5min":3.0419999999999998,"15min":40.426000000000002},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.68000000000000005,"15min":1.091},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.83499999999999996,"5min":3.0099999999999998,"15min":45.401000000000003},"last":0.38700000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757556,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":894968,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":893950,"kb_avail":92850184,"statfs":{"total":95995035648,"available":95078588416,"internally_reserved":0,"allocated":1032192,"data_stored":709864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":915405144},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":1.0449999999999999,"15min":1.113},"min":{"1min":0.34000000000000002,"5min":0.222,"15min":0.21099999999999999},"max":{"1min":0.97399999999999998,"5min":25.09,"15min":25.09},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":1.1379999999999999,"15min":1.0860000000000001},"min":{"1min":0.309,"5min":0.23899999999999999,"15min":0.20999999999999999},"max":{"1min":1.0409999999999999,"5min":23.628,"15min":23.628},"last":0.623}]},{"osd":1,"last update":"Wed Sep 11 09:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":1.601,"15min":1.274},"min":{"1min":0.41699999999999998,"5min":0.23999999999999999,"15min":0.216},"max":{"1min":0.98499999999999999,"5min":49.965000000000003,"15min":49.965000000000003},"last":0.44},{"interface":"front","average":{"1min":0.52200000000000002,"5min":1.601,"15min":1.2589999999999999},"min":{"1min":0.32100000000000001,"5min":0.24399999999999999,"15min":0.23400000000000001},"max":{"1min":0.94399999999999995,"5min":48.692,"15min":48.692},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed Sep 11 09:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":1.7410000000000001,"15min":1.296},"min":{"1min":0.34999999999999998,"5min":0.26100000000000001,"15min":0.253},"max":{"1min":0.93000000000000005,"5min":48.331000000000003,"15min":48.331000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":1.52,"15min":1.2749999999999999},"min":{"1min":0.33300000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.92700000000000005,"5min":50.441000000000003,"15min":50.441000000000003},"last":0.58799999999999997}]},{"osd":3,"last update":"Wed Sep 11 09:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":1.9350000000000001,"15min":1.6870000000000001},"min":{"1min":0.41399999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.94699999999999995,"5min":51.340000000000003,"15min":85.299999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":1.988,"15min":1.7150000000000001},"min":{"1min":0.374,"5min":0.29399999999999998,"15min":0.29099999999999998},"max":{"1min":0.999,"5min":51.889000000000003,"15min":85.941000000000003},"last":0.65100000000000002}]},{"osd":4,"last update":"Wed Sep 11 09:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.51100000000000001,"15min":0.84199999999999997},"min":{"1min":0.27300000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.94499999999999995,"5min":2.5009999999999999,"15min":16.021999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.61599999999999999,"15min":0.83799999999999997},"min":{"1min":0.28599999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.96299999999999997,"5min":3.0369999999999999,"15min":9.6140000000000008},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed Sep 11 09:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65500000000000003,"15min":0.92600000000000005},"min":{"1min":0.318,"5min":0.253,"15min":0.253},"max":{"1min":1.246,"5min":2.4489999999999998,"15min":8.8059999999999992},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.66900000000000004,"15min":0.94299999999999995},"min":{"1min":0.30399999999999999,"5min":0.27900000000000003,"15min":0.27300000000000002},"max":{"1min":0.753,"5min":3.9470000000000001,"15min":8.1920000000000002},"last":0.66900000000000004}]},{"osd":7,"last update":"Wed Sep 11 09:02:08 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.68000000000000005,"15min":1.25},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.98099999999999998,"5min":3.448,"15min":88.513000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.72799999999999998,"15min":1.256},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.29599999999999999},"max":{"1min":1.0609999999999999,"5min":3.306,"15min":86.963999999999999},"last":0.63300000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986488,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":889972,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":889214,"kb_avail":92855180,"statfs":{"total":95995035648,"available":95083704320,"internally_reserved":0,"allocated":765952,"data_stored":446005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":910555480},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":2.919,"15min":1.8979999999999999},"min":{"1min":0.42699999999999999,"5min":0.20999999999999999,"15min":0.20499999999999999},"max":{"1min":0.78400000000000003,"5min":206.922,"15min":206.922},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":2.9409999999999998,"15min":1.9850000000000001},"min":{"1min":0.35299999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.79200000000000004,"5min":206.636,"15min":206.636},"last":0.51000000000000001}]},{"osd":1,"last update":"Wed Sep 11 09:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":1.119,"15min":1.544},"min":{"1min":0.34200000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.65500000000000003,"5min":24.946000000000002,"15min":48.976999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":1.2729999999999999,"15min":1.5700000000000001},"min":{"1min":0.36699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.80200000000000005,"5min":25.035,"15min":45.787999999999997},"last":0.49199999999999999}]},{"osd":2,"last update":"Wed Sep 11 09:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":1.3169999999999999,"15min":2.1360000000000001},"min":{"1min":0.38600000000000001,"5min":0.248,"15min":0.21099999999999999},"max":{"1min":0.77800000000000002,"5min":44.311,"15min":255.88900000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":1.3640000000000001,"15min":2.214},"min":{"1min":0.40100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.85299999999999998,"5min":40.161999999999999,"15min":255.41200000000001},"last":0.60299999999999998}]},{"osd":3,"last update":"Wed Sep 11 09:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":1.3520000000000001,"15min":1.601},"min":{"1min":0.40200000000000002,"5min":0.26900000000000002,"15min":0.252},"max":{"1min":0.77000000000000002,"5min":21.652999999999999,"15min":45.268000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":1.181,"15min":1.504},"min":{"1min":0.35499999999999998,"5min":0.28599999999999998,"15min":0.25900000000000001},"max":{"1min":0.81499999999999995,"5min":21.556000000000001,"15min":44.234000000000002},"last":0.47499999999999998}]},{"osd":5,"last update":"Wed Sep 11 09:02:27 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.63900000000000001,"15min":0.873},"min":{"1min":0.316,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.73599999999999999,"5min":3.524,"15min":11.045999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.68200000000000005,"15min":0.86699999999999999},"min":{"1min":0.28999999999999998,"5min":0.26500000000000001,"15min":0.254},"max":{"1min":0.747,"5min":4.7119999999999997,"15min":10.526999999999999},"last":0.63300000000000001}]},{"osd":6,"last update":"Wed Sep 11 09:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.73499999999999999,"15min":1.2110000000000001},"min":{"1min":0.29999999999999999,"5min":0.27000000000000002,"15min":0.26800000000000002},"max":{"1min":0.61499999999999999,"5min":3.3519999999999999,"15min":72.100999999999999},"last":0.443},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.79800000000000004,"15min":1.272},"min":{"1min":0.33200000000000002,"5min":0.28000000000000003,"15min":0.22800000000000001},"max":{"1min":0.622,"5min":6.5359999999999996,"15min":74.527000000000001},"last":0.42699999999999999}]},{"osd":7,"last update":"Wed Sep 11 09:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.80700000000000005,"15min":0.87},"min":{"1min":0.31900000000000001,"5min":0.31,"15min":0.253},"max":{"1min":0.69899999999999995,"5min":5.2560000000000002,"15min":6.907},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.78600000000000003,"15min":0.98499999999999999},"min":{"1min":0.35499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73099999999999998,"5min":3.6560000000000001,"15min":6.8029999999999999},"last":0.55300000000000005}]}]},{"osd":3,"up_from":26,"seq":111669150014,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":904720,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":903358,"kb_avail":92840432,"statfs":{"total":95995035648,"available":95068602368,"internally_reserved":0,"allocated":1384448,"data_stored":1037954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":925038936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":1.038,"15min":1.022},"min":{"1min":0.20399999999999999,"5min":0.17499999999999999,"15min":0.16500000000000001},"max":{"1min":0.54200000000000004,"5min":15.601000000000001,"15min":24.280000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.34799999999999998,"5min":1.3100000000000001,"15min":1.1599999999999999},"min":{"1min":0.192,"5min":0.192,"15min":0.184},"max":{"1min":0.53000000000000003,"5min":16.492999999999999,"15min":22.225000000000001},"last":0.30199999999999999}]},{"osd":1,"last update":"Wed Sep 11 09:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":1.321,"15min":1.4630000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.20100000000000001},"max":{"1min":0.501,"5min":16.559000000000001,"15min":84.284999999999997},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":1.4239999999999999,"15min":1.4790000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20499999999999999},"max":{"1min":0.60199999999999998,"5min":13.686,"15min":85.832999999999998},"last":0.26200000000000001}]},{"osd":2,"last update":"Wed Sep 11 09:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":1.3120000000000001,"15min":1.45},"min":{"1min":0.23699999999999999,"5min":0.23100000000000001,"15min":0.22500000000000001},"max":{"1min":0.57199999999999995,"5min":13.627000000000001,"15min":86.766000000000005},"last":0.317},{"interface":"front","average":{"1min":0.36299999999999999,"5min":1.3939999999999999,"15min":1.45},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.22700000000000001},"max":{"1min":0.58499999999999996,"5min":15.621,"15min":85.367999999999995},"last":0.35599999999999998}]},{"osd":4,"last update":"Wed Sep 11 09:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":1.393,"15min":1.393},"min":{"1min":0.44,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.84899999999999998,"5min":13.663,"15min":13.663},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":1.2030000000000001,"15min":1.1459999999999999},"min":{"1min":0.36599999999999999,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":0.875,"5min":17.079999999999998,"15min":17.079999999999998},"last":0.57199999999999995}]},{"osd":5,"last update":"Wed Sep 11 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":1.321,"15min":1.288},"min":{"1min":0.41799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.74299999999999999,"5min":13.736000000000001,"15min":13.736000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":1.274,"15min":1.2010000000000001},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.28899999999999998},"max":{"1min":0.77200000000000002,"5min":17.048999999999999,"15min":17.048999999999999},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed Sep 11 09:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":1.266,"15min":1.446},"min":{"1min":0.42199999999999999,"5min":0.34599999999999997,"15min":0.29999999999999999},"max":{"1min":0.86799999999999999,"5min":13.755000000000001,"15min":33.106999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":1.371,"15min":1.365},"min":{"1min":0.41399999999999998,"5min":0.33200000000000002,"15min":0.29299999999999998},"max":{"1min":0.79400000000000004,"5min":16.527000000000001,"15min":33.076000000000001},"last":0.67000000000000004}]},{"osd":7,"last update":"Wed Sep 11 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":1.3580000000000001,"15min":1.5589999999999999},"min":{"1min":0.41499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94199999999999995,"5min":13.712999999999999,"15min":35.073},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":1.327,"15min":1.4379999999999999},"min":{"1min":0.41399999999999998,"5min":0.39300000000000002,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":17,"15min":34.844999999999999},"last":0.69399999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411650,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":892776,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":891774,"kb_avail":92852376,"statfs":{"total":95995035648,"available":95080833024,"internally_reserved":0,"allocated":1015808,"data_stored":708514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":913176920},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":2.3759999999999999,"15min":1.5589999999999999},"min":{"1min":0.25,"5min":0.192,"15min":0.189},"max":{"1min":0.47599999999999998,"5min":130.703,"15min":130.703},"last":0.439},{"interface":"front","average":{"1min":0.377,"5min":2.3730000000000002,"15min":1.6240000000000001},"min":{"1min":0.249,"5min":0.186,"15min":0.17399999999999999},"max":{"1min":0.52600000000000002,"5min":130.64699999999999,"15min":130.64699999999999},"last":0.34399999999999997}]},{"osd":1,"last update":"Wed Sep 11 09:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":1.6000000000000001,"15min":1.571},"min":{"1min":0.307,"5min":0.182,"15min":0.182},"max":{"1min":0.58399999999999996,"5min":51.231999999999999,"15min":72.867000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":1.8400000000000001,"15min":1.75},"min":{"1min":0.28899999999999998,"5min":0.20499999999999999,"15min":0.20399999999999999},"max":{"1min":0.53600000000000003,"5min":47.674999999999997,"15min":67.799000000000007},"last":0.40100000000000002}]},{"osd":3,"last update":"Wed Sep 11 09:02:39 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":2.3500000000000001,"15min":1.841},"min":{"1min":0.25900000000000001,"5min":0.219,"15min":0.186},"max":{"1min":0.54900000000000004,"5min":50.173000000000002,"15min":64.795000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":2.1890000000000001,"15min":1.7470000000000001},"min":{"1min":0.27900000000000003,"5min":0.23000000000000001,"15min":0.216},"max":{"1min":0.54900000000000004,"5min":49.935000000000002,"15min":64.632999999999996},"last":0.42799999999999999}]},{"osd":4,"last update":"Wed Sep 11 09:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":1.133,"15min":1.2549999999999999},"min":{"1min":0.41299999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.94299999999999995,"5min":7.3650000000000002,"15min":52.113999999999997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.98099999999999998,"15min":1.2150000000000001},"min":{"1min":0.40899999999999997,"5min":0.28899999999999998,"15min":0.27300000000000002},"max":{"1min":0.82099999999999995,"5min":7.0810000000000004,"15min":52.097999999999999},"last":0.49399999999999999}]},{"osd":5,"last update":"Wed Sep 11 09:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":1.0740000000000001,"15min":1.276},"min":{"1min":0.46600000000000003,"5min":0.311,"15min":0.29799999999999999},"max":{"1min":0.88700000000000001,"5min":7.1219999999999999,"15min":52.145000000000003},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.62,"5min":1.034,"15min":1.1519999999999999},"min":{"1min":0.41399999999999998,"5min":0.29399999999999998,"15min":0.28999999999999998},"max":{"1min":0.77500000000000002,"5min":6.5880000000000001,"15min":52.216000000000001},"last":0.44500000000000001}]},{"osd":6,"last update":"Wed Sep 11 09:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":3.5470000000000002,"15min":2.1070000000000002},"min":{"1min":0.45900000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.97299999999999998,"5min":246.482,"15min":246.482},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.625,"5min":3.5249999999999999,"15min":2.1299999999999999},"min":{"1min":0.48699999999999999,"5min":0.34899999999999998,"15min":0.33500000000000002},"max":{"1min":0.79000000000000004,"5min":246.184,"15min":246.184},"last":0.55900000000000005}]},{"osd":7,"last update":"Wed Sep 11 09:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":1.0369999999999999,"15min":1.264},"min":{"1min":0.40300000000000002,"5min":0.33800000000000002,"15min":0.32200000000000001},"max":{"1min":0.78000000000000003,"5min":6.6289999999999996,"15min":84.108999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.626,"5min":1.135,"15min":1.4199999999999999},"min":{"1min":0.47599999999999998,"5min":0.30599999999999999,"15min":0.30299999999999999},"max":{"1min":0.91100000000000003,"5min":7.415,"15min":85.938000000000002},"last":0.55100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575175,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":904464,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":903678,"kb_avail":92840688,"statfs":{"total":95995035648,"available":95068864512,"internally_reserved":0,"allocated":794624,"data_stored":446965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":925366616},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 09:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.89100000000000001,"15min":1.244},"min":{"1min":0.22,"5min":0.192,"15min":0.16600000000000001},"max":{"1min":0.41899999999999998,"5min":7.8220000000000001,"15min":56.613},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.99399999999999999,"15min":1.329},"min":{"1min":0.23300000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.501,"5min":7.0439999999999996,"15min":55.533999999999999},"last":0.24399999999999999}]},{"osd":2,"last update":"Wed Sep 11 09:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":1.3029999999999999,"15min":1.885},"min":{"1min":0.219,"5min":0.219,"15min":0.19},"max":{"1min":0.64600000000000002,"5min":12.596,"15min":169.607},"last":0.318},{"interface":"front","average":{"1min":0.34799999999999998,"5min":1.1659999999999999,"15min":1.7969999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20000000000000001},"max":{"1min":0.47499999999999998,"5min":10.243,"15min":169.762},"last":0.21199999999999999}]},{"osd":3,"last update":"Wed Sep 11 09:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":2.5150000000000001,"15min":1.577},"min":{"1min":0.29499999999999998,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.48199999999999998,"5min":70.478999999999999,"15min":70.478999999999999},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":2.9460000000000002,"15min":1.679},"min":{"1min":0.26300000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.76400000000000001,"5min":70.337999999999994,"15min":70.337999999999994},"last":0.28799999999999998}]},{"osd":4,"last update":"Wed Sep 11 09:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":1.163,"15min":1.109},"min":{"1min":0.35499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.115,"5min":21.786000000000001,"15min":21.786000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":1.2070000000000001,"15min":1.1220000000000001},"min":{"1min":0.40000000000000002,"5min":0.32300000000000001,"15min":0.23599999999999999},"max":{"1min":0.78600000000000003,"5min":21.815999999999999,"15min":21.815999999999999},"last":0.54200000000000004}]},{"osd":5,"last update":"Wed Sep 11 09:02:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":1.1910000000000001,"15min":1.2849999999999999},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.27600000000000002},"max":{"1min":1.1739999999999999,"5min":15.664999999999999,"15min":38.098999999999997},"last":0.496},{"interface":"front","average":{"1min":0.60699999999999998,"5min":1.1779999999999999,"15min":1.335},"min":{"1min":0.38900000000000001,"5min":0.307,"15min":0.307},"max":{"1min":1.2270000000000001,"5min":15.199,"15min":38.130000000000003},"last":0.63400000000000001}]},{"osd":6,"last update":"Wed Sep 11 09:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":1.0509999999999999,"15min":1.1559999999999999},"min":{"1min":0.376,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.1279999999999999,"5min":7.8840000000000003,"15min":7.8840000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.622,"5min":1.202,"15min":1.1479999999999999},"min":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.315},"max":{"1min":0.82999999999999996,"5min":6.1740000000000004,"15min":6.2869999999999999},"last":0.67900000000000005}]},{"osd":7,"last update":"Wed Sep 11 09:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":1.3180000000000001,"15min":1.125},"min":{"1min":0.41399999999999998,"5min":0.34799999999999998,"15min":0.31900000000000001},"max":{"1min":1.1459999999999999,"5min":14.994,"15min":14.994},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.624,"5min":1.421,"15min":1.2150000000000001},"min":{"1min":0.46700000000000003,"5min":0.35999999999999999,"15min":0.30199999999999999},"max":{"1min":1.093,"5min":22.215,"15min":22.215},"last":0.65900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738699,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":906172,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":905086,"kb_avail":92838980,"statfs":{"total":95995035648,"available":95067115520,"internally_reserved":0,"allocated":1097728,"data_stored":774282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":926808408},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 09:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":1.548,"15min":1.427},"min":{"1min":0.22900000000000001,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.46400000000000002,"5min":42.197000000000003,"15min":56.685000000000002},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":1.573,"15min":1.474},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.188},"max":{"1min":0.56000000000000005,"5min":41.822000000000003,"15min":61.012999999999998},"last":0.24199999999999999}]},{"osd":2,"last update":"Wed Sep 11 09:02:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":2.4100000000000001,"15min":1.466},"min":{"1min":0.25800000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":5.4020000000000001,"5min":111.41200000000001,"15min":111.41200000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":2.6120000000000001,"15min":1.556},"min":{"1min":0.247,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":4.79,"5min":111.637,"15min":111.637},"last":0.316}]},{"osd":3,"last update":"Wed Sep 11 09:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":1.823,"15min":1.5640000000000001},"min":{"1min":0.27200000000000002,"5min":0.20000000000000001,"15min":0.19800000000000001},"max":{"1min":0.58099999999999996,"5min":41.131999999999998,"15min":56.095999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":1.4770000000000001,"15min":1.502},"min":{"1min":0.26700000000000002,"5min":0.24199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52100000000000002,"5min":41.173999999999999,"15min":59.651000000000003},"last":0.30099999999999999}]},{"osd":4,"last update":"Wed Sep 11 09:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.68700000000000006,"15min":1.665},"min":{"1min":0.35499999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.84699999999999998,"5min":5.2119999999999997,"15min":168.72200000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.81399999999999995,"15min":1.752},"min":{"1min":0.40699999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.84399999999999997,"5min":5.8579999999999997,"15min":168.88999999999999},"last":0.42699999999999999}]},{"osd":5,"last update":"Wed Sep 11 09:02:26 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":1.0129999999999999,"15min":1.163},"min":{"1min":0.38600000000000001,"5min":0.311,"15min":0.30599999999999999},"max":{"1min":4.6639999999999997,"5min":7.2649999999999997,"15min":7.7830000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.79900000000000004,"15min":1.145},"min":{"1min":0.36099999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.85899999999999999,"5min":5.2359999999999998,"15min":9.5239999999999991},"last":0.501}]},{"osd":6,"last update":"Wed Sep 11 09:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":1.1899999999999999,"15min":1.304},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.30099999999999999},"max":{"1min":0.96499999999999997,"5min":16.196000000000002,"15min":25.027000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.90300000000000002,"15min":1.2789999999999999},"min":{"1min":0.47699999999999998,"5min":0.32200000000000001,"15min":0.29399999999999998},"max":{"1min":0.84499999999999997,"5min":16.585999999999999,"15min":25.076000000000001},"last":0.52300000000000002}]},{"osd":7,"last update":"Wed Sep 11 09:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":1.264,"15min":1.484},"min":{"1min":0.40100000000000002,"5min":0.33300000000000002,"15min":0.32800000000000001},"max":{"1min":5.8929999999999998,"5min":20.399000000000001,"15min":56.119},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":1.468,"15min":1.6830000000000001},"min":{"1min":0.443,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":4.7610000000000001,"5min":20.286999999999999,"15min":56.100000000000001},"last":0.44}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:03:06.910 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-11T09:03:06.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 211625d4-7018-11ef-bcea-c7b262605968 -- ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-11T09:03:07.151 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T09:03:07.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:07 smithi092 ceph-mon[29647]: 3.0 scrub starts 2024-09-11T09:03:07.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:07 smithi092 ceph-mon[29647]: 3.0 scrub ok 2024-09-11T09:03:07.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:07 smithi092 ceph-mon[29647]: 6.13 scrub starts 2024-09-11T09:03:07.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:07 smithi092 ceph-mon[29647]: 6.13 scrub ok 2024-09-11T09:03:07.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:07 smithi092 ceph-mon[29647]: from='client.26180 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T09:03:07.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:07 smithi092 ceph-mon[29647]: 6.12 scrub starts 2024-09-11T09:03:07.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:07 smithi092 ceph-mon[29647]: 6.12 scrub ok 2024-09-11T09:03:07.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:07 smithi092 ceph-mon[41114]: 3.0 scrub starts 2024-09-11T09:03:07.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:07 smithi092 ceph-mon[41114]: 3.0 scrub ok 2024-09-11T09:03:07.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:07 smithi092 ceph-mon[41114]: 6.13 scrub starts 2024-09-11T09:03:07.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:07 smithi092 ceph-mon[41114]: 6.13 scrub ok 2024-09-11T09:03:07.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:07 smithi092 ceph-mon[41114]: from='client.26180 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T09:03:07.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:07 smithi092 ceph-mon[41114]: 6.12 scrub starts 2024-09-11T09:03:07.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:07 smithi092 ceph-mon[41114]: 6.12 scrub ok 2024-09-11T09:03:07.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:03:07 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug there is no tcmu-runner data available 2024-09-11T09:03:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:07 smithi145 ceph-mon[33384]: 3.0 scrub starts 2024-09-11T09:03:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:07 smithi145 ceph-mon[33384]: 3.0 scrub ok 2024-09-11T09:03:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:07 smithi145 ceph-mon[33384]: 6.13 scrub starts 2024-09-11T09:03:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:07 smithi145 ceph-mon[33384]: 6.13 scrub ok 2024-09-11T09:03:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:07 smithi145 ceph-mon[33384]: from='client.26180 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T09:03:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:07 smithi145 ceph-mon[33384]: 6.12 scrub starts 2024-09-11T09:03:07.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:07 smithi145 ceph-mon[33384]: 6.12 scrub ok 2024-09-11T09:03:08.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[29647]: 4.12 scrub starts 2024-09-11T09:03:08.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[29647]: 4.12 scrub ok 2024-09-11T09:03:08.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[29647]: 3.1a scrub starts 2024-09-11T09:03:08.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[29647]: 3.1a scrub ok 2024-09-11T09:03:08.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[29647]: purged_snaps scrub starts 2024-09-11T09:03:08.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[29647]: purged_snaps scrub ok 2024-09-11T09:03:08.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[29647]: osdmap e465: 8 total, 8 up, 8 in 2024-09-11T09:03:08.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[29647]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:03:08.566 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[29647]: pgmap v1115: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:03:08.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[41114]: 4.12 scrub starts 2024-09-11T09:03:08.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[41114]: 4.12 scrub ok 2024-09-11T09:03:08.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[41114]: 3.1a scrub starts 2024-09-11T09:03:08.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[41114]: 3.1a scrub ok 2024-09-11T09:03:08.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[41114]: purged_snaps scrub starts 2024-09-11T09:03:08.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[41114]: purged_snaps scrub ok 2024-09-11T09:03:08.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[41114]: osdmap e465: 8 total, 8 up, 8 in 2024-09-11T09:03:08.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[41114]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:03:08.567 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:08 smithi092 ceph-mon[41114]: pgmap v1115: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:03:08.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:08 smithi145 ceph-mon[33384]: 4.12 scrub starts 2024-09-11T09:03:08.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:08 smithi145 ceph-mon[33384]: 4.12 scrub ok 2024-09-11T09:03:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:08 smithi145 ceph-mon[33384]: 3.1a scrub starts 2024-09-11T09:03:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:08 smithi145 ceph-mon[33384]: 3.1a scrub ok 2024-09-11T09:03:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:08 smithi145 ceph-mon[33384]: purged_snaps scrub starts 2024-09-11T09:03:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:08 smithi145 ceph-mon[33384]: purged_snaps scrub ok 2024-09-11T09:03:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:08 smithi145 ceph-mon[33384]: osdmap e465: 8 total, 8 up, 8 in 2024-09-11T09:03:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:08 smithi145 ceph-mon[33384]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T09:03:08.662 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:08 smithi145 ceph-mon[33384]: pgmap v1115: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:03:09.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:09 smithi092 ceph-mon[29647]: 4.1a scrub starts 2024-09-11T09:03:09.565 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:09 smithi092 ceph-mon[29647]: 4.1a scrub ok 2024-09-11T09:03:09.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:09 smithi092 ceph-mon[41114]: 4.1a scrub starts 2024-09-11T09:03:09.565 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:09 smithi092 ceph-mon[41114]: 4.1a scrub ok 2024-09-11T09:03:09.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:09 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:09:03:09] ENGINE Bus STOPPING 2024-09-11T09:03:09.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:09 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:09:03:09] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T09:03:09.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:09 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:09:03:09] ENGINE Bus STOPPED 2024-09-11T09:03:09.566 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:09 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:09:03:09] ENGINE Bus STARTING 2024-09-11T09:03:09.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:09 smithi145 ceph-mon[33384]: 4.1a scrub starts 2024-09-11T09:03:09.661 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:09 smithi145 ceph-mon[33384]: 4.1a scrub ok 2024-09-11T09:03:09.980 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:09 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:09:03:09] ENGINE Serving on http://:::9283 2024-09-11T09:03:09.980 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:09 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:09:03:09] ENGINE Bus STARTED 2024-09-11T09:03:10.004 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T09:03:10.015 INFO:tasks.cephadm:Teardown begin 2024-09-11T09:03:10.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T09:03:10.043 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T09:03:10.089 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T09:03:10.090 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T09:03:10.105 DEBUG:teuthology.orchestra.run.smithi145:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T09:03:10.123 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T09:03:10.124 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T09:03:10.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-211625d4-7018-11ef-bcea-c7b262605968@mon.a 2024-09-11T09:03:10.288 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:10 smithi092 systemd[1]: Stopping Ceph mon.a for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:03:10.411 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:10 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/838140976' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-11T09:03:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:10 smithi145 ceph-mon[33384]: from='client.? 172.21.15.92:0/838140976' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-11T09:03:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:10 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T09:03:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:10 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T09:03:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:10 smithi145 ceph-mon[33384]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T09:03:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:10 smithi145 ceph-mon[33384]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:03:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:10 smithi145 ceph-mon[33384]: pgmap v1116: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:03:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:10 smithi145 ceph-mon[33384]: 3.1b deep-scrub starts 2024-09-11T09:03:10.412 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:10 smithi145 ceph-mon[33384]: 3.1b deep-scrub ok 2024-09-11T09:03:10.561 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/838140976' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-11T09:03:10.561 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[41114]: from='client.? 172.21.15.92:0/838140976' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-11T09:03:10.561 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T09:03:10.561 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T09:03:10.561 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[41114]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T09:03:10.561 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[41114]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:03:10.562 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[41114]: pgmap v1116: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:03:10.562 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[41114]: 3.1b deep-scrub starts 2024-09-11T09:03:10.562 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[41114]: 3.1b deep-scrub ok 2024-09-11T09:03:10.562 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/838140976' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-11T09:03:10.562 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[29647]: from='client.? 172.21.15.92:0/838140976' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-11T09:03:10.562 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T09:03:10.562 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T09:03:10.562 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[29647]: from='mgr.14553 172.21.15.92:0/1956375572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T09:03:10.562 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[29647]: from='mgr.14553 ' entity='mgr.y' 2024-09-11T09:03:10.562 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[29647]: pgmap v1116: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T09:03:10.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[29647]: 3.1b deep-scrub starts 2024-09-11T09:03:10.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-mon[29647]: 3.1b deep-scrub ok 2024-09-11T09:03:10.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a[29624]: 2024-09-11T09:03:10.469+0000 7f9ae863f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T09:03:10.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:10 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a[29624]: 2024-09-11T09:03:10.469+0000 7f9ae863f640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T09:03:10.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:10 smithi092 podman[238906]: 2024-09-11 09:03:10.561922076 +0000 UTC m=+0.228287515 container died b8d1f6a92bc5fb3bbb38871f1a17f3f9e58c0d061442f5473a6cf4984684f20b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T09:03:11.316 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:11 smithi092 podman[238906]: 2024-09-11 09:03:11.109660185 +0000 UTC m=+0.776025620 container cleanup b8d1f6a92bc5fb3bbb38871f1a17f3f9e58c0d061442f5473a6cf4984684f20b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-11T09:03:11.317 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 11 09:03:11 smithi092 bash[238906]: ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-a 2024-09-11T09:03:11.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T09:03:11.627 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T09:03:11.627 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T09:03:11.627 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-11T09:03:11.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-211625d4-7018-11ef-bcea-c7b262605968@mon.c 2024-09-11T09:03:11.815 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:11 smithi092 systemd[1]: Stopping Ceph mon.c for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:03:11.815 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:03:11] "GET /metrics HTTP/1.1" 200 222501 "" "Prometheus/2.43.0" 2024-09-11T09:03:12.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-c[41091]: 2024-09-11T09:03:11.907+0000 7ff7eb7a8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T09:03:12.315 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-c[41091]: 2024-09-11T09:03:11.907+0000 7ff7eb7a8640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T09:03:12.316 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:12 smithi092 podman[239038]: 2024-09-11 09:03:12.133487181 +0000 UTC m=+0.325403249 container died 656e8e0df9f5ae5871efdde6209e2d63d3fa4fbde717060d12806f8da813c188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T09:03:12.316 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:09:03:11] ENGINE Bus STOPPING 2024-09-11T09:03:12.316 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:09:03:11] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T09:03:12.316 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:09:03:11] ENGINE Bus STOPPED 2024-09-11T09:03:12.316 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:11 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:09:03:11] ENGINE Bus STARTING 2024-09-11T09:03:12.316 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:12 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:09:03:12] ENGINE Serving on http://:::9283 2024-09-11T09:03:12.316 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:12 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: [11/Sep/2024:09:03:12] ENGINE Bus STARTED 2024-09-11T09:03:12.706 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:12 smithi092 podman[239038]: 2024-09-11 09:03:12.531288621 +0000 UTC m=+0.723204689 container cleanup 656e8e0df9f5ae5871efdde6209e2d63d3fa4fbde717060d12806f8da813c188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-c, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-09-11T09:03:12.707 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 11 09:03:12 smithi092 bash[239038]: ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-c 2024-09-11T09:03:12.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T09:03:12.967 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T09:03:12.967 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-11T09:03:12.967 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-11T09:03:12.967 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-211625d4-7018-11ef-bcea-c7b262605968@mon.b 2024-09-11T09:03:12.994 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:03:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x[35391]: ::ffff:172.21.15.145 - - [11/Sep/2024:09:03:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T09:03:13.296 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:13 smithi145 systemd[1]: Stopping Ceph mon.b for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:03:13.603 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-b[33349]: 2024-09-11T09:03:13.295+0000 7f7de1138640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T09:03:13.603 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-b[33349]: 2024-09-11T09:03:13.295+0000 7f7de1138640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T09:03:13.603 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:13 smithi145 podman[163731]: 2024-09-11 09:03:13.337528408 +0000 UTC m=+0.167294348 container died 01d29144d2b3858ee0df800dc612ee1deddb3c99cdfc863cd8bded066e77fe12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T09:03:13.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:13 smithi145 podman[163731]: 2024-09-11 09:03:13.794954223 +0000 UTC m=+0.624720162 container cleanup 01d29144d2b3858ee0df800dc612ee1deddb3c99cdfc863cd8bded066e77fe12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux ) 2024-09-11T09:03:13.911 INFO:journalctl@ceph.mon.b.smithi145.stdout:Sep 11 09:03:13 smithi145 bash[163731]: ceph-211625d4-7018-11ef-bcea-c7b262605968-mon-b 2024-09-11T09:03:14.234 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T09:03:14.323 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T09:03:14.323 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-11T09:03:14.323 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T09:03:14.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-211625d4-7018-11ef-bcea-c7b262605968@mgr.y 2024-09-11T09:03:14.604 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:14 smithi092 systemd[1]: Stopping Ceph mgr.y for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:03:14.936 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:14 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y[29898]: 2024-09-11T09:03:14.603+0000 7fe4c1a69640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T09:03:14.936 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:14 smithi092 podman[239179]: 2024-09-11 09:03:14.635225091 +0000 UTC m=+0.113465671 container died 417d2bf1fce87da9b4bf42d263b3aa91be1477b0ee0be6e0a26800c1a124978c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731) 2024-09-11T09:03:15.303 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:15 smithi092 podman[239179]: 2024-09-11 09:03:15.027888607 +0000 UTC m=+0.506129163 container cleanup 417d2bf1fce87da9b4bf42d263b3aa91be1477b0ee0be6e0a26800c1a124978c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-11T09:03:15.304 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Sep 11 09:03:15 smithi092 bash[239179]: ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-y 2024-09-11T09:03:15.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T09:03:15.482 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T09:03:15.483 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T09:03:15.483 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T09:03:15.483 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-211625d4-7018-11ef-bcea-c7b262605968@mgr.x 2024-09-11T09:03:15.789 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:03:15 smithi145 systemd[1]: Stopping Ceph mgr.x for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:03:16.059 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:03:15 smithi145 podman[163861]: 2024-09-11 09:03:15.790382826 +0000 UTC m=+0.122115697 container died d360d7b619c6ebd588320c3d0c91085180219a2cca6e2ef83a656a655ee51894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T09:03:16.411 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:03:16 smithi145 podman[163861]: 2024-09-11 09:03:16.216880414 +0000 UTC m=+0.548613343 container cleanup d360d7b619c6ebd588320c3d0c91085180219a2cca6e2ef83a656a655ee51894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T09:03:16.411 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:03:16 smithi145 bash[163861]: ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x 2024-09-11T09:03:16.680 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T09:03:16.702 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:03:16 smithi145 podman[163879]: 2024-09-11 09:03:16.452182578 +0000 UTC m=+0.656497207 container remove d360d7b619c6ebd588320c3d0c91085180219a2cca6e2ef83a656a655ee51894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T09:03:16.702 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:03:16 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-09-11T09:03:16.702 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:03:16 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@mgr.x.service: Failed with result 'exit-code'. 2024-09-11T09:03:16.702 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:03:16 smithi145 systemd[1]: Stopped Ceph mgr.x for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T09:03:16.702 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Sep 11 09:03:16 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@mgr.x.service: Consumed 11.518s CPU time. 2024-09-11T09:03:16.715 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T09:03:16.715 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T09:03:16.715 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T09:03:16.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.0 2024-09-11T09:03:17.033 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 09:03:16 smithi092 systemd[1]: Stopping Ceph osd.0 for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:03:17.315 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 09:03:17 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-0[45480]: 2024-09-11T09:03:17.033+0000 7ff516a8f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T09:03:17.315 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 09:03:17 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-0[45480]: 2024-09-11T09:03:17.033+0000 7ff516a8f640 -1 osd.0 465 *** Got signal Terminated *** 2024-09-11T09:03:17.315 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 09:03:17 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-0[45480]: 2024-09-11T09:03:17.033+0000 7ff516a8f640 -1 osd.0 465 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T09:03:22.790 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 09:03:22 smithi092 podman[239310]: 2024-09-11 09:03:22.502657268 +0000 UTC m=+5.596357624 container died bc43e195ea99025f7ec2ee533ef06038a0c6ee4cfa2a18e70039882c524101fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2) 2024-09-11T09:03:23.065 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 09:03:22 smithi092 podman[239310]: 2024-09-11 09:03:22.973570704 +0000 UTC m=+6.067271052 container cleanup bc43e195ea99025f7ec2ee533ef06038a0c6ee4cfa2a18e70039882c524101fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T09:03:23.065 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 09:03:22 smithi092 bash[239310]: ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-0 2024-09-11T09:03:23.739 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 09:03:23 smithi092 podman[239330]: 2024-09-11 09:03:23.392715202 +0000 UTC m=+0.893972641 container remove bc43e195ea99025f7ec2ee533ef06038a0c6ee4cfa2a18e70039882c524101fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-09-11T09:03:23.995 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 09:03:23 smithi092 podman[239424]: 2024-09-11 09:03:23.643294609 +0000 UTC m=+0.015550467 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T09:03:23.996 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 09:03:23 smithi092 podman[239424]: 2024-09-11 09:03:23.743526318 +0000 UTC m=+0.115782238 container create a7486f0adbce1375d546200a58642690bcabf1a793fa750a500229797efbdf88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-11T09:03:24.315 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 09:03:23 smithi092 podman[239424]: 2024-09-11 09:03:23.996044035 +0000 UTC m=+0.368299900 container init a7486f0adbce1375d546200a58642690bcabf1a793fa750a500229797efbdf88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T09:03:24.316 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 09:03:24 smithi092 podman[239424]: 2024-09-11 09:03:24.001672723 +0000 UTC m=+0.373928582 container start a7486f0adbce1375d546200a58642690bcabf1a793fa750a500229797efbdf88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731) 2024-09-11T09:03:24.316 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 09:03:24 smithi092 podman[239424]: 2024-09-11 09:03:24.129307968 +0000 UTC m=+0.501563839 container attach a7486f0adbce1375d546200a58642690bcabf1a793fa750a500229797efbdf88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-11T09:03:24.316 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 11 09:03:24 smithi092 podman[239424]: 2024-09-11 09:03:24.238165267 +0000 UTC m=+0.610421128 container died a7486f0adbce1375d546200a58642690bcabf1a793fa750a500229797efbdf88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T09:03:24.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T09:03:24.858 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T09:03:24.858 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T09:03:24.858 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T09:03:24.859 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.1 2024-09-11T09:03:25.065 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 09:03:24 smithi092 systemd[1]: Stopping Ceph osd.1 for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:03:25.565 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 09:03:25 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-1[51569]: 2024-09-11T09:03:25.181+0000 7f3db8b2c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T09:03:25.565 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 09:03:25 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-1[51569]: 2024-09-11T09:03:25.181+0000 7f3db8b2c640 -1 osd.1 465 *** Got signal Terminated *** 2024-09-11T09:03:25.565 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 09:03:25 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-1[51569]: 2024-09-11T09:03:25.181+0000 7f3db8b2c640 -1 osd.1 465 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T09:03:30.970 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 09:03:30 smithi092 podman[239551]: 2024-09-11 09:03:30.675766806 +0000 UTC m=+5.592249609 container died b2e8238e0b4786db01b7f593a1d158b3b77d24d3bfb1eae6187343acbfb31863 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-1, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T09:03:31.315 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 09:03:31 smithi092 podman[239551]: 2024-09-11 09:03:31.162224836 +0000 UTC m=+6.078707626 container cleanup b2e8238e0b4786db01b7f593a1d158b3b77d24d3bfb1eae6187343acbfb31863 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-11T09:03:31.315 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 09:03:31 smithi092 bash[239551]: ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-1 2024-09-11T09:03:31.709 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 09:03:31 smithi092 podman[239574]: 2024-09-11 09:03:31.397054307 +0000 UTC m=+0.718002876 container remove b2e8238e0b4786db01b7f593a1d158b3b77d24d3bfb1eae6187343acbfb31863 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T09:03:31.999 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 09:03:31 smithi092 podman[239670]: 2024-09-11 09:03:31.613393505 +0000 UTC m=+0.015524080 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T09:03:32.000 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 09:03:31 smithi092 podman[239670]: 2024-09-11 09:03:31.75648402 +0000 UTC m=+0.158614633 container create 611105f02ca97f10bef1c30bb3df29e061040a09085bc4b0c353fb9b218d1af8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-11T09:03:32.000 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 09:03:32 smithi092 podman[239670]: 2024-09-11 09:03:32.000692002 +0000 UTC m=+0.402822583 container init 611105f02ca97f10bef1c30bb3df29e061040a09085bc4b0c353fb9b218d1af8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-11T09:03:32.315 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 09:03:32 smithi092 podman[239670]: 2024-09-11 09:03:32.005637946 +0000 UTC m=+0.407768522 container start 611105f02ca97f10bef1c30bb3df29e061040a09085bc4b0c353fb9b218d1af8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T09:03:32.316 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 09:03:32 smithi092 podman[239670]: 2024-09-11 09:03:32.117409643 +0000 UTC m=+0.519540230 container attach 611105f02ca97f10bef1c30bb3df29e061040a09085bc4b0c353fb9b218d1af8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T09:03:32.316 INFO:journalctl@ceph.osd.1.smithi092.stdout:Sep 11 09:03:32 smithi092 podman[239670]: 2024-09-11 09:03:32.178604884 +0000 UTC m=+0.580735459 container died 611105f02ca97f10bef1c30bb3df29e061040a09085bc4b0c353fb9b218d1af8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T09:03:32.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T09:03:32.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T09:03:32.815 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T09:03:32.815 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T09:03:32.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.2 2024-09-11T09:03:33.065 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 09:03:32 smithi092 systemd[1]: Stopping Ceph osd.2 for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:03:33.565 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 09:03:33 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-2[57505]: 2024-09-11T09:03:33.184+0000 7f2b836c5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T09:03:33.565 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 09:03:33 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-2[57505]: 2024-09-11T09:03:33.184+0000 7f2b836c5640 -1 osd.2 465 *** Got signal Terminated *** 2024-09-11T09:03:33.565 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 09:03:33 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-2[57505]: 2024-09-11T09:03:33.184+0000 7f2b836c5640 -1 osd.2 465 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T09:03:38.949 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 09:03:38 smithi092 podman[239798]: 2024-09-11 09:03:38.645812018 +0000 UTC m=+5.600289877 container died b90a674f7255ee2b12727bc44037fcb4f2a0c61ccdb1bf93f170a9a6ad33de13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T09:03:39.315 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 09:03:39 smithi092 podman[239798]: 2024-09-11 09:03:39.131622165 +0000 UTC m=+6.086100009 container cleanup b90a674f7255ee2b12727bc44037fcb4f2a0c61ccdb1bf93f170a9a6ad33de13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0) 2024-09-11T09:03:39.315 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 09:03:39 smithi092 bash[239798]: ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-2 2024-09-11T09:03:39.730 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 09:03:39 smithi092 podman[239818]: 2024-09-11 09:03:39.425179892 +0000 UTC m=+0.774510887 container remove b90a674f7255ee2b12727bc44037fcb4f2a0c61ccdb1bf93f170a9a6ad33de13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T09:03:40.065 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 09:03:39 smithi092 podman[239913]: 2024-09-11 09:03:39.633867075 +0000 UTC m=+0.015864273 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T09:03:40.065 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 09:03:39 smithi092 podman[239913]: 2024-09-11 09:03:39.742687915 +0000 UTC m=+0.124685114 container create ad84a83a9181fe0bc639afb21afb1fcf0e2249029eabe6fed49dda1cde811d28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T09:03:40.065 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 09:03:39 smithi092 podman[239913]: 2024-09-11 09:03:39.953077257 +0000 UTC m=+0.335074456 container init ad84a83a9181fe0bc639afb21afb1fcf0e2249029eabe6fed49dda1cde811d28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T09:03:40.065 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 09:03:39 smithi092 podman[239913]: 2024-09-11 09:03:39.958338897 +0000 UTC m=+0.340336093 container start ad84a83a9181fe0bc639afb21afb1fcf0e2249029eabe6fed49dda1cde811d28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T09:03:40.423 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 09:03:40 smithi092 podman[239913]: 2024-09-11 09:03:40.067599107 +0000 UTC m=+0.449596337 container attach ad84a83a9181fe0bc639afb21afb1fcf0e2249029eabe6fed49dda1cde811d28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T09:03:40.423 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 09:03:40 smithi092 podman[239913]: 2024-09-11 09:03:40.13146687 +0000 UTC m=+0.513464067 container died ad84a83a9181fe0bc639afb21afb1fcf0e2249029eabe6fed49dda1cde811d28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T09:03:40.674 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 11 09:03:40 smithi092 podman[239956]: 2024-09-11 09:03:40.673249608 +0000 UTC m=+0.535067658 container remove ad84a83a9181fe0bc639afb21afb1fcf0e2249029eabe6fed49dda1cde811d28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T09:03:40.696 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T09:03:40.736 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T09:03:40.736 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T09:03:40.736 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T09:03:40.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.3 2024-09-11T09:03:41.061 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:40 smithi092 systemd[1]: Stopping Ceph osd.3 for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:03:41.315 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3[63650]: 2024-09-11T09:03:41.061+0000 7f087c507640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T09:03:41.316 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3[63650]: 2024-09-11T09:03:41.061+0000 7f087c507640 -1 osd.3 465 *** Got signal Terminated *** 2024-09-11T09:03:41.316 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:41 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3[63650]: 2024-09-11T09:03:41.061+0000 7f087c507640 -1 osd.3 465 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T09:03:44.065 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:43 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3[63650]: 2024-09-11T09:03:43.777+0000 7f0879b21640 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:16.933519+0000 front 2024-09-11T09:03:16.933530+0000 (oldest deadline 2024-09-11T09:03:42.833623+0000) 2024-09-11T09:03:44.082 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:43 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:43.673+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:44.911 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:44 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:44.646+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:45.065 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:44 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3[63650]: 2024-09-11T09:03:44.778+0000 7f0879b21640 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:16.933519+0000 front 2024-09-11T09:03:16.933530+0000 (oldest deadline 2024-09-11T09:03:42.833623+0000) 2024-09-11T09:03:45.911 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:45 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:45.635+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:46.065 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:45 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3[63650]: 2024-09-11T09:03:45.745+0000 7f0879b21640 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:16.933519+0000 front 2024-09-11T09:03:16.933530+0000 (oldest deadline 2024-09-11T09:03:42.833623+0000) 2024-09-11T09:03:46.876 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:46 smithi092 podman[240042]: 2024-09-11 09:03:46.608405118 +0000 UTC m=+5.641787700 container died 641efd4a3386901484b81377e6e91fa075f7fa3e75e49f85de8cb6b1e2e30b0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T09:03:46.911 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:46 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:46.592+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:47.259 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:47 smithi092 podman[240042]: 2024-09-11 09:03:47.042138633 +0000 UTC m=+6.075521280 container cleanup 641efd4a3386901484b81377e6e91fa075f7fa3e75e49f85de8cb6b1e2e30b0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T09:03:47.259 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:47 smithi092 bash[240042]: ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3 2024-09-11T09:03:47.565 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:47 smithi092 podman[240064]: 2024-09-11 09:03:47.260156464 +0000 UTC m=+0.645688709 container remove 641efd4a3386901484b81377e6e91fa075f7fa3e75e49f85de8cb6b1e2e30b0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T09:03:47.911 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:47 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:47.556+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:47.912 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:47 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:47.798+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.444063+0000 front 2024-09-11T09:03:21.443860+0000 (oldest deadline 2024-09-11T09:03:47.343892+0000) 2024-09-11T09:03:48.029 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:47 smithi092 podman[240162]: 2024-09-11 09:03:47.60006644 +0000 UTC m=+0.015581415 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T09:03:48.029 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:47 smithi092 podman[240162]: 2024-09-11 09:03:47.702720421 +0000 UTC m=+0.118235388 container create 1845c86f8175f65adfb1d54842af0f358116357bda884971ce545ac853a2b32a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T09:03:48.029 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:47 smithi092 podman[240162]: 2024-09-11 09:03:47.938787501 +0000 UTC m=+0.354302476 container init 1845c86f8175f65adfb1d54842af0f358116357bda884971ce545ac853a2b32a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-11T09:03:48.029 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:47 smithi092 podman[240162]: 2024-09-11 09:03:47.943921 +0000 UTC m=+0.359435979 container start 1845c86f8175f65adfb1d54842af0f358116357bda884971ce545ac853a2b32a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T09:03:48.029 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:48 smithi092 podman[240162]: 2024-09-11 09:03:48.029985078 +0000 UTC m=+0.445500054 container attach 1845c86f8175f65adfb1d54842af0f358116357bda884971ce545ac853a2b32a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T09:03:48.315 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:48 smithi092 podman[240162]: 2024-09-11 09:03:48.116193852 +0000 UTC m=+0.531708819 container died 1845c86f8175f65adfb1d54842af0f358116357bda884971ce545ac853a2b32a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T09:03:48.675 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 11 09:03:48 smithi092 podman[240205]: 2024-09-11 09:03:48.658466984 +0000 UTC m=+0.537047288 container remove 1845c86f8175f65adfb1d54842af0f358116357bda884971ce545ac853a2b32a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T09:03:48.680 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T09:03:48.718 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T09:03:48.719 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T09:03:48.719 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T09:03:48.719 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.4 2024-09-11T09:03:48.748 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:48 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:48.580+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:48.748 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:48 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4[38644]: 2024-09-11T09:03:48.483+0000 7f9935942640 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:22.017782+0000 front 2024-09-11T09:03:22.017845+0000 (oldest deadline 2024-09-11T09:03:47.917742+0000) 2024-09-11T09:03:48.748 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:48 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:48.495+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:03:49.044 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:48 smithi145 systemd[1]: Stopping Ceph osd.4 for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:03:49.044 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:48 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:48.798+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.444063+0000 front 2024-09-11T09:03:21.443860+0000 (oldest deadline 2024-09-11T09:03:47.343892+0000) 2024-09-11T09:03:49.411 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:49 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4[38644]: 2024-09-11T09:03:49.044+0000 7f993ab2d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T09:03:49.411 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:49 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4[38644]: 2024-09-11T09:03:49.044+0000 7f993ab2d640 -1 osd.4 465 *** Got signal Terminated *** 2024-09-11T09:03:49.411 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:49 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4[38644]: 2024-09-11T09:03:49.044+0000 7f993ab2d640 -1 osd.4 465 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T09:03:49.797 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:49 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4[38644]: 2024-09-11T09:03:49.457+0000 7f9935942640 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:22.017782+0000 front 2024-09-11T09:03:22.017845+0000 (oldest deadline 2024-09-11T09:03:47.917742+0000) 2024-09-11T09:03:49.797 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:49 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:49.551+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:49.798 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:49 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:49.543+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:03:50.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:49 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:49.797+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.444063+0000 front 2024-09-11T09:03:21.443860+0000 (oldest deadline 2024-09-11T09:03:47.343892+0000) 2024-09-11T09:03:50.814 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:50 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4[38644]: 2024-09-11T09:03:50.494+0000 7f9935942640 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:22.017782+0000 front 2024-09-11T09:03:22.017845+0000 (oldest deadline 2024-09-11T09:03:47.917742+0000) 2024-09-11T09:03:50.815 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:50 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:50.505+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:50.815 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:50 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:50.567+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:03:51.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:50 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:50.814+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.444063+0000 front 2024-09-11T09:03:21.443860+0000 (oldest deadline 2024-09-11T09:03:47.343892+0000) 2024-09-11T09:03:51.853 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4[38644]: 2024-09-11T09:03:51.505+0000 7f9935942640 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:22.017782+0000 front 2024-09-11T09:03:22.017845+0000 (oldest deadline 2024-09-11T09:03:47.917742+0000) 2024-09-11T09:03:51.853 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:51.555+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:03:51.854 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:51.534+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:51.854 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:51.534+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:03:52.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:51.853+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.444063+0000 front 2024-09-11T09:03:21.443860+0000 (oldest deadline 2024-09-11T09:03:47.343892+0000) 2024-09-11T09:03:52.162 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:51 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:51.853+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:27.344366+0000 front 2024-09-11T09:03:27.344425+0000 (oldest deadline 2024-09-11T09:03:51.444314+0000) 2024-09-11T09:03:52.860 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:52.579+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:03:52.861 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:52.517+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:52.861 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:52.517+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:03:52.861 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4[38644]: 2024-09-11T09:03:52.541+0000 7f9935942640 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:22.017782+0000 front 2024-09-11T09:03:22.017845+0000 (oldest deadline 2024-09-11T09:03:47.917742+0000) 2024-09-11T09:03:52.861 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4[38644]: 2024-09-11T09:03:52.541+0000 7f9935942640 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:27.918350+0000 front 2024-09-11T09:03:27.918360+0000 (oldest deadline 2024-09-11T09:03:52.018290+0000) 2024-09-11T09:03:53.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:52.860+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.444063+0000 front 2024-09-11T09:03:21.443860+0000 (oldest deadline 2024-09-11T09:03:47.343892+0000) 2024-09-11T09:03:53.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:52 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:52.860+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:27.344366+0000 front 2024-09-11T09:03:27.344425+0000 (oldest deadline 2024-09-11T09:03:51.444314+0000) 2024-09-11T09:03:53.838 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:53 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:53.626+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:03:53.838 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:53 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4[38644]: 2024-09-11T09:03:53.583+0000 7f9935942640 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:22.017782+0000 front 2024-09-11T09:03:22.017845+0000 (oldest deadline 2024-09-11T09:03:47.917742+0000) 2024-09-11T09:03:53.838 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:53 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4[38644]: 2024-09-11T09:03:53.583+0000 7f9935942640 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:27.918350+0000 front 2024-09-11T09:03:27.918360+0000 (oldest deadline 2024-09-11T09:03:52.018290+0000) 2024-09-11T09:03:53.838 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:53 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:53.498+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:53.839 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:53 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:53.498+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:03:54.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:53 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:53.838+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.444063+0000 front 2024-09-11T09:03:21.443860+0000 (oldest deadline 2024-09-11T09:03:47.343892+0000) 2024-09-11T09:03:54.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:53 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:53.838+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:27.344366+0000 front 2024-09-11T09:03:27.344425+0000 (oldest deadline 2024-09-11T09:03:51.444314+0000) 2024-09-11T09:03:54.885 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:54 smithi145 podman[164009]: 2024-09-11 09:03:54.596514604 +0000 UTC m=+5.672915834 container died 409023eeddb89d937ab9b4d01af8dba51e53428f14ad18b73ab2e4c0afe9ae26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T09:03:54.886 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:54 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:54.658+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:03:54.886 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:54 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:54.506+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:54.886 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:54 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:54.506+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:03:55.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:54 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:54.885+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.444063+0000 front 2024-09-11T09:03:21.443860+0000 (oldest deadline 2024-09-11T09:03:47.343892+0000) 2024-09-11T09:03:55.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:54 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:54.885+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:27.344366+0000 front 2024-09-11T09:03:27.344425+0000 (oldest deadline 2024-09-11T09:03:51.444314+0000) 2024-09-11T09:03:55.161 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:55 smithi145 podman[164009]: 2024-09-11 09:03:55.051320916 +0000 UTC m=+6.127722145 container cleanup 409023eeddb89d937ab9b4d01af8dba51e53428f14ad18b73ab2e4c0afe9ae26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T09:03:55.162 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:55 smithi145 bash[164009]: ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4 2024-09-11T09:03:55.640 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:55 smithi145 podman[164030]: 2024-09-11 09:03:55.386513591 +0000 UTC m=+0.781375601 container remove 409023eeddb89d937ab9b4d01af8dba51e53428f14ad18b73ab2e4c0afe9ae26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T09:03:55.640 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:55 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:55.547+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:55.640 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:55 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:55.547+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:03:55.911 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:55 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:55.889+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.444063+0000 front 2024-09-11T09:03:21.443860+0000 (oldest deadline 2024-09-11T09:03:47.343892+0000) 2024-09-11T09:03:55.911 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:55 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:55.889+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:27.344366+0000 front 2024-09-11T09:03:27.344425+0000 (oldest deadline 2024-09-11T09:03:51.444314+0000) 2024-09-11T09:03:55.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:55 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:55.640+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:03:55.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:55 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:55.640+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:03:55.912 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:55 smithi145 podman[164126]: 2024-09-11 09:03:55.598345239 +0000 UTC m=+0.015947397 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T09:03:55.912 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:55 smithi145 podman[164126]: 2024-09-11 09:03:55.712810823 +0000 UTC m=+0.130412965 container create 8d2195579bc74214bff78646b593bb2cab9d9f7d33f4c01e8501151f30a590ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T09:03:56.411 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:55 smithi145 podman[164126]: 2024-09-11 09:03:55.940371048 +0000 UTC m=+0.357973191 container init 8d2195579bc74214bff78646b593bb2cab9d9f7d33f4c01e8501151f30a590ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T09:03:56.412 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:55 smithi145 podman[164126]: 2024-09-11 09:03:55.945341116 +0000 UTC m=+0.362943260 container start 8d2195579bc74214bff78646b593bb2cab9d9f7d33f4c01e8501151f30a590ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T09:03:56.412 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:56 smithi145 podman[164126]: 2024-09-11 09:03:56.03186959 +0000 UTC m=+0.449471741 container attach 8d2195579bc74214bff78646b593bb2cab9d9f7d33f4c01e8501151f30a590ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-11T09:03:56.412 INFO:journalctl@ceph.osd.4.smithi145.stdout:Sep 11 09:03:56 smithi145 podman[164126]: 2024-09-11 09:03:56.171131978 +0000 UTC m=+0.588734124 container died 8d2195579bc74214bff78646b593bb2cab9d9f7d33f4c01e8501151f30a590ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T09:03:56.805 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:56.546+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:56.805 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:56.546+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:03:56.805 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:56.609+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:03:56.806 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:56.609+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:03:56.825 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T09:03:56.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T09:03:56.905 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T09:03:56.905 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T09:03:56.905 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.5 2024-09-11T09:03:57.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:56.918+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.444063+0000 front 2024-09-11T09:03:21.443860+0000 (oldest deadline 2024-09-11T09:03:47.343892+0000) 2024-09-11T09:03:57.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:56 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:56.918+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:27.344366+0000 front 2024-09-11T09:03:27.344425+0000 (oldest deadline 2024-09-11T09:03:51.444314+0000) 2024-09-11T09:03:57.162 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:56 smithi145 systemd[1]: Stopping Ceph osd.5 for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:03:57.588 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:57.256+0000 7f1da33cc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T09:03:57.589 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:57.256+0000 7f1da33cc640 -1 osd.5 465 *** Got signal Terminated *** 2024-09-11T09:03:57.589 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:57.256+0000 7f1da33cc640 -1 osd.5 465 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T09:03:57.911 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:57.588+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:57.912 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:57.589+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:03:57.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:57.627+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:03:57.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:57.627+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:03:58.411 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:57.918+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.444063+0000 front 2024-09-11T09:03:21.443860+0000 (oldest deadline 2024-09-11T09:03:47.343892+0000) 2024-09-11T09:03:58.411 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:57 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:57.918+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:27.344366+0000 front 2024-09-11T09:03:27.344425+0000 (oldest deadline 2024-09-11T09:03:51.444314+0000) 2024-09-11T09:03:58.912 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:58 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:58.553+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:58.912 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:58 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:58.553+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:03:58.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:58 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:58.615+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:03:58.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:58 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:58.615+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:03:59.411 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:58 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:58.914+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.444063+0000 front 2024-09-11T09:03:21.443860+0000 (oldest deadline 2024-09-11T09:03:47.343892+0000) 2024-09-11T09:03:59.411 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:58 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:58.914+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:27.344366+0000 front 2024-09-11T09:03:27.344425+0000 (oldest deadline 2024-09-11T09:03:51.444314+0000) 2024-09-11T09:03:59.876 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:59 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:59.588+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:03:59.877 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:03:59 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:03:59.588+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:03:59.877 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:59 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:59.605+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:03:59.877 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:03:59 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:03:59.605+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:00.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:59 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:59.874+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.444063+0000 front 2024-09-11T09:03:21.443860+0000 (oldest deadline 2024-09-11T09:03:47.343892+0000) 2024-09-11T09:04:00.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:59 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:59.874+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:27.344366+0000 front 2024-09-11T09:03:27.344425+0000 (oldest deadline 2024-09-11T09:03:51.444314+0000) 2024-09-11T09:04:00.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:03:59 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:03:59.874+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:33.745364+0000 front 2024-09-11T09:03:33.745420+0000 (oldest deadline 2024-09-11T09:03:59.045186+0000) 2024-09-11T09:04:00.866 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:00 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:00.572+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:04:00.866 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:00 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:00.572+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:04:00.866 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:00 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:00.572+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:36.791167+0000 front 2024-09-11T09:03:36.791045+0000 (oldest deadline 2024-09-11T09:04:00.291037+0000) 2024-09-11T09:04:00.866 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:00 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:00.633+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:00.866 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:00 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:00.633+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:00.867 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:00 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:00.633+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:01.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:00 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:04:00.866+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.444063+0000 front 2024-09-11T09:03:21.443860+0000 (oldest deadline 2024-09-11T09:03:47.343892+0000) 2024-09-11T09:04:01.162 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:00 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:04:00.866+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:27.344366+0000 front 2024-09-11T09:03:27.344425+0000 (oldest deadline 2024-09-11T09:03:51.444314+0000) 2024-09-11T09:04:01.162 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:00 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:04:00.866+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:33.745364+0000 front 2024-09-11T09:03:33.745420+0000 (oldest deadline 2024-09-11T09:03:59.045186+0000) 2024-09-11T09:04:01.833 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:01 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:01.548+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:04:01.833 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:01 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:01.548+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:04:01.833 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:01 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:01.548+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:36.791167+0000 front 2024-09-11T09:03:36.791045+0000 (oldest deadline 2024-09-11T09:04:00.291037+0000) 2024-09-11T09:04:01.833 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:01 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:01.608+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:01.833 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:01 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:01.608+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:01.833 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:01 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:01.608+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:02.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:01 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:04:01.833+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.444063+0000 front 2024-09-11T09:03:21.443860+0000 (oldest deadline 2024-09-11T09:03:47.343892+0000) 2024-09-11T09:04:02.161 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:01 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:04:01.833+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:27.344366+0000 front 2024-09-11T09:03:27.344425+0000 (oldest deadline 2024-09-11T09:03:51.444314+0000) 2024-09-11T09:04:02.162 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:01 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5[44735]: 2024-09-11T09:04:01.833+0000 7f1da09e6640 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:33.745364+0000 front 2024-09-11T09:03:33.745420+0000 (oldest deadline 2024-09-11T09:03:59.045186+0000) 2024-09-11T09:04:02.789 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:02.513+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:04:02.789 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:02.513+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:04:02.789 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:02.513+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:36.791167+0000 front 2024-09-11T09:03:36.791045+0000 (oldest deadline 2024-09-11T09:04:00.291037+0000) 2024-09-11T09:04:02.789 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:02.653+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:02.789 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:02.653+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:02.789 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:02 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:02.653+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:03.089 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:02 smithi145 podman[164254]: 2024-09-11 09:04:02.78977194 +0000 UTC m=+5.654973964 container died 1ae82425167c23d38454df105411b737667154507e60f1ffe7536c79d43b837a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T09:04:03.411 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:03 smithi145 podman[164254]: 2024-09-11 09:04:03.180304688 +0000 UTC m=+6.045506701 container cleanup 1ae82425167c23d38454df105411b737667154507e60f1ffe7536c79d43b837a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T09:04:03.411 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:03 smithi145 bash[164254]: ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5 2024-09-11T09:04:03.749 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:03 smithi145 podman[164276]: 2024-09-11 09:04:03.443619145 +0000 UTC m=+0.647425201 container remove 1ae82425167c23d38454df105411b737667154507e60f1ffe7536c79d43b837a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-11T09:04:03.749 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:03.667+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:03.749 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:03.667+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:03.749 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:03.667+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:03.750 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:03.507+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:04:03.750 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:03.507+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:04:03.750 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:03 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:03.507+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:36.791167+0000 front 2024-09-11T09:03:36.791045+0000 (oldest deadline 2024-09-11T09:04:00.291037+0000) 2024-09-11T09:04:04.084 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:03 smithi145 podman[164366]: 2024-09-11 09:04:03.652918925 +0000 UTC m=+0.015870034 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T09:04:04.084 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:03 smithi145 podman[164366]: 2024-09-11 09:04:03.776814755 +0000 UTC m=+0.139765854 container create b1b9f96dee92ceaf3dfd0d39ebdbce78745309b8273c910696560c3c70ae0f8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-11T09:04:04.084 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:03 smithi145 podman[164366]: 2024-09-11 09:04:03.993682538 +0000 UTC m=+0.356633638 container init b1b9f96dee92ceaf3dfd0d39ebdbce78745309b8273c910696560c3c70ae0f8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-11T09:04:04.084 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:03 smithi145 podman[164366]: 2024-09-11 09:04:03.999091057 +0000 UTC m=+0.362042159 container start b1b9f96dee92ceaf3dfd0d39ebdbce78745309b8273c910696560c3c70ae0f8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T09:04:04.084 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:04 smithi145 podman[164366]: 2024-09-11 09:04:04.085034935 +0000 UTC m=+0.447986053 container attach b1b9f96dee92ceaf3dfd0d39ebdbce78745309b8273c910696560c3c70ae0f8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T09:04:04.411 INFO:journalctl@ceph.osd.5.smithi145.stdout:Sep 11 09:04:04 smithi145 podman[164366]: 2024-09-11 09:04:04.173207077 +0000 UTC m=+0.536158190 container died b1b9f96dee92ceaf3dfd0d39ebdbce78745309b8273c910696560c3c70ae0f8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T09:04:04.746 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:04.619+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:04.746 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:04.619+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:04.746 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:04.619+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:04.746 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:04.461+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:04:04.746 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:04.461+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:04:04.746 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:04 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:04.461+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:36.791167+0000 front 2024-09-11T09:03:36.791045+0000 (oldest deadline 2024-09-11T09:04:00.291037+0000) 2024-09-11T09:04:04.768 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T09:04:04.808 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T09:04:04.808 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T09:04:04.808 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T09:04:04.808 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.6 2024-09-11T09:04:05.126 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:04 smithi145 systemd[1]: Stopping Ceph osd.6 for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:04:05.411 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:05 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:05.126+0000 7fefc8140640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T09:04:05.411 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:05 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:05.126+0000 7fefc8140640 -1 osd.6 465 *** Got signal Terminated *** 2024-09-11T09:04:05.411 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:05 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:05.126+0000 7fefc8140640 -1 osd.6 465 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T09:04:05.911 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:05 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:05.486+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:04:05.911 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:05 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:05.486+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:04:05.911 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:05 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:05.486+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:36.791167+0000 front 2024-09-11T09:03:36.791045+0000 (oldest deadline 2024-09-11T09:04:00.291037+0000) 2024-09-11T09:04:05.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:05 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:05.620+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:05.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:05 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:05.620+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:05.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:05 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:05.620+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:06.911 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:06.467+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:04:06.912 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:06.467+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:04:06.912 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:06.467+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:36.791167+0000 front 2024-09-11T09:03:36.791045+0000 (oldest deadline 2024-09-11T09:04:00.291037+0000) 2024-09-11T09:04:06.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:06.592+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:06.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:06.592+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:06.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:06 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:06.592+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:07.911 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:07 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:07.470+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:04:07.912 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:07 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:07.470+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:04:07.912 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:07 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:07.470+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:36.791167+0000 front 2024-09-11T09:03:36.791045+0000 (oldest deadline 2024-09-11T09:04:00.291037+0000) 2024-09-11T09:04:07.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:07 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:07.545+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:07.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:07 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:07.545+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:07.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:07 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:07.545+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:07.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:07 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:07.545+0000 7fbc92183640 -1 osd.7 465 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24413.0:1479 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e465) 2024-09-11T09:04:08.911 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:08 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:08.513+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:04:08.912 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:08 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:08.513+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:04:08.912 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:08 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:08.513+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:36.791167+0000 front 2024-09-11T09:03:36.791045+0000 (oldest deadline 2024-09-11T09:04:00.291037+0000) 2024-09-11T09:04:08.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:08 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:08.549+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:08.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:08 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:08.549+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:08.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:08 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:08.549+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:08.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:08 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:08.549+0000 7fbc92183640 -1 osd.7 465 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24413.0:1479 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e465) 2024-09-11T09:04:09.911 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:09.465+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:18.888087+0000 front 2024-09-11T09:03:18.888111+0000 (oldest deadline 2024-09-11T09:03:43.587875+0000) 2024-09-11T09:04:09.912 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:09.465+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:28.089677+0000 front 2024-09-11T09:03:28.089778+0000 (oldest deadline 2024-09-11T09:03:50.989568+0000) 2024-09-11T09:04:09.912 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6[50639]: 2024-09-11T09:04:09.465+0000 7fefc575a640 -1 osd.6 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:36.791167+0000 front 2024-09-11T09:03:36.791045+0000 (oldest deadline 2024-09-11T09:04:00.291037+0000) 2024-09-11T09:04:09.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:09.599+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:09.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:09.599+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:09.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:09.599+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:09.913 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:09 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:09.599+0000 7fbc92183640 -1 osd.7 465 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24413.0:1479 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e465) 2024-09-11T09:04:10.911 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:10 smithi145 podman[164496]: 2024-09-11 09:04:10.698235806 +0000 UTC m=+5.671398049 container died 38c0513808099365e205f6b38bead70400a0321bc117e1f885d846649cd3df41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T09:04:10.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:10 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:10.573+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:10.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:10 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:10.573+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:10.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:10 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:10.573+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:10.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:10 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:10.573+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6830 osd.3 since back 2024-09-11T09:03:45.888636+0000 front 2024-09-11T09:03:45.888603+0000 (oldest deadline 2024-09-11T09:04:09.988513+0000) 2024-09-11T09:04:10.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:10 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:10.573+0000 7fbc92183640 -1 osd.7 465 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24413.0:1479 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e465) 2024-09-11T09:04:11.324 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:11 smithi145 podman[164496]: 2024-09-11 09:04:11.065953313 +0000 UTC m=+6.039115545 container cleanup 38c0513808099365e205f6b38bead70400a0321bc117e1f885d846649cd3df41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T09:04:11.324 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:11 smithi145 bash[164496]: ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6 2024-09-11T09:04:11.599 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:11 smithi145 podman[164520]: 2024-09-11 09:04:11.325453346 +0000 UTC m=+0.618969099 container remove 38c0513808099365e205f6b38bead70400a0321bc117e1f885d846649cd3df41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-11T09:04:11.911 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:11 smithi145 podman[164612]: 2024-09-11 09:04:11.571629496 +0000 UTC m=+0.015867311 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T09:04:11.911 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:11 smithi145 podman[164612]: 2024-09-11 09:04:11.693492981 +0000 UTC m=+0.137730796 container create 96cccab029e178c0b81a4f42bf043f61078c30c3b7604a672f3759ad4d62f84c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-11T09:04:11.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:11.600+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:11.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:11.600+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:11.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:11.600+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:11.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:11.600+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6830 osd.3 since back 2024-09-11T09:03:45.888636+0000 front 2024-09-11T09:03:45.888603+0000 (oldest deadline 2024-09-11T09:04:09.988513+0000) 2024-09-11T09:04:11.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:11 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:11.600+0000 7fbc92183640 -1 osd.7 465 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24413.0:1479 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e465) 2024-09-11T09:04:12.398 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:11 smithi145 podman[164612]: 2024-09-11 09:04:11.920953206 +0000 UTC m=+0.365191023 container init 96cccab029e178c0b81a4f42bf043f61078c30c3b7604a672f3759ad4d62f84c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731) 2024-09-11T09:04:12.398 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:11 smithi145 podman[164612]: 2024-09-11 09:04:11.925711096 +0000 UTC m=+0.369948913 container start 96cccab029e178c0b81a4f42bf043f61078c30c3b7604a672f3759ad4d62f84c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T09:04:12.398 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:12 smithi145 podman[164612]: 2024-09-11 09:04:12.012640933 +0000 UTC m=+0.456878751 container attach 96cccab029e178c0b81a4f42bf043f61078c30c3b7604a672f3759ad4d62f84c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-11T09:04:12.398 INFO:journalctl@ceph.osd.6.smithi145.stdout:Sep 11 09:04:12 smithi145 podman[164612]: 2024-09-11 09:04:12.108023838 +0000 UTC m=+0.552261657 container died 96cccab029e178c0b81a4f42bf043f61078c30c3b7604a672f3759ad4d62f84c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T09:04:12.661 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:12.563+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:12.661 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:12.563+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:12.662 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:12.563+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:12.662 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:12.563+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6830 osd.3 since back 2024-09-11T09:03:45.888636+0000 front 2024-09-11T09:03:45.888603+0000 (oldest deadline 2024-09-11T09:04:09.988513+0000) 2024-09-11T09:04:12.662 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:12 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:12.563+0000 7fbc92183640 -1 osd.7 465 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24413.0:1479 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e465) 2024-09-11T09:04:12.725 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T09:04:12.766 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T09:04:12.766 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T09:04:12.767 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T09:04:12.767 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.7 2024-09-11T09:04:13.087 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:12 smithi145 systemd[1]: Stopping Ceph osd.7 for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:04:13.411 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:13.086+0000 7fbc98370640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T09:04:13.457 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:13.086+0000 7fbc98370640 -1 osd.7 465 *** Got signal Terminated *** 2024-09-11T09:04:13.458 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:13.086+0000 7fbc98370640 -1 osd.7 465 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T09:04:13.911 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:13.593+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:13.911 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:13.593+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:13.911 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:13.593+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:13.911 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:13 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:13.593+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6830 osd.3 since back 2024-09-11T09:03:45.888636+0000 front 2024-09-11T09:03:45.888603+0000 (oldest deadline 2024-09-11T09:04:09.988513+0000) 2024-09-11T09:04:14.911 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:14 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:14.631+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:14.911 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:14 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:14.631+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:14.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:14 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:14.631+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:14.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:14 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:14.631+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6830 osd.3 since back 2024-09-11T09:03:45.888636+0000 front 2024-09-11T09:03:45.888603+0000 (oldest deadline 2024-09-11T09:04:09.988513+0000) 2024-09-11T09:04:15.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:15 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:15.659+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:15.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:15 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:15.659+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:15.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:15 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:15.659+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:15.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:15 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:15.659+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6830 osd.3 since back 2024-09-11T09:03:45.888636+0000 front 2024-09-11T09:03:45.888603+0000 (oldest deadline 2024-09-11T09:04:09.988513+0000) 2024-09-11T09:04:16.911 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:16.659+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:16.911 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:16.659+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:16.911 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:16.659+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:16.911 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:16 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:16.659+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6830 osd.3 since back 2024-09-11T09:03:45.888636+0000 front 2024-09-11T09:03:45.888603+0000 (oldest deadline 2024-09-11T09:04:09.988513+0000) 2024-09-11T09:04:18.161 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:17 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:17.672+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-11T09:03:21.885557+0000 front 2024-09-11T09:03:21.885737+0000 (oldest deadline 2024-09-11T09:03:47.785582+0000) 2024-09-11T09:04:18.162 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:17 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:17.672+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6814 osd.1 since back 2024-09-11T09:03:29.486643+0000 front 2024-09-11T09:03:29.486607+0000 (oldest deadline 2024-09-11T09:03:55.287029+0000) 2024-09-11T09:04:18.162 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:17 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:17.672+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6822 osd.2 since back 2024-09-11T09:03:35.287514+0000 front 2024-09-11T09:03:35.287480+0000 (oldest deadline 2024-09-11T09:03:59.987470+0000) 2024-09-11T09:04:18.162 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:17 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:17.672+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.92:6830 osd.3 since back 2024-09-11T09:03:45.888636+0000 front 2024-09-11T09:03:45.888603+0000 (oldest deadline 2024-09-11T09:04:09.988513+0000) 2024-09-11T09:04:18.162 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:17 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7[57034]: 2024-09-11T09:04:17.672+0000 7fbc92183640 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.145:6804 osd.4 since back 2024-09-11T09:03:52.289430+0000 front 2024-09-11T09:03:52.289442+0000 (oldest deadline 2024-09-11T09:04:17.589441+0000) 2024-09-11T09:04:18.911 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:18 smithi145 podman[164739]: 2024-09-11 09:04:18.661997678 +0000 UTC m=+5.678594334 container died 43552565b1caa145ef8632917c0916ab9440574364867ccdb59cb672f4f7b52f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T09:04:19.411 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:19 smithi145 podman[164739]: 2024-09-11 09:04:19.026975034 +0000 UTC m=+6.043571679 container cleanup 43552565b1caa145ef8632917c0916ab9440574364867ccdb59cb672f4f7b52f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T09:04:19.411 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:19 smithi145 bash[164739]: ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7 2024-09-11T09:04:19.411 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:19 smithi145 podman[164774]: 2024-09-11 09:04:19.194882799 +0000 UTC m=+0.527744345 container remove 43552565b1caa145ef8632917c0916ab9440574364867ccdb59cb672f4f7b52f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T09:04:19.911 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:19 smithi145 podman[164868]: 2024-09-11 09:04:19.628948721 +0000 UTC m=+0.109309311 container create 201df3f8ec3d7eaababab1a7dd7412efd20c06e9c9215285a770649fa0693dc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True) 2024-09-11T09:04:19.911 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:19 smithi145 podman[164868]: 2024-09-11 09:04:19.535935434 +0000 UTC m=+0.016295981 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T09:04:19.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:19 smithi145 podman[164868]: 2024-09-11 09:04:19.847800731 +0000 UTC m=+0.328161275 container init 201df3f8ec3d7eaababab1a7dd7412efd20c06e9c9215285a770649fa0693dc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T09:04:19.912 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:19 smithi145 podman[164868]: 2024-09-11 09:04:19.852643831 +0000 UTC m=+0.333004366 container start 201df3f8ec3d7eaababab1a7dd7412efd20c06e9c9215285a770649fa0693dc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T09:04:20.396 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:19 smithi145 podman[164868]: 2024-09-11 09:04:19.947674501 +0000 UTC m=+0.428035033 container attach 201df3f8ec3d7eaababab1a7dd7412efd20c06e9c9215285a770649fa0693dc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T09:04:20.396 INFO:journalctl@ceph.osd.7.smithi145.stdout:Sep 11 09:04:20 smithi145 podman[164868]: 2024-09-11 09:04:20.039777918 +0000 UTC m=+0.520138449 container died 201df3f8ec3d7eaababab1a7dd7412efd20c06e9c9215285a770649fa0693dc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T09:04:20.782 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T09:04:20.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T09:04:20.821 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T09:04:20.821 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-11T09:04:20.821 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-211625d4-7018-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-11T09:04:21.132 INFO:journalctl@ceph.rgw.foo.a.smithi092.stdout:Sep 11 09:04:20 smithi092 systemd[1]: Stopping Ceph rgw.foo.a for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:04:21.565 INFO:journalctl@ceph.rgw.foo.a.smithi092.stdout:Sep 11 09:04:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-rgw-foo-a[68815]: 2024-09-11T09:04:21.131+0000 7f7192d08640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T09:04:21.565 INFO:journalctl@ceph.rgw.foo.a.smithi092.stdout:Sep 11 09:04:21 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-rgw-foo-a[68815]: 2024-09-11T09:04:21.131+0000 7f71972107c0 -1 shutting down 2024-09-11T09:04:31.428 INFO:journalctl@ceph.rgw.foo.a.smithi092.stdout:Sep 11 09:04:31 smithi092 bash[240290]: time="2024-09-11T09:04:31Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-211625d4-7018-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-11T09:04:31.428 INFO:journalctl@ceph.rgw.foo.a.smithi092.stdout:Sep 11 09:04:31 smithi092 podman[240290]: 2024-09-11 09:04:31.149262158 +0000 UTC m=+10.129289373 container died 83137f9367192a5a2175d1c1308cd9ea8d736611f6ca985e43454072112eb74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T09:04:31.815 INFO:journalctl@ceph.rgw.foo.a.smithi092.stdout:Sep 11 09:04:31 smithi092 podman[240290]: 2024-09-11 09:04:31.594567376 +0000 UTC m=+10.574594632 container cleanup 83137f9367192a5a2175d1c1308cd9ea8d736611f6ca985e43454072112eb74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-rgw-foo-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0) 2024-09-11T09:04:31.816 INFO:journalctl@ceph.rgw.foo.a.smithi092.stdout:Sep 11 09:04:31 smithi092 bash[240290]: ceph-211625d4-7018-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-11T09:04:31.816 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 09:04:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[79001]: ts=2024-09-11T09:04:31.624Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi092.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.92:8443: connect: connection refused" 2024-09-11T09:04:31.816 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 09:04:31 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[79001]: ts=2024-09-11T09:04:31.625Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi145.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.145:8443: connect: connection refused" 2024-09-11T09:04:32.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-11T09:04:32.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T09:04:32.049 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-11T09:04:32.049 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-11T09:04:32.049 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-211625d4-7018-11ef-bcea-c7b262605968@prometheus.a 2024-09-11T09:04:32.362 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 systemd[1]: Stopping Ceph prometheus.a for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:04:32.636 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T09:04:32.363Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T09:04:32.636 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T09:04:32.363Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T09:04:32.636 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T09:04:32.363Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T09:04:32.636 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T09:04:32.363Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T09:04:32.637 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T09:04:32.363Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T09:04:32.637 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T09:04:32.363Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T09:04:32.637 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T09:04:32.363Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T09:04:32.637 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T09:04:32.363Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T09:04:32.637 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T09:04:32.371Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T09:04:32.637 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T09:04:32.375Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T09:04:32.637 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T09:04:32.375Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T09:04:32.637 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a[67857]: ts=2024-09-11T09:04:32.375Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T09:04:32.637 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 podman[164995]: 2024-09-11 09:04:32.384198313 +0000 UTC m=+0.138161204 container died 5220b1a6b4555900dfa461815783f66aa11382e4016ec968d4d8fffd4ba5126d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T09:04:32.911 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 podman[164995]: 2024-09-11 09:04:32.769667771 +0000 UTC m=+0.523630723 container cleanup 5220b1a6b4555900dfa461815783f66aa11382e4016ec968d4d8fffd4ba5126d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T09:04:32.911 INFO:journalctl@ceph.prometheus.a.smithi145.stdout:Sep 11 09:04:32 smithi145 bash[164995]: ceph-211625d4-7018-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T09:04:33.159 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-211625d4-7018-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-11T09:04:33.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T09:04:33.194 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-11T09:04:33.194 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 211625d4-7018-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T09:04:35.168 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 09:04:34 smithi092 systemd[1]: Stopping Ceph alertmanager.a for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:04:35.168 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 09:04:35 smithi092 ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a[79001]: ts=2024-09-11T09:04:35.127Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T09:04:35.469 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 09:04:35 smithi092 podman[240748]: 2024-09-11 09:04:35.168547615 +0000 UTC m=+0.149848635 container died 1fa23353947c5624b6186edf67da740fecf1ed25cc5f090fe45ee547b0f37af2 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T09:04:35.815 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 09:04:35 smithi092 podman[240748]: 2024-09-11 09:04:35.658753719 +0000 UTC m=+0.640054733 container cleanup 1fa23353947c5624b6186edf67da740fecf1ed25cc5f090fe45ee547b0f37af2 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T09:04:35.815 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 09:04:35 smithi092 bash[240748]: ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T09:04:36.151 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 09:04:35 smithi092 podman[240767]: 2024-09-11 09:04:35.901604971 +0000 UTC m=+0.725135013 container remove 1fa23353947c5624b6186edf67da740fecf1ed25cc5f090fe45ee547b0f37af2 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T09:04:36.151 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 09:04:35 smithi092 podman[240767]: 2024-09-11 09:04:35.977327042 +0000 UTC m=+0.800857082 volume remove b4d34c6f250d71a87dc252cf9c275a0865491d75f9ecdbba25271dede582b418 2024-09-11T09:04:36.152 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 09:04:36 smithi092 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T09:04:36.152 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 09:04:36 smithi092 systemd[1]: Stopped Ceph alertmanager.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T09:04:36.152 INFO:journalctl@ceph.alertmanager.a.smithi092.stdout:Sep 11 09:04:36 smithi092 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.843s CPU time. 2024-09-11T09:04:36.565 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 09:04:36 smithi092 systemd[1]: Stopping Ceph node-exporter.a for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:04:36.856 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 09:04:36 smithi092 podman[240882]: 2024-09-11 09:04:36.568146811 +0000 UTC m=+0.116694220 container died 4f4e713d1f2589253d2b282470b0bd4d5c440efa367cc801450a8531088c8b37 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T09:04:37.289 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 09:04:37 smithi092 podman[240882]: 2024-09-11 09:04:37.022156776 +0000 UTC m=+0.570704240 container cleanup 4f4e713d1f2589253d2b282470b0bd4d5c440efa367cc801450a8531088c8b37 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T09:04:37.289 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 09:04:37 smithi092 bash[240882]: ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a 2024-09-11T09:04:37.565 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 09:04:37 smithi092 podman[240899]: 2024-09-11 09:04:37.290063906 +0000 UTC m=+0.726479023 container remove 4f4e713d1f2589253d2b282470b0bd4d5c440efa367cc801450a8531088c8b37 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T09:04:37.565 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 09:04:37 smithi092 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T09:04:37.565 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 09:04:37 smithi092 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-11T09:04:37.565 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 09:04:37 smithi092 systemd[1]: Stopped Ceph node-exporter.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T09:04:37.565 INFO:journalctl@ceph.node-exporter.a.smithi092.stdout:Sep 11 09:04:37 smithi092 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 4.876s CPU time. 2024-09-11T09:04:38.091 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 211625d4-7018-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T09:04:39.911 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:39 smithi145 systemd[1]: Stopping Ceph iscsi.iscsi.a for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:04:39.912 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:39 smithi145 ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[62637]: debug Shutdown received 2024-09-11T09:04:49.964 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:49 smithi145 bash[165378]: time="2024-09-11T09:04:49Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-11T09:04:49.964 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:49 smithi145 podman[165378]: 2024-09-11 09:04:49.711936621 +0000 UTC m=+10.111353717 container died d07f9d6332065432b61b7f59ebe9920edb3d0ded130bb854fa8ae00dcd1789f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T09:04:50.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:50 smithi145 podman[165378]: 2024-09-11 09:04:50.055821953 +0000 UTC m=+10.455239046 container cleanup d07f9d6332065432b61b7f59ebe9920edb3d0ded130bb854fa8ae00dcd1789f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-09-11T09:04:50.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:50 smithi145 bash[165378]: ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-11T09:04:50.597 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:50 smithi145 podman[165403]: 2024-09-11 09:04:50.240471632 +0000 UTC m=+0.526241742 container remove d07f9d6332065432b61b7f59ebe9920edb3d0ded130bb854fa8ae00dcd1789f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T09:04:50.597 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:50 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-11T09:04:50.859 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:50 smithi145 podman[165459]: 2024-09-11 09:04:50.598138716 +0000 UTC m=+0.109650190 container died d513187baac9a3e6e430da0be54c3acf39476921565b59802636792c8fb0132e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T09:04:51.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:50 smithi145 podman[165459]: 2024-09-11 09:04:50.951211165 +0000 UTC m=+0.462722640 container cleanup d513187baac9a3e6e430da0be54c3acf39476921565b59802636792c8fb0132e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T09:04:51.161 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:50 smithi145 bash[165459]: ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-11T09:04:51.161 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:50 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-11T09:04:51.161 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:50 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 62462 (conmon) remains running after unit stopped. 2024-09-11T09:04:51.161 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:50 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 165476 (podman) remains running after unit stopped. 2024-09-11T09:04:51.161 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:50 smithi145 systemd[1]: Stopped Ceph iscsi.iscsi.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T09:04:51.161 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:50 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 3.171s CPU time. 2024-09-11T09:04:51.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi145.stdout:Sep 11 09:04:51 smithi145 podman[165476]: 2024-09-11 09:04:51.411263732 +0000 UTC m=+0.816906167 container remove d513187baac9a3e6e430da0be54c3acf39476921565b59802636792c8fb0132e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T09:04:51.743 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 09:04:51 smithi145 systemd[1]: Stopping Ceph grafana.a for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:04:52.054 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 09:04:51 smithi145 podman[165604]: 2024-09-11 09:04:51.743947151 +0000 UTC m=+0.120810265 container died c1e40a1789f917c84c910f2f7fec27534d7edbb37fc606af3e0c8b491194bf05 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-04-29T11:48:11.055897, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.32.0, io.k8s.display-name=Red Hat Universal Base Image 8, release=239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , description=Ceph Grafana Container, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, version=8.5, vcs-type=git, distribution-scope=public, io.openshift.tags=base rhel8, architecture=x86_64) 2024-09-11T09:04:52.055 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 09:04:52 smithi145 podman[165604]: 2024-09-11 09:04:52.056034834 +0000 UTC m=+0.432897947 container cleanup c1e40a1789f917c84c910f2f7fec27534d7edbb37fc606af3e0c8b491194bf05 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a, version=8.5, com.redhat.component=ubi8-container, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, description=Ceph Grafana Container, architecture=x86_64, name=ubi8, release=239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, distribution-scope=public, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Paul Cuzner , io.buildah.version=1.32.0) 2024-09-11T09:04:52.411 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 09:04:52 smithi145 bash[165604]: ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a 2024-09-11T09:04:52.411 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 09:04:52 smithi145 podman[165620]: 2024-09-11 09:04:52.248803185 +0000 UTC m=+0.505402247 container remove c1e40a1789f917c84c910f2f7fec27534d7edbb37fc606af3e0c8b491194bf05 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-grafana-a, com.redhat.component=ubi8-container, maintainer=Paul Cuzner , io.openshift.tags=base rhel8, vcs-type=git, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.buildah.version=1.32.0, name=ubi8, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.expose-services=, distribution-scope=public, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.display-name=Red Hat Universal Base Image 8, vendor=Red Hat, Inc., version=8.5, architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664) 2024-09-11T09:04:52.411 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 09:04:52 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T09:04:52.880 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 09:04:52 smithi145 systemd[1]: Stopping Ceph node-exporter.b for 211625d4-7018-11ef-bcea-c7b262605968... 2024-09-11T09:04:52.880 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 09:04:52 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-11T09:04:52.880 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 09:04:52 smithi145 systemd[1]: Stopped Ceph grafana.a for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T09:04:52.880 INFO:journalctl@ceph.grafana.a.smithi145.stdout:Sep 11 09:04:52 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@grafana.a.service: Consumed 13.752s CPU time. 2024-09-11T09:04:53.161 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 09:04:52 smithi145 podman[165735]: 2024-09-11 09:04:52.881374556 +0000 UTC m=+0.100942793 container died 256664afc9fccad27937179d5fcf67aa40753a56ff735c49f2ed613770544542 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T09:04:53.493 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 09:04:53 smithi145 podman[165735]: 2024-09-11 09:04:53.251547079 +0000 UTC m=+0.471115311 container cleanup 256664afc9fccad27937179d5fcf67aa40753a56ff735c49f2ed613770544542 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T09:04:53.493 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 09:04:53 smithi145 bash[165735]: ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b 2024-09-11T09:04:53.764 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 09:04:53 smithi145 podman[165755]: 2024-09-11 09:04:53.494477484 +0000 UTC m=+0.616792352 container remove 256664afc9fccad27937179d5fcf67aa40753a56ff735c49f2ed613770544542 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-211625d4-7018-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T09:04:53.765 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 09:04:53 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-11T09:04:53.765 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 09:04:53 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-11T09:04:53.765 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 09:04:53 smithi145 systemd[1]: Stopped Ceph node-exporter.b for 211625d4-7018-11ef-bcea-c7b262605968. 2024-09-11T09:04:53.765 INFO:journalctl@ceph.node-exporter.b.smithi145.stdout:Sep 11 09:04:53 smithi145 systemd[1]: ceph-211625d4-7018-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 4.540s CPU time. 2024-09-11T09:04:54.291 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T09:04:54.323 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T09:04:54.351 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T09:04:54.353 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901289/remote/smithi092/crash 2024-09-11T09:04:54.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/crash -- . 2024-09-11T09:04:54.405 INFO:teuthology.orchestra.run.smithi092.stderr:tar: /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T09:04:54.405 INFO:teuthology.orchestra.run.smithi092.stderr:tar: Error is not recoverable: exiting now 2024-09-11T09:04:54.407 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901289/remote/smithi145/crash 2024-09-11T09:04:54.408 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/crash -- . 2024-09-11T09:04:54.443 INFO:teuthology.orchestra.run.smithi145.stderr:tar: /var/lib/ceph/211625d4-7018-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T09:04:54.443 INFO:teuthology.orchestra.run.smithi145.stderr:tar: Error is not recoverable: exiting now 2024-09-11T09:04:54.444 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T09:04:54.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-09-11T09:04:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:grep: /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-11T09:04:54.476 INFO:tasks.cephadm:Compressing logs... 2024-09-11T09:04:54.476 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T09:04:54.519 DEBUG:teuthology.orchestra.run.smithi145:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T09:04:54.549 INFO:teuthology.orchestra.run.smithi145.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T09:04:54.549 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-11T09:04:54.549 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T09:04:54.549 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose --/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/tcmu-runner.log: /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T09:04:54.549 INFO:teuthology.orchestra.run.smithi145.stderr: 63.5% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-11T09:04:54.549 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T09:04:54.549 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T09:04:54.550 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T09:04:54.551 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T09:04:54.551 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T09:04:54.551 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T09:04:54.560 INFO:teuthology.orchestra.run.smithi092.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T09:04:54.561 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T09:04:54.562 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T09:04:54.562 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T09:04:54.562 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T09:04:54.562 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T09:04:54.563 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T09:04:54.563 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T09:04:54.563 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-11T09:04:54.564 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T09:04:54.564 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T09:04:54.564 INFO:teuthology.orchestra.run.smithi092.stderr: 58.8% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-11T09:04:54.605 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 95.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T09:04:54.621 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 91.2% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T09:04:54.667 INFO:teuthology.orchestra.run.smithi145.stderr: 92.3% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T09:04:54.715 INFO:teuthology.orchestra.run.smithi092.stderr: 91.2% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T09:04:54.753 INFO:teuthology.orchestra.run.smithi145.stderr: 94.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T09:04:55.699 INFO:teuthology.orchestra.run.smithi092.stderr: 89.6% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T09:05:00.944 INFO:teuthology.orchestra.run.smithi145.stderr: 91.2% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T09:05:02.749 INFO:teuthology.orchestra.run.smithi092.stderr: 91.2% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T09:05:16.912 INFO:teuthology.orchestra.run.smithi092.stderr: 90.4% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T09:05:48.607 INFO:teuthology.orchestra.run.smithi092.stderr: 93.6% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T09:06:34.073 INFO:teuthology.orchestra.run.smithi145.stderr: 93.8% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T09:08:05.096 INFO:teuthology.orchestra.run.smithi092.stderr: 92.8% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T09:09:21.131 INFO:teuthology.orchestra.run.smithi145.stderr: 93.7% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T09:09:37.293 INFO:teuthology.orchestra.run.smithi145.stderr: 93.1% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T09:09:43.852 INFO:teuthology.orchestra.run.smithi092.stderr: 93.1% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T09:10:02.259 INFO:teuthology.orchestra.run.smithi145.stderr: 93.4% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T09:10:02.260 INFO:teuthology.orchestra.run.smithi145.stderr: 2024-09-11T09:10:02.260 INFO:teuthology.orchestra.run.smithi145.stderr:real 5m7.727s 2024-09-11T09:10:02.261 INFO:teuthology.orchestra.run.smithi145.stderr:user 4m35.096s 2024-09-11T09:10:02.261 INFO:teuthology.orchestra.run.smithi145.stderr:sys 0m12.341s 2024-09-11T09:10:17.600 INFO:teuthology.orchestra.run.smithi092.stderr: 93.2% -- replaced with /var/log/ceph/211625d4-7018-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T09:10:17.601 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-11T09:10:17.601 INFO:teuthology.orchestra.run.smithi092.stderr:real 5m23.069s 2024-09-11T09:10:17.602 INFO:teuthology.orchestra.run.smithi092.stderr:user 5m12.855s 2024-09-11T09:10:17.602 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m14.010s 2024-09-11T09:10:17.602 INFO:tasks.cephadm:Archiving logs... 2024-09-11T09:10:17.602 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901289/remote/smithi092/log 2024-09-11T09:10:17.604 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T09:11:06.107 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901289/remote/smithi145/log 2024-09-11T09:11:06.108 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T09:11:46.989 INFO:tasks.cephadm:Removing cluster... 2024-09-11T09:11:46.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 211625d4-7018-11ef-bcea-c7b262605968 --force 2024-09-11T09:11:47.854 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 211625d4-7018-11ef-bcea-c7b262605968 --force 2024-09-11T09:11:48.677 INFO:tasks.cephadm:Removing cephadm ... 2024-09-11T09:11:48.677 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T09:11:48.696 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T09:11:48.711 INFO:tasks.cephadm:Teardown complete 2024-09-11T09:11:48.712 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T09:11:48.721 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T09:11:48.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T09:11:48.739 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T09:11:48.858 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T09:11:48.866 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T09:11:48.867 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T09:11:48.869 DEBUG:teuthology.orchestra.run.smithi145:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T09:11:48.883 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-09-11T09:11:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-11T09:11:48.896 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T09:11:48.897 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-09-11T09:11:48.897 INFO:teuthology.orchestra.run.smithi145.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 -2820us[-2795us] +/- 59ms 2024-09-11T09:11:48.897 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 +1679us[+1703us] +/- 53ms 2024-09-11T09:11:48.897 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +1222us[+1246us] +/- 71ms 2024-09-11T09:11:48.897 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:11:48.906 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T09:11:48.906 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-11T09:11:48.906 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 -2802us[-2709us] +/- 59ms 2024-09-11T09:11:48.906 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 12 +1704us[+1796us] +/- 53ms 2024-09-11T09:11:48.906 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 81 +1240us[+1331us] +/- 70ms 2024-09-11T09:11:48.906 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:11:48.907 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T09:11:48.917 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T09:11:48.917 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T09:11:48.981 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T09:11:49.163 DEBUG:teuthology.orchestra.run.smithi145:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T09:11:49.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T09:11:49.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:11:49.290 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T09:11:49.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:11:49.318 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T09:11:49.327 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T09:11:49.361 INFO:teuthology.task.internal:Duration was 2831.901529 seconds 2024-09-11T09:11:49.362 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T09:11:49.401 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T09:11:49.401 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T09:11:49.404 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T09:11:49.466 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T09:11:49.473 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T09:11:49.624 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T09:11:49.624 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-09-11T09:11:49.624 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T09:11:49.646 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi145.front.sepia.ceph.com 2024-09-11T09:11:49.646 DEBUG:teuthology.orchestra.run.smithi145:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T09:11:49.696 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T09:11:49.696 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T09:11:49.699 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T09:11:49.725 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T09:11:49.725 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T09:11:49.766 DEBUG:teuthology.orchestra.run.smithi145:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T09:11:51.302 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T09:11:51.317 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T09:11:51.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T09:11:51.343 DEBUG:teuthology.orchestra.run.smithi145:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T09:11:51.375 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T09:11:51.388 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T09:11:51.390 DEBUG:teuthology.orchestra.run.smithi145:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T09:11:51.429 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-09-11T09:11:51.443 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = core 2024-09-11T09:11:51.461 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T09:11:51.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:11:51.505 DEBUG:teuthology.orchestra.run.smithi145:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T09:11:51.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:11:51.520 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T09:11:51.530 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T09:11:51.531 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901289/remote/smithi092 2024-09-11T09:11:51.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T09:11:51.618 DEBUG:teuthology.misc:Transferring archived files from smithi145:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901289/remote/smithi145 2024-09-11T09:11:51.618 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T09:11:51.658 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T09:11:51.658 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T09:11:51.661 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T09:11:51.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T09:11:51.719 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T09:11:51.719 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T09:11:51.758 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T09:11:51.758 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T09:11:51.761 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T09:11:51.773 INFO:teuthology.orchestra.run.smithi092.stdout: 265932 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 09:11 /home/ubuntu/cephtest 2024-09-11T09:11:51.776 INFO:teuthology.orchestra.run.smithi145.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 09:11 /home/ubuntu/cephtest 2024-09-11T09:11:51.777 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T09:11:51.789 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T09:11:51.856 INFO:teuthology.run:Summary data: description: orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} duration: 2831.901529073715 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-11T09:11:51.856 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T09:11:51.935 INFO:teuthology.run:pass